Tip of the Iceberg

Now that my busiest month since Lotusphere is drawing to a close, I wanted to share just a few of my thoughts about two recent conferences I had the pleasure to attend: IamLUG and MWLUG.

Both of these conferences were extremely well organized.
Although each was only two days (not counting peripheral elements, such as the welcome reception at MWLUG and the follow-on day at IamLUG), these weren't user group meetings... they were definitely conferences. As I mentioned in person to some of the organizers of each, what amazes me most is that, while IBM had some involvement in the form of sponsorship, keynotes and the like, the actual organization of the events - choosing and scheduling sessions, communicating with speakers and attendees, coordinating with venue personnel, and all of the many details I can't even think of that are crucial to making events like these useful and enjoyable - all of these tasks were performed solely by members of the community who are passionate enough to sacrifice whatever time, sanity, and other personal comforts they had to in order to not only pull it off, but make it look easy.

No hotel's ISP can survive a horde of Yellowbleeders.
At both events, the wireless held up pretty well the first day, then died spectacularly that night. Then came back up. Then crashed again. Repeatedly. Eventually, whoever was trying to resuscitate it went to sleep, and the external connection was down the rest of the night, revived just barely before sessions resumed in the morning. It was eerie how similarly this played out at both events. Seriously, if someone were to show me crash logs from the two routers that showed identical timeframes between, and duration of, the outages, I wouldn't be surprised. By the time I left Cleveland on Saturday, I was even half tempted to cut Disney some slack for their perennial failure to accommodate our hunger for bandwidth... but only half tempted. If I can get 20 Mb/s down, 4 Mb/s up for $50 a month - as an individual consumer with no bargaining clout - surely a Hilton Garden Inn can afford more than a 384 Kb/s DSL line... and "the award-winning Walt Disney World Swan and Dolphin" has absolutely no excuse: they charge about 4 times that much each night for a single room during Lotusphere, and most of us that stay onsite stay for at least 4 nights. It baffles me that Internet is still such a problem for almost every hotel, yet Chick-fil-A can figure it out... and offer it for free.

I get by with a little help from my friends.
I always enjoy getting to hang out with my Lotus pals. It might surprise some that only know my blog persona, but I'm an introvert. It's often a struggle of Herculean proportions to force myself to interact with others, especially face-to-face. Not so when I'm with Yellowbleeders. Perhaps knowing that we have a common interest reassures me that this is one place where I don't have to fear awkward silence, because we're never going to run out of things to talk about. But I've also been fortunate to encounter at events like these numerous folks whom I would genuinely enjoy being around, even if we didn't share professional interests. So each time I get to attend a user group meeting, a mid-year conference, or Lotusphere, I leave socially rejuvenated as well as professionally inspired. For me, at least, that's the true value of these events: I come back to my day job, keenly aware of why I do what I do; in our industry niche, that awareness alone can be powerful.

A track-within-a-track can be very effective.
A couple of the MWLUG speakers noticed in advance that there seemed to be a natural progression inherent in the XPage-related sessions, and suggested to the organizers that the agenda be scheduled in a specific order to allow each session to segue into the next, creating a sort of "XPages boot camp". The organizers liked the idea, and, aside from a tiny bit of overlap (very difficult to avoid when you're squeezing 45 sessions across 3 tracks into a 2 day event), the result was perfect. In fact, the result was so effective that, in retrospect, I'm actually glad that my "grand finale" demo crashed on me. I know that sounds strange, but it allowed me to witness something I wouldn't otherwise have been able to see firsthand.

You see, the entire goal of my session ("XPages: the Evolution of Possible") was to challenge the attendees to discard all assumptions of limitation - to believe that we can now do anything with Domino, even if we don't yet know how. Some of the demonstrations illustrated functionality that was possible but difficult or hacktastic in prior versions, but the final demo would have shown something that I've never even seen attempted without XPages... something that is a fundamental departure from our very definition of the word "collaboration" in the context of the platform. Those of you who saw it in action at IamLUG know what I'm talking about. To be perfectly honest, if someone had told me that XPages make this not only possible but easy back when I was first learning XPages, I would have been extremely skeptical. Point of fact, I suspect I wouldn't have believed it. After all, at the time, I just thought the big win with XPages was that they let us do joins in views.

I really, really wanted to be able to show this demo, because I feel it perfectly illustrates why I get so excited about this stuff. When I realized it wasn't going to happen, I confessed my disappointment to the attendees, stating that a mere verbal description of the functionality just wouldn't do it justice (which is why I'm now being intentionally vague about what the demo was). But then something fascinating happened: as I proceeded to provide a mere verbal description of the functionality, I glanced around the room... undistracted by slides, browser windows, Designer, or anything else I'd been intending to call to their attention, I could simply look people in the eye and tell them an incredible story about what XPages had empowered me to create. And here's the punchline: having listened for two days to some amazing speakers (among others, Scott Good, Dave Leedy, Mike McGarel, and Roy Rumaner,) explaining what XPages are, what technologies are fundamental to working with them (including some we've long been familiar with, like Notes Formula and LotusScript), and what features we can add to our applications that we never could before... I could tell that they believed me. I didn't even have to show them; they didn't ask me to "prove" to them that this crazy thing we would previously never have dared attempt actually works. I simply looked them in the eye and told them, and they knew I was telling the truth. I think that's amazing, and I don't think that would have been possible had the sequence and quality of the presentations leading up to mine not prepared them to discard the very assumptions of limitation I'd been hoping to convince them to abandon. In short, while most (if not all) speakers want every presentation to go exactly as planned, this one unexpected failure turned out to be the best proof I could have hoped for that my session achieved - for at least a few folks - precisely what I'd hoped.

So I want to thank the organizers, fellow presenters, and attendees of both events for helping to make this one of the most fun and inspiring months I've had in a very, very long time. I hope to see many of you again soon...

Posted by Tim Tripcony At 9:35 AM Location Home | Permalink | Comments (2)

Like Nathan, I am now rocking a shiny new dedicated GROUP Live instance, so going forward, you can get to this blog via xmage.gbs.com. The blog database on that server is just a replica of the one that was already on Frodo, so for the foreseeable future, you'll still be able to get here via TimTripcony.com - no need to update your feed readers, bookmarks, tattoos, etc., but the xmage server is generally going to be faster and comparatively impervious to Canton's frequent thunderstorms, so you might want to anyway.

Posted by Tim Tripcony At 9:56 PM Location Home | Permalink | Comments (4)

Posted by Tim Tripcony At 6:45 PM Location Canton | Permalink | Comments (5)

Posted by Tim Tripcony At 3:08 AM Location Home | Permalink | Comments (5)

Posted by Tim Tripcony At 2:22 AM Location Home | Permalink | Comments (6)

In summary, let's recap some of the most unconventional characteristics of this approach to authentication:

• We're storing a hashed representation of the user's password directly within the application, but do not store their user ID within the same record.

• Despite not storing the user ID in the record that stores their password, we can rapidly retrieve the password when we need to because the UNID for that record adheres to a format that is both predictable and inexpensive to calculate.

• Once the user has been registered, their password never needs to be sent to our application again. As long as the consumer knows the user's ID and password, it can generate a temporary API key based on tokens it requests from the API.

• Our token and key format causes tokens to expire after a period of time of our choosing, allowing each consumer to cache tokens temporarily but not indefinitely. This allows users to perform multiple operations during a single session without having to request a new token for each. This approach minimizes both the total traffic to our application and the response time for each operation, but also minimizes the period of time during which an intercepted key can be used maliciously.

• We never store either the token that we send to the consumer or the API key we expect to receive in return, but because we know what that key must be, we assign the token record a predictable UNID, allowing both the validity and the identity of the user to be rapidly calculated.

• The use of a predictable UNID for both account records and token records removes the need to create views that display either document type. This reduction of the indexing burden that the application places on the Domino server, in addition to the rapid retrieval of data when requested, provides superior scalability and performance compared to traditional Domino data structures.

• Most importantly, by preventing all users – both anonymous and conventionally authenticated – from accessing any data or design elements other than the XPage design elements used to surface our API to consumer applications, we are able to keep our application's data entirely secure despite being unable to leverage the entire security model that is built into the Domino application model.

I hope that you've found this article informative, and that this approach to authentication gives you yet another way to use Domino to provide amazing application experiences to your users.

FIN

Download this entire article as:

• PDF
  
• ODF
  
• DOC
  

For additional convenience, here are links to each section of this article:

1. Putting your data to REST
   
2. Making a Token gesture
   
3. Anonymity is a warm blanket
   
4. Welcome to the club
   
5. The Keymaster and the Gatekeeper
   
6. The sum of the parts
   

Posted by Tim Tripcony At 1:41 AM Location Home | Permalink | Comments (4)

Thus far, we've only looked at the "back end" code that can be used within an XPage to register and authenticate users. So let's finish off by exploring how all the concepts this article has discussed can come together to create a fully functional application API.

The following is the entire source XML of registerUser.xsp:

The first characteristic of this element that is atypical for an XPage is the rendered attribute of the UIViewRoot node: by setting it to false, we tell Domino not to send any markup of its own. This allows us to "print" to the browser only the specific data that we want to send. Since we are surfacing an API to consumer applications that will provide its own visual representation of the data, we can send just the data with no visual characteristics of our own. We do that using the sendJson method of our base API class:

This method obtains a handle on the servlet response, which allows us to not only write our desired content directly to the consumer, but also to set the content type of the response and set a header that prevents caching, just in case the consumer would otherwise choose to cache the response.

We use the afterPageLoad event of the XPage to call this method, but before doing so, we use the native importPackage function of SSJS to allow us to use an abbreviated syntax. Since all SSJS code is ultimately interpreted by Java at runtime, we would normally have to use the full Java namespace (com.puncrock.api.API.sendJson) to allow the runtime to load the correct method; importPackage provides the class loader a shortcut for finding the method, so as long as we haven't defined any conflicting variables, we can simply call API.sendJson() and the correct code will be executed.

The registerUser method returns an instance of a UserRegistration class. We've previously examined its register method; let's take a look now at its toJson method:

This method uses the org.json package, which can be downloaded from (as you might expect) json.org. Although the XPage runtime automatically includes multiple packages for generating JSON, when generating – and parsing – basic JSON structures, the org.json package is far simpler to use, as the above example illustrates.

If the registration request succeeded, the consumer will receive the following response:

{"success":true}

If the request was unsuccessful, however, the response will be similar to the following:

{"success":false, "error":"User Id timtripcony already exists"}

Either response provides the consumer all of the information it needs to provide appropriate feedback to the user.

The source of getToken.xsp is quite similar:

Once again, we block the default rendering of the page, and in its place, send back a JSON response – this time, by calling an almost identical toJson method of the TokenRequest class we examined earlier:

This time, if a valid user ID was provided, the consumer should receive a response similar to the following:

{"token":"AEB5929ED34F602FD98DB7917098AC5B"}

Finally, let's allow a user to actually do something in our application. To be precise, let's allow a user to store their current GPS position in their account record. So we'll create setUserLocation.xsp:

This page calls the setUserLocation method of our base API class, and passes it several request parameters, including the API key that should identify a valid user:

Because this is the longest single block of code we've yet seen, but also because it finally demonstrates a real use case for this authentication technique, let's examine it in excruciating detail.

1. We create an empty JSON object that will be sent back to the consumer to notify the user of the result of the attempted operation.

2. We call the getUserId method we examined earlier, passing it the API key received with the request. If the key is expired – or was never valid – we update the JSON object to convey that the operation was not successful because the user is not authenticated. This provides the consumer an opportunity to request a new token and try the same operation again.

3. If, however, the key we received was valid, we now know the user ID associated with the token used to generate the key. We can, therefore, rapidly obtain a handle on their account record.

4. Having now located their account record, we write the latitude and longitude data received with the API request and save the record.

5. Finally, we update the JSON object to convey that the operation was successful.

Expanding on this basic premise would allow your service to do more interesting things: instead of just storing the user's current location by overwriting item values in their account record, you could create a separate "check-in" document each time, storing the user ID, location data, and current date-time. This might allow consumer applications to display a chronological map of a user's travels, perhaps even calculate patterns of changes in physical distance between members of a given social circle.

What is significant about this example is not what is done with the data that was submitted to the application; rather, the most pertinent facet of the above method is how the application is determining whether the data should be accepted in the first place – and if so, on whose behalf.

Posted by Tim Tripcony At 10:25 PM Location Home | Permalink | Comments (0)

Now that our application has users, we need to allow them to log in. As we discussed earlier, this process essentially just consists of two steps: generating temporary tokens, and matching tokens that come back in subsequent API requests to the corresponding user. The latter step also includes determining whether the token is even valid to begin with and, if so, whether it has expired. We'll now take a look at exactly how to do this.

Just as the data transport format you choose for your API is a decision you must make based on priorities you set for your application, you can choose any format you wish for constructing tokens. In order to provide one example of this technique, we will be modeling our authentication scheme based on some principles used by Toodledo, a cloud-based task management service. To be precise, we will construct a temporary token upon request to send to the consuming application. Instead of simply sending that token back, however, the consumer will be required to send back an API "key" based on that token accompanying any subsequent API request.

The format that Toodledo uses for an API key is as follows:

md5 ( md5 ( password ) + token + userId )

Let's express this format briefly in paragraph form. First, the user's password is hashed using the MD5 algorithm. Next we append the temporary token and the user ID to that result. Finally, we MD5 hash the entire combination one more time. The result is a 32-character mixed case key that is a representation not only of the user's credentials, but also of the period of time during which the key is valid.

In order to illustrate how we can generate the temporary token that will be used to construct the API key, let's take a look at the constructor of a TokenRequest class:

This code attempts to locate the user's account record. Remember, we don't have to search a view to locate this document: because the UNID of an account record is predictable, we know how to locate it even though the document that we'll retrieve doesn't actually store the user ID. Hence, if the hashed user ID does not match the UNID of an existing document, the specified user does not exist. Otherwise, the handle we now have on the account record was retrieved rapidly, no matter how much data our application currently contains. This performance implication is fundamental to why this technique is so useful when designing an application to be as scalable as possible.

If the specified user ID is valid, we create a document that represents the temporary token that is being requested. As before, we set some convenience field values – this time, both the form associated with the document and the creation date of the token. Similar to our user registration process, however, we have no need to ever store the token. As you can see, we're overwriting the UNID of the token document with a value of our own choosing, but until we do, it has a default UNID automatically generated by Domino; for the sake of convenience, this will be the token that we send back to the consumer. We could instead generate our own random or sequential identifier, of course, but Domino has already generated a sequential identifier for us, which we are about to discard anyway, so we may as well put it to good use while we have it.

Having set our return value to be the temporary, automatically-generated UNID, we retrieve the user's password from their account record, and pass it along with the user ID and token to a method that will determine what the corresponding API key must be when it is sent back by the consumer. Because we're already storing the MD5 hash of the user's current password, we don't need to hash it again when calculating the API key (NOTE: this is a slight change from what was discussed in the previous section in response to a suggestion from Karsten; the PDF version of this article will reflect this change in section 4):

We don't need to calculate this yet, of course: it's the consumer's responsibility to provide this key, not the token generation process. However... immediately before we save the token document, we set its new UNID to be an MD2 hash of the expected API key. As a result, when we do receive an API request that includes a key, determining whether that key is valid – and, if so, the identity of the corresponding user – is easy:

We simply hash the key, and try to find a document with a matching UNID. If none exists, that key was never valid. If a document does match, we check the last modified date of the userId item; if it's older than 4 hours (here, again, is an arbitrary portion of this approach that can be changed to suit your own priorities), then it's expired. But if the document exists, and the token it represents hasn't expired, then the userId item tells us on whose behalf the action specified in the API request should be performed. Because each requested token is valid for a specific period of time, we can inform all API consumers how long they can cache tokens within their own application before having to request a new one; they need not request a new token prior to every API request.

In the next section, we'll pull all of these concepts together to create a functional API.

Posted by Tim Tripcony At 9:56 PM Location Home | Permalink | Comments (2)

Before we can allow users to log in, we must first, of course, have users. Since we're using a custom authentication scheme, we obviously need a custom registration mechanism as well.

For the purposes of this authentication approach, a user consists only of a document within our application that stores a single item value: the user's password. This, again, may seem to be unconventional if not heretical. Remember, however, that no end user will have direct access to any data. As long as our API does not programmatically expose a user's password, our system will remain secure. But you may also have noticed that we're not storing the user's ID in the document that stores their password... there's no need, as you'll soon see.

Earlier we discussed an option built into Domino for obtaining an MD2 hash of any string value. Because this implementation is specific to Domino, it is actually a variation on the standard algorithm, and will not return the same encoded result that would be returned by other implementations. Furthermore, the token format used in the authentication technique this article proposes relies upon the use of MD5 hashes; not only is there no implementation of this algorithm built directly into Domino, but we need to be absolutely sure that the token value we receive from third-party applications is the precise value we have calculated in advance that we are expecting to receive. We need, therefore, a more generic method for hashing string values.

One of the key benefits of XPages is that it is now easier than ever to incorporate external code into our applications: specifically, Java code written by developers with no knowledge of Domino can be imported directly into, and leveraged within, Domino applications – nearly always without any additional modification needed. In many cases, features built into the base Java language can be very useful in our applications. As an initial example, let's take a look at a small custom utility class, called HashMaster:

This class not only allows us to generate an MD2 hash for reasons already mentioned, but the same class can also obtain an MD5 hash, which will be useful later, both in generating and in validating authentication tokens. In the meantime, let's take a look at a method from another class (DominoUtil), which will be used extensively in our API:

This method does the following:

1. Obtains an MD2 hash of the "primary key" passed to the method

2. Since the result of the hash is a syntactically valid UNID, it attempts to locate a document within the specified database that has the same UNID

3. Although the hash result might be the UNID of an existing document but also might not be, there is the possibility that an exception will be thrown. In that scenario we may optionally create a new document and set its UNID to match the hash. Take note that, if a new document is created, it is not yet saved. In either scenario, we end up with a document whose UNID is an encoded version of the text used to request it... in other words, the UNID is meaningful: if I pass "Tim Tripcony" as the key, the resulting UNID is an encoded representation of my name.

Finally, let's examine how this concept is put to good use within the register() method of our UserRegistration class:

As you can see, this method attempts to locate a document that has a UNID matching the MD2 hash of the user ID we are attempting to register. If the document it returns is not a new note, then the user has already been registered: the database already contains a document with a UNID representative of the specified user ID. If, however, the document is a new note, then we can safely create the new account.

As was previously mentioned, we're not storing the user ID in their account record – just their password. For the sake of convenience, we are writing a Form item, in case we want to create a view to display all user accounts, though it's not likely we ever would. Similarly, you may choose in your own application to store additional information about each user to allow users to search for other users and view information about them, such as a profile picture or contact information. For the purposes of authentication, however, we never need to search any index for a user's account record, because we can always obtain a hash of their ID and navigate straight to the account record via the resulting UNID.

One concept that should be noted is the format used for the user ID. This could be a canonical name (i.e. "CN=John Doe/OU=..."), a shortname (i.e. "jdoe"), an email address... when using this style of authentication, the name format is inconsequential as long as it's consistent. The key limitation is that – unlike traditional Domino authentication – this particular technique would not allow the same user to enter their canonical name, shortname or email address and still be authenticated as the same user... you have to choose one.

An additional implication of using a hash of the user ID as the UNID for the account record is that this makes the user ID case sensitive, which is atypical; most authentication systems expect the user's password to be entered using the same case as the stored value, but do not require the case to match for the user ID. Because an hash of a lowercase string will not return the same value as an MD2 hash of an upper- or mixed-case version of the same string, however, our implementation requires that the case of the ID match each time it is evaluated – at least, in the low-level code. You can certainly shield your users from this requirement by converting the case of the ID to ensure it is always the same prior to being hashed. If, however, you prefer that the ID be case-sensitive, hashing the ID as passed without case modification will automatically enforce that requirement for you.

Posted by Tim Tripcony At 12:07 PM Location Home | Permalink | Comments (8)