Rolling your own authentication for REST API's, part 4 - Welcome to the club
Before we can allow users to log in, we must first, of course, have users. Since we're using a custom authentication scheme, we obviously need a custom registration mechanism as well.
For the purposes of this authentication approach, a user consists only of a document within our application that stores a single item value: the user's password. This, again, may seem to be unconventional if not heretical. Remember, however, that no end user will have direct access to any data. As long as our API does not programmatically expose a user's password, our system will remain secure. But you may also have noticed that we're not storing the user's ID in the document that stores their password... there's no need, as you'll soon see.
Earlier we discussed an option built into Domino for obtaining an MD2 hash of any string value. Because this implementation is specific to Domino, it is actually a variation on the standard algorithm, and will not return the same encoded result that would be returned by other implementations. Furthermore, the token format used in the authentication technique this article proposes relies upon the use of MD5 hashes; not only is there no implementation of this algorithm built directly into Domino, but we need to be absolutely sure that the token value we receive from third-party applications is the precise value we have calculated in advance that we are expecting to receive. We need, therefore, a more generic method for hashing string values.
One of the key benefits of XPages is that it is now easier than ever to incorporate external code into our applications: specifically, Java code written by developers with no knowledge of Domino can be imported directly into, and leveraged within, Domino applications – nearly always without any additional modification needed. In many cases, features built into the base Java language can be very useful in our applications. As an initial example, let's take a look at a small custom utility class, called HashMaster:
- import java.math.BigInteger;
- import java.security.MessageDigest;
- import java.security.NoSuchAlgorithmException;
- public class HashMaster {
- return encode(unhashed, "MD2");
- }
- return encode(unhashed, "MD5");
- }
- byte[] defaultBytes = unhashed.getBytes();
- try {
- algorithm.reset();
- algorithm.update(defaultBytes);
- byte messageDig est[] = algorithm.digest();
- nsae.printStackTrace();
- }
- return hashed;
- }
- }
This class not only allows us to generate an MD2 hash for reasons already mentioned, but the same class can also obtain an MD5 hash, which will be useful later, both in generating and in validating authentication tokens. In the meantime, let's take a look at a method from another class (DominoUtil), which will be used extensively in our API:
- try {
- unid = HashMaster.md2(key);
- result = source.getDocumentByUNID(unid);
- } catch (NotesException ne) {
- if (ne.id == 4091 && createOnFail) {
- try {
- result = source.createDocument();
- result.setUniversalID(unid);
- e.printStackTrace();
- }
- }
- e.printStackTrace();
- }
- return result;
- }
This method does the following:
Obtains an MD2 hash of the "primary key" passed to the method
Since the result of the hash is a syntactically valid UNID, it attempts to locate a document within the specified database that has the same UNID
Although the hash result might be the UNID of an existing document but also might not be, there is the possibility that an exception will be thrown. In that scenario we may optionally create a new document and set its UNID to match the hash. Take note that, if a new document is created, it is not yet saved. In either scenario, we end up with a document whose UNID is an encoded version of the text used to request it... in other words, the UNID is meaningful: if I pass "Tim Tripcony" as the key, the resulting UNID is an encoded representation of my name.
Finally, let's examine how this concept is put to good use within the register() method of our UserRegistration class:
- public boolean register() {
- try {
- DominoUtil util = new DominoUtil(getDatabase());
- true);
- if (userRecord.isNewNote()) {
- userRecord.replaceItemValue("userPass", getPassword());
- userRecord.replaceItemValue("Form", "user");
- setSuccess(userRecord.save());
- } else {
- setError("User Id " + getUserId() + " already exists");
- }
- } catch (NotesException e) {
- setError("An unexpected error occurred while attempting to register user");
- e.printStackTrace();
- }
- return isSuccess();
- }
As you can see, this method attempts to locate a document that has a UNID matching the MD2 hash of the user ID we are attempting to register. If the document it returns is not a new note, then the user has already been registered: the database already contains a document with a UNID representative of the specified user ID. If, however, the document is a new note, then we can safely create the new account.
As was previously mentioned, we're not storing the user ID in their account record – just their password. For the sake of convenience, we are writing a Form item, in case we want to create a view to display all user accounts, though it's not likely we ever would. Similarly, you may choose in your own application to store additional information about each user to allow users to search for other users and view information about them, such as a profile picture or contact information. For the purposes of authentication, however, we never need to search any index for a user's account record, because we can always obtain a hash of their ID and navigate straight to the account record via the resulting UNID.
One concept that should be noted is the format used for the user ID. This could be a canonical name (i.e. "CN=John Doe/OU=..."), a shortname (i.e. "jdoe"), an email address... when using this style of authentication, the name format is inconsequential as long as it's consistent. The key limitation is that – unlike traditional Domino authentication – this particular technique would not allow the same user to enter their canonical name, shortname or email address and still be authenticated as the same user... you have to choose one.
An additional implication of using a hash of the user ID as the UNID for the account record is that this makes the user ID case sensitive, which is atypical; most authentication systems expect the user's password to be entered using the same case as the stored value, but do not require the case to match for the user ID. Because an hash of a lowercase string will not return the same value as an MD2 hash of an upper- or mixed-case version of the same string, however, our implementation requires that the case of the ID match each time it is evaluated – at least, in the low-level code. You can certainly shield your users from this requirement by converting the case of the ID to ensure it is always the same prior to being hashed. If, however, you prefer that the ID be case-sensitive, hashing the ID as passed without case modification will automatically enforce that requirement for you.

Comments
It's not all bad news, however. If you're mapping a "real user" to each user's API ID (either by storing that ID in canonical format to begin with or defining a relationship somewhere between their API user ID and their actual Notes name), then you can at least determine some information about their permissions to documents. For example:
var isAuthor = function(doc, userId) {
var result = false;
var docAuthors = session.evaluate("@Author", doc);
if (docAuthors.contains(userId)) {
result = true;
} else {
var roles = signerDatabase.queryAccessRoles(userId);
for (var i = 0; i < roles.size(); i++) {
if (docAuthors.contains(roles.get(i)) {
result = true;
break;
}
}
}
return result;
}
In other words, you can logically determine whether the user has author access to a certain document. Technically this will still work even if the user isn't listed anywhere in the NAB but is still listed in Authors fields on the document; of course, if you are using Authors fields, you'll need to ensure that the signer also has Author access... but I'm assuming you already know that anyway.
And we definitely lose the convenience of using Readers fields to cause records to simply "disappear" from anyone who shouldn't see them. We have to use the same type of logic to determine record by record whether the user has Reader access - but the logic is even more convoluted because there's no "@Reader" we can call to determine that; we have to loop through all the items in the record, check to see if each is a Readers field, and then perform the same evaluation to see if the user is either explicitly listed, or implicitly included by role or (ideal in most situations, but atrocious in this case) as a member of some cascaded group. When faced with the need to perform all this logic ourselves, it becomes easy to see why using Readers fields always slows down the entire application... often to the point that the security is not worth the degradation; if we're using the built-in security, Domino still has to perform these evaluations, even though we don't have to explicitly instruct it to do so.
So I guess the question becomes: in situations where it makes sense to implement a REST API for an application - which will certainly be an exception for most organizations - which of the following options is ideal?
A) "Settle" for an inferior alternative logic-based security scheme, which forces us to manually do a portion of the work that Domino would otherwise do automatically for us, in order to continue to allow Domino to do everything it does do well that we can still leverage.
B) Since we'd just have to implement our own security anyway, which we can do in any platform, abandon ALL of our existing investment and just migrate the application to another platform.
C) Simply choose not to bother: keep the application locked behind the API's that Domino supports out of the box, limiting its adoption to those who are able and willing to access it via those interfaces.
That list isn't quite as snarky as it probably comes across: those really are our options. There's been an abundance of discussion lately about what IBM or its customers and partners should do to ensure that Domino outlives us all, and those discussions have yielded many suggestions. But none of us (myself included) has created something on Domino that has achieved adoption on the scale of Twitter or Facebook... and then announced to the world that we did so. I know that's a rather high bar to set, but it's possible. But the only way it could happen is if either it has a web front-end that is so utterly compelling that everyone with any computing device capable of an Internet connection wants, for some reason, to navigate to it on a regular basis... or if it uses a globally standard format to surface an API that allows third-party applications and devices to turn the service the application represents into something everyone wants to use. That's certainly within our reach.
And imagine what it would be like if that happened... if I were to write an app called... I dunno, maybe "MyTwitFace". And suddenly that's the app that everyone's using and talking about and posting to from 5 different devices, and CNN is telling their viewers, "to keep up on the latest news, follow us on MyTwitFace"... and then, when a half billion people are using it, I tell the world that it's running on Domino, and that's why it doesn't crash every two hours or suddenly get slow just because it's lunchtime and everybody's hitting our servers. I can't help thinking it would be rather difficult for Microsoft to convince a CIO that they could ever pull that off with Sharepoint. And, while most businesses aren't in the business of creating that type of application, why would they ever abandon Domino in favor of Sharepoint if they're already using a platform that can scale to support one twelfth of the world's population?
Posted by Tim Tripcony At 03:43:09 PM On 06/23/2010 | - Website - |
Posted by Karsten Lehmann At 12:28:49 PM On 06/23/2010 | - Website - |
Additionally, with the new page serialization settings slated to be available in 8.5.2, we'll be able to balance raw speed vs. overall scalability. While I'm going to be too busy between now and the end of August to be able to justify playing with this, I'm planning to create a free iOS (and possibly Android) app this fall that uses the authentication technique described here to connect to data stored on a Domino server sitting in my living room, and just see what happens. It won't be an enterprise app, purely a novelty one. If it goes viral, I'll up the price to $0.99 just so I can pay to move the server to an EC2 instance, then put up an actual website for it that publishes the API so that third parties can build their own apps to connect to it. In the meantime, I'm not going to mention here what the app is... I want to first try to prove the point that Domino deserves to be the back end for some of these trendy apps - sure, the enterprise is its natural home, but who says it can't play games too?
Posted by Tim Tripcony At 06:45:13 PM On 06/23/2010 | - Website - |
Posted by Karsten Lehmann At 05:37:03 PM On 06/23/2010 | - Website - |
md5 ( md5 ( password ) + token + userId )
The only time we ever query the password is when we're determining what the temporary API should be based on the temporary token we send to the consumer, so if we're always storing the password as an MD5 of the actual value, we can skip hashing it each time we calculate the API key. It's an inexpensive calculation to perform, but if our user base were to grow into the millions, any additional efficiency would add up over time.
Similarly, if we were to implement a password change feature, where the user provides the existing password and a new password, we can hash both, and if the hash of the existing matches what we've stored, then it's okay to store the new password - alternatively, we could require that the consumer hash both prior to sending them so that the plain text password is never sent over the wire.
Posted by Tim Tripcony At 02:34:11 PM On 06/23/2010 | - Website - |
Posted by Niklas Heidloff At 11:13:30 AM On 06/23/2010 | - Website - |
I'm just wondering how many of the Domino features you're going to re-implement. The Lotus Notes/Domino security model with it's very fine granularity is one of the things that keeps me from moving to another platform. So far the authentication methods you've described could be implemented in most any environment.
I use reader and author fields in all of my designs. The fact that I can put a role in a reader field gives me fantastic control over who can view documents with very little development overhead. I have Java classes that I've developed over many years that let me automate updated groups, roles and ACLs from web interfaces as well as on a date/time basis. It would take a lot of persuasion for me to all of that up and move to a new model.
Or maybe I'm not clear on where you're going with this. It kind of reads like a nerd suspense novel. I can hardly wait for the next installment. And your writing is very clear.
Peace,
Rob:-]
Posted by RobShaver At 02:28:01 PM On 06/23/2010 | - Website - |
Soon, we'll have the army of interns we need...
Posted by Nathan T. Freeman At 11:58:37 PM On 06/23/2010 | - Website - |
I'm wondering why you're using MD2 for creating the unid and not MD5. From my own tests MD5 is a bit quicker and also produces unid-like hashes.
Warning: The function HashMaster.md2 doesn't always produce 32 char unids. For instance test with "The quick brown fox jumps over the lazy dog" and it returns the 31 char string "3d85a0d629d2c442e987525319fc471". I've added a "unid" method to the HashMaster class:
public static String unid(String unhashed) {
String hashed = encode(unhashed, "MD2");
while (hashed.length() < 32) {
hashed = "0" + hashed;
}
return hashed;
}
Posted by Thimo Jansen At 02:36:45 AM On 06/14/2011 | - Website - |