Tip of the Iceberg

Category xpages java
Several years ago, I wrote a utility Java class designed to make it easy to search for components within the current XPage instance based on various criteria. I've found it enormously useful, and, apparently, so has Keith Strickland, because he added it to org.openntf.xsp.extlib, complete with a few refinements. As an example of how you might use this, examine the following line of code:

List<UIComponent> requiredFields = new XspQuery()
  .addInstanceOf(UIInput.class)
  .addEquals("required", true)
  .locate();

Just in case the above isn't self explanatory, this line of code returns an iterable collection of all components in the tree that extend the UIInput class and have a value of true for the required attribute... in other words, all required fields on the page.

The basic premise upon which this utility class operates is that you feed it one or more "filters", then call locate(), optionally passing it a context component. It starts at that context component (or the view root, if you didn't specify one), and spiders through all its descendants (its children, their children, and so on) looking for any that match all the criteria you specified. It flattens pointers to any matches it finds into a single List, which you can then process however you want. In short, it's effectively a server-side equivalent of dojo.query.

Here are all the filters it currently supports, expressed as method signatures:

public XspQuery addContains(String propertyName, Object content);
public XspQuery addEndsWith(String propertyName, String content);
public XspQuery addEquals(String propertyName, Object content);
public XspQuery addInstanceOf(Class klass);
public XspQuery addStartsWith(String propertyName, String content);


These filter methods behave, respectively, as follows:

• The value of the specified property contains the specified text
• The value of the specified property ends with the specified text
• The value of the specified property is equal to the specified object
• The component can be cast as an instance of the specified class
• The value of the specified property begins with the specified text

Disclaimer 1: each of these filter addition methods returns the XspQuery instance against which they're called as an optional convenience: as demonstrated above, returning the current object from each method allows chaining of the query's construction from declaration of the object iself straight through to the eventual execution of the query. As a general rule, you don't want to do this in Java. In theory, any method of any class could throw an Exception. When you chain methods (instead of performing each operation on its own line), it can be more difficult to debug problems when they arise, because it's not immediately apparent which of the methods you called failed in the line flagged as the point of failure. In the case of XspQuery, none of its chainable methods perform any operations that could fail (with the possible exception -- sorry for the pun -- of an out of memory error), so it's almost guaranteed that if anything does fail, it's in the call to locate() at the end. This type of chaining is nearly always going to be safe. But if you're concerned that this will cause you to form bad habits elsewhere, it would be wise to restructure the above example as follows:

XspQuery requiredFieldLocator = new XspQuery();
requiredFieldLocator.addInstanceOf(UIInput.class);
requiredFieldLocator.addEquals("required", true);
List<UIComponent> requiredFields = requiredFieldLocator.locate();

This obviously looks and feels more verbose, but ensures that if any individual operation fails, you know right away which operation failed. That's why it's best to stay in the habit of not chaining method calls. But in case you're comfortable that chaining your uses of XspQuery won't be a bad influence on your overall coding style, the terser syntax is supported and relatively safe.

Disclaimer 2: the tree navigation used by XspQuery is based on the children (and facets) of each component it examines. So if your page contains repeat controls, it will only find one instance of each descendant thereof, unless you've set repeatControls to true, which you should never do unless you're content with preventing said repeat from becoming completely unaware of any changes to the collection to which the repeat is bound. So just keep that in mind if you're trying to use XspQuery to mess with components inside a repeat... and, if you are, there's almost definitely a better way.

In addition to the filters listed above, this class supports several convenience methods, each of which call locate() internally:

public UIComponent byClientId(String id, UIComponent root);
public UIComponent byId(String id, UIComponent root);
public UIInput locateField(String id, UIComponent root);
public List locateInputs(UIComponent root);


As with locate(), each of these use the view root as the context component if an alternative is not passed to the method, and they behave, respectively, as follows:

• Returns the component whose client ID (as rendered in the markup sent to the browser) matches the specified text
• Returns the first component whose server ID matches the specified text
• Returns the first editable field (instance of UIInput) whose server ID matches the specified text
• Returns all editable fields

So you could, for example, update the server-side value of a single field from client-side JavaScript via JSON-RPC:

new org.openntf.xsp.extlib.query.XspQuery()
  .byClientId(componentId)
  .getValueBinding("value")
  .setValue(newValue);

Or empty the values of all fields on the page:

for (UIInput eachField : new XspQuery().locateInputs()) {
    ValueBinding vb = eachField.getValueBinding("value");
    if (!(vb == null || vb instanceof ValueBindingEx)) {
        vb.setValue(null);
    }
}

Finally, Keith added a class called ComponentTreeMassage, which is a wrapper for a bunch of convenient static methods (meaning that you can just call them directly against the class itself without having to construct a separate instance each time) that use XspQuery to do all sorts of crazycool things... the easiest way to sum up all of what this class facilitates is to equate it to server-side DOM manipulation. Basically, if you want to construct (or restructure) your component tree on the fly, instead of defining the entire page at design time, this is the way you want to do it.

You can view the source of all the files in this package here. If you want to use this functionality without installing the entire extension library this project represents, just open the history page for each class in the package, then click the "Raw" button, and save the .java file that button redirects you to.

Posted by Tim Tripcony At 2:00 AM Location Home | Permalink | Comments (7)

Category musings
I've noticed a pattern emerging when I'm asked for help with XPages. Here's a representative conversation:

"I'm trying to do [X] and it's not working. How can I do that?"

"What are you trying to accomplish?"

"I already told you. I'm trying to do [X]."

"No, that's how you're trying to do it. What are you trying to do?"

For example, replace "[X]" with "reach into a repeat control from outside it" (since this has become the most frequent topic I'm asked about when an XPage developer is frustrated because what they think they're trying to do isn't working). That's a how, not a what. But I'm finding that this distinction is difficult to discern. The reason is that there's a trap that we all fall into from time to time: I was asked to implement a feature, and the moment I decided how I wanted to implement that feature, I committed to making that approach work. But that's not what I was asked to do... I was asked to make the feature work. Ergo, if the approach I chose to pursue isn't working, then it's possible -- perhaps likely -- that a better approach exists.

So I've recently switched to a different wording of the same question. Because we all occasionally fall into the trap of confusing what we're trying to do with how we're trying to do it, I've found that the following seems to dispel that confusion:

If this works, what will the user think they're doing?

Users don't know what repeat controls are. Or data sources. Or Custom Controls. Or any other terminology that we tend to think in. If what we do works, when the user accesses the application, they think they're submitting a travel request; or approving an expense report; or purchasing something online. In my experience, when you're struggling to implement some very specific functionality, if you step back and think about what you're trying to accomplish in terms of what the user needs to be able to do, a simpler solution will nearly always present itself. The end result might not seem as cool to you, but you will have met your users' needs. In this business, that's all that truly matters.

Posted by Tim Tripcony At 1:30 PM Location Home | Permalink | Comments (6)

Category
I will be presenting at two upcoming conferences, Developer2013 and IamLUG.

Developer2013 will be held at the MGM Grand in Las Vegas April 30 to May 2, and is organized by THE VIEW. I will be presenting the following sessions:

Doing more with less code in XPages

"Work smarter, not harder." We're all expected to, but are rarely told how. In XPages, however, we have many opportunities to do precisely that. This session will equip you with techniques for writing less code to achieve the same result. Learn the surprising power of Expression Language and discover how its syntax allows simple expressions to produce complex results. See new ways of letting data sources do the work for you instead of manually moving data from place to place. Most powerfully of all, step through real examples of how to consume code written by others so you can skip solving problems others have already addressed and focus entirely on delivering solutions that are unique to your application. As a bonus, learn how implementing any of these techniques even improves the performance of your application so that your users can spend less time waiting and more time working.

The mobile interface you're already developing

A challenge that has faced Domino developers for over a decade is maintaining "hybrid" applications: user interfaces that cater both to the Notes client and to web browsers. As phones and tablets augment -- even replace -- the use of laptops and desktops to access applications, it can appear that this challenge has grown even more difficult. Thanks to the evolution of the Domino platform, however, the opposite is true. This session will focus on two ways you can minimize duplication of effort while ensuring that your application is indifferent to how your users choose to access it. First, learn how to structure your design elements to allow them to be used in numerous user interface contexts with little or no targeted customization. Then, take a look at how XPages empower us to leverage emerging web development techniques to produce a "responsive design": a single user interface that is both attractive and functional, whether accessed via the Notes client, a desktop web browser, or a mobile device.

IamLUG will be held at HoteLumiere in St. Louis May 6 - 7, and is (as always) completely free to attend. I will be presenting the following session:

Become a Control Freak: discover XPage components that will make your apps awesome

In the early days of Lotus Notes, most end users had no home computing experience. Today, nearly everyone has a laptop and a mobile device -- if not several of each. Applications developed behind the firewall must now compete for users' mindshare with their daily use of Facebook, Twitter, LinkedIn, and other publicly available services that they perceive to be modern. Providing user experiences that delight users, regardless of their screen resolution or physical location, can seem like a daunting task... but it doesn't have to be. This session will take you on a tour of XPage components that make it easy to develop -- and maintain -- compelling, modern user interfaces.

If you're not already registered for either -- or both -- of these events, you should seriously consider signing up. The abstracts listed above are just a representative sample of the type of guidance these events provide. Not only is it always energizing to get together with a bunch of folks who share your professional passion, the technical content is likely to be just the nudge in the right direction that you need to spare yourself massive headaches later on, as well as providing some ideas for how to make your user experiences amazing. If your boss needs convincing, Kathy Brown has already gone to the trouble of creating a form letter that spells out rather clearly the value of IamLUG; THE VIEW has done the same for Developer2013... just click the "Convince Your Boss" link on the Developer2013 site.

Posted by Tim Tripcony At 7:00 PM Location Home | Permalink | Comments (1)

Category musings

"We go about our daily lives understanding almost nothing of the world. We give little thought to the machinery that generates the sunlight that makes life possible, to the gravity that glues us to an earth that would otherwise send us spinning off into space, or the atoms of which we are made and on whose stability we fundamentally depend. Except for children (who don’t know enough not to ask the important questions), few of us spend much time wondering why nature is the way it is; where the cosmos came from, or whether it was always here; if time will one day flow back backward and effects precede causes; or whether there are ultimate limits to what humans can know. There are even children, and I have met some of them, who want to know what a black hole looks like; what is the smallest piece of matter; why we remember the past and not the future; how it is, if there was chaos early, that there is, apparently, order today; and why there is a universe."

- Carl Sagan

Posted by Tim Tripcony At 1:54 AM Location Home | Permalink | Comments (3)

Category xpages java openntf
Suppose you had a motivation to learn a new spoken language. As an example, let's imagine that you live in the U.S., but your job requires you to occasionally visit Paris, so you've decided to learn French. But you're not planning to move there, just spend a week there every couple months. So you don't want to learn the entire language, just enough to facilitate basic interaction whenever you're there. So you briefly considered taking a semester-long course at a local community college, but that seemed like an unnecessary hassle when you remembered that a close friend of yours speaks fluent French. So instead you just ask him to expose you to the basics... just enough to get by. He's happy to oblige, and teaches you a few catch phrases he promises will be useful. But the first time you try speaking to actual French people, they look a bit confused. They can guess what it is you're actually trying to say, but something about the way you're saying it just isn't natural. And every time you ask where the nearest public restroom is, whoever you ask seems deeply offended.

By the end of that trip, you give up on French. After all, everyone you really need to talk to understands enough English. It should be their responsibility to understand you, shouldn't it?

Here's the problem: the guy who taught you this small sampling of French was raised German. So he told you he was teaching you French, but the French he taught you feels like German. All the words are technically valid French words, but in a real life context, using them in the way he taught you can cause confusion... and, in some cases, serious problems.

That's not entirely unlike using the original* Domino Java API. Yes, it's technically valid Java, but it carries with it some serious baggage from both C++ and LotusScript. As such, it doesn't feel like Java. In fact, the more comfortable you become with that tiny window into the Java world, the more foreign any other Java becomes. And in the meantime, trying to speak this subset of the language without understanding what you're really "saying" can cause some serious problems.

Enter org.openntf.domino.

Paul Withers hinted about it, Ulrich Krause has been tweeting about it, and Nathan Freeman has now posted an overview of the goals. What "it" is, quite simply, is an open source replacement for the lotus.domino API.

If you want all the details of why this will make every Domino developer's life easier, check out Nathan's post and the project wiki. But the quickest way I know how to demonstrate the benefits is with an actual code example.
Here's the proper way to iterate all the documents in the current database using the standard Domino API (assuming your run context is an XPage in an application that uses the XPages Extension Library):



...and here's an exact functional equivalent using the org.openntf.domino API:



Forgive the rhetorical question, but which of these two seems easier?

Here are the reasons I'm most excited about this new API (and enjoying being part of the project team):

1. No recycling. None. Nada. Never again. Just like LotusScript, Java has garbage collection. We shouldn't have to manually clean up C++ object handles just because we switch from LotusScript to Java. And now we don't have to. Make no mistake: this isn't a "pure Java" vs. SSJS issue. If you're iterating view entries today in SSJS code and you're not using the first pattern shown above (set temp object to next, recycle current, set current to temp), you are leaking C++ handles, and eventually your app will blow up. org.openntf.domino allows you to completely forget about recycling with impunity.
2. Iterable collections. lotus.domino collections aren't Java collections. They're Java implementations of LotusScript collections. Or, perhaps more accurately, they're Java implementations of C++ collections using LotusScript patterns. Any way you choose to look at it, they make your code way more verbose than it needs to be. True Java collections are iterable... they allow "foreach" behavior. Or, for those of us accustomed to LotusScript, ForAll behavior. In my opinion, rule 1 of Java API design should be, if you're going to call something a collection, make it a Collection. So we did.
3. Not everything throws an error. In lotus.domino, absolutely every useful method declares that it throws NotesException. So if you're writing code that uses that API, you often have to wrap code in a try (or bubble the throw) that can't possibly fail. That's just ridiculous. Your try blocks should wrap code that has at least a possibility of failure, implement a workaround if possible, and if not, facilitate presenting useful feedback to the user. As such, best practice in Java is to only wrap very specific operations (often, only a single line of code) in a try, so that it's fairly easy to decide how to handle an error, because you were only trying to do one thing when the error was encountered. Because absolutely everything in lotus.domino claims that it might fail, it has engendered the awful habit in many Domino developers of wrapping entire procedures in a single try just to avoid having to wrap every line in a separate try. Actually handling errors when writing code that way would be a nightmare, so Domino developers usually don't handle Java errors... they just spew a stack trace to the log and then either ignore the error and pick up where they left off, or bail out of the entire procedure, causing any remaining work to be skipped. With org.openntf.domino, your error handling is driven by your business process, not by the API.
4. Interactive documentation. Just about everyone else in the Java world has the benefit of Javadoc. When they're writing code, and content assist helps them choose which method to call on an object, it gives them information that helps them to know which method they should choose and how to properly call it. This information typically includes intelligently named method parameters, and a description of what the method does, how, and why. By contrast, if you have a handle on a Domino view, and want to call the FTSearchSorted method, does this help you at all?
   
   
   
   This might make things a bit easier:
   
   
   
   For the methods we've already added the necessary Javadoc comments to, you also get this off to the side of the suggested methods list:
   
   
   
   A couple members of the project team have been musing that it makes a sad statement about the current API that we're really excited that this new API will have this kind of interactive documentation. But it truly is easier to write code when the tool you use to write the code can guide you in deciding what the code should be, and nearly all Java developers just take for granted that any API they consume will provide them such guidance. Designer has had the capacity for over four years to do this, but lotus.domino allows that capacity to go to waste... org.openntf.domino remedies this.

When does this all become a reality? Well, you're welcome to try it out already, if you don't mind that it's not entirely polished yet. Some early adopters are already using it to test CRUD against tens of millions (yes, millions) of documents in a single procedure, so it's rather stable. But we're planning to release a version in 11 days that would be perfectly safe to use in real production apps. And we have some pretty cool features on the horizon shortly after that initial release. For now, I'll just leave you with a teaser: Designer's capacity to understand Javadoc syntax isn't the only portion of Domino that's currently going to waste.

*The original Java API in Lotus Notes was the lotus.notes package, added to Notes in 4.5. This was replaced in 4.6 by the lotus.domino package. It's one of the few times anything has ever been deprecated in Notes / Domino.

Posted by Tim Tripcony At 12:15 AM Location Home | Permalink | Comments (3)

Category xpages java

Before we dive in to this first topic, I should mention Declan's series, "XPage Java Roots". Declan has been shifting more of his code to Java, so just as he did with his epic "Learning XPages" series, where he documented his initial experiences with XPages itself, he is now documenting his experience of learning how to take advantage of Java in XPage development. It's a safe bet that this series will be a very useful reference, so whether or not my own perspective on this topic proves valuable, you should definitely check out his articles on this subject.

As I previously mentioned, my goal with this series is to shed light on easy ways to incorporate Java into XPage development, and my intended approach is to try to answer the questions, "When?" "How?" and "What?". Because I believe the process of asking -- and answering -- these questions to be cyclical, I've chosen to start with a How topic: knowing the entry points for telling your XPage to run Java code.

You're already doing this.

If you've written even a single XPage, then the first two entry points are techniques I can guarantee you're already using.

The Eclipse platform includes a concept of a "builder": this is a task that runs when an Eclipse project is compiled. Designer includes several builder tasks. One of them scans all XPage and Custom Control design elements in the current application and generates a Java class that corresponds to each, then stores the compiled class file in design notes within the application. The first entry point, then, is simply writing an XPage... no matter what that page does: by creating any XPage or Custom Control, you're instructing Designer to generate Java code that will be run when your users access the page. So you're already producing Java code... you're just trusting Designer to write it for you.

The second technique uses a JSF concept known as a "resolver". The job of a resolver in XPages is essentially to parse text in real time and guess what Java code should be executed in its place. This means that we're now dealing with an interpreter instead of a compiler. If you want to get pedantic, Java is interpreted too, in the sense that it is stored in a format known as "bytecode", which must then be interpreted by the Java Virtual Machine, not executed directly by the computer itself. The JVM, however, is a layer between your code and the server's hardware that is incredibly optimized. JSF resolvers, on the other hand, are interpreters executed by that bytecode, so even if you consider Java to be an interpreted language, then we're nesting interpreters... no doubt that's enough computer science jargon to illustrate that there might be some performance implications here.

If you've ever looked at the source XML of an XPage and seen an expression like "#{currentDocument.Subject}", you've already seen a resolver instruction. This type of expression gets parsed on the fly, as your users access and interact with your XPage, and if Domino can successfully figure out what the expression "means", then it knows what Java code to run. If not, the result is an error. The portion of the XPages runtime that does the parsing is known as an EL resolver; the syntax we use for these expressions is called Unified Expression Language, or Expression Language for short... or just EL, for even shorter. In some cases, a $ is used instead of a #, but in the context of this topic, the difference that causes at runtime is irrelevant, because either way, the expression still serves as a shorthand instruction to run Java code.

The parsing of this syntax is highly efficient because there's a very small set of permutations of what it can contain. It packs a lot of power into a very limited set of operators. If you see a colon in one of these expressions, however, all bets are off. Well, some bets are off. This is because whatever precedes the colon is instructing the runtime to use an alternate EL resolver. For nearly all of us, by far the most common example of this is the "javascript:" prefix, which we typically refer to as SSJS. Any expression using this prefix tells the server to construct an abstract syntax tree based on what the text of that expression represents, according to the ECMAScript specification, then run Java code that approximates equivalent operations. This approach to parsing is still impressively efficient, but far less so than a "prefixless" expression, because there are so many more possibilities for what the expression can contain. The resolver must determine what everything means in context. So, for example, if it encounters parentheses, do they surround the definition of a new function's argument list, or arguments being passed to an already defined function? Or are they simply enforcing order of execution?

Regardless of which resolver is used, though, this is all EL: SSJS, therefore, is not an alternative to EL, it is a subset of EL. It just uses a very complex resolver instead of the default resolver.

But this is probably the most crucial point in this particular explanation: a key aspect of any EL resolver's behavior is called the variable resolver. The variable resolver's job is to determine what Java object every variable in your expression refers to. Once it knows what each object is, then it can determine what to do with and to those objects.

So let's take a second look at the example expression from above:

#{currentDocument.Subject}

When parsing this expression, the first thing the runtime does is look for a colon. None is present, so it uses the default resolver. The default resolver determines that the entire expression consists of two pieces: one variable named "currentDocument", and a property of that variable named "Subject". So it needs to figure out what "currentDocument" refers to.

There are a couple places it looks. One of them is the requestScope. So if you manually store something in the requestScope, the key you used will remain a valid variable for the remainder of that request. There are cases where variables automatically created by Domino are only valid for a subset of request processing... for instance, the row variable for a view panel or repeat control, or a dataContext assigned to a specific panel. But in general, an object stored in the requestScope can be accessed by key without explicitly referring to requestScope, because the variable resolver treats requestScope keys as variable names.

If everything goes as planned, the variable resolver should determine that "currentDocument" refers to an instance of a Java class called com.ibm.xsp.model.domino.wrapped.DominoDocument. Now it needs to decide what to do about the "Subject" property. For this specific example, you'd most likely create this expression to assign as the "value" attribute of a data component, such as an Edit Box. If this is the case, then the same expression can mean one of two things; either the page is being "rendered" (sent to the browser), which means this is a read operation, or data is being posted to the server, so this is a write operation. Once the server has determined which, it needs to decide what methods to run to handle the property.

The DominoDocument Java class is an instance of a Java interface called com.ibm.xsp.model.DataObject. LotusScript supports the definition of classes, but does not have a notion of interfaces, so this is a new concept we can take advantage of in Java. An interface is basically a promise: specifically, that any class claiming to "implement" it will have all of the methods that interface defines. However, it may choose to implement those methods very differently from how other classes that implement the same interface implement its methods. In the case of DataObject, the two methods you'll most want to be aware of are getValue() and setValue(). Because these are part of the DataObject interface, that means that any data source in XPages has these two methods. What these methods do behind the scenes will typically be wildly different based on which type of data source you're dealing with. In the case of DominoDocument, they handle getting data to and from the "back end" document.

So, in a read operation, #{currentDocument.Subject} ends up running as something approximately equivalent to the following Java code:

VariableResolver resolver = ApplicationEx.getInstance().getVariableResolver();
FacesContext context = FacesContext.getCurrentInstance();
DominoDocument doc = (DominoDocument) resolver.resolveVariable("currentDocument", context);
return currentDocument.getValue("Subject");

In a write operation, however, the last line will be slightly different:

doc.setValue("Subject", newValue);

This is one of the handy benefits of EL: not only is it very efficient, it allows us to use a single expression for a property that is read/write, and just let the platform figure out what to do. But this is what is really happening in XPages... all of our EL expressions tell Domino to run Java code on our behalf. It's just not necessarily our Java code.

Stop guessing.

When you call methods on any object in SSJS, therefore, you're already using Java... you're just using a bizarre IDE to do so. Every reference to session or database or context.getUser().getRoles().contains("[Admin]") is causing Java to execute... Domino just has to temporarily interpret all that code as JavaScript in order to guess what Java code to execute.

Similarly, when you define your own objects in SSJS, the variable resolver turns them into instances of Java classes that approximate the behavior of JavaScript objects in a browser. These objects have characteristics in common with portions of the Java Collections Framework but include some extra overhead. There are also classes in that framework that do a lot of useful things for you automatically without you having to tell them to, like ensure that members of a list are unique and / or automatically sorted. Paul Withers found a handy cheatsheet that tersely describes what each is useful for.

So one of the easiest ways to start getting additional benefits from Java code is to occasionally allow Domino to stop guessing what you want, just for an individual object, by explicitly stating that it should be an instance of a specific Java class.

This is done by using the full canonical name of the class. Just like Domino user names have a common format (e.g. "John Doe") and a canonical format (e.g. "CN=John Doe/O=ACME"), so do Java classes. The canonical name of the HashMap class, for example, is java.util.HashMap. The portion prior to the common name (e.g. java.util) is known as its "package". Java packages are just sets of related classes, similar to how we tend to bundle related LotusScript code into a Script Library.

So if you were planning to write the following code:

var userInfo = {}; // because you already know never to use 'new Object()'
userInfo.firstName = userDoc.getItemValueString("firstName");

You might choose to tell Domino that you don't need the extra overhead of a JavaScript object (like support for maintaining closures) by writing this code instead:

var userInfo = new java.util.HashMap();
userInfo.put("firstName", userDoc.getItemValueString("firstName"));

So instead of getting an object that is like a HashMap, but with features you know you're not going to use anyway, you get an object that is a HashMap.

If you know you're going to be creating a bunch of objects from the same package, you can save a few characters on each line by using the importPackage() statement:

importPackage(java.util);
var firstMap = new HashMap(); // common name is now enough
var anotherMap = new HashMap();
var sortedMap = new TreeMap(); // different class, but same package

At last count, the XPages runtime includes over 1,500 packages that just ship with the platform. Not 1,500 classes... 1,500 packages. For a lot of what you want to do in an app, Domino already makes code available to you that will do it for you instead of you having to reinvent the wheel (or call a LotusScript agent because you have old code that does the same thing). Using only the syntax above, you can skip writing (or maintaining) your own custom code for doing common things like parsing or generating JSON and XML or encoding and decoding Base64. Stuff like that is just built in to the platform now.

The only tricky part here is actually finding out that what you want to do has already been done for you, so it's very tempting to ignore that all this code is just right here waiting (with apologies to Richard Marx). But if you do choose to ignore it, either you are less productive (because your code takes longer to write) or your users are (because your code takes longer to execute)... or both. So I encourage you to discover what is already available to you. The difficulty is that there's no quick and easy way to do this. The most comprehensive list I've found is the Extensibility API Javadoc, which is already a bit outdated, because it's for 8.5.2, and they don't seem to have published an updated Javadoc for 8.5.3. Additionally, this only covers the couple dozen pacakges that are considered part of the Extensibility API... so who knows what's in the other 1,500 packages and how to interact with them.

So at present the best -- and far from ideal -- way to get a feel for what is already available to us without having to install anything else or bundle additional libraries into our applications is to switch to the Package Explorer, and underneath the root for any open application, expand the section labeled "Plug-in Dependencies". Every .jar file listed there is a collection of packages, each of which is a collection of classes, each of which is a collection of methods. I feel comfortable guaranteeing that, if you're still calling LotusScript agents to avoid having to rewrite all that code, at least some of it has already been rewritten for you in those bundled packages.

And the good news is that even if you only learn about one class that allows you to skip running a LotusScript agent against an in-memory document, your app will perform better than it otherwise would, and you have less code to maintain because you're now calling platform code instead of a custom equivalent. The platform API now includes so many features that none of us will ever discover, much less master, every one of them, but don't let that become an excuse for never learning any of it.

To be continued...

This post is already longer than I'd planned, so I'm going to wander off and pick this up again later. Note that so far we have neither written any custom Java code ourselves, nor have we added anything to the platform from external sources, such as open-source Java libraries or a commercial product API. We've just been removing a tiny bit of the ambiguity around which type of Java objects our XPages will use.

In the next post I'll cover how to allow the exact same pattern to add capabilities to the platform that don't ship with it... again, without writing any actual Java yourself. That can wait for later.

Posted by Tim Tripcony At 3:20 AM Location Home | Permalink | Comments (6)

Category java xpages
The discussion following my last post made stark the need for greater availability of information that makes the nature of Java more accessible to Domino developers. Credit for the title of this post goes to Declan, who is considering writing a series of blog posts on this topic. I will be doing the same; hopefully there will be a fair amount of duplication. As David Leedy is fond of stating, it's a good thing when several people share the same information, because that makes it easier for the people who need it to find it. And that's what's missing: an easy path to using Java in XPages.

My hope is to answer three questions:

When?

How?

What?

There's plenty of Why on my blog already. And I'm not the only one who's been advocating an increased use of Java in favor of other languages. But when does it make sense to define your business logic in Java instead of an alternative? In my opinion, that's the first question that needs to be more clearly answered. As several folks stated in the aforementioned discussion, if I simply tell you to use it for everything, that can be an intimidating answer. And, frankly, it's not realistic. At least, not right away. For most of you, the first time you directly make use of Java, it will be for a single use case within the context of an entire application that meets the users' needs via other means. Over time, you may find that you've reached a point where the majority of your code is written in Java. But that shift is likely to be gradual. And that's perfectly fine.

Suppose you've identified a specific use case for Java... now you need to know how to gain the benefits of Java just for that use case without impacting the rest of what your application does. Do you call Java classes that are already bundled with Domino? Or do you download a library that is entirely unrelated to Domino and call classes that are defined in that library? Or do you write your own classes entirely from scratch? Once you've decided, how do you actually do that?

Finally, it is crucial to know what the code needs to be. In time, you'll find that this is often the first question you'll be asking, not the last. And this entire process is recursive: as your exposure to Java increases, you'll become aware of the types of tasks it excels at, as well as the common programming problems that others have used it to solve; this will give you fresh opportunities to ask when it is appropriate to create a custom solution and when to just stand on someone else's shoulder, and to ask how to best integrate the resulting solution into your application and your broader Domino ecosystem.

I do understand that, for many, the prospect of learning Java seems difficult. I've been there too. My goal is to provide answers to these questions that demonstrate that XPages really do make it easy to put the power of Java to work for you. The truth is, it does this every day: XPages are Java... when we write code in any other language, we're just introducing another layer of abstraction that has to be translated back into Java anyway. And, if your experience is similar to mine, you'll find that the more that you discard those extra layers and just speak to the platform in its native tongue, the easier it becomes to do the very things that already feel difficult.

If the last 4.5 years have taught me anything about Domino, it's that when something I'm trying to do in XPages seems hard, it's because I'm fighting the platform somehow. Once I identify where I'm resisting it, where I'm not allowing it to be what it already is, whatever I was trying to accomplish instantly becomes easy. So if at times it seems like I'm disproportionately nonchalant, that's the reason. I'm not just hoping to convince you that Java is nothing to be afraid of, I'm hoping to show you just how easy it makes everything you want to do.

Posted by Tim Tripcony At 11:40 PM Location Home | Permalink | Comments (12)

Category

<disclaimer>

I've been debating for quite a while whether I should write this post. It obviously makes a potentially controversial statement. A fellow developer who knew I was drafting it put my hesitance into perspective: "you really want to be that guy?" This was my response:

I want to be the guy who saves people pain. But sometimes to do that, you have to tell your friend if she wants to stop being punched in the face, she needs to leave the guy who keeps punching her in the face.

This post is just about making short-term programming decisions that have long-term benefits, so obviously the above is hyperbole. But that's where I'm coming from: a lot of you won't want to hear what's coming, but to the best of my knowledge this is the truth, awkward or unpleasant as it might perhaps seem. I'm making this statement only because I firmly believe that any Domino developer who takes it to heart will at some point be relieved in retrospect.

</disclaimer>

As I've occasionally mentioned, I spend a bit of time each week on stackoverflow. This is partly selfless: I enjoy helping people. It's partly selfish: the more successful we all are at developing XPage applications, the more the market realizes the platform's potential, which makes it easier for me to keep a job without having to drastically alter my career path. The bulk of the questions I notice on the site are indicative of someone embracing the platform but struggling with a specific aspect of it. That's a natural part of the learning curve, and it's great that they have a place to go for assistance with overcoming those obstacles.

There's another style of question that, quite frankly, concerns me. With apologies to those who I'm calling out here, what inspired me to finally publish this post is that lately I've noticed quite a few with a similar theme:

• Lotus Notes - @Transform - Workaround available for XPages?
• SSJS equivalent of LSI_Info from LotusScript
• agent.runWithDocumentContext(document) retrieve document “body” field
• Calling Java agent from LotusScript in XPages upload
• How to use IBM SBT in background LotusScript code

I've listed those in a non-chronological order for a reason. Like anyone who's supremely convinced of how right he is, I'm going to frame the evidence in a way that supports my existing conclusions.

What I consider to be common about all of these is an understandable, but limiting, mindset: "I want to do new things, but somehow keep doing them the way I did the old things." If you happen to believe this is an effective approach, I recommend a personal experiment: surround yourself entirely with people you've known since childhood, and then try to grow as an individual.

While I don't have direct, personal knowledge of why @Transform was added to Notes just over a decade ago, it's probably safe to assume that it's because somebody at IBM acknowledged that what it does was already trivial to accomplish in languages in common use outside of the Notes and Domino world. I want to stress that a bit: list manipulation was already trivial in most languages more than a decade ago. Formula was lacking because achieving the same result had always required comparative hacks. Many of us were aware of this difficulty, but weren't aware that it was atypical, because we weren't using those other languages. So now that our beloved platform has evolved to center around some of those other languages, it's quite natural to want to continue using @Transform, because we didn't know everyone else already had a better way. The same is true of @For, @While, and several other bolt-ons that seemed magical to us when they arrived in Notes, but in actuality were bastardizations of patterns that everyone else just takes for granted.

Continuing to use Formula language within SSJS, therefore, is a crutch. I don't even mean that in a pejorative sense; it's a direct metaphor. We have muscles that are aching to do the work they're designed to do, but those muscles will never become strong if we cling to a tool that essentially performs the same function, but in a comparatively clumsy manner.

Another example of this, however, is when we either try to convince SSJS to be LotusScript, or punt entirely and spin up Agent Manager on every document save in order to keep using outdated code... or, worse, continue to write new code in a language that is, syntactically, nearly identical to Visual Basic 3 (circa 1993) and unlikely to ever again receive significant updates. I can sympathize with the argument that it can be difficult to justify manually refactoring large amounts of LotusScript in Java, especially if the existing code already "works", but only because I've made that argument myself in the past. I only hear that argument now from those who haven't actually done it yet. In my direct, personal experience, those who have made this investment have all had two common experiences: it's not as time consuming as they anticipated, and once they've made the switch, they don't want to go back. Buyer's remorse simply does not occur; at least, again, not in my experience.

How is this possible? Because if your destination language is Java, there's nothing in LotusScript -- or Formula, for that matter -- to miss.

Yes, if you port all your LotusScript to SSJS, of course you're going to be frequently annoyed that JavaScript's error management is sorely lacking compared to LotusScript's. But compared to LotusScript, Java's error management is magnificent.

You're bound to be occasionally annoyed that SSJS is a language primarily devoid of type safety executed in an environment where it tries to maintain type safety anyway. Java does not have this problem.

The SSJS editor can't reliably warn you that you misspelled a variable or function name, so you won't find out until runtime. Java does not have this problem.

I have a theory. I believe that IBM implemented SSJS as a transitional language. I don't believe they did this consciously, although it honestly wouldn't surprise me if that were the case. The evolution of the platform, the projects shared by IBM on OpenNTF, and the conference sessions I've recently attended all point to an increase in the significance of the role of Java within XPages itself as well as the broader IBM software portfolio. This actually mirrors what I've witnessed in my own development and that of the developers I know who have been developing XPages the longest: more and more code that would have been, or used to be, SSJS is instead now being written in Java. Anyone who is now writing the bulk of their code in SSJS is transitioning to Java, even if they don't yet realize it. The trouble with deploying transitional code, of course, is that you must support it. This is why SSJS is also a crutch: if you've ever dreaded converting your LotusScript, but then convert it to SSJS, you'll dread converting it to Java. That's simply inevitable.

So skip SSJS. Just learn Java.

Sure, you'll occasionally write quick SSJS evaluations in isolated component attributes because it doesn't make sense to spin up an entire Java class just to handle that simple expression. But complex business logic? Just tear off the Band-Aid. At some point, you will eventually want to refactor that code in Java. So do it now. Save yourself pain and your employer money.

I have a decent comfort level in about a half dozen programming languages, but I only speak one. So, having never learned a foreign language, I literally cannot speak from experience. But I always hear that there is no substitute for immersion. When learning a new programming language, I've certainly found that to be true. And I can vividly remember being not only hesitant, but terrified, of making the switch to Java. If you look hard enough, you can find evidence of that terror on this very blog. What removed that terror was, plain and simple, immersion. I chose a project (well, okay, Nathan chose it for me), bit the bullet, and just dove right in. That was over four years ago, so I frequently now forget that Java wasn't always my favorite language. I don't miss LotusScript at all. And I certainly don't miss relying on SSJS to define my business logic. I set that crutch aside long ago... and I hope, in the coming months, to provide those of you with a similar career focus both inspiration and assistance in doing the same.

Posted by Tim Tripcony At 12:45 AM Location Home | Permalink | Comments (39)

Category xpages
Paul Withers posted a thorough article explaining the differences between namespaced XPage components (e.g. <xp:div />) and their corresponding passthru elements (e.g. <div />), providing numerous examples of what actually happens when these objects are constructed. I've always heard (and often repeated) that passthru elements are more efficiently processed than their namespaced equivalents, so Paul's post inspired me to offer my own perspective.

Simply put, there's practically no difference... but there are a couple subtle discrepancies.

The three passthru elements I use most in my own development are div, span, and br. As Paul mentioned, any passthru element becomes an instance of UIPassthroughTag. This class extends UIOutputEx, which also happens to be the same base class from which links, labels, script blocks, computed fields, and several viewpanel-related components are derived. UIOutputEx, in turn, extends UIOutput, which itself extends UIComponentBase. As a result, passthru elements are components, and go through the encode and decode cycle just like any other component.

Similarly, XspDiv, XspSpan, and XspLineBreak all extend UIComponentTag (as do XspParagraph, XspTable, XspTableRow, and XspTableCell). UIComponentTag extends UIComponentBase. So as far as the JVM is concerned, all of these things are very much alike.

There are, however, two potentially significant differences, and since none of the components mentioned above can accept data, both differences are associated with the rendering phase.

The first difference relates to how component attributes are rendered. For these three elements (divs, spans, and line breaks), the same renderer class is used regardless of whether the element is namespaced or passthru. However... as of Domino 9, XspDiv supports 17 named attributes of which the renderer is aware. If, for example, you set a value for the align attribute, the component will be constructed as follows:

XspDiv result = new XspDiv();
result.setAlign("right");
return result;

If, on the other hand, you set a value for the align attribute on a passthru div, its construction will be as follows:

UIPassThroughTag component = new UIPassThroughTag();
component.setTag("div");
component.addAttribute("align", "right");
return component;

Fairly similar, yes? But with one key difference: in an XspDiv, the attribute value is stored in a named String property; in a passthru div, the attribute name and value are set as properties of a TagAttribute instance and simply added to a List of arbitrary attributes. This is the reason that you can essentially set any attribute you want on a passthru element, whereas on a namespaced component you must adhere to the named properties (although, in recent versions, you can use the attrs property to define your own). In other words, at page load, it's actually slightly more work to construct a passthru div that has at least one attribute defined than it would be to construct an XspDiv with the same attribute / value mapping. When rendering, however, it's far less. Just for fun, let's compare the process for encoding a div with only an align attribute by pretending the HtmlDivRenderer is self-aware:

Passthru Div

I've started the open div tag, so now I need to send all the attribute values.
I'll loop through this internal List.
The first attribute is named align, and its value isn't null. So I'll send a space, the attribute name, an equals sign, a quote, the attribute value, and another quote. That was easy. Okay, on to the next.
Oh, that was the last list member. I'll send a > to end the open div tag.

XspDiv

I've started the open div tag, so now I need to send all the named property values.
I'll ask for the value of the align property. It's not null, so now let me check its length. It's greater than 0, so I need to send this attribute.
I'll send a space, the attribute name, an equals sign, a quote, the attribute value, and another quote. That was easy. Okay, on to the next.
I'll ask for the value of dir property. Oh, it's null. Okay, nothing to send.
Does lang have a value? Nope.
Okay, how about onclick? Nope.
ondblclick?
onkeydown?
I'm bored.
(10 more properties later...)
Well, does it at least have a title? Nope.
Phew, glad that's over with. Now I have to check for any arbitrary attributes, so I'll loop through this internal List.
Oh, the List is empty. I'll send a > to end the open div tag.

See the difference? The renderer actually has a lot more to do on a component with named properties than it does for a passthru element with arbitrary attributes simply because it has to ask for every possible property value and find out that it's null, rather than just looping through the properties that do have a value. Keep in mind, of course, that the difference between the two is likely around a millisecond each time, and the more attributes you specify on any given element, the less the differential between asking and just looping.

The second difference between how these types of components are rendered is that passthru elements cannot be manipulated via a theme. The UIPassthroughTag class does support the getStyleKitFamily method (which is how the theme determines the shared theme ID for each type of component), but it literally just returns null. Since this class does not include a getter or setter for the themeId property that most components share, there is no way to target any passthru element with a theme. When rendering any themable component, the renderer must consult the theme to determine if any property values are impacted based on the default theme ID for that type of component and any custom theme ID which may have been specified for this specific component instance. In contrast, all theme-related processing can simply be skipped for all elements with no namespace.

Neither of these differences are likely to impact performance enough for your users to notice, so in most cases, switching to passthru simply to gain performance benefits is going to be an unnecessary optimization. In apps where you're explicitly setting the page serialization and scope timeout options, and changing as many events as possible to use partial execution, because you're anticipating (or already encountering) a need for atypical speed or scalability, then you'd be well-served by switching to passthru for any elements that are purely structural and not behavioral - in other words, if you don't need to bind SSJS events to the onclick of a div, might as well just use passthru. But thanks to find and replace, it's much faster to remove the xp: from elements that don't need it after you've already gotten the page functional than it is to add it back in on those few elements where you do need the JSFish nature of a component.

Here's where you can achieve far more incremental performance gain: stop using panels and use divs instead - passthru or otherwise. The div component doesn't show up in the palette by default, so most developers just use a panel for every block element because it's readily accessible for dragging onto the page. Don't. Every time a panel is encoded, it checks for a component-specific ACL, and component-specific data sources. If either or both exist, then some other magic happens. For instance, if you've defined a data source local to a panel, the request scope (and, when the data source is initially created, the scope the data is associated with... which, by default, is the view scope for all data sources) must be updated to include pointers to the data source instance. At the end of encoding the panel, it has to check again for a local ACL or data sources so that it can clean up after itself... remove any scope pointers it added, and so forth. Compared to a div, a panel is a complicated beast. This differential is still small enough that your users won't notice unless the application is under heavy load, but this is simply a good habit to be in so that you don't have to suddenly change your behavior for one app that needs to be atypically scalable: unless a block element must be a panel, because you need localized data or security, just use a div. Besides, when you get in the habit of using the source tab instead of settling for the visual approximation of the design tab, it's less typing anyway.

Posted by Tim Tripcony At 11:35 PM Location Home | Permalink | Comments (4)

Category xpages openntf
About 18 months ago, I created an OpenNTF project called Community Control Library. The fundamental reason for creating the project was my belief that the single factor keeping the Domino community from realizing the true potential of the platform is the assumption that the XPages Extension Library is the extension library, not an extension library. Let's briefly revisit its history:

1. IBM starts an internal project, code named "Porus" (in reference to the Greek / Roman god of plenty), intending to share the code on OpenNTF. The primary intent of this project, if I recall correctly, was to demonstrate the power of the XPages Extensibility API, which was added to Domino in 8.5.2. Publishing the source code for an expanded set of components, data sources, and more would provide Domino developers some best practice examples of how the platform can now be extended to our hearts' content.
2. IBM creates an OpenNTF project to distribute what had previously been called Porus. The project is now known as the XPages Extension Library. As more and more developers download and explore the features made possible by this library, those whose organizational policies prevent them from doing so begin to demand an "officially supported" version from IBM.
3. IBM releases Upgrade Pack 1 for Domino 8.5.3, which consists almost entirely of a version of the XPages Extension Library that they are able to officially support. They continue to periodically publish new releases to the OpenNTF project.
4. IBM releases a public beta of Domino 9, which will bundle the newest officially supported version of the XPages Extension Library... no Upgrade Pack required.

When I created the Community Control Library project in 2011, I had lofty intentions of turning the project into the central place where the XPage development community goes to contribute, and consume, extensions to the platform beyond those already provided by IBM. But life got in the way and I never actually created the library. Recently, however, I was reflecting on the irony of the last item in the above list, and decided to revisit the idea.

You may well be wondering why I consider the above progression ironic. Or, quite possibly, chiding me for my incorrect use of the term. Unless I imagined this, the whole point of the XPages Extension Library was originally to demonstrate that we can add globally reusable artifacts to the platform. Then it became an add-on to the product that customers could file an SPR against, relinquishing all control over the library's behavior and stability back to IBM. Now (or rather, whenever Domino 9 is released), it's just part of the product. It's no longer really an extension library: it's just more features developed by IBM, supported by IBM. The platform is more than what it was before - as it is with every new release - but it's still the base platform. If you install Domino, you have this library.

DISCLAIMER: The intent of this rant is not to be ungrateful. The capabilities provided by this library are amazing. The features we can now rapidly and easily include in our applications continue to boggle my mind when compared to the original set of 38 components and 2 data sources that shipped with 8.5.0. I'm also intensely glad that this library will be natively included in 9; as long as a customer is running that version, I will no longer have to persuade them of the value of installing the library, because they will have already installed it. I can simply deliver more value to them than I otherwise could. This is all awesome.

But it still dilutes the truth that XPage applications can include components and data sources (and EL resolvers, view handlers, phase listeners, resource providers...) that IBM didn't develop. The Extensibility API is there for us. Yes, IBM can also use that to make it easier for them to add new features that they will support. But the primary reason the API exists is to give us more control over what the platform becomes.

The XSP Starter Kit makes it dead simple to create your own extension library. What it doesn't include, however, is an abundance of obviously useful artifacts that IBM will almost certainly never fold back into the product. So that's what I want to create: a library that, by virtue of always remaining a separate add-on, underscores the point that anyone can create this type of library. Here are just a few examples of what I envision including in this library:

• Custom Controls already published in the Apache Catalog on OpenNTF, converted to library components to allow them to be used in any application without having to paste them into each NSF and manage design inheritance.
• Components corresponding to HTML5 tags such as audio, video, and canvas (the latter of which is already part of the XSP Starter Kit).
• Data sources that provide simple consumption of YQL and other APIs.

I'm maintaining this library out on GitHub. Declan has already added some controls to it. Let me know if you have any feature requests (e.g. which Custom Controls from OpenNTF are already your favorites) or, even better, if you'd like to be granted commit access to the repository so you can donate features of your own. I'll periodically post releases to OpenNTF that will include built plugins in case you just want to install the library, but you can pull the source from GitHub at any time if you just want to look at the code or build the plugins yourself.

Posted by Tim Tripcony At 7:07 AM Location Home | Permalink | Comments (4)