Currently, I am working on a mailing list browser based on Sling and just thought, I should post the code before it gets complicated (i.e. before error handling and UI chrome get into the way).

The content is spidered form a couple of public mailing lists and stored (almost) in the format described in this post. There is an attachment to this post with test content in XML (system view format) that you can import. Import it into /mailinglists_tmp (if you use some other path you need to adapt the link to get started below).

The Sling application is written in esp and allows users to browse lists, mails and search over all mails. All scripts are attached as well. Store the scripts in the /apps folder. Please see here and here for a description on web dav, Sling scripts and so on. Also, Betrand Delacretaz' 15 minutes guide to Sling is well worth reading if you are new to Sling.

After installation you can get started by pointing the browser at (yourserver)/mailinglists_tmp.html. This should show the list of available mailing lists.

Some things are to be noted in the code:

• The  syntax for outputting a value cannot be used inside of markup attributes. In there one must use expression language syntax ${...}
• The current node that is being worked on is always available as "currentNode" (this was "resource.node" before).
• Through currentNode you have access to further repository functionality, like the QueryManager: var queryManager = currentNode.getSession().getWorkspace().getQueryManager();
• Node properties can be access in JavaScript-style, like node["propertyname"]

You need to have a quite recent Sling build, at least from yesterday, actually, to get this code running. Since it is only a few lines, I suspect it should be easy to understand. If not, just leave a comment or drop me an email.

apps.zip

sysview.zip

David Dossot has just released the JCR transport for Mule. For the uninitiated: Mule is an open source Enterprise Service Bus. The JCR transport allows users to attach a JCR to Mule so that content is routed between the JCR and other sources of information. I think this is a great piece of software that will be useful in many enterprise projects where systems integration plays a big role. I was happy that I could ask David a couple of questions about JCR development:

Q: David, congratulations to the release of the JCR transport. I looked at the list of transports on MuleForge and saw that your JCR transport is one of only two projects that has reached production level - you were fast. Did you find JCR to be a good fit to Mule?

Thank you! The JCR Transport was one of the first projects to be hosted on MuleForge, so it had a little more time for maturing than the other ones. In fact, the very first version of the transport went out at the end of July 2007. This new version, with full support for reading and writing from JCR, took seven months and a couple of milestone releases to be finalized.

JCR has proven to be an incredible good fit for Mule. I was initially not convinced that it would make sense to go further than an simple Observations-based event listener, but comments on The Server Side and Mule's users mailing list decided me to go for a full fledged transport. Indeed, it was proven to be a good idea: the capacity to read and write to JCR from an ESB opens up new possibilities of content-oriented integration scenarios, thanks to the solid and comprehensive architecture of Mule combined with the huge amount of both officially and community supported transports.

Even if the current version of the transport does not support transactions, users can already enjoy advanced functionalities that leverage diverse JCR features like: multi-syntax queries, content streaming and custom node types.

Q: I believe the JCR transport for Mule relies a lot on JCR observations. Would you like to share some of your experiences on working with the Observations API with us?

The Observations mechanism is a simple yet powerful way to monitor changes in a JCR. It was very straightforward to create a Mule message receiver that leverages Observations for creating messages the ESB can then propagate to whatever component or endpoint that needs to receive it.

There are several aspects of the Event object that are somewhat challenging for the ESB, the main one being that it is a "connected" object (bound to the current Session) which can be problematic if you send it over to an asynchronous or distant consumer. This is why I had to create an optional transformer that "detaches" an Event, making it a fully resolved serializable object better apt to be used outside of the JCR context.

Finally, if I had one comment on Observations for the upcoming JSR-283, I would ask for the possibility to listen to only one sub-level below a node path, as currently it is either the whole sub-tree or a single node that can be observed.

Q: What tools did you use for your JCR-related development?

My main Java development environment is Eclipse Europa on Kubuntu Gutsy Gibbons, complemented with the usual plug-ins: FindBugs, CheckStyle, EclEmma, Subclipse... I use JackRabbit's TransientRepository for all the integration tests and the transport examples: it is really convenient as you have full control on the initial state of the repository before running each test.

I am also using Maven for my builds, which is made easy thanks to Mule's modularization in a wealth of specialized Maven artifacts. It is to be noted that Sun's opposition to have javax binaries distributed in the central repository could have complicated the build but, thanks to Day and their legacy Maven repository that contains the JCR archive, anyone can access the required dependency without first installing it by hand!

On the server side, I have been leveraging the great tools offered by MuleSource to the community via its MuleForge platform, which has proven to be a successful place for fostering Mule related extensions. These tools comprise Subversion, the Atlassian full suite (Bamboo, Jira, Confluence, Fisheye), forums and mailing lists.

Q: For the benefit of the JCR community let me ask you about your general experiences with JCR. What did you learn about JCR architecture or technology? What did you miss? What took you by surprise?

I started first to explore JCR in 2006 when helping a client I was contracting for to investigate alternatives to their proprietary and expensive document management system. Since this time, it seems that I can not escape this technology, as I am now working with Communique/CRX!

From the beginning I have been impressed by the clarity of the JCR API, something that is alas rare enough to be mentioned. But JCR is a twofold specification as it not only specifies an API, but also mandates pre-defined mixin and primary node types. I initially thought that, for the sake of cross-vendor portability, JCR should go further in normalizing content models (maybe by deprecating nt:unstructured and by forcing users to stick to a limited set of mixins and node types), but I realized it was as naive as willing to normalize, say, standard database models in the JDBC specification!

I must add that "David's Model", from JackRabbit's Wiki, has been a great source of inspiration for the way the Mule JCR transport handles custom node types.

Q: Coming back to Mule: did you get some user feedback on the JCR transport? Are you aware of any real-world use cases you can share?

As with all open source projects, users tend to only manifest themselves when they face an issue or need a feature. So far, none of this has happened so I do not know who, out there, is using or experimenting with the transport. As JCR got listed by Carlos E. Perez as one of the top 5 Java technologies to learn in 2008, I am pretty sure this transport will get more traction in the coming months!

These days I get asked quite often about the status of Communiqué 5 and when a release can be expected. However, it is not so simple to answer this question with a simple date. That is, because (unlike some other software companies) our release plan is quality based. What does that mean?

Our software development process involves several iterations. At the end of each iteration CQ5 has to pass a quality gate. Of course, there is an ETA for each of our iterations. But when push comes to shove time is considered lower priority than achieving quality gates and will therefore bend accordingly.

On a very high level there are two iterations of interest to our customers:

1. We are currently in the first phase which is called “tech preview phase”. In the tech preview we deliver one single project together with a visionary customer. In this phase we focus exclusively on this one customer.
2. The second phase is a closed beta program. In the course of the beta we will deliver three complete customer projects including new projects or rewrites as well as a migration from an earlier version of CQ. After successful completion of the three projects CQ5 will become generally available.

There are a number of requirements from our side on the beta projects, e.g. relating to size and scope. But there are still slots available. If you are interested let us have a chat if your project is a good fit.

JBoss has started a new project called DNA. As far as I understand it DNA is a new implementation of the concepts of the MetaMatrix software they bought last year. They describe it as:

a repository and set of tools that make it easy to capture, version, analyze, and understand the fundamental building blocks of information

Take a look at the architecture diagram of DNA. There are a number of interesting aspects on this project:

• DNA uses Java Content Repositories for information storage
  
  

  JBoss DNA manages its information in JCR repositories

  I could not find any information about the actual implementation that is used, though.
• More important, in DNA JCR is also seen as an API to all other sorts of existing content:
  
  

  Integrate multiple JCR repositories. Use relational databases. Access applications and services. JBoss DNA can federate and integration information from multiple JCR repositories, external databases, applications and services - all in real time without having to make copies.

  This is very much in line with the original vision of JSR-170: "JCR the API" is more important than "JCR the implementation". It is also very similar to the"everything is content" vision Day has been sharing for a very long time.
• One last aspect I want to mention is displayed in the upper part of the architecture diagram: access to the repository is possible through HTTP, REST and ATOM (as well as JDBC!). The JCR-ATOM love affair seems to have a new offspring.

It looks to me that this project might produce pieces of infrastructure that are useful far beyond the use case the project currently targets.

Peter Svensson has published a presentation about his ideas on thin server architectures (see also this post).

I am usually not exactly advocating Object Content Mapping in many of our internal discussions since it implies more structure.
People usually know me more as a "data first" guy.

Olafur Gauti Gudmundsson pointed me today to his effort called JCROM (pronounced "Jack-rom").

I am thrilled with the lightweight, annotation based approach he has chosen and I think it is great to have a very direct relation to JCR. I think the "2 Minute Intro" demonstrates the power and elegance of JCROM.

Anyway: Kudos to Gauti.

As I described in this post I use Jukka Zittings JCRStoreBean to write emails into my content repository. In this bean an incoming mail with only one recipient is stored with a single-valued property named "to" whereas mails with multiple recipients are stored in a multi-valued property named "to". This might cause problems for the consuming application because the API calls to read a single-valued or a multi-valued properties are different.

Of course, this can easily be fixed in the JCRStoreBean, but this fact can also be used as an excuse to further explore scripting a JCR with JRuby. The aim is to convert the "to" property into a multi-valued property for all mails that I have already stored in my CRX repository.

If you have set up JRuby like in this previous post you need to have the additional jar crx-rmi-1.3.2.jar file on your classpath to access a CRX repository through RMI. Find it in your CRX installation here server\runtime\0\_crx\WEB-INF\lib.

Afterwards, getting a connection to your CRX repository is simple:

require 'java'
include_class('java.lang.String') 
  {|package,name| "J#{name}" }
include_class 'javax.jcr.Repository'
include_class 'javax.jcr.SimpleCredentials'
include_class 'org.apache.jackrabbit.rmi.client.
  ClientRepositoryFactory'

factory = ClientRepositoryFactory.new 		
repository = factory.
  getRepository('//localhost:1234/crx')
session = repository.login(SimpleCredentials.
  new("admin", JString.new("adminpassword").
  toCharArray))
name = repository.
  getDescriptor(Repository::REP_NAME_DESC);
user = session.getUserID
puts "logged in as " + user + " in " + name

Running this like described here will yield something like

logged in as admin in CRX

The code should be straight-foward apart from the weird JString include which is also explained in my previous post. Now, for converting the "to" property from a single-valued to a multi-valued property I intend to

1. copy the value of "to" to a new multi-valued property "tom"
2. delete the single-valued "to"
3. create a new multi-valued "to"
4. copy the value of "tom" to "to"
5. and delete "tom" for all mail nodes

Deleting a node: null fun

Deleting a JCR property is (unfortunately) not done through a method like removeProperty(). Instead, you need to use setValue and pass null as a parameter. However, setValue is overloaded. So, in Java the code would look like:

mailNode.getNode("jcr:content").
  setProperty("tom", 
  new String[]{mailNode.getNode("jcr:content").
  getProperty("to").getString()});
mailNode.getNode("jcr:content").
  setProperty("to", (String)null);
mailNode.getNode("jcr:content").
  setProperty("to", 
  mailNode.getNode("jcr:content").
  getProperty("tom").getValues());
mailNode.getNode("jcr:content").
  setProperty("tom", (String[])null);

Note that in the 2nd and the 4th line the null is casted into a String and a String[] array respectively (to be honest, until recently I did not know that null can be casted). This needs to be done in order to let the compiler know which version of the overloaded method setValue to use.

If you want to do this in JRuby you need to map Ruby strings to Java strings, null to nil and Java arrays to Ruby arrays.

mail_node.getNode("jcr:content").
  setProperty("tom", 
  [mail_node.getNode("jcr:content").
  getProperty("to").getString].to_java(JString))
mail_node.getNode("jcr:content").
  setProperty("to", nil)
mail_node.getNode("jcr:content").
  setProperty("to", mail_node.
  getNode("jcr:content").
  getProperty("tom").getValues)
mail_node.getNode("jcr:content").
  setProperty("tom", [nil].to_java(JString))

If you look at the second line you note that nil is OK to pass instead of Java's (String)null. In the first and the forth line, however, you can see that JRuby's to_java method must be used to convert an array of Ruby strings to an array of Java strings (JStrings). R.J. Lorimer has written a tutorial on how to convert arrays between JRuby and Java.

The code above needs to be wrapped into a iterator loops (depending on how you structured your mail archive) which is quite simple, but for completeness here it is:

list = session.getRootNode.
  getNode("incubator-sling-dev")
month_iterator = list.getNodes
while month_iterator.hasNext
  month_node_iterator = month_iterator.
  nextNode.getNodes
  while month_node_iterator.hasNext
    mail_node = month_node_iterator.nextNode
    to = mail_node.getNode("jcr:content")
      .getProperty("to")
    if to.getDefinition.isMultiple
      puts mail_node.getPath+" is OK"
    else
      # code from above		
      session.save
    end
  end	
end

Adding a Sling Resource Type

In order to use Sling to render a JCR node it is often beneficial if the node has a "sling:resourceType" property set. Sling can also use the node type to find the appropriate rendering script. But in our case with unstructured nodes we need the resourceType property. So, let's loop over the mail nodes and set it.

Rather than invoking boring old setProperty on a node I would like to show how to utilize (J)Ruby's dynamic language features to achieve this goal. In Ruby you can dynamically extend a class. This even works for Java's built-in classes. For example:

class ArrayList
  def foo; "Hello!"; end
end
ArrayList.new.foo => "Hello!"

javax.jcr.Node is an interface. For interfaces the syntax looks slightly different:

JavaUtilities.extend_proxy("java.util.List") do
  def bar; "Goodbye!"; end
end
ArrayList.new.bar => "Goodbye!"

So for our mail nodes this looks like:

JavaUtilities.extend_proxy("javax.jcr.Node") do
  def resource_type=value
    self.getNode("jcr:content").
      setProperty("sling:resourceType", value)
  end
  def has_resource_type?
    begin
      self.getNode("jcr:content").
        getProperty("sling:resourceType").
	 length > 0
      rescue
        false
      end
    end
  end
end

resource_type=value is the setter method for the resource type. has_resource_type? returns a boolean value if the resource type is set already (Ruby method names can contain characters like "?"). This allows us to set the property "sling:resourceType" like this:

if !mail_node.has_resource_type?
  mail_node.resource_type = "email"
end

Ruby has also another trick up its sleeves that allows us to do funny things with JCR nodes: method_missing. Override the method_missing method to catch all method calls to undefined methods. Like this we can create a new setter that catches attempts to set a new property and write it down into our mail node as a string property. The definition looks like this:

JavaUtilities.extend_proxy("javax.jcr.Node") do
  def method_missing(name, *args)
    # get the name of the called method
    # and check if it is a setter
    name = name.to_s
    setter = /=$/ === name
    expected_args = 0
    if setter
      name = name[0...-1]
      expected_args = 1
    end
    # this is soemthing we do not want to catch
    unless args.size == expected_args
      err = "wrong number of arguments 
        (#{args.size} for #{expected_args})"
      raise ArgumentError.new(err)
    end
    # write it to the repository
    self.getNode("jcr:content").
      setProperty(name.to_s, args.to_java(JString))
    end	
end

In the above code name is the name of the variable the user wants to set. setter is a boolean that checks whether it is actually a setter method which is called (rather than another method). args is an array that contains the passed arguments. Armed with this extension we can add properties to the node like this:

mail_node.mynewprop = "myvalue"

Hmm, syntactic sugar.

If you are interested in more Ruby goodies: Daniel Spiewak has written a piece on JRuby scripting in Java projects and Ola Bini has a nice overview post on Ruby's meta programming techniques.