Java on Roland Huß

Elegant Camel route configuration

Tue, 03 Jul 2018 00:00:00 +0000

I’m a big fan of the Camel Java DSL for defining Camel routes with a RouteBuilder. This is super easy and slim. However, in this blog post I show you a nerdy trick how this can be done even more elegant.

If you are a Camel user, you know, that defining a route for a given Camel context ctx ist just a matter to implement the configure() method of the abstract RouteBuilder class:

ctx.add(new RouteBuilder {

 @Override
 public void configure() throws Exception {
 from("file:data/inbox?noop=true")
 .to("file:data/outbox");
 }

});

Its really simple and you can use the whole Camel machinery from within your configure() method.

However, this kind of configuration can be performed even simpler. Let’s assume that you have a no-op default implementation of RouteBuilder called Routes:

public class Routes extends RouteBuilder {
 @Override
 public void configure() throws Exception { }
}

Then, the configuration can be rewritten simply as

ctx.add(new Routes {{
 from("file:data/inbox?noop=true")
 .to("file:data/outbox");
}});

This trick just uses Java’s object initializers, a not so well known language feature. The inspiration for providing the DSL context like this comes from JMockit which defines its mock expectations the same way. I think object initializers are really an elegant albeit hipster way to implement DSLs.

Although you can easily define the Routes class on your own, you might vote for this Camel issue or pull request if you want to have this in upstream Camel, too.

Java EE Management is dead

Mon, 10 Oct 2016 00:00:00 +0000

Now that some weeks has been passed we all had time to absorb the revised Java EE 8 proposal presented at Java One. As you know, some JSRs remained, some things were added and some stuff was dropped. Java EE Management API 2.0, supposed to be a modern successor of JSR 77, is one of the three JSRs to be dropped.

What does this mean for the future of Java EE management and monitoring ?

First of all it’s fair to state that JSR 373 never really took off. Since February 2015 there were not more than 86 mails on the expert group mailing list, half of them written in March 2015 during incubation. At the latest of January 2016 it was clear that JSR-373 is not on Oracle’s focus anymore. To be honest, even we members of the expert group, we were not able to push this JSR further.

How did it come that far ? Let’s have a look back into history.

All starts with JSR 3 back in 1999. This first JMX specification is the foundation of all Java resources management. As it can been seen by its age, Java folks took care about Management and Monitoring from the very beginning on. And even better, since JS2E 5 JMX is integral part of Java SE so its available on every JVM out there.

Over the years, additional JSRs were added on top of this base:

JSR 160 defines a remote protocol for JMX,, which is based on RMI. This might have been a good decision in 2003, but turned out to be awful to use especially for non-Java based monitoring systems.
JSR 262 was started to overcome this by defined a “WebServices Connector for Java Management Extensions Agents” which was mostly around SOAP services. However although even an initial implementation existed, it was withdrawn before the final release. It’s not completely clear why it was stopped in 2008 and later withdrawn, as the public review ballot has been approved, although it was a tight result. The biggest objections were on dependencies on “proprietary” WS-* specifications.
“J2EE Management” JSR 77 was finished in 2002 and defines a hierarchy how Management and Monitoring resources exposed by a Java EE server is structured. It allows a uniform interface for how to access the various Java EE resources, like web applications or connector pools. Beside this it also defines how statistics are exposed by defining various metrics formats. However, implementing the StatisticsProvider model is not mandatory and from my personal experience it was implemented only rarely by some vendors and if so, not for every resource.
JSR 88 complements JSR 77 and defines a common format for deploying Java EE artefacts.
JSR 255 was started to be the next version of JMX and supposed to be included in Java 7. Although it was already nearly finished and integrated, it didn’t make it into Java 7 (nor Java 8). The spec was then dormant until it was finally withdrawn in 2016.

With the dead of JMX 2.0 in 2009 the evolution of JMX as a standard for Java SE has stalled. But what’s about Java EE Management ? At least JSR 77 is still part of Java EE 7 and for Java EE 8 the successor was supposed to be JSR 373. JSR 373 tackles the problem of remote access, whereas JSR 77 still relies on RMI as a standard implementation protocol as defined in JSR 160.

The two major goals of JSR 373 were:

Provide an update of the hierarchal resource and statistics structure as defined by JSR 77
Provide a REST access to these resources independent of JMX

In the often cited Java EE 8 Community Survey more than 60% were in favour of defining a new API for managing application, which should be based on REST (83% pro-votes). This finally lead to JSR 373. However, as it seems in retrospective, a deep interest in this topic was not really given and probably lead to this final decision to drop JSR 373 from Java EE 8.

So, what is the state of Monitoring and Management of Java and in particular Java EE applications nowadays and what can be expected in the future ? Let’s have a look into the crystal ball.

JMX is here to stay. It is part of Java SE and I don’t know of any plans for removing it from future Java editions. Ok, it feels a bit rusty but it is still rock solid and gives you deep insight in the state of your JVM. With tools like Jolokia you can overcome most of the restrictions JSR 160 imposes. (Disclaimer: since I’m the author of Jolokia all my personal opinions given here should be evaluated in this light :)
It is not clear how the Management API of Java EE 8 and beyond looks like. It does not look like that JSR 77 will survive. Will there be a standard for Java EE management at all ? Probably not, and so there is the danger that vendors will push their proprietary management APIs, which already happens to some extent. Luckily, most of these proprietary APIs are also mirrored in JMX these days.
On the other hand, it could be also a good thing that there is no other Management API which is not based on JMX. That’s because you will always need JMX to monitor the basic aspects like Heap Memory usage or Thread count, which are covered by Java SE. Adding a different, REST like protocol for Java EE monitoring requires operators to access a Java EE server with two different protocols (JMX and Rest), duplicating configuration efforts on the monitoring side. This can only be avoided if the Java EE resources are mirrored in JMX, too.

To sum it up, I think its a shame that Management and Monitoring, which played a prominent role over the whole evolution of Java EE, will probably be dropped completely in Java EE 8. As a replacement the new Health Check API has been announced, but to be honest, that can’t be a full replacement for classical management and monitoring where the evaluation of a system’s health is done on a dedicated monitoring platform (e.g. like Nagios or Prometheus). These platforms take the plain metrics data exposed by the application and does the data evaluation on their own.

The good thing is still that you have JMX to the rescue and I’m pretty sure that this technology will survive also this storm. Especially if vendors are willing to support it for their application server metrics, too.

Even without a Java EE standard.

Docker for (Java) Developers

Mon, 25 Aug 2014 00:00:00 +0000

Recently I gave a Meetup talk for the Docker Munich Meetup Group which explained how Docker can help developers to improve integration tests and to ship applications.

The slides are online as well as the demo project.

During the demo I used Butterfly for an in-browser shell, which was quite cool, I guess ;-) (This is obviously not enabled in the online slides).

I’m going to continue to celebrate my Docker-♡ with another two talks in autumn:

Boosting your developer toolbox with Docker at JBoss One Day Talk, September, 29. in Germering (Munich, Germany)
Docker für Java Entwickler (in german) at W-JAX 14, November, 3. - 7. in Munich

And there is a slight chance (since the CFP has not yet been declined ;-) to talk at Devoxx about Docker. JavaZone unfortunately declined my CFP, I guess there are already too many riding the docker horse (which is a good thing).

Nevertheless I will attend both conferences with talks about Jolokia, and I’m really looking forward to it.

‘guess it will become a hot autumn (hotter than this german 2014 summer for sure) ….

Removing attachments with JavaMail

Mon, 29 Mar 2010 00:00:00 +0000

If you have ever sent or received mail messages via Java, chances are high that you have used JavaMail for this task. Most of the time JavaMail does an excellent job and a lot of use cases are described in the JavaMail FAQ. But there are still some additional quirks you should be aware of when doing advanced mail operations like adding or removing attachments (or “Parts”) from existing mails retreived from some IMAP or POP3 store. This post gives a showcase for how to remove an attachment from a mail at an arbitrary level which has been obtained from an IMAP store.

It points to the pitfalls which are waiting and shows some possible solutions. The principles laid out here are important for adding new attachments to a mail as well, but that’s yet another story.

JavaMail objects
#

Before we start manipulating mail messages it is important to understand how these are represented in the JavaMail world.

The starting point is the Message. It has a content and a content type. The content can be any Java object representing the mail content, like a plain text (String) or raw image data. But it can also be a Multipart object: this is the case when a message’s content consists of more than a single item. A Multipart object is a container which holds one ore more BodyPart objects. These BodyParts, like a Message, have a content and a content type (in fact, both Message and BodyPart implement the same interface Part which carries these properties).

Beside plain content, A BodyPart can contain another Multipart or even another Message, a so called nested message (e.g. a message forwarded as attachment) with content type message/rfc822.

As you can see, the structure of a Message can be rather heterogenous, a tree with nodes of different types. The following picture illustrates the tree structure for a sample message.

This object tree can be navigated in both directions:

getContent() on Parts like Message or BodyPart to get to the child of this node. The return type is a java.lang.Object and in case of a plain BodyPart can be quite huge. Before calling Part.getContent() be sure to check whether it contains a container by checking for its content type via Part.isMimeType("multipart/*") or Part.isMimeType("message/rfc822")
getParent() on Multipart or BodyPart returns the parent node, which is of type BodyPart. Note that there is no way to get from a nested Message to its parent BodyPart. If you need to traverse the tree upwards with nested messages on the way, you first have to extract the path to this node from the top down. E.g. while identifying the part to remove you could store the parent BodyParts on a stack.

First approach
#

Back to our use case of removing an attachment at an arbitrary level within a mail. First, a Message from the IMAP Store needs to be obtained, e.g. by looking it up in an IMAPFolder via its UID:

Session session = Session.getDefaultInstance(new Properties());
Store store = session.getStore("imap");
store.connect("imap.example.com",-1,"user","password");

IMAPFolder folder = (IMAPFolder) store.getFolder("INBOX");
IMAPMessage originalMessage = (IMAPMessage) folder.getMessageByUID(42L);

Next, the fetched message is copied over to a fresh MimeMessage since the IMAPMimeMessage obtained from the store is marked as read-only and can’t be modified:

Message message = new MimeMessage(originalMessage);

// Mark original message for a later expunge
originalMessage.setFlag(Flags.Flag.DELETED, true);

Now the part to be removed needs to be identified. The detailed code is not shown here, but it is straight forward: You need to traverse the cloned Message top down to identify the Part, e.g. by its part number (a positional index) or by its content id. Be careful, though, not to call getContent() except for BodyParts of type multipart/* or message/rfc822, since this would trigger a lazy fetch of the part’s content into memory. Probably not something you want to do while looking up a part. I think, I already said this. ;-)

MimePart partToRemove = partExtractor.getPartByPartNr(message,"2.1");

It’s time to remove the body part from its parent in the hierarchy and store the changed message back into the store. You can mark the original message as DELETED and expunge it on the folder. If you have the UIDEXTENSION available on your IMAP store, you can selectively delete this single message, otherwise your only choice is to remove all messages marked as deleted at once (“Empty Trash”).

Multipart parent = partToRemove.getParent();
parent.removeBodyPart(partToRemove);

// Update headers and append new message to folder
message.saveChanges();
folder.appendMessages(new Message[] { message });

// Mark as deleted and expunge
originalMessage.setFlag(Flags.Flag.DELETED, true);
folder.expunge(new Message[]{ originalMessage });

We are done now.

But wait, that’s not good enough …
#

If you try the code above, you will probably be a bit surprised. If you fetch back the newly saved message from the folder, you will find that the attachment has not been removed at all.

Dragons are waiting here.

The problem is that a JavaMail Part does heavy internal caching: it keeps a so called content stream until a new content is set for it. So even if you modify the hierarchy of objects as described above, the original content is kept until you update the content of the parents yourself and the cache is thrown away. Our part has not been removed because the cached content stream has not yet been invalidated. The solution is to get rid of the cached content stream (aka ‘refresh the message’). You could set the content directly via Part.setContent(oldPart.getContent(),oldPart.getContentType()), but this is dangerous in so far as it will load the part content into memory. (Did I already mention this?) That’s really not something you are keen on if you want to remove this Britney Spears Video to save some IMAP space. The alternative is to work on the wrapped DataHandler only. A DataHandler (defined in Java Activation) is not much more than a reference to the content stream. Setting the DataHandler on a Part via Part.setDataHandler() also causes it to invalidate its cached content, so a later Part.writeTo() will stream out the new content. Unfortunately, this has to be done on every parent up to the root. A brute force solution is to start from the top and refresh every content with

// Recursively go through and save all changes
if (message.isMimeType("multipart/*")) {
 refreshRecursively((Multipart) message.getContent());
}
Multipart part = (Multipart) message.getContent();
message.setContent(part);
message.saveChanges();

...

void refreshRecursively(Multipart pPart)
 throws MessagingException, IOException {

 for (int i=0;i<pPart.getCount();i++) {
 MimeBodyPart body = (MimeBodyPart) pPart.getBodyPart(i);
 if (body.isMimeType("message/rfc822")) {
 // Refresh a nested message
 Message nestedMsg = (Message) body.getContent();
 if (nestedMsg.isMimeType("multipart/*")) {
 Multipart mPart = (Multipart) body.getContent();
 refreshRecursively(mPart);
 nestedMsg.setContent(mPart);
 }
 nestedMsg.saveChanges();
 } else if (body.isMimeType("multipart/*")) {
 Multipart mPart = (Multipart) body.getContent();
 refreshRecursively(mPart);
 }
 body.setDataHandler(body.getDataHandler());
 }
}

However, we can be smarter here: Since we already identified the part to remove, we can make our way upwards to the root message via the getParent() method on Multipart and BodyPart (which, by the way are not connected via any interface or inheritance relationship).

BodyPart bodyParent = null;
Multipart multipart = parent;
do {
 if (multipart.getParent() instanceof BodyPart) {
 bodyParent = (BodyPart) multipart.getParent();
 bodyParent.setDataHandler(bodyParent.getDataHandler());
 multipart = bodyParent.getParent();
 } else {
 // It's a Message, probably the toplevel message
 // but could be a nested message, too (in which
 // case we have to stop here, too)
 bodyParent = null;
 }
} while (bodyParent != null);

Finally you need to update the uppermost message headers, too with a

MimeMessage.saveChanges()

As you might have noticed, this works as long as there is no nested message in the chain of BodyParts up to the root. Since a Message doesn’t have any parent, we need some other means to get the BodyPart which is the parent of an enclosed Message. One way is to keep track of the chain of parent BodyParts when identifying the part to remove e.g. by extending the part extractor to support a stack of parent BodyParts which will be in:

Stack<MimeBodyPart> parentBodys = new Stack<MimeBodyPart>();
MimePart partToRemove = partExtractor.getPartByPartNr(message,"2.1",parentBodys);
 ....

This example could be extended to remove multipart containers on the fly, if only one part is left after removal and replace the multipart with its then last child or remove an empty multipart altogether when its last child has been removed.

Summary
#

Hopefully, I could sketch out that there are several points to take care of when manipulating existing JavaMail Messages (it’s not that difficult if you build up one from scratch). The code shown above is only a starting point, but it hopefully saves you some time when you start wondering why on earth your nicely trimmed message isn’t stored correctly on the IMAP store.

Java on Roland Huß

Elegant Camel route configuration

Java EE Management is dead

Docker for (Java) Developers

Removing attachments with JavaMail

JavaMail objects #

First approach #

But wait, that’s not good enough … #

Summary #

JavaMail objects
#

First approach
#

But wait, that’s not good enough …
#

Summary
#