“it could be bunnies”

During SoTR 2011 I presented on using XFA PDF forms (a.k.a. LiveCycle forms) with ColdFusion. Slides and materials are now available for download.

A after-conference addition is that Chandan Kumar from Adobe confirmed that the issue with overwritedata=”yes” in the cfpdf tag is resolved in ColdFusion so you don’t need to add it to all cfpdf populate operations anymore once the fix is released / installed.

Last week I had the second Flex 4 Crash Course session at the Adobe office in Amsterdam as an introduction to Flex for people with no previous Flex experience. (Although there were some familiar faces in the audience.) The training material was provided by Adobe and I am not allowed to publish all the originals, but I can share the slides with all the links to external resources.

flex4_crash_course_slides_extract

In a recent discussion on cf-talk the question was asked how to improve the performance of ColdFusion when working with very large XML documents. One of the solutions proposed was to use StAX and that got me thinking. StAX is a stream processor works very different from what you may be used to from other XML processors. Instead of viewing an XML document as a whole and elements in context to their parents, children and siblings, it just treats the whole document as a sequence of items. Each of these elements can be of type elementstart, elementend, comment, entity etc. The way you work with this is you iterate through all the items in your document and process them one by one. Working that way is sufficiently different to make it necessary to rewrite all your processing from scratch if you want to switch from the built-in processor to StAX which makes it a solution that is not so attractive.

But what if we combine a preprocessing step in StAX to split the large XML document into smaller pieces with the regular processing in ColdFusion? StAX is Java so it is easy to integrate it into ColdFusion and to test this I wrote a sample implementation to test if this would help. It has some limitations such as only handling elements, element text and attributes, but it seems to work just fine (and the code is open for improvement). With this I benchmarked some XML files I downloaded from internet with the following results:

As you can see how you are splitting a document has a significant impact. I presume this is mostly due to the impact the write operations have on my laptop with a slow 5400 rpm harddisk. On the other hand in the best case scenario the parsing speed is over 4 MB per second. Memory consumption stayed under 200 MB for the whole server so it looks like there are some scenario’s where this might be useful.

Code for xmlSplitter.cfc, tested on CF 9.01, 64-bit with StAX 1.2.0 and Java 1.6u24 64-bit.

Product:               Seapine TestTrack Pro
Vulnerable versions:   2010.x, 2011.x
Vulnerability:         predictable session cookies
Vendor informed:       2010-09-07
Fix available:         no

Info:
TestTrack Pro is an issue tracking application from Seapine

Vulnerability:
TestTrack Pro offers a SOAP interface which works as follows:
- connect with username and password to retrieve a list of available
  projects: getProjectList(username, password);
- connect with username and passsword to retrieve a session login cookie
  on a project: projectLogon(project, username, password);
- query the system to retrieve project data using the session login
  cookie to authenticate: getRecordListForTable (cookie, .....);
- log off the session: databaseLogoff(cookie).

The session login cookies generated by the server are predictable. Below
is a log file from the connections showing the date and time of a log
entry, and then the cookie used for authentication:
"09/07/10","11:18:19","1246111"
"09/07/10","11:18:22","1246115"
"09/07/10","11:18:44","1246123"
"09/07/10","11:18:46","1246127"
"09/07/10","11:18:51","1246132"
"09/07/10","11:18:53","1246139"
"09/07/10","11:19:16","1246144"
"09/07/10","11:19:18","1246151"
"09/07/10","11:19:33","1246156"
"09/07/10","11:19:35","1246163"
"09/07/10","11:19:51","1246167"
"09/07/10","11:19:53","1246175"

The absolute value of the session cookie is related to the server
uptime, starting near 0 when the server is just started and increasing
monotonic afterwards.

History:
2010-09-07 Seapine was informed and assigned case number 121426
2010-09-08 Seapine confirmed the issue as a known issue and scheduled a
           fix in 'an upcoming 2011.0.x maintenance release'.
2010-12-20 TestTrack 2011.1 was released without a fix.
2010-12-24 Seapine was asked to publish a security bulletin detailing
           risks and mitigations despite no fix being availale
2011-02-02 Seapine was informed this issue would be publicly disclosed
2011-02-13 Submitted to bugtrack and published on my blog

One of the ways to translate LiveCycle Designer PDFs is using XLIFF. Since an XDP file is essentially an XML file the process is to run an XSLT transformation over the XDP to extract all the text strings. Not just the captions, but also all the tooltips, screenreader texts, image alt’s etc. Then you can send the generated document with strings to a translator, and some time later you get a translation back that you can merge into your template using a different XSLT transformation. The whole process is described pretty well in Using XLIFF for translating Adobe® LiveCycle® Designer ES form designs on Adobe DevNet.

While putting together a few extra exercises for the LiveCycle ES2 Designer Specialist training I stumbled over something weird in the XSLT templates from Adobe. While the tools for translating XLIFF files all presume the original string is in the source element and the translated string is in the target element, the Adobe XSLT templates presume the translated string is put back into the source element. So for an English to Dutch translation tools create:

<trans-unit id="09DD30F3-CE03-4FF1-92D3-067126FF904E" resname="09DD30F3-CE03-4FF1-92D3-067126FF904E">
    <source>Awesome!</source>
    <target>Keigaaf!</target>
</trans-unit>

But the XSLT templates Adobe distributes with LiveCycle Designer (look in the%programfiles%\Adobe\Adobe LiveCycle Workbench ES2\Adobe LiveCycle Designer ES2\FormTranslation\ folder) expect:

<trans-unit id="09DD30F3-CE03-4FF1-92D3-067126FF904E" resname="09DD30F3-CE03-4FF1-92D3-067126FF904E">
    <source>Keigaaf!</source>
</trans-unit>

Luckily this is easy to fix (once you figure out what is going on) by changing the XSLT template for merging the translations back into the XDP file to select the target node instead of the source node at line 66 of mergestrings.xslt:

<xslt:variable name="translatedNode" select="$s2x[@id=$idToGet][1]/target" />

A version of mergestrings.xslt with this change is available for download.

This was written in response to a forum question, but I figured it might be useful for more people.

There are many ways to deploy ColdFusion code to a server. Probably the most prevalent, especially considering shared hosting, is using FTP to upload CFML templates to the server.  Tools such as DreamWeaver and CFBuilder allow you to do so right from your IDE. Another way to do it is to run some Ant script or batch file and extract the sources straight from source control to the server. With a little bit of effort you get much more control and much more reproducible results.  At Prisma IT we prefer to go a step further and deliver the ColdFusion applications we build as Enterprise ARchives (EARs) to our clients.  This allows us even more control, especially when we don’t have any.

Let me explain that a bit. We have several clients where we do all their development, but final deployment is done on the clients infrastructure. If we are lucky, we may have read-only access to the User Acceptance Testing servers, but sometimes we don’t even have that. In those cases deploying an application is completely up to the client (or their hosting partner). That leaves us no wiggle room to deal with stuff that could go wrong during a deployment. With EAR files we eliminate a huge number of risks from the process. An EAR file is a full application, so there is no risk that some files get forgotten. And we have the MD5 to prove it. Since it gets deployed to its own temporary folder, there is no chance of any old files remaining on the server ans slipping in to the server (the cfclasses folder is famous for that).

The one thing you need to solve for this is how to configure your application. If the client had to log in to the ColdFusion Administrator after deploying the application to configure datasources, mappings etc., it would be just as easy to do something wrong. So what we do instead is to have the client place a properties file on the class path with a bunch of configuration settings. Standard ones, such as the IP address of the outgoing mail server and the folder for logfiles, and application specific ones such as the location on the SAN where all the documents are stored. Then in the onApplicationStart() the application parses that and configures itself. Each of these settings is checked when it is loaded into the application, so if there is a path configured, a directoryExists() wil make sure it actually exists.

The added benefit is that it becomes very easy to move an application around. Once you have written your properties files for test, QA and production, they stay the same. You just move an EAR file with a release from one environment to the next and it configures itself as soon as it starts. The EAR files themselves get generated by Ant on our build server to make sure they are completely reproducible. And to protect our intellectual property and deter others from mucking around in them, they only contain compiled source code. And since an EAR is a standard format, it works on different JEE servers too. (Mostly JRun and occasionally JBoss for us.)

As any solution, this process has downsides. Working with compiled EAR files is obviously not a good idea if you push small changes to a live server three times a day. It is a very ‘heavy’ process, because in each EAR you are packaging ColdFusion as well (100+ MB). And building EAR files without a ColdFusion Administrator does not just mean the client can not mess the configuration up anymore, it also means you can not fix the configuration anymore either. But all in all, it is serving us well.

Since I am in Bangalore for a training I dropped in on the “Adobe Flash Platform Tools Preview” this evening. The agenda promised short sessions on Flash Builder 4, Flash Catalyst and LiveCycle ES, followed by food and networking. The Flash Builder (previously known as Flex Builder) session was solid. It showed Data Centric Development, where services are defined in Flash Builder and can then easily be wired into the UI because all the code for the services and value objects is generated. (See Raghu’s blog for the demo screencast). Next up was Flash Catalyst, showing a design - development workflow where a .psd file was transformed to a .fxp, which was then imported in Flash Builder to wire the data in through the new services management. Last was LiveCycle ES. Unfortunately, but understandably considering the audience, this was all about data management and not process management. What was new for me was that apparently this now wires directly into Hibernate so you don’t need to write any server side code anymore, you can have everything generated.

The Q&A focused mainly on the designer - developer workflow with Flash catalyst. The main question that was repeated several times in different words was whether this workflow put any additional constraints on the designer. And each time the answer was that good development on the design side, including the judicious use of layers, was all it took. I think this reflected the audience of architects and project managers, from developers I would expect more technical oriented questions.

Afterward the food and networking were great. Not just the Flash team from Adobe was there, but also people from the LiveCycle team and the ColdFusion team., so I got an opportunity to thank some people in person for fixes and new features I am not allowed to mention yet. And I also met up with some of the people we do business with in India.

Every year we all go on a trip together for a few days with everybody from Prisma IT. This year the trip is to Iceland. This is all of us having lunch at Schiphol airport before boarding.

Prisma IT having lunch at Schiphol airport

So for the next two days we will all be out of office. Well, not really all of us, Richard is staying in the Netherlands to answer the phone …

I am currently in the UK for a LiveCycle training and when I looked out of my hotel window the view was very nice.

Winter view from Fleet (UK)

Then just before I was going for breakfast my boss called and asked whether I still was in the hotel. Then he told me to “Go make snow dolls”. Apparently the snow is a bit more then the people in the UK are used to and is causing major travel issues. The students are from all over the country (apparently some had to drive 4 hours under normal conditions), and due to the weather the first day of the training is canceled. So now I have my first snow day ever! (In the Netherlands we only have ice days to go ice-skating, when it has frozen enough and the ice is thick enough, but no snow days.)

UPDATE: training cancelled for the whole week based on the weather forecast, but no way to go home since all flights to Amsterdam have been cancelled at Heathrow.

Today was the CF Insider Workshop in Brussels. Line-up was the same as last week in Amsterdam, Claude Englebert from Adobe and Simon Slooten and me from Prisma IT. Mark van Hedel was supposed to join us for the ColdFusion and AJAX session, but he broke his leg last week so he couldn’t be there and Simon took over his session. The event in Amsterdam was pretty well attended with 20+ attendees, in Brussels we had a few less (but they have another event in French tomorrow). But the good news is that after years of a slumber Adobe is finally talking about ColdFusion in Europe again. Some of the attendents were even Adobe employees from other departments who wanted to know what that ColdFusion thing was about.

I have uploaded my slides from the CF Insider workshop to the Prisma IT website:

What’s new in ColdFusion 8

The ColdFusion Server Monitor