Jamie Craane's Blog

Monitoring an IBM JVM with VisualVM

2012-01-23T20:48:00.002+01:00

JDK6 update 7 and onward include a tool called VisualVM. VisualVM is a visual tool with monitoring and profiling capabilities for the JVM. With VisualVM you can:

Monitor heap usage
Monitor CPU usage
Monitor Threads
Initiate garbage collections
Profile CPU and memory
And more…

Although VisualVM is distributed with the Oracle JDK, it can also be used to monitor IBM JVM’s. VisualVM is not able to connect to the IBM JVM locally. JMX must be used instead.

To enable JMX monitoring on the IBM JVM open the WebSphere administrative console and:

navigate to: Server -> Server Types -> WebSphere application servers ->[SERVER_NAME]
Expand Java and Process Management and click Process definition
Click Java Virtual Machine
In the Generic JVM arguments field append the following properties: -Djavax.management.builder.initial= -Dcom.sun.management.jmxremote -Dcom.sun.management.jmxremote.authenticate=false -Dcom.sun.management.jmxremote.ssl=false -Dcom.sun.management.jmxremote.port=1099
Restart the server

To start monitoring the JVM with VisualVM start VisualVM by navigating to [JDK_HOME]\bin and start jvisualvm.exe (please note that when VisualVM is downloaded as a separate package the executable is called visualvm.exe instead). In VisualVM click File -> Add JMX Connection. Specify localhost:1099 in the connection field and click OK.

If everything went OK, you should see the localhost:1099 connection under the Local node in the tree on the left. Double-click this node to start monitoring. See the following screenshot for an example:

When using a JMX connection to monitor the JVM please be aware that not all functionality can be used compared to monitoring a local JVM. Profiling memory is for example not possible.

The above configuration was tested on:

Windows 7 64-bit
IBM JDK1.6 64 bit
WebSphere Application Server version 7.0

Note: Before JDK6 update 7, VisualVM can also be downloaded separately from http://visualvm.java.net/download.html

Note: To actually test if port 1099 is listening for connections use (on Windows) the netstat –a command and check wether the port is present and listening.

Improved IntelliJ keymap

2011-12-28T20:05:00.001+01:00

At my current client the whole team recently switched from an Eclipse based IDE to IntelliJ. Although some of us have used IntelliJ for a long time, most had not. Switching from Eclipse to IntelliJ can be hard. It took me about three to four weeks before I was more productive with IntelliJ. Now I do not want to go back anymore.

To make the transition more smooth I am looking into several ways to accomplish this. One of the things I made is a new keymap reference for IntelliJ. The default keymap reference contains a whole list of shortcuts but I find them sometimes hard to find. The new, improved, keymap should make the most used IntelliJ features really easy to find. The most used features are based upon my own experience and the IntelliJ productivity guide of my colleagues. The productivity guide shows which features of IntelliJ are used most often.

To create the improved keymap reference I basically did the following:

Stripped about half of the features from the default keymap reference. The features present on the new keymap reference are the most used features.
Made a clearer distinction between the different types of functionality the features belong to.
In each feature group I made a distinction between individual features which belong together.
Used a larger font and more contrast to make things stand out more.

The Windows keymap reference can be found here: IntelliJ Keymap Windows

The OSX keymap reference can be found here: IntelliJ Keymap OSX

Hope you like it.

If you have any suggestions for the keymap reference please let me know.

WebSphere Portal 6.x Locale Resolution

2011-08-02T20:21:00.002+02:00

When developing applications for WebSphere Portal there will be some point in time that the application should support I18N. For every language an application supports a resource bundle is provided. If a given language is not supported the application falls back to its default resource bundle. To format currencies and dates the JSTL formatting tags can be used. Based on the locale selection process of WebSphere Portal, the translation of dates and currencies in the correct locale may not what be exactly what you expect/want. This article provides a portal wide solution to handle locales in a consistent and transparent way.

Language selection process
WebSphere Portal server 6.x uses the following language selection process taken from the portal documentation:

The language encoded in the URL takes highest priority. The portal does not encode a locale into the URL by default.
If the user has logged in, the portal displays in the preferred language selected and stored in the user repository by the user.
If no preferred user language can be found, the portal looks for the language defined in the user's browser. If the portal supports that language, it displays the content in that language. If the browser has more than one language defined, the portal uses the first language in the list to display the content.
If no browser language can be found, for example if the browser used does not send a language, the portal resorts to its own default language.
If the user has a portlet that does not support the language that was determined by the previous steps, that portlet is shown in its own default language.

By looking at step 3, the portal should accept the browser language if it supports this language. The same is true for the portlet itself. Unfortunately, it is not clear how the portal determines if a given language is supported or not. The documentation is not clear about this either.

By executing extensive tests (removing languages with XML access and removing the resource bundles of those specific language from portal) with different locales sent from the browser, the portal always took the language from the browser if the user did not have a specific language set in its user profile. This is not an issue with the resource bundles since the application falls back to its default resource bundle if the language is not supported. The JSTL tags however did not use the proper language which resulted in dates and currencies formatted in the locale of the browser. Since we wanted tight control of the language used, this behaviour was not an option for us. We wanted a Portal wide solution which did not impact application design/implementation.

The portal documentation mentions something about a locale filter. Portal itself provides two implementations of this filter: ExtendedLocaleFilter and SortingLocaleFilter. It is also possible to implement a custom filter.

The code of the provided locale filters can be found in C:\IBM\WP61\PortalServer\base\wp.engine.impl\shared\app\wp.engine.impl.jar file.

Decompiling those filters you can see that the implementation reflects the above language selection process. The filters use the Puma API to determine the user’s locale. Also, by looking at this implementation you get a good idea of how to efficiently build a custom locale filter.

The following filter is an example filter which always fixes the locale to en_US regardless of the user profile or browser setting.

   1:  public class FixedLocaleFilter implements Filter {

   2:      public void init(final FilterConfig aFilterConfig) throws ServletException {

   3:      }

4:

   5:      public void doFilter(final ServletRequest aServletRequest, final ServletResponse aResponse, final FilterChain aFilterChain) throws IOException, ServletException {

   6:          if (aServletRequest instanceof HttpServletRequest) {

   7:              aFilterChain.doFilter(new FixedLocaleHttpServletRequest((HttpServletRequest) aServletRequest), aResponse);

   8:          } else {

   9:              aFilterChain.doFilter(aServletRequest, aResponse);

  10:          }

  11:      }

12:

  13:      public void destroy() {

14:

  15:      }

  16:  }

17:

  18:  public class FixedLocaleHttpServletRequest extends HttpServletRequestWrapper {

  19:      private static final Locale EN = new Locale("en", "US");

  20:      private static final Vector<Locale> LOCALES = new Vector<Locale>() {{

  21:          add(EN);

  22:      }};

  23:      private static final Vector<String> ACCEPT_LANGUAGE_EN_US = new Vector<String>() {{

  24:          add("en-US");

  25:      }};

26:

  27:      public FixedLocaleHttpServletRequest (HttpServletRequest request) {

  28:          super(request);

  29:      }

30:

  31:      @Override

  32:      public Locale getLocale() {

  33:          return EN;

  34:      }

35:

  36:      @Override

  37:      public Enumeration getLocales() {

  38:          return LOCALES.elements();

  39:      }

40:

  41:      public String getHeader(String header) {

  42:          if (isAcceptLanguageHeader(header))

  43:              return "en-US";

  44:          else

  45:              return super.getHeader(header);

  46:      }

47:

  48:      public Enumeration getHeaders(String s) {

  49:          if (isAcceptLanguageHeader(s))

  50:              return ACCEPT_LANGUAGE_EN_US.elements();

  51:          else

  52:              return super.getHeaders(s);

  53:      }

54:

  55:      private boolean isAcceptLanguageHeader(String s) {

  56:          return s.intern() == "Accept-Language" || s.equalsIgnoreCase("Accept-Language");

  57:      }

  58:  }

The above FixedLocaleFilter just wraps the incoming request in a FixedLocaleHttpServletRequest. The next step is to configure this filter in the web.xml of the Portal application web.xml. The web.xml can be found in wp_profile_root/config/cells/node-name/applications/wps.ear/deployments/wps/wps.war/WEB-INF.

See the following example configuration:

   1:  <filter>

   2:          <filter-name>Locale Filter</filter-name>

   3:          <!--<filter-class>com.ibm.wps.engine.ExtendedLocaleFilter</filter-class>-->

   4:          <filter-class>FixedLocaleFilter</filter-class>

   5:      </filter>

Please note that the above sample code is just an example unless you want a fixed locale of en_US for the whole profile where the portal is deployed. A practical scenario would be a custom filter which uses the following language selection process:

If the user has set a locale in its profile use that one.
If the user has not set a locale use the browsers locale if the locale is supported.
If the browser locale is not supported use the default locale of the portal.

The above steps can easily be implemented using a custom locale filter. You can also use init parameters to make the filter configurable.

Conclusion

Letting all applications in a WebSphere portal behave the same regarding locale resolution can be hard/impossible if you do not want to implement custom locale handling code within the applications itself. By using a custom locale filter in WebSphere portal, the locale resolution code is kept in one place. This makes locale resolution transparent and clear to all applications running in the portal.

search engine meta information

2011-06-06T20:37:00.001+02:00

Microsoft, Google and Yahoo, the companies behind the three major search engines Bing, Yahoo and Google, have launched a new initiative to add meta data to an HTML document. This meta data is used by the aforementioned search engines to improve the displaying of search results.

Consider the following example:

<html>
    <div class="product">
        <!-- product information comes here. -->
    <div>
</html>

The reader of this HTML fragment immediately sees that this is something about product information. A computer program, like a search engine instead, cannot determine that this information is actually about products.

To help search engines, and potentially other software, understand the meta information behind the information, http://www.schema.org describes a shared vocabulary which can be used to add meta data to HTML tags.

Consider the revisited example:

<html>
    <div class="product" itemscope itemtype="http://schema.org/Product">
        <!-- product information comes here. -->
    <div>
</html>

By adding the itemscope attribute, you are telling the information between the div element is about a particular item. The itemtype attribute narrows this item further as a Product. This way, search engine can potentially add this information to an online product catalog making your products easier to find. Other types are available as wel, for example:

Movie
Event
Organisation

I implemented this meta tags for a new webshop I am building. These attributes are part of the HTML5 standard and therefore cannot be used by an XHTML1.1 strict syntax without passing the W3C validation. To use these attributes, just declare your document as HTML5. You can then use the W3C validator to fix any validation errors which may arise. I think this is worth the exercise.

One step further to the semantic web.

Deploying Hippo on JBoss 4.2.3GA and PostgresQL 8.3

2011-01-21T21:31:00.005+01:00

For a client of us we developed a public website with the Hippo CMS. Hippo CMS is a Java based Content Management System based on the Content Repository API. Hippo runs standard on Apache Tomcat and Mysql. We wanted to deploy Hippo on JBoss 4.2.3GA with Postgres 8.3. Below is a step-by-step guide to deploy Hippo 7, the cms.war artifact, on Jboss with Postgres 8.3.

1. Create a PostgresQL database
Before you can deploy the cms.war file a PostgresQL database must be created. Hippo demands a database encoding of Latin1. If your PostgresQL database server is installed with a different encoding, excluding C or POSIX, you may not be able to create a Latin1 encoded database. If this is the case you can do one of the following:

Re-install the database server with a Latin1, C or POSIX locale. By using the C or POSIX locale you can create a database with any encoding you like, including Latin1.
Create a database with an encoding the current database server supports, for example UTF-8, and change the encoding of the database with the following SQL statement: ALTER DATABASE dbX SET client_encoding TO latin1; Make sure you restart the database server.

2. Add the postgress JDBC-driver to the /server/default/lib

3. Create a datasource
Create a datasource in Jboss by creating a postgres-ds.xml in the /server/default/deploy directory with the following contents:

<?xml version="1.0" encoding="UTF-8"?>
<datasources>
    <local-tx-datasource>
        <jndi-name>jdbc/repositoryDS</jndi-name>
        <connection-url>jdbc:postgresql://localhost:5432/hippo</connection-url>
        <driver-class>org.postgresql.Driver</driver-class>
        <user-name>postgres</user-name>
        <password>postgres</password>-->
        <blocking-timeout-millis>5000</blocking-timeout-millis> 
          <idle-timeout-minutes>5</idle-timeout-minutes> 
     <prepared-statement-cache-size>50</prepared-statement-cache-size>
       </local-tx-datasource>
</datasources>

4. Add a resource-ref to the cms.war web.xml

Add the following resource-ref to the web.xml of the cms.war file:

 <resource-ref>
        <res-ref-name>jdbc/repositoryDS</res-ref-name>
        <res-type>javax.sql.DataSource</res-type>
        <res-auth>Container</res-auth>
    </resource-ref>

You can add this element just before the closing </web-app> element.

5. Add a jboss-web.xml file to root of the WEB-INF directory of the cms.war

The jboss-web.xml defines the mapping between the datasource and the resource-ref. This file should have the following contents:

<?xml version="1.0" encoding="UTF-8"?>
<jboss-web>
<resource-ref>
  <res-ref-name>jdbc/repositoryDS</res-ref-name> 
  <jndi-name>java:/jdbc/repositoryDS</jndi-name> 
</resource-ref>
</jboss-web>

6. Modify the location of the repository.xml in the web.xml of the cms.war

We have to create a custom repository.xml file which configures Hippo for using PostgresQL. The location of the repository path should be modified so that it can find out repository.xml file. Change the location of the repository.xml file in de web.xml file to the following:

<servlet>
        <servlet-name>Repository</servlet-name>
        <servlet-class>org.hippoecm.repository.RepositoryServlet</servlet-class>
        <init-param>
            <param-name>repository-config</param-name>
            <param-value>repository.xml</param-value>
            <description>The location of the repository configuration file.
                Unless the location
                starts with file://, the location is retrieved from within the application
                package as
                resource.</description>
        </init-param>
        <load-on-startup>4</load-on-startup>
    </servlet>

7. Add the customized repository.xml file to the cms.war file
The following repository.xml file should be added to the the cms.war file in the directory: WEB-INF/classes/org/hippoecm/repository. The bundle-cache is enabled in the following repository.xml which greatly improves the performance of the Hippo cms system. The cache size is set to 256MB but can be altered when needed.

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE Repository PUBLIC "-//The Apache Software Foundation//DTD Jackrabbit 1.5//EN"
      "http://jackrabbit.apache.org/dtd/repository-1.5.dtd">
<Repository>

    <FileSystem class="org.apache.jackrabbit.core.fs.db.DbFileSystem">
        <param name="url" value="java:comp/env/jdbc/repositoryDS" />
        <param name="driver" value="javax.naming.InitialContext" />
        <param name="schemaObjectPrefix" value="repository_" />
        <param name="schema" value="postgresql" />
    </FileSystem>

    <Security appName="Jackrabbit">
        <SecurityManager class="org.hippoecm.repository.security.SecurityManager" />
        <AccessManager class="org.hippoecm.repository.security.HippoAccessManager" />
        <LoginModule class="org.hippoecm.repository.security.HippoLoginModule" />
    </Security>

    <Workspaces rootPath="${rep.home}/workspaces"
        defaultWorkspace="default" />
    <Workspace name="${wsp.name}">
        <FileSystem class="org.apache.jackrabbit.core.fs.db.DbFileSystem">
            <param name="url" value="java:comp/env/jdbc/repositoryDS" />
            <param name="driver" value="javax.naming.InitialContext" />
            <param name="schemaObjectPrefix" value="${wsp.name}_" />
            <param name="schema" value="postgresql" />
        </FileSystem>

        <PersistenceManager
            class="org.apache.jackrabbit.core.persistence.bundle.PostgreSQLPersistenceManager">
            <param name="driver" value="javax.naming.InitialContext" />
            <param name="url" value="java:comp/env/jdbc/repositoryDS" />
            <param name="schemaObjectPrefix" value="${wsp.name}_" />
            <param name="externalBLOBs" value="true" />
            <param name="consistencyCheck" value="true" />
            <param name="consistencyFix" value="true" />
            <param name="bundleCacheSize" value="256"/>
        </PersistenceManager>

        <SearchIndex
            class="org.hippoecm.repository.FacetedNavigationEngineThirdImpl">
            <param name="indexingConfiguration" value="indexing_configuration.xml" />
            <param name="indexingConfigurationClass"
                value="org.hippoecm.repository.query.lucene.ServicingIndexingConfigurationImpl" />
            <param name="path" value="${wsp.home}/index" />
            <param name="useCompoundFile" value="true" />
            <param name="minMergeDocs" value="1000" />
            <param name="volatileIdleTime" value="10" />
            <param name="maxMergeDocs" value="1000000000" />
            <param name="mergeFactor" value="5" />
            <param name="maxFieldLength" value="10000" />
            <param name="bufferSize" value="1000" />
            <param name="cacheSize" value="1000" />
            <param name="forceConsistencyCheck" value="true" />
            <param name="enableConsistencyCheck" value="true" />
            <param name="autoRepair" value="true" />
            <param name="analyzer"
                value="org.apache.lucene.analysis.standard.StandardAnalyzer" />
            <param name="queryClass" value="org.apache.jackrabbit.core.query.QueryImpl" />
            <param name="respectDocumentOrder" value="false" />
            <param name="resultFetchSize" value="2147483647" />
            <param name="extractorPoolSize" value="0" />
            <param name="extractorTimeout" value="100" />
            <param name="extractorBackLogSize" value="100" />
            <param name="textFilterClasses"
                value="org.apache.jackrabbit.extractor.PlainTextExtractor,org.apache.jackrabbit.extractor.MsWordTextExtractor,org.apache.jackrabbit.extractor.MsExcelTextExtractor,org.apache.jackrabbit.extractor.MsPowerPointTextExtractor,org.apache.jackrabbit.extractor.PdfTextExtractor,org.apache.jackrabbit.extractor.OpenOfficeTextExtractor,org.apache.jackrabbit.extractor.RTFTextExtractor,org.apache.jackrabbit.extractor.HTMLTextExtractor,org.apache.jackrabbit.extractor.XMLTextExtractor" />
        </SearchIndex>
    </Workspace>

    <Versioning rootPath="${rep.home}/version">
        <FileSystem class="org.apache.jackrabbit.core.fs.db.DbFileSystem">
            <param name="url" value="java:comp/env/jdbc/repositoryDS" />
            <param name="driver" value="javax.naming.InitialContext" />
            <param name="schemaObjectPrefix" value="version_" />
            <param name="schema" value="postgresql" />
        </FileSystem>

        <PersistenceManager
            class="org.apache.jackrabbit.core.persistence.bundle.PostgreSQLPersistenceManager">
            <param name="driver" value="javax.naming.InitialContext" />
            <param name="url" value="java:comp/env/jdbc/repositoryDS" />
            <param name="schemaObjectPrefix" value="version_" />
            <param name="externalBLOBs" value="true" />
            <param name="consistencyCheck" value="true" />
            <param name="consistencyFix" value="true" />
            <param name="bundleCacheSize" value="256"/>
        </PersistenceManager>
    </Versioning>

    <SearchIndex class="org.apache.jackrabbit.core.query.lucene.SearchIndex">
        <param name="path" value="${rep.home}/repository/index" />
        <param name="forceConsistencyCheck" value="true" />
        <param name="enableConsistencyCheck" value="true" />
    </SearchIndex>

    <DataStore class="org.apache.jackrabbit.core.data.db.DbDataStore">
        <param name="url" value="java:comp/env/jdbc/repositoryDS" />
        <param name="driver" value="javax.naming.InitialContext" />
        <param name="databaseType" value="postgresql" />
        <param name="minRecordLength" value="1024" />
        <param name="maxConnections" value="5" />
        <param name="copyWhenReading" value="true" />
    </DataStore>
</Repository>

8. Optionally: change the heap and permsize settings of JBoss.

I use the following memory settings for my configuration. Modify this in the run.bat startup script: set JAVA_OPTS=%JAVA_OPTS% -Xms256m -Xmx768m -XX:MaxPermSize=256m

Thats it!

Second Swiz workshop

2011-01-12T11:43:00.000+01:00

On February the 16th I will be giving the second Swiz workshop for the Dutch Flex User Group. The workshop has been upgraded to Swiz 1.0 instead of 0.6. In this workshop I will show you how to apply the MVC pattern to Flex applications using the Swiz framework. More information about Swiz can be found here.

Registration is still open since there are still some seats available. You can register here.

See you there!

20 Useful development tools

2011-01-04T11:04:00.000+01:00

In this post I will take a look at 20, mostly free, useful tools which I use during my day-to-day software development. I am not talking about IDE's (I use various ones) here but small useful tools to increase developer productivity. The tools are listed in no particular order.

IETester
IITester can be used to test websites in various versions of Internet Explorer. A lot of website still require IE6 which can be difficult to test under Windows Vista or Windows 7. IETester provides a viable alternative which let you test your website in IE5.5, 6, 7 and 8. IETester can be found here.
PuTTY
PuTTY is an open source terminal emulator which can act as a client for SSH, Telnet and other protocols. Useful tool for the remote management of computers. PuTTY can be found here.
Notepad++
Notepad++ is a free source and text file editor which I use daily for text file editing. I do not use Notpad++ as a source code editor but it does support syntax highlighting for various languages. I find this useful as I sometimes use Notepad++ to look at source files. Notepad++ supports various plugins and does also have a column editing mode. To enable column editing mode just hold the "Alt" key and select the text with your left mouse button. Notepad++ can be found here.
ArgoSoft Mail Server
ArgoSoft mail server is a mail server which you can use on your localhost to sent and receive mails using POP3 en SMTP. This is a very useful tool when testing e-mail functionality in any application. ArgoSoft mail server Freeware edition can be found here.
GEDIS Studio
GEDIS studio is a test data generation tool. They also have a free community edition which you can use to generate test data. The community edition also comes with text files which contains useful testdata which you can use to generate your own test- data/files. GEDIS studio can be found here.
FileZilla client
FileZilla client is a free FTP client which I use a lot to transfer files to other machines using FTP. The FTP client is available for multiple platforms and can be found here.
MySQL Workbench
For some projects I use MySQL. When working with MySQL I usually use the MySQL workbench for SQL development and server administration. The tool can also be used for data modeling. MySQL workbench can be found here.
JRebel
When doing Java development I find the time it takes to reload changes in Java source files to publish back to the server frustratingly slow. JRebel enables the changes you make in Java source files to immediately be reflected in the runtime environment for example Tomcat, JBoss or WebSphere. Although JRebel is not free the money is really worth it. JRebel can be found here.
VMWare player
I mainly develop on Windows 7. Sometimes I need to check something in another operating system, for example Ubuntu. VMWare player is a useful tool to run other operating systems in an easy way. I also use VMWare player when I give workshops. Instead of installing the prerequisites on every client machine, I distribute a VMWare image which participants use during the workshop. This saves me a lot of setup time and frustration. VMWare player can be found here.
Paint.NET/Gimp
Although graphic design is not my primary expertise I sometimes have the need to edit images for my web projects. I use Paint.NET and Gimp for this depending on what I need to do. Paint.NET can be found here and Gimp can be found here.
SQLite Administrator
SQLite Administrator is a powerful tool if you easily want to create, design or administrate SQLite database files. I use SQLite Administrator for example when I use SQLite with Adobe Air. SQLite Administrator can be found here.
Selenium IDE/iMacro
When testing certain functionality during web application development you often must perform a sequence of steps before you can test the actual functionality. Selenium and iMacro let you record these steps and automate this process. This saves you a lot of time when testing the same functionality several times. SeleniumIDE can be found here and iMacro can be found here.
Firebug
Firebug is useful tool when doing web application development. This Firefox plugin enables you to inspect the HTML and modify it in real time and analyze network traffic. Firebug also has a JavaScript debugger. Firebug can be found in the Firefox add-on window.
Chrome
Chome also has a lot of useful tools when doing web application development. The tools in Chrome I use most are the element inspector, JavaScript debugger and network analysis tool.
YSlow
YSlow is a Firebug extension which analyzes a web application based on client side performance indicators developed by Yahoo. YSlow makes suggestions in how to optimize the client side performance of your web application. YSlow can be found in the Firefox add-on window.
PageSpeed
PageSpeed is a similar tool as YSlow but PageSpeed is developed by Google. I use both YSlow and PageSpeed. PageSpeed can be found here.
SoapUI
SoapUI is a Java based tool for testing web services. SoapUI has a basic version for free which can be found here.
Tamper data
Tamper data is Firefox plugin to view and modify HTTP/HTTPS header and modify POST parameters. I use this tool for inspecting traffic sent from the browser to the server and for testing the security of web applications. Tamper data can be found here.
REST Client
REST Client is a Firefox plugin to easily test REST services and can be found here.
Fiddler
Fiddler is a web debugging proxy which can be used inspect the traffic sent from your browser over the Internet. I use this tool a lot in analyzing the traffic between the web application under development and the backend. It also supports HTTPS. Fiddler can be found here.

This list is not exhaustive so I would like to know which tools you use?

JRebel rocks!

2010-09-29T13:24:00.003+02:00

Today I installed JRebel because I was tired of all the Republish stuff when working with JBoss in Eclipse. I am glad I did! I am not sure how much time it saved me exactly (when working for 4 hours) but looking at the JBoss server.log showed the following:

Maybe a little too optimistic, but a republish costs one minute at minimum. At 40 redeploys, this saves me about 45 minutes. This is 1.5 hour for a full work day!

Flex4 DateField bug when using locale nl_NL

2010-09-27T10:20:00.005+02:00

When porting a Flex 3 application to Flex 4 (4.1 SDK), I had a little issue with my DateField components. The selected date in the DateField component did not have any formatting applied to it and when accessing the selectedData property in Actionscript returned null. For example selecting 7th September 2010 resulted in '0709092010' but it should be '07/09/2010'.

Some searching revealed that this is a known bug which is hopefully resolved in SDK4.5. See http://forums.adobe.com/message/2834803 for more details.

A quick workaround is to specify the formatString explicitly in the DateField component as in the following example:

<mx:DateField id="date" formatString="DD/MM/YYYY"/>
<s:Button click="trace(date.selectedDate);"/>

FlexCamp presentation

2010-09-24T20:37:00.002+02:00

Wednesday 22nd of September, I gave a presentation for the Dutch Flex User Group at FlexCamp at Adobe in Amsterdam. You can find this presentation below:

Structuring your flex application with swiz

View more presentations from Jamie Craane.

Printing in Vaadin

2010-09-03T13:49:00.005+02:00

In one of my recent projects for one of our clients, I have used the Vaadin framework to develop the application. One of the requirements of this application was to print certain content. I basically evaluated three possible solutions to implement this functionality:

Create a new window with content specifically designed for printing and use the JavaScript print() method to print the contents of this window;
Create a PDF with the printable contents and open this PDF in a new browser window;
Create a PDF with the printable contents and use the Embedded and Window components to display the PDF.

I found option 1 not compatible and consistent enough between browsers. To create a consistent look&feel of the printable contents I used PDF. One way to display this PDF to the end-user is to open a new browser window. Unfortunately this was hindered by popup blockers which is not acceptable to end-users of the application.

Option 3 involved using an Embedded component to display the PDF inside a native Vaadin popup window (which is a regular div). This way, the printable contents are always consistent (because PDF is used) and the popup with the PDF contents is not hindered by browser popup blockers because the popup is just a regular div. By using this method, the print window also integrated nicely with the visual appearance of the rest of the application. The following code demonstrates how to implement this functionality:

/**
     * This class creates a PDF with the iText library. This class implements
     * the StreamSource interface which defines the getStream method.
     */
    public class Pdf implements StreamSource {
        private final ByteArrayOutputStream os = new ByteArrayOutputStream();

        public Pdf() {
            Document document = null;

            try {
                document = new Document(PageSize.A4, 50, 50, 50, 50);
                PdfWriter.getInstance(document, os);
                document.open();

                document.add(new Paragraph("This is some content for the sample PDF!"));
            } catch (Exception e) {
                e.printStackTrace();
            } finally {
                if (document != null) {
                    document.close();
                }
            }
        }

        @Override
        public InputStream getStream() {
            // Here we return the pdf contents as a byte-array
            return new ByteArrayInputStream(os.toByteArray());
        }
    }

And the code for displaying a popup with the actual PDF viewer:

Window window = new Window();
        ((VerticalLayout) window.getContent()).setSizeFull();
        window.setResizable(true);
        window.setWidth("800");
        window.setHeight("600");
        window.center();
        Embedded e = new Embedded();
        e.setSizeFull();
        e.setType(Embedded.TYPE_BROWSER);

        // Here we create a new StreamResource which downloads our StreamSource,
        // which is our pdf.
        StreamResource resource = new StreamResource(new Pdf(), "test.pdf?" + System.currentTimeMillis(), this);
        // Set the right mime type
        resource.setMIMEType("application/pdf");

        e.setSource(resource);
        window.addComponent(e);
        getMainWindow().addWindow(window);

A full, working example including PDF generation, can be found here. The example project demonstrates displaying the PDF with and without a popup.

What is your experience with printable content in a Vaadin application?

FlashBuilderCamp

2010-09-03T12:25:00.003+02:00

On September 22nd, the Dutch Flex User Group organizes FlashBuilderCamp at Adobe in Amsterdam, The Netherlands. I will be giving a session about presentation patterns and how to implement these patterns with the Swiz framework. Swiz is a micro-architecture and dependency injection framework for structuring Flex applications which I used in a couple of major Flex application.

Make sure you attend the conference. Entrance and dinner is free. More information can be found at Flugr.

Stream erorr 2032 in Flex using HttpService

2010-08-05T08:05:00.007+02:00

Lately I developed an application for a workshop I was giving. The application was a Flex application which communicated with a SpringMVC 3.0 annonation based based web application using Xstream as XML serialization library. All seemed well until in specific browsers and operation systems I got a 'Error #2032: Streamfault'. I did not found a solution on the Internet other than verifying that my URL was correct and disable HTTP caching on the server. None of these solutions worked.

When I executed this specific request in the browser, I got a HTTP 406 status code. The 406 status code says:

The resource identified by this request is only capable of generating responses with characteristics not acceptable according to the request "accept" headers ().

Inspecting the request the browser sent, showed an accept-header of '*/*'. I compared this to the accept-header FireFox sent which was: 'text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8'.

The solution turned out to set the accept-header explicitly on the HttpService. I used the following code for this:

var httpService:HTTPService = new HTTPService();
httpService.url = url + "/tasks";
httpService.requestTimeout = 10;
// Set the content type expected by the server
httpService.contentType = "application/xml";
// Explicitly set the accept header
httpService.headers = { accept: "text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8" };
httpService.resultFormat = "e4x";

As a sidenote, to disable HTTP caching on the server, you can use the following code:

response.addHeader("Pragma", "no-cache");
response.addHeader("Cache-Control", "no-cache");
response.addHeader("Cache-Control", "no-store");
response.addHeader("Cache-Control", "must-revalidate");
response.addHeader("Expires", "Mon, 8 Aug 2006 10:00:00 GMT");

Flex workshop

2010-05-10T10:04:00.006+02:00

At Wednesday the 2nd of June I will be giving a Flex/Swiz workshop. During this workshop I will show you how to develop a Flex application with the Swiz framework. Swiz is a micro architecture and dependency injection framework for Flex. Swiz does not restrict you in any way which enables you to use Swiz in almost any application. by using Swiz and, for example the MVCS pattern, the application becomes better testable and maintainable.

The application we are building is a simple contact/task management application based on experiences from clients when building complex Flex applications. The application communicates with a Java backend using BlazeDS and XML/HTTP. The full workshop contents are presented on a VMWare image. The only prerequisite from the attendee is a laptop with VMWare player and at least 8 GB of free hard disk space.

Agenda:
June 2nd, 2010
15:00 – 15:30 Welcome and introduction
15:30 – 18:30 workshop
18:30 – 19:00 Conclusion and dinner

Location:
QNH Business Integration
Driebergseweg 2
3708 JB Zeist

There are still seats available so be quick!

See you there!

Flex Camp 2010

2010-02-25T09:14:00.005+01:00

Update: The presentations are online and can be found here.

The first Flex Camp 2010 is a fact and was a great success. Flex Camp is organized by the Dutch Flex User Group (FLUGR) and is a half-day Flex event with a variety of speakers and subjects. This time, the location of the event was Amsterdam. The following sessions were given:

Designer/Developer workflow with Flash Catalyst and Flash Builder

This session was all about how Flash Catalyst and Flash Builder approach the designer /developer workflow. In a traditional development environment a design usually takes the form of a Photoshop document which is sliced by the developer and incorporated in the Flex application. Flash Catalyst takes a new approach in that this product is able to import and slice the design itself. When imported, Flash Catalyst is able to convert individual elements to read-to-use Flex components.

One thing that is not incorporated in the product yet, is an iterative design/development cycle although this can partly be accomplished with the use of libraries.

Set your code on fire

Flex developers with a Java background are always disappointed by the lack of features of Flex Builder compared to working with Java editors. Luckily there are alternatives for Flex Builder and one of the alternatives is FDT from Powerflasher. During the session, an application was developed to show a lot of the capabilities of FDT. Very impressive. Some of FDT’s features are:

Rename and move refactoring

Extended use of code templates and quick fixes to:

Generate classes

Generate functions

Generate fields with getters and setters

Loop over arrays

Etc.

Superb code navigation possibilities like type hierarchy, dependency structure, outline view with search capabilities.

Debugger

Version 4 of FDT, which is coming in 2010, has even more features like a memory and performance profiler. To learn more about version 4 goto http://fdt.powerflasher.de/fdt4comfort

Flex Builder action script projects

Flex Builder is not only good in Flex projects but also suitable for Action script only projects. The speaker showed how he developed a pure Action script only website with Flex Builder. During the session a lot of practical tips were given which he encountered during the development of the website. Unfortunately, the site is not yet live at the time of writing. To be continued.

Flex in a JEE environment

In this session the speaker demonstrated a Flex application which is used to display information about geographical locations. The application integrates with the backend using web services and Blazeds and is deployed on a JBoss application server. The speaker showed how Maven is used to build and distribute the application and how the Flex application can be configured using server side services. It was a very practical and good session, especially for Java and Flex developers.

When all presentations are online, I will post the link to these presentations here.

Conclusion
Flex Camp 2010 was a big success. The subjects of the sessions were diverse enough to please everyone who attended Flex Camp. From Flex designer/developer to hardcore Java developer. On to the next Flex Camp!

Flex: How to create a different rollover color for the header in a datagrid component?

2009-10-30T08:38:00.009+01:00

Update 01-02-2010: Monkey patching the Flex framework does not work when RSL's are used. This is because the code of the RSL is loaded in the first frame wether the monkey patched code is instantiated in the second frame and hence does not override the RSL classes. This can be solved by instantiating the monkey patched class in a custom preloader as described here. Depending on the size of the included classes, this can invalidate the choice to use RSL's in the first place, namely to reduce application size. To include the monkey patched class in a custom preloader depends solely on the patch itself and the increased download size. These two factors should be taken into account.

In my current project I have the following requirement for styling a DataGrid component:

The roll-over color of the header should be of a different color than the roll-over color of the datagrid rows. To make things even more interesting, the roll-over color must be a gradient instead of a solid color.

Unfortunately there is no easy way to accomplish this. We can set the default background color of the header with the headerColors style, but this color is overwritten when the mouse hovers over the header. We can also disable the rollover highlighting with the useRollOver property but this does not prevent from changing the rollover color of the header! Even if we could disable the header rollover color this way, it is not a solution because we want a different rollover color for the header and the rows containing the data.

After some digging in the Flex SDK source code, the solution I came up with is to provide a customized DataGridHeader class. Please note that I customized the Flex 3.2.0 SDK because I use the 3.2.0 SDK. If you use the Flex 3.4.0 SDK you should modify the DataGridHeader class of the 3.4.0 SDK instead. To customize the DataGridHeader class I did the following:

Created a mx.controls.dataGridClasses package in my own source folder
Copied the DataGridHeader.as file from the Flex 3.2.0 SDK to my own mx.controls.dataGridClasses package
Located the mouseOver- and mouseDownHandlers and replaced the drawHeaderIndicator method with my own drawGradientBackGround method.
Added two new style attributes to the DataGridHeader class, named: startColor and endColor which defines the start- and endcolor of the gradient fill. These attributes are queried in the drawGradientBackGround method. For simplicity I did not provide default values so the default color is black.
Declared the startColor and endColor attributes in css and gave them a color value.

A full working example can be downloaded from http://code.google.com/p/jc-examples/ as a zip file or from the svn repository. The following picture is taken from the sample project which demonstrates the gradient header color when the mouse is over the header. The mouse is over the name column in this case. Please note that this example uses the Flex 3.2.0 SDK.

If there are more straightforward solutions to this problem please share them.

Resources

http://code.google.com/p/jc-examples/

Sample code as a zip file

To inject or not to inject

2009-10-03T16:17:00.001+02:00

Last week I was a couple of days at a client to look at some strange behavior in the application. The application was a calculator used by clients of the company to calculate services the client offers. The application was written in Java6 and used Tapestry for the view and a Spring/Hibernate/AspectJ combination for the backend which talked against an Oracle database. Tomcat was used as the runtime container for the application.

The first problem that had to be tackled was that sometimes the dependency injection did not work properly. Sometimes dependencies were injected, and other times not. When I looked at the application I noticed that an aspect was responsible for injecting repositories into domain objects. As soon as a domain object is deserialized or instantiated, the AnnotationBeanConfigurerAspect injected the repository. At least that was the intention. But the injection did not always take place. I further discovered that load-time weaving was used to inject the aspect’s behavior. I looked at the configuration and did not see anything that was wrong. The proper class loader in Tomcat was used and the necessary Spring configuration was present. What was causing this awkward behavior? Because time was limited I decided to switch to compile-time weaving. We compiled the classes with the AspectJ compiler and redeployed the application. This time the dependency injection was working as expected. We did not further investigate why load-time weaving did not work properly. If you have any suggestions please mention them.

The second problem was that after a specific amount of inactivity at the application side, the application crashed when it was first accessed. The stack trace revealed that it was a problem with the underlying database connection. The message indicated that the connection was already closed. What was happening was that Oracle closes all idle connections after a specific amount of time, in our case 30 minutes. This is configured in the sqlnet.ora file with a parameter called sqlnet.expire. To overcome this, we added the following configuration to the Commons DBCP connection pool:

testOnBorrow=true

validationQuery=SELECT 1 FROM DUAL

The result of this configuration is that all connections returned from the pool are first tested with the specified validation query. If the test fails the connection is discarded from the pool and a new one is returned. This solved our problem. I must mention that this configuration can have a negative impact on performance because every returned connection is checked for validity. Because of this, the simplest Oracle query is used to test the connection. Since the application did not have many concurrent transactions, this setting was acceptable.

If performance is an issue, I would recommend using an eviction strategy where a specified eviction policy periodically checks all idle connections for validity. See the Commons DBCP configuration guide for an explanation to configure this.

I hope some of you may find these solutions helpful in your own projects.

Melody composition using genetic algorithms

2009-06-16T10:52:00.016+02:00

Introduction

Several techniques exist to create computer generated musical melodies. One of those techniques is genetic algorithms. Because the diversity of melodies over a specific range of notes is so large, genetic algorithms are a good candidate to help in composing melodies. This article describes how genetic algorithms can be used to compose musical melodies. this is explained following the steps needed to apply a genetic algorithm. These steps are:

Define the genetical representation of the problem
Determine the fitness function
Determine the parameters used for the run
Determine the termination criteria

The implementation of the genetic algorithm is done using the JGAP framework, a Java-based framework for implementing genetic algorithms. The generated melody itself is converted to MIDI which can then be played by the internal MIDI device or by a musical instrument attached to the computer

For more information about using genetic algorithms in Java please see my previous article which can be found here: introduction to genetic algorithms. For more information about music theory which is applied in this article please refer to the excellent site http://www.musictheory.net.

Define the genetical representation

Before the genetic algorithm can do it's work, the genetical representation of the problem must be defined. To make things slightly less complicated the following constraints are introduced:

Only harmonic melodies are supported. This means that only one note is played at ones at any given time in the melody.
The generated melodies do not adhere to a specific measure. It is just a sequence of notes.

In a future version these constraints could be loosened.

A Melody can be seen as a composition of individual notes and rests. Those individual notes have properties which define how a certain note must be played. The following properties are used which are modified by the genetic algorithm:

Pitch
The pitch determines which notes on the grand staff is played. Possible values are C,D,E,F,G,A and B and all possible variants using sharps (#) and flats (b).
Octave
The octave determines in which octave a certain note is played. A piano has seven full octaves.
Duration
The length determines how long a certain note is played. Possible values are, whole notes, half notes, quarter notes, eight notes, sixteenth notes.

Notes have more properties than pitch and duration alone. Velocity, for example, indicates how hard a note is played. These properties are not used in this article. Besides notes, there are also rests in a melody. Rests can only have a duration.

A solution in the search space of melody generation is a melody consisting of individual notes. A solution is represented as a chromosome with a fixed number of genes. The genes in the chromosome represent the individual notes and rests, each with it's own characteristics. A note can be represented by using a composite gene with three integer gene's. The three integer gene's represent pitch, octave and duration. Integer gene's are chosen in favor of a custom gene implementation to make mutation easy and simple across the individual gene's. Each gene is described in detail here:

Pitch: The pitch can be represented as a number from 1 to 12. Since there are twelve semitones in an octave, each semitone can be represented by a number from 1 to 12. Every following number is equal to one semitone. The mapping of the notes is:

Value	Note
1	C
2	C# or Db
3	D
4	D# or Eb
5	E
6	F
7	F# or Gb
8	G
9	G# or Ab
10	A
11	A# or Bb
12	B
0	rest

Because all gene's are treated the same, a special case for the a rest in a melody is introduced. A rest has the value of 0 for the pitch. Since a rest is not associated with an octave, the octave property is ignored.

Octave: an octave can be represented by a number, the octave in which a certain note is played. The octave is indicated by a number from 1 to 7 where 1 represents the lowest octave on a piano keyboard and 7 the highest.

Duration: the duration can be represented by a number between 1 and 5. The mapping of the numbers are:

Value	Note
1	Whole note
2	Half note
3	Quarter note
4	Eighth note
5	Sixteenth note

The duration applies to both notes and rests. See the following code for the creation of the initial population of chromosomes:

Configuration.reset();
Configuration gaConf = new DefaultConfiguration();
gaConf.resetProperty(Configuration.PROPERTY_FITEVAL_INST);
gaConf.setFitnessEvaluator(new DeltaFitnessEvaluator());

gaConf.setPreservFittestIndividual(true);
gaConf.setKeepPopulationSizeConstant(false);

gaConf.setPopulationSize(40);
CompositeGene gene = new CompositeGene(gaConf);
// Add the pitch gene
gene.addGene(new IntegerGene(gaConf, 0, 12), false);
// Add the octave gene
gene.addGene(new IntegerGene(gaConf, 1, 7), false);
// Add the length (from 3 - 5 is from quarter to sixteenth)
gene.addGene(new IntegerGene(gaConf, 1, 5), false);

// A size of 16 represent 16 notes
IChromosome sampleChromosome = new Chromosome(gaConf, gene, 16);
gaConf.setSampleChromosome(sampleChromosome);

gaConf.setFitnessFunction(melodyFitnessFunction);

return Genotype.randomInitialGenotype(gaConf);

Determine the fitness function

The fitness function determines how good a specific melody is, relative to other melodies. The fitness function is the most complicated part of this problem since the fitness of a melody is subjective. Because of this nature, two approaches for fitness determination are presented.

Computer generated fitness

The computer generated fitness is purely based on certain algorithms to measure the fitness of a melody. Since there are a lot of different parameters, the fitness function combines the fitness value of several different strategies which can be easily added to the fitness function. Some of these parameters, but not limited to, are:

80% of the notes in a melody may not have more than than 7 semitones difference.
A melody may not span more than 2 octaves
A melody must be in C-major (or minor scale)
Only 10% of a melody may consist of rests
Two consecutive notes may not lie more than 5 semitones from each other
...

In the proof-of-concept implementation the following rules regarding the fitness of a melody are implemented. These rules are implemented as separate classes which all implement the MelodyFitnessStrategy interface. This makes it easy and straightforward to add more rules which measure the fitness of a melody.

ScaleStrategy
Calculates if a given melody adheres to a specific scale, for example C major. The scale can be set as a parameter on this class.
IntervalStrategy
Calculates if a given melody has one or more major and/or perfect intervals. The number of major and perfect intervals can be set as parameters on this class.
GlobalPitchDistributionStrategy
Calculates if the lowest and highest pitch of a given melody fall within the margins specified by this class. The margin is indicated as the number of semitones and a percentage about how much of the notes must fall between the given semitones. These values can be set as parameters on this class.
RepeatingNotesStrategy
Calculates if a given melody has repeating notes or rests. The maximum number of repeating notes and/or rests can be set as parameters on this class.
PropertionRestAndNotesStrategy
Calculates the proportion between the notes and rests in a given melody. The propertion of notes/rests can be set as parameter on this class.
ParallelIntervalStrategy
Calculates the number of parallel intervals in this melody. Some parallel intervals are supposed to sound good, like thirds and sixths. The number of good sounding parallel intervals can be set as a parameter on this class.

A builder class exists which helps in the creation of a valid fitness function.

Please note that all rules calculate the deviation between the generated melody and the specified rules. A lower fitness value means less deviation which means a better melody. In the future it is planned to add more strategies to calculate the fitness of a given melody. For example:

ContourStrategy. Strategy which calculates if a given melody has a specific contour in the pitch of the individual notes.

Below is the implementation of the ScaleStrategy:

public final class ScaleStrategy extends AbstractMelodyFitnessStrategy {
  private static final int ERROR_COUNT_WHEN_NOTE_NOT_ON_SCALE = 1;

  @Override
  // The more difference between current note and note in scale the higher the error count
  public double calculateErrors(IChromosome melody) {
     double errors = 0.0D;
     for (Gene gene : melody.getGenes()) {
        Note note = GeneNoteFactory.fromGene((CompositeGene) gene);
        if (Pitch.REST != note.getPitch() && !super.scale.contains(note.getPitch())) {
           errors += ERROR_COUNT_WHEN_NOTE_NOT_ON_SCALE;
        }
     }

     // Adhering to a given scale is quite important so square the result
     return (errors * errors) * 10;
  }

  public String toString() {
     return "[ScaleStrategy[scale: " + this.scale + "]]";
  }
}

A different approach

The above paragraph describes how the fitness of a melody can be computed based on the knowledge of music theory. Instead of using music theory to compute the fitness of a melody, a different approach can be used. This approach looks at the melody as audio data, an array of bytes, instead as a sequence of notes. From this audio data, different information can be extracted. For example, by using a Fast Fourier Transform (FFT), the audio data can be viewed in the frequency domain. By analyzing existing melodies using FFT, an algorithm can be constructed which measures the fitness of generated melodies based on this knowledge. A future version of the application may use the technique described here.

Human intervention based fitness

Since melodies are very subjective it is hard to come up with a computer based mechanism to measure how good a specific melody is. Another approach which can be used is a human based intervention fitness function. In this approach the user is represented with a fixed set of melodies generated by the genetic algorithm. The user selects two or three melodies which participate in the next evolution. Although this approach is not implemented in the current version, expect a future release to use this approach.

Determine the run parameters

When using a computer based fitness function, several parameters which affect the generated melody can be provided by the user. See the section about the computer generated fitness function which parameters can be supplied. Based on some sample runs, the number of evolutions to execute is 250. 250 evolutions seems like a good trade off between quality of the generated melody and computation time. The quality is measured in terms of the fitness value of the melody. With 250 evolutions, the fitness value of a 24-note melody almost always approaches zero.

Determine the termination criteria

The run ends when 250 evolutions are executed.

Some sample runs

The program can be executed in two different ways:

By executing the MelodyGeneratorMain class file from the command line. Modify and recompile this class to alter the parameters of the run, for example the fitness function.
Via the Swing UI. This is a very simple Swing UI build with Groovy's SwingBuilder. All of the parameters of the fitness function can be modified with this UI. Please note that this is a very simple UI implementation and not an example of how to write production quality Swing code.

To generate a melody from the UI, just click the generate button. When finished, the application plays back the melody and gives the option to replay, save or generate a new melody. Just experiment with the different settings and listen to the various generated melodies. In the UI you can specify the path to write the MIDI files to. Make sure this path exists on disk.

Conclusion

This article explains how genetic algorithms can be used to compose melodies. Genetic algorithms seem like a viable alternative for melody generation since they are very well suited to search for specific solutions in very large search spaces. In this case the search space are all possible combination of notes. The difficulty in generating melodies with genetic algorithms is the specification of the fitness function since this is very subjective.
Although the melodies generated by this program are nowhere near hit melodies, they can be used as inspiration when composing melodies. The performance of the algorithm can be improved by writing additional (and more complex) strategies to measure the fitness of a melody.

Let me know what ideas you have to measure the fitness of the generated melodies to improve the quality of the melodies.

The full source code of this application can be found on Google code, see the resources.
Resources

http://code.google.com/p/jc-examples/

www.musictheory.net

J-Spring 2009 part 2

2009-04-20T12:50:00.004+02:00

The second keynote was given by Jos Warmer, a Model Driven Architecture evangelist. Jos has a lot of experience with MDA and you can see that from his talk. His talk was about Mod4j. According to the website: “Mod4j (Modelling for Java) is an open source DSL-based environment for developing administrative enterprise applications.”

Mod4j is an environment in which DSL’s are used to generate specific parts of a Java application. At the moment three DSL’s are implemented (and a fourth is on its way) which focuses on specific architectural layers of the application. These DSL’s are:

Business Domain DSL
Data Contract DSL
Service DSL

The business domain DSL implements the concepts to specify the business objects in your application. These are: business objects with properties, associations between business objects and constraints. Consider the following example:

association Order order one -> many OrderLine orderLines ordered;

The above sample DSL generates a one-to-many association from the Order to the OrderLines class including the necessary Hibernate mapping files. A lot of boilerplate code is generated by using the business domain DSL.

The Service DSL is used to specify services for your domain objects. Examples are the CRUD operations of a specific domain object and finder methods. The code generator for this DSL takes care of generating the required Java code. Consider the following code:

from RecordShopDataContract import SimpleCustomerDto;
create createCustomer for SimpleCustomerDto ;
read readCustomer for SimpleCustomerDto ;
update updateCustomer for SimpleCustomerDto ;
delete deleteCustomer for SimpleCustomerDto ;

The preceding code generates the CRUD methods for the SimpleCustomerDto in the service class.

What’s nice about Mod4j is that no visual models are used for code generation. Domain Specific Languages are used instead. The DSL’s can be put in a version control system where you are able to use the version control system’s merge and history abilities; something which is hard to implement when using visual models. There are also a lot of extension points to modify the generated code. But when you find yourself modifying too much of the generated code, you must ask yourself if the code generator should be modified instead. With mod4j this is possible because it is open source. What’s also nice is that there is an Eclipse plug-in to facilitate Mod4j development. With this plug-in you get code completion on the provided DSL’s.

Although I am not a big fan of Model Driven Architecture tools, I like the concept and principles of Mod4j mainly because coding is central. When should you use Mod4j? I find this a difficult question. When working on a large multi-layered application in Java with technologies like Hibernate, Mod4j can make sure that a consistent architecture is used across all layers. New developers can be more easily brought up to speed.

But when looking at the DSL’s used for generating code, I immediately thought of the Grails framework, based on Groovy. Grails uses a lot of DSL’s. Consider the following Grails example which creates an Author business object:

class Author {
   static hasMany = [ books : Book ]
   String name
   static constraints = {
       name(size:5..15,blank:false)
   }
}

The above code is the Author business object in Grails with a one-to-many association to the Book class. It also implements a constraint on the name property. Because of the dynamic nature of Groovy, you get the CRUD operations, dynamic finders and the relationship handling for free. If you want to customize the mappings you can modify the Hibernate mapping files if you want.

So the DSL Grails uses is what I call an executable DSL whereas the DSL’s that Mod4j uses are used to generate Java code.

Despite this there is still place for a technology like Mod4j. Not everyone is in the position to use Grails (or Rails) to facility high speed web development. When plain Java web development is used, Mod4j provides a viable alternative to speed-up this development. I will definitely keep an eye out for Mod4j!

Service Oriented Architecture

The next session I went to was about Service Oriented Architectures from Tijs Rademakers. His presentation turned out to be a very practical session which included a lot of his own experiences when implementing Service Oriented Architectures. I like this approach very much. He clarified some misunderstandings when using ESB’s. When an ESB is used, often a lot of code has to be written inside the ESB. Think of transformation and routing logic. This code has to be maintained as well which is often underestimated.

Tijs also talked about how to handle XML messages in a Service Oriented Architecture. I always thought that Groovy has superior XML handling capabilities compared to Java. Tijs mentioned Groovy as one of the most effective ways for handling XML. I agree! So when working with XML in Java (Service Oriented Architecture or not), consider using the Groovy language for the XML handling logic.

Conclusion
Despite the economical situation, 4 people of my company (including me) went to the J-Spring. At QNH we find it very important to continue to invest in knowledge. Individuals are the most important part of a company in which you should invest.

I found the J-Spring 2009 a very good day with lots of good speakers and sessions with a great variety of subjects well worth the investment. If you’re a Dutch Java developer and was not part of J-Spring, make sure you will be at J-Fall later this year.

Keep up the good work!

J-Spring 2009 part 1

2009-04-17T14:26:00.008+02:00

April 15th I went to the Dutch Java user group conference named J-Spring. This is the first of two blog entries about this conference. The J-Spring is a popular 1 day conference about Java and related technologies. It is a good place to get an overview about the current state of technology and meet fellow developers.

The introductory keynote was given by Sherali Karimov from Atlassian who talked about how agile is implemented at Atlassian.

Two things in his presentation I found particular interesting. These were:

The disturbed
Blitz tests

The disturbed
We are all familiar with it. Sometimes you and your team members get so many questions from different people that your own work suffers from it. At Atlassian they introduced the role: the disturbed. Whenever someone has the disturbed role, he/she is expected NOT to do any work but instead answer all questions from outside. The other team members can focus on delivering. The disturbed role is changed every week to another team member. Every week a new team member is responsible for answering questions from people outside the team. Besides shielding the team from outside questions, the disturbed role has another advantage. By switching team members to answer questions, gradually each team member gains the knowledge to answer those questions. It is an effective and practical way to share knowledge within the team.

Blitz tests
Another concept that was introduced is Blitz tests. Atlassian uses there own products inside there own company. Whenever a new release of a particular product is released, it is deployed on the company’s production environment. People from within the company can subscribe to test the new functionality. The experience at Atlassian was that a lot of people subscribed and were eager to test the new functionality. They gave a lot of good feedback to the development team which resulted in higher quality software. I believe you gain a lot when you have the possibility to test every new release by certain people in your own company before releasing the product to your customers.

JEE and JavaFX
The next session I went to was from Paul Bakker. This session was about the future of web development with JEE6 and JavaFX. Paul gave a very complete overview about all the technologies involved in JEE6 and JavaFX. A couple of technologies used in JEE6 are JSF 2.0, JAX-RS and JPA 2.0. After giving a brief overview about every technology, Paul showed how to build a web application with JSF and JavaFX together. Basically, JavaFX was used for the highly interactive parts and JSF for the HTML forms. JavaScript was used to act as a bridge between the HTML page and the JavaFX applet. He also showed how to pass parameters between the applet and the HTML page and vice versa. This seems like an effective way to combine JavaFX with regular web development. JEE6 and JavaFX definitely look very promising!

Look out for the second post of the J-Spring which will highlight some interesting points from the other sessions I attended.

J-Spring

2009-04-06T12:56:00.001+02:00

Next week, April 15th, I am going to the Dutch Java User Group conference named J-Spring. This one-day conference is a good place in helping to keep your knowledge up-to-date. There are a lot of interesting sessions which you can attend and a lot of fellow developers to have discussions with. I Am planning to attend the following sessions:

Developing web applications of the future: Combining JavaFX & JEE 6
Java Programming in a Multicore World
SOA, it's a hard knock life
REST, the web as the database?
Hop on Board the Java Troubleshooting Platform

I will post my experience of the J-Spring next week.

Introduction to Genetic Algorithms with JGAP

2009-02-23T15:13:00.023+01:00

Out of interest I am familiarizing myself in genetic algorithms, in short GA. My interest in GA came when I first heard about the JGAP project. As mentioned on the project's site "JGAP (pronounced "jay-gap") is a Genetic Algorithms and Genetic Programming component provided as a Java framework.". For a newcomer I found it difficult to get a good overview about all the concepts introduced in genetic algorithms. Before diving into JGAP, I think it is essential that these concepts are well understood. This post is an introduction to genetic algorithms (GA) with JGAP and is explained with a concrete example. In one of my next posts I will demonstrate solving a problem with genetic programming (GP).

So what is a genetic algorithm? Given is the following definition from John R. Koza:

The genetic algorithm is a probabilistic search algorithm that iteratively transforms a set (called a population) of mathematical objects (typically fixed-length binary character strings), each with an associated fitness value, into a new population of offspring objects using the Darwinian principle of natural selection and using operations that are patterned after naturally occurring genetic operations, such as crossover (sexual recombination) and mutation.

In genetic algorithms, a potential solution is called a chromosome. A chromosome consists of a fixed length of genes. A gene is a distinct component of a potential solution. During the evolution of the genetic algorithm, multiple solutions (chromosomes) are combined (crossover and mutation) to form, potentially, better solutions. The evolution is done over a population of solutions. The population of solutions is called a genotype and consists of a fixed-length of chromosomes. During each evolution, natural selection is applied to determine which solutions (chromosomes) make it to the next evolution. The input criteria for the selection process is the so-called fitness of a potential solution. Solutions with a better fitness value are more likely to appear in the next evolution than solutions with a worse fitness value. The fitness value of a potential solution is determined by a user-supplied fitness function.

Although it is possible to implement the above concepts yourself, JGAP already took care of this. Because the best way to learn is by example, let me first introduce the problem domain which I am going to solve with genetic algorithms. During the example, the concepts mentioned above are further clarified.

Consider a moving company which is specialized in moving boxes (with things in it) from one location to another. These boxes have varying volumes. The boxes are put in vans in which the boxes are moved from location to location. To reduce transport costs, it is crucial for the moving company to use as minimal vans as possible. Problem statement: given a number of boxes of varying volumes, what is the optimal arrangement of the boxes so that a minimal number of vans is needed? The following example shows how to solve this problem with genetic algorithms and JGAP.

First: with the arrangement of the boxes I mean the following: consider 5 boxes with the following volumes (in cubic meters): 1,4,2,2 and 2 and vans with a capacity of 4 cubic meters. When the boxes are put in the vans based on the initial arrangement, the distribution of the boxes in the vans is like this:

Van	Boxes	Space wasted
Van 1	Box 1	3
Van 2	Box 4	0
Van 3	Box 2, Box 2	0
Van 4	Box 2	2

Fitness value = 3+2 * 4 = 20. See section fitness function for an explanation of the fitness function for this particular problem.

A total of 4 vans is needed. But when the number of vans needed is calculated, which is the total volume of the boxes divided by the volume of the vans, the optimal number of vans is: 11 / 4 = 2.75. Because no partial vans can be used the optimal number of vans needed is 3. The optimal arrangement of the boxes is the following: 1,2,2,2,4. Based on this arrangement the distribution looks like this:

Van	Boxes	Space wasted
Van 1	Box 1, Box 2	1
Van 2	Box 2, Box 2	0
Van 3	Box 4	0

Fitness value of 1 * 3 = 3.

Before implementing the actual solution, the following preparatory steps must be taken. These preparatory steps are always needed if genetic algorithms is used to solve a particular problem.

Define the genetical representation of the problem domain. The boxes which must be put in the vans are represented by an array of Box instances. The genetic algorithm must find the optimal arrangement in the array as how to put the boxes in the vans. A chromosome is a potential solution and consists of a fixed-length of genes. A potential solution in this example consists of a list of indexes where each index represents a Box in the box array. To represent such index, I use an IntegerGene. As mention earlier, a gene is a distinct part of the solution. In this example, a solution (chromosome) consists of as many genes as there are boxes. The genes must be ordered by the genetic program in such a way that it represents a (near) optimal arrangement. For example: if there are 50 boxes, a chromosome with 50 IntegerGene's is constructed, where each gene's value is initialized to an index in the box array, in this case from 0 to 49.
Determine the fitness function. The fitness function determines how good a potential solution is compared to other solutions. In this problem domain, a solution is fitter when fewer vans are needed so less space is wasted.
Determine the parameters used for the run. For the run I use a population size of 50 and a total number of 5000 evolutions. So the genotype (the population) initially consists of 50 chromosomes (potential solutions). These values are chosen based on some experimentation and can vary based on the specific problem.
Determine the termination criteria. The program ends when 5000 evolutions are reached or when the optimal number of vans needed is reached. The optimal number of vans can be calculated by dividing the total volume of the boxes by the capacity of the vans and rounding the result up (because no partial vans can be used).

Initialization
The Box class has a volume. In this example 125 boxes are created with varying volumes between 0.25 and 3.00 cubic meters. The boxes are stored in an array. The following code creates the boxes:

Random r = new Random(seed);
this.boxes = new Box[125];
for (int i = 0; i < 125; i++) {
 Box box = new Box(0.25 + (r.nextDouble() * 2.75));
 box.setId(i);
 this.boxes[i] = box;
}

Before we configure JGAP we must first implement a fitness function. The fitness function is the most important part in genetic algorithms as it determines which populations potentially make in to the next evolution. The fitness function for this problem looks like this:

package nl.jamiecraane.mover;

import org.jgap.FitnessFunction;
import org.jgap.IChromosome;

/**
* Fitness function for the Mover example. See this
* {@link #evaluate(IChromosome)} for the actual fitness function.
*/
public class MoverFitnessFunction extends FitnessFunction {
  private Box[] boxes;
  private double vanCapacity;

  public void setVanCapacity(double vanCapacity) {
          this.vanCapacity = vanCapacity;
  }

  public void setBoxes(Box[] boxes) {
          this.boxes = boxes;
  }

  /**
   * Fitness function. A lower value value means the difference between the
   * total volume of boxes in a van is small, which is better. This means a
   * more optimal distribution of boxes in the vans. The number of vans needed
   * is multiplied by the size difference as more vans are more expensive.
   */
  @Override
  protected double evaluate(IChromosome a_subject) {
          double wastedVolume = 0.0D;

          double sizeInVan = 0.0D;
          int numberOfVansNeeded = 1;
          for (int i = 0; i < boxes.length; i++) {
                  int index = (Integer) a_subject.getGene(i).getAllele();
                  if ((sizeInVan + this.boxes[index].getVolume()) <= vanCapacity) {
                          sizeInVan += this.boxes[index].getVolume();
                  } else {
                          // Compute the difference
                          numberOfVansNeeded++;
                          wastedVolume += Math.abs(vanCapacity - sizeInVan);
                          // Make sure we put the box which did not fit in this van in the next van
                          sizeInVan = this.boxes[index].getVolume();
                  }
          }
          // Take into account the number of vans needed. More vans produce a higher fitness value.
          return wastedVolume * numberOfVansNeeded;
}
}

The above fitness function loops through all the genes in the supplied potential solution (where each gene in the chromosome represents an index in the box array) and calculates how many vans are needed for this arrangement of boxes to fit in the vans. The fitness value is based on the space wasted in every van when a new van is needed, called the wasted volume. The total volume wasted is multiplied by the number of vans needed. This is done to create a much worse fitness value when more vans are needed. In the above, simplified, example the fitness value of the first solution is 20 and the fitness value of the second, optimal, solution is 3. One term deserves more explanation and that is allele. In the above code the getAllele method on the gene is called. Allele is just another word for the value of the gene. Because all genes all IntegerGene's, the value of each gene is of type Integer.

Next it is time to setup JGAP:

private Genotype configureJGAP() throws InvalidConfigurationException {
 Configuration gaConf = new DefaultConfiguration();
// Here we specify a fitness evaluator where lower values means a better fitness
Configuration.resetProperty(Configuration.PROPERTY_FITEVAL_INST);
gaConf.setFitnessEvaluator(new DeltaFitnessEvaluator());

// Only use the swapping operator. Other operations makes no sense here
// and the size of the chromosome must remain constant
gaConf.getGeneticOperators().clear();
SwappingMutationOperator swapper = new SwappingMutationOperator(gaConf);
gaConf.addGeneticOperator(swapper);

// We are only interested in the most fittest individual
gaConf.setPreservFittestIndividual(true);
gaConf.setKeepPopulationSizeConstant(false);

gaConf.setPopulationSize(50);
// The number of chromosomes is the number of boxes we have. Every chromosome represents one box.
int chromeSize = this.boxes.length;
Genotype genotype;

// Setup the structure with which to evolve the solution of the problem.
// An IntegerGene is used. This gene represents the index of a box in the boxes array.
IChromosome sampleChromosome = new Chromosome(gaConf, new IntegerGene(gaConf), chromeSize);
gaConf.setSampleChromosome(sampleChromosome);
// Setup the fitness function
MoverFitnessFunction fitnessFunction = new MoverFitnessFunction();
fitnessFunction.setBoxes(this.boxes);
fitnessFunction.setVanCapacity(VOLUME_OF_VANS);
gaConf.setFitnessFunction(fitnessFunction);

// Because the IntegerGenes are initialized randomly, it is neccesary to set the values to the index. Values range from 0..boxes.length
genotype = Genotype.randomInitialGenotype(gaConf);
List chromosomes = genotype.getPopulation().getChromosomes();
for (Object chromosome : chromosomes) {
    IChromosome chrom = (IChromosome) chromosome;
    for (int j = 0; j < chrom.size(); j++) {
        Gene gene = chrom.getGene(j);
        gene.setAllele(j);
    }
}

return genotype;
}

In the above code we setup the JGAP library. The provided Javadoc should be self-explanatory. A population (genotype) of 50 potential solutions (chromosomes) is created where every chromosome consists of the same number of genes as there are boxes. Because in this example a lower fitness value is better, theDeltaFitnessEvaluator is used.

Next, it is time to evolve the population. The population is evolved 5000 times. When the optimal amount of vans is reached earlier, the run ends. The following code demonstrates the evolution of the problem solution:

private void evolve(Genotype a_genotype) {
int optimalNumberOfVans = (int) Math.ceil(this.totalVolumeOfBoxes / VOLUME_OF_VANS);
LOG.info("The optimal number of vans needed is [" + optimalNumberOfVans + "]");
     
double previousFittest = a_genotype.getFittestChromosome().getFitnessValue();
numberOfVansNeeded = Integer.MAX_VALUE;
for (int i = 0; i < NUMBER_OF_EVOLUTIONS; i++) {
   if (i % 250 == 0) {
       LOG.info("Number of evolutions [" + i + "]");
   }
   a_genotype.evolve();
   double fittness = a_genotype.getFittestChromosome().getFitnessValue();
   int vansNeeded = this.numberOfVansNeeded(a_genotype.getFittestChromosome().getGenes()).size();
   if (fittness < previousFittest && vansNeeded < numberOfVansNeeded) {
       this.printSolution(a_genotype.getFittestChromosome());
       previousFittest = fittness;
       numberOfVansNeeded = vansNeeded;
   }
             
   // No more optimal solutions
   if (numberOfVansNeeded == optimalNumberOfVans) {
       break;
   }
}
IChromosome fittest = a_genotype.getFittestChromosome();

List<Van> vans = numberOfVansNeeded(fittest.getGenes());
printVans(vans);
this.printSolution(fittest);
}

Because we set the preserveFittest property on the JGAP configuration object to true, we have access to the most fittest chromosome with the getFittestChromosome() method. The fittest chromosome consists of 125 genes, the indexes of the boxes in the array, in the arrangement of how to put the boxes in the vans. The actual evolution is performed by JGAP. The fitness value determines which populations have the highest chance to make it to the next evolution. Eventually a (near) optimal solution is formed. This indicates the importance of a well chosen fitness function as it is used in the selection process of the chromosomes. Below is the output of a sample run:

The total volume of the [125] boxes is [210.25989987666645] cubic metres.
The optimal number of vans needed is [49]
Number of evolutions [0]
Fitness value [4123.992085987977]
The total number of vans needed is [63]
Fitness value [3458.197333300851]
The total number of vans needed is [61]
Fitness value [3138.2899569572887]
The total number of vans needed is [60]
Fitness value [2865.5105375433063]
The total number of vans needed is [59]
Fitness value [2562.282028584251]
The total number of vans needed is [58]
Fitness value [2267.7135196251966]
The total number of vans needed is [57]
Fitness value [1981.8050106661412]
The total number of vans needed is [56]
Fitness value [1704.5565017070858]
The total number of vans needed is [55]
Fitness value [1479.769464870246]
The total number of vans needed is [54]
Number of evolutions [250]
Fitness value [1215.9278601031112]
The total number of vans needed is [53]
Fitness value [1002.6487336510297]
The total number of vans needed is [52]
Number of evolutions [500]
Fitness value [774.5352329142294]
The total number of vans needed is [51]
Number of evolutions [750]
Number of evolutions [1000]
Fitness value [535.1696373214758]
The total number of vans needed is [50]
Number of evolutions [1250]
Number of evolutions [1500]
Number of evolutions [1750]
Number of evolutions [2000]
Number of evolutions [2250]
Fitness value [307.8713731063958]
The total number of vans needed is [49]
Van [1] has contents with a total volume of [4.204196540671411] and contains the following boxes:
Box:0, volume [2.2510465109421443] cubic metres.
Box:117, volume [1.9531500297292665] cubic metres.
Van [2] has contents with a total volume of [4.185233471369987] and contains the following boxes:
Box:17, volume [1.0595047801111055] cubic metres.
Box:110, volume [0.5031165156303853] cubic metres.
Box:26, volume [2.6226121756284955] cubic metres.
Van [3] has contents with a total volume of [4.312990612147265] and contains the following boxes:
Box:91, volume [1.8897555340430203] cubic metres.
Box:6, volume [2.423235078104245] cubic metres.
...Further output omitted

As seen in the above output, the optimal number of vans is reached between 2250 and 2500 evolutions. The program also outputs the distribution of the boxes in the vans. The complete source code of the above example can be downloaded from http://code.google.com/p/jc-examples/. The down-loadable artifact is called ga-moving-example-1.0.jar.

Conclusion
Genetic algorithms and programming is an exciting technology but with a lot of concepts which are hard to grasp if you are new to the field. This post is an introduction of the concepts for genetic algorithms and showed how to implement a GA solution with JGAP. In one of my next posts, genetic programming is explained with a concrete example.

The silver bullet rule applies to genetic algorithms as well. To give you an impression, the following problems are good candidates to solve with GA:

Problems where it is hard to find a solution but once a solution is found, measure how good this particular solution is.
Problems where the search space is very large, complex or poorly understood.
Problems where a near optimal solution is acceptable.
Problems where no mathematical analysis is available.

Although I am not an expert on the subject, feel free to ask your questions and I will try to answer them as best as possible.

Resources
[1] Field guide to genetic programming
[2] http://jgap.sf.net
[3] http://en.wikipedia.org/wiki/Genetic_algorithm
[5] http://www.genetic-programming.org/
[6] http://code.google.com/p/jc-examples/.
[7] http://en.wikipedia.org/wiki/Knapsack_problem.
[8] http://en.wikipedia.org/wiki/Bin_packing_problem.

2009-01-28T21:41:00.004+01:00

I finally found the time to finish my FolderVisualizer application. To recap: FolderVisualizer draws a Tree Map of a selected folder which makes it easy to see which folders/files take up the most space. You can also delete these folders from within the application. I started this project with a colleague to do a comparison between Adobe Air and JavaFX.

Version 2.0.0RC1 is down-loadable from the project's website. The following screenshot gives an impression of the application:

The application implements the following functionality:

Select a folder which must be visualized in the tree map

drill down to other folders

change the number (density) of the folders in the tree map

provide a breadcrumb navigation path for easy navigation

Delete the selected folder/file

Visual effects when changing view stacks

More appealing theme/colors than the previous version

Version 2 is a complete rewrite of the whole application. Because a lot of new functionality is added, I restructured the application to make it more maintainable and transparent. As a guideline, I used the principles of MVCS, which is basically a set of design guidelines to implement the Model View Controller Service architecture in Flex applications. It is NOT a framework unlike for example Cairngorm. I Used Cairngorm on another project but I had the impression that the framework gets in your way. I found myself writing to much boilerplate code like commands, custom events, controllers etc. I'm NOT saying that Cairngorm is a bad framework, but I found it to complicated for the task at hand: structuring a Flex application in a well defined matter.

On the other hand, I really like the clarity and simplicity of the MVCS approach and the freedom to adapt the principles to your application's needs. The FolderVisualizer application consists of a couple controllers, a model class which contains references to the application's model objects, a FileService and components which make up the view. To handle user interaction, I used the event bubbling approach. In the event bubbling approach, when the user clicks on a button, a UIEvent is dispatched. This event "bubbles" up to the parent who handles it, usually a controller. This controller is responsible for handling the business logic associated with this specific user action. Consider the following example, taken from the FolderVisualizer application:

<?xml version="1.0" encoding="utf-8"?>
<mx:HBox xmlns:mx="http://www.adobe.com/2006/mxml">
    <mx:Metadata>
        [Event(name="uiEvent",type="nl.foldervisualizer.events.UIEvent")]
    </mx:Metadata>
    
    <mx:Script>
        <![CDATA[
           ... Some code omitted
            
            [Bindable]
            public var model : Model;
            
            private function handleBack(event:Event):void {
                dispatchEvent(new UIEvent(UIEventKind.BACK))
            }
            
          ... Some code omitted
        ]]>
    </mx:Script>
    
    <mx:Spacer width="5"/>
    <mx:Button label="Back" click="handleBack(event)"/>
    ... Some code omitted
</mx:HBox>

The above code is taken from the NavigationControlBar.mxml which implements the navigational actions a user can perform. When a user clicks on the back button, a UIEvent is dispatched of kind UIEventKind.BACK. Also note that this component defines an event "uiEvent" which parent containers can use to add an eventListener which listens for UIEvents. See the following code-snippet taken from FolderVisualizer.mxml, which is the parent container of the NavigationControlBar:

<?xml version="1.0" encoding="utf-8"?>
<mx:HBox xmlns:mx="http://www.adobe.com/2006/mxml"
    xmlns:toolbox="http://www.flextoolbox.com/2006/mxml"
    xmlns:view="nl.foldervisualizer.view.*" 
    xmlns:controller="nl.foldervisualizer.controller.*"
    xmlns:model="nl.foldervisualizer.model.*"
    creationComplete="handleCreationComplete()">
    
    ... Some code omitted
            
    <!-- Controllers used in the application, could also use dependency injection here (for example Prana) -->            
    <controller:AppController id="appController" model="{model}"/>
    <controller:NavigationController id="navController" model="{model}"/>
    
    <view:ControlBar width="100" height="100%" uiEvent="appController.handleUIEvent(event);"/>
        
    <mx:VBox width="100%" height="100%" verticalGap="0">        
        <view:NavigationControlBar height="35" width="100%" verticalAlign="middle" horizontalAlign="left" horizontalGap="0" model="{model}" uiEvent="navController.handleUIEvent(event);"/>
        ... Some code omitted
    </mx:VBox>
</mx:HBox>

As you can see in the above code, the navController.handleUIEvent method handles all UIEvents dispatched from the NavigationControlBar. One way to handle the events in the handleUIEvent method is the use of a switch statement. For every UIEventKind which the controller must handle, a separate case in the switch statement is used which calls the correct method. Consider the following example:

public function handleUIEvent(event:UIEvent) : void {
    switch (event.kind) {
        case UIEventKind.BACK:
            // handle event
        break;
        case UIEvent.SHUTDOWN:
            // Handle event
        break;
        default:
            trace("No event handler defined")
    }
}

What I don't like about this implementation is the excessive amount of switch-case code to handle the events. For every controller in your application which must handle UIEvents, a switch block must be implemented. Instead of using a switch block, I used a different approach for handling UIEvents in controllers. I implemented a MultiActionController which acts as a base class of all controllers which must be able to handle UIEvents. The following code snippet comes from the MultiActionController (some code omitted):

public class MultiActionController extends BaseController
{
        /**
        * Mappings between UIEvent.kind (see UIEventKind) to functions. Those functions are defined in
         * the subclass.
        */
        protected var _handlerMappings:HashMap = new HashMap();
        
        public function MultiActionController()
        {
                super();
        }
        
        public function handleUIEvent(event:UIEvent) : void {
                var funct:Function = _handlerMappings.getValue(event.kind)
                if (funct==null) {
                        trace("No function for [" + event.kind + "] is defined in the handlerMappings. Make sure you define this event to function mapping in the constructor of you base class. See the ASDocs of the MultiActionController.")
                        throw new IllegalOperationError("No function for [" + event.kind + "] is defined in the handlerMappings. Make sure you define this event to function mapping in the constructor of you base class. See the ASDocs of the MultiActionController.")
                }
                funct.call(this, event)
                event.stopPropagation()
        }               
}

The MultiActionController defines a _handlerMapping variable of type HashMap. This hashmap contains the mapping between UIEventKind and functions (functions are objects in ActionScript 3). When the handleUIEvent is called, this method looks up the function belonging to the UIEvent.kind property. If this function exists, it is called passing the UIEvent as a parameter and stops propagation for this event. Otherwise an IllegalOperationError is thrown. Now take a look at the NavigationController which extends MultiActionController:

{
    public class NavigationController extends MultiActionController
    {
            private var _model:Model;
    
            public function set model(model:Model):void {
                    _model = model
            }
            
            public function NavigationController()
            {
                    super();
                    super._handlerMappings.put(UIEventKind.BACK, this.handleBack)
                    super._handlerMappings.put(UIEventKind.DIRECT_LINK, this.handleDirect)
                    super._handlerMappings.put(UIEventKind.DENSITY_CHANGE, this.handleDensityChanged)
                    super._handlerMappings.put(UIEventKind.DRILLDOWN, this.handleDrilldown)
            }
            
            private function handleBack(event:UIEvent):void {
                    ... Do actual work here
            }      
            
           ... Some code omitted 
                           
    }
}

The constructor of the NavigationController defines the mappings between the UIEvents this controller handles, and the corresponding functions to call. As you can see, there is no need to write blocks of switch statements in every controller to handle UIEvents. Just define the mapping between the UIEvents and functions in the constructor of the corresponding controller. One thing to be aware of is that all functions that are called by the MultiActionController accept a single argument of type UIEvent. This should not be a problem because the only way to call these functions is by dispatching a UIEvent. The UIEvent.payload property should contain all data required for the function to do its work.

Note that this is NOT a framework but an implementation variant of the MVCS principle for handling UIEvents. Feel free to use this code in your own project.

Conclusion

Structuring and organizaing a Flex (and any other!) application gives you the following benefits:

easier to maintain the application

easier to extend the application with new funcitonality

easier to test the application

easier to understand the application for new team members

Implementing the principles of the MVCS approach is an effective way of organizing and structuring Flex applications. The principles of MVCS are not restricted to a particular implementation. Instead, they can be implemented in various ways that best suites the needs of your application. In this post, I showed you a variation of handling UIEvents which reduces the amount of code necessary to handle these events. When you work on moderate to large Flex applications, have a look at the MVCS guidelines to structure the application. The time you invest does get pay back. The complete source code of the FolderVisualizer application can be found here.

One final note: although I did not use a DI framework, for example Prana, using a DI framework can be an effective way to decrease the coupling between components and increase testability. At the moment, my DI framework of choice for Flex applications is Prana.

Resources

Folder Visualizer
MVCS
Tree map component

text-image-generator example

2009-01-20T15:48:00.003+01:00

As a follow-up on my previous post, here is an example of how to use the text-image generation library. Consider the following code:


        TextImage   testImage = new TextImageImpl(400, 250, new Margin(0, -5));
    
        // Declare or read the fonts you need
        Font header = new Font("Sans-Serif", Font.BOLD, 15);
        Font plain = new Font("Sans-Serif", Font.PLAIN, 12);
        
        // Read in any images you need
        InputStream is = Test.class.getResourceAsStream("/warning.jpg");
        Image warning = ImageIO.read(is);
        // Put close() in try-finally block with production code! 
        is.close();
        
        // 1. specify font and write text with a newline
        testImage.useFont(header).writeLine("Example usage text-image-generator.").newLine();
        // 2. enabled the wrapping of text and write text which is automatically wrapped 
        testImage.useFont(plain).wrap(true).write("This is an example of how to use the text-image generation library. With this library you can dynamically generated textbased content as images (png or jpeg). Notice how this sentence is automatically wrapped. See the code at http://code.google.com/p/textimagegenerator/ for more examples.").newLine();
        
        // 3. Disable text-wrapping again. Write underlined text.
        testImage.wrap(false).useColor(Color.BLUE).useFontStyle(Style.UNDERLINED).write("This line of text is underlinedand blue.").useFontStyle(Style.PLAIN).newLine();
        testImage.writeLine("You can also embed images in your generated image:");
        // 4. embed the actual image. Here we use absolute positioning
        testImage.write(warning, 175, 175);
        
        // 5. Write the image as a png to a file
        OutputStream os = new FileOutputStream(new File("example.png"));
        testImage.createPng(os);
        os.close();

The code above generates the following image:

The text-image generation code can be downloaded at http://code.google.com/p/textimagegenerator/. Here you will find more examples of how to use this library. Hope you enjoy.

Creating Dynamic Images with Java and GridGain

2009-01-13T15:52:00.006+01:00

At my current client we have a requirement to dynamically create images with a specific text, for example an authorization code. These images are created in Java and encoded as a PNG file. For the generation of text based images I created a library which is available on Google Code, see [1]. These images are generated by a servlet of which the URL is embedded in the HTML as an image tag or in a background of a div element.

During performance tests I saw that the generation of images is a very CPU intensive task. Because there are a lot of users who request a dynamically generated image, this functionality can cripple the CPU's which has a negative impact on the throughput and the stability of the system. Because this is a specific, CPU intensive task, it is a perfect candidate to off-load this process to another machine so the CPU intensive task is distributed over a number of machines.

One way to distribute the processing of CPU intensive tasks is to use the open source grid computing platform GridGain. GridGain is "an open source computational grid framework that enables Java developers to improve general performance of processing intensive applications by splitting and parallelizing the workload." See also [3]. Image generation is a perfect candidate to distribute with GridGain.

I have created the following setup to do some first tests with GridGain in this specific scenario:

Created a web application with a servlet which is called by clients. This servlet is responsible for executing the image generation task on the grid and streaming the result back to the client as a PNG.

Created a GridGain task which creates a job which is responsible for the actual image generation. This job can be executed on any of the nodes participating in the grid.

Used two physical nodes in my first setup, node 1: PentiumM 2 GHz with 2 GB's RAM, node 2: Pentium4 2.2 GHz with 1 GB RAM.

Used Apache JMeter to create a load test [4].

See the following diagram for an overview of my setup:

I used 5 concurrent threads (users) in my JMeter script with a ramp-up time of 0ms which means all 5 users are active at once.

My first test run was used to create a baseline. This version did not use the grid at all and the image generation logic was implemented directly in the servlet. This test was run on node 1. With a couple of tests there was an average of 6.7 transactions per second.

My second test was used to determine the overhead of the grid. I expect the throughput to be a little less than in my first test. The same node was used as in 1. With a couple of tests there was an average of 6.5 transactions per second. Almost as good as without the grid. In this specific scenario the overhead of the grid is negligible. However, this can vary based on your specific situation.

In my third test I added a second node, node 2, to the grid. With a couple of tests there was an average of 9.5 transactions per second. This is a throughput increase of 46%. Pretty impressive. In my test, both nodes had 100% CPU utilization.

This is really impressive considering the amount of work I had to install and configure GridGain, almost nothing. With the default configuration, GridGain uses IP multi-cast to discover all the grid nodes. I just had to start gridGain on the second node and this node automatically participated in the grid. Other strategies can be used to implement the discovery process, for example JMS topics. When a given task is executed for the first time on a given node, the grid takes care of deploying this task on that specific node. No need for manual deployment of tasks. Failover and load balancing of tasks to other nodes when a node has crashed is enabled by default. Everything is also well documented which really helps creating a consistent package.

Conclusion
Distributing CPU intensive tasks across several physical machines is an effective way to increase the throughput and reducing the risk that this process will become a bottleneck or has a negative impact on the stability of the system as a whole. GridGain is a platform which enables (Java) developers to create a computing grid and execute tasks on this grid. When you have computational intensive tasks in your application, make sure you check out GridGain.

Resources
[1] Image Text library
[2] GridGain
[3] Highscalability.com
[4] JMeter

Radrace impressions

2008-10-02T09:00:00.009+02:00

On the 26th and 27th of September our company (QNH Application Development & Solutions) held the second internal Rapid Application Development race. There were 15 teams each consisting of 2 people competing against each other. As you already have read, a variety of technologies were used. The technology of our choice was Grails and the IDE we used was IntelliJ. Here are my impressions.

25th of September
23:30: The final preparations for the game tomorrow were made. I double checked the setup on my laptop, from Subversion to IntelliJ and all possible documentation we might need. I also did a final check on our pre-build application to see if everything still worked as expected.

26th of September, the first day
09:00: There was a briefing of the application that had to be build. The application was about the registration of driver licences which contain points. Points are subtracted when a driver makes an offence, like speeding or drinking while driving. Points are added when the driver shows good behaviour like taking a practical exam.
The functionality that was needed consisted of the following:

Login functionality for John Europe. John Europe must be able to see his driver licences and for each driver license the details. Details consist of his points left and the offences he made. He could also go into appeal for some of the offences. When a civilian goes into appeal, an amount of 100 Euro should be paid. A payment module with credit card check should be build.

Login functionality for the judge which treat appeals and determine if an appeal is valid or not. An e-mail is sent to the applicant of the appeal to inform him/her about the decision of the judge.

Login functionality for police officers which see an overview of all civilians which should have returned his/her driver license to the police.

Business rules which state how and when points are subtracted and added to the points on a driver license.

Make use of the existing database which is a Microsoft access database. The number of records in this database was close to 40.000 and consisted of multiple tables with relationships between those tables.

A couple of reports which display the civilians who needed to take practical exams and eye tests.

10:00: Let the games begin! We used the first 15 minutes to determine our strategy and made sure that we understood the functionality. After that we started working. I used the Mysql migration wizard to migrate the Access database to Mysql. We created a separate database for the original data and a new database for the application. Paul started working on the domain model and database and I started working on the authentication/authorization functionality. Whenever Paul had created a domain object and migrated the corresponding data, I started working on the controller and view for that functionality.

12:00: Lunch. We already had the login functionality, the eye test report and the data model for the rules working. For the report functionality we used straight sql with Spring’s JDBC template. There was not anything that had yet gone wrong.

13:00: We started coding again. I began working on the driver license information and the functionality for the judge and Paul continued to migrate the data. I used some Ajax to display the driver license details. We made sure we used the unique ids from the original data to maintain the relationships. Groovy scripts were used to migrate the data.

19:00: We completed the functionality for John Europe and the Judge. Both reports were finished. There were some improvements left to be made which we did the next day. We decided not to do anything with the rules yet because there were no real deliverables regarding those rules. Our expectation was that the changes, which were issued the next day, were regarding those rules.

20:00: We had a nice dinner with all of teams in a restaurant a talked a lot about this day. We tried to figure out what the rest of the teams were doing to measure our competition.

27th of September, the second day
08:00: We started with the optimizations of the use cases we finished the day before and made sure all functionality was working correctly. A total of three changes were issued. One of the changes was implementing the rules regarding speeding and parking in restricted areas. The rules were developed by me and Paul concentrated on completing the judge and police functionality. Some changes to the css were made to make the application a little more attractive than it already did.

11:45: We made sure everything was checked in and did a final test to make sure everything worked as expected.

12:00: Development time is over. We were pretty confident that we completed a lot of the functionality and scored a lot of points. However, we were not exactly sure if this would be enough to win the first prize. The jury began examining each application. Because of the huge amount of teams, the jury was divided in two so two teams could be judged at the same time to speed up the process.

15:00: When the first jury round was over, the best three teams had to demonstrate their application for the grand jury. We were one of the three best teams! We passed the test script and almost all of the functionality worked as expected.

16:00: Finally the awards. There were a couple of awards like the “best dressed award” which we did not win:). We already knew we were among the best three teams. When we heard we had not won the third prize we fell relieved. We could only be number one or two. The tension raised and eventually we heard that we came out first out of the total of 15 teams and both won a brand new Xbox 360 Elite console! All the preparations weren’t for nothing.

17:00: It was a great day and had a lot of fun with everybody. Till next year!

To see a video impression of the rad race check You Tube
Paul Bakker also wrote an impression which you can find here

Our first Open Space session

2008-09-23T19:47:00.003+02:00

Our first Open Space session was held on the 15th of September 2008. As you may have read, our internal RAD race is on the 26th and 27th of September. One of the open space subjects was how you can prepare yourself for the RAD race. This was the outcome:

Preparation is the most important thing

Database setup

E-mail client/server

Styling

Authentication/Authorization module

Print module

Payment module

Code generation

Books, pdf’s

Source Control System

Networking equipment

Working hardware

Report generator

As you can see there is a lot you can do to prepare yourself for the RAD race. What is your opinion about the preparation for a Rapid Application Development race?

Internal QNH Rad Race 2008

2008-09-12T09:19:00.003+02:00

26th and 27th of September, our company organizes the second internal Rad (Rapid Application Development) race. A Rad race is a competition in which several teams compete against each other to build an application in 1.5 days. The team which completes the most functionality wins. The teams consist of two people which work together and are responsible for creating their own development environment including version control, choice of IDE etc. They also choose their own technology to build the application in. It is your own responsibility to choose a technology you are familiar with. The technology we use to build the application is Grails and IntelliJ as our IDE. We find the Grails framework (and Groovy) very productive and a pleasure to work with (which is partly due to IntelliJ, one of the best IDE’s I have worked with).

The competition is tough though. There are 12 other teams which use the following technology:

PHP / Ruby on Rails

.NET

Oracle / APEX

Flex, Spring

Spring MVC

GWT, Spring and Hibernate

Wicket, Spring and Hibernate

Although several factors come into play which determine who is the winner, it is nice to see the pros and cons of each technology during the competition.

We will keep you updated!

Support for image acquisition devices in Flex/Air

2008-07-28T09:20:00.002+02:00

I would like to see support for image acquisition devices in Flex/Air. Currently there is no such support for image acquisition devices, like scanners that operate via the TWAIN API. However you can use sockets in combination with a Java application but this is far from an ideal scenario.

I filed an issue request which you can see at https://bugs.adobe.com/jira/browse/SDK-16239. Please vote on this issue and we may see support for image acquisition devices in the future.

Open Space sessions

2008-07-18T08:05:00.004+02:00

Upcoming august 2008 me and a colleague organize the first internal Open Space session for our company. Although the concept is not new, Open Space sessions date back from 1985, they gained momentum at Java Polis 2007. Here, Bruce Eckel held an excellent presentation about Open Space sessions.

I think knowledge and/or experience sharing is a key point in our profession as a software developer/engineer/architect etc. We constantly seek new and effective methods to share our knowledge. The most effective method for sharing knowledge is the one with the best cost/benefit tradeoffs. Costs have the form of the amount of time invested to prepare a session and the benefit is the amount of knowledge and experience shared amongst other colleagues.

The cost/benefit ratio of a traditional session/presentation is not very high. Usually, the preparation of a session costs far more than the benefit it gets. When this session is held multiple times, the cost/benefit ratio gets better. The format of a traditional session is of a very static nature. You have one or more presenters and a whole lot of attendees. The attendees do not know if the session is of any value for them. For some this may be the case and for others this may be not. Also, the attendees usually are not in the position to determine the subjects of the session.

Open Space sessions try to address these shortcomings. In an Open Space session the subject is determined by the attendees. The attendees seek a space where they can discuss/contribute and learn about the subject. One thing to remember in an open space session is the 'Law of the two feet'. This means: if you feel you do not learn anything or cannot contribute something meaningful to the session, use your two feet and seek another session which may interest you. This way you and the other attendees get the most value from a session.

I will post our experiences and the way we organize our Open Space session in the next month. Stay tuned...

More information about Open Space sessions can be found at:

Open Space world

Wikipedia

JavaFX and Air: a brief comparison

2008-06-30T12:55:00.002+02:00

Paul Bakker has wrote an excellent post of our first impressions when developing the FolderVisualizer. Check it out and also don't forget to install and try both applications!

Efficiently reclaim you disk space

2008-06-10T21:51:00.003+02:00

We started a new project on Google code named FolderVisualizer.

The application is able to visualize the relative size of files and folders below a given directory. With this tool it is easy to see which files and folders take up the most disk space which you then can use to efficiently clean your hard drive. See Olav Maassen's post about efficiency :).

Although such tool already exists, it is fun to see how different implementations compare to each other. At the moment, the following technologies are planned:

A Groovy implementation with a command line client

A Java implementation with a command line client

A JavaFX client

An Adobe AIR client

It is nice to see that the JavaFX client can make use of the Groovy implementation of the visualizer algorithm. For the AIR implementation we have to rewrite the algorithm in Action Script.

The project can be found here.

Once again: Assumptions are the mother of all fuck ups!

2008-06-09T09:57:00.000+02:00

Even the most obvious assumptions aren't all that obvious. Who believes that a software tester tests a new release in the production environment! This has actually happened. When a new release was installed in the acceptance environment, the acceptance tester was signalled.

When he finished testing, a whole list of issues was reported via the issues tracker. We had absolutely no idea of what was going on until we asked for the URL and user accounts with which the tester had tested. The URL that was used was actually the URL of the production environment! After that it was totally clear to us what went wrong.

Lesson learned: even if your assumption is obvious, do not assume!

Code review

2007-07-04T08:22:00.002+02:00

During a code review I stumbled accross the following code:

SomeObject someObject; 
if(o instanceof SomeObject) {
 someObject = (SomeObject)o;
} else {
 throw new ClassCastException("Cannot cast=[" + o.getClass() + "] to expected class=["+ SomeObject.class + "]");
}

Obviously the throwing of the ClassCastException is redundant. You can replace the above code with the following statement:

SomeObject someObject = (SomeObject)o;

This statement doas semantically the exact same thing! At runtime a ClassCastException is thrown when object o cannot be cast to SomeObject. By looking at the stacktrace you can figure out where in your code things go wrong.

Moral of this story: use code reviews to improve the quality of your code.

Using Spring elements and attributes in your custom namespace

2007-06-27T14:58:00.001+02:00

With Spring 2 it is possible to define your own namespace and have your own custom elements which you can use in the application context. Sometimes, allthough I do not promote this, you want to use Spring elements and attributes on your custom elements. For example: suppose I have a custom element named "foo:myCustomFooElement" with one attribute named "text". By default, you can't write something like this:

< foo:mycustomfooelement text="test" abstract="true">
   < property name="property" value="value">
< /foo:myCustomFooElement>

This is because the property element and abstract attribute are not defined in your xsd which describes your custom element. Following is the source of a sample application I wrote to demonstrate the use of Spring properties and attributes on custom elements. The reason I do not promote this, is because you will loose the encapsulation of implementation details which your custom element hides. So use it with care.

public class Foo {
    // This property gets injected by our custom FooParser
    private String text;

    // This property gets set by using Spring's < property> element
    private Bar bar;

    // getters and setters omitted

}

We will write a custom element for the Foo class.

public class Bar {
    private int count;

        // getter and setters omitted
}

Bar is just a regular class that gets injected in Foo. Below is the xsd I wrote to define myCustomFooElement:

< ?xml version="1.0" encoding="UTF-8"?>
< xsd:schema xmlns="http://www.mynamespace.com/foo"
    xmlns:xsd="http://www.w3.org/2001/XMLSchema"
    xmlns:beans="http://www.springframework.org/schema/beans"
    targetNamespace="http://www.mynamespace.com/foo"
    elementFormDefault="qualified" attributeFormDefault="unqualified">

    < xsd:import
        schemaLocation="http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans-2.0.xsd"
        namespace="http://www.springframework.org/schema/beans" />

    < xsd:element name="myCustomFooElement">
        < xsd:complextype>
            < xsd:complexcontent>
                < xsd:extension base="beans:identifiedType">
                    <!-- Here we re-use Spring elements and attributes -->
                    < xsd:group ref="beans:beanElements">
                    < xsd:attributegroup ref="beans:beanAttributes">
                    < xsd:attribute name="text" type="xsd:string"
                        use="required" />
                < /xsd:extension>
            < /xsd:complexContent>
        < /xsd:complexType>
    < /xsd:element>
< /xsd:schema>

The XML file containing the bean definitions looks like this:

< beans xmlns="http://www.springframework.org/schema/beans"
    xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
    xmlns:foo="http://www.mynamespace.com/foo"
    xsi:schemaLocation="http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans-2.0.xsd
    http://www.mynamespace.com/foo http://www.mynamespace.com/foo/foo.xsd">
   
    < foo:mycustomfooelement id="foo" text="test" init="true">
        <!-- Here we use a Spring property element inside our custom element -->
        < property name="bar">
            < bean class="nl.sample.customnamespace.Bar">
                < property name="count" value="100">
            < /bean>
        < /property>
    < /foo:myCustomFooElement>
< /beans>

To parse the myCustomFooElement we must create a namespacehandler and register a beandefinition parser like this:

public class FooNameSpaceHandler extends NamespaceHandlerSupport {
    private static final String NAME_CUSTOM_FOO_ELEMENT = "myCustomFooElement";

    public void init() {
        registerBeanDefinitionParser(NAME_CUSTOM_FOO_ELEMENT, new FooParser());
    }

    class FooParser extends AbstractBeanDefinitionParser  {
        @Override
        protected AbstractBeanDefinition parseInternal(Element element, ParserContext parserContext) {
            // Here we parse the Spring elements such as < property>
            BeanDefinitionHolder holder = parserContext.getDelegate().parseBeanDefinitionElement(element);
            BeanDefinition bd = holder.getBeanDefinition();
            bd.setBeanClassName(Foo.class.getName());
           
            // Here we parse our custom attributes
            String text = element.getAttribute("text");
            bd.getPropertyValues().addPropertyValue("text", text);
            parserContext.getRegistry().registerBeanDefinition(NAME_CUSTOM_FOO_ELEMENT, bd);
            return (AbstractBeanDefinition) bd;
        }
    }
}

Don't forget to register the namespacehandler in a file called spring.handlers in the META-INF directory. The contents of de spring.handlers file looks like this:

http\://www.mynamespace.com/foo=nl.sample.customnamespace.FooNameSpaceHandler

Also, specify in the META-INF/spring.schemas file where Spring can find the foo.xsd:

http\://www.mynamespace.com/foo/foo.xsd=nl/sample/customnamespace/foo.xsd

Write a class to verify everything works:

public class Driver {
    public static void main(String[] args) {
        ClassPathXmlApplicationContext appContext = new ClassPathXmlApplicationContext("classpath:nl/sample/customnamespace/customelements.xml");
        Foo foo = (Foo) appContext.getBean("foo");
        System.out.println("foo.text = " + foo.getText());
        System.out.println("foo.bar.count = " + foo.getBar().getCount());
    }
}

Extensive documentation about extensible XML authoring can be found in the reference manual. Eriks Wiersma has also written an excellent piece on his [URL="http://erik.jteam.nl/?p=23"]blog[/URL] about implementing custom namespaces.

Comparing two XML documents

2007-03-06T09:21:00.000+01:00

When writing unit tests involving XML you often want to compare two XML documents. This can easily be done using a simple String compare as in:

expectedXML.equals(responseXML);

Where ‘expectedXML’ is the XML we expect to get back and ‘responseXML’ is the actual XML that is given as a response.

The problem with this approach is that it not only compares the actual XML but also any unimportant whitespace, tabs etc. This is not the desired behavior because it breaks our test when, for example, whitespaces are added in de reponseXML.

One solution to this problem is to use a handy utility class named DocumentComparator, which is part of the JIBX distribution. The javadoc of the DocumentComparator says the following:

XML document comparator. This uses XMLPull parsers to read a pair of documents in parallel, comparing the streams of components seen from the two documents. The comparison ignores differences in whitespace separating elements, but treats whitespace as significant within elements with only character data content.

This is exactly the functionality we need. With this class we can write the following code to compare two XML documents:

ByteArrayOutputStream out = new ByteArrayOutputStream();
DocumentComparator comp = new DocumentComparator(new PrintStream(out));
boolean result =comp.compare(new StringReader(responseXml), new StringReader(expectedXml));
assertEquals(out.toString(), true, result);

The DocumentComparator class uses the PrintStream to report any differences found when comparing the two documents.

Core Spring training experience

2007-02-26T11:49:00.000+01:00

The training was given by Arjen Poutsma. Arjen works for Interface21 and is the lead developer of the Spring-WS sub project.

Day 1, 20 february 2007

Day 1 is all about the core concepts of the Spring framework like Inversion of Control and Dependency Injection. IOC and DI are essential parts to understanding Spring's architecture. Besides some basic subjects about IOC and DI there were also advanced concepts that were given attention: autowiring, life cycle callbacks, property editors, factory beans, post processors etc. The Lab's in day 1 were all about configuring application contexts. However there was also a bonus section which programmatically used AOP via ProxyFactory(s) with dynamic TargetSource(s).

If you already know Spring than the subjects in day 1 are familiar.

Day 2, 21 february 2007

Day 2 was all about effective middle tier architecture with Spring. Subjects that got attention were:

Persistence with Spring JDBC and ORM tools
Data Access Objects with Spring
Brief introduction about transaction management
Test Driven Development with Spring
Introduction to AOP
Advanced data access concepts such as: batch updates, blob handling and stored procedures.

The labs were about using Spring in conjunction with Hibernate and using Spring utilities for creating integration tests.

Day 3, 22 february 2007

After a brief introduction about transaction management in day 2, this day continued with the more advanced topics about transaction management. Specifically:

local versus global transactions
declarative versus programmatic
transaction propagation
isolation levels

The rest of the day was about Spring-MVC including several labs covering handler mappings, input forms and interceptors.

The last subject for this day was an introduction about Acegi security system for Spring.

Day 4, 23 february 2007

For me, day 4 was most interesting. This was mainly because AOP got in depth-attention. Arjen focused on Spring 2's support for AOP with Spring and AspectJ. Some advanced concepts such as introductions also got attention.

The remainder of the day was used to tell something about Spring's support for JMX and remoting. Arjens's project Spring-WS got little attention. The main point about Spring-WS is that it promotes contract-first development.

Things to remember

Spring 2.1 will not be compatible anymore with JDK 1.3
The upcoming new version of Spring IDE will support dependency listing for aspects build with Spring-AOP. With this feature you can view all aspects that are applied to a particular class.
Migrating from Spring-AOP to AspectJ is easier when using the @Aspect annotations in Java classes rather than using XML to configure aspects. This is because AspectJ understands these annotations.

Conclusion

Although I had experience with Spring before taking this training, I am glad I took it. A lot of concepts used in Spring were covered in detail. I can recommend this training to anyone who would like to gain more in depth knowledge of the Spring framework.

When taking this training, I found it very helpful to have knowledge about enterprise application patterns, transactions, J2EE, TDD, DDD, Hibernate etc.

Core Spring training

2007-02-19T09:03:00.001+01:00

This week I am happy to attend the Core Spring training in Amsterdam given by Interface21. The Core Spring training is a full 4 day training covering all aspects on Spring from IOC and dependency injection to Spring MVC and testing with Spring. This week I will post my day-to-day experiences on this training. More information on this training can be found at: www.interface21.com.

Service contracts

2007-02-16T16:06:00.000+01:00

XML is a common format for message exchange in Service Oriented Architectures. One of the most important things is to formalize the messages exchanged between systems. This can be done using XML schema definitions. XML schema defines the rules to which a XML message must conform when it is constructed. A well defined schema brings the following advantages:

Validate messages when they are constructed and thus before they are sent.
Generate Java classes from the XML schema for use in XML binding frameworks.
Easily construct mock XML messages with a tool like XMLSpy for use in (unit)tests or mock implementations of services.

With “well defined schema” I mean a schema definition which is constructed before the actual service is implemented so every element in the schema is well thought out. Another example: when some element’s value is an enumeration of some values, define this in the schema definition and not in a separate document (I,ve seen it happen). This way you restrict the values an element can contain and it becomes much easier to construct mock XML messages for testing purposes.

The bottom line is: invest in the construction of well thought out service contracts.

A good reference for designing XML schema definitions can be found here.

Starting up...

2007-02-16T09:29:00.000+01:00

Finally created my Blog today to post my ideas, thoughts and experiences about Enterprise Java development. Although Java is the primary field of interest, it is not the only field of interest. For example, I will also post my latest experiences with Adobe Flex to build highly interactive user interfaces on top of Java backends. I Already can say that I am very impressed. But more on that in one of my next posts.

The only thing for now I would like to say is: Happy reading!