Another victim of The Wall of Erasure

Ahh, Erasure....  Today I was implementing an extension to a class that performs traversing of tree-like object structures (think UI component hierarchies).  The parent class extends another type-parameterized version of an abstract class which provides general tree traversal logic.  The problem is that I wanted to check that the current node is of a type beyond that of the type parameter provided to the parent class.  So to put that into something that might actually be understandable:

public class ParentFinder<T extends Component> extends ComponentVisitor {

 private T parent = null;

 public ParentFinder() {
  // Set up things to travel up the tree...
  ...
 }

 public void visit(Component node) {
  if(T.class.isAssignableFrom(node.getClass())
   parent = (T)node;
 }
}

Survey says! *BZZZZ* XXX *BZZZZ* This line is no good thanks to erasure:

  if(T.class.isAssignableFrom(node.getClass())

And, no...you cannot do "instanceof" either. The JVM couldn't tell you if it wanted to. That info is G-O-N-E, gone!

How dare you want to know what you are working with!

@BFNH This is a Big Fat Nasty Hack. Be ye' warned!

The above is my recent favorite @ tag. We all come upon those times when, no matter how much we try to avoid it, a Big Fat Nasty Hack is required. We try to make them pretty. We roll them in perfume, wrap them in silk and adorn them with flowers, but they remain BIG, FAT and NASTY. It's like a warthog in a wedding dress. Let's just call them what they are. The @BFNH tag calls these hacks out and exposes them to the world. Use it wisely.

I ran across this situation most recently when trying to insert cells into an Excel workbook that contained functions with references to named ranges. It would seem that Apache POI does not really support this. POI can read these cells in, but it is unable to provide a valid "function value" for the cell when calling HSSFCell.getFunctionValue(). Instead of a good function string, it gives something like "COUNT( NO IDEA )" or some such nonsense. Not being able to produce a function string for this type of function call, it also fails to parse a function of this type for insertion. I really needed this functionality. As I described in my previous post, I came upon a way to provide dynamic charts in Excel from Java. Seeing what could be done with this, my users went a little over-the-top. Excel reports with charts are something they have been requesting for a while, only to be rejected. So, I set upon the creation of the plethora of new "must have" reports. Many of the reports they want involved breaking up data into ranges and reporting on range statistics. They also wanted to be able to play with the raw data and see the results in the charts. This was a task.

What I decided to do was create named ranges referencing the desired "raw" data ranges for periods. Because this data is being queried based on the users parameters, the ranges must be calculated and created on the server. I planned to then reference these ranges in the cells of a table of period statistics. This would free me from calculating the statistics on the server and allow Excel to alter the charts as data in the ranges is changed. The only thing that was missing was how to go about setting a cell value to something like "MEDIAN("MyWorkbook.xls"!MyNamedRange)". I looked everywhere for a solution. I scoured the POI mailing lists and APIs, I tried a number of variations on the formula string, I searched Krugle like mad and I got nowhere. Not getting any response on lists or forums, and being one to take defeat likely, I took up my trusty debugger and set to probing the depths of POI. Let me warn those of weak constitution to stop reading here. The resulting code involves accessing the inaccessible, breaking the rules that the makers have set out for us and generally doing nasty things.

I created a number of cells that contained references of the type that I required, then created expressions in the debugger to expose the internal representation of the file that POI uses. I found that formulas are parsed into a list of references (to functions, ranges, etc.). I found that I could create formulas that resembled the ones that I desired, using a dummy static reference as a place holder and then swap the static reference for a dynamic one of my creation. This converts a formula like "AVERAGE(A:A)" to something like "AVERAGE("MyWorkbook.xls"!MyNamedRange)". The thing is, the POI guys/gals didn't really intend these things to be accessed directly. So, naturally, I promptly went against their wishes. The following is the result. Use it at your own risk, observe the comments and stated limitations, expand it, generalize it, whatever...but I would appreciate it if you relay any improvements back to me as well. ;)

    /**
     * The purpose of this method is to HACK OUT a static range reference from
     * POI's internal representation of a single range parameter function cell
     * and replace it with a reference to a existing named range. Because the
     * necessary fields and methods are not meant for general use, they are
     * inaccessible. As a result, reflection was used to obtain the necessary
     * references. This provides an opportunity for undetected errors to creep
     * in, as changes to these methods would not be detectable at compile time.
     * Furthermore, as non-accessible methods, they are more likely to be
     * subject to change with future releases.
     * 
     * The process for replacement of references is as follows: - Retrieve the
     * FormulaRecordAggregate from the target cell. - Reduce the FormulaRecord's
     * expression length by 2 (area references are larger than name references). -
     * Set the calculate on load option for the cell to true (options = 2). -
     * Create a new named range reference and set field_2_ilbl to the index of
     * the desired name in the workbook's name table. - Remove the static area
     * reference from the parsed expression (index = 0). - Insert the newly
     * created named range reference into the parsed expression (index = 0). -
     * Smile.
     * 
     * @BFNH  This is a Big Fat Nasty Hack. Be ye' warned!
     * @param results  the workbook being dealt with
     * @param targetcell  the cell in which the formula reference
     * @param nameName the name of the named range to reference
     */
    @SuppressWarnings("unchecked")
    protected void hackFunctionCellToSetNewNamePointer(HSSFWorkbook results,
     HSSFCell targetcell, String name) {
 try {
     short nameIndex = (short) (results.getNameIndex(name) + 1);

     // Get the original formula record aggregate. This must be done
     // via reflection, as it is not an accessible method.
     Method method = HSSFCell.class.getDeclaredMethod(
      "getCellValueRecord", new Class[] {});
     method.setAccessible(true);
     FormulaRecordAggregate original = (FormulaRecordAggregate) method
      .invoke(targetcell);

     // A static area reference is a couple of bytes larger than a name
     // reference. So, if we don't reduce the formula record size, we
     // will get corruption of the produced file.
     List parsedExpression = original.getFormulaRecord()
      .getParsedExpression();
     short length = (short) (original.getFormulaRecord()
      .getExpressionLength() - 2);
     original.getFormulaRecord().setExpressionLength(length);

     // Now we want to make sure that we cause the cell to be calculated
     // on-load. The following line should accomplish that. Note, that
     // if you have other options on the cell, they will be wiped out by
     // this. Chances are that won't effect you though.
     original.getFormulaRecord().setOptions((short) 2);

     // Now we create a name pointer...the constructor I'm using here has
     // not been implemented yet. If it were, we would likely not need to
     // set the "field_2_ilbl" value. From what I can tell, this field is
     // essentially the pointer to the name table in the workbook. We
     // give it the name index and leave the rest alone.
     NameXPtg nameptr = new NameXPtg(name);
     Field nameTableIndex = NameXPtg.class
      .getDeclaredField("field_2_ilbl");
     nameTableIndex.setAccessible(true);
     nameTableIndex.setShort(nameptr, nameIndex);

     // Now we remove the static area reference from the parsed
     // expression and add our name pointer in it's place. Please note
     // that I have only tried this with functions that take a single
     // name pointer and of those, only functions with 1 or 2 total
     // parameters. In the case of two parameters, I have only tested
     // when the second parameter is a constant. For example,
     // This should work with: COUNT("MyFile.xls"!MyName) or
     // AVERAGE("MyFile.xls"!MyName) or even
     // QUARTILE("MyFile.xls"!MyName, 1). I would imagine that the
     // process is similar for more complex function cells, but this is
     // all I have needed, so that's as far as I've gone. My guess is
     // that the length would have to be decremented once for each Area
     // to Name conversion you do and I have no speculation about where
     // the other area pointers might land in the parsed expression (in
     // terms of their index).
     parsedExpression.remove(0);
     parsedExpression.add(0, nameptr);
 } catch (Exception e) {
     // You screwed up and I don't care...write your own error handling.
     // ;)
     log.error("Bad developer...BAD!");
 }
    }

Excel Charting in Java

Ahh, Excel...  I hate it.  I find it rather unintuitive, to be honest.  No, no no...that is not quite saying enough.  I find it nearly impossible to use.  Business folk however, love it.  In every company that I have worked for, every user group I have served has, eventually, asked for something to be output to Excel.  Of course, for us Java folk, there is good old Apache POI.  Now, POI is pretty good.  I've used it in a number of projects and have not had any stability issues.  Performance is generally acceptable when dealing with reasonable amounts of data.  It does however, have a good number of limitations.  A large subset of the Excel functions are still unimplemented and there is almost no support for charts (To be specific: charts will be preserved but cannot be altered).  This is often a big stumbling block.  Furthermore, building spreadsheets with method calls is not a very efficient process for the developer.  Creating just a single spreadsheet can require a lot of code.

This is the problem I found myself up against recently.  In searching for a possible solution, I came up with a couple of really useful finds.  The first of which was jXLS.  jXLS is an excel templating framework of sorts.  It allows a designer to insert tags into a template spreadsheet which will be interpreted by jXLS and populated appropriately.  The tags utilize an expression language very similar to the EL used in JSPs or XSLT.  Developers who are comfortable with either of these technologies should feel right at home with jXLS.

Basically, you can create an excel sheet and throw in some expressions like so:



Then you run in through jXLS with something similar to:

Map beans = new HashMap();
beans.put("item", items);
XLSTransformer transformer = new XLSTransformer();
HSSFWorkbook results  = transformer.transformXLS(Thread.currentThread().getContextClassLoader()
    .getResourceAsStream("daysToProvideInstructions.xls"), beans);

As you can see by the fact that we get a HSSFWorkbook back from this method call, jXLS is based around POI, but it does the having lifting for the data insertion.  If you want to shove that file back out via a HttpServletResponse, you don't need much:

response.setContentType("application/vnd.ms-excel");
response.setHeader("Content-Disposition","attachment; filename=warrantyAgingReport.xls");
results.write(response.getOutputStream());

Put all this in your web app and you'll get:



So, jXLS gives me a nice and easy solution for populating a spreadsheet with data...but, we still cannot edit the ranges of any tables that we create.  Furthermore, if we want functions to be evaluated when the spreadsheet is opened, we have to evaluate them in POI, right?  Named Ranges to the rescue!  It just so happens that we can define a dynamic range in Excel and give it a name.  We can then use those named ranges in charts, functions, etc.  This results in a those items being dynamically evaluated and gives us the results we want.  So following Micro$oft's own instructions on the matter, we can create a couple of ranges in our template spreadsheet like so:



Using those ranges in a chart gives us the following:



Now we have the dynamic Excel charts we need, generated via our Java web application with very little excess effort.  So, while I still hate Excel...today I hate it from Java just a little less. :)

Drink More Guice...

Okay, okay...this post is, seriously, a long time coming. I have not posted anything for a while for a number of reasons. I could give the details, but there aren't enough of you out there who are going to care. So, I'll skip it and get into the goodies.

Reading this may actually kill two birds with a single stone for many of you. For the example project that I put together here, I made use of the Echo2 framework. There are a number of reasons that I chose to do this. First of all, I think that dependency injection (D.I.) can be particularly beneficial when working in a component model. I think the switch to a real component model in Java GUI development is long overdue, particularly in the world of web UI development. So, I am hoping that this example will convey the ease with which a simple UI can be created using a framework like Echo2 or GWT as well as provide a few simple, yet useful, ways to leverage D.I. when creating a user interface.

We have all seen dependency injection brought into the main stream by the Spring Framework. As many of its proponents will tell you, Spring is a very powerful and highly configurable tool, whose usefulness goes way beyond simple dependency injection. Many people see Guice as a competitor to Spring, and in some ways it is. However, Guice is not nearly as broad in functionality as Spring. Guice is aimed at being fast and simple for dependency injection only. One immediately evident difference between Spring and Guice is Guice having fully embraced annotations in its implementation of DI. While I must point out that while the JavaConfig project has brought annotation based configuration to Spring, the implementation is, arguably, more complex and results in a more intrusive presence in the most common circumstances where DI is being used. If you are familiar with Spring and JavaConfig, much of this will become evident as I continue.

Enough of this blathering, let's look at some code. I am going to start with what many tutorials will save to the end: the point of injection. Ideally, D.I. should be something that is as invisible as possible. Far too often you see people directly accessing a framework's DI context directly in several different places in their application code. This is a situation that I will try to avoid in this tutorial. To that end, I am going to try to find a place where injection can happen once and the effects will flow throughout the rest of the application. In an Echo2 application this choice is simple, every Echo2 application instance is feed off and attached to the user session via a WebContainerServlet implementation:

package com.amateuratbest.examples.guice;

import nextapp.echo2.app.ApplicationInstance;
import nextapp.echo2.webcontainer.WebContainerServlet;

import com.amateuratbest.examples.guice.layout.LayoutModule;
import com.google.inject.Guice;
import com.google.inject.Injector;
import com.google.inject.Module;
import com.google.inject.servlet.ServletModule;

/**
 * This is where all the Guice-iness goes on.  We will look for our injector
 * in the ServletContext, create it if necessary and then inject our newly
 * constructed ApplicationInstance.  All done!
 * 
 * @author mkimberlin
 */
public class TestGuiceServlet extends WebContainerServlet {

   private static final long serialVersionUID = 2715828643772899566L;

   /* (non-Javadoc)
    * @see nextapp.echo2.webcontainer.WebContainerServlet#newApplicationInstance()
    */
   @Override
   public ApplicationInstance newApplicationInstance() {
      TestGuiceApplication newApp = new TestGuiceApplication();
      // This uses the Injector class name as the attribute name...
      Injector injector = (Injector) getServletContext().getAttribute(
            Injector.class.getName());
      if (injector == null) {
         injector = Guice.createInjector(new Module[] {new LayoutModule(), new ServletModule() });
         getServletContext().setAttribute(Injector.class.getName(), injector);
      }
      injector.injectMembers(newApp);
      return newApp;
   }
}

In order to avoid constructing the injector more than once, I am keeping it in the ServletContext. If one isn't in context when I need it, it can be instructed by specifying the configuration modules that contain the desired configuration. Yes, you read that right. The configuration is done in Java, not XML. For the purposes of this example, I have a set of two modules that will be used for injection. In this case, I have one application specific module (LayoutModule) and one Guice-provided module (ServletModule). If you are working with a web application, you will almost definitely want to include this second module, as it defines injection scopes for request and session as well as providing bindings for injection into the request, session or response. It should be noted that there is nothing preventing the use of different modules as dictated by your applications needs. Module selection could be done based on user preferences, branding, or any number of other concerns. The main thing here is that, if I do things right, we should never see an injector again.

Now that we've seen our "point of entry", as it were, let us grab a quick look at those configuration modules:

package com.amateuratbest.examples.guice.layout;

import nextapp.echo2.app.Column;
import nextapp.echo2.app.SplitPane;

import com.amateuratbest.examples.guice.annotations.NamedAnnotation;
import com.google.inject.AbstractModule;

/**
 * This class contains the DI bindings for the main layout components.
 * 
 * @author mkimberlin
 */
public class LayoutModule extends AbstractModule {
   public static final String BANNER = "banner";
   public static final String NAVIGATION = "navigation";

   @Override
   protected void configure() {
      bind(Column.class).annotatedWith(new NamedAnnotation(BANNER))
         .to(BannerColumn.class);
      bind(SplitPane.class).annotatedWith(new NamedAnnotation(BANNER))
         .to(BannerSplitPane.class);
      bind(SplitPane.class).annotatedWith(new NamedAnnotation(NAVIGATION))
         .to(NavigationSplitPane.class);

      
      // Note that if you uncomment the following lines you will get a compile
      // time error.  Guice's use of generics means a far less error prone
      // configuration process.  If this were done in a config file, you would
      // have to wait until run time to discover your mistake.
      //
      //bind(Column.class).annotatedWith(new NamedAnnotation(BANNER))
      //   .to(BannerSplitPane.class);
   }
}

This module, lovingly dubbed "LayoutModule" extends the abstract class AbstractModule that is provided as a part of Guice. As you can see, you only need to override the configure() method and specify the bindings that you would like Guice to honor when doing injection based on this configuration. The first two calls to bind() produce bindings for the given classes for any case where the target is annotated with a named annotation constructed with the name "banner". Using named annotations like this is simply one way to accomplish multiple bindings for the same class. The third call to bind() produces a second binding for the SplitPane class. However, this binding only applies when the target was annotated with a the NamedAnnotation for "navigation".

The section of code that has been commented out can be uncommented to see how Guice's "config in code" approach results in a less error-prone configuration. Static typing results in a compile time error if you have configured a binding that doesn't make sense.

This next piece of code has a couple of methods that are requesting injection. What is worth attention in these is the fact that there was no binding necessary. If you look at the earlier configuration module, you will not find anything that creates a binding for these classes. Crazy Bob Lee and his ilk at Google appear to have put a good deal of effort in to minimizing the amount of configuration you have to do (of which this is but one example). In this particular case, I just need to make sure that any new instances get a rootWindow and layoutPane. Since Window has an available default constructors, Guice will just take care of things for us without having to be told. LayoutPane is a similar, but not identical circumstance, but I will investigate that in a moment.

package com.amateuratbest.examples.guice;

import nextapp.echo2.app.ApplicationInstance;
import nextapp.echo2.app.Window;
import nextapp.echo2.app.componentxml.ComponentXmlException;
import nextapp.echo2.app.componentxml.StyleSheetLoader;

import org.apache.commons.logging.Log;
import org.apache.commons.logging.LogFactory;

import com.amateuratbest.examples.guice.layout.LayoutPane;
import com.google.inject.Inject;

/**
 * Very simple implementation of the Echo2 ApplicationInstance.  Normally, we
 * would not want to have a check that is "injection aware", per se...but I did
 * here for "instructional" purposes.  Now I think I'll use some "more quotes",
 * "just because" i think they are "groovy".
 * 
 * @author mkimberlin
 */
public class TestGuiceApplication extends ApplicationInstance {
   private static Log log = LogFactory.getLog(TestGuiceApplication.class);

   private Window rootWindow;
   
   private LayoutPane layoutPane;
   
   /* (non-Javadoc)
    * @see nextapp.echo2.app.ApplicationInstance#init()
    */
   @Override
   public Window init() {
      try {
         setStyleSheet(StyleSheetLoader.load("/style.xml", Thread.currentThread().getContextClassLoader()));
      } catch (ComponentXmlException e) {
         log.warn("Could not load default styles for routine builder.");
      }
       
      if(rootWindow == null) {
         rootWindow = new Window();
         log.debug("Injection Failed...");
      } else {
         log.debug("Injection Worked...");         
      }
      rootWindow.setContent(layoutPane);
       return rootWindow;
   }

   public Window getRootWindow() {
      return rootWindow;
   }

   @Inject
   public void setRootWindow(Window rootWindow) {
      this.rootWindow = rootWindow;
   }

   public LayoutPane getLayoutPane() {
      return layoutPane;
   }

   @Inject
   public void setLayoutPane(LayoutPane layoutPane) {
      this.layoutPane = layoutPane;
   }
}

Unlike with Window, Guice is not using a default constructor for LayoutPane:

package com.amateuratbest.examples.guice.layout;

import nextapp.echo2.app.ContentPane;
import nextapp.echo2.app.SplitPane;

import com.amateuratbest.examples.guice.ComponentUtil;
import com.google.inject.Inject;
import com.google.inject.name.Named;

/**
 * Constructs and contains the main layout components of the application.
 * You will notice the constructor parameters are named.  In this example
 * I'm using named mostly to avoid writing a bunch of custom annotations.
 * This may or may not be the right approach for your project.
 * 
 * @author mkimberlin
 */
public class LayoutPane extends ContentPane {
   private SplitPane bannerSplitPane;
   private SplitPane navigationSplitPane;

   @Inject
   public LayoutPane(
            @Named(LayoutModule.BANNER) SplitPane bannerSplitPane,
            @Named(LayoutModule.NAVIGATION) SplitPane navigationSplitPane) {
      super();
      this.bannerSplitPane = bannerSplitPane;
      this.navigationSplitPane = navigationSplitPane;
      
      add(bannerSplitPane);
      bannerSplitPane.add(navigationSplitPane);
   }

   /**
    * @return the navigationSplitPane
    */
   public SplitPane getNavigationSplitPane() {
      return navigationSplitPane;
   }

   /**
    * @param navigationSplitPane the navigationSplitPane to set
    */
   public void setNavigationSplitPane(SplitPane navigationSplitPane) {
      ComponentUtil.replaceChild(bannerSplitPane, this.navigationSplitPane,
            navigationSplitPane);
      this.navigationSplitPane = navigationSplitPane;
   }

   /**
    * @return the bannerSplitPane
    */
   public SplitPane getBannerSplitPane() {
      return bannerSplitPane;
   }

   /**
    * @param bannerSplitPane the bannerSplitPane to set
    */
   public void setBannerSplitPane(SplitPane bannerSplitPane) {
      ComponentUtil.replaceChild(this, this.bannerSplitPane, bannerSplitPane);
      this.bannerSplitPane = bannerSplitPane;
   }
}

In this case I have made Guice aware of a constructor available for use during injection with the @Inject annotation. Furthermore, I have annotated the parameters of said constructor in such a way as to trigger the bindings that I presented in the LayoutModule.

Another instance of injection in this example application that deserves calling is in the NavigationSplitPane. Let's take a look at its constructor:

@Inject
public NavigationSplitPane(Column buttonColumn, ContentPane mainPane, ContentDAO contentDAO) {
   ...
}

Here again, there is a constructor being marked for injection, and again there are default constructors being used without configuration. The last parameter deserves a few words, and not only because it is highly irregular to construct a GUI component by handing it a DAO...but that's not the point here. ;)

As you might expect, ContentDAO is an interface. So, how did Guice know what to do for it?

package com.amateuratbest.examples.guice.dao;

import com.amateuratbest.examples.guice.dao.mock.ContentDAOImpl;
import com.google.inject.ImplementedBy;

/**
 * Simple DAO interface with a default implementation annotated for Guice.
 * Using the ImplementedBy annotation allows the developer to supply a default
 * implementation in the event that there is no explicit binding for an
 * interface.  This is the case 90% of the time for most projects.
 * 
 * @author mkimberlin
 */
@ImplementedBy(ContentDAOImpl.class)
public interface ContentDAO {
 public String getContent();
}

Some people are going to balk at this right off of the bat. "WHY does my interface know about its implementation?!?" However, I think you have to see this as something that is available for your convenience (and boy is it convenient) and not a required part of working with Guice. If we developers are honest with ourselves, or simply inspect our code, it is clear that most interfaces we create use a single implementation 80+ percent of the time. This makes those 80+ percent easy. Obviously if this is an interface that you are going to package as a public API for some B2B system, you probably want to avoid this type of dependency. Most of the time that is not the case, however. Most of your interfaces only exist on the server and only have one implementation in the first place. So, it is up to you to decide when using this "default implementation" feature is appropriate

I have not presented all of the code contained in the example application in this tutorial. I would encourage you to download and deploy the application for yourself. Plop the war in tomcat and off you go...all of the dependencies are included in the project zip file. You will then be the proud papa (or mama) of a rich web application developed with only a few Java classes, and without touching any javascript directly, which manages to hydrate itself successfully with only one initial point of injection.