Because I've Got Issues ...

I have to hand it to the guys over at Atlassian, JIRA is a pretty killer app (although I know now that Love is the REAL Killer App:) ).

I've worked with JIRA for, well, a really long time. I've always worked in companies where you needed to wear many hats, and I'm one of those developers that doesn't get snobby when I'm asked to step outside of my comfy Java home and help out the IT folks. So it's usually me that gets those prize winning projects like migrating forums, internationalizing wikis, or looking for new software to streamline our internal processes. I've spun JIRA around the dance floor several times, XSLT'ing crazy aggregate reports from XML backup formats, writing plugins to support externalizing JIRA data, customizing schemes, changing workflows. Every time, the same epiphany gets me - JIRA just works, exactly how you would think it should.

While some who are not so in the know might think, "Gretchen, you simpleton, it's a series of instructions to a processor, of course that's how it works". But those of us who bend software over and around daily know that few apps are actually written with quality, exceptional exception handling and in an intuitive manner that doesn't require years of higher learning and great tolerance for pain to adopt. (This is a very familiar concept particularly for those who use a certain unreasonable operating system).

This time, we need to move Mondrian's tracker issues from their original home on Sourceforge over to JIRA, which is our tool of choice for managing work and issues at Pentaho. With an assist from my other favorite killer app, Kettle, it has been a dreamy couple of days putting together the pieces to get Mondrian's issues to their new home. OK, maybe not dreamy, but certainly pain free.

Kudos, my Atlassian friends. You Aussies got it going on.

Where have you been??

When I say "you", of course I mean me! I silently fell off the radar about 7-8 months ago, and am now finally re-emerging. Well, let me tell you what I've been doing in my "off" time:

• First, let me introduce you to Jack David, my new baby boy. Doug and I were blessed with this little guy August 15th, 2008. He is the primary cause for my hiatus. I have been loving every minute of being home with my peanuts (Anthony, 13, Bella, 3 and baby Jack)! Alas, the lure of olap cubes, ETL and bug squashing safaris was just too compelling to resist, and it's time to return to the business world.

• 
• I also joined the board of directors as secretary of the Brevard Rescue Mission, a faith based ministry that provides whole-life transition resources for near-homeless moms and their kids. My dear friend Stacia Glavas is the founder, and I have been privileged to be able to handle her communications, marketing and administrative needs in between diaper changes.
  
• I have been dabbling in a bit of graphic design and found that while I have no natural talent, the Adobe suite allows me to appear semi-talented in creating fun and compelling designs. I have since designed the web site for rescue mission mentioned above, as well as the logo for my daughter's new preschool, several business cards and stationery for friends, and my latest, most daring adventure: skinning a mySpace page for my photographer and friend, Yvette Gioia! Where WILL my curiosities take me?????
• I also have managed to talk a friend of mine into letting me "borrow" his home renovation crew to renovate the entire exterior of our 25 year old home. That's right, I've decided that I also have some sort of qualifications as a contractor. Or possibly just a penchant for frustration and pain, we'll soon see!

As you can see, when Doug and I decided that it would be a good idea for me "take some time off" to adjust and organize our growing family, well, I may have misread the "time off" instructions :) I have thoroughly enjoyed my very full, engaged foray into being a stay-at-home mom. I actually just this month officially earned my soccer mom title, as Bella joined the local soccer team. I never knew there was a position for grass-pickers in soccer, but sure enough, my daughter has the exalted title!

While I will miss much of the freedom that comes from being my own manager, I miss my career more. So expect to hear from me soon, as I mull over my next career adventure! And of course, immerse myself back into the Pentaho Nation!

Kindest regards!
Gretchie

Pentaho Meetup in Mainz, Germany

Pentaho is hosting its first community meetup in Mainz, Germany in a few short weeks. I like the format we've chosen, as it combines informal sessions with lots of food, drink and some touring. And everyone is encouraged to bring their ideas and latest projects for a show and tell.

Find out details and register here. It's sure to be informative and provide an opportunity to make some great Pentaho contacts.

Creating a Minimum Width Constraint on an SWT SashForm

My friend Matt encouraged me to blog a quick snippet of code that many SWT coders might find useful.

If you've used the SWT SashForm, you know it's a nice widget in that it saves you a bit of work that you would do using a traditional Sash. One of the things I expected to get for "free" was the ability to limit how far the sash could expand or collapse a single control, or eseentially constrain the sash from completely hiding any number of it's child controls. To my surprise, I found that this is not a feature of the SashForm.

I have figured out how to create that behavior, by adding a selectionListener to the sash on the sashform. I have to give most of the credit to Duong Nguyen, who posted a partial solution to my problem here.

Without further adue, here's the code:

// Set a minimum width on the sash so that the
// controls on the left are always visible.

// First, find the sash child on the sashform...
Control[] comps = sashform.getChildren();
for (Control comp : comps){

if (comp instanceof Sash){

// My limit is derived from the size of a
// toolbar; yours can be any size you wish ...

int limit = 10;
for (ToolItem item : view.getParent().getItems()){
limit += item.getWidth();
}

final int SASH_LIMIT = limit;
final Sash sash = (Sash)comp;

sash.addSelectionListener (new SelectionAdapter () {
public void widgetSelected (SelectionEvent event) {
Rectangle rect = sash.getParent().getClientArea();
event.x = Math.min (Math.max (event.x, SASH_LIMIT), rect.width - SASH_LIMIT);
if (event.detail != SWT.DRAG) {
sash.setBounds (event.x, event.y, event.width, event.height);
sashform.layout();
}
}
});
}
}

Hope this is useful!
kindest regards,
G

Kettle, XUL, SWT and a Bit of Theory...

I've been working on a very fun and challenging proof of concept / usability project that involves Pentaho Data Integration (Kettle), XUL, SWT, Swing and a very cool, slightly controversial theory: with a lot of work frontloaded, we should be able to port user interfaces to any UI technology, and expect consistent behavior AND look and feel without much, if any, additional code .

The project I'm in the middle of encompasses three different but equally compelling goals:

1. To move Pentaho forward in providing common layers across our core pillars (reporting , analysis, data mining, ETL and dashboards).
2. To provide a proof of concept for the Pentaho XUL Framework, an architecture built to help us support common UIs across all of our applications and tools .
3. To provide a common way of describing and managing the information needed to connect to a database.

So, we began by taking Pentaho Data Integration's connection dialog ( a powerhouse of detail and metadata concerning over 30 different database connection types), applied some usability design to it, and are in the middle of rewriting the dialog using XUL and some new design concepts.

This required a lot of up front work on the part of our lead Pentaho XUL developer, Nick Baker. Nick has plumbed an architecture for Pentaho that makes it possible for us to define a user interface in XML (specifically, using the Mozilla XUL specification), and use a common set of wrapper libraries to render the UI in SWT or Swing, without rewriting any of the UI, and reusing a great amount of the business logic code. There are also plans to begin a XUL-to-Web 2.0 library as well, which will cover most of the UI technologies that we use for the Pentaho projects.

Interested? Download the source and demo, take it for a ride, inspect the code and let me know what you think. Here are the home pages for both the Pentaho XUL Framework project, and the Common Database Connection Dialog proof of concept.

Common Database Connection Dialog
Pentaho XUL Framework Project

Go ahead and post here, on the wiki, or shoot me an email. I look forward to your reactions and suggestions:)

kind regards,
Gretch

PS. All you Kettle users & developers, here's a chance to weigh in on some needed UI changes for Kettle's database UI... don't miss the opportunity!

Pentaho on Oracle's App Server (OC4J)

Wow! It has really been 4 months since my last post?? Moving over to development has cut into the time I had for blogging, documenting, communicating, you name it! We are coding like crazy.

Well, I'm back because I am heading out to ODTUG Kaleidoscope next week, and in preparation for the show, I decided to setup Pentaho on Oracle's Java Edition App Server, which is OC4J, which is based on the Orion app server. I was pleased that I managed the migration in less than a day, and I wanted to share the steps with all those folks who are too impatient to wait for this to get into our J2EE deployment distribution :)

Mind you, it takes a bit of tweaking, but it is certainly very do-able, and all server features are stable (minus the portal stuff, I didn't get a chance to address moving the portal over). Here is the repro of where I started, what I tweaked and what I came out with. Enjoy!

Where I Started

I started out by downloading a Pentaho J2EE deployment distribution from the Pentaho downloads site. The version I used for this exercise was pentaho_j2ee_deployments-1.5.4.716-0.zip. This distribution is found on our downloads page under Pentaho Open BI Suite Web Archive (.war). I know, the name implies a .war distribution, but trust me, its the deployments zip file.

Unpack this distribution to a work directory of your choosing. This distribution has an Ant build script that lets you build several different .war files and .ear files configured for different app servers and RDBMSs. Next, I'll detail the tweaks I had to make to get the Orion build target working, which is sufficient to buid an .ear file appropriate for Oracle's OC4J.

I also downloaded the sample solutions distribution and the sample data (Hypersonic) distribution, so I would have stuff to test against. You can get both of these distributions from the Pentaho downloads site as well.

What I Tweaked

About two years ago, we had started to incorporate build scripts for the Orion application server in anticipation of great community demand for this build. However, with the multitude of projects we have taken on, and a surprising lack of banging from the community, we never found time nor priority to finish that build, until now.

I really want to be able to demo our stuff on Oracle at the user conference, so I took on the task of cleaning up and repairing the Orion build. NOTE that you will be able to get these fixes in a near future build, as soon as we get our M4 release out the door and I can check this stuff in.

The Build

For the brave and impatient, here's what it needs out of the gate:

1. We'll call the root of your work directory [pentaho_j2ee_deployments]. In this directory, you will find a build.xml file. Open that file in your favorite text editor.
   
2. Find the ant target named "build-orion2.0.5-ear". Delete that target entirely. Replace the target with the following XML:
   
   <!-- ===================================================================
   target: build-orion2.0.5-ear
   =================================================================== -->
   <target name="build-orion2.0.5-ear" depends="zip-pentaho-style-war">
   
   <antcall target="war-pentaho-tomcat">
   <param name="rdbms" value="hsqldb" />
   </antcall>
   <mkdir dir="${build.ears.dir}/orion" />
   
   <ear destfile="${build.ears.dir}/orion/pentaho.ear"
   appxml="${ear.dir}/application.xml">
   <manifest>
   <attribute name="Implementation-Title"
   value="${impl.title}" />
   <attribute name="Implementation-Version"
   value="${impl.version}" />
   <attribute name="Implementation-Vendor"
   value="${impl.vendor}" />
   </manifest>
   <fileset dir="${build.wars.dir}">
   <include name="pentaho-style.war" />
   <include name="sw-style.war" />
   </fileset>
   <fileset dir="${build.wars.dir}/tomcat/hsqldb"
   includes="pentaho.war" />
   <metainf dir="pentaho-res/orion" includes="*.xml" />
   </ear>
   
   </target>
   
3. Save and close the build.xml file.
4. In the same directory, open the override.properties file. Add the following line to this file:
   ear.dir=pentaho-res/ear
5. Save and close the override.properties file.

Those few steps fix up the build files so that you can build a pentaho.ear for OC4J. You of course will have to have Ant in your system path, and a JDK available of at least 1.4.2. I will assume if you are climbing this mountain, those are easy steps you already know how to set up:)

Don't run the build yet! There are several configuration file tweaks that have to be added in order for this app to be configured properly.

The web.xml File

We need to make a few minor changes in the Pentaho web.xml.

First, the OC4J container's default application is running on port 8888. Pentaho's default port is set to 8080. So I changed Pentaho's default port to be 8888, since that seemed to be the easiest road. NOTE that you want to add your changes to two web.xml files, since it is duplicated in the deployer distribution.

You need to modify BOTH

- [pentaho_j2ee_deployments]/pentaho-precompiled-jsps/pentaho.war/WEB-INF/web.xml

AND

- [pentaho_j2ee_deployments]pentaho-webapp/WEB-INF/web.xml !!

1. Open both web.xml files.
2. Find the base-url param-name, with the value http://localhost:8080/pentaho/. Change the value to http://localhost:8888/pentaho/.

We also need to make sure the server can find the Pentaho solutions directory. If you haven't yet, unpack the solutions distribution that you downloaded to a work directory of your choice.

1. Find the param-name pentaho-solutions, and replace the value with the absolute path to the pentaho-solutions directory that you just unpacked. The value should look something like d:\work\pentaho-solutions.
2. Save and close both web.xml files.

The application.xml File

The application.xml file lives in the [pentaho_j2ee_deployments]/pentaho-res/ear directory.

1. Open the application.xml file.
2. Delete all modules under the comment <!-- additional web apps --> , as well as the web-uri module for the sw-style.war above the comment. The only modules your application.xml should have left is the pentaho.war and the pentaho-style.war.
3. Save and close the application.xml file.

Note that the sw-style.war provides the styles for the Steel Wheels samples in the solutions, but I did this quick and dirty, so I left out as much extras as possible. You may want to include the sw-styles.war, and see if it works. I left it out.

The orion-web.xml File

The orion-web.xml file is non-existent, you need to create one.

1. Add an orion-web.xml file to [pentaho_j2ee_deployments]/pentaho-webapp/WEB-INF.
2. Add the following XML to the file:
   <orion-web-app>
   <resource-ref-mapping location="jdbc/HibernateDS" name="jdbc/Hibernate"/>
   <resource-ref-mapping location="jdbc/SampleDataDS" name="jdbc/SampleData"/>
   <resource-ref-mapping location="jdbc/QuartzDS" name="jdbc/Quartz"/>
   <resource-ref-mapping location="jdbc/SharkDS" name="jdbc/Shark"/>
   <resource-ref-mapping location="jdbc/SampleDataAdminDS" name="jdbc/SampleDataAdmin"/>
   <resource-ref-mapping location="jdbc/SampleDataDS" name="jdbc/datasource1"/>
   <resource-ref-mapping location="jdbc/SampleDataDS" name="jdbc/datasource2"/>
   <resource-ref-mapping location="jdbc/SampleDataDS" name="jdbc/datasource3"/>
   <resource-ref-mapping location="jdbc/SampleDataDS" name="jdbc/datasource4"/>
   <resource-ref-mapping location="jdbc/SampleDataDS" name="jdbc/solution1"/>
   <resource-ref-mapping location="jdbc/SampleDataDS" name="jdbc/solution2"/>
   <resource-ref-mapping location="jdbc/SampleDataDS" name="jdbc/solution3"/>
   <resource-ref-mapping location="jdbc/SampleDataDS" name="jdbc/solution4"/>
   </orion-web-app>
3. Save and close the orion-web.xml.

Build the pentaho.ear File

You've got everything you need now to build the pentaho.ear file. Go to a command prompt, navigate to your [pentaho_j2ee_deployments] directory, and execute:

ant build-orion2.0.5-ear

OC4J Server Configuration Changes

Now that you have your .ear file, you will obviously want to deploy it through OC4J Server Console. First, shut down your OC4J instance, because we have a few mods that we need to make to the server configuration.

TopLink Conflicts

Now, I know this is not nice, but since we have no use for Toplink with Pentaho, and the antlr.jar conflicts with our hibernate3 library, you will need to delete the Toplink directory under [OC4J_Home]. This causes no harm as long as you're not using Toplink for some other reason. If so, then maybe you can investigate further with Oracle how to avoid the library conflicts.

This will cause the java_sso webapp in the default module deployed in the server to fail. I simply undeployed the app.

Out of Memory Errors

Pentaho is a big engine, and as such uses a bit of memory. I found that I was running out of memry early and often. So I modified the JVM_ARGS parameter in the [OC4J_Home]/bin/oc4j.cmd file to include:

set JVMARGS=%OC4J_JVM_ARGS% -Xms128m -Xmx512m -XX:MaxPermSize=128m

Add the hsqldb.jar to the [OC4J_Home/lib] Directory

Since we will be testing against the Hypersonic sample data, we need to add the JDBC driver for Hypersonic to the [OC4J_Home/lib] directory. You can find the hsqldb.jar in [pentaho_j2ee_deployments]/pentaho-third-party.

Note that when you deploy the .ear file, you MUST set an additional entry on the classloader's classpath for this jar!

Mod the data-sources.xml File

And finally, we need to add our datasources to the [OC4J_Home]/j2ee/home/config/data-sources.xml file. Add the following xml to the file, before the <datasources/> end tag:

<data-source
class="com.evermind.sql.DriverManagerDataSource"
name="Hibernate"
location="jdbc/HibernateDS"
xa-location="jdbc/xa/HibernateDS"
ejb-location="jdbc/Hibernate"
connection-driver="org.hsqldb.jdbcDriver"
username="hibuser"
password="password"
url="jdbc:hsqldb:hsql://localhost/hibernate"
inactivity-timeout="30"
/>
<data-source
class="com.evermind.sql.DriverManagerDataSource"
name="SampleData"
location="jdbc/SampleDataDS"
xa-location="jdbc/xa/SampleDataDS"
ejb-location="jdbc/SampleData"
connection-driver="org.hsqldb.jdbcDriver"
username="pentaho_user"
password="password"
url="jdbc:hsqldb:hsql://localhost/sampledata"
inactivity-timeout="30"
/>
<data-source
class="com.evermind.sql.DriverManagerDataSource"
name="SampleDataAdmin"
location="jdbc/SampleDataAdminDS"
xa-location="jdbc/xa/SampleDataAdminDS"
ejb-location="jdbc/SampleDataAdmin"
connection-driver="org.hsqldb.jdbcDriver"
username="pentaho_admin"
password="password"
url="jdbc:hsqldb:hsql://localhost/sampledata"
inactivity-timeout="30"
/>
<data-source
class="com.evermind.sql.DriverManagerDataSource"
name="Quartz"
location="jdbc/QuartzDS"
xa-location="jdbc/xa/QuartzDS"
ejb-location="jdbc/Quartz"
connection-driver="org.hsqldb.jdbcDriver"
username="pentaho_user"
password="password"
url="jdbc:hsqldb:hsql://localhost/quartz"
inactivity-timeout="30"
/>
<data-source
class="com.evermind.sql.DriverManagerDataSource"
name="Shark"
location="jdbc/SharkDS"
xa-location="jdbc/xa/SharkDS"
ejb-location="jdbc/Shark"
connection-driver="org.hsqldb.jdbcDriver"
username="sa"
password=""
url="jdbc:hsqldb:hsql://localhost/shark"
inactivity-timeout="30"
/>

Problem in the Solutions' XSLTs

I don't get this at all, but Oracle has some sort of built-in XSLT processor that complains when your XSLT tries to reference a Java class as the value for a namespace. So what you need to do is go through the [pentaho-solutions]/system/custom/xsl directory, and check every xsl for this occurrence (I would say at least 20 of them have it) and make the following mod to those XSLTs:

1. Open the xslt file.
2. Find the java class reference in the namespace declaration. They are always at the top of the file, and look simialar to xmlns:msg="org.pentaho.messages.Messages" . Note that xmlns:msg isn't the only possibility, just an example for you to follow.
3. Prepend the java classname value with http://www.oracle.com/XSL/Transform/java/. So the fixed example would be xmlns:msg="http://www.oracle.com/XSL/Transform/java/org.pentaho.messages.Messages" .
4. Repeat the steps above for every xslt file in that directory that has the namespace occurence.

There is one syntax problem in one of the xslt's as well. JBoss and Xalan don't seem to care about it, but OC4J will error out, so we need to fix that.

1. Open the file [pentaho-solutions]/system/custom/xsl/files-list.xml.
2. Search for the following lines of XML, and delete them. Note they are not right next to each other.
   <xsl:param name="target" />
   <xsl:with-param name="target" select="$target"/>
   

Deploying the Pentaho .ear File

Finally!! You can deploy your .ear file now. Start your OC4J server back up, and use the Server Console to deploy. Here are a few notes on the deployment options:

1. You do not need to set up a security provider in the deployer wizard. Pentaho uses the standard J2EE security via ACEGI, and no custmo extensions to the configuration are necessary.
2. Make sure that you map a path on the classloader to the hsqldb.jar that you moved into OC4J's lib directory!
3. You may see some log4j exceptions on start-up, but as long as you see a statement toward the end of the console log that says "Pentaho BI Platform ready", then you should be ready to go!

Navigate in your browser to http://localhost:8888/pentaho/Home, and test it out!

What I've Got - Pentaho on OC4J!

So, now I'm stoked because I can talk intelligently to Oracle tools users about Pentaho on Oracle. I think I might even dive in and use Kettle to move our sample data over to Oracle Express.

Very soon all this should be doc'ed and available through our SVN repository... hopefully any early comers who use this tip will let meknow how it goes for them:)

And please, if you will be at the conference in Daytona next week, stop by the Pentaho booth and say hey! Can't wait to meet you all!

Gretchie

Wrap up on the Pentaho Implementation Workshop

All in all, I had great expectations for the 3 day implementation workshop, and I wasn't disappointed!

The last session on Thursday, Dashboards and AJAX, was crammed full of great technical information. The most important info that I can give you all is that the current JSP based dashboards are going away, in exchange for dashboards built upon Pentaho AJAX components, currently under development. Of course, we all were very excited to hear about the new AJAX component architecture, and happy to hear that we can get to the code. The ETA for GA delivery is somewhere in the 3rd or 4th quarter of this year (with milestone builds available earlier most certainly), but always stomping on that hairy edge, I can't wait to dive into the code and contribute to implementation (in my spare time, ha ha! ).

James Dixon, Chief Geek for Pentaho, also went into a bit of the history of Pentaho Dashboards, which was really helpful in gaining perspective on why Dashboards require so much coding today. The philosophy and design goals for Dashboards (really for the platform, in general) is to remain delivery agnostic - meaning we want the platform output to be delivered via the client's choice of technology, not our own. So if you are a JSF shop, .NET shop, or Java applet guru, it won't matter to us, since we deliver the content from the platform in XML. You can take it from there, and transform that XML any way you wish. Well, that design goal is tough to stick to when you are implementing a Dashboard architecture, since Dashboards are heavy on UI, usually containing reports, charts, dials, gauges and numerous other widgets, in some portal type fashion specific to the user's point of view. So we went about component-izing all of the above mentioned widgets, and used a simple JSP (or not so simple JBoss Portal) to demonstrate what COULD be done. The response from our community has been to make Dashboards easier to build, and fortunately, AJAX has conme along (or been there, depending on how you look at AJAX) so that we can deliver that ease of implementation.

I haven't had a chance to say much about the group that attended the class with me. I was tickled to finally meet some folks that I have been chatting with via email, some for more than a year now! Nice, intelligent, talented and truly passionate about BI - I could spend alot more time with these folks, we share so many traits and interests (I know, I think very highly of me ;))! The attendees came from Canada, the Netherlands, Germany, France and the US, which speaks for the demand that our training generates, as well as the global presence that Pentaho has earned in a short 2.5 years. I can't believe sometimes that I am a part of something this big, and this bold! On a day to day basis, it feels like we are just a bunch of guys doing what we have done best for a long time - building BI. But when you gather your community, partners and teammates in a room like we did last week, it sure feels a whole lot bigger, a lot more significant. And so, I can someday explain to my daughter why my job makes me proud :)

Workshop, Day 3

I know, I know, what happened to Day 2? Well, between busy sessions, picking up the kids, and teaching my night class at Brevard Community College, blogging fell to the way side. For my three avid fans, I sincerely apologize, but by now you are used to it :)

So Day 2 of the workshop, in all honesty was a bit of a fire fighting exercise. Since we are working with the very latest code for the platform (and I mean VERY latest), we ran into a couple of problems during the Subscriptions session that prevented us from seeing the results of the subscribed bursting examples that we set up. But the content was good and subscriptions in the stable builds is very very powerful. We have the ability with the platform to relieve the administration and information overload that occurs with the typical scheduled reporting process. Subscriptions also prove to be flexible and easy to use, which makes it a nice tool for consumers of Pentaho reports, analysis, ETL and processes.

The Day 2 afternoon session was all about advanced deployments of the platform, as well as customizing deployments for each user's environment. Brian Hagan walked through the complex details of manually deploying web application through JBoss and Tomcat, focusing on the touch points that are required when you have your own app server installation already in place. Overall a good session that could be helpful to anyone that struggles with J2EE deployments today.

Day 3 started out with Bill Seyler, a stellar Pentaho engineer, presenting the life cycle management features within Pentaho. For anyone who is not familiar with the term and what it means in Pentaho, life cycle management is versioning solution content for the platform. Bill covered the architecture of how Pentaho interacts with version control systems, which seems to be a very clean and simple implementation. The beauty of life cycle management in Pentaho is that due its simple interface, any version control system can be used, as well as using multiple systems for one Pentaho deployment.

Next up was Anthony DeShazor, our engineering wrangler, talking about scalability and clustering. Much of this session covered the general topology and infrastructure issues that prevent almost any application from scaling. The point I took away is we can control what the Pentaho application does, but how you get at your data and how you deliver it out to consumers can bottleneck any good app. It was great to participate in tis discussion, since many in the room are experienced in the field and had much to contribute. Anthony then took us through the JBoss Clustering presentation that James Dixon presented at JBossWorld late last year. It was a simple architectural discussion covering JBoss Clustering, ending with some pretty impressive benchmarks that proved Pentaho's ability to scale. The most interesting news that came out at the end of this session is that Pentaho has started to build a BI benchmarking bundle, based on the Transaction Processing Performance Counsel processes, and plans to release it to the open source community for benchmark responses. Feel free to email James (communityconnection@pentaho.org and it will get forwarded) if you are interested in participating in that effort!

Our last session after lunch is Dashboarding and AJAX, a session that all the trainees, including myself are looking forward to. I'll fill you in tonight on how that session goes and how this all wraps up.

Implementation Workshop: Metadata

"My users think I'm a God." , Matt Harbert, DivX

I had to start with that quote, I was so pleased to hear it during discussions over lunch this afternoon during the workshop. I was talking with a couple of classmates about Kettle, the Pentaho Data Integration tool, reveling in stories of leaping tall buildings with Kettle as my booster pack. From across the room, Matt tossed the aforementioned words of glory, and I thought "well, if that doesn't just sum it all up". Kettle really is that good, and you'll only know if you dive into it, because, of course, I am a Pentaho-an, making me slightly biased ;)

These were the topics of discussion that opened the Metadata module of the workshop. Metadata is another project being architected and driven by Matt Casters, founder of the Kettle project. Jake Cornelius led the module, and did a nice job of showing us the Pentaho metadata Editor, a handy tool that assists in building Pentaho metadata models. The Metadata Editor is not quite 2 months old, and from what we were shown, is proving to hold lots of potential. The core functionality is there, and the user interface is intuitive, once you learn the new jargon. Amidst a short array of funky behavior and a few bugs, the Metadata Editor's power shined through with it's ability to model not only mappings to physical tables, but also extended formulas, formatting and style properties in a hierarchical fashion (termed "concepts"), and internationalization functions.

This is the end of day one, time to pop over and visit the dev guys on my way home. I have a renewed sense of excitement today, partly because this training is turning out to be even better than I had expected (and my expectations were pretty high), and partly because I'm training up for a brand new seat on the Pentaho ride :)

Implementation Workshop: Security Simplified

Well, we just finished the security module of the workshop, and I have to say, sans the network issues, I am really impressed. Mat Lowry, a Pentaho engineer who focuses on security during his day job, put together the content for the module today. Mat took a pretty complex set of topics (LDAP, Acegi, CAS and J2EE Container Security) and delivered just enough content to understand easily what Pentaho Security is made up of, and what Pentaho adds to the standard technologies available to you in a J2EE environment.

It seems we have done a very nice job of separating the wrangling of authentication and authorization from the functionality of the BI platform. I plan to follow up this workshop with a deeper dive into Acegi, as Mat has gotten me really excited about what it and the Spring framework can do. I'm taking away the relief that Acegi can handle a good 80% to 90% of my web resource security problems, without me having to write more code. I like it, I like it.

My thoughts on the hands on lab is it really made me think and I was pleasantly surprised to find that I understood the concepts Mat covered, and could apply them in the 60 minute lab exercise that was given to us. This was not your typical training class exercise that with loads of screenshots and step-by-step instructions, you could achieve one simple implementation of security. This was more like "A train leaves Tampa at 400 miles an hour at the same time a train leaves Daytona at 200 miles an hour, when will they meet" type of exercise. Now, I think I've mentioned before that I'm a very bright, but pretty simple person, and frankly this type of exercise reminds me how little focus I have. Once we stopped chatting, and I could read the lab carefully, I had no trouble implementing my own switch over from memory based security to LDAP based security within the Pentaho platform.

I have to admit, I feel a tad bit smarter than I used to :)

Pentaho Implementation Workshop

This week, I get the pleasue of sitting in on the Pentaho Implementation Workshop, a hands-on in depth training session covering many advanced implementation features in the Pentaho BI platform.

This training kicks off my new role as a Pentaho developer! I decided to move over to the engineering side after a year and a half leading the Pentaho community, which was a very rewarding experience. You knnow what they say, you have to go where your heart takes you :)

Here's the workshop agenda:

Security
Metadata
Dashboards and AJAX
Reporting User Interface
Life Cycle Management
Advanced Deployments
Scalability

Community members here - Roland, Samuel, Fabrizio, welcome, and it's so nice to finally meet you in person :)

I'll be blogging on the workshop all week, so stay tuned!

Follow-on to Internationalization.... Vote!

If you read my previous post on internationalization, you know that I'm looking for a great Confluence solution for handling multiple translations of the Pentaho documentation inside our new wiki.

Go here, login and VOTE for Atlassian to help us solve the problem!

And my apologies to Atlassian regarding my comment that they may not be responsive in their forums - I wasn't monitoring the thread they responded to, only four short days after I posted my dilemma. I will have crow for dinner :)

Thanks everybody!
-G

The Hefty Job of Internationalization

When I worked in the purely commercial software world, internationalization of a product was a critical, magical event that happened somewhere offsite in the hands of contracted translation companies. Having your software translated into as many different languages as possible is just as important in the open source arena. Arguably, open source projects NEED translations MORE to grow into a global presence. The PROCESS of translating software and documentation for an open source project is an entirely different experience from the commercial event depicted above, as I have learned over my last two years with Pentaho.

This topic is at the forefront of my writings today because I have been tasked with figuring out how to manage localizing the Pentaho documentation in a wiki that has no feature support for internationalization of content! Did I mention that Pentaho is moving all of their documentation to a wiki? Well, there it is. The cat's out of the bag. As of our 1.2 GA release, the documentation will be maintained, by community and development team, in a Confluence wiki.

Confluence is a great tool and has lots of integration points with our case tracking system, JIRA. That said, it seems that Atlassian (the company behind Confluence and JIRA) is a little behind in the internationalization game. Confluence only recently started to support language packs for translation of the wiki itself, and has no support for content translation.

So here we are, and I need to figure out a solution to satisfy three very important groups. The first group are the translators of our documentation. They are community members who contribute those translations to our projects. We need to set up the internationalization in the wiki so that it is easy for these folks to do the initial translation, and also have a mechanism for notifying them when the master language version of the doc has changed. The second group is the users of the documentation. It should be easy for me, if I am French, to find the French documentation, but also be able to peruse the English documentation. And the third group is the poor guys in house that have to maintain the organization of this documentation. When you consider we have over 10 projects, translated (so far) into 8 languages, that get a new revision of documentation for every version of the project released... well, that's alot to manage.

So my initial attempt at a design for this conglomeration that we need to support was to try to stuff all of the languages in the same document in the wiki using DIV type tags and such for separation. I'd then use some custom code to hide the other translations based on the user's language setting in the browser. This would make my translators happy initially, because they can do the initial translation almost inline with the master language version. My users would be happy because our wiki respects their browser's language settings, and if a particular piece of content hasn't been translated to the user's language of choice, we would default to the master language version. Of course, this solution does not address the translator notification of master language changes, and well, it would be a bit of a pain to determine whether it was an master language version change or a translation change, with all that content in the same document. Also, this only addresses translation of the content. What about the document titles? Since the navigation of a wiki (by definition) is based on document name, we have a big problem to solve there. And the largest point of failure in my grand plan is the fact that the merging capabilities are not so hot in our wiki of choice, so the translators would have to line up and take turns translating. Ick, in a word.

So the next path we steer down is that path that takes us to completely separate repositories (called Spaces in Confluence) for the different languages. This gives us autonomy for the purposes of editing and not having the language content intermingled, but at a pretty large synchronization and maintenance cost. We now need to figure out, do we populate the French repository with all of the English documentation, to assist our translators in translating? Well, then we have up to eight copies of the doc, that is changing realtime, and is sure to get out of sync. So, perhaps we should let the translators populate their language wiki from scratch, organically? This isn't very accommodating to our translators, and documents will surely be placed out of order with the master language wiki, making it confusing for the users.

Yikes. It's at this point, having discussed the plethora of less-than-stellar options with a few clever guys on our team, that I decided to step back and write to the community. In my mental gymnastics over this problem, I made many assumptions about what our users and translators really want.

For the translators in our community, have you worked at translating in a wiki before? What did you like about it? What did you hate? Is it easier for you to translate everything in your format of choice, or do you like the idea that once you translate it it would be available, without having to wait for the Pentaho team to publish it? Of the two scenarios I detailed above, which is the lesser of two evils for you?

And for all of the rest our community that must USE and update our documentation - would it be more frustrating for you to work with translations inline in the wiki (in edit mode only) or to have to go hunt around someplace else to find those translations?

And, of course, for any other open source project that holds the silver bullet to this problem - feel free to share your solution here!! Heck, I'd even take well intentioned guesses and good ideas :)

Google Maps Demo, Check It Out

You really need to check out our demo of Pentaho's Google Maps integration, put together courtesy of Jake Cornelius, Product Manager for Pentaho Dashboards.

I like this demo for alot of reasons, but first for it's content quality. I will be the first to admit, I have a short attention span for any type of sales or marketing related collateral. Unless it's funny or can incorporate a bit of irony, I tend to hear the adults in Charles Schulz's Peanuts cartoons - wah, wah, wah, wah, wah.

Jake's demo is a great, get-to-the-point example of location intelligence - solving a problem that has a geographical component in the equation. Jake takes you through a common case where a business executive takes a trip to the East Coast, and wants to know who are the key customers by sales volume he should visit during his travels. They need to be big spenders and - that geographic component - should be located relatively close to one another so he can maximize the number of customers he can visit.

The solution was simple with Pentaho BI and Google Maps. Check it out - you will be impressed.

From a developer's point of view, I liked the demo because I know I can download this solution in the Pre-Configured Install, and not only do I have a great example Pentaho Dashboard that I can pick apart, but I also get to see AJAX and the Google Maps API in action. For the hands-on learner like myself, this demo is a nice introduction to some very cool tools I can now put in my BI toolbox.

So go get it. Download it, listen to it (it's less than 8 minutes long) and come back and tell me what you think. Not your typical demo, right?

A Whole Lot of Community Goin' On...

I haven't had the chance to get back to blogging lately. This really is a quandry - there is so much going on at Pentaho and around the Pentaho community, I have so much to blog about... but there is so much going on at Pentaho, I haven't found the time to blog! :) So today's a fun day, because I get to catch you all up on so many great things that are happening to and for the Pentaho community.

SourceForge.net Project of the Month

Let's start with Pentaho being honored as the SourceForge Project of the Month. If you have worked with open source for even a short while, you have heard of SourceForge.net. It is THE place to go to download open source software and source code. I have heard the number of projects hosted on SourceForge.net to be somewhere around 130,000, so to be chosen and highlighted as the project of the month for October, well, we take that as a very big deal.

This is a huge win for us the community because this showcase should draw alot of attention from savvy developers who are looking for an interesting and well-architected project to work on. I just can't convey all the great possibilities that could come out of this highlight. We are very excited! I will keep you up to date on the happenings around this win, as they unfold :)

New Web Sites for Mondrian, JFreeReport and Kettle

The next great community move is the new web sites for Mondrian, JFreeReport and Kettle. We have built new sites for the projects and the web team will be maintaining them and providing the TLC that the sites deserve.

This a big plus for the leads of the projects, since that takes the maintenance burden off of them and they get to concentrate on doing what they love - implementing the vision for their projects! The content for the sites is roughly the same, mostly just the look and feel, and the URLs have changed. Content changes will come slowly over time, when it makes sense to update and re-organize it for each team. The new URLS for the sites are listed below:

Mondrian: http://mondrian.pentaho.org
JFreeReport: http://jfreereport.pentaho.org
Kettle: http://kettle.pentaho.org

Pentaho and MySQL Webinars

We have had an awesome series of webinars running with the MySQL team. Since MySQL requests in the community seem to dominate, this series has proven very valuable to both the Pentaho community and the MySQL community. If you haven't viewed them, you can watch the replays here:

Business Intelligence with MySQL and Pentaho

ETL for MySQL using Pentaho Kettle

And coming up October 24, you can register for the latest webinar, OLAP for MySQL using Pentaho Mondrian.

Contributor Update

As always, we have recieved numerous contributions from different corners of the world over the past few months. I would like to share a few of them, just to demonstrate what is POSSIBLE for any community member, and what is being accomplished today.

A few weeks back, I recieved an email from Marc pointing me to a forum thread where Steve sent an example of his solution, which contains .NET consumption of Pentaho web services, producing an ADO dataset. Hoowah! I still need to contact Steve and see if he would write us a a nice technical article on his solution, but you know, so many hours in a day:)

Radek, a regular contributor, sent us the code for outputting multiple reports into different worksheets of the same Excel workbook. If your interested in that feature, you can track progress on it here.

By far our biggest avenue for contributing is in the translation of the platform. Last week alone, I recieved updates to 4 different language translations, 3 new Brazilian Portuguese documentation translations (thanks Fabrizio!), and a request to start an Italian translation of the project!

Thanks to the Pentaho Community!

At the end of the day, we know that our success is attributed to the the hard work of the Pentaho team, and the dedication and commitment of our community. We would like to say thanks again for the support. Running an open source project is a great experiment in trusting teams to provide resources for a common goal. It's with pleasure that we participate in this process, and see people at their best, making things work to achieve a common benefit.

Check Out Pentaho Training

I recently sat in a meeting with our new Product Manager, Jake Cornelius, listening to some great feedback on a Pentaho training session he recently attended.

I worked with Jake back in my Hyperion days, so I can confidently say that Jake is pretty great guy who isn't one to blow wind up your shorts for heck-all's sake. He's a strong leader and has deep insight into BI, having worked in the industry for umpteen (a great figure when you don't have a precise number, but it's been pretty much forever) years.

I asked Jake to send me his thoughts on the session, so I could share them with my ever growing blog-base :) So here's the review, and I hope this helps anyone on the fence about investing in Pentaho training come on over.

Jake:

We recently rolled out our new training series on Building Analytic Solutions using the Pentaho BI Platform. The 4-day event, held September 18-22 in Orlando, was a huge success and confirmed Pentaho’s position as the global leader in Open Source Business Intelligence. Attendees from Japan, Columbia, Canada and around the US were present to learn how Pentaho’s BI Platform will help their organizations unleash the valuable information currently held hostage in proprietary data stores and transactional systems. The course was facilitated by Dave Reinke and Brian Senseman of OpenBI. In four short days, Dave and Brian lead attendees through an end-to-end BI implementation example from ETL using Pentaho Data Integration Services, to reporting with Pentaho Reporting Services and the BI Platform, to interactive analysis using Pentaho Analysis Services. Unlike conventional product training courses we’ve all sat through that focus on “this feature does x, and that feature does y”, OpenBI leveraged real world examples and best practices from their years in BI consulting to teach the class how to build enterprise-class solutions using Pentaho. The feedback was overwhelmingly positive, here are just a few of the comments we received from the attendees:

“Great class!”

“Thanks for the course. There were a lot of interesting ideas talked about.”

“This has been a great class and I have learned quite a bit. Perhaps more importantly, I have gained confidence in both the quality of the software and my ability to use it efficiently.”

If you are currently investigating Pentaho for an upcoming BI project, I strongly recommend you consider attending one of our upcoming training courses. A current list of planned training courses can be found on our website at http://www.pentaho.com/services/training/ .

Jake Cornelius
Product Manager

Hopefully we'll here more from Jake in the near future, as he will be a key player in helping Pentaho deliver project roadmaps out to the community. The entire team (me especially:) is insanely excited about that endeavor, so you can imagine how happy we are to have Jake on board!

Of course, COMMUNITY voices are heard the loudest, so I invite the Pentaho Nation to comment here, email me at communityconnection@pentaho.org, or post in our forums what it is that YOU need on those roadmaps!

BlogOrlando Un-conference: Just a Fantastic Friday

You couldn't ask for a better Friday.

I started out early driving in to Winter Park, Florida, about 60 miles from home, to attend the first BlogOrlando Un-conference at Rollins College. I'm not saying starting my day with a 60 mile commute is all that fantastic, but it does give me time to catch the news on the radio, enjoy my favorite cup of coffee (7-11, if you haven't tried it, you don't know what you're missing!!), and we are experiencing unseasonally mild, sunshine filled weather right now, so the drive isn't even a consideration anymore :)

The BlogOrlando event was hosted on Rollins Campus, and drew a mix of PR professionals, marketing gurus, technical entrepreneurs and various other business folk. I was pleasantly surprised and interested to chat with such a diverse group, especially since landing in this role that has me straddling the fence between marketing and the development groups at Pentaho.

The first session I attended was on Corporate Blogging. Dave Coustan, the corporate blogger for Earthlink.net, facilitated the session and had some very interesting lessons learned and success stories to share about his journey building the very popular Earthlink.net corporate blog. While Dave made many points, the advice I liked the best was "Write so that any smart 14 year old can understand it". This helped me in numerous ways - I no longer feel embarrassed that I am incapable of writing pretty prose, and many of my Pentaho co-workers refuse to claim a birthday past their 14th. So you can see how this advice works so well for me:)

Some other interesting and valid bits from Dave on running a successful corporate blog:

• Find an angle for boring stuff
  
• Draw a crowd, figure out the content to draw it
  
• Advises to separate blogs for personal and corporate
  
• Get the umbrella knowledge of what is happening in the company - be your own "beat cop"
  
• Research and fact-check your work!!!
  
• Regularly perform outreach and follow ups
  
• Capitalize on the ability to handle anything that falls out of traditional channels - edge cases
  

Some helpful points that were made regarding those who are just trying to convince their companies that corporate blogging is a net plus:

• Cheaper, more trusted, lasts longer than ads
  
• Start a dialogue with your customers
  
• Valuable content for search
  
• Rapid publishing
  
• Gives press something to link to
  
• Ear to the ground feedback loop
  
• Safety valve for customer service/pr/media
  
• Free syndication
  

The next session was "PR and Blogging", where there was much discussion around bloggers versus journalists. Is there a difference? Are journalists held to a higher standard than bloggers? And, of course the accountability of bloggers was introduced here and carried over in detail in the Legal session. Most of the conversation was speculative, and it seems that the PR folks who blog know the by-laws of good journalism, and use them whether they are blogging or running a corporate campaign. This of course is not always the case, especially for the casual blogger. I think the most useful point for me that was discussed in this session was the fact that the amount of blogger traffic you generate is only one measure of a blog's success, and will peak and valley. The true strength of your blog is in internet indexing. The indexing of a blog entry is much farther reaching than the direct audience of the blog.

The last session I was able to attend was "Blogging and Legal Issues", facilitated by Andrea Weckerle, a vibrant and very well-presented attorney and PR professional. I was very impressed with Andrea and wish I had a chance to connect with her to talk more about the specifics, but lack of time prevented me from introducing myself. I will have to stop by her blog. The legal issues that we discussed covered things like libel, slander (spoken and slander both start with "s", hence the difference:), copyright law and privacy issues. I was not surprised that the law covers these issues in much the same way for blogging as for written content. Regarding copyrights specifically, it seems the consensus is that in blogging, alot more violations happen, and alot less action is taken (think splogging).

I had to leave Rollins around 2:30, but if you want to read more about the afternoon sessions and wrap up, you can find all sorts of blogs and details at www.blogorlando.com, or google for BlogOrlando.

Thanks to Josh at Hyku for putting on such a nice event, in such a nice place. It made my Friday:)

Analytics For Bugzilla

If you are a Bugzilla user, you really should check this out. My friend Nick Goodman came up with a Pentaho solution that takes Bugzilla data, ETL's it into palatable OLAP format, and uses the rest of the Pentaho Open BI Suite to provide analytics and reporting - that you could NEVER get out of Bugzilla alone.

For example, how about historical based analysis, such as running total net open bugs by product, year over year, with the capability to drill into details? Or current issues, by status and product? These are the reports that product managers, project managers, development managers and software engineers need to manage the timeline, scope and releases for a software project.

Let Bugzilla do what it does best - track your bugs. Let Software Quality Reporting for Bugzilla unlock the information captured in your bug tracking. To try it out, go to the SourceForge project at http://sourceforge.net/projects/qareports/, and download the solution and the Getting Started Guide. It takes about 15 minutes to set up, and the demo is up and running. Another 30 minutes or so and you can have the solution hooked up to your Bugzilla data!

Get Your Open Source Here...

Talk about a really great resource - a site that provides open source alternatives to the most popular commercial software packages. And very nicely put together I might add, which in my opinion DOES contribute to its net usefulness:)

The site is osalt.com, "Open Source as an Alternative" and can be found at http://www.osalt.com/. The premise behind the site is a listing of the most popular proprietary software matched to their open source alternative offerings. Just type in Cognos in the search engine, and osalt.com brings you Pentaho! Niiiccce:)

Of course, this site is useful in so many ways - I can now get the top listed open source alternatives for my photo editing software, my HTML wysiwyg editor, my mapping software, oh the list goes on.

Working in open source, I forget what it was like, being a newbie in the open source world, and trying to figure out what project does what. But I can bet that osalt.com makes it a bit nicer for open source novices to get a grip... which from all we hear, there are droves of new open source adoptees everyday.

By the way, I can't take all the credit for finding osalt.com - my co-worker Brian forwarded it to me, thanks a bunch Brian!

Dynamically Generating a Cross Tab Report, Part 2

So, I thought it would take me a week or two to finish up this article from Nic Guzaldo, a strong supporter of the Pentaho platform. It ended up taking almost a month, which tells me I am long overdue for updating my calendar and planner.

With many apologies for the delay, I present to the Pentaho community the long-awaited second half to "Dynamically Generating a Cross Tab Report". In Part 2, we pick up where we left off and use the query that we derived in the first article to feed into a Pentaho report template. Actually, we will use several mini templates that demonstrate template re-use and assembling with parameters, very cool stuff. Our template will have several replaceable parameters that will accommodate our variable number of columns, a dynamically generated report title, and a nice name for any output files we generate, should the user choose to output their report to a format such as Excel, RTF, PDF, etc.

This exercise is a giant hand up for Pentaho users who really need to see it all come together - JNDI, dynamic queries, parameter passing, report templates, variable output formats, and much more! Great luck and please do send me some feedback! I'd love to know if these articles are helpful, and what topics you would like to see in future articles:)

all the best,
Gretch