Gretchen and the Pentaho Nation

Pentaho and OpenMRS Integration

2011-10-08T18:26:00.001-07:00

We have a great opportunity to explore how Pentaho can provide ETL, analytics, and reporting benefits to OpenMRS, an open source medical records platform and community interested in global health care.

Check out the first projects underway, and decide if you have time to participate:

Pentaho ETL and Designs for Dimensional Modeling

Cohort Queries as Pentaho Reporting Datasource
This project still needs a lead developer; we'd like to have these projects run in tandem.

To get involved, feel free to email me directly, or contact any of the OpenMRS mentors listed in the projects.

kindest regards and in His grace,
Gretchen

PCM11: Continuity and Change @ Pentaho

2011-10-04T08:43:00.000-07:00

Last week, I enjoyed my third (of four) Pentaho Community meetup, this year held in Rome (Frascati), Italy. Jan Aertsen did a fantastic job summarizing the presentations, you can review them all here, including access to the presentation materials. At this particular juncture, I find myself in my longest commitment to a single company in my career. The entire ride has this very cool thread of continuity through tides of swift and constant change that comes with being a bleeding edge software company.

When I look back over the past seven years, many times I focus solely on Pentaho milestones and growth, the markets we've entered and enjoyed success with, the new initiatives that take hold. PCM11 gave me a look at the global reach of success that Pentaho enjoys, creating opportunity and economy beyond the bounds of the company official. This is what makes open source make sense to me. This appeals to me.

The people that make up the Pentaho community are a talented, committed group of individuals who are growing in their own endeavors, many based on the community edition of the Pentaho BI Suite of tools. Many of our community colleagues have been committed to Pentaho from the earliest releases of 2004 and 2005. Their efforts are paying off, and while Pentaho the company doesn't get everything right, we've managed to earn the respect and partnership of some incredibly driven and talented people.

Another interesting phenomena - community members becoming Pentaho employees, blurring any lines that get drawn at times between community and corporate.

From the ranks of the Pentaho community a well of talent has sprung - Slawo, Roland, Jan, Jens, and a handful of others. Pentaho is incredibly savvy in hiring from the community. Our community is the hotbed of Pentaho, DBA, big data, analytic and reporting knowledge, both from a project development perspective and from a solutions development perspective. How many software projects suffer from the writers not understanding the use cases? Not eating their own dog food? Well, the newest Pentaho developers have been at that bowl for some time, and the internal developers can help them keep that commitment with internal initiatives delivering Pentaho solution driven information.

And what of the other direction? Those leaving the formal Pentaho realm and working entirely community based? Well, that would be me. It's not like this is new news - I'm now infamous for my off-again, on-again relationship with formal employment :) Don't mistake me for irresponsible; I just have higher priorities. We all should be so blessed, right?

The great news is I also have reaped the benefits of a long series of lessons in BI, big data, analytics, reporting, visualizations, problem solving and code writing. So I take these lessons learned into the community and can begin to give back a little. To my fellow community members, to other open source projects, to Pentaho.

One project that has caught my attention is the OpenMRS project. OpenMRS is a medical records system platform widely deployed throughout the compromised countries of the world. OpenMRS is open source, and has a thriving community of developers, implementers, users and observers from well established world health organizations.

I intend to spend the last quarter of this year investigating integration points between Pentaho tooling and OpenMRS. OpenMRS could use more insight into their data; Pentaho is an excellent set of tools for turning raw data into information. I see synergies here :)

Soon, there will be a project page to stay informed if you're interested or would like to participate. I'll post back as soon as I have the leg work done. In the meantime, checkout http://www.openMRS.org. It's a very rational site that gets you up to speed quickly on the project.

Cheers & all in His grace,
Gretchen

Jimmy D. takes a look at 2010 and where Pentaho is present

2010-11-15T07:53:00.000-08:00

I couldn't resist re-posting this link to James' blog - these numbers are sooo exciting! More so for me since I remember when Pentaho was largely comprised of a small rented space and some beanbag chairs :)

James takes a look at 2010 and where Pentaho is present.

Kindest regards,
-G

Troubleshooting Localization

2010-09-24T13:04:00.000-07:00

I've been gathering some interesting and useful information when dealing with Pentaho Reporting, Pentaho Metadata and characters not represented in the standard ASCII character set. This bucket of tips will make it into our documentation ASAP, but I thought it prudent to share it with our community even sooner.

IMPORTANT CAVEAT: Note that where I specify UTF-8, I am only doing that as a reference encoding... the encoding I speak of in most cases can represent any extended character set; UTF-8 is a common one for multi-national apps, because it represents multi-national characters.

Character encoding is key to displaying multi-byte or special characters from character sets outside of the standard ASCII character set. Any text-based files that contain special characters in their glyph form must be encoded as at least UTF-8, or in the character encoding for the language you are attempting to display.

The character encoding is significant no matter where these characters reside or travel - if the file or database stores the characters as UTF-8, then Java must handle those characters as UTF-8 and where ever the characters' destination is, be it a browser window or system file, the destination must also render the characters using the same character encoding.

So, this means:

1. Check any and all TXT or CSV files in an appropriate editor to verify that they are encoded in the correct character encoding. In a pinch, Notepad will do, but if you are seriously dealing with localization, it's in your best interest to invest in or download a good unicode text editor.

2. Make sure that your HTML and XML files have a meta tag specifying your chosen as the character set. For example:

<?xml version="1.0" encoding="UTF-8"?>
<meta http-equiv="Content-Type" content="text/html; charset="UTF-8" >

And, if it actually appears in an xhtml document (as suggested by the xml declaration) the content type should probably text/xhtml, and the meta tag should be closed in itself like so:

<meta http-equiv="Content-Type" content="text/xhtml; charset="UTF-8" />

3. The Pentaho BI Server allows you to specify a default encoding in a context parameter in the web.xml file of the webapp. This "default encoding" applies to any XML documents that the server generates. The platform adds an xml prologue to these documents and sets the encoding to that of the BI Platform, which comes from web.xml. By default, the server assumes this is UTF-8. If you want a different default encoding, specify it in the web.xml.

4. You also want to make sure that the default encoding that Java (specifically, the JVM that is running the Pentaho application) is using matches the encoding that the Pentaho application is using. We just mentioned that the default encoding for the Pentaho BI Server is UTF-8. So, what is the default encoding for the JVM? The JVM determines it's encoding from the system property "file.encoding". As of Java 1.4.2, this property is available and set from as the default OS locale. However, on Windows systems, this default locale may not exist, so Java makes a best guess. As you can see, knowing what the default encoding is can be a bit nebulous, so we recommend setting the encoding for Java on the command line:

java -Dfile.encoding=UTF-8

You will want to add this command line parameter to any Pentaho application startup script that you are attempting to use internationally. Specifically for the Pentaho BI Server, you would want to set this command line parameter in the start-pentaho.bat | .sh script.

It's important to note that we don't demand UTF-8. We do (for now) demand that whatever file.encoding is specified is what the web.xml context parameter "encoding" says. So - as long as this param says ISO-8859-1 and file.encoding says ISO-8859-1, you're still good.

Next, understand that common fonts do not have all of the characters possibly represented in the UTF-8 character set or other extended character sets. So, while your encoding may be correct, if you specify a font that doesn't include the glyph for a multi-byte character, it's likely to render as a square, question mark, or some other seemingly unrelated character.

A good test font on Windows systems is "Arial Unicode MS", which is distributed with MS Office and is claimed to have every UTF-8 character glyph available. It's ability to represent every character makes it a good TEST font, but comes with a price - the font is nearly 24 MB. You do not want to recommend this as a production font, since as a best practice guideline, we tell customers to embed their fonts with certain output formats, and this font would equate to staggering overhead in download sizes. The proper recommendation is to tell customers to find the font that best represents the consumer base's languages for that report.

So how do we control which fonts and encodings are used in Pentaho Reports? It's a bucket of valuable information I'm attempting to summarize here:

First encodings:

In Pentaho reports, there are global configuration properties for the different output formats. The global report engine configuration can be found in the Pentaho BI Server installation under the pentaho webapp: pentaho/WEB-INF/classes/classic-engine.properties.

org.pentaho.reporting.engine.classic.core.modules.output.table.html.Encoding=UTF-8
org.pentaho.reporting.engine.classic.core.modules.output.pageable.pdf.Encoding=UTF-8
org.pentaho.reporting.engine.classic.core.modules.output.table.csv.Encoding=UTF-8

And fonts:

1. If you have a metadata model in play, make sure that the metadata concept properties for the font-family are all set to a font that is installed on the server serving up the model and is capable of rendering the special characters you need represented. There is a Base concept (found in the Concept Editor) that has a default font-family that you will want to verify/modify is configured correctly.

2. If you are using any of the templates designed for Report Design Wizard or Web Adhoc Query and Reporting, you will want to verify/modify those templates to use a font that is capable of rendering the special characters you need represented. The templates for Report Design Wizard are found in the Report Designer's /templates directory. The templates for WAQR are found in the Pentaho BI Server solutions directory under pentaho-solutions/system/waqr/templates.

3. On Windows, what determines whether Pentaho can find an installed font? A few things! First, look in the Windows Control Panel (or modern equivalent), under Fonts... these are the fonts that should be available to the reports generated by the Pentaho BI Server. If for some reason you want to include a font not in the system fonts directory, you can add additional directories of fonts.

This is done using a configuration file that you would create and place in the Pentaho webapp WEB-INF/classes directory, which basically creates an override for the configuration file that is found in the libfonts-x.x.x.jar library in the Pentaho webapp primary classpath. The name of the libfont report configuration is libfont.properties. Create this file, place it in the classes directory and add the following configuration property to it, with your font location of course.

org.pentaho.reporting.libraries.fonts.extra-font-dirs.myNewDir=c:/myNewDir/myFonts

Note: There is an open issue with this property that should be fixed with the SUGAR release of the Pentaho BI Server: http://jira.pentaho.com/browse/PRD-2145.

4. Our best practice recommendation for ensuring the proper rendering of special characters in PDF reports is:
a. Embed the font. This can be accomplished using the following global reporting configuration property: org.pentaho.reporting.engine.classic.core.modules.output.pageable.pdf.EmbedFonts=true
b. The font should be a TrueType font.

Also important to note is that you can confirm what fonts the Pentaho BI Server is aware of, as the reporting engine creates a cache of the fonts it has registered. If you are at all concerned that the server hasn't correctly registered a new font from the system, you can blow away the cache, restart the server, and the reporting engine will load all system fonts anew.

The cache exists at $HOME/.pentaho/cache/libfonts.

Pentaho Architect's Bootcamp Training Now Available

2010-08-20T06:26:00.000-07:00

Last week, I had the pleasure of offering our very first 5-day session of the Pentaho Architect's Bootcamp training, and overall, it was a big success!

There comes a point in many advanced deployments of the Pentaho BI Suite where some feature or requirement pushes the boundaries of the out-of-the-box product capabilities. Since the beginning of Pentaho time, we've marched to the beat of "make it possible first, then make it pretty/easy", and it's this scenario where our approach pays big dividends to our customers/community/users. Because the platform/server/tools were built for extensibility, there are numerous places where you can roll up your sleeves and leverage a simple API to implement a customization that suits your specific requirements.

The Pentaho Architect's Bootcamp is geared for developers, partners, customers, consultants that are ready to roll up their sleeves and understand the complex problems that are surfacing in large scale BI implementations, and how to extend the Pentaho suite of products to answer far more questions and customizations than the boundaries of the out-of-the-box product.

Here's a sampling of some of the questions that are answered during Architect Bootcamp training:

How do I integrate my own custom visualizations (maps, charts, gauges, etc) into Pentaho?
How do I accommodate multiple companies/groups/organizations' data in my solutions, while maintaining each companies/groups/organizations' personal point of view of the data?
How do I dynamically drive row level security in Pentaho Analysis and Pentaho Metadata?
How do I integrate Pentaho solution content into my own application?
How do I customize security across the Pentaho platform and pillars?
How do I create integrated solutions using ETL, reporting, analysis and metadata to deliver my customer's specific solutions?
How do I plug my custom content into the Pentaho BI Server? How do I then integrate my custom data/functionality with the Pentaho pillars (ETL, reporting, analysis, metadata, etc)?

I really believe this course is a game-changer for Pentaho users and solution developers. I teach the course, so don't take just my word for it. Here's some feedback we received from the first course offering:

"We traveled from India to Florida for this course, and we are extremely glad that we did. The information is very valuable."

"The presentation was excellent, [I e]specially liked the interactive nature of the class."

"All of the lessons uncovered new boundaries; all topics had more priority!"

"YEEEEAAAA.. just built my first Pentaho BI Server plugin... Pentaho Architect's Bootcamp RULES!!! #Pentaho @Pentaho"

"Day 4 Pentaho Architect's Bootcamp brought so many excitements, I loved it. even though my brain feels like it's going to explode!! #Pentaho"

Barcelona Pentaho Community Meetup 2009 Pics

2009-09-20T11:39:00.001-07:00

We fly home from Barcelona tomorrow, a lovely vacation and another fantastic Pentaho community gathering.

I'll be chatting with many of you online in the near future, and hopefully will see everyone again next year - Vienna is it? Sweet!

As promised, here's my pics :)

kindest regards,
G

Hola from Barcelona, Community Gathering 2009

2009-09-19T05:17:00.000-07:00

We're close to restarting the sessions for the afternoon, just dropping in to update my fellow Pentaho colleagues on the gathering:)

We got a bit of a late start this morning, mostly because the community started the meetup last night, and the socializing lasted into the wee hours for the group. A great time was had by all :) The morning's speakers had great content, covering a variety of topics, from Mozilla statistics presentation with CDF to the latest revision of PAT, the community analysis tool. Roland and Jos are here, our celebrated Pentaho Solutions authors, signing books and presenting the basics of developing custom CDF components.

And in traditional European holiday style, we're late in getting the afternoon sessions started, most of our group (around 40 attendees with community and Pentaho included)is still in earnest roadmap discussions at the cervesseria :)

Next post, some pics for posterity.

buenos tarde!
Gretchen

Pentaho Community, Together in Barcelona

2009-09-14T17:10:00.001-07:00

Doug and I arrived in Barcelona this morning, early enough to see some of this beautiful city before the Pentaho Community Meetup this weekend.

This is the second annual community meetup, an event that is organized and planned completely by Pentaho community for Pentaho community. No fluffy corporate speak, just a full weekend of Pentaho community developers and users showing off their stuff, talking through their current projects and solutions, and having a few beers and some fun. Many thanks to Tom Barber for planning and sponsoring much of this year's event.

We look forward to seeing familiar faces, and new community as well:0) See you all very soon!

Thanks Roland and Jos: Pentaho Solutions IN PRINT!

2009-08-21T07:46:00.000-07:00

I received my pre-ordered copy yesterday of Pentaho Solutions: Business Intelligence and Data Warehousing with Pentaho and MySQL. A huge congratulations and thank you to Roland Bouman and Jos van Dongen, two long time Pentaho community members who wrote the book.

I can' t tell you how excited I am to see this book! For many years, developers and project managers that I've worked with have felt that a book like this one is the missing link to helping customers achieve success with their warehouse and business intelligence strategies. Most books on business intelligence are either too abstract or offer guidance only on select pillars (for example, only reporting solutions), which leave the reader with unfulfilled requirements and no direction for filling in the gaps.

With Pentaho Solutions, the reader gets a concrete explanation and best-of-breed Pentaho implementation of ETL, reporting, analysis, dashboarding and data mining solutions; 5 core pillars and their concepts that contribute to a healthy, whole, successful BI strategy and implementation.

You can pre-order your copy at Amazon.com :)

Roland and Jos, the team has already sunk their teeth in, and they love what they're reading. Well, the picture says it all :)

Development and Debugging with GWT and Javascript

2009-08-12T06:43:00.000-07:00

Java code is the bread and butter of what I do, but as most Java developers know, there is a plethora of good frameworks and technologies that surround Java and provide a means to build very powerful and extendable software.

Lately, I've popped back into GWT land and have been dealing with lots of Javascript, both original and generated. I originally introduced myself to GWT building a small volunteer information submittal form for Brevard Rescue Mission. This tiny application only scratched the surface of what GWT could do in its earliest stages. The magic that the Pentaho development team have performed with Pentaho Dashboarding is a new level of web-goodness, fully capitalizing on the power of GWT. I've been dabbling in the chart rendering layers of dashboards, and have learned some simple, effective means of making life a bit easier when dealing with debugging and developing Javascript and GWT generated Javascript. I hope you find these tips useful, and it's certainly nice to have them aggregated in one place! I have to give credit for this info to Nick Baker and Mike D'Amour, two of my colleagues at Pentaho. Use this blog post as a starting point for googling the original sources for more details on each tip:)

Helpful tips for Developing and Debugging with GWT & Javascript in General

Limit the user.agent Property

GWT compiling is resource intensive due to the number of compilations that happen for the browsers supported. At times, you will run out heap space or other resources before the compile can finish (this usually manifests itself as a StackOverflowError).

The following entries in your *.gwt.xml file can help by only compiling for the single browser you may be testing on:

<inherits name="com.google.gwt.user.UserAgent"/> 
  <set-property name="user.agent" value="ie6" />

Valid values for the user.agent property are: ie6,gecko,gecko1_8,safari,opera

Limit the gwt.compile.localWorkers

You can also scale back the number of threads to use for running parallel compilation. While this may hurt performance, you will be able to finish the compilation without running out of resources. This property, gwt.compile.localWorkers, can be added to the compile option in your ant script.

Bump the GWT version from 1.6.4 to 1.7.0

GWT 1.7.0 seems to have resolved many of the compilation resource issues with GWT.

GWT Pretty Print Compile

By default, we obfuscate our GWT compiled Javascript. To debug readable GWT compiled Javascript, compile with pretty print turned on. This property, gwt-style, can be added to the compile option in your ant script. Valid values include OBF, PRETTY, and DETAILED.

IE Javascript Debugging Help

If you need to debug Javascript in IE, it is highly recommended that you get IE8. You can install IE8 for the duration of your testing, then uninstall it when you no longer need it, as it has conflicts with GWT. IE8 has a new set of features called Developer Tools that make debugging Javascript very easy.

Helpful In Line Javascript Alerts

You can use the following line of code to send alert windows whereever you like in your Javascript code:

$wnd.alert("Hello World")

A good example from Nick:

This is literally saving me hours. By adding a line to the the end of the
printStackTrace() function you can alert out the stacktraces that normally do nothing when compiled.

Open up the gwt script file (xxxxxxxxxxxxxxxxxxxxxxxxx.cache.html) for your particular browser. I find it by seeing what's loaded in firebug.

Search for "function $printStackTrace"

Add a new line right before the function returns:

$wnd.alert(msg.impl.string);

It should now look like this.

function $printStackTrace(this$static){
var causeMessage, currentCause, msg;
msg = $StringBuffer(new StringBuffer());
currentCause = this$static;
while (currentCause) {
causeMessage = currentCause.getMessage();
if (currentCause != this$static) {
msg.impl.string += 'Caused by: ';
}
$append_4(msg, currentCause.getClass$().typeName);
msg.impl.string += ': ';
msg.impl.string += causeMessage == null?'(No exception detail)':causeMessage;
msg.impl.string += '\n';
currentCause = currentCause.cause;
}
$wnd.alert(msg.impl.string);
}

In Code Breakpoints

Rather than sifting through the script debugger window trying ot figure out where to put a breakpoint, you can use the following line of code to embed a breakpoint:

debugger;

Feel free to comment and send your favorite tricks for working with GWT and Javascript.

Pentaho Analysis Tool Integrated as a Pentaho Plugin

2009-05-19T12:33:00.000-07:00

I had the chance this week to play around with the still-under-construction Pentaho plugin architecture in the Citrus code line. The new architecture is just what BI developers have been waiting for: totally flexible with several new ways to integrate with the server, simple to use and allows for building nicely decoupled extensions.

With Aaron Phillip's help, I got my head around the new features in less than a day, and had my first plugin written shortly after: The Pentaho Analysis Tool (PAT) plugin. Before I get into the details of the PAT plugin, let's first talk about the new tools and capabilities in the Pentaho BI server's plugin layer.

The plugin architecture consists of several different fun ways you can hook into the Pentaho BI Server, without having to modify server code or disturb the platform deployment. All avenues for leveraging the plugin architecture expect that the necessary files and code will be found in the solutions folders. The layer currently has the following capabilities:

Customization of the menu system of the "classic" and more recent PUC (Pentaho User Console) user interfaces
Customization of various page contents (overlays)
New types of content to be added to the solution repository and operated upon in the user console
New Java classes that generate UI pages to be dynamically added to the server
(new in 3.0) Add your own BI Component to the platform without having to modify system files and paths

You can get more details about these features and how they work by reading the documentation here. Aaron also created a sample plugin that demonstrates each of these features in a simple plugin mockup, that also is a great template to use for new plugin creation.

So that's exactly what I did. Here's a screenshot of the results of my plugin:

Using the EchoPlugin sample as a guide, I created a new content type (.xpav, for Pentaho Analysis View) which is the first notion of a view definition file for PAT. When you "open" this new content type in PUC, it initializes and launches PAT, which is a separately deployed web application. This is accomplished by creating a new content generator in the plugin that delegates the generation to the PAT webapp. It takes a bit to put it all together: you need a bleeding edge Citrus BI Server download, the latest PAT code and the plugin project. If you are interested in seeing it in action, read the integration instructions here.

I only took advantage of a couple of the new plugin layer's capabilities in my first plugin. I'm looking forward to playing with the new web services as well as the component that allows my plain old Javabean to look like a BI component automagically. I can foresee great extensions coming fast for the Pentaho BI Server with this new architecture!

I've listed some good references for those who are ready to take a look at plugins:

Here is the documentation:
http://wiki.pentaho.com/display/ServerDoc2x/BI+Platform+Plugins+in+V2

and here is the Plugin Depot, where you can show others the cool new extensions you've built:
http://wiki.pentaho.com/display/ServerDoc2x/Plugin+Depot

and if you have questions, comments or problems, or think you may have spotted a bug, chat with some of the developers about it here:
http://forums.pentaho.org/forumdisplay.php?f=73

kindest regards,
Gretchen

Maven: The Definitive Guide

2009-05-10T02:02:00.000-07:00

My one true nerdy tendency: I like writing technical documentation. It's ironic then (or a bit of a hypocrisy) that I loathe reading it. Chalk it up to my lack of passion for technology. I am a passionate problem solver; technology is a sometimes rewarding, sometimes frustrating means to help effectively get the problem solving job done.

Recently, I began reading Maven: The Definitive Guide while getting a pedicure at my local salon (laugh it up guys, I can guess where most of you do your leisure reading). I strongly recommend that any developer approaching Maven for the first (or tenth) time give chapters 3 through 8 a read. This guide is what you hope most technical guides or books would be, but then usually quite early on, they disappoint.

The guide starts with a quick, understandable introduction to Maven terminology and concepts, via a short step by step example. As I was reading this from a "make this worth my while" perspective, I had specific use case questions that immediately popped into my head ... and then, I was pleasantly surprised to find the answers in the next few paragraphs.

For example, the guide mentions early on that "support for transitive dependencies is one of Maven's most powerful features". To that my questions were "What about conflicts in dependency hierarchies?" and "What about compile time dependencies that I don't want to package?". The rest of the chapter addresses exactly those questions with explanations on dependency exclusions and scoping. Finally. A book that thinks like I do :)

A quick summary of the rest of the meat of the guide: chapters 4 through 8 build on the core concepts introduced in 3, with bite size chunks of additional functional explanation in each chapter. The material is presented as a hands on example, building in feature complexity little by little. Chapter 4 shows you how to add new dependencies to your project; 5 introduces simple web application features; chapters 6 and 7 cover multi-module project and enterprise project features. This presentation worked for me on a few different levels:

The graduated approach to the introduction of new materials made a large amount of Maven terminology, concepts and finally usage documentation digestible.
The authors take great care in describing WHY they arrange and refactor the projects as they do, in a very modular fashion. This approach, in practice, lends itself not only to Maven's default conventions, but also to best practices for software project layout. Note that this introduced complexity to the examples that wasn't necessary to explain the features at hand. But the authors bit that bullet in order to present a good and useful way of developing a project.
The example projects described in the guide are immediately relevant for me. I write Java code. I use Spring. I use Hibernate. This, of course, will not be the case for every reader, but it was a nice bonus for me.

So after all of my monologue thoughts, I leave you with a few tids:

Read the guide (at least Part 1), then decide how problematic you perceive Maven to be. I know my perception changed dramatically.
Note that it is a bit outdated, deprecated goals and such (the guide is updated for Maven 2.0.10, I downloaded Maven 2.1). This really didn't distract me at all.
The chapters I reference are only the tip of the iceberg. Part 2 of the guide includes another 200 reference pages that I have yet to use. I'll let you know how that goes for me:)

Every Developer Needs a Roadshow

2009-04-06T19:18:00.000-07:00

It's been a few days since returning from the Pentaho Partner Summit. When I get the chance to attend company events, conferences or seminars (the opportunities are rare), I try to sum for myself the benefits of having traveled, gathered and given my attention to the occasion at hand.

In the case of the Partner Summit, I thought of several key revelations that came about as a result of the trip. The one that stands at the forefront of my mind: every developer needs a roadshow.

Not as a roadie in a product tour, or as booth Bob at a trade show, but as an interested attendee at an event that showcases whatever you have been working on as a developer. Mind you, this is not a NEW revelation for me; I've had the privilege of representing Hyperion Analyzer at Java One as a developer on that project, and talked to many talented Oracle folks about Pentaho at ODTUG, as well as many other roadshows of my own. I always have come back saying the same thing to my peers - "You guys have to hear what they are saying! You have to feel the excitement!". ( Yes, their was a maddening amount of energy and excitement around the Pentaho Partner Summit!)

The benefits to sending developers out to events that have nothing to do with development and everything to do with the project or product are many. The first benefit that I got excited about in Menlo Park was that I was able to hear how our partners and customers were using Pentaho. I'm committed to focusing on what questions BI users are asking as I re-enter the BI space as a developer, and this was a prime audience. During networking opportunities, partners told stories about customers with big data on Vertica, MySQL, and InfoBright; in intranets, in DMZs, and of course, now in the Cloud. Pentaho partners OpenBI had an attentive and boisterous audience as they discussed their Cloud implementation with client Nutricia.

I also really enjoyed having face time with the consumers of the fruits of my previous efforts. I have been away for some time, but I think some parts of the Pentaho projects are still riddled with my signature:) It's OK that many, but not all comments were glowing; that's the point, right? I feel like I understand just a little bit better some of our users' pain points. And that puts me in a better place to alleviate some of that pain. (No worries, Brian and Nick and Domingo ... Will and Thomas will get right on native crosstabs!!!)

The Partner Summit event gave me the opportunity to lift my head up from the details of our projects and see the field from our partners' perspective. Can I get a lot of the same information surfing the web or hitting the forums? Sure. The perspective is unique though, to spending time with the people who are providing business intelligence solutions in the market. That, I believe, only comes on the road.

Pentaho Partner Summit: Menlo Park, CA

2009-04-03T07:28:00.000-07:00

It's a fortunate coincidence that I'm in California at the same time the Pentaho Partner Summit is going on. The event is packed, with partners and interests attending from more than 15 countries. The speakers yesterday were really quality, talking about everything from business intelligence in the Cloud to commercial open source business strategy.

So I've been able to spend lots of quality time connecting with old friends and colleagues, and have met some new, really talented folks. More details on the event later, but for now, check out some pics of the event here.

Cheers, -G

Because I've Got Issues ...

2009-03-26T23:38:00.001-07:00

I have to hand it to the guys over at Atlassian, JIRA is a pretty killer app (although I know now that Love is the REAL Killer App:) ).

I've worked with JIRA for, well, a really long time. I've always worked in companies where you needed to wear many hats, and I'm one of those developers that doesn't get snobby when I'm asked to step outside of my comfy Java home and help out the IT folks. So it's usually me that gets those prize winning projects like migrating forums, internationalizing wikis, or looking for new software to streamline our internal processes. I've spun JIRA around the dance floor several times, XSLT'ing crazy aggregate reports from XML backup formats, writing plugins to support externalizing JIRA data, customizing schemes, changing workflows. Every time, the same epiphany gets me - JIRA just works, exactly how you would think it should.

While some who are not so in the know might think, "Gretchen, you simpleton, it's a series of instructions to a processor, of course that's how it works". But those of us who bend software over and around daily know that few apps are actually written with quality, exceptional exception handling and in an intuitive manner that doesn't require years of higher learning and great tolerance for pain to adopt. (This is a very familiar concept particularly for those who use a certain unreasonable operating system).

This time, we need to move Mondrian's tracker issues from their original home on Sourceforge over to JIRA, which is our tool of choice for managing work and issues at Pentaho. With an assist from my other favorite killer app, Kettle, it has been a dreamy couple of days putting together the pieces to get Mondrian's issues to their new home. OK, maybe not dreamy, but certainly pain free.

Kudos, my Atlassian friends. You Aussies got it going on.

Where have you been??

2009-03-20T17:10:00.000-07:00

When I say "you", of course I mean me! I silently fell off the radar about 7-8 months ago, and am now finally re-emerging. Well, let me tell you what I've been doing in my "off" time:

First, let me introduce you to Jack David, my new baby boy. Doug and I were blessed with this little guy August 15th, 2008. He is the primary cause for my hiatus. I have been loving every minute of being home with my peanuts (Anthony, 13, Bella, 3 and baby Jack)! Alas, the lure of olap cubes, ETL and bug squashing safaris was just too compelling to resist, and it's time to return to the business world.

I also joined the board of directors as secretary of the Brevard Rescue Mission, a faith based ministry that provides whole-life transition resources for near-homeless moms and their kids. My dear friend Stacia Glavas is the founder, and I have been privileged to be able to handle her communications, marketing and administrative needs in between diaper changes.
I have been dabbling in a bit of graphic design and found that while I have no natural talent, the Adobe suite allows me to appear semi-talented in creating fun and compelling designs. I have since designed the web site for rescue mission mentioned above, as well as the logo for my daughter's new preschool, several business cards and stationery for friends, and my latest, most daring adventure: skinning a mySpace page for my photographer and friend, Yvette Gioia! Where WILL my curiosities take me?????
I also have managed to talk a friend of mine into letting me "borrow" his home renovation crew to renovate the entire exterior of our 25 year old home. That's right, I've decided that I also have some sort of qualifications as a contractor. Or possibly just a penchant for frustration and pain, we'll soon see!

As you can see, when Doug and I decided that it would be a good idea for me "take some time off" to adjust and organize our growing family, well, I may have misread the "time off" instructions :) I have thoroughly enjoyed my very full, engaged foray into being a stay-at-home mom. I actually just this month officially earned my soccer mom title, as Bella joined the local soccer team. I never knew there was a position for grass-pickers in soccer, but sure enough, my daughter has the exalted title!

While I will miss much of the freedom that comes from being my own manager, I miss my career more. So expect to hear from me soon, as I mull over my next career adventure! And of course, immerse myself back into the Pentaho Nation!

Kindest regards!
Gretchie

Pentaho Meetup in Mainz, Germany

2008-06-02T14:00:00.000-07:00

Pentaho is hosting its first community meetup in Mainz, Germany in a few short weeks. I like the format we've chosen, as it combines informal sessions with lots of food, drink and some touring. And everyone is encouraged to bring their ideas and latest projects for a show and tell.

Find out details and register here. It's sure to be informative and provide an opportunity to make some great Pentaho contacts.

Creating a Minimum Width Constraint on an SWT SashForm

2008-05-15T13:30:00.000-07:00

My friend Matt encouraged me to blog a quick snippet of code that many SWT coders might find useful.

If you've used the SWT SashForm, you know it's a nice widget in that it saves you a bit of work that you would do using a traditional Sash. One of the things I expected to get for "free" was the ability to limit how far the sash could expand or collapse a single control, or eseentially constrain the sash from completely hiding any number of it's child controls. To my surprise, I found that this is not a feature of the SashForm.

I have figured out how to create that behavior, by adding a selectionListener to the sash on the sashform. I have to give most of the credit to Duong Nguyen, who posted a partial solution to my problem here.

Without further adue, here's the code:

// Set a minimum width on the sash so that the
// controls on the left are always visible.

// First, find the sash child on the sashform...
Control[] comps = sashform.getChildren();
for (Control comp : comps){

if (comp instanceof Sash){

// My limit is derived from the size of a
// toolbar; yours can be any size you wish ...

int limit = 10;
for (ToolItem item : view.getParent().getItems()){
limit += item.getWidth();
}

final int SASH_LIMIT = limit;
final Sash sash = (Sash)comp;

sash.addSelectionListener (new SelectionAdapter () {
public void widgetSelected (SelectionEvent event) {
Rectangle rect = sash.getParent().getClientArea();
event.x = Math.min (Math.max (event.x, SASH_LIMIT), rect.width - SASH_LIMIT);
if (event.detail != SWT.DRAG) {
sash.setBounds (event.x, event.y, event.width, event.height);
sashform.layout();
}
}
});
}
}

Hope this is useful!
kindest regards,
G

Kettle, XUL, SWT and a Bit of Theory...

2008-03-27T17:35:00.000-07:00

I've been working on a very fun and challenging proof of concept / usability project that involves Pentaho Data Integration (Kettle), XUL, SWT, Swing and a very cool, slightly controversial theory: with a lot of work frontloaded, we should be able to port user interfaces to any UI technology, and expect consistent behavior AND look and feel without much, if any, additional code .

The project I'm in the middle of encompasses three different but equally compelling goals:

To move Pentaho forward in providing common layers across our core pillars (reporting , analysis, data mining, ETL and dashboards).
To provide a proof of concept for the Pentaho XUL Framework, an architecture built to help us support common UIs across all of our applications and tools .
To provide a common way of describing and managing the information needed to connect to a database.

So, we began by taking Pentaho Data Integration's connection dialog ( a powerhouse of detail and metadata concerning over 30 different database connection types), applied some usability design to it, and are in the middle of rewriting the dialog using XUL and some new design concepts.

This required a lot of up front work on the part of our lead Pentaho XUL developer, Nick Baker. Nick has plumbed an architecture for Pentaho that makes it possible for us to define a user interface in XML (specifically, using the Mozilla XUL specification), and use a common set of wrapper libraries to render the UI in SWT or Swing, without rewriting any of the UI, and reusing a great amount of the business logic code. There are also plans to begin a XUL-to-Web 2.0 library as well, which will cover most of the UI technologies that we use for the Pentaho projects.

Interested? Download the source and demo, take it for a ride, inspect the code and let me know what you think. Here are the home pages for both the Pentaho XUL Framework project, and the Common Database Connection Dialog proof of concept.

Common Database Connection Dialog
Pentaho XUL Framework Project

Go ahead and post here, on the wiki, or shoot me an email. I look forward to your reactions and suggestions:)

kind regards,
Gretch

PS. All you Kettle users & developers, here's a chance to weigh in on some needed UI changes for Kettle's database UI... don't miss the opportunity!

Pentaho on Oracle's App Server (OC4J)

2007-06-13T10:59:00.000-07:00

Wow! It has really been 4 months since my last post?? Moving over to development has cut into the time I had for blogging, documenting, communicating, you name it! We are coding like crazy.

Well, I'm back because I am heading out to ODTUG Kaleidoscope next week, and in preparation for the show, I decided to setup Pentaho on Oracle's Java Edition App Server, which is OC4J, which is based on the Orion app server. I was pleased that I managed the migration in less than a day, and I wanted to share the steps with all those folks who are too impatient to wait for this to get into our J2EE deployment distribution :)

Mind you, it takes a bit of tweaking, but it is certainly very do-able, and all server features are stable (minus the portal stuff, I didn't get a chance to address moving the portal over). Here is the repro of where I started, what I tweaked and what I came out with. Enjoy!

Where I Started

I started out by downloading a Pentaho J2EE deployment distribution from the Pentaho downloads site. The version I used for this exercise was pentaho_j2ee_deployments-1.5.4.716-0.zip. This distribution is found on our downloads page under Pentaho Open BI Suite Web Archive (.war). I know, the name implies a .war distribution, but trust me, its the deployments zip file.

Unpack this distribution to a work directory of your choosing. This distribution has an Ant build script that lets you build several different .war files and .ear files configured for different app servers and RDBMSs. Next, I'll detail the tweaks I had to make to get the Orion build target working, which is sufficient to buid an .ear file appropriate for Oracle's OC4J.

I also downloaded the sample solutions distribution and the sample data (Hypersonic) distribution, so I would have stuff to test against. You can get both of these distributions from the Pentaho downloads site as well.

What I Tweaked

About two years ago, we had started to incorporate build scripts for the Orion application server in anticipation of great community demand for this build. However, with the multitude of projects we have taken on, and a surprising lack of banging from the community, we never found time nor priority to finish that build, until now.

I really want to be able to demo our stuff on Oracle at the user conference, so I took on the task of cleaning up and repairing the Orion build. NOTE that you will be able to get these fixes in a near future build, as soon as we get our M4 release out the door and I can check this stuff in.

The Build

For the brave and impatient, here's what it needs out of the gate:

We'll call the root of your work directory [pentaho_j2ee_deployments]. In this directory, you will find a build.xml file. Open that file in your favorite text editor.
Find the ant target named "build-orion2.0.5-ear". Delete that target entirely. Replace the target with the following XML:


<target name="build-orion2.0.5-ear" depends="zip-pentaho-style-war">

<antcall target="war-pentaho-tomcat">
<param name="rdbms" value="hsqldb" />
</antcall>
<mkdir dir="${build.ears.dir}/orion" />

<ear destfile="${build.ears.dir}/orion/pentaho.ear"
appxml="${ear.dir}/application.xml">
<manifest>
<attribute name="Implementation-Title"
value="${impl.title}" />
<attribute name="Implementation-Version"
value="${impl.version}" />
<attribute name="Implementation-Vendor"
value="${impl.vendor}" />
</manifest>
<fileset dir="${build.wars.dir}">
<include name="pentaho-style.war" />
<include name="sw-style.war" />
</fileset>
<fileset dir="${build.wars.dir}/tomcat/hsqldb"
includes="pentaho.war" />
<metainf dir="pentaho-res/orion" includes="*.xml" />
</ear>

</target>
Save and close the build.xml file.
In the same directory, open the override.properties file. Add the following line to this file:
ear.dir=pentaho-res/ear
Save and close the override.properties file.

Those few steps fix up the build files so that you can build a pentaho.ear for OC4J. You of course will have to have Ant in your system path, and a JDK available of at least 1.4.2. I will assume if you are climbing this mountain, those are easy steps you already know how to set up:)

Don't run the build yet! There are several configuration file tweaks that have to be added in order for this app to be configured properly.

The web.xml File

We need to make a few minor changes in the Pentaho web.xml.

First, the OC4J container's default application is running on port 8888. Pentaho's default port is set to 8080. So I changed Pentaho's default port to be 8888, since that seemed to be the easiest road. NOTE that you want to add your changes to two web.xml files, since it is duplicated in the deployer distribution.

You need to modify BOTH

- [pentaho_j2ee_deployments]/pentaho-precompiled-jsps/pentaho.war/WEB-INF/web.xml

AND

- [pentaho_j2ee_deployments]pentaho-webapp/WEB-INF/web.xml !!

Open both web.xml files.
Find the base-url param-name, with the value http://localhost:8080/pentaho/. Change the value to http://localhost:8888/pentaho/.

We also need to make sure the server can find the Pentaho solutions directory. If you haven't yet, unpack the solutions distribution that you downloaded to a work directory of your choice.

Find the param-name pentaho-solutions, and replace the value with the absolute path to the pentaho-solutions directory that you just unpacked. The value should look something like d:\work\pentaho-solutions.
Save and close both web.xml files.

The application.xml File

The application.xml file lives in the [pentaho_j2ee_deployments]/pentaho-res/ear directory.

Open the application.xml file.
Delete all modules under the comment  , as well as the web-uri module for the sw-style.war above the comment. The only modules your application.xml should have left is the pentaho.war and the pentaho-style.war.
Save and close the application.xml file.

Note that the sw-style.war provides the styles for the Steel Wheels samples in the solutions, but I did this quick and dirty, so I left out as much extras as possible. You may want to include the sw-styles.war, and see if it works. I left it out.

The orion-web.xml File

The orion-web.xml file is non-existent, you need to create one.

Add an orion-web.xml file to [pentaho_j2ee_deployments]/pentaho-webapp/WEB-INF.
Add the following XML to the file:
<orion-web-app>
<resource-ref-mapping location="jdbc/HibernateDS" name="jdbc/Hibernate"/>
<resource-ref-mapping location="jdbc/SampleDataDS" name="jdbc/SampleData"/>
<resource-ref-mapping location="jdbc/QuartzDS" name="jdbc/Quartz"/>
<resource-ref-mapping location="jdbc/SharkDS" name="jdbc/Shark"/>
<resource-ref-mapping location="jdbc/SampleDataAdminDS" name="jdbc/SampleDataAdmin"/>
<resource-ref-mapping location="jdbc/SampleDataDS" name="jdbc/datasource1"/>
<resource-ref-mapping location="jdbc/SampleDataDS" name="jdbc/datasource2"/>
<resource-ref-mapping location="jdbc/SampleDataDS" name="jdbc/datasource3"/>
<resource-ref-mapping location="jdbc/SampleDataDS" name="jdbc/datasource4"/>
<resource-ref-mapping location="jdbc/SampleDataDS" name="jdbc/solution1"/>
<resource-ref-mapping location="jdbc/SampleDataDS" name="jdbc/solution2"/>
<resource-ref-mapping location="jdbc/SampleDataDS" name="jdbc/solution3"/>
<resource-ref-mapping location="jdbc/SampleDataDS" name="jdbc/solution4"/>
</orion-web-app>
Save and close the orion-web.xml.

Build the pentaho.ear File

You've got everything you need now to build the pentaho.ear file. Go to a command prompt, navigate to your [pentaho_j2ee_deployments] directory, and execute:

ant build-orion2.0.5-ear

OC4J Server Configuration Changes

Now that you have your .ear file, you will obviously want to deploy it through OC4J Server Console. First, shut down your OC4J instance, because we have a few mods that we need to make to the server configuration.

TopLink Conflicts

Now, I know this is not nice, but since we have no use for Toplink with Pentaho, and the antlr.jar conflicts with our hibernate3 library, you will need to delete the Toplink directory under [OC4J_Home]. This causes no harm as long as you're not using Toplink for some other reason. If so, then maybe you can investigate further with Oracle how to avoid the library conflicts.

This will cause the java_sso webapp in the default module deployed in the server to fail. I simply undeployed the app.

Out of Memory Errors

Pentaho is a big engine, and as such uses a bit of memory. I found that I was running out of memry early and often. So I modified the JVM_ARGS parameter in the [OC4J_Home]/bin/oc4j.cmd file to include:

set JVMARGS=%OC4J_JVM_ARGS% -Xms128m -Xmx512m -XX:MaxPermSize=128m

Add the hsqldb.jar to the [OC4J_Home/lib] Directory

Since we will be testing against the Hypersonic sample data, we need to add the JDBC driver for Hypersonic to the [OC4J_Home/lib] directory. You can find the hsqldb.jar in [pentaho_j2ee_deployments]/pentaho-third-party.

Note that when you deploy the .ear file, you MUST set an additional entry on the classloader's classpath for this jar!

Mod the data-sources.xml File

And finally, we need to add our datasources to the [OC4J_Home]/j2ee/home/config/data-sources.xml file. Add the following xml to the file, before the <datasources/> end tag:

<data-source
class="com.evermind.sql.DriverManagerDataSource"
name="Hibernate"
location="jdbc/HibernateDS"
xa-location="jdbc/xa/HibernateDS"
ejb-location="jdbc/Hibernate"
connection-driver="org.hsqldb.jdbcDriver"
username="hibuser"
password="password"
url="jdbc:hsqldb:hsql://localhost/hibernate"
inactivity-timeout="30"
/>
<data-source
class="com.evermind.sql.DriverManagerDataSource"
name="SampleData"
location="jdbc/SampleDataDS"
xa-location="jdbc/xa/SampleDataDS"
ejb-location="jdbc/SampleData"
connection-driver="org.hsqldb.jdbcDriver"
username="pentaho_user"
password="password"
url="jdbc:hsqldb:hsql://localhost/sampledata"
inactivity-timeout="30"
/>
<data-source
class="com.evermind.sql.DriverManagerDataSource"
name="SampleDataAdmin"
location="jdbc/SampleDataAdminDS"
xa-location="jdbc/xa/SampleDataAdminDS"
ejb-location="jdbc/SampleDataAdmin"
connection-driver="org.hsqldb.jdbcDriver"
username="pentaho_admin"
password="password"
url="jdbc:hsqldb:hsql://localhost/sampledata"
inactivity-timeout="30"
/>
<data-source
class="com.evermind.sql.DriverManagerDataSource"
name="Quartz"
location="jdbc/QuartzDS"
xa-location="jdbc/xa/QuartzDS"
ejb-location="jdbc/Quartz"
connection-driver="org.hsqldb.jdbcDriver"
username="pentaho_user"
password="password"
url="jdbc:hsqldb:hsql://localhost/quartz"
inactivity-timeout="30"
/>
<data-source
class="com.evermind.sql.DriverManagerDataSource"
name="Shark"
location="jdbc/SharkDS"
xa-location="jdbc/xa/SharkDS"
ejb-location="jdbc/Shark"
connection-driver="org.hsqldb.jdbcDriver"
username="sa"
password=""
url="jdbc:hsqldb:hsql://localhost/shark"
inactivity-timeout="30"
/>

Problem in the Solutions' XSLTs

I don't get this at all, but Oracle has some sort of built-in XSLT processor that complains when your XSLT tries to reference a Java class as the value for a namespace. So what you need to do is go through the [pentaho-solutions]/system/custom/xsl directory, and check every xsl for this occurrence (I would say at least 20 of them have it) and make the following mod to those XSLTs:

Open the xslt file.
Find the java class reference in the namespace declaration. They are always at the top of the file, and look simialar to xmlns:msg="org.pentaho.messages.Messages" . Note that xmlns:msg isn't the only possibility, just an example for you to follow.
Prepend the java classname value with http://www.oracle.com/XSL/Transform/java/. So the fixed example would be xmlns:msg="http://www.oracle.com/XSL/Transform/java/org.pentaho.messages.Messages" .
Repeat the steps above for every xslt file in that directory that has the namespace occurence.

There is one syntax problem in one of the xslt's as well. JBoss and Xalan don't seem to care about it, but OC4J will error out, so we need to fix that.

Open the file [pentaho-solutions]/system/custom/xsl/files-list.xml.
Search for the following lines of XML, and delete them. Note they are not right next to each other.
<xsl:param name="target" />
<xsl:with-param name="target" select="$target"/>

Deploying the Pentaho .ear File

Finally!! You can deploy your .ear file now. Start your OC4J server back up, and use the Server Console to deploy. Here are a few notes on the deployment options:

You do not need to set up a security provider in the deployer wizard. Pentaho uses the standard J2EE security via ACEGI, and no custmo extensions to the configuration are necessary.
Make sure that you map a path on the classloader to the hsqldb.jar that you moved into OC4J's lib directory!
You may see some log4j exceptions on start-up, but as long as you see a statement toward the end of the console log that says "Pentaho BI Platform ready", then you should be ready to go!

Navigate in your browser to http://localhost:8888/pentaho/Home, and test it out!

What I've Got - Pentaho on OC4J!

So, now I'm stoked because I can talk intelligently to Oracle tools users about Pentaho on Oracle. I think I might even dive in and use Kettle to move our sample data over to Oracle Express.

Very soon all this should be doc'ed and available through our SVN repository... hopefully any early comers who use this tip will let meknow how it goes for them:)

And please, if you will be at the conference in Daytona next week, stop by the Pentaho booth and say hey! Can't wait to meet you all!

Gretchie

Wrap up on the Pentaho Implementation Workshop

2007-02-03T13:25:00.000-08:00

All in all, I had great expectations for the 3 day implementation workshop, and I wasn't disappointed!

The last session on Thursday, Dashboards and AJAX, was crammed full of great technical information. The most important info that I can give you all is that the current JSP based dashboards are going away, in exchange for dashboards built upon Pentaho AJAX components, currently under development. Of course, we all were very excited to hear about the new AJAX component architecture, and happy to hear that we can get to the code. The ETA for GA delivery is somewhere in the 3rd or 4th quarter of this year (with milestone builds available earlier most certainly), but always stomping on that hairy edge, I can't wait to dive into the code and contribute to implementation (in my spare time, ha ha! ).

James Dixon, Chief Geek for Pentaho, also went into a bit of the history of Pentaho Dashboards, which was really helpful in gaining perspective on why Dashboards require so much coding today. The philosophy and design goals for Dashboards (really for the platform, in general) is to remain delivery agnostic - meaning we want the platform output to be delivered via the client's choice of technology, not our own. So if you are a JSF shop, .NET shop, or Java applet guru, it won't matter to us, since we deliver the content from the platform in XML. You can take it from there, and transform that XML any way you wish. Well, that design goal is tough to stick to when you are implementing a Dashboard architecture, since Dashboards are heavy on UI, usually containing reports, charts, dials, gauges and numerous other widgets, in some portal type fashion specific to the user's point of view. So we went about component-izing all of the above mentioned widgets, and used a simple JSP (or not so simple JBoss Portal) to demonstrate what COULD be done. The response from our community has been to make Dashboards easier to build, and fortunately, AJAX has conme along (or been there, depending on how you look at AJAX) so that we can deliver that ease of implementation.

I haven't had a chance to say much about the group that attended the class with me. I was tickled to finally meet some folks that I have been chatting with via email, some for more than a year now! Nice, intelligent, talented and truly passionate about BI - I could spend alot more time with these folks, we share so many traits and interests (I know, I think very highly of me ;))! The attendees came from Canada, the Netherlands, Germany, France and the US, which speaks for the demand that our training generates, as well as the global presence that Pentaho has earned in a short 2.5 years. I can't believe sometimes that I am a part of something this big, and this bold! On a day to day basis, it feels like we are just a bunch of guys doing what we have done best for a long time - building BI. But when you gather your community, partners and teammates in a room like we did last week, it sure feels a whole lot bigger, a lot more significant. And so, I can someday explain to my daughter why my job makes me proud :)

Workshop, Day 3

2007-02-01T08:19:00.000-08:00

I know, I know, what happened to Day 2? Well, between busy sessions, picking up the kids, and teaching my night class at Brevard Community College, blogging fell to the way side. For my three avid fans, I sincerely apologize, but by now you are used to it :)

So Day 2 of the workshop, in all honesty was a bit of a fire fighting exercise. Since we are working with the very latest code for the platform (and I mean VERY latest), we ran into a couple of problems during the Subscriptions session that prevented us from seeing the results of the subscribed bursting examples that we set up. But the content was good and subscriptions in the stable builds is very very powerful. We have the ability with the platform to relieve the administration and information overload that occurs with the typical scheduled reporting process. Subscriptions also prove to be flexible and easy to use, which makes it a nice tool for consumers of Pentaho reports, analysis, ETL and processes.

The Day 2 afternoon session was all about advanced deployments of the platform, as well as customizing deployments for each user's environment. Brian Hagan walked through the complex details of manually deploying web application through JBoss and Tomcat, focusing on the touch points that are required when you have your own app server installation already in place. Overall a good session that could be helpful to anyone that struggles with J2EE deployments today.

Day 3 started out with Bill Seyler, a stellar Pentaho engineer, presenting the life cycle management features within Pentaho. For anyone who is not familiar with the term and what it means in Pentaho, life cycle management is versioning solution content for the platform. Bill covered the architecture of how Pentaho interacts with version control systems, which seems to be a very clean and simple implementation. The beauty of life cycle management in Pentaho is that due its simple interface, any version control system can be used, as well as using multiple systems for one Pentaho deployment.

Next up was Anthony DeShazor, our engineering wrangler, talking about scalability and clustering. Much of this session covered the general topology and infrastructure issues that prevent almost any application from scaling. The point I took away is we can control what the Pentaho application does, but how you get at your data and how you deliver it out to consumers can bottleneck any good app. It was great to participate in tis discussion, since many in the room are experienced in the field and had much to contribute. Anthony then took us through the JBoss Clustering presentation that James Dixon presented at JBossWorld late last year. It was a simple architectural discussion covering JBoss Clustering, ending with some pretty impressive benchmarks that proved Pentaho's ability to scale. The most interesting news that came out at the end of this session is that Pentaho has started to build a BI benchmarking bundle, based on the Transaction Processing Performance Counsel processes, and plans to release it to the open source community for benchmark responses. Feel free to email James (communityconnection@pentaho.org and it will get forwarded) if you are interested in participating in that effort!

Our last session after lunch is Dashboarding and AJAX, a session that all the trainees, including myself are looking forward to. I'll fill you in tonight on how that session goes and how this all wraps up.

Implementation Workshop: Metadata

2007-01-30T12:30:00.000-08:00

"My users think I'm a God." , Matt Harbert, DivX

I had to start with that quote, I was so pleased to hear it during discussions over lunch this afternoon during the workshop. I was talking with a couple of classmates about Kettle, the Pentaho Data Integration tool, reveling in stories of leaping tall buildings with Kettle as my booster pack. From across the room, Matt tossed the aforementioned words of glory, and I thought "well, if that doesn't just sum it all up". Kettle really is that good, and you'll only know if you dive into it, because, of course, I am a Pentaho-an, making me slightly biased ;)

These were the topics of discussion that opened the Metadata module of the workshop. Metadata is another project being architected and driven by Matt Casters, founder of the Kettle project. Jake Cornelius led the module, and did a nice job of showing us the Pentaho metadata Editor, a handy tool that assists in building Pentaho metadata models. The Metadata Editor is not quite 2 months old, and from what we were shown, is proving to hold lots of potential. The core functionality is there, and the user interface is intuitive, once you learn the new jargon. Amidst a short array of funky behavior and a few bugs, the Metadata Editor's power shined through with it's ability to model not only mappings to physical tables, but also extended formulas, formatting and style properties in a hierarchical fashion (termed "concepts"), and internationalization functions.

This is the end of day one, time to pop over and visit the dev guys on my way home. I have a renewed sense of excitement today, partly because this training is turning out to be even better than I had expected (and my expectations were pretty high), and partly because I'm training up for a brand new seat on the Pentaho ride :)

Implementation Workshop: Security Simplified

2007-01-30T09:08:00.000-08:00

Well, we just finished the security module of the workshop, and I have to say, sans the network issues, I am really impressed. Mat Lowry, a Pentaho engineer who focuses on security during his day job, put together the content for the module today. Mat took a pretty complex set of topics (LDAP, Acegi, CAS and J2EE Container Security) and delivered just enough content to understand easily what Pentaho Security is made up of, and what Pentaho adds to the standard technologies available to you in a J2EE environment.

It seems we have done a very nice job of separating the wrangling of authentication and authorization from the functionality of the BI platform. I plan to follow up this workshop with a deeper dive into Acegi, as Mat has gotten me really excited about what it and the Spring framework can do. I'm taking away the relief that Acegi can handle a good 80% to 90% of my web resource security problems, without me having to write more code. I like it, I like it.

My thoughts on the hands on lab is it really made me think and I was pleasantly surprised to find that I understood the concepts Mat covered, and could apply them in the 60 minute lab exercise that was given to us. This was not your typical training class exercise that with loads of screenshots and step-by-step instructions, you could achieve one simple implementation of security. This was more like "A train leaves Tampa at 400 miles an hour at the same time a train leaves Daytona at 200 miles an hour, when will they meet" type of exercise. Now, I think I've mentioned before that I'm a very bright, but pretty simple person, and frankly this type of exercise reminds me how little focus I have. Once we stopped chatting, and I could read the lab carefully, I had no trouble implementing my own switch over from memory based security to LDAP based security within the Pentaho platform.

I have to admit, I feel a tad bit smarter than I used to :)

Pentaho Implementation Workshop

2007-01-30T05:52:00.000-08:00

This week, I get the pleasue of sitting in on the Pentaho Implementation Workshop, a hands-on in depth training session covering many advanced implementation features in the Pentaho BI platform.

This training kicks off my new role as a Pentaho developer! I decided to move over to the engineering side after a year and a half leading the Pentaho community, which was a very rewarding experience. You knnow what they say, you have to go where your heart takes you :)

Here's the workshop agenda:

Security
Metadata
Dashboards and AJAX
Reporting User Interface
Life Cycle Management
Advanced Deployments
Scalability

Community members here - Roland, Samuel, Fabrizio, welcome, and it's so nice to finally meet you in person :)

I'll be blogging on the workshop all week, so stay tuned!

Follow-on to Internationalization.... Vote!

2007-01-12T08:55:00.000-08:00

If you read my previous post on internationalization, you know that I'm looking for a great Confluence solution for handling multiple translations of the Pentaho documentation inside our new wiki.

Go here, login and VOTE for Atlassian to help us solve the problem!

And my apologies to Atlassian regarding my comment that they may not be responsive in their forums - I wasn't monitoring the thread they responded to, only four short days after I posted my dilemma. I will have crow for dinner :)

Thanks everybody!
-G

The Hefty Job of Internationalization

2006-12-14T05:26:00.000-08:00

When I worked in the purely commercial software world, internationalization of a product was a critical, magical event that happened somewhere offsite in the hands of contracted translation companies. Having your software translated into as many different languages as possible is just as important in the open source arena. Arguably, open source projects NEED translations MORE to grow into a global presence. The PROCESS of translating software and documentation for an open source project is an entirely different experience from the commercial event depicted above, as I have learned over my last two years with Pentaho.

This topic is at the forefront of my writings today because I have been tasked with figuring out how to manage localizing the Pentaho documentation in a wiki that has no feature support for internationalization of content! Did I mention that Pentaho is moving all of their documentation to a wiki? Well, there it is. The cat's out of the bag. As of our 1.2 GA release, the documentation will be maintained, by community and development team, in a Confluence wiki.

Confluence is a great tool and has lots of integration points with our case tracking system, JIRA. That said, it seems that Atlassian (the company behind Confluence and JIRA) is a little behind in the internationalization game. Confluence only recently started to support language packs for translation of the wiki itself, and has no support for content translation.

So here we are, and I need to figure out a solution to satisfy three very important groups. The first group are the translators of our documentation. They are community members who contribute those translations to our projects. We need to set up the internationalization in the wiki so that it is easy for these folks to do the initial translation, and also have a mechanism for notifying them when the master language version of the doc has changed. The second group is the users of the documentation. It should be easy for me, if I am French, to find the French documentation, but also be able to peruse the English documentation. And the third group is the poor guys in house that have to maintain the organization of this documentation. When you consider we have over 10 projects, translated (so far) into 8 languages, that get a new revision of documentation for every version of the project released... well, that's alot to manage.

So my initial attempt at a design for this conglomeration that we need to support was to try to stuff all of the languages in the same document in the wiki using DIV type tags and such for separation. I'd then use some custom code to hide the other translations based on the user's language setting in the browser. This would make my translators happy initially, because they can do the initial translation almost inline with the master language version. My users would be happy because our wiki respects their browser's language settings, and if a particular piece of content hasn't been translated to the user's language of choice, we would default to the master language version. Of course, this solution does not address the translator notification of master language changes, and well, it would be a bit of a pain to determine whether it was an master language version change or a translation change, with all that content in the same document. Also, this only addresses translation of the content. What about the document titles? Since the navigation of a wiki (by definition) is based on document name, we have a big problem to solve there. And the largest point of failure in my grand plan is the fact that the merging capabilities are not so hot in our wiki of choice, so the translators would have to line up and take turns translating. Ick, in a word.

So the next path we steer down is that path that takes us to completely separate repositories (called Spaces in Confluence) for the different languages. This gives us autonomy for the purposes of editing and not having the language content intermingled, but at a pretty large synchronization and maintenance cost. We now need to figure out, do we populate the French repository with all of the English documentation, to assist our translators in translating? Well, then we have up to eight copies of the doc, that is changing realtime, and is sure to get out of sync. So, perhaps we should let the translators populate their language wiki from scratch, organically? This isn't very accommodating to our translators, and documents will surely be placed out of order with the master language wiki, making it confusing for the users.

Yikes. It's at this point, having discussed the plethora of less-than-stellar options with a few clever guys on our team, that I decided to step back and write to the community. In my mental gymnastics over this problem, I made many assumptions about what our users and translators really want.

For the translators in our community, have you worked at translating in a wiki before? What did you like about it? What did you hate? Is it easier for you to translate everything in your format of choice, or do you like the idea that once you translate it it would be available, without having to wait for the Pentaho team to publish it? Of the two scenarios I detailed above, which is the lesser of two evils for you?

And for all of the rest our community that must USE and update our documentation - would it be more frustrating for you to work with translations inline in the wiki (in edit mode only) or to have to go hunt around someplace else to find those translations?

And, of course, for any other open source project that holds the silver bullet to this problem - feel free to share your solution here!! Heck, I'd even take well intentioned guesses and good ideas :)

Google Maps Demo, Check It Out

2006-10-31T10:56:00.000-08:00

You really need to check out our demo of Pentaho's Google Maps integration, put together courtesy of Jake Cornelius, Product Manager for Pentaho Dashboards.

I like this demo for alot of reasons, but first for it's content quality. I will be the first to admit, I have a short attention span for any type of sales or marketing related collateral. Unless it's funny or can incorporate a bit of irony, I tend to hear the adults in Charles Schulz's Peanuts cartoons - wah, wah, wah, wah, wah.

Jake's demo is a great, get-to-the-point example of location intelligence - solving a problem that has a geographical component in the equation. Jake takes you through a common case where a business executive takes a trip to the East Coast, and wants to know who are the key customers by sales volume he should visit during his travels. They need to be big spenders and - that geographic component - should be located relatively close to one another so he can maximize the number of customers he can visit.

The solution was simple with Pentaho BI and Google Maps. Check it out - you will be impressed.

From a developer's point of view, I liked the demo because I know I can download this solution in the Pre-Configured Install, and not only do I have a great example Pentaho Dashboard that I can pick apart, but I also get to see AJAX and the Google Maps API in action. For the hands-on learner like myself, this demo is a nice introduction to some very cool tools I can now put in my BI toolbox.

So go get it. Download it, listen to it (it's less than 8 minutes long) and come back and tell me what you think. Not your typical demo, right?

A Whole Lot of Community Goin' On...

2006-10-10T07:52:00.000-07:00

I haven't had the chance to get back to blogging lately. This really is a quandry - there is so much going on at Pentaho and around the Pentaho community, I have so much to blog about... but there is so much going on at Pentaho, I haven't found the time to blog! :) So today's a fun day, because I get to catch you all up on so many great things that are happening to and for the Pentaho community.

SourceForge.net Project of the Month

Let's start with Pentaho being honored as the SourceForge Project of the Month. If you have worked with open source for even a short while, you have heard of SourceForge.net. It is THE place to go to download open source software and source code. I have heard the number of projects hosted on SourceForge.net to be somewhere around 130,000, so to be chosen and highlighted as the project of the month for October, well, we take that as a very big deal.

This is a huge win for us the community because this showcase should draw alot of attention from savvy developers who are looking for an interesting and well-architected project to work on. I just can't convey all the great possibilities that could come out of this highlight. We are very excited! I will keep you up to date on the happenings around this win, as they unfold :)

New Web Sites for Mondrian, JFreeReport and Kettle

The next great community move is the new web sites for Mondrian, JFreeReport and Kettle. We have built new sites for the projects and the web team will be maintaining them and providing the TLC that the sites deserve.

This a big plus for the leads of the projects, since that takes the maintenance burden off of them and they get to concentrate on doing what they love - implementing the vision for their projects! The content for the sites is roughly the same, mostly just the look and feel, and the URLs have changed. Content changes will come slowly over time, when it makes sense to update and re-organize it for each team. The new URLS for the sites are listed below:

Mondrian: http://mondrian.pentaho.org
JFreeReport: http://jfreereport.pentaho.org
Kettle: http://kettle.pentaho.org

Pentaho and MySQL Webinars

We have had an awesome series of webinars running with the MySQL team. Since MySQL requests in the community seem to dominate, this series has proven very valuable to both the Pentaho community and the MySQL community. If you haven't viewed them, you can watch the replays here:

Business Intelligence with MySQL and Pentaho

ETL for MySQL using Pentaho Kettle

And coming up October 24, you can register for the latest webinar, OLAP for MySQL using Pentaho Mondrian.

Contributor Update

As always, we have recieved numerous contributions from different corners of the world over the past few months. I would like to share a few of them, just to demonstrate what is POSSIBLE for any community member, and what is being accomplished today.

A few weeks back, I recieved an email from Marc pointing me to a forum thread where Steve sent an example of his solution, which contains .NET consumption of Pentaho web services, producing an ADO dataset. Hoowah! I still need to contact Steve and see if he would write us a a nice technical article on his solution, but you know, so many hours in a day:)

Radek, a regular contributor, sent us the code for outputting multiple reports into different worksheets of the same Excel workbook. If your interested in that feature, you can track progress on it here.

By far our biggest avenue for contributing is in the translation of the platform. Last week alone, I recieved updates to 4 different language translations, 3 new Brazilian Portuguese documentation translations (thanks Fabrizio!), and a request to start an Italian translation of the project!

Thanks to the Pentaho Community!

At the end of the day, we know that our success is attributed to the the hard work of the Pentaho team, and the dedication and commitment of our community. We would like to say thanks again for the support. Running an open source project is a great experiment in trusting teams to provide resources for a common goal. It's with pleasure that we participate in this process, and see people at their best, making things work to achieve a common benefit.

Check Out Pentaho Training

2006-09-29T05:09:00.000-07:00

I recently sat in a meeting with our new Product Manager, Jake Cornelius, listening to some great feedback on a Pentaho training session he recently attended.

I worked with Jake back in my Hyperion days, so I can confidently say that Jake is pretty great guy who isn't one to blow wind up your shorts for heck-all's sake. He's a strong leader and has deep insight into BI, having worked in the industry for umpteen (a great figure when you don't have a precise number, but it's been pretty much forever) years.

I asked Jake to send me his thoughts on the session, so I could share them with my ever growing blog-base :) So here's the review, and I hope this helps anyone on the fence about investing in Pentaho training come on over.

Jake:

We recently rolled out our new training series on Building Analytic Solutions using the Pentaho BI Platform. The 4-day event, held September 18-22 in Orlando, was a huge success and confirmed Pentaho’s position as the global leader in Open Source Business Intelligence. Attendees from Japan, Columbia, Canada and around the US were present to learn how Pentaho’s BI Platform will help their organizations unleash the valuable information currently held hostage in proprietary data stores and transactional systems. The course was facilitated by Dave Reinke and Brian Senseman of OpenBI. In four short days, Dave and Brian lead attendees through an end-to-end BI implementation example from ETL using Pentaho Data Integration Services, to reporting with Pentaho Reporting Services and the BI Platform, to interactive analysis using Pentaho Analysis Services. Unlike conventional product training courses we’ve all sat through that focus on “this feature does x, and that feature does y”, OpenBI leveraged real world examples and best practices from their years in BI consulting to teach the class how to build enterprise-class solutions using Pentaho. The feedback was overwhelmingly positive, here are just a few of the comments we received from the attendees:

“Great class!”

“Thanks for the course. There were a lot of interesting ideas talked about.”

“This has been a great class and I have learned quite a bit. Perhaps more importantly, I have gained confidence in both the quality of the software and my ability to use it efficiently.”

If you are currently investigating Pentaho for an upcoming BI project, I strongly recommend you consider attending one of our upcoming training courses. A current list of planned training courses can be found on our website at http://www.pentaho.com/services/training/ .

Jake Cornelius
Product Manager

Hopefully we'll here more from Jake in the near future, as he will be a key player in helping Pentaho deliver project roadmaps out to the community. The entire team (me especially:) is insanely excited about that endeavor, so you can imagine how happy we are to have Jake on board!

Of course, COMMUNITY voices are heard the loudest, so I invite the Pentaho Nation to comment here, email me at communityconnection@pentaho.org, or post in our forums what it is that YOU need on those roadmaps!

BlogOrlando Un-conference: Just a Fantastic Friday

2006-09-23T05:44:00.000-07:00

You couldn't ask for a better Friday.

I started out early driving in to Winter Park, Florida, about 60 miles from home, to attend the first BlogOrlando Un-conference at Rollins College. I'm not saying starting my day with a 60 mile commute is all that fantastic, but it does give me time to catch the news on the radio, enjoy my favorite cup of coffee (7-11, if you haven't tried it, you don't know what you're missing!!), and we are experiencing unseasonally mild, sunshine filled weather right now, so the drive isn't even a consideration anymore :)

The BlogOrlando event was hosted on Rollins Campus, and drew a mix of PR professionals, marketing gurus, technical entrepreneurs and various other business folk. I was pleasantly surprised and interested to chat with such a diverse group, especially since landing in this role that has me straddling the fence between marketing and the development groups at Pentaho.

The first session I attended was on Corporate Blogging. Dave Coustan, the corporate blogger for Earthlink.net, facilitated the session and had some very interesting lessons learned and success stories to share about his journey building the very popular Earthlink.net corporate blog. While Dave made many points, the advice I liked the best was "Write so that any smart 14 year old can understand it". This helped me in numerous ways - I no longer feel embarrassed that I am incapable of writing pretty prose, and many of my Pentaho co-workers refuse to claim a birthday past their 14th. So you can see how this advice works so well for me:)

Some other interesting and valid bits from Dave on running a successful corporate blog:

Find an angle for boring stuff
Draw a crowd, figure out the content to draw it
Advises to separate blogs for personal and corporate
Get the umbrella knowledge of what is happening in the company - be your own "beat cop"
Research and fact-check your work!!!
Regularly perform outreach and follow ups
Capitalize on the ability to handle anything that falls out of traditional channels - edge cases

Some helpful points that were made regarding those who are just trying to convince their companies that corporate blogging is a net plus:

Cheaper, more trusted, lasts longer than ads
Start a dialogue with your customers
Valuable content for search
Rapid publishing
Gives press something to link to
Ear to the ground feedback loop
Safety valve for customer service/pr/media
Free syndication

The next session was "PR and Blogging", where there was much discussion around bloggers versus journalists. Is there a difference? Are journalists held to a higher standard than bloggers? And, of course the accountability of bloggers was introduced here and carried over in detail in the Legal session. Most of the conversation was speculative, and it seems that the PR folks who blog know the by-laws of good journalism, and use them whether they are blogging or running a corporate campaign. This of course is not always the case, especially for the casual blogger. I think the most useful point for me that was discussed in this session was the fact that the amount of blogger traffic you generate is only one measure of a blog's success, and will peak and valley. The true strength of your blog is in internet indexing. The indexing of a blog entry is much farther reaching than the direct audience of the blog.

The last session I was able to attend was "Blogging and Legal Issues", facilitated by Andrea Weckerle, a vibrant and very well-presented attorney and PR professional. I was very impressed with Andrea and wish I had a chance to connect with her to talk more about the specifics, but lack of time prevented me from introducing myself. I will have to stop by her blog. The legal issues that we discussed covered things like libel, slander (spoken and slander both start with "s", hence the difference:), copyright law and privacy issues. I was not surprised that the law covers these issues in much the same way for blogging as for written content. Regarding copyrights specifically, it seems the consensus is that in blogging, alot more violations happen, and alot less action is taken (think splogging).

I had to leave Rollins around 2:30, but if you want to read more about the afternoon sessions and wrap up, you can find all sorts of blogs and details at www.blogorlando.com, or google for BlogOrlando.

Thanks to Josh at Hyku for putting on such a nice event, in such a nice place. It made my Friday:)

Analytics For Bugzilla

2006-09-15T06:38:00.000-07:00

If you are a Bugzilla user, you really should check this out. My friend Nick Goodman came up with a Pentaho solution that takes Bugzilla data, ETL's it into palatable OLAP format, and uses the rest of the Pentaho Open BI Suite to provide analytics and reporting - that you could NEVER get out of Bugzilla alone.

For example, how about historical based analysis, such as running total net open bugs by product, year over year, with the capability to drill into details? Or current issues, by status and product? These are the reports that product managers, project managers, development managers and software engineers need to manage the timeline, scope and releases for a software project.

Let Bugzilla do what it does best - track your bugs. Let Software Quality Reporting for Bugzilla unlock the information captured in your bug tracking. To try it out, go to the SourceForge project at http://sourceforge.net/projects/qareports/, and download the solution and the Getting Started Guide. It takes about 15 minutes to set up, and the demo is up and running. Another 30 minutes or so and you can have the solution hooked up to your Bugzilla data!

Get Your Open Source Here...

2006-09-14T09:30:00.000-07:00

Talk about a really great resource - a site that provides open source alternatives to the most popular commercial software packages. And very nicely put together I might add, which in my opinion DOES contribute to its net usefulness:)

The site is osalt.com, "Open Source as an Alternative" and can be found at http://www.osalt.com/. The premise behind the site is a listing of the most popular proprietary software matched to their open source alternative offerings. Just type in Cognos in the search engine, and osalt.com brings you Pentaho! Niiiccce:)

Of course, this site is useful in so many ways - I can now get the top listed open source alternatives for my photo editing software, my HTML wysiwyg editor, my mapping software, oh the list goes on.

Working in open source, I forget what it was like, being a newbie in the open source world, and trying to figure out what project does what. But I can bet that osalt.com makes it a bit nicer for open source novices to get a grip... which from all we hear, there are droves of new open source adoptees everyday.

By the way, I can't take all the credit for finding osalt.com - my co-worker Brian forwarded it to me, thanks a bunch Brian!

Dynamically Generating a Cross Tab Report, Part 2

2006-09-12T11:36:00.000-07:00

So, I thought it would take me a week or two to finish up this article from Nic Guzaldo, a strong supporter of the Pentaho platform. It ended up taking almost a month, which tells me I am long overdue for updating my calendar and planner.

With many apologies for the delay, I present to the Pentaho community the long-awaited second half to "Dynamically Generating a Cross Tab Report". In Part 2, we pick up where we left off and use the query that we derived in the first article to feed into a Pentaho report template. Actually, we will use several mini templates that demonstrate template re-use and assembling with parameters, very cool stuff. Our template will have several replaceable parameters that will accommodate our variable number of columns, a dynamically generated report title, and a nice name for any output files we generate, should the user choose to output their report to a format such as Excel, RTF, PDF, etc.

This exercise is a giant hand up for Pentaho users who really need to see it all come together - JNDI, dynamic queries, parameter passing, report templates, variable output formats, and much more! Great luck and please do send me some feedback! I'd love to know if these articles are helpful, and what topics you would like to see in future articles:)

all the best,
Gretch

Designing Reports in Pentaho: Save As versus Publish

2006-08-31T11:21:00.000-07:00

As part of my job at Pentaho, I occasionally get to help build solutions using the Pentaho BI platform and the solution building toolset, Design Studio, Report Wizard and Designer. I really enjoy that time, in part because I miss my developer days, but also because using the software we build makes me feel like I can connect and relate more to our community. The end product is I can do my job better, because I know the project and the software better.

I recently started creating some reports against our case tracking data, which lives in a JIRA database. JIRA is a phenomenal issue tracker, but I just can't get the reports and data analysis out of it that I really need. And that's OK, that's what we have the Pentaho platform for. One thing I came across that I thought would be good to explain a bit is the difference between "Save As" and "Publish" in our report designing tools.

Both the Report Designer and the Report Design Wizard have a "Save As" option and a "Publish" option. The "Save As" option is pretty straight forward, and "Publish" is a very powerful feature, if you know how to use it:) I have to warn you at this point that these options do produce slightly different results when you are in Report Design Wizard than when you are using Report Designer, and I will spell out what those differences are in a moment. If you are wondering WHY this is the case, read my previous blog post:)

Let's start by talking about what Save As and Publish do in Report Design Wizard.

If you choose the Save As option in the Report Design Wizard, you are saving the entire state of the wizard session you are in. The file will have a .xreportspec extension, which is the file extension that the Report Design Wizard expects when you select the File | Open option in the wizard. It may not seem intuitive at first to "save" the state of a wizard. However the wizard allows you to set so many options for your report, that you may want to go back to your wizard session and tweak a few report features. This makes less sense when you have the Report Designer, which gives you much more powerful report tweaking capabilities. The Save As option in the Report Design Wizard was built at a time when Pentaho didn't have a Report Designer, and so, as the two tools become more tightly integrated, I would hazard a guess that this Save As feature may go away.

The real power of the wizard is in its Publish feature. When you choose the Publish option from the Report Design Wizard, you get a number of files that amount to everything you need to run that report in the Pentaho BI platform. Here is a list of the files that would be generated with a Publish, and what they are for:

JFreeReport-WizardReport.xreportspec - the saved wizard session. This is the same file you would have if you chose the Save As option.

JFreeReport-WizardReport.xml - this is the JFreeReport report definition.

JFreeReport-WizardReport.xaction - this file is the action sequence built to run this report in the platform. If you are not familiar with action sequences, you can get a great overview from the Pentaho Creating Solutions Guide.

JFreeReport-WizardReport.properties - this is the resource file, used for holding the action sequence's extracted strings. This file facilitates internationalization of your action sequence.

PentahoReporting.png - a sample image, referenced in the action sequence, that can be used in to represent this report in your solution's UI.

The Report Designer's Save As and Publish functions serve the same purposes, but result in slightly different outputs.

When you select the Save As option in Report Designer, you are saving the designer session you are working in. The file will have a .report extension, which is the type of file that the Designer expects on opening of a file.

When you choose Publish with the Report Designer, there are fewer files than what the Report Designer produces, but the same end goal is achieved. Here are the files produced by the Report Designer and what they are for:

Report.xml - this is the JFreeReport report definition.

Report.xaction - this file is the action sequence built to run this report in the platform.

Why, you might ask, does the Report Designer generate less files, but accomplish the same task? There are few differences between the outputs that I should note:

First, the Report Design Wizard still gives you the wizard session (.xreportspec), even when you Publish, whereas the Report Designer does not give you the designer session (.report) on Publish.

The Report Design Wizard extracts strings from the action sequence file for translations into a .properties file, whereas the Report Designer does not.

The Report Design Wizard generates a sample icon for the action sequence where the Designer does not.

Another difference worth noting, that is not apparent by examining just the list of output files, is where the query for the report lives when either tool publishes. The Report Design Wizard will put the query in the action sequence file, which is the best practice. The Report Designer embeds the query in the JFreeReport definition.

These differences constitute minor inconsistencies in implementation that I'm sure will be addressed very soon. That is everything I had hoped to cover.

I hope that this helps a few people get up and running with Pentaho Reporting, because it really is quite an extraordinary set of tools, and just keeps getting better (not that I'm biased or anything :).

Pentaho Report Designer versus Pentaho Report Design Wizard

2006-08-28T19:53:00.000-07:00

The Pentaho Reporting client toolset can look a bit confusing at first glance. We have the Pentaho Report Designer, and then we have the Pentaho Report Design Wizard. Two products that perform seemingly very similar functions, but are separate tools. What's THAT all about??

Well, first, I'm here to tell you it won't be this way for very long:) And second, these two tools are powerful and unique, and compliment each other very well. Our vision for Reporting in the Pentaho BI Suite is well, sweet. With a little history, it's easier to understand how we got where we are, what should be used when, and where we are going with all this.

So how did we get here? Well, we really loved JFreeReport. JFreeReport provides a robust, full-featured reporting foundation for the Pentaho BI platform. However, at the time that we brought JFreeReport on board the Pentaho ship, the available open source report-building UIs for JFreeReport were lacking. So, that provoked our captain (Mr. Dick Daley) to hail the crew and shout "Go forth yee and build a Report Design Wizard!". Aargghhh. And thus the Report Design Wizard was born (thanks to the mighty efforts of one rather savvy shipmate, Mike D'Amour).

<Insert A.D.D. tangent here>
If at this point, you all are wondering about the sea-faring dialog, my husband often describes the thoughts in his head as a scene from Moulan Rouge. Mine are a bit more Pirates of the Carribean:)
</Insert A.D.D. tangent here>

At this point in the story, we are very excited about the Report Design Wizard, when we are introduced to a company out of Germany that has built a JFreeReport Report Designer. This tool is pretty spiffy and approaches report design a bit differently than the wizard, so Pentaho purchased the Report Designer, and donated it to the world as open source. And that is how we ended up with two tools, that initially look the same.

In reality, the only similarities that the Report Designer and the Report Design Wizard are their names, and their end goal, being to generate a JFreeReport to use in the Pentaho Open BI Suite. These tools actually bring unique sets of features together to provide a complimentary toolset, that will in the near future be much more tightly coupled. Let's take a look at each tool on it's own.

The Report Design Wizard was designed with to be just that - a wizard. A tool to get you from a dataset to a report in the shortest number of steps possible, easing the startup time for building JFreeReport reports. This makes the Report Design wizard inherently focused on the data. You start building your report by telling the wizard where your data is and what query you wish to perform. The next steps allow you to tweak numerous report features, as well as incorporate charts into your final product.

The Report Designer on the other hand, while it can be dataset-centric, provides more power in allowing you to define your report without specific data, as well as giving you the abillity to customize every feature and aspect of your report. The Report Designer supports the vast majority of JFreeReport features, and can render them through it's UI. It shields the user from the JFreeReport XML, almost completely.

My favorite way to use these tools is to start my report creation with the Report Design Wizard ( I can't think in terms as abstract as a report layout without data - too many years as a programmer, I guess), building up as many report features as I can. If I still need to tweak the report, I import it into the Report Designer and polish it there.

And where is all of this going? In the future, we envision the Report Design Wizard as a feature of the Report Designer, as well as our Pentaho Design Studio. Definitely an integration that makes sense, and will make for a nice experience building Pentaho solutions and reports. As the Pentaho Community leader, it's my job to know what's important to you, our community. I'd love to hear about your experiences using these tools, what you think of our plan, and what great ideas you have for the toolset. Please comment, or feel free to email me at communityconnection@pentaho.org :)

Blogging on Pentaho, a Favorite

2006-08-23T08:37:00.000-07:00

The Pentaho team spans several countries, and it's not always easy to keep up with the mountainous amount of activity among our projects and people. This has spurred a number of team members to begin blogging.

Here is a short list of Pentaho bloggers, some new, some not so new. These are definitely a few of the people who can tell you where Pentaho is and where we are going:

http://www.nicholasgoodman.com
Nick Goodman is Pentaho Director of Solutions, and just an a
all around knowledgable guy when it comes to open source AND business intelligence.

http://www.ibridge.be/
Matt Casters is the founder of Kettle, the Pentaho Data Integration piece and Chief Architect of Data Integration for Pentaho. In a past life, Matt was an independent BI consultant for many years and implemented numerous data warehouses and BI solutions for large companies.

http://julianhyde.blogspot.com
Julian Hyde is the founder of Mondrian, THE open source OLAP server, and the OLAP server component of Pentaho. Julian brings his expertise to Pentaho as Chief Architect of OLAP Technologies.

Dynamically Generating a Pentaho Cross Tab Report, Part 1

2006-08-17T17:51:00.000-07:00

We recently received a monster (in a great way!!) of a tech tip from Nic Guzaldo, a self-taught Pentaho expert and supporter. I spent a day or so getting my head around what Nic had accomplished (with many exclamations of "wow, I didn't know we could do THAT!"), and figured I should break it up into a series of articles for easier consumption.

This first article of the series focuses on creating a query that will not only demonstrate setting up regular data for a cross tab report, but also use user input to determine the number of columns in the query.

In the second half of this solution, we will continue with Nic's example and use the generated resultset as input to a dynamically generated report spec, to create a Pentaho report that allows a variable number of columns, dynamically generated report name, nice output names for external files and other goodies!

Start test driving this jam-packed solution at here.

Moving Pentaho Sample Data to MySQL

2006-08-11T13:45:00.000-07:00

MySQL has been one of the most popular databases amongst the Pentaho community. We receive questions and comments regularly about setting up and writing solutions for Pentaho on MySQL, which tells me people are putting MySQL to work in the business intelligence space. These questions were the catalyst for our Tech Tips, a collection of very useful how-to's for the Pentaho platform and Pentaho tools.

Today, I published a short tech tip regarding how to move the Pentaho sample data from HSQLDb to MySQL. Not rocket science, yet definitely made much easier with a Kettle transformation donated by Nic Guzaldo - thanks again Nic! Check it out, it's a good example of moving data with Kettle and also is a great utility tool to stick in your Pentaho toolbox. That one transformation can be tweaked to move the Pentaho sample data to just about any JDBC compliant database.

Nic's full contribution is actually a complete reporting solution that comprehensively covers a plethora of Pentaho features - secure filters, parameters, variable replacements, more MySQL 5.0 tricks just to name a few. In my next posts I'll talk more about the "Guzaldo mini-series", look for it!

As Good a Time As Any...

2006-07-20T07:03:00.000-07:00

So, it's time to participate in this blogging rage. I lead the community for open source business intelligence, the Pentaho Nation, and I am always looking for new ways to get "the message" out.

Of course, today is the day before I leave to tour the Nova Scotia area of Canada for ten days, so blogging on business intelligence may have to take the backseat to the review of my Canada experience. We are looking forward to whales and puffins, amazing state parks, lots of natural beauty and some well-earned downtime. Yes, I will be taking my laptop. A day in the life of a start-up engineer:)

When I get back, look forward to a series of experiments with the Pentaho BI Platform I intend to take on. Mostly involving better intelligence for the Pentaho community. Things like, how does the community help steer the Pentaho roadmap? How many steps from Kevin Bacon are some other open source communities? Where are the specific projects that the Pentaho community can run with?