Pentaho Kettle for Flow-Based Programming

Question

Is Pentaho Data Integration aka Kettle suitable for Flow-Based Programming FBP?

Kettle is an ETL(Extraction, Transformation and Loading) Tool and is based on FBP concepts

There is User Defined Java Class step and has a SDK so you can extend Kettle and you can integrate Kettle with Java Web Apps.

Is possible to build a web application using only Kettle with custom plugin?

Worth it write a custom plugin for web apps?

Which are the missing steps in Kettle for web development?

I should go for noflojs and leave Java?

JavaFBP is a java implementation of FBP. I can use JavaFBP to build a component that for example reads an Excel file. But this component already exist in Kettle!

My purpose is do some research and build a prototype not a real application.

Answer 1

I would say so yes. Kettle is being used in a lot of use cases these days which are not typical ETL.

Examples:

1. At the recent PCM event in Lisbon we saw the use of PDI to manage "builds" and deployments of solutions.

2. Sparkl - This is a biggie. Sparkl is a plugin builder framework for Pentaho which lets you use CDE to build the UI and Kettle to do the server based work. It's very clever and I recommend looking out for the video from the Pentaho Community Meetup. When you've built your app it can be zipped up as a pentaho platform plugin and distributed on the Marketplace.

3. Streaming - There are various cases now of kettle behaving more like an ESB where you have an endless transformation or job processing data as it arrives.

Also Kettle 5 has a whole bunch of new plugin technology, and "extension points" so kettle is starting to become a platform in itself, in which you can do any manner of data related activities!