Integrating Classification Algorithms in OBIEE 11g?

Question

Currently, I've been involved in a Data warehouse based Banking Analytics project. We've been using Oracle as the database and Oracle Business Intelligence tool (OBIEE 11g) as the front-end or presentation layer for the user through weblogic server.

Here, I need to implement the Support Vector Machine (SVM) algorithm for the ATM card fraud pattern detection & CART for churn prediction and feed the result to the user through OBIEE. So my question is how can I integrate these algorithms (implemented in Python) to the OBIEE?

I've seen about Oracle Data Miner integration with OBIEE but it implements its own algorithm.

Answer 1

There is a new feature in 11g external tables called PREPROCESSOR (white papaer for 11gR2 can be found here) External table enables you read flat files directly into tables in oracle. the new preprocessos command enables you to specify a script name instead of a file and oracle will use the output stream as input. this way you can get algorithem results directly into oracle.

lets take you CART algorithm as an example

1. create a working directory in oracle

   create or replace directory dmdir as "/some_folder_on_your_os"

2. create a script that executes the algorithm and prints the output in a comma delimited format. lets say that for the CART algorithm you can output the tree as level,attribute,value,outcome,confidence.

3. create the external table

   CREATE TABLE cart_tree level number, attribute varchar2(2000), value varchar2(2000), outcome varchar2(2000), confidence number ) ORGANIZATION EXTERNAL ( TYPE ORACLE_LOADER DEFAULT DIRECTORY dmdir ACCESS PARAMETERS ( RECORDS DELIMITED BY NEWLINE FIELDS TERMINATED BY ',' MISSING FIELD VALUES ARE NULL PREPROCESSOR execdir:'my_cart_algorithem.py' FIELDS ( level, attribute, value, outcome, confidence

   )) LOCATION ('')) REJECT LIMIT UNLIMITED

   • notice that location is the parameter for the script. in this case non.

4. query

   select * from cart_tree

It's important to understand that every time you query the table oracle executes the script. so - what i usually do with external tables is create a materialized view on top of them.

A second option you might find useful is using the existing integration oracle has with R. both CART and SVM algorithm are available in R and has build in integration with oracle. you can read about it here here