Any specific rules for converting MySQL data to Prolog rules for exploratory mining?

Question

I have three tables (for now), one with 2000 rows and the other two with approx 1.6 million rows each. They have columns that relate them to each other but they are not formal FK fields. I have written a C++ program to produce rule files based on the source MySQL data like this:

if T{ C1, C2...Cn } is the table definition
then the rule would be:
    T(C1, C2, Cn).

My conversion utility leaves integers unwrapped and anything else is put inside single quotes so INT(n), DECIMAL etc become Prolog numbers ad everything else is an atom.

My question then is this: if I want to write a search rule for a table/rule that has 26 fields, is there a "meta-level" way to express this:

findStuffById(I,FieldIWant1,FieldIwant2etc) :-
    tablename(_,_,I,_,FieldIWant1,_,_,_FieldIWant2etc,_,_,....).

Or is it that I have to create "simpler" rules that will then have to use "_" or Variables to capture what I want?

The prospect of writing rules like...

findThisById(X)     :- tablename(X,_,_,_,_,This,_,_).
findThatById(X)     :- tablename(X,_,_,That,_,_,_,_).
findTheOtherById(X) :- tablename(X,_,_,_,_,_,_,TheOther).

...ad nauseum... is unsettling!

My only thoughts so far are that I might need to create compound terms within the rules to group things together i.e. to reduce the number of variables in the rules but that would maybe restrict the "freedom" of a future query? I am no Prolog expert; I've played with it for quite a few hours and and dying to find a real world use in my own day job for it.

I guess that my conversion program could also code generate the drudge rules as well so that I wouldn't have to hand code it. The big tables have 26 and 28 columns each so you can see where this is going!

Any advice on how to proceed would be welcome; I don't get to use Prolog as much as I would like and I am always wanting to learn some more!

I want to create a SWI-Prolog web server that is going to go head-to-head against an ElasticSearch server using the same raw data to see which is the fast to respond to ad-hoc queries. It seems ES doesn't do traditional joins either unless you pre-create a compound document with the indices embedded which also might be something in favor of using Prolog.

Thanks. Sean.

Answer 1

You could maybe use nth1/1 and the "univ" operator, doing something like this:

fieldnames(t, [id,this,that]).
get_field(Field, Tuple, Value) :- 
    Tuple =.. [Table|Fields],
    fieldnames(Table, Names),
    nth1(Idx, Names, Field),
    nth1(Idx, Fields, Value).

You'd need to create fieldnames/2 records for each table structure, and you'd have to pass the table structure along to this query. It wouldn't be terrifically efficient, but it would work.

?- get_field(this, t(testId, testThis, testThat), Value)
Value = testThis

You could then build your accessors on top of this pretty easily:

findThisById(X, This) :- get_field(this, X, This).

Edit: Boris points out rightly that arg/3 will do this with even less work:

get_field(Field, Tuple, Value) :-
    functor(Tuple, Table, _),
    fieldnames(Table, Names),
    nth1(Idx, Names, Field),
    arg(Idx, Tuple, Value).

Prolog is so awesome.

Answer 2

On an actual e-Commerce DB I used code like

update_price(File, Pid, Cn, Manu) :-
    product(Pid, [tax_class_id = Tid, /*cost = Cost,*/ price = CurrPrice]),
    tax_rate(_, [tax_class_id = Tid, rate = R]),
    manufacturer(Manu, name = NameM),
    (   ( NameM == 'Gruppo Aboca' ; NameM == 'L\'Amande' )
        ->  % Pr is Cost * 3 / 2
        togli_iva(R, Cn, Pr)
    ;   togli_iva(R, Cn, NoIva),
        Pr is NoIva * 2
    ),
    Delta is abs(CurrPrice - Pr),
    (   Delta > 0.01
    ->  Prx is round(Pr * 100) / 100,
        format(File, 'update product set price=~w where product_id=~d~@', [Prx, Pid, eol])
    ;   true
    ).

product, tax_rate, manufacturer are actual table names, with known structure. For instance, product has 26 columns, tax_rate has 8,....

I have declarations

:- dynamic 
    ...
    product/2,product/26,
    ...
    tax_rate/2,tax_rate/8,
    ...

Data is read from a SQL dump, and when asserted in memory I build the product/2 accessor, that take care of field translation from name to position.

I'm not suggesting to use this approach, because would be too much slow for your data. Instead you could use goal_expansion/2 and translate - at compile time - any call to table(field1=Value1, field2=Value2) to a corresponding call to table(,,Value1,_,Value2) - Prolog positional way.

This should squeeze best performances from SWI-Prolog indexing, recently updated to work on all columns.

Of course, to get more detail, you should post your metadata format...

edit: if you are interested in simple single fields accessors (as I understand from comments to Daniel answer), you could try library(record).