Selecting distinct values from key/value pairs in a seperate table

Question

I have 2 database tables

JOBS(JOB_ID, JOB_TIME, JOB_NAME,...), JOB_PARAMETERS(JOB_ID,NAME,VALUE)

where JOB_PARAMETERS is essentially a map containing job parameter key value pairs. Every job may have a unique parameter key/value pairs.

I am looking to pragmatically build a query that will return distinct job id's that contain key/value combinations. Where the values are actually a list of values, comparison operators. For example:

JOB_PARAMETERS: NAME = 'OUTPUT_FILENAME', VALUE LIKE "ALEX%", "JAX%"
                NAME = 'PRIORITY' , VALUE > 7

The above example would automatically filter out all jobs that don't have the OUTPUT_FILENAME and PRIORITY key. Returning All jobs that meet both conditions.

I also need to be able to support pagination and order by.

I was planning on using Perl with DBIx::Class, But I can do it in pure Perl/SQL as well.

I am open to changing the database schema, but every job can have different key/value pairs, so I cant just make them columns in the jobs table.

Thanks in advance.

Answer 1

When using DBIx::Class you can generate a DBIC schema by using Schema::Loader.

After connecting to the database you get a $schema object you can use to get a ResultSet filtered to return the Result objects you want:

my $rs_job_parameters = $schema->resultset('Job_Parameters')->search({
    -or => [
        {
            'name' => 'OUTPUT_FILENAME',
            'value' => [{ like => 'ALEX%'}, { like => 'JAX%' }].
        },
        {
            'name' => 'PRIORITY',
            'value' => [{ '>' => 7}].
        }
    ]},
    {
        columns  => [qw( job_id )],
        group_by => [qw( job_id )], # alternative you can use distinct => 1 to group_by all selected columns
        having   => \[ 'COUNT(*) = ?', [ 2 ] ],
    }
);

my @job_ids = $rs_job_parameters->get_column('job_id')->all;

Answer 2

One can do it in SQL, by grouping JOB_PARAMETERS by JOB_ID and filtering the groups accordingly. For example, if there is a uniqueness constraint over (JOB_ID, NAME), one can query as follows:

SELECT   JOB_ID
FROM     JOB_PARAMETERS
WHERE    (NAME='OUTPUT_FILENAME' AND (VALUE LIKE 'ALEX%' OR VALUE LIKE 'JAX%'))
      OR (NAME='PRIORITY' AND VALUE > 7)
GROUP BY JOB_ID
HAVING   COUNT(*) = 2

Absent such a uniqueness constraint, COUNT(*) would have to be replaced e.g. with COUNT(DISTINCT NAME).