SSIS - How do I join two sources on a complex expression?

Question

I have one data source which has a column Part_Number.

My second data source has Product_Family_ID, Product_ID and Part_Mask.

Mask is a pattern string used in a like expression where part_number like mask. For instance 'NXA%' or '001-[abcd][456]9-121%'

Usually we look up legitimate part numbers for a product family based on the available product part masks, but in this instance I need to go in the other direction. Based on a part number, I must find all the related products in that product family and store that in a summary table.

Simulating this in T-SQL:

declare @partlist table (partnumber varchar(100))

insert into @partlist (partnumber) values ('nxampvg1')

select distinct pl.partnumber, match.Product_ID
from @partlist pl
join (  select m.masks, p.product_id from  MCS_ProductFamily_PartMasks m
    join Product p on m.ProductFamilyID = p.ProductFamily_ID) match
    on pl.partnumber like match.Masks

Desired output:

Part_Number Product_ID
----------- ----------
nxampvg1    15629
nxampvg1    15631
nxampvg1    15632
nxampvg1    15633
nxampvg1    15634
nxampvg1    15635
nxampvg1    15636
nxampvg1    15637
nxampvg1    15638
nxampvg1    15639

How can I accomplish this in an SSIS data flow task?

Answer 1

Create 2 data sources, one that gets all the part numbers from source one and another that gets all the product id's, product family's and masks using the inner query select m.masks, p.product_id from MCS_ProductFamily_PartMasks m join Product p on m.ProductFamilyID = p.ProductFamily_ID from source two. The data destinations for these should be staging tables on the same sql server. Then simply use a derivation of the SQL you stated in your example in such a way: select distinct pl.partnumber, match.Product_ID from STG_SOURCE_ONE pl join STG_SOURCE_TWO match on pl.partnumber like match.Masks