(kdb+/Q language) Creating separate table files from CSV import Parser(edited)

Question

I have a (big) CSV file with some data. And I have a importer from code.kx.com using .Q.fsn

colnames:`Symbol`Date`Time`Sequence`Exchange`Type`Level`Condition`Price`Size`BuyerID`SellerID
.Q.fsn[{`:newCreatedFile upsert flip colnames!("SDTISSISFISS";",") 0:x};`:C:/myDir/data.csv 5000000]

This code creates a file named newCreatedFile with the data from data.csv, assuming a big data file, processed the data in chunks of 5000000 bytes.

Question:

I want to create 2 separate files from this data, and let's say the basis for the distinction is the values in the "Condition" column. For each row, if the value in Condition column is x,y,or z put in file A.csv, else B.csv.

Here is a pseudocode for the if statement:

$[Condition in `x`y`z; Afunction ; Bfunction]
Afunction:{`:newA upsert flip ...};
Bfunction:{`:newB upsert flip ...};

How would I have to set up the if statement? I currently have:

$[datatable.Condition = `SomeCondition; fileA;fileB]

But I get a type error. How do I match each value of a certain column?

Should it check while importing the original data file or after the data file has been created into a table in kdb?

Answer 1

You haven't parsed the column yet in your example when you check the condition, this needs to be done first. You could use something like the following:

foo:{t:flip colnames!("SDTISSISFISS";",") 0:x;
     `:newA upsert select from t where Condition = `SomeCondition;
     `:newB upsert select from t where not Condition = `SomeCondition}

.Q.fsn[foo;`:C:/myDir/data.csv;5000000]