How does q cache data?

Question

I was trying to measure data access times for a db I had created. It was taking ~1s for one day's data. To aggregate I ran the following code. I am using kdb studio and there are ~1MM trades in total every day

\t ans: raze {select from trade where date=x, sym=`ABC} each 20#dtl

dtl was the entire list of dates. I closed the server and ran it again and surprisingly this took <1 second. As this was contrary to what I had observed above, I ran this

\t ans: raze {select from trade where date=x, sym=`ABC} each 20#20_dtl

and now it took ~21 seconds. My question is if i close the kdb server, is it possible for q to still cache some of the previous results?

Answer 1

KDB+ does not cache anything. It's all hardware caching if you see a difference in speed like this. You can confirm this if you flush the cache (in unix systems there are a set of commands to do this, but you'll need root access). Bottom line is that KDB+ doesn't do any caching at all. (unless you tell it to of course... a la .Q.fu)

As an aside, not sure how your query worked here - 20#dtl will give a list of dates, and date=x would then give a length error. I assume you meant "date in x". In which case you may get skewed results due to multithreading, if you're using -s on the command line.

Answer 2

This is likely due to your operating system caching the data it reads from disk. Kdb+ provides no inbuilt caching by default.