Dictionary: hard-coded vs. external file

Question

I have a java application which is started and stopped multiple times per second over hundreds of millions of items (called from an external script).

Input: String key
Output: int value

The purpose of this application is to look for a certain key in a never ever ever changing Map (~30k keys) and to return the value. Very easy.

Question: what is more efficient when used multiple times per second:

1. hard-coded dictionary in a Map
2. Read an external file with a BufferedReader
3. ...amaze me with your other ideas

I know hard-coding is evil but sometimes, you need to be evil to be efficient :-)

Answer 1

Read in the dictionary from file. Store it in a Map. Set up your Java application as a service that runs continuously (since you said it gets called many times per second). Then your Map will be cached in RAM.

Answer 2

The fastest is a hard coded map in memory. if u a have a huge file you can use a Memory Mapped file :

MappedByteBuffer in = new FileInputStream("map.txt").getChannel().map(
FileChannel.MapMode.READ_ONLY, 0, LENGTH);
StringBuilder bs = new StringBuilder();
//read 1/4 of the file   
while (i < LENGTH/4)
      bs.append((char)in.get(i++));

This approach is a bit problematic though,in practice you will want to partition the file on line breaks i.e read until the 100th line clean the buffer and read some more.

Answer 3

I would load the file into a Map at startup of the application, and then use it as you describe.

I would store the data in a database for faster load times.

Definitely do not have the application startup and shutdown every time it is called; have it as a service that waits for IO, using Asynchronous I/O, such as netty