Read in the dictionary from file. Store it in a Map
. Set up your Java application as a service that runs continuously (since you said it gets called many times per second). Then your Map
will be cached in RAM.
Dictionary: hard-coded vs. external file
-
14-04-2022 - |
Frage
I have a java application which is started and stopped multiple times per second over hundreds of millions of items (called from an external script).
Input: String key
Output: int value
The purpose of this application is to look for a certain key in a never ever ever changing Map
(~30k keys) and to return the value. Very easy.
Question: what is more efficient when used multiple times per second:
- hard-coded dictionary in a
Map
- Read an external file with a
BufferedReader
- ...amaze me with your other ideas
I know hard-coding is evil but sometimes, you need to be evil to be efficient :-)
Lösung
Andere Tipps
The fastest is a hard coded map in memory. if u a have a huge file you can use a Memory Mapped file :
MappedByteBuffer in = new FileInputStream("map.txt").getChannel().map(
FileChannel.MapMode.READ_ONLY, 0, LENGTH);
StringBuilder bs = new StringBuilder();
//read 1/4 of the file
while (i < LENGTH/4)
bs.append((char)in.get(i++));
This approach is a bit problematic though,in practice you will want to partition the file on line breaks i.e read until the 100th line clean the buffer and read some more.
I would load the file into a Map
at startup of the application, and then use it as you describe.
I would store the data in a database for faster load times.
Definitely do not have the application startup and shutdown every time it is called; have it as a service that waits for IO, using Asynchronous I/O, such as netty