Fetch a Unique Number from a file containing list of numbers concurrently

Question

I have a list of unique numbers in a file. Whenever user of my sites makes a request i need to fetch a value from that list , such that no other user gets the same value. There could be many concurrent requests, and hence the major issue is that of guaranteeing that no two users get the same value. I am not that concerned with performance in the sense that user doesn't expect any reply after making a request and so everything can happen in the background. From what i have read, this could be implemented using file locks or i could store data in the db and use db locks. I just want to get an ideas what is the best way of doing this thing. I am using postresql as my database and was wondering if this could be done using sequences where i store the current row number in the sequence so that my program knows which row to read from the db. But again i am not sure how to prevent multiple processes reading the same row before any one of them has a chance to update it.

Answer 1

Databases don't actually do many things, but what they do, they do very well. In particular, they handle concurrent access to data. You could use this to your advantage:

• Create a table: create table special_number (id serial, num varchar(100) not null);
• Ensure uniqueness: create unique index special_number_num on special_number(num);
• Load your numbers into the table using COPY, letting id count up from 1 automatically
• Create a sequence: create sequence num_seq;
• Use the fact that postgres's nextval function is guaranteed concurrent-safe to safely pick a number from the list: select num from special_number where id = (select nextval('num_seq'));. This will return null if you run out of numbers.

This approach is 100% safe and easy - postgres's internal strengths as a database does all the heavy lifting.

Here's the SQL extracted from above:

create table special_number (id serial, num varchar(100) not null);
create unique index special_number_num on special_number(num);
copy special_number(num) from '/tmp/data.txt'; -- 1 special number per file line
create sequence num_seq;
select num from special_number where id = (select nextval('num_seq'));

This SQL has been tested on postgres and works as expected.