Preventing an action from being called multiple times by different servers

Question

I have a small table (<100 rows) which containts an ordered list of items. I want to randomly permute those items. The way I want to do this is select the 5 least recently used items and pick one of those 5 at random. However, I only want to do this once in a while.

I was thinking of doing this using a stored procedure, and then the query simply becomes something like SELECT TOP 1 * FROM myTable ORDER BY LastUsedDate DESC.

Unfortunately, this solution isn't great. If the time between each permutation (each time I run the stored proc) is variable, a SQL-Server job that runs every X minutes will not work. If I let my servers perform the permutation, multiple servers might end up doing the permutation.

This is the logic I was thinking of doing on the servers:

1. Get the most recently used item from DB
2. If it's time to perform a permutation, try getting a lock on the table
3. If it's already locked, it means that another server is already performing the permutation (goto 1)
4. Perform the permutation
5. Return the most recently used item.

However, I can imagine that locking the table is not such a great solution. So I'm looking for suggestions :).

I'm using Java on the servers with Hibernate.

Thanks!

Update:

I ended up trying to lock the rows using hibernate instead of having a stored proc (easier to debug, easier to push). However, I don't think hibernate is properly locking the necassary rows. Here's the code I have:

Session s = sessionFactory.openSession();
Transaction tx = null;
try {
    tx = s.beginTransaction();

    //Check whether the most recent tournament is expired or not. If it's not, abort (another server already updated it)
    TournamentTemplateRecord lastActiveTournament = (TournamentTemplateRecord) s.createCriteria(TournamentTemplateRecord.class)
    .addOrder(Order.desc("lastUse"))
    .setMaxResults(1)
    .uniqueResult();

    long startTime = lastActiveTournament.getLastUse().getTime();
    long tournamentDurationMillis =  lastActiveTournament.getDurationInSec() * 1000;
    if ((startTime + tournamentDurationMillis) < System.currentTimeMillis()){
        //Tournament is still active and valid. Abort.
        System.out.println("Tournament is still active");
        tx.rollback();
        return;
    }

    // Fetch the 5 least recently used tournaments
    List<TournamentTemplateRecord> leastRecentlyUsedTournaments = s.createCriteria(TournamentTemplateRecord.class)
    .addOrder(Order.asc("lastUse"))
    .setMaxResults(5)
    .setLockMode(LockMode.PESSIMISTIC_WRITE)
    .setTimeout(0) //If rows are locked, another server is probably already doing this.
    .list();

    Random rand = new Random();

    // Pick one at random
    TournamentTemplateRecord randomTournament = leastRecentlyUsedTournaments.get(rand.nextInt(leastRecentlyUsedTournaments.size()));

    randomTournament.setLastUse(new Date());

    s.update(randomTournament);

    tx.commit();
} catch (Exception e) {
    if(tx != null) {
        tx.rollback();
    }
} finally {
    s.close();
}

However Hibernate is not generating a SELECT ... FOR UPDATE NOWAIT. Any ideas?

Here's the generated HQL:

Hibernate: 
    WITH query AS (select
        ROW_NUMBER() OVER (
    order by
        this_.lastuse desc) as __hibernate_row_nr__,
        this_.combattemplateid as id89_0_,
        this_1_.combattypeid as combatty2_89_0_,
        this_1_.combattargetid as combatta3_89_0_,
        this_1_.resourcenameid as resource4_89_0_,
        this_1_.resourcedescriptionid as resource5_89_0_,
        this_1_.rewardloottemplateid as rewardlo6_89_0_,
        this_1_.combatcontainertypeid as combatco7_89_0_,
        this_.requirementtemplateid as requirem2_90_0_,
        this_.assetid as assetid90_0_,
        this_.durationinsec as duration4_90_0_,
        this_.lastuse as lastuse90_0_ 
    from
        tournament_tournamenttemplate this_ 
    inner join
        readyforcombat_combattemplate this_1_ 
            on this_.combattemplateid=this_1_.id ) SELECT
            * 
    FROM
        query 
    WHERE
        __hibernate_row_nr__ BETWEEN ? AND ?
Hibernate: 
    WITH query AS (select
        ROW_NUMBER() OVER (
    order by
        this_.lastuse asc) as __hibernate_row_nr__,
        this_.combattemplateid as id89_0_,
        this_1_.combattypeid as combatty2_89_0_,
        this_1_.combattargetid as combatta3_89_0_,
        this_1_.resourcenameid as resource4_89_0_,
        this_1_.resourcedescriptionid as resource5_89_0_,
        this_1_.rewardloottemplateid as rewardlo6_89_0_,
        this_1_.combatcontainertypeid as combatco7_89_0_,
        this_.requirementtemplateid as requirem2_90_0_,
        this_.assetid as assetid90_0_,
        this_.durationinsec as duration4_90_0_,
        this_.lastuse as lastuse90_0_ 
    from
        tournament_tournamenttemplate this_ 
    inner join
        readyforcombat_combattemplate this_1_ with (updlock, rowlock) 
            on this_.combattemplateid=this_1_.id ) SELECT
            * 
    FROM
        query 
    WHERE
        __hibernate_row_nr__ BETWEEN ? AND ?
Hibernate: 
    update
        Tournament_TournamentTemplate 
    set
        RequirementTemplateId=?,
        AssetId=?,
        DurationInSec=?,
        LastUse=? 
    where
        combatTemplateId=?

Answer 1

Locking the table will work just fine and if every operation needs to go through these steps will have no cost (as you need to lock on something anyway).

Really you need to revisit your requirements and architecture though, this has the smell of something that's going to get horribly messy very fast. Just have something responsible for the randomizing, run it on your server, have it randomize when required. Keep things simple.

Answer 2

This was an interesting puzzle. Since I don't know how once in a while is determined, it is a parameter to the stored procedure. The decision logic should implemented in the database, though. That would be far simpler than trying to coordinate the decision across multiple servers. If you can be more specific, I can improve my answer.

The idea is to store the five least recently used rows in a table variable with a computed ordinal. Compute @pick as a random number between 0 and 4 and update the row in myTable whose computed ordinal was picked.

CREATE PROCEDURE [dbo].[GetPermutedMostRecentlyUsed]
AS
BEGIN

    DECLARE @isTimeToPermute BIT
    SELECT TOP 1 @isTimeToPermute=CASE WHEN Expiry<GETDATE() THEN 1 ELSE 0 END FROM MyTable ORDER BY LastUsedDate DESC

    IF @isTimeToPermute = 1
    BEGIN
        BEGIN TRAN
            SELECT @isTimeToPermute=CASE WHEN Expiry<GETDATE() THEN 1 ELSE 0 END FROM MyTable WITH (TABLOCKX) ORDER BY LastUsedDate DESC
            IF @isTimeToPermute = 1
            BEGIN
                DECLARE @P TABLE
                (
                    ID INT PRIMARY KEY NOT NULL,
                    Ordinal INT NOT NULL
                )

                INSERT @P (ID, Ordinal)
                SELECT TOP 5 ID, ROW_NUMBER() OVER(ORDER BY LastUsedDate ASC) - 1 AS Ordinal
                FROM MyTable WITH (TABLOCKX)

                DECLARE @Pick INT; SET @Pick = FLOOR(RAND() * 5)

                UPDATE MyTable SET LastUsedDate=GETDATE(), Expiry=DATEADD(SECOND, 300, GETDATE())
                FROM MyTable AS T
                INNER JOIN @P AS P ON P.ID=T.ID AND P.Ordinal=@Pick
            END
        COMMIT TRAN
    END

    SELECT TOP 1 * FROM MyTable ORDER BY LastUsedDate DESC

END
GO

Just call it without any arguments.

EXEC GetPermutedMostRecentlyUsed

You will have to adjust column names and so forth to match your table schema.

Edited to compute @isTimeToPermute internally and perform table locking. A table lock is taken only when it is time to permute the records.