Possible reasons for irregularly high query execution time?

Question

The above is a graph showing the average response times per second (including error bars) for an experiment where I simulate 30 clients which are all invoking the same query multiple times. I ran this experiment over and over and over again using also different queries but the result is always that around 20 seconds there is this one peak.

I'm not that familiar with postgres. The only thing that I thought of which could be responsible for this behavior is the vacuum daemon. However I enabled logs for the deamon and it turns out that it's not active at all in the period of the experiment. Is there anything else running at arbitrary points which could cause a high query time for a short period?

The function:

CREATE OR REPLACE FUNCTION insert_message(_queue_id BIGINT, _source_client_id BIGINT, _receiver_client_id BIGINT, _content TEXT, OUT status SMALLINT) AS $$
    BEGIN
                INSERT INTO message VALUES(DEFAULT, _queue_id, _source_client_id, _receiver_client_id, _content, DEFAULT);
                status = 0;
    END;
$$ LANGUAGE plpgsql;

The respective table:

CREATE TABLE message(
    id BIGINT DEFAULT nextval('message_id_seq'),
    queue_id BIGINT NOT NULL,
    sender_client_id BIGINT NOT NULL,
    receiver_client_id BIGINT NOT NULL,
    content TEXT NOT NULL,
    insertion_time TIMESTAMP NOT NULL DEFAULT current_timestamp,
    PRIMARY KEY(id),
    FOREIGN KEY(queue_id) REFERENCES queue(id) ON DELETE CASCADE,
    FOREIGN KEY(sender_client_id) REFERENCES client(id) ON DELETE CASCADE,
    FOREIGN KEY(receiver_client_id) REFERENCES client(id) ON DELETE CASCADE
);

Index on the table:

CREATE INDEX message_index ON message(queue_id, receiver_client_id, sender_client_id);

EDIT: Ran it on AWS. Same pattern. Thus it's no hardware thing.

Answer 1

That spike is probably caused by dirty data pages being flushed to disk. Raise the checkpoint_segments parameter in the postgresql.conf file.

http://www.postgresql.org/docs/current/static/runtime-config-wal.html#RUNTIME-CONFIG-WAL-CHECKPOINTS

http://www.postgresql.org/docs/current/static/wal-configuration.html