Time series database for java?

Question

I want to store millions of time series entries (long time, double value) with Java. (Our monitoring system is currently storing every entry in a large mySQL table but performance is very bad.)

Are there time series databases implemented in java out there?

Answer 1

• checkout http://opentsdb.net/ as used by StumbleUpon?
• checkout http://square.github.com/cube/ as used by square

I hope to see additional suggestions in this thread.

Answer 2

The performance was bad because of wrong database design. I am using mysql and the table had this layout:

+-------------+--------------------------------------+------+-----+-------------------+-----------------------------+
| Field       | Type                                 | Null | Key | Default           | Extra                       |
+-------------+--------------------------------------+------+-----+-------------------+-----------------------------+
| fk_category | smallint(6)                          | NO   | PRI | NULL              |                             |
| method      | enum('min','max','avg','sum','none') | NO   | PRI | none              |                             |
| time        | timestamp                            | NO   | PRI | CURRENT_TIMESTAMP | on update CURRENT_TIMESTAMP |
| value       | float                                | NO   |     | NULL              |                             |
| accuracy    | tinyint(1)                           | NO   |     | 0                 |                             |
+-------------+--------------------------------------+------+-----+-------------------+-----------------------------+

My fault was an inapproriate index. After adding a multi column primary key all my queries are lightning fast:

+-------+------------+----------+--------------+-------------+-----------+-------------+----------+--------+------+------------+---------+---------------+
| Table | Non_unique | Key_name | Seq_in_index | Column_name | Collation | Cardinality | Sub_part | Packed | Null | Index_type | Comment | Index_comment |
+-------+------------+----------+--------------+-------------+-----------+-------------+----------+--------+------+------------+---------+---------------+
| job   |          0 | PRIMARY  |            1 | fk_category | A         |          18 |     NULL | NULL   |      | BTREE      |         |               |
| job   |          0 | PRIMARY  |            2 | method      | A         |          18 |     NULL | NULL   |      | BTREE      |         |               |
| job   |          0 | PRIMARY  |            3 | time        | A         |   452509710 |     NULL | NULL   |      | BTREE      |         |               |
+-------+------------+----------+--------------+-------------+-----------+-------------+----------+--------+------+------------+---------+---------------+

Thanks for all you answers!

Answer 3

You can take a look at KDB. It's primarily used by financial companies to fetch market time series data.

Answer 4

What do you need to do with the data and when?

If you are just saving the values for later, a plain text file might do nicely, and then later upload to a database.