InfluxDB with Zabbix or Telefraf?

Question

I'm considering the use of InfluxDB to store thousands of measurements per second from routers in a network. Each measurement is lightweight (between 10-20 bytes). Now, I'm wondering what is the best strategy: InfluxDB + Zabbix? Or Telegraf? Or another alternative?

1. InfluxDB + Zabbix: according to

https://www.zabbix.com/integrations/influxdb

there is a possible integration. However, it seems the integration involves taking data that was collected in a Zabbix database and moving it to an InfluxDB to take advantage of InfuxDB using much less storage. So they meant it to be for archiving large amounts of data.

However, we believe Zabbix itself is too slow, no? The integration above would not work for online data collection, right? We need fast storage, and we do not want Zabbix to be a bottleneck. Are there known benchmarks and recommendations about Zabbix + InfluxDB integration, for online collection + storage?

2. InfluxDB + Telegraf: according to

   https://www.influxdata.com/blog/monitoring-openwrt-with-telegraf/

there is also the possibility of integrating InfluxDB + Telegraf for data collection. Are there best practices and/or benchmarks about data collection with Telegraf + storage with InfluxDB?

3. Are there other alternatives and/or suggestions for large scale data collection and integration with InfluxDB?

Answer 1

You might want to look into using Spark, and if are able to introduce AWS into the architecture, Kinesis Data Firehouse.

Spark: https://medium.com/analytics-vidhya/processing-time-series-data-in-real-time-with-influxdb-and-structured-streaming-d1864154cf8b

Another article on Spark: https://www.influxdata.com/blog/zeppelin-spark-and-influxdb-for-big-data-time-series-scenarios/

Kinesis Data Firehose: https://aws.amazon.com/kinesis/data-firehose/?kinesis-blogs.sort-by=item.additionalFields.createdDate&kinesis-blogs.sort-order=desc

Spark is fairly widely used these days and integrates with InfluxDB, and Kinesis Data Firehose is used for heterogeneously ingesting large amounts of data in real time.