Python (PyRRD) and RRD with while loop?

Question

SOLVED, ADDED SOLUTION FURTHER DOWN I've been trying to create a graph with PyRRD from the output of a thermal sensor I have connected to my Raspberry Pi, but with no luck getting the actual data to show up in the graph (but the png-file is created). I'm not sure if this is the right way to do it, but this code spits out the temperature every second, so the while loop works at least.

import os
import glob
import time
import subprocess

# RDD-imports
from pyrrd.graph import DEF, CDEF, VDEF
from pyrrd.graph import LINE, AREA, GPRINT
from pyrrd.graph import ColorAttributes, Graph
from pyrrd.rrd import DataSource, RRA, RRD

# Sensor-stuff 
os.system('modprobe w1-gpio')
os.system('modprobe w1-therm')

base_dir = '/sys/bus/w1/devices/'
device_folder = glob.glob(base_dir + '28*')[0]
device_file = device_folder + '/w1_slave'

# RRD-stuff
startTime = int(time.time())
filename = 'temptest.rrd'
dataSources = []
rras = []
dataSource = DataSource(dsName='temp', dsType='DERIVE', heartbeat=5)
dataSources.append(dataSource)
rra1 = RRA(cf='AVERAGE', xff=0.5, steps=1, rows=5)
rra2 = RRA(cf='AVERAGE', xff=0.5, steps=6, rows=10)
rras.extend([rra1, rra2])
myRRD = RRD(filename, ds=dataSources, rra=rras, start=startTime)
myRRD.create()

# Graph-making
graphfile = 'tempgraf.png'
def1 = DEF(rrdfile=myRRD.filename, vname='mytemp', dsName=dataSource.name)
# Data going into green field
cdef1 = CDEF(vname='temp', rpn='%s,3600,*' % def1.vname)
# Line for max value
line1 = LINE(value=30, color='#990000', legend='Max temp allowed')
# Green area
area1 = AREA(defObj=cdef1, color='#006600', legend='Temp')

def read_temp_raw():
    catdata = subprocess.Popen(['cat',device_file], stdout=subprocess.PIPE, stderr=subprocess.PIPE)
    out,err = catdata.communicate()
    out_decode = out.decode('utf-8')
    lines = out_decode.split('\n')
    return lines

def read_temp():
    lines = read_temp_raw()
    while lines[0].strip()[-3:] != 'YES':
        time.sleep(0.2)
        lines = read_temp_raw()
    equals_pos = lines[1].find('t=')
    if equals_pos != -1:
        temp_string = lines[1][equals_pos+2:]
        temp_c = float(temp_string) / 1000.0
        myRRD.bufferValue(int(time.time()), int(temp_c))
        myRRD.update()
        return temp_c

while True:
    print(read_temp())
    g = Graph(graphfile, start=startTime, end=int(time.time()), vertical_label='Temp(c)')
    g.data.extend([def1, cdef1, line1, area1])
    g.write()
    time.sleep(1)

I have been experimenting around, reading ALOT in the RRD manual and tutorial for beginners but i just can't get this right. I'm very unsure about the rpn-stuff in the #Graph-making part. Please help me :) Also if there is a better way to do this, please tell me!

SOLUTION (to my problem): Dropped PyRRD and tried out rrdtools own python implementation. http://oss.oetiker.ch/rrdtool/prog/rrdpython.en.html

I created the database outside of the program and set the Step correctly like this in terminal (Linux):

rrdtool create dailyTemp.rrd --step 5      \
DS:temp:GAUGE:10:-100:200                  \
RRA:AVERAGE:0.5:1:2880 RRA:MAX:0.9:1:2880  \

I then dropped all code connected to PyRRD and just added some import lines and ONE row for rrdtool update. MUCH cleaner and now i can create my graphs :D Here is the "final" code:

import os
import glob
import time
import subprocess
import sys
sys.path.append('/usr/local/lib/python2.7/site-packages/')
import rrdtool, tempfile

# Sensor-stuff 
os.system('modprobe w1-gpio')
os.system('modprobe w1-therm')

base_dir = '/sys/bus/w1/devices/'
device_folder = glob.glob(base_dir + '28*')[0]
device_file = device_folder + '/w1_slave'

# RRD-stuff, not specific
startTime = int(time.time())

def read_temp_raw():
    catdata = subprocess.Popen(['cat',device_file], stdout=subprocess.PIPE, stderr=subprocess.PIPE)
    out,err = catdata.communicate()
    out_decode = out.decode('utf-8')
    lines = out_decode.split('\n')
    return lines

def read_temp():
    lines = read_temp_raw()
    while lines[0].strip()[-3:] != 'YES':
        time.sleep(0.2)
        lines = read_temp_raw()
    equals_pos = lines[1].find('t=')
    if equals_pos != -1:
        temp_string = lines[1][equals_pos+2:]
        temp_c = float(temp_string) / 1000.0
        print(int(temp_c))
        print(int(time.time()))
        rrdtool.update('dailyTemp.rrd','N:' + `temp_c`)
        return temp_c

while True:
    print(read_temp())
    time.sleep(5)

I have not implemented the graph creating into the code yet but can print it out the readings for the last 2 hours with:

rrdtool graph temp120.png --end now --start end-7200s --width 400   \
    DEF:ds0a=dailyTemp.rrd:temp:AVERAGE             \

Graph result (work in progress): Adding picture as soon as i got the needed reputation (10)

Answer 1

it looks like you are getting confused with rrdtool. I would advice you to use rrdtool first to generate the graph manually. Once you have understood the functioning you can move over to pyrrd.

These urls contain pretty good documentation on rrd

http://oss.oetiker.ch/rrdtool/doc/index.en.html

If you already have rrd generated use the following url and try to generate the graph manually.

http://oss.oetiker.ch/rrdtool/doc/rrdgraph.en.html

Since you mentioned empty png file is getting generated please make sure data is getting updated to rrd properly. Use rrdtool fetch to figure that out whether any real data got updated in rrd.

http://oss.oetiker.ch/rrdtool/doc/rrdfetch.en.html

Answer 2

Here's another way to grab the temperature values and then update your RRD ...

#!/usr/bin/python

# reads 1-wire DS18B20 temperature sensors and outputs options for rrdupdate

import os
import glob
import rrdtool

sensors = ( {'aab8': {                 # last four characters of sensor ID
                      'DS': 'DSName1', # DS name in rrd
                      'value': 0
                     },
             '5cc3': {
                      'DS': 'DSName2',
                      'value': 0
                     },
             '9ce0': {
                      'DS': 'DSName3',
                      'value': 0
                     }
          } )

RRD='/path/to/my.rrd'

base_dir = '/sys/bus/w1/devices/'
device_folder = glob.glob(base_dir + '28*')

DS_list = ""
value_list = "N:"

for index, sensor in enumerate(device_folder):
    for k, v in sensors.items():
        if sensor[-4:] == k:
            f = open(sensor + '/w1_slave', 'r')
            lines = f.readlines()
            f.close()
            if lines[0].strip()[-3:] == 'YES':
                v['value'] = float(lines[1][lines[1].find('t=')+2:])/1000.0
                DS_list += v['DS'] + ':'
                value_list += str(v['value']) + ':'

DS_list = DS_list[:-1]
value_list = value_list[:-1]

rrdtool.update(RRD, '--template', DS_list, value_list)

Answer 3

I have hanged on "step" setup when creating rrd file like you.

It have to be setup when creating RRD file:

myRRD = RRD(filename, step=5, ds=dataSources, rra=roundRobinArchives, start=datetime.fromtimestamp(time.time()), step=60)
myRRD.create()

not in the datasource like I first thought...

I've found the solution in the file example5.py provided with PyRRD