How to implement a data historian?

Question

A friend has asked if I could implement a data historian. I am busy doing research, googling around, reading UPC Unified Architecture - but it's a lot to get through, so I will ask if anyone here has ever gone down that road (while still continuing my research).

Approx how many man months for a 20+ year developer (or two) to get at least a demonstrable working prototype - and how long to completion?

Which programming language? Is C++ good, or what?

What resources are available to me? (I thought I saw an Open OPC framework, but can't find it again). Any FOSS, libraries or free code which I can base upon? Maybe a sourceForge project?

How best to test?

Any other hints?

Answer 1

It would be interesting if you can do a write up of your project when you complete it.

For OPC libraries your pretty limited, but OPC Connect has a good list of UA development kits otherwise you'll need to be a corporate member with the OPC Foundation.

Answer 2

In my company we use mostly to brand of Historian, PI developped by OSISoft and GE Profecy Historian. Ge Profecy now offers a 25 tag version of their latest Historian 4.5. The way it works is you got an historian server that collects from data collector pc's. Depending what piece of equipment you are communicating with you'll need different OPC driver.

Matrikon and Kepware are the 2 references in that field. At Matrikon you'll find almost anything related to OPC. We mostly use Kepware, cause we felt their solutions are more stable on the long run.

Depending of your knowledge of the PLC's you have in place and the number of point you want to acquire. It might take a day to a week implementing an historian. I'll be more than happy to help you if you provide us with more details.

Answer 3

This is an old topic, but I'm interested in the subject.

There is a Python library for OPC: openopc at SourceForge.net (I use a proprietary OPC client because it is provided by my automation supplier, Yokogawa.)

For short-term data grabs you can use a delimited text file, but for a historian you ought to use a database. I use SQLite for speed, size, and portability. Other DB solutions have advantages. Of course if you collect 400 points every second then over time your DB grows quite quickly, so efficient data storage is important.

Language used is influenced by your choice of OPC package. OpenOPC for Python is, well yes, for Python. I've used Graybox's free OPC client with .Net. The OCX I use at work is easiest to use with VB6. Not sure about others.

The time required to build a historian depends entirely on how complete the application needs to be. You can probably put together a data grabber in a few hours. A long-term historian with interfaces to view data, to add and delete points, to maintain data integrity, to handle bad data and interrupted communications gracefully -- all that will take days, not hours.