how is Orion context data persisted in Cosmos?

Question

I've heard about a connector persisting Orion context data in Cosmos in order to create a historical view of such data.

What's the name of such connector and where can I find documentation about it? What's the format of the persisted data? Where is the data stored within Cosmos?

Answer 1

The connector you are asking for is Cygnus, a Flume-based software able to receive notifications from Orion and persist them in Cosmos.

You can read a complete description and installation, configuration and usage guides within the README.

The persisted data is stored as text files in the configured Cosmos user space, i.e. a directory in HDFS such as /user/myuser/mydataset. For each (entity,attribute) pair received by Cygnus, a line is persisted in a text file. In Cygnus 0.1 this line is written in CSV style ('|' separator):

ts|iso8601date|entityId|entityType|attributeName|attributeType|value

in a text file called (a file per (entity,attribute) pair):

/user/myuser/mydataset/entityId-entityType-attributeName-atributeType.txt

Nevertheless, from release 0.2 (inclusive) the lines are written in Json format:

{"ts"="xxx", "iso8601date"="xxx", "entityId"="xxx", "entityType"="xxx", "attributeName"="xxx", "attributeType"="xxx", "value"="xxx"|{...}|[...]}

in a text file called (a file per entity):

/user/myuser/mydataset/cygnus-myuser-mydataset-entityId-entityType.txt