Make Orion fetch data from Cosmos and publish

Question

I have set up a subscription between Orion ContextBroker and Cosmos BigData using Cygnus, and data is properly persisted in Cosmos when an update is made to Orion.

But I want to analyze the data in Cosmos and return the results to Orion, and finally access the result data in Orion from "outside".

How would one do this? Of course, I would like the solution I build to be as "automated" as possible, but mostly I just want to solve this problem.

Any advise is much appreciated!

Answer 1

As general response (as also the question is very general ;), what you need is a process that access to information stored in Cosmos (either using HDFS APIs -such as WebHDFS or HttpFs-, Hive queries, general MapReduce jobs on top of Hadoop, etc.), then implement the client side of the NGSI API that Orion implements in order to inject context elements into Orion based in the information you retrieved from Cosmos. The key operation to do so in the Orion API is updateContext.

The automation degree would depend on how you implement that process. It can be as automated as you want.

EDIT: considering this answer comments, I will try to add more detail.

What I mean is to develop a piece of software (let's call it APOS -A Piece Of Software) implementing the following behaviour:

1. APOS will grab data from Cosmos any of the interfaces provided by Cosmos, i.e. WebHDFS/HttpFs, Hive, MapReduce jobs, etc.
2. APOS will process the data to produce some result
3. APOS will inject that result in Orion, using the Orion REST API described in the Orion user manual. It is particularly useful for that task the updateContext operation. From a client-server point of view, Orion is a server exposing a REST API and APOS is the client interacting with that server.

It is completely up to you how to implement this APOS and how orchestrate the flow from 1 to 3 (e.g. it can run in batch mode all midnights, be triggered by user interaction on a web portal, etc.).

At the present moment, FI-WARE doesn't provide any generic enabler to convert from Cosmos data to NGSI given that each particular realization of the steps 1 to 3 above is different and depends on the use case. However, note that there is software component named Cygnus which implements the other way: from NGIS to Cosmos.