ruby: Extracting fields from nested json

Question

I am trying to teach myself ruby and solve a problem at work. My ultimate goal is to extract out three of the many fields in JSON response from an API, manipulate and dump to CSV for executive reporting.

The structure of the JSON is:

{
  "status": 200,
  "data": {
    "total": 251,
    "alerts": [
      {
        "dataPoint": "x",
        "ackedBy": "x",
        "dataSourceInstance": "x",
        "dataSource": "x",
        "host": "x",
        "endOn": 0,
        "ackedOn": 1385085190,
        "dataSourceInstanceId": 588384,
        "hostId": 935,
        "type": "alert",
        "dataSourceId": 694,
        "ackedOnLocal": "2013-11-21 17:53:10 PST",
        "id": 6170384,
        "startOn": 1385084917,
        "thresholds": "!= 1 1 1",
        "endOnLocal": "",
        "level": "error",
        "ackComment": "x",
        "value": "No Data",
        "hostDataSourceId": 211986,
        "acked": true,
        "hostGroups": [{
          "alertEnable": true,
          "createdOn": 1362084592,
          "id": 21,
          "parentId": 78,
          "description": "",
          "appliesTo": "",
          "name": "2600hz",
          "fullPath": "x"
        }],
        "startOnLocal": "2013-11-21 17:48:37 PST"
      },

To be specific I want to extract out dataPoint,startOn,ackedOn.

I am thinking I need to extract the total value first, so I know how many alerts I have. That will help me loop through the alert instances.

*url = "https://xyz"
uri = URI(url)
http = Net::HTTP.new(uri.host, 443)
http.use_ssl = true
http.verify_mode = OpenSSL::SSL::VERIFY_NONE
req = Net::HTTP::Get.new(uri.request_uri)
response = http.request(req)

# Make sure we had a proper web response
if response.code == '200' then

# Set up that the total is two levels deep in the json
path = JsonPath.new('$.total')

# Extract out total field into variable
totalAlerts = path.on(response)

# Print hash to confirm value
pp totalAlerts

end*

I am stuck trying to just extract out the total. Output shows nothing:

sudo ruby alerts.rb
[]

Is there a better way to do this?

Answer 1

Try this:

# just for demonstration, you would probably do json = JSON.parse(response)
json = {
  "status": 200,
  "data": {
    "total": 251,
    "alerts": [
      {
        "dataPoint": "x",
        "ackedBy": "x",
        ...

For just the total:

total = json['data']['total']

For the other values you asked about:

json['data']['alerts'].each do |alerts|
  # do whatever you want with these...
  alerts['dataPoint']
  alerts['startOn']
  alerts['ackedOn']

Now the question is, what do you want to do with the results? Do you want to collect them into a new hash? The json['data']['alerts'] is an array so you have to decide how you want to collect them. You could do:

collected_alerts = { 'dataPoints' => [], 'startOns' => [], 'ackedOns' => [] }
json['data']['alerts'].each do |alerts|
  collected_alerts['dataPoints'] << alerts['dataPoint']
  collected_alerts['startOns'] << alerts['startOn']
  collected_alerts['ackedOns'] << alerts['ackedOn']
end

Now you can get all those values in collected_alerts

Answer 2

In my case, I had to convert a Logstash event into a JSON array to get a value from one of its objects, so what did it for me was writing these lines on a Ruby filter:

sources_arr = event.get('[message][body][sources]')
sources_json = JSON.parse(sources_arr.to_json)
event.set('my_source_uri', sources_json[0]['uri'])

Notice the JSON.parse(<array>.to_json)