Convenient way to extract data from the MtGox/PubNub JSON API?

Question

I'm using the PubNub API with Java for pulling data from MtGox.

When retrieving data, the API delivers it in form of a JSONObject, which represents a tree structure of JSON data. Trying to extract bits of information directly from a JSONObject produces ugly code with lots of calls to getJSONObject(String), for which again exceptions might need to be handled.

Therefor, I'm looking for a convenient way to extract information from the JSONObject responses. So far, I've come across the possibility to convert the JSONObject into a POJO and then access the POJO. For conversion, I found the ObjectMapper from the Jackson library, which does a nice job here:

public void successCallback(String channel, Object message) {
    JSONObject messageJson = (JSONObject) message;

    ObjectMapper mapper = new ObjectMapper();
    Message myMessage = mapper.readValue(messageJson.toString(), Message.class);

    // do stuff with myMessage here
}

This approach has the disadvantage that I have to write my own POJO classes, e.g. the Message class in the above example, because I could not find these classes ready to use anywhere.

How to conveniently access the information stored in the JSONObject?

Answer 1

PubNub Java Class for MtGox JSON API

It's easy to create a ready made Java Class for ingesting the live feed provided by Mt.Gox This is a work-in-progress post to show you how to access the PubNub Data Feed from Mt.Gox as shown in the Dev Console live feed!

Official Bitcoin Wiki JSON Streaming API

We will be working from the Bitcoin wiki feed instructions provided by Bitcoin official Wiki: https://en.bitcoin.it/wiki/MtGox/API/Pubnub - continue reading below the screenshot to continue.

To see the live real-time data feed we will be using, please checkout the following two links:

1. Live Feed Trade Events (Buy/Sell Feed): https://www.pubnub.com/console?sub=sub-c-50d56e1e-2fd9-11e3-a041-02ee2ddab7fe&pub=demo&channel=dbf1dee9-4f2e-4a08-8cb7-748919a71b21&origin=pubsub.pubnub.com&ssl=true
2. Live Feed Ticker Updates (Price Changes): https://www.pubnub.com/console?sub=sub-c-50d56e1e-2fd9-11e3-a041-02ee2ddab7fe&pub=demo&channel=d5f06780-30a8-4a48-a2f8-7ed181b4a13f&origin=pubsub.pubnub.com&ssl=true
3. Trade Lag Example: https://www.mtgox.com/lag.html

PubNub Java SDK Docs

We will be using the PubNub Java SDK Docs http://www.pubnub.com/docs/java/javase/overview/data-push.html

Specifically we'll be using the mtgox.subcribe(...) instance method to focus our efforts which looks like the following:

Download JAR or Source: https://github.com/pubnub/mtgox

import org.json.JSONObject;
import com.pubnub.mtgox.MtGox;
import com.pubnub.mtgox.MtGoxCallback;

public class PubnubMtGoxSample {

    public static void main(String[] args) {
        MtGox mtgx = new MtGox();

        mtgx.subscribe("ticker.BTCUSD", new MtGoxCallback(){

            @Override
            public void callback(JSONObject data) {
                try {
                    String channel_name = data.getString("channel_name");
                    String avg_value = data.getJSONObject("ticker").getJSONObject("avg").getString("value");
                    System.out.println(channel_name + " : " + avg_value);
                } catch (Exception e) {}

            }});
    }
}

See Full MtGox Example with Java Source Code - https://github.com/pubnub/mtgox/blob/master/java/examples/PubnubMtGoxSample.java

To compile the example got to https://github.com/pubnub/mtgox/tree/master/java and run

javac -cp Pubnub-MtGox.jar:libs/json-20090211.jar   examples/PubnubMtGoxSample.java

And then to RUN:

java -cp .:examples/:Pubnub-MtGox.jar:Pubnub-StandardEdition-3.5.6.jar:libs/json-20090211.jar:libs/bcprov-jdk15on-1.47.jar:libs/slf4j-api-1.7.5.jar:libs/slf4j-nop-1.7.5.jar PubnubMtGoxSample

Answer 2

The Concept

For me, the best solution was to convert the JSONObjects from the PubNub API to bean classes which I found in the MtGox module of the XChange library.

Admitted, this approach adds quite a bit of glue code as you can see at the end of this answer, but I think it's worth the trouble, because after the conversion, the code gets much simpler. E.g. for getting the rate and currency from the ticker at which BTC was last traded, you can simply write

ticker.getLast().getValue()

and

ticker.getLast().getCurrency()

How To Do It

The mtgox module of the XChange library is available as a maven artifact, which is very convenient. You only need to add this module as a dependency to your project and the project setup is done.

In the xchange-mtgox module, you will find the package com.xeiam.xchange.mtgox.v2.dto.marketdata with the two calsses MtGoxTrade and MtGoxTicker.

Converting from JSONObject to one of these classes is easy with the Jackson ObjectMapper. As an advantage, the Jackson library is automatically imported as a transitive dependency of the xchange-mtgox maven artifact. That means that if you're using maven, you don't even have to write a single line of code to add it to your project.

Below is a complete runnable Example. Most is standard code for using PubNub. The important bits are between the marks // BEGIN OF IMPORTANT PART and // END OF IMPORTANT PART.

public class PubNubMtGoxBeanExample {

    private static final String MTGOXPUBNUB_SUBSCRIBE_KEY = "sub-c-50d56e1e-2fd9-11e3-a041-02ee2ddab7fe";
    private static final String MTGOXPUBNUB_BTCEUR_CHANNEL = "0bb6da8b-f6c6-4ecf-8f0d-a544ad948c15";

    private static final ObjectMapper OBJECT_MAPPER = new ObjectMapper();
    static {
        OBJECT_MAPPER.configure(DeserializationFeature.FAIL_ON_UNKNOWN_PROPERTIES, false);
    }

    public static void main(String[] args) throws PubnubException {
        Pubnub pubnub = new Pubnub("demo", MTGOXPUBNUB_SUBSCRIBE_KEY);

        pubnub.subscribe(MTGOXPUBNUB_BTCEUR_CHANNEL, new Callback() {

            @Override
            public void successCallback(String channel, Object message) {
                // BEGIN OF IMPORTANT PART

                JSONObject messageJson = (JSONObject) message;

                JSONObject tickerJson;
                try {
                    tickerJson = messageJson.getJSONObject("ticker");
                } catch (JSONException e) {
                    throw new RuntimeException(e);
                }

                MtGoxTicker ticker;
                try {
                    // the following line is the most important, because it maps from the JSONObject to the MtGoxTicker class
                    ticker = OBJECT_MAPPER.readValue(tickerJson.toString(), MtGoxTicker.class);
                } catch (JsonParseException e) {
                    throw new RuntimeException(e);
                } catch (JsonMappingException e) {
                    throw new RuntimeException(e);
                } catch (IOException e) {
                    throw new RuntimeException(e);
                }

                String currency = ticker.getLast().getCurrency();
                BigDecimal value = ticker.getLast().getValue();
                System.out.println(currency + " " + value);

                // END OF IMPORTANT PART
            }

            @Override
            public void connectCallback(String channel, Object message) {
                System.out.println("connectCallback on channel:" + channel + " : " + message.getClass() + " : " + message.toString());
            }

            @Override
            public void disconnectCallback(String channel, Object message) {
                System.out.println("disconnectCallback on channel:" + channel + " : " + message.getClass() + " : " + message.toString());
            }

            @Override
            public void reconnectCallback(String channel, Object message) {
                System.out.println("reconnectCallback on channel:" + channel + " : " + message.getClass() + " : " + message.toString());
            }

            @Override
            public void errorCallback(String channel, PubnubError error) {
                System.out.println("errorCallback on channel " + channel + " : " + error.toString());
            }
        });
    }
}

For clarity, I've removed the imports, which you can add back in with the appropriate hotkeys in most IDEs (it's Ctrl+Shift+O in Eclipse).

Morevoer, note that there is a performance penalty in this code, which can be avoided by following the answer to the question How to efficiently map a org.json.JSONObject to a POJO?