Converting byte array to json in Compojure

Question

I'm working on a Compojure application that serves json files. I'm using clojure.java.jdbc library to retrieve the files from sqlite database:

(defn grid-result [z x y]
  (j/with-connection db-specs
    (j/with-query-results results ["SELECT grid FROM grids WHERE zoom_level = ? AND tile_column = ? AND tile_row = ?" z x y]
      (doall results))))

The result of the query is this:

({:grid #<byte[] [B@7e88dd33>})

In order to display it as json, I update the headers, and strip the query result to byte array and use ByteArrayInputStream:

(defn grid-response [grid]
  {:status 200
   :headers {"Content-Type" "application/json"}
   :body (new java.io.ByteArrayInputStream (:grid (first grid)))})

This is how the request is handled:

(defroutes app-routes
  (GET "/api/:z/:x/:y.grid.json" [z x y] (grid-response (grid-result z x y))
  (route/not-found "Page not found"))

All above results in:

"xœ«VJ/ÊLQ²ŠVR (éŒ1jĨ£FŒ1jĨ£FŒ1jĨÄEÊPP¨¨ªQl„
Å®€ƒQ#F5b¤«£”ZYÈ�gh`jndaXƒf†æ0†Œa¨[�7ð+k"

Json that I'm trying to retrieve has a format of:

grid({"keys": ["""], "data": {"105803": {"predicti_9": 0.0257, "prediction": "3B2", "OA01CDOLD": "15UFGH0011"}, "106178": {"predicti_9": 0.0265, "prediction": "6B1", "OA01CDOLD": "15UHFE0001"}, "106171": {"predicti_9": 0.0257, "prediction": "3B2", "OA01CDOLD": "15UHFC0001"}, "105721": {"predicti_9": 0.0257, "prediction": "3B2", "OA01CDOLD": "15UFGC0013"}, "106170": {"predicti_9": 0.0257, "prediction": "3B2", "OA01CDOLD": "15UHFB0001"}}, "grid": ["  ", "  "]});

What I've also tried:

Changed grid-response function to convert byte array to String:

:body (String. (:grid (first grid)) "UTF-8")

Results in ((with UTF-8 or without):

x??VJ/?LQ??VR?(?1j??F?1j??F?1j??E?PP????Ql?
???Q#F?5b?????ZY?�gh`jndaX?f??0??a?[�7?

Mapped each char to str:

(apply str (map #(char (bit-and % 255)) (:grid (first grid))))

Results in:

xœ«VJ/ÊLQ²ŠVR (éŒ1jĨ£FŒ1jĨ£FŒ1jĨÄEÊPP¨¨ªQl„
Å®€ƒQ#F5b¤«£”ZYÈ�gh`jndaXƒf†æ0†Œa¨[�7ð+k

(Same as with ava.io.ByteArrayInputStream).

Any advice on how to convert that byte stream to json will be greatly appreciated.

Answer 1

The grids are compressed with zlib. So this is the solution I used:

(defn zlib-decompress
  [input]
  (with-open [input (-> input io/input-stream InflaterInputStream.)]
    (slurp input)))

This function returns a string representation of the json, so when I set the response's content type to "application/json", everything works.

Answer 2

You're trying to read GZIP compressed data. You can try using something like:

(java.io.BufferedInputStream.
 (java.util.zip.GZIPInputStream.
  (new java.io.ByteArrayInputStream (:grid (first grid)))))

to decompress it before displaying it.