Invalid serialised value error when reading a map of stringified json objects from Cassandra 1.2 using DataStax 1.0.5 java driver and Cassaforte

Question

I'm using cassaforte 1.3.0-beta9, Cassandra 1.2 (DataStax), and com.datastax.cassandra/cassandra-driver-core 1.0.5 (since 2.0 does not work with Cassandra 1.2). I have a table with the following schema:

  CREATE TABLE entities (
  id text,
  address_country text,
  address_county text,
  address_region text,
  address_street_two text,
  csv_uploads list<text>,
  documents list<text>,
  name text,
  notes list<text>,
  photos list<text>,
  project_id text,
  property_code text,
  property_data text,
  retrofit_completion_date text,
  metering_point_ids text,
  devices map<text,text>,
  user_id text,
  PRIMARY KEY (id)
)

This is an example of what's stored in a row:

7b237da126d39ccf98f80d78b6145a10828970f8 |
{'b9d88015c84312bf24cef0e7f1a0baf98dbc8d8d':
  '{"metadata":{"passivrole":"zone 1 temperature,zone 1 temperature"},
    "readings":
      [{"type":"temperature","resolution":"120","accuracy":"0.1","period":"INSTANT"},
       {"type":"interpolatedTemperatureAir","resolution":100,"accuracy":"0.1","period":"INSTANT"}],
   "description":"External air temperature sensor",
   "entity-id":"3e0730d4672538307056a30615aaaa333107d94e"}', 
'd2fe3dca47b760dd67759a56ee0ebbc2cb3395f4': 
  '{"metadata":{"passivrole":"zone 1 humidity"},
    "readings":[{"type":"blah","resolution":"120","accuracy":"0.1","period":"INSTANT"}],
  "description":" air humidity sensor",
  "entity-id":"3e0730d4672538307056a30615aaaa333107d94e"}'} |
alice

When I try to read the devices column I get the following error:

com.datastax.driver.core.exceptions.InvalidTypeException: Invalid serialized value for type map<text, text> (String didn't validate.)
    at com.datastax.driver.core.DataType.deserialize(DataType.java:503)

I can read devices column with no problems using cqlsh (which is shown above). The values stored in the map are stringified json objects.

Here's the sample workflow:

(require '[clojurewerkz.cassaforte.client :as cassaclient]
         ' [clojurewerkz.cassaforte.query :as cassaquery]
         ' [clojurewerkz.cassaforte.cql :as cql])

;; SELECT on empty devices:
(binding [cassaclient/*default-session* session]
      (let [entity (cql/select :entities (cassaquery/where :id "3e0730d4672538307056a30615aaaa333107d94e"))]
        (prn "retrieved entity: " entity)))

;; Returns vector of results, no errors:
"retrieved entity: " [{:retrofit_completion_date nil, :notes nil, :property_code "3e49de0-12af-012e-4f3a-12313b0348f8", :devices nil, :name nil, :user_id "alice", :csv_uploads nil, :project_id "0b4f512fb7834a0878252ab144a525d424445ba2", :photos nil, :property_data nil, :metering_point_ids "[\"1\" \"2\"]", :address_street_two nil, :documents nil, :address_region nil, :address_country nil, :address_county nil, :id "3e0730d4672538307056a30615aaaa333107d94e"}]

;; UPDATE using cassaforte:
(binding [cassaclient/*default-session* session]
  (cql/update :entities {:devices  {"0edf9e09bce895caf0342ac6ae12511555930362" "{:metadata {\"passivrole\" \"zone 1 humidity\"}, :readings [{\"type\" \"tba\", \"resolution\" \"90\", \"accuracy\" \"0.2\", \"period\" \"INSTANT\"}], :description \"lalalala\", :entity-id \"3e0730d4672538307056a30615aaaa333107d94e\"}"}} (cassaquery/where :id "3e0730d4672538307056a30615aaaa333107d94e")))

;; update worked ok, cqlsh (select devices from entities where id = '3e0730d4672538307056a30615aaaa333107d94e';)
;; returns correct map: 
;; {'0edf9e09bce895caf0342ac6ae12511555930362': '{:metadata {"passivrole" "zone 1 humidity"}, :readings [{"type" "tba", "resolution" "90", "accuracy" "0.2", "period" "INSTANT"}], :description "lalalala", :entity-id "3e0730d4672538307056a30615aaaa333107d94e"}'}

;; SELECT using cassaforte
(binding [cassaclient/*default-session* session]
      (let [entity (cql/select :entities (cassaquery/where :id "3e0730d4672538307056a30615aaaa333107d94e"))]
        (prn "retrieved entity: " entity)))

;; Returns this error:
;; InvalidTypeException Invalid serialized value for type map<text, text> (String didn't validate.)  com.datastax.driver.core.DataType.deserialize (DataType.java:503)

I've tried the same workflow using raw CQL and Cassaforte, and the results were exactly the same:

(require '[clojurewerkz.cassaforte.client :as cassaclient])

;; SELECT on empty devices:
(cassaclient/execute session "SELECT * FROM entities WHERE id = '3e0730d4672538307056a30615aaaa333107d94e';")

;; Returns vector of results, no errors:
[{:retrofit_completion_date nil, :notes nil, :property_code "3e49de0-12af-012e-4f3a-12313b0348f8", :devices nil, :name nil, :user_id "alice", :csv_uploads nil, :project_id "0b4f512fb7834a0878252ab144a525d424445ba2", :photos nil, :property_data nil, :metering_point_ids "[\"1\" \"2\"]", :address_street_two nil, :documents nil, :address_region nil, :address_country nil, :address_county nil, :id "3e0730d4672538307056a30615aaaa333107d94e"}]

;; UPDATE using cassaforte:
(cassaclient/execute session "UPDATE entities SET devices = {'0edf9e09bce895caf0342ac6ae12511555930362' : '{:metadata {\"passivrole\" \"zone 1 humidity\"}, :readings [{\"type\" \"tba\", \"resolution\" \"90\", \"accuracy\" \"0.2\", \"period\" \"INSTANT\"}], :description \"lalalala\", :entity-id \"3e0730d4672538307056a30615aaaa333107d94e\"}'} WHERE id = '3e0730d4672538307056a30615aaaa333107d94e';")

;; update worked ok, cqlsh (select devices from entities where id = '3e0730d4672538307056a30615aaaa333107d94e';)
;; returns correct map: 
;; {'0edf9e09bce895caf0342ac6ae12511555930362': '{:metadata {"passivrole" "zone 1 humidity"}, :readings [{"type" "tba", "resolution" "90", "accuracy" "0.2", "period" "INSTANT"}], :description "lalalala", :entity-id "3e0730d4672538307056a30615aaaa333107d94e"}'}

;; SELECT using cassaforte
(cassaclient/execute session "SELECT * FROM entities WHERE id = '3e0730d4672538307056a30615aaaa333107d94e';")

;; Returns this error:
;; InvalidTypeException Invalid serialized value for type map<text, text> (String didn't validate.)  com.datastax.driver.core.DataType.deserialize (DataType.java:503)

Any help and/or suggestions on how json objects should be stored in Cassandra's map collection will be greatly appreciated.

Answer 1

(Cassaforte Maintainer here)

Haven't seen this question until now,

This was an issue with 1.0.5 version of DataStax driver. Cassaforte is currently at 2.0.0, issue is closed wit any version above beta12. More info here: https://github.com/clojurewerkz/cassaforte/issues/41

Use latest version of Cassaforte and try sticking to the version of Java Driver that comes bundled with Cassaforte and it'll be all great.