How to unmarshal an escaped JSON string

Question

I am using Sockjs with Go, but when the JavaScript client send json to the server it escapes it, and send's it as a []byte. I'm trying to figure out how to parse the json, so that i can read the data. but I get this error.

json: cannot unmarshal string into Go value of type main.Msg

How can I fix this? html.UnescapeString() has no effect.

val, err := session.ReadMessage()
if err != nil {
break
}
var msg Msg

err = json.Unmarshal(val, &msg)

fmt.Printf("%v", val)
fmt.Printf("%v", err)

type Msg struct {
    Channel string
    Name    string
    Msg     string
}

//Output
"{\"channel\":\"buu\",\"name\":\"john\", \"msg\":\"doe\"}"
json: cannot unmarshal string into Go value of type main.Msg

Answer 1

You might want to use strconv.Unquote on your JSON string first :)

Here's an example, kindly provided by @gregghz:

package main

import (
    "encoding/json"
    "fmt"
    "strconv"
)

type Msg struct {
    Channel string
    Name string
    Msg string
}

func main() {
    var msg Msg
    var val []byte = []byte(`"{\"channel\":\"buu\",\"name\":\"john\", \"msg\":\"doe\"}"`)

    s, _ := strconv.Unquote(string(val))

    err := json.Unmarshal([]byte(s), &msg)

    fmt.Println(s)
    fmt.Println(err)
    fmt.Println(msg.Channel, msg.Name, msg.Msg)
}

Answer 2

You need to fix this in the code that is generating the JSON.

When it turns out formatted like that, it is being JSON encoded twice. Fix that code that is doing the generating so that it only happens once.

Here's some JavaScript that shows what's going on.

// Start with an object
var object = {"channel":"buu","name":"john", "msg":"doe"};

// serialize once
var json = JSON.stringify(object); // {"channel":"buu","name":"john","msg":"doe"}

// serialize twice
json = JSON.stringify(json); // "{\"channel\":\"buu\",\"name\":\"john\",\"msg\":\"doe\"}"

Answer 3

Sometimes, strconv.Unquote doesn't work.

Heres an example shows the problem and my solution. (The playground link: https://play.golang.org/p/Ap0cdBgiA05)

Thanks for @Crazy Train's "encodes twice" idea, I just decoded it twice ...

package main

import (
    "encoding/json"
    "fmt"
    "strconv"
)

type Wrapper struct {
    Data string
}

type Msg struct {
    Photo string
}

func main() {
    var wrapper Wrapper
    var original = `"{\"photo\":\"https:\/\/www.abc.net\/v\/t1.0-1\/p320x320\/123.jpg\"}"`

    _, err := strconv.Unquote(original)

    fmt.Println(err)

    var val []byte = []byte("{\"data\":"+original+"}")
    fmt.Println(string(val))
    err = json.Unmarshal([]byte(val), &wrapper)


    fmt.Println(wrapper.Data)

    var msg Msg
    err = json.Unmarshal([]byte(wrapper.Data), &msg)
    fmt.Println(msg.Photo)
}

Answer 4

As Crazy Train pointed out, it appears that your input is doubly escaped, thus causing the issue. One way to fix this is to make sure that the function session.ReadMessasge() returns proper output that is escaped appropriately. However, if that's not possible, you can always do what x3ro suggested and use the golang function strconv.Unquote.

Here's a playground example of it in action:

http://play.golang.org/p/GTishI0rwe

Answer 5

The data shown in the problem is stringified for some purposes, in some cases you can even have \n in your string representing break of line in your json.

Let's understand the easiest way to unmarshal/deserialize this kind of data using the following example:

1. Next line shows the data you get from your sources and want to derserialize

   stringifiedData := "{\r\n \"a\": \"b\",\r\n \"c\": \"d\"\r\n}"

2. Now, remove all new lines first

   stringifiedData = strings.ReplaceAll(stringifiedData, "\n", "")

3. Then remove all the extra quotes that are present in your string

   stringifiedData = strings.ReplaceAll(stringifiedData, "\\"", "\"")

4. Let's now convert the string into a byte array

   dataInBytes := []byte(stringifiedData)

5. Before doing unmarshal, let's define structure of our data

   jsonObject := &struct{
   A string `json:"a"`
   C string `json:"c"`
   }

6. Now, you can dersialize your values into jsonObject

   json.Unmarshal(dataInBytes, jsonObject)}

Answer 6

You got into infamous escaped string pitfall from JavaScript. Quite often people face (as I did) the same issue in Go, when serializing JSON string with json.Marshal, e.g.:

in := `{"firstName":"John","lastName":"Dow"}`
bytes, err := json.Marshal(in)

json.Marshal escapes double quotes, producing the same issue when you try to unmarshal bytes into struct.

If you faced the issue in Go, have a look at How To Correctly Serialize JSON String In Golang post which describes the issue in details with solution to it.