Json dumping a dict throws TypeError: keys must be a string

Question

I am attempting to convert the following dict into JSON using json.dumps:

 {
     'post_engaged': 36,
     'post_impressions': 491,
     'post_story': 23,
     'comment_count': 6,
     'created_time': '03:02 AM, Sep 30, 2012',
     'message': 'Specialities of Shaktis and Pandavas. \n While having power, why there isn\\u2019t',
     < built - in function id > : '471662059541196',
     'status_type': 'status',
     'likes_count': 22
 } {
     'post_engaged': 24,
     'text': '30 Sept 2012 Avyakt Murlli ( Dual Voice )',
     'post_story': 8,
     'comment_count': 3,
     'link': 'http:\\/\\/www.youtube.com\\/watch?v=VGmFj8g7JFA&feature=youtube_gdata_player',
     'post_impressions': 307,
     'created_time': '03:04 AM, Sep 30, 2012',
     'message': 'Not available',
     < built - in function id > : '529439300404155',
     'status_type': 'video',
     'likes_count': 7
 } {
     'post_engaged': 37,
     'post_impressions': 447,
     'post_story': 22,
     'comment_count': 4,
     'created_time': '03:11 AM, Sep 30, 2012',
     'message': '30-09-12 \\u092a\\u094d\\u0930\\u093e\\u0924:\\u092e\\u0941\\u0930\\u0932\\u0940 \\u0913\\u0',
     < built - in function id > : '471643246209744',
     'status_type': 'status',
     'likes_count': 20
 } {
     'post_engaged': 36,
     'post_impressions': 423,
     'post_story': 22,
     'comment_count': 0,
     'created_time': '03:04 AM, Sep 29, 2012',
     'message': 'Essence: Sweet children, whenever you have time, earn the true income. Staying i',
     < built - in function id > : '471274672913268',
     'status_type': 'status',
     'likes_count': 20
 } {
     'post_engaged': 16,
     'text': 'Essence Of Murli 29-09-2012',
     'post_story': 5,
     'comment_count': 2,
     'link': 'http:\\/\\/www.youtube.com\\/watch?v=i6OgmbRsJpg&feature=youtube_gdata_player',
     'post_impressions': 291,
     'created_time': '03:04 AM, Sep 29, 2012',
     'message': 'Not available',
     < built - in function id > : '213046588825668',
     'status_type': 'video',
     'likes_count': 5
 }

But it leads me to

TypeError : keys must be a string

The error is likely due to the dict containing, keys like:

 <built-in function id>: '213046588825668'

Can someone please guide me, with how should I remove these elements from the dict?

Answer 1

You could try to clean it up like this:

for key in mydict.keys():
  if type(key) is not str:
    try:
      mydict[str(key)] = mydict[key]
    except:
      try:
        mydict[repr(key)] = mydict[key]
      except:
        pass
    del mydict[key]

This will try to convert any key that is not a string into a string. Any key that could not be converted into a string or represented as a string will be deleted.

Answer 2

Modifying the accepted answer above, I wrote a function to handle dictionaries of arbitrary depth:

def stringify_keys(d):
    """Convert a dict's keys to strings if they are not."""
    for key in d.keys():

        # check inner dict
        if isinstance(d[key], dict):
            value = stringify_keys(d[key])
        else:
            value = d[key]

        # convert nonstring to string if needed
        if not isinstance(key, str):
            try:
                d[str(key)] = value
            except Exception:
                try:
                    d[repr(key)] = value
                except Exception:
                    raise

            # delete old key
            del d[key]
    return d

Answer 3

I know this is an old question and it already has an accepted answer, but alas the accepted answer is just totally wrong.

The real issue here is that the code that generates the dict uses the builtin id function as key instead of the literal string "id". So the simple, obvious and only correct solution is to fix this bug at the source : check the code that generates the dict, and replace id with "id".

Answer 4

Maybe this will help:

your_dict = {("a", "b"):[1,2,3,4]}
# save
with open("file.json","w") as f:
    f.write(json.dumps(list(your_dict.items())))

# load
with open("file.json","r") as f:
    your_dict = dict([tuple((tuple(x[0]), x[1])) for x in json.loads(f.read())])

Answer 5

Nolan conaway's answer gives this result for example

{"b'opening_hours'": {"b'1_from_hour'": 720, "b'1_to_hour'": 1440, "b'1_break_from_hour'": 1440, "b'1_break_to_hour'": 1440, "b'2_from_hour'": 720, "b'2_to_hour'": 1440, "b'2_break_from_hour'": 1440, "b'2_break_to_hour'": 1440, "b'3_from_hour'": 720, "b'3_to_hour'": 1440, "b'3_break_from_hour'": 1440, "b'3_break_to_hour'": 1440, "b'4_from_hour'": 720, "b'4_to_hour'": 1440, "b'4_break_from_hour'": 1440, "b'4_break_to_hour'": 1440, "b'5_from_hour'": 720, "b'5_to_hour'": 1440, "b'5_break_from_hour'": 1440, "b'5_break_to_hour'": 1440, "b'6_from_hour'": 720, "b'6_to_hour'": 1440, "b'6_break_from_hour'": 1440, "b'6_break_to_hour'": 1440, "b'7_from_hour'": 720, "b'7_to_hour'": 1440, "b'7_break_from_hour'": 1440, "b'7_break_to_hour'": 1440}}

while this amended version

import time
import re
import json
from phpserialize import *


class Helpers:
   def stringify_keys(self,d):
    """Convert a dict's keys to strings if they are not."""
    for key in d.keys():
        # check inner dict
        if isinstance(d[key], dict):
            value = Helpers().stringify_keys(d[key])
        else:
            value = d[key]
        # convert nonstring to string if needed
        if not isinstance(key, str):
            try:
                d[key.decode("utf-8")] = value
            except Exception:
                try:
                    d[repr(key)] = value
                except Exception:
                    raise

            # delete old key
            del d[key]
    return d

will give this cleaner version..

{"opening_hours": {"1_from_hour": 720, "1_to_hour": 1440, "1_break_from_hour": 1440, "1_break_to_hour": 1440, "2_from_hour": 720, "2_to_hour": 1440, "2_break_from_hour": 1440, "2_break_to_hour": 1440, "3_from_hour": 720, "3_to_hour": 1440, "3_break_from_hour": 1440, "3_break_to_hour": 1440, "4_from_hour": 720, "4_to_hour": 1440, "4_break_from_hour": 1440, "4_break_to_hour": 1440, "5_from_hour": 720, "5_to_hour": 1440, "5_break_from_hour": 1440, "5_break_to_hour": 1440, "6_from_hour": 720, "6_to_hour": 1440, "6_break_from_hour": 1440, "6_break_to_hour": 1440, "7_from_hour": 720, "7_to_hour": 1440, "7_break_from_hour": 1440, "7_break_to_hour": 1440}}

Answer 6

Ideally you would want to clean your data so you comply to the data types supported by JSON.

If you simply want to suppress/ or remove these elements from the dict while serializing, you can use skipkeys argument, description can be found in json.dump section

If skipkeys is true (default: False), then dict keys that are not of a basic type (str, int, float, bool, None) will be skipped instead of raising a TypeError.

json.dumps(obj, skipkeys=True)

This solution is much cleaner and allows the standard library handle erroneous keys for you.

WARNING: You must fully understand the implications of using such a drastic method as this will result in data loss for non-compliant data types as JSON keys.

Answer 7

Maybe this will help the next guy:

strjson = json.dumps(str(dic).replace("'",'"'))