'dict' object has no attribute 'union'

Question

I have wdict as a dictionary and i want to add all unique words to it which are scanned from the text files stored at path and converted to list of words by CleanDoc(). I an getting error AttributeError: 'dict' object has no attribute 'union'. What should i do?

import collections
import os.path
import glob
import nltk

wdict = {}
path = "C://Python27//Corpus Files//*.*"


#this function cleans up a doc (removes stopwords etc)
def cleanDoc(doc):
    stopset = set(nltk.corpus.stopwords.words('english'))
    stemmer = nltk.PorterStemmer()
    tokens = nltk.WordPunctTokenizer().tokenize(doc)
    clean = [token.lower() for token in tokens if token.lower() not in stopset and len(token) > 3 and token.isalpha() and not 'reuter']
    final = [stemmer.stem(word) for word in clean]
    return final

for text in glob.glob(path):
    f = open(text)
    data= f.read()
    words = cleanDoc(data)
    wdict = wdict.union(words)
    print wdict

Answer 1

You can use a set instead of a dict:

wdict = set() # `wset` will make a better name

Also probably wdict.update(words) will look better than wdict = wdict.union(words)

Answer 2

Python Dictionary object does not have a union method. As suggested in the error. Union method is only available over Sets.

You should look at the SO answer :- How to merge two Python dictionaries in a single expression?

My favorite is :-

       w_dicts.update(words)

But this is purely because of personal choice.

Hope this helps.

Answer 3

Change wdict = {} to wdict = set().