Question

I'm trying to clear some tokens of the following punctuation by starting with an empty array then adding cleaned tokens to that array.

def main():
    tokens = readTokens("words.txt")
    depunctuatuate = depunctuateToken(tokens)

def readTokens(fileName):
    s = Scanner(fileName)
    items = []
    token = s.readstring()
    while (token != ""):
        items.append(token)
        token = s.readstring()
    print(items)
    s.close()
    return items

def depunctuateToken(string):
    result = []
    s = Scanner(string)
    token = s.readstring()
    punct = [",",".","?",":","'","-"]
    for i in range(0,len(string),1):
        if(not(string[i] in punct)):
            result += [string[i]]
    return result
Was it helpful?

Solution

You don't need Scanner in the depunctuateToken function.

You are passing it an array of strings, Just iterate through them and do your punct filtering on each one. You probably also want to modify your return statement, right now it looks like it will return the first item only.

Licensed under: CC-BY-SA with attribution
Not affiliated with StackOverflow
scroll top