最好的方式来条的标点符号从一个字符串

StackOverflow https://stackoverflow.com/questions/265960

  •  06-07-2019
  •  | 
  •  

题

这似乎是应该有一个更简单的方法：

import string
s = "string. With. Punctuation?" # Sample string 
out = s.translate(string.maketrans("",""), string.punctuation)

是那里吗？

有帮助吗？

解决方案

从效率的角度来看，你不会打败

s.translate(None, string.punctuation)

对于更高版本的Python，请使用以下代码：

s.translate(str.maketrans('', '', string.punctuation))

它正在使用查找表在C中执行原始字符串操作 - 除了编写自己的C代码之外，没有什么能比这更好。

如果速度不是担心，另一个选择是：

exclude = set(string.punctuation)
s = ''.join(ch for ch in s if ch not in exclude)

这比使用每个char的s.replace更快，但是不能像非纯python方法那样运行，例如正则表达式或string.translate，正如您可以从下面的时间中看到的那样。对于这种类型的问题，尽可能低的水平做到这一点是值得的。

时间码：

import re, string, timeit

s = "string. With. Punctuation"
exclude = set(string.punctuation)
table = string.maketrans("","")
regex = re.compile('[%s]' % re.escape(string.punctuation))

def test_set(s):
    return ''.join(ch for ch in s if ch not in exclude)

def test_re(s):  # From Vinko's solution, with fix.
    return regex.sub('', s)

def test_trans(s):
    return s.translate(table, string.punctuation)

def test_repl(s):  # From S.Lott's solution
    for c in string.punctuation:
        s=s.replace(c,"")
    return s

print "sets      :",timeit.Timer('f(s)', 'from __main__ import s,test_set as f').timeit(1000000)
print "regex     :",timeit.Timer('f(s)', 'from __main__ import s,test_re as f').timeit(1000000)
print "translate :",timeit.Timer('f(s)', 'from __main__ import s,test_trans as f').timeit(1000000)
print "replace   :",timeit.Timer('f(s)', 'from __main__ import s,test_repl as f').timeit(1000000)

这给出了以下结果：

sets      : 19.8566138744
regex     : 6.86155414581
translate : 2.12455511093
replace   : 28.4436721802

其他提示

正则表达式很简单，如果你知道的话。 

import re
s = "string. With. Punctuation?"
s = re.sub(r'[^\w\s]','',s)

在上面的代码中，我们用空字符串替换（re.sub）所有NON [字母数字字符（\ w）和空格（\ s）]。
因此。和？通过正则表达式运行s变量后，变量's'中不会出现标点符号。

为了方便使用，我总结了Python 2和Python 3中字符串条带化标点符号的注释。请参阅其他答案以获取详细说明。

Python 2 

import string

s = "string. With. Punctuation?"
table = string.maketrans("","")
new_s = s.translate(table, string.punctuation)      # Output: string without punctuation

Python 3 

import string

s = "string. With. Punctuation?"
table = str.maketrans({key: None for key in string.punctuation})
new_s = s.translate(table)                          # Output: string without punctuation

myString.translate(None, string.punctuation)

我通常使用这样的东西：

>>> s = "string. With. Punctuation?" # Sample string
>>> import string
>>> for c in string.punctuation:
...     s= s.replace(c,"")
...
>>> s
'string With Punctuation'

string.punctuation 仅为ASCII ！更正确（但也更慢）的方法是使用unicodedata模块：

# -*- coding: utf-8 -*-
from unicodedata import category
s = u'String — with -  «punctation »...'
s = ''.join(ch for ch in s if category(ch)[0] != 'P')
print 'stripped', s

如果你对这个家庭更熟悉，不一定更简单，但不一样。 

import re, string
s = "string. With. Punctuation?" # Sample string 
out = re.sub('[%s]' % re.escape(string.punctuation), '', s)

对于Python 3 str 或Python 2 unicode 值， str.translate（） 只需要一本字典;在该映射中查找代码点（整数），并删除映射到 None 的任何内容。

要删除（某些？）标点符号，请使用：

import string

remove_punct_map = dict.fromkeys(map(ord, string.punctuation))
s.translate(remove_punct_map)

dict.fromkeys（）类方法使创建映射变得微不足道，根据键序列将所有值设置为 None 。

要删除所有标点符号，而不仅仅是ASCII标点符号，您的表格需要更大一些;请参阅 J.F.塞巴斯蒂安的回答（Python 3版）：

import unicodedata
import sys

remove_punct_map = dict.fromkeys(i for i in range(sys.maxunicode)
                                 if unicodedata.category(chr(i)).startswith('P'))

string.punctuation 错过了载的标点符号的标记，通常是用在现实世界中。如何解决工作的非ASCII标点符号？

import regex
s = u"string. With. Some・Really Weird、Non？ASCII。 「（Punctuation）」?"
remove = regex.compile(ur'[\p{C}|\p{M}|\p{P}|\p{S}|\p{Z}]+', regex.UNICODE)
remove.sub(u" ", s).strip()

就个人而言，我相信这是最好的方式来删除的标点符号从一个字符串中蟒蛇，因为：

  • 它消除了所有Unicode的标点符号
  • 这很容易可改变的，例如你可以除去 \{S} 如果你想删除的标点符号，但保留符号像 $.
  • 你可以得到真正的具体关于什么你要保留什么你想要删除，例如 \{Pd} 将只删除破折号。
  • 这regex也恢复正常的空白。这地图，标签，运输返回，以及其他怪异，好的，单个空间。

这种使用Unicode character properties， 你可以读取更多关于在维基百科.

这可能不是最佳解决方案，但这就是我的方法。

import string
f = lambda x: ''.join([i for i in x if i not in string.punctuation])

这是我写的一个函数。这不是很有效，但它很简单，您可以添加或删除任何您想要的标点符号：

def stripPunc(wordList):
    """Strips punctuation from list of words"""
    puncList = [".",";",":","!","?","/","\\",",","#","@","<*>quot;,"&",")","(","\""]
    for punc in puncList:
        for word in wordList:
            wordList=[word.replace(punc,'') for word in wordList]
    return wordList

这是Python 3.5的单行代码：

import string
"l*ots! o(f. p@u)n[c}t]u[a'ti\"on#$^?/".translate(str.maketrans({a:None for a in string.punctuation}))

我还没有看到这个答案。只需使用正则表达式;除去单词字符（ \ w ）和数字字符（ \ d ）之外的所有字符，后跟空格字符（ \ s ）： 

import re
s = "string. With. Punctuation?" # Sample string 
out = re.sub(ur'[^\w\d\s]+', '', s)

这里有一个解决方案，而不regex.

import string

input_text = "!where??and!!or$$then:)"
punctuation_replacer = string.maketrans(string.punctuation, ' '*len(string.punctuation))    
print ' '.join(input_text.translate(punctuation_replacer).split()).strip()

Output>> where and or then
  • 替换的标点符号与空间
  • 替代多的空间在词语之间有一个单一的空间
  • 删除末尾的空间，如果有任何与 条()

就像更新一样，我在Python 3中重写了@Brian示例并对其进行了更改以在函数内部移动正则表达式编译步骤。我的想法是为了使功能发挥作用所需的每一步。也许您正在使用分布式计算，并且不能在您的工作者之间共享正则表达式对象，并且需要在每个工作者处执行 re.compile 步骤。另外，我很想为Python 3的两个不同的maketrans实现计时

table = str.maketrans({key: None for key in string.punctuation})

vs 

table = str.maketrans('', '', string.punctuation)

另外我添加了另一种使用set的方法，其中我利用交集函数来减少迭代次数。

这是完整的代码：

import re, string, timeit

s = "string. With. Punctuation"


def test_set(s):
    exclude = set(string.punctuation)
    return ''.join(ch for ch in s if ch not in exclude)


def test_set2(s):
    _punctuation = set(string.punctuation)
    for punct in set(s).intersection(_punctuation):
        s = s.replace(punct, ' ')
    return ' '.join(s.split())


def test_re(s):  # From Vinko's solution, with fix.
    regex = re.compile('[%s]' % re.escape(string.punctuation))
    return regex.sub('', s)


def test_trans(s):
    table = str.maketrans({key: None for key in string.punctuation})
    return s.translate(table)


def test_trans2(s):
    table = str.maketrans('', '', string.punctuation)
    return(s.translate(table))


def test_repl(s):  # From S.Lott's solution
    for c in string.punctuation:
        s=s.replace(c,"")
    return s


print("sets      :",timeit.Timer('f(s)', 'from __main__ import s,test_set as f').timeit(1000000))
print("sets2      :",timeit.Timer('f(s)', 'from __main__ import s,test_set2 as f').timeit(1000000))
print("regex     :",timeit.Timer('f(s)', 'from __main__ import s,test_re as f').timeit(1000000))
print("translate :",timeit.Timer('f(s)', 'from __main__ import s,test_trans as f').timeit(1000000))
print("translate2 :",timeit.Timer('f(s)', 'from __main__ import s,test_trans2 as f').timeit(1000000))
print("replace   :",timeit.Timer('f(s)', 'from __main__ import s,test_repl as f').timeit(1000000))

这是我的结果：

sets      : 3.1830138750374317
sets2      : 2.189873124472797
regex     : 7.142953420989215
translate : 4.243278483860195
translate2 : 2.427158243022859
replace   : 4.579746678471565

>>> s = "string. With. Punctuation?"
>>> s = re.sub(r'[^\w\s]','',s)
>>> re.split(r'\s*', s)


['string', 'With', 'Punctuation']

import re
s = "string. With. Punctuation?" # Sample string 
out = re.sub(r'[^a-zA-Z0-9\s]', '', s)

在非常严格的情况下，单行可能会有所帮助：

''.join([c for c in s if c.isalnum() or c.isspace()])

#FIRST METHOD
#Storing all punctuations in a variable    
punctuation='!?,.:;"\')(_-'
newstring='' #Creating empty string
word=raw_input("Enter string: ")
for i in word:
     if(i not in punctuation):
                  newstring+=i
print "The string without punctuation is",newstring

#SECOND METHOD
word=raw_input("Enter string: ")
punctuation='!?,.:;"\')(_-'
newstring=word.translate(None,punctuation)
print "The string without punctuation is",newstring


#Output for both methods
Enter string: hello! welcome -to_python(programming.language)??,
The string without punctuation is: hello welcome topythonprogramminglanguage

with open('one.txt','r')as myFile:

    str1=myFile.read()

    print(str1)


    punctuation = ['(', ')', '?', ':', ';', ',', '.', '!', '/', '"', "'"] 

for i in punctuation:

        str1 = str1.replace(i," ") 
        myList=[]
        myList.extend(str1.split(" "))
print (str1) 
for i in myList:

    print(i,end='\n')
    print ("____________")

使用Python

从文本文件中删除停用词 
print('====THIS IS HOW TO REMOVE STOP WORS====')

with open('one.txt','r')as myFile:

    str1=myFile.read()

    stop_words ="not", "is", "it", "By","between","This","By","A","when","And","up","Then","was","by","It","If","can","an","he","This","or","And","a","i","it","am","at","on","in","of","to","is","so","too","my","the","and","but","are","very","here","even","from","them","then","than","this","that","though","be","But","these"

    myList=[]

    myList.extend(str1.split(" "))

    for i in myList:

        if i not in stop_words:

            print ("____________")

            print(i,end='\n')

这是如何将我们的文档更改为大写 或小写。

print('@@@@This is lower case@@@@')

with open('students.txt','r')as myFile:

    str1=myFile.read()
    str1.lower()
print(str1.lower())

print('*****This is upper case****')

with open('students.txt','r')as myFile:

    str1=myFile.read()

    str1.upper()

print(str1.upper())

我喜欢使用这样的函数：

def scrub(abc):
    while abc[-1] is in list(string.punctuation):
        abc=abc[:-1]
    while abc[0] is in list(string.punctuation):
        abc=abc[1:]
    return abc
许可以下： CC-BY-SA 和 归因
不隶属于 StackOverflow
scroll top