蟒蛇符号化的UnicodeDecodeError

Question

我试图来标记一些文件，但我有这个错误

UnicodeDecodeError: 'ascii' codec can't decode byte 0xef in position 6: ordinal not in range(128)

import nltk 
import pandas as pd 

df = pd.DataFrame(pd.read_csv('status2.csv')) 
documents = df['status'] 

result = [nltk.word_tokenize(sent) for sent in documents] 

我认为这是unicode的问题，所以我说

documents = unicode(documents, 'utf-8') 

另一个错误

TypeError: coercing to Unicode: need string or buffer, Series found

print documents 

1  Brandon Cachia ,All I know is that,you're so n... 
2  Melissa Zejtunija:HAM AND CHEESE BIEX INI??? *... 
3       .........Where is my mind????? 
4  Having a philosophical discussion with Trudy D... 

Answer 1

unicode对字符串或字节进行操作，但documents是一个熊猫系列。

可能：

result = [nltk.word_tokenize(unicode(sent, 'utf-8')) for sent in documents]