使用Python从html表中提取数据

Question

我想解析来自网站的数据。例如，我试图从中提取数据的SRC代码部分看起来像这样。

<table summary="Customer Pending and Vendor Pending Table"> 
    <tr> 
     <th> <a href="/team.php?wrkgrp=Somedata&Escalationsort=Level&Escalationorder=0#Escalation" class="headlink"> 
    <img src="/images/rat/up_selected.png" width="11" height="9" border="0" alt="up">Risk   </a> </th> 
     <th> <a href="/team.php?wrkgrp=Somedata&Escalationsort=AvgLastUpd&Escalationorder=1#Escalation" class="headlink"> 
    Avg Last Updated   </a> </th> 
     <th> <a href="/team.php?wrkgrp=Somedata&Escalationsort=AvgDaysOpen&Escalationorder=1#Escalation" class="headlink"> 
    Avg Days Open   </a> </th> 
     <th> <a href="/team.php?wrkgrp=Somedata&Escalationsort=Srs&Escalationorder=1#Escalation" class="headlink"> 
    # of Cases   </a> </th> 
     <th> <a href="/team.php?wrkgrp=Somedata&Escalationsort_pct=1&Escalationorder=1#Escalation" class="headlink">% of Total Cases</a> </th> 
    </tr> 
     <tr > 
    <td><a href="/snapshot.php?statusrisk=2&wrkgrp=Somedata&function=statusrisk&statuses=CustomerPending"><img src="/images/rat/severity_2.gif" alt="Very High Risk" title="Very High Risk" border="0"></a></td> 
    <td> 8.0</td> 
    <td> 69.0</td> 
    <td>1</td> 
    <td> 3.1</td> 
    </tr> 

我需要从上表中提取值8.0,69.0和3.1。我的Python代码看起来像这样。

from lxml import html 
import requests 

page = requests.get('http://rat-sucker.abc.com/team.php?wrkgrp=somedata') 
tree = html.fromstring(page.text) 
Stats = tree.xpath(//*[@id="leftrat"]/table[1]/tbody/tr[2]/td[2]) 

print 'Stats: ', Stats 

我已经检查使用多种方法和Xcode的模拟器我的Xpath，这是正确的（如果你对上面的部分代码则可能无法正常运行），但运行我的Python脚本时，它不产生任何输出。

[根@测试平台testhost]＃蟒蛇scrapper.py 统计

[根@测试平台testhost]＃

Answer 1

你可以使用BeautifulSoup parser。

>>> s = '''<table summary="Customer Pending and Vendor Pending Table"> 
    <tr> 
     <th> <a href="/team.php?wrkgrp=Somedata&Escalationsort=Level&Escalationorder=0#Escalation" class="headlink"> 
    <img src="/images/rat/up_selected.png" width="11" height="9" border="0" alt="up">Risk   </a> </th> 
     <th> <a href="/team.php?wrkgrp=Somedata&Escalationsort=AvgLastUpd&Escalationorder=1#Escalation" class="headlink"> 
    Avg Last Updated   </a> </th> 
     <th> <a href="/team.php?wrkgrp=Somedata&Escalationsort=AvgDaysOpen&Escalationorder=1#Escalation" class="headlink"> 
    Avg Days Open   </a> </th> 
     <th> <a href="/team.php?wrkgrp=Somedata&Escalationsort=Srs&Escalationorder=1#Escalation" class="headlink"> 
    # of Cases   </a> </th> 
     <th> <a href="/team.php?wrkgrp=Somedata&Escalationsort_pct=1&Escalationorder=1#Escalation" class="headlink">% of Total Cases</a> </th> 
    </tr> 
     <tr > 
    <td><a href="/snapshot.php?statusrisk=2&wrkgrp=Somedata&function=statusrisk&statuses=CustomerPending"><img src="/images/rat/severity_2.gif" alt="Very High Risk" title="Very High Risk" border="0"></a></td> 
    <td> 8.0</td> 
    <td> 69.0</td> 
    <td>1</td> 
    <td> 3.1</td> 
    </tr>''' 
>>> soup = BeautifulSoup(s) 
>>> [i.text.strip() for i in soup.find_all('td', text=True)] 
['8.0', '69.0', '1', '3.1']