Python XHR请求格式

Question

我想从使用AJAX的网站获取信息。当我使用REST插件进行chrome时，我可以发送请求并接收所需的答案。我想编写一个python脚本，但我坚持如何以正确的格式编写请求。目前，我收到了未经授权的页面响应，我认为这是由于未使用XHR发送它？

代码：

import json 
import requests 

payload = {"stores":"3650","products":{"31445761":[{"sku":"6000197536050","upc":["4549659075"]}]},"origin":"pip","csrfToken":"d707af2ed8b79a78a669b38dff593c909f6b6262-1507764346644-ebc72845dfa30431a8f7b1c4"} 

with requests.Session() as session: 
    session.get("https://www.walmart.ca") 
    r = session.post('https://www.walmart.ca/ws/en/products/availability', data=json.dumps(payload), 
       headers={"Content-Type": "application/x-www-form-urlencoded; charset=UTF-8"}) 

    print(r.content) 

当我在REST Chrome扩展I型：

stores=["3650"]&products={"31445761":[{"sku":"6000197536050","upc":["4549659075"]}]}&origin=pip&csrfToken=d707af2ed8b79a78a669b38dff593c909f6b6262-1507764346644-ebc72845dfa30431a8f7b1c4 

这给了我希望的结果。

所需的输出：

{ 
31445761: { 
"online": [ 
    { 
"maxRegularPrice": 0, 
"minRegularPrice": 0, 
"mapPrice": 0, 
"minCurrentPrice": 99.96, 
"maxCurrentPrice": 99.96, 
"inventory": 0, 
"sku": "6000197536050", 
"clearance": false, 
"offerId": "6000197536050", 
"limited": false, 
"reducedPrice": false, 
"offerType": "1P", 
"limitedStock": false, 
"sellerId": "0", 
"rollback": false, 
"date": "", 
"status": "OutOfStock", 
"eligible": false, 
"sellerName": "Walmart", 
"asAdvertised": false 
} 
], 
"stores": [ 
    { 
"minRegularPrice": 0, 
"maxRegularPrice": 0, 
"minCurrentPrice": 99.96, 
"maxCurrentPrice": 99.96, 
"inventory": 0, 
"sku": "6000197536050", 
"clearance": false, 
"limited": false, 
"limitedStock": false, 
"rollback": false, 
"date": "", 
"status": "OutOfStock", 
"storeNumber": "3650", 
"eligible": false, 
"asAdvertised": false 
} 
], 
"onlineSummary": { 
"status": "OutOfStock", 
"date": "", 
"eligible": false, 
"clearance": false, 
"rollback": false, 
"asAdvertised": false, 
"limited": false, 
"limitedStock": false, 
"minRegularPrice": 0, 
"maxRegularPrice": 0, 
"minCurrentPrice": 99.96, 
"maxCurrentPrice": 99.96 
}, 
"storeSummary": { 
"status": "OutOfStock", 
"date": "", 
"eligible": false, 
"clearance": false, 
"rollback": false, 
"asAdvertised": false, 
"limited": false, 
"limitedStock": false, 
"minRegularPrice": 0, 
"maxRegularPrice": 0, 
"minCurrentPrice": 99.96, 
"maxCurrentPrice": 99.96 
} 
} 
}` 

Answer 1

你只是缺少一个标题：

'X-Requested-With': 'XMLHttpRequest' 

更新代码：

import json 
import requests 

payload = {"stores":"3650","products":{"31445761":[{"sku":"6000197536050","upc":["4549659075"]}]},"origin":"pip","csrfToken":"d707af2ed8b79a78a669b38dff593c909f6b6262-1507764346644-ebc72845dfa30431a8f7b1c4"} 

with requests.Session() as session: 
    session.get("https://www.walmart.ca") 
    r = session.post('https://www.walmart.ca/ws/en/products/availability', data=json.dumps(payload), 
       headers={"Content-Type": "application/x-www-form-urlencoded; charset=UTF-8", 'X-Requested-With': 'XMLHttpRequest'}) 

    print(r.content) 

而且，通常CSRF令牌的变化。 因此，您应该检索HTML，获取CSRF令牌，然后执行您的XHR POST请求。

我会推荐beautifulsoup在HTML中查找CSRF标记。