对于一次性代码爬虫的需求, 使用 postman + chrome / edge + python 可以几分钟内解决请求本身. 省下时间来写逻辑

举steam页面为最最简单的例子 https://store.steampowered.com/category/action/

想获取这个页面的游戏信息, 打开F12调试面板选择 xhr 过滤 ajax 请求 ( 大多数前后端分离的网站数据都用 ajax 获取

刷新一下页面, 然后从选择里随便点点看有没有需要的信息, 然后看字段大致猜内容.

锁定到需要的请求报文. 右键复制为 curl

---

下载一个 postman ( 一个接口测试工具 https://www.postman.com/

把复制的 curl 导入 postman

---

打卡右边 tab, 到处为 python request 格式

好了, 得到的 python 代码, 其他更复杂的需求就可以拿这个模板改了

  import requests

  url = "https://store.steampowered.com/dynamicstore/userdata/?id=0&cc=CN&origin=https:%2F%2Fstore.steampowered.com"

  payload = {}
  headers = {
    'Accept': 'application/json, text/plain, */*',
    'Accept-Language': 'zh-CN,zh;q=0.9,en;q=0.8,en-GB;q=0.7,en-US;q=0.6',
    'Connection': 'keep-alive',
    'Cookie': 'browserid=3973951764346379062; timezoneOffset=28800,0; recentapps=%7B%222299510%22%3A1729311143%2C%222153420%22%3A1728196597%2C%221831250%22%3A1727516663%2C%223107870%22%3A1726332337%2C%222093920%22%3A1725887532%2C%222543350%22%3A1719641908%2C%222453360%22%3A1718427416%2C%221710100%22%3A1718426242%2C%222002970%22%3A1718426235%2C%22517710%22%3A1718426165%7D; steamCountry=CN%7C951bb454afe61d6715cee4a6db6f3b20; sessionid=61ccccaf7fa4ba51119d362f; app_impressions=730%401_4_4__129_1%7C2023650%401_4_4__100735%7C473930%401_4_4__125%7C374040%401_49_4_1592_43%7C440%401_49_4_1592_1601%7C2050650%401_49_4_1592_1601%7C2995920%401_49_4_1592_201%7C230410%401_49_4_1592_201%7C2358720%401_49_4_1592_201%7C550%401_49_4_1592_201%7C271590%401_49_4_1592_201%7C1938090%401_49_4_1592_201%7C431960%401_49_4_1592_201%7C730%401_49_4_1592_201%7C570%401_49_4_1592_201%7C1091500%401_49_4_1592_1601%7C1517290%401_49_4_1592_1601%7C2138330%401_49_4_1592_1601%7C730%401_49_4_1592_salesmartpopularpurchased%7C2933620%401_49_4_1592_salesmartpopularpurchased%7C2429640%401_49_4_1592_salesmartpopularpurchased%7C1845910%401_49_4_1592_salesmartpopularpurchased%7C578080%401_49_4_1592_salesmartpopularpurchased%7C570%401_49_4_1592_salesmartpopularpurchased%7C1091500%401_49_4_1592_salesmartpopularpurchased%7C2246340%401_49_4_1592_salesmartpopularpurchased%7C1203220%401_49_4_1592_salesmartpopularpurchased%7C230410%401_49_4_1592_salesmartpopularpurchased%7C2139460%401_49_4_1592_salesmartpopularpurchased%7C1517290%401_49_4_1592_salesmartpopularpurchased%7C1172470%401_49_4_1592_salesmartpopularpurchased%7C381210%401_49_4_1592_salesmartpopularpurchased',
    'Referer': 'https://store.steampowered.com/category/action/',
    'Sec-Fetch-Dest': 'empty',
    'Sec-Fetch-Mode': 'cors',
    'Sec-Fetch-Site': 'same-origin',
    'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/130.0.0.0 Safari/537.36 Edg/130.0.0.0',
    'sec-ch-ua': '"Chromium";v="130", "Microsoft Edge";v="130", "Not?A_Brand";v="99"',
    'sec-ch-ua-mobile': '?0',
    'sec-ch-ua-platform': '"Windows"'
  }

  response = requests.request("GET", url, headers=headers, data=payload)

  print(response.text)