爬取谷歌商店各搜索词排名时,报错“由于连接方在一段时间后没有正确答复或连接的主机没有反应,连接尝试失败”,请帮忙! | python | python 技术论坛-380玩彩网官网入口


//以下是我的代码

import requests
import httplib2
import lxml
import re
import time
from lxml import etree
from urllib.parse import urlencode
from httplib2 import socks
app_id = '146'
top_cnt = 200
gp_search_url = 'http://play.google.com/store/search'
proxy_host = '115.29.170.73'
proxy_port = 2080
def get_pack(url_data):
    proxy = httplib2.proxyinfo(socks.proxy_type_socks5, proxy_host, proxy_port)
    http = httplib2.http(proxy_info=proxy)
    uri = '%s?%s' % (gp_search_url, urlencode(url_data))
    response, content = http.request(uri, 'get')
    return content
def parse_page(content, keyword):
    tree = etree.html(content)
    pack_names = tree.xpath(u'//div(@class="card-listtwo-cards")/div/@data-docid')
    rankings = []
    for i,pack_name in enumerate(pack_names,start=1):
        if keyword in pack_name:
            rankings.append((pack_name, i))
    return rankings
if __name__ == "__main__":
    keyword = "your_keyword_here"
    url_data = {'q':keyword}
    content = get_pack(url_data)
    rankings = parse_page(content, keyword)
    for app_info in rankings:
        print(f"app:{app_info[0]},rank:{app_info[1]}")
讨论数量: 1
jason990420

to detect whether the proxy is effective

ping command: test network connection and delay. if the return value is normal, it means that the proxy connection is normal and can be used; otherwise, you need to change other proxy.

d:\>ping 115.29.170.73
pinging 115.29.170.73 with 32 bytes of data:
request timed out.
request timed out.
request timed out.
request timed out.
ping statistics for 115.29.170.73:
    packets: sent = 4, received = 0, lost = 4 (100% loss),
2个月前

讨论应以学习和精进为目的。请勿发布不友善或者负能量的内容,与人为善,比聪明更重要!
网站地图