Ryen's Blog

知识就是力量

python爬虫UserAgent伪装库

在找资料的时候我发现一个库fake_useragent，这个库可以伪装请求头，具体用法 from fake_useragent import UserAgent ua = UserAgent() #ie浏览器的user agent print(ua.ie) #opera浏

Posted by Ryen's Blog on Thursday, September 9, 2021

Python3查看字符编码方法

查看字符编码，需要用到chardet模块一、查看网页编码 #coding=utf-8 import urllib . request import chardet url = 'http://www.baidu.com' a = urllib . request . urlopen ( url ) encode = chardet . detect ( a . read ( ) ) print ( encode [ 'encoding' ] ) 二、查

Posted by Ryen's Blog on Thursday, March 4, 2021

Python爬虫编码格式问题GB2312转换utf8

在最近的图片爬虫中遇到的问题是：爬取网页得到的结果如下（部分）里面的中文出现乱码。 <!DOCTYPE html> <html lang='zh-CN'> <head> <meta charset='gb2312'> <meta content='IE=edge' http-equiv='X-UA-Compatible'> <title>2017Äê

Posted by Ryen's Blog on Thursday, March 4, 2021