五、其他常见垃圾爬虫(社区长期反馈的"流氓 UA")(11 条)
邮箱收割、内容盗取、无视 robots.txt 的口碑恶劣爬虫。上游库内描述明确含有 spam / harvest / fake / abusive 等特征词,或被社区长期反馈。表格中的:"匹配正则"可直接用于做访问日志分析或 WAF 拦截规则。
| # | 匹配正则 | 典型 UA 实例 | 说明 |
|---|---|---|---|
| 1 | antibot | Antibot web crawler for content discovery | |
| 2 | AP3A\.240617\.008 | Mozilla/5.0 (Linux; Android 15; CPH2557 Build/AP3A.240617.008; wv) AppleWebKit/537.36 (KHTML, like … | 80legs web scraping service |
| 3 | AspiegelBot | Mozilla/5.0 (Linux; Android 7.0;) AppleWebKit/537.36 (KHTML, like Gecko) Mobile Safari/537.36 (comp… | Aspiegel web crawler for content discovery |
| 4 | Bark[rR]owler | Barkrowler/0.5.1 (experimenting / debugging - sorry for your logs ) http://www.exensa.com/crawl - a… | Barkrowler web crawler for content discovery |
| 5 | binlar | binlar_2.6.3 binlar2.6.3@unspecified.mail | Binlar web crawler for content discovery |
| 6 | EmailWolf | EmailWolf 1.00 | Email address extraction and contact scraping crawler bot |
| 7 | ip-web-crawler\.com | IP Web Crawler web crawler bot | |
| 8 | PetalBot | Mozilla/5.0 (compatible;PetalBot;+https://webmaster.petalsearch.com/site/petalbot) | Petal search engine web crawler bot |
| 9 | TombaPublicWebCrawler | Mozilla/5.0 (compatible; TombaPublicWebCrawler/1.0; +https://tombascraper.com) | Tomba web crawler for email discovery |
| 10 | YisouSpider | YisouSpider | Yisou search engine web crawler bot |
| 11 | ZoominfoBot | ZoominfoBot (zoominfobot at zoominfo dot com) | ZoomInfo web crawler for business intelligence |
