Live Search 改进爬虫系统
由. Ghost 将文章归档于 Windows Live
Live Search Webmaster Center Blog 近日公布了 Live Search 改进爬虫系统的一些细节。这次 Live Search 爬虫系统的更新改进了抓取和索引网站的效率。Live Search 团队始终致力于寻找能够帮助站长的方式，并希望这次能让使 Live Search 往正确的方向发展：
- HTTP Compression: HTTP compression allows faster transmission time by compressing static files and application responses, reducing network load between your servers and our crawler…
- Conditional Get: We support conditional get as defined by RFC 2616 (Section 14.25), generally we will not download the page unless it has changed since the last time we crawled it…
除了这两个更新，其他还有许多性能上的改进。另外，Live Search 团队也升级了 MSNbot User Agent，现已经是 "msnbot/1.1"。(via livesino)
- Conditional Get: We support conditional get as defined by RFC 2616 (Section 14.25), generally we will not download the page unless it has changed since the last time we crawled it…
除了这两个更新，其他还有许多性能上的改进。另外，Live Search 团队也升级了 MSNbot User Agent，现已经是 "msnbot/1.1"。(via livesino)
Permalink: Live Search 改进爬虫系统
Tags:
WindowsLive
Search
搜索引擎
爬虫系统
改进更新
Trackback: http://www.creative-weblogging.com/cgi-bin/mt-tb.pl/113882
Mr Wong
| 
