大家在查：飞掠凯金斯 qmvj dunekiller 摆龙门阵腐朽没落 etiopathogenesis 慈姑 AyatollahAliKhamenei xxsy

Web爬虫

Web爬虫

Web爬虫

Web爬虫

2

If you avoid non-local links and dig down only into local Web pages , you provide a Web crawler for a single Web site, as shown in Listing 7 .

如果能够避免非本地的链接而只访问本地 Web 页面，就可以为这个单一 Web 站点提供 Web 爬虫了，如清单 7 所示。

3

The final step is to allow this XML document to be made accessible to crawlers through a Web server .

最后一步需要使爬虫程序能通过 Web 服务器访问这个 XML 文档。

4

This paper studies the subject of gathering information relevant to the subject, to significantly reduce the amount of web pages dealing .

本文研究的主题爬虫致力于收集主题相关信息，达到极大降低网页处理量的目的。

5

Beacons, also known as "Web bugs" and "pixels , " are small pieces of software that run on a Web page .

beacon 也称为 “网络爬虫（Web bug ）”和 “像素 ”，是可以在网页上运行的小段软件。

6

An archiving crawler simply crawls a Web site pulling content locally to be stored on a long-term storage medium .

归档爬虫会简单地对 Web 站点进行遍历，将其网站的本地内容存储到一个长期的存储介质上。

9

10

This information helps the Web crawler determine what the set of pages is and when to crawl them.

这一信息能帮助 web 爬虫程序决定要爬行哪些页面以及爬行的时间。

11

Web servers can exclude crawlers using a file called robot. txt that tells the crawler what can and can't be crawled.

Web 服务器可以使用一个名为 robot.txt 的文件来执行爬虫，它会告诉爬虫不能搜索的内容。

13

How can the meta robots tag impact how search engines crawl, index and display content on a web page ?

metarobots 标签是如何影响搜索引擎爬虫抓取、索引并显示网页的？

14

But all the major and legitimate Web crawling engines obey the requests in robots. txt.

不过，所有主要的合法 Web 爬虫引擎都会遵从 robots.txt 内的要求。

15

Other useful specialized crawlers include Web site checkers .

其他有用的专用爬虫包括 Web 站点检查器。

16

However the current two ways of web crawling used by focus crawler are low efficiency .

然而目前的主题爬虫所采用的两种基本抓取网页的方式效率比较低下。

18

19

We designed a deep web crawler base on the most efficient queries .

本文提出一种基于最优查询词的深度网络爬虫。

20

It was developed within Google as a mechanism for processing large amounts of raw data , for example , crawled documents or web request logs .

Google 对这个模型进行了实现，用来处理巨量的数据，例如网络爬虫得到的文档和 web 访问到的记录。

热词推荐

大家在看