Tag web-crawler - This is page 285 - GeneraCodice

How, using .NET RegEx, do I parse an HTML file and find 1. External links. 2. Internal links

https://www.generacodice.com/en/articolo/170632/how-using-net-regex-do-i-parse-an-html-file-and-find-1-external-links-2-internal-links

c# - regex - web-crawler

Web crawler links/page logic in PHP

https://www.generacodice.com/en/articolo/167978/web-crawler-links-page-logic-in-php

PHP - hyperlink - logic - web-crawler

Save a deque in a text file

https://www.generacodice.com/en/articolo/163009/save-a-deque-in-a-text-file

python - web-crawler

Best way to store data for Greasemonkey based crawler?

https://www.generacodice.com/en/articolo/158265/best-way-to-store-data-for-greasemonkey-based-crawler

storage - persistence - greasemonkey - web-crawler - xmlhttprequest

robots.txt: disallow all but a select few, why not? [closed]

https://www.generacodice.com/en/articolo/158141/robots-txt-disallow-all-but-a-select-few-why-not-closed

seo - web-crawler - robots.txt

How to get list of URLs for a domain

https://www.generacodice.com/en/articolo/158135/how-to-get-list-of-urls-for-a-domain

dns - url - screen-scraping - web-crawler

Is it possible to discover plugged disks from Java?

https://www.generacodice.com/en/articolo/151952/is-it-possible-to-discover-plugged-disks-from-java

java - io - disk - web-crawler - discovery

How to find “equivalent” texts?

https://www.generacodice.com/en/articolo/146788/how-to-find-equivalent-texts

data-structures - web-crawler - data-mining

Possible to prevent search engine spiders from infinitely crawling paging links on search results?

https://www.generacodice.com/en/articolo/145761/possible-to-prevent-search-engine-spiders-from-infinitely-crawling-paging-links-on-search-results

seo - web-crawler - robots.txt - nofollow

Creating a simple 'spider'

https://www.generacodice.com/en/articolo/143385/creating-a-simple-spider

c# - web-crawler

Results found: 2920