Question

I want to create a page that "silently" queries other pages. It then crawls them for a result. When it's done querying all the pages, then it should calculate its own result according to the results retrieved.

What I meant with silent is,

  1. the other web site's code shall not appear on my page, nor affect it in any way
  2. I want each other page to be queried in a different session (like when I open a new tab for each in my browser) or something similar. So that there will be no namespace problems.

I heard chrome would offer something that might be helpful for that?

edit This is NOT about crawling web pages. It is for fetching data from other local stand-alone projects

edit2 I am just looking for an alternative to simply looping URLs and querieng them, because there are namespace issues

Was it helpful?

Solution

You can't access pages from other domains with JavaScript unless the domain explicitly allows your domain.

Typically you would use a server-side language for this, or better, use the website's public API. If they don't have a public API they probably wont appreciate you crawling their site.

Licensed under: CC-BY-SA with attribution
Not affiliated with StackOverflow
scroll top