Best way to take products off a page

Question

I am thinking of making a little program for myself that takes products off a page like amazon and similar. What would be the best way to get all the info about the product off the page? Beautiful soup? Is there anything that would be better? Btw I am using Python

Answer 1

If you are looking for a general purpose library, a scraper such as Beautiful Soup or Scrapy would work.

Specifically, Amazon a powerful API that you might want to take a look at (which works well with Scrapy), which could make the process easier. Don't forget to check whether your usage falls within Amazon's usage guidelines.