How to Store html content in database

Question

I want to store the temperatures for a year from a weather forecast web site like this one into a database to can use it later in an android application. I tried to use Jsoup, but i only get pieces of the table containing temperatures. Is there any way to get that html table content to can store it?

Answer 1

It would be a whole lot better if you used the API provided by wunderground instead of using jsoup in order to screen scrape the page. The main reasons are that the implementation will be a lot cleaner and also your implementation will be immune to stylistic changes in wunderground web pages. Here is guide on how to consume a REST web service with Spring.

Once you have retrieved the data from the API you could easily store the data in a database using an ORM framework like Hibernate since you would have already created the objects to retrieve the data. You can make your life even easier if you use Spring with Hibernate integration to save the data. Check out this guide.

The guides mentioned above use Spring Boot to make it extremely easy to get started with the Spring framework (gone are the days where it would be almost impossible for a novice to get started with a Spring project all alone)

Answer 2

Broadly speaking, the HTML document displayed on the website would have to be parsed programmatically, tokenized, converted to suitable data types and finally stored into the database. However it should be checked whether the data on the website could be read via a SOAP webservice or something similar, as the interface would be cleaner and the approach more robust.