Submitting a search query using jsoup

Question

<form action="http://www.lyricsfreak.com/search.php">
    <input name="a" value="search" type="hidden">
    <input type="hidden" name="type" value="song">
    <input type="text" name="q" class="searchinp" placeholder="Search artist, albums and songs">
    <input value="Search" type="submit" class="srchbtn searchst" data-tracking='["Search","Lyrics","Search button"]'>
    <a href="http://www.lyricsfreak.com/search.php" class="srchadv" data-tracking='["Search","Lyrics","Advanced search"]'>Advanced search</a>
</form>

I know this question has already been asked but that does not work form me. Actually at this site when I even copy url and paste to other tab it shows error first then after a few seconds it reload again and show the desired page, so I can not get the value at first attempt. It would be great if anybody can give me any solution. As you see my reputation I am very new to stackoverflow if I did any kind of mistake then sorry...

Answer 1

The link http://www.lyricsfreak.com/search.php is not available that is why its showing error. When you say it lists after sometimes, what actually happens is a redirect to home page. So instead of using http://www.lyricsfreak.com/search.php you could use home page link http://www.lyricsfreak.com/

A sample based on comment

Ok. I suppose you want to read the tracks after a search from a java program using jsoup. the problem with lyricsfreak is it checks for source before returning result. If the source querying is not lyricsfreak.com it redirects to a access_error.htm and from there using the below script it submits again to get the result after 3.5 seconds

<script type="text/javascript">
    var redirect_url = document.location.href.match(/\?(.*)/);
    redirect_url = redirect_url ? '/search.php?' + redirect_url[1] : '/';
    setTimeout(function() { document.location.href = (redirect_url); }, 3500);
</script>

So what you could do here is set the referrer as 'lyricsfreak.com' via jsoup and it'll give you proper result with out access_error. So a sample code would be as follows. Of course you could parse the page in a better way but this is just to give you an idea.

import java.util.ArrayList;
import java.util.List;

import org.jsoup.Jsoup;
import org.jsoup.nodes.Document;
import org.jsoup.nodes.Element;

public class LyricsParser {

    public static void main(String[] args) throws Exception {

        String SEARCH_STRING = "Madonna";
        String URL = "http://www.lyricsfreak.com/search.php?a=search&type=song&q=";

        List<String> artists = new ArrayList<String>();
        List<String> songs = new ArrayList<String>();
        List<String> hits = new ArrayList<String>();

        Document doc = Jsoup.connect(URL + SEARCH_STRING)
                .referrer(URL + SEARCH_STRING).get();

        for(Element tracks : doc.select("td.colfirst")){
            for(Element links : tracks.getElementsByTag("a")){
                artists.add(links.text());
            }
        }

        for(Element tracks : doc.select("td > a.song")){
            for(Element links : tracks.getElementsByTag("a")){
                songs.add(links.text());
            }
        }

        for(Element tracks : doc.select("td.colast")){
            hits.add(tracks.text());
        }

        int length = artists.size();
        for(int i=0; i<length; i++){
            System.out.println("[" + artists.get(i) + ",\t" + songs.get(i) + ",\t" + hits.get(i+1) + "]");
        }
    }

}