Fetch twitter page title,description and keywords in java

Question

I want to fetch Twitter page title, description and keywords in Java.

I goggling many times for this but I could not get solution. All gave response me in some ISO-8859 charset format. Please help me to have response in UTF-8 charset format.

I used below code for this,

public class TitDesKey
{
        public static void main ( String[] args ) throws IOException 
        {
            String inputLine,source= null,result_tit= null,result_des= null,result_key= null;
                try 
                {
                        URL url = new URL("http://www.twitter.com");

                        URLConnection conn =  url.openConnection();
                        conn.setRequestProperty("User-Agent","Mozilla/5.0 (X11; U; Linux x86_64; en-GB; rv:1.8.1.6) Gecko/20070723 Iceweasel/2.0.0.6 (Debian-2.0.0.6-0etch1)");
                        BufferedReader in = new BufferedReader(new InputStreamReader(conn.getInputStream()));                        


                        while ((inputLine = in.readLine()) != null) 
                        {
                            source=source+" "+inputLine;
                            if(inputLine.contains("</head>"))
                            {
                                break;
                            }
                        }
                }
                catch (MalformedURLException e)
                {
                    System.out.println("Please Enter Write Information");
                }
                catch (IOException e) 
                {
                    System.out.println("Please Enter Write Information");
                }


//              Title Data
                Pattern PATTERN_tit = Pattern.compile("<title>(.*?)</title>", Pattern.CASE_INSENSITIVE|Pattern.DOTALL);

                Matcher m_tit = PATTERN_tit.matcher(source);
                while (m_tit.find()) 
                {
                    result_tit = m_tit.group(1);
                    result_tit = result_tit.replace("/", "").trim();
                    System.out.println(result_tit);
                }       

//              Description Data
                Pattern Pattern_dis = Pattern.compile("<meta name=\"description\" content=(.*?)>", Pattern.CASE_INSENSITIVE|Pattern.DOTALL);

                Matcher m_dis = Pattern_dis.matcher(source);
                while (m_dis.find()) 
                {
                    result_des = m_dis.group(1);
                    result_des = result_des.replace("/", "").trim();
                    System.out.println(result_des);
                }   

//              Keyword Data
                Pattern Pattern_key = Pattern.compile("<meta name=\"keywords\" content=(.*?)>",Pattern.CASE_INSENSITIVE|Pattern.DOTALL);

                Matcher m_key = Pattern_key.matcher(source);
                while (m_key.find()) 
                {
                    result_key = m_key.group(1);
                    result_key = result_key.replace("/", "").trim();
                    System.out.println(result_key);
                }   
        }
}

Thank You in Advance.

Answer 1

If the page that you are fetching is already in UTF-8, then use overloaded InputStreamReader constructor which accepts Charset as well. Use UTF-8 and you should be ok.

Documentation reference.