can't find stock price in yahoo finance using selenium webdriver

Question

I've been trying to find a way to pull stock data from yahoo finance for the past two days. I've used selenium webdriver before and I find it very useful, however I can't seem to get it to locate the price for stocks. The html is rather complicated in finance.yahoo.com but I think you can isolate the tags so my code looks something like this:

public void writeSheet() throws WriteException, Exception{
    int r = 0;
    for(String s : listOfFunds){
        Label stockLabel = new Label(0,r,s);
        Number stockPrice = new Number(1,r,0.00);
        driver.get("http://finance.yahoo.com/q?s=" + s + "%2C+&q1=q");
        Thread.sleep(200);
        WebElement price = driver.findElement(By.tagName("span"));
        stockPrice.setValue(Double.parseDouble(price.findElement(By.id("yfs_184_" + s.toLowerCase())).getText()));
        id=\"yfs_184_" + s.toLowerCase() + "\""));
        sourceSheet.addCell(stockLabel);
        sourceSheet.addCell(stockPrice);
        r++;
    }
    sourceBook.write();
    sourceBook.close();
}

I'm really just sort of checking to see I didn't miss anything stupid or if I'm searching by the tag name. The one HTML line I'm looking for looks like this:

<span id="yfs_184_" + insert symbol to lowercase here + ">stock price</span>

Answer 1

I tried this and it worked very well.

WebElement stockpriceText = driver.findElement(By.cssSelector("span[id=\"yfs_l84_ctsh\"]"));
String price = stockpriceText.getText();

Output: 51.59

I believe your locating strategy is a bit wrong. You are trying to first find a span in:

WebElement price = driver.findElement(By.tagName("span"));

which could be the problem. WebDriver will find the very first <span> element on the page. It could be some other span in the DOM.
Plus when you are executing this:

price.findElement(By.id("yfs_184_" + s.toLowerCase())).getText();

That time you are trying to find another element inside the span element which will not work.