Read data from a webpage in D?

Question

how to simply open an url and read the data from a webpage with D? (I prefer phobos over tango, if needing to use standard lib functionality)

Answer 1

curl is in the standard library. You can fetch a url pretty easily like this:

import std.net.curl;
string content = get("d-lang.appspot.com/testUrl2");

http://dlang.org/phobos/std_net_curl.html#get

If you need to parse html, I wrote a dom library that is pretty good at it. https://github.com/adamdruppe/misc-stuff-including-D-programming-language-web-stuff

grab dom.d and characterencodings.d then you can:

import arsd.dom;
auto document = new Document();
document.parseGarbage(content); // content is from above, the html string

writeln(document.title); // the <title> contents
auto paragraph = document.querySelector("p");
if(paragraph is null)
     writeln("no paragraphs in this document");
else
     writeln("the first paragraph is: ", paragraph.innerText);

and so on. If you've used javascript dom api, this is pretty similar (though expanded in a lot of ways too).

Answer 2

I think std.net.curl bindings are your best bet, specifically its get/post methods (example is in the docs): http://dlang.org/phobos/std_net_curl.html#get

After all, curl is designed specifically for this kind of tasks and bindings are part of phobos.