You can iterate on each child URI and create a new JSOUP Document and then collect all the child links
example pseudocode:
download(String toplevelURI, int level) {
if (level > MAX_LEVEL) {
return; //termination condition
}
Document doc = Jsoup.connect(url).get();
Elements links = doc.getElementsByTag("a");
for (Element link : links) {
String url = link.absUrl("href");
link.add(url); //store the current level of link
download(url, level++); //get all children of current link
}
}
There are few things to take care of
- Till what depth you want to download (MAX_DEPTH)
- The links needs to be stored in a tree structure to keep parent link / child link relationship
- The code is doing depth first i.e. it will follow a link until it reaches a page where there are no link or reaches MAX_DEPTH. try changing it so its depth first