It's generally not the best decision to blanket cache collections for which there are tons of different combinations of queries. Chances are, certain query combinations are accessed much more often than others (for instance combinations for which there's lots of SEO juice, or combinations that you distribute/share links to/have links to on your site, or are just more relevant to your users), so those should be selectively cached. The problem with just caching everything for a long ttl is if the url space is big, is that you may run out of memory and nuke resources that are frequently accessed in favor of caching things that are infrequently accessed.
There is no limit of ESI includes per page, and the approach you describe is a good strategy assuming the hit rate on the xml subdocuments will be very high. Cache hits in varnish are very lightweight so even if a page is a composite of 50 cache hits I think it will perform quite well compared to no caching. If the hit rate on esi included subdocuments is low, and there are tons of them on each page, it will result in worse performance than just having the backend rendering the subdocuments each time. I would definitely recommend doing some load testing on the following scenarios so that you can make an educated decision:
- No caching, backend renders subdocuments every time.
- ESI to render subdocuments as fragments, 0% cache hits.
- ESI to render subdocuments as fragments, 50% cache hits.
- ESI to render subdocuments as fragments, 100% cache hits.
This will give a nice picture of how performance will degrade as your hit rate goes down (it may not be linear, hence doing 0%, 50%, 100%) and also tell you how much caching can improve the performance in theory. To me it seems likely that the best solution is some combination of esi:including fragments in a "working set" of regularly accessed subdocuments and rendering the subdocuments directly on the backend if they are not in the working set.