I'm investigating using Varnish with ESIs to cache page content for a high traffic forum-like website.
Context : I want to cache content for visitors only (connected users will have a really different display and need absolute fresh content). Still, for visitors, some parts of a page need to be dynamic : - not cachable, for example for a visitor-dependant module (think a 'suggestion' widget that is fed with a real-time analysis of the pages viewed, thanks to a beacon) - cachable with a small TTL of 15mn, for example for a 'latest posts' widget or for very changing ads campaigns.
Currently we are using Apache/PHP/symfony/memcache to cache pages and have in-house ESI-like mecanism : a page from cache is parsed and some specific tags are interpreted (including calls to web services and/or databases). This is not performant enough since server time is then around 700ms.
In remplacement of this solution, we can have Varnish+ESIs. The total nb of ESIs included in a page can reach 15. The real number of ESIs to fetch will be less than that but not so much given the ESI's TTLs. The critical problem is that Varnish fetches the ESIs sequencially instead of parallel and this is not acceptable. This feature is somewhere late in Varnish's roadmap.
So,
What is your experience with Varnish and ESIs ? How many ESIs, response time gain that you have ?
Do you know workarounds or other serious and configurable (VCL was nice) reverse-proxies with parallel ESI fetching ?
If not, what kind of good caching strategy do you use for equivalent use-cases ?
Thanks, P.