I dropped the shs404sef component, installed Speedcache from Joomunited and used the core SEF. Site speed improved a lot, less crashes.
Update june 14th, 2018: After trying JoomUnited's speedcache, Varnish, sh404sef, JCH Optimize... we came into a conclussion that LiteSpeed Cache is the BEST cache provider. Of course that you need to be hosted on a server with LTS installed instead of Apache, Nginx, IIS. We offer LTS to all our clients as well as LTS cache.
Comparission between Varnish and LTS? Wow... Varnish is the closest contender to LTS but still a 2nd place with a huge gap in front of it. Not only the configuration of the default.vcl is a pain in the butt, but the hated Guru errors that showed up to clients harming your ranking, traffic and reputation.
Litespeed cache is now available for Joomla!: First of all, make sure you have it on your hosting (hey, we offer it b default to all our clients!) and it's configuration is super fast. We've been trying the ESI modules on some of our clients and on this page too and the loading time drops almost by half.