In the case of my scenario, The loading issue seems to have been caused by making HTTP requests with no HTTP timeout set to our CDN. The synchronous requests held the page execution for many minutes and then the load balancer timed out the request.
The requests are for images on the article pages. We make sure they exist before rendering, if one doesn't exist we show a generic one. An example is https://cloudfront.qwery.com/CDNSource/teasers/56628.jpg there is 1 image per article. Could anyone provide alternative solutions to this?
Currently, the resources are in S3 so perhaps a CRON that syncs data hourly to the webservers, and the webservers take a copy of the S3 bucket when building. For that solution though we'd need an EBS to scale to our total image size. Could anyone guide me on how to effectively calculate it & what that would be? Could anyone also please advise if there would be a better solution for this? Ideally, a local file resource would be available to check if something does/doesn't exist.
We had tried to use EFS for session storage previously but found its cost was far too high for us to use in production, +$10,000/mo. Could anyone advise on how much you would foresee this costing for image storage, and/or how much effort you envision it taking to configure this in production?