What's the best, simple way of deploying a lot of containers to AWS? It's necessary to be scalable but cheap if a container sees little or no use.
I have a method of generating docker containers that serve data (using fastAPI). Each container runs some basic numpy/ML, nothing too intensive. More importantly, they might not receive any requests for days on end, we need to able to deploy lots of these for minimal cost and not worry about keeping a whole EC2 instance up for them.
Appreciate anyone with an opinion on docker & AWS. Here's my understanding of the solutions (and why I'm not satisfied with them):
This means uploading your container to ECR, then setting up a load-balancer environment on EB with an endpoint. This is the current solution I use, it's inconvenient that it's endpoint is a public URL, but I've already gone through the challenge of getting that on my virtual cloud. However, I suspect that while this can scale up as it needs to, it always keeps at least one EC2 instance running per container, which means paying like 1000x the cost of what you actually need, maintaining an EC2 instance that just sits there.
I can just deploy an EC2 instance and run my containers on there. But this seeems extremely manual in terms of load (how many containers can fit on one instance?) and in terms of endpoints (I'd have to track the name and assign each container its own port), which seems messy and error prone.
- Lambda Function uploaded as docker container
This means altering the docker container to implement a lambda_function method as it's api, then deploying it as a lambda function. This seems to be the only way to maintain containers in a zero-use means zero-cost (or minimal cost) fashion, paying only as requests come in. But, it means modifying the container for this use case and setting up a bunch of extra plumbing; I've got to somehow meld the typical lambda function dockerfile with my current web-api based dockerfile, and work out how to translate API Gateway requests into docker-lambda invocations.
- Lambda Function, no longer using docker at all
Now I've completely deconstructed the docker containers, but maybe docker is just the wrong thing to use if you need 24/7 access but don't want to keep them running?
Conclusion
So basically I suspect I need to morph my containers into lambda-functions, which is doable but kind of defeats the purpose of having our contractors use docker (I'm now modifying the build procedure in a way that fits with my target environment, the exact thing docker is supposed to prevent!)
