Join Log in
Score:0
avatar Server

Autoscaling images with EBS attached

kr flag
Learning from masters

We have an AWS EC2 instance that has an EBS gp3 SSD mounted with around 70GB. Sometimes, we do some scp orders to copy new files to this EBS, but for the rest of the time, the instance will only do reading operations in the EBS.

This instance gets requests from internet, and for each requests it has to read 2000 files (1000 of ~ 60kb and 1000 of ~ 414b). Now we want to include this instance in an autoscaling group. What should we do with this EBS? As far as I have read, I can:

  1. Create the new instances with a new EBS that copies the original EBS each time they are created -> It copies GB and does IOPS that at the end means to spend $ + time to copy the EBS.
  2. Use multi-attach EBS -> higher costs of storage (it's provisioned GB, not general)
  3. Use EFS. Lower speed and higher latency. Higher price than EBS, but when multiple are created it will be cheaper.
  4. Use NFS on a micro instance with a autoscaling group with min:1 max:1 to avoid failures, and attach the EBS each time it is created.
  5. Use GlusterFS. I think it's quite expensive on AWS. Is it?

I don't think at long run we will never have more than 100GB shared. What do you think is the best approach in this scenario? I was thinking in 5) but because of costs I was thinking in 4).

19
0 + 0
Score:1
Tim avatar Server
gp flag
Tim

EFS would usually be used in this scenario, it's made to be shared network storage. You can also use multi-attach EBS if EFS cost or performance doesn't work for you.

1 / 4 / 5 are not good options IMHO. Use services rather than servers.

0 + 0
Tim P avatar
af flag
Tim P
I was thinking EFS, but another option would be to bundle the files into an archive (and compress if it makes sense). Think tar, gzip'd tar or zip. Then on instance start, have a script or application that pulls the files from S3 and writes them locally.
0
Reply
Tim avatar
gp flag
Tim
@TimP "Sometimes, we do some scp orders to copy new files to this EBS" made me think that wasn't a good solution. It also means paying for an extra 70GB x # of instances worth of EBS.
0
Reply
ธงชาติไทย
Elon Musk
I sit in a Tesla and translated this thread with Ai:

mangohost

Post an answer

Most people don’t grasp that asking a lot of questions unlocks learning and improves interpersonal bonding. In Alison’s studies, for example, though people could accurately recall how many questions had been asked in their conversations, they didn’t intuit the link between questions and liking. Across four studies, in which participants were engaged in conversations themselves or read transcripts of others’ conversations, people tended not to realize that question asking would influence—or had influenced—the level of amity between the conversationalists.