Score:0

Benchmarking AWS outbound Internet bandwidth (egress) "up to 25 Gbps"

co flag

We conducted our tests on c6gn.2xlarge AWS instances located is us-east-1 region, which are advertised in AWS documentation to have a network performance of "Up to 25 Gbps" with a baseline bandwidth of 12.5 Gbps.

We ran UDP tests with iperf3, from a client VM in Europe, outside AWS network.

On the server side: iperf3 -s -p 45000

On the client side: iperf3 -c <server_public_IPv4> -p 45000 -u -i 1 -b 500M -P 5 -R -t 3600

(sending 5 streams of 500 Mbps each, every second for 1 hour)

After a few minutes (depending on previous usage), the bandwidth will collapse to 250 Mbps, and 90% of packets will get lost.

Yes it's 1/100th of the advertised bandwidth.

Has anyone experienced similar behaviour?

Are you aware of other limitations at the VPC level, rather than per instance?

iperf3 UDP test towards AWS c6gn instance showing network degradation

Tim avatar
gp flag
Tim
Did you test bandwidth to one other server, or to a variety of other servers? It might be that each flow is limited by AWS / TCP but that overall throughput can be maintained to multiple destinations. The question of throughput should probably go to AWS support.
co flag
Hello Tim, thank you for your comment. This benchmark reproduces a behaviour we observed running a live video broadcast on two AWS EC2 servers, hosting 500 viewers spread across North America, that degraded badly after 10 minutes. Yes we contacted AWS support and I'll keep you posted of what they say.
Tim avatar
gp flag
Tim
Have you considered using CloudFront to distribute your content rather than direct from the server? CloudFront supports some streaming protocols. It may be cheaper than direct as well.
co flag
Thank you Tim! In this case we have UDP video live streams that can't go through CloudFront, but for other HTTP content we do use CloudFront, and it's effective.
Score:0
id flag
MLu

I'm afraid that it's a very poorly executed benchmark with worthless results.

  • Does your client VM support 25 Gbps bandwidth? (I very much doubt it)
  • Is your VMs local network 25 Gbps all the way to your ISP? (I very much doubt it)
  • Does your ISP have 25 Gbps bandwidth to AWS PoP? (I very much doubt it)
  • The latency between US and EU may hinder it as well.

The bottom line is - you can't measure 25 Gpbs bandwidth from a single client that doesn't support this speed.

Run the test between two c6gn instances in the same VPC and see. Or run a distributed speed test from many clients with combined bandwidth of 25 Gpbs or more. Otherwise your bottleneck is your client, not the c6gn instance.

co flag
Hello @MLu, as the title reads, it's all about egress *Internet* bandwidth. You are right that within the same VPC (AWS network), the 25Gbps burst and the baseline are achieved. As I described, we ran a 2.5 Gbps test of UDP packets towards a VM outside AWS (Google Cloud precisely) that easily handles 7 Gbps download (we of course tested it separately). With our recent experience of running a live video event, added to these tests, we are 100% positive that the bottleneck comes from the AWS Network (Internet Gateway) and nowhere else.
MLu avatar
id flag
MLu
According to [this announcement](https://aws.amazon.com/about-aws/whats-new/2021/09/amazon-ec2-increases-instance-network-bandwith/) only instances with 32 vCPU or more have guaranteed IGW bandwidth, being 50% of the instance bandwidth. Your c6gn.xlarge has only 8 vCPU so not really covered by that. IMO that "up to 25 Gbps" means just that - "up to but no guarantees". Try with a bigger instance. I'd be keen to know if it helps.
co flag
We performed the same test on c6gn.8xlarge (32 vCPUs) and got throttled the exact same way to 250Mbps. I haven't found any announcement from AWS that discuss actual egress performance, unfortunately. If someone has any insight, please share!
MLu avatar
id flag
MLu
Well then that would be a question for AWS Support. Or use CloudFront - the AWS CDN offering. It may help achieve a better performance.
mangohost

Post an answer

Most people don’t grasp that asking a lot of questions unlocks learning and improves interpersonal bonding. In Alison’s studies, for example, though people could accurately recall how many questions had been asked in their conversations, they didn’t intuit the link between questions and liking. Across four studies, in which participants were engaged in conversations themselves or read transcripts of others’ conversations, people tended not to realize that question asking would influence—or had influenced—the level of amity between the conversationalists.