Join Log in
Score:2
automatictester avatar Crypto

AWS signature v4 key derivation

ht flag
automatictester

Generating Authorization header with AWS signature v4 involves deriving signing key as follows:

https://docs.aws.amazon.com/general/latest/gr/signature-v4-examples.html

static byte[] HmacSHA256(String data, byte[] key) throws Exception {
    String algorithm="HmacSHA256";
    Mac mac = Mac.getInstance(algorithm);
    mac.init(new SecretKeySpec(key, algorithm));
    return mac.doFinal(data.getBytes("UTF-8"));
}

static byte[] getSignatureKey(String key, String dateStamp, String regionName, String serviceName) throws Exception {
    byte[] kSecret = ("AWS4" + key).getBytes("UTF-8");
    byte[] kDate = HmacSHA256(dateStamp, kSecret);
    byte[] kRegion = HmacSHA256(regionName, kDate);
    byte[] kService = HmacSHA256(serviceName, kRegion);
    byte[] kSigning = HmacSHA256("aws4_request", kService);
    return kSigning;
}
  1. What is the benefit of calculating HMAC as shown above, in multiple subsequent steps, instead of doing doing just one HmacSHA256 calculation over concatenated input (dateStamp, regionName and serviceName)?
  2. What is the benefit of the above approach instead of using standard HKDF?
115
1 + 0
mac
Score:3
Gilles 'SO- stop being evil' avatar Crypto
cn flag
Gilles 'SO- stop being evil'

The staggered scheme allows each step to be performed by a different entity.

  1. A central server stores the master key and does not communicate it to anyone else. This central server should have high availability and high security, but can have low throughput. The key can be stored in a hardware security module.
  2. The first step combines the key with a protocol version number in an unambiguous way ("AWS4" + key). This allows a protocol upgrade to use the same key, so that you don't have to distribute new keys at the same time as the protocol upgrade.
  3. The second step combines the key with the date (HMAC(dateStamp, "AWS4" + key); the protocol specifies a date stamp with a 1-day resolution). The key changes every day. Quicker changes require more throughput on the central server, whereas slower changes mean a larger impact in case something down the chain is compromised.
  4. Every day, each region server requests the day's region key kRegion = HMAC(regionName, HMAC(dateStamp, "AWS4" + key)). The central server can authenticate the region server to ensure that only a given region's server can obtain the region key.
  5. Each service that uses the key requests its own key kService = HMAC(kRegion, serviceName) from the region where it's running. The region server can authenticate the service to ensure that a service doesn't learn another service's key.
  6. The service combines its key with a purpose: kSigning = HMAC(kService, "aws4_request"). This allows the same master key to be used for different purposes if needs be.

The advantage of this staggering is that each participant only has access to its own keys. It's impossible to find the master key given kRegion, or to find the region key given kService, or to find the service key given kSigning. This minimizes the impact of a compromise: only participants down the chain are affected. For example, if a breach is discovered in the American datacenter on 2022-01-05 and the logs show that the breach occured on 2022-01-01 then only signatures made after 2022-01-01 in the American region are suspicious and may need to be revoked. Signatures made in Europe or before 2022-01-01 are still good.

Of course only large deployments will ever make use of this flexibility. In a majority of applications, the whole calculation is done in the same piece of software in a single function. But the protocol does scale to large deployments that use compartmentalization to mitigate the impact of breaches.

Combining the inputs all at once:

0 + 0
us flag
gusto2
@automatictester Indeed, this was explained in one of the keynotes this year re:Invent. The main purpose is to delegate the authentication to individual regions /services. Each service would receive a derived key for the user once valid for the day, so the service itself doesn't get access to the secret key, yet the service woudn't need to contact the central IAM for each request to authenticate.
0
Reply
Score:1
SEJPM avatar Crypto
us flag
SEJPM

So the problem AWS is solving here is that they have a tuple of values, namely the service, the region, and the current date and they would like to derive a different secret key that depends on all of these values.

You could go ahead and do this with a regular PRF but then you run into a problem: Namely you don't want to have a situation where bad values of any of the tuple entries suddenly yield key collisions. A classic mistake would e.g. be to use straight concatenation and if they're not careful you end up with (to,day) and (tod,ay) yielding the same key. Obviously the strings in play here have more structure than the example, but the underlying worry should be clear.

Now, the most efficient way to resolve the above issue is to use a pairing function that guarantees that no two distinct input tuples yield the same output and thus the same input to the PRF. However, designing such functions is also mildly error-prone.

I guess it is because of this that AWS chose to use this convoluted chain of HMAC invocations. Now they don't have to worry about complicated pairing functions and all that is required is a bunch of HMAC evaluations which should be possible in almost all languages / environments. The fact that this then requires no less than HMAC operations is offset by the fact that all the inputs to this key derivation are somewhat static and you can cache the keys for about a day if HMAC operations cost considerable compute time for you.

0 + 2
automatictester avatar
ht flag
automatictester
Isn't calculating HMAC once over structured string like `[dateStamp=...;regionName=...;serviceName=...]` an easier solution to this problem?
0
Reply
us flag
gusto2
@automatictester it's just another solution, a design decision. When the "structured" input may contain any client input (s3 key, db row id,.) then I would rather use nested hash than risking any potential conflict.
1
Reply
Gilles 'SO- stop being evil' avatar
cn flag
Gilles 'SO- stop being evil'
@automatictester Why do you think it would be easier? Compare the number of lines of code for the two solutions, and note how if the structured string is easy to construct, it means you're adding a large data formatting library as a dependency.
0
Reply
ธงชาติไทย
Elon Musk
I sit in a Tesla and translated this thread with Ai:

mangohost

Post an answer

Most people don’t grasp that asking a lot of questions unlocks learning and improves interpersonal bonding. In Alison’s studies, for example, though people could accurately recall how many questions had been asked in their conversations, they didn’t intuit the link between questions and liking. Across four studies, in which participants were engaged in conversations themselves or read transcripts of others’ conversations, people tended not to realize that question asking would influence—or had influenced—the level of amity between the conversationalists.