Join Log in
Score:3
ubiquibacon avatar Crypto

Confusion over PKCS#1 and "traditional" options with OpenSSL

cn flag
ubiquibacon

I'm attempting to generate a private/public key pair in a few different standards and formats for testing an encryption library I'm working on (i.e. keys with the same "guts", but different formats and standards). Specifically I want a keys generated by OpenSSL in the DER and PEM formats, using the PKCS#1, PKCS#8, and "traditional" standards for each. The "traditional" standard is because apparently OpenSSL did its own thing at one point in time and kind of invented a standard, and it is still possible to generate keys using that "standard" with the -traditional option. So in the end I would have 12 unique files:

DER PEM
PKCS#1 pub/priv pub/priv
PKCS#8 pub/priv pub/priv
Traditional pub/priv pub/priv

I first generated a private/public key pair using the following commands:

openssl genrsa -out private.key 2048
openssl rsa -in private.key -pubout -out public.key

And was given this private key (ASN.1 decoding here):

-----BEGIN RSA PRIVATE KEY-----
MIIEowIBAAKCAQEAr2xKE7fuT/VV2Lk7gfCkA4xOTcFXWboTJ6ZGx1zWCP8d1pY5
mYPx/dTUgDtUjaYGIRJy6G8xYLZvj22aY3l/DdfgLfk4Br9katexMSmKR0C9hVBW
DbCk6ROK9dqEXuzGmpXhfcYs/9dL2N+CptjsS3PcBjxslcBJhUM60jLV+13No95D
Bw1f1PCEb3QNffxxVBEYLzv12xgafSjaCo+uY/BUgKbmU3OO6W1w+8z817t+n11m
ufobCHpyx5f9x7O66gEcT8YT6FtYEPSYVbxPqXveBZaVAUe0uKlvd7yZE5ZAfyKH
LNpT85ay/yfA6O4B9hwslM2El5ge3FKL53jVFQIDAQABAoIBACoD+QsXgPX4OB9A
IFtbOGFcK84OSn5kZZ2XwviRiPcfKXWhxfp5oo5t+Racf0As/WdS7KA0r6IvF/HF
qdZ8/VwisiA4wn1FPocjkZ5JiYPO4wWo6+97+UeU5XHErHMy6U+RqfAutMzzM0im
ofuV1aStw4tf0g8c/s7y/wXk5KD/XNQtdaIqIS3lipIFhvVLSdPsUFU8KhiNKU99
QbGNC7rFvrjdU0jbVgIWPuF7ffT3c0aR6x68Y8b8FOLiD140n7grvllSklnUia2S
zctC4mhZT/9Vfa8cUio+ODGeaXiPqmPsWKJ2kUPSOSRXyG69E5feOXb4uSK4nEWm
7kRXMDkCgYEA6f38oU9/QLjrmjBxC8onVvLpro0V/B7j3PC+IAJynxwwr4dWb5P6
gOVNx+hUxAmDyTg5afmhJXXj9U0ZH5h+cLYN5d9tXHVZQsnozMp2RDGsJxxHZqgN
/hG/EaWe/B1M3XRb+1FV/PjrRZXONdtDfJAw5uZtJ3eUx9tVUjtb9B0CgYEAv+wW
Xsttdw9aARcQqF9LUltpLMiHHYasYju/M2g/axyWQ6S20DVl4rZtURnbUSjn6ppV
9Zr+0puhJWwQGbqMmyDEXlV8myV5+KMM1TfmBrffwcuZLu8bJ2RKZDdxQ+HUKGWs
sxNqzdurmFxIiEjY/hg2kvZ2tJunB8lHLA7VI1kCgYAt9/rIigCa63lFqlybD1TZ
LRGhfBAknsDvJ9CCI1j3Tyd1ZGYjt3OQHPxB2K/Gb4QXZNdKrYLuBBILn+Depyu3
4twqG9G1R0yI5Xe5u9CuJwAGEvUoAr30+vGJevsX1n3CR9jGL71v3EsEOaDwTaod
b4pb4krxZPmypbFGXWj8NQKBgQCX3GaDEbKHQAV20Vpj8Ct4ek5Jmk6XhWXHwQD0
22s7BP69fYsOSwJYlwL+5lfM63I/B2o3EjLKUUz8gHpo8Vyqrw3SmxAi70+I9vOD
j1ybOkl6XfDS224ACHJ4xAoCraGjaXhypk2foE8yGutls0bIID6moRCirntHRPPl
H8N02QKBgAKc7vSuvEnmlhlMZt5bipCclTdtKbYELtbvslk2eNSqya3n3BO3t7FW
jyfvT56ZJ5UXsS0TYtq5XcVWr3ywdwsr5lwb/2lgtGv40NVfKG2OWqTY4s342HCo
GtWJ8KdHv7ZgDmkuSR/fJka8sqjTVzeWvzyKCz7kSx+K0QvIPHqj
-----END RSA PRIVATE KEY-----

And this public key (ASN.1 decoding here):

-----BEGIN PUBLIC KEY-----
MIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEAr2xKE7fuT/VV2Lk7gfCk
A4xOTcFXWboTJ6ZGx1zWCP8d1pY5mYPx/dTUgDtUjaYGIRJy6G8xYLZvj22aY3l/
DdfgLfk4Br9katexMSmKR0C9hVBWDbCk6ROK9dqEXuzGmpXhfcYs/9dL2N+Cptjs
S3PcBjxslcBJhUM60jLV+13No95DBw1f1PCEb3QNffxxVBEYLzv12xgafSjaCo+u
Y/BUgKbmU3OO6W1w+8z817t+n11mufobCHpyx5f9x7O66gEcT8YT6FtYEPSYVbxP
qXveBZaVAUe0uKlvd7yZE5ZAfyKHLNpT85ay/yfA6O4B9hwslM2El5ge3FKL53jV
FQIDAQAB
-----END PUBLIC KEY-----

Based on my reading I was expecting the commands I ran to generate PEM formatted PKCS#1 standard keys, but looking at the ASN.1 decoder, I'm seeing extra pieces of structure I wasn't expecting. My expectations were set by reading the Mbed TLS docs here, because I haven't been able to find any better source that clearly identifies what exactly should be expected in the various PKCS structures. The structure of the public key is identical except the sequence which I drew a box around only contains the exponent and modulus:

enter image description here

I then run the following command to see what a "traditional" key looks like:

openssl rsa -in private.key -traditional -out private_traditional.key

And was given this private key (ASN.1 decoding here):

-----BEGIN RSA PRIVATE KEY-----
MIIEowIBAAKCAQEAr2xKE7fuT/VV2Lk7gfCkA4xOTcFXWboTJ6ZGx1zWCP8d1pY5
mYPx/dTUgDtUjaYGIRJy6G8xYLZvj22aY3l/DdfgLfk4Br9katexMSmKR0C9hVBW
DbCk6ROK9dqEXuzGmpXhfcYs/9dL2N+CptjsS3PcBjxslcBJhUM60jLV+13No95D
Bw1f1PCEb3QNffxxVBEYLzv12xgafSjaCo+uY/BUgKbmU3OO6W1w+8z817t+n11m
ufobCHpyx5f9x7O66gEcT8YT6FtYEPSYVbxPqXveBZaVAUe0uKlvd7yZE5ZAfyKH
LNpT85ay/yfA6O4B9hwslM2El5ge3FKL53jVFQIDAQABAoIBACoD+QsXgPX4OB9A
IFtbOGFcK84OSn5kZZ2XwviRiPcfKXWhxfp5oo5t+Racf0As/WdS7KA0r6IvF/HF
qdZ8/VwisiA4wn1FPocjkZ5JiYPO4wWo6+97+UeU5XHErHMy6U+RqfAutMzzM0im
ofuV1aStw4tf0g8c/s7y/wXk5KD/XNQtdaIqIS3lipIFhvVLSdPsUFU8KhiNKU99
QbGNC7rFvrjdU0jbVgIWPuF7ffT3c0aR6x68Y8b8FOLiD140n7grvllSklnUia2S
zctC4mhZT/9Vfa8cUio+ODGeaXiPqmPsWKJ2kUPSOSRXyG69E5feOXb4uSK4nEWm
7kRXMDkCgYEA6f38oU9/QLjrmjBxC8onVvLpro0V/B7j3PC+IAJynxwwr4dWb5P6
gOVNx+hUxAmDyTg5afmhJXXj9U0ZH5h+cLYN5d9tXHVZQsnozMp2RDGsJxxHZqgN
/hG/EaWe/B1M3XRb+1FV/PjrRZXONdtDfJAw5uZtJ3eUx9tVUjtb9B0CgYEAv+wW
Xsttdw9aARcQqF9LUltpLMiHHYasYju/M2g/axyWQ6S20DVl4rZtURnbUSjn6ppV
9Zr+0puhJWwQGbqMmyDEXlV8myV5+KMM1TfmBrffwcuZLu8bJ2RKZDdxQ+HUKGWs
sxNqzdurmFxIiEjY/hg2kvZ2tJunB8lHLA7VI1kCgYAt9/rIigCa63lFqlybD1TZ
LRGhfBAknsDvJ9CCI1j3Tyd1ZGYjt3OQHPxB2K/Gb4QXZNdKrYLuBBILn+Depyu3
4twqG9G1R0yI5Xe5u9CuJwAGEvUoAr30+vGJevsX1n3CR9jGL71v3EsEOaDwTaod
b4pb4krxZPmypbFGXWj8NQKBgQCX3GaDEbKHQAV20Vpj8Ct4ek5Jmk6XhWXHwQD0
22s7BP69fYsOSwJYlwL+5lfM63I/B2o3EjLKUUz8gHpo8Vyqrw3SmxAi70+I9vOD
j1ybOkl6XfDS224ACHJ4xAoCraGjaXhypk2foE8yGutls0bIID6moRCirntHRPPl
H8N02QKBgAKc7vSuvEnmlhlMZt5bipCclTdtKbYELtbvslk2eNSqya3n3BO3t7FW
jyfvT56ZJ5UXsS0TYtq5XcVWr3ywdwsr5lwb/2lgtGv40NVfKG2OWqTY4s342HCo
GtWJ8KdHv7ZgDmkuSR/fJka8sqjTVzeWvzyKCz7kSx+K0QvIPHqj
-----END RSA PRIVATE KEY-----

And when I load that key into the ASN.1 decoder I see what I was expecting to see when I loaded the PKCS#1 structured key:

enter image description here

I guess my most pressing question is, what is the exact structure expected of PKCS#1 and PKCS#8? I've looked, but I must be searching the wrong term or something because the Mbed TLS docs I linked to earlier was the best description I've found, but at this point I'm not sure it is correct.

My second question is, do the keys I've identified in the table make sense?

Examples:

  1. Would no one ever use a PEM PKCS#8 key (OpenSSL will only output a PKCS#8 structured key it in DER format unless it is encrypted).
  2. Would a public key always be PKCS#1 format and never PKCS#8 or "traditional" format?
  3. Any other similar question that would make the format/structure combination of a key pair identified in the table nonsensical.
2525
1 + 6
dave_thompson_085 avatar
cn flag
dave_thompson_085
OpenSSL supports 'generic' PKCS8 private and X.509/PKIX/SPKI public formats for all algorithms and 'traditional' format(s) for _some_ algorithms: for RSA it uses PKCS1 both private and public, for DSA (and DH) something Eric invented for private only, for ECDSA/ECDH SECG SEC1 (aka rfc5915) for private only, and for Bernstein algorithms there is no traditional. Only private keys are encrypted (except PKCS12 which OpenSSL commandline can export or import but not use directly); PKCS8 in either PEM or DER but traditional only in PEM.
1
Reply
ubiquibacon avatar
cn flag
ubiquibacon
@dave_thompson_085 Would you agree the first pic in my OP is PKCS#8 and the last pic is PKCS#1?
0
Reply
dave_thompson_085 avatar
cn flag
dave_thompson_085
Also the default formats written vary for different subcommands in `openssl` commandline, and in some case have changed over time, so you need to be clear what version(s) you did or will use.
0
Reply
dave_thompson_085 avatar
cn flag
dave_thompson_085
Yes, and that's why the PEM labels are BEGIN/END PRIVATE KEY (= PKCS8 unencrypted) and BEGIN/END RSA PRIVATE KEY (=PKCS1, either unencrypted or encrypted but in your case unencrypted).
0
Reply
ubiquibacon avatar
cn flag
ubiquibacon
Then would you also agree that this key (though public) is also PKCS#8? https://i.stack.imgur.com/bqJq5.png
0
Reply
dave_thompson_085 avatar
cn flag
dave_thompson_085
No, the generic (= AlgId + BITSTRING) public format is as I referenced the 'SPKI' (SubjectPublicKeyInfo) portion of X.509/PKIX defined e.g. in rfc5280 4.1 and 4.1.2.7 which for RSA contains the structure that actually is PKCS1 RSAPublicKey although rfc3279 2.3.1 referenced by Maarten doesn't call it by that name.
1
Reply
Score:2
Maarten Bodewes avatar Crypto
in flag
Maarten Bodewes

There are basically four PKCS#8 formats: the unencrypted "inner" format, which is what you displayed first, then there is the binary encrypted PKCS#8 format - PEM encoded or not - and there is the SSLeay format which has the encryption details in the PEM/text instead of in the binary DER. What you've shown is the inner PKCS#8 format which also contains the privateKeyAlgorithm (in this case just the OID for rsaEncryption) key type.

You have drawn a rectangle around the PKCS#1 structure of the private key which is contained in the algorithm-agnostic PKCS#8 structure. The PKCS#1 structure is apparently what is output if you specify -traditional in OpenSSL versions 3.0 and 3.1 (in newer versions the option is not described anymore):

-traditional: When writing a private key, use the traditional PKCS#1 format instead of the PKCS#8 format.

Example 1: Would no one ever use a PEM PKCS#8 key (OpenSSL will only output a PKCS#8 structured key it in DER format unless it is encrypted).

It is a private key; generally the entity that creates it is the one to decide how you store it. Preferably you'd store it in a hardware device or an OS key container. If stored in software you'd normally store it in a PKCS#12 key store together with the certificate chain; usually we require some kind of PKI for "static" key pairs - and obviously we don't have to store ephemeral private keys.

Example 2: Would a public key always be PKCS#1 format and never PKCS#8 or "traditional" format?

PKCS#8 is literally the "Private-Key Information Syntax Specification Version 1.2". Obviously a private key is not a public key so you cannot (just) store a public key in PKCS#8 format. Usually you can extract a public key from a private key (structure) though.

When storing public keys we generally try and use the comparable Subject Public Key Info or spki structures defined in X.509 as those also contain the algorithm. Note that the inner SEQUENCE that contain the modulus and the public exponent is basically the public key structure you find in PKCS#1.

However, generally we wrap all that with a signed certificate :)

Example 3: Any other similar question that would make the format/structure combination of a key pair identified in the table nonsensical.

Nah, but I would call it the "traditional SSLeay encrypted private key format". I don't know why you would want to test that old stuff though, please don't use it. If anybody is still using it then let them generate a new key pair. There are conversion routines, but at this point you should uproot the entire thing and start over.

Beware that passwords are usually pretty weak, use a password manager and a generated password if you go for either PKCS#8 or PKCS#12 encrypted private keys. Usually the key strengthening that is used for these key container formats is not sufficiently strong.

Note that it is really easy to distinguish between PKCS#1, PKCS#8 and SPKI / X.509 formatted keys when encoded as PEM.

If it contains:

  • RSA PUBLIC KEY: this is a PKCS#1 public key;
  • RSA PRIVATE KEY: this is a Pkcs#1 private key;
  • PUBLIC KEY: this is an SPKI key, the key type is in the DER OID within it;
  • PRIVATE KEY: this is a PKCS#8 key inner structure, the key type is in the DER OID within the inner structure;
  • ENCRYPTED PRIVATE KEY: this is a PKCS#8 encrypted private key, the type is now part of the ciphertext, which in turn is surrounded by information on how to decrypt the ciphertext
  • PRIVATE KEY with additional headers before the base64 encoded data: this is the SSLeay format of the original OpenSSL authors.
+ 11
ubiquibacon avatar
cn flag
ubiquibacon
You are blowing my mind right now! Everywhere I've read online says that, by default, OpenSSL generates PKCS#1 keys in PEM format, but it is really PKCS#1 wrapped in PKCS#8 (per the link you shared)? Also, I updated my question with a link to the ASN.1 decoded public key because though you state `PKCS#8 is literally the "Private-Key Information Syntax Specification Version 1.2"`, the public key has the exact same structure, but (obviously) only contains the modulus and exponent. Can you explain why that is?
0
Reply
ubiquibacon avatar
cn flag
ubiquibacon
As for "traditional" support, I can drop that per your recommendations, but supporting it was fairly easy since it just didn't have the "outside" structure shown in my ASN.1 pic. Does the "traditional" format represent some security risk, or is there some other reason you recommend against using it? I guess keep in mind too that I'm trying to make a library that offers as much generic support as cheaply as possible, without compromising security.
0
Reply
Maarten Bodewes avatar
in flag
Maarten Bodewes
More complexity without additional benefit is by definition detrimental to security if you ask me. If you allow all kinds of stupid things you get Internet Explorer. I've updated my information w.r.t. PEM headers. Uh, yeah, both are just PKCS#1 structures, why wouldn't the public key and private key use approximately the same structure?
0
Reply
ubiquibacon avatar
cn flag
ubiquibacon
I guess it is just confusing to me, because having created several file and data formats throughout the years, I would never create a structure which we refer to as PKCS#1 which is really the inner structure of another format (PKCS#8), but we don't call it that. Seems it should be one, or the other, not somehow both.
0
Reply
Maarten Bodewes avatar
in flag
Maarten Bodewes
PKCS#1 , i.e. the first Public Key Cryptography Standard (PKCS, originally by RSA Laboratories) obviously preceded PKCS#8. PKCS#8 relies on standards like PKCS#1 and similar ANSI defined standards for Elliptic Curve keys. As far as I understood, ASN.1 wasn't new, it was already being used by the telco insdustry.
0
Reply
ubiquibacon avatar
cn flag
ubiquibacon
Let me expand on my confusion: `openssl genrsa -out private.key 2048` gives a private key in PKCS#8 "inner" format (is it PKCS#1, PKCS#8, both?, neither?). Now run `openssl pkcs8 -in private.key -out private_pkcs8.key` which, because of the lack of the `-topk8` option expects a PKCS#8 key input. The command doesn't fail, but the output is exactly the same as the input, not the PKCS#1 key I expected. I check my sanity by running `openssl pkcs8 -topk8 -in private.key -out private_pkcs8-2.key -nocrypt` which specifies the input is not a PKCS#8 key, but the output is again the same as the input.
0
Reply
ubiquibacon avatar
cn flag
ubiquibacon
I think my comment above shows something is not right. OpenSSL is not doing what its documentation states it should be doing, OR there are some shenanigans' going on with formats, or both, or I'm crazy.
0
Reply
Maarten Bodewes avatar
in flag
Maarten Bodewes
I'm not sure what you are talking about. You expect a command `topk8`, i.e. which means "to PKCS#8" to output PKCS#1? Or you think that the latter command will not work even if the structure is clearly a PKCS#8 data structure? I've got the feeling that you are kind of looking *for* problems instead of solving them. What's your goal?
0
Reply
ubiquibacon avatar
cn flag
ubiquibacon
I'm really just trying to wrap my head around what seems bad nomenclature so that I may give my functions names that make sense (i.e. I don't want to name a function `load_pkcs1` if what it is really doing is loading PKCS#8). As far as `-topk8` goes, if it is specified, the docs indicate a key will be converted from some format that is not PKCS#8 to PKCS#8, but the command never fails when given a private key which (based on what you have said) is PKCS#1. With that key, the output of `openssl pkcs8` commands is always the same as the input. That makes no sense unless PKCS#1 and 8 are the same.
0
Reply
ubiquibacon avatar
cn flag
ubiquibacon
Let us [continue this discussion in chat](https://chat.stackexchange.com/rooms/141792/discussion-between-ubiquibacon-and-maarten-bodewes).
0
Reply
dave_thompson_085 avatar
cn flag
dave_thompson_085
PEM BEGIN/END 'PRIVATE KEY' is always unencrypted (5208 sec5) and 'ENCRYPTED PRIVATE KEY' is always encrypted (sec 6); see rfc7468. The PEM-level encryption with additional headers applies only to traditional: BEGIN/END '{RSA,DSA,DH,EC} PRIVATE KEY' but not 'PRIVATE KEY' or 'ENCRYPTED PRIVATE KEY', and is indeed one of the awfulest PBEs around (EVP_BytesToKey with MD5 and ONE iteration).
2
Reply
ธงชาติไทย
Elon Musk
I sit in a Tesla and translated this thread with Ai:

mangohost

Post an answer

Most people don’t grasp that asking a lot of questions unlocks learning and improves interpersonal bonding. In Alison’s studies, for example, though people could accurately recall how many questions had been asked in their conversations, they didn’t intuit the link between questions and liking. Across four studies, in which participants were engaged in conversations themselves or read transcripts of others’ conversations, people tended not to realize that question asking would influence—or had influenced—the level of amity between the conversationalists.