The good
The method in the question leads to a signature system that always successfully verifies the message it signs; and there are combinations of asymmetric encryptions schemes and hashes where that signature system meets the EUF-CMA security property, including:
- RSAES-OAEP close to as practiced, and a standard cryptographic hash like SHA-256. Two tweaks are needed to the RSA part, though:
Choosing $e$ random and of large bit size (half the bit size of $n$ seems OK) rather than $e=65537$ as common (or other lower guessable value, or random but too low for security¹ of the question's method)
Expressing the decryption key $K_{dec}$ as $(n,d)$ rather than as the usual $(n,e,d,p,q,d_p,d_q,q_\text{Inv})$, used by standard implementations that care for performance.
Both changes are necessary to keep $e$ of $K_{enc}=(n,e)$ secret when we make $K_{dec}=(n,d)$ public, as in the question.
- RSA encryption as in the original article except with $n$ large enough to resist modern factorization methods², and a hash considerably larger than above in order to resist the Desmedt and Odlyzko attack.
The bad
The method is unsafe when applied to most asymmetric cryptosystems
Observe that $K_{dec}$ must be made public since it's the verification key, and $K_{enc}$ must be kept secret since it is the signing key. Thus the relation between the two keys must be such that we can make $K_{dec}$ public while preserving the secrecy of $K_{enc}$. RSA is the only family of encryption schemes I know well that can have that property (and as explained above, it's practical implementations do not).
For many asymmetric encryption systems, the property can't be made to apply. For example, in ElGamal and it's modern descendant ECIES, $K_{dec}$ is an integer that encodes how to reach group element $K_{enc}$ by applying the internal group law to a certain public group element; therefore revealing $K_{dec}$ unavoidably reveals $K_{enc}$, breaking the security of the question's construction.
More generally, using a secure asymmetric encryption system and a secure hash is no good indication that the signature system obtained by the question's method is secure, for a variety of reasons.
When we consider implementation security, there's a further issue: an implementation of $\texttt{encrypt}$ needs no protection against side channel leakage of it's key input, since it's normally public. However such protection is required for the question's signature system.
The method tends to build signature schemes with substandard characteristics
Comparing RSA signature (including by the question's method) to EdDSA at standard security level:
- The signature is rather large (256 bytes compared to 64 bytes)
- The public key is large (typicaly 260 bytes compared to 32 bytes)
- Signature generation is slow (like dozens times slower)
- Key generation is painfully slow (like hundreds times slower).
And the method is not quite what's used to build RSA signature, even the close cousin RSA-FDH. That uses the textbook RSA private-key function $h\mapsto s=h^d$ to sign a wide hash $h=H(M)$; and the textbook RSA public-key function $s\mapsto h=s^e\bmod n$ followed by a comparison $h\overset?=H(M)$ in verification. Contrary to the question's method, the public key $(n,e)$ remains public, and the private key $(n,e,d,p,q,d_p,d_q,q_\text{Inv})$ remains secret. Compared to that, or RSASSA-PKCS1-v1_5 or RSASSA-PSS, the question's signature method:
- Signs nearly four times slower, because it can't use the $(n,e,d,p,q,d_p,d_q,q_\text{Inv})$ form for $K_{dec}$
- Verifies like a hundred to a thousand times slower, because it can't use a small $e$.
¹ With $n$ known, if one of $e$ or $d$ is known and less that $n^{0.292}$, then the other can be found, see Boneh and Durfee. Therefore the customary limit of 256-bit $e$ (e.g. built into the key generation method of FIPS 186-4) is such that revealing $d$ allows to find $e$.
² The original recommendation “that $d$ should be chosen from a large enough set so that a cryptanalyst cannot find it by direct search” then compute $e$ from $d$ was inadequate, and allowed Wiener's attack, later improved by Boneh and Durfee. But keeping $d$ secret and making $e$ public as the question's method requires fixes that issue.
