How do I sign my own SSL certificates so that they cover www and non-www domain names?

When using a commercial Certificate Authority, generating a csr for the common name www.mysite.com and sending it to them will result in a certificate being issued that works for both www.mysite.com and mysite.com.

The signing request is a single name request- just www.mysite.com, so nothing special happens at the csr level:

openssl genrsa -des3 -out mysite.com.key 4096

openssl req -new -key mysite.com.key -out mysite.com.csr
common name, ie your name: www.mysite.com

But what comes back from the commercial CA is a certificate that works on both www and non-www.

Question: How can I take a csr that is just for www.mysite.com and, using openssl with my own certificate authority, issue a certificate that works for both www.mysite.com and mysite.com, just like the commercial companies do?

I know you can modify the csr to add multiple domains with a config file, but only the www version is needed in the csr when using a commercial company. No multi-domain config files are necessary.

Are the commercial CAs modifying the submitted csr to include both versions? Or is there a flag in the signing command that makes the www optional?

Can I modify this command to add both www and non-www versions, without changing the csr?

openssl x509 -req -days 365 -in mysite.com.csr -CA Authority.crt -CAkey Authority.key -set_serial 12345 -out mysite.com.crt

Or is there a simple way to add a second domain to a csr without a config file?

openssl req -new -key mysite.com.key -out mysite.com.csr
common name, ie your name: mysite.com, www.mysite.com

The CSR doesn't need to be modified. The certificate is not the CSR, is not a copy of the CSR, and is not created by signing the CSR. The cert is a new object that contains some data copied from the CSR, and some new data.

In particular, 'commercial' CAs (LetsEncrypt and CACert among others don't charge money, so aren't really commercial) set the SubjectAlternativeName (SAN) extension (in the cert) to support multiple 'DNS' names (it can support other types of names also, but you don't want that here). The CommonName attribute in the Subject name must not contain multiple domain names, because that can't be matched.

While OpenSSL can do SAN several ways, using openssl x509 -req -CA/CAkey makes it simpler: the only option there is to specify an -extfile that contains the desired settings. This may need to be an actual file you explicitly create, or on Unix in some shells you can use the <(command) syntax to create a hidden temporary file. See:
Chrome not trusting self-signed cert
Chrome desktop and android refuses to accept trusted self-signed cert
https://stackoverflow.com/questions/61125319/node-js-self-signed-certificate-is-still-showing-as-not-trusted-in-my-browser
and many more linked at each. (Many involve Chrome because Chrome was a major reason to use SAN, but the methods to implement SAN don't depend on the reason.)