The question is for the standard 256-bit Elliptic Curve secp256k1. It wants the key in the format described in sec1v2, section 2.3.3 Elliptic-Curve-Point-to-Octet-String Conversion, with point compression, that is per 2.1/2.2.1/2.3/2.4
The desired compressed public key starts with byte 02 or 03 (with the same parity as the $Y$ coordinate), followed by the $X$ coordinate in big-endian notation over 32 bytes. Further conversion to hexadecimal makes that a 66-character string.
To decode the DER-encoded public key SubjectPublicKeyInfo or X.509 certificate, one can use this online tool, which accepts hex and base64. As this comment points out, it will normally show the (point representative of) the public key either
- in a bitstring which data is in the uncompressed format of 65 bytes (520 bits), the first of which is
04, followed by the $X$ coordinate and the $Y$ coordinate in big-endian notation each over 32 bytes. In order to convert to compressed format, we can replace the initial 04 with 02 or 03 (determined so that it has the same parity as the last byte), and remove the last 32 bytes.
- in a bitstring which data already is in the desired compressed format of 33 bytes (264 bits), the first of which is
02or 03.
Note: In other conversion contexts I have met $X$ and $Y$ coordinates encoded as ASN.1 integer in the public key. Facing this, the textbook solution is to use an ASN.1 library. When doing without, be careful that the ASN.1 encoding of integers has variable length, so that it may be needed to remove one leading 00 (≲50% of the cases), or add one 00 (<0.4% of the cases) or even more (<0.002%).
Note: One might want to check that $X$, and $Y$ if available, form a valid secp256k1 public key, that is: for $p=2^{256}-2^{32}−977$ (the prime field order) it holds
- $0<X<p$
- if $Y$ is available, that $0<Y<p$ and $X^3+7-Y^2\bmod p=0$
- otherwise, that $(X^3+7)^{(p-1)/2}\bmod p=1$
