my question starts here I saw many, many websites for setting up SSH via a public key and private keys (including ssh.com, windows website, raspberry pi documentation(as my server)). But I didn't see any Source that says, put the ssh public key of the server to the user's machine.
RFC 4251
"4. Architecture
4.1 Host Keys
Each server host SHOULD have a host key. Hosts MAY have multiple host keys using multiple different algorithms. Multiple hosts MAY share the same host key.
[...]
The server host key is used during key exchange to verify that the client is really talking to the correct server. For this to be possible, the client must have a priori knowledge of the server's public host key.
Two different trust models can be used:
o The client has a local database that associates each host name (as typed by the user) with the corresponding public host key. This method requires no centrally administered infrastructure, and no third-party coordination. The downside is that the database of name-to-key associations may become burdensome to maintain.
o The host name-to-key association is certified by a trusted certification authority (CA). The client only knows the CA root key, and can verify the validity of all host keys certified by accepted CAs.
The second alternative eases the maintenance problem, since ideally only a single CA key needs to be securely stored on the client. On the other hand, each host key must be appropriately certified by a central authority before authorization is possible. Also, a lot of trust is placed on the central infrastructure."
https://www.rfc-editor.org/rfc/rfc4251#page-4