sftp - give password access to one user only & chroot

First I kniow this kind of question has been asked a lot, and I tried a lot of solution here and on SuperUser but none worked and I can't find why.

I want to configure my ssh to only accept key login except for one user that could login with sftp only (no ssh cli access, if possible, I'm not sure)

Here is my config

AllowUsers user1 user2
LoginGraceTime 0
PubkeyAuthentication yes
AuthorizedKeysFile      %h/.ssh/authorized_keys
PasswordAuthentication no
ChallengeResponseAuthentication no
UsePAM yes
X11Forwarding yes
UsePrivilegeSeparation sandbox
ClientAliveInterval 120
Subsystem sftp internal-sftp
Match Group sftpusers
        ChrootDirectory /home/%u
        ForceCommand internal-sftp
Match user user2
        PasswordAuthentication yes
        ChrootDirectory /home/%u

When I try to login with FileZilla with SFTP I have this log

Status: Connecting to *my_server_ip*:*my_ssh_port*...
Response:   fzSftp started, protocol_version=11
Command:    open "user2@*my_server_ip*" *my_ssh_port*
Status: Using username "user2". 
Command:    Pass: ************
Error:  Could not connect to server

When I try to connect with ssh

ssh user2@*my_server_ip* -p *my_ssh_port*
user2@*my_server_ip*'s password:
client_loop: send disconnect: Connection reset

I can connect with user1 just fine using pubkey but I can't make user2 able to login with password

For those with same problem as me, here is how I managed to chroot the sftp user in their home directory

Here is my sshd_config

Subsystem sftp internal-sftp
Match Group sftpusers
        X11Forwarding no
        AllowTcpForwarding no
        ChrootDirectory /home
        ForceCommand internal-sftp -d /%u
Match user user2
        PasswordAuthentication yes

You need to chroor /home, and set just enough permission for the connection chmod 711 /home your /home/user2 dir can have normal permission for user2 to be able to create/upload files.

With this config, the user will be moved automatically in their dir, the chroot will be in /home dir but the user won't have the permission to .. and he will be jailed in his own home dir.