Install and initialize the services as root:

pkg install ccid opensc pcsc-lite
sysrc pcscd
service pcscd restart

Start ssh agent and add provider to it as user:

eval $(ssh-agent)
ssh-add -s /usr/local/lib/opensc-pkcs11.so

That’s it, SSH should work with YubiKey now.