SSH with ecdsa_sk key very slow confirmation

If I use the Trezor to SSH (on Linux), it takes a very long time (5+ mins) for the confirmation screen to show up on the Model T.
The Trezor suite software OTOH manages to work without delays.

It does seem like an HID timeout issue, since if I replug the device, SSH works right away.
How do I troubleshoot this?