The performance loss can be substantial on modern NVMe drives, up to 20 times slower. But I was wrong about the reason for the performance loss, it's not TRIM but how VeraCrypt handles I/O operations. You can see some numbers real numbers in this Github issue: https://github.com/veracrypt/VeraCrypt/issues/136
first of all trim only affects write speed (somewhat), which is not really all that important for non-server use.
it also has some impact on wear which is probably more interesting than its performance impact.