When you experience high packet losses, then the audio stream is going to suffer.  Packets that aren't available will create gaps within the audio stream.  Even if packets are retransmitted, the resent packet may arrive too late.

Packet loss is probably very low within your network.  However, packet loss from your location to your destination can happen if ISP loads and rules are forced to drop some packets because of your service tier or because of other rules.