If the source IP address is used for authentication then the attacker can use the one-sided communication to break into the server