How do I make sure my headphones are configured correctly? – Naughty Dog

If you are using stereo (not surround) sound headphones make sure that the game is configured for stereo headphones mode. This is not automatic if they have been plugged into a TV or receiver or external headphone amp connected to the optical output instead of USB or the DualShock controller. The localization of sound is not accurate for headphones if this is not set correctly.

The setting can be found by going to Options → Audio → Device → Stereo Headphones

Users of surround headphones that support 5.1 from the optical output or 5.1/7.1/3D from a USB adapter, should leave the Channels option in 5.1/7.1/3D. Those types of headphones have built-in HRTF processing that only works if you leave the game in surround headphone mode. If your game is not displaying these options then your headphones are not reading as Surround or 3D.