As you've already tested that the hardware is not broken by running the system with another OS, you can probably fix the issue by adjusting hardware mixer settings. I prefer to use alsamixer for this because it has all the settings even though the UI is a bit weird.
Basically open a terminal, run alsamixer (if that's missing, you can install it with command sudo apt install alsa-utils).
Once the alsamixer has started, press F6 to select the soundcard your headphones are connected to, the press F5 to make all channels and settings visible and start adjusting channels one by one until you hear a change in the noise.
For every channel you can use key M to toggle mute, space to switch capture source and up arrow / down arrow to adjust the channel volume or enable/disable feature. The capture flag is supposed to select only the recording source(s) but some audio hardware has weird interlinked limitations between input and output channels and changing the recording sources may affect the sound output, too.
It's a good idea to restore the same settings for each channel if that channel doesn't seem to affect the problem you're experiencing.
Some features and channel names will have cryptic values because most Linux audio drivers have been created via reverse engineering and officially marketed names are typically not known to the device driver developers.
I'd guess your problem is that some analog input source is not muted but the only way to figure it out is to try one-by-one until you find the culprit.
You can exit alsamixer by pressing ESC.
When you shutdown your system normally, the current hardware state should be stored and restored during next boot so you should only need to do this once.
Other people may recommend using pavucontrol but that doesn't adjust the hardware directly but via pulseaudio API which may hide some settings that alsamixer can adjust. If you think alsamixer is too hard to use, you can try with pavucontrol first, if you find that that easier to use. In most cases the pavucontrol is good enough.
(Also note that if your audio device is based on Intel HDA chipset, it supports reconfiguring the chip to use any soldered plug as input and output as you wish. In theory, it's possible to configure the chipset (accidentally) so that it uses any plugin as microphone source, no matter what the hardware says in the plugs.)
And if your audio setup is totally messed up, you could try removing /var/lib/alsa/asound.state after booting from USB memory stick to make sure the previously stored hardware configuration is not messing things up. That file contains (in both machine and human readable format) the current settings of the hardware at the last system shutdown.
