Learning to Sit with Noise
When Radiohead recorded Kid A, they spent months trying to write a proper follow-up to "OK Computer" and kept scrapping everything. What they ended up with sounds unfinished, and really, it is.
Remove the distortion and you'd have a different album, probably a worse one. There's no "clean" version underneath.
Compulsions and Certainty
In most things worth paying attention to, the signal and noise aren't separable. What you'd throw out often has the thing you actually need, and stripping it out just leaves you with something uninteresting.
I do the same thing with code, spending hours refactoring things that already work, looking for a version that "feels" right. More than once I've made working code worse by trying to make it cleaner. For me, that discomfort with leaving things alone probably means it's time to stop. But the compulsion to fix comes from wanting certainty, a feeling that the right configuration has been reached.
Ironically, "Everything In Its Right Place" sounds like everything is in the wrong place.
The longer you sit with it, the less it feels like noise though; eventually it sounds alright.