I'm no expert in feedback loop stability issues, but it seems to me that C3 that you removed is drawn wrong, it should be from amp + input to ground, i.e., to form a low-pass RC filter with R2 to limit the audio bandwidth getting in. With unlimited bandwidth and a lot of gain, any noise can start the oscillation.
Secondly, I think that integration action in the feedback loop, i.e., reducing voltage gain on high frequencies, would increase stability. C6 does just that, but R5 is too high for it to work properly.
I found out the hard way with such an integrated amp that even the exact application schematic and even the PCB layout supplied in the datasheet was completely unstable and oscillating in the megahertz range, causing the integrated protection to pop. Adding these two small caps, one at input and one in the feedback (without a series resistor like R5 at all) solved the problem.