Touch Synth WIP: Atmega328pb unstable outputting audio over PWM into a speaker

[chiptune]

The A-Star board has a 4.7uF decoupling capacitor - would that make that much of a difference?

If you mean C3 4.7 μF and R2 (1 Ω), then they are probably a requirement of the specific LDO datasheet (I can't see the LDO part number), as it specifies (I suspect) a minimum ESR (equivalent series resistance, some types of capacitor can have a value, which is too low, to work well), for that capacitor, to ensure it damps down oscillations (for stability).

So, it shouldn't matter, as you are not using an LDO.

In your PCB layout.  Are things like the crystal, its supporting (load) capacitors and the decoupling capacitors, very, very close to the IC pin(s), that need them?

Did you source all your parts from reliable sources?

I.e. Not from AliExpress/Ebay Chinese sellers, with x10 .. x50 parts for $0.25 total, for things that normally cost at least $2 each, everywhere else?
(Because of possible, fakes, rejected components, substandard-clones, reused components, faulty parts, etc).

Very, very close, on both the MCU and the amp. All my parts are from LCSC, the parts provider for JLCPCB.

I am now wondering specifically about the value 4.7uF because I can see it on the Nano schematic as well https://www.arduino.cc/en/uploads/Main/Arduino_Nano-Rev3.2-SCH.pdf

One last update: No more issues when I disable the crystal and use the internal oscillator instead (but of course the audio quality is now terrible). I am not sure what to make of that.

Should I have a ground plane behind a 20MHz oscillator and its copper traces?
Would the 4.7uF polarized capacitor have helped?

[MK14]

I'd suggest posting the PCB layouts.  Then people here can check it out, especially as regards things like the crystal section, missing or poorly placed decoupling capacitors (as already mentioned by another poster), and other things.

[chiptune]

I'd suggest posting the PCB layouts.  Then people here can check it out, especially as regards things like the crystal section, missing or poorly placed decoupling capacitors (as already mentioned by another poster), and other things.

I went over my PCB design and evaluated it against various posted guidelines and two things are clear violations from what I can tell:
1) I used vias on my crystal signal lines, didn't cross the audio output though
2) I violated the length tolerance for the audio differential pair - not sure I can do better because of the switching jack
3) not clear: I have a ground plane under the crystal - lots of differences of opinion here

What I don't understand is why I need the combination of 3 things to fail:
1) unpowered speaker
2) using the external crystal
3) high volume PWM output - either through the pot or through the code

It must be all 3 happening for things to fail - I can see how 2) and 3) together might cause interference, but I don't see what role 1) would play here.