You *ALWAYS* need the recommended load capacitance, whether as physical capacitors or as PCB trace capacitance. Also in the classic single gate Pierce oscillator configuration, the capacitors are essential to provide 180 deg phase inversion across the crystal.
With Cl=4pF, you need 8pF either end of it as the load capacitors are in series from the crystal's viewpoint. As double sided 1.5mm FR4 will give you about 5pF* per cm2 of trace area above a ground plane, neglecting capacitance between traces, that's easily achievable if you keep the crystal layout tight near the IC pins. If you come in under the target capacitance, just expand the crystal pads, but limit their solderable area with soldermask.
* varying the geometry from long and skinny to short and stubby will significantly affect this value, I got it from a microstrip calculator that includes fringing fields, using a 2mm trace width. Your crystal traces will be mush narrower but 2mm isn't unreasonable as an average including the pads.