Decoupling capacitors. The traces delivering power to the chip will have some inductance, so if it takes a sudden gulp of current, the voltage at the power pin will drop. A capacitor is put very close to each power pin to provide a local store of power to counteract this.
Another reason to parallel capacitors is that smaller capacitors typically perform better at high frequency. If you need 100uF, but you need it to work up to high frequency, you'll have to put some small ceramic capacitors next to the big electrolytic capacitor.