So to sum it up.
ICSP is a connector that brings together several wires that are needed to program a microcontroller with a programmer. Usually, the minimum amount is voltage, ground, reset signal, data and clock.
A programmer powers up the microcontroller, sends a reset signal and then sends data which gets written in the onboard memory.
Arduino has that connector in case you want to write the programs directly to it using a programmer.
If you write the programs through USB, what actually happens is that there is already code (a firmware) inside the Arduino microcontroller which was put there using ICSP when the product was manufactured and this code takes what you give it through usb, writes it in the memory in the microcontroller and then resets itself to allow your program to run.
The firmware and the Arduino software sending data through usb to the arduino are designed in such way designed that the firmware remains in the Arduino and always starts first, and then gives up control to your program. Think of it like the microcontroller having 16 KB of disk space and the first 1 KB of disk space holds that firmware and your program is always written after the first 1 KB.
If you actually want to overwrite that firmware and write data into ALL the Arduino memory, you have to use that ICSP connector and a programmer to actually write data (almost) anywhere on the chip flash memory.
ICSP is just a method of programming chips, there's other methods like JTAG, aWire etc
Here's more info about ICSP (In Circuit Serial Programming) :
http://en.wikipedia.org/wiki/In-circuit_serial_programminghttp://en.wikipedia.org/wiki/In-system_programming