If i was going to make it i'd probably put one of those 2x2mm Atmel microcontrollers under each led and inside each switch. Switch --> led communication could be done with led pulses too short to be visible.
In fact, a tiny micro doing software definition radio transmission could make some interesting layout possible