I just finished a project with the exact same scenario. 1 master and 3 slaves in my case. I used the device UART but interfaced them with RS485 differential full duplex transceivers. The master had transmitter and receiver always enabled. The slaves had their receivers enabled and transmitter under uC control.
The master would send out a packet consisting of device destination, number of bytes to follow, and the message. If the device destination matched one of the slave devices, it would enable its transmitter, respond, and then disable the transmitter. All slaves would receive the packet from the master and would ignore it if the device destination byte didn't match.