Well, when you connect to a computer, you are going to need a USB to serial adapter anyway, because modern computers aren't going to have enough serial ports to be useful. A USB serial chip is expensive, but cheaper than an external adapter and gets you power, too. For lower power devices that could save you an extra power supply, which will more than make up for the cost. For internal communication, I would use I2C rather than 5V RS232.
Is your BOM really $15 when you include enclosure, power supply, cables, and connectors? Or does that just count the PCB and the components mounted on it? These costs tend to dominate in really simple devices. If you can increase the integration level somewhat, you get more bang for your buck out of the mechanical and power components, and also reduce the fraction of the cost spend on USB to serial adapters. You loose some flexibility by including more in a single instrument, but also reduce complexity and cable clutter.
This is all very generally speaking, if you can provide some specifics of what you would like to build, how you want to use it, and how many units you plan to make, we might be able to help more.
Finally, when looking at the cost of the USB to serial converter, remember that it is allowing you to completely eliminate the cost of front panel displays and controls. A few dollars spent making sure that you can talk to it easily is money well spent.