Sure. I think that the android hardware developer kits give these options as well. Although using them as-is, seems to be a bit overkill.
So I could get away with using a processor (i.MX6UL or similar that could house the kernel) + some graphics chipset (to run the LCD). Sure, the software (blank Android) might be able to access more, but since there would be no hardware, some (or significant) software modification (disabled features) would be needed. But currently it seems, that's pretty much all what is needed...
Starting to wonder if it's maybe easier to make my own UI on some Graphic LCD, instead of trying to use Android for the interface.