One simple solution that might work but won't reduce on space that much is to get a set of modules that suit your purpose, take a close look at the PCB layout and copy that exactly onto your own PCB design, de-solder the components from the module and reuse them on your own PCB. That way there is a higher certainty that it works, but still no guarantee. Not very handy for serial production, but as others stated, a new product that uses radio waves needs certification before it can be legally sold.
The field of electronics is very wide, ranging from analog to digital, high/low voltage, high/low current, high/low frequency, making it very difficult to shine in every aspect of it. I myself are more a programmer, but have learned the basics of electronics and forgot a lot of it over the many years. I'm getting back into the game, but know my limitations. RF is a very difficult field that I rather leave to the experts, but still like to do remote control stuff. For that I just buy ready made modules known to be working and connect them to microcontrollers. Even when designing my own PCB's, I rather put a connector on it to connect the module to then put the components directly on the PCB and have to redo it several times because I missed some specific aspect about RF.
So if you really want to learn about RF, I applaud you, but if it is just about this simple project, I would say stick to the modules and call it the day. Otherwise you should start at the beginning and get some books about basic RF design and what it all entails.