I do a bit of work on multi bit CNC drilling machines. These things can easily pack up with chips or saw dust if something is off. It's often due to the natural resins or binders in the wood melting and then building up on the bit. Anything you can do to reduce temperature is good. Maybe drill slower and let the bit cut more rather than wear its way thru.
Twist drills aren't the best geometry for wood - it's better to go to brad point boring bits. Here's an example for a high volume machine:
https://www.amanatool.com/products/boring-drilling-bits/brad-point-boring-bits.html If the countersink you have is a collar, you should be able to find a brad point bit to replace the twist drill. It probably won't have the reduced diameter the boring bit has.
Compressed Air is good - most CNC wood drilling machines have some sort of air blast. Even a vacuum nozzle helps because it takes away debris so there isn't recutting and it can provide some cooling air.
If there isn't too much build up, DriCote can help:
https://www.rockler.com/bostik-dricote-aerosol-spray If you can get silicone grease or other lube, it can delay buildup. Silicone is nice because it usually sticks well to metal, but nothing sticks to it. Dino based lubes can work too if you can't get silicone.
A small metal welder's brush can be used to clean the bit before it gets too bad:
https://www.usaweld.com/Stainless-Steel-Wire-Brush-p/wirebrushes.htm These things are about the size of tooth brushes and usually have twisted wire bristles. Mineral spirits or turpentine is a good cleaner. After cleaning, put on the lube.
Best o' luck,