Let's back up and look at this again. First, the schematic I posted earlier is not correct. Looking at your pictures, it's clear that pin 8 (VBUS) of the 2104 is connected to the USB power line, and pin 7 (REGIN) is directly connected to VBUS. However, it's not clear that pin 6 (VDD) is connected to anything except the capacitors. It is not connected to the module's 3v3 pin. So it appears the 2104 draws current only when USB is connected. If that's true, it would not account for your sleep current.
However, on my copy of this module, pin 5 (VIO) is directly connected to the 3v3 pin of the module. So you have a situation where the 2104 is powered down, but 3.3V is being applied to VIO. I don't know if the datasheet tells us what happens in that circumstance. In addition, there is the issue of the ESP pins which are connected to the 2104, but I think you can control the state of those pins during deep sleep in software (but it's complicated).
If VIO is a problem, one possibility would be to cut its connection to 3v3, and connect it instead to VDD. But that's just a guess. I'll try to measure current through VIO when 3.3V is applied and everything else is off.