Not near a cad program at the moment, but try the following:
Connect output of U8 / capacitor C34 pin 2, to pin 1 of D1, connect J1 pin 4 (vcc) to D1 pin 2, and attach your +5V to D1 pin 3. This way the source with the highest voltage will supply your circuit.. The actual VCC will be Vin - drop voltage over D1 (0.3V usually), that is 4.7V in your example. But that is next to nothing..