I'm sorry to let you know that you are using Atmega328p outside the manufacturer-guaranteed specifications.
My apology again
But your super "Whisper NODE AVR" with onboard booster is very nice, but this booster "boost" to 3,3V.
And 16Mhz is out of range for ATmega328p at 3.3V
It wants a lot of courage to sell such a product for boys from Wisen
I believe that it works in most cases, but there can be situations - noise, EMI, when a problem occurs for this reason.
Something like PRO-MINI with 8MHz is safer for me and I do not have problems with random events.