The pinchange interupt works fine.
I found the offending things on my board. It was a combination of a couple of things. First it was the BME280 that was constantly active. Change to forced mode dropped the consumption from 0.4mA to 0.1 mA. I stopped using the internal pullup for my windspeedsensor which is based on a reedswitch. I noticed when the switch was open the consumption dropped to 0.04mA so I mounted a 1Mohm resistor instead of the internal pullup. Turning off the ADC dropped a about 10-20 uA. And lastly it seems that my nimh batterys where bad. They could not keep the charge. Changed to 2 x aaa and now it looks better.
Current now on average 0.035 mA in sleepmode
I will buy new nimh's to see if my solar panel circuit will work as expected. I hope to keep it running for about 5 years without changing the battery.
Only time will tell now.