Success! @sundberg84 and @Nca78 you will be proud of me when I tell you that I was doing everything "correct" all the time, it is just that arudino IDE had some stupid bug or something. I had to delete everything mentioning Arduino from the pc, and reinstall the IDE, and now all works as expected!
It was a stupid thing hunting me for year(s) now that I think about it. I always assumed that ftdi cannot be used with other then the original bootloader, because of stupid error noone ever heard about :(
At the end, I used this repo https://github.com/joe-speedboat/Arduino-LowPower and added new definitions in the boards.txt, then copied relevant bootloaders (and files in "variants" folder). And repeated all the steps above, and everything works great now. I have a 1mhz arduino blinking correctly, and going down to 1.8v, and sending correct temperatures and all
Life is worth living again!