Bootloading a barebones arduino
-
@OldSurferDude said in Bootloading a barebones arduino:
...you got me thinking...
This forum is built for that.
Thank you for posting and the link to Arduino Bootloaders. I read that again. In summary of my reading, and in my own words: Bootloaders are the gravel under pavement. So I pose this as a question: If one uses "Sketch/Upload Using Programmer" it will load the program without the bootloader, right? Is that your understanding? Kind of like all pavement, no gravel.
Over several of these exercises bootloading and programming with ICSP's using the Arduino IDE I've noticed that once a program is loaded with the ICSP using "Sketch/Upload Using Programmer", no further USB-TTL (USART) programming can occur until "Tools/Burn Bootloader" is subsequently executed. So in my limited experience, the bootloader is the gateway to normal USART programming. Too many words... I know. But the nice thing about the link you provide is that the Bootloader program is only 512 bytes. So one shouldn't fret over economizing over space.
@OldSurferDude said in Bootloading a barebones arduino:
...skill and experience...
I assume the skill that is required to remove the bootloaded 328P's from the Nano is a higher challenge. Soldering a new chip, as I did, has to be easy-peasy compared to removing and resoldering an existing chip. As it was, I think I killed my new chips because of the soldering & bridge removal. More on that to come in my next post.
@OldSurferDude said in Bootloading a barebones arduino:
...pretty awesome and I wish you success
Kind of you. At this point, I'm kind of a dog with a bone. Given the contributions of NeverDie, AlphaHotel and others, I just have to press on or all the time & effort is for not. Perhaps the dialog is of use to others.
@Larson said in Bootloading a barebones arduino:
I assume the skill that is required to remove the bootloaded 328P's from the Nano is a higher challenge. Soldering a new chip, as I did, has to be easy-peasy compared to removing and resoldering an existing chip.
Honestly, you'll be surprised how easy it is to desolder a 328P if you use enough Chipquik. You basically lower the melting point to where all 32 pins are in liquid metal all at the same time, and then you just push it aside. I'm sure there are other techniques as well that don't rely on Chipquik, but this method is very easy. If it's still sticking, melt more Chipquik onto the pads and keep everything heated until the whole thing is swimming in liquid chipquik. If you follow that heuristic, you can't fail.
-
@Larson said in Bootloading a barebones arduino:
I assume the skill that is required to remove the bootloaded 328P's from the Nano is a higher challenge. Soldering a new chip, as I did, has to be easy-peasy compared to removing and resoldering an existing chip.
Honestly, you'll be surprised how easy it is to desolder a 328P if you use enough Chipquik. You basically lower the melting point to where all 32 pins are in liquid metal all at the same time, and then you just push it aside. I'm sure there are other techniques as well that don't rely on Chipquik, but this method is very easy. If it's still sticking, melt more Chipquik onto the pads and keep everything heated until the whole thing is swimming in liquid chipquik. If you follow that heuristic, you can't fail.
@NeverDie said in Bootloading a barebones arduino:
Honestly, you'll be surprised how easy it is to desolder a 328P if you use enough Chipquik
I"ll give it a go. What can I lose? Good excuse to use my Chipquick kit that arrived some time ago. I'm going to use my heavier iron. I won’t reinstall another 328P until the clam shell device arrives.
-
@Larson said in Bootloading a barebones arduino:
I assume the skill that is required to remove the bootloaded 328P's from the Nano is a higher challenge. Soldering a new chip, as I did, has to be easy-peasy compared to removing and resoldering an existing chip.
Honestly, you'll be surprised how easy it is to desolder a 328P if you use enough Chipquik. You basically lower the melting point to where all 32 pins are in liquid metal all at the same time, and then you just push it aside. I'm sure there are other techniques as well that don't rely on Chipquik, but this method is very easy. If it's still sticking, melt more Chipquik onto the pads and keep everything heated until the whole thing is swimming in liquid chipquik. If you follow that heuristic, you can't fail.
@NeverDie said in Bootloading a barebones arduino:
Honestly, you'll be surprised how easy it is to desolder a 328P if you use enough Chipquik.
Before undoing my solder job on the barebones, I had to try again. I think my problem is solved. The short story is that the chips I bought are looking for a clock. (See This is a symptom of the target chip's clock not running.) Once I gave the chip a clock (thanks AlphaHotel for the insight of LED driver pins 20/21 hack) wala ... I'm now talking to the chip in CMD terminal. More to come.
DogWithABone
-
@OldSurferDude said in Bootloading a barebones arduino:
...you got me thinking...
This forum is built for that.
Thank you for posting and the link to Arduino Bootloaders. I read that again. In summary of my reading, and in my own words: Bootloaders are the gravel under pavement. So I pose this as a question: If one uses "Sketch/Upload Using Programmer" it will load the program without the bootloader, right? Is that your understanding? Kind of like all pavement, no gravel.
Over several of these exercises bootloading and programming with ICSP's using the Arduino IDE I've noticed that once a program is loaded with the ICSP using "Sketch/Upload Using Programmer", no further USB-TTL (USART) programming can occur until "Tools/Burn Bootloader" is subsequently executed. So in my limited experience, the bootloader is the gateway to normal USART programming. Too many words... I know. But the nice thing about the link you provide is that the Bootloader program is only 512 bytes. So one shouldn't fret over economizing over space.
@OldSurferDude said in Bootloading a barebones arduino:
...skill and experience...
I assume the skill that is required to remove the bootloaded 328P's from the Nano is a higher challenge. Soldering a new chip, as I did, has to be easy-peasy compared to removing and resoldering an existing chip. As it was, I think I killed my new chips because of the soldering & bridge removal. More on that to come in my next post.
@OldSurferDude said in Bootloading a barebones arduino:
...pretty awesome and I wish you success
Kind of you. At this point, I'm kind of a dog with a bone. Given the contributions of NeverDie, AlphaHotel and others, I just have to press on or all the time & effort is for not. Perhaps the dialog is of use to others.
@Larson said in Bootloading a barebones arduino:
once a program is loaded with the ICSP using "Sketch/Upload Using Programmer", no further USB-TTL (USART) programming can occur until "Tools/Burn Bootloader" is subsequently executed.
That is exactly right! Once you have all the wiring (I made up a wire harness for just that purpose) the processes are equally easy.
-
@NeverDie said in Bootloading a barebones arduino:
Honestly, you'll be surprised how easy it is to desolder a 328P if you use enough Chipquik.
Before undoing my solder job on the barebones, I had to try again. I think my problem is solved. The short story is that the chips I bought are looking for a clock. (See This is a symptom of the target chip's clock not running.) Once I gave the chip a clock (thanks AlphaHotel for the insight of LED driver pins 20/21 hack) wala ... I'm now talking to the chip in CMD terminal. More to come.
DogWithABone
@Larson said in Bootloading a barebones arduino:
I'm now talking to the chip in CMD terminal.
Well done! Let us know how it goes from here.
AH
-
@alphaHotel and @NeverDie
It has been a long journey. Since being able to hack my barebones with an external clock onto the D20 and D21 nets, I’ve been able to talk to my barebones MCU in CMD window and Arduino IDE. Once talking, then I could bootload using internal crystal. But the major breakthrough today was trying to get the LEDs blinking. That brought me back to post 110 of https://forum.mysensors.org/topic/11917/anyone-using-tried-the-e28-2g4m27s-2-4ghz-lora-sx1280-27db-module/110?_=1659129333013. And here I re-learn that I’ve been on the wrong path along. After installing MCUDude in board manager, bootloading, and reprogramming (USART via USB-TTL) my LEDs are blinking away.
But along the way I’ve learned a couple of things:
- The barebones Atmega 328P running without an external clock can’t exactly keep up with the IDE’s baud rate of 9600. Clock accuracy, I’m told, is the cost of using the low-power internal clock setting. So after some iterative looping, I’ve discovered that coding Serial.begin (8800) works on one board, and Serial.begin(9000) works on the other. Your chip will probably be different.
- To make the crystal hack noted above, I had unsoldered the two resistors on one of my barebones. On my second board, I simply held the crystal against the resistor base (facing D20, D21) while resetting fuses to internal clock setting; it worked.
- Before finding MCUDude, I had made edits to Board.txt file and copied one of the Pro Mini boards, renamed it, re-fused it, and that worked do. But as mentioned, the LED’s didn’t.
I have been around the block a coupe of times with these boards and learned a bunch. Soon, I hope to actually mount a radio which originally looked like it would be fast work.
Thank you to the contributors and readers of this post. For me, this forum has become a great source of discussion and documentation for my own record.
-
@alphaHotel and @NeverDie
It has been a long journey. Since being able to hack my barebones with an external clock onto the D20 and D21 nets, I’ve been able to talk to my barebones MCU in CMD window and Arduino IDE. Once talking, then I could bootload using internal crystal. But the major breakthrough today was trying to get the LEDs blinking. That brought me back to post 110 of https://forum.mysensors.org/topic/11917/anyone-using-tried-the-e28-2g4m27s-2-4ghz-lora-sx1280-27db-module/110?_=1659129333013. And here I re-learn that I’ve been on the wrong path along. After installing MCUDude in board manager, bootloading, and reprogramming (USART via USB-TTL) my LEDs are blinking away.
But along the way I’ve learned a couple of things:
- The barebones Atmega 328P running without an external clock can’t exactly keep up with the IDE’s baud rate of 9600. Clock accuracy, I’m told, is the cost of using the low-power internal clock setting. So after some iterative looping, I’ve discovered that coding Serial.begin (8800) works on one board, and Serial.begin(9000) works on the other. Your chip will probably be different.
- To make the crystal hack noted above, I had unsoldered the two resistors on one of my barebones. On my second board, I simply held the crystal against the resistor base (facing D20, D21) while resetting fuses to internal clock setting; it worked.
- Before finding MCUDude, I had made edits to Board.txt file and copied one of the Pro Mini boards, renamed it, re-fused it, and that worked do. But as mentioned, the LED’s didn’t.
I have been around the block a coupe of times with these boards and learned a bunch. Soon, I hope to actually mount a radio which originally looked like it would be fast work.
Thank you to the contributors and readers of this post. For me, this forum has become a great source of discussion and documentation for my own record.
@Larson said in Bootloading a barebones arduino:
The barebones Atmega 328P running without an external clock can’t exactly keep up with the IDE’s baud rate of 9600. Clock accuracy, I’m told, is the cost of using the low-power internal clock setting. So after some iterative looping, I’ve discovered that coding Serial.begin (8800) works on one board, and Serial.begin(9000) works on the other. Your chip will probably be different.
Something is amiss then. Maybe you're running with very long wires? I can't remember having problems running UART at 115200 baud without a crystal, and that's true across many atmega328p's and across many different boards. For sure I've never had a problem at 9600 baud.
Are you using either an official FTDI or a CP2102? Either of those works great. The cheap UART alternative chips which aren't those might be a problem problem though. I avoid those.
-
@Larson said in Bootloading a barebones arduino:
The barebones Atmega 328P running without an external clock can’t exactly keep up with the IDE’s baud rate of 9600. Clock accuracy, I’m told, is the cost of using the low-power internal clock setting. So after some iterative looping, I’ve discovered that coding Serial.begin (8800) works on one board, and Serial.begin(9000) works on the other. Your chip will probably be different.
Something is amiss then. Maybe you're running with very long wires? I can't remember having problems running UART at 115200 baud without a crystal, and that's true across many atmega328p's and across many different boards. For sure I've never had a problem at 9600 baud.
Are you using either an official FTDI or a CP2102? Either of those works great. The cheap UART alternative chips which aren't those might be a problem problem though. I avoid those.
@NeverDie said in Bootloading a barebones arduino:
Something is amiss then.
Probably true, and thank you for your commenting.
I think all my USB-TTL converters are based on cheapo CH340G chips. But, to date, except for my blunders, I've not had a mismatch between the SW and FW baud rate that is so slight and I'm using the same wire length and converter as before(cheap & near, pluggable wire).
I think I misstated the idea that that the bare bones "can't exactly keep up". The reverse is the case: the barebones MCU is faster than the IDE/USB-TTL combo. None-the-less, the timing is off so that Serial.print yeilds some ASCII characters instead of numbers. Incrementally changing the baud rate in the loop enabled me to test different rates. And what the serial monitor in the IDE thinks is 9600 baud, it is best expressed when the chip hits a baud rate of about 9000. Make sense? I'm imagining that the same is true if I were at 115,200, only relatively higher.
In development, as I am, I always default to 9600 until I gain some confidence. Sometimes going slow is the source of my problem when chips are going so fast.
-
I think I know what you mean. On the nRF52805, one of the nominal UART Tx baud rates is 921,600, but the datasheet states that the "actual" baudrate at that setting is 941,176, so I set my terminal emulator to that, the actual baud, rather than the nominal rate. I don't ever remember encountering that kind of distinction on the atmega328p, but maybe there are some that I've just never run across. At slower speeds it's probably more forgiving. There's also the issue of hardware flow control, but it often isn't used by some link in the chain, which ruins it.
-
@NeverDie said in Bootloading a barebones arduino:
Something is amiss then.
Probably true, and thank you for your commenting.
I think all my USB-TTL converters are based on cheapo CH340G chips. But, to date, except for my blunders, I've not had a mismatch between the SW and FW baud rate that is so slight and I'm using the same wire length and converter as before(cheap & near, pluggable wire).
I think I misstated the idea that that the bare bones "can't exactly keep up". The reverse is the case: the barebones MCU is faster than the IDE/USB-TTL combo. None-the-less, the timing is off so that Serial.print yeilds some ASCII characters instead of numbers. Incrementally changing the baud rate in the loop enabled me to test different rates. And what the serial monitor in the IDE thinks is 9600 baud, it is best expressed when the chip hits a baud rate of about 9000. Make sense? I'm imagining that the same is true if I were at 115,200, only relatively higher.
In development, as I am, I always default to 9600 until I gain some confidence. Sometimes going slow is the source of my problem when chips are going so fast.
@Larson Thinking about it now, I suspect maybe what you're describing is why I previously couldn't get lesser UART chips "to work" and instead took refuge in, generally, a CP2102, which could maybe handle such inconsistency better at nominal baud rates. So, if that's the case, good on ya for sticking with it and figuring out how to make cheaper UART chips work. Evidently they do work after all if you dial-in the actual rather than the nominal baud rate.
As a side note, I just now looked at the CP2102 datasheet, and it finally explains why I seemingly couldn't get the nominal 1mbps baudrate out of the nRF52805, even though the nRF52805 datasheet offers that as a possible setting. It's because the CP2102 is maxed out supporting a nominal 921600. :face_palm:
With that mystery now solved, the remaining mystery is why I can't seem to get a USB virtual com port on a PC to do hardware flow control. Do I literally need a physical serial port rather than a USB to get hardware flow control working? It's breaking somewhere in the chain; I'm just not sure where. For now I've worked around the lack of proper hardware flow control at high speeds by just padding in some extra time between transmissions, which does allow communication to happen, but obviously slower than what it could be with proper RTS/CTS protocol working. Who knows? Maybe it's a factor in your situation as well, depending on what assumptions you're making or how you're configuring things. As it turns out, without HWFC, at higher baud rates you actually need less of a catch-up delay between transmissions than at lower baud rates, so running at max baudrate is a double win of sorts.
-
@Larson Thinking about it now, I suspect maybe what you're describing is why I previously couldn't get lesser UART chips "to work" and instead took refuge in, generally, a CP2102, which could maybe handle such inconsistency better at nominal baud rates. So, if that's the case, good on ya for sticking with it and figuring out how to make cheaper UART chips work. Evidently they do work after all if you dial-in the actual rather than the nominal baud rate.
As a side note, I just now looked at the CP2102 datasheet, and it finally explains why I seemingly couldn't get the nominal 1mbps baudrate out of the nRF52805, even though the nRF52805 datasheet offers that as a possible setting. It's because the CP2102 is maxed out supporting a nominal 921600. :face_palm:
With that mystery now solved, the remaining mystery is why I can't seem to get a USB virtual com port on a PC to do hardware flow control. Do I literally need a physical serial port rather than a USB to get hardware flow control working? It's breaking somewhere in the chain; I'm just not sure where. For now I've worked around the lack of proper hardware flow control at high speeds by just padding in some extra time between transmissions, which does allow communication to happen, but obviously slower than what it could be with proper RTS/CTS protocol working. Who knows? Maybe it's a factor in your situation as well, depending on what assumptions you're making or how you're configuring things. As it turns out, without HWFC, at higher baud rates you actually need less of a catch-up delay between transmissions than at lower baud rates, so running at max baudrate is a double win of sorts.
@NeverDie Talking about virtual ports, HWFC, and chain breakage is way out of my experience. But I'm impressed that you are thinking at that level.
On Baud Rate missmatch: Lacking an OScope, I suppose I could hook the PPKII to the TX of the barebones and transfere values of (2^8)-1, or 255, to see what the exact timing looks like. Serial.print(255, BIN) should do it, right? The datasheet tells me the max sampling speed of the PPKII is 100,000 S/s, so 9600 baud should show pretty clearly and enable me to fine tune what baud the MCU is generating. In a forum post I asked you what the trade-off is for using the internal crystal; you answered "accuracy". That is what got me thinking and looping on different baud rates. Serial Monitor was giving me 12#$, or the like, when I was expecting 1234 via Serial.print, so I knew it was close.
How tight are the radios to baud support they get from the 328P? I would imagine the internal crystal baud rate is dependent on voltage. Maybe some formula exists in the 328P datasheet that allows for adapting this on the fly.
-
@NeverDie Talking about virtual ports, HWFC, and chain breakage is way out of my experience. But I'm impressed that you are thinking at that level.
On Baud Rate missmatch: Lacking an OScope, I suppose I could hook the PPKII to the TX of the barebones and transfere values of (2^8)-1, or 255, to see what the exact timing looks like. Serial.print(255, BIN) should do it, right? The datasheet tells me the max sampling speed of the PPKII is 100,000 S/s, so 9600 baud should show pretty clearly and enable me to fine tune what baud the MCU is generating. In a forum post I asked you what the trade-off is for using the internal crystal; you answered "accuracy". That is what got me thinking and looping on different baud rates. Serial Monitor was giving me 12#$, or the like, when I was expecting 1234 via Serial.print, so I knew it was close.
How tight are the radios to baud support they get from the 328P? I would imagine the internal crystal baud rate is dependent on voltage. Maybe some formula exists in the 328P datasheet that allows for adapting this on the fly.
@Larson said in Bootloading a barebones arduino:
How tight are the radios to baud support they get from the 328P? I would imagine the internal crystal baud rate is dependent on voltage. Maybe some formula exists in the 328P datasheet that allows for adapting this on the fly.
The radios are connected over SPI, with the radio as slave and the atmega328p as master. The master sends out the clock signal, and the slave is literally "slave" to that clock signal, even if it isn't steady. Therefore, within normal parameters, it's not something you need to worry about, because the shared clock signal synchronizes their communication. Naturally there's some limit on how fast you could theoretically run a master's clock before it becomes too fast for the slave to follow, but AFAIK there is no limit on how slow you can run the clock. You could probably even toggle the clock pulses manually by hand if you wanted to. The slave isn't allowed to get bored and time out no matter how long it takes. The protocol simply says that whenever there is a clock transition, the next bit of data has to be ready for reading on the datalines (both MOSI and MISO). i.e. the protocol is agnostic as to the clock speed and even its consistency. You and a group of friends could sit around a table and pass bits and bytes to each other using an SPI communication protocol. Better than telephone!
-
@Larson said in Bootloading a barebones arduino:
How tight are the radios to baud support they get from the 328P? I would imagine the internal crystal baud rate is dependent on voltage. Maybe some formula exists in the 328P datasheet that allows for adapting this on the fly.
The radios are connected over SPI, with the radio as slave and the atmega328p as master. The master sends out the clock signal, and the slave is literally "slave" to that clock signal, even if it isn't steady. Therefore, within normal parameters, it's not something you need to worry about, because the shared clock signal synchronizes their communication. Naturally there's some limit on how fast you could theoretically run a master's clock before it becomes too fast for the slave to follow, but AFAIK there is no limit on how slow you can run the clock. You could probably even toggle the clock pulses manually by hand if you wanted to. The slave isn't allowed to get bored and time out no matter how long it takes. The protocol simply says that whenever there is a clock transition, the next bit of data has to be ready for reading on the datalines (both MOSI and MISO). i.e. the protocol is agnostic as to the clock speed and even its consistency. You and a group of friends could sit around a table and pass bits and bytes to each other using an SPI communication protocol. Better than telephone!
@NeverDie said in Bootloading a barebones arduino:
You and a group of friends could sit around a table and pass bits and bytes to each other using an SPI communication protocol.
Passing bits and bytes is probably more fun that sitting around a table and passing digs and barbs! I don't know: maybe my friends ARE binary!:angel:
[Edit: I forgot to say anything about the content of your SPI protocol story. That was a great story; I feel badly for not thinking about the SPI connection at all. Now it is ingrained. ]
