Best PC platform for running Esxi/Docker at home?
-
I get that efficieny is important for a device that runs 24/7, but looking at the options you have with a 5800U, I think you sacrifice a lot. How do you add hard drives for storage, via USB3? What about RAID? How do you know if the laptop's thermal design is sufficient for 24/7 uptime?
Do not focus too much on the nominal TDP. It is basically just an approximation for the cooling solution required to achieve the rated performance of the CPU. In the specific case of AMD Ryzen CPUs, the rated TDP roughly equals the power draw in P0 state, which is the highest active power state the CPU can be in.
For example, a Ryzen 5 5600X rated at 65W may draw 60 to 75W (once the thermal limit is reached and everything is settled in) under high demanding workloads (prime95 and such). Modern CPUs are smart enough to switch to quite efficient P-states when they have nothing to do, lowering the clock to the sub-GHz range, drastically reducing the core voltage. So if the CPU is idling, it may only draw 10 to 15% of the rated TDP - and a home server will likely idle 95% of the time.
At this point, other components (mainboard chipset, hard drives and other peripherals, power losses of the power supply, ...) might start to contribute more to the total power consumption of the PC than the CPU itself. And I haven't even mentioned that you could underclock (which doesn't void warranty) the CPU the increase the efficiency even further.
@BearWithBeard said in Best PC platform for running Esxi/Docker at home?:
Modern CPUs are smart enough to switch to quite efficient P-states when they have nothing to do, lowering the clock to the sub-GHz range, drastically reducing the core voltage. So if the CPU is idling, it may only draw 10 to 15% of the rated TDP - and a home server will likely idle 95% of the time.
I can believe that may be true of some CPU's, but I wonder if that really does hold true for the Ryzen 5 5600X. Instead of a min, max, and "default" TDP, it's specs list just one TDP at 65w:
https://www.amd.com/en/products/cpu/amd-ryzen-5-5600xIn contrast, the ryzen 7 5800u gives a range of 10-25w, with a "default" of 15w:
https://www.amd.com/en/products/apu/amd-ryzen-7-5800uThat said, maybe there's a better place I could look where I could confirm that it draws only 10 to 15% of its nominal power draw while idling?
I do very much like that the Ryzen 5 5600X has a very good single-thread passmark score for its roughly $299 price, which is quite favorable compared to much of its competition. If its power draw really were 10-15% of its nominal 65w about 95% of the time, then I think it would be a great choice. But how to confirm that without actually buying it and testing it? Surely it must be documented somewhere. But where exactly?
-
The cTDP is generally only provided for APUs (CPU + GPU on a single chip). It allows OEMs to easily limit the power draw (mainly the all core boost clock) to match the CPU to their thermal design requirements. That's why two equally equipped notebooks can have vastly different performance results. It's a pity that OEMs almost never advertise their cTDP setting.
The desktop CPU market is a little different. Thermal design isn't that critical, CPUs are also sold directly to end users, etc. While it may depend on the mainboard manufacturer to enable the cTDP setting in their UEFI, it is generally possible to adjust it on desktop Ryzens as well. Many do that to find their efficiency sweet spot. I guess nothing could stop you from setting a 5600X's cTDP to 15W. That's what I did with my Athlon as well, albeit only from 25W to 15W. Besides the cTDP, there are other ways to increase efficiency: Lowering the PPT threshold, setting a negative vCore offset, etc. But that whole topic is too complex to explain or discuss here. You may search on PC hardware websites like AnandTech, Tom's Hardware, GamersNexus, Guru3D and such for more details if you're interested.
Here's a 5600X review by Tom's Hardware. They suggest that it draws 13W in idle at stock settings, which are 20% of the nominal 65W TDP. Other reviewers may get different results, due to hundreds of different combinations of configurations, hardware choice and measurement methods. I assume that you can reduce those 13W even further with some adjustments, but don't take my word on that. Unfortunately, I don't own a Ryzen right now to test that myself.
That being said, I don't think you can realistically underclock a desktop CPU to match the power draw of their mobile pendants. So the mobile CPU may always be more economical.
Maybe the embedded lineup of CPUs like the Intel Xeon D or AMD V / Epic Embedded may suit you? They are generally more targeted towards industrial use and have far more connectivity options than those NUC-style mini PCs or notebooks. Commercial NAS vendors like Synology use those. Unfortunately, at first glance, it looks like both Intel and AMD solutions seem rather outdated at the moment.
-
Look at this thread, if you haven't already: https://forums.serverbuilds.net/t/guide-nas-killer-4-0-fast-quiet-power-efficient-and-flexible-starting-at-125/
-
Results are all over the map, but it looks as though idle generally is low across a range of platforms:
These are all mini-pc's, so it's not surprising that the idle power numbers are low.At any rate, it looks as though the idle wattage for just the Ryzen 5 5600X cpu is around 26 watts:
Not particularly awesome, but it can do a lot more work when needed than the mini-pc's above.
26w is manageable with a quiet fan. Total system draw will, of course, be a higher number.And here are the loaded numbers for contrast and comparison:
image url)
source: https://thinkcomputers.org/amd-ryzen-5-5600x-processor-review/8/ -
Results are all over the map, but it looks as though idle generally is low across a range of platforms:
These are all mini-pc's, so it's not surprising that the idle power numbers are low.At any rate, it looks as though the idle wattage for just the Ryzen 5 5600X cpu is around 26 watts:
Not particularly awesome, but it can do a lot more work when needed than the mini-pc's above.
26w is manageable with a quiet fan. Total system draw will, of course, be a higher number.And here are the loaded numbers for contrast and comparison:
source: https://thinkcomputers.org/amd-ryzen-5-5600x-processor-review/8/@NeverDie said in Best PC platform for running Esxi/Docker at home?:
At any rate, it looks as though the idle wattage for just the Ryzen 5 5600X cpu is around 26 watts:
See what I mean with "other reviewers may get different results"? ThinkComputers says 26W idle. Tom's Hardware says 13W. AnandTech says 11W. TPU says 50W (whole system). Who is right? It's unfortunate that there is no standard testing procedure every reviewer adheres to.
Part of the different results may be due to mainboard selection. Some (I don't know how many and if they are rare or the majority) AM4 mainboards seem to be rather power hungry. Especially those with an X570 chipset. Some may add another 10 to 20W to the total power consumption, even if you use the same CPU. See here for example.
26w is manageable with a quiet fan.
Oh yeah, absolutely! Even under load. My 80W Xeon sitting right next to me here is inaudible for most people unless they hold their ear close to the PC case.
Oh, and please note that I'm not trying to sell you the 5600X just because I used it as a comparison a few times. IMHO, all CPUs in that category are grossly overpowered for the average home server unless you know that you will need it sooner or later.
-
It seems that a minimum of 3 disks is required for running ProxMox: one to boot from, one for ISO's, and one for VM's. I'm surprised it's that literal and not able to partition one disk into 3 equivalent disks. It advises not to use a USB flash for the boot disk.
-
It seems that a minimum of 3 disks is required for running ProxMox: one to boot from, one for ISO's, and one for VM's. I'm surprised it's that literal and not able to partition one disk into 3 equivalent disks. It advises not to use a USB flash for the boot disk.
-
Well, good to know for the future. Right now I'm not finding good beginner instructions for a single drive system, or even a two drive system, so I guess I'll just use a cheap SSD over USB 3.0 as the bootdrive. I'll add a second SSD to my Intel NUC, and in theory that should work well enough in the short-term to evaluate ProxMox.
Presently, running just ProxMox with no VM's and no load: it's drawing 8 watts at idle on on an Intel NUC-6i5SYS (that's a i5-6260u CPU) with 16GB memory and a 1TB Samsung NVME. After adding a SATA SSD and a USB SSD, it should meet the nominal minimum criteria for ProxMox's beginner setup. Not an ideal arrangement, so I'll consider something bigger with more drive bays as a better setup in the future.
-
Well, good to know for the future. Right now I'm not finding good beginner instructions for a single drive system, or even a two drive system, so I guess I'll just use a cheap SSD over USB 3.0 as the bootdrive. I'll add a second SSD to my Intel NUC, and in theory that should work well enough in the short-term to evaluate ProxMox.
Presently, running just ProxMox with no VM's and no load: it's drawing 8 watts at idle on on an Intel NUC-6i5SYS (that's a i5-6260u CPU) with 16GB memory and a 1TB Samsung NVME. After adding a SATA SSD and a USB SSD, it should meet the nominal minimum criteria for ProxMox's beginner setup. Not an ideal arrangement, so I'll consider something bigger with more drive bays as a better setup in the future.
-
On second thought, I'll try ProxMox on a system I previously built for Esxi that has 9 drives:
Out with the old hypervisor and in the with the new! This way I'll have plenty of drives to try BTRFS in a Linux VM and whatever else.
-
On second thought, I'll try ProxMox on a system I previously built for Esxi that has 9 drives:
Out with the old hypervisor and in the with the new! This way I'll have plenty of drives to try BTRFS in a Linux VM and whatever else.
-
@mfalkvidd said in Best PC platform for running Esxi/Docker at home?:
@NeverDie looks like a Fractal Design Refine case?
Nope. It's the Nanoxia Deep Silence 6 Super Tower Case. It promised the most soundproofing of nearly any computer case on the market at the time, and the large size makes wiring things up easier than working in cramped quarters. That was six years ago, though, so maybe there's something better available now.
The one case that stood out as being better was one that was completely sealed and which had a lot of heatsink fins milled into the outside of its case. Being sealed, I've got to believe it would have had the best soundproofing by far. It looked beautiful but was hugely expensive.
I imagine these days a bunch of M.2. nvme's would be the way to go--faster, smaller, and less power.
I recently set up a Ryzen 3 3100 as a gaming computer for my son, and it's nearly silent even with just a cheap run-of-the-mill computer case. Come to think of it, maybe later tonight I should test its at-the-wall power draw when it's idle.... :smile:
-
@mfalkvidd said in Best PC platform for running Esxi/Docker at home?:
@NeverDie looks like a Fractal Design Refine case?
Nope. It's the Nanoxia Deep Silence 6 Super Tower Case. It promised the most soundproofing of nearly any computer case on the market at the time, and the large size makes wiring things up easier than working in cramped quarters. That was six years ago, though, so maybe there's something better available now.
The one case that stood out as being better was one that was completely sealed and which had a lot of heatsink fins milled into the outside of its case. Being sealed, I've got to believe it would have had the best soundproofing by far. It looked beautiful but was hugely expensive.
I imagine these days a bunch of M.2. nvme's would be the way to go--faster, smaller, and less power.
I recently set up a Ryzen 3 3100 as a gaming computer for my son, and it's nearly silent even with just a cheap run-of-the-mill computer case. Come to think of it, maybe later tonight I should test its at-the-wall power draw when it's idle.... :smile:
@NeverDie said in Best PC platform for running Esxi/Docker at home?:
Ryzen 3 3100 as a gaming computer
Reporting back: the entire computer draws 60 watts from the wall when idle, whereas it draws 305 watts if running a fractal graphics benchmark. The takeaway is that even 60 watts can still be nearly silent if the fans and power supply are carefully chosen to be quiet. This particular build uses a refurbished Corsair Ax860i for the power supply, and so at 60 watts its fan doesn't even spin. For the case fan it uses a Phantek PH-F120MP_BK_PWM, which is a very quiet fan and which furthermore spins only when the motherboard gets warm. The CPU fan is just the stock AMD Wraith Stealth cooler that came packaged with the CPU, but also with PWM controlled by the motherboard.
-
Building quiet PCs is so addictive. Once you're used to it, every little rattling becomes an annoyance. It didn't took long until I had to get rid of all spinning HDDs in favor of SSDs. And as soon as everything was dead silent... I bought myself a clicky mechanical keyboard, oh well.. :D
Nanoxia Deep Silence and Fractal Design Define are indeed nice sound-insulated cases for quiet builds with plenty of space and features.
For silent cooling, I can recommend be quiet (especially the Silent Wings fans) and Noctua (basically everything). The latter is quite pricy though.
-
It seems that a minimum of 3 disks is required for running ProxMox: one to boot from, one for ISO's, and one for VM's. I'm surprised it's that literal and not able to partition one disk into 3 equivalent disks. It advises not to use a USB flash for the boot disk.
@NeverDie said in Best PC platform for running Esxi/Docker at home?:
It seems that a minimum of 3 disks is required for running ProxMox: one to boot from, one for ISO's, and one for VM's. I'm surprised it's that literal and not able to partition one disk into 3 equivalent disks. It advises not to use a USB flash for the boot disk.
Not true. I have 1 SSD running proxmox. And two more disks passed trough to vm's for storage, but the proxmox part is entirely on the small SSD. It is just partitioned using LVM.
-
@NeverDie said in Best PC platform for running Esxi/Docker at home?:
It seems that a minimum of 3 disks is required for running ProxMox: one to boot from, one for ISO's, and one for VM's. I'm surprised it's that literal and not able to partition one disk into 3 equivalent disks. It advises not to use a USB flash for the boot disk.
Not true. I have 1 SSD running proxmox. And two more disks passed trough to vm's for storage, but the proxmox part is entirely on the small SSD. It is just partitioned using LVM.
@monte said in Best PC platform for running Esxi/Docker at home?:
It is just partitioned using LVM
@monte How did you do it? Did you pre-partition the disk before installing Proxmox, and then Proxmox just found what you did and adopted it, or is it more involved than that?
I started with the ProxMox beginner video, which used 3 disks and gave no hints on how to use fewer:
-
@monte said in Best PC platform for running Esxi/Docker at home?:
It is just partitioned using LVM
@monte How did you do it? Did you pre-partition the disk before installing Proxmox, and then Proxmox just found what you did and adopted it, or is it more involved than that?
I started with the ProxMox beginner video, which used 3 disks and gave no hints on how to use fewer:
@NeverDie to be honest, I don't rememver :) It was so long ago, I was just updating the system from then. But I guess there might be an option in the installer... I will try it in vm, you made me curious :)
UPDATE: I've just ran the install and it plain and simple, no extra options. It automatically partitioned the disk. The only drawback is that you don't have ability to fine tune the sizes of partitions, but that can be done later with some LVM magic :)
-
@NeverDie to be honest, I don't rememver :) It was so long ago, I was just updating the system from then. But I guess there might be an option in the installer... I will try it in vm, you made me curious :)
UPDATE: I've just ran the install and it plain and simple, no extra options. It automatically partitioned the disk. The only drawback is that you don't have ability to fine tune the sizes of partitions, but that can be done later with some LVM magic :)
@monte said in Best PC platform for running Esxi/Docker at home?:
@NeverDie to be honest, I don't rememver :) It was so long ago, I was just updating the system from then. But I guess there might be an option in the installer... I will try it in vm, you made me curious :)
UPDATE: I've just ran the install and it plain and simple, no extra options. It automatically partitioned the disk. The only drawback is that you don't have ability to fine tune the sizes of partitions, but that can be done later with some LVM magic :)
That's what I started with too, but then when I went to upload ISO's or create VM's, ProxMox said there was no disk available for that.
-
So to be precise, Proxmox installer creates LMV VG (volume group) called
pve, in which it creates LV(logic volume)root,swapand andother volume, which in fact is a pool for another thin volumes that will be created for each VM and container, calleddata. So ISO's, backups and other stuff related to proxmox itself will be located atrootvolume. -
@monte said in Best PC platform for running Esxi/Docker at home?:
@NeverDie to be honest, I don't rememver :) It was so long ago, I was just updating the system from then. But I guess there might be an option in the installer... I will try it in vm, you made me curious :)
UPDATE: I've just ran the install and it plain and simple, no extra options. It automatically partitioned the disk. The only drawback is that you don't have ability to fine tune the sizes of partitions, but that can be done later with some LVM magic :)
That's what I started with too, but then when I went to upload ISO's or create VM's, ProxMox said there was no disk available for that.
