2023 sorta quieter 1U/40mm cooling ~110W?

xoa

Ars Legatus Legionis
12,061
Subscriptor++
So I'm super rusty on this area of things, and up until now every single time I've set up systems for clients the racks have always been in basements or other out of living/working space areas where, like data centers, noise isn't a massive concern. But now I've got a client with a 1U super micro system being used as an OPNsense edge, and the rack is unavoidably only semi-isolated from part of the office. This is unchangeable for at least the next year, so need to adapt. The system is a short depth and has a single E-series Xeon (95W tdp) and Chelsio NIC and that's it, so not talking a lot of thermal load here, and there is zero need for "silence", there is plenty of noise around, it just needs to be non-irritating. Think 20-40dB, not 60+ dB but also not 10-20, with no grating whine. Right now the system is both running a little hot and loud even at very low load, and looking inside the cooling is pretty mediocre even for rack stuff. Exactly two typical cheap 40mm random fans aimed at the CPU, and that's it, they didn't even bother to plug anything into the third fan header, there is literally no fan pointed at the PCIe 90° and for bonus points the design ensures boards are put in facing down. The heat sink is a basic LGA1150/1200 one. It is what it is. One positive is unlike the misery that is HP or the like, no proprietary crap here, bog standad PWM, swaps are easy.

Obviously the best would be to either isolate it or have it in a bigger case since density isn't actually the problem here, but as that isn't an option for now my first thought is to just throw fans at the problem. My recollection is Arctic and Noctua are decent brands, with the former having a server focused 40/28mm that straddles "quiet" with typical 20k heavy weights and the latter focused purely on minimal noise. Both are dirt cheap, like $11-15 a pop, so reasonable first step seems like it'd be just getting a few splitters and throwing 6 of them in there and that'd probably be gudenuf™? It's short depth without any blockage going on inside or much load, so I don't think high static pressure is critical. Are those two still good brands? If I needed a bit more, is there anyone doing better performing low profile heat sinks, or perhaps low profile heat pipe designs that allow more dissipation area?

I'm probably overthinking this but it might come up again with more load so figured I'd try to learn a bit. Most fan designs I am finding quickly are naturally oriented around 120mm+, serious GPUs etc. Thanks so much for any pointers on good companies.
 

continuum

Ars Legatus Legionis
94,897
Moderator
What's the spec on your current fans and how many are installed currently? You mention 6, which per the link below and your TDP mentioned above sounds like you could get away with six lower flowing, hopefully less noisy fans.

Tangential but interesting given it also talks about fan size and cooling:
 

xoa

Ars Legatus Legionis
12,061
Subscriptor++
What's the spec on your current fans and how many are installed currently?
Two currently installed, looks like around 20cfm max at I think 13k RPM. They cool way way more then adequately at full power, looks like they're sized for heavier duty applications, but are also jet engines, like ~60dB when I measure with NoiseLab-Pro and an i437L mic with some irritating harmonics on top. At lowest RPM things are on the edge of too warm, touching around 60C, which is well below Tjunction or even Tcase but that's not under 100% load either. So tripling the fans including proper cooling of the NIC seems on the face of it like it'd be reasonable.
You mention 6, which per the link below and your TDP mentioned above sounds like you could get away with six lower flowing, hopefully less noisy fans.
Thanks for the sanity check. And it's so low effort and cheap vs trying anything else might as well do that first. I'll grab a 5pack of the arctics at 6k rpm, they're 7.1cfm each so decent top end with an impressively wide range, all the way down to 250rpm. Can play with powerd after. May also be some tunables that will have an effect, I'll be turning off hyperthreading anyway.

Tangential but interesting given it also talks about fan size and cooling:
That is indeed interesting, thank you. And though I'm in a different universe of scale I'll admit I've been a touch surprised over the last decade at how even nominally SMB-targetted kit insists on acting as if it's going to be in an ultra high density data center, and even that seems more dubious with the kind of CPUs and GPUs now available packing enormous dies. But tons of places have lots and lots of physical space to spare, but are more power or cooling limited, and could really benefit from how much easier a lot of design and work is in a 2U/3U/4U form factor. It's not like a bigger case is radically more expensive. Dunno.
 

xoa

Ars Legatus Legionis
12,061
Subscriptor++
Your plan sounds pretty good. You mention at lowest RPM but don't say what that is on your current fans - am curious what that is?
I'd have to double check, but I think 3000-ish is as low as I've seen it go, at idle, as reported by the readings in IPMI. Though some of that may be a futile effort to cool the NIC which another fan would certainly help. There are irritating harmonics even then although the absolute noise isn't bad at that level. The system is some white box build, I think it's one of the endless rebrands based off a supermicro layout, though no component info was loaded into bmc either so would have to go hunt down fan part numbers if they exist. It's not a standard A+ series so can't just grab it all off of SM's site. The steppings in how things rev don't seem to be very smooth either, so there's a lot of "idling along at 3-4k, not awful, gets a touch hot, ramps immediately to 7-9k now it's irritating, ramps back down, then back and forth".

Having looked at it a touch more last night I'll add I think there is probably something on the software side as well in terms of the high idle. I mean, obviously when it's under real load routing/filtering 20gbps that won't be the issue the fans will be needed, but when it's just sitting there I wonder if if it's not dropping frequency or something? I'll need to refresh myself on how to dig into that in FreeBSD.
 

malor

Ars Legatus Legionis
16,093
I'd probably do three Noctua 40x20s, ideally with at least one pushing air in from outside the case, and then see how things went. If more cooling was needed, then I'd look into fan splitters and buying some more.

Remember the case needs spots to mount them, so you may have a hard upper limit that's lower than you want.

I've been using Noctua 40x10s for my little devices, and they've been really good, nice and quiet, and with a reasonable amount of airflow. The x20s should move more air, and the extra thickness shouldn't be a problem in most 1U cases.

Also, their bearings should last for a long, long time, probably a decade or more, so you shouldn't have to revisit the problem later.
 

xoa

Ars Legatus Legionis
12,061
Subscriptor++
Just to follow up, fans arrived, a few Noctua 40x20s and a 5 pack of the 6K rpm Arctic S4028s, along with a few pwm splitters/extensions. I tried a few Noctuas first and not quite enough so ended up fitting 4x of the Arctics in the front (don't tell anyone I used 3M command strips to stick 2 of 'em in there :whistle:) and then a few Noctuas elsewhere, one pointed right at the NIC. Temperatures did drop a solid 10°C overall, still not exactly "cool" but no longer so much into the concern zone and it's still plenty quiet. Sensors report fans running 2300-3000rpm which is completely fine vs other noise.

Now that I've actually opened it back up and put in new stuff vs operating by memory, it's definitely a supermicro board in there, and I do think the CPU cooler is probably something of a bottleneck:
cpu_fan.jpg
It's this blower deal with plate over the sink, direction of airflow front to back of case is the arrows, so a lot of the fan air just runs directly into the blower itself, and heat has to go via the plate vs copper. Noctua makes a very low profile (37mm, 1U is 44.5mm so might fit) fan for LGA1200 in the NH-L9i, or maybe there's something using heat pipes feeding a bigger sink, or even just full passive, a pure copper sink would offer a lot more surface and there should be plenty of front-back airflow now. So I might give that a shot too for a bit more leeway. Still, major improvement and gudenuf for the time being. Thanks both continuum and malor!
 
Last edited:

xoa

Ars Legatus Legionis
12,061
Subscriptor++
Probably the way I'd try first.
Yeah even more so after I slept on it. I realized durr, the Noctua one and similar designs for example are fine in HTPCs but no good in a 1U even though they'd "fit" technically because of course the fan would be right up against the top of the case! Silly. Plain jane copper it is unless anyone makes a low profile heat sink using vapor chambers or something worth looking at. I think Dynatron themselves do, but only for socket 2011.
Glad it's helping!
Thank you, me too ;). Though serves yet again to illustrate how much time/money savings it can be to just do something properly upfront vs after the fact! I guess that's a lot of jobs though.
 

malor

Ars Legatus Legionis
16,093
Yeah, 1U cases are a real pain in the butt to cool, unless they're specially engineered. (usually costs big bucks.) Even then, they're usually super loud, something you'd only want in a colo facility. I prefer using 4U cases for home rackmount stuff, as those can generally take 120mm fans. They can potentially be as quiet as a desktop, although most of them are louder. They take more space, but that's rarely an issue with a home rack.

Another option is to use a nice desktop case sitting on a rack shelf; that's usually very quiet, but looks hacky.

There's quite a bit of surface area on the existing cooler, so it should be fairly effective, I would think. But it looks like that exhaust goes both ways, so it's going to be partially fighting a front-to-back airflow. If you do want to replace it with a passive unit, remember that cooler surface area, airflow over that surface, and volume of exhausted air will be your limiting factors. At the very least, be sure you've got quite a bit of exhaust volume before replacing the existing solution, and remember to get something thin enough to leave room for air to pass overhead. If that area is too choked, you may end up with side-to-back airflow or something else similar, which will really put a hit on cooling potential.
 
Last edited:

xoa

Ars Legatus Legionis
12,061
Subscriptor++
Huh. I thought all high powered 1U cases do fans in front and duct to whatever full case height heatsink needs airflow... TIL.
Yeah that's what I've commonly seen and indeed have myself in my NAS (though that's 2U), it's a sensible design pattern. And that thing with a more powerful processor (not a ton granted, but 120W vs 95W) stays quietly in the 30s at idle instead of 50s or 60s.

But there's an entire galaxy of second tier integrators who do their own thing I guess and haven't all gotten the memo. In fairness there are also big players off doing their own worse BS. Part of the reason I've dumped all HP was after deploying some gen10 stuff that as well as other misbehavior would just decide to scream full power 100% of the time with FreeBSD or Linux on random boots with zero fan speed options, which used to at least be possible via BMC in gen8/9 and previous but HP went out of their way to remove. And of course there you also get to enjoy a nice proprietary connector so no easy replacement either (and then get gouged on basic ipmi features and having a physical port and... grumblegrumblegrumble). Even when isolated off its in own physical space so it isn't a bother in day to day work it's still irritating to have to unnecessarily wear hearing protection just to go do something on a different system in the same rack. I started briefly down the path of "identify which wires can correspond to pwm and cut/strip/join those with a separate fan controller" before realizing this was absolute madness and life is too short.