Solar power and F@H (or DC in general)

Very nice! Are you on no-export plan, or does your power company just not care if you don't export?

JimboPalmer said:

My municipal power utility will disconnect me if I ever generate power. So I use a minimal 40 watts of their power constantly so I am always a customer, never a generator. It is too bad, on sunny days I could make 20 kW more than I do. As it is, I try to run appliances and dream of the Toyota RAV4 Prime.

Click to expand...

Is this your formal legal arrangement with them through contract, or an unpermitted install that they don't know about?

Talking to the power guys around here, they often enough get sent to go look at a big solar install somewhere that has no solar install on record - satellite imagery or just line guys doing maintenance makes it easy enough to find that sort of thing, and "You have a large solar install interconnected that we don't know about" tends to lead to them taking their meter and going home until you fix the problem to their satisfaction. A few panels are probably messing around, 24 is hard to hide. They were curious about my office when looking at the solar install on the other side of the property, but it's very clearly not touching the power system.

It sounds like it at least monitors loads so it won't export even if your internet connection goes down or folding boxes crash, but... eh.

JimboPalmer said:

But after 6 months he advised us that he now saw no hope of getting an agreement through the city's legal department. He said to turn it on, but never make the meter turn backwards. So yes, the Utility knows, and yes, I have no legal contract.

Click to expand...

Weird... OK! Well, sounds like you're in the same general situation as my office is - lots of use-it-or-lose-it power to find ways to burn.

Any particular reason you went with the "half cell" panels for your roof? Doesn't look like you've got much of a shade issue they'd resolve.

Post on the Agora and see if anyone has weird old servers that aren't power efficient but are still beefy. I got my dual Xeon box for free (I think I paid shipping?) from someone on Ars because it was way too power hungry to justify for their use, but it's still a respectable system - just a power hog. I don't mind power hogs out here as much, though I'm still limited by my inverter output (and a bit of browning out some systems if the inverter is loaded up too high when the air conditioner hits - it's not a gentle starting unit and I've not installed a soft start on it). I've considered moving some systems outside or venting them directly outside in the summer, but I'm not sure it's worth the hassle and extra holes in my office...

Or scrounge parts on eBay. I built an eDRAM based Broadwell box a while back that seems to work pretty well, though hard to estimate performance gains of the eDRAM exactly because my board doesn't let me disable it and I haven't gone poking around at MSRs to find the way to disable it. It's not a big deal for workloads that fit in L3, but should have some nice gains on memory-heavy workloads. You can get used stuff for not an awful lot of money, and again, power efficiency doesn't matter if you're just losing the power anyway.

I'd say get some older GPUs to fold on, but WOW those got expensive... I've debated selling a 980, because it would fetch an absurd amount of money for a several generation old GPU.

My home arrangement with net metering is somewhat similar in that I'll never get paid out for surplus power (I can run a kWh credit until about Dec 2045, then I assume surplus is just discarded/donated, so I'm trying to avoid getting too absurd if I can). I heated somewhat on resistive-efficiency heaters last winter (compute), and will likely do so more in the future - I figure using a bit more energy for heat and doing something useful in the process is better than using somewhat less energy purely for pumping heat around, though... one could argue either way there. We still used the heat pump some, but depending how far ahead I run this year, I may use more next winter. Once I swap a heat pump water heater in, I'll have a few more MWh/yr to play around with. I expect at some point our energy use for transportation will go up... but still not by a huge amount compared to production.

In any case, optimizing for that sort of solar is hard. If I hang another inverter on my office bank, I could run more compute, but then I'd have to replace the charge controller wiring, and... it's still my office, not just a compute farm. Though at this rate, when I replace the batteries, I might very well just build a compute farm somewhere on the hill and automate the whole thing. Ambient venting year round, computer controlled fans, load following, etc.

Step 1, move current guts to larger cases with larger PSUs after a PSU fails, Step 2, replace new guts in larger cases?

Or find a scrap Bitcoin miner that isn't worth running at retail rates and run that.

"No, honey, I didn't pay anything for it, they gave it to me for free to use!" is another valid option, hopefully!

I'm right at the limit of what I can run in my office without pretty serious upgrades - unless I scrap out that 3820/980/1060 system and use the power budget for something else. It's quite power hungry, and I'm not sure if it's still particularly fast. I should really sit down and benchmark the stuff across a range of metrics and figure out which are the best ones to keep around, and if I should rotate some out and use the power budget for other stuff.

I don't have a great feel for what's more valuable, CPU or GPU power. The stuff I tend to run on CPUs obviously can't run on GPUs (climateprediction.net is particularly brutal, 12+ day WUs that don't like being stopped and started very much, so I'm limited in what I can run those on to things that will sleep sanely). But I've got about 600-700W in that 3820 box I could use for other loads.

Of course, in the winter, if I build up enough surplus with my arrays, I could use the excess systems in the house for heat. That's a few months a year, and while the heat pump is more efficient, "doing something productive with the power and also heating" is a useful path as well. At this rate, I may have 5-6MWh surplus by winter to use, and this last winter we only used 3 (with plenty of resistive-type heating).

I wonder if anyone would object to a "Hey, got older server hardware for free? A few of us are interested!" post over in the Agora.

\o/ More solar compute!

You... like, actually found a 3060? Or this is just the plan?

Hard to tell until you actually play with the systems in a range of conditions.

My advice would be to drop stuff down to a lower power, more efficient configuration after dark, and try to stretch battery that way - auto-underclocking, or cutting down core count or something. As we get into the longer summer days, I can leave some of the smaller systems on overnight in here (my little NUC), but I can't run all the stuff all night. Just not enough power for it.

Or just let it rip, either way.

I generally run less in the summer as I need to run the A/C out here more, but... I can still run a lot.

Fancy... grounds?

As weather warms up, my production is dropping a hair, I'd rather keep my office sane temperature-wise, and if I have more than about 1300W running, the air conditioner start browns out stuff, so I can't run past about 1100W if I want a clean AC start.

I would assume those are kWh - seems a reasonable amount for mostly solar based usage.

Ours is rolling back at a very good clip. We're under 1MWh positive from the winter (down from a forward peak of 3.1MWh), and should go sailing past zero here long before the cooling season really starts. Once we do that, I might light up a few more DC tasks in the house, but... I think I'm going to bank kWh for the summer, and heat more on "resistive equivalent" heating in the winter.

I'd put a newer GPU in, but, lol.

Well, that dual Xeon is temporarily out of service, as one of the new rigs I picked up has shown up - a fancy, modern, i5-6600K.

On the Mapping Cancer Markers task, which mostly fits in L3, the 12C/24T Xeon (Westmere) was putting away around 60G instructions per second across all the cores on... oh, 300W.

This 6600K (4C/4T) is putting away 40G. On less than 100W. Not bad!

How many CPU thread did you have running?

I typically leave one or two cores (hyperthreads) available for feeding the GPUs - on my main F@H rig, it was 4C/8T, I ran... 6 or 7 BOINC tasks, left the other threads to feed the GPUs. You shouldn't see a reduction in GPU performance if you do that, and you only lose one or two CPU threads vs all of them.

Woah... there's something wrong, or you're seriously overvolted if you idle at 70C...

Well, set up more F@H this winter!

I'm substantially offline for this heatwave, my office struggles a bit with 105F if I have a ton of extra load running in here dumping heat, and I don't have an outdoor compute area configured yet. However, I've been rotating out some of the less efficient hardware (the Xeons) and replacing it with some more efficient stuff... so I should be able to wedge another machine or two out here, and move the less efficient stuff to the house for winter heating this year.

Current state of the solar compute in my office, which is probably stable for while:

- 1x i7-6770HQ CPU @ 2.60GHz, 4C/8T, stable on all cores at 3.2GHz with my Kerbaled bonus heatsink, 32GB RAM. Desktop NUC/compute box, generally runs 24/7 for now, though will be sleeping more this winter.
- 1x i5-6600K CPU @ 3.50GHz, 4C/4T, stable at 4.4GHz, 16GB RAM.
- 1x i7-5775C CPU @ 3.30GHz, 4C/8T, eDRAM box, stable at 3.8GHz, 16GB RAM.
- 1x i5-4400S CPU @ 2.80GHz, 4C/4T, stable at 2.9GHz, 16GB RAM.

I've paired the 6600K and 5775C on a single ATX PSU, which seems to work - they can suspend independently, and while there are some quirks about fan rails, doesn't seem to cause any problems (if one is asleep, the other fans are still turning - only when both suspend does it shut down properly, but they typically suspend at the same time anyway). I haven't bothered with any diodes or such in the splitter, so... hopefully it doesn't damage anything with one commanding on and the other not caring, but seems to work so far. Frees up a PSU and should improve efficiency a bit (loading a single PSU up more, vs two running at fairly low load).

These can run most of the solar day, they're getting around 8-10h/day of compute in, on surplus from the office system.

Once it cools down enough that I'm starting to use heat in the house, I'll light up 4-6 cores on the homeserver with more compute, running on home solar surplus (I have 5MWh credit right now, will likely get another 1-2 before winter hits, and need about 3 for heating over the winter on top of the production if it's like last winter).

There are also a couple cores crunching at the local datacenter in my colo'd box, as that's not really loaded heavily right now.

I've still got a 12C/24T dual socket Xeon I might put to some work in the house, but it's just insanely power hungry for what it does (though admittedly most of that power is at idle - I may pull some RAM to see if that improves power use, I don't need 72GB).

But all the office stuff is fully "use it or lose it" surplus power on my increasingly comical array out here.

I'm... not great at "stable for a while," apparently.

Of the previous compute boxes, the only one still doing compute is the i7-5775C.

The i5-6600K was sold to a friend for a light house compute/gaming box base.

The NUC (i7-76770HQ) is still present, but not doing compute tasks. It's been doing the normal NUC thing and getting less stable, so I put it back in the case and have stopped pushing it.

The 4440S is still present, but is serving as my daughter's desktop at the moment.

And I've added a pair of 3900X based systems, one with 32GB of RAM, the other with 16GB. Though the 16GB box needs to get more RAM in it soon... and the other one is doing some weird things with clocks dropping every now and then. I'll have to check the BIOS settings once I get a better heatsink on it. But they're churning away during the daylight hours now, and have a quite respectable amount of CPU chops behind them!

Turns out, the better heatsink solved it nicely. Even with temps that... should have been sane, it was pulling clocks back every now and then. With a Wraith Prism spun up, it's holding steady.

AND, I found out how to monitor instructions retired per core with MSRs, so I'll write a utility for that - I'm curious as to how many WUs of particular types I can run before I get net negative throughput, and this will tell me. I'm more interested in optimizing throughput over latency on these rigs.

continuum said:

Glad the better heatsink fixed it. Is it still on the original case or did you swap it back? I forget your setup.

Click to expand...

"Case"?

Currently, I have the 5775C and a 3900X laying on a shelf and sharing a PSU with a splitter, which leads to some interesting behavior in that when one board is asleep, fans are still spun up on the other board. To actually drop fans offline, you have to have both boards asleep. Which I can do on low power nights, but I've got one of them turning 24/7 right now because days are long, nights are short.

The other 3900X is also laying on a different shelf, bare, with a 120mm fan moving air into that corner - it was really heating up the corner under my bench over time and running a bit warm. That's another PSU that can run multiple systems, I just don't have another board for it right now.