Round Robin Scheduling Not Power-Efficient

Via_Patrino writes "While having to distribute load between several servers, round robin, or any other technique that balances load equally, is the most common approach because of its simplicity. But a recent study shows that trying to accumulate load on some servers can improve energy efficiency because the other servers will be mostly unused during off-peak periods and then able to make better use of power saving methods. Specially, where load involves lots of concurrent power-consuming TCP connections, which was the case in the study, a new load-balancing algorithm resulted in an overall 30% power savings. Here's the paper (PDF)."

Logical conclusion

So if we're willing to sacrifice speed for energy savings, shouldn't we just use the bare minimum number of computers that can handle the workload without crashing?

Re:Logical conclusion

What this means is someone needs to architect an intelligent loading system. Ideally, it would manage the load on your base load servers (that are on all the time), and when those servers reach 85-95% of capacity (numbers from my ass) other servers should be brought out of low power/sleep mode to start serving.

Of course, if you use Amazon EC2, this is all moot, as they can shift load around to have their cluster run at peak efficiency.

Re:Logical conclusion

Good luck with that. Really.
 

Re:Logical conclusion

Ahh, Slashdot, where anything Microsoft is thought of as a piece of shit. Mailtrust (formerly Webmail.us) has hundreds of thousands users using their mail product. The backend? Linux running mail daemons and their storage system. The front end? Windows Server 2008.

Get over yourself. Use the right tool for the job.

Re:Logical conclusion

We've been sacrificing computing power for efficiency for years. New Server CPUs tout thier energy savings atleast as much, and quite often more than they tout their computational power. As electricity gets more expensive and data centers continue to grow this trend can only continue; it's simply too expensive to a warehouse full of server racks unless you focus on efficiency.

I'm waiting for the first company to put a data center a few hundred feet under water, where the water temp is low. You'd be surrounded by the worlds biggest heat sink. The environmentalists would have a hissy fit but that's never stopped industry before, and of course you could argue that you are saving electricty on cooling.

Re:

Citations for the idea that the ocean holds heat while the air radiates it harmlessly?

Equity or efficiency?

Confronted with distributing food rations to hungry orphans, people would rather be fair than efficient, even if it means letting some of the food go to waste, a US study shows.

But the tests demonstrated that most people preferred equity in distributing food -- that all the hungry mouths got fed equally -- rather than an efficiency that perhaps meant that one orphan got almost nothing but also that no food went to waste.

http://news.yahoo.com/s/afp/20080509/ts_alt_afp/ussciencepsychologymoralityresearch_080509123210 [yahoo.com]

This problem shows up in many places.

Another Scheduling Flamewar

I don't think that we should go down this road again - why don't we talk about religion or politics, instead?

Re:Another Scheduling Flamewar

I don't think that we should go down this road again - why don't we talk about religion or politics, instead?

Can we just stick to politics? Why spoil a perfectly good Friday with a vi vs emacs debate?

Managed power supplies...

Just switch them off...

If the load on your boxes is below a threshold, remove one of them from the load balance list, wait for connections to end, or migrate the processes off to another machine, and switch it off. When the load is above a certain threshold, you power on an additional node, configure it for whichever service and add it to the load balancer.

Oh come on people, you call yourselves engineers? It really isn't that difficult.

 

Re:Managed power supplies...

If the load on your boxes is below a threshold, remove one of them from the load balance list, wait for connections to end, or migrate the processes off to another machine, and switch it off. When the load is above a certain threshold, you power on an additional node, configure it for whichever service and add it to the load balancer.

Sure, that's not too difficult to do. But it does add complexity. And it does mean your system can't respond to increased load as quickly, as you have to wait for your additional boxes to boot up. If the increased load is predictable, you can anticipate, but that adds more complexity. It doesn't save you on capital costs as you still have to size your power and A/C systems for peak load. Powering the boxes on and off may shorten their lives or reduce their reliability. The question isn't whether it can be done; it's whether it's worth it.

Re:

Agreed. How hard is it to understand that if you use 50% load on 10 servers, you will probably be using more energy than a 100% load on 5 servers. It's common sense when you realize that a 50% load != 50% power consumption.

I am starting to think I did

IT discovers boiler scheduling

Operators of multiple steam boilers have been dealing with this problem for a century. The number of boilers fired up is adjusted with demand, with the need for some demand prediction because it takes time to get steam up. This was done manually for decades; now it's often automated.

The same thing applies to multiple HVAC compressors. Usually there's a long-term round-robin switch so that the order of compressor start is rotated on a daily or weekly basis to equalize wear.

More and more, IT is becoming like stationary engineering.

Re:

Similar idea to modern fuel efficient engines shutting down cylinders when you're idling as well (probably oversimplifying there but you know what I mean)

Re:IT discovers boiler scheduling

Operators of multiple steam boilers have been dealing with this problem for a century. The number of boilers fired up is adjusted with demand, with the need for some demand prediction because it takes time to get steam up. This was done manually for decades; now it's often automated.

Which, alas, won't stop someone from patenting it with respect to servers. Even if it's already been done with computers too.

Incidentally, I've seen descriptions of currently available HVAC control systems for office buildings which takes into account the season, the direction the building faces, the thermal mass of the building, demand, etc, and even learns some of these parameters while running, rather than forcing the installer to calculate them. But every office building I've worked in has had crappy systems which amount to running the compressors on a timer and using individually controlled dampers to provide even cooling (poorly). It seems that we have the technology, but not the will (or the capital) to use them.

The real question is what "fully loaded" means

The only thing that makes this hard is a metric of what "fully loaded" means for a server. With generators and boilers, you have a single number which represents output, and you know what the capacity of each unit is, so you know when to start up the nex

Pound, haproxy

We're running a no-frills OpenBSD load balancer at work. Right now, it's running Pound (the quickest thing we could get up once traffic spiked a few weeks ago), but we're considering other approaches too. haproxy's load balancing knobs look interesting. It looks like you can configure it so the maximum number of clients scales with the current load. The problem is that there's no feedback system.

Some kind of loadavg-based, or even response-time, feedback mechanism would be great! Pound has that (I believe), but since Pound requires downtime for every configuration change, we want to move away from it ASAP.

Re:

pen [siag.nu] can perform some configuration changes on the fly using an optional control service; you can set server weightings at least. It's also event driven rather than the thread-per-connection model I believe pound uses, so it should scale better.

Very cool, but obvious now that we see it

This is a very cool idea, and I don't think it will affect usability too much either. As long as the load balancer keeps tabs on system loading, via snmp or something, it can turn on/off machines based on need.

Assuming your system scales smoothly, i.e. gets proportionally slower as the system load starts to exceed processing capacity. For example, a process will always take 100ms as long as there is CPU time to spare, but once the CPU gets to 100% utilization, you have to start time slicing more processes, that 100ms starts to be 150ms. The load balancer can spin up a new server an start bring down the processing times.

This is an obvious solution to an obvious problem, but until now, we've just never had to examine it.

BigIP

BigIP's can use round robin and use prioritizing, in other words one server receives the most connections over the others. So how is this new?

Could apply to CPUs as well

I think it's probably simplistic to simply distribute a load to all cores of a CPU evenly. Although asymmetrical might be tougher, I could see a system with one low-power always-on core to deal with system requests and organization (Maybe even low enough power to remain on during a suspend), One to handle all GUI threads and interact with the GPU on a private bus, a couple normal cores to handle typical user threading, one of which doesn't come on until the first is like 50% loaded, and one or two high-speed high-power cores that run all-out when the system is plugged in and needs them for intensive processing.

It would take some targeted software design to take advantage of this, but I think we could be looking at a moores law style increase in power...

Soccer moms and scheduling.

14 soccer moms are taking the team of 14 kids to a game. They have two options:
A. Spread the kids among all the cars, and drive all the cars (14 cars)
  or
B. Fill up a car, and send off. Repeat until done. (6 cars)

What is more energy efficient?

Soccer moms have solved this without statistical analysis or engine torque curves.

Re:

Parent post has it all.
Car analogy? Check.
Soccer Moms? Check. Check. (no mention of how many are single though)

But... a lot of soccer moms don't care. They're busy with their other kids and errands too (each server runs more than just apache), so th

Re:Soccer moms and scheduling.

I believe your calculations are wrong. It's understandable, though, since soccer parenting is a fairly unique branch of mathematics.

First off, you're assuming a standard car with 1 adult driver and 4 passengers; instead, you should be using an SUV with a capacity of 6-8, including driver.
(Result: 4-5 vehicles)

Next, you have to consider that not all parents will attend every game. The primary reason that soccer moms drive SUVs is that they must occasionally transport several of their child's teammates to a game (or, worse, to practice!) when their turn comes up in the rotation. Therefore, you only need enough SUVs to cover the number of child passengers, and the number of adults will follow.
(Result: 2-3 vehicles)

However, you might recall that the other reason that soccer moms drive SUVs is that they often have additional children that have not yet reach sports playing age, and must be transported along with the parent, in a car seat (which, in the case of a standard car, would reduce passenger capacity by at least 20% by rendering the back center seat useless.) Assume that approximately 1 in 3 soccer moms have an additional child to transport, and the child adds to the overall passenger count.
(Result: 3-4 vehicles)

Finally, realizing that the overloaded schedule and priorities of child + parent create scheduling conflicts, it is impossible to get optimal performance. At least 1 child per SUV will be late, leaving a seat empty and requiring another parent with car to tranport them.
(Result: 6-8 vehicles)

The result is a range of possible values, but your initial calculation of 6 vehicles is optimistic at best.

New Slashdot Headline Needed

"Round Robin Scheduling Not Power-Efficient when using Windows Live Messenger"

RTFA, in the abstract, "In this paper, we characterize unique properties, performance, and power models of connection servers, based on a real data trace collected from the deployed Windows Live Messenger."

The research itself appears pretty solid. I'd be interested if they publish a followup paper where the model was based off of a variety of applications which utilize round-robin, not just one.