top | item 19409902

Facebook announces next-generation Open Rack frame

125 points| el_duderino | 7 years ago |code.fb.com | reply

44 comments

order
[+] mlthoughts2018|7 years ago|reply
This strikes me as a gigantic coordinated form of commoditize your complement.
[+] walrus01|7 years ago|reply
DC power wise, the original reason why they did this is that it's grossly inefficient to have a 42RU worth of 1U servers, each with its own individual 110-240VAC to 12/5VDC power supply in it.

Or the equivalent of that with four-servers-in-2RU type setups, but also with AC to DC power supplies.

They centralized the AC to DC conversion in a single unit in the rack, feeding either 277VAC or 480VAC to each rack, and ran 12VDC to each server. The new system wisely moves from 12VDC to 48VDC (same as most telecom equipment), and probably has basic DC-DC converters in each server unit for 48 to 12VDC conversion.

They've also gone with custom motherboards that entirely eliminate the 5VDC rails which are distributed by 'normal' ATX server power supplies.

[+] noir_lord|7 years ago|reply
Makes sense but has nice second order effects.

If we could standardise more we can optimise for power efficiency, things like more DC in DCs

[+] meruru|7 years ago|reply
>commoditize your complement

You talk like that's a well-known concept, but as far as I know it has been coined only one year ago in this essay: https://www.gwern.net/Complement

[+] kayhi|7 years ago|reply
how so? like commoditizing Oracle databases?
[+] acd|7 years ago|reply
Could ocp make a rack that behaves like a blade server?

Construct a server rack that works like a network Patch panel and server power bus? Thus one would just install the server in the rack slot, no more Cables to the server. Bonus award if a robot could slide in the server in the rack.

[+] ajdecon|7 years ago|reply
There are rack designs out there built on this idea, but they are usually pretty specialized. For example, the Cray XC series has racks with built in power, network, and out of band monitoring.

A downside of this kind of thing is that it makes upgrades and maintenance harder, and you often have to do any hardware work on a whole rack at once. Heterogeneous setups get really hard. And it’s usually very vendor specific.

[+] zamadatix|7 years ago|reply
You'd either need a different rack design per particular server or a rack that is overdesigned for 99% of servers plugged into it (e.g. different speeds/counts of network connections). If the thought is the servers will be static for the lifetime of the rack does wiring up a preset rack to a switch really save any time from wiring up preset server designs to a switch?

I think it's probably best kept modular.

[+] acd|7 years ago|reply
A rack but that has two power feeds in standardized position for each rack unit. When you slide in the server in the rack, you provide power. It would slide in the power connector provided by the rack. More specifically I mean that the server would get automatically connected to IEC 320 C13 female provided by the rack. The modifications to a standard server would be too standardize the position of two 230V/110V power feeds.

Alternatively, connect the server to a 48DC bus when its racked in providing +48 volt direct current and ground which can be through the server chassis.

Network connectivity should be provided in two standardized positions as well.

[+] akhilcacharya|7 years ago|reply
What’s the current industry practice for blades? Does AWS GCP, Azure use them?
[+] baybal2|7 years ago|reply
I wonder, when will facebook's hardware people start using their own inventions?

I see them and other dotcoms pouring inordinate amount of money designing own hardware without actually manufacturing or using anything of it.

As I know, facebook still buys very plain OEM servers from Quanta

[+] electrum|7 years ago|reply
Facebook does use it. They design their own hardware, but they are do not manufacture it themselves, they aren't in the hardware manufacturing business. They parter with companies like Quanta to do that.

Here's an older article that explains the relationship: https://venturebeat.com/2014/01/29/facebook-quanta/

[+] jauer|7 years ago|reply

    without actually manufacturing or using anything of it.

    As I know, facebook still buys very plain OEM servers from Quanta
This is false, as you can see from publicly available pictures of FB datacenters. Facebook purchases OCP servers from ODMs.
[+] dana321|7 years ago|reply
The sooner this centralized model of storing data ends, the better.
[+] Twirrim|7 years ago|reply
What does that have to do with an open specification for racks of computers? This is a design that anyone could potentially adopt, and gain from the engineering efforts that those involved in OCP have done.

There's even a marketplace you can purchase Open Rack components through (and you could likely also go more direct to those companies rather than via OCP's marketplace): https://www.opencompute.org/products