These big switch boxes typically end up being about thermal management and this box looks like thermal design was an afterthought. Also I'm not sure the power entry design is really all that smart. Generally speaking if you're in the market for a 640Gb/s switch (or 3.8Tb/s switch), does your data center really not have access to 48V power? The AC/DC conversion wastes power and space.
The QSFP's are "spaced for optimal airflow." However this spacing seems to neglect cooling the QSFPs themselves. Belly-to-belly mounting of modules is usually the most thermally challenging way to arrange them. The heat dissipated by the QSFP's is generally directed towards the top of the module. By placing open air channels between modules, they have effectively ensured that little to no air flows over the QSFP heat sinks (which is not shown). So there is probably a limitation on which reach codes are supported. My guess is that because of the thermal limitations of this design, it's not truly non-blocking in all reach configurations.
This is true, however: 48V circuits are generally priced 'per-amp' just like 120V or 240V circuits. That translates into much higher cost-per-watt and with the efficiency of AC switching supplies being well over 90% you're probably better off from an economics point of view of getting the highest voltage supply that you can get and then to step down in your rack mounted devices.
This is rather cool, and I'm glad to see some work being done in the networking gear space utilizing open designs and firmware like this. Unfortunately, unless I find myself needing to network a datacenter in the near future, it's not immediately useful to me.
That said, I do hope other developments in network gear that will be useful in other markets emerge from this effort.
From my perspective, there's a gap in network gear between the unmanaged, low port-count switches in plastic enclosures, targeted to home and small office consumers, and the lower tiers of Cisco's catalog, targeted toward top-of-rack or wiring-closet-of-a-larger-building type uses. I would love to see a managed switch with say, 8-24 ports, supporting features such 802.11Q VLANs. I would love to be able segment my network at home so different devices with different performance and security needs aren't all stepping on each other's toes. And I'd like the firmware and hardware designs to be open source, so it can readily patched when bugs are found, and easily adapted to new use cases.
I realize that I'm an outlier and that my needs are not common, or there'd probably be equipment on the market that met them. But it is my hope that as a result of Facebook's work here, and similar efforts, that building such a device will become feasible.
I've the exact same needs, and by occasion got hold of cisco SG300-10 switch, which I think fits the description - so I thought to share my experience, in case it is useful.
I use it for the home lab, where I have a one-armed router serving multiple VLANs, and I have 3x MacMinis running Linux as a "server farm" (The latter I use because they are quite a good gear power-management wise, scaling from ~18W at idle up to ~250W when all cores are busy, and because they are very very quiet, which is handy when the "lab" is next to the bedroom).
The biggest complain I have about this box is that the only way to manage is the Web UI, and especially the 802.1q configuration is a bit unintuitive (though I just learned the firmware is actually upgradable to something with decent IOS CLI, so I will try it out and update here the impressions).
Otherwise, needing just a very simple L2 switching and 802.1q trunking at gigabit speeds, and fanless operation - I am pretty happy with it.
8 ports works well in my setup (the main segmentation/trunking is really in the lab, the rest is either wireless, or directly connected to the "border router").
EDIT: the upgrade to the latest firmware indeed unearthed the checkboxes to enable telnet/ssh, as well as quite a few new features, comparable if not more than the "bigger brothers". What's pleasant is a quite comprehensive IPv6 support.
I also wanted something in the category you're describing and after shopping around a bit I stumbled across Mikrotik's products. I bought one of their 24 port smart switches and it was exactly what I was looking for. Not a lot more expensive than some of the nicer consumer-targeted gear, but it has way more functionality.
There are some good cheap managed switches out there. I have a Linksys SFE2000-something 24 port PoE switch for my PoE devices (security cams, IP phone, other stuff) and a TP-Link [something] 16 port switch. Both support basic VLANs, SNMP, etc.
I have 1 2port VLAN that connects my cable modem to my router on the TP-Link. From that switch I have 1 port cables to a machine with wireshark, and I can configure that port to monitor various VLANs for whatever reason.
I run MRTG for several things, including basic traffic graphing. I display the graphs from the router uplink port and a couple of other key ports in a window on the VMS that also has security cameras on it. From that monitor I can keep on eye on key things (cameras, Internet I/O, some home automation stuff).
Anyway, I haven't found an affordable "perfect" switch for home stuff, but there are a lot of cheap, decent managed switches that give you a lot more flexibility beyond "everything on 1 network".
My home net is essentially segmented into Primary LAN, Security Devices, Guest LAN (mostly just a wifi bridge) and LAB LAN.
This isn't really suitable for the purposes you're describing, but it has a niche for firewalls and routers that may interest someone.
I've been using PC-Engines Alix[1] boards for this purpose since 2008. They are 1-3 ports depending on model, but with VLAN tagging a few ports can do a lot of routing or other traffic processing. I run OpenBSD on mine, but they should work well with anything that can run headless.
Wow. That switching fabric is so much smaller than the HIPPI crossbar backplanes we used in the 100baseT/pre-GigE switches that I used to write firmware for... almost 20 years ago now. Of course, complexity means higher cost, so it's obvious why nobody bought our stuff. Hopefully, this open hardware project will have better luck!
/* it's easy to forget just how many iterations of Moore's Law have happened since the mid-90s */
I get the feeling this is a bit like Bugatti building the Veyron, its super cool and impressive but the vast majority of us will never be able to or have a need to drive it.
It is still fun watch this switch go around the proverbial track, but I'm happy knowing that I'll never have to configure, build and test a switch of this complexity unless I really absolutely have to, with my largest caveat being AWS disappearing from the face of the earth.
You're not the target market. You shouldn't be making an analogy with a Veyron, which remains a luxury car for an individual. You should rather be comparing it with a firetruck, a vehicle custom-built for a purpose you as an individual you will never need.
I currently work in the network hardware industry (think Cisco, Juniper, etc). Our boxes sell in the 6-figure price range, each [1]. We sell to your ISP, wireless carrier, datacenter constructor. We're the competition this kind of box is aiming at.
[1] it's highly specific hardware and software for a low-volume market. Individual chips used in the hardware can cost multiple thousands of dollars each.
One of the great things about computing is that stuff which was insanely high-end N years ago is high-end of affordable now and will be commodity in another N years.
And N years after that you get that functionality for free along with your new TV.
The only datapoint I have is that it's in the SF bay area, since I am acquainted with someone who got poached from Amazon specifically to work on this and had to relocate from Seattle to there.
I just want a fully open source switch that's ready for enterprise, from the h/w to s/w. Linux preferred but bsd based is acceptable.
I've been a "cisco guy" since 2001 or so, but I am so tired of them. Licensing fees kill budgets that could be used on other things, and you end up surrounded by consultants that only ever touch Lozedoze systems insisting that "nobody ever got fired for buying Cisco". Smartnet is a must for some equipment, yes, but I'm so ready for a paradigm shift in networking.
I've really been watching ubiquiti and their switching/routing products, they seem very promising but not quite prod ready. I am impressed with Dells open switches too.
Edit: The Microtik stuff being linked elsewhere here is looking pretty awesome too. Not quite FOSS but still.
As someone who runs a 6509 (almost fully populated with line cards and specialized controllers) as my HOME core switch (with a second one on the way for full redundancy), an open modular switch is very cool.
(I'm currently hacking on OpenFlow -> NETCONF bridging, to bring typical SDN capabilities to legacy Cisco environments). Hence my rather..... extensive home network.
This is very similar to what Cisco is doing with Nexus and the "fabric extender" TOR replacement kit. I'll have to see if Facebook has any of this stuff in GIT and stand it up in a VM environment and play with it (I already do a bunch of OpenFlow stuff on OpenWRT and am looking at implementing an open southbound API on FPGA on the parallela board).
1) low individual component cost (which is great when you need 100's of them for a datacenter build out)
2) no frills, high throughput non-blocking backplane which are normally only available from the enterprise grade network vendors at top dollar per individual component, and come with a mountain of features and bugs you DONT NEED
3) software defined networking stack: if you can imagine a scenario/feature that would improve your life, nothing but development costs will get in your way. contrast with enterprise vendors, which requires explaining the problem to begin with, and dangling a wad of money explaining how it will be worth their while to develop it
This looks great. They also have the advantage that James Hamilton mentioned AMZ got from building their own hardware - the get to do a cleansheet s/w design and not have to support all the gazillion options that Cisco does.
Am I the only one who read the title and thought it referred to a 'pushbutton' style electronics hardware switch? I would have been quite excited to see that!
btoptical|11 years ago
The QSFP's are "spaced for optimal airflow." However this spacing seems to neglect cooling the QSFPs themselves. Belly-to-belly mounting of modules is usually the most thermally challenging way to arrange them. The heat dissipated by the QSFP's is generally directed towards the top of the module. By placing open air channels between modules, they have effectively ensured that little to no air flows over the QSFP heat sinks (which is not shown). So there is probably a limitation on which reach codes are supported. My guess is that because of the thermal limitations of this design, it's not truly non-blocking in all reach configurations.
wmf|11 years ago
No, they really don't. I've never heard of 48V outside of telco.
jacquesm|11 years ago
rsync|11 years ago
However, at no single site (especially my home) do I need a 6U chassis full of switch ports.
Is there a 1U version of this on the horizon ?
kijiki|11 years ago
Some easy ways to buy are: https://bm-switch.com/ http://whiteboxswitch.com/ http://ipfabrics.net/ http://www.penguincomputing.com/products/network-switches/
- nolan co-founder/CTO Cumulus
aaronbee|11 years ago
actaeon169|11 years ago
For example:
http://www.edge-core.com/ProdDtl.asp?sno=425&AS4600-54T
or
http://www.edge-core.com/ProdDtl.asp?sno=423&AS5600-52X
gonzo|11 years ago
pingswept|11 years ago
From the responses, it looks like "open hardware" means something like "can install any OS on it" in the network switch world.
sschueller|11 years ago
korethr|11 years ago
That said, I do hope other developments in network gear that will be useful in other markets emerge from this effort.
From my perspective, there's a gap in network gear between the unmanaged, low port-count switches in plastic enclosures, targeted to home and small office consumers, and the lower tiers of Cisco's catalog, targeted toward top-of-rack or wiring-closet-of-a-larger-building type uses. I would love to see a managed switch with say, 8-24 ports, supporting features such 802.11Q VLANs. I would love to be able segment my network at home so different devices with different performance and security needs aren't all stepping on each other's toes. And I'd like the firmware and hardware designs to be open source, so it can readily patched when bugs are found, and easily adapted to new use cases.
I realize that I'm an outlier and that my needs are not common, or there'd probably be equipment on the market that met them. But it is my hope that as a result of Facebook's work here, and similar efforts, that building such a device will become feasible.
ay|11 years ago
I use it for the home lab, where I have a one-armed router serving multiple VLANs, and I have 3x MacMinis running Linux as a "server farm" (The latter I use because they are quite a good gear power-management wise, scaling from ~18W at idle up to ~250W when all cores are busy, and because they are very very quiet, which is handy when the "lab" is next to the bedroom).
The biggest complain I have about this box is that the only way to manage is the Web UI, and especially the 802.1q configuration is a bit unintuitive (though I just learned the firmware is actually upgradable to something with decent IOS CLI, so I will try it out and update here the impressions).
Otherwise, needing just a very simple L2 switching and 802.1q trunking at gigabit speeds, and fanless operation - I am pretty happy with it.
8 ports works well in my setup (the main segmentation/trunking is really in the lab, the rest is either wireless, or directly connected to the "border router").
Where I needed to add more access ports, I used http://www.conrad.com/ce/en/product/976050/CE-Port-Network-S... to help. Also based on my experience seems to be quite a solid building block for a small home/lab network.
full disclosure: I do work for cisco.
EDIT: the upgrade to the latest firmware indeed unearthed the checkboxes to enable telnet/ssh, as well as quite a few new features, comparable if not more than the "bigger brothers". What's pleasant is a quite comprehensive IPv6 support.
gh02t|11 years ago
http://routerboard.com/CRS125-24G-1S-IN
brk|11 years ago
I have 1 2port VLAN that connects my cable modem to my router on the TP-Link. From that switch I have 1 port cables to a machine with wireshark, and I can configure that port to monitor various VLANs for whatever reason.
I run MRTG for several things, including basic traffic graphing. I display the graphs from the router uplink port and a couple of other key ports in a window on the VMS that also has security cameras on it. From that monitor I can keep on eye on key things (cameras, Internet I/O, some home automation stuff).
Anyway, I haven't found an affordable "perfect" switch for home stuff, but there are a lot of cheap, decent managed switches that give you a lot more flexibility beyond "everything on 1 network".
My home net is essentially segmented into Primary LAN, Security Devices, Guest LAN (mostly just a wifi bridge) and LAB LAN.
GeorgeBeech|11 years ago
See the bottom of this page for models the 24 and 8 switches are what you are looking for. http://www.cisco.com/c/en/us/products/switches/catalyst-2960...
Also, if you want a cheaper options the Dell Basic switches are fully managed and have just about any feature you could want.
http://www.dell.com/us/business/p/powerconnect-2800/fs
Unless I'm missing something that you are looking for.
DrPhish|11 years ago
[1]http://www.pcengines.ch/alix.htm
charlesnw|11 years ago
http://www.ubnt.com/accessories/toughswitch/
and
http://www.ubnt.com/unifi-switching-routing/unifi-switch/
That's exactly what you are looking for. SOHO friendly (fanless, cheap, supports vlans).
pdkl95|11 years ago
/* it's easy to forget just how many iterations of Moore's Law have happened since the mid-90s */
sudioStudio64|11 years ago
rustyconover|11 years ago
It is still fun watch this switch go around the proverbial track, but I'm happy knowing that I'll never have to configure, build and test a switch of this complexity unless I really absolutely have to, with my largest caveat being AWS disappearing from the face of the earth.
AceJohnny2|11 years ago
I currently work in the network hardware industry (think Cisco, Juniper, etc). Our boxes sell in the 6-figure price range, each [1]. We sell to your ISP, wireless carrier, datacenter constructor. We're the competition this kind of box is aiming at.
[1] it's highly specific hardware and software for a low-volume market. Individual chips used in the hardware can cost multiple thousands of dollars each.
dsr_|11 years ago
And N years after that you get that functionality for free along with your new TV.
gdooraccess|11 years ago
hamburglar|11 years ago
aristus|11 years ago
arca_vorago|11 years ago
I've been a "cisco guy" since 2001 or so, but I am so tired of them. Licensing fees kill budgets that could be used on other things, and you end up surrounded by consultants that only ever touch Lozedoze systems insisting that "nobody ever got fired for buying Cisco". Smartnet is a must for some equipment, yes, but I'm so ready for a paradigm shift in networking.
I've really been watching ubiquiti and their switching/routing products, they seem very promising but not quite prod ready. I am impressed with Dells open switches too.
Edit: The Microtik stuff being linked elsewhere here is looking pretty awesome too. Not quite FOSS but still.
charlesnw|11 years ago
As someone who runs a 6509 (almost fully populated with line cards and specialized controllers) as my HOME core switch (with a second one on the way for full redundancy), an open modular switch is very cool.
(I'm currently hacking on OpenFlow -> NETCONF bridging, to bring typical SDN capabilities to legacy Cisco environments). Hence my rather..... extensive home network.
This is very similar to what Cisco is doing with Nexus and the "fabric extender" TOR replacement kit. I'll have to see if Facebook has any of this stuff in GIT and stand it up in a VM environment and play with it (I already do a bunch of OpenFlow stuff on OpenWRT and am looking at implementing an open southbound API on FPGA on the parallela board).
VLM|11 years ago
http://opencores.org/project,mac_layer_switch
and some links to digilent dev boards for hardware, however the linked project is a bit more ambitious, all full of custom ASICs and such.
An open source switch made out of COTS FPGA dev boards would be interesting. So you'd use something like
http://www.digilentinc.com/Products/Detail.cfm?NavPath=2,400...
But what to use as a COTS "just unpack it all, plug it all together, upload the firmware and go" backplane?
virtuallynathan|11 years ago
fidotron|11 years ago
Almost all "open" projects seem to hit a point where the line between open and closed is quite arbitrary, even if they choose not to see it that way.
wmf|11 years ago
If Facebook wants to give me that value for free I won't complain.
spydum|11 years ago
1) low individual component cost (which is great when you need 100's of them for a datacenter build out) 2) no frills, high throughput non-blocking backplane which are normally only available from the enterprise grade network vendors at top dollar per individual component, and come with a mountain of features and bugs you DONT NEED 3) software defined networking stack: if you can imagine a scenario/feature that would improve your life, nothing but development costs will get in your way. contrast with enterprise vendors, which requires explaining the problem to begin with, and dangling a wad of money explaining how it will be worth their while to develop it
MrZipf|11 years ago
The talk was on HN a little while ago, worth watching: https://www.youtube.com/watch?v=JIQETrFC_SQ
The scale at FB means they're probably saving hundreds of millions doing this.
aaronsnoswell|11 years ago
unknown|11 years ago
[deleted]
cordite|11 years ago