Currently I am working on integrating some 10Gbe switches into an existing 1Gbe network. Being completely new to 10 Gbe I wasn’t prepared for the volley of new terms, acronyms, and gotcha’s that were thrown my way.
Initially I had a very hard time finding answers to some basic questions. So I figured I would write a quick post for everyone else having the same struggles… Excuse my laymen explanations and gross oversimplifications which are to follow… To some I may sound the dunce, so be it. I hope this is helpful to all the other dunces :).
1. What is SFP/SFP+ ?
A. In a typical 1 Gbe switch you have a box with a bunch of ports into which you can plug in Cat5/6 network cables. You may or may not have some “uplink” ports on the right or left-hand side of your switch, but if you do, there is a good chance they are SFP or SFP+. SFP/SFP+ ports look like big empty square holes. They are “modular” by design. Which means you need a device called a Transceiver that plugs into them and the Transceiver can support several different kinds of mediums based on what kind of switch you have and what kind of Transceiver you plug in. And that leads to our next question…
2. Is SFP/SFP+ Optical/Fiber or Copper?
A. It usually can be either depending on the type of transceiver you plug in. If you want to use optical cabling to transmit data, you plug in an optical transceiver. If you want to use standard Cat5/6 copper to transmit data, then you plug in a Cat5/6 transceiver. And then there is TwinAx…
3. What is TwinAx?
A. TwinAxial cable is copper – However unlike Cat5/6 it is significantly lower-latency. Why? I don’t know, there are technical explanations if you do a bit of googling though.
4. Where do I get a TwinAx Transceiver?
A. Now this is what confused the heck out of me until I looked at enough pictures that it finally clicked. Unlike every other medium, most TwinAx cables are DAC (Direct Attach Copper). Which means both ends of the cable already have a Transceiver attached to them. So if you are getting TwinAx DAC cables you don’t need to get separate Transceivers. This is part of the reason why TwinAx is so much more cost-effective vs. Optical because with Optical you have to get the cabling + the Transceivers separately.
5. What is “active” TwinAx and why is it so much more expensive?
A. TwinAx DAC cables have transceivers on the end. If you have a particularly long cable run you may need to go “active.” Passive TwinAx is only rated for like 9 – 11 Meters. If you get an active TwinAx cable then there are additional electronics built into the Transceivers on both ends that boost the signal (and therefore the range), hence the higher price.
6. Can I just get any Transceiver and/or any TwinAx DAC cable that I want?
A. Largely… no… SFP/SFP+ I think is an open standard, however different manufacturers (Dell, Cisco, etc.) like to put in some for of microcode into the transceivers and if the device you are attaching to (Switch, SAN, etc.) detects a “non-branded” transceiver it may not work or will at least generate an error message in the device log. That being said, you can theoretically safely go with an off-brand as long as they list your device/switch model as compatible.
7. What about 10Gbe Base-T?
A. Ahh… yes. 10Gbe Base-T is 10 gig networking over Cat 6a cabling. It looks, works, and feels very similar to what we have all been used to in the 1 Gbe world. TwinAx Copper and Optical (via SFP+) ports are superior is many ways (drastically lower power consumption, drastically lower latency) but I think 10 Gbe Base-T is on the rise because it reduces real and perceived complexity. I think it is more of the latter than the former. Most of what I read though said TwinAx was the best way to go all around. It has latency that is comparable to Optical in many ways and significantly lower power draw.
8. Okay, so SFP and SFP+ ports, what’s the difference?
A. Grossly stated, SFP+ supports 10 Gbe networking, SFP (being older) typically (I think) only supports 1 Gbe networking (although the spec goes up to 4 Gbe). I think they have the same physical form-factor and are backwards compatible though. But double-check that before you buy if you are mixing and matching in your environment.
9. So… where have I landed on all of this?
A. I think I am going to stick with 10 Gbe SFP+ for my switch as well as my server NIC’s. From there, I will go TwinAx as the price point for the performance is excellent, being careful to get cabling that is listed as compatible with my switch. Once you get above the physical-layer, the rest of the job of getting networking working should be more or less the same as the more familiar Base-T medium. TCP/IP is still TCP/IP. To wire my 10 Gbe SFP+ switches back into the rest of my network, I am going to get 1 Gbe Cat5/6 SFP Transceivers for my switch and then just run Cat5 cabling from my 10 Gbe switch back to my 1 Gbe switch. That’s part of the modular beauty of SFP/SFP+ – you can mix and match ports on the same device with different Layer-1 mediums.
Okay, now there is a bevy of other technical terms and topics when it comes to all of the above to sort through. More so I think if you end up going with Optical vs. Twinax. Also, in your research you may come across other modular networking form-factors and types… things like Xenpak, XFP, etc. What I found though, at the end of the day, is that either those form-factors are “dated” and/or the industry has standardized around SFP/SFP+ -or- fixed-port 10Gbe Base-T (Cat6a cabling). So most of my other reading on things like Xenpak was more historical curiosity vs. practical usefulness.
Finally, I found that many 10 Gbe switches have 40 Gbe modular uplink ports using the QSFP+ standard (Quad SFP+) formfactor. Depending on the module you get you can break these up into 4 x 10 Gbe ports or connect to QFSP+ ports on other switches or devices for 40 Gbe throughput. Think switch-stacking and network trunks.
So those are the rather crude questions (with my crude answers) that I had a hard time getting all of the information together for. Hopefully this is a help to other struggling admins dipping their toes into the ocean of 10 Gbe networking.