Scribed by Matt Fichman
The Role of FCOE in I/O Construction
Summary
Recently there has been a trend in data center networks toward the consolidation of multiple traffic types over a unified data network. Currently, several different high-speed networking technologies exist. Ethernet is used primarily for the LAN. Fibre Channel is used for the SAN, or storage-area network. Other technologies, like Infiniband, are used for interprocess communication for high-performance computing clusters.
A new technology, Fibre Channel over Ethernet (FCoE) is beginning to become a popular replacement for separate Ethernet LANs and FC SANs. The advantages of FCoE are as follows: 1) reduced number hardware components, 2) reduced power consumption, and 3) simplified cabling. FCoE is made possible by the recent introduction of 10 Gigabit Ethernet. However, FC was a lossless protocol, while Ethernet is not. Thus, Ethernet had to be enhanced to carry FC traffic by introducing “pause” messages to control congestion over Ethernet links. In addition, the Ethernet implementation needs to support “jumbo” frames (many implementations do this already) because an FC frame is larger than a traditional Ethernet frame (1500 bytes).
FCoE is superior to other technologies, like iSCSI because it does not require saving state at the gateway between the SAN and the Ethernet LAN. This means it can use cut-through routing and as a result scales much better. In addition, FCoE takes advantage of the ubiquity of Ethernet, which will eventually mean lower cost and better interchangeability of parts as 10GE is adopted.
Opinion
After reading the paper, I am convinced that FCoE is the best solution (compared to Infiniband, iSCSI, etc). I thought it was interesting that Ethernet had to be extended to provide lossless (or nearly lossless) communication, and I wonder why Ethernet did not support this from the beginning. In any case, Ethernet is essentially lossless already. The small amount of lossy behavior might encourage developers of disk access software for managing SANs to write more redundant code, and maybe Ethernet should not be extended as lossless after all.
Discussion
We discussed the topology and basic operation of the Fibre Channel network. When nodes attach to the network, they issue an FLOGI request to login to the switch, and then a PLOGI to login to the disk array. Also, there are 3 types of ports in a Fibre Channel network: extension (used for top-level switches), fabric (used to connect to a node), and node. Fibre Channel uses buffer-to-buffer credits to perform flow control. Basically, the receiver leases a certain amount of credits to the sender, who is allowed to transmit a number of packets equal to the number of credits. We also discussed the advantages/disadvantages of Fibre Channel. Advantages: low overhead, multi-path routing, low state requirements, fast performance. Disadvantages: difficult setup, proprietary, weak interoperability, no layer-2 routing, and the fact that it doesn’t work over long distance.
