Containers Expo Blog Authors: Yeshim Deniz, Liz McMillan, Pat Romanski, Zakia Bouachraoui, Elizabeth White

Related Topics: Recurring Revenue, Containers Expo Blog, @CloudExpo

Recurring Revenue: Article

Virtualization Journal: FCoE’s Role in Data Center Convergence

Or why a range of multi-protocol fabrics will be around for a while

It's a long-standing goal in enterprise networking to cost-effectively converge all local and storage area network (LAN and SAN) traffic on a single powerful infrastructure via one flexible, reliable, high-performance and low-latency protocol. And the future data center will, indeed, one day rely on some form of converged fabric with server and storage virtualization.

Only two of today's technologies stand to keep pace with Moore's Law and potentially serve as that sought-after unifying fabric: InfiniBand and Ethernet. This makes emergent Fibre Channel over Ethernet (FCoE) an especially promising interconnect candidate. Proponents of the protocol feel FCoE one day could shoulder all LAN and SAN traffic on the same, commonly managed, low-cost infrastructure - while delivering performance as fast, reliable and low latency as offered via highly sophisticated protocols such as InfiniBand.

There is a compelling value proposition for the adoption of FCoE on some types of midrange and rack-mounted servers. But, for many years to come, the real-world enterprise data center based on high-end servers will continue to support a range of existing multiprotocol fabrics. As always, the challenge faced by data center managers will be smartly matching technologies with applications to most affordably satisfy technical requirements and business objectives. Protocol-agnostic Wavelength Division Multiplexing (WDM) will continue to serve as the convergence cloud for an array of valuable protocols linking together multiple enterprise-class data centers as well as transporting converged fabrics where required.

For managers of enterprise data centers, FCoE's promise for cost-effectively converging Fibre Channel and Ethernet - the two dominant enterprise networking protocols - is too great to be overlooked. At the same time, its hype as today's single unifying fabric for all enterprise LAN and SAN traffic must be closely scrutinized.

A Precedent Has Been Set
Data center managers can be forgiven if they suspect they have read FCoE's story before.

In the excitement surrounding the rollout of a number of technologies - Fibre Channel, Asynchronous Transfer Mode (ATM) and InfiniBand, among them - there has acclaim for the arrival of the unifying interconnect fabric for all LAN and SAN services run across enterprises. This is compelling language to data center managers, who would gladly welcome the simplicities and cost efficiencies delivered by convergence given the dazzling pace of innovation in Internet, mobile, wireless, storage and other business capabilities that the data center must somehow accommodate.

Each of these protocols have served - and, in many cases, continue to serve - critical roles in enterprise networking. But the truth is that they have failed to deliver the complete transformation of convergence. In some instances, the various specific requirements of competing protocols have not been satisfied, or proprietary features have been too heavily emphasized. Or, in some cases, cost points have not been met because production volumes were insufficient.

Total LAN and SAN unification - managers of data centers have come to learn - is not achieved lightly.

Sorting the Hype from Reality
Today, there's a great deal of hype about the potential of FCoE to bring about the converged world of SAN and LAN. Could FCoE succeed in a way that the other technologies have not?

It must be noted that FCoE does have a lot going for it. Conventional Ethernet is being enhanced in areas such as flow control, Quality of Service (QoS), and packet loss, and developers are leveraging the lessons learned from previous convergence efforts in working toward an industry-standard FCoE protocol. The promise of this technology is considerable.

The open version of this new standard is called Converged Enhanced Ethernet, though some vendors use proprietary versions with different names. Thus, the preferred implementation of this standard would be Fibre Channel over Converged Enhanced Ethernet (FCoCEE), but for simplicity we will refer to this market as simply FCoE for now.

At the same time, it must be noted that the FCoE standards are just being finalized this year, which is an important factor for data center managers who prefer to base their infrastructure on open, international standards rather than proprietary implementations. As the industry prepares to roll out the first FCoE installations, there are a number of obstacles which must be overcome.

First, data center managers must select their converged fabric switch provider with care. For at least some switches, convergence of the SAN and LAN onto FCoE will require an upgrade of the existing Ethernet infrastructure with new low-latency Ethernet switches. This could amount to a major, disruptive and costly overhaul of the enterprise network core. Migration will not be a simple story of taking the existing corporate SAN, shifting it to the LAN and deploying the FCoE protocol. Data center managers will need to carefully re-examine their infrastructure partners, including several new companies just entering the market, to obtain the best value with minimal disruption.  Companies would be well advised to partner with a server provider who understands when FCoE is appropriate, rather than those who tout this technology as one size fits all.

Beyond that, we already know that some of the proposed non-standard FCoE implementations on the table fail to sufficiently address real world issues such as latency, synchronous recovery and continuous availability - key requirements for some of today's most important enterprise data center applications. Once again, partnering with a server company that understands the end-to-end data center application will be important in the design of FCoE enabled data centers.

Server and Storage Virtualization Via FCoE?
Consider the questions surrounding convergence on FCoE as an enabler to the promising application of server and storage virtualization.

Server virtualization allows enterprises to consolidate applications onto less hardware in order to lower costs. Prior to server virtualization, it was difficult to convert applications between different operating systems. By virtualizing the server it could, in principle, run any operating system (OS), enabling applications to be consolidated without converting them. This is one reason server virtualization has taken off. Storage virtualization simplifies today's increasingly complex storage environment, allowing organizations to more easily manage their infrastructures and consolidate storage systems from different vendors into one pool of storage.

Storage virtualization usually requires a SAN to go between the storage and servers, and has proven to cause the least disruption to these systems. This approach of placing the virtualization in the SAN between the storage and the server has been around for years yet somehow has languished while server virtualization has become more widely accepted. The problem again lies in the support of so many heterogenous storage and management systems that the "virtualized SAN" must support in a scalable fashion. Thus, the preferred location for virtualization services remains unclear, and will likely require a combination of server and network features to fully deliver the promise of converged networking.

Storage virtualization via FCoE is an intriguing concept, which some large data centers will soon be attempting to demonstrate across a spectrum of customer installations where Fibre Channel is currently the dominant protocol. The practical implications are far-reaching and hard to foresee. For instance, sharing one physical device among several logical partitions brings up a number of technical issues across links, servers, storage adapters, switches and points across the network - congestion, prioritization and queuing among them.

These technical problems are certainly not unsolvable, and the industry is working on an extension of networking standards and new features in server hardware and software to achieve such goals. Achieving these benefits will require data center managers who understand the full range of problems, and implement phased solutions to evolve the data center from its current design into something more energy and cost efficient. Initial deployments of FCoE will likely limit the use of FCoE to a chassis, such as within a rack or within a Blade Server. This usage model can be applied across multiple racks to obtain significant capital and operational cost savings. In the long term, as FCoE standards, hardware, software and management evolve, FCoE may one day enable full data center fabric convergence with associated economies of scale and resource savings. In the meantime, 8G Fibre Channel is an option on some of the latest FCoE switches that provides customers the flexibility to move to FCoE at their own pace.

Understanding Where FCoE Fits Today
Data center managers must evaluate FCoE in the same way they evaluate any other protocol for their networks:

  • Start with the business objectives to be achieved.
  • Consider the solutions available that can meet the technical requirements.
  • Cost out and compare the options which fit today and will effectively scale into the future.

The traditional data center compute model, especially in the case of x86 servers, has consisted of lightly utilized servers running a bare metal operating system or a hypervisor with a small number of virtual machines. In this environment, lower bandwidth links were sufficient for attachment to server resources. We see the industry moving to a dynamic infrastructure model that has highly utilized servers running many virtual machines per server, and using higher bandwidth links to communicate with virtual storage. This compute model offers several short-term business advantages to customers, including lower capital expenses through higher utilization of midrange servers, storage, and converged network fabrics, as well as lower operational expenses that are achieved through automated and integrated management that optimizes the end-to-end data center infrastructure. By contrast, enterprise servers that already offer high link utilization and make extensive use of virtual machine partitions will realize other long-term benefits.

There are scenarios where a data center manager might see FCoE as a wise choice today. For example, in cases where an enterprise needs to move a lot of raw data over a network for storage backup, then FCoE might be acceptable. Greenfield opportunities where FCoE's performance, reliability and latency levels would be acceptable might be a good place to experiment with the new protocol, by initially deploying FCoCEE at the chassis level as described earlier. Or, large, Internet-scale data centers might take a look at FCoE as a way to consolidate the LAN and SAN and reduce servers, adapters and cables. Finally, environments where low-cost connectivity is a higher priority than reliability - social networking sites and search engines, for instance - also might make a good fit for FCoE, where high-end "cloud computing" could represent a new market opportunity by utilizing existing data center assets. These sites always operate on a low-cost commodity model. FCoE might well be their answer to a reliable transport without ever investing in a separate Fibre Channel SAN network. FCoE is also expected to play well in the analytics and high-performance computing market. As FCoE matures and meets the performance, reliability and quality requirements of larger enterprise customers, we expect this technology will come to play well in these areas; this will also require addressing related technical issues, such as ensuring the enterprise server I/O subsystem design provides sufficient start rates and takes advantage of opportunities to increase link utilization.

One of Many Valuable Protocols
Other enterprise applications have shown renewed enthusiasm for InfiniBand, a protocol experiencing intensified use for high-end, low latency applications such as real-time stock trading, medical image processing, clustering, business continuity, disaster recovery, and grid computing. Research and education, government and other enterprise users have found InfiniBand as the ideal enabler for high availability architectures such as Geographically Dispersed Parallel Sysplex (GDPS), which has recently announced Parallel Sysplex Infiniband (PSIFB) with Server Time Protocol (STP) messaging. Very high bandwidth and low latency are must-have attributes for these services, and InfiniBand delivers in cost-effective, proven and available solutions.

InfiniBand is a perfect example of a mission-critical high-end enterprise protocol of unique benefits that FCoE is not likely to render obsolete any time soon. Managers of most enterprise data centers will need to strategically and cost-effectively balance a host of factors - technology capabilities, distances to be linked, access options and cost issues, among them - and continue to run multiple protocols to most cost-effectively satisfy business objectives.

WDM, then, effectively becomes the convergence foundation for a data center's FCoE, InfiniBand and other legacy protocols when multiple data centers need to be interconnected over any significant distance. Flexibly and transparently multiplexing applications of any standard or non-standard protocol in "virtual channels" across optical fiber networks, WDM delivers the reliable, native-speed performance for multiprotocol traffic that real-world data centers will require moving forward.

Imagine that the manager of an enterprise data center reads a white paper about the indisputable promise of FCoE and is understandably intrigued: "I know Ethernet ... I know Fibre Channel ... They're easy and virtualization will be transparent... I'll consolidate all of our LAN and SAN traffic on FCoE and save money. I'll also start deployment with non-standard hardware. Perhaps I'll hand my storage network management over to the network administrators, surely they'll maintain the bandwidth allocations needed to support my servers."

For a number of critical reasons, it's not quite that easy. Some companies may release pre-standard (CEE and FCoCEE) products in early 2009. Clients who are evaluating deployment of such products should ask themselves several questions. Does the product support input from the CEE Authors group and the T11 FC-BB-5 standard? If not, will the vendor provide a free upgrade to the de-facto standard version in the future? Does the product support intermediary switches (i.e., a CEE Cloud), and if not, will the vendor provide a free upgrade to enable a CEE cloud in the future ?

Finally, there are real issues of management tools, behavior and organization that simply cannot be ignored in the convergence conversation. The management tools need to be vetted in real-world 24x7 environments. Today, the SAN and LAN are managed by two different organizations. Convergence implies that the server and storage groups within an enterprise need to entrust their GDPS STP, Enterprise System Connection (ESCON), Fiber Connection (FICON) and other well-established, mission-critical channels to the network group. The not-inconsequential political ramifications of such a transformation across the enterprise must be managed carefully, gracefully and gradually. Managers of large data centers should work with an IT company that has the service skills necessary to undergo this organizational transformation in careful steps.

For the foreseeable future, large data centers will continue to manage a variety of protocols when it comes to interconnecting over an extended distance. And that means WDM will continue to play its integral role in enterprise networking - as the flexible, protocol-agnostic cloud for converging all of the multi-protocol traffic, interconnecting critical data centers, and delivering necessary levels of performance, bandwidth and protection.

More Stories By Casimer DeCusatis

Dr. Casimer DeCusatis is an IBM Distinguished Engineer and Technical Executive based in Poughkeepsie, N.Y.

He is an IBM Master Inventor with over 70 patents, and recipient of several industry awards. He holds an MS and PhD from Rensselaer Polytechnic Institute, and a BS in the Engineering Science Honors Program from the Pennsylvania State University.

More Stories By Todd Bundy

Todd Bundy has 26 years of experience in the storage networking industry. He is a recognized expert in SAN and optical networking, and specializes in storage applications over various types of networks to meet corporate contingency plans. Throughout his career, Todd has participated in many successful large-scale disaster recovery and data center consolidation projects with companies like IBM, EMC, HDS, HP and Sun using ADVA FSP (Fiber Service Platform) WDMs. He has a degree in communications and business administration from Boston College.

Comments (0)

Share your thoughts on this story.

Add your comment
You must be signed in to add a comment. Sign-in | Register

In accordance with our Comment Policy, we encourage comments that are on topic, relevant and to-the-point. We will remove comments that include profanity, personal attacks, racial slurs, threats of violence, or other inappropriate material that violates our Terms and Conditions, and will block users who make repeated violations. We ask all readers to expect diversity of opinion and to treat one another with dignity and respect.

IoT & Smart Cities Stories
Dion Hinchcliffe is an internationally recognized digital expert, bestselling book author, frequent keynote speaker, analyst, futurist, and transformation expert based in Washington, DC. He is currently Chief Strategy Officer at the industry-leading digital strategy and online community solutions firm, 7Summits.
Digital Transformation is much more than a buzzword. The radical shift to digital mechanisms for almost every process is evident across all industries and verticals. This is often especially true in financial services, where the legacy environment is many times unable to keep up with the rapidly shifting demands of the consumer. The constant pressure to provide complete, omnichannel delivery of customer-facing solutions to meet both regulatory and customer demands is putting enormous pressure on...
IoT is rapidly becoming mainstream as more and more investments are made into the platforms and technology. As this movement continues to expand and gain momentum it creates a massive wall of noise that can be difficult to sift through. Unfortunately, this inevitably makes IoT less approachable for people to get started with and can hamper efforts to integrate this key technology into your own portfolio. There are so many connected products already in place today with many hundreds more on the h...
The standardization of container runtimes and images has sparked the creation of an almost overwhelming number of new open source projects that build on and otherwise work with these specifications. Of course, there's Kubernetes, which orchestrates and manages collections of containers. It was one of the first and best-known examples of projects that make containers truly useful for production use. However, more recently, the container ecosystem has truly exploded. A service mesh like Istio addr...
Digital Transformation: Preparing Cloud & IoT Security for the Age of Artificial Intelligence. As automation and artificial intelligence (AI) power solution development and delivery, many businesses need to build backend cloud capabilities. Well-poised organizations, marketing smart devices with AI and BlockChain capabilities prepare to refine compliance and regulatory capabilities in 2018. Volumes of health, financial, technical and privacy data, along with tightening compliance requirements by...
Charles Araujo is an industry analyst, internationally recognized authority on the Digital Enterprise and author of The Quantum Age of IT: Why Everything You Know About IT is About to Change. As Principal Analyst with Intellyx, he writes, speaks and advises organizations on how to navigate through this time of disruption. He is also the founder of The Institute for Digital Transformation and a sought after keynote speaker. He has been a regular contributor to both InformationWeek and CIO Insight...
Andrew Keys is Co-Founder of ConsenSys Enterprise. He comes to ConsenSys Enterprise with capital markets, technology and entrepreneurial experience. Previously, he worked for UBS investment bank in equities analysis. Later, he was responsible for the creation and distribution of life settlement products to hedge funds and investment banks. After, he co-founded a revenue cycle management company where he learned about Bitcoin and eventually Ethereal. Andrew's role at ConsenSys Enterprise is a mul...
To Really Work for Enterprises, MultiCloud Adoption Requires Far Better and Inclusive Cloud Monitoring and Cost Management … But How? Overwhelmingly, even as enterprises have adopted cloud computing and are expanding to multi-cloud computing, IT leaders remain concerned about how to monitor, manage and control costs across hybrid and multi-cloud deployments. It’s clear that traditional IT monitoring and management approaches, designed after all for on-premises data centers, are falling short in ...
In his general session at 19th Cloud Expo, Manish Dixit, VP of Product and Engineering at Dice, discussed how Dice leverages data insights and tools to help both tech professionals and recruiters better understand how skills relate to each other and which skills are in high demand using interactive visualizations and salary indicator tools to maximize earning potential. Manish Dixit is VP of Product and Engineering at Dice. As the leader of the Product, Engineering and Data Sciences team at D...
Dynatrace is an application performance management software company with products for the information technology departments and digital business owners of medium and large businesses. Building the Future of Monitoring with Artificial Intelligence. Today we can collect lots and lots of performance data. We build beautiful dashboards and even have fancy query languages to access and transform the data. Still performance data is a secret language only a couple of people understand. The more busine...