TRILL in the Data Center: Look Before You Leap

White Paper

TRILL in the Data Center:
Look Before You Leap
Understanding Fundamental Issues with TRILL

Copyright © 2012, Juniper Networks, Inc. 1

White Paper - TRILL in the Data Center: Look Before You Leap

Table of Contents
Executive Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3
Introduction. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3
What is TRILL?. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3
What Do Modern Data Centers Need?. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4
Fundamental Issues with TRILL. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5
Early Adopters of TRILL Face Vendor Lock-In. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6
Conclusion. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6
About Juniper Networks. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6

Table of Figures
Figure 1: Typical TRILL-based network in the data center. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4

2 Copyright © 2012, Juniper Networks, Inc.


Executive Summary
Juniper Networks has a long history of supporting industry standards that are mature and enable customers to
effectively solve their networking problems. Therefore, before jumping on the Transparent Interconnection of Lots of
Links (TRILL) bandwagon, Juniper took a close look at how the protocol proposes to solve the fundamental issues
plaguing the data center, specifically as they relate to connecting infrastructure designed for cloud computing. This
white paper outlines the results of those reviews and presents Juniper’s conclusions about TRILL.

Introduction
TRILL was originally intended to fix specific scaling deficiencies inherent to Ethernet bridging while retaining its plug-
and-play nature. While its primary application was for large distributed campus networks or large Layer 2 networks,
TRILL is now being heralded by some experts as a data center solution. However, the requirements of the modern data
center—such as exceedingly high levels of performance, any-to-any connectivity, virtual machine (VM) mobility, and
more—are dramatically different than the needs of the campus.
The current hype around TRILL is primarily generated by legacy vendors who have a significant presence in the data
center—an installed base they are desperate to protect. Vendors promoting TRILL in the data center will eventually
discover that there are a number of problems that it doesn’t solve, and they will attempt to fix those problems through
patches, resulting in the same level of complexity that disqualified legacy Ethernet as a data center solution in the first
place. Customers who implement TRILL in their data centers will pay dearly for this incremental approach, since they
are required to replace their existing infrastructure, which merely increases their costs.
While TRILL is a step in the right direction, it increases complexity and prevents fundamental advances that could
truly improve the data center network. Customers considering TRILL must ask themselves: Why should we invest in a
technology that only supports Layer 2—one that introduces additional cost and complexity to the network—when the
majority of our data center application traffic is Layer 3?

What is TRILL?
TRILL is a network protocol invented by Radia Perlman to remove certain deficiencies of bridged Ethernet networks
in large Layer 2 campuses. The context of TRILL’s creation is important, because it speaks directly to the problems it
can—and cannot—solve. The following quote, from a 2006 interview with Perlman in Network World, is revealing:
“A couple of years back there was this Boston Globe article about a hospital network melting down and in the
middle of it was mentioned the spanning tree algorithm…eventually we tracked down the company providing
the switches and indeed it was a giant bridged network… One of the things I’m trying to do now, given that
we’re stuck with IP, is come up with something that gives you the advantages of bridging so it can be all zero-
configuration within a campus and all look like one big prefix and not be confined to just transmitting data
along the spanning tree…”
TRILL has since been taken up by the IETF in an attempt to create a new standard. In the meantime, various
implementations by various Ethernet switching vendors include a number of differences, even as they operate under
the cloak of a single “open standard” called TRILL. Ironically, contrary to what a “standard” is supposed to do, these
different implementations don’t even interoperate with one another. TRILL introduces a new encapsulation layer, a set
of associated control plane protocols, and a new network device type called an RBridge that sits midway between a
router and a bridge. The encapsulation layer adds a hop count, a VLAN tag, and ingress and egress RBridge identifiers,
along with a few additional control bits.
Figure 1 shows a typical TRILL-based network deployment in the data center.



Layer 2 Only

RBridge RBridge
Core Core

RBridge RBridge RBridge
Edge Edge Edge

Servers Compute Servers Compute
and Storage and Storage
Figure 1: Typical TRILL-based network in the data center

In a typical TRILL application, an IP packet encapsulated in an Ethernet frame would have two additional
encapsulation layers added—a TRILL header and an outer Ethernet header, assuming that an Ethernet cloud were used
to connect the RBridges.

What Do Modern Data Centers Need?
Today’s data centers are complex, inefficient, unreliable, power hungry, expensive, and difficult to operate. To a large
extent, these problems arise from limitations in the networking technologies used to interconnect computing, storage,
and services inside the data center. A careful examination of these problems shows that any networking technology
designed for the data center today must provide the ability to treat infrastructure resources as fully fungible pools that
can be dynamically and rapidly partitioned without the infrastructure or the applications knowing details about each
other1. This capability is the key to simplicity, efficiency, and security. The networking technology in the data center
must also be able to connect resources to each other at very high speeds with no apparent limitations in the network.
This capability is the key to high performance as well as efficiency improvements. Finally, the technology must be able
to scale to provide these capabilities across a wide range of data centers of all sizes without requiring a redesign.
These high-level capabilities can be broken down into seven specific characteristics that a data center network
technology must provide:
1. Any-to-any connectivity with fairness and full non-blocking: A set of interfaces must be able to send and
receive packets to and from any other set with no restrictions or preplanning. Specifically, this includes the ability
to absorb and adapt to rapid changes in the rate of transmission, the number of active senders, and the number
of active receivers. In all cases, the full bandwidth of a target interface (or interfaces) is shared equally by all
contending interfaces instantaneously and continuously, including the special case of one interface sending to
just one other interface; this equal sharing is referred to as “fairness.” Finally, the only apparent congestion is due
to the limited bandwidth of ingress and egress interfaces. Any congestion of egress interfaces does not affect
ingress interfaces sending to uncongested interfaces; this noninterference is referred to as “non-blocking.”
2. Low latency and jitter: The technology must provide low interface-to-interface latency—on the order of a
handful of microseconds. Latency should also grow slowly with offered traffic load. Finally, the technology
should provide low jitter, which is an instantaneous variation of latency.

1
In a multi-tenant data center, it is completely impractical to assume that the infrastructure knows about the details of any application, or that applications know about
details of the infrastructure.



3. No packet drops under congestion: When the instantaneous rate of incoming packets exceeds the
instantaneous rate of outgoing packets, the technology must signal the source (e.g., servers or VMs) causing the
congestion to slow down sufficiently so that the arrival rate matches the departure rate. This throttling should
occur rapidly and continuously to ensure that input and output rates are matched in a smooth manner.
4. Linear cost and power scaling: The cost and power consumption of the network infrastructure must increase
linearly with the number of server ports “N” it needs to support. This is in sharp contrast to traditional
approaches, where cost and power grow nonlinearly as they scale up.
5. Support of virtual networks and services: The technology must implement virtual Layer 2 and Layer 3 networks
to support multiple tenants, each running multitier applications. Complex security and services requirements
should be supported by the insertion of Layer 4-7 processing at any point in an application’s workflow. Full
mobility of VMs from any interface to any other interface must also be supported. Virtual network support should
not compromise any of the other properties.
6. Modular distributed implementation that is highly reliable and scalable: The technology should be built
using modular hardware and software components that are distributed and federated to provide high levels of
redundancy. The modular implementation should be designed to permit increasing or decreasing the number of
interfaces while the system is running, a property called “dynamic scalability.”
7. Single logical device: Despite its distributed implementation, the technology should act as a single, logical
packet switching device. The complexity of its distributed implementation should be transparent without
removing any of the desirable properties such as high reliability or dynamic scalability.

Fundamental Issues with TRILL
The capabilities provided by any networking technology span four dimensions:
• The data plane is responsible for forwarding bits.
• The control plane is responsible for producing the data structures that the data plane uses to forward packets.
• The services plane specifies how Layer 4-7 services are provided.
• The management plane supports the way the networking technology is operated by the facilities manager.
While TRILL does address some of the issues for the data and control planes, it does nothing for the services and
management planes. Specifically, when considered as a solution for data center networking problems, TRILL exhibits
the following deficiencies:
• Layer 2 only: TRILL completely ignores IP packet forwarding. This is a problem, since most packets in the data center
originate as IP. One-armed routers attached to the TRILL core perform IP forwarding (unicast and multicast), where they
quickly become a bottleneck. The additional cost of these IP routers is never mentioned by vendors pushing TRILL.
• Multitier architecture with poor economics: Any TRILL-based solution has at least two tiers—edge and core—where
packet processing occurs. Packet processing in the RBridges is more complex than Ethernet forwarding because it
involves additional processing steps on both ingress and egress. In real-world deployments, packet processing in the core
is just as complex as with traditional core Ethernet switches as requirements that go beyond Layer 2 unicast are added.
In large networks, the core itself requires multiple levels of forwarding. This additional complexity means that TRILL can
never compete economically with a “clean sheet design” that is not burdened with this extra processing.
• Multicast scaling issues: While TRILL does reduce the Layer 2 unicast forwarding state in the network core, it does not
reduce the multicast forwarding state. Additionally, the control plane complexity associated with setting up multicast
trees is bound to present severe operational and troubleshooting challenges in real deployments.
• Large broadcast domains: Since crossing VLANs is expensive in a TRILL solution, TRILL inherently pushes a network
designer to artificially increase the size of VLANs. This has two consequences: first, large VLANs create flooding issues
that are hard to handle; second, this artificial inflation diminishes the effectiveness of VLANs as a mechanism for
separating resources belonging to different applications, organizations, or tenants. Security is a grave concern in TRILL-
based architectures.
• Congestion management remains unsolved: TRILL says nothing about how congestion is handled in the network. In
fact, it makes the congestion problem much harder to solve by creating a network with multiple layers of forwarding.
The TRILL core elements are in a different address plane than the TRILL edge elements and endpoints, so the core
has no idea how to forward congestion messages to endpoints. Additionally, TRILL networks will encounter the same
fundamental problem that early Backward Congestion Notification (BCN) schemes faced—the time constant of the
congestion control loop is greater than the time constant of the congestion events themselves.



• Multi-tenancy out of scope: Until today, TRILL has not tackled the problem of overlapping IP and Ethernet addresses
between organizations, nor has it addressed how to provide for a large number of VLANs. Both of these are prerequisites
for data centers that host multiple tenants.
• Operational simplicity of a fabric: TRILL edges and cores don’t present the abstraction of a single logical device.
Instead, these TRILL network elements must be configured and managed as individual devices, posing a significant
operational burden to data center operators. Orchestration is also a challenge in a TRILL architecture, as the complexity
of the network is fully exposed to management applications.
• Host of other inefficiencies: Dual homing servers, flooding, VM mobility within and across data centers, multi-pathing for
Layer 3 and Fibre Channel over Ethernet (FCoE)—these are just some of the areas where TRILL is insufficient.

Early Adopters of TRILL Face Vendor Lock-In
Vendors frequently hijack the standards setting process in order to give the impression of openness for a new
technology. In fact, legacy vendors have no interest in creating open standards because this would create a level
playing field for their competitors.
Customers, on the other hand, like open standards, so the challenge for vendors is to operate under the guise of an
open standard while making incompatible or proprietary changes to the technology.
Such is the case with TRILL. Incumbent vendors are encouraging customers to rip and replace their existing network
gear with proprietary implementations of TRILL, promising that they will be compatible with the eventual TRILL
standard in the future. In doing so, incumbent vendors lock customers into their proprietary version of the protocol,
effectively forcing them to buy their gear in the future, since the proprietary technology won’t cleanly interoperate with
the final standard.

Conclusion
The problem that TRILL sets out to solve was relevant in the days of “switch when you can, route when you must”—a
time when routing was so much more expensive than switching that Ethernet networks had to be made as large as
possible. Today, forwarding technology has advanced to the point where IP packets can be forwarded as easily and
efficiently as Ethernet packets, so there is no need to maximize the scale of Layer 2 networks.
In the final analysis, the problems that TRILL was designed to solve represent only a fraction of those faced by
modern data centers. While TRILL is a positive step in the right direction, it favors an incremental approach that
increases complexity while leaving many issues unresolved. In its present form, TRILL is an immature technology that
unnecessarily creates new issues without substantially solving any of the problems in the data center. Juniper will
reevaluate this technology when its scope has been broadened to address the networking challenges found in cloud
data centers.

About Juniper Networks
Juniper Networks is in the business of network innovation. From devices to data centers, from consumers to cloud
providers, Juniper Networks delivers the software, silicon and systems that transform the experience and economics
of networking. The company serves customers and partners worldwide. Additional information can be found at
www.juniper.net.

Corporate and Sales Headquarters APAC Headquarters EMEA Headquarters To purchase Juniper Networks solutions,
Juniper Networks, Inc. Juniper Networks (Hong Kong) Juniper Networks Ireland please contact your Juniper Networks
1194 North Mathilda Avenue 26/F, Cityplaza One Airside Business Park representative at 1-866-298-6428 or
Sunnyvale, CA 94089 USA 1111 King’s Road Swords, County Dublin, Ireland authorized reseller.
Phone: 888.JUNIPER (888.586.4737) Taikoo Shing, Hong Kong Phone: 35.31.8903.600
or 408.745.2000 Phone: 852.2332.3636 EMEA Sales: 00800.4586.4737
Fax: 408.745.2100 Fax: 852.2574.7803 Fax: 35.31.8903.601
www.juniper.net

Copyright 2012 Juniper Networks, Inc. All rights reserved. Juniper Networks, the Juniper Networks logo, Junos,
NetScreen, and ScreenOS are registered trademarks of Juniper Networks, Inc. in the United States and other
countries. All other trademarks, service marks, registered marks, or registered service marks are the property of
their respective owners. Juniper Networks assumes no responsibility for any inaccuracies in this document. Juniper
Networks reserves the right to change, modify, transfer, or otherwise revise this publication without notice.

2000408-003-EN May 2012 Printed on recycled paper


TRILL in the Data Center: Look Before You Leap

Recomendados

Recomendados

Más contenido relacionado

Destacado

Destacado (7)

Más de Juniper Networks

Más de Juniper Networks (20)

Último

Último (20)

TRILL in the Data Center: Look Before You Leap