Lippis Report 184: Network Services to Differentiate Next Generation of Campus Core Switches

During the middle of 2012, a few firms will introduce core switches for campus networking. Many of these products will be based upon merchant silicon such as HP Networking’s A10500 Series Enterprise Core Switch. While these products will boast performance advantage, they will find it difficult to win share against established firms such as Cisco’s Catalyst 6500, thanks to its investment in network services. In this Lippis Report Research Note 184, we explore the importance of network services and their role in campus network design.

Catalyst 6500 Sup2T Network Virtualization via MPLS/VPLS Performance

Watch the Video

Modern corporate networks are under increasing pressure to support a wider variety of applications, thanks to mobile and cloud computing, desktop virtualization plus video traffic having skyrocketed. Not only are bandwidth rates increasing from 1 to 10 to 40 GbE, but most importantly, network services are needed to manage and support a different application portfolio mix and network access methods. Network services such as firewalls, WLANs, network diagnostics and monitoring plus application performance acceleration are needed to deliver a consistently excellent user experience. Cisco recently announced an upgrade to its popular Catalyst 6500 with the availability of the Supervisor 2T or Sup2T that included re-vamped high performance service modules to deliver these network services.

By all counts, Cisco’s upgrade of the Catalyst 6500 via its new Sup2T is its most ambitious and thoughtful yet for the venerable platform. The Sup2T is a 2-Terabit (Tb) platform that triples the previous Sup720 performance. Thanks to the support of Virtual Switching System (VSS), the platform allows two 2 Tbps switches to combine into a single 4 Tbps virtual switch. The Sup2T is a major upgrade to the most widely-deployed switching platform in campus and data center networking in the industry. But while these performance numbers are impressive, it’s the new Catalyst 6500’s network services that deliver most of the value, which is partially found in the Sup2T’s Policy Feature Card or PFC that increases NetFlow monitoring and a new TCAM design offering improved Access Control List (ACL), Quality of Service design options, encryption security and many other features.

Alcatel-Lucent OmniSwitchTM 6900-X40

Watch the Video

Cisco’s Catalyst 6500 is the firm’s most successful product with over 700,000 systems and110 million ports installed, worth some $42 billion in revenue over the years. This product’s success increases the stakes for Cisco as it introduces a major upgrade. Cisco had to consider backward and forward customer migration, increased competition and pricing pressure, especially as competitors are starting to offer core switches based upon merchant silicon. In short, Cisco had to eliminate the trade-off of innovation versus investment protection and find a way to deliver both simultaneously. The Lippis Report conducted the most comprehensive testing of the Catalyst 6500 Sup2T at Ixia’s iSimCity in November 2011 to verify Cisco’s performance and upgradability claims. While it’s impossible to test all of the Catalyst 6500’s new 200-plus features within the Sup2T, we rather focus on a select few that will have the widest impact on IT business leaders’ product acquisition decision process. The full report is found here; below are highlights.

Compatibility, Upgradeability and Investment Protection Test

In this test, we look to measure how smooth the upgrade from Sup720 to Sup2T is. What IT business leaders are looking for are incremental network upgrades with minimal disruption versus major disruption that usually accompanies a significant and, at times, a not so significant network upgrade. Therefore, we swap out Sup720 for Sup2T and bring up existing service modules and line cards. Remember that line cards represent the largest investment in switching equipment, so we demonstrate that older line cards interoperate at high performance when the new Sup2T replaces the Sup720.

Building a Smart Virtual Network Infrastructure with IBM

Listen to the Podcast

Results: We found that upgrading the Catalyst 6500 from Sup720 to Sup2T within the 6513-E chassis was straightforward and compatible with existing line cards and service modules. Those who invested in the E series chassis (i.e., 6503-E to 6513-E) and purchased line cards and service modules will find that this investment is protected and enhanced as new network services such as NetFlow, TCAM architecture improvements, encryption, deeper QoS granularity, Access Control Lists (ACLs), dry-run and atomic commit, et al, are added during supervisor upgrade from 720 to 2T.

We verified backward compatibility of the 6513-E Catalyst 6500 Sup2T with existing service modules, bus-based and CFC-based line cards along with feature and performance benefits afforded by the Sup2T (PFC4). We further verify the upgradability of existing modules which currently employ the DFC3 (B and C) daughter card with feature and performance benefits afforded by the DFC4 upgrade. We also verify the migration of current IOS configuration (as applicable to existing line cards) as well as their use of existing interface transceivers (e.g., SFP & X2). Finally, we verify the Sup2T when combined with the 6513-E chassis enables high-performance (dual-fabric) line cards to operate in the upper 6 slots.

Virtualization Comes to the WAN with Cost and Performance Improvements

Listen to the Podcast

In the same 6513-E chassis, we replaced the Sup720 for Sup2T, upgraded the line cards in slots 1 and 2 for the new 6908s, upgraded the DFC4 daughter cards in slots 12 and 13 and kept the same service modules. All of this was done while the Catalyst 6500 was operational. The Sup2T triples the performance of Sup720 while adding greater network service features such as Flexible NetFlow monitoring, Mac-Sec of 802.1ae based encryption security, WLAN integration and firewall protection.

Switching Performance Test

Switching performance in enterprise networks is becoming increasingly important, as IT responsibility has been split between employees and IT departments, thanks to BYOD or Bring Your Own Device, and IT consumerization. As a result, the number of devices on the network has increased significantly as employees bring smartphones and other mobile devices into the work force. These devices and their applications are driving unforeseen network requirements in terms of performance and support of both IPv4 and IPv6 as many mobile devices are now set for IPv6 as the default.

A Comprehensive Testing of Cisco Systems Catalyst 6500 Sup2T

Get the White Paper

For IPv4 and IPv6, dual stack implementations are most popular where desktops and mobile devices run both IPv4 and IPv6, therefore, the network infrastructure needs to support both equally at high performance. IPv6 performance has not been on par with IPv4 until now. To demonstrate how the Catalyst 6500 upgrade with Sup2T has improved IPv6 performance, we measure IPv4 and IPv6 unicast and bidirectional traffic performance via RFC 2544.

Results: We test the Catalyst 6500 for throughput between popular enterprise network frame sizes ranging from 256 to 9216 byte size packets. We find that each WS-X6908-10G delivers IPv4 and IPv6 throughput at the theoretical maximum possible for packet sizes ranging from 256 to jumbo size 9216 at 10GbE.

Real Estate Firm Boosts Bandwidth, Cuts Cost with Talari

Get the White Paper

IP Multicast Test

IP Multicast traffic has been on the rise, thanks to the increased use of video services within the enterprise. Efficient use of multicast is important to interactive video, video surveillance, video dissemination, etc. Consider 500 to 1000 video surveillance cameras that need to stream their video to five or more locations within the enterprise, for regulation, storage, monitoring, etc. This is a popular requirement in gaming, retail, healthcare, etc. Streaming five streams per camera consumes a lot of bandwidth; therefore, using IP multicast reduces bandwidth consumption making video and other point-multipoint services efficient. Therefore, we test IP Multicast performance on the new catalyst 6500 Sup2T. This test stresses the packet replication ASIC built into the 6908-10G line cards for both point-multipoint and mesh or multipoint-multipoint configurations.

Results: For the point-multipoint configuration, the Catalyst 6500 Sup2T demonstrated zero packet loss or 100% throughput at line rate while a single 10GbE source was broadcast to 92 receivers.

For mesh multipoint-multipoint configuration, the Catalyst 6500 Sup2T demonstrated throughput performance that ranged from 49.8 Mpps to .53 Mpps for packet sizes that varied between 256 bytes to jumbo size or 9216 bytes. We find that the replication engine that is resident on Catalyst 6500 6908-10G line cards delivers multicast performance scale as there is no performance penalty for point-multipoint and multipoint-multipoint. This is due to the Sup2T having an improved hashing algorithm to support larger IP Multicast flows over the Sup720.

Top 5 Network Performance Management Mistakes and How to Avoid Them

Get the White Paper

Access Control List Test

Access Control List or ACL are important tools in the configuration and customization of network attributes, especially with the Catalyst 6500. In the Catalyst 6500 upgrade with Sup2T, the TCAM has been both increased and its architecture improved. For ACL, one major concern was the lack of visibility of overflowing the TCAM when new ACL scripts were submitted, which would disrupt network operation. Updating ACLs occur infrequently and over a long period of time. As such multiple network engineers working on the same network may not even be aware of previous ACL updates. Further, an ACL update may drive multiple ACE (ACE = Access Control Entries), which occupy more TCAM resources than anticipated and thus over consume this resource. Therefore, Cisco developed the ACL Dry Run and ACL Atomic Commit to mitigate this scenario from occurring.

Results: We verify that this new efficient use of TCAM and
ACL safeguards perform as stated.

System Network Test Configuration: MPLS/VPLS/VSS

To test MPLS/VPLS and VSS throughput performance, we populate two Catalyst 6500 WS-C6513-Es with eight 10GbEports each via 6908-10G modules connected directly to Ixia test equipment. The Catalyst 6500s are connected via8 x 10G Distributed EtherChannels. This configuration created a full end-end 80Gbs path of full-mesh traffic; typical in the real world.

Wireless on the Wall New Converged Wireless and Wired Edge for Mobile Users

Get the White Paper

The test data result show that throughput performance is consistent independent upon protocol that being MPLS, VPLS and VSS. A contributing factor to the differences in throughput is found in different headers associated for each protocol. This result could not occur in the older generation of Catalyst 6500 with Sup720 with its 40Gbs per module backplane access speed.

Network Encryption with 802.1ae MACSec

We tested performance for 802.1ae MACSec to verify that there was no throughput performance degradation when encryption was enabled minus the additional 16 byte overhead of 802.1ae keys. MACSec encryption has become increasingly popular and important to campus network design, but previous switch performance degraded when forwarding encrypted traffic. Here we verify that the Catalyst 6500 does not suffer throughput performance degradation while MACSec traffic is being forwarded.

We tested the Catalyst 6500 via the cPacket Networks cTapSmart 10G passive probe to verify traffic flows were either MACsec encrypted or unencrypted. We found that there is no material difference in throughput performance, other than 802.1ae encryption key overhead, thanks to 16 additional bytes per packet.


We found that upgrading the Catalyst 6500 from Sup720 to Sup2T was straightforward and added significant value in the areas of MACsec encryption, improved ACL capabilities and IPv4/IPv6/MPLS/VPLS/VSS throughput performance. In addition, we found that the Sup2T supported existing service models, such as Network Analysis (NAM), Wireless (WiSM), Application Control Engine (ACE20), Firewall Service Module (FWSM) plus 6148A-GE, 6148E-GE with POE/POE+, 6724-SFP line cards plus 6704 and 6716 line cards after a trivial DFC3 to DFC4 daughter card swap. We found that line cards can be swapped and upgraded while the Sup2T is operational, avoiding off-hour scheduled downtime. In addition, we found that existing interface transceivers SFP and X2 being used in a Sup720 Catalyst 6500 can be reused with the Sup2T. Finally, we found that Sup720 IOS configurations may be copied and migrated to a Sup2T via a flash drive successfully upon boot up.

Much of the throughput performance advantages and scale of network services is due to custom ASICs resident in theSup2T, 6908-10G line cards and DFC4 daughter cards. We were particularly impressed with the ease of upgrade, the new ACL dry run and atomic commit plus MACsec performance.

For existing customers of Cisco’s Catalyst 6500 Sup720, we anticipate upgrade experiences similar, if not simpler, than ours as this test was conducted under tight time constraints with limited resources. It’s no wonder why the Catalyst 6500 is so popular as it offers a wide variety of network design options such as MPLS/VPLS/VSS. With the new upgrade to Sup2T and supporting line cards, we verify that throughput performance doubles over the Sup720 for IPv6, IP Multicast, MPLS/VLPS and VSS.

New entrants in the campus core market such as HP Networking A10500 later this year that boast pure performance without network services will find a chilly reception awaits them.

One Response to Lippis Report 184: Network Services to Differentiate Next Generation of Campus Core Switches

  1. Michael said:

    Let´s talk about pricing. Take a Cisco 6500 with Sup2T, add interface modules, redundant PSU, fabric and management modules. Add service contracts and compare against an A10500 setup.

    And of course, do not take list prices…