Tweet

Computer networking vendors have been increasing the speed and port density of their Ethernet switches while reducing power draw and price per port. But while Ethernet switching hardware marches on linearly, thanks to 10, 40 and 100GbE, networking software is taking a different historical path as the pace of compute and network technology evolution has diverged, with networking lagging. Highly virtualized server deployment has broken traditional networking approaches on multiple levels, for example. In response, the industry is now developing a “virtualized infrastructure” or “stack” to add network flexibility. To close the technology gap, Software-Defined Networking (SDN) is promoted as the new “organizing principle” to deliver network software and service value. While it will be, likely, years before SDN’s organizing principles take hold, I propose that these two industry activities are inexplicably linked and phased; here’s why…

Software-Defined Networking

There are multiple definitions of SDN. Making it even harder to pin down SDN, the definitions are evolving too. But this is common in a new breakout space for the computer networking industry that’s evolving fast. For this Lippis Report Research Note, we take the SDN definition that is based upon splitting the data plane or the forwarding hardware of an Ethernet switch from its control plane or the logic that controls how packets flow from ingress to egress. This split of data and control planes opens up an innovation injection point into networking that has not been previously available.

During 2011, a market has opened up for controllers. Currently Big Switch Networks, Nicira Networks and NEC are offering standalone centralized controllers. But limited controllers are also available in open source software, OpenStack and VMware’s vSphere/vCloud too. In addition Cisco’s IOS, Juniper’s Junos, Arista’s EOS, etc., are distributed controllers that may interoperate with centralized controllers in the future. In fact, Arista’s EOS already supports OpenFlow, OpenStack and vSphere/vCloud.

The link between the separated data and control plane is an open interface called OpenFlow. Now some end their SDN definition here, but this is just the beginning as the real promise of SDN are the applications that will reside upon the controller to address a wide range of networking issues and opportunities. In fact researchers at Princeton and Cornell are developing the Frenetic programming language that provides high-level network abstraction that gives programmers direct control over the network, allowing them to specify what they want the network to do without worrying about how to implement it.

One can imagine a wide range of applications residing upon a controller such as WAN optimization, traffic engineering optimization, load balancing, security services, etc. In essence, the control plan allows network services that are currently deployed as appliances to be virtualized appliances/applications much like applications that reside on top of a VM. It gets even more interesting, as a centralized control plane can be easily split in to many little control planes, each of which sees its own slice of the data plane topology. In traditional networking where control and data planes are one and the same and in each box, it is much harder to merge control planes and split data planes. It’s possible, but harder to keep complexity and stability in check over the long term. Splitting control plans can have huge value in public cloud multi-tenant or private cloud multi-team networking.

SDN and OpenFlow are at the early stages of its industry matriculation. But one thing is clear: SDN is an organizing principle whereas network software is developed by both network vendors and third parties, and network services are virtualized. SDN thus represents a new industry order and structure as to how value is added to networks. But I digress. The real issue today is solving network inflexibility in the face of highly virtualized data centers.

Enter the “Virtualized Stack” or Virtualized Infrastructure”

Virtualized server deployment has been propelled en masse, thanks to increased data center efficiency, by delivering the same or greater application workload with a reduced number of servers. While this is good, many IT business leaders are now realizing huge consequences to highly virtualized data centers that span from IP address change management to application management.

At the IP address level, networking has become extremely rigid within virtualized environments, slowing down process, limiting moves and changes as well as elongating the time to spin up an application that resides within a VM. Necessary network services to support the virtual cloud infrastructure, such as IP address assignment and management, are still performed largely with manual tools and processes, such as spreadsheets shuffled between various departments or operational groups, which can result in days of delay for something as simple as assigning an IP address to a VM. Contrast that with the virtual server administrator. Virtual instances of servers and machines can be dynamically provisioned, migrated and shut down by a virtual server administrator in minutes.

Moving up the stack, challenges are rooted in application management plus Layer 4-7 services such as WAN optimization, Application Delivery Controllers and security, especially in environments that include multiple hypervisors, a wide variety of workload types and shifting virtual machines.

For example, the new challenges of enterprise application management in virtualized data centers include: what type of and location of network intelligence is required when multiple hypervisors and various workloads exist and shift? Also how do operations groups maintain consistent security policy across both virtualized and non-virtualized environments consistently? And how do operations groups monitor and maintain application flow visibility?

Cisco

Cisco, for example, is addressing these issues via its Virtualization Stack and is now organizing its products around this initiative. Three components define Cisco’s virtualization stack, those being: 1) virtual networking, 2) virtual security and application networking services and 3) orchestration and provisioning. An important part of Cisco’s strategy is the virtualization of appliances such as its VSG or Virtual Security Gateway, the ASA 1000v, the support of VXLAN, the Nexus 1000v, etc.

Brocade, F5, Citrix

But F5, Citrix and Brocade are all virtualizing their appliances, moving away from physical single application appliances to an integrated virtualized suite. One can imagine that these virtualized applications will some time reside upon an SDN controller as their next stage of evolution. In addition each application delivery vendor has a way for programmers to control application network behavior. For example, Brocade recently launched OpenScript, a Perl-based scripting language used to modify the content of and control delivery of packets at Layer 4 through Layer 7 on its ServerIron ADX products. These scripting languages could be standardized and reside within an SDN controller.

Embrane

A good example of what the virtualized Layer 4-7 future may hold is that of a start-up firm called Embrane.

Embrane has virtualized server load balancing, firewalls and VPN termination and placed them upon a distributed software platform called heleos. Heleos runs on x86 servers and any hypervisor. It leverages a distributed virtual architecture that decouples network services functionality from the underlying physical infrastructure and hypervisor technology that it says provides high scalability, flexibility and performance.

IBM & NEC

IBM and NEC offer the best example of a commercial SDN offering with OpenFlow. NEC’s pFlow OpenFlow controller that resides within an IBM server manipulates IBM System Networking G8264 OpenFlow switch’s flow table. The link between the two is OpenFlow 1.0.0. The NEC pFlow controls traffic, discovers topology, gathers stats and other functions while the G8264 forwards traffic based upon these flow commands.

What’s impressive about the IBM/NEC SDN solution is that it has customers such as: Tervela validated the IBM and NEC OpenFlow solution ensures predictable performance of Big Data for complex and demanding business environments. Selerity’s IBM and NEC’s OpenFlow solution improved real-time
decision-making for global financial markets. Stanford’s IT Department chose IBM and NEC’s OpenFlow solution to deliver network capacity on-demand to its academic community. What’s important about these use cases is that IBM is communicating SDN via OpenFlow’s value in business terms, which will only increase as industry adoption accelerates.

In essence the SDN market has started, and as its technology underpinnings solidify, many of today’s network services will fall under the SDN umbrella. In fact, nearly all network vendors are launching SDN programs as a new way to communicate existing product value and their evolution into a SDN. Just like the Appian Way where all roads lead to Rome, all network services may very well lead to an SDN.

Tweet

During the middle of 2012, a few firms will introduce core switches for campus networking. Many of these products will be based upon merchant silicon such as HP Networking’s A10500 Series Enterprise Core Switch. While these products will boast performance advantage, they will find it difficult to win share against established firms such as Cisco’s Catalyst 6500, thanks to its investment in network services. In this Lippis Report Research Note 184, we explore the importance of network services and their role in campus network design.

Modern corporate networks are under increasing pressure to support a wider variety of applications, thanks to mobile and cloud computing, desktop virtualization plus video traffic having skyrocketed. Not only are bandwidth rates increasing from 1 to 10 to 40 GbE, but most importantly, network services are needed to manage and support a different application portfolio mix and network access methods. Network services such as firewalls, WLANs, network diagnostics and monitoring plus application performance acceleration are needed to deliver a consistently excellent user experience. Cisco recently announced an upgrade to its popular Catalyst 6500 with the availability of the Supervisor 2T or Sup2T that included re-vamped high performance service modules to deliver these network services.

By all counts, Cisco’s upgrade of the Catalyst 6500 via its new Sup2T is its most ambitious and thoughtful yet for the venerable platform. The Sup2T is a 2-Terabit (Tb) platform that triples the previous Sup720 performance. Thanks to the support of Virtual Switching System (VSS), the platform allows two 2 Tbps switches to combine into a single 4 Tbps virtual switch. The Sup2T is a major upgrade to the most widely-deployed switching platform in campus and data center networking in the industry. But while these performance numbers are impressive, it’s the new Catalyst 6500’s network services that deliver most of the value, which is partially found in the Sup2T’s Policy Feature Card or PFC that increases NetFlow monitoring and a new TCAM design offering improved Access Control List (ACL), Quality of Service design options, encryption security and many other features.

Cisco’s Catalyst 6500 is the firm’s most successful product with over 700,000 systems and110 million ports installed, worth some $42 billion in revenue over the years. This product’s success increases the stakes for Cisco as it introduces a major upgrade. Cisco had to consider backward and forward customer migration, increased competition and pricing pressure, especially as competitors are starting to offer core switches based upon merchant silicon. In short, Cisco had to eliminate the trade-off of innovation versus investment protection and find a way to deliver both simultaneously. The Lippis Report conducted the most comprehensive testing of the Catalyst 6500 Sup2T at Ixia’s iSimCity in November 2011 to verify Cisco’s performance and upgradability claims. While it’s impossible to test all of the Catalyst 6500’s new 200-plus features within the Sup2T, we rather focus on a select few that will have the widest impact on IT business leaders’ product acquisition decision process. The full report is found here; below are highlights.

Compatibility, Upgradeability and Investment Protection Test

In this test, we look to measure how smooth the upgrade from Sup720 to Sup2T is. What IT business leaders are looking for are incremental network upgrades with minimal disruption versus major disruption that usually accompanies a significant and, at times, a not so significant network upgrade. Therefore, we swap out Sup720 for Sup2T and bring up existing service modules and line cards. Remember that line cards represent the largest investment in switching equipment, so we demonstrate that older line cards interoperate at high performance when the new Sup2T replaces the Sup720.

Results: We found that upgrading the Catalyst 6500 from Sup720 to Sup2T within the 6513-E chassis was straightforward and compatible with existing line cards and service modules. Those who invested in the E series chassis (i.e., 6503-E to 6513-E) and purchased line cards and service modules will find that this investment is protected and enhanced as new network services such as NetFlow, TCAM architecture improvements, encryption, deeper QoS granularity, Access Control Lists (ACLs), dry-run and atomic commit, et al, are added during supervisor upgrade from 720 to 2T.

We verified backward compatibility of the 6513-E Catalyst 6500 Sup2T with existing service modules, bus-based and CFC-based line cards along with feature and performance benefits afforded by the Sup2T (PFC4). We further verify the upgradability of existing modules which currently employ the DFC3 (B and C) daughter card with feature and performance benefits afforded by the DFC4 upgrade. We also verify the migration of current IOS configuration (as applicable to existing line cards) as well as their use of existing interface transceivers (e.g., SFP & X2). Finally, we verify the Sup2T when combined with the 6513-E chassis enables high-performance (dual-fabric) line cards to operate in the upper 6 slots.

In the same 6513-E chassis, we replaced the Sup720 for Sup2T, upgraded the line cards in slots 1 and 2 for the new 6908s, upgraded the DFC4 daughter cards in slots 12 and 13 and kept the same service modules. All of this was done while the Catalyst 6500 was operational. The Sup2T triples the performance of Sup720 while adding greater network service features such as Flexible NetFlow monitoring, Mac-Sec of 802.1ae based encryption security, WLAN integration and firewall protection.

Switching Performance Test

Switching performance in enterprise networks is becoming increasingly important, as IT responsibility has been split between employees and IT departments, thanks to BYOD or Bring Your Own Device, and IT consumerization. As a result, the number of devices on the network has increased significantly as employees bring smartphones and other mobile devices into the work force. These devices and their applications are driving unforeseen network requirements in terms of performance and support of both IPv4 and IPv6 as many mobile devices are now set for IPv6 as the default.

For IPv4 and IPv6, dual stack implementations are most popular where desktops and mobile devices run both IPv4 and IPv6, therefore, the network infrastructure needs to support both equally at high performance. IPv6 performance has not been on par with IPv4 until now. To demonstrate how the Catalyst 6500 upgrade with Sup2T has improved IPv6 performance, we measure IPv4 and IPv6 unicast and bidirectional traffic performance via RFC 2544.

Results: We test the Catalyst 6500 for throughput between popular enterprise network frame sizes ranging from 256 to 9216 byte size packets. We find that each WS-X6908-10G delivers IPv4 and IPv6 throughput at the theoretical maximum possible for packet sizes ranging from 256 to jumbo size 9216 at 10GbE.

IP Multicast Test

IP Multicast traffic has been on the rise, thanks to the increased use of video services within the enterprise. Efficient use of multicast is important to interactive video, video surveillance, video dissemination, etc. Consider 500 to 1000 video surveillance cameras that need to stream their video to five or more locations within the enterprise, for regulation, storage, monitoring, etc. This is a popular requirement in gaming, retail, healthcare, etc. Streaming five streams per camera consumes a lot of bandwidth; therefore, using IP multicast reduces bandwidth consumption making video and other point-multipoint services efficient. Therefore, we test IP Multicast performance on the new catalyst 6500 Sup2T. This test stresses the packet replication ASIC built into the 6908-10G line cards for both point-multipoint and mesh or multipoint-multipoint configurations.

Results: For the point-multipoint configuration, the Catalyst 6500 Sup2T demonstrated zero packet loss or 100% throughput at line rate while a single 10GbE source was broadcast to 92 receivers.

For mesh multipoint-multipoint configuration, the Catalyst 6500 Sup2T demonstrated throughput performance that ranged from 49.8 Mpps to .53 Mpps for packet sizes that varied between 256 bytes to jumbo size or 9216 bytes. We find that the replication engine that is resident on Catalyst 6500 6908-10G line cards delivers multicast performance scale as there is no performance penalty for point-multipoint and multipoint-multipoint. This is due to the Sup2T having an improved hashing algorithm to support larger IP Multicast flows over the Sup720.

Access Control List Test

Access Control List or ACL are important tools in the configuration and customization of network attributes, especially with the Catalyst 6500. In the Catalyst 6500 upgrade with Sup2T, the TCAM has been both increased and its architecture improved. For ACL, one major concern was the lack of visibility of overflowing the TCAM when new ACL scripts were submitted, which would disrupt network operation. Updating ACLs occur infrequently and over a long period of time. As such multiple network engineers working on the same network may not even be aware of previous ACL updates. Further, an ACL update may drive multiple ACE (ACE = Access Control Entries), which occupy more TCAM resources than anticipated and thus over consume this resource. Therefore, Cisco developed the ACL Dry Run and ACL Atomic Commit to mitigate this scenario from occurring.

Results: We verify that this new efficient use of TCAM and
ACL safeguards perform as stated.

System Network Test Configuration: MPLS/VPLS/VSS

To test MPLS/VPLS and VSS throughput performance, we populate two Catalyst 6500 WS-C6513-Es with eight 10GbEports each via 6908-10G modules connected directly to Ixia test equipment. The Catalyst 6500s are connected via8 x 10G Distributed EtherChannels. This configuration created a full end-end 80Gbs path of full-mesh traffic; typical in the real world.

The test data result show that throughput performance is consistent independent upon protocol that being MPLS, VPLS and VSS. A contributing factor to the differences in throughput is found in different headers associated for each protocol. This result could not occur in the older generation of Catalyst 6500 with Sup720 with its 40Gbs per module backplane access speed.

Network Encryption with 802.1ae MACSec

We tested performance for 802.1ae MACSec to verify that there was no throughput performance degradation when encryption was enabled minus the additional 16 byte overhead of 802.1ae keys. MACSec encryption has become increasingly popular and important to campus network design, but previous switch performance degraded when forwarding encrypted traffic. Here we verify that the Catalyst 6500 does not suffer throughput performance degradation while MACSec traffic is being forwarded.

We tested the Catalyst 6500 via the cPacket Networks cTapSmart 10G passive probe to verify traffic flows were either MACsec encrypted or unencrypted. We found that there is no material difference in throughput performance, other than 802.1ae encryption key overhead, thanks to 16 additional bytes per packet.

Conclusion

We found that upgrading the Catalyst 6500 from Sup720 to Sup2T was straightforward and added significant value in the areas of MACsec encryption, improved ACL capabilities and IPv4/IPv6/MPLS/VPLS/VSS throughput performance. In addition, we found that the Sup2T supported existing service models, such as Network Analysis (NAM), Wireless (WiSM), Application Control Engine (ACE20), Firewall Service Module (FWSM) plus 6148A-GE, 6148E-GE with POE/POE+, 6724-SFP line cards plus 6704 and 6716 line cards after a trivial DFC3 to DFC4 daughter card swap. We found that line cards can be swapped and upgraded while the Sup2T is operational, avoiding off-hour scheduled downtime. In addition, we found that existing interface transceivers SFP and X2 being used in a Sup720 Catalyst 6500 can be reused with the Sup2T. Finally, we found that Sup720 IOS configurations may be copied and migrated to a Sup2T via a flash drive successfully upon boot up.

Much of the throughput performance advantages and scale of network services is due to custom ASICs resident in theSup2T, 6908-10G line cards and DFC4 daughter cards. We were particularly impressed with the ease of upgrade, the new ACL dry run and atomic commit plus MACsec performance.

For existing customers of Cisco’s Catalyst 6500 Sup720, we anticipate upgrade experiences similar, if not simpler, than ours as this test was conducted under tight time constraints with limited resources. It’s no wonder why the Catalyst 6500 is so popular as it offers a wide variety of network design options such as MPLS/VPLS/VSS. With the new upgrade to Sup2T and supporting line cards, we verify that throughput performance doubles over the Sup720 for IPv6, IP Multicast, MPLS/VLPS and VSS.

New entrants in the campus core market such as HP Networking A10500 later this year that boast pure performance without network services will find a chilly reception awaits them.

Tweet

In this Lippis Report Research Note 183, we provide our very popular annual top 10 2012 industry predictions that were provided by Andre Kindness, senior analyst at Forrester Research, Nick Lippis, CEO of Lippis Enterprises, and Zeus Kerravala, principal at ZK Research. We take a look into the year ahead and provide our view as to what will come to pass. This Research Note is based upon the “2012 Networking Industry Predictions” Lippis Report podcast.

The following are our top 10 2012 Networking Industry Predictions.

Prediction One

2012 Is the Year of Data Center Fabrics: The back half of 2012 starts the kick-off of aggressive data center fabric deployments. While Cisco has been shipping FabricPath and Arista has been shipping MLAG and ECMP, Juniper will join the market with its long anticipated Qfabric, Avaya will be shipping a broader VINA enabled product set along with Brocade’s expanded VDX switches with VCS. Alcatel-Lucent and Huawei too will be shipping their version of SPB. In short, there will be plenty of product and options available from which to choose.

Prediction Two

Voice over LTE Goes Live: Verizon will aggressively deploy Voice over LTE to match AT&T’s talk while you surf on mobile devices functionality. AT&T will then respond with a Voice over LTE initiative. This will drive a huge wave of growth for internet infrastructure companies as VoIP enters the mobile market. Expect to see a robust year for Acme Packet, BroadSoft, Infoblox, Tekelec and many others.

Prediction Three

Repatriation of Holiday: Obama grants a repatriation holiday allowing large IT firms such as Cisco to bring billions of dollars back to the US market, and puts it to work by making two large acquisitions; one of them will be security and the other storage related.

Prediction Four

Wither Polycom: Amidst tremendous pressure from Cisco’s video communication and telepresence business, Polycom continues its slide. Polycom ends 2012 as an acquired company.

Prediction Five
 
The Year of Software-Defined Network Marketing: The SDN/OpenFlow industry marketing machine kicks in with all major networking companies wrapping their existing products around the SDN message. In addition, the SDN controller market starts up with data center switches equipped with controller plug-ins. All new networking concerns seeking VC dollars have SDN/OpenFlow in their business plan. Case in point: look at “Embrane.” While there is will few SDN revenue dollars made in 2012, marketing messaging will be loud.

Prediction Six

Huawei Enterprise Business Division Comes On Line: Huawei will climb to the number four spot in worldwide network switching at the expense of low cost providers. HP Networking will be the hardest hit, losing at least two points of Asia market share.

Prediction Seven

Network IPO Market Comes Back: At least four large networking IPOs occur, including Arista Networks, Ruckus Wireless, Infoblox and Palo Alto Networks, fueling liquidity into the networking market once again.

Prediction Eight

IBM Becomes a Networking Thought Leader: IBM System Networking will coalesce its networking investments around virtualized network infrastructure and SDN, renewing its place as a thought leader in the networking industry.

Prediction Nine

Brocade Gets into WLAN Market: Brocade will buy into the WLAN market by either acquiring Meru Networks, Aerohive or Meraki to shore up its enterprise network switch by offering a unified access value proposition.

Prediction Ten

Application Acceleration Market Fundamentally Changes: Citrix, Riverbed, Cisco, Brocade and F5 will start to compete in the application acceleration or delivery market by offering integrated WAN acceleration, Application Delivery Controllers or ADCs and security network services in both appliance and virtual form factors. Those who are able to tag and steer applications to network services while adding policy will win a larger percentage of market share.

In addition to the top ten predictions above, software network engineers will be the new rage in 2012 as the market shifts toward a value proposition rooted in software and network services. In addition, Cisco will dominate market share and thought leadership.

Tweet

The Fall 2011 Open Industry Network Performance and Power Test Report is now available. Since our Spring 2011 test, we added four products from three vendors to the 11 products from eight vendors already tested. We now have data on 15 data center switching products from nine vendors in the new report to be released after Thanksgiving. Our cloud networking test of 10 and 40GbE is now the industry benchmark for cloud networking. In fact, only those companies that are sure of their product(s) enter the test at Ixia’s iSimCity. We found that 40GbE is hard, and thus you have to give credit to the vendors that go through the testing—in this test, those vendors are Extreme Networks, Brocade and Alcatel-Lucent. These firms have high performance data center switching product that is Enterprise and Cloud service provider ready. In this Lippis Report Research Note, we share our the top 10 findings from this round of testing. Lippis Report subscribers can download the 125-page report here, free of charge.

To assist IT business leaders with the design and procurement of their private or public data center cloud fabric, the Lippis Report and Ixia have conducted an open industry evaluation of 10GbE and 40GbE data center switches. These test were conducted at the Ixia iSimCity Santa Clara, CA, laboratories.

The Lippis Report test, based on independent validation, communicates credibility, competence, openness and trust to potential buyers of 10GbE and 40GbE data center switching equipment as the tests are open to all suppliers and are fair, thanks to RFC and custom-based tests that are repeatable. The private/public data center cloud 10GbE and 40GbE fabric test was free for vendors to participate and open to all industry suppliers of 10GbE and 40GbE switching equipment, both modular and fixed configurations.

While Lippis Report subscribers can download the full report here, below are our top ten findings from conducting these three rounds of testing. The Fall Lippis/Ixia test proved to show that the industry is advancing at a breakneck pace. And we do expect to see more products being submitted for test in the March 26th Spring 2012 test. Based upon three series of industry test, the following top ten findings have become evident.

1) 10GbE Top of Rack (ToR) and Core Switches: 10GbE ToR and core switches are ready for mass deployment. There have been 15 new switches since Interop 2011, and there will be 15 more launched during 2012.

2) Fastest Ethernet Switches under the Milky Way: We are in the 500 ns ToR and 2 us core switch era. For core switches, the Extreme x8 is two to nine times faster than any other core switch we have tested. ToR switch latency will decline to 100s of ns within two years, thanks to better merchant silicon plus Phy-less designs. Core switch latency will decline to ns area with 40 & 100GbE speeds plus the next generation of merchant silicon.

3) Merchant Silicon Proves Its Value: Most switches entering the Lippis Report test are based upon a new generation of merchant silicon. They are based upon a single chip design from Broadcom, Fulcrum or Marvell. Broadcom currently leads this space and is becoming the Intel of the networking industry.

4) Switch Vendors Differentiate Products Mostly on Software: There are differences between suppliers at both the box and system level. At the box level, we find latency, congestion, power and software differences. We also find differences in how these vendors propose building cloud networks. There is difference in cloud network architecture approach, such as support for TRILL, SBP, MLAG and/or ECMP. There are differences in network services and virtualization aware support.

5) Ability to Support Storage Engagement: Most core and ToR switches demonstrated throughput performance without loss and low latency variability to support storage enablement. Most switching firms will be offering a range of convergence options during 2012, including ToR switches with direct fiber channel connections and/or FCoE, ATA over Ethernet and iSCSI over Ethernet support.

6) 40GbE Is Ready: 40GbE support as a downlink from ToR to End of Row (EoR) and in the core at density is here, ready and performs as advertised. In addition 40GbE cost is approximately 3 to 4 times that of 10GbE, making 40GbE favorable from a pricing point of view too. There are plenty of ToR switches that support multiple 40GbE options such as Alcatel-Lucent OmniSwitch 6900-X40, IBM BNT RackSwitch G8264, Arista 7504 Series Data Center Switch, Dell/Force10 S-Series S4810, etc. In the core switch market, there is only one company with high-density 40GbE, and that’s Extreme BlackDiamond X8 with 192-40GbE. But we expect at least four more high-density 40GbE core switches to be launched in 2012. Note, at times, we did some observe difficulty with preamble and equalization at the physical QSFP+ level causing packet loss, but this we mitigated through software control.

7) Low Power Consumption: Power consumption in networking devices is dropping precipitously. All ToR and core switches offer low power consumption with energy cost over three-years estimated between 1.3% and 4% of acquisition cost. Two of the most impressive results we observed was that of Extreme Networks’ BlackDiamond X8 Core switch and Brocade’s VDXTM 6730-32 Data Center ToR Switch. The Extreme X8 consumed a low 5.2W/10GbE; that’s nearly as low as a Christmas bulb. Brocade’s VDXTM 6730-32 Data Center ToR switch consumed a low 1.5W/10GbE; that’s about 20% of the power a Christmas bulb consumes! In addition to power consumption all switches support front-rear or rear-front airflow in support of hot/cold aisle designs.

8) Virtualization Scale Support: All switches in this test are able to support far greater numbers of VMs and physical servers than their physical ports allow; that is, their logical networking scales to support very large virtualized data center infrastructure.

9) 10/40GbE Recommended as Cloud Network Fabric: From server connections to ToR to core switching plus storage enablement and virtualization aware software 10GbE is recommended as the fabric for cloud networking environments. We recommend that IT business leaders take full advantage of server I/O at 10Gbps bandwidth and low latency as it will provide the highest performance and greatest data center design options moving forward. With 10GbE ToR switch cost per port in the $350 to $670 range, core switch cost per 10GbE port in the $1.2K to $6K range plus 40GbE cost per port in the 3 to 4 times of 10GbE, Ethernet technology is well segmented for data center needs. 10GbE and 40GbE switches have the logical networking to support highly virtualized infrastructure with dense VM:physical server ratios of 30:1 to 60:1. With ToR and core switch latencies in the 500 ns to 2 microsecond range, the industry’s 10GbE switches possess the raw performance and capacity to support storage enablement, albeit this area is evolving.

10) Software-Defined Networking/OpenFlow: While not tested during these past three rounds of test, software-defined networking or SDN and OpenFlow will be increasingly important during 2012 as companies seek to differentiate their high performance switch products with increased features and functionality. SDN with OpenFlow promises to offer such added value.

The next Lippis Report test at iSimCity is scheduled for the week of March 26, 2012. We expect more 40GbE products in the Spring.

Tweet

During the weeks of October 10 and October 31, 2011, at Ixia’s iSimCity, the Lippis Report conducted its third industry test of cloud networking data center switches operating at 10 and 40GbE. In just six short months, the industry has moved forward by breaking all previous records of data center switch speed, power consumption, port density and bandwidth. We added four products from three vendors to the eleven products from eight vendors already tested. We now have data on fifteen data center switching products from nine vendors in the new report to be released after Thanksgiving. During May 2011 Interop, we had eleven vendors provide verbal commitment to participate in this Fall industry test (remember it is free for vendors to submit products to test). As the deadline for signed agreements came, this field of eleven dropped to three because their products were simply not ready. 40GbE is hard, and thus you have to give credit to the vendors that go through the testing—in this test, those vendors are Extreme Networks, Brocade and Alcatel-Lucent. These firms have high performance data center switching product that is Enterprise and Cloud service provider ready. In this Lippis Report Research Note, we share our insights gained from testing all these products and provide the topic cloud networking industry trends taking shape now.

To assist IT business leaders with the design and procurement of their private or public data center cloud fabric, the Lippis Report and Ixia have conducted an open industry evaluation of 10GbE and 40GbE data center switches. These test were conducted at the Ixia iSimCity Santa Clara, CA, laboratories. The resources available for this test at Ixia’s iSimCity are out of reach for nearly all corporate IT departments with test equipment on the order of $9.5M, devices under test on the order of $2M, plus costs associated with housing, power and cooling the lab plus 22 or so engineers from around the industry. It’s our hope that this industry effort will remove performance, power consumption and latency concern from the purchase decision, allowing IT architects and IT business leaders to focus on other vendor selection criteria, such as post sales support, platform investment, vision, company financials, etc.

The Lippis Report test, based on independent validation at Ixia’s iSimCity, communicates credibility, competence, openness and trust to potential buyers of 10GbE and 40GbE data center switching equipment as the tests are open to all suppliers and are fair, thanks to RFC and custom-based tests that are repeatable. The private/public data center cloud 10GbE and 40GbE fabric test was free for vendors to participate and open to all industry suppliers of 10GbE and 40GbE switching equipment, both modular and fixed configurations.

Ixia supplied all test equipment needed to conduct the tests while Leviton provided optical SPF+ connectors and optical cabling, and Siemon provided copper and fiber optic QSFP+ cables and transceivers for 40GbE connections. Each 10GbE supplier was allocated lab time to run the test with the assistance of an Ixia engineer. Each switch vendor configured its equipment while Ixia engineers ran the test and logged the resulting data.

While we can’t just yet release data on the latest round of testing, we can share with some of the records that were broken. We measured for the first time core switch latency in single digit microseconds and single digit Watts/10GbE power consumption. Also for the first time, we measured power consumption in top of rack switches power consumption in very low single digits. We measured how fast core switches can forward packets at very high density being 256 0GbE plus 24 40 GbE ports, and this was only a third of this switch’s port density. We measured congestion, IP Multicast, cloud simulation, latency and throughout for 24 40GbE, a first in this series of industry test.

In just six short months, data center Ethernet core switching has increased in speed by nearly a factor of 10, its power consumption dropped by nearly 50% while port density increased by nearly 3 times. In ToR switching, power consumption is down by over 50% while these products add 40GbE uplines and storage enablement such as direct Fiber Channel and/or Fiber Channel over Ethernet connections. With all of these advances, the one thing that is holding steady is pricing as the industry serves up more features for the same or slighly more dollars.

The Fall Lippis/Ixia test proved to show that the industry is advancing at a breakneck pace. And we do expect to see more products being submitted for test in the Spring 2012 test. Based upon three series of industry test, the following trends have become evident.
Faster Forwarding: While the Fall test showed new records in latency measurements—that is, how fast a switch can forward packets at zero packet loss or 100% wire speed throughput—switching products will get even faster. While it’s anticipated that the Fall core switch latency records will not be broken in 2012, ToR switches will show significant improvement getting into the range of 100ns with 100Mbs Etherent uplinks.

Hybrid Cut-Through and Store and Forward Switching: To make switches faster, merchant silicon vendors have taken a new look at packet forwarding. It used to be that Ethernet switches were either cut-through (CT)—where packets were not stored for processing—or store and forward (S&F)—where packets were stored, processed then forwarded. Now switches use both forwarding techniques, where the first few hundred packets are forwarded via S&F and the rest, CT.

IP Multicast Rises in Importance. With the huge increase in video traffic, IP Multicast performance and, in particular, how switch replicator chips perform will be increasingly scrutinized. We tested the lowest latency of IP Multicast during the Fall test, indicating that switch speed of forwarding IP Multicast is becoming an important product selection criteria.

40GbE Arrives in 2012: Due to 40GbE component shortages in Asia, most vendors could not participate in the Fall test. These shortages will abate over the next quarter, creating a wave of new 40GbE modules and products during 2012. With 40GbE being 3 to 4 times the cost of 10GbE, look for a quick ramp up in ToR uplink and core switching modules.

The Rise of Merchant Silicon: Merchant silicon from Broadcom, Fulcrum MicroSystems and Marvell manufacture low-cost chips for network switches that have lowered the risks for new entrants into the hot data center Ethernet fabric market. In the last few months alone, 10 companies announced new products based upon one of the above merchant silicon 10 and 40 Gbps Ethernet chips. We expect to see enhancements to network virtualization, support for software-defined networking and a focus on buffer architecture.

New Set of Best of Breed Products: With merchant silicon competing with custom ASICS, a new class of best of breed products has emerged with more to follow during 2012. These products will be pushing the envelope on packet forwarding speed, power consumption, port density, storage enablement and network virtualization, thanks to VXLAN/NVGRE support and software-defined networking.

Software-Defined Cloud Networking: As best of breed Ethernet data center switches get more powerful while consuming less power, these products will need to tap into a growing software base to add value to these networking products. Software-Defined Cloud Networking or SDCN promises to ignite a cycle of innovation that shifts competitiveness to network software that enables firms like Cisco, HP, Extreme Networks, IBM, Arista Networks, Force10/Dell, Avaya, Huawei, Brocade, Juniper Networks, Alcatel-Lucent, Enterasys and others to compete by rapidly adding software features to low-cost merchant silicon-based network products. There are two approaches to SDCN: 1) OpenFlow based that defines an open interface between switches and a controller or 2) hypervisor virtual network controllers that plug directly into switches.

The next Lippis Report test at iSimCity is scheduled for the Spring of 2012. We expect more 40GbE products plus the observation and measurement of the above trends.

Tweet

By all counts, Cisco’s upgrade of the Catalyst 6K via its new Supervisor 2T, or Sup2T, is its most ambitious and thoughtful yet for the venerable platform. The Sup2T is a 2 Terabit (Tb) platform that triples the previous Sup720 performance. Thanks to the support of Virtual Switching System (VSS), the platform allows two 2 Tbps switches to combine into a single 4 Tbps virtual switch. The Sup2T is a major upgrade to the most widely-deployed switching platform in campus and data center networking in the industry. But while these performance numbers are impressive, it’s the new Cat6K’s network services and pricing that deliver most of the value. From a services’ point of view, the Cat6K stands alone.

Cisco’s Cat6K is the firm’s most successful product with over 700,000 systems and 110 million ports installed, worth some $42 billion. This product’s success increases the stakes for Cisco as it introduces a major upgrade. Cisco had to consider backward and forward customer migration, increased competition and pricing pressure especially as many firms are starting to offer core switches based upon merchant silicon. In short, Cisco had to eliminate the trade-off of innovation versus investment protection and find a way to deliver both simultaneously. A detailed review of the new Cat6K with Sup2T finds that Cisco has navigated well by incorporating customer feedback from multiple theaters and industry segments in the form of some 200 features, most of which are incorporated into ASICs, something with which merchant silicon based switching firms cannot compete.

Merchant Silicon versus Custom ASIC

There will be an increase in the number of core switches offered from various vendors during 2012 thanks to the availability of merchant silicon, but these products, for the most part, will be focused on primarily performance while falling short on network services. Network services are hardware and software features that provide the tools, customization and design options for IT architects to optimize their networks and applications to either run faster and maintain secure, reliable, high-quality user experiences whether it’s for video traffic, virtualized desktops, general purpose office productivity or client facing web traffic.

For example, consider something as mundane as counters. In the Cat6K Sup2T and new modules, there are more than two million counters, enough to have separate counters for every protocol, including IPv4, IPv6, multicast, unicast, MPLS, etc. What this says is that Network Operations engineers will be afforded a level of granularity and visibility into the network well beyond anything they previously could gather. But I digress; let’s focus on the big picture of the new Cat6K.

The New Cat6K by the Numbers

The last major upgrade for the Cat 6K was the Sup720-10G in 2007, which was the first management module with 10GbE uplinks. The Sup2T enables 40GbE interoperability and interface speed transition as the Cat6K will support 100MbE, 1GbE, 10GbE and now 40GbE in a modular chassis platform. The performance leap on the 2 Tb portfolio is complemented by a quadrupling, or more, of the NetFlow, Access Control List and Quality of Service capacities of the platform to meet the increasing manageability, security and service demands of enterprise networks. The platform now offers 720 Mpps of IPv4 and 360 Mpps of IPv6 performance, roughly a twofold increase over the previous generation. In a word, the Cat6K scales logically.

What Cisco engineering has done is tripled the performance, quadrupled the platform scalability and added new network services—several of which are industry firsts and all of which protect investment by being backward compatible with these forward innovations. For example, central forwarding line cards that started shipping in 2003 are supported in the Sup2T. The E-series chassis and power supplies that started shipping in 2004 are supported with the Sup2T. For a large segment of the Cat6K installed base, all that is required is the install of the new Sup2T to gain increased performance, scale and network services. This is perhaps one of the easiest refresh offers Cisco has ever made.

Network Services Rich

As for network services, the Cat6K supports some 2,600 features that the market has demanded. Most of these features were developed over time with many firms depending upon them to run their networks. In addition to hardware backward compatibility, Cisco had to be software backward compatible too by supporting these 2,600 features, which are supported in the Sup720 and the wiring closet Sup32, in the Sup2T. Some of these features include IPv6, multicast, NetFlow, MPLS, etc. But clearly the market does not stand still, and Cisco engineering has added some 200 new innovations to the Sup2T, some of which will also be supported on previous versions of supervisor engines.

Interestingly enough is that with backward support of new network services supported on the Sup720, IT architects can choose to move these Cat6Ks down a network layer and place the Sup2T Cat6Ks in the distribution and core, extending the entire portfolio of network services from access, distribution and core. Some of these new innovations are Flexible NetFlow, Role-based Access Control, Virtual Private LAN Service (VPLS), Bridged Domain Technology, etc. Following are a few of the next generation innovations introduced with the Sup2T.

NetFlow: NetFlow scalability in the Cat6K Sup2T has increased fourfold with larger tables being supported in the ASICs. Up to 13 million NetFlow entries are possible in a single system. That is up to eight times the visibility afforded by the previous generation of NetFlow hardware. Over time, most networks will have a mix of 1GbE, 10GbE and 40GbE; this new version of NetFlow introduced sample NetFlow so NetOps does not have to export all traffic to collector, a huge complexity and time reduction. Also NetFlow visibility is now protocol independent, meaning that it does not matter if a network is running IPv4, IPv6, MPLS, Unicast, Multicast, etc. In addition, select modules, rather than the central supervisor, are able to export NetFlow to the NetFlow collector offering yet another way to scale.

MACsec: From a security perspective, the Cat6K Sup2T natively supports MACsec, or IEEE 802.1AE, embedding it within line cards offering line-rate, hop-by-hop encryption and decryption. In addition to the new Cat6K, the Nexus 7K, Cat 3K and Cat 4K currently support MACsec, thereby enabling end-to-end secure communications much like IPSec and SSL but over the LAN.

Role-Based Access Control List (RBACL): Access Control Lists, or ACLs, can now be programmed in role-based scenarios controlling user access to IT resources. Roles can be finance, human resources, marketing, engineering, sales, executive management, etc. Role-based access control allows NetOps to configure which IT resources each user is allowed to access for each type of job role, thereby controlling their access to servers, applications, WAN connections, etc. Role-based access control is an addition to the Sup2T’s ACL Dry Run, which first tests if ACL changes will fit in the ACL Ternary Content-Addressable Memory or TCAM before they go live with the configuration. Using ACL Dry Run will help avoid potential network disruption since NetOps engineers will know whether the ACL changes will be supported in hardware before implementing them. If an ACL change does not pass the Dry Run, then the system will indicate which resources are being exhausted, allowing the NetOps staff to adjust the ACL accordingly.

Network Virtualization: The new Cat6K Sup2T boosts its network virtualization capabilities that enables physical infrastructure to be logically divided. For example, airports, such as Zurich, Munich, Toronto, etc., use network virtualization to change gate attributes as an airline carrier completes the boarding process and transitions the gate to another carrier. They also use network virtualization to separate out kiosk vendors from operations from WLAN AP guest access to airline carrier support, etc. Governments network virtualization to logically segment departments while they share the same physical building/floors/office spaces. Universities use network virtualization to logically segment administration, research, faculty and student interests. Just as with other previously-mentioned capabilities, Sup2T increases the scalability for network virtualization up to fourfold with support for up to 4K MPLS VPNs, 32 instances of (VPN Routing and Forwarding) VRF-lite, native VPLS in hardware, allowing for VPLS-facing interfaces to be any interface in the system, and more.

New Service Modules

Admittedly, the Cat6K with the Sup2T is not the fastest Ethernet switch on the market with 2 Tbps of switching capacity. Cat6K doesn’t need to be the fastest given its place in campus networking and mid-range data centers. However, it does need more than enough performance to never be the bottleneck in IT delivery while providing a wide range of software options to control traffic and optimally design enterprise IP networks. Cisco engineering has done this with 2 Tbps, and 4Tbps with VSS, far greater capacity of most, if not all, campus and mid-range data center networks operating at a range of 10/100/100, 10GbE and soon 40GbE. For higher performance, Cisco offers the Nexus 7K with 9 Tbps of switching capacity for data center switching designs.

To increase performance in the Cat6K, it’s not just the supervisor engine that’s been upgraded. New service modules, such as the new Wireless Service Module 2 (WiSM-2), Adaptive Security Appliance Service Module (ASA-SM) firewall, Network Analysis Module 3 (NAM-3) and Application Control Engine 30 (ACE30) load balancing were introduced to take the Cat6K with Sup2T to the next level of hardware-based services processing. Remember, service modules allow IT business leaders to reduce the number of devices in their network they need to manage, improving energy efficiency and reducing carbon footprint. These new service modules have been upgraded for performance and scalability, as services performance has to scale with network performance. For example, the ASA-SM offers a threefold increase in performance with 15-20 Gbps of stateful application firewalling. NAM-3 has been upgraded in performance by a factor of fifteen, allowing application visibility and analysis at 15 Gbps. The WiSM-2 scales up to 20 Gbps of throughput and support for up to1,000 centrally-managed access points, a threefold increase in performance and scalability.

Integrated and Virtualized Network Services

Unique to a Cisco environment is that service modules and appliances basically share the same operating system, meaning that there is operational consistency between the two platforms. For example, if an IT architect implements an ASA appliance and ASA-SM, NetOps will experience the same operating system, management and look and feel between the appliance and service module. This consistency allows NetOps to best utilize and manage network services independent of physical packaging and network location, thereby increasing operational efficiency and innovation injection. Thanks to network services being integrated into the Cat6K, and the ability to virtualize services, IT architects are afforded design choices where they can regulate the number of appliances versus service modules in their network by choosing to utilize service modules more over time and obtain their green benefits too. Note that the ASA-SM and ACE-30 can be virtualized or divided between users/groups, thereby extending their reach throughout a corporate network and reducing the number of appliances in the process.

Cat6K with Sup2T Pays to Upgrade to 10GbE

From a pricing point of view, it’s best to think of the Cat6K with Sup2T as the device to transition a campus and mid-range data center network from 1GbE to 10GbE. With 1GbE in the access layer, via upgraded Cat4K with Sup7-E and/or Cat3K / 3750X, connected to a Cat6K with Sup2T in the distribution layer providing 10GbE to the core, Cisco estimates that this configuration will be 20% less costly than a similar configuration utilizing the Sup720 and older versions of the Cat4K and 3K. This design provides for 10GbE between access, distribution and core. In essence, Cisco is paying IT leaders 20% to upgrade to 10GbE with a new generation of switching.

Economics plays a large role in network design. From an economics perspective, Cisco is responding to competitive pressure with new pricing and design options with this Cat6K upgrade. While the Cisco Cat6K Sup2T represents increased performance, what IT business leaders will find is that for typical configurations independent of data center or campus, 1GbE, or 10GbE, the overall cost of a Cat6K network is actually reduced by 20 to 25%. For example, the 48 port 10/100/1000 copper line cards were sold in two versions: centralized and distributed forwarding modes. The centralized forwarding mode is priced at $15K and comes with 256MB of memory, while distributed forwarding is $22.5K. New Ethernet line cards (6800 Series) have Distributed Forwarding Card 4 (DFC4) daughtercards by default and come with 1GB of memory that are priced at the same $15K as the centralized forwarding mode cards, closing the price gap between centralized and distributed forwarding mode to the lower cost centralized pricing. IT architects are offered distributed forwarding performing line cards, which are higher performance throughout the system, at a third of previous generation cards. This is but one important example that demonstrates that the Sup2T is a price reduction over Sup720 around 10GbE.

New Network Design Options and Economics

Campus networking traffic patterns are dominated by north-to-south flows, thanks to the centralization of IT application delivery within data centers. While over time, an increase in east-to-west flows may occur thanks to peer-to-peer applications, north-to-south flows are getting thicker and denser especially as the industry adopts virtualized desktop computing and real time video communications. These thicker north-to-south flows are being accentuated as more applications are being hosted in corporate data centers and private cloud facilities for IT complexity and cost reduction. At the same time, enterprise mobile computing has skyrocketed with the adoption of iPhones, Android-based devices and iPads. For example, Gartner predicts that 55 million tablets will be sold worldwide by the end of 2011. Thanks to lower power output antennas on these new mobile devices, the density of WLAN APs are also increasing to provide coverage. This is creating a challenge to roam seamlessly without user experience interruption.

Mobile and cloud computing economics and increasing traffic volume are driving a new model for campus networking. It’s a model that seeks to increase wired and wireless network bandwidth, scale logical networking and extend network services such as security throughout the enterprise network via centralized management control methods. It’s a model that also seeks greater visibility and control of flows to optimize performance and apply resources where needed. Network virtualization, where physical network infrastructure is logically segmented to assign different network attributes to various groups/departments/entities, has become a mandatory requirement in some industry segments. And from a design point of view, high reliability needs to be systemic as all corporate productivity is flowing across this IT asset.

For those with Cat6K-based networks, installing the Sup2T offers a range of new network design options and economics. For example, encryption is now embedded and integrated. Network services are increasingly becoming virtualized, offering greater reach, cost effectiveness and lower carbon footprint. 10GbE and 40GbE speeds can be strategically placed where bandwidth is needed. NetOps is offered a common look and feel between appliances and service modules, reducing operational cost and increasing efficiency. Logical networking can scale to support more IPv6, more WLAN APs and users, greater visibility into the network via NetFlow, greater stateful application firewalling, etc. It’s clear that Cisco engineering has made tremendous efforts on security with TrustSec, taking ACLs to the next level, NetFlow’s deeper visibility, network virtualization via MPLS or VPLS for segmentation and bringing parity to IPv6 and IPv4.

Cisco is paying customers to upgrade to both the Cat6K Sup2T and 10GbE. Obviously, there’s additional capital cost to spend to gain the return, but from a historic perspective, the upgrade cost is a fraction of previous switch generations. With the Cat6K Sup2T upgrade, IT business leaders gain a wide range of network services, some of which are mentioned above, that will prove to be invaluable as IT marches on toward an IT delivery model dominated by mobile and cloud computing with nearly everything becoming virtualized.

Tweet

Back in November of 2009, I wrote Lippis Report Research Note 136 titled “HP Plans to Acquire 3Com Accelerating a New IT Convergence Era.” In that Research Note, I wrote

“When 3Com is fully integrated into HP what kind of networking revenue and market share can HP gain? ProCurve + 3Com is approximately $2B of revenue now. With the existing product lines can HP generate $5B, $10B or more of network revenue over five years? Time will tell.”

Well after nearly two years, HP Networking or HPN’s North America (NA) layer 2/3 Ethernet switch market share by revenue is nearly the same, bouncing between 5% and 6.1%, according Dell’Oro, with HPN’s Q2CY11 NA switch revenue share being down to 6%. Considering HPN’s limited results after significant investments in sales, channels and marketing, including its “proof-of-concept” plus “A Catalyst for Change” Cisco Trade-in program, not to mention engineering investment, the question is can HP make it in networking? We attempt to answer that question in this Lippis Report Research Note.

Market Share Analysis: 2% Growth Comes from Asia and RoW

HP had approximately 6% WW (Worldwide) layer 2/3 Ethernet switch market revenue share with its ProCurve product line before the 3Com acquisition, according to Dell’Oro. Post 3Com acquisition, HPN’s WW Ethernet switch revenue market share rose to approximately 10%, thanks to 3Com’s 4% share contribution, and stayed that way for three quarters until Q1CY11 where an additional 2% was gained thanks to increases in APR (Asia and Pacific Rim) and RoW (Rest of the World) theaters, according to Dell’Oro. In short, HPN’s NA switch market share has been flat since it acquired 3Com. From a WW switching perspective, HPN’s share of ports has also been flat with 20% share in Q1CY10 to 20.2% share in Q1CY11, according to Dell’Oro. In this same period, NA share of ports has been on a steady decline but with HPN maintaining share thanks to gains in APR and RoW.

In short, in nearly two years, HP gained 2% of WW layer 2/3 Ethernet switch revenue market share, all of which came in Q1CY11 and held during Q2CY11, according to Dell’Oro, and is directly attributed to APR and RoW markets. Its bright spots are in routing and WLANs, which increased 2.5% and 2.2% in revenue share, respectively, between Q1CY10 to Q1CY11. Its IPS/IDS revenue share has been steadily declining, losing .3% share over the same period.

Yes, it’s very difficult to gain share in an established market as HPN has discovered. HPN’s value proposition has been grounded as a lower cost alternative to Cisco, a firm that’s greater than 20 times HPN but sells architected solutions.

Huawei Could Shut Down APR and RoW

HPN’s growth is coming from APR and RoW theaters, which is understandable considering that HP obtained H3C, the once Huawei/3Com joint venture (JV) when HP acquired 3Com. Remember that Huawei and 3Com entered into a JV back in the early 2000s called H3C with the hope that H3C could produce lower cost networking products that 3Com would sell in NA while opening up the Chinese market. In Lippis Report Research Note 16, Bruce Claflin, 3Com’s then President and CEO, had hoped that H3C would deliver success much like Amdahl did over IBM in the 1980s and 1990s when Amdahl gained huge market share from IBM in the Front End Processor (FEP) business by offering similar products priced well below IBM.

Fast forward to late 2006 when Huawei agreed to sell its stake in H3C to 3Com. Huawie had a non-compete agreement with 3Com post the sale of its stake in H3C, which has since expired, allowing Huawie to more aggressively and organically pursue the Ethernet switch market. And it has, as in early 2011, Huawie announced a new Enterprise Business Division.

Surprisingly H3C’s massive product portfolio has not made it into the HPN NA channel, partly explaining HPN’s flat NA share growth. H3C’s products were to be HPN’s competitive advantage. More alarming for HP, however, is the prospect that Huawie’s Enterprise Business Division will bring its enterprise product portfolio right to H3Cs Asian customers, cutting off HPN from this bright spot. Also when H3C was partly owned by Huawei, the Chinese government was tremendously supportive of H3C, but since H3C is 100% owned by HP, the Chinese government has no incentive to support H3C and will more than likely shift its support to Huawie when its Enterprise portfolio is ready. The danger here is that in the quarters to come, HPN’s APR and RoW market could start to dry up. Much of the future growth for H3C had been pinned on continuing its China dominance. But wait it gets worse.

Huawie is threatening to hijack Bruce Claflin’s and now HPN’s low cost networking value proposition and use it for its own advantage. First Huawie will more than likely go after the H3C installed base in Asia then onward to NA and Europe. One possible scenario has HPN competing with Huawie as to who is the lowest cost provider of networking. This would push HPN up market and force it to change its value proposition to an architected solution, where it will find Cisco. HPN has started to move in this direction with its recently announced FlexNetwork Architecture. This scenario would, in essence, squeeze HPN between Huawie on the low end and Cisco on the high end. If networking gets into a price war game, Huawei could out low price HPN and that should be the major concern to HPN as it represents an estimated $800 million a year in revenue.

But Huawie will face stiff headwinds in NA as Huawei has a credibility problem with most North American buyers. IT business leaders know it as a low cost provider and that Cisco did a good job of raising the visibility of how Huawei tried to steal intellectual property source code. Therefore, while Huawei could have some impact in NA, the most immediate opportunity for Huawei enterprise is in China, specifically the install base that H3C had built.

Lacking Data Center Network Strategy and Products

HP certainly has product to support one of the most comprehensive data center visions in the industry. HP has servers, storage, a huge services group and network products. HPN’s FlexNetwork architecture is an interesting vision if an IT architect wishes to extend a fabric across an entire campus, branch and data center but the underlying architectural detail and products are missing. The A12500 series has been available for two years, but not in NA in any great numbers. HPN recently said that it will be available in the 2H2011. The new A10500 data center switch was announced in May but is scheduled to ship some time in the second half of 2012. HP’s networking strategy in highly virtualized data centers is limited to its Virtual Connect product. HPN’s data center networking share according to Infonetics, and UBS is estimated at 6% versus Cisco’s 81%. This is where the networking market is at its hottest versus HPN’s strong hold in education and low cost networking.

For a company with the portfolio size of HP and its strength in data centers, it’s curious that HP is the only mainstream network vendor that doesn’t have a good data center fabric story. Cisco clearly does, as does Brocade, Juniper, Extreme, Dell/Force10, Arista Networks, Alcatel Lucent, IBM, Mellanox, etc. HP doesn’t, and it’s surprising, considering its large position in the data center market. It would be refreshing to hear HP communicate what a unique HP data center architecture looks like tied into mainstream industry pain points.

How Can HPN Win?

How can HPN turn this around and participate in an effective way, utilizing its deep assets of broad product line, services, software, support, brand, financial strength and low price points to bring value to both customer and shareholders? Certainly HPN has product but it needs to bring the H3C products to NA and wrap the services group around them. HPN needs high performance and low latency 10GbE and 40GbE data center switching products since 10GbE represents some 25% of the total Ethernet switch market and growing, according to Infonetics. HPN recently announced a family of Top of Rack (ToR) switches called the 5830-switch family targeted for 2H2011 availability, but few details are available. HPN should consider acquiring Arista Networks, which may cost it two quarters of switching revenue but would add between 5 and 10% to its switch revenue and plug a major hole in its networking product line.

In addition, HPN needs leadership consistency as HPN has transitioned leadership from Marius Haas, previous HPN GM who left HP for KKR in May, to now Bethany Meyer, a marketing executive who is interim SVP and GM of HPN. Bottom line: HPN needs to create leadership stability. The first order of business for whomever is to lead HPN should be to communicate what the unique HPN vision is as it’s still not clear to the market. In short, what is it about the HP data center and HPN that’s going to create a competitive advantage over Cisco, IBM, Dell and Oracle other than low cost. For example, consider Cisco’s data center vision, which is very clear. Cisco’s data center business advantage architecture is a system’s approach that bundles products together to deliver business outcomes.

The above is a straight-line approach to winning an established game, but HP needs to do something big and radical that is out of the box but meets market needs. It could consider acquiring Xsigo, a firm that recently released its server-based fabric as an alternative to processing at the network layer. This could be an approach that disrupts what networking actually is in the data center. HP would best be served to develop a compute centric view of the world. Clearly some IT business leaders will buy into this model while others may not, but one thing is certain and that is data center computing buyers tend to be closer to the CIO, offering HP a potential competitive advantage.

HPN needs to develop a new vision for computing and networking, and deliver it via a bold strategy and vision that’s disruptive rather than “we sell cheaper than everybody else.” HP has the brainpower and financials to develop a disruptive approach to data center networking; they just need the thought and executive leadership. In short, HPN needs to lead this industry and not just be a fast follower.

Tweet

It seems like every week or so there is news of a massive cyber attack where criminals get away with stealing credit card and other personal data on the order of tens of millions of individual records. Sony, Bank of America, Epsilon, Nintendo, the International Monetary Fund, the US Senate and CIA are but a few of the targets for high-profile cyber attacks that took place in 2011. According to a recent study by the Ponemon Institute, “cyber attacks have recently become more harsh and recurrent. At least 90% of the IT practitioners surveyed claimed that they had experienced one or more cyber breaches within the last year, and 89% of these respondents could not identify the source of these breaches.”

To mitigate and avoid these breaches and protect credit card information, the Payment Card Industry (PCI) Security Standards Council issued PCI Data Security Standard (DSS) 2.0 in late 2010. The emphasis of PCI DSS 2.0 is two-fold: 1) provide increased protections not addressed in the previous standard (i.e., wireless and virtualized infrastructure) and 2) maintain compliance. As all of the breached organizations above were in compliance at some time but failed to maintain it, this exposed their customers to hackers and ultimately being breached. In short PCI DSS 2.0 is about being vigilant about maintaining security.

In the data center, virtualized servers are now defined within PCI and guidance is given on how to secure them given that all hypervisors are deemed insecure. In addition, wireless detection methods were expanded to address the variety of retailer capabilities.

IT business leaders who support any organization that stores, processes or transmits credit card data are required to ensure PCI 2.0 compliance not only during an assessment but continually to avoid the fate of the above-mentioned organizations. The key to a successful PCI assessment is to simplify this major effort. Some tech firms are assisting this effort through validation and assessment of compliance prior to installation. In this Research Note, we review Cisco’s PCI Solution 2.0 as it offers a unique network-based approach that is comprehensive, holistic and end-to-end. It has been tested in a simulated retail environment and assessed for compliance by a Qualified Security Assessor, QSA, and Verizon Business.

Cisco’s PCI Solution 2.0

The Cisco PCI Solution 2.0 is built on network security best practices, proven Cisco products and partner technologies that meet Payment Card Industry security standards. Because PCI covers many parts of the network, no single product or technology meets all PCI technology requirements. Therefore Cisco’s updated PCI Solution 2.0 is an architectural approach that maps to the updated PCI DSS 2.0 requirements. This comprehensive perspective allows retailers to see the bigger picture to prepare and design across the relevant parts of the enterprise. Cisco’s PCI Solution 2.0 is a holistic approach as it spans an end-end architecture.

Cisco’s approach provides templates and services that simplify PCI compliance. This simplification enables customers to maintain compliance year round, not just during assessments. Detailed information, including product configurations from validation efforts, is included in the Cisco PCI Solution 2.0 Design and Implementation Guide (DIG) to provide additional guidance and best practices.

Simplifying PCI Compliance

As a first step toward simplifying compliance, Cisco recommends segmenting the IT infrastructure and isolating cardholder data from the rest of the network. As with any complex problem, breaking a problem down into smaller solvable pieces reduces the complexity and simplifies the solution. Cisco’s approach reduces the scope of audit via network segmentation. Without network segmentation, the entire IT infrastructure is in PCI scope, which drives cost and complexity significantly upward. While segmentation sounds easy, it’s a bit more challenging in a virtualized data center infrastructure.

PCI Compliance in the Virtualized Data Center

Most IT business leaders are challenged with complex PCI audits within virtualized infrastructure as well as rogue wireless access detection. These two areas, virtualized infrastructure and rogue wireless access detection, tend to be the two largest pain points. Confusion around virtualization and security has existed for several years until the PCI standards body clarified that all hypervisors are considered insecure. With so many organizations having virtualized their data centers, this detail results in extra compliance considerations to protect cardholder data. Before virtualization, traditional infrastructure could be easily protected with a firewall appliance, as this device was placed directly in the path of traffic. In highly-virtualized environments, traffic is not as well-behaved, offering IT managers a challenge to restrict cardholder data.

Cisco’s Virtual Security Gateway (VSG), along with its Nexus 1000k virtual switch, intercepts and steers traffic to either VSG or firewall appliances before it gains access to cardholder data, providing a means for segmentation and access restriction in virtualized data centers.

Therefore to be PCI DSS 2.0 compliant, both physical and virtualized infrastructure need to secure and restrict access to cardholder data. Cisco does this with both its own VSG solution as well as with technology partners such as EMC, VMware, VCE and HyTrust.

Rogue Wireless Access Detection

Rogue access point detection is a PCI requirement. Even if a merchant does not use wireless technology within its stores, it still must have a method for detecting unauthorized access points that may have been inadvertently or maliciously deployed. The PCI Council expanded the flexibility of the requirement to allow for several methods, including Wireless IDS and NAC/802.1x to detect rogue wireless access points.

Unified Wireless and Cisco’s Identity Services Engine (ISE) technology offer technical solutions for these methods that have been validated by Verizon Business to successfully address these requirements. In addition, Cisco offers CleanAir technology, which monitors the entire frequency spectrum, surpassing the security requirements of PCI.

Risk Management

While a portion of PCI compliance is addressed through technology, it’s also addressed with process and compliance audits. One of the largest challenges is to maintain compliance between audits. Many retailers seek the lowest cost solution to achieve PCI compliance during the audit, but this may very well be penny wise and pound foolish. For example, some retailers conduct a visual inspection of Ethernet switches quarterly to ensure that unauthorized wireless access points are not connected into the corporate network, thereby opening a door to rogue access. The difficulty of this approach is that quarterly physical scans only work during inspection day. The day after the quarterly scan someone can plug in a wireless access point, putting the site and cardholder data at risk until the next quarterly inspection. A more continuous and secure approach is the implementation of wireless IDS, IPS, CleanAir and ISE, where every single wave is monitored and wireless devices plugged into the corporate network are detected assuring continual PCI compliance.

How to Approach PCI Compliance?

PCI can be an overwhelming topic. How do IT and small business leaders approach PCI compliance? To simplify PCI, Cisco offers three recommendations.

Recommendation One: Reduce PCI Scope. Scope means all systems and people that are touching cardholder data (i.e., firewalls and IT administrators). Are there people accessing cardholder data who shouldn’t be? If they are, then remove their access by restricting access to the systems that contain cardholder data. Are there systems or applications or networks that are touching cardholder data that don’t need to? Segment and narrow the scope of the Cardholder Data Environment (CDE) with network addressing and filters to decrement the risk as much as possible. If the CDE is smaller, the cost of the audit will be smaller as will be the complexity of maintenance. Standardizing network and system architectures across branches can also decrease cost and complexity as it allows auditors to sample same store/branch footprints and data center designs.

Recommendation Two: Secure the Perimeter. With a new smaller PCI scope implemented, the perimeter of that scope needs to be secure. Firewalls configured to only allow business-justified access to the cardholder data environment and IDS need to be installed. In addition, administrative access to this environment needs to be locked down to the bare minimum with complete logging for audit trails.

Recommendation Three: Maintain and Simplify. It’s not good enough just to segment and reduce the scope of cardholder data and then protect the perimeter. IT business leaders need to maintain and simplify their PCI recommended implementation. Cisco’s solution utilizes RSA technology to provide real-time alerts, tuned logs and compliance management dashboards that assist in maintaining compliance. The firms mentioned in the opening paragraph were all in compliance at some point in time, but they were not when they were breached. So take these requirements seriously.

Implementing a PCI Solution 2.0

The above three recommendations will go a long way toward reducing cost and keeping an organization’s systems PCI compliant. Cisco has made a huge commitment in its thoughtful approach to PCI DSS 2.0 compliance where it offers an end-end architecture that has been assessed and documented. A critical element of the Cisco PCI Solution for Retail 2.0 is Cisco network architecture and validated network designs. Cisco network architectures have been designed for stores, enterprise data centers and the Internet edge to support e-commerce operations, store employees, customers and teleworkers. Cisco’s PCI solution also supports wireless 3G technology deployments and multiple store formats, including pop-up stores, and convenience stores, in addition to typical small, medium and large stores.

Cisco’s PCI Solution 2.0 offers thought leadership for those seeking to simplify their PCI deployments; Cisco’s new PCI DIG is an in-depth, roadmap for organizations looking to achieve PCI compliance. It addresses technologies such as virtualization, wireless and mobile payments. As the number of high profile and alarming plus brazen cyber attacks occur, IT business leaders would be well-served to review Cisco’s PCI Solution 2.0 and Design and Implementation Guide.

Tweet

The tech sector is at a crossroads. In just 18 short months, mobile and cloud computing has fundamentally changed business assumptions and technical underpinnings of IT delivery. And in the process IT business leaders are fundamentally changing their buying requirements and corporate IT investments challenging existing vendor relationships. The tech sector served up corporate IT along technical lines of computing, networking, storage and applications, but these lines are blurring as every major multi-billion dollar IT firm now seeks to deliver vertical offerings comprised of a single rack of compute, storage and networking to address scale and simplicity associated with the new mobile and cloud computing models. Cisco, IBM, HP, Dell and Oracle all are repositioning their data center offers to address the market opportunity and shift to assist IT leaders building iBusinesses. In this Lippis Report Research Note, we dive into Cisco’s Data Center Fabric as it’s the furthest along at integrating compute, networking and storage access for corporate advantage offering a glimpse of IT’s future.

What’s driving a new fabric or structure of data centers is rooted in the interplay between technology and business opportunity. The efficiency of server virtualization to reduce energy consumption and increase server utilization drove its massive deployment that was boosted by an economic cycle starving for efficiency. At nearly the same time, mobile computing, thanks in large part to Apple’s iPhone and iPad plus Google’s android- based devices, introduced a new tier of computing that unleashed increased corporate productivity, evident in today’s productivity boom. Equipped with a new IT delivery model that is both more flexible and centralized, IT business leaders have begun en masse to build private cloud facilities.

The iBusiness

The end result is the construction of iBusinesses that possess simultaneously lower IT cost and the ability to quickly address market dynamics, thanks to faster application deployments plus a nimbler and mobile workforce. While it’s too early to aggregate the benefits of iBusiness in terms of productivity improvements, market share gains, IT expense as a percentage of corporate revenue and other metrics, early adopters are experiencing improvements that span IT departments and most importantly, corporate operations.

In short, a Data Center Fabric of compute, networking and storage reduce IT operational cost, the largest budget component of IT Total Cost of Ownership (TCO) and provide the foundation for a faster responding business that is able to exploit the value of mobile and cloud computing to corporate advantage.

Data Center Fabric Requirements

A core set of data center fabric requirements is emerging, thanks to early adopter deployments that possess the following attributes fundamental to iBusinesses.

Scale: Computational density is increasing at a fast pace with the ability to support hundreds to hundreds of thousands of servers per data center. This increased density of computing is also driving higher virtualization ratios as the ratio of virtual to physical servers is increasing from 10:1 to soon 60:1, which taxes the logical network of MAC address, /32 IP host route table size and ARP entry size. The ability to support both east-west and north-south traffic flows over an increasingly 10GbE and 40GbE low latency, non-blocking, high performance network fabric has become paramount as small queries from mobile devices drive a tsunami of east-west plus north-south data center traffic flows, all of which must be combined and transmitted back to the mobile device at millisecond speeds.

Mobility: As virtual machines (VMs) are moved within and between racks of computing and between data centers plus between private and public cloud facilities, the ability of the Data Center Fabric to support such moves is fundamental. VM aware Data Center Fabrics support VM mobility, allowing IT business leaders to maximize efficiency while enabling a degree of freedom to move containers of IT workloads (data, applications, VMs) as business requirements demand.

Consolidated IO: A significant cost reduction strategy and performance enhancement is the deployment of a single physical 10GbE and soon 40GbE network that supports both storage and network traffic. Cost savings is found in reduced cabling requirements, storage and network switches as well as server network and storage interface cards.

Consolidated Management: As compute, storage and networking converge into a single virtualized Data Center Fabric, the ability to manage these resources across operational groups become increasingly important. Not only is the technology converging, but IT organizational design is under review to focus this human resource into a services organization rather than siloed technology departments. The ability to manage the Data Center Fabric as a centralized resource that is partitioned to unique IT departments is an aid to organization re-design. It’s very helpful that a common look and feel for all resources be available so as to hasten a learning curve and accelerate cross-discipline service delivery.

Cloud Enabled: The combination of the above attributes results in a Data Center Fabric that is cloud-enabled, meaning that containers of workload are movable not only within a data center but also between them and into private and public cloud facilities. The ability to move workloads provides IT leaders with the tools to expand and contract their IT resources and shop their IT needs from a wide range of cloud providers, assuring executive management that their IT cost is competitive.

iBusiness Outcomes

Those who have deployed a Data Center Fabric are rewarded with favorable business outcome results. Cisco’s Data Center Fabric unifies network services, networking and storage plus computing through its Unified Network Services (UNS), Unified Fabric (UF) and Unified Computing System (UCS), respectively. Early adopters have benefited by viewing and procuring their data center assets from this unified holistic perspective versus compute, network and storage separately. For example, Kindred Healthcare saved approximately $6.6M on just cabling cost for a 1,000-server data center, thanks to its deployment of a Data Center Fabric. Additional operational savings was gained by a reduction in the number of management points the operations group has to manage too. To Kindred’s surprise and delight they noticed that the Data Center Fabric enabled different groups—the virtualization team, the network team, and the storage team—to work together as one on a common platform versus in silos; a huge help to hasten deployments especially as Kindred has been growing through acquisitions.

Other early adopters are Almaviva wine producers that saw its revenue increase 2 to 3%, thanks to its data center fabric deployment that also reduced its cabling and power consumption cost by 70% and 60%, respectively. Tutor Perini Corporation was able to reduce its device count and power consumption by 60% and 38%, respectively. Coca Cola was able to consolidate 80 servers down to four, plus reduced cabling 30 to 60%. Terremark saw a 30% improvement in application performance and server density increased by a factor of four. The Apollo Group, owner of the University of Phoenix and other educational properties, doubled the size of its network without an increase in IT staff, lowered per-port switching cost while increasing port volume and freed up several rows of space in its data centers. Avago Technologies, a manufacturer, accelerated batch processing by 30 to 40%, increased business flexibility and decreased operational cost by 40% while adding a third data center. CareCore National, a health benefit management concern, increased business agility by being able to launch new lines of business in just two weeks, down from six months. These iBusinesses’ benefits were gained, in large part through the insight and leadership of IT executives and their deployment of Cisco’s Data Center Fabric architecture.

Cisco has been investing heavily in its Data Center Fabric portfolio. It owns some 80% of the data center switching market and in just two short years, possesses the number three-market share ranking for x86 blade servers worldwide, behind HP and IBM, according to an IDC report released in May. Over the past quarter, Cisco has added to its UF portfolio with the new Nexus 3000, 5548 and 5596 switches. It has expanded its Fabric Extender (FEX) offering to include the adapter and VM FEX, a key technology in converged IO plus virtualization aware networking. To increase mobility of workloads, it has added IP address location independence with its OTV (Overlay Transport Virtualization) and LISP (Location ID/Separation Protocol) features to its Nexus Operating System. Fiber Channel over Ethernet (FCoE) can traverse more devices, thanks to a new director-class multihop FCoE feature available on the Nexus 7000 and MDS 9500. Data Center LANs, SANs and virtualization infrastructure can now be managed via a single pane of glass, thanks to the Cisco Data Center Network Manager. On the computing side, Cisco has expanded the UCS server portfolio with multiple form factors, including Blade and Rack-Mounted, and in the process, has broke three world performance records. Cisco has followed up that with a new set of I/O components for UCS, which was just announced on July 13th.

At the crossroads of the tech industry are two paths; one is a legacy approach of building data centers by acquiring compute, storage and networking gear separately with IT professionals integrating these components. The other road is one of vertically-integrated offerings of compute, storage and networking where IT professionals focus on automating business processes turning their corporation into an agile iBusiness. I advise choosing the latter.

Tweet

In Lippis Report 172, I mentioned three huge trends that are starting to interact with each other creating a perfect storm that is gripping the tech industry. One of those trends is the creation of a software ecosystem in the networking market, thanks to the Clean Slate program out of Stanford University that has spawned the Software Defined Network (SDN) initiative and open controller protocol called OpenFlow. I spent a week in the Valley talking to people at Stanford and many industry executives from Cisco, Juniper, Marvell, Big Switch, Nicira, Arista, IBM and others. In this Lippis Report Research Note, I share with you what I learned. OpenFlow-based SDN is being both hyped and in its current state, limited, but it does represent a new paradigm that has the industry abuzz, filled with possibilities.

Centralized Controller Model

OpenFlow is a protocol, or API, that modifies forwarding tables in network switches. It sits between a switch and controller. The controller can run on a centralized computer/server that has an Über view of the network and its topology. When a packet enters a switch and the forwarding table does not contain a path for the packet, it’s passed to the controller. The controller then searches the packet’s destination address and defines a table entry with associated attributes to create a path through the network, which the packet and subsequent packets are to follow. The controller then sends a message to each switch in the path the packet will traverse via the switch’s OpenFlow API, which modifies the switch’s forwarding table. Every subsequent packet with the same destination address will then be forwarded based upon this table in cut-through mode. The first store-and-forward stage takes about 50ms; yes, a long time, but it can be significantly shortened. Subsequent packets being forwarded in cut-through mode travel at switch latency, which for 10GbE Top-of-Rack (ToR) switches is between 500ns and a few microseconds.

Now this search method is a bit controversial as some claim that all that the controller needs is a large TCAM to compute the table flow. Some worry that a Cartesian explosion may occur, corrupting the calculation, but this is an engineering problem with an engineering solution, perhaps via multi-staging the flow tables.

This centralized controller model can scale as has been proven in distributed computing models used by all the major cloud providers. An example at Stanford demonstrated that a network of 35,000 PCs with approximately 2,000 switches generated 15 to 20k flows/sec. A controller can support 2M flows/sec at half a 2007 PC processor capacity. Further, modern 48-port ToR switches can request 100s of flows/sec with controllers supporting 2M flows/sec, which means that a single controller can support 10s of thousands of ToR switches. In short, a centralized controller-based OpenFlow SDN can theoretically scale.

How an OpenFlow SDN Is Different Than Today’s Network Architecture

The above model departs significantly from today’s network architecture in a few key ways. First there is the concept of a centralized controller(s) versus a distributed packet forwarding architecture based upon topology discovery. There may be separate links for control and data plane communications, which would also be a significant departure from today’s single physical network that supports both control information and data forwarding. There is no layer 2 and 3 construct in an OpenFlow SDN, which has been the semantics of computer networking over the past twenty plus years.

Software Defined Network Ecosystem

Further, on top of the controller is another API, yet to be fully defined, that enables application developers to write network applications without knowledge of the underlying network structure. In short, the API abstracts the network, allowing the programmer to focus on what she/he needs to accomplish versus how to configure the network to comply. The creation of a software ecosystem creates the possibility of a new network paradigm where low cost Asian switches populated with SDN software force an economic collapse of the existing network market. While this is highly unlikely, it does warrant careful observation and mitigation planning on the part of established vendors.

An OpenFlow SDN offers significant differences, which is why there is such excitement surrounding OpenFlow. The genius of the approach is the separation of data and control plain so that SOA-based application developers and researchers can layer applications onto the network, injecting innovation at speed via a software ecosystem. Further centralized controller-based networks such as the national cellular network plus dense compute management have proven to reduce operational cost and increase control in complex systems.

There is an industry group called the Open Network Foundation, or ONF, that is promoting the use and interoperability of OpenFlow SDN enabled switches. The above OpenFlow SDN example is primarily an academic description as OpenFlow is well regarded as the leading open implementation to date for providing SDNs within the research community. But there will be many networking concerns introducing controllers that reside in the switch. Further, the definition of a controller is a bit vague as some define it as a network operating system, such as Cisco’s IOS or NX-OS, Juniper’s JUNOs, Arista’s EOS, etc., while others define it as a management entity, performing configuration changes. But before we dive into this, let me explain a few problems that an OpenFlow SDN may solve.

Innovation at Speed: The institutions that were created to assure interoperability and inject innovation into our industry have become too cumbersome and slow such that networking has fallen behind compute and storage advances. The way innovation is injected into networking today is that a proposal is made to a standards group, such as the IETF, IEEE, etc., and all interested parties compete for the best ideas or technical advantage. This process can take a few years just to modify a few bits in the header of a packet. Then, once the standard is completed, companies build to it, which can take another eighteen to twenty-four months. This approach is not serving the industry any longer, and there needs to a more rapid way to inject innovation. An OpenFlow SDN promises such an approach where applications can be added to the network rapidly, thanks to the abstraction of layer 2 and 3 forwarding.

Traffic Engineering: Fine-grained traffic engineering utilizing a variety of forwarding actions is an application that service providers and enterprises seek to optimize application performance.

Tagging vs. Table Manipulation: There is much agreement in the industry that the network has become too ridged in virtualized data centers, restricting the movement of VMs between racks, data centers, etc. Further, as appliances such as firewalls, load balancers, IPS, etc., have become virtualized, there needs to be a method to steer traffic to them to service an application. The industry has responded to this by proposing the placing of tags on packets to guide its path to the right VM, appliance. An OpenFlow SDN implementation could simply modify switch-forwarding tables to guide the application through a chain of appliances mitigating tagging and offering applications appliance servicing within highly virtualized infrastructures.

The Real World

An OpenFlow SDN is new, and it’s unrealistic to think that it’s without challenges; here are some OpenFlow challenges.

Trust: The single largest issue an OpenFlow SDN has is trust. Will IT business leaders trust it within their networks, especially their data center? If a controller is sourced from a new company, how comfortable will the IT team be that it’s modifying switch-forwarding tables? How many controllers are needed for a particular load? What will the support model be? How complicated will it be to manage multiple controllers?

Interoperability: The current construct of OpenFlow requires knowledge of the switch’s hardware semantics of L2/L3/VLAN architecture; therefore, each controller implementation may be different and thus unclear how controller interoperability is achieved. Further, it’s unclear how applications written for one controller will work on another.

Network Stability: This issue may be linked with trust, but it’s unclear why a third-party controller should search packets to define a path through the network topology. Rather, why not use existing network operating systems for what they are good at– topology discovery, etc.–so that IT business leaders are more comfortable running OpenFlow-based SDN applications on top of a stable network. In short, will OpenFlow controllers introduce instability?

Controller Placement: If we take the definition of a controller to include existing network operating systems, then there will be both distributed and centralized controllers within a network. From a design point of view, how does an IT architect approach distributed versus centralized controllers and what are the trade-offs?

It’s unfair to expect that a new approach to networking would have the above issues all sorted out before deployment. These are not barriers to entry but rather challenges that the OpenFlow SDN community will work on over the next one to two business cycles. Let me be clear…OpenFlow-based SDN is a very big deal and is being embraced by all vendors including established firms and start-ups. What is driving most companies is the promise of a software ecosystem to inject innovation and value into their network products.

Established firms will support OpenFlow SDN via OpenFlow client reference implementation within their switches but will add proprietary extensions that differentiate their OpenFlow version from others. Cisco, Juniper, Arista, et al, will differentiate based upon how much of their network operating system they expose. Established firms should have an advantage over smaller ones in attracting software developers as their installed base is much larger.

New companies such as Big Switch Networks and Nicira will focus on solving particular problems in the data center, service provider and enterprise network that existing layer 2/3 networks either don’t solve or don’t solve easily. Virtualization of both servers and desktop are two prime areas, and I expect a suite of SDN Virtualized Applications to emerge from these firms and others.

The service provider market is perhaps the biggest OpenFlow SDN winner as early experiments have shown that the existing three-tier service provider architecture of packet switching, optical core and edge may shrink over time to just two, thanks to traffic management applications.

OpenFlow SDN has successfully introduced the concept of controller-based networking and the controller market. OpenFlow 1.1 is in standardization process and once completed, will be the first defined open controller API to communicate between network and controller, offering greater control of cloud network resources and management. But perhaps the greatest contribution an OpenFlow SDN will offer is the potential to usher in a wave of fast-paced innovation not seen before in the networking industry.

Tweet

Three strong trends are taking shape that are so powerful they threaten the status quo of the networking industry. These trends are more like storms than new markets; in fact they represent a major industry discontinuity. The first storm is happening now and is represented by merchant silicon for 10 and 40 GbE chips lowering the barrier of entry for new entrants in the Ethernet switch market. The second storm is much weaker but promises to be just as big, or bigger, than the first. This second storm is the creation of a software ecosystem in the networking industry, thanks to initiatives such as Software Defined Networks (SDN), OpenFlow, Arista Network’s EOS Central, etc. The third storm is the paradigm shift in enterprise IT spending thanks to mobile and cloud computing. These three storms are starting to interact and feed upon each other, forming a perfect storm in the networking industry. The perfect storm is already doing damage, as all major IT firms position product portfolios to navigate through it and prepare for its aftermath of making existing networking legacy.

IBM, for example, sees the perfect storm as an opportunity to optimize performance of IBM systems for new and emerging workloads like cloud computing and analytics that require instant access to information by investing in networking. In this Lippis Report Research Note, we focus on IBM’s networking strategy and analyze its potential impact.

IBM created the System Networking group to organize its network resources and execute its strategy. It’s a strategy to implement a data center fabric that ushers in a smart compute model that federates storage, compute, memory and I/O into pools of resources that are brought together to meet business requirements. It recently acquired BLADE Network Technologies (BNT), which produces blade and Top of Rack (ToR) data center switching gear, network-aware virtualization technology, load-balancing and management software. From an organizational point of view, IBM System Networking includes BNT and an IBM group that used to be called Data Center Networking that possesses Fiber Channel and InfiniBand assets. System Networking also maintains working relationships with networking leaders such as Brocade, Cisco, Juniper Systems and Mellanox.

IBM has been selling system networking solutions with its servers and storage offerings for decades. Systems and networking are now more interconnected, making it important to continue partnering with core networking providers like Brocade, Cisco and Juniper while enabling closer connections with IBM servers by increased investment in systems networking technology.

But why did IBM enter the System Networking business and why now? In short, IBM executives saw an opportunity to gain control of a critical data center asset, address customer needs, and add a key component to deliver on its vision of Smarter Computing. From discussions with IBM executives, they stress common concerns of their largest data center customers, which have propelled IBM into the System Networking business. Clearly, Cisco’s launch of Unified Computing System or UCS and the forecasted perfect storm also factor heavily into IBM’s calculus. IBM is hearing demand and seeing a shift in the networking industry that has opened a door for it to be a leader in data center enterprise networking, or System Networking, as IBM now calls it.

Cloud Spec Scale

The largest data center customers are implementing cloud spec facilities that are boosting up their infrastructure spend and deployment by an order of magnitude in many cases. Yes, that’s ten times the size of their normal data centers. This scale has created unique problems that challenge linear approaches and are focusing IT business managers to seek alternative solutions to scale.

The old model of increasing capacity of memory, compute, I/O, and storage, etc., by acquiring more servers does not work any longer. IBM seeks to solve this scale problem with Smarter Computing that delivers elastic services to federate a pool of resources that are brought together to meet business needs for Big Data analytics and private and public clouds. Resources could be memory, I/O, compute or storage. The goal is to bring together the right proportion of resources to solve a particular workload.

Why Networking Is Important to IBM

To deliver on Smarter Computing, IBM realized that to offer a federated pool of resources, it needs a network fabric that connects these assets, and thus this is what System Networking is all about. IBM let other industry players connect high-density blade and rack system with their network gear. This left IBM out of the innovation loop and control allowing others to set the rate and pace of network innovation.

The need to own the network and provide IT business leaders with vertical IT expertise has become apparent. If the data center rack is the new computer, and multiple racks are the new super pod, how does a supplier make this system look and feel like one large computer? It all starts with connecting these elements together in a very smart fashion using physical connections and software to orchestrate resources and infrastructure simpler than today’s approach.

How can IBM make dense IT infrastructure simpler to deploy and manage as its largest customers deploy ten times more infrastructure? Most IT business leaders translate this into the need for rack infrastructure management, configuration management, and database technology to keep track of IT assets, etc. While IBM has director and utility tools, System Networking is a critical component of Smarter Computing. IBM executive management figured that System Networking will play an even more important role in solving new IT business leader requirements that include simplifying massive amounts of IT infrastructure installation and orchestration, be it physical or virtual.

At the high end of the enterprise computing market, IT business leaders are acquiring IT assets like airlines buy airplanes and hotel builders buy property. Both airlines and hospitality concerns worry about the same thing: use or occupancy rate management. Airlines want to ensure that they have the right size aircraft for a particular flight route so that few, if any, seats are left empty.

As IT business leaders scale up their data centers to cloud spec, thanks to IT service demand, how do they ensure that the capacity acquired is effectively utilized and not over or under designed? Most, if not all, IT business leaders have embraced server virtualization as the key technology affording efficiency gain.

Without System Networking, IBM management realized that it was unable to address IT business leaders’ full virtualization requirements. The data center network needs to be virtual machine aware. In fact, this is one of the biggest reasons why IBM acquired BNT as IBM needed BNT’s network virtualization expertise.

More Business Goes Online

The reason why IT business leaders are deploying so much more infrastructure is that more of their business is going online. Just think about your average day. When communicating to each other we text, email, VoIP and videoconference. When you want to go see a movie, you book it online. You bank online. You pay your bills online. You trade stock online. You make airline reservations online. You read news online, your photos are stored online, office productivity tools are online, etc. As more and more business goes online, the scale of IT infrastructure needed increases.

In addition to more business going online, IBM’s big analytics business needed networking too to be first class. IT business leaders are putting in place more analytic systems, decision support systems and data warehousing systems so they can mine their depositories of vast amount of information that they have about customers, business, products, competitors and supply chain, etc., so they can make smart important business decisions.

This is why data warehousing, data mining, smart analytics or solving the big data equation is so important to IBM. This is why IBM acquired Netezza. Now, what is the difference between a good data warehousing engine and a great one? The answer: how fast data can be transported to and from the analytic engine, or how fast is the network. For IBM to be a successful player in smart analytics and be recognized as the clear leader in this large and very important market, it realized that it needed to be in the networking business.

Controlling TCO at Scale

As data centers have been scaling up, so too has Total Cost of Ownership or TCO. For every dollar that CFOs spend on servers and storage, they spend between 15 and 25 cents on networking. IBM is not able to control a customer’s TCO as it has no control over 15 to 25% of the IT budget. Therefore, how could IBM profess to solve the TCO equation when it can’t provide a credible solution to 15 to 25% of the TCO problem? IBM needed to have a voice and solution for TCO, thus this too factored into its thinking of re-entering the networking industry.

The change in IT buying requirements is the first of three storms that IBM saw as IT business leaders are building private clouds and experiencing scale issues associated with them. Data center buying criteria is changing as scale, density, deployment, orchestration management, efficiency and utilization, security, being able to extract meaningful decision support information out of information repositories, as well as cost of ownership become high priority items. The merchant silicon storm stirred up by companies such as Broadcom and Fulcrum Microsystems got IBM’s attention. IBM got a close up look at this storm, as BNT built its new ToR switches with Broadcom’s Trident-1 10GbE and 40GbE chips and decided to invest by acquisition. It was these two storms and its forecast of a third in the creation of a network software ecosystem emerging that in the end tipped IBM’s hand and led it into the data center system networking industry, or System Networking, as IBM now calls it.

The New IBM

IBM realized that not having System Networking was a competitive disadvantage especially in its analytical systems business. There was an underlying reliance on the network that IBM didn’t control. IBM realized that System Networking is a strategic asset, and it needed to invest.

IBM is now a three-stack business with its platform business including compute, storage and networking, then software and lastly, services. Software is the biggest business followed by services, and then its platform business. Without networking, IBM’s business model was incomplete. How can you drive innovation in software and smart analytics, etc., and all the services to go around it, if you have one or two missing pieces in the platform equation?

Others to Follow

IBM is not the only large vertical IT player to beef up its networking business. Clearly there is HP, Oracle, IBM, Dell and Cisco. Cisco possesses a different portfolio mix than the others with its dense networking portfolio. HP, on the other hand, possesses approximately $2.5 billion worth of networking products/revenue, but lacks data center networking.

Consider Oracle and IBM—they are both focused on the data center. With Oracle’s recent acquisition of Sun, it too is viewing the perfect storm as an opportunity to enter the networking market. But the fundamental thing that is different about IBM is that it is singularly focused on the data center. This contrasts with Cisco’s network focus while HP strives to be the low cost alternative to Cisco, plus its huge consumer line of products, such as printing and personal computing. Dell, on the other hand, is focused on transitioning away from the personal computing market into higher margin businesses, networking being one of them.

What all of these firms are searching for is a new networking model to emerge, and the perfect storm may very well provide it. With low cost merchant silicon that competes with custom ASICs, network switching is fast, low latency, low power consuming and low cost. With software defined networking (SDN), a new software ecosystem could emerge that challenges established network services and in the process, starts an innovation race between established vendors and a new software industry. SDN is critical if a new networking model is to emerge as it could enable innovation that differentiates common merchant silicon-based network switches. In short, the perfect storm could enable the large IT vendors to leapfrog into a new system networking paradigm.

IBM has its work cut out for itself. BNT has expanded from Ethernet embedded blade server switches to ToR switches. IBM will enter the aggregation space with the implementation of technologies such as TRILL (Transparent Interconnection of Lots of Links) and 802.1Qbg, the Edge Virtual Bridging (EVB) standard that will seek to break the model of large centralized mainframe like modular switches. And, through partners such as Brocade, Cisco, Juniper and Mellanox, IBM System Networking offers a portfolio of Fibre Channel and Infiniband as well as Ethernet solutions, for servers and storage from network edge to core. IBM’s point is that if servers and storage can scale out then why can’t networking?

IBM is developing new networking products that it hopes would enable it to change the networking landscape and how people think of networking. It seems that IBM System Networking is working on a scale out networking model that allows IT business leaders to start smaller and expand as needed without large upfront capital outlays. It is looking to make networking a bit smarter.

IBM System Networking is focused on building what it calls “a scalable fabric,” which connects servers, storage and networking. Thus IBM advocates to keeping network intelligence close to servers and storage making its fabric fast, low cost, virtual and reliable.

Time will tell how successful IBM is in System Networking, but one thing is for sure, cloud computing has kicked up quite a perfect storm for it.

Tweet

In the Lippis Report, we have discussed the fundamental changes shaping a new data center network architecture. These drivers are massive virtualization, a sea change in traffic patterns that are now dominated with east-west flows on top of existing north-south traffic, ultra low latency, the emergence of cloud spec data centers, etc. As a result, data center networking attributes are changing with requirements of traffic, steering in virtualized infrastructure, avoiding manual network changes as VMs move, removing oversubscription (thanks to spanning tree), streamlining network tiers to hasten east-west traffic flows, etc. The industry is responding to these changes and requirements with new approaches to data center networking, such as the Open Networking Foundation, Cisco’s FabricPath, Juniper’s QFabric, Brocade’s VCS, Avaya’s VENA, Nicira Networks’ network virtualization software, etc. In this Lippis Report Research Note, we explore a key technology to enabling two-tier network fabrics, and that’s link aggregation and its various approaches, including Multi-Chassis Link Aggregation Group, Transparent Interconnection of Lots of Links (TRILL) and Shortest Path Bridging (SPB).

Over the past year, firms such as BLADE Network Technologies, an IBM Company, Force10 Networks, Juniper Networks and Voltaire/Mellanox have introduced 48 port10GbE top-of-rack (ToR) switches. Before Interop in May, there will be six more companies making similar announcements. With 10GbE priced at $300 per port and below for server connections, the transition from 1 to 10GbE is on its way in the data center. Now most, if not all, of these switches possess two 40GbE uplinks. Also by Interop, at least two firms will announce Core switches with dense 40GbE capability. So the question is how are these ToR products being connected so as to address the changes mentioned above?

One Thousand Plus Servers Connected at 10GbE
Consider a 1,024-server data center where all servers are dual home connected into the fabric via 10GbE. This example could be a Global 2000 company data center, but many Global 2000 companies and service provider hosting companies have larger scale requirements in the tens of thousands of servers to over one hundred thousand. In this example, approximately 2,048 10GbE connections are needed. Consider this requirement using traditional approaches.

If designing this data center fabric with traditional spanning tree protocol (STP)-based networking, there would be blocked links between access and distribution. The IT architect would rely upon a three-tier structure that forces an oversubscription of nearly 8:1 between access and aggregation, and 2:1 between aggregation and core, or a total of 16:1 oversubscription. There would be 64 access switches, 8 aggregation switches and two Core switches required and four pods to house access and aggregation switches. In addition, east-west traffic flows are forced to traverse these network tiers, incurring delay with every passage of a switch.

To eliminate the oversubscription and reduce latency, a two-tier network architecture can be utilized. One approach is to use 43 of the new 10GbE ToR switches to connect servers. Connecting ToR switches would be some number of Core switches with enough capacity to support 512 40GbE or 2,048 10GbE connections, if non-blocking is a requirement. The Core switches would need to be connected together too at very high speeds and densities. Yet another approach would be to use Core switches to connect servers. Assuming a Core switch capable of supporting 256 10GbE ports, then eight Core switches would connect servers. Now, if the IT architect required non-blocking, then a Core switch would need to terminate 48 10GbE for each ToR switch or 256 10GbE links for each server facing Core switch. There lies the rub; with such large numbers of parallel 10 or 40 and eventually 100GbE links, there needs to be a way to aggregate and route between ToR and Core switches.

Enter link aggregation. The two-tier architecture allows the level of oversubscription and blocking to be designed and managed by choosing the number of links to be aggregated.

Key to this design is the elimination of STP with some number of multi-links between ToR and Core that eliminate oversubscription, and enable a non-blocking fabric, assuming the switches are designed with enough backplane capacity to support packet forwarding equal to the sum of leaf ingress bandwidth. High spine switch performance is fundamental in the two-tier leaf-spine architecture as it collapses the aggregation layer in the traditional three-tier network. Further, by connecting every switch together in a full mesh via link aggregation connections, every server is then one hop away from each other, reducing latency and providing VM mobility service.

There are multiple approaches for connecting ToR/leaf and Core/spine switches at high bandwidth via some type of link aggregation.

Multi-Chassis Link Aggregation Group or MC-LAG covered in project IEEE 802.3ad allows one or more links to be aggregated together to form a Link Aggregation Group. MC-LAG is a method of inverse multiplexing over multiple Ethernet links as if it were a single link. This layer 2 transparency is achieved by the LAG using a single MAC address for all the device’s ports in the LAG group. LAG can be configured as either static or dynamic. Dynamic LAG uses a peer-to-peer protocol for control, called the Link Aggregate Control Protocol LACP.

TRILL or Transparent Interconnection of Lots of Links is an emerging IETF protocol based upon a link state routing algorithm IS-IS that broadcast routes available to all TRILL connected devices for pair-wise optimal unicast paths. TRILL is invisible to routers as it runs over layer 2 links such as Ethernet and PPP.

Shortest Path Bridging or SPB is an IEEE 802.1aq standard solution for shortest path frame routing in multi-hop Ethernet networks with arbitrary topologies. SBP, like TRILL uses IS-IS link-state routing protocol to advertise both topology and logical network membership. SPB packets are encapsulated at the edge either in mac-in-mac 802.1ah or tagged 802.1Q/802.1ad frames and transported only to other members of the logical network. Unicast and multicast are supported, and all routing is on symmetric shortest paths.

MC-LAG vs. TRILL vs. SPB

As you would expect, there is debate over which approach is best, MC-LAG vs. TRILL vs. SPB. It doesn’t help that TRILL is an IETF standard, while SPB and MC-LAG are IEEE. Picking a winner is complex as there are pros and cons to each, and all protocols have their supporters in the vendor community. MC-LAG may be the most widely supported protocol but lacks link state routing to define paths. Some even question if you need IS-IS at this level of the network.

From an implementation point of view, many firms are betting on SPB, such as Brocade in its VCS, Avaya in the VENA, Alcatel-Lucent in its OmniSwitches. These firms like SPB for its following advantages:
SPB scales to support 100s to 1000s of multi Terra bit switching enabling a non-blocking two-tier network fabric;
SPB creates logical trees, which can be extended out of the data center and into the campus increasing SPB’s usefulness.

SPB service provider deployments are planned for 2011 and they believe SPB offers increased scalability over TRILL. Further, SPB will interoperate with carrier infrastructure to allow seamless data center-data center connections in the near future. This is an interesting and compelling option in that SPB could be the link that connects private and public clouds via a single data center fabric.

SPB advocates boast that for network architects/designers and operations, there is a quick learning curve as SPB uses the existing IS-IS protocol, and for service providers, SPB is already available through OAM (Operations, Administration and Maintenance), enabling it to be managed through existing management services.

Perhaps the biggest proponents of TRILL is IBM and Cisco, which has its FabricPath offering based upon it, and Data Center Bridging before that. Its proponents point to TRILL’s advantages of multi-pathing that delivers higher throughput between leaf and spine connections. TRILL too can be extended out of the data center into the campus and cloud as service providers offer TRILL connections. It’s also backward compatible with classic bridges, and was developed by Radia Perlman, the inventor of Spanning Tree Protocol.

Juniper’s QFabric

Then, in addition to the above, Juniper recently announced its QFabric architecture, which disaggregates the data, control and management planes. Its QFNodes are ToR switches, which are connected to its QF Interconnect chassis and managed via QF Director management platform. There are two separate connections for data and control plane traffic, with control traffic on a 10GbE link while data traffic runs at 40GbE. It’s not clear if the QFabric is a cell based data gram architecture, or if it uses Ethernet data grams. If QFabric is a cell based architecture, then it would not utilize TRILL, SPB or MC-LAG for inter-switch high-speed aggregated connections and routing.

Enter the Open Networking Foundation

Then, there’s Open Networking Foundation (ONF) started by Deutsche Telekom, Facebook, Google, Microsoft, Verizon and Yahoo with 17-member companies, including major equipment vendors, networking and virtualization software suppliers, and chip technology providers. ONF is proposing a new approach to data center networking it calls Software-Defined Networking (SDN).

SDN comprises of two basic components: a software interface (called OpenFlow) for controlling how packets are forwarded through network switches, and a set of global management interfaces upon which more advanced management tools can be built. The first task of ONF will be to adopt and then lead the ongoing development of the OpenFlow standard (www.openflow.org), and encourage its adoption by freely licensing it to all member companies. ONF will then begin the process of defining global management interfaces. The hope is that SDN will help networks become both more secure and more reliable.

Nicira Networks is a real world provider of SDN type solutions. Nicira proposes splitting control and data planes so that the data center network can be completely virtualized, like VMware did for servers. That is the right operational model for networking, where you treat the physical infrastructure as a generalized resource pool of switching capacity, and all of the services intelligence is done at edge in software, and the physical network does one thing and one thing only…forwards IP packets. 

MC-LAG, TRILL and SPB offer a linear approach to scaling data center networking while Juniper’s QFabric and ONF’s SDN offer new departures in the design and architecting of data center and cloud computing networking. While QFabric and SDN are interesting, they need to be developed and understood, but represent a new approach to networking that our industry has not seen. Over the next several years most IT architects will choose the linear approach as QFabric and SDN become fleshed out and their pros and cons articulated.

Tweet

Cisco recently launched its SecureX architecture that extends perimeter-based network security to secure modern IT, recognizing the huge growth in mobile and cloud computing. SecureX is a multi-layer architecture built upon Cisco’s AnyConnect client, its global footprint in real-time threat intelligence found in SIO (Security Intelligence Operation), Cisco TrustSec, including policy servers of NAC manager and server appliances, ASA firewall and the security enforcement features of its switches and routers. SecureX is an architecture to Cisco’s network security products and service to work together in an effort to create deeper defenses and contain exploit infestation if, and when, they occur. Fundamental to SecureX is the concept of “context aware” policy across the enterprise, including remote endpoint devices, centralized policy creation with distributed security device and network enforcement. SecureX provides for innovation injection points through APIs (Application Programming Interfaces) for management and SIEM or Security Information and Event Management. In this Lippis Report Research Note, we explore SecureX with a focus on how context increases defenses and keeps IT assets safer.

SecureX offers something for everyone…such as a simpler, yet richer, management model for SecOps, deeper levels of security for users within and outside the corporate network, centralized policy creation that extends beyond the corporate firewall, and increased protections for users as they utilize mobile endpoints to access corporate and cloud-based applications. IT business leaders should be pleased with better protections and compliance tools, especially as their vulnerabilities increase with mobile endpoints seeking network access growing.

SecureX is not just about extending security to mobile devices but to capturing contextual information in the use of policy creation. Contextual information includes user and device identity plus location, login time of day, plus which specific applications users attempt to access too, and this information is not only collected upon login but during their entire network connected session. Context aware policy allows IT leaders to use this information in the creation of policy with the end result of either allowing or denying access to IT resources, independent upon endpoint device and method of which access is attempted. And this context aware policy attribute of SecureX, over time, will be extend beyond normal data traffic streams to apply consistent unified policies to application, video or voice traffic also.

And while SecureX is security, in reality, it’s bigger than just security, because security is a necessary integrated attribute to enable mobility, video, voice and web collaboration, etc. To create a secure IT environment, IT services need to interact with security services with minimum to no user intervention that steals productivity. In short, SecureX seeks to make Cisco security and network devices work better together through context aware policy so access and deny decisions are improved, and are built upon so that anomalistic behavior remediation is automated post access through traffic monitoring.

Use cases have changed dramatically since a new tier of computing has emerged, that being smartphones and tablets. For example, a laptop could be plugged into an iPhone, which is streaming video into the corporate network. The network should be able to differentiate between data traffic, video traffic, phone traffic and even iPhone application traffic, then monitor all of those traffic types for behavior so if a Virtual Machine (VM) is launched on the laptop, the network recognizes this new entity and performs a new series of monitoring. Security needs to be much smarter as the combinations and permutations of acceptable user behavior are fundamentally changing.

So where does this monitoring come from? Is it centralized, distributed, within appliances, in the cloud? The answer is all of the above. It’s in the network infrastructure and highly distributed. The SIEM ecosystem plays a role, TrustSec provides monitoring as does SIO, ASA, IPS, etc. The network infrastructure itself is monitoring behavior that’s outside of parameters/rules/policy that have been established for each network connection, and can take defined action when anomalistic behavior is identified. With monitoring and enforcement being so highly distributed, the chances of capturing anomalistic behavior increases significantly. Anomalistic behavior can occur anywhere, so depending upon where alerts are triggered, what type of traffic is involved, the kind of device being used, the location, the identity of the user, the time of day, etc., it’s this contextual information that adds color to tripping anomalistic behavior and remediation options.

SecureX is much like Cisco’s self-defending network concept, but with a global perspective and tools to extend contextual base security to the Cloud, virtualized environments and out to the growing mobile workforce. And this extension of security services is the biggest challenge with which IT business leaders struggle. IT leaders want to push context aware policy into their virtualized datacenters, their Cloud(s) and to mobile users, because it solves a large set of security problems. In fact, security concerns is one of the primary gating factors limiting enterprises from deploying these new innovative IT services that offer favorable business processes outcomes.

Context Is Fundamental to Access Decisions

We already have perimeters and defenses within the enterprise, but IT has gone mobile, thanks to smartphones, iPads, tablets, etc. Also, applications are selectively moving into the Cloud as well. SecureX is a security architecture delivering control to SecOps and IT business leaders to extend their IT services to mobile workers, enabling them to embrace a new tier of computing and a new way of application delivery via the Cloud.

SecureX adds the concept of context aware policy to the principles of visibility and control as context provides insight into threats as employees are working outside of defined enterprise perimeters. The type of context that’s important includes identity—such as who are you, where are you located, the device that you’re using and can I trust the device—and what resources are you seeking to access. All of this contextual information needs to be considered when a firewall is determining network resources it will allow access to. In addition, contextual information may also instruct the network to enforce encryption on a session based on who you are and where you’re trying to go.

Policy Driven

To make contextual information work, a policy wrapper needs to surround context elements of personal identity, device identity, location, time of day and application access request. That is empowering the network to being able to create a uniform policy, such that the network is able to intelligently negotiate a variety of context options that are being considered when individuals attempt to access IT resources. This is the perfect job for a policy appliance.

To add context information to firewall decisions, Cisco is leveraging key pieces of its security product portfolio. For example, its TrustSec architecture provides access control plus encryption, which is the first and most critical piece of context information. Within access control, a device’s security posture is assessed, the end user is identified, and their device is profiled, all of which is used to make an intelligent decision to grant or deny network access. In addition, the network can “tag” a user’s data stream, so that as the stream transverses throughout the enterprise IT infrastructure, the network can enforce defined policy independent upon the stream’s destination(s). For example, once the user has passed access control, should this user decide to search for a payroll server location, the network may recognize that he/she is not allowed access, thanks to defined policy, and the network can drop the requests and log the event. This set of sequences is a benefit of TrustSec.

Access Control and Contextual Information

With trusted systems on the inside of an enterprise network providing enforcement through policy of mostly fixed endpoints, such as desktops and IP phones, the question on most IT business leaders’ minds is how to extend these protections to the exponentially-growing mobile community and non-user network devices. IT leaders are confronted with an increasing number of both mobile endpoints and non-user endpoints, such as printers, video surveillance, wireless access points, etc., attempting to access their network and IT assets. To protect IT assets, IT leaders are seeking a process in which all devices connecting to the network, independent upon inside or outside the perimeter, are profiled to analyze device function and apply appropriate policy. For example, an IP camera may be identified during profiling and then a policy applied that allows IP cameras to transmit data, but not allowed to request data. In addition, during post access control, the network then monitors the IP camera to assure policy is applied while the IP camera is connected to the network.

This type of contextual information to build another level of defense is also extended to the virtualized data center environment. For example, once a virtual server comes online, policy can be applied to it, which is then communicated to the entire infrastructure. Policy may allow a virtual server to pass traffic between VMs on a select number of hypervisors. In addition, these VMs may also recognize that the new virtual server can do X and Y with these VMs but not Z. This level of control granularity enables SecOps to define virtual environment behavior in a meaningful way.

The Network Can Be the Firewall

Clearly policy management is an integral component of SecureX. To define policy, Cisco offers the Cisco TrustSec solution, which can be deployed using the NAC Appliance or with a network-centric 802.1X strategy, combined with the Access Control Server. These solutions offer posture assessment, remediation and quarantine functionality. Device profiling for non-authenticating devices such as IP Cameras, printers, WLAN access points, etc., are placed on guest services with triple-A services. The aggregate of these features with the ability to create centralized policy that can be pushed out to the entire network infrastructure creates, in essence, a highly-distributed firewall. If a firewall’s job is to allow or deny access to IT resources, then SecureX turns the entire network into a highly-distributed firewall, where every component of the network is now analyzing and processing traffic.

Enforcement and Layers of Context

Context aware policy enforcement is performed with network infrastructure such as network switches, routing, firewalls, IPS, VPN, etc. There are layers of context: who are you, and should you be allowed to go to this website; or who are you, and what should I do with the types of email that you’re creating, or the traffic you’re generating based on who you are? It’s a meta context environment that asks, “Who are you in a dynamic environment?” In this dynamic environment, a higher-level policy may ask, “When you’re inside the network, there’s one set of rules. But if you leave the network, policy moves and perhaps changes with you.” For example, an exchange between two users may be allowed while both are inside the network. The network could allow certain content to pass between the users. But if one moves outside the network, then the network could stop some content from moving between them. Another example of enforcement due to anomalistic behavior could be a user logging in from within his/her New York network while another login request comes in from the same user located in Shanghai, China; the network needs to make a decision about which one of these users is authentic, and what action to take upon both users.

Networking Is Much More than a Connectivity Service

Enforcement is performed in both security appliances and network infrastructure. This elevates the network beyond a connectivity service to a secure IT service where it provides visibility, context and control, thanks to SecureX. When a network utilizes 802.1X for access control, the network is not only providing connecting, but also enforcement, for example. A SecureX network is creating and analyzing policy tags, performing enforcement of policy, dynamically identifying new devices, monitoring traffic, communicating with policy server(s) and making decisions about which access rules to apply to a device.

Protecting Mobile Users

The key architectural approach to SecureX is that the mobile device is equipped with a thin client, that being AnyConnect with the heavy processing burden of threat intelligence, mitigation and enforcement left in the Cloud or at the corporate head-end. Cisco’s AnyConnect plays an important role in SecureX to protect mobile devices as it leverages a huge resource of threat intelligence. SIO collects and analyzes traffic of approximately 5 billion emails per day, 3 billion Web requests per day and 700,000 network sensors or IPS; expand that to include approximately 100 million endpoint devices that are equipped with an AnyConnect client, and SecureX provides the most comprehensive real-time threat intelligence telemetry and mitigation to endpoints.

All of these numbers can be boiled down through a few examples. Consider a user—with a laptop equipped with an AnyConnect client—is attempting to log into her/his corporate network. At the point of login, the network will identify the user, her/his role and which resource she/he is attempting to access. For example, Bill from finance is requesting access to the payroll server. Policy may be defined as Bill can only have access while he’s inside the network perimeter, but not outside. Further, if Bill’s inside the network perimeter, policy may dictate that access to financial servers are encrypted via MACsec. No need for Bill to take any action, as a MACsec tunnel is established automatically as a matter of policy.

Mobile Internet Browsing

Consider an AnyConnect iPhone mobile user browsing the Internet with Cisco’s ScanSafe dynamically managing the Web interaction. With the endpoint’s VPN connection terminated on an ASA firewall, behavior is monitored. If anomalistic behavior occurs, such as malware activity traversing terminated VPN connections, ASA, in conjunction with ScanSafe and SIO, can extract that information and analyze it. In the event that a virus is propagating on iPhone-based smartphones, SecOps can be notified with a message such as “This is a warning. There’s something big happening on iPhone smartphones, and it’s happening in this part of the world. SIO is analyzing this information, will create and distribute a signature fix shortly.” This type of message can be pushed to all AnyConnect VPN terminating devices: “There’s an iPhone virus coming on. SecOps is blocking it for the moment, and in the next few minutes, we’ll distribute a signature to destroy this virus.”

A SecureX Ecosystem Is in the Works

There are two innovation inject points into SecureX to enable an ecosystem for management and SIEM. The management API offers an approach to a wider and consistent management view of network and security resources. SecOps often requested a super management platform where visibility and control is available from one tool. Unfortunately there is just too much information to display in one management window. But if multiple management tools/windows consulted the same policy data and shared this information, then a more consistent view of network assets can be obtained. An API to enable this type of information sharing would enable NetOps to manage its switched environment and be able to control not only switches, but also gain visibility in a security context of what policies have been applied to that switch. This concept can be extended to all network element management where they share policy information.

While not detailed in Cisco’s SecureX architecture, Cisco did announce a new SIEM ecosystem last month as it placed CS-MARS in end-of-life. This SIEM ecosystem will contribute to the contextual element of SecureX. For example, there are a number of ecosystem partners in place providing sophisticated types of analysis as they deepen their interaction with Cisco’s network infrastructure products. These partners collect and gather real-time alarm information and are correlative to global SIO. The combination of Cisco’s SecureX and its SIEM ecosystem will be able to span threat intelligence from local machines to the global footprint of SIO, offering an expanse of security information that can be put to work to protect assets and mitigate threats once detected. These real-time local and global threat intelligence assets can also be interfaced with a policy engine to not only identify and control devices requesting network access, but to monitor behavior within and outside a corporate network.

The value benefit to a SIEM ecosystem and SIO feeding real-time global information to a policy server is best described through example. Should a device suddenly begin behaving anomalistically, the network can automatically identify the device and its closest switch, and take action, such as lock the device and redirect it to a remediation server. That is, SecureX will be able to perform infection containment and control, thanks to adding real-time local intelligence to the policy sever, thereby changing policy on the fly based upon contextual information.

SecureX is Cisco’s latest attempt at integrating security deep into the network infrastructure as this infrastructure expands to mobile devices, cloud service providers and virtualized infrastructure. Its core component is context aware policy that is centrally administrated with enforcement highly distributed. SecureX is a modern security architecture for a new age of mobile and cloud computing.

Tweet

The data center switching market is heating up. To address the scale issues posed by mobile and cloud computing nearly every network vendor is launching its own version of a 10/40/100 GbE fabric to connect servers and storage to the internet. At the heart of this fabric is a two-tier (Fat-Tree) network made up of leaf/ToR and spine/Core switches. Here leafs connect servers and spines connect leafs while also being interconnected in a logical mesh. The protocols to create this logical mesh are based upon IS-IS link state routing, but each vendor is taking a unique approach with Cisco using its FastPath, Alcatel-Lucent and Avaya using SPB (802.1aq Shortest Path Bridging) while Brocade VDX is based upon TRILL (Transparent Interconnection of Lots of Links). Juniper recently announced QFabric but has not detailed what it’s using for logical meshing. At the center of new data center design are leaf and spine switches. In Lippis Report Research Note 166, we detailed the latest ToR switches. In this Lippis Report Research Note 167, we dive into performance and power consumption measurements plus the use of SPB of Alcatel-Lucent’s OmniSwitch 10K, a new entry into spine/core data center switching market.

During December 6-10, 2010, the Lippis Report and Ixia conducted the industry’s first 10GbE data center switching evaluation of Top-of-Rack and Core Ethernet switches at the modern iSimCity lab in Santa Clara, CA. We evaluated Alcatel-Lucent’s OmniSwitch 10K, Arista’s 7504 Series Data Center Switch, BLADE Network Technologies’, an IBM Company, IBM BNT RackSwitch G8124 and IBM BNT RackSwitch G8264, Force10 Network’s S-Series S4810, Hitachi Cable’s Apresia 15000-64XL-PSR, Juniper Network’s EX Series EX8216 Ethernet Switch and Voltaire®’s Vantage™ 6048. We are conducting a second round of test scheduled for the week of April 4-8 at iSimCity, and it is open to all suppliers of 10 and 40 GbE data center switching.

There were three Core/Spine Switches evaluated for performance and power consumption in the Lippis/Ixia test. These participating vendors were:

Alcatel-Lucent OmniSwitch 10K
Arista 7504 Series Data Center Switch
Juniper Network EX Series EX8216 Ethernet Switch

These switches represent the state-of-the-art of computer network hardware and software engineering, and are central to private/public data center cloud computing infrastructure.

If not for this category of Ethernet switching, cloud computing would not exist. The Lippis/Ixia public test was the first evaluation for every Core switch tested. Each supplier’s Core switch was evaluated for its fundamental performance and power consumption features. The Lippis/Ixia test results demonstrate that these new Core switches provide state-of-the-art performance at efficient power consumption levels not seen before. The port density tested for these Core switches ranged from 128 10GbE ports to a high of 256 10GbE.

IT business leaders are responding favorably to Core switches equipped with a value proposition of high performance, high port density, competitive acquisition cost, virtualization aware services, high reliability and low power consumption. These Core switches currently are in high demand with quarterly revenues for mid-size firms in the $20 to $40M plus range. The combined market run rate for both ToR and Core 10GbE switching is measured in the multibillion-dollar range. Further, Core switch price points on a 10GbE per port basis are a low of $1,200 to a high of $6,093.

Their list price varies from $230,000 to $780,000 with an average order usually being in the million plus dollar range. While there is a large difference in list price as well as price per port between vendors, the reason is found in the number of network services supported by the various suppliers and 10GbE port density.

We compare each of the above firms in terms of their ability to forward packets: quickly (i.e., latency), without loss or their throughput at full line rate, when ports are oversubscribed with network traffic by 150%, in IP multicast mode and in cloud simulation. We also measure their power consumption.

Alcatel-Lucent launched its new entry into the enterprise data center market on December 17, 2010, with the OmniSwitch ™ 10K. The OmniSwitch was the most densely-populated device tested with 256 ports of 10GbE. The test numbers below represent the first public performance and power consumption measurements for the OmniSwitch™ 10K running software version 7.1.1.R01.1638. The Alcatel-Lucent OmniSwitch™ 10K Modular Ethernet LAN Chassis is the first of a new generation of network adaptable LAN switches. It exemplifies Alcatel-Lucent’s approach to enabling what it calls Application Fluent Networks, which are designed to deliver a high-quality user experience while optimizing the performance of legacy, real-time and multimedia applications. So how did the OmniSwitch 10K do?

RFC 2544 Layer 2 and 3 Latency Test

The OmniSwitch 10K was tested across all 256 ports of 10GbE. Its average latency ranged from a low of 20,561 ns or 20 μs to a high of 36,823 ns or 36 μs at jumbo size 9216 Byte size frames for layer 2 traffic. Its average delay variation ranged between 5 and 10 ns, providing consistent latency across all packet sizes at full line rate. What this means is that the OmniSwitch 10K can be counted on to forward packets at these latencies without much variation which is extremely important for predictable performance.

For layer 3 traffic, the OmniSwitch 10K’s measured average latency ranged from a low of 20,128 ns or 20μs at 64Bytes to a high of 45,933 ns or 45μs at jumbo size 9216 Byte size frames. Its average delay variation for layer 3 traffic ranged between 4 and 10 ns, providing consistent latency across all packet sizes at full line rate.

RFC 2544 Layer 2 and 3 Throughput Test

The OmniSwitch 10K demonstrated 100% throughput as a percentage of line rate across all 256 10GbE ports. In other words, not a single packet was dropped while the OmniSwitch 10K was presented with enough traffic to populate all of its 256 10GbE ports at line rate simultaneously for both L2 and L3 traffic flows. Not a single packet was dropped while 2.5 Tbps of traffic passed through its line cards and backplane.

RFC 2889 Congestion Test

The OmniSwitch 10K demonstrated nearly 80% of aggregated forwarding rate as percentage of line rate during congestion conditions. A single
10GbE port was flooded at 150% of line rate. The OmniSwitch did not use HOL blocking, which means that as the 10GbE port on the OmniSwitch became congested, it did not impact the performance of other ports. There was no back pressure detected as the Ixia test gear did not receive flow control frames. This was not the same for the Arista 7504. See the full test report here.

RFC 3918 IP Multicast

The OmniSwitch 10K demonstrated 100% aggregated throughput for IP multicast traffic with latencies ranging from 9,596 ns at 64 Byte size packets to 28,059 ns at 9216 Byte size packets. The OmniSwitch 10K demonstrated the lowest multicast latencies of all vendors.

Cloud Simulation Test

The one test that was not RFC based is a cloud simulation that was developed by the Lippis Report and Ixia. This test determines the traffic delivery performance of the DUT (device under test) in forwarding a variety of north-south and east-west traffic in cloud-computing applications. This test measures the throughput, latency, jitter and loss on a per application traffic type basis across M sets of 8-port topologies. The following traffic types are used: web (HTTP), database-server, server-database, iSCSI storage-server, iSCSI server-storage, client-server plus server-client. The north-south client-server traffic simulates Internet browsing; the database traffic simulates server-server lookup and data retrieval, while the storage traffic simulates IP-based storage requests and retrieval. When all traffic is transmitted, the throughput, latency, jitter and loss performance are measured on a per traffic type basis.

The OmniSwitch 10K performed extremely well under cloud simulation conditions by delivering 100% aggregated throughput while processing a large combination of east-west and north-south traffic flows. Zero packet loss was observed as its latency stayed under 28μs.

Power Consumption Test

The OmniSwitch 10K represents a new breed of cloud network spine switches with power efficiency being a core value. The OmniSwitch consumes 13.3 Watts/10GbE port with a TEER (Telecommunications Energy Efficiency Ratio) value of 71. TEER is a measure of network-element efficiency quantifying a network component’s ratio of “work performed” to energy consumed. Larger TEER values are better and the OmniSwitch is second only to Arista in TEER value while Juniper’s EX8216 measured a 44 TEER. You can download the OmniSwitch 10K test report here:

The OmniSwitch 10K power cost per 10GbE is estimated at $16.26 per year. The three-year cost to power the OmniSwitch is estimated at $12,485.46 and represents less than 3% of its list price. Keeping with data center best practices, its cooling fans flow air front to back, which is the norm except for Juniper’s EX8216 which pushes air from side to side unless a third party cabinet from vendors, such as Chatsworth, enclose the EX8216 to support hot-aisle and cold-aisle deployments.

Discussion:

The OmniSwitch™ 10K seeks to improve application performance and user experience with deep packet buffers, lossless virtual output queuing (VOQ) fabric and extensive traffic management capabilities. This architecture proved its value during the RFC2889 layer 2 and layer 3 congestion test with a 78% aggregated forwarding rate when a single 10GbE port was oversubscribed at 150% of line rate. The OmniSwitch™ 10K did not use HOL blocking, back pressure or signal back to the Ixia test equipment with Aggregated Flow Control Frames to slow down traffic flow. Not tested but notable features are its security and high availability design for uninterrupted uptime. The OmniSwitch™ 10K was found to have low power consumption, front-to-back cooling, front-accessible components and a compact form factor. The OmniSwitch™ 10K is designed to meet the requirements for mid- to large-sized enterprises data centers.

To demonstrate how the OmniSwitch™ 10K operates as a lossless fabric plus its ability to deliver carrier class quality of service (QoS), Alcatel-Lucent conducted two separate sets of tests; its data is available here. The lossless fabric test configured 256 x 10GbE ports connected with fully meshed traffic running at wire-speed via Ixia test equipment. The objective of this test was to demonstrate that as fabric and management modules were pulled and inserted into the OmniSwitch™ 10K chassis, zero loss at 100% load would result, and the fabric would be lossless. With fully meshed traffic running through all 256 GbE ports, the following modules were changed.

1. Fabric module was pulled out.
2. Fabric module was inserted back.
3. Management module (a fabric resides on this module) was pulled out.
4. Management module was inserted back.
5. Management module was pulled out causing a management failover in addition to fabric failover.
6. Management module was inserted back.

The result of the lossless fabric was that the fabric was lossless as the above modules were pulled and inserted.

The carrier class QoS objective was to demonstrate no packet loss at wire-speed with P0-P7 (priority) traffic running in fully meshed scenario as in the test above. The carrier class QoS test configured 256 x 10GbE ports connected with fully meshed traffic, priority 0 to 7, running at wire-speed via Ixia test equipment. In this scenario, the OmniSwitch™ 10K delivered zero loss with consistent store-and-forward average latency in range of 132,357 ns to 139,448 ns. See this test report details.

Cloud Network Architecture

There are three approaches to connect spine switches together to create a network fabric. MC-LAG (Multi-Chassis Link Aggregation Group), which allows one or more links to be aggregated together to form a Link Aggregation Group, TRILL and/or SPB are emerging standards that design a solution for shortest path frame routing in multi-hop Ethernet networks with arbitrary topologies, using an existing link-state routing protocol technology.

While there is debate over which approach is best, SPB has the following advantages. SPB deployments are planned for 2011 and offer increased scalability than TRILL. Further, SPB will interoperate with carrier infrastructure to allow private-public or private-private or public-public data center-to-data center connections. For network architects/designers and operations, there is a quick learning curve as SPB uses the existing IS-IS protocol, and for service providers, SPB is already available through OAM (Operations, Administration and Maintenance), enabling it to be managed through existing management services.

Paramount in the two-tier leaf-spine architecture is high-spine switch performance, which collapses the aggregation layer in the traditional three-tier network connecting spine switches together. The above captures the major trends and demands that IT business leaders are requiring from the networking industry. The underpinnings of private and public data center cloud network fabric are 10GbE switching with 40GbE and 100GbE ports/modules. 40GbE and 100GbE are in limited availability now but will be increasingly offered and adopted during 2011. Network performance including throughput performance and latency are fundamental switch attributes to understand and review across suppliers, because if the 10GbE switches an IT leader selects cannot scale performance to support increasing traffic volume plus shifts in traffic profile, not only will the network fail to be a fabric unable to support converge storage traffic, but business processes, application performance and user experience will suffer too.

During 2011, an increasing number of servers will be equipped with 10GbE LAN on Motherboard (LOM) driving 10GbE network requirements, and in 2012, high-end servers will be equipped with 40GbE LOM starting 40GbE’s growth curve. In addition, with nearly 80% of IT spend being consumed in data center infrastructure with all IT assets eventually running over 10GbE switching, the stakes could not be higher to select the right product upon which to build this fundamental corporate asset. Further, data center network equipment has the longest life span of all IT equipment; therefore, networking is a long-term investment and vendor commitment.

We review the Alcatel-Lucent OmniSwitch 10K from a perspective of performance and power measurement, mesh protocol support and key product features. Alcatel-Lucent has entered the data center switching market with a very competitive Core/spine switch. Clearly there are differences between Core switch vendors, and it’s advised to conduct a detailed review. For starters Click here for a copy of Alcatel-Lucent’s OmniSwitch 10K plus cross-vendor test results report.

Tweet

During December 6-10, 2010, the Lippis Report and Ixia conducted the industry’s first 10GbE data center switching evaluation of Top-of-Rack and Core Ethernet switches at the modern iSimCity lab in Santa Clara, CA. We evaluated Alcatel-Lucent’s OmniSwitch 10K, Arista’s 7504 Series Data Center Switch, BLADE Network Technologies’, an IBM Company, IBM BNT RackSwitch G8124 and IBM BNT RackSwitch G8264, Force10 Network’s S-Series S4810, Hitachi Cable’s Apresia 15000-64XL-PSR, Juniper Network’s EX Series EX8216 Ethernet Switch and Voltaire®’s Vantage™ 6048. We are conducting a second round of test scheduled for the week of April 4-8 at iSimCity, and it is open to all suppliers of 10GbE data center switching. We learned a lot about these products, both in the lab and out. In this Lippis Report Research Note, we dive into the Top-of-Rack 10GbE switches we tested as they represent a new generation of products that exhibit low power consumption, low latency, high performance and are all based upon new single chip designs from Broadcom, Marvell or Fulcrum Micro.

The Top-of-Rack (ToR) switches tested at iSimCity were the:

BLADE Network Technologies, an IBM Company, IBM BNT RackSwitch G8124 and IBM BNT RackSwitch G8264;
Force10 Network’s S-Series S4810;
Hitachi Cable’s Apresia 15000-64XL-PSR;
Voltaire®’s Vantage™ 6048.

All of these ToR switches utilize a new single chip design, but mostly from different silicon suppliers. With a single chip provided by chip manufacturers Broadcom, Marvell or Fulcrum Micro, vendors are free to invest resources other than ASIC development, which can consume much of a company’s engineering and financial resources. With merchant silicon providing a forwarding engine for their switches, these vendors are free to choose where to innovate, be it in buffer architecture, network services such as virtualization support, 40GbE uplink or fan-in support, etc.

The Lippis/Ixia test results demonstrate that these new chip designs provide state-of-the-art performance at efficient power consumption levels not seen before. In addition, price points on a 10GbE per port basis for ToR switches are a low of $351 to a high of $520.

IT business leaders are responding favorably to ToR switches equipped with a value proposition of high performance, low acquisition price and low power consumption. These ToR switches currently are the hot boxes in the industry, with quarterly revenues for mid-size firms in the $10 to $15M plus. We compared each of the above firms in terms of their ability to forward packets: quickly (i.e., latency) without loss or their throughput at full line rate, when ports are oversubscribed with network traffic by 150 percent, in IP multicast mode and in cloud simulation. We also measured their power consumption. Click Here for a copy of BLADE’s G8124 and G8264 plus cross-vendor test results report and Click Here for a copy of Force10’s S4810 plus cross-vendor specific report.

Latency Measurement Anomalies

When evaluating five products from four companies, there are bound to be anomalies. One anomaly was found during latency measurement. As both BLADE and Force 10 use the same Broadcom chip in their G8264 and S4810 ToR switches, respectively, one would expect their latency measurements would be close, but the S4810 showed lower latency values. As it turns out, the Broadcom chip allows switches to forward in cut-through and/or store-and-forward mode. The G8264 was configured and tested in cut-through mode while the S4810 and all other switches were configured and tested in store-and-forward. Test equipment, such as Ixia and others, measure latency very differently in these two forwarding modes.

During store-and-forward testing, test equipment subtract packet transmission latency, decreasing actual latency measurements by the time it takes to transmit a packet from input to output port. This makes comparisons between the two-latency measurement testing methodologies difficult. Also other potential device specific factors can impact latency too. But looking at the bigger picture, latency is being measured in the hundreds to thousands of nanoseconds across various packet sizes, making these switches the fastest forwarding engines in the market.

One of the biggest surprises was Voltaire’s Vantage 6048 ToR latency results, which were the highest of the group by nearly a factor of 2. Voltaire, now owned by Mellanox, used the Marvell 10GbE single chip code named Lion. The Hitachi Apresia 15000-64XL-PSR showed low latency results but it had other difficulties. For example, the largest frame size supported is 9044, excluding it from the 9216 byte size packet tests. Further, there is no latency data for the Apresia 15000-64XL-PSR at 64 bytes due to configuration difficulties during testing. The 15000-64XL-PSR could not be configured to maintain a VLAN at 64 bytes which eliminated packet signature to measure latency at this packet size.

A big surprise and delight found was how low the average delay variation was for all suppliers. Average delay variation was in the 5 to 10ns range, meaning that all of the above ToR switches deliver their latency results reliably.

Throughput

The results of RFC 2544 throughput testing should be boring with all ToR switches showing 100% throughput at line rate. The only anomaly here was the Apresia 15000-64XL-PSR during layer 2 forwarding, dropping packets at between 128 to 2176 packet sizes.

Congestion Testing

RFC 2889 congestion testing was telling too. Here the IBM BNT RackSwitch G8124 and IBM BNT RackSwitch G8264, Force10 Network’s S-Series S4810 and Voltaire®’s Vantage™ 6048 performed as expected, that is, offering 100% line rate under congestion conditions without head of line blocking and using back pressure or pause messages to control the flow of traffic. Here again, Hitachi Cable’s Apresia 15000-64XL-PSR showed head of line blocking and low throughput especially at the higher packet sizes of 2176 bytes.

IP Multicast

For RFC 3918 IP Multicast Throughput No Drop Rate testing, the IBM BNT RackSwitch G8124 and IBM BNT RackSwitch G8264 and Force10 Network’s S-Series S4810 performed flawlessly, exhibiting 100% line rate throughput and nanosecond latency with the G8124’s average latency 700ns and below. The IBM BNT RackSwitch G8264 and Force10 Network’s S-Series S4810 IP multicast performed as expected as they are both based upon the same Broadcom chip. The G8264 demonstrated a slight advantage of 100ns at the higher packet sizes while Force10 showed approximately 100ns advantage at the lower packet sizes. The Apresia 15000-64XL-PSR and Vantage 6048 do not support IP Multicast at this time.

Cloud Simulation

The one test that was not RFC based is a cloud simulation that was developed by the Lippis Report and Ixia. This test determines the traffic delivery performance of the DUT (device under test) in forwarding a variety of north-south and east-west traffic in cloud-computing applications. This test measures the throughput, latency, jitter and loss on a per application traffic type basis across M sets of 8-port topologies. The following traffic types are used: web (HTTP), database-server, server-database, iSCSI storage-server, iSCSI server-storage, client-server plus server-client. The north-south client-server traffic simulates Internet browsing, the database traffic simulates server-server lookup and data retrieval, while the storage traffic simulates IP-based storage requests and retrieval. When all traffic is transmitted, the throughput, latency, jitter and loss performance are measured on a per traffic type basis.

This test is telling too as it’s designed to be a simulation of real-world cloud-computing traffic. The results here show that the IBM BNT RackSwitch G8124 and G8264 delivered the lowest latency consistently across all protocol types. The Apresia 15000-64XL-PSR performed very well in this test too, followed by Force10’s S4810 followed by Voltaire’s Vantage 6048. Anomalistically, both the Force10 S4810 and Vantage 6048 spiked in terms of latency for east-west database-server, HTTP and iSCSI-Storage traffic flows. Both IBM BNT RackSwitches and Force10’s S4810 tested in cut-through mode.

Power Consumption

Power consumption or energy efficiency has become a paramount concern in data centers as the cost of power and cooling start to dominate TCO (total cost of ownership) over a three-year period. The ToR switches tested offer the lowest power consumption of switching products evaluated in public industry test. Their power consumption measured in WATTS per 10GbE via ATIS methodology ranged from 3.6 to 5.5. We then projected annual cost per 10GbE to be between $4.36 to $6.70, with the Apresia 15000-64XL-PSR offering the lowest power consumption. The IBM BNT RackSwitch G8264 and Force10’s S4810 were very close at $4.78 and $4.91, respectively, with the G8264 having a slight advantage. Of the 48-port 10GbE ToR switches, Voltaire’s Vantage 6048 consumed the most energy at 5.5Watts/10GbE.

While not confirmed, the IBM BNT RackSwitch G8124 may be based upon the Fulcrum single chip set, code named Bali, as well as Arista’s 7124 and Force10’s S2410. The Apresia 15000-64XL-PSR may be based upon the Broadcom Trident single chip. There are rumors in the industry too that large networking firms may start to utilize merchant silicon rather than build their own, as these chips offer a quicker path to market and are delivering solid performance, latency and power efficiency results.

While I detail ten recommendations in the test report, here I’ll focus on one. 10GbE ToR switches are ready for mass deployment, delivering full line rate
throughput at zero packet loss and nanosecond latency plus single- to double-digit delay variation. In addition, these ToR switches offer low power consumption with energy cost over a three-year period estimated between 3 and 4% of acquisition cost. Clearly there are differences between vendors, and it’s advised to conduct a detailed review. For starters Click here for a copy of BLADE’s G8124 and G8264 plus cross-vendor test results report and Click Here for a copy of Force10’s S4810 plus cross-vendor specific report.

Tweet

There are powerful market forces changing IT delivery. IT application delivery is becoming increasingly centralized thanks to data center server virtualization plus mobile and cloud computing. Desktops are being virtualized, too, thanks to network speeds that deliver low latency and high bandwidth, creating a thin client user experience that is indistinguishable from a thick client but at lower desktop management cost. One serious implication of this concentration of IT in data centers is that a new IT security model is needed as mobility brings greater threat exposure while virtualization changes traffic patterns and the rules of security appliance placement. In this Lippis Report Research Note, we present a new model for IT security in the virtualized mobile and cloud-computing era.

Users are demanding IT support commercial mobile computing platforms in the enterprise market, driving nearly exponential growth of these devices within corporations. And while commercial mobile computing use, that is Apple’s iPhone/iPad and Android smartphones and tablets, rises, it’s pushing applications, data and IT critical resources into private and public data center cloud facilities. In short, IT is shifting toward both mobile and cloud computing simultaneously, as the two are inextricably linked. Factor in the need for geographically and time independent access to IT services on any end point device, and you have the making of a major shift of centralizing application delivery to geographically dispersed end points that can scale globally.

This pull to centralize IT applications is driven by technology innovation of mobile and cloud computing with financial and performance gains afforded virtualization. But while there are material business benefits to this IT transition, there are risks too. Threats continue to increase, especially as mobile computing expands the diameter of access to data center resources. Virtualization provides huge efficiency benefits but changes the way in which security devices, such as firewalls, need to work to secure applications.

For example, traditional network services are frequently placed in-line or in the flow of traffic, that is firewall, IPS, VPN tunneling etc., forming a line of layer 4-7 network services. But as applications are virtualized, their movement may take them out of the path of traffic flow, thus creating difficulty to maintain network services to Virtual Machines (VMs) and their applications. In most data centers, a mix of physical and virtual network services is emerging as well as a mix of virtual servers and physical servers based upon old and new investment. What IT business leaders demand is that their investment in physical and/or virtual network services support both virtualized and non-virtualized applications, so they may extract the highest value from their IT dollars and that the same level of security services are applied to both virtualized and non-virtualized applications. This is a hard problem to solve and requires new thinking in network security.

The New Approach to Network Security

Before we dive into security architecture, a new approach to network security thinking is in order. Traditionally, network security was based upon the hard-shell and soft-core concept; that being, build a perimeter of firewalls and IPS equipment creating a hard shell around IT assets, but keep the internal network free of security services—that is a soft core. Then security layering was added to this model by offering defenses in depth to harden the soft core. While these approaches are still valid, thinking needs to be expanded in step with the directions of IT.

Modern day network security architecture needs to defend, extend, prevent and comply. By defend, we mean mitigate threats as the number of exploits/malware, etc., continue to rise. Network security services need to be extended to support virtualized data centers as well as mobile users and cloud-computing facilities. Network services need to prevent business loss, be it data loss prevention and business continuity. And lastly network security needs to assure compliance of government legislation/regulation/orders to mitigate risks of non-compliance.

Applying this new thinking in network security to major user behavior scenarios and IT assets creates both a broad security blanket that is also deep. For example, systemic across the enterprise, progressive IT business leaders are developing cloud security, desktop virtualization security and, for those engaged in on-line transactions, a PCI solution. These three security services support IT assets in need of protections, such as application security, mobile user experience security, virtualization security, service security such as encryption plus infrastructure security, e.g., firewall, IPS, VPN.

Cisco’s Data Center Virtualization Security Approach

There are only a few IT firms that can deliver the depth and breadth of this type of a security approach. These firms are Cisco, IBM, HP, Microsoft, Oracle and perhaps CA. For this Research Note, we focus on Cisco as it possesses all the technologies to deliver on a broad data center virtualization security solution. In the above example, Cisco’s ScanSafe would provide email and web application security. Its AnyConnect mobile client provides mobile security for VPN and cloud access. Service security is delivered via TrustSec, an architecture providing policy, identify and encryption services. For infrastructure security, its ASA (or Adaptive Security Appliance) security product combines firewall, IPS and VPN, while infrastructure security services are also embedded in its switch and router product lines. While all of the above products have been in production for some time, Cisco has launched an innovative approach to solving one of the biggest virtualization security problems, and that is to virtualize firewall services and to steer traffic to it as application flow changes from in-line to off-line as occurs when applications become virtualized.

Virtual Security Gateway

Within Cisco’s Unified Network Services (UNS) umbrella of products, it has launched its data center firewall called VSG or Virtual Security Gateway, and provided it management and policy services via its VNMC or Virtualized Network Management Center software. VSG is an example of a virtual service node, as compared to physical ASA security appliance. The key underpinning technology to VSG is the Nexus 1000V and vPATH, which enable traffic to be re-routed or steered to the virtual firewall nodes…more on this below.

VSG is a proof-point of Cisco’s ability to solve the firewall problem within virtualized infrastructure; that is how to provide firewall services to flows destined to and between various VMs. vPATH, a software module within the Nexus 1000V softswitch, steers traffic to VSG, which blocks or allows traffic flow to its destination. Further, VSG assures that the correct network security service is applied, and a VM’s policies follow it as it moves between physical servers. VSG policy is centrally managed through the VNMC umbrella management platform.

By inserting vPATH technology/software into the Nexus 1000V virtual switch, hypervisors and VM’s traffic is re-directed as needed to deliver network services, such as firewall.

vPATH

In the case of VSG, through VNMC, policy is created to define what type of traffic needs to be redirected, and then what action to take upon that traffic once it arrives at the firewall. As traffic reaches a server or Nexus 1000V, it is intercepted as it’s destined for a particular VM by vPATH, which redirects it to VSG for inspection. VSG then performs its network security service, then forwards the traffic, if allowed, to its destination just like a firewall appliance operates. vPATH intercepts traffic and sends it to VSG while VSG performs its security service and decides if traffic will be forwarded to the destination VM.

Fast Path

vPATH also benefits from a concept called fast path. Fast path is similar to a cut-through method in that once traffic has been forwarded to VSG for firewall services, for example, the remaining traffic flow, it’s routed directly to its VM destination. Note that fast path can be utilized for most network services. Fast path obviates the need to route all traffic through VSG once the first packet of the flow has been processed by the firewall. Therefore, all traffic does not require packet-by-packet inspection, speeding up flows and reducing processing and latency.

For example, if the first packet of a flow passes through VSG without alteration then the rest of the flow should pass uninspected as the security rules are the same. However, this wouldn’t be the case for an IPS system, where the entire payload is inspected to assure there is no malware residing in the flow.

A key benefit of vPath is that it intelligently steers traffic via flow classification and redirection to associated VSGs to implement security policies in a virtual environment. Fast path offload: Policy enforcement of flows are offloaded by VSG to vPath thanks to Fast path and deliver improved efficiency and performance of firewall services to virtualized applications. These capabilities, along with physical firewalls, help IT leaders to regulate how virtualized and non-virtualized applications receive firewall services. In addition, as VMs move between physical servers, firewall settings do not need to change as they follow the VM move within the data center. Thus VSG is mobility aware and is VLANs and topology agnostic enabling flexibility not seen before in virtualized data center environments.

Going back to the need for a modern approach to network security, the combination of Cisco’s ASA, VSG, AnyConnect and Security Intelligence Operations or SIO start to deliver the attributes of defend, extend, prevent and comply to IT business leaders concerned with protecting modern IT business assets. For example, AnyConnect 3.0 provides security services for remote and mobile end points via client software on laptops, tablets and smartphones with centralized policy control. In short, AnyConnect provides protections against the increased network diameter afforded by mobile and cloud computing. SIO is one of the most comprehensive and globally expansive threat detection services that update Cisco IPSs with exploit signatures in near real time, thanks to its global threat correlation service. SIO is based upon over 1 million sensors (Cisco IPS) distributed around the globe from which it sends and receives updates and is staffed with over 500 security experts.

So as servers and applications are virtualized and computing goes mobile and to the cloud, a new modern approach to network security is taking hold. With Cisco, its network security architecture and products of ASA, VSG, AnyConnect and SIO span the new nature of borderless IT to offer business leaders protections as they manage their business and exploit the value created by this new cycle in Information Technology.

Tweet

Any IT business leader knows that the single most important technology driving data center design change is server virtualization to the point that a virtual machine (VM) is now the data center building block. As server virtualization marches on until nearly every physical server has been virtualized, networking in a virtualized environment is being forced to fundamentally change too. By networking, I mean not only layer 2 and 3 forwarding but network services too, such as application controllers, WAN optimizes, firewalls, etc., which are fundamental for mission critical application performance, cost reduction and high application availability especially where service level agreements are required.

Adding new applications to a data center has become highly complex, thanks to all the routing paths that need to be set-up to provide connectivity and reach of network services plus the configuration and policy set-up for network services specific to the application. Then, once the application is operational, it’s hard to virtualize it and move it via v-motion, et al, while keeping set-up and policies intact, especially routing paths. The current state of rigid networking consumes time and cost, but most importantly limiting the speed and agility in which new applications can be delivered and businesses react to market dynamics. This is a nasty problem, riddled with complexity and associated cross-administrative operational cost limiting the number of applications that can be virtualized until this problem is solved.

An entirely new approach to deploying, provisioning and managing data center network services in a virtualized environment is needed, and Cisco is addressing this need with its Unified Network Services or UNS. Cisco’s UNS is not just a suite of its layer 4-7 network service offerings such as ACE, WAAS, etc., but a framework for transparently inserting network services into a virtual server environment for steering traffic to network services on a per-VM basis plus an extensible and integrated policy management architecture. The key word in UNS is “unified,” as UNS makes network services available to both physical and virtual servers and their associated applications via steering traffic to network services hosted in appliances/modules/blades or within a VM. UNS promises to help reduce the costs to deploy new applications plus to enable more applications to be virtualized. In short, UNS offers an approach to deploy, provision and manage new applications without the network set-up complexity mentioned above. In addition, it also promises to remove network complexity associated with virtualizing applications and their moves. UNS is a main pillar of Cisco’s Data Center Business Advantage architecture, along with Cisco’s Unified Fabric and Unified Computing Services. These pillars combine to form the tightly-integrated next generation data center components including the network, storage, application services, virtualization layers and network services.

Cisco’s UNS is addressing mobile (v-motion) applications and their associated changing or dynamic network topology requirements by steering traffic to appropriate network services that are centrally controlled via policy. These network services such as firewalls, application controllers, WAN acceleration, load balancing, etc., can be packaged in appliances, modules, server blades and/or other form factors and/or increasingly as a virtualized service. UNS is a modern approach to applying layer 4-7 network services to both non-virtualized applications and VMs, while in the process solving some of the most complex problems associated with virtualized infrastructure.

Dedicated Hardware Services to Virtualized Network Services

Traditional network services are frequently placed in-line or in the flow of traffic, that is firewall, IPS, load balancing, application controllers, WAN acceleration, etc., forming a line of layer 4-7 network services. But as applications are virtualized, their movement may take them out of the path of traffic flow, thus creating difficulty to maintain network services to VMs and their applications. In most data centers, a mix of physical and virtual network services is emerging as well as a mix of virtual servers and physical servers based upon old and new investment. What IT business leaders demand is that their investment in physical and/or virtual network services support both virtualized and non-virtualized applications so they may extract the highest value from their IT dollars. This is a hard problem to solve and requires new thinking in networking which is what UNS is focused upon delivering. In short, UNS allows a mix and matching of physical and virtual network services to support either virtualized or non-virtualized applications through a more flexible approach to networking and policy management. So how do IT architects create this level of flexibility?

In a UNS environment, the physical placement of network services in appliance/modules/server blades, etc., or virtualized form is moot, offering IT architects a new degree of freedom to access these services anywhere in a virtualized infrastructure. A network service can be offered to a VM and its associated traffic, independent upon its form factor, be it a physical appliance, dedicated module or virtualized network service as long as the VM and softswitch send traffic to the appropriate service as the application moves around the data center.

That’s important as traffic patterns have shifted from primarily north-south to a mix of east-west and north-south, resulting in the need for network services to offer far greater flexibility in their reach to service VMs and the applications they contain. And as network services are logically wrapped around a VM via policy, they receive the benefit of all moving together, solving one of the biggest virtualization problems in the industry, manually intensive change management. Parallel to making network services accessible independent upon location and its packaging is the added benefit of virtualizing network services as this will decrease the number of hardware appliances in a data center, reducing complexity, total cost of ownership and energy consumption.

Unified Network Services Is a Platform for Inter-Cloud Mobility and On-Demand Provisioning

But perhaps even more important than solving the immediate change management problem is that unified network services deliver a set of attributes that put in place the tools and ability to deliver elastic IT services between clouds—the holy grail of cloud computing. With core network services unified, a degree of flexibility is gained far beyond current technology and offers a platform in which service advertising and registry can occur so that a “provision proxy” can automate network service configuration to meet new IT service delivery needs in near real time; but this is a topic for another day. The important point is that a unified network service is a platform that all large IT firms, cloud providers and enterprises will be investing in over the next business cycle.

Cisco’s Unified Network Services or UNS

In this Research Note, we review Cisco’s UNS, the most comprehensive approach to data center and cloud network service deployments in the industry thus far. UNS addresses the on-demand provisioning problem so sought after in virtualized infrastructure. That is when IT leaders need to allocate resources from within or between a private or public cloud on demand and quickly, UNS will respond to a capacity request so that network services are provisioned in the right order, at the right capabilities and within minutes rather than months. In short, UNS’s vision is to enable on-demand network service delivery and on-demand provisioning to accommodate VM container workload mobility within the construct of an Enterprise’s IT model or service architecture.

The Virtual Security Gateway

UNS is both a vision of on-demand service provisioning and the products that enable its construct. Within UNS, Cisco has launched its data center firewall called VSG or Virtual Security Gateway, and is on a path of virtualizing its data center service products including the Wide Area Application Services or WAAS, et al, and providing them with consistent policies via its VNMC or Virtualized Network Management Control software. VSG is an example of a virtual service node, as compared to physical ASA security appliances. The key underpinning technology to VSG is the Nexus 1000v and vPATH, which enable traffic to be re-routed or steered to the virtual firewall nodes; more on this below.

Cisco’s VSG offers a model of how network services are virtualized and in the process, solves some of the biggest server virtualization problems while delivering added flexibility value. VSG is a proof-point of Cisco’s ability to solve the firewall problem within virtualized infrastructure; that is how to provide firewall services to flows destined to and between various VMs. vPATH, a software module within the Nexus 1000v softswitch, steers traffic to VSG, the firewall, which blocks or allows traffic flow to its destination. Further, VSG assures that the correct network security service is applied and a VM’s policies follow it as it moves between physical servers. VSG policy is centrally managed through the VNMC umbrella management platform.

Central to UNS is vPATH technology that confers the same VSG benefits discussed above to Cisco’s new Virtual WAAS or vWAAS WAN acceleration offering. vPATH is fundamental to UNS as it delivers unification by being the same underlying infrastructure for both VSG and vWAAS. Therefore, by inserting vPATH technology/software into the virtual switch, hypervisors and VM’s traffic is re-directed as needed to deliver network services, such as firewall, WAN acceleration, etc.

vPATH

In the case of VSG, through VNMC, policy is created to define what type of traffic needs to be redirected, and then what action to take upon that traffic once it arrives at the firewall. As traffic reaches a server or Nexus 1000v, it is intercepted as it’s destined for a particular VM by vPATH, which redirects it to VSG for inspection. VSG then performs its network security service then forwards the traffic, if allowed, to its destination just like a firewall appliance operates.

The closest analogy to describe vPATH’s function is network-based application recognition. That is NBAR analyzes traffic and classifies it, and then performs a function such as prioritization. Thus, vPATH intercepts traffic and sends it to VSG while VSG performs its security service and decides if traffic will be forwarded to the destination VM.

Fast Path

vPATH also benefits from a concept called fast path. Fast path is similar to a cut-through method in that once traffic has been forwarded to VSG for firewall services, for example, the remaining traffic flow, it’s routed directly to its VM destination. Note that fast path can be utilized for most network services. Fast path obviates the need to route all traffic through VSG once the first packet of the flow has been processed by the firewall. Therefore, all traffic does not require packet-by-packet inspection, speeding up flows and reducing processing and latency.

For example, if the first packet of a flow passes through VSG without alteration, then the rest of the flow should pass uninspected as the security rules are the same. However, this wouldn’t be the case for an IPS system, where the entire payload is inspected to assure there is no malware residing in the flow. Fast path will evolve to support various traffic scenarios too.

Network Service Chaining

Cisco’s UNS provides a solution to the challenge of providing network services to traffic flows within a virtualized infrastructure that stick to VMs as they move and change physical location in the data center. The next challenge is to provide virtualized network service chaining. Chaining network services is the ability to create a single policy for traffic flows as it ingresses to a VM for multiple network services. For example, a policy may apply firewall, load-balancing, WAN-optimization, etc., to a flow and route that traffic through subsequent services, as opposed to having to create unique policies, intercept each one and route traffic accordingly. Chaining is a huge operational time saver, and it hastens the flow of traffic within the data center. vPATH is one underlying mechanism that can steer traffic to services in the right chain/order.

The UNS Value Proposition

From a data center network design perspective, UNS is developing a set of network service building blocks that brings physical network service appliances and virtual service nodes into virtualized environments along with the tools to apply policies to govern their use. As more and more data centers become virtualized so too will network services. In addition, as physical and virtual data centers will co-exist for many years to come, the ability to offload physical network appliances with virtualized ones as well as pass traffic between them offers a transition path and a means to extend the life of existing appliance investments.

As mentioned above, physical data centers are equipped with stacks of appliances offering load balancing, WAN acceleration, firewalls, IPS, etc. Now with service chaining and vPATH, all of these physical and virtualized appliances can be put to work servicing VMs and their applications. Most importantly though is that UNS offers a way to control network services so that VMs, virtual applications and mobile workloads can be scaled up and down plus moved within a dynamic network that allows provisioning services easily. For all intents and purposes, the industry has not had a multi-service chaining mechanism in the physical world. IT operations have done this manually via provisioning VLANs, policy routing, Web Cache Communications Protocol or WCCP, etc. But the old approach is static, and when servers, applications, appliances, etc., move or change, manual intervention is required. The beauty is that chaining network services in a virtualized infrastructure enables elastic scale-up and scale-down much more seamlessly.

Why Unify Network Services

One of the key strategic elements behind UNS is to change the mindset in which IT leaders deploy network services. Traditionally network service appliances were deployed at the edge of the data center or in front of a specific application server. But servers and application are often moved creating the manual re-configuration problem discussed above. Having common accessible network services in private and public data center clouds could offer huge provisioning benefits. For example, there could be, potentially, a vWAAS instantiation in Amazon EC2, Rackspace, GoGrid, etc, which IT leaders who have deployed WAAS in their branch offices could leverage, meaning their WAN would be accelerated thanks to a common WAAS image in the branch and cloud providing that network service independent upon these two application deployment models. This new network services deployment model attempts to blend the worlds of Cisco’s borderless and data center initiatives to the fullest extent.

What’s the intrinsic value of making a network service virtualization? In the case of vWAAS, Cisco is able to give IT leaders flexibility of placement and IT delivery. vWAAS is easier to scale up, licensed in a “pay as you grow” model, offers fewer devices to manage with less power and cooling cost plus is overall more flexible in its placement. In addition, vWAAS and WAAS can both offer WAN acceleration services to virtualized applications thanks to vPATH increasing the usefulness and value to both. vWAAS may be deployed by cloud providers too, which could offer IT leaders a WAN acceleration option independent upon application hosting.

Distributed Deployment with Centralized Management

Value is gained by being able to deploy network services in a distributed fashion, thanks to UNS. UNS changes network service deployment from a centralized model to distributed. But while virtualized network services are distributed, its management is centralized, offering operational efficiency and deployment flexibility. Distributed network service deployment with centralized management is the only approach that works as virtualized network services tend to be distributed widely. In fact, large data centers and clouds will see their instantiations of a particular service grown from a few hundred to thousands, if not more. Therefore, centralized management of virtualized network services provide the control knobs to provision, develop policy, steer traffic, etc., for thousands of virtualized network services distributed throughout a virtualized infrastructure. For example, in Cisco’s UNS, vWAAS and VSG run in their own VM, either on a single physical server or multiple physical servers, offering a highly distributed network service option.

Other companies, such as A10 and at least five others, are virtualizing their application delivery offering too. And cloud service providers are seeking virtualized network services, which will offer IT business leaders the ability to deploy applications from either private or public clouds with a common set of network services over time. For example, many public cloud providers would like to place load-balancing services on top-of-rack and deploy it in a small-medium-large type format. Further, many would also like to place load-balancing services on a compute platform to give customers the ability to deploy load-balancing pseudo-traditionally. That is to deploy network services where a compute platform would be largely dedicated to that service, or, alternatively, distributed so that it does not necessarily reside top-of-rack, or centralized, but resides “logically” next to a VM or sets of VMs so that as VMs move the network service benefit followings.

UNS: A Product Set or Next Evolution of Networking and Computer Services

Now Cisco isn’t the only IT firm developing a unified network service framework, but it is the only company that has all the components to deliver a comprehensive and thoughtful solution. For example, HP, IBM and Oracle do not develop load balancing, application delivery, WAN acceleration or softswitch network services, placing them at a disadvantage. Oracle, HP and IBM usually partner with others for these services such as F5, Riverbed, VMWare, etc., eliminating the opportunity for this level of virtualization and unification development. In HP’s case, its networking gear is increasingly made in China which lacks the forward-looking foresight to get in front of this opportunity. IBM usually does a really good job here, but it’s limited on these major network service components.

Many of the niche players, such as F5, Riverbed, Infoblox, A10, et al, will and are virtualizing their network service appliances and will do it very well, emerging as feature functional leaders. But these firms’ virtualization strategies will lack the broad view of multiple network services and most importantly, how the network nodes (L2-3 infrastructure) or hypervisor can steer traffic to them. To gain a broader UNS view and solution, these firms could organize a consortium to develop a comprehensive UNS strategy and implementation that matches Cisco’s UNS. But consortium is driven by committee, which usually moves slowly. Cisco’s UNS framework will be emulated by others while key technology layers can be standardized, such as Cisco’s proposed VN-Link for traffic steering to physical devices from a virtual/softswitch. Hopefully, an ecosystem can be created that allows all vendors to participate, because UNS is not just another vision and product line, but it’s the next evolution of networking and computing services.

Tweet

In an effort to offer a multi-vendor SIEM (Security Information and Event Management) solution, Cisco is placing its SIEM product, CS-MARS, in end-of-life and in its place, offering the industry its first SIEM ecosystem. Cisco acquired MARS six years ago in December 2004. MARS provided traditional event management and security monitoring along with limited forensic capabilities and compliance reporting. But the market demanded a broader cross-vendor SIEM solution rather than a SIEM focused primarily on Cisco products. In response Cisco has launched a SIEM ecosystem to support deep event monitoring, forensics and compliance reporting across a heterogeneous enterprise network. IT has also expanded the role of its Cisco Security Manager or CSM to support policy management and troubleshooting across a wider range of Cisco products. In this Lippis Report Research Note, we examine the new distribution of security responsibilities that now stretch across Cisco CSM and its new SIEM ecosystem with an eye toward stronger defense of IT assets.

IT business leaders were requesting Cisco develop deeper forensics and compliance across multiple areas within MARS. But the MARS architecture was not designed for such long-term storage, long-term data indexing and look-ups required for conducting forensics and compliance in a manner that IT business leaders are demanding. So in June of 2010, Cisco launched a SIEM ecosystem to provide a scalable and cross-vendor approach for IT business leaders to conduct deep forensics and compliance capabilities. Real-time security monitoring capabilities, which MARS provided, are being blended into the CSM.

CSM started as a policy manager for multiple Cisco devices such as routers, switches, firewalls, VPN, IPS, etc. But Cisco recently announced its 4.1 image for CSM that incorporates security-monitoring capabilities that enable policy troubleshooting. For example, essentially event logs will flow into CSM. CSM will determine if a stream of event logs rise to the level of a security problem or if it needs to make policy changes and execute those changes in real time via a closed-loop system. CSM does not deliver forensics or long-term compliance reporting. This is province of the Cisco SIEM ecosystem.

The SIEM Ecosystem

Both MARS and CSM have been missing the capability to conduct broad multi-vendor security monitoring, compliance reporting and forensics in a heterogeneous vendor environment. In fact, most, if not all, security vendors are guilty of this. Clearly market reality dictates that most enterprise IT organizations utilize multiple devices and/or software that contribute to IT security defense.

Therefore, to align its security products and IT defense approach with the reality of the market, Cisco has started a SIEM ecosystem consisting of the five largest SIEM suppliers. The five vendors in the ecosystem are RSA, ArcSight, LogLogic, Splunk and netForensics. Cisco’s exit of the SIEM market has created the opportunity for it to partner with these top SIEM providers covering 75% +/- of the enterprise market.

The power of a SIEM is to accept logs from multiple devices and make sense of them, meaning it weaves them together by way of correlation. The larger the number of log streams to a SIEM from various security appliances, the greater its ability to correlate. The goal of a SIEM is to gather data from all deployed security appliances, which ends up delivering an exponential lift with respect to the security intelligence gain obtained from correlating large streams of data.

With the Cisco SIEM ecosystem, Cisco is now able to deliver heterogeneous capabilities that cover security monitoring analysis, compliance and forensics capabilities, and some specifically, LogLogic, deliver long-term log management capabilities. To assure confidence that Cisco security and networking equipment interoperate with these five SIEM suppliers, Cisco has conducted extensive interoperability testing with each supplier. This is key for IT business leaders who have an operational SIEM deployed need to be assured that either the introduction of a new SIEM or security device will interoperate with their existing SIEM. This is key for Cisco CS-MARS customers who will be looking to transition to a new SIEM. Note that end-of-life is a multi-year process so co-existence and transition are important attributes for the ecosystem to contain.

Conduit between SIEM and Cisco Security Products

The interface or conduit that enables information transfer between Cisco products and its SIEM partners is device specific. The interface could be SysLog, SDEE or Security Device Event Exchange, and depends upon what conduit the end security device uses, be it an IPS, firewall, switch, router, etc. The conduits have not evolved yet, although at some point in time, they may.

The Interoperability, Validation and Testing Lab

To demonstrate Cisco interoperability, Cisco has created a Cisco-compatible logo, which a partner earns after they have passed through what is called the “IVT Lab” meaning Interoperability, Validation and Testing Lab. One of the key outputs of the IVT Lab is interoperability assurance plus license rights to display the Cisco-compatible logo, and a set of deployment guides to assist a Systems Engineer (SE) or an IT security department to deploy a partner’s SIEM product alongside Cisco’s firewalls, switches, routers or email plus web security products, etc. The detailed deployment guides offer various configurations of the SIEM ecosystem partners and Cisco products.

To gain the Cisco-compatible logo, a partner needs to be tested against Cisco security products, which are approximately eight devices in its latest software versions. These include Cisco Cross-Device, Firewall, IPS, ASA, E-mail Security Appliance (ESA), Web Security Appliance (WSA), etc. The Cisco-compatible logo says that each partner has been tested for that set of core security devices. Over time Cisco plans to test SIEMs across the entire Cisco security product line.

The IVT Lab and associated Cisco-compatible logo essentially level-sets SIEM partners so all have validated and verified support for core Cisco security products. From a support perspective, Cisco’s TAC can take the lead on support. Cisco has developed relationships with its ecosystem partners by tying them into its TAC processes. In the event that SECOPS has an issue with, say, Splunk or RSA, Cisco TAC has a streamlined process that places customers in touch with the right person at RSA, Splunk and its other partners.

Greater Defense through Faster Innovation Absorption

Clearly Cisco products bring value to their ecosystem partners. For example, Cisco’s firewall team produces the number one firewall in the world, developing features or functionality nearly every quarter or at least twice a year.

Before the ecosystem was in place, a lag between Cisco innovation launch and SIEM ability to support new features was common. For example, SIEM vendors may not understand what the new features are meant to do or how they’re used. Therefore, as part of the SIEM ecosystem, Cisco is committing to assure that as new innovations/features are rolling out across its security portfolio, SIEM partners understand how Cisco recommends they be used which will speed SEC OPS innovation absorption.

Pulling It All Together

Cisco’s new approach to heterogeneous network security is based upon an ecosystem of SIEM providers that it provides interoperability testing, new feature training, TAC support and deployment guides. The SIEMs will aggregate event logs from a wide range of Cisco and other company security appliances to deliver cross-vendor IT forensics and compliance reports. Cisco’s CSM is the policy manager and troubleshooting platform going forward and will enjoy expanded support of Cisco’s security products. Therefore, policy management and troubleshooting services will be delivered through CSM, while the SIEM ecosystem delivers broader cross-vendor IT forensics, event monitoring and compliance reports.

IT business leaders are benefited with a broader multi-vendor approach to event monitoring, forensics and compliance reports as well as centralized policy management and troubleshooting of Cisco products. This new approach should increase IT defenses while simplifying the management of their Cisco security products.

Tweet

Ethernet networking is now the single most important data center technology to assure the new IT economic model of centralized application delivery. Yes that’s right—Ethernet as the data center fabric is the stability point in data center design that will dictate if a data center or cloud facility can scale to support huge application and storage traffic loads. And if you think that Ethernet switch performance is not important then you would be as right as the engineers who designed the Tacoma Narrows Bridge. In this Lippis Report Research Note, we explain why network performance of data center Ethernet switching products matter more now than ever.

Data centers are becoming IT black holes where no application can escape the gravity of its economic force. A few facts are in order:

Mobile Applications and Devices Soar: Mobile application use is expanding exponentially, thanks to the popularity of the iPhone and increasingly Android smartphones. Most important about this is the traffic load these applications are placing on data center Ethernet fabrics. The vast majority of mobile applications are hosted in data centers and/or public cloud facilities. The application model of mobile devices is not to load them up with thick applications like Microsoft Word, PowerPoint, Excel, etc, but to load them with thin clients that access their application and data in data centers, private and/or public cloud facilities. As of this writing, there are some 205,000 plus smartphone applications.

A New Tier of Computing Emerges: A new and rapidly growing tier of computing has emerged in 2010. This tier is the Android tablet and iPad. According to the Wall Street Journal, sales of tablet devices (Android plus iPad) are expected to hit 19.5 million units in 2010 and 54.8 million in 2011. In contrast, Garter predicts that PC shipments will be 352 million units in 2010. In just six short months, tablets now represent some 6% of PC shipments and are expected to displace nearly 10% of PC shipments by 2014!

What is important about this new tier of computing is its application model, which is nearly the same as smartphones. That is these 10s of millions and growing numbers of tables are relying on data centers plus private/public cloud facilities for their applications placing further traffic load on Ethernet data center fabrics.

Virtualized Desktops: 2011 will be the year of the virtualized desktop. Frustrated with Microsoft’s enterprise application licensing, plus desktop support model, IT business leaders will turn toward virtualizing desktops at increasing numbers in 2011. The application model of virtualized desktops is to deliver a wide range of corporate applications hosted in data centers and/or private/public clouds over the enterprise network. While there are no estimates to the traffic load this will place on campus and data center Ethernet networking, one can only assume it will be huge.

Storage Traffic over Ethernet Fabric: Converged I/O or unified networking where storage and network traffic flow over a single Ethernet network will increasingly be adopted in 2011. A single converged network adaptor or CNA plugged into a server provides the conduit for storage and application traffic flows to traverse over an Ethernet fabric. The number of suppliers offering CNAs has grown significantly, including Intel, HP, Emulex, IBM, ServierEngines, QLogic, Cisco, Brocade, etc. In addition, the IEEE opened up the door for mass deployment as it has ratified the key Ethernet standards for lossless Ethernet. What will drive converged I/O is the reduced cost of cabling, NIC and switching hardware.

The above trends are just starting to take hold. Over the next five years, a sea change in IT delivery will occur. It’s clear that the number of mobile smartphones and tablets will only increase as will their reliance data center hosted applications. Virtualized desktops too will force an increase in centralized application delivery while storage traffic increasingly flows over Ethernet fabrics. Corporate application portfolios will change dramatically as will their application traffic profiles with loads being ever more unpredictable. There will be surprises or unforeseen changes that may very well accelerate these trends.

From a data center design point of view, IT architects discovered over three years ago that they can scale compute resources to nearly unlimited dimension thanks to multi-core processors, virtualization and cloud spec design. And with centralization comes huge corporate advantage that being centralized complexity to manage IT more effectively. But more importantly is the fact that IT represents on average only 2% of corporate revenue but has a profound impact on the other 98% of corporate operational spend and competitiveness. With application centralization, IT business leaders can more easily control IT and target it toward reducing corporate operational spend through streamlined business processes or launch new services to respond to market dynamics.

At the center of this massive application centralization transition is networking as it ties compute, storage and internet access together. Ethernet networking, in particular, is now the single most important data center technology to assure the new IT economic model of centralized application delivery. Now most corporations and cloud providers are scaling up their data center bandwidth with 10GbE. In fact, over the last quarter, many networking companies have reported greater than 60% shipment growth in their layer 2 and layer 3 fixed and modular Ethernet switches. So the above trends are driving network demand.

But IT architects and business decision makers need to understand the underlying performance and power consumption metrics of the switches they deploy. The only way to be assured that the Ethernet fabric that is being deployed now in the data center will scale to support increasing application load and storage traffic is to review public, independent, credible and repeatable network throughput and latency performance numbers across multiple vendors.

During the mid 1990s, Scott Bradner of Harvard University and Nick Lippis of the Lippis Report offered independent comparative Ethernet switch performance test evaluations to guide IT business leaders with their purchase decisions. But network purchase decisions have much greater weight to them now as over 80% of IT budgets are spent in the data center. Further, HP wouldn’t have purchased 3Com or IBM wouldn’t have purchased BLADE if they didn’t realize how critically important networking has become to successful data center and cloud computing design.

It’s for the above reasons the Lippis Report has teamed with Ixia to deliver an open data center fabric evaluation of 10GbE switches. Several network equipment manufacturers will participate in this industry-first evaluation, including Alcatel-Lucent, Apresia, Arista, Blade, Juniper Networks and Voltaire. The testing, which is taking place at Ixia’s iSimCity location in Santa Clara, will use Ixia’s Xcellon-Flex load modules to evaluate the performance of the participating vendors’ top-of-the-line 10 GE data center devices.

We’ll publish a report on our findings in mid January so stay tuned.

Tweet

For as long as I have been following Avaya—and it’s been a decade since it was spun out of Lucent back in October of 2000—it has undergone three fundamental transitions. First, Don Peterson, Avaya’s first CEO, managed to fix Avaya’s balance sheet after Lucent saddled it with heavy debt. He also pointed the way toward IP telephony in his six years at the helm. Then came Louis D’Ambrosio, with high energy and confidence, to point Avaya in the direction of unified communications, and a software and services business model, while bringing the company private in 2007 through TPG Capital and Silver Lake Partners. In 2008, Charlie Giancarlo became chairman, while Kevin Kennedy took the helm, ushering in a new wave of innovation and nimbleness while re-engineering sales and channels plus absorbing the Nortel enterprise business. Yes, what a long, strange trip it’s been, but Avaya is now the most innovative in its history and well positioned for the post-recession business cycle. In this Lippis Report Research Note, we examine Avaya’s prospects and challenges.

If Peterson’s contribution to Avaya was “Righting the Ship,” and D’Ambrosio’s was “Energy and Purpose,” Kennedy is ushering in “Nimbleness and Innovation.” With each phase of executive leadership came a resetting of corporate culture. Peterson and the executive management team nearly all had AT&T/Lucent culture where the enterprise business was a rounding error. D’Ambrosio brought a customer focus, energy and big blue reliability. Both of these cultures were grounded in East Coast high tech. Kennedy reset the culture button to a West Coast pace of “go, go, go” with phased product roadmaps, advanced technologies and broad channels to market. The new Avaya has taken shape with a slew of product announcements and new technologies. Here are a few of its most novel directions:

The Flare Experience: In September of 2010, Avaya introduced the Flare Experience, which is a new human-machine metaphor to easily conduct videoconferences and collaboration. Flare seeks to provide a seamless video experience from desktop to softphone to video conferencing systems to android tablets, etc. The most notable aspect of Avaya Flare™ is the introduction of the Avaya Desktop Video Device, an android tablet that creates video sessions with the ease of dragging and dropping contacts from an address book to the center of the screen via touch screen technology. Key to Flare’s innovation is the linking of presence, directory and call establishment/tear down between Avaya one-x Communicator 6.0, the Avaya Video Conferencing solutions based upon joint development work with LifeSize, the Avaya Desktop Video Device, Avaya Video professional and managed services as well as Avaya’s web.live. But it’s the Avaya Aura collaboration server in the back end providing the magic code to create an enterprise wide video experience.

What’s impressive about Flare is that Avaya has created a user interface that integrates voice, video, web conferencing, IM, presence, email, contacts, calendar, messaging, browsing, business applications and social networking that’s controlled by touch. Desktop or user interface design is usually not offered by communications companies, other than phones, so this is a significant innovation point for Avaya.

The Skype Relationship: With Skype and Avaya being owned in part by Silver Lake Partners, a friendly business channel was easily created. For years, most industry observers and IT business leaders sort a way to integrate Skype calls into enterprise communications and collaboration. Avaya was the first to do so by granting access to U.S. customers to Skype Connect™, from their existing UC endpoint via a SIP connection between Avaya Aura and Skype. The Avaya-Skype link becomes more feature rich in the second half of 2011 when federation is established so that Avaya and Skype business users can engage and interact via presence, IM plus voice and video. Beyond the cool factor, there are hard economic reasons why a Skype connection makes sense for Avaya customers. There are three value points…those being low international calling rates, access to Skype’s global community and inter-company collaboration via modern communications.

The SME Roadmap: To show how nimble Avaya has become, when it acquired the Nortel enterprise business in late 2009, it had six products focused at the same Small to Medium Enterprise (SME) market. Those products were Avaya IP Office, Integral 5, PARTNER ACS, Norstar, BCM and SCS. The road to a single product was introduced in January 2010, and the Avaya IP Office was chosen as the SME platform. In 10 short months, Avaya has integrated the full feature sets of PARTNER ACS and has added support for BCM IP handsets into the Avaya IP Office 6.1 image. The next major software revision for IP Office is 7.0, due out in early 2011, and if all goes well, it will include complete BCM and Norstar features plus handset support. The integration value is huge as there are fewer products with overlapping features to support the large SME market simplifying IT executives’ lives, plus channel partners and Avaya’s businesses.

The Avaya Virtual Enterprise Network Architecture or VENA: With the Nortel acquisition, Avaya picked up the enterprise data networking group and associated products that include Ethernet switching, unified branch, WLAN, network access and network management portfolios. To organize these products and demonstrate an investment cycle, Avaya recently launched Virtual Enterprise Network Architecture (VENA). VENA focused the Avaya product set on a major inflection point occurring in the industry; that is virtualization in the data center as well as on the desktop via VDI and storage. There are clear problems with existing network architecture and design that has focused on physical versus virtual ports since the mid 1980s. New thinking in network design is needed if IT business leaders are to reap the benefits of virtualization as it spreads throughout an enterprise.

VENA defines a virtual service network layer that maps IT services to unique virtual networks that run over a virtual services fabric, which is built upon enhanced IEEE Shortest Path Bridging. According to Avaya, this provides resiliency, simplicity and a consistent interconnect that transparently supports co-existing services. In short, applications and IT resources are assigned to virtual networks that are independent of physical ports, allowing more freedom and much less operator intervention during changes to applications, Virtual Machines, etc.

Avaya’s Prospects

One thing is clear, and that’s Avaya is not tree hugging any technologies or products from the past. It has aligned its UC, contact center and data businesses with major market demands. The Flare Experience is a bold new approach to UC and video collaboration matched only by Cisco and in part Microsoft. It has executed the Nortel integration with speed rivaled only by much larger high-tech firms Cisco and IBM. It could not have picked a better market inflection point than virtualization to add value and investment for its data-networking portfolio. Avaya seems to be firing on all pistons from an operations, engineering, channel expansion and product innovation points of view. If its bets are right, it should be rewarded with market share gain.

Avaya’s Challenges

Avaya does have challenges too. By keeping the data networking group, it has, in essence, become a little Cisco, being about one eighth its size. But Avaya does enjoy very loyal data networking customers, particularly in the financial services industry, which date back to the days of Wellfleet and Synoptics. Avaya and Cisco could not be more different, however. If Avaya is a voice company with some data networking, Cisco is a data networking concern with voice technology. While Cisco and Microsoft have significant pull through sales of communications for their data networking and software, respectively, Avaya does not. Avaya has to compete for data networking and communications business, by and large separately, unless and until it provides a compelling value proposition to supply an architected solution consisting of networks, communications, collaboration and contact center.

Opportunity: Service Delivery Process

One of Avaya’s biggest opportunities lies within its ability to add value to a company’s “service delivery process,” thanks to its rich customer data afforded by Avaya’s Contact Center (CC) business. For example, just this past July, the Avaya CC business introduced the Avaya Aura CC Suite, which is designed to enable end-to-end service experience management. The Aura CC Suite’s Assisted and Automated Experience categories include multi-channel work assignment, self-service and proactive contact applications that drive communications and transactions with customers via voice, email, web chat, SMS or social media. Aura CC Suite also delivers a Performance Management category that includes Avaya’s analytics and reporting platforms, Avaya Call Management System and Avaya IQ, which provide companies detailed customer information that helps to improve profitability and customer retention. In addition workforce optimization and workforce management capabilities were added under the Avaya Aura WFO category.

Service is a key competitive differentiator during this economic cycle where the service economy is the bright spot, and 82% of millennials stop doing business with a company after one bad CC experience. Avaya’s CC customers are equipped with vast customer touch points with every interaction, be it voice, chat, IM, email, etc., being a data point of needs. And Avaya does have a loyal CC customer base. Paradoxically Avaya and Harley Davidson are similar in the fact that both enjoy customers who never switch products. A Harley rider would be hard pressed to switch to another motorcycle as would an Avaya CC customer.

It’s all of these tools to monitor and control customer touch points that deliver so much value to customers and Avaya that it can now be leveraged to another level. Avaya can add value to its enterprise customers by synthesizing, aggregating and monitoring the huge number of customer touch points it offers its customers to afford them deeper market knowledge and allow them to be more adaptive, responsive organizations that deliver differentiated experiences and favorable business outcomes.

Key Avaya platforms are Aura—especially as it invests to make Aura a media-agnostic application platform with special attention to video—and Agile Communications Environment (ACE). ACE facilitates the development of communications-enabled business applications to speed workflow. ACE 2.2 includes an Event Response Manager—a new packaged application that reduces downtime and increases efficiency by automatically notifying the right people with the right skills to respond to and manage unexpected events, such as inventory shortages, security breaches, disasters, stock crashes, etc. A new ACE developer toolkit seeks to make it easier to embed timely and personalized communications into business applications. With Avaya ACE, enterprises can communications-enable their business applications up to 80% faster than by using traditional methods, according to Avaya.

Leveraging Avaya’s huge DevConnect community to write applications around Aura that leverage Avaya’s UC and CC resources while riding over VENA is one sure way to elevate Avaya’s importance and consideration in the Enterprise market. With Avaya’s new nimbleness and innovation, it clearly has the ability to weave its UC, Collaboration, CC and Data products and services around “service delivery process” for its customers, as well as differentiate itself in a significant way. If there isn’t a value proposition around all of Avaya’s products and services, then it may find itself competing in four separate markets, with four separate customers, channels and competitors.

Tweet

Nick Lippis of the Lippis Report announces the participating vendors in the 10GbE Data Center Network Fabric test at the iSimCity lab during the week of Dec 6-10, 2010.  Watch it here

Tweet

A Comprehensive Approach to Corporate and Government Energy Cost Savings and Carbon Reduction

Being green is increasingly being forced upon IT business leaders from their management, government regulations and societal pressures. Ask a recent college grad what is the number one societal contribution they would like to make with their career and the answer is “make the world greener.” The workforce is changing worldwide with a sense of personal and corporate social responsibility to reduce carbon emissions, and choose sustainable materials and processes to power our lives and deliver products and services. And being green is no longer a luxury that IT leaders can choose as governments, boards of directors and presidential directives issue mandates forcing energy efficiency upon IT executives.

From an IT perspective, much work has been done to reduce data center energy consumption and cooling by virtualizing servers and consolidating data centers. In addition, IT vendors continually work to deliver products with increased feature sets that consume less energy. But one company in particular has taken its core competency and found a way to not only make its own products more energy efficient but everything its products touch, too. That company is Cisco Systems.

A Broader View of Energy Management

Cisco is providing tools and knowledge to IT business leaders to assist them in complying with energy efficiency mandates. And while much attention has been focused on data center energy reduction, a much larger target for energy conservation is IT and non-IT energy consuming assets that are sprawled throughout enterprise and government facilities—this means networks, personal computers, printers, lighting, HVAC, etc. But in addition to energy management of electrical device sprawl, energy consumption can also be avoided by using communication and collaboration tools such as Webex, virtual office teleworking and TelePresence. These collaboration tools allow users to work at home and engage in meetings over the web or via high definition videoconferencing versus traveling, thus avoiding dollar and carbon emission cost of travel. These concepts and initiatives are part of Cisco’s Borderless Networks Green service, one of the key network services within Cisco’s Borderless Networks Architecture.

The key concept of Cisco’s Borderless Networks Architecture is the removal of boundaries or borders that create common trade-offs and compromises IT business leaders and users have come to despise. Cisco’s Borderless Networks Architecture is comprised of five pillars that enable borderless connections of anyone, anytime, anywhere and from any device securely, reliably and seamlessly: 1) Mobility through the Motion service, 2) Green or enabling energy cost savings and carbon reduction through EnergyWise, 3) integrated network Security via TrustSec, 4) Application Performance to increase network and application agility, visibility and control with Application Velocity Network Service and 5) Video/Voice services to offer the best possible video experiences to users via the Medianet technologies. These borderless network services are delivered by core infrastructure including switching, routing, security, wireless and wide area application services (WAAS) infrastructure products. It’s the integration of these services into existing network infrastructure and their control via policy and management that enables a borderless experience to occur. In short, a borderless network eliminates friction points and user plus operational frustration associated with common IT use cases such as application access from desktop, laptop, tablet, smartphone, etc. For example, the Borderless Networks Green service enables IT executives to reduce their carbon emissions, save on energy costs, transform their business while satisfying increased IT demand. In this Lippis Report Research Note, I focus on the Borderless Networks Green service as it offers a comprehensive approach to energy management.

Borderless Networks Green Service

There are three main drivers why organizations are looking for ways to be greener—those being cost reduction, sustainability mandates and corporate responsibility. Being a green, socially-responsible organization improves corporate image, which is usually accompanied by increased revenue opportunities. And many companies are in search for effective ways to achieve operational cost savings through green IT practices, especially during the past three years given economic conditions. That is why corporate executives seek to enhance their firms’ image/brand and comply with energy reduction mandates while reducing operational costs, all through green initiatives.

To help customers achieve their green goals, Cisco’s Borderless Networks Green service exploits the network as a platform to extend green borders. This is done in three ways: 1) transform the workforce by making it more flexible with collaboration applications such as TelePresence, Webex, Virtual Office, etc., 2) enable energy cost savings with innovations such as EnergyWise that measures and manages energy usage, and 3) improve network efficiency through virtualization, consolidation plus product and system life-cycle management. As Cisco EnergyWise is a fundamental and unique green enabler, we focus on this technology first.

Cisco EnergyWise is a system-wide framework for energy management that is integrated into Cisco Catalyst switches, routers and building controllers. Every device that connects into the network can eventually have its energy managed, monitored and optimized by Cisco EnergyWise. This concept of using the network as a system to coordinate activities which provide benefits that aren’t available from a single device is a key principle of the Cisco’s Borderless Networks Architecture. EnergyWise delivers on this principle by adding energy management to Cisco’s Borderless Networks services.

Cisco EnergyWise

Cisco EnergyWise is being released in phases. The first phase was launched in January, 2009, and focused on reducing energy usage of Power over Ethernet (PoE) devices. These devices include IP phones, wireless access points, security cameras, etc. The second phase, launched in March, 2010, added the ability to control PC and laptop power. PC and laptop power control is accomplished with a product called Cisco EnergyWise Orchestrator. Orchestrator is a client-server architecture designed to scale up for large organizations. A small software client runs on each PC, collects energy usage information and allows Cisco EnergyWise Orchestrator to distribute centrally-managed, time-based energy policies to each workstation such as shut down after 6:00 p.m. and power up after 8:00 a.m. In addition, EnergyWise Orchestrator can request “on-demand” power reductions. EnergyWise Orchestrator also receives power usage statistics from PCs distributed throughout an enterprise or government facilities, which can be aggregated and displayed in different variations via its sustainability dashboard. As PCs and laptops are sprawled throughout enterprise and government facilities, Cisco EnergyWise Orchestrator is able to manage up to 60% of power used by IT devices, thus the impact of Cisco’s energy management solution is material.

Cisco is extending the reach of EnergyWise to control power of more IT and non-IT devices. The EnergyWise framework includes open APIs that enable an ecosystem of partners to offer comprehensive energy management solutions to meet customer needs of all kinds. For example, recently Cisco announced partners that allow EnergyWise to manage Smart Power Distribution Units from Schneider APC, WTI (Western Telematic, Inc.), Server Technology, Raritan and CyberSwitching. These partnerships extend energy monitoring and reporting to data centers, and expand energy management capabilities to clientless devices like printers, copy machines and digital media displays. .

Business Transformation Applications that Reduce Energy Consumption

While most, if not all, networking concerns stress energy efficiency of their products, Cisco’s Borderless Networks Green service takes this to an entirely different level through energy efficient collaboration applications that transform how corporations conduct business. Collaboration applications, such as Cisco’s WebEx, TelePresence and Virtual Office, reduce travel needs and improve productivity while achieving great in-person work experiences. Underneath these collaboration applications is Cisco’s Borderless Networks infrastructure that ensures security, availability and performance of these business applications with services such as Medianet and Cisco TrustSec.

Product Power Efficiency Gains

Cisco’s Borderless Networks Green service addresses reduced energy consumption of IT assets, such as PCs, laptops, PoE devices, and networking equipment such as routers and switches, plus collaborative applications. And while offering this broad view and tool set for IT business leaders to manage energy policy, Cisco has not taken its eye off the ball of engineering innovations and improvements in network products to ensure energy efficiency. For example, StackPower is a new innovation for the Cisco Catalyst fixed switching products that distribute power across a stack of switches in a unique and efficient way. Further, Cisco recently introduced a 48-port switch that consumes only 40 watts of power…that’s less power consumption than most light bulbs.

Virtualized Data Center Infrastructure Delivers Energy and Resource Efficiency

In addition to EnergyWise, product energy improvements and collaborative applications, Cisco’s Borderless Networks Green service extends green initiatives to the data center too via virtualization. Data center consolidation and server virtualization are solutions that help IT business leaders maximize the usage of existing resources while contributing to data center efficiency. These solutions include VMware and Cisco’s UCS (Unified Computer System). In addition to server virtualization, firewall and WAAS services have become virtualized as well as bandwidth via Storage Area Networking. Desktops too are being virtualized. All of these initiatives contribute to reduced footprint for rack space, cabling and HVAC requirements. Less power is consumed while the data center is more efficient with improved operations, thanks to more flexible use of resources and bandwidth.

Some text to space apart the download boxes

The benefits of Borderless Networks Green service are workforce flexibility and improved productivity, energy cost savings and network efficiency. While some of these improvements are difficult to measure, there are solid ROI examples. GE, for example—a Fortune 500 company that adopted Cisco’s TelePresence—reduced its travel and lodging expenses by 40% while reducing executive management wear and tear. Parque Escolar works with the Portugal Ministry of Education and was able to reduce Portugal schools’ energy consumption by more than 33% by implementing Cisco EnergyWise Orchestrator. Brunel University is saving $143,908 per year thanks to energy control of power usage through EnergyWise.

Cisco’s Borderless Networks Green service offers a range of options to manage corporate and government energy consumption, and the value/cost savings that EnergyWise brings to IT business leaders today will continue to multiply as Cisco delivers more platforms and partner devices that can be monitored and managed from centralized management applications such as Cisco EnergyWise Orchestrator or LMS. While IT executives are implementing virtualization and collaboration applications based upon their own merit, much can be gained by viewing these IT projects through a green prism. For it’s the totality of device energy management along with business transformation collaborative applications and virtualization that may very well define a modern green business.

Tweet

One significant trend that has emerged during the current business/economic cycle is that IT projects that reduce cost are winners. This savings trend is as strong as I have experienced in my twenty-five years within the IT industry. In particular, it’s propelling data center consolidation, server virtualization and mobile computing projects. As enterprises consolidate data centers and miniaturize them with virtualization, cloud-computing providers are busy offering a new lower cost IT delivery economic model. In short, a new tier of computing has emerged were endpoint devices are mobile and applications are delivered via corporate data centers and cloud computing facilities. This new model of computing that also increases convenience and productivity is lacking in one important area; network security for both mobile endpoints and the ability of data center security appliances to keep up with application demand.

And keeping up with application demand is one of the most challenging tasks IT business leaders are encountering. Not only has information demand skyrocketed during this business cycle but content in the form of web pages has become dynamic, where a single page request opens a multitude of connections pulling content from various sources to satisfy user expectations of real time information access. For example, a single web page request can easily spawn more than fifty network connections over physical and virtual infrastructure placing extraordinary demands on network speed, latency, reliability and security. For the uninitiated, just point your browser to any of these sites—disney.com, cnn.com, nytimes.com, et al—and notice rich content in action. As the page is presented, it serves up video, photos, audio, rich text and more, all of which are pulled from various sources within a data center fabric over virtual and physical infrastructure. The calculus IT leaders are seeking to solve includes massive growth in information demand plus Brownian motion traffic flows, thanks to dynamic content plus densely packed data centers, thanks to virtualization. Even with consolidation and virtualization information/application, demand is forcing the overall data center market size to expand from 108 million sq. ft. in 2009 to a projected 117 million sq. ft. by year end 2010, according to Frost & Sullivan. Part of the solution to IT leaders’ calculus problem is found in a data center network fabric that supports millions of connections/session of east-west and north-south traffic flows securely.

To put the mobility trend into perspective, Apple sold over 3.3 million iPads in its first 3 months; the highest uptake of any endpoint device. Google activates 100,000 Android-based phones per day. Cisco recently announced its CIUS android-based table for business use with tight links to its unified communications (UC) and videoconference systems. Every major UC provider will be offering similar devices while traditional computer vendors serve up android-based tablets over the next few quarters. The iPad and Android tablet is a new tier of computing, which are driving users to access applications over mobile and wireless networks in addition to their wired and VPN networks.

And therein lays the rub. In today’s modern IT world, applications are being extended over multiple networks, e.g., wired, wireless, mobile and remote, where users shift their application access back and forth between these different network access methods and expect the same or consistent experience. Security is paramount to user experience and IT asset protection. While IT security executives have fortified their defenses of IT assets within corporate boundaries or perimeters, exponentially growing numbers of mobile endpoints being connected into corporate networks and data centers present significant security challenges that are unfortunately outside the control of IT.

The nature of mobile smart phone endpoints is to combine personal and business IT services, thereby creating a unique user experience. Part of that experience includes information access from a plethora of online destinations, such as public WIFI hotspots, SaaS applications, e.g., Salesforce.com, workday.com, netsuite.com, etc, corporate VPN, and a wide range of personal sites for social networking, banking, music, videos, news, communications, etc. Therefore, for every employee equipped with a mobile endpoint, security vulnerabilities and threats are opened unless IT mitigates with network security. Clearly mobile devices are becoming ubiquitous, and there are security solutions available, such as VPN support, data wipe after loss, cloud-based security services, etc. But mobile devices need a security solution that works in real time, meaning it’s always-on protection and provides comprehensive coverage.

For example, mobile endpoints, and thus corporate assets, need to be protected from users accessing the corporate network from insecure home WIFI networks and hackers. Internal applications need to be secured against attacks such as SQL injection/data leakage, request forgery/impersonation, cross site scripting/phishing, etc. SaaS access needs to be secure against unauthorized access, exposure from password reuse, layer 7 attacks and more. Also the same level of reporting for mobile users as wired users needs to be supported to assure activity/audit trail, regulatory compliance plus governance and reporting. In short, IT needs the same level of control over mobile endpoints as it does over devices within the corporate perimeter without ruining the mobile experience.

Mobile Endpoint Policy and Enforcement

The most important aspect of real-time mobile security is policy enforcement as it places control of corporate asset and SaaS access back into the hands of IT. Not only does policy and enforcement mitigate threats from being transmitted from mobile endpoints onto corporate networks, it makes them safer devices, too, by providing a means to adhere to corporate policy as corporate devices, even though they are used for business and pleasure. This is important as many mobile devices are purchased by employees, part of the huge consumerization trend that has been building over the last five years. With IT able to administer policy with a means of enforcement, mobile devices can deliver personal and business IT services. Employees may purchase mobile devices but if they require access to corporate IT, then the endpoint has to comply with corporate policy and IT needs a means to enforce such policy. In short, policy and enforcement enables IT to extend the corporate perimeter around mobile devices to creating a virtual perimeter around IT assets.

Consider the following example of policy and enforcement creating a virtual perimeter… A user may be accessing an SaaS application while at his/her desktop. This flow traverses the corporate firewall with associated policy and enforcement. When this user is outside the corporate perimeter, he/she could access the SaaS application directly without corporate policy or enforcement opening vulnerabilities. However, with mobile policy and enforcement, this same user could access the SaaS application with the same policy, enforcement and protections as available when within the corporate perimeter mitigating any vulnerability. Solutions to this usually require the mobile device to first pass through the corporate firewall or a security cloud service where IT controls policy before the user connects to the SaaS application.

New Security Performance Demands

With mobile endpoints under corporate IT policy and enforcement, this huge security vulnerability can now be managed and mitigated. At the same time that mobile devices are becoming ubiquitous, data center security appliances are failing to keep up with the huge demand for information and application access. As more compute power is concentrated into smaller spaces, traffic volume increases exponentially, and security appliances need to adjust accordingly.
Consider how web sites serve up a rich media web page. Every time a user requests a webpage, its server typically needs to request 50 to 100 different objects just to display the one webpage requested. Now consider a data center with thousands of servers and five-thousand connections per second of requests each spawning 50 to 100 server requests. The backend east-to-west traffic flows between servers are one to two orders of magnitude larger than the north-to-south user request flows with the combination of both flows being immense.

New Firewall/IPS Performance Metrics Needed

From a security point of view, not only is firewall throughput an important performance metric, but “connections per second” is becoming more important. A high number of “connections per second” supported assures IT that backend server flows are being screened without delaying user experience. In addition to the number of connections per second, another performance measurement is “maximum connections” supported per second to assure that the number of server-to-server flows to deliver a webpage can be securely delivered. The combination of throughout, connections per second and maximum number of connections can be defined as “true scale performance.” Typically a firewall can deliver hundreds of thousands of connections per second, but this is too slow for most demanding data centers by at least a factor of 2 to 3. Typical maximum number of simultaneous connections supported per firewall is around a few million, which is too low by at least a factor of 4 to 6. Also consider a more realistic throughput measurement other than a range of UDP packet sizes, which is common in the industry. Real world throughput performance numbers that represent a mixture of traffic profiles is a better measurement to assure throughout quoted is throughput experienced.
In addition to raw security performance, data center rack space too needs to be carefully managed as IT executives quickly start running out of rack space as they consolidate. Security appliances need to reduce their footprint as many appliances occupy 16 to 24 RU or a half rack of space and more consuming footprint, energy and cooling resources. Expect security appliances to start delivering on the above performance metrics at up to an 8th of their size or 2 RU high if not smaller.

Threat Protection

To assure this security infrastructure protects IT assets at the rate in which cybercriminals and hackers wish to penetrate it, the industry is serving up cloud-based threat protection. A few suppliers have launched cloud-based security services, which collect anomalistic data throughout the internet and corporate networks via sensors, analyze/correlate the anomalies with reputation scores and when a new exploit’s signature is detected, the cloud transmits mitigation code/signature updates to corporate IPSs. The speed in which this process takes place is a competitive differentiation. Those that send updates every five or so minutes have the best chance of mitigating exploits from cybercriminals which tend to change IP address every hour to avoid detection. IT business leaders will know when cloud-based threat protection becomes highly reliable. It’s at that point that suppliers will start offering “guaranteed protection” that incorporate penalties to suppliers if protection is penetrated.
Policy and enforcement of mobile devices creates a virtual perimeter while true scale performance enables security appliances to keep up with application demand and new traffic flow realities. Smaller security appliance footprint allows IT executives to maximize data center space while minimizing energy and cooling. Cloud-based threat protection keeps the security infrastructure updated in near real time with signatures to mitigate threats throughout the corporate and virtual perimeter. In short, IT business leaders gain control and manage mobile security vulnerabilities while delivering applications to users securely at speed with small footprint consumption. Mobile, data center consolidation and virtualization plus cloud computing are powerful trends rooted in economic efficiency and increased information demand. To maximize the value of these investments, a new security model is needed.

Tweet

Major IT Delivery Transitions IT Business Leaders Are Managing
Application owners and developers have been deploying and writing applications as if networks had no boundaries or were borderless. By “application owners” I mean IT departments chartered with IT application delivery and management. By “application developers” I mean in-house corporate software developers, independent software vendors (or ISVs) and software companies. There has always been a disconnect between applications and network architects where developers write applications to run over a network as long as there is connectivity. In addition, service-oriented architecture (SOA) based applications call for greater application componentization, which increases messaging between application components, resulting in the network having a direct impact on application performance. In essence, application owners, developers and application standard bodies assume that networks are borderless as the industry is organized around the OSI model where knowledge and skills at one layer, e.g., the network is not necessarily taken into account at another layer, i.e., the application. Therefore, the normal state of affairs is that network designers have been tasked to optimize applications to improve user experience especially when the application was not written to run over a particular kind of network. This status quo does not scale and needs to be re-thought.

Business Drives Applications that Drive Computing that Drive Networking

Every cycle of computing has brought with it this discontinuity between applications and networks with the possible exception of mainframe computing and SNA. Minicomputer applications designed for local ASCII terminal connections were extended over the Wide Area Network (WAN) and via virtual terminals. Client-server computing applications designed to run over Local Area Networks (LANs) were extended over the WAN. At first the internet was text based until the mid 1990s when the web was developed, bringing graphics, audio and video to a network that needed a massive upgrade to support new media rich applications.

IT today is no different. Application developers are writing mobile applications at a frenzied pace thanks to Apple’s iPhone and iPad, Google’s Android, RIM’s Blackberry and now Cisco’s CIUS plus Avaya’s Flare, etc. Legacy enterprise applications are being extended to mobile platforms too with the assumption of a suitable network for delivery. At the same time, applications are being increasingly centralized into consolidated data centers creating greater distance between users and their applications plus data. Some estimate that over 80% of enterprises have undergone a data center consolidation process, which is significant, but we are just at the beginning of the centralization trend.

Thanks to the economics and performance offered by server virtualization, much more consolidation will occur with associated challenges. For example, IT leaders require application tracking as applications are moved from Virtual Machine (VM) to VM as they tune/optimize their virtual infrastructure or respond to peak loads as well as manage VM failovers. In addition to virtualization, massive data centers we call cloud-computing facilities are being built to host applications at scale plus offer infrastructure, platform and other IT services. According to the Yankee Group, 56% of IT business leaders seek to take advantage of cloud-computing technology and build their own private cloud center while 24% seek a fully-managed cloud-computing facility. In the same study, 32% of IT business leaders will seek a hybrid cloud approach that is, connect their private cloud to a service provider’s public cloud. While these market numbers are impressive, they could be much higher as IT leaders express that their top three concerns as they consider cloud services is application performance issues, according to IDC.

In addition to increased mobile and cloud-computing trends, video communications, both on-demand and real-time, have become the largest percentage of internet traffic type. In fact, Cisco Systems recently predicted that by 2014 video traffic will be greater than 94% of all global internet traffic!

This disconnect between applications and network architects will more than likely continue as application owners/developers/standards continue to view networks without borders and boundaries. However, for most network architects, there is no single network, but a wired network, wireless, campus, wide area, data center, branch office network, telecommuting network, mobile network, etc. In fact, most enterprises have a diverse infrastructure in which they are tasked to delivery applications over and for those applications to perform at high standards. The good news is that network designers and architects are starting to build borderless networks that anticipate unforeseen application changes, are equipped with a portfolio of application performance features and simplify deployment and management of IT services…more on this below.

Application Performance Challenges

From the above discussion, it’s clear that enterprise-computing applications are being demanded and stretched over increasingly borderless networks. Consider that the number of small or remote offices and mobile employees are increasing significantly. It’s impossible to argue the mobile computing surge with over 3.3 million iPads shipped in the first three months of its launch, and new entrants such as Cisco and Avaya offering CIUS and Flare tablets, respectively, for business users. In addition, data centers are being consolidated with cloud computing, offering further consolidation and centralization of applications. Applications are changing too as developers add rich media features, and video becomes a dominate application type. Employees, customers, partners and suppliers will be accessing applications over ever-larger distances, via a plethora of endpoints and different networks.

To assure applications perform their task and deliver an excellent user experience, network architects and designers will be increasingly challenged with network capacity being taxed as a wider application portfolio competes for network resources. Today’s model of application performance optimization is to implement appliances within remote sites and data centers, which increases certain application performance, but at the high capital and operational expense of increased network complexity. In addition to network capacity and complexity issues, latency or application transaction delay and how to efficiently utilize data center resources are challenges faced by network architects as they seek to maintain high application performance over a borderless network. Relating specific application transaction problems to network behavior to ascertain if a correlation exists is yet another challenge.

Application Performance Creates Corporate Value

At the center of application performance is corporate performance. The ability of IT leaders to respond to executive management directives is directly linked to corporate performance. Executive management may be challenged with a competitive threat or a new market opportunity, etc., requiring fast corporate response. IT leaders who can execute directives quickly have built an agile business capable of changing when markets or customers shift under them, placing their corporation in a better competitive position to serve its customers and prospects. For example, consider a retail store under competitive pricing pressure where executive management decides to respond with an alternative offer. IT may be able to display the new offer via digital signage quickly allowing the business to respond.

Key to business agility is the IT attribute of rapid innovation absorption–that is, the capability to deploy new applications and technologies at the speed of business opportunity. Most IT infrastructures consist of innovation and features which are already in place, but IT organizations require knowledge, skills and tools to put them to work when needed.

A borderless network that is capable of application performance delivers these attributes of innovation absorption and business agility. In addition, IT resource utilization can be optimized, and most important to users is that they gain an excellent IT experience independent of geographic location, endpoint device or application, which in the end improves productivity.

As an example of optimal resource utilization, consider Cisco’s ISR G2 branch office router that integrates unified communications, wide area application optimization, network security, LAN/WAN networking plus supports its AXP (or Application eXtension Platform), which run applications at the branch office router. In one branch office, an IT manager can deliver networking, security, voice and video communications and host applications while gaining visibility to applications. This type of resource utilization not only saves on capital cost and energy spend, but offers IT operational efficiency, rapid application deployment and innovation absorption.

To gain the full value of corporate applications, their performance must deliver excellent user experience. An excellent experience should not only occur while working in the office or at home, but anywhere in between, even while talking on a mobile endpoint. Independent of geographic location, a user accessing his/her business services and/or personal services should be the same seamless experience. Application performance is key to excellent experience and should be consistently good whether sitting at a desktop watching a video or engaged in a Web conference, and then immediately transitioning to an iPhone for example. The user should have an excellent experience at the highest level afforded by his/her endpoint. To deliver this seamless user experience, application performance technology needs to be incorporated in corporate IT infrastructure, endpoint devices or a combination of both.

That is, networking silos need to become an integrated network without borders. For applications to offer the best possible user experience, then the use of application acceleration technology as appliances or an overlay needs to be integrated into the network fabric and into network operating systems. This technology, which has improved application delivery for specific applications, needs to become systemic and fully distributed throughout the network fabric. The integration or pervasiveness of application acceleration technology within networks and endpoints is its natural evolutionary next step. Over the next few months we’ll see vendors such as Cisco, HP Networking, Juniper, Riverbed, Citrix, Blue Coat, et al, start to deliver on this vision.

Tweet

Networking is entering a new phase or era. During the 1990s, new networking markets opened up, creating multi-billion dollar opportunities for the vendor community and corporate cost savings for IT business leaders. First, it was shared LANs and routing, then switched LANs, then Frame Relay to speed up WANs, then SNA over IP, then remote access via dial-up and VPN, then MPLS, then IP telephony, then Wireless LANs etc… and now, it’s video and cloud networking. You get the picture. But what we didn’t realize as we build these networks is that they are silos with disparate management systems and unique access methods resulting in operational cost overlap and, most importantly, user frustration as they transition application use from desktop, to mobile end point, to remote endpoint. In short, we built boundaries around applications in the form of networks and it is the dismantling of these borders that vendors are now starting to deliver and differentiate upon. It’s not just Cisco that communicates borderless networks, but HP Networking, Juniper, Brocade, Extreme, Avaya, Force10 and others too. Why is the industry entering a new age of borderless networking and what’s in it for IT business leaders, is explained in this Lippis Report Research Note.

As each new wave of computing entered corporate IT departments, a new set of networking requirements arose. To connect remote 3270 terminals via SNA to mainframes, IT implemented an analog multipoint wide area network or WAN. To connect remote ANSI terminals to minicomputers, IT departments implemented pools of dial-up modems and private line WANs. To connect personal computers (PCs) via Client-Server computing, IT departments implemented Local Area Networks or LANs via LAN switches, which we now call wired connections. To connect multiprotocol LANs over the corporate WAN, IT departments implemented routed networks. To gain access to LAN based applications while remote, IT departments implemented Virtual Private Networks or VPNs. And, as computing and applications go mobile, IT has been implementing Wireless Local Area Networks or WLANs. In short, each network was deployed to service a certain computing style and application set. These networks are silos, and with advances in technology, IT business leaders can now design one borderless network to provide a broad array of common access methods to support a plethora of endpoints and applications.

Siloed networking frustrates users, as each access network performs differently depending upon its access method. Siloed networking also frustrates IT, as each siloed network has its own management system creating inefficient IT operations. In addition, siloed networking does not meet today’s IT “any access” requirements.

There are boundaries or silos that need to be broken down in many places of the network. In today’s modern IT world, applications are being extended over multiple networks e.g., wired, wireless, cellular, remote, virtual, etc where users need to shift their application access back and forth between these different network access methods and expect the same or consistent experience. In short, networks need to be borderless so that applications can be accessed independent upon network entry point and IT operations efficient. This “any access” trend is accelerating as IT business leaders seek to connect not only traditional desktops and laptops, but smartphones, notebooks, tablets, iPads, cameras and building control systems into a common general purpose network that support multiple logical network topologies.

Crossing purpose-built silos is difficult for applications, as bandwidth and quality of service issues limit application portability thus their usefulness. These different access methods offer limited consistency resulting in user frustration when they shift application access from desktop to mobile smartphone to VPN and back again.

And this shifting of application access between different networks and endpoints is only going to increase. Apple sold over 3.3 million iPads in its first 3 months, the highest uptake of any endpoint device. Google activates 100,000 Android based phones a day. Cisco recently announced its CIUS android-based table for business use with tight links to its unified communications (UC) and videoconference systems. Every major UC provider will be offering similar devices while traditional computer vendors serve up android-based tablets over the next few quarters. The iPad and Android tablet is a new tier of computing which will drive users to access their applications over mobile and wireless networks in addition to their desktop and VPN networks.

If IT business leaders are unable to get ahead of this curve and think of network access from an architected and unified design point of view, than unfortunately, their users and IT cost will be more frustrated and expensive, respectively, than others. Siloed networks are friction points as they create boundaries between network access types degrading user experience, which results in decreased productivity and increased IT operational cost. The result is a high total cost of ownership and less then optimal user experience, and thus decreased corporate productivity. The status quo of siloed networking is about to change.

Cisco’s Borderless Network Architecture

From a design point of view, borderless networking requires three core attributes: 1) reliability, 2) security and 3) seamlessness. Cisco was the first to articulate a vision for borderless networks, which has resonated with IT business leaders as it represents a solution to their pain. For example, Cisco’s borderless network architecture is built upon five services: 1) mobility or users in motion, 2) Energy efficiency called EnergyWise, 3) integrated network security via its TrustSec architecture, 4) application performance and 5) video management, control and distribution via its MediaNet. These borderless network services are built within switching, routing, security, wireless and wide area application services or WAAS infrastructure products. It’s the integration of these services into existing network infrastructure and their control via policy and management that enable a borderless experience to occur.

Juniper’s New Network

But Cisco is not the only supplier to grasp the problem siloed networks create. Juniper Networks is working to a similar end, albeit it hasn’t articulated it well. It provides VPN, LAN Switching, mobile security through its acquisition of SMobile and is working toward a flat cloud Ethernet fabric through its project Stratus and New Network initiatives. For example, Juniper plans to integrate SMobile security into its JUNOS Pulse endpoint software for network connectivity and acceleration breaking down the boundary between LAN based and mobile network access.

HP Networking’s Converged Infrastructure

When HP Networking launched its comprehensive network portfolio in April of this year it emphasized the elimination of network silos. The HP Networking portfolio strives to eliminate redundant equipment by integrating wired and wireless environments with security from edge to core. From an IT operations perspective, this translates into a “single pane of glass” for management, configuration, deployment and monitoring these networks as if one. HP Networking hopes to implement a common policy management to reduce human error of network operations while creating a consistent user experience across access mediums.

Brocade One

Brocade has jumped on the borderless bandwagon also in June of this year with the introduction of its “Brocade One”. Brocade One emphasizes the convergence of wired, wireless and cellular networking to offer a seamless user experience. In addition, Brocade One describes its view of a simplified virtualized data center network fabric that scales to cloud spec. In essence, Brocade One is about eliminating the boundaries around wired, wireless and data center networking.

Arista Network’s VM Tracer

Arista Networks doesn’t use the terminology of borderless networking either, but its recent VM Tracer strives to eliminate the boundaries between physical and virtual networking environments. VM Tracer does this by being integrated into Arista’s EOS linking Arista switches to VMware’s vCenter. This linkage creates an adaptive infrastructure in which the network responds to changes in the VM network while also providing complete visibility into the virtual machine network.

Extreme’s DirectAttach

Extreme Networks has focused on removing two network boundaries; the wired and wireless boundary and the physical to virtual network boundary. For the latter, Extreme has introduced its Direct Attach approach to data center networking that eliminates the virtual switch layer, simplifying the network and improving performance.

Force10’s Open Automation

Force10’s focus in eliminating boundaries is in the data center between physical and virtual networks. Force 10′s Open Automation initiative seeks to align dynamic data center changes with network configuration and policies, a huge barrier to virtualized data center management and scale.

While each of the above suppliers are at different points in their borderless network initiatives, the direction is clear. The boundaries between siloed networking are coming down be it in the data center, campus, branch office or home. For IT business leaders this means simplified operations and management as a key attribute is the “single pane of glass” approach to network management for siloed networks. The big surprise and delight will be found in enhanced user experience, as borderless networking strives to deliver a common access method for all networking types while enabling applications to be extended across a plethora of different endpoints, depending upon endpoint capabilities and network resources.

In essence, borderless networking’s value proposition is that it enables a corporation to be more adaptive or agile while increasing user experience and reducing operational cost. With the majority of IT business leaders trading off reductions in operational spend for an increase in capital expenditure, borderless networking is the right solution at the right time.

Tweet

It hasn’t been since the mid 1990s that the networking industry was focused on multi-protocol integration or convergence. But the industry is gearing up for a major innovation and competitive cycle fueled by the multi-billion dollar addressable market for data center network fabrics. Over the last eighteen months, every major Ethernet infrastructure provider has been talking about two and three tier network fabrics for high-end data centers.

Companies such as Cisco, Arista Networks, HP/3Com, Force10, Voltaire, Extreme, Brocade, Juniper et al have announced network fabrics for data centers with five thousand and more servers with and without storage enablement. Juniper talks of a one-tier fabric through their Project Stratus work with IBM to be available some time in the future. Brocade recently introduced its Brocade One, which is a converged data center fabric. Extreme Networks launched its DirectAttachTM that eliminates virtual plus blade switch layers. HP has FlexFabric, a virtualized fabric for the data center. Cisco launched its FabricPath Switching System or FSS for the Nexus 7000 that enables massive scale of a two-tier fabric.

In this Lippis Report Research Note, we review the architectural attributes of two tier network fabrics.

The IT industry is at an inflection point as service delivery is becoming more and more centralized thanks to data center consolidation, virtualization, cloud and mobile computing. It is estimated that a third of all IT spend is concentrated in the data center, and this trend is only building thanks to favorable economics, motivating IT business leaders to centralize IT delivery.

The impact of this trend is more and more dense data centers made up of servers in the thousands to tens of thousands and higher. It is at the scale of 5,000 plus servers that a new network fabric is required for high-end data centers. High-end data center design is challenged with increasing complexity, the need for greater workload mobility and reduced energy consumption. Traffic patterns have also shifted significantly, from primarily client-server or as commonly referred to as north-to-south flows, to a combination of client-server and server-server or east-to-west plus north-to-south streams. These shifts have wreaked havoc on application response time and end user experience, since the network is not designed for these Brownian motion type flows.

The main requirements for high-end data center network fabric are low latency, large flat layer 2 domains to enable workload mobility, low power consumption, simplicity of design and significant bandwidth. Storage enablement, meaning consolidated I/O or virtualized I/O, is a growing priority and a new fabric that can support FiberChannel over Ethernet, iSCSI over Ethernet, iWARP over Ethernet or Infiniband over Ethernet, is a major plus. One salient observation is that it’s pretty clear that Ethernet is the network fabric of choice, as it is the only network protocol that enjoys continual innovation such as TRILL, Data Center Bridging, IEEE’s 802.1AQ, link aggregation, multi-pathing, and as recently ratified by the IEEE 40 Gbs and 100 Gbs speeds.

With the above requirements in mind, let us review data center network design options.

Three Tier Data Center Fabric

A three-tier network architecture is the dominant structure in data centers today and will likely continue as the optimal design for many networks. For most network architects and administrators, this type of design provides the best balance of asset utilization, layer 3 routing for segmentation, scaling and services, plus efficient physical design for cabling and fiber runs. By three tiers we mean, access switches/Top-of-Rack (ToR) switches, or modular/End-of-Row (EoR) switches that connect to servers and IP based storage. These access switches are connected via Ethernet to aggregation switches. The aggregation switches are connected into a set of core switches or routers that forward traffic flows from servers to an intranet and internet, and between the aggregation switches. It’s common in this structure to over-subscribe bandwidth in the access tier, and to a lesser degree, in the aggregation tier, which can increase latency and reduce performance. Inherent in this structure is the placement of layer 2 versus layer 3 forwarding that is Virtual Local Area Networking or VLANs and IP routing. Also common, is that VLANs are constructed within access and aggregation switches, while layer 3 capabilities in the aggregation or core switches route between them.

But within the high-end data center market, where the number of servers is in the thousands to tens of thousands plus and where north-south plus east-west traffic is significant, is where a new structure is needed. It is within these data centers where applications need a single layer 2 domain.

Two-tiers of network fabric

A two-tier fabric is designed with two kinds of switches: one that connects servers, and the second that connect switches creating a non-blocking, low latency fabric. In short, there are server facing and fabric facing switches. We use the terms ‘leaf’ switch to denote server facing or connecting switches and ‘spine’ to denote fabric facing or switches that connect leaf switches into the fabric. Together, leaf and spine switches create the fabric.

Many IT leaders in Global 2000 firms will have deployed both two and three tier network structure, as different deployment models are used for different applications. For these leaders, a network equipment supplier that possesses product architecture flexibility, meaning an end-to-end product solution that accommodates tier two and three fabrics would be advantageous. This flexibility is found in product that supports layer 2 and layer 3 forwarding, as well as, a variety of line cards to offer design options.

A common network Operating System (OS) of products configured for two and three tier structure is important as IT operations gain efficiency to manage fabrics, as configuration and management are consistent. In addition, a common network OS offers rapid absorption of innovation to IT operations, as new OS features are available at the same time to all fabrics. The benefit of using a common product set to build tier two or three fabrics offers value around operational efficiency, training, sparing and ease of evolution between fabric deployments. In short, the network fabric needs to be simple and general purpose versus purpose built, which a common set of products creating tier two or three fabrics offer.

A Unified/Converged Fabric

The concept of a unified fabric is to virtualize data center resources and connect them through a high bandwidth network that is very scalable, high performance and enables the convergence of multiple protocols onto a single physical network. These IT resources are compute, storage and applications, which are connected via a network fabric. In short, the network is the unified fabric and the network is Ethernet.

The industry tends to focus on storage transport over Ethernet as the main concept behind a unified/converged fabric with technologies such as Fiber Channel over Ethernet or FCoE, iSCSI over Ethernet, iWARP over Ethernet and even Infiniband over Ethernet. But this is a narrow view of a unified/converged fabric which is being expanded, thanks to continual innovation of Ethernet by the vendor community and standards organizations such as the IEEE and IETF.

Ethernet innovations such as FCoE, Data Center Bridging or DCB, IETF’s Transparent Interconnection of Lots of Links or TRILL, CEE or Converged Enhanced Ethernet, link aggregation, IEEE’s 802.1AQ have enhanced Ethernet networking to support a wide range of new data center fabric design options. In addition to these protocol enhancements, the IEEE has ratified its work on defining 40Gb and 100Gb Ethernet, significantly increasing Ethernet’s ability to scale bandwidth. To demonstrate how Ethernet is evolving to be the unified fabric for high-end data centers, we explore Cisco’s new FabricPath Switching System innovation in this white paper.

The decision to implement a two or three tier network structure comes down to scale. For high-end data centers, a two-tier structure meets the requirements of low latency, movable workloads, scale, simplicity, etc. Many global 2000 concerns will have deployed both a two and three tier network fabric for their high end and less dense data centers.

When shopping for network equipment to construct two and three tier network fabrics, look for suppliers that support both rich Layer 3 routing services and scalable Layer 2 Ethernet capabilities to ensure choice and flexibility of three tier and scalable two tier fabric implementations. Such suppliers offer products that can be configured in multiple use cases and topologies where modules are inter-changeable, skills transferable and operations common between both fabric approaches.

But make no mistake about it, it’s a two-tier network fabric that IT business leaders and data center architects have gravitated toward for high performance computing, cloud scale data centers and just plain high end data centers of 5,000 and above servers.

Tweet

Networking has become “rigid”. Yes I know it’s almost absurd to attribute inflexibility or rigidity to networking. Look what TCP/IP has done for us. There are nearly 2 billion people connected to the internet and according to the Internet World Stats internet user growth rate increased by 380% between 2000-2009. With 2 billion people and growing online, accessing a plethora of applications via a wide range of end-points there is no doubt that the internet and TCP/IP has been a much bigger success than anyone would have imagined back in the early ’90s. But there’s always a give and take between computing and networking where one drives and changes the other. Right now we are in a compute innovation cycle that’s driving a fundamental change in networking which screams out the need for more flexibility.

Sure networking has increased from a bandwidth point of view and the IETF has added new protocols and network services, but it hasn’t kept up with compute innovation. As data centers pack more compute power and operating systems (OS) per physical server, thanks to virtualization, the need to move containers of OS plus applications and data around have sky rocked. In addition, traffic patterns have shifted tremendously as client-server or north-south flows are layered on top of server-server or east-west flows. And yes, there are new networking approaches being offered by vendors and standard organizations such as Cisco’s FlexPath, Juniper’s Stratus, Brocades VCS, Extreme’s Direct Attach, Force 10’s Open Automation, Arista’s Multi-Chassis Link Aggregation, BLADE’s Unified FabricArchitecture, the IETF’s TRILL and LISP and IEEE’s 802.1AQ, but these may be short term solutions to a much bigger networking problem.

Computing has always driven network design as mainframes drove SNA and analog multi-point wide area networks (WANs) during the ’70s. Mini-computers drove peer-to-peer networking protocols like DecNet, OSI and TCP/IP in the ’80s. Client-Server computing drove LANs and TCP into the mainstream in the early ’90s. The Web drove the internet in the 2000s and now server virtualization and cloud computing is once again changing fundamental networking requirements to make them more flexible.

The rigid label is a powerful one as it creates frustration by not addressing or enabling new business processes. Every time a network protocol or architecture was labeled as too rigid it was replaced and in the process a new market emerged on the scale of tens of billions of dollars. SNA was labeled as too rigid to support peer-to-peer networking. The T1 multiplexer market of the late ’80s and early ’90s was too rigid to support data traffic and thus routing replaced it. The PSTN and TDM were too rigid as they doled out bandwidth in 56Kbs chunks and were unable to support internet and VoIP traffic. The national entertainment network is rigid too as it doesn’t support two-way communications and it also will be replaced slowly but surely.

So where is networking not flexible enough? It’s in virtualized data centers. Some analyst groups estimate that 30% of workloads are virtualized and increasing. Since virtualization or a VM is the new atomic layer of data centers, networking is falling short in public as well as private clouds. Ideally, all resources (compute, storage, and networking) would be pooled, with services dynamically drawing from the pools to meet demand. Virtualization techniques have succeeded in enabling processes to be moved between machines, but constraints in the data center network continue to create barriers that prevent agility, for example, VLANs, ACLs, broadcast domains, Load Balancers, Firewall/IPS Security settings and service-specific network engineering.

The well understood problem is that when a VM is moved from one physical machine to another the network, load balancers, firewalls/IPS, broadcast domains, etc., have to be reconfigured. There is no automation in place, meaning that the network is not flexible or agile enough to make the changes required. Now this problem has scale to it as it’s a growing requirement of both IT executives managing corporate IT assets and service/cloud providers.

There are market solutions available today and more are coming that address “network automation” which enable the network to reconfigure itself as a VM and/or workload is moved within a data center. Cisco’s Nexus 1000V, HP Network Automation software and its Virtual Connect approach, Force 10’s Open Automation, Blade Network Technologies VMReady Network Virtualization, Arista Network’s Virtualized Extensible Operating System or vEOS and others are addressing the problem of network agility or lack thereof in virtualized environments.

But the problem gets bigger and more complex when distance and cloud provider entities become engaged. None of the solutions above address moving a VM from one physical server to another over large distance, be it around town, across state lines, across the country or the globe. Some are using IF-MAP as a registry, sort of like facebook for computers that publish their resources and use this information to automate network configuration to support large distance VM moves.

The problem gets larger yet when workloads move from a private cloud to a public cloud. (Definition note: There is no single definition of a workload, so for my purpose here I assume a container including a VM and associated applications and data that can be moved as simply as drag and drop or some other string of instructions). In short, all the software that is needed to compile and run an application for a set of users is a workload. The network inflexibility problem grows even larger when moving workloads between public clouds.

Now is this a real problem? You bet it is. Consider the value also of portable or mobile workloads to Enterprise and service providers. Workload mobility means capacity on demand, business continuance, and disaster recovery, etc. In addition, as IT leaders explore public and private cloud alternatives, they will want to move workloads from their data center to a provider’s and move the workload back when and if required. For reasons of security and trust, IT business leaders will demand mobility. For example, if your cloud provider goes bankrupt, then you will want to move your workload out quickly. If your cloud provider’s performance drops again then you could move your workload out. If your cloud provider is the target of a terrorist attack or is turned into a large botnet then you can move your workload out.

In addition to security and piece of mind, mobile workloads will fundamentally change IT delivery, capital structure and most importantly business models and processes. Once IT can move workload anywhere in their data center, across their data centers or to a provider they have tiered with, the question becomes when and how fast does IT move workload? If IT can perform all the provisioning in software and enable workload moves to occur transparently and safely with address, identity, security preservation, enabled trust, control and interoperability across providers, then the question is when does IT need to move workload? This level of mobility is an industry-wide initiative as it offers significant and material business value. Business value is created as IT could move workload in a follow- the-sun model, following the lowest cost per kilowatt-hour model; workload could move to avoid a disaster, or for capacity on demand, or for lowest cost of workload execution, etc.

So how can data center networks become more flexible? A key element of the solution is agility or the ability to dynamically grow and shrink resources to meet demand and to draw those resources from the most optimal location. Today, the network stands as a barrier to agility and increases the fragmentation of resources, which leads to low server utilization and prevents portable or mobile workloads.

Tweet

In Lippis Report 151: A Two or Three Tier High-End Data Center Ethernet Fabric Architecture? we detailed the new two tier data center Ethernet fabric that is becoming conventional wisdom amongst business leaders of high end data centers and cloud computing service providers. The networking industry is headed for a major innovation and competitive cycle fueled by a multi-billion dollar addressable market for data center network fabrics. Over the last eighteen months, every major Ethernet infrastructure provider has announced or taken a position on two tier network fabrics for high-end data centers. Companies such as Cisco, Arista Networks, Force10, Voltaire, HP/3Com, Juniper, Extreme, Brocade, BLADE Network Technology, et al have announced network fabrics for data centers with two thousand and more servers that either support storage enablement or not. In this Lippis Report Research Note, we review why it is Ethernet that will be the network fabric of high performance computing or HPC and cloud computing deployments.

For high-end data centers, HPC plus private and public cloud computing networks connecting thousands of servers, a new set of requirements have emerged. Low latency and high performance are the two driving requirements. Yes, there are more, especially when the fabric needs to enable converged storage, but let’s focus on latency and performance for now. Traditional three tier (server access, distribution and core) fabrics designed primarily for north-south traffic flows, that is client-server computing utilized spanning tree protocol (STP) and slower speed Ethernet (100Mbs to 1Gbs). Thanks to web 2.0, mash-ups and social networking sites east-to-west or server-server traffic flows have spiked requiring networks to support both north-south and east-west flows.

As most network engineers know, STP was designed to avoid loops that confused Ethernet as it was designed as a bus topology. STP shuts down redundant links between common switches to maintain the bus. Therefore, connecting access switches to distribution switches utilizing STP would require that network engineers over-subscribe the links between switches as only half of the bandwidth could be used. Oversubscription would also create blocking of packets between points too. To avoid this design, nearly every major switch manufacturer offered link aggregation that is the ability to shut off STP and aggregate links between switches. While this was and is a benefit, the down side has been that vendors only offered the ability to aggregate two links, which still drove oversubscription and blocking.

Recently, industry players such as Cisco and Arista Networks have offered the ability to scale up aggregation of links from 16 to 32, while at the same time delivering multipathing that allows packets to be forwarded across multiple links to arrive at its intended destination. Switch-processing capacity to support these massive inter-switch links have been increased too. These design changes, along with Ethernet’s innovation march, has ushered in the two-tier network design fabric option.

A two-tier fabric is designed with two kinds of switches; one that connects servers and the second that connect switches creating a non-blocking, low latency fabric. We use the terms ‘leaf’ switch to denote server connecting switches and ‘spine’ to denote switches that connect leaf switches. Together a leaf and spin architecture create the network fabric.

In late June 2010, Cisco announced its’ FabricPatch Switching System or FSS and its’ F-Series modules that support 32 ports of 10GbE of auto-sensing 1/10GbE and is essentially for server access and aggregation. FabricPath provides a new level of bandwidth scale to connect Nexus switches and delivers a new fabric design option with unique attributes for IT architects and designers. FabricPath is a NX-OS innovation, meaning that its’ capabilities are embedded within the NX-OS network OS for the data center. FabricPath essentially is multipath Ethernet; a scheme that provides high-throughput, reduced and more deterministic latency, and greater resiliency compared to traditional Ethernet.

FabricPath combines today’s layer 2 or Ethernet networking attributes and enhances it with layer 3 capabilities. In short, FabricPath brings some of the capabilities available in routing into a traditional switching context. For example, FabricPath offers the benefits of layer 2 switching such as low cost, easy configuration and workload flexibility. What this means is that when IT needs to move VMs and/or applications around the data center to different physical locations, it can do so in a simple and straightforward manner without requiring VLAN, IP address and other network reconfiguration. In essence, FabricPath delivers plug and play capability, which has been an early design attribute of Ethernet. Further, large broadcast domains and storms inherent in layer 2 networks that occurred during the mid 1990s have been mitigated with technologies such as VLAN pruning, Reverse Path Forwarding, Time-to-Live, etc.

The layer 3 capabilities added to FabricPath deliver scalable bandwidth allowing IT architects to build much larger layer 2 networks with very high cross-sectional bandwidth eliminating the need for oversubscription. In addition, FabricPath affords high availability as it eliminates STP, which only allows one path and blocks all others, and replaces it with multiple paths between endpoints within the data center. This offers increased redundancy as traffic has multiple paths in which to reach its final destination.

FabricPath employs routing techniques such as building a route table of different nodes in a network. It possesses a routing protocol, which calculates paths that packets can traverse through the network. What is being added to FabricPath is the ability for the control plane or the routing protocols to know the topology of the network and choose different routes for traffic to flow. Not only can FabricPath choose different routes, it can use multiple routes simultaneously so traffic can span across multiple routes at once. These layer 3 features enable FabricPath to use all links between switches to pass traffic as STP is no longer used and would shut down redundant links to eliminate loops. Therefore, this would yield incremental levels of resiliency and bandwidth capacity, which is paramount as compute and virtualization density continue to raise driving scale requirements up.

Designing A 160 Tbps Data Center Fabric

As an example to how multi link aggregation, the elimination of STP, high switching capacity and 10GbE connections create a highly scalable two-tier layer 2 Ethernet fabric, we use Cisco’s FSS and its’ F-Series module in the Nexus 7000. The following details the design of a 160 Tbps switching fabric with FabricPath and the F-Series module for high performance data centers using Cisco’s Nexus 7000 switches. This architecture can support over 8,000 servers connected at 10GbE or 4,000 servers dual homed at 10GbE with attributes of being non-blocking, low latency (5 microseconds), high bandwidth, reliability, plus simplicity of workload movement.

To build a 160 Tbps two-tier fabric, thirty-two Nexus 7018 switches populated with F-Series 10GbE modules would connect servers. These thirty switches are leaf switches. Each leaf chassis provides 256 10GbE ports to connect servers and another 256 10GbE ports to connect into spine switches. Therefore, each leaf is directly connected to each spine with sixteen FabricPath ports at 10GbE equaling a total of 256 10GbE ports for each leaf switch. There are sixteen spine switches each accepting 512 10GbE FabricPath ports. A single leaf chassis connects 256 10GbE ports into a spine equaling approximately 2.5Tbs. Multiplying each thirty-two leaf’s contribution into the fabric yields 80Tbs. As Ethernet is full-duplex, the total fabric switching capacity is 160
Tbps. Therefore, 160Tbps of switching fabric is available across all thirty-two leaf chassis. As 256 10GbE equals 2.5 Tbs, which also equals 16 FabricPath links to each one of sixteen spine switches, yields 2.5 Tbs, the fabric is non-blocking.

As for layer 2 and layer 3 forwarding, the job of the spine is to forward packets from leaf switches at layer 2, creating a single tier fabric. A key attribute of this architecture is that each 16-way FabricPath links are Equal Cost Multipathing or ECMP. What 16-way FabricPath ECMP provides are two benefits: 1) It delivers more paths for traffic to flow, which increases available bandwidth in the fabric and 2) as they’re distributed across all switches, diversity of routes is enabled to distribute packet forwarding. In essence what 16-way FabricPath ECMP provides is a very low latency, high bandwidth approach to supporting both north-to-south and east-to-west traffic flows simultaneously.

While the above is a Cisco deployment example Arista’s new 7500 series of Ethernet switches support 6 Billion packets per second at wire speed. The 7500s can be configured into a massive two-tier network fabric thanks to it support of 32 port MLAG (Multi-Chassis Link Aggregation) affording the connection of 18,000 to 30,000 servers.

Ethernet continues to evolve. The IEEE recently ratified the 40 and 100 GbE standard with vendors such as Force 10, Cisco, Arista, Extreme, BLADE, Brocade, Voltaire, HP et al announcing support and scheduling product delivery. While the above two-tier network example provides the perspective from the large switch provider, below is BLADE Network Technologies perspective, a company focused on server connectivity.

BLADE Network Technologies believes that as Ethernet delivers new levels of speed and intelligence, it will be the dominant two-tier network fabric for high-end next-generation data centers.
For many applications, low latency is a key requirement, and latency is an area where two-tier networks excel. Studies of stock trading exchanges have shown that tens of milliseconds of delay in data delivery can represent a ten percent drop in revenues, and delays of even five microseconds per trade can cost hundreds of thousands of dollars. Industry-specific requirements for uncompressed data and end-to-end deterministic latency within tens of microseconds make attaining such performance even more difficult. These factors have combined to make raw switching speed a top priority, and today’s best-of-breed 10 Gigabit Ethernet switches achieve can operate with under 700 nanoseconds of port-to-port latency while consuming a miniscule amount of power equivalent to that of standard light bulbs.

As next-generation networks get flatter – driven by latency and bandwidth requirements – emerging Layer 2 technologies such as the IETF’s Transparent Interconnection of Lots of Links or TRILL, enable this trend. The idea behind TRILL is to replace spanning tree as a mechanism to find loop free trees within Layer 2 broadcast domains. Using a routing protocol to build forwarding trees within a Layer 2 broadcast domain enables the flexibility and efficiency to route Layer 2 traffic, just like one would Layer 3 traffic, without the overhead associated with Layer 3 packet processing. TRILL will offer important features, such as support for both broadcast and multicast, load splitting along multiples paths, support for multiple points of attachment, and no tangible delay in service after attachment.

In the data center, bottlenecks are moving from the CPU and memory access to the I/O of the servers. Today’s multi-core servers are now able to sustain a great amount of traffic, requiring fast, flat networks, especially now that virtualization is widely deployed. Analysts have predicted that the 10G market will double year-to-year in 2010 and 2011. More servers using 10G increases the requirement for 40G and 100G in upstream networks. With 10G widely available and 40G coming online, Ethernet networks can enable data and storage traffic to use a single wire, using FCoE or iSCSI for example, and provide the raw speed that makes Ethernet with its economies of scale, to supplant InfiniBand for HPC requirements.

The reason Ethernet will be the network fabric for high-end data center networks is that the vendor community continues to innovate and build upon this protocol. Ethernet innovations are many and are beyond bandwidth increases from 10Mbs, 100Mbs, 1Gbs, 10Gbs, 40Gbs and 100Gbs, which are obvious. Link aggregation, multi-pathing and so much more propel Ethernet’s relevance and suitability to new challenging networking requirements.

Tweet

In the Lippis Report Research Note 150, we discussed the new industry group called Unified Communications Interoperability Forum or UNIF and compared it to other industry consortium charted to deliver interoperable solutions. While interoperability is sorely needed in the UC industry, it looks like Microsoft killed its changes of broad industry success before it started. What I hear from both UCIF members and non-members is that UCIF is controlled by Microsoft, and thus, lacks a large cross section of industry players as well as major UC providers. With its current structure, UCIF will make limited headway on its charter. In this Lippis Report Research Note, we review UCIF and its’ opportunities.

There is no doubt that the unified communications and collaboration industry needs interoperable solutions. Video traffic, in particular, is growing exponentially, which will not abate anytime soon. Driving growth is the new mobile video market with devices being equipped with real time video applications from companies such as Apple with its’ iPhone 4.0 FaceTime feature and Cisco’s Cius tablet. There is a real-time mobile video chat for Android too via the Movicha client application. In addition, every major UC supplier will launch a tablet based, end user device this year with tight links into its UC and video collaboration infrastructure. In short, the next generation office phone is a tablet. The combination of consumer and business mobile video device options will drive demand for interoperability, not only between mobile end points, but into corporate video conferencing systems too.

There needs to be a base line of interoperability standards for presence and call management also. Yes SIP or session initiation protocol does provide a base line, but many have built proprietary extensions minimizing interoperability options.

Now is a great time for an industry wide consortium of suppliers, service providers, IT executives and analysts to contribute to a set of interoperability standards with associated certification testing. Before UCIF was established Microsoft drove the initiative with limited to no input or invitation from its competitors. This approach has alienated nearly every major UC supplier from participating in UCIF, and therefore, don’t expect to see Cisco, Avaya, ShoreTel, Mitel, NEC et al to contribute. From this point of view, Microsoft killed UCIF before it even started.

But UCIF can make a contribution especially in the area of real time video collaboration between mobile, desktop and video conferencing system end points. For example, Microsoft could open up its’ Real Time Video (RTV) and Real Time Audio (RTA) codec protocols so that mixed vendor video endpoints can communicate with Office Communicator endpoints natively. With LifeSize, Polycom, HP and Microsoft being the UCIF founding members, their contribution to video collaboration interoperability could have a large impact on the real time video conferencing market.

For example, I use a LifeSize Express 220 video conferencing system, and as a standalone device that connects to other video conferencing systems via IP, H.323 or SIP, it’s magnificent. It would be great to connect with clients that have video enabled their desktop and mobile endpoints too. The larger the universe of potential video endpoints that one can connect to, the greater the value a real time video system provides. This would be a great charter for UCIF, which is to contribute open standards and certification testing that enable mobile, desktop and corporate video conferencing systems to interoperable.

However, for UCIF to deliver on its charter, it would have to dissolve and restart with Cisco, Avaya, Mitel, ShoreTel, and a larger role for Siemens, plus service providers, analysts and IT executives all being stake holders. You cannot have a closed group defining open standards. It just does not work that way.

Tweet

It hasn’t been since the mid 1990s that the networking industry was focused on multi-protocol integration or convergence. The industry is gearing up for a major innovation and competitive cycle fueled by the multi-billion dollar addressable market for data center network fabrics. Over the last eighteen months, every major Ethernet infrastructure provider has been talking about two and three tier network fabrics for high-end data centers. Companies such as Cisco, Arista Networks, HP/3Com, Force10, Voltaire, Extreme, Brocade, Juniper et al have announced network fabrics for data centers with five thousand and more servers with and without storage enablement. Juniper talks of a one-tier fabric through their Project Stratus work with IBM to be available some time in the future. Brocade recently introduced its’ Brocade One, which is a converged data center fabric. Cisco just launched its’ FabricPath Switching System or FSS for the Nexus 7000 that enables massive scale of a two-tier fabric. In this Lippis Report Research Note, we review the architectural attributes of two and three tier network fabrics and review FSS and its accompanying F-Series 10GbE module.

The IT industry is at an inflection point as service delivery is becoming more and more centralized thanks to data center consolidation, virtualization, cloud and mobile computing. It is estimated that a third of all IT spend is concentrated in the data center and this trend is only building thanks to favorable economics, motivating IT business leaders to centralize IT delivery.

The impact of this trend is more and more dense data centers made up of servers in the thousands to tens of thousands and higher. It is at the scale of 5,000 plus servers that a new network fabric is required for high-end data centers. High-end data center design is challenged with increasing complexity, the need for greater workload mobility and reduced energy consumption. Traffic patterns have also shifted significantly, from primarily client-server or as commonly referred to as north-to-south flows, to a combination of client-server and server-server or east-to-west plus north-to-south streams. These shifts have wreaked havoc on application response time and end user experience, since the network is not designed for these Brownian motion type flows.

The main requirements for high-end data center network fabric are low latency, large flat layer 2 domains to enable workload mobility, low power consumption, simplicity of design and significant bandwidth. Storage enablement, meaning consolidated I/O or virtualized I/O, is a growing priority and a new fabric that can support FiberChannel over Ethernet, iSCSI over Ethernet, iWARP over Ethernet or Infiniband over Ethernet is a major plus. One salient observation is that it’s pretty clear that Ethernet is the network fabric of choice as it is the only network protocol that enjoys continual innovation such as TRILL, Data Center Bridging, link aggregation, multi-pathing, and soon, 40 Gbs and 100 Gbs speeds. With the above requirements in mind, let us review data center network design options.

Two and Three Tier Fabrics

A three-tier network architecture is the dominant structure in data centers today and will likely continue as the optimal design for many networks. For most network architects and administrators, this type of design provides the best balance of asset utilization, layer 3 routing for segmentation, scaling and services, plus efficient physical design for cabling and fiber runs. By three tiers, we mean access switches/Top-of-Rack (ToR) switches, or modular/End-of-Row (EoR) switches that connect to servers and IP based storage. These access switches are connected via Ethernet to aggregation switches. The aggregation switches are connected into a set of core switches or routers that forward traffic flows from servers to an intranet and internet, and between the aggregation switches. It’s common in this structure to over-subscribe bandwidth in the access tier, and to a lesser degree, in the aggregation tier, which can increase latency and reduce performance. Inherent in this structure is the placement of layer 2 versus layer 3 forwarding that is Virtual Local Area Networking or VLANs and IP routing. Also common, is that VLANs are constructed within access and aggregation switches, while layer 3 capabilities in the aggregation or core switches route between them.

But within the high-end data center market, where the number of servers is in the thousands to tens of thousands plus and east-west bandwidth is significant, is where a new structure is needed. It is within these data centers where applications need a single layer 2 domain.

Two-tiers of network fabric

A two-tier fabric is designed with two kinds of switches: one that connects servers, and the second that connect switches creating a non-blocking, low latency fabric. In short, there are server facing and fabric facing switches. We use the terms ‘leaf’ switch to denote server facing or connecting switches and ‘spine’ to denote fabric facing or switches that connect leaf switches into the fabric. Together, a leaf and spine architecture create the fabric.

Many IT leaders in Global 2000 firms will have deployed both two and three tier network structure, as different deployment models are used for different applications. For these leaders, a network equipment supplier is needed that possesses product architecture flexibility, meaning an end-to-end product solution that accommodates tier two and three fabrics. This flexibility is found in product that supports layer 2 and layer 3 forwarding, as well as, a variety of line cards to offer design options.

A common network Operating System (OS) of products configured for two and three tier structure is important as IT operations gain efficiency to manage fabrics, as configuration and management are consistent. In addition, a common network OS offers rapid absorption of innovation to IT operations, as new OS features are available at the same time to all fabrics. The benefit of using a common product set to build tier two or three fabrics offers value around operational efficiency, training, sparing and ease of evolution between fabric deployments. In short, the network fabric needs to be simple and general purpose versus purpose built, which a common set of products creating tier two or three fabrics offer. This type of flexibility will enable IT leaders to address the challenges of scale outlined above.

In addition to product flexibility, some networking suppliers take a systems approach to their fabric design, meaning that a solution is built and pre-tested before it arrives on site. This ensures that IT does not have to perform system integration. With the increased concentration of computing and IT dollars into data centers, it’s only obvious that data centers are long-term corporate commitments. Therefore, it is only appropriate that the networking supplier of choice also has a proven long-term commitment to their product architecture.

Perhaps the best example of this is Cisco’s Catalyst 6000 switching architecture and its’ two-year-old Nexus product line. The Catalyst investment protection is well documented as it has been in operation for over a decade, which Cisco customers enjoy continued innovation and value added to this platform. Competitors view its’ longevity as a weakness. The Nexus product line has a similar investment protection philosophy with a fifteen-year plus lifespan expectation. Common to both Catalyst and Nexus is the fact that these products are built on silicon, developed at Cisco, affording investment protection from one generation of the hardware to the next.

A Unified Fabric

The concept of a unified fabric is to virtualize data center resources and connect them through a high bandwidth network that is very scalable, high performance and enables the convergence of multiple protocols onto a single physical network. These IT resources are compute, storage and applications, which are connected via a network fabric. In short, the network is the unified fabric and the network is Ethernet.

The industry tends to focus on storage transport over Ethernet as the main concept behind a unified fabric with technologies such as Fiber Channel over Ethernet or FCoE, iSCSI over Ethernet, iWARP over Ethernet and even Infiniband over Ethernet. But this is a narrow view of a unified fabric, which is being expanded thanks to continual innovation of Ethernet by the vendor community and standards organizations such as the IEEE and IETF. Ethernet innovations such as FCoE, Data Center Bridging or DCB, link aggregation, Cisco’s VN-Link, FEX-Link and virtual PortChannel or vPC have enhanced Ethernet networking to support a wide range of new data center fabric design options. In addition to these protocol enhancements, the IEEE is scheduled to complete its’ work on defining 40Gb and 100Gb Ethernet during the summer of 2010, significantly increasing Ethernet’s ability to scale bandwidth. To demonstrate how Ethernet is evolving to be the unified fabric for high-end data centers, we explore Cisco’s new FabricPath Switching System innovation in this white paper.

The decision to implement a two or three tier network structure comes down to scale. For high-end data centers, a two-tier structure meets the requirements of low latency, movable workloads, scale, simplicity, etc. Many global 2000 concerns will have deployed both a two and three tier network fabric for their high end and less dense data centers.

When shopping for network equipment to construct two and three tier network fabrics, look for suppliers that support both rich Layer 3 routing services and scalable Layer 2 Ethernet capabilities to ensure choice and flexibility of three tier and scalable two tier fabric implementations. Such suppliers offer products that can be configured in multiple use cases and topologies where modules are inter-changeable, skills transferable and operations common between both fabric approaches.

Tweet

Unified communications is becoming an integration point for traditional dial tone, FAX, video and messaging as it adds increased access and functionality to these important forms of communications. Sagemcom is by far the industry leader in IP FAX, having successfully integrated FAX capability into Cisco, Avaya, Alcatel-Lucent and Microsoft’s UC platforms.  Its’ XmediusFAX has proven to reduced cost, increase access and security of FAX communications.  In this Lippis Report podcast, I talk with Bob Wood, Executive Director at Sagemcom, about how FAX communications can be integrated into the UC platform of your choice and the business outcomes it delivers.

Listen to the Podcast

Tweet

In mid May of this year HP, Juniper Networks, Microsoft, Logitech / LifeSize and Polycom established a forum to develop a set of interoperability test methodologies and certification programs along with specifications and guidelines that enable mixed vendor Unified Communications UC solutions to work with each other. In short, the UC Interoperability Forum or UCIF is trying to define what it means for multi-vendor UC implementations to interoperate. Since its establishment, membership has grown by thirteen vendors, but blaringly obvious is the omission of Cisco, Avaya, Mitel, ShoreTel and other major UC providers. This begs the question of motivation. Is the UCIF interested in interoperability or changing the market landscape to gain advantage on the established leaders? In this Lippis Report Research Note we explore this question.

UC interoperability is a very big deal. In fact, back in early April of this year, Zeus Kerravala, SVP of the Yankee Group and I addressed this issue in a Lippis Report podcast titled What is Holding UC Back?. Our answer was lack of interoperability standards and the vendor community’s minimal interest of embracing the ones we have. The UC market has evolved in a peculiar way as it brings together traditional voice communication companies, data networking firms, computing corporations and software concerns. UC is now at the epicenter of video communications, social networking and mobile computing too. UC represents one of the largest cross sections of disparate markets second only to the Internet. It’s here, within this cross section, that UC gains its enormous value.

UC offers to control real time communications and collaboration. Put another way, all real time business processes will be accessed and control by UC over time. Need to call a colleague? It’s via your UC client. Need to schedule a meeting? It’s via your UC calendar client. Need to video chat with a customer? It’s via your UC video client. Need to bring a group of people together for an emergency meeting? Yes, you guessed it! It is via your UC collaboration client. And common to all those UC clients is presence enabled directory to you, so you can find someone and know if they are available, a communications management system that sets up and tears down connections over intranet, internet and mobile nets. To make UC work ubiquitously, like the public telephone network or the Internet, the vendor community needs a forum or place where it can work out interoperability standards. In addition, for this next evolution in human communications to live up to its promise, it needs motivated vendors to allow their equipment to work together.

Yes, UC does have key interoperability standards such as SIP or Session Initiation Protocol that offer both end-point and communications manager interoperability, but many vendors add proprietary extensions to SIP reducing its value in multi-vendor networks. So the UCIF is to be applauded for taking the first step in creating an organization among the vendor community to usher in an era of interoperable UC. But the problem with UCIF is which companies established its formation. Clearly suppliers are businesses looking for sustainable competitive advantage that comes with large market share and innovative, albeit proprietary technologies. It’s no surprise then that when UCIF is established by firms with limited UC market share one’s mind jumps to the obvious assumption that the founding members of the UCIF are perhaps more interested in market share re-distribution than interoperability.

I’ve observed many industry forums and consortiums in the past that used interoperability as a convenient cause to hide a group’s true intentions. For example, Bay Networks, 3Com and IBM established the Network Interoperability Alliance or NIA in May of 1996 to foster interoperability between Local Area Network (LAN) switch vendors. NIA had limited success in competing with Cisco’s increasing market share gains of the enterprise router and switch market.

UCIF feels a lot like NIA to me. The shear fact that it’s mission statement, board and legal structure was done without any of the UC market leaders input and participation is unfortunate, as it has alienated them. It’s also unfortunate that Polycom and LifeSize are founding UCIF partners, but Cisco/Tandberg is not involved as this has a hint of Polycom/LifeSize fear of Cisco breaking away with the Telepresence market; UCIF seems like a way of mitigating this threat. The timing is very close with Cisco closing the Tandberg acquisition in April and UCIF being launched in May.

If UCIF is not able to entice and recruit Cisco, Avaya, Mitel, and ShoreTel et al in a meaningful and authoritative way, then its fate may very well be the same as NIA. What the industry does need is true interoperability standards so that a Cisco, Avaya, Microsoft, Siemens, HP et al UC implementations are able to work with each other in the same way that multi-vendor email systems work with each other. But without full industry participation, it seems that UCIF may be doomed and not able to deliver on its promise of interoperability. For UCIF to be meaningful it needs the UC market leaders full participation as well as Enterprise IT architects and planners plus service providers too, for without them, UCIF is NIA.

Tweet

In Lippis Report 148 we reviewed the major drivers and trends that are propelling the high-end data center Ethernet switch market to well over a $1B annual run rate. In this Lippis Report Research Note, we review the major suppliers of these switches. We review Cisco, Arista Networks Force10 Networks, BLADE Network Technologies, HP/3Com/H3C, Voltaire, Avaya, Brocade, and Juniper and identify their unique positions and offerings to participants in the burgeoning market. Our focus is the high-end, high density 10GbE switches that are enabling virtualized cloud computing data centers thanks to Terabits per second of back plane switching capacity, billions of packets per second of layer 2/3 forwarding, hundreds of 10GbE port connectivity per chassis, a new two-tier architecture, microsecond level latency, low power consumption, non-stop operation and software hooks that eliminate network barriers to large scale server virtualization. The engineering in these switches should be celebrated, as they represent the state-of-the-art in computer and network design. In short, they represent the fundamental building block of a new generation of IT delivery based upon cloud computing and virtualization. This Research Note is a must read for any IT executive designing a data center.

After finishing this Research Note, it became evident that this market needs a set of industry neural 10GbE switch test to independently verify vendor claims. We hope to make such a contribution this Fall.

Cisco Systems Nexus Family of Switches

Cisco’s approach to data center Ethernet switching is rooted in its Data Center 3.0 strategy which seeks to scale server virtualization while introducing a platform to enable a unified fabric or converged network and storage running on one physical Ethernet network. Cisco’s data center Ethernet switch portfolio is primarily the Nexus family of switches including the 7000, 5000, 2000 and 1000v. NX-OS is a purpose built data center operating system that runs across the entire Nexus family. NX-OS integrates a number of higher system availability functionalities such as virtual port- channel (vPC), and the capability to upgrade software without disrupting traffic. The Nexus 1000v is a softswitch that resides in a VM hypervisor. The Nexus 1000v’s main job is to eliminate network configuration barriers that exist when moving a VM from one physical machine to another. To accomplish this, the 1000v creates a port profile including VLAN, ACL, policy, security, etc. with persistence, which moves with a VM as a virtualization administer moves a VM from one physical machine to another.

The Nexus 2000 family of Fabric Extenders (FEX) introduces the concept of a remote line card of the parent Nexus 5000 switches and sits on the top-of-rack connecting servers to the switch fabric. The extender concept allows the 2000 and 5000 to be managed as one switch. This configuration reduces cabling requirements and offers an economical approach to server connection, thus providing the benefits of both end-of-row and top-of-rack deployments. The Nexus 5000 Series is 10 Gb Ethernet and Unified Fabric capable switches, connecting Nexus 2000s and servers directly at 100/1/10GbE/FCoE, while providing layer 2 forwarding. Providing layer 3 forwarding, dense 1/10GbE connectivity is the Nexus 7000 Series. The Nexus 7000 Series is available in a 10 and 18 slot chassis and is Cisco’s flagship data center Ethernet switch series. As a point of reference, the Nexus 7000 is now on an annualized run rate of $1B for Cisco, which is more than 10 times greater than any other switch supplier in the data center switch market. The high end 7000 connects 512 10GbE ports with 128 line-rate 10 Gigabit Ethernet ports. The Nexus 7000 Series switches can be segmented into virtual devices, delivering true segmentation of network traffic, context-level fault isolation, and management through the creation of independent hardware and software partitions. Overlay Virtualization Transport (OTV) provides customers a simplified DCI solution by extending layer 2 VLANs over existing IP networks. We have profiled the Nexus 7000 when first released and is available here. The Nexus switches can create a two-tier architecture with the 2000/5000, providing server connectivity and layer 2 forwarding between servers. The Nexus 7000 connects the 2000/5000 to each other and the internet/intranet with high density, high reliability layer 2/3 forwarding.

Arista Networks 7500 Family of Modular Switches

Arista Networks is a new comer to the data center Ethernet market, but its management team is seasoned and customer base growing. It provides six fixed 10GbE switches; five 1/10GbE 7100 and the 1GbE 7048 along with the new Best of Interop awarding winning 7500 modular switch. The 7100/7048 switches connect servers in a Top-of-Rack configuration while the 7500 aggregates these switches and connects them to the internet and intranet. This is a two-tier, “leaf-spine” architecture. The 7500 boasts ultra high performance layer 2/3 1/10 Gb Ethernet switching for high performance computing and cloud computing data centers. The 7500 supports 384 10GbE ports, 5.7Bpps at layer 2 or 3, high packet buffers 18GB deep, ultra low port-port latency of 4.5 microseconds and 10Terabit loss less switch fabric connecting modules.

The 7500 is 10GbE port dense, compact, cloud spec fast, green and prepared for 40 and 100GbE, with a price tag 50% below competitive offerings, according to Arista. While the 7500’s hardware architecture is impressive, its operating system EOS, Extensible Operating System, offers another set of uniqueness. For example, all Arista switches run the same binary image of EOS, easing administration while hastening switch feature upgrades. EOS is a modular OS that allows partners to run their software in the Arista switch, consolidating the number of management and network appliances required, thus increasing performance while reducing energy consumption and physical space. Arista’s EOS modularity was designed as a unique state sharing architecture that separates switch state from protocol processing and application logic. EOS is built on top of a standard Linux kernel. All EOS processes run in their own protected memory space and exchange state through an in-memory database. This multi-process state sharing architecture provides the foundation for in-service-software updates and self-healing resiliency. You can listen to a podcast interview with Douglas Gourlay, VP Marketing and Anshul Sadana, VP Customer & Systems Engineering from Arista on the introduction of the 7500 Series of Ethernet switches here

HP/3Com/H3C’s A12500 Core Data Center Switches

HP has spent 25 years building and selling networking products to its worldwide client base and is currently #2 in the market, with a 21% port count share and the fastest growing networking company in the industry. The combined HP/3COM acquisition brings core switching products, the #1 market share position in China, TippingPoint Intrusion Prevention System and ProCurve edge switches, representing a new choice for clients who are frustrated by today’s current offerings. HP will combine these two entities and operate under the banner of “HP Networking.”

The HP Converged Infrastructure Architecture and FlexFabric blueprint approach the modern data center with a vision that places networking at the center of an integrated data center solution and accelerates deployment of enterprise services and applications. It is designed to drive simplicity through streamlined network designs and centralized management, enhance agility with high performance security, and accelerated provisioning, and reduce cost with energy efficiency and low total cost of ownership. Central to HP FlexFabric is policy-driven network provisioning tightly integrated with server and storage management in an end-to-end data center converged infrastructure.

HP data center solutions are purpose built, using the latest advanced systems and ASIC technologies. “A” family data center networking platforms leverage a common operating system, Comware™ and are managed with a single-pane manager, Intelligent Management Center (IMC). HP switches make use of an HP-developed technology – Intelligent Resilient Framework (IRF) – to create a resilient virtual switching fabric. IRF delivers geographic independence, distributed high-availability, resiliency and millisecond re-convergence across layer 2 and layer 3 protocols. These innovations allow customers to build a simplified, high performing, highly resilient and flat (two-tier) data center network design. They overcome the limitations of low performance/scale, high cost/latency inherent in legacy solutions, which rely on multi-tier network designs, disjointed platform operating systems and complex resiliency protocols.

A key enabler of this transformational design flexibly is the HP next-generation data center switching architecture. This starts with the flagship HP A12500 core data center switch – which is based on a 100G design that uses a multi-level, multi-plane, non-blocking switching architecture to provide high performance and scalability. The A12500 supports 6.66 Tbps of high-performance switching capacity (future support for 13.32 Tbps) and scales to 2.2 billion packets per second of forwarding performance. The A12518 supports 512 10 Gigabit Ethernet or 864 Gigabit Ethernet ports in a single chassis. Its future-proof design accommodates 40/100 Gigabit Ethernet and emerging unified network requirements such as end-to-end FCoE/Data Center Ethernet.

Force10 Networks ExaScale E Series

Force10 Networks was one, if not the first company to offer 1 and 10Gb switching solutions for high-performance computing and data center markets in Fortune 100 companies, Internet portals, global carriers, leading research laboratories and government organizations. It offers a wide range of Ethernet switching and routing products that deliver high port density and resiliency to help customers deploy a high-availability, agile and standards-based GbE and 10 GbE network fabric, while reducing power and cooling costs. Its Ethernet switching products are designed to leverage virtualized data center environments and automate Ethernet networking. For example, its VirtualScale enables management of virtual chassis. Its VirtualControl enables virtualizing logical switching and routing boundaries. For automation, Force10 has developed an architecture, which automates network resource allocation as applications and services spin up and down. This architecture is built upon its HyperLink and SwitchLink technology, two new software features implemented within its Force10 Operating System (FTOS). HyperLink provides real-time communication between Force10 switches and hypervisors or virtual switches to enable automatic provisioning of one or many virtual LANs (VLANs) across multiple switches simultaneously. The SwitchLink feature provides real-time communication with middleware orchestration tools to enable automatic provisioning and management of virtual devices anywhere in the network.

Force10’s modular Ethernet switch data center product portfolio includes the ExaScale E-Series, optimized for core deployments in large-scale, high-performance 10GbE data centers, and the C-Series, optimized for mid-range data centers. Both the E-Series and C-Series come in multiple form factors, run FTOS and are dense high performance switching platforms equipped with redundancy, availability, fault-tolerant operations and many line card options. In addition, Force10 offers the fixed configuration S-Series product line for GbE and 10 GbE ToR configurations. Force10 promotes a vision of simplified data center topologies, using integrated switching and routing in the core, using chassis based E-Series or C-Series products, and fixed configuration ToR access products allowing both 1 tier and 2 tier designs. One tier can be achieved with high density E-Series platform for server aggregation, switching at the server edge, and routing off the same platform to the Internet / WAN. The two-tier architecture can be achieved leveraging ToR switching for server aggregation along with Force10’s chassis based systems in the core. In addition to a large direct sales force, IBM OEM’s Force10’s ExaScale platform as part of IBM’s iDataPlex clustering solution. You can listen to a podcast interview with Steve Garrison, VP Marketing of Force10 on their 40 GbE offering here.

BLADE Network Technologies RackSwitch Family of Ethernet Switches

BLADE Network Technologies (BNT) has been working in the data center switch market since 2006 with much success providing 1/10Gb Ethernet switches for blade servers and top-of-rack configurations. BLADE was launched from Nortel and made up of the successful Alteon Networks group. Their success stems from their ability to identify the top-of-rack and blade switch market in ’06, along with an OEM go to market strategy that included all of the top tier blade server providers such as HP, IBM and NEC. The result is that BLADE has shipped over 8m ports, achieved 25% growth from 2008 to 2009 (in a down economy), owns 50+ % of the blade switch market, is number 3 in the Fixed 10GbE market according to Dell’Oro Group, and has demonstrated scale with at least one customer installing over 16,000 of its switches.

BLADE offers the RackSwitch family of Ethernet switches, which are ToR, 1U high switches. They include the 24-port 360ns latency RackSwitch G8100 10GbE, 48-port RackSwitch G8000 1/10 GbE aggregation and the 24-port 700ns latency RackSwitch G8124 10GbE. Over a year ago, BLADE released its virtualization software called VMready that automates network settings for VM movement ensuring that network settings migrate when a VM is moved from one physical server to another. VMready scales to a 1000 virtual port switch, is based on standards and works with most popular hypervisors.

In addition to VMready, RackSwitch’s unique attributes are found in the fact that they were designed for the data center versus being a wiring closet switch re-formatted for the data center. For example, the RackSwitch BLADEOS supports CEE for unified fabrics, uplink failure detection, virtualization, dual homing for servers, low (80-170Watts) power consumption, back-to-front or front-to-back airflow and very low latency in the 700-360 nanosecond range.

Voltaire’s Vantage 8500

Voltaire has a long history in high performance computing and data center networking as it is one of the key leaders in the InfiniBand market. Voltaire enjoys distribution relationships with HP and IBM, as well as Bull, Fujitsu, NEC, SGI and Oracle. The result is a 100% + year over year revenue growth for Q1 as reported on May 5th. Last October, Voltaire entered the 10 GbE market with the introduction of its Vantage 8500 Ethernet layer 2-core switch. The Vantage 8500 boasts less than 1 microsecond of latency, a low 10 watts per port power consumption and 288 wire speed 10GbE ports in a 15U high chassis. The Vantage 8500’s unique industry contribution is that it’s based on converged enhanced Ethernet (CEE) technology providing InfiniBand-like capabilities to the Ethernet data center. In fact, Voltaire has ported many of InfiniBand’s key characteristics to the Vantage 8500 such as a lossless switching fabric, multi-pathing, virtualization, fabric-wide congestion management and QoS.

From a network design point of view, Voltaire supports a two tier network architecture that enables a simplified, ‘flat’ data center network and puts an end to the era of the over-provisioned network. Voltaire’s design centered on the Vantage 8500 is to support a two-tier data center network that scales from hundreds to a few thousand core ports, which requires high capacity, non-blocking 10 Gigabit Ethernet core switches. By clustering up to twelve Vantage 8500 switches together, IT business leaders can expand their data center to many thousands of servers while preserving the efficiency and price-per-port, without degrading performance or latency which occurs in traditional hierarchical network designs. To support ToR implementations, Voltaire and BLADE Network Technologies announced recently a partnership where BLADE ToR RackSwitches are aggregated by Voltaire’s Vantage 8500, rounding out the two-tier data center Ethernet network architecture.

The Vantage 8500 also features software-based capabilities to address virtualized and converged data center environments. Voltaire’s Unified Fabric Manager™ (UFM) software, application acceleration software and management OS (VT-OS) provide management and performance enhancement tools. These tools were developed and optimized in InfiniBand environments and are now available for Ethernet-based data centers. Voltaire’s recently introduced Unified Fabric Manager™ (UFM™) 3.0 software orchestrates physical and virtual switches delivering guaranteed levels of service per application. It’s the first and only Ethernet fabric management software that dynamically orchestrates end-to-end virtual machine connectivity for multi-vendor, scale-out data center networks.

Avaya’s VSP 9000

During the April 2009 Las Vegas Interop trade show, Nortel committed to the data center Ethernet market with the announcement of its Virtual Services Platform or VSP 9000 switch, which supports up to 27 Terabits per second (Tbps) of backplane switching and 240 10GbE ports per chassis at first release. Avaya announced their commitment to the VSP 9000 and said that it will be generally available in the second half of 2010 while already in controlled availability. The VSP 9000 is built upon the Ethernet Routing Switch 8600/8800 software providing a proven software foundation, mid-plane architecture, a fully programmable network processor unit for flexible data forwarding and carrier-grade Linux.

The VSP 9000 is designed to deliver high-density 10GbE, 40GbE and 100GbE. Its design center is rooted in highly dense connectivity environments that are all mission critical, by definition. Early testing validation of the VSP 9000 promises to provide ultra-high reliability and availability delivering below 50ms failover support, which is critical to eliminate application disruption thanks to its patented hardware failure detection differentiation. The VSP 9000 switch fabrics are lossless Ethernet capable and therefore well positioned to support the next generation Data Center requirements for convergence of storage onto the Ethernet infrastructure.

The VSP 9000’s unique network architecture is found in its ability to cluster four switches together, in that the total architecture exceeds 100 Tbs, with the number of 10GbE ports per rack being up to 720. Avaya continues to invest in Switch Clustering technology (Active/Active resiliency model) such as SMLT (split multi-link trunking) and RSMLT (routed-SMLT), which provides link, switch and router redundancy mechanisms. Three modules are being introduced in the first VSP 9000 release, a 24 port SFP+ for 1 GbE and 10 GbE connectivity, a 48-port of SFP module in addition to a 48-port 10/100/1000 TX module. Future plans include 40GbE and 100GbE interfaces, and even higher-capacity Switch Fabric modules.

Juniper Networks’s EX8200 & EX4500

In January of 2008, Juniper Networks launched its much-anticipated entry into the enterprise Ethernet switch market. Juniper’s focus is on the enterprise data center, campus and branch, as well as the service provider market. Juniper provides a suite of Ethernet switch products, including the EX4200 with Virtual Chassis technology for GbE Top-of-Rack (ToR) and End-of-Row (EoR) data center access, the EX2500 24-port and new EX4500 48-port 10GbE ToR switches, and the EX8200 high-density, high-performance line of modular Ethernet switches.

According to Juniper, it simplifies customer enterprise LAN architectures and advances the economics of networking via its most recently launched initiative called the “new network” for data centers. Juniper’s “new network” promises critical innovations in automation, virtualization and fabric technologies. These innovations are to reduce time to operation by up to 50 percent and eliminate up to 35 percent of data center networking capital expenditures. One aspect of the “new network” is a simplified two-tier network architecture, which may be reduced to one when “Project Stratus” is completed with IBM. The reduction of a three-tier architecture to two is accomplished by utilizing Juniper’s Virtual Chassis fabric technology in the access layer, in conjunction with its high-density, high-performance platforms such as EX8200 and EX4500 in the LAN core, thus eliminating the aggregation or distribution layer. According to Juniper, collapsing the distribution layer reduces complexity in the data center as well as campus networks by reducing the number of managed devices by up to 89%, providing up to 39% savings in space, 44% savings in power and reducing the number of switch interactions by up to 99% compared to three-layer networks. According to Juniper, this approach improves application performance by also reducing latency up to 77% compared to three-layer networks. Note that these claims and numbers are Juniper’s and not mine.

At the core of Juniper’s data center Ethernet product family is the EX8200 line of modular switches. The EX8208 and EX8216 are eight and sixteen-slot modular switches. The EX8216 sports a maximum of 640 10GbE ports and 1.92Bpps and 6.2Tbps backplane speed. The EX8200 is said to support 40GbE and 100GbE interfaces in the future. The EX8200s connect either EX4200 GbE or EX2500 and EX4500 10GbE ToR switches together while providing access to internet/intranet. All Juniper switches run Junos, the network operating system that provides reliability and availability features, developed for the high-performance enterprise and service provider market.

Brocade’s NetIron MLX Series of Switches

In July of 2008, Brocade had purchased Foundry Networks, catapulting them into the Ethernet switch market as one of the top five Ethernet switch/router vendors by revenue. Brocade, with its long history of data center storage, saw that converged I/O was going to happen and prepared the company to participate in this market. At the high end of Brocade’s data center Ethernet switch products is the NetIron MLX-4, MLX-8, MLX-16 and MLX-32 routers, which support 4, 8, 16 and 32 I/O module slots, respectively. We’ll focus on the h