In Lippis Report 151: A Two or Three Tier High-End Data Center Ethernet Fabric Architecture? we detailed the new two tier data center Ethernet fabric that is becoming conventional wisdom amongst business leaders of high end data centers and cloud computing service providers. The networking industry is headed for a major innovation and competitive cycle fueled by a multi-billion dollar addressable market for data center network fabrics. Over the last eighteen months, every major Ethernet infrastructure provider has announced or taken a position on two tier network fabrics for high-end data centers. Companies such as Cisco, Arista Networks, Force10, Voltaire, HP/3Com, Juniper, Extreme, Brocade, BLADE Network Technology, et al have announced network fabrics for data centers with two thousand and more servers that either support storage enablement or not. In this Lippis Report Research Note, we review why it is Ethernet that will be the network fabric of high performance computing or HPC and cloud computing deployments.

For high-end data centers, HPC plus private and public cloud computing networks connecting thousands of servers, a new set of requirements have emerged. Low latency and high performance are the two driving requirements. Yes, there are more, especially when the fabric needs to enable converged storage, but let’s focus on latency and performance for now. Traditional three tier (server access, distribution and core) fabrics designed primarily for north-south traffic flows, that is client-server computing utilized spanning tree protocol (STP) and slower speed Ethernet (100Mbs to 1Gbs). Thanks to web 2.0, mash-ups and social networking sites east-to-west or server-server traffic flows have spiked requiring networks to support both north-south and east-west flows.

As most network engineers know, STP was designed to avoid loops that confused Ethernet as it was designed as a bus topology. STP shuts down redundant links between common switches to maintain the bus. Therefore, connecting access switches to distribution switches utilizing STP would require that network engineers over-subscribe the links between switches as only half of the bandwidth could be used. Oversubscription would also create blocking of packets between points too. To avoid this design, nearly every major switch manufacturer offered link aggregation that is the ability to shut off STP and aggregate links between switches. While this was and is a benefit, the down side has been that vendors only offered the ability to aggregate two links, which still drove oversubscription and blocking.

Recently, industry players such as Cisco and Arista Networks have offered the ability to scale up aggregation of links from 16 to 32, while at the same time delivering multipathing that allows packets to be forwarded across multiple links to arrive at its intended destination. Switch-processing capacity to support these massive inter-switch links have been increased too. These design changes, along with Ethernet’s innovation march, has ushered in the two-tier network design fabric option.

A two-tier fabric is designed with two kinds of switches; one that connects servers and the second that connect switches creating a non-blocking, low latency fabric. We use the terms ‘leaf’ switch to denote server connecting switches and ‘spine’ to denote switches that connect leaf switches. Together a leaf and spin architecture create the network fabric.

In late June 2010, Cisco announced its’ FabricPatch Switching System or FSS and its’ F-Series modules that support 32 ports of 10GbE of auto-sensing 1/10GbE and is essentially for server access and aggregation. FabricPath provides a new level of bandwidth scale to connect Nexus switches and delivers a new fabric design option with unique attributes for IT architects and designers. FabricPath is a NX-OS innovation, meaning that its’ capabilities are embedded within the NX-OS network OS for the data center. FabricPath essentially is multipath Ethernet; a scheme that provides high-throughput, reduced and more deterministic latency, and greater resiliency compared to traditional Ethernet.

FabricPath combines today’s layer 2 or Ethernet networking attributes and enhances it with layer 3 capabilities. In short, FabricPath brings some of the capabilities available in routing into a traditional switching context. For example, FabricPath offers the benefits of layer 2 switching such as low cost, easy configuration and workload flexibility. What this means is that when IT needs to move VMs and/or applications around the data center to different physical locations, it can do so in a simple and straightforward manner without requiring VLAN, IP address and other network reconfiguration. In essence, FabricPath delivers plug and play capability, which has been an early design attribute of Ethernet. Further, large broadcast domains and storms inherent in layer 2 networks that occurred during the mid 1990s have been mitigated with technologies such as VLAN pruning, Reverse Path Forwarding, Time-to-Live, etc.

The layer 3 capabilities added to FabricPath deliver scalable bandwidth allowing IT architects to build much larger layer 2 networks with very high cross-sectional bandwidth eliminating the need for oversubscription. In addition, FabricPath affords high availability as it eliminates STP, which only allows one path and blocks all others, and replaces it with multiple paths between endpoints within the data center. This offers increased redundancy as traffic has multiple paths in which to reach its final destination.

FabricPath employs routing techniques such as building a route table of different nodes in a network. It possesses a routing protocol, which calculates paths that packets can traverse through the network. What is being added to FabricPath is the ability for the control plane or the routing protocols to know the topology of the network and choose different routes for traffic to flow. Not only can FabricPath choose different routes, it can use multiple routes simultaneously so traffic can span across multiple routes at once. These layer 3 features enable FabricPath to use all links between switches to pass traffic as STP is no longer used and would shut down redundant links to eliminate loops. Therefore, this would yield incremental levels of resiliency and bandwidth capacity, which is paramount as compute and virtualization density continue to raise driving scale requirements up.

Designing A 160 Tbps Data Center Fabric

As an example to how multi link aggregation, the elimination of STP, high switching capacity and 10GbE connections create a highly scalable two-tier layer 2 Ethernet fabric, we use Cisco’s FSS and its’ F-Series module in the Nexus 7000. The following details the design of a 160 Tbps switching fabric with FabricPath and the F-Series module for high performance data centers using Cisco’s Nexus 7000 switches. This architecture can support over 8,000 servers connected at 10GbE or 4,000 servers dual homed at 10GbE with attributes of being non-blocking, low latency (5 microseconds), high bandwidth, reliability, plus simplicity of workload movement.

To build a 160 Tbps two-tier fabric, thirty-two Nexus 7018 switches populated with F-Series 10GbE modules would connect servers. These thirty switches are leaf switches. Each leaf chassis provides 256 10GbE ports to connect servers and another 256 10GbE ports to connect into spine switches. Therefore, each leaf is directly connected to each spine with sixteen FabricPath ports at 10GbE equaling a total of 256 10GbE ports for each leaf switch. There are sixteen spine switches each accepting 512 10GbE FabricPath ports. A single leaf chassis connects 256 10GbE ports into a spine equaling approximately 2.5Tbs. Multiplying each thirty-two leaf’s contribution into the fabric yields 80Tbs. As Ethernet is full-duplex, the total fabric switching capacity is 160
Tbps. Therefore, 160Tbps of switching fabric is available across all thirty-two leaf chassis. As 256 10GbE equals 2.5 Tbs, which also equals 16 FabricPath links to each one of sixteen spine switches, yields 2.5 Tbs, the fabric is non-blocking.

As for layer 2 and layer 3 forwarding, the job of the spine is to forward packets from leaf switches at layer 2, creating a single tier fabric. A key attribute of this architecture is that each 16-way FabricPath links are Equal Cost Multipathing or ECMP. What 16-way FabricPath ECMP provides are two benefits: 1) It delivers more paths for traffic to flow, which increases available bandwidth in the fabric and 2) as they’re distributed across all switches, diversity of routes is enabled to distribute packet forwarding. In essence what 16-way FabricPath ECMP provides is a very low latency, high bandwidth approach to supporting both north-to-south and east-to-west traffic flows simultaneously.

While the above is a Cisco deployment example Arista’s new 7500 series of Ethernet switches support 6 Billion packets per second at wire speed. The 7500s can be configured into a massive two-tier network fabric thanks to it support of 32 port MLAG (Multi-Chassis Link Aggregation) affording the connection of 18,000 to 30,000 servers.

Ethernet continues to evolve. The IEEE recently ratified the 40 and 100 GbE standard with vendors such as Force 10, Cisco, Arista, Extreme, BLADE, Brocade, Voltaire, HP et al announcing support and scheduling product delivery. While the above two-tier network example provides the perspective from the large switch provider, below is BLADE Network Technologies perspective, a company focused on server connectivity.

BLADE Network Technologies believes that as Ethernet delivers new levels of speed and intelligence, it will be the dominant two-tier network fabric for high-end next-generation data centers.
For many applications, low latency is a key requirement, and latency is an area where two-tier networks excel. Studies of stock trading exchanges have shown that tens of milliseconds of delay in data delivery can represent a ten percent drop in revenues, and delays of even five microseconds per trade can cost hundreds of thousands of dollars. Industry-specific requirements for uncompressed data and end-to-end deterministic latency within tens of microseconds make attaining such performance even more difficult. These factors have combined to make raw switching speed a top priority, and today’s best-of-breed 10 Gigabit Ethernet switches achieve can operate with under 700 nanoseconds of port-to-port latency while consuming a miniscule amount of power equivalent to that of standard light bulbs.

As next-generation networks get flatter – driven by latency and bandwidth requirements – emerging Layer 2 technologies such as the IETF’s Transparent Interconnection of Lots of Links or TRILL, enable this trend. The idea behind TRILL is to replace spanning tree as a mechanism to find loop free trees within Layer 2 broadcast domains. Using a routing protocol to build forwarding trees within a Layer 2 broadcast domain enables the flexibility and efficiency to route Layer 2 traffic, just like one would Layer 3 traffic, without the overhead associated with Layer 3 packet processing. TRILL will offer important features, such as support for both broadcast and multicast, load splitting along multiples paths, support for multiple points of attachment, and no tangible delay in service after attachment.

In the data center, bottlenecks are moving from the CPU and memory access to the I/O of the servers. Today’s multi-core servers are now able to sustain a great amount of traffic, requiring fast, flat networks, especially now that virtualization is widely deployed. Analysts have predicted that the 10G market will double year-to-year in 2010 and 2011. More servers using 10G increases the requirement for 40G and 100G in upstream networks. With 10G widely available and 40G coming online, Ethernet networks can enable data and storage traffic to use a single wire, using FCoE or iSCSI for example, and provide the raw speed that makes Ethernet with its economies of scale, to supplant InfiniBand for HPC requirements.

The reason Ethernet will be the network fabric for high-end data center networks is that the vendor community continues to innovate and build upon this protocol. Ethernet innovations are many and are beyond bandwidth increases from 10Mbs, 100Mbs, 1Gbs, 10Gbs, 40Gbs and 100Gbs, which are obvious. Link aggregation, multi-pathing and so much more propel Ethernet’s relevance and suitability to new challenging networking requirements.

ShareThis

In the Lippis Report Research Note 150, we discussed the new industry group called Unified Communications Interoperability Forum or UNIF and compared it to other industry consortium charted to deliver interoperable solutions. While interoperability is sorely needed in the UC industry, it looks like Microsoft killed its changes of broad industry success before it started. What I hear from both UCIF members and non-members is that UCIF is controlled by Microsoft, and thus, lacks a large cross section of industry players as well as major UC providers. With its current structure, UCIF will make limited headway on its charter. In this Lippis Report Research Note, we review UCIF and its’ opportunities.

There is no doubt that the unified communications and collaboration industry needs interoperable solutions. Video traffic, in particular, is growing exponentially, which will not abate anytime soon. Driving growth is the new mobile video market with devices being equipped with real time video applications from companies such as Apple with its’ iPhone 4.0 FaceTime feature and Cisco’s Cius tablet. There is a real-time mobile video chat for Android too via the Movicha client application. In addition, every major UC supplier will launch a tablet based, end user device this year with tight links into its UC and video collaboration infrastructure. In short, the next generation office phone is a tablet. The combination of consumer and business mobile video device options will drive demand for interoperability, not only between mobile end points, but into corporate video conferencing systems too.

There needs to be a base line of interoperability standards for presence and call management also. Yes SIP or session initiation protocol does provide a base line, but many have built proprietary extensions minimizing interoperability options.

Now is a great time for an industry wide consortium of suppliers, service providers, IT executives and analysts to contribute to a set of interoperability standards with associated certification testing. Before UCIF was established Microsoft drove the initiative with limited to no input or invitation from its competitors. This approach has alienated nearly every major UC supplier from participating in UCIF, and therefore, don’t expect to see Cisco, Avaya, ShoreTel, Mitel, NEC et al to contribute. From this point of view, Microsoft killed UCIF before it even started.

But UCIF can make a contribution especially in the area of real time video collaboration between mobile, desktop and video conferencing system end points. For example, Microsoft could open up its’ Real Time Video (RTV) and Real Time Audio (RTA) codec protocols so that mixed vendor video endpoints can communicate with Office Communicator endpoints natively. With LifeSize, Polycom, HP and Microsoft being the UCIF founding members, their contribution to video collaboration interoperability could have a large impact on the real time video conferencing market.

For example, I use a LifeSize Express 220 video conferencing system, and as a standalone device that connects to other video conferencing systems via IP, H.323 or SIP, it’s magnificent. It would be great to connect with clients that have video enabled their desktop and mobile endpoints too. The larger the universe of potential video endpoints that one can connect to, the greater the value a real time video system provides. This would be a great charter for UCIF, which is to contribute open standards and certification testing that enable mobile, desktop and corporate video conferencing systems to interoperable.

However, for UCIF to deliver on its charter, it would have to dissolve and restart with Cisco, Avaya, Mitel, ShoreTel, and a larger role for Siemens, plus service providers, analysts and IT executives all being stake holders. You cannot have a closed group defining open standards. It just does not work that way.

ShareThis

It hasn’t been since the mid 1990s that the networking industry was focused on multi-protocol integration or convergence. The industry is gearing up for a major innovation and competitive cycle fueled by the multi-billion dollar addressable market for data center network fabrics. Over the last eighteen months, every major Ethernet infrastructure provider has been talking about two and three tier network fabrics for high-end data centers. Companies such as Cisco, Arista Networks, HP/3Com, Force10, Voltaire, Extreme, Brocade, Juniper et al have announced network fabrics for data centers with five thousand and more servers with and without storage enablement. Juniper talks of a one-tier fabric through their Project Stratus work with IBM to be available some time in the future. Brocade recently introduced its’ Brocade One, which is a converged data center fabric. Cisco just launched its’ FabricPath Switching System or FSS for the Nexus 7000 that enables massive scale of a two-tier fabric. In this Lippis Report Research Note, we review the architectural attributes of two and three tier network fabrics and review FSS and its accompanying F-Series 10GbE module.

The IT industry is at an inflection point as service delivery is becoming more and more centralized thanks to data center consolidation, virtualization, cloud and mobile computing. It is estimated that a third of all IT spend is concentrated in the data center and this trend is only building thanks to favorable economics, motivating IT business leaders to centralize IT delivery.

The impact of this trend is more and more dense data centers made up of servers in the thousands to tens of thousands and higher. It is at the scale of 5,000 plus servers that a new network fabric is required for high-end data centers. High-end data center design is challenged with increasing complexity, the need for greater workload mobility and reduced energy consumption. Traffic patterns have also shifted significantly, from primarily client-server or as commonly referred to as north-to-south flows, to a combination of client-server and server-server or east-to-west plus north-to-south streams. These shifts have wreaked havoc on application response time and end user experience, since the network is not designed for these Brownian motion type flows.

The main requirements for high-end data center network fabric are low latency, large flat layer 2 domains to enable workload mobility, low power consumption, simplicity of design and significant bandwidth. Storage enablement, meaning consolidated I/O or virtualized I/O, is a growing priority and a new fabric that can support FiberChannel over Ethernet, iSCSI over Ethernet, iWARP over Ethernet or Infiniband over Ethernet is a major plus. One salient observation is that it’s pretty clear that Ethernet is the network fabric of choice as it is the only network protocol that enjoys continual innovation such as TRILL, Data Center Bridging, link aggregation, multi-pathing, and soon, 40 Gbs and 100 Gbs speeds. With the above requirements in mind, let us review data center network design options.

Two and Three Tier Fabrics

A three-tier network architecture is the dominant structure in data centers today and will likely continue as the optimal design for many networks. For most network architects and administrators, this type of design provides the best balance of asset utilization, layer 3 routing for segmentation, scaling and services, plus efficient physical design for cabling and fiber runs. By three tiers, we mean access switches/Top-of-Rack (ToR) switches, or modular/End-of-Row (EoR) switches that connect to servers and IP based storage. These access switches are connected via Ethernet to aggregation switches. The aggregation switches are connected into a set of core switches or routers that forward traffic flows from servers to an intranet and internet, and between the aggregation switches. It’s common in this structure to over-subscribe bandwidth in the access tier, and to a lesser degree, in the aggregation tier, which can increase latency and reduce performance. Inherent in this structure is the placement of layer 2 versus layer 3 forwarding that is Virtual Local Area Networking or VLANs and IP routing. Also common, is that VLANs are constructed within access and aggregation switches, while layer 3 capabilities in the aggregation or core switches route between them.

But within the high-end data center market, where the number of servers is in the thousands to tens of thousands plus and east-west bandwidth is significant, is where a new structure is needed. It is within these data centers where applications need a single layer 2 domain.

Two-tiers of network fabric

A two-tier fabric is designed with two kinds of switches: one that connects servers, and the second that connect switches creating a non-blocking, low latency fabric. In short, there are server facing and fabric facing switches. We use the terms ‘leaf’ switch to denote server facing or connecting switches and ‘spine’ to denote fabric facing or switches that connect leaf switches into the fabric. Together, a leaf and spine architecture create the fabric.

Many IT leaders in Global 2000 firms will have deployed both two and three tier network structure, as different deployment models are used for different applications. For these leaders, a network equipment supplier is needed that possesses product architecture flexibility, meaning an end-to-end product solution that accommodates tier two and three fabrics. This flexibility is found in product that supports layer 2 and layer 3 forwarding, as well as, a variety of line cards to offer design options.

A common network Operating System (OS) of products configured for two and three tier structure is important as IT operations gain efficiency to manage fabrics, as configuration and management are consistent. In addition, a common network OS offers rapid absorption of innovation to IT operations, as new OS features are available at the same time to all fabrics. The benefit of using a common product set to build tier two or three fabrics offers value around operational efficiency, training, sparing and ease of evolution between fabric deployments. In short, the network fabric needs to be simple and general purpose versus purpose built, which a common set of products creating tier two or three fabrics offer. This type of flexibility will enable IT leaders to address the challenges of scale outlined above.

In addition to product flexibility, some networking suppliers take a systems approach to their fabric design, meaning that a solution is built and pre-tested before it arrives on site. This ensures that IT does not have to perform system integration. With the increased concentration of computing and IT dollars into data centers, it’s only obvious that data centers are long-term corporate commitments. Therefore, it is only appropriate that the networking supplier of choice also has a proven long-term commitment to their product architecture.

Perhaps the best example of this is Cisco’s Catalyst 6000 switching architecture and its’ two-year-old Nexus product line. The Catalyst investment protection is well documented as it has been in operation for over a decade, which Cisco customers enjoy continued innovation and value added to this platform. Competitors view its’ longevity as a weakness. The Nexus product line has a similar investment protection philosophy with a fifteen-year plus lifespan expectation. Common to both Catalyst and Nexus is the fact that these products are built on silicon, developed at Cisco, affording investment protection from one generation of the hardware to the next.

A Unified Fabric

The concept of a unified fabric is to virtualize data center resources and connect them through a high bandwidth network that is very scalable, high performance and enables the convergence of multiple protocols onto a single physical network. These IT resources are compute, storage and applications, which are connected via a network fabric. In short, the network is the unified fabric and the network is Ethernet.

The industry tends to focus on storage transport over Ethernet as the main concept behind a unified fabric with technologies such as Fiber Channel over Ethernet or FCoE, iSCSI over Ethernet, iWARP over Ethernet and even Infiniband over Ethernet. But this is a narrow view of a unified fabric, which is being expanded thanks to continual innovation of Ethernet by the vendor community and standards organizations such as the IEEE and IETF. Ethernet innovations such as FCoE, Data Center Bridging or DCB, link aggregation, Cisco’s VN-Link, FEX-Link and virtual PortChannel or vPC have enhanced Ethernet networking to support a wide range of new data center fabric design options. In addition to these protocol enhancements, the IEEE is scheduled to complete its’ work on defining 40Gb and 100Gb Ethernet during the summer of 2010, significantly increasing Ethernet’s ability to scale bandwidth. To demonstrate how Ethernet is evolving to be the unified fabric for high-end data centers, we explore Cisco’s new FabricPath Switching System innovation in this white paper.

The decision to implement a two or three tier network structure comes down to scale. For high-end data centers, a two-tier structure meets the requirements of low latency, movable workloads, scale, simplicity, etc. Many global 2000 concerns will have deployed both a two and three tier network fabric for their high end and less dense data centers.

When shopping for network equipment to construct two and three tier network fabrics, look for suppliers that support both rich Layer 3 routing services and scalable Layer 2 Ethernet capabilities to ensure choice and flexibility of three tier and scalable two tier fabric implementations. Such suppliers offer products that can be configured in multiple use cases and topologies where modules are inter-changeable, skills transferable and operations common between both fabric approaches.

ShareThis

In mid May of this year HP, Juniper Networks, Microsoft, Logitech / LifeSize and Polycom established a forum to develop a set of interoperability test methodologies and certification programs along with specifications and guidelines that enable mixed vendor Unified Communications UC solutions to work with each other. In short, the UC Interoperability Forum or UCIF is trying to define what it means for multi-vendor UC implementations to interoperate. Since its establishment, membership has grown by thirteen vendors, but blaringly obvious is the omission of Cisco, Avaya, Mitel, ShoreTel and other major UC providers. This begs the question of motivation. Is the UCIF interested in interoperability or changing the market landscape to gain advantage on the established leaders? In this Lippis Report Research Note we explore this question.

UC interoperability is a very big deal. In fact, back in early April of this year, Zeus Kerravala, SVP of the Yankee Group and I addressed this issue in a Lippis Report podcast titled What is Holding UC Back?. Our answer was lack of interoperability standards and the vendor community’s minimal interest of embracing the ones we have. The UC market has evolved in a peculiar way as it brings together traditional voice communication companies, data networking firms, computing corporations and software concerns. UC is now at the epicenter of video communications, social networking and mobile computing too. UC represents one of the largest cross sections of disparate markets second only to the Internet. It’s here, within this cross section, that UC gains its enormous value.

UC offers to control real time communications and collaboration. Put another way, all real time business processes will be accessed and control by UC over time. Need to call a colleague? It’s via your UC client. Need to schedule a meeting? It’s via your UC calendar client. Need to video chat with a customer? It’s via your UC video client. Need to bring a group of people together for an emergency meeting? Yes, you guessed it! It is via your UC collaboration client. And common to all those UC clients is presence enabled directory to you, so you can find someone and know if they are available, a communications management system that sets up and tears down connections over intranet, internet and mobile nets. To make UC work ubiquitously, like the public telephone network or the Internet, the vendor community needs a forum or place where it can work out interoperability standards. In addition, for this next evolution in human communications to live up to its promise, it needs motivated vendors to allow their equipment to work together.

Yes, UC does have key interoperability standards such as SIP or Session Initiation Protocol that offer both end-point and communications manager interoperability, but many vendors add proprietary extensions to SIP reducing its value in multi-vendor networks. So the UCIF is to be applauded for taking the first step in creating an organization among the vendor community to usher in an era of interoperable UC. But the problem with UCIF is which companies established its formation. Clearly suppliers are businesses looking for sustainable competitive advantage that comes with large market share and innovative, albeit proprietary technologies. It’s no surprise then that when UCIF is established by firms with limited UC market share one’s mind jumps to the obvious assumption that the founding members of the UCIF are perhaps more interested in market share re-distribution than interoperability.

I’ve observed many industry forums and consortiums in the past that used interoperability as a convenient cause to hide a group’s true intentions. For example, Bay Networks, 3Com and IBM established the Network Interoperability Alliance or NIA in May of 1996 to foster interoperability between Local Area Network (LAN) switch vendors. NIA had limited success in competing with Cisco’s increasing market share gains of the enterprise router and switch market.

UCIF feels a lot like NIA to me. The shear fact that it’s mission statement, board and legal structure was done without any of the UC market leaders input and participation is unfortunate, as it has alienated them. It’s also unfortunate that Polycom and LifeSize are founding UCIF partners, but Cisco/Tandberg is not involved as this has a hint of Polycom/LifeSize fear of Cisco breaking away with the Telepresence market; UCIF seems like a way of mitigating this threat. The timing is very close with Cisco closing the Tandberg acquisition in April and UCIF being launched in May.

If UCIF is not able to entice and recruit Cisco, Avaya, Mitel, and ShoreTel et al in a meaningful and authoritative way, then its fate may very well be the same as NIA. What the industry does need is true interoperability standards so that a Cisco, Avaya, Microsoft, Siemens, HP et al UC implementations are able to work with each other in the same way that multi-vendor email systems work with each other. But without full industry participation, it seems that UCIF may be doomed and not able to deliver on its promise of interoperability. For UCIF to be meaningful it needs the UC market leaders full participation as well as Enterprise IT architects and planners plus service providers too, for without them, UCIF is NIA.

ShareThis

In Lippis Report 148 we reviewed the major drivers and trends that are propelling the high-end data center Ethernet switch market to well over a $1B annual run rate. In this Lippis Report Research Note, we review the major suppliers of these switches. We review Cisco, Arista Networks Force10 Networks, BLADE Network Technologies, HP/3Com/H3C, Voltaire, Avaya, Brocade, and Juniper and identify their unique positions and offerings to participants in the burgeoning market. Our focus is the high-end, high density 10GbE switches that are enabling virtualized cloud computing data centers thanks to Terabits per second of back plane switching capacity, billions of packets per second of layer 2/3 forwarding, hundreds of 10GbE port connectivity per chassis, a new two-tier architecture, microsecond level latency, low power consumption, non-stop operation and software hooks that eliminate network barriers to large scale server virtualization. The engineering in these switches should be celebrated, as they represent the state-of-the-art in computer and network design. In short, they represent the fundamental building block of a new generation of IT delivery based upon cloud computing and virtualization. This Research Note is a must read for any IT executive designing a data center.

After finishing this Research Note, it became evident that this market needs a set of industry neural 10GbE switch test to independently verify vendor claims. We hope to make such a contribution this Fall.

Cisco Systems Nexus Family of Switches

Cisco’s approach to data center Ethernet switching is rooted in its Data Center 3.0 strategy which seeks to scale server virtualization while introducing a platform to enable a unified fabric or converged network and storage running on one physical Ethernet network. Cisco’s data center Ethernet switch portfolio is primarily the Nexus family of switches including the 7000, 5000, 2000 and 1000v. NX-OS is a purpose built data center operating system that runs across the entire Nexus family. NX-OS integrates a number of higher system availability functionalities such as virtual port- channel (vPC), and the capability to upgrade software without disrupting traffic. The Nexus 1000v is a softswitch that resides in a VM hypervisor. The Nexus 1000v’s main job is to eliminate network configuration barriers that exist when moving a VM from one physical machine to another. To accomplish this, the 1000v creates a port profile including VLAN, ACL, policy, security, etc. with persistence, which moves with a VM as a virtualization administer moves a VM from one physical machine to another.

The Nexus 2000 family of Fabric Extenders (FEX) introduces the concept of a remote line card of the parent Nexus 5000 switches and sits on the top-of-rack connecting servers to the switch fabric. The extender concept allows the 2000 and 5000 to be managed as one switch. This configuration reduces cabling requirements and offers an economical approach to server connection, thus providing the benefits of both end-of-row and top-of-rack deployments. The Nexus 5000 Series is 10 Gb Ethernet and Unified Fabric capable switches, connecting Nexus 2000s and servers directly at 100/1/10GbE/FCoE, while providing layer 2 forwarding. Providing layer 3 forwarding, dense 1/10GbE connectivity is the Nexus 7000 Series. The Nexus 7000 Series is available in a 10 and 18 slot chassis and is Cisco’s flagship data center Ethernet switch series. As a point of reference, the Nexus 7000 is now on an annualized run rate of $1B for Cisco, which is more than 10 times greater than any other switch supplier in the data center switch market. The high end 7000 connects 512 10GbE ports with 128 line-rate 10 Gigabit Ethernet ports. The Nexus 7000 Series switches can be segmented into virtual devices, delivering true segmentation of network traffic, context-level fault isolation, and management through the creation of independent hardware and software partitions. Overlay Virtualization Transport (OTV) provides customers a simplified DCI solution by extending layer 2 VLANs over existing IP networks. We have profiled the Nexus 7000 when first released and is available here. The Nexus switches can create a two-tier architecture with the 2000/5000, providing server connectivity and layer 2 forwarding between servers. The Nexus 7000 connects the 2000/5000 to each other and the internet/intranet with high density, high reliability layer 2/3 forwarding.

Arista Networks 7500 Family of Modular Switches

Arista Networks is a new comer to the data center Ethernet market, but its management team is seasoned and customer base growing. It provides six fixed 10GbE switches; five 1/10GbE 7100 and the 1GbE 7048 along with the new Best of Interop awarding winning 7500 modular switch. The 7100/7048 switches connect servers in a Top-of-Rack configuration while the 7500 aggregates these switches and connects them to the internet and intranet. This is a two-tier, “leaf-spine” architecture. The 7500 boasts ultra high performance layer 2/3 1/10 Gb Ethernet switching for high performance computing and cloud computing data centers. The 7500 supports 384 10GbE ports, 5.7Bpps at layer 2 or 3, high packet buffers 18GB deep, ultra low port-port latency of 4.5 microseconds and 10Terabit loss less switch fabric connecting modules.

The 7500 is 10GbE port dense, compact, cloud spec fast, green and prepared for 40 and 100GbE, with a price tag 50% below competitive offerings, according to Arista. While the 7500’s hardware architecture is impressive, its operating system EOS, Extensible Operating System, offers another set of uniqueness. For example, all Arista switches run the same binary image of EOS, easing administration while hastening switch feature upgrades. EOS is a modular OS that allows partners to run their software in the Arista switch, consolidating the number of management and network appliances required, thus increasing performance while reducing energy consumption and physical space. Arista’s EOS modularity was designed as a unique state sharing architecture that separates switch state from protocol processing and application logic. EOS is built on top of a standard Linux kernel. All EOS processes run in their own protected memory space and exchange state through an in-memory database. This multi-process state sharing architecture provides the foundation for in-service-software updates and self-healing resiliency. You can listen to a podcast interview with Douglas Gourlay, VP Marketing and Anshul Sadana, VP Customer & Systems Engineering from Arista on the introduction of the 7500 Series of Ethernet switches here

HP/3Com/H3C’s A12500 Core Data Center Switches

HP has spent 25 years building and selling networking products to its worldwide client base and is currently #2 in the market, with a 21% port count share and the fastest growing networking company in the industry. The combined HP/3COM acquisition brings core switching products, the #1 market share position in China, TippingPoint Intrusion Prevention System and ProCurve edge switches, representing a new choice for clients who are frustrated by today’s current offerings. HP will combine these two entities and operate under the banner of “HP Networking.”

The HP Converged Infrastructure Architecture and FlexFabric blueprint approach the modern data center with a vision that places networking at the center of an integrated data center solution and accelerates deployment of enterprise services and applications. It is designed to drive simplicity through streamlined network designs and centralized management, enhance agility with high performance security, and accelerated provisioning, and reduce cost with energy efficiency and low total cost of ownership. Central to HP FlexFabric is policy-driven network provisioning tightly integrated with server and storage management in an end-to-end data center converged infrastructure.

HP data center solutions are purpose built, using the latest advanced systems and ASIC technologies. “A” family data center networking platforms leverage a common operating system, Comware™ and are managed with a single-pane manager, Intelligent Management Center (IMC). HP switches make use of an HP-developed technology – Intelligent Resilient Framework (IRF) – to create a resilient virtual switching fabric. IRF delivers geographic independence, distributed high-availability, resiliency and millisecond re-convergence across layer 2 and layer 3 protocols. These innovations allow customers to build a simplified, high performing, highly resilient and flat (two-tier) data center network design. They overcome the limitations of low performance/scale, high cost/latency inherent in legacy solutions, which rely on multi-tier network designs, disjointed platform operating systems and complex resiliency protocols.

A key enabler of this transformational design flexibly is the HP next-generation data center switching architecture. This starts with the flagship HP A12500 core data center switch – which is based on a 100G design that uses a multi-level, multi-plane, non-blocking switching architecture to provide high performance and scalability. The A12500 supports 6.66 Tbps of high-performance switching capacity (future support for 13.32 Tbps) and scales to 2.2 billion packets per second of forwarding performance. The A12518 supports 512 10 Gigabit Ethernet or 864 Gigabit Ethernet ports in a single chassis. Its future-proof design accommodates 40/100 Gigabit Ethernet and emerging unified network requirements such as end-to-end FCoE/Data Center Ethernet.

Force10 Networks ExaScale E Series

Force10 Networks was one, if not the first company to offer 1 and 10Gb switching solutions for high-performance computing and data center markets in Fortune 100 companies, Internet portals, global carriers, leading research laboratories and government organizations. It offers a wide range of Ethernet switching and routing products that deliver high port density and resiliency to help customers deploy a high-availability, agile and standards-based GbE and 10 GbE network fabric, while reducing power and cooling costs. Its Ethernet switching products are designed to leverage virtualized data center environments and automate Ethernet networking. For example, its VirtualScale enables management of virtual chassis. Its VirtualControl enables virtualizing logical switching and routing boundaries. For automation, Force10 has developed an architecture, which automates network resource allocation as applications and services spin up and down. This architecture is built upon its HyperLink and SwitchLink technology, two new software features implemented within its Force10 Operating System (FTOS). HyperLink provides real-time communication between Force10 switches and hypervisors or virtual switches to enable automatic provisioning of one or many virtual LANs (VLANs) across multiple switches simultaneously. The SwitchLink feature provides real-time communication with middleware orchestration tools to enable automatic provisioning and management of virtual devices anywhere in the network.

Force10’s modular Ethernet switch data center product portfolio includes the ExaScale E-Series, optimized for core deployments in large-scale, high-performance 10GbE data centers, and the C-Series, optimized for mid-range data centers. Both the E-Series and C-Series come in multiple form factors, run FTOS and are dense high performance switching platforms equipped with redundancy, availability, fault-tolerant operations and many line card options. In addition, Force10 offers the fixed configuration S-Series product line for GbE and 10 GbE ToR configurations. Force10 promotes a vision of simplified data center topologies, using integrated switching and routing in the core, using chassis based E-Series or C-Series products, and fixed configuration ToR access products allowing both 1 tier and 2 tier designs. One tier can be achieved with high density E-Series platform for server aggregation, switching at the server edge, and routing off the same platform to the Internet / WAN. The two-tier architecture can be achieved leveraging ToR switching for server aggregation along with Force10’s chassis based systems in the core. In addition to a large direct sales force, IBM OEM’s Force10’s ExaScale platform as part of IBM’s iDataPlex clustering solution. You can listen to a podcast interview with Steve Garrison, VP Marketing of Force10 on their 40 GbE offering here.

BLADE Network Technologies RackSwitch Family of Ethernet Switches

BLADE Network Technologies (BNT) has been working in the data center switch market since 2006 with much success providing 1/10Gb Ethernet switches for blade servers and top-of-rack configurations. BLADE was launched from Nortel and made up of the successful Alteon Networks group. Their success stems from their ability to identify the top-of-rack and blade switch market in ’06, along with an OEM go to market strategy that included all of the top tier blade server providers such as HP, IBM and NEC. The result is that BLADE has shipped over 8m ports, achieved 25% growth from 2008 to 2009 (in a down economy), owns 50+ % of the blade switch market, is number 3 in the Fixed 10GbE market according to Dell’Oro Group, and has demonstrated scale with at least one customer installing over 16,000 of its switches.

BLADE offers the RackSwitch family of Ethernet switches, which are ToR, 1U high switches. They include the 24-port 360ns latency RackSwitch G8100 10GbE, 48-port RackSwitch G8000 1/10 GbE aggregation and the 24-port 700ns latency RackSwitch G8124 10GbE. Over a year ago, BLADE released its virtualization software called VMready that automates network settings for VM movement ensuring that network settings migrate when a VM is moved from one physical server to another. VMready scales to a 1000 virtual port switch, is based on standards and works with most popular hypervisors.

In addition to VMready, RackSwitch’s unique attributes are found in the fact that they were designed for the data center versus being a wiring closet switch re-formatted for the data center. For example, the RackSwitch BLADEOS supports CEE for unified fabrics, uplink failure detection, virtualization, dual homing for servers, low (80-170Watts) power consumption, back-to-front or front-to-back airflow and very low latency in the 700-360 nanosecond range.

Voltaire’s Vantage 8500

Voltaire has a long history in high performance computing and data center networking as it is one of the key leaders in the InfiniBand market. Voltaire enjoys distribution relationships with HP and IBM, as well as Bull, Fujitsu, NEC, SGI and Oracle. The result is a 100% + year over year revenue growth for Q1 as reported on May 5th. Last October, Voltaire entered the 10 GbE market with the introduction of its Vantage 8500 Ethernet layer 2-core switch. The Vantage 8500 boasts less than 1 microsecond of latency, a low 10 watts per port power consumption and 288 wire speed 10GbE ports in a 15U high chassis. The Vantage 8500’s unique industry contribution is that it’s based on converged enhanced Ethernet (CEE) technology providing InfiniBand-like capabilities to the Ethernet data center. In fact, Voltaire has ported many of InfiniBand’s key characteristics to the Vantage 8500 such as a lossless switching fabric, multi-pathing, virtualization, fabric-wide congestion management and QoS.

From a network design point of view, Voltaire supports a two tier network architecture that enables a simplified, ‘flat’ data center network and puts an end to the era of the over-provisioned network. Voltaire’s design centered on the Vantage 8500 is to support a two-tier data center network that scales from hundreds to a few thousand core ports, which requires high capacity, non-blocking 10 Gigabit Ethernet core switches. By clustering up to twelve Vantage 8500 switches together, IT business leaders can expand their data center to many thousands of servers while preserving the efficiency and price-per-port, without degrading performance or latency which occurs in traditional hierarchical network designs. To support ToR implementations, Voltaire and BLADE Network Technologies announced recently a partnership where BLADE ToR RackSwitches are aggregated by Voltaire’s Vantage 8500, rounding out the two-tier data center Ethernet network architecture.

The Vantage 8500 also features software-based capabilities to address virtualized and converged data center environments. Voltaire’s Unified Fabric Manager™ (UFM) software, application acceleration software and management OS (VT-OS) provide management and performance enhancement tools. These tools were developed and optimized in InfiniBand environments and are now available for Ethernet-based data centers. Voltaire’s recently introduced Unified Fabric Manager™ (UFM™) 3.0 software orchestrates physical and virtual switches delivering guaranteed levels of service per application. It’s the first and only Ethernet fabric management software that dynamically orchestrates end-to-end virtual machine connectivity for multi-vendor, scale-out data center networks.

Avaya’s VSP 9000

During the April 2009 Las Vegas Interop trade show, Nortel committed to the data center Ethernet market with the announcement of its Virtual Services Platform or VSP 9000 switch, which supports up to 27 Terabits per second (Tbps) of backplane switching and 240 10GbE ports per chassis at first release. Avaya announced their commitment to the VSP 9000 and said that it will be generally available in the second half of 2010 while already in controlled availability. The VSP 9000 is built upon the Ethernet Routing Switch 8600/8800 software providing a proven software foundation, mid-plane architecture, a fully programmable network processor unit for flexible data forwarding and carrier-grade Linux.

The VSP 9000 is designed to deliver high-density 10GbE, 40GbE and 100GbE. Its design center is rooted in highly dense connectivity environments that are all mission critical, by definition. Early testing validation of the VSP 9000 promises to provide ultra-high reliability and availability delivering below 50ms failover support, which is critical to eliminate application disruption thanks to its patented hardware failure detection differentiation. The VSP 9000 switch fabrics are lossless Ethernet capable and therefore well positioned to support the next generation Data Center requirements for convergence of storage onto the Ethernet infrastructure.

The VSP 9000’s unique network architecture is found in its ability to cluster four switches together, in that the total architecture exceeds 100 Tbs, with the number of 10GbE ports per rack being up to 720. Avaya continues to invest in Switch Clustering technology (Active/Active resiliency model) such as SMLT (split multi-link trunking) and RSMLT (routed-SMLT), which provides link, switch and router redundancy mechanisms. Three modules are being introduced in the first VSP 9000 release, a 24 port SFP+ for 1 GbE and 10 GbE connectivity, a 48-port of SFP module in addition to a 48-port 10/100/1000 TX module. Future plans include 40GbE and 100GbE interfaces, and even higher-capacity Switch Fabric modules.

Juniper Networks’s EX8200 & EX4500

In January of 2008, Juniper Networks launched its much-anticipated entry into the enterprise Ethernet switch market. Juniper’s focus is on the enterprise data center, campus and branch, as well as the service provider market. Juniper provides a suite of Ethernet switch products, including the EX4200 with Virtual Chassis technology for GbE Top-of-Rack (ToR) and End-of-Row (EoR) data center access, the EX2500 24-port and new EX4500 48-port 10GbE ToR switches, and the EX8200 high-density, high-performance line of modular Ethernet switches.

According to Juniper, it simplifies customer enterprise LAN architectures and advances the economics of networking via its most recently launched initiative called the “new network” for data centers. Juniper’s “new network” promises critical innovations in automation, virtualization and fabric technologies. These innovations are to reduce time to operation by up to 50 percent and eliminate up to 35 percent of data center networking capital expenditures. One aspect of the “new network” is a simplified two-tier network architecture, which may be reduced to one when “Project Stratus” is completed with IBM. The reduction of a three-tier architecture to two is accomplished by utilizing Juniper’s Virtual Chassis fabric technology in the access layer, in conjunction with its high-density, high-performance platforms such as EX8200 and EX4500 in the LAN core, thus eliminating the aggregation or distribution layer. According to Juniper, collapsing the distribution layer reduces complexity in the data center as well as campus networks by reducing the number of managed devices by up to 89%, providing up to 39% savings in space, 44% savings in power and reducing the number of switch interactions by up to 99% compared to three-layer networks. According to Juniper, this approach improves application performance by also reducing latency up to 77% compared to three-layer networks. Note that these claims and numbers are Juniper’s and not mine.

At the core of Juniper’s data center Ethernet product family is the EX8200 line of modular switches. The EX8208 and EX8216 are eight and sixteen-slot modular switches. The EX8216 sports a maximum of 640 10GbE ports and 1.92Bpps and 6.2Tbps backplane speed. The EX8200 is said to support 40GbE and 100GbE interfaces in the future. The EX8200s connect either EX4200 GbE or EX2500 and EX4500 10GbE ToR switches together while providing access to internet/intranet. All Juniper switches run Junos, the network operating system that provides reliability and availability features, developed for the high-performance enterprise and service provider market.

Brocade’s NetIron MLX Series of Switches

In July of 2008, Brocade had purchased Foundry Networks, catapulting them into the Ethernet switch market as one of the top five Ethernet switch/router vendors by revenue. Brocade, with its long history of data center storage, saw that converged I/O was going to happen and prepared the company to participate in this market. At the high end of Brocade’s data center Ethernet switch products is the NetIron MLX-4, MLX-8, MLX-16 and MLX-32 routers, which support 4, 8, 16 and 32 I/O module slots, respectively. We’ll focus on the high end NetIron MLX-32 here, which has been in production since August 2006.

The NetIron MLX-32 boasts a total of fully redundant non-blocking 7.68 Tbps switch fabric capacity. Brocade says that the MLX-32 can forward some 2.284 Bpps of Layer 2/3 packets and support 1,536 and 256 non-blocking 1 GbE and 10 GbE ports, respectively. Note that the new high density 10 GbE was announced the same day as this Research Note was made public. All four NetIron MLX systems are designed for non-stop operation, supporting 1:1 management module redundancy, N+1 switch module redundancy, M+N power module redundancy and N+1 fan redundancy. The NetIron MLX architecture is an adaptive self-routing Clos switch fabric with a virtual output queue (VOQ) design. This non-blocking architecture is optimized for maximum throughput and low latency for all packet sizes.

ShareThis

During last week’s Cisco Q3 FY10 quarterly financial conference call, John Chambers, Cisco’s CEO, said something that impressed and shocked me. The company has been quiet about the growth rates for its Nexus line of data center switches until this call. What shocked me was that the Nexus 7000 is now on an annualized run rate of $1B, yes that’s Billion with a B! I remember being interviewed by John Markoff of the NY Times in Jan ’08 about the Cisco’s Nexus and Juniper’s yet to be announced Ethernet switches. In just 27 short months, the Nexus product line including the 7000, 5000 and 2000 represents a $1.4 B run rate of revenue to Cisco. Another insight gained from this ramp up is that the data center networking trends that we’ve discussed here in various Lippis Report Research Notes are powerful demand drivers for Cisco and other companies participating in this lucrative emerging market and its just starting! Companies such as Arista Networks, Force10 Networks, Blade Network Technologies, HP/3Com/H3C, Voltaire, Avaya, Brocade, Juniper, et al, have unique positions and offerings to participants in the burgeoning market. In this Lippis Report Research Note, we review the mega trends driving high market growth. We save a product review of each of the suppliers for our next Lippis Report Research Note.

In addition to the run rate numbers above, Cisco also posted a milestone of 1 million 10 GbE ports shipped, providing a strong indicator that the 10GbE market is nearing a tipping point to high volume, as pricing drops and its use accelerates. The following are mega trends driving this tremendous market growth. Traffic demand drives bandwidth and that’s the first mega trend.

Traffic Profile Changes: Gone are the days when data center networks primarily shuffle asymmetric email messages and low bandwidth client-server computing applications between endpoints and servers. Best effort data delivery, where latency was secondary to delivering data accurately, has changed to being a paramount design element where 10 milliseconds means the difference between losing a customer or capturing revenue. Traffic is now highly mixed, moving around a data center in near Brownian motion between servers, storage, internet and intranet thanks to a plethora of old and new applications such as mash-ups, VoIP, search, backups, storage access, emerging converged I/O etc. In addition to Brownian motion traffic flows and low latency requirements, the volume of traffic continues to skyrocket and shows no sign of abating. Remember when the Dow dropped by 1000 points in early May of this year? Financial services firms saw an average of 40 times the amount of traffic in their data centers as traders responded to the drop. There is no better driver for traffic volume as financial markets in turmoil. The traditional model of over subscribing data center bandwidth by as much as 80:1 is the norm, and IT business leaders are looking for a more efficient model.

Workload Mobility: With the advent of server virtualization IT leaders are able to decouple an operating system from its underlying server hardware and increase the number of instances an operating system can be replicated on a single server. Server virtualization reduced the number of physical servers needed and in the process reduced energy and cooling requirements. Now that an operating system only needs to know which hypervisor it’s running on, that operating system instance and the applications it services can be moved from one physical server to another in near real-time with the click of a mouse, thus providing workload mobility or portability as well as a rapid application procurement tool.

So what does all of this have to do with networking? A lot, first moving these workloads around a data center consumes huge bandwidth and has low latency requirements to driving raw bandwidth requirements. Secondary, and most importantly to the industry, is that networking or should I say the rigid structure of IP addressing/VLANs, etc are impeding the automation of these workload moves. In short, the data center network needs to be reconfigured when VMs are moved from one physical server to the next in the same data center and it simply does not work if a VM is moved between data centers separated over distance, between a data center and a cloud provider and between cloud providers. This is the area of the infrastructure 2.0 working group.

Doug Goulay said it best in his recent Network World post.

“When moving VMs between machines there is a caveat:  if you want your TCP connections and IP addressing to stay intact the receiving physical host must be capable of supporting the same IP address that the VM moving to it is actively using.  This means that both physical hosts have to be in the same subnet or in the same VLAN depending which layer of the network you are looking at.  Since the largest number of physical servers that can be supported doing this is around 64 it doesn’t change the addressing architecture too much, unless the servers are in different data centers, or are connected to different access layer switches that talk to different aggregation layer switches.  If this is the case the network architecture all of a sudden starts dramatically impeding the movement of VMs:  either VM mobility is impeded, or the network is redesigned. 

Some people often ask me, “can’t I do this with DNS?’  In short, no.  DNS is cached at many client sites, ignoring your TTL.  Additionally, DNS is cached on many PCs for the life of an application session.  If you try to change the IP address of your backup server while you are in the middle of a 2GB backup do not expect the connection to continue.  TCP doesn’t work this way.”

Increased Density: It’s no secret that data centers are bursting from the seams as the economic down turn kicked large IT capital outlays down the road until economic conditions improved. Business leaders have been postponing increasing data centers space, that is square footage, while power density has grown exponentially, until very recently, as cooling requirements increase unabated. Power and cooling capacity are the primary constraints to data center expansion. To deal with these realities, IT business leaders are left with only one option, appropriate capital to either upgrade power and cooling systems or build a new data center. The impact of high energy densities is that server hardware is no longer the primary cost component of a data center. The purchase price of a new (1U) server is now exceeded by the capital cost of power and cooling infrastructure to support that server and will soon be exceeded by the lifetime energy costs alone for that server. In short, energy costs are on their way to dominate data center economics.

To help mitigate these trends, the new data center switches offer increased server connection density at lower energy consumption levels. In addition, their own energy consumption to shuffle packets around has been reduced, for some by as much as 50%. To connect an every increasing dense set of servers, new generation of data center switches boast a two tier network architecture to support thousands to tens of thousands to hundreds of thousands of servers. To deal with high server density connectivity, server access is via a leaf switch, while leaf switches and storage connect to a modular spine switch. The two-tier approach offers efficient connectivity density, low latency albeit this depends highly upon the internal switch design, and is ready to support consolidated I/O.

Consolidated I/O while early in its adoption cycle will go a long way in reducing power consumption of servers as they will have a single network interface for both storage and networking. In addition, consolidated I/O promises to reduce the need for a separate storage switch too again reducing capital, energy and cooling cost.

Back to server density. Server density will only get, well, more dense. If the industry trajectory of cloud computing is realized any where near what the conventional wisdom dictates, then there will be more and more highly dense cloud computing sites supporting an ever increasing number of enterprise, government and consumer applications. How many cloud computing sites does the US need to support all IT applications? With nearly 16 million servers installed nation wide, according to IDC, and with each cloud computing site supporting hundreds of thousands of servers, then perhaps the number of cloud computing sites would be in the hundreds. While its unrealistic that all US enterprises and governments will be hollowed out of their data centers and applications via cloud computing with today’s technology and business control believes; the trend line is clear, there will be a smaller number of very large cloud providers delivering applications to a wide range of customers. Almost like a supernova transforms into a black hole, applications will not be able to escape the gravitational pull of the scale and economics of cloud computing if the industry gets anywhere near this size scale.

The networking industry has been busy adapting to these powerful trends with new internal switching architectures, data center network architecture and automation. Internal switching architectures are being designed with high internal switching capacity in the terabit rage, lower energy consumption in the 10W/port range, low latency and of course high port density. The data center network architecture most are progressing toward is a two –tier leaf-spin approach mentioned above. These switches possess the highest levels of reliability, serviceability and redundancy, as networking is at the center of this massive server connectivity density.

Network automation is another area of investment where VMs can be moved within and between data centers, as well as between data centers and cloud providers, plus between cloud providers. A few companies are addressing network automation, but this is a huge issue that the industry needs to wrap its arms around and provide a scalable solution.

In the next Lippis Report Reseach note, we’ll review Cisco, Arista Networks, Force10 Networks, Blade Network Technologies, HP/3Com/H3C, Voltaire, Avaya, Brocade, Juniper, et al, and highlight their unique positions and offerings to participants in the burgeoning market.

ShareThis

This past Interop in Las Vegas was one of the best I have attended, since even before the economy took a noise dive in 2008. The tone and level of excitement of the industry’s growth potential was refreshingly up beat from the hundreds of IT and vendor executives I talked with. While the size of Interop is a small fraction of what it was in the late 1990s, (70k attendees with over 600 exhibitors to ~ 15K attendees with ~ 200 exhibitors) it still provides a pulse of the networking industry. In fact, Interop has come full circle, back to being a networking event even though it has added other topics. You have to give Dan Lynch credit for creating such a long lasting venue for our industry. Congratulations to Cisco, Arista Networks, HP/3Com, Mallonx for winning best of show in their respective categories and for Arista for winning Best of Interop. In this Lippis Report Research Note I provide the key industry themes that were evident at Interop this year.

The following are my observations of Interop 2010 in LV.

Network Infrastructure Takes Center Stage: Even though Interop provided attendees with thirteen educational content areas including cloud computing, IT security, Enterprise 2.0, etc., it’s the changes taking place in the network infrastructure business that was front and center, loud and clear. The following was the topic of conversations throughout Interop:

• Cisco’s introduction of its Best of Show winning Aironet 3500 Series Access Point with CleanAir technology,
• Arista Networks’ introduction of and winning Best of Show and Best of Interop for its Arista 7500 10Gb modular Ethernet cloud computing switch,
• HP’s closing of its acquisition of 3Com and winning Best of Show for its TippingPoint Virtual Controller,
• HP’s planned acquisition of Palm,
• Avaya’s reassertion in the network business with the introduction of its Ethernet Routing Switch 8800, WLAN 8100 and Advanced Gateway 2330,
• Voltaire’s new Vantage™ 8500, 10 GbE Layer 2 core Ethernet switch,
• Force10’s open network automation demonstrations and 40GbE module

With the above announcements and accomplishments, two thoughts come to mind. First is that Interop is finally back to core networking issues, and second, the above announcements provide a window into the huge changes that are taking place in our industry.

New Industry Structure Emerges: The networking industry has been consolidating for some time now and will only continue. Corporations have some $2T in cash and equivalents on their books, which will be put to work acquiring companies and investing in growth markets. The big growth market in our industry is the fundamental change IT is starting to progress through. HP’s actions last week provided a preview of what’s to come.

HP stole the headlines last week with their shorter then expected closing of their 3Com acquisition, in addition to their intent to purchase Palm. HP realizes that the IT industry is structurally changing away from fixed desktop computing accessing corporate applications hosted in data centers, to mobile computing accessing applications hosted in corporate data centers and cloud computing facilities. The big winner in this transition is networking, as without it, cloud and mobile computing will not happen. Palm gives HP a smartphone platform to participate in the mobile computing market while 3Com expands its corporate networking portfolio significantly.

HP vs Cisco: The buzz at Interop around HP was how it will compete with Cisco. The HP executives and booth personnel were the most energized I have ever seen. HP views their competitive advantage along the lines of innovation, open network architecture and economics. Thinking it through however, HP’s focus will be more on supply chain efficiencies to drive down their cost of producing networking gear close to server economics while leveraging their massive and productive channel to gain market share.

The supply chain efficiency is a great idea, but will take at least a year if not more to deliver. The thinking here is that a 40 Watt power supply is the same, independent of its final designation, as long as it powers a server, router, etc. So can HP redesign their product lines for common components where they gain huge cost efficiency thanks to volume purchasing? Perhaps, but this will take time. Their channel strength should deliver results in the short term. If HP executives are correct and that the market wants a strong number two networking provider, then its channel should produce fairly quickly. If it doesn’t, then this premise is questionable. HP networking is about $5B now; if it doesn’t grow faster then the industry by a significant amount next year, then something is wrong.

Remember HP is competing with a $40B powerhouse that is Cisco Systems, which has a massive and productive channel too that are energized to sell, not only networking gear, but also unified communications, Cisco’s new server platform UCS and video equipment. As for innovation, HP is a great operational company therefore expect them to take cost out of their products. Nevertheless, Cisco is the innovation king, thanks to its systemic incorporation of innovation in product development, plus its ability to integrate acquisitions quickly and materially. Cisco does not only innovate in its products, but around them, offering architected solutions. Examples of this are everywhere, including its borderless network architecture, EnergyWise, UCS, the new 3000 series stackables, Power over Ethernet Plus, its’ ISR G2, the Nexus line of data center switches, its’ approach to integrated network security, etc.

Here’s an example of the power of innovation. A client and Lippis Report subscriber has funded a new $20M data center. During their due diligence, they visited Dell, HP, IBM and Cisco. This CIO will go with Cisco’s UCS. The reason is that during the customer visit, Cisco first described the major direction and trends in data center virtualization and cloud computing in such a way that my client said “Cisco looked into the future and designed UCS to exploit these changes while all the other vendors were selling their old blade systems”. Now this is significant, as this CIO only purchased equipment from market share leaders, that is, he would buy from HP for servers, Dell for desktop systems, Cisco for networking, Avaya for communications etc. Cisco’s innovation in UCS changed his long-standing principal of buying only from market share leaders and will buy UCS for this new data center. So the basis of competition between Cisco and HP will fall into three categories; innovation, supply chain management and channel productivity.

A Mobile and Cloud Computing IT Model Is Disrupting The Status Quo

The Interop announcements above were aligned with this new world order of IT. For example, Arista Networks delivers a massively powerful 10GE switch for cloud spec data centers and high performance data center environments. Clearly investment in cloud infrastructure is a growth market which motivated Voltaire to enter the Ethernet market and leverage its Infiniband experience to deliver converged I/O for both Infiniband and Fiber Channel Over Ethernet (FCoE). As computing is in a rapid technology innovation stage thanks to server virtualization, networking has lagged in its ability to automate network changes brought on by VM moves. This has motivated Force10, F5 and Infoblox to demonstrate innovative approaches to automating network changes so that network administrators do not have to be involved in the process of VM moves and/or the provisioning of new IT services as demand is increased and/or decreased.

It’s clear that HP networking products has gained awareness and will receive consideration. As HP opens the consideration door, Avaya wishes to enter too with its refreshed and new data networking products. Avaya is now lead by experienced IP networking executives that understand voice and data. The Nortel channel also understands voice and data. Ever since Avaya closed its acquisition of Nortel, those channel partners that put selling Nortel gear on hold, have started to come back. They are comfortable now as stability, R&D funding and a strong financially viable company has emerged.

The networking industry is an upside down pyramid with Cisco at the top followed by a few others in the billion-dollar range. Then there are a number of $100M sized firms followed by a few start-ups. The successful firms will be the ones that embrace the new world order of IT that is being brought on as IT leaders de-emphasizes desktop computing and invest in mobile plus cloud computing.

ShareThis

During a podcast with Zeus Kerravala of the Yankee Group, we came to the conclusion that the unified communications market is in a funk and the only way out is for suppliers to adhere to industry standards that allow interoperability. To demonstrate this achievement, UC providers would be well advised to participate in industry wide interoperability testing. In this Lippis Report, we discuss the issues that are holding back UC and video conferencing adoption.

It’s important to understand that standards and interoperability mean different things. A supplier can be open, but not standards based. A supplier can be standards based, and not open. And then a supplier can be standards based and build a range of extensions to the standard, which then makes their implementation nonstandard. And this is where the UC industry is right now. Nearly every supplier will tout how open they are; that is how standards based they are, but what it all comes down to is we really don’t have a common standard UC that allows IT business leaders to deploy UC solutions and work in a mixed vendor and service provider environment. This is the single most important issue to IT business leaders that is creating pause in their UC deployments and extending sales cycles.

It’s disappointing. Our industry has been developing UC since 1996. It seems as if UC suppliers are not ready to implement standards based UC solutions, as they haven’t figured out how to maneuver as the basis of competition changes toward interoperable UC. The question is if a UC supplier makes their offering open and interoperable will they lose important functionality and compete on features above standard UC services?

The UC market is built primarily off of a telecom heritage in which none of the PBX phone system vendors had interest in interoperable solutions, and as a result, the PBX market was frozen with 30% share each going to Lucent/Avaya, Nortel and Siemens for decades. Voice over IP or VoIP thawed that market by radically changing it with a new approach to voice and based upon the openness of IP.

It’s because of this PBX heritage that many of the suppliers view being open and truly standards based as a threat. Thinking this way masks the bigger picture. UC suppliers are missing the larger picture, which is this. If UC endpoints truly worked as plug-n-play, and IT business leaders knew that whatever UC systems they deployed would interact and work with different UC suppliers, then UC usage would go through the roof. The market would expand and service providers could offer standard UC services too.

The big picture of plug and play universal UC would change market share. Perhaps large suppliers would have a lower percentage of share, but of a much bigger addressable market and associated dollar value. In short, the pie would get much bigger. In addition, the big picture would create a much larger UC ecosystem, with more winners than the current industry structure, and that is healthy.

Point in case. Most IT business leaders have relationships and large investment with both Cisco and Microsoft. Many Lippis Report subscribers voice concern that they can’t get their Cisco and Microsoft UC solutions to work properly together. If two of the largest vendors in the UC space don’t work together, than what hope do most IT leaders have of actually getting their UC investments to work in a mixed vendor environment?

This is systemic, because without adherence to basic UC standards overall market size, growth rates, adoption rates and adjacent markets will be limited. A closely aligned UC adjacent market is video communications. While there are companies promoting various different standards, there’s no interoperability within the three-tier enterprise video communications structure. The three-tiers are 1) desktop video, 2) a pedestrian video conferencing system and 3) Telepresence rooms. There are little to no standards that would allow different vendors to be providing each of the three-tiers and offer users the same simple set-up that allows video communications to work between the three tiers. Today’s solution is to buy a single vendor, but no video conferencing supplier offers all three-tiers. Cisco may soon offer all three tiers thanks to their Tandberg acquisition, but Microsoft still owns the desktop and they are not opening up their RTA/RTE protocol any time soon.

Another closely aligned UC adjacent market are smartphones, such as the iPhone, Android, blackberry, the Palm Pre etc. There are only limited UC extensions being offered to mobile endpoints but they lack standards, presence, directory and fixed mobile convergence

In short, the biggest drawback is that it’s too hard to get systems, sometimes-even systems from the same vendor to talk to each other. Getting different systems from different vendors to talk to each other is nearly non-existent today. The directory problem is a huge industry problem, because it’s very different to know who has video communications and who doesn’t. Think of it in terms of telephony. I know you’ve got a phone and a phone number that I can call you on. I know you’ve got an email address. However, I don’t know if you have video, and if I do, I don’t know how to connect to you. So, if that barrier doesn’t fall, video will remain a niche application with relatively low utilization even though high definition video and Telepresence utilization has increased substantially during the downturn.

We are calling the telecos to task on this. The telecos hold a lot of the keys to success because video conferencing systems are connect over teleco networks, which is the perfect place to apply interoperability standards. And while a number of telecos now support inter-company Telepresence on their own backbone, they need to step that up and provide inter-company video cross-backbone, and be willing to work with all video conferencing providers.

Again, here’s the case where the telecos probably look at this interoperable video service as threatening, in that they don’t want to open their network up and allow other provides to provide service with our network. Yet if they did, usage would go up and everybody would benefit. So the network operators really need to step up here.

The big picture plug and play model of UC will change business models. As the industry becomes open and standards based, truly standards based, an innovative ecosystem will flourish. Money flows will shift as the big picture UC market becomes much more ISV (independent software vendor) driven. In this model, from a vendor perspective, what’s important is less about the tools you have or the applications you provide, and more about your willingness to support the ecosystem that surrounds you and the development tools you provide them. In essence, the developer community winds up leading your organization.

This is a big shift. In the world of applications, the platform is the important asset and how a company supports its ecosystem will become a key basis of competition and a barrier of entry, as there are only a limited number of ISVs. The open UC market will move the value proposition to one of a platform delivering innovative UC applications. In this model, revenue generation shifts where money comes from and how vendors get it. Avaya understands it very well, with its Dev Connect community, Cisco with its CDN and Siemens with its UC Server 2010 UC platform, but all suppliers need to put much more energy into open standards and going to market through a developer ecosystem.

To accelerate the industry to the big picture UC market expansion, the industry needs to embrace a public semi-annual interoperability testing and demonstration event. It was this public testing that drove TCP/IP into the success of the Internet with the industry trade show and conference called Interop. We need a UC Interop to move this technology to mainstream.

ShareThis

Many IT leaders are striving to understand who is on their network and what they are doing. These are two simple questions and yet, in many cases, IT business leaders do not have a good way to answer them. And once IT leaders are able to obtain this information the question then becomes what else I can do with the data: obtain a history report, perform statistics for analysis and planning, generate compliance reports and much more. To tightly link business processes with networked applications, IT leaders need to wrap policy, identity and security around users and IT assets.

This is the essence of Cisco’s TrustSec; that TrustSec provides security services as its primary value proposition but the data and insight it generates assist IT business leaders with network design to meet future growth. Cisco’s TrustSec organizes and simplifies existing authentication and policy schema allowing administrators to configure and maintain identity-based access to IT resources while identifying and applying policy based on a user’s role in the organization. TrustSec also provides encrypted links between end-points and servers. TrustSec is an architecture which builds upon existing network services embedded into network infrastructure, addressing not only security issues but delivering certain business services too.

A key pillar of strength for TrustSec is its ability to create a consistent and unified set of policies across the entire network. Its second pillar is the ability to identify users; from the moment a user accesses the network, everything about this user is known and it follows them wherever they go. TrustSec identity is embedded in the traffic that the user generates, which goes well beyond initial Network Access Control (NAC) and offers unique design capabilities that we’ll discuss below. The third pillar is security, which is reflected in a number of areas such as NAC, encryption, etc.

TrustSec is an architecture delivering network access control, policy, identity and encryption. Policy is the glue that ties business processes to network behavior and thus TrustSec has expanded its role in policy creation. TrustSec policy is segmented into three areas:

Authentication: The foundation of the technologies is authentication as it defines user identity. Authentication is how TrustSec understands users; who they are, what roles they have in the organization and what type of credentials they possess as well as confirmation of these attributes. TrustSec provides multiple authentication approaches, such as 802.1x, web authentication and MAC authentication bypass (MAB). All three approaches are implemented and supported on Cisco Catalyst or Cisco Nexus switches. Cisco uses the term “Flexible Authentication” to represent these three methods. What’s unique about Cisco’s TrustSec authentication approach is that it is providing all three methods together and they are completely adjustable. What this means is that IT administrators can configure these authenticating methods in any sequence of their choice, in one place, to host all authentication configurations, greatly simplifying the process of configuration and change management. There is yet another TrustSec authentication method, namely appliance-based network authentication provided by the Cisco NAC Appliance. This method expands beyond LAN switches to include wireless and remote access as well.

A powerful feature is that once authentication is configured on a centralized policy server all switches receive this data, easing deployment while providing consistency and scale. No more authentication configuration on a per switch basis but rather a consistent policy is realized. For IT leaders not ready to implement Catalyst or Nexus switch policy enforcement but who would rather use an appliance there is an in- and out-of-band NAC appliance approach to policy enforcement.

Authorization: Once a user has been authenticated and their organizational role confirmed then services could be designed specifically for them, implemented via control mechanisms. It’s common in the industry to typically assign a VLAN or ACL for the user depending upon a layer 2 or 3 construct. TrustSec supports both VLAN and ACL implementations. What’s unique about TrustSec is that it allows IT administrators to create a security group tag or SGT. SGT essentially allows every single packet to be tracked throughout the entire infrastructure so user control is not relegated to the initial network entry point that VLAN and ACLs dictate. SGT enables user control and support deep down in the interior of the network. For example, to strictly control access to a critical file server, an IT administrator can enable SGT to filter network egress to that server for only those allowed access. The control point is on the switch so that when traffic leaves the switch trying to reach the file server, authorized users via SGT are able to egress.

Value-Added Services: With user authentication and authorization configured along with control, IT administrators can now design specified user services that are linked to business processes. Services such as IP telephony integration and IP phone end-points that need to be authenticated and authorized but are non-user devices, meaning that they don’t possess an 802.1x supplicant and there is no human behind the device. TrustSec utilizes aspects of 802.1x to authenticate and authorize the IP phone’s user taking into account various scenarios such as when the IP phone is powered down or its behind a PC, etc. Other services are guest access, device profiling, device posture and link encryption via MACSec, an IEEE standard that specifies how encryption may be used to secure links within local area networks.

TrustSec’s MACSec implementation is supported on the Nexus switches and on the new Cisco Catalyst 3560-X and 3750-X series switches that connect desktops, WLAN access points and laptops. In short, with MACSec supported on Nexus 7000 and Catalyst 3560-X and 3750-X switches Cisco is working towards full native layer 2 encryption as the Nexus switches are located in the data center while the Catalyst 3000s are closet switches connecting desktops. This is a welcome development for high security environments such as government agencies, certain research and development laboratories and other environments that require a higher level of security.

TrustSec Innovations
Cisco is announcing a set of new TrustSec features and innovations such as Security Group Access Control List that allows IT administrators to control group access based upon MACSec key technology. Security group Tag Exchange Protocol (SXP) is useful for Catalyst switches that do not have the processing power to support SGT today. So Cisco developed SXP to insure Cisco customers can use their existing Catalyst switches to participate in the overall SGT implementation. Flexible Authentication is another innovation for scenarios when end-points do not have an 802.1x supplicant and require access to an 802.1x network. Flexible Authentication offers web authentication which is useful for printers, guest access, etc.

Open Mode offers additional options or modes to being simply denied network access, a dramatic event when it occurs. Cisco TrustSec designed multiple modes to ease this transition. For example, monitor mode is like an audit mode. IT is able to monitor all users and their traffic thus allowing IT to view network dynamics before turning on 802.1x.

In addition to monitor mode there is ‘low impact’ mode. In this case 802.1x authentication is engaged but allows certain types of traffic to pass onto the network even if authentication denies access. This is useful for DNS or maintenance related network traffic; for example, allowing this specific traffic to pass even if it didn’t pass authentication. There are configurable options for “low impact” mode. There is also a “high security” mode where only authenticated users/devices are granted access.

Value-Added Services:

There are tools to automate the process of adding value-added services such as device profiling which recognizes defined end-points such as a printer which is very handy when the printer is moved, replaced or a new one is added, thus saving IT operations configuration time. Automated device profiling tracks devices by monitoring these end-points as they boot up on the network. TrustSec identifies that the new device is a printer, and then loads the printer policy placing the printer in the right VLAN, ACL or SGT; then it updates the device database, saving IT a lot of effort.

Guest services are now integrated with the Cisco NAC appliance guest server, streamlining guest account creation and user notification. The integration of guest services into the NAC Appliance allows report creation; for example, history tracking. Guest services now works in both 802.1x and NAC environments offering IT choice, convenience and simplified operations, an industry first. Thus any worker with authorization can create a guest account, reducing dependence on IT or the helpdesk which often fielded guest access requests.

Posture assessment provides device compliance status, such as which version of Anti-Virus, spyware scan, network configuration assessment, etc., which is added to authentication services.

Cisco has enhanced end-to-end troubleshooting and monitoring capabilities into TrustSec for 802.1x environments. When an 802.1x end-point attempts to access the network a string of exchanges occur between that end-point and the network. There is a protocol exchange to obtain user information while the authenticator or network switch transfers the information to the authentication policy server. During this protocol exchange between the three entities there could be a number of reasons why things do not work. Typically when things went wrong there was limited information available to IT administrators to troubleshoot and resolve the issue. To fix this problem TrustSec collects user supplicant information from the network, the policy server and switch as a log message, which is passed through certain algorithms or scripts to isolate the problem. This increased visibility enables quick problem identification and resolution, pin pointing the trouble to the switch configuration, supplicant issue or determining whether it’s simply a wrong password. These scripts are not only useful with troubleshooting, but also compliance as collected information can generate reports. These scripts are available in Cisco’s ACS 5.1 policy server.

Implementing TrustSec

There are currently two TrustSec deployment scenarios: 1) 802.1x and 2) Appliance based. In 802.1x environments ACS server is the policy server with Catalyst and Nexus switches providing enforcement with Radius as the control plane. In the appliance-based approach Catalyst switches provide enforcement, NAC Manager is the policy server while SNMP is the control plane. The appliance-based approach does not support SGT but it provides posture assessment which 802.1x does not.

TrustSec features and attributes are implemented across many Cisco products such as the Cisco Catalyst and Nexus switches providing policy enforcement and encryption services. Policy is defined in the Cisco ACS (Access Control System) while its key authentication and authorization are implemented in the NAC Manager, Server, Profiler and Guest Server. There are two TrustSec end-point clients, those being Cisco’s or any 802.1x supplicant and its NAC client. It’s not a stretch to see that Cisco will consolidate the end-point clients and policy components over time to minimize the number of appliances needed to fully utilize TrustSec. ACS already works with the NAC Profiler and Guest Server plus directory services such as active directory or LDAP. Knowing Cisco the NAC manager may also hold all this functionality for those who choose to deploy TrustSec in an appliance form factor. Over time these two TrustSec approaches will consolidate to one, allowing 802.1x and NAC users and devices connect to the network with one policy server, and either switch or appliance enforcement method leaving choice to IT departments. The end-point clients would fit nicely into Cisco’s AnyConnect client offering both LAN and remote security services in one client.

TrustSec has expanded to include 802.1x and NAC environments offering customer choice to either proceed with one approach or a combination of the two. TrustSec’s attributes are based on policy, identity and security. Over time we expect that many of the TrustSec attributes will be integrated into the network allowing its services to be ubiquitous throughout the corporate network fabric, significantly adding to corporate security architecture.

To make TrustSec truly successful Cisco should add more support for mobile and remote access end-points in addition to LAN-based end-points to the architecture. In addition video end-points will require TrustSec services too and will have to be supported. There are slight tradeoffs between 802.1x and NAC clients such as posture assessment and SGT support. These two client features should blend over time and converge into one to simplify TrustSec client software.

ShareThis

With all the investment in IT security over the years, one would think that threats would have subsided; but they have only increased and largely increased with exploits and iframes (redirection on a reputable website to infect its visitors) up nearly by a factor of 2000 over the past two years. This has resulted in an increase in data theft Trojans over the same period by a factor of 6000, according to the 2009 ScanSafe Global Threat Report, enriching hackers and cybercriminals. What’s driving this exploit growth is that hackers and cybercriminals are automating successful techniques for mass website infection. In addition, hackers increasingly collaborate, sharing best practices to infect websites for personal gain. In short, IT and business leaders are not confronting individual hackers, but a community of cybercriminals working together to steal corporate data that is increasingly organized as a traditional business with suppliers, resellers and end users. And this community’s opportunities to attack individuals and corporations have only increased with the huge growth in mobile access and deep corporate reliance of web-based applications to automate business processes.

IT leaders, especially those in small- to medium-sized companies are at a disadvantage with limited and even decreased IT staff and capital budgets, making it difficult for them to keep up with an ever-increasing volume of threats and complex exploit profiles. To mitigate these fears and concerns IT leaders have been turning to Cloud Web Security offerings by Cisco, BlueCoat, Websense, McAfee and others. While limited at first to URL filtering, Cloud Web Security is becoming sophisticated enough to identify threats by analyzing content in a contextual basis. Further, Cloud Web Security is in essence a SaaS offering affording on premises and mobile threat defense by extending a corporate perimeter around its mobile workforce.

The Web has become fundamental to business and the overall economy. The use of the internet has evolved from a static research tool to a dynamic communication platform, with corporate revenue directly linked to Web availability. Second, Web access is wide and varied in terms of end-points used, be it desktops, laptops, netbooks, smartphones, kiosks, etc., and networks providing access such as corporate networks, broadband, WLAN, hotspots. From a security point of view exploits infect corporate IT assets primarily through malicious content on web sites, email and blended email/web combinations. The Web will be used increasingly as the threat vector of choice by hackers and cybercriminals to distribute malware and perpetuate identity theft, financial fraud, and corporate espionage. As networks have become borderless, security vulnerabilities have increased by opening up doors or entry points that hackers can exploit, be those doors end-point devices, web sites, bad sections of web sites, applications, email, etc.

To mitigate these vulnerabilities IT leaders have deployed Web Security services in their enterprises in an effort to control which web sites employees’ access. But with the huge growth of laptops and smartphones, Cloud Web Security has been introduced beyond the corporate perimeter to protect all users and mobile devices too. Cloud Web Security threat prevention is getting much smarter by incorporating both content analysis with context offering, a powerful defense against zero-day exploits for all users regardless of location.

Cisco ScanSafe

To make these points, I focus on Cisco’s Cloud Web Security offering through their acquisition of ScanSafe. Prior to Cisco’s acquisition of ScanSafe, IDC’s “Worldwide Web Security 2009-2013 Forecast and 2008 Vendor Shares” ranked it as the worldwide market leader with over 30% share with Websense in second place at 7%. ScanSafe’s suite of services includes Web Malware Scanning, Web Filtering and Anywhere+ for roaming user protection. Unlike other solutions, which rely on URL databases and signatures to filter and identify malicious sites, ScanSafe, through its Outbreak Intelligence engine scans all Web requests in real time, so IT leaders receive comprehensive protection from all threats, including threats that appear before an anti-virus signature is available – and that’s a huge advantage.

What’s unique about Cisco ScanSafe is the sheer volume of data – billions of web requests daily – it processes for threat identification. The visibility gained from ScanSafe is also fed into Cisco’s Security Intelligence Operations (SIO) that incorporates data from IntelliShield, SensorBase and the huge footprint from participating Cisco customers who have opted into send their IPS appliance security data to SIO, creating the largest threat collection network on the planet. SIO’s broad threat collection and exploit mitigation dissemination will only increase the accuracy of the entire Cisco security portfolio, including ScanSafe.

Since ScanSafe is a Cloud Web Security service consisting of over 15 data centers deployed across the world, access is independent of geographic location. In essence a user connecting to the Web will have their traffic pass through one of ScanSafe’s data centers. In the ScanSafe data center the requested Web page is split into its basic components such as Java, PDF, Windows EXE, etc., and scanned within an analysis engine called Outbreak Intelligence for zero-day exploits via twenty-six specialized scanlets. The output of the scanlets is processed by a meta scanner that processes contextual information to decide if the content should be blocked or allowed to pass. This process of content scanning takes less than 5ms assuring user performance is not impeded. What’s impressive about ScanSafe is its scale. It sees billions of web requests per day and all of this scanning and filtering of traffic is captured within Outbreak Intelligence that provides real time harvesting of data that allows it to identify and stop an exploit well before anti-virus vendors can produce a signature and propagate it to their customers.

Signatures Defense Is Not An Effective Zero Day Threat Mitigation Technique

For example, during the Zeus Botnet and Gumblar exploit ScanSafe was blocking these exploits from propagating to clients well before anti-virus firms developed and distributed a signature. This lapse of time between exploit identification, signature development and mitigation is reduced to zero in ScanSafe’s Outbreak Intelligence, offering a much better approach to defense. Consider Gumblar, which first spiked near the 16th of April 2009 and took anti-virus vendors nearly a week to develop a signature, all the while ScanSafe was blocking it from clients. After anti-virus vendors released a Gumblar signature Gumblar traffic did indeed decline, but the hacker modified his/her exploit and near the 23rd of April Gumblar spiked again forcing the anti-virus vendors to identify it, analyze it, write a new signature and finally distribute it. During this time ScanSafe had been blocking the mutated Gumblar from its clients. This cycle continued for nearly six weeks starting from threat outbreak and included four hacker mutations and subsequent signatures until the anti-virus vendors delivered consistent protection.

The above is an example of ScanSafe’s ability to detect and block exploits in scale. The more content ScanSafe’s data centers scan the smarter its Outbreak Intelligence gets. This is important for two reasons. First in this market the suppliers with the largest market share are rewarded with the greatest visibility into exploits and thus offer the quickest and most potent defenses. Thus with its dominant share ScanSafe has a level of threat visibility that allows it to accurately and quickly mitigate exploits. Second since ScanSafe is a cloud-based service it can deliver a solution for on-premise and mobile users quickly and easily. This combination is not only powerful for large enterprises but for small- to medium-sized business as well, where IT skills and capital constraints had precluded them from offering the same protections as larger firms, until now. In fact the small to medium enterprise (SME) market can offer its employees the same level of protection as large enterprises when using ScanSafe.

ScanSafe’s data centers not only offer scale of processing but fault tolerance and redundancy are built into their design so that in the case of a data center outage, the data center that’s nearest in proximity is equipped with enough capacity to support all users without negatively impacting performance. ScanSafe has a track record of 100% availability over the past 7 years. For traveling mobile users their protection follows them anywhere in the world. For example a traveling mobile worker may deplane in Singapore connecting to the ScanSafe Singapore data center, but upon arrival in the U.K. the London data center will service this mobile user so that his/her policy is consistent worldwide while performance is maximized.

Reporting Is A Key ScanSafe Differentiator

ScanSafe reporting is arguably the most detailed in the market at analyzing web security threats and offers depth unattainable by enterprise system thanks to its position in the cloud. There are over 5000 customizable reports with 75 reporting attributes and 11 categories with comprehensive drill downs. This reporting flexibility allows administrators to define important data too. There are virtually no report design restraints offering great insight and visibility into web activity. The reports are based on a data warehouse infrastructure providing cumulative, trending and forensic reports being processed and maintained by ScanSafe’s storage, compute and network infrastructure. Its reporting is SaaS-based, meaning that IT leaders do not need to purchase or run reporting software on-premise. Reporting is key as IT leaders are provided with visibility for both on-premise and off-premises Web usage, offering them tools for charge back, forensics, application planning, etc.

Consistent or Different Policy

Policy is an enabler for IT leaders to gain control over Web use by in office and mobile workers. ScanSafe delivers IT leaders control knobs over content such as URL filtering, dynamic classifications of websites, end-user education through threat labeling of search engine results before employees click on links plus other traditional policy settings. In addition, ScanSafe’s Anywhere+ allows IT Security leaders to set flexible on- and off- premises policy. For example, in-office employees may have policy set for both acceptable use and malware prevention; however, off-premises employees may have policy set for malware prevention. As Anywhere+ becomes integrated with Cisco’s AnyConnect client, this capability will be pushed to the millions of users that use the AnyConnect client. Providing a consistent policy framework for on- and off-premises is a work in progress at Cisco, but they do have the product breadth to deliver on its implementation.

Cloud Web Security has primarily been focused on URL filtering as its primary control. But URL filtering has become less effective as a control or security technique due to large quantities of dynamic content delivered over the internet. URL filtering schemes are unable to identify different types of content within pages especially within Web 2.0 sites. This is where content analysis has blossomed as an accurate approach to identify every component of web page content that is attempting to traverse a corporate firewall or reach a mobile end-point independent of website categorization.

Cloud Web Security offerings are delivering a network approach to zero-day exploit mitigation that is faster and more accurate than traditional client-based anti-virus signature approaches. Cloud Web Security offerings that are based upon content analysis with a contextual basis are best positioned to mitigate exploits. As these offerings are cloud-based their use is naturally extended to static and mobile locations offering protection to both desktop and mobile users with consistent reporting and customizable policy creation. Another large benefit is that Cloud Web Security solutions are well within the reach of small- to medium-sized businesses, offering these firms an effective way to close the gap between effective defense and budget plus staff limitations. Cloud Web Security should be considered as part of IT’s overall arsenal to defend workers and corporate assets from hacker and cybercriminal threats.

ShareThis

No matter where you look today the structure of IT is fundamentally changing. Applications are increasingly being accessed from mobile devices along with traditional laptop, desktop and even kiosk machines. SaaS has taken off and is far more prevalent than most executives realize as they are acquired by line of business and divisional budgets, leaving many IT leaders blind-sided and out of control with their relevance coming into question. As a result corporate application portfolios are shifting in their mix under IT leaders from one of total control to partial control to none. In short, IT leaders are finding that the largest application growth in their corporation is coming from outside of their traditional perimeter and with no control knobs. In essence applications and networks are becoming borderless.

While borderless networks offer productivity improvements allowing work to follow individuals, IT leaders are concerned about its security implications, that being are corporate assets secure when applications are being accessed and used within and outside of corporate perimeter? Can IT leaders deliver the ease of use afforded by borderless networks securely? In this Lippis Report Research Note we review Cisco’s New AnyConnect approach to securing mobile devices, which promises invisible use along with safeguards, visibility, control and relevance for IT security leaders.

With mobility comes productivity. As users work anywhere through a wide range of devices or end-points business productivity accelerates. This has been the case with every cycle of computing, from mainframes, minis, PCs, internet-connected PCs to now mobility; a correlated significant jump in productivity at a macro-economic level occurred and the mobile computing cycle will be no different. But to cease this productivity IT leaders need to be comfortable with mobile computing security. And they do have a lot to be concerned about as securing a plethora of different devices accessing both corporate and Web/SaaS applications from a vast array of locations and network access methods is a challenge.

Three major mobile computing themes stand out:

Theme one: Increase Productivity: IT business leaders need employees to be productive, so they provide access to information, making that access as seamless as possible so employees obtain the tools they need and information they require to do their jobs. A central component to this is providing consistency between out-of-office and in-office IT experience.

Theme two: Deliver Mobile Security: Many IT leaders feel this way: “I built all of this infrastructure to protect my users when they’re sitting within the organization. When they leave and are remote what is protecting them and corporate assets? I protect them eight hours a day, then they go home with their laptop and get infected for 16 hours.” In short a disproportionate amount of security investment has been made within the corporate perimeter that needs to be extended to remote and mobile access.

Theme three: End-point Agnostic: Consumerization of the enterprise is forcing IT business leaders to not only support traditional remote devices such as laptops, but also IPhones, Android, Blackberry, netbooks and other end-points that are on the horizon such as the iPad. Consumerization is focusing IT business leaders to deliver seamless network access with always-on security and protection across a broad array of devices to enable business productivity.

Securing Mobile End-points With Existing Defense Techniques
From a security point of view, IT defense for mobile devices share many of the same concerns as securing fixed end-points. Unique to mobility is the security issue of lost mobile devices/end-points. To address this concern IT leaders typically need complementary product that can enforce PIN locks/encryption and support remote data wipe. Common to mobile and desktop security are concerns with acceptable use and threat protection. Malware plus web-based threats have spiked over the past 18 months, increasing threat awareness as business press coverage of exploits have expanded. IT leaders have data security on the top of their minds too. Therefore, access control, threat protection, data security, etc., are common security concerns to fixed and mobile computing with IT leaders and vendors seeking to expand/extend existing defenses to this new wave of computing.

Legacy VPNs Too Cumbersome: A New Generation of Remote Access Emerges
Clearly existing technologies such as Virtual Private Networks (VPN) is a remote access approach that seeks to provide a solution to mobile computing, but it falls short. The challenge with legacy VPNs is its cumbersome use model with multiple boxes to check, tokens and keys to exchange plus certificates to obtain. The process is not transparent and as a result is too painful to use resulting in legacy VPNs use only when absolutely necessary. This use difficulty is both a lost productivity opportunity and security vulnerability.

The vast majority of time a user is outside the corporate network its end-point is unconnected to that network and thus largely unprotected and invisible to IT. Laptops in essence have no security except perhaps a desktop anti-virus (AV) client, which is becoming less and less effective over time due to signature-based defenses lagging exploit propagation. Connectivity may even be so rare that end-points spend much of their time out-of-compliance on patch levels. SaaS makes the problem even worse. Many use SaaS applications such as Salesforce.com, et al., to conduct business-critical or business-relevant tasks by simply accessing these sites over the internet where IT doesn’t have visibility let alone control over these sessions. Most don’t use VPNs to access SaaS applications, which would route traffic through the corporate network, due to the use hassle.

With corporate applications having moved rapidly to both HTTP/Web/SaaS web security is an increasing threat breeding ground that requires a new defense model. There are web security solutions in the market such as Websense and BlueCoat, but their current models are limited to URL-filtering clients, which enforce approved URLs to each end-point. Further, their current operating system support for clients is limited to Windows XP omitting MAC OS X and smartphone mobile platforms. And while URL-filtering does provide limited acceptable use and malware security it does not address data loss, access control and thus full threat prevention, particularly given the nature and mechanism used by hackers to propagate threats today.

Enter Cisco AnyConnect Secure Mobility

To address mobile computing, Cisco has announced its Cisco AnyConnect Secure Mobility to combine access control and web security, which in essence creates a flexible perimeter around a corporation’s mobile end-points providing them the safeguards and security that desktop systems enjoy behind the corporate firewall. AnyConnect Secure Mobility combines Cisco’s AnyConnect client, Cisco’s ASA (VPN, Firewall, IPS, content switch appliance), IronPort (Web security), ScanSafe (Cloud Web Security), and SIO (Security Intelligence Operation) to deliver the next generation of remote access and security for mobile end-points.

While AnyConnect utilizes and integrates much of Cisco’s security technology, the real innovation is how the mobile client captures ease of use and simplicity, allowing users to access both corporate and Web/SaaS applications without the hassle of traditional VPNs for any type of end-point, be it laptop, smartphone, netbook, etc., while protecting corporate assets. In many cases the user experience will be far superior to existing remote access solutions as they don’t need to be concerned with network access type, be it VPN, internet, 3G, WLAN, 4G, etc. The hope is that AnyConnect will provide IT leaders with the assurances they need to enable employees to embrace mobile computing allowing their corporations to exploit its productivity advantages.

Making Remote Access Secure and Invisible

AnyConnect is a pervasive end-point controlling network access and security. The idea is that it fades away into the background, versus the very manual VPN configuration of today. AnyConnect decides where to connect and establishes the connection when the end-point needs to network. If a laptop or iPhone moves from WiFi to the 3G network, AnyConnect figures out what it needs to establish the connections. In addition, AnyConnect provides persistence, keeping all session state. The more intelligent AnyConnect gets over time the more it will fade into the background, being invisible to the user. Cisco is committing to a broad range of device support. Support for Windows XP, Vista, Windows 7, MAC OS X laptops has been made. Smartphones from Apple’s iPhone, Android and Windows Mobile are rapidly changing the enterprise mobility landscape which has been dominated by BlackBerry thus far and it seems logical that these end-points will be supported by Cisco at some point.

Flexible Policy Creation

For web security clients AnyConnect delivers an innovation around policy so that specific policies for remote workers can be distinguished and reported differently than desktop policies. This is important from a compliance point of view as IT leaders often set policy for workers within the network perimeter around “acceptable use” and from a compliance and liability standpoint IT leaders need to be concerned with “where” users go on the web. However, when an employee is home on their own time using their laptop to browse the internet, IT Security leaders don’t care “as much” about which web sites they visit, only that they are secure and protected from propagating threats. Therefore, AnyConnect allows IT Security leaders to set flexible on- and off-premises policy. For example, in-office employees may have policy set for both acceptable use and malware prevention; however, off-premises employees may have policy set for malware prevention.
Device Collaboration Takes Complexity Away From Mobile End-point

AnyConnect promises to deliver an end-to-end user experience, thanks to the engineering that Cisco has done to enable the above mentioned security products to collaborate between each other. One example of this value is during AnyConnect user authentication via the ASA configured for remote access VPN headend. The ASA authentication information along with the fact that the user is mobile is passed to the web security appliance so that both can apply the right policy without delivering another prompt to the user; thus allowing mobile-specific policy to be applied to the remote access session. For the mobile user this process streamlines their access as he/she is not greeted with two different screens (ASA and Web security) during authentication, just one.

Hybrid Hosting: The Way We Work

Backhauling internet destined traffic from remote sites over the corporate network is unfortunately more often done for security reasons. As many security leaders are requiring remote or mobile users to pass through the corporate perimeter to access SaaS applications and other Web content, application performance may suffer. AnyConnect performs performance optimization between VPN and Web access scenarios to significantly lower latency improving user experience even during backhaul scenarios. But as internet video traffic has skyrocketed there’s increased pressure and demand to maintain high user experience by allowing these flows to bypass backhauling and go straight to internet, or “enforcement points” such as a ScanSafe cloud. AnyConnect promises to seamlessly find the closest network attach point and optimal enforcement point, whether that’s the backhaul path, a ScanSafe cloud or even a Cisco ISR G2 running in a branch office equipped with web security capabilities. It’s logical that Cisco will release these capabilities over time.

Securing mobile/remote users via cloud-based services and desktop users with on premise security appliances have emerged as an important security design approach. Security services delivered to mobile and desktop users via on premises and cloud solutions respectively are what some call “hybrid hosting”. Policy consistency is important to a successful hybrid hosting implementation. That is the ability to define user access policy on one policy server and propagate it to on-premises and cloud providers, providing common enforcement, single consolidated reporting and a better user experience.

Key to hybrid hosting is the mobile client. Cisco has built connection intelligence into the Cisco AnyConnect Secure Mobility Client. AnyConnect manages connections by finding a trusted network, meaning assessing if the connection is a secure enforcement point. If an end-point is currently connected to an unsecured public internet link, but the user application requires a secure connection, Secure Mobility Client will find it without operator intervention. Optimal gateway detection is another feature that automatically finds the fastest gateway for VPN access and connects to it.

Security For Thin Client End-points: Full Context Awareness

As end-point devices become thinner and thinner, meaning devices with less processing power and memory, the harder it is to enforce security on the end-point. Laptops can run sophisticated AV and scanning software to protect the end-point, but this software will not run on iPhones, BlackBerries, Android, etc., as they don’t possess adequate resources to run the code. Therefore as end-points become thinner and their numbers balloon while threats continue to be more sophisticated and web-based the question is how to protect these devices and corporate IT assets from them if they become infected? The answer is to leverage the processing power that resides within the network. With the network providing security services on behalf of thin client mobile end-points, a consistency across devices is gained that is independent of end-point type. Malware or exploits are identified along with web site destinations, policy can be enforced, reporting is captured and in the process IT Security leaders gain visibility.

For web security AnyConnect has integrated Cisco’s Web Security Appliance, which provides malware security, acceptable use, access control, and data security for web traffic. By performing this in the network rather than the end-point it’s possible to obtain powerful security capabilities such as multiple layers of malware defense and web application controls which are very difficult to deliver, especially across a breadth of end-points via an end-point solution.

Malware defense includes Web reputation, which is delivered by Cisco’s Security Intelligence Operation (SIO), and is effectively a risk rating for how likely a specific Web object is to be hosting malware. Additionally, multiple AV signature sets are run in parallel on suspicious traffic providing better coverage than any single engine. Currently Cisco offers Webroot and McAfee, and is planning to offer Sophos in the near future.

For acceptable use, Cisco offers standard URL filtering. But URL filtering has become less effective as the number of pages on the Web is exploding, making it impossible for URL lists to keep up. To address this, Cisco dynamically categorizes web sites in real-time. In addition, Web 2.0 sites and tunneling applications mean that a URL filter is not enough to protect users or create meaningful policy. Enter application control. What Cisco has done to expose web traffic is build an engine that understands web traffic and applications that traverse within it. That is to be able to identify if the traffic is IM, WebEX, Facebook, Facebook chat, an application running on Facebook such as Mafia Wars, Twitter, streaming media, etc. With all traffic being distinguished Web Security Appliance’s application control can “block” or “allow” the traffic but more importantly provide greater policy granularity.

Consider this. An IT leader can develop a policy that allows chat on IM, but it’s a data security violation if a user attempts to send a file via IM. Or a user can participate in a WebEx session but he/she can’t relinquish remote control of his/her desktop because it’s a security violation. A user may be allowed to go to Facebook and read, but not post as this may be a potential DLP risk. Cisco’s AnyConnect Web Security Appliance offers this deep application control thanks to its parsing of web traffic and subsequent policy granularity.
It’s difficult if not impossible to obtain this level of security and policy enforcement even on a traditional mobile end-point like a laptop. Imagine trying to make it possible for all of those smartphones that are flooding into the enterprise; virtually impossible. This is the value of Cisco’s network-based approach.

With SaaS Growth, IT Managers May Become Less Relevant

With the large number of mobile devices that access SaaS applications that are out of an IT leader’s control and visibility, IT leaders have become concerned with their own relevance. Most SaaS purchases are in fact not from IT departments but from business unit or line of business managers. Therefore, IT becomes less relevant as IT leaders don’t see this surge in SaaS application use, how to secure it and protect existing IT assets from potential threats. As SaaS use grows so does this challenge to IT.

To address this challenge, Cisco is building in SAML (Security Assertion Markup Language) assertion into the Cisco IronPort Web Security Appliance, in addition to authenticating web traffic as it egresses the enterprise. IronPort already works with AD (Active Directory) and LDAP to authenticate users. Therefore, Cisco is adding the capability to create a SAML token, which will offer a better user experience by delivering single sign-on into SalesForce, WebEx, Concur, Google Docs, and all SaaS applications that support SAML.

SaaS Access Control

What this does for IT leaders is provide control back as IT can demand that their SaaS providers support SAML token, meaning that users can’t access the SaaS application directly but through the corporate network. So if a user is at home he/she can’t go directly to SalesForce.com and download a customer list onto his/her home PC or onto an unmanaged end-point. Users have to come back through the corporate infrastructure via AnyConnect to obtain their token. This provides IT leaders with both control and visibility independent upon where applications are hosted; be it in their data center or the cloud. With this link to all applications IT leaders can apply access control policy, data security policy and in the event of data loss or theft IT leaders now have granular forensic evidence too. With SAML token in IronPort, IT leaders have both control and great visibility that gives them the confidence to enable SaaS applications for workers and remain relevant. This is a huge point as many companies don’t know how many SaaS applications are being used. Cisco for example has over 350 SaaS application in use throughout their corporation, which is more than likely the rule rather than the exception.

One critical challenge SaaS presents is when employees leave or are terminated from their employer. How does IT remove access to these SaaS applications? It’s easy if there are only a few SaaS applications in use, but when the number of SaaS applications grows to the tens and hundreds the process becomes daunting and DLP vulnerabilities increase. With Cisco’s Web Application Controls IT can simply implement a zero day revocation; that is pull the terminated employee’s credential out of the AD and all access to every SaaS application is terminated.

What AnyConnect is offering IT leaders is the assurances and safeguards to say yes to employees to use the IT tools they desire, be it a laptop, iPhone, SaaS applications, Android, Blackberry, etc. For users, they get a simplified way to connect to applications independent upon where they are hosted along with the protections and safeguards once only available to them while in their offices behind the corporate perimeter. From a security leader perspective they get increased control and more security as AnyConnect extends out to that entire mobile workforce. Cisco’s AnyConnect promises to successfully thread the needle to avoid the typical tradeoffs that accompany security products such as security versus business process or security versus user experience. With AnyConnect IT leaders will be able to enable business mobility, increased user experience, and protect corporate assets through strong security services. In short the AnyConnect Secure Mobility Client offers a simple use model for mobile workers that leverages Cisco’s ASA, IronPort Web Security Appliance, SIO, and more then likely in the future ScanSafe, to wrap a corporate perimeter around its mobile workforce.

For existing Cisco customers that utilize ASA and WSA their implementation of AnyConnect is straightforward and the ability to absorb this innovation fast. These IT organizations would install AnyConnect Secure Mobility Client on end-points with required configuration changes to ASA and WSA. AnyConnect can be implemented piece meal too starting with AnyConnect Secure Mobility Client and ASA adding other security defenses when appropriate.

But to make AnyConnect a success Cisco needs to expand its smartphone support and prove that its AnyConnect Secure Mobility Client is indeed as simple and invisible as it claims. Also IT leaders will have to get comfortable with and trust the various enforcement points and its policy granularity. AnyConnect will have to work in conjunction with other security technology such as anti-malware engines, PIN locks and data encryption, plus remote data wipe to protect against lost devices. Look for Cisco to partner with others to deliver these aspects of mobile security. The key value proposition of AnyConnect is a simple yet powerful user experience. The success of AnyConnect rests upon Cisco’s ability to deliver on the promise of an exceptional user experience with an always-connected remote access and security architecture.

ShareThis