The Lippis Report

While business decision makers are increasingly layering real time applications such as voice and video over their IP networks thanks to its economic and business value, technical professionals have grown increasingly alarmed at the performance of their converged networks. It turns out that many have not assessed their IP networks to support real time flows and thus find performance degradation of their new converged networks robbing them of benefits originally sought. As performance falls, technical managers try tweaking configuration setting on switches, routers, communication managers, wide area services, appliances, and buying more local and wide area bandwidth to alleviate the problem. All the while the business decision makers scratch their heads and wonder, at times out loud, did we make the right decision?

As converged network performance degrades so to does all application delivery, relying upon the network for its connectivity service. If converged networks can’t be controlled, then the industry will take a giant step backwards, and giant it would be. There are approximately 45 million IP phone lines installed. It took 3 years for the industry to ship some 8 million IP phones; it ships that many plus in 3 months now and the growth rate is increasing. 22 million
IP phone lines are forecasted to ship this year alone. Gartner estimates that more than 80% of enterprises worldwide will install VoIP over the next five years and that 85% of data networks aren’t ready for it. It’s estimated that for every IP telephony system installed there is a major network infrastructure (switches and routers) upgrade which is usually 50% or higher of the IP telephony system acquisition cost. If service quality cannot be controlled and
guaranteed, then all of this progress will slow to a crawl.

The Crisis is Real

Consider MDRC, a nonprofit research organization, who implemented a converged IP videoconferencing and VoIP network with 200+ users across two remote sites. The network included Cisco Call Manager servers, Cisco IP phones, and Polycom 8000 series videoconferencing systems. To simplify WAN management, MDRC outsourced their IP video and VoIP network to a service provider. The service provider’s MPLS-based managed service with QoS monitoring and contracted
guaranteed SLAs were implemented. The outsourced network worked at first, but videoconferencing picture quality suddenly degraded precipitously with pixilation, freeze-frames and garbled voice.

A Lehman Brothers IT executive had discussed with me their early efforts to implement a converged network. They had two major converged network “hiccups”, which brought down their IP telephony network for some 5+ minutes, halting their trading floor. At Lehman Brothers’ minutes equal millions if not 10s of millions of dollars.

These are but just two stories, which are repeated over and over again throughout corporations in the global economy. Why don’t we hear more stories about converged network performance problems in the trade press and elsewhere? Because these stories are unflattering and embarrassing. Most corporate communication departments forbid IT departments from talking about these problems for fear of damaging their corporate brand or worse.

Gaining control of converged network performance includes implementing Quality of Service (QoS), measuring traffic flow performance, voice quality, delay, jitter, packet loss, etc. But to deliver even one of these components, QoS, in converged networks the industry has served up a complex set of protocols, which unfortunately becomes even more complex as different vendors implement different QoS approaches with little interoperability. Switches, routers, firewalls, communication managers, end-points, access points, supplicants, appliances, service provider services, etc., all have their own approach and job to do to assure that converged network performance is high. Many, if not all, IT departments first look to the large vendors such as Cisco, Avaya, Siemens, ProCurve by HP, 3Com, Nortel et al., to deliver the tools and features necessary to keep performance high. But there are very few corporations who have implemented a converged network infrastructure with just one vendor. Each of the large vendors can solve only a part of the converged network problem. The only real solution has risen from a series of start-ups that are neutral and can address mixed vendor converged network performance issues.

Finding Performance Solutions

There are solutions which allow IT departments to gain control over converged network performance and support real and non-real time traffic flows in production networks. A large group of companies are offering solutions to tame this complexity and reduce its associated high operational cost. These firms traditionally fell into three categories: 1) test; 2) assess; and 3) monitor. But the market has changed and many firms provide all three services and a few have started to enter into a new category of “fix”. It’s easier to categorize these firms based upon their core competency as many have added IP telephony or convergence performance services to their existing platforms. For example, firms who specialize in application flow management are adding IP telephony monitoring, those in packet testing are adding VoIP testing too, to mention just a few.

So who are these companies? They are firms such as Prominence Networks who focus on QoS and those in the traffic shaping/policy management market such as Converged Access, Packeteer, Peribit, and Quirtum. Then there are those with network test background and strong IP telephony offerings such as Brix, Corvil, Qovia, Viola Networks, Integrated Research, InfoVista, Ixia, NetIQ, Network General, Network Instruments, SyncVoice Communications, Clearsight, Spirent,
Candela Technologies, Clarus, Empirix and WildPackets. Another class of companies are those with deep roots in application flow management which have or are adding VoIP to their existing platforms, such as Network Physics, Fluke/Visual Networks, Apparent Networks, and NetScout.

There are a lot of companies in this space all vying for a piece of the $1 billion market. Frost & Sullivan expects the worldwide VoIP monitoring and management market to grow at a CAGR of 55.6% from $50.7MM in 2004 to $297.1MM in 2008. The broader market for convergence assessment
and management tools is expected to be over $1.0B by 2008.

We don’t have space to cover all of the above firms so we’ll dive into just two here and cover
more over the year.

Prominence Networks

Prominence Networks develops and provides products to accelerate, enhance, and automate the deployment and management of real time traffic such as VoIP and IP video (IPV) in converged enterprise networks. Prominence’s Clear Call converged network management products service delivery of IP voice and video through an integrated set of assessment, engineering, provisioning, monitoring, and management tools which provide enterprises cost-effective, secure,
and guaranteed end-to-end QoS.

Prominence Networks provides the Clear Call Assessor/Controller appliance and Clear Call Director management and reporting system. With the Assessor and Director enterprises can undergo a Convergence Assessment which provides proactive planning and monitoring of the converged networks to ensure its readiness to support real-time flows. To assure converged network performance, the Clear Call Controller and Director work together gathering statistics, analyzing flows and tweaking configuration settings to guarantee end-to-end QoS allowing wide area bandwidth to scale dynamically with traffic flows/requirements.

Prominence improves the business case for VoIP and IPV by offering guaranteed service quality, dynamic bandwidth control, application awareness, real-time monitoring, end-to-end traffic engineering, and automated detection and correction of network problems. These features combine to enable significant cost savings, increase operational efficiencies, and accelerate the implementation of convergence for enterprises, service providers, and government agencies.

Viola Networks

Viola Networks provide a software-based architecture to assess, monitor and manage a converged network’s performance. Viola has market presence in over 25 countries with over 200 customers. Their NetAlly solution is certified interoperable with nearly all the major IP telephony vendors including Cisco, Avaya, Mitel, Ericsson, ShoreTel and Inter-tel.

Viola found early success with service providers being part of their hosted and managed IP services offerings. Recently they expanded their focus to sell directly to the enterprise market. Their flag ship product is NetAlly Lifecycle Manager, a distributed software-based agent architecture which remotely monitors and diagnoses service levels while controlling and correlating metrics on a central service management system. NetAlly combines active and passive
agent technology providing a wide view of the performance of converged networks. Because the architecture is entirely software based the price barrier of entry is low and preservative monitoring is cost effective allowing call quality analysis between all VoIP end-points and network segments isolated in the connection path.

Active testing is based on intelligent agents deployed at the user, site, or network device level to monitor and troubleshoot network aggregation points, branch offices, or internal user departments. Active agents simulate VoIP calls and measure service quality using Mean Opinion Score (MOS), identifying call quality impairment (such as delay, loss and jitter).

Passive monitoring is based on SNMP polling of network devices and collection of information from IP telephony and IP phone proprietary APIs. Call Data Record (CDR) collection from IP-PBXs. Passive data when correlated with active agent’s test results provide insight of network transport between VoIP endpoints with call quality problems. CDR data provide call history of specific users in context with call quality analysis produced by active testing.

In short, active testing allows proactive service management and troubleshooting while passive monitoring allows correlation with real users and network devices to determine the impact footprint of a service-affecting outage or degradation.

Controlling Converged Network Performance

Lehman Brothers’ IT staff recommends that the converged network should be rock solid and that data and voice staff should be in unified organization. Converged network technology needs constant TLC (tender loving care) to get the most out of it; it’s not the same as TDM PBXs.

MDRC tried for months to diagnose where the problems were occurring, but they had no visibility into the WAN and were in need of a specialized solution that would provide real-time assessment, monitoring and management tools for videoconferencing and voice quality and validate SLAs to help bring back network reliability.

For tools to control network performance, IT staffs have to look beyond the large IP telephony and network infrastructure players and sort out the above vendors to address unique convergence performance issues. Out of the firms above, there is not just one company that can solve all performance problems. Most, if not all, IT staffs will deploy a set of solutions to assess, test, monitor and fix performance degradation problems within converged networks. The key reality to keep in mind is that on converged networks performance degradation impacts application flows plus voice and video communications, which is why multiple companies will have to be deployed to keep users and business decision makers happy.