August, 2000 Volume 18.8
Netwatcher (ISSN 0890-5800) is a monthly publication of CIMI Corporation. Subscription information is available here. Copyright © 2000, CIMI Corporation. All rights reserved. No publication or reproduction of this document is permitted without the express written consent of CIMI Corporation.
 |
Management Briefing |
The crazy nature of the financial markets these days has created many a monster in the tech space, but none with nobler inten-tions and less chance of success than the CLECs. Born of the Telecom Act, the CLEC concept as it was originally formulated was more a role than a total business. Hype turned it into a kind of prospective money machine, and dozens of non-players were launched. Now, dozens are going to die off, but just how that's going to happen is something for our subscribers only!
 |
In the Know |
There is no question that network management has been the poor stepchild of networking, and the network management beat has been an assignment that most reporters dread. One long-time reporter in this space told us wryly that the only way he’d get onto the front page would be to die.
Well, we may have a better strategy for our long-suffering reporter. As we enter this new decade, there’s a good chance that we’re also entering the age when software defines network services, and thus perhaps becomes the most important thing in networking.
Software has always been something that people knew was at the root of most network functionality. Cisco even branded its router software as “IOS”, causing other vendors to begin to productize their own imbedded code. But network management remained outside this new and at least somewhat exciting area. Most still pictured it as a bunch of grizzled old standards-writers doing the TMN thing over in some foreign land.
Well, it’s not your mother’s network management any more, folks. The industry is in the process of remaking the whole space, and that process will require some education and adjustment on the part of those in the industry. In fact, we’re breaking up what used to be called “network management” into two distinct pieces—network management and service management. It’s the latter that we plan to address, because it’s the latter that nobody really understands—though everyone in the industry must eventually come to an understanding of it.
We hope to help that process of understanding along. This is the first piece in a series on network management that will parallel our fall, 1999, series on VPNs. That means that we’ll provide a couple of lead-in pieces, and then throw the debate open to all vendors. We’ll publish the results in a free PDF file at the conclusion of the series, probably next summer.
In this segment, we’re going to look at the “management software” space as it applies to carrier network applications. We’ll trace the roots of each of today’s product segments, and explain what’s happening there. We’ll also develop a picture of why the sum of the management parts isn’t necessarily adding up to a glorious whole, and what’s being done about that shortcoming. In the next segment, we’ll develop a reference architecture for service management, and it’s that architecture that we’ll invite vendors to respond to—with comments, product information, etc.
Vendors, even non-clients, please take note! We will provide the two “set-up” issues on this topic at no charge to any vendor who wishes to make a submission to our review process. All that is required is that the vendor have a web site that validates the fact that they do have a product, and that the material supplied for our publication be offered for publication without restriction. We’ll publish the material “as-is”, without correcting even grammar or spelling errors. Please limit your submissions to 3,500 words and four illustrations, with the latter in PowerPoint format.
As noted above, we’ll make all this material available from our website on a no-charge basis, when we’ve collected it into a single PDF. This issue, as the first introduction to the topic, will also be provided on an open basis on the Internet, but the remainder of this series is for subscribers only.
Network management software evolved out of the growing intelligence in network hardware, particularly from the trend toward non-TDM networking. With protocol-oriented networking, the processes of maintaining a network shifted from one of recording manual activity to one of controlling devices directly to bring about desired behavior on a per-customer, and complete network, basis.
The original focus of network management was device management, the control of the smarter hardware boxes that, even in those days, made up networks. Early router products were set up through command-line interfaces, and similar tools were also used on some of the early packet switches. It was quickly determined that this approach was too time-consuming and error-prone, and so vendors began to develop software that would handle the management chores using a more forgiving interface, and with better process automation.
Standardization appeared first in the form of network management protocols like SNMP, designed to get and set variables in a standard management information base (MIB) that controlled device operations. Unfortunately, most vendors extended the “standard” MIB, or continued to rely on other proprietary interfaces to exercise the full capabilities of their devices. Still, device management was an improvement over command lines and text-oriented parameterization.
As devices were built into networks that had to cooperate to create routes and balance traffic, the problems of managing devices reappeared at the network level. A higher-level process was needed to allow operators to mold a set of devices into supporting a common mission. Initially, this was accomplished through extensions to the basic device management systems, but by the early 1990s it was clear that some structure had to be brought to the matter. The result was the Telecommunications Management Network (TMN) standard set, promulgated by what at the time was called the CCITT (now the ITU-T). TMN created a layered management architecture, with the bottom layer the device or “element” management layer, the middle layer the “network” management layer, and the top layer the “service” management layer.
The TMN management model was generally adopted by the more traditional equipment vendors (ATM players, for example), but more slowly by router and other data-specific players. Nevertheless, it’s safe to say that nearly all carrier-oriented equipment today is associated with a TMN-architected software suite, even if the vendor only provides the lowest layer in the model.
TMN has its shortcomings, however. The following is what could be called a “short list” of them:
1. TMN doesn’t really standardize the interfaces between the layers. Thus, if a vendor adheres to the model but doesn’t supply the full suite of software, there’s no guarantee the lower-layer that is provided will work with someone’s higher-layer components.
2. TMN was really designed to support the OSI management standards (the Common Management Information System, or CMIS). OSI-based management protocols are nearly extinct in the US, and not doing all that well in the rest of the world.
3. TMN works best for networks that produce a single service from the native behavior of the included devices, and in a single-vendor environment. If these conditions aren’t met, a so-called “TMN-based” approach won’t necessarily do everything needed, and may not do anything at all useful.
4. TMN doesn’t deal effectively with “back office” facilities like the operations support systems that provide billing, order processing, etc.
For all its problems, TMN doesn’t seem to be going away. In fact, nearly every new player in the carrier equipment space kisses TMN babies with every management launch. But TMN doesn’t seem to be getting any better either.
Another type of management software that’s been evolving over time is the operations support system or OSS. These software systems were originally built for the voice networks by AT&T when the Bell System was still a glorious whole, and Bellcore (now Telcordia) still has a strong position in that market.
In their original concept, OSSs were what were called “batch” systems, making them more record-keepers than real-time creators of services or service elements. OSS systems provided for journaling calls and billing, for circuit inventory, for order entry, and for problem management.
Just because you decide to replace circuit-switched networks with packet-mode networks doesn’t mean you don’t bill for services, obviously. Nearly all the OSS functions could be viewed as “back office” or administrative elements of a service provider’s network, and so nearly all have a counterpart in modern networks.
Most of the original OSS players, including Telcordia, have moved their products into the packet world. In addition, literally hundreds of smaller outfits and startups have developed back-office OSS elements designed for packet networks. In some cases, these products are linked to specific transport technologies, such as IP. One of the most popular new software startup areas, for example, is service billing software for ISPs to provide for usage or class-of-service billing.
You can’t get away from OSSs, but that doesn’t mean that they deserve a key role in service management. The problem that OSS products introduce lies in their disconnect from the process of creating the service on the network. Billing, order entry, and other back-office OSS activities are derived from the operation of the service, meaning that the order entry system supports service creation or provisioning, and the billing system operates on parameters journaled either from order entry (fixed-metric bills such as billing by port speed) or from network statistics (usage bills). If the OSSs don’t provide the mechanism to actually create the service, then how effective can the integration of the OSS with the actual service creation and support process be?
Billing systems for modern communications services must support the concept of a credit or bill-back from the customer service charge, should the service fail to meet contractual requirements expressed in the service level agreement (SLA). For older TDM services, the SLA was expressed in simple digital error form; error-free seconds, severely errored seconds, etc. In newer packet services, the SLA parameters are more complicated, but the real challenge comes in how one measures some of the parameters at all.
How do you measure packet loss? A missing packet isn’t replaced by some kind of virtual object that follows along through the network announcing that there used to be a data packet here! Packet loss can be measured by counting packets, but the precision of this process is limited by the ability to synchronize clocks to measure the same period at both ends of the connection.
Another strategy with less technological baggage is the concept of a “telemetry sub-channel”, popularized by Visual Networks. A packet channel (a frame relay virtual circuit, in the early Visual products) is divided into a main channel and a low-volume telemetry channel through the addition of a new encapsulation header. Packets with sequence numbers are sent periodically over this sub-channel, and loss there can be correlated with loss on the connection overall.
The telemetry approach also answers the problem of how to measure delay. Since packets aren’t necessarily time-stamped, it’s hard to know how much they may be held up in the network. Telemetry packets with timestamps can be used to determine end-to-end delay, again subject to clock synchronization on the part of the two stations.
Because the problems of service assurance have been vexing to users and providers alike, many have viewed their solution as the main issue in service management, and thus take the position that companies like Visual or Concord have the needed technology to be called a service management system. The problem is that this view is valid only if one decides that the solution to an SLA problem is to bill a credit to the customer for the failure. Both customers and providers reject this; they’d rather have the service work as promised.
It doesn’t take rocket science to see that perhaps the key recurring issue in our comments above is the issue of integration of the elements of a management system. We have developed band-aid solutions to a number of network, customer, and service management problems, but not an integrated solution.
One of the “indicators” that integration is a real problem is the extent to which it is being solved by explicit network integration projects using high-priced international players. Big consulting houses are providing the bodies needed to do object-oriented programming to mate various network software elements to a common object middleware toolkit, thus providing a means of integrating their activities.
The overall approach to this middleware integration is the construction of workflow control through the use of an object bus and a bus manager or director. The manager provides the sequencing of tasks by having a state/event table of objects and activities that can be referenced to insure that the output of order entry, for example, becomes an input into provisioning.
Object-oriented software built on the CORBA standard can often be integrated with other similar components using a combination of middleware tools and custom coding. Clearly, though, this isn’t the approach the market would like to take. For one thing, there’s no guarantee that the elements you’d like to integrate will really fit together, even if object-level compatibility is created via customization.
It’s also true that the simple need for integration may be an indicator of the overriding problem here. Software doesn’t fit together naturally, thus there’s a need for integration. Does the software fit conceptually? Is there a common vision of a “service” and a “network” that drives the software elements? If not, it’s hard to see how effective simple integration at the object level could be, and how effective any more complex integration efforts could be.
If there’s a place where a vision of a service and a network have to exist, or at least would seem to have to exist, it’s the place where provisioning or service creation is done. As usual, though, what’s logical and obvious isn’t necessarily what’s true.
The problem lies in defining what a “service” is. Remember that in old TDM networks, the network and the service were really sort of the same; provisioning effectively created a service by defining a subset of network resources that were linked to a customer along a traffic route. Service creation is route creation. That simplistic vision makes “service management” a layer of the TMN model (as we’ve already noted) and also makes it easier for “provisioning software” players to build their products.
Three popular service creation models have emerged in what we’ve perhaps unkindly called the “simplistic” vision, as we’ve migrated into the packet world. All, on the surface, seem reasonable, but we’ll dive below the surface on each to expose their limitations.
The first service model is what we could call the “Internet” model. In this model, there is no explicit resource allocation because the network is connectionless. Customers vie for network capacity in a random way, and capacity shortages result in generally unpredictable performance impacts. These impacts can be eliminated by providing enough resources to insure no congestion occurs. It’s not that there’s no QoS, but that there’s no QoS control. Everybody is equal; network Marxism, in short.
The second model is the “Directory-Enabled Networking” or DEN model, supported by giants Cisco and Microsoft, among others. In the DEN model, the network is assumed to be connectionless, as before, but mechanisms exist in the network to provide traffic priorities to some users or flows. These mechanisms are invoked from the customer interface point through either a control-plane protocol (RSVP, for example) or through packet tagging (TOS bits). The process of setting TOS bits or authorizing RSVP-directed flow prioritization is accomplished through a directory lookup. Edge-driven networking, in short.
A version of this can be applied in connection-oriented networks as well. If we assume that the underlying network has the ability to set up switched virtual circuits (SVCs), then customer connectivity and QoS could be assured by using SVC setup to create paths with the characteristics needed.
In both these “model two” approaches, the benefit we’ve realized is disconnecting the process of service creation from the management of network resources. Inside the network’s devices there is a set of processes that discover topology and inventory resources. These processes can then thread a path from device to device, given only where the path starts and ends (source and destination address for IP, for example, or called and calling party address for frame relay or ATM). The network is effectively self-routing and self-controlling and we need only frame our requests in a language it can understand to invoke these procedures.
But what if it isn’t? A network that supports connections and that doesn’t support SVCs must (by default) support permanent virtual circuits (PVCs). Setting these up is a process of defining a route through the network based on customer connection points and network topology/resources, and then commanding the network devices to “harden” this route plan into a set of resource commitments. That’s “hop-by-hop routing” and it’s also the third model of service creation.
Early in the development of ATM and frame relay network equipment, vendors recognized this need for hop-to-hop routing of PVCs, and many responded with what might be called a “soft PVC”. In this approach, the underlying network does have route topology discovery capabilities, and also the ability to maintain a resource inventory. The ability to signal for route setup, however, is removed from the network’s edge and emplaced in the management software. This shields the network from having to validate billing authority for SVCs, and also limits the ability of network subscribers to create arbitrary routes that might not follow expected network expansion patterns, thus stressing the network’s performance.
The problem with this approach was its single-vendor nature. While there have been standards developed for inter-carrier routing and network-to-network interfaces, they are not widely implemented or tested. Furthermore, they don’t generally apply to very small enclaves of devices, or to individual nodes. To provide a generalized way to do hop-to-hop routing, a multi-vendor approach is required.
A small Canadian firm, Syndesis, created what was probably the first “hop provisioning” systems with its Activator software. Activator might be viewed as a network management layer function of TMN that could overlay a variety of element (device) management systems from a variety of vendors, thus enabling global route creation.
For all these advances, however, a single problem remained. Networks are OSI-modeled entities, and thus services could be said to exist at any OSI layer at which the network could function. Since the highest OSI layer in the “network” is Level 3, services could exist at Level 2 or 3 (Level 1 is physical-layer interfacing). That suits the DEN model of provisioning, but not the connection-oriented models, since these models create only Level 2 services. What’s a network to do?
In fact, the problem of Level 3 services is simply the tip of a very ugly iceberg. The nature of a network service is often more complex than meets the eye. Is the Internet a service? To virtually all its users, yes it is. And yet, the Internet “service” clearly consists of more than routing traffic. We need to get an address to get on the Internet, so is DHCP a service element? We need to decode URLs to access web sites, so is DNS likewise a service element? How about email, or web hosting, or ASP application hosting, or video serving, or a host of other things? Services, in short, are in the eye of the beholder—or, more correctly, the payer. The buyer defines the market.
Ah, but who then defines the service on the network? This new definition of services breaks down the symmetry of DEN and self-routing SVC or soft-PVC connectionless networks, because DEN networks could imbed not only the features of Level 3 routing lacking in switched networks, but also the elements of network applications like DHCP and DNS.
Well, then, why not just adopt DEN and connectionlessness and forget all this virtual circuit stuff? One service provider noted to us recently that there has never been, anywhere in the world, a truly profitable connectionless service. Service providers earn hundreds of billions of dollars from connection-oriented networks, and these incumbent earners are interested in developing a model of connectionless services from connection-oriented networks. This would keep them in their craft and planning comfort zone, maximize their re-use of existing infrastructure, support a logical customer service evolution, and reduce churn in the customer base arising from what might appear otherwise as a radical shift in provider strategy.
There are also potential problems with the DEN model even in the case where an IP network creates IP services. The problem is that DEN presupposes that the behavior of a service is a subset of the behavior of the network, and that the factors allowing the selection of the subset can be determined at the edge. The problems with this can be:
1. Some services may involve features not present in the network. ASP services are clear examples of this, but so is the ability to assign, route, and decode “private” or non-unique IP addresses.
2. The process of policy validation at the edge may involve considerable real-time directory activity that could impact network performance. Do we do a DEN lookup on each packet, for example?
3. It’s not clear how easily the service provider could determine if the resource requirements presented to the network by the DEN process were within the limits of the network to support, since no explicit resource reservation is attempted in some cases. Do all these high-TOS service classes still fit within the overall capacity limits, for example? How, given that TOS bits are assigned by distributed directory lookup, would we know?
It should be clear that these problems are not all that dissimilar to the problems faced in attempting to create a Level 3 service on a virtual circuit infrastructure. In fact, what we’d like to have is an architecture for “service” creation that would reduce to DEN in simple cases, but extend DEN as needed to meet the most generalized cases of service needs. This architecture should also support the range of infrastructure likely to be found in networks today, so service providers aren’t forced to displace equipment to play in the new service market.
We finally arrive at a key point; the main ingredient in a true “service management system” is a conceptual model that links customer services to network infrastructure over the full range of both service and infrastructure that might exist in the market. This model should reflect the evolving nature of services, particularly their increased reliance on what might be called imbedded processes (DNS, etc.).
Service creation, under this new SMS agenda, is the generation of a service model that accurately represents both the communications and process requirements of the customers’ services. A customer has a “service model” that represents each different service, meaning each service that has a different OSI structure or connection/process ingredient set. We believe that this service creation and service modeling concept is evolving in the industry, but that there are as yet no true incumbents in the space.
The network would then be similarly modeled, presumably based on information gathered from the network management system. Provisioning, then, is the matching of the service model to the network model, so that service resource requirements can be mapped to the actual network.
Model-to-model mapping could offer elegant solutions to a number of problems. First, the process could be made largely rules-based, reducing or eliminating the manual activity that often forms the basis for modern data network service creation. Second, the approach could provide an easy way to relate service and network issues and problems, from quality management to fault and failure management, and fault correlation. Finally, the framework could be extended to cover not only the services we already buy and sell, but those we’re contemplating as well.
Given a conceptual model of services (in fact, of all services being supported), and a similar conceptual model of the network, we have a framework for integrating back-office activities (normally service-related) and network management activities (hardware and circuit-related). In fact, we can now talk about the overall harmonization of the network software process. That’s the mission of service management systems, as we see it. A service management system is a software formulation of both a network and a service model, structured to permit the creation of services and the integration of service sales with the process of network support.
But will it harmonize SLA enforcement and OSSs? Will it simplify the current integration process, making life easier for service providers and less lucrative for integrators? How much of this is available today? How well would the current products vendors call “service management” products fit our definition? In order to establish this, we’ll have to look much more closely at the topic, and the products.
In our next issue, we’ll introduce a service management framework that will attempt to lay out the functional elements and requirements of a service management system. This will be modeled on our VPN approach of last fall. We’ll invite vendors to submit their own offerings, hopefully keeping at least somewhat to our framework. We expect that some of these submissions will be published late this year, and the balance into next spring. Again, none of the vendor submissions will be put on the Internet until we’ve developed the entire series and published the results as a single PDF.
We hope you'll find this as interesting as the VPN series was!
 |
Strategies |
This section is being omitted this month to save space for the Service Management series introduction in the section above.
 |
Down the Line |
Next month, we’ll be developing the template for
service management consideration, the framework on which we’ll be evaluating
vendor offerings in this space. We
still have our RBOC subsidiary status on hold, pending some stabilization in
this key area. We also expect to be
looking at the area of service switches or service point-of-presence hardware
this fall.