Email a colleague    

March 2011

ILEC Business in the Balance: The Case for Probe-Based Traffic Monitoring

ILEC Business in the Balance: The Case for Probe-Based Traffic Monitoring

For the last 20 years, most small ILECs and rural LECs in North America have relied on the tandem companies to provide them with their source billing usage data.  It’s been a marriage of convenience.

Rather than go through the expense of collecting their own interexchange (IXC) billing usage, why not have the large tandem company such as an RBOC, etc., supply billing tapes to the ILECs to enable the ILECs to turn around and bill the IXCs and other exchange carriers for access to their local networks?

This happily organized world is steadily crumbling under the weight of massive industry change.  Today, non-Bellcore-compliant switch technology is being deployed in both fixed and mobile networks.  And this has opened cracks — even gaping holes — in the venerable Automatic Message Accounting (AMA) system that for so many years has kept telecom peace on the continent.

Here to tell us about these developments is Darrell Merschak, national sales manager from Carrier Management Systems Inc., a small company with a large installed base of 120 small to mid-sized carriers.  CMSI’s flagship product, NAMS Firefly, mediates AMA records, collects/polls switches, and is now migrating its customers to a full-fledged SS7 analysis system that correlates and reconciles AMA billing data streams that are often missing critical call identification data.

Dan Baker: Darrell, your company’s history is a fascinating one because you started off as a telephone company and only then became a probe manufacturer and billing analytics supplier.  Can you give us a quick background on the industry problem that caused you to migrate to SS7 solutions?

Darrell Merschak: Yes, and thank you.  Here’s the issue, Dan.  New switches from foreign countries are entering the market and some of those switches are not compliant with North American standards for the proper creation of call detail records.

Even the rural ILECs, who you might think would be the last to catch the trend, are deploying foreign made switches in their networks because the prices are low in relation to the overwhelming need for the rural ILECs to deploy broadband technology and find a cheap way to counter competitive threats in their service territory.

The growth of mobile/wireless is also contributing to the problem.  Wireless providers are the lost stepchildren of intercarrier billing.  Many wireless carriers don’t even think Carrier Access Billing (CABS) affects them.  However, wireless radio network traffic is often downloaded to wireline networks for backhaul and the tandems pick up that traffic and deliver it for termination.  The trouble is that wireless traffic is harder to identify because — once again — its call record structure didn’t grow out of North American CABS standards.

So essentially when the tandem sees this traffic it says: “Gee, this record looks strange to me.  The CIC field is all zeros, and the calling party is not even included.  I can’t make sense of this call traffic, but I’ll pass it along anyway.“

It’s become so confusing that the tandem companies, whose job it is to figure this out for the smaller operators, have thrown up their hands, and ironically, some even remain in denial of the problem.  As a consequence, today the large LECs or tandems have small operators sign Annex agreements, 12-page addendums to the 300-page interconnection agreements that basically states: “You will now pay us a transit fee for your traffic, and from here on out we will create billing records for you only on a “best efforts basis.“

Therefore, though the large LECs are having a hard time identifying wireless traffic, it’s the rural LECs and CLECs who are most adversely affected and caught without a reliable and accurate source of usage data for billing.

Now I understand you’re proposing ILECs deploy SS7 network probes to solve this problem.  But why is that the best alternative?  Why not simply turn on the old usage collection systems?

Well, if you turn on your AMA collectors after 15 years of inactivity, you suddenly find that it’s hard to do.  The knowledge and expertise is gone, and it’s costly to retrain people with the appropriate skills.

I once worked at a company that relied on the tandem company billing records for 12 years, and when they attempted to begin billing record creation using their own switch equipment, I discovered that it was 16 months since the last time they updated their switches with the latest NPA NXXs from Telcordia’s LERG database.  It was obvious to me in that period of time there are a ton of telephone-number changes and you can’t possibly create complete and valid billing records.

As the industry evolves and matures, you’re going to need to move to probe technology anyway to accommodate more foreign manufactured switch equipment, new softswitch technology, and bandwidth-hungry consumer devices that will operate on LTE and WiMAX, which are pure IP packet technologies.

We believe the only viable option today is to deploy probe technology and have the probe stand at the signaling-network level to act as a uniform source to monitor record every one of the multiple handshakes required before a voice call goes through.  In short, it is a detailed and comprehensive record of precisely what the network actually did.  The switches are communicating with each other so there’s no mistake about it.  In fact, for years the central office engineers took all that very robust SS7 data and used it to groom their networks.

Moreover, probes are versatile enough to monitor both SS7 in TDM networks and SIP in IP packet networks at the same time.  Therefore, they not only provide robust detailed data because it’s based on the network handshakes that actually occurred during the set up and transmission of a call, but they are also switch-agnostic and provide a uniform source of network monitoring data irrespective of the types or combinations of legacy switches, soft switches, routers, brouters, or bridges deployed in the network.

How big an issue is this?  What percent of usage records can’t be classified for billing?

I don’t have a precise number on the industry error gap because it varies tremendously by company and service area setting, but I do know some extreme examples such as the former e.spire Communications, whereby 42 percent of their traffic was phantom traffic — traffic they couldn’t properly identify.  The fields were stripped so they didn’t know who to bill.  It was personally conveyed to me by the former vice president of Rates and Regulation that e.spire was told: “Don’t worry, it’s a transitory traffic spike that will go away.“ Well, it didn’t go away and e.spire was forced to file for Chapter 11 bankruptcy protection.

Another issue is “bypass,“ which NECA (National Exchange Carrier Association) calls “access avoidance.” In this situation you have toll-type traffic sneaking onto local or EAS (Extended Area Service) trunks disguised as local traffic.

However, with SS7 data, you can reliably say, “Oh no, that should be a toll call,“ because you can point to an originating OCN code and a JIP (Jurisdicational Information Parameter).  Moreover, there are often other SS7 fields that can be examined which are either not spoofed correctly or inconsistently spoofed relative to the other SS7 fields and even some AMA fields.

For example, depending on what networks are involved, you can often directly determine where calls originate from SS7 by examining the OPC (originating point code) data and therefore at the least you’re getting an idea of the interconnect IDs.  Many times if a field in the AMA record is stripped, it may still be available in SS7.  Also, SS7 has some additional fields that can be queried as surrogates for AMA’s calling party number such as the charge number, the originating LRN, etc.

Moreover, when you begin to correlate and compare the AMA or EMI switch generated record data to the SS7 data you notice some obvious differences.  Regarding the obvious differences you only need to be as skillful as, let’s say, the English novel crime investigator Dr.  Watson, to solve and reverse this type of phantom or bypass access avoidance revenue loss.  With respect to the more complex cases you need to develop greater skill and become a Sherlock Holmes, but this is possible in short time with the tools available now.

Who are your competitors in the market for SS7 analysis software?

Well if you insist, to name a few: Agilent, Tekelec, Tektronix, Tekno Telecom, GL Communications are all in the market.  Tekelec is the 800-pound gorilla now because they serve as the Service Transfer Point (STP) for the network.

Agilent is also well-established, especially at the large LECs.  In fact, I’ve seen clauses in Annex Agreement contracts that stated — “if we disagree on billing exchange record accuracy or record jurisdictionalization, Agilent SS7 technology will be used to resolve the dispute.“

You can bet these probe manufacturers are eager to showcase how you can leverage their SS7 technology for revenue recovery but the hard truth is that SS7 probe technology has tended to work against the smaller players because they couldn’t afford to own it.

However, our NAMS product is far more affordable for the lower tier markets — and yet we also serve the needs of tier 2 or even larger carriers, and this is software that a carrier owns from Day 1 and they don’t have to pay rent on.  The beauty is it allows you to directly correlate an AMA or EMI record to an SS7 record.

Moreover, with our NAMS system you don’t even have to repair the damaged record if you don’t want to.  You can convert the exception SS7 records (which correlate to AMA error bucket records) to AMA or EMI format which allows any CABS billing company or system to directly upload and invoice.

In the last two years, the FCC has been pushing for the legislation for wider deployment of broadband services to rural America.  What’s your take on that?

In my opinion, broadband provisioning by the rural ILECs is a double-edged sword.  You are essentially creating a packet information superhighway into your network, which is exactly what the major VoIP providers need to steal your subscribers.

Also, as proposed by the new regulation, the original rate-support mechanisms such as USF and ICC (intercarrier compensation) will be diverted to support broadband and many of the traditional subsidies may be dramatically reduced or diminished and in time ultimately eliminated.

If the ILECs don’t win the FCC regulatory battle and also catch on and start monitoring their network traffic, understanding the nature of the traffic, and figuring out how to earn revenue on the traffic, their businesses will suffer considerable harm.  The logical conclusion is that these carriers will become so financial weak that either they will need to declare Chapter 11 bankruptcy protection or become a target for business takeover by another carrier.

I have already witnessed evidence of this because as a national salesman I call on a lot of telecom companies and an increasing number are no longer in business.  You argue that this is a Darwinian survival of the fittest, but the prospect of becoming extinct is not appealing.

The good news from a technology standpoint is that carriers can protect themselves and defend their networks because the same probe technology can monitor SS7 TDM or SIP-based IP networks and produce billing data CDRs (call detail records).

There are also a few companies that suggest and point to DPI (deep packet inspection) as a way to look at the contents of the traffic, but the volumes are huge and I don’t believe that all this extra data extract work is necessary for a carrier to obtain sufficient billed revenue compensation.All you really need to inspect is the SS7/SIP stream of that traffic — sometimes called SPI (shallow packet inspection) — and with that you’ll have enough information.  You’ll know that a communication or bandwidth window was open for a certain duration and you will be able to render a bill.

In addition, using probe technology to your maximum advantage, you can understand the nature of the traffic — how much is VoIP and from where it originates.  So if an operator is not paying his freight, you can exercise your right to degrade their service when the network is congested and protect paying customer quality of service which has been guaranteed by you under SLA contract.  You cannot block a call, but if network service is impaired, you can take care of your paying customers first using the network traffic monitoring products that exist today.

In closing, permit me to be blunt.  Irrespective of present regulatory vagaries or even the final direction of FCC regulation, every carrier is a duck on a fencepost if they rely on exchange record or switch data alone for source billing data.  Today carriers must defend their networks against exchange record billing data errors.  Increasingly, networks that lack probe equipment to perform traffic monitoring will find themselves overwhelmed by the relentless advance of IP and new packet technologies.

This article first appeared in Billing and OSS World.

Copyright 2011 Black Swan Telecom Journal


About the Expert

Darrell Merschak

Darrell Merschak

Darrell G.  Merschak is National Sales Manager for CMSI (Carrier Management Systems Inc.). and has worked in the telecom industry for 25 years in jobs such as: director of revenue, director of billing operations, revenue assurance manager, and regulatory and rates manager.  He has worked for AT&T, SBC, Ameritech, MCI, Pathnet, Looking Glass Networks, Tekno Telecom.  Darrell has an MBA in Finance and Economics form Loyola University of Chicago and a BA in Economics from Knox College.   Contact Darrell via

Related Articles

  • Dynamic Reference Data: How Symmetry Helps O2 UK Simplify its Processes, Reduce Errors, and Keep its Businesses in Synch interview with Nev Phillips & Luke Taylor — How a master data system can enable an a operator to exploit its reference data to: 1) update linchpin data across the enterprise; 2) simplify complex enterprise processes; and 3) get quick answers to assurance and marketing questions.
  • Multi-Vendor SDN/NFV Services: How Integrators Collaborate to Drive the Next Era of Network Automation interview with Bartosz Michalik & Marcin Paszkiewicz — What technical steps are required to orchestrate NFV/SDN and legacy multi-vendor networks?  This tutorial explains the role of key development standards like OpenDaylight and  integration challenges in the new era.
  • Monetizing NFV/SDN: How Policy Will Allow Virtual Networks to Play in the Big Leagues interview with Ari Banerjee — What’s it going to take to transform a bunch of VNFs into a large, synchronized network of VNFs that delivers a huge commercial advantage for operators?  A leading supplier of OSS/BSS solutions explains why closed loop automation and policy are key.
  • Pumping Crude Intelligence: How to Manage Telco Big Data before you Monetize It interview with Thomas Vasen — Mobile telecoms are eager to sell their time- and location-sensitive big data to marketers in other industries.  But a key bottleneck exists: finding a way to efficiently access and manage the huge data voluimes involved.  In this article, a supplier of mediation software explains his firm’s approach to tackling the problem.
  • When Big Data is Too Big: The Value of Real-Time Filtering and Formatting interview with Rick Aguirre — The volume of telecom network traffic is often so huge it outstrips the ability of even “big data” engines to analyze it fast enough.  In this article, you’ll learn about a business that filters and formats very large data sets and delivers the relevant data for applications like: data monetization, network optimization, network peering monitoring, and unstructured data storage.
  • How to Blaze an Access Path to Siloed Enterprise Data and Avoid the High Cost of Systems Integration by Dan Baker — Systems integration has always a big issue in the telecom back office.  And history tells us that it’s a messy and risky exercise when done on a large scale.  But now there’s an alternative to systems integration called “semantic search” and the concept is fully explained in a white paper.
  • Crusaders Clash: The Battle for Control of Telco 2.0 Service Delivery, Billing & Policy interview with Stephen Rickaby — Mobile is still reeling from the shock of being taken out of the driver’s seat in terms of services offered on the handset.  But  will telecoms make a services come-back?  This interview with an expert in the thick of Telco 2.0 transformation action discusses the strategic issues involved and also analyzes Oracle’s recent moves to acquire Acme Packet and Tekelec.
  • Clever Integration: The Sherlock Holmes Way to Tease Solutions from a Fog of Misaligned Data interview with Benedict Enweani — Tired of risky and expensive “big bang” systems integration projects that don‘t solve problems?  In this article you’ll learn how deductive reasoning can be used to weave through mountains of tangeled and misaligned data.  The approach uses a repeatable method to orchestrate solutions at relatively low cost and can be applied to many B/OSS and  assurance problems.
  • Telecom Mediation: Time to Move Back into the Limelight? by Dan Baker — While mediation technology remains crucial to assurance applications, solution vendors have been relatively quiet in recent years.  This article points to reasons why the mediation market may soon get more active.  Among the factors discussed are: consolidation, big data, group merger activity and the offload of mobile transactions to cheaper platforms.
  • Service Assurance Hits the Fashion Runway: The Power of Tailored Visuals interview with Richard Thomas — Some wonderful value is created by presenting data in a way that allows enterprises or internal departments to best extract meaning.  This point is a amplified through a Verizon case study where service assurance data was presented to a high-end fashion retailer during a multimedia event delivered live in over 200 global stores.  The data was presented in differently, keyed to the invidual needs of the retailer’s internal teams.
  • ILEC Business in the Balance: The Case for Probe-Based Traffic Monitoring interview with Darrell Merschak — As new, low-cost  and non-Bellcore switches penetrate the North American market, interexchange carrier are no longer delivering accurate billing usage data to rural ILECs and wireless providers.  This article explains the issue and proposes SS7 probes as a uniform source for establishing accurate usage data handshakes between carriers.
  • Putting a Database at the ‘Nexus’ of Service and Revenue Assurance interview with Michael Olbrich — Closing the B/OSS gap — getting network-facing OSS systems to communicate with customer-facing business systems is one of telecom’s greatest challenges.  This article shows the virtue of unifying B/OSS data and  processes under a single database.  Also discussed is the issue of vendor management and choosing trusted supplier to grow with.