Email a colleague    

November 2012

Big Data Financial Analytics: Creating Business Value for Tier 1 Telecoms

Big Data Financial Analytics: Creating Business Value for Tier 1 Telecoms

“I do not know how I may appear to the world, but to myself I seem to have been only like a boy playing on the sea-shore, and diverting myself -- now and then finding a smoother pebble or a prettier shell than ordinary, whilst the great ocean of truth lay all undiscovered before me.”

Sir Isaac Newton (1643 — 1727)
perhaps the world’s most influential scientist

Kids finding cool pebbles and shiny shells at the seashore.  Sounds like the mission of an analytics company.  I’ll bet that’s more fun than finding needles in haystacks :- )

What exactly is an analytics company?  They’re actually pretty rare, but with the era of affordable big data platforms upon us, I suspect more and more analytics firms will begin to sprout up.

SAS Institute, one of the most successful high tech companies ever, is probably your classic analytics firm.  Over the years, whenever a Tier 1 carrier in the Americas or Europe needed help with churn reduction or subscriber analytics, they would often turn to SAS.

Everybody classifies SAS as a software firm, but I don‘t agree.  Sure, it employs software to do its exploratory work, but SAS is not really known for any particular software product it leaves behind.  SAS isn’t a consulting firm either, though throwing industry-savvy consultants and systems experts at problems is key to its work.

A few weeks ago I drove down from Pennsylvania to see another analytics firm, TEOCO, a telecom analytics firm, at its Fairfax, Virginia offices outside Washington DC.

I knew TEOCO’s marketing people pretty well, but I was particularly looking forward to this meeting because I would get a chance to converse with their top executives including Atul Jain, the CEO and founder of TEOCO, who I had never actually met before.

They sat me down in the middle of a 25-foot long conference table, then surrounded me with a dozen people from their top executive and marketing team.  What?  Am I going to be grilled with tough questions?  Thankfully, no.  Most of the people were there to listen in and contribute when they could; the conversation was led by Atul and General Manager, John Devolites.

It was great.  We dispensed with PowerPoints and other formalities and just talked.  I turned on my audio recorder and listened in on some of the best big carrier “war stories” I’d ever heard.  TEOCO’s perspective on the telecom analytics business is equally enlightening I think.

Given the upward trajectory of its business, its expansion into Europe, its vast experience working with Tier 1 telcos, and the “kids at the seashore” excitement I felt from their executive team, TEOCO has more than a fighting shot of becoming the SAS Institute of telco financial analytics.  Read the interview and see if you don‘t agree.

Atul, the last three acquisitions TEOCO has made have been all over the map.  First it was Vero Systems in 2006, a supplier of Least Cost Routing solutions.  That deal made sense to me because optimal routing fits nicely with your historic focus on cost management.  But then you bought TTI Telecom, a service assurance software vendor out of Israel, and then Schema, a specialist in wireless spectrum.  It’s kind of hard to figure out what the last two acquisitions have to do with financial and cost analytics, which is your specialty.

Atul Jain: Dan, our acquisitions only seem scattered if you assume our objective is to fill a particular hole in a portfolio of applications.  We actually don‘t think that way.  We are an analytics company who looks to deliver fast ROI projects to customers.  Only secondarily do we deliver on-going applications.

TEOCO’s Acquisitions of Vero Systems and TTI Telecom

Acquiring Vero Systems taught us an important lesson: the closer we get to the network, the closer we are to the place where telcos put a lot of time, money and effort.  With Vero, we could monitor that network from the standpoint of completing calls and routing them cost effectively.

It’s true, TTI Telecom and TEOCO are coming from different worlds.  TTI’s value proposition to the carrier is on the operational or OSS side.  By contrast, TEOCO delivers value on the BSS side.

But acquiring TTI was very exciting for us because they gave us a perspective we were missing — network element data from a fault and performance management point of view.  We now have access to network elements we have never spoken to before.  And so our challenge now is to take the OSS information and create a business value out of it.

And with TTI we are playing to our strength: analyzing huge volumes of data.  We are looking at TTI’s operational data and figuring out what may be going wrong so we can an ROI or something that will have a financial impact for a customer.

John Devolites: Dan, what do you think is the hardest problem for a company like us to deal with?

Dan Baker: Finding the right people who understand the systems you’re looking at?

John Devolites: Yes, that’s on the right track.  Actually our toughest challenge is simply getting hands on the data we need.

Hardest Job: Gaining Access to Data

So what we have done is to either buy companies or build products that get us access to data that is very difficult to get your hands on.

Give you an example.  On the cost management side, many of the North American carriers run their invoices through us.  That gives us a good outlook on costs.  And costs are nice: you can do a lot of good things with cost -- even figure out how to reduce it.  But costs alone don‘t really give you the complete picture of what’s going on.

Vero’s knowledge was pivotal because if it gave us access to CDRs.  Now we had CDRs before but we never really had CDRs at the deeper level we got when we acquired Vero.  Why?  Because with Vero every call that gets routed we can see a cost component and see a price.  That’s the way the wholesale model works.

Now you can certainly ask a telephone company to give you access to that data, but they almost never do that.  Best case: they spend 6 months trying to find the one guy who knows how to get what you need.  We know of a larger carrier whose top network guy doesn‘t quite know who to get his CDRs from.

Our strategy, then, is pretty simple.  We want to have applications that get in the way of data and that give us access to that data.  And it’s much easier to get access to the data by acquiring an application that already exists in the market than for us to go to a customer and ask for it.

If we go to any of the major carriers and ask them for access to their subscriber database, what’s the chance they will give it to us?  But if we’re already in the business of getting quality by subscriber data , that becomes a lot easier.

So our mission is to get access to data then enrich it to a level where it’s become more meaningful and shows people information in a way they haven‘t seen it before.

In the end it’s all about data access.  Getting applications into a position where we can collect that information so we can do something with it.  That’s the secret sauce.

Thanks John.  I get It now.  When you buy a company, their current application is of secondary value.  It’s the ROI business application you develop on top of the data sources where the real value lies.

Atul Jain: Every time we figured the secret was to invest in probes to go after specific data, the answer turned out to be “no”.  Our goal is to get data streams, mostly data streams that have not been used for business value generating activities and have very valuable information associated with them.

That’s how we create the ROIs.  It’s pretty cool.  I mean for us guys who love data, it’s the best thing in the world.

I understand why TTI Telecom was attractive now, but what about the Schema acquisition?

Atul Jain: Well, Schema gives us access to another kind of data — data in the Radio Access Network (RAN).  And this gives us access to dropped calls.  Those c alls didn‘t go through and are not known because they never made into CDRs.

In the Schema application, that’s the primary focus, knowing how well the radio access network is working.  Once again, TEOCO had access to logs we never had access to before.  And the logs give you access to all the dropped calls and bad quality calls.

That’s the data that’s used to optimize a RAN network.  Earlier we were focused on the calls that were actually completed, but now we see all the calls even the ones never completed.

Now does that play into customer retention?  Absolutely, because you can determine which customers you are apt to lose based on your bad quality and dropped call statistics.

As we move to LTE, they will see a slow session and sporadic session in the RAN and this will become a more important factor from a subscriber quality point of view.  And knowing about the quality of service in detail will be the key to churn reduction.

A year ago I heard TEOCO present at the Telestrategies conference about what you did at a tier 1 U.S. provider.  It’s actually pretty amazing to be able to implement a company-wide profit assurance solution like that.  The idea of getting a large operator to cooperate across many internal organizations is maybe the most impressive part.

John Devolites: The biggest mistake you see integrators make is to try to do everything as a big bang project.  You can‘t just say, “Here’s the end goal, so let’s go after it.” You have to use an incremental approach.  It’s got to be phased, very clear, and measure results after Phase 1, Phase 2, etc.

Phased Projects, Measured Results

And we generally do that by data sources.  First phase is switch records.  Second phase is billing records, for example.  And we make sure we have clear cases so we can measure the return and compare it to the beginning state.

A nirvana solution delivered all at once is doomed to fail.

Every phone company we deal with in North America operates the same way.  There has to be a business case.  That’s where it starts.  We can come in and tell you what great things we can do, it all boils down to a business case that is going up to senior management for approval.

And in virtually every deal we have done recently, there is some promise where they have a way out of the program if we don‘t make our numbers.  And that is incremental — each phase.

So when Atul talks about ROI, the foundation of TEOCO is the ROI equation.

And we tend to love business cases because they show pretty well what we do.  And it’s interesting how clients tend to get greedier during the process.

If every time I give you a dollar, you get three back, you’d think that was a good deal.  Well, no.  It’s doesn‘t work that way.  Because now they want $5 back for every dollar of investment!  Generally we try to get a 90 day ROI on the short projects.  But on the analytics projects, they tend to be longer because the data sources are harder to get at and set up.

John , I know you guys are champions of big data analytics, but when your data set runs into billions of records, isn‘t a statistical sampling just as good?  Why is looking at all the data important?

John Devolites: As an aside, when we acquired Vibrant Solutions several years ago, we found out they actually sold the first (Serial number 0001) Neteeza box (now IBM), so you could say our roots in big data go back a ways.

Switch-to-Bill Analytics

Let me give you a case study.  One of our large customers had an issue with switch-to-billing information.  The switch said one thing and the billing information said another.

Now if you wanted to do a deep every-CDR investigation back in 2006, your only choice was Teradata.  It was the only data warehouse that could handle that kind of scale.  But by the time you paid licenses to Teradata and went through all the effort to insert yourself into the carrier’s data warehouse ecosystem, you could never make any money as an analytics firm.

When we eventually did get the project to do the work using our Netezza box, we took all the switch records and billing records, and pulled them together.  The analysis work paid for itself very quickly.  In many cases, it turned out the switches didn‘t have CDRs turned on.  They ended up having to clean up various network elements to get it right.

But one of the problems was their sampling methods didn‘t identify underlying issues.  You could get into an argument with a statistician (there’s one sitting next to you [Atul] (laughing in the room) , but the correct answer is that nothing substitutes for analyzing all the data.

And part of the reason is there’s no argument about what statistical method you used — it is what it is.  If you start off with a billion minutes on the switches and you end up with 900 million in billing, you definitely want to know why.  And the way you do that is look at both sets of records and figure out why that’s actually occurring.  Some of it was legitimate.  Turns out that in a few of their states, you got an exemption if you were a Native American.  Same goes if you were blind, disabled, or over 65, you didn‘t have to pay for a phone.  And the list went on.

So we ended up finding about 150 of these exemptions that people had forgotten about.  And nobody had ever reporting on how much those exemptions were costing the business.  So when you go back to the Public Utility Commission (PUC) [regulator] and say, this is what people asked us to do, and here’s what it’s costing us, that has a big impact.

Traffic Pumping

Traffic pumping is a big issue that if a carrier doesn‘t do things right, it can really turn into some serious revenue issues.  This is an area where we saved carriers hundreds of millions of dollars.  But you can’t actually get there without the technology to see the record detail.

For instance, if you did sampling, you’d end up sampling the wrong place.  And the traffic pumpers would set up operations so they could traffic pumping overnight.  They set up a partnership with a local LEC in North Dakota who gave them some phone numbers to use.  And that’s where they set up their porn and chat lines to use those phone lines.

Well, many of the largest operators are paying for it.  The number got to be in the $300 to $400 million a year range for just about all the Tier 1 providers.

Atul Jain: Actually, it’s the legal advantages of using all the data that really seals the deal.

The Legal Advantages of Big Data vs.  Sampling

Let’s say someone is doing traffic pumping and you’ve decided to sue them.  Now when you’re in court, a statistician stands up and says: we sampled this data and found out that X% of the traffic was traffic pumping.

If you extrapolate the numbers and say the money owed is $20 million, I don‘t think the court is going to buy that.  The court is going to say: show me the evidence.  Don’t give me sampling that adds to $20 million.  Show me the actual records that add up to $20 million.  Don‘t base you number on a 2% sample that was only $40,000.

Now believe it or not, the statistical analysis is quite valid, but getting a court to buy into that is another story.  Frankly, the statistical approach isn‘t admissible in court anyway.  We’ve been a major source for this type of information, so our guys are the ones doing the expert testimony.  The statistical guys doing sampling are just chewed up by the defense lawyers.

Another problem with the statistical method is a lot of calls are dropped, and there’s a difference here between intra-state and inter-state calls that doesn‘t show in the sampling.  So the problem would not have been caught if you relied on sampling alone.

I’ve always considered TEOCO to be delivering most of your value in the cost management or inter-carrier reconciliation space.  But the stories I’m hearing now in switch-to-bill and traffic pumping are revenue assurance and fraud issues.

John Deloites: Often the problems large carriers have would be simple to solve if the huge scale of their operations didn‘t get in the way.

Analysing the Outliers of Wireless Usage

For one large carrier we went in and examined two months of records to look at what their top 100 subscribers were doing.  You’d think they’d have the ability to do that, but they didn‘t at the time.

So we ran off the top 100 wireless subscribers and found the top 20 had 20,000 minutes of usage a month.  Now that’s a lot of minutes a month.  I’m not even sure there are 20,000 minutes in a month.  Anyway, TEOCO called the numbers to investigate and we found out it was a taxi dispatch service in Seattle, Washington.  Each taxi driver basically just turned their mobile phone on all day long and used it as a walky-talky.

So this shows another reason for not sampling.  Looking at all the data, you have a comprehensive view of the outliers.  With sampling you might miss a few of those.

Sampling is basically like using training wheels.  That isn‘t the way fraudsters play the game in North America.  They play at the individual phone number level.  And if you can’t get down to that level, you’re not going to detect those bad guys.


Editor's Note: TEOCO promises to contribute a byline story detailing a few of its customer analytics stories.  I‘m going to hold them to that promise :-)

Copyright 2012 Black Swan Telecom Journal

 

About the Experts

Atul Jain

Atul Jain

Atul Jain is Chairman and CEO of TEOCO, the corporation he founded in 1994 with the dream of building a successful business model that didn‘t compromise its values, not just a successful business.  Prior to starting TEOCO Corporation, Atul was with a Silicon Valley firm called TIBCO for seven years.  At TIBCO, Atul’s focus was to work with Fortune 500 clients to design and build state-of-the-art software solutions leveraging the company’s trademark TIB platform.

John Devolites

John Devolites

John Devolites is the General Manager for TEOCO.  Prior to the company in a full time capacity in February 2004, he served on TEOCO’s Board of Directors for four years.  Previously, Mr.  Devolites served as President of Professional Services for Telcordia.  His other work experiences include executive positions at PricewaterhouseCoopers, E-Commerce Industries, Andersen, American Management Systems (AMS), Alexander Proudfoot PLC, and Booz Allen Hamilton.  He is the author of multiple US patents focused on data mining and storage.

Related Stories

  • Atlanta-lytics: New Telecom Conference Features Revenue and Customer Analytics by Dan Baker — A new telecom conference on analytics in Atlanta (January 29 — 31) is shaping up to be a key event for people who want to stay abreast of this fast moving field.  Black Swan is hosting a half-day, pre-conference workshop entitled, “Financial and Revenue Analytics” featuring seven speakers, many of whom are contributors to this magazine.  Detailed program guide is enclosed.
  • Big Data Financial Analytics: Creating Business Value for Tier 1 Telecoms interview with John Devolites & Atul Jain — What exactly is an analytics company?  This interview with executives at TEOCO helps defines this new breed of telecom solution vendor — a cross between software, consulting, and assurance supplier.  Along with a few Tier 1 case studies, the interview discusses: big-data vs. sampling, analytics project management, and the importance of data access.
  • Margin Analysis: Bolting Profit Assurance onto a Revenue Assurance Platform by Efrat Nissimov — Margin analysis is one of the hottest trends in business assurance.  But what’s all the excitement about?  This article explains the benefits, the market forces driving its adoption, and its advantages over Excel and heritage data warehouse approaches in terms of flexibility, data granularity and quicker time to access.  The author also touts major cost savings if margin analysis is deployed as a module within an existing revenue assurance suite.
  • Is Offering Free International LD on Your Horizon?  Follow These Checkpoints by Hulya Altinsoy — Free is a magic word for consumers.  So how about adding free international long distance (LD) to your list of services?  While free international LD serivce is fraught with risk, the article presents a  5-point checklist to help get your bearings before you make the leap.  The article also presents an analysis of billions of minutes in international LD calls by U.S. wireless carriers.

Related Articles

  • Webinar: From Wholesale Settlement  to Global Partner Management by Dan Baker — A 40 minute webinar providing a sweeping view of the challenges and opportunities service providers face as they try to manage a far more complex wholesale and partnering scene.
  • Special Report: Wholesale Systems for Advanced Services by Dan Baker — Plenty of technology and market changes are disrupting the telecom wholesale market: advanced services, new kinds of partners,,and the need to monitor relationships and services more closely than was ever required in the circuit voice world.  This article introduces a TRI special report discussing the kind of Wholesale Systems needed in this challenging marketplace.
  • Data Integrity Issues Go Global in the Telecom Wholesale Exchange interview with Vic Bozzo — Wholesale systems enable operators to expand their global reach and work efficiently with partners in an increasingly interdependent telco business.  In this article, you’ll learn about: how many subsystems work together in support of the global trading exchange, why data integrity has become critical, and how wholesale business trends are affecting wholesale systems at Tier 1 and mobile operators.
  • Network Asset Choreography: Subex Teaches Analytics to Dance with Discovery & Life Cycle Management interview with John Brooks — Network provisioning and asset life cycle management is a foreign world to most business assurance professionals because it’s a domain that lives outside the order-to-cash stream.  This article explains how new analytics and network discovery techniques are enabling operators to better track assets, plan capacity, and pave the way for strategic network deployments and decommissioning.
  • Atlanta-lytics: New Telecom Conference Features Revenue and Customer Analytics by Dan Baker — A new telecom conference on analytics in Atlanta (January 29 — 31) is shaping up to be a key event for people who want to stay abreast of this fast moving field.  Black Swan is hosting a half-day, pre-conference workshop entitled, “Financial and Revenue Analytics” featuring seven speakers, many of whom are contributors to this magazine.  Detailed program guide is enclosed.
  • Big Data Financial Analytics: Creating Business Value for Tier 1 Telecoms interview with John Devolites & Atul Jain — What exactly is an analytics company?  This interview with executives at TEOCO helps defines this new breed of telecom solution vendor — a cross between software, consulting, and assurance supplier.  Along with a few Tier 1 case studies, the interview discusses: big-data vs. sampling, analytics project management, and the importance of data access.
  • Margin Analysis: Bolting Profit Assurance onto a Revenue Assurance Platform by Efrat Nissimov — Margin analysis is one of the hottest trends in business assurance.  But what’s all the excitement about?  This article explains the benefits, the market forces driving its adoption, and its advantages over Excel and heritage data warehouse approaches in terms of flexibility, data granularity and quicker time to access.  The author also touts major cost savings if margin analysis is deployed as a module within an existing revenue assurance suite.
  • Partner Settlement: The Adaptable Chameleon that Lives Between Wholesale and Retail Billing interview with Arun Kalavath — Partner settlement has been around a long time, but its profile is on the rise because wholesalers are offering new services and content services are bringing settlements into the retail sector.  In this article, you’ll learn about the market forces changing the wholesale business, the expanded portfolio of services that require settlement, and advice on how to pick a settlements solution vendor.
  • Carrier Usage Audits: Why Regular Checkups Are the Key to a CSP’s Bottom-Line Health by Brian Silvestri — Many telecoms are wholly unaware of the risks of not having a solid usage auditing program for interconnect bills.  Rather than a once-a-year or once-a-quarter audit, this article makes the case for regular monthly audits.  Also discussed are: the different types of audits and organizations involved; the dangers of not auditing often; and factors to consider when selecting a cost-management vendor.
  • Partners in Carrier Management: The Success Story Behind T-Mobile’s Fiber Rollout in Wireless Backhaul interview with Bryan Fleming — Wireless backhaul is the unsung hero of the smartphone’s success.  This interview with T-Mobile’s carrier management architect for backhaul reveals the behind the scenes game plan for one of the most ambitious wireless interconnect programs ever.  You’ll learn about: the reasons for adopting a full-scale fiber strategy; the challenge of finding carrier partners; the clever techniques T-Mobile used to simplify and cut costs; advice on building great relationships with suppliers; and the key role that analytics, assurance, and visualization software played.
  • Will Apple Take a Bite Out of Carrier SMS Revenues?  Likely. by Brian Silvestri — Text messaging generates $20 billion of revenue each year for U.S. carriers alone.  But Apple’s lastest iPhones come with iMessage, a free embedded messaging service.  But while the long term threat to SMS revenue is clear, what strategies should operators be following today?  This article discusses the many factors operators need to consider, offering advice on how mobile operators can keep SMS profits rolling in as long as possible.
  • Invoice Audit Productivity: From 19th Century Textile Mill to SaaS by Peter Yelle — When it comes to invoice reconciliation, are your processes and tools modern and efficient, or is your company stuck in a 19th century time warp?  This article show how operators can vastly reduce their manual audit steps by automating via a SaaS solution.  The  detailed case study and diagram show which tasks were automated to achieve an 80%+ productivity gain.
  • Least Cost Routing and Rate Audits Not Delivering the Savings You Expected?  Here’s Maybe Why by Brian Silvestri — Least cost routing (LCR) is supposed to be one of telecom’s greatest cost-saving innovations.  Yet Connectiv Solution’s research suggests that the LCR programs at many U.S. carriers are losing millions of dollars.  The story explains the traffic analytics steps service providers must take to ensure profitable LCR and other cost assurance programs.
  • Wireless Backhaul: Preparing for the Cleanup When the Dust Settles by Charlie Thomas — Wireless backhaul represents perhaps the largest telecom build-out in the last decade.  This article lays out a strategy for operators as they sooner or later must groom, optimize and re-engineer their backhaul networks based on actual capacity needs, new pricing and improved supplier agreements.  The article lays out a 6-point plan for ensuring success.
  • Is Offering Free International LD on Your Horizon?  Follow These Checkpoints by Hulya Altinsoy — Free is a magic word for consumers.  So how about adding free international long distance (LD) to your list of services?  While free international LD serivce is fraught with risk, the article presents a  5-point checklist to help get your bearings before you make the leap.  The article also presents an analysis of billions of minutes in international LD calls by U.S. wireless carriers.
  • Real-Time Network Intelligence: The New Way to Read Telecom Tea Leaves by Suren Nathan — Real-time network intelligence is the key to deciding which products to launch, whose facilities to lease, and where to route traffic.  The article explains why telecoms — and especially enhanced service providers --  should ideally be equipped with both a fine-grained margin analysis solution and a SaaS platform, offering an upgrade path that requires no internat IT support.
  • International Call Routing: A Challenging Area to Master, But Payoffs Can Be Big interview with John Fitzpatrick — International call routing offers a completely different set of challenges than the common practices found in North America.  This article explains the many issues faced in international LCR from imprecise routing and number portability headaches to dispute problems and the flood of wholesaler rate sheets that must be rapidly made sense of and implemented in the routing plan.
  • Dynamic Call Routing: The Market Enabler that Allows VoIP to Improve with Age interview with Neal Axelrad — Least cost routing (LCR) has matured greatly in recent years.  Today’s routing is far more dynamic because of new real-time capabilities.  This article explains technology advances that instaneously re-routie calls based on real-time intellgence gathering from the network.  Also discussed are template-driven rate addendums, SaaS delivery, and the economics of LCR.
  • Network Inventory Integrity: Taking Cost Management to a Higher Level interview with Suren Nathan — Recovering stranded assets and capacity is essential for minimizing CAPEX and leasing costs.  This article makes the case for reconciling network inventory through the many ordering, billing, and other systems that interact with that inventory.
  • An Automated Self-Audit Approach to Telecom Cost Assurance interview with Jim Buttafuoco — What’s the value of an automated approach to invoice validation?  This article explains the power of the SaaS model where the vendor supplies the data-processing expertise, relieves the operator of tedious manual work and boosts auditor productivity so more money is saved.
  • Meet Your New Cost Management Solution: A Tandem Hub Carrier interview with Surendra Saboo — In the telco interconnect game, frustrations are many: high tolls, jurisdiction scams, and the expense of connecting to a partner.  This article explains how tandem or aggregation carriers are making life easier and saving operators a boatload of money.  Learn the difference between peering and a “tandem hub” and why this trend is becoming increasingly global.
  • Cost Assurance: Dealing with Rating Variety in Wholesale Bills by David West — Validation of intercarrier wholesale invoices seems like a fairly straight forward process.  But regulatory rules and a variety of bill rating methods have made it very complex.  This article details the challenge by walking through a wireless carrier case, then laying out the argument why operators should opt for a custom solution based on a core reporting engine.