Email a colleague    

October 2012

Clever Integration: The Sherlock Holmes Way to Tease Solutions from a Fog of Misaligned Data

Clever Integration: The Sherlock Holmes Way to Tease Solutions from a Fog of Misaligned Data

Big Data?  Forgive me for being cynical, but these days I’m suspicious of any noun with the “Big” adjective in front of it.

“Big” things often lead to big disappointments: Big Banks, Big Government, Big Debt, “Too Big to Fail” automobile companies, and “Big Transformation” projects that cost a ton of money but fail to make a telecom’s operating environment simpler or less costly to maintain.

If it was my choice, I’d substitute “Big Data” with something a little more descriptive like “Real-Time Data” or “Fine-Grained Data”?  In analytics terms, those are the chief benefits you get from these cheaper and faster boxes: the chance to act on data quicker and view it in far greater detail.

Now while Big Data enables many new capabilities, its value is greatly enhanced when you can combine it with “Big Thinking”.

Big Thinking is what Benedict Enweani and his UK-based software firm Ontology are all about.  And we’re pleased that Benedict has agreed to be interviewed in Black Swan.

First off, I must warn you to not be intimidated by that obscure word, “ontology”.  Not to worry.  Benedict does a great job of explaining his firm’s technology here.  The Ontology guys are brainy, down-to-earth people, not philosophy professors tossing $10 words around to put you to sleep.

I’m not going to steal Benedict’s thunder, but I will say that the analogy to Sherlock Holmes style thinking is very apropos.  You can apply deductive reasoning to investigate anything from mysterious murders to complex telecom systems.

As opposed to attacking integration issues with an army of Scotland Yard investigators though, Ontology uses a clever, repeatable method to orchestrate solutions at relatively low cost.  My hunch is you can apply this stuff to many business assurance problems.

Benedict, it would be great if you explained the origins of your firm.

Sure, Dan.  So, the genesis of Ontology, myself and my co-founder, Leo Zancani worked together at Orchestream, a mid-90s startup that focused on IP, VPNs, and internet services provisioning.  Orchestream was acquired in 2003 by Metasolv, then soon after by Oracle.

And the thing we noticed in each and every deployment was it would take 8 out of 10 days to align data coming out of billing, CRM, and inventory systems so that we could just create the IP VPNs for the customers and put them on to the network.

This data alignment issue wasn‘t simply an Orchestream problem.  In fact, every single deployment a service provider makes raised this issue.  And the problem stretches from the Tier 1 giants to very small operators.  BT, for instance, has about 3,000 different OSS systems for their core operations and even small service providers and ISPs have 10 to 20 different systems.

If you have a large array of clean systems, traditional integration technologies can actually bring data together quite nicely.  Likewise, if you only have a few systems and the data is dirty, you can manually clean up those systems and get a joined up view of your customers, bills, and equipment at relatively little cost.

But here’s the point: whether you have 10 systems with very dirty data or 100 systems with moderately dirty data, there is not a single technical solution you can turn to that will bring those data together because the many misalignments are so hard to pin down.

And in many cases, the inventory and CRM suppliers are more than happy to take on these data cleanup projects.  Some get rich from that type of work.

Wouldn‘t it be nice if telecoms were like Amazon.com where it is like one-to-one relationship between a book sold and something in physical inventory?  In telecom, things are never that simple.

Exactly, we’re now helping Level 3 Communications to align their inventory and CRM.  At face value, it sounds rather straightforward, but the problem is very complicated because there are something like 12 different systems where inventory data is stored.

The other complication we have in the telecom industry is that different internal consumers of data look at it in entirely different ways.  In service provisioning, it’s about configuring ports on a Cisco router.  In a Siebel CRM, you focus on a catalog of products with certain customer-restricted parameters.  In a billing system, you worry about a line item associated with a particular customer order.  In a performance management system, you’re collecting events and matching that with a service model and IP addresses.

So how do you begin to address this immense complexity?  How do you decide where to even start?

Well, suppose we wanted somebody with a fresh perspective to come in and solve the problem.  What if we went to the owner of a Mom and Pop shoe store and said, “Here’s our network operations center.  It’s a mess and I want you to go fix it.”

Five days later, the guy from the Mom and Pop store returns and says, “Yeah, I fixed it.” And you stand there incredulous and ask him what he did and he says:

“I started with the network and saw where the actual services were and wrote those down.  Then I went to the CRM system to look at the ”adds“ and ”deletes“ to determine what the network configuration should be.  Then I noticed how you had forgotten to disconnect a bunch of circuits, so I freed those up.  Finally, I updated the billing records to match the CRM.”

OK, what’s the point of my story?  It’s to say that even the toughest integration problems can be solved in step by step fashion.  Aligning systems is like solving a jigsaw puzzle where each piece you connect gives you a hint as to what to look for next.  You arrive at answers by conducting a series of searches from one system to the next, checking the inconsistencies and reconciling views.

But aren‘t you relying on an imprecise method of discovering where the errors lie?  In a traditional systems integration approach, you’re examining the systems as they were originally designed and conceived.  Isn’t that a more logical approach?

Traditional integration may be the more logical, but it is not the most efficient path -- as the history of telecom integration proves.  Integration is often an enormously expensive and risky business.

Consider this: the world’s biggest misaligned data system is not at BT, Verizon, or China Telecom.  The world’s biggest misaligned data system is the internet itself.  And yet the internet is very effective.  People will often examine seven or eight web sites before they decide to buy something.  The internet has become a key ordering channel for airlines, hotels, and many other industries, and yet nobody requires a fully integrated and joined view of the internet to get useful work done or to find answers.

The same goes for the systems at telecoms large and small.  The point is to deliver as much alignment as is required by the mission, not to align all systems and spend a fortune doing so.

What are some of the clues you use to connect the dots.  How do you how do you achieve accuracy with your system?

Well the first place to look is field names and id numbers.  You try to match these across multiple systems and see what you find.  An IP address is a unique identifier within a network.  And wherever you have an IP address, it implies the existence of an interface.  Likewise, an IP address implies the existence of a router or switch.  If an IP address can‘t be associated with a network element, that breaks the “ontology” or rules of existence for things and therefore indicates an error in the data.

Along the way, we can correct a slightly wrong identifier.  The whole point is to present the matches the system finds in a way that makes it easy for the network or system expert to make the final classification decisions.

In the area of network reconciliation, we can identify circuits that are likely to be stranded assets.  We may not be able to find a match in the network side, but we can investigate services being leased with no associated bill to a customer.

Other companies, such as Subex, are out there offering network inventory reconciliation solutions to identify stranded assets.  How would you distinguish your approach from theirs?

I know about TrueSource and it’s a fundamentally different methodology.  Their starting point involves building a very extensive model of the underlying systems, and then pulling data from those systems so you can look for inconsistencies

Rather than start with a model, Ontology’s approach is to recognize the structure that is implicit in the data systems that people already have.  Instead of pulling data from existing systems into another database, we pull it into a graph.

When you say “graph” I think of a probability graph with a normal distribution curves.  Is that what you mean?

No, I’m using graph in the sense of a map, a fully connected node and edge structure that stores relationships: one entity connects to another entity through this particular relationship.  And you can scan the nodes and examine all the relationships, so what you see is a kind of spider web called a “graph”.

What we are really storing is metadata — tags, pointers, and relationships among various databases and objects.

The Ontology Architecture

The Ontology Architecture

And as we examine the patterns in that graph, we look for consistencies, inconsistencies and when we find them, we classify things.  In short, our approach deduces the structure of network and service delivery systems without having to know very much about the kind of data we’re looking at.  And using this method we are able to deliver operational value in six to eight weeks, and the reason we can do that is we do not do a big integration at the beginning of the project.

Actually, Leo and I first got the idea for this approach from the intelligence community who pore through massive data sets trying to find terrorist cells.  In that scenario, you have no control over where and when the data will change.  Investigative work like that is a great example of teasing out the details of underlying systems using a centralized search method.

Benedict, this is highly interesting stuff.  It would be great if you could recap your perspective and value proposition.

Well, Dan, it’s no secret that the telecom business is getting more complex.  New services and new network technology are expanding.  In addition, many new partners are entering the scene, be they: virtual network operators, cloud providers, IT outsource partners, over-the-top content providers -- even managed service providers.

But no matter how complex back office systems and partner relationships get, a telecom’s ability to squeeze out greater profits and efficiency depends on knowing a lot about customer behavior and network utilization that only a finer grain level of inter-system analytics can deliver.

So there’s a tremendous need to join data across multiple misaligned systems for business intelligence , optimization, and assurance purposes.  But given the nature of today’s business, you need to do that at very low cost.  And that tends to rule out those expensive and risky full-scale integration projects that were the norm in the past.

We tackle the problem a different way.  We use orchestration and discovery techniques to align systems because they don‘t require you to change out or remodel your existing infrastructure.

What more can I say?  Fortunately we became profitable last year and doubled our revenue the last two.  So I encourage your readers to check us out.

Copyright 2012 Black Swan Telecom Journal

 
Benedict Enweani

Benedict Enweani

As CEO and co-founder of Ontology Systems, Benedict aims to make his company the leading innovator of semantic solutions in the data center and networking marketplace.  Before founding Ontology in 2005, Benedict was CTO at Corvil Networks and CTO at Orchestream (now Oracle).   Contact Benedict via

Black Swan Solution Guides & Papers

cSwans of a Feather

Related Articles

  • Black Swan Guide: Araxxe’s Revenue Assurance Consulting, Testing, and High Definition Billing Analysis Service by Dan Baker — How Araxxe’s end-to-end revenue assurance complements switch-to-bill RA  through telescope RA (external and partner data) and microscope RA (high-definition analysis of complex services like bundling and digital services).
  • Subex’s IDcentral Monetizes Telco & Enterprise Data to Deliver Digital ID & Risk Metric Services for Financing, KYC & More interview with Shankar Roddam — A new digital intelligence service that monetizes the idle data of telecoms and enterprises while also earning a good return for the owner of the data.
  • Opportunities & Obstacles: Consultant Luke Taylor Muses on the State of the Telecom Risk Assurance Business interview with Luke Taylor — A rambling discussion on the state of the risk assurance business with Luke Taylor, independent consultant in telecom revenue/fraud assurance and solution requirements and marketing.
  • LATRO’s Tips for Launching a Successful Revenue & Fraud Assurance Program for Mobile Money Operations in Developing Countries interview with Don Reinhart — A company building mobile money RA/FM tools and  managed services gives a concise, but detailed tutorial on how the Mobile Money Ecosystem works.  Revenue assurance pros will get tips on  what to look for in analytics/assurance tools, controls, and professional services.
  • A WeDo Conference Talk: Consulting & Analytics: Improving your Business Today, Enhancing it Tomorrow interview with Carla Cardoso & Bernado Lucas & Thomas Steagall — Leading risk management consultants explain their mission and walk-through RA, subscription fraud, and collections cases.  They also explain how analytics and machine learning can supplement process optimization.
  • PrologMobile’s Simple and Brilliant Plan to Save US MNOs Billions a Year in Recovered Phones & Retained Customers interview with Seth Heine — An expert in the mobile phone reverse supply chain explains how MNOs — via a neutral third party information exchange — can recover their original phones on the used market and save huge sums in multi-year customer retention.
  • WeDo Explores the IoT Ecosystem in Search of Tomorrow’s Pivotal Fraud & Business Assurance Solutions interview with Carlos Marques — A veteran product manager scans the IoT terrain, discusses key fraud and assurance challenges, and explains the preparatory steps WeDo is taking to become a key player in this emerging market.
  • New Report: Telecom Fraud & Business Assurance Solutions, Services & Strategies by Dan Baker & Luke Taylor & Colin Yates — TRI publishes a new market research report, Telecom Fraud & Business Assurance Solutions, Services & Strategies.  Free executive summary available.
  • Subex Juggles a Wide Variety of Business Assurance and Big Data Analytics Use Cases interview with Rohit Maheshwari — A expert in business assurance solutions explains top use cases such as: IoT security, big data analytics/AI, network asset optimization, multi-player gaming assurance, onboarding mobile subs, and AI customer analytics.
  • MTN Agility: Mastering Exponential Technologies in Revenue/Fraud Assurance and Beyond interview with Danie Maritz & Tony Sani & Luke Taylor — An in-depth look at RAFM operations and innovation at the MTN Group.  Topics discussed include RA/fraud control challenges, strategies, and MTN’s journey to exploit exponential tech (AI, robotics, and ML) in its RAFM program and support of internal non-telco businesses.
  • From Byzantine Software Contracts to Simple & Flexible RA Managed Services interview with Philippe Orsini — Is the way B2B/enterprise software is sold and delivered today progressive — or is it Byzantine in the age of cloud?  An expert lays out the case for managed services in RA and billing verification.
  • Premiere Experts Set to Speak at Summer RAG Conference in London, July 7th and 8th by Dan Baker — The Risk and Assurance Group (RAG) has announced that its 2016 summer conference will expand into a two-day event and feature many premiere experts. 
  • WeDo Hosts Revenue Assurance & Fraud Management Conference in Washington DC by Dan Baker — Black Swan is pleased to announce what looks to be a first class revenue assurance and fraud management conference being put on by WeDo Technologies, on October 1st and 2nd in beautiful Washington DC.
  • Test Call Generators: An Essential Test & Debugging Tool in Mobile Billing Assurance interview with Steffen Öftring — An “active” test call generator (TCG) can see problems that a “passive” revenue assurance system is blind to.  Here’s a discussion on the test call RA  process, over-the-air calls versus core call injection, and test call networks in global roaming RA.
  • The Revenue Assurance Game: How the Rules Change in the Era of IoT & Mobile Broadband interview with Rene Felber & Gadi Solotorevsky — Revenue assurance is perhaps the hardest of telecom functions to define because the term is used in so many different senses.  This discussion on the evolving role of revenue assurance was catalyzed by a survey of experts in the profession.
  • Day in the Life of a Revenue Assurance Analyst interview with Michael Lazarou — Revenue assurance is much more than a software category.  It’s individual analysts struggling to help their larger organizations get a handle on system errors and coordination problems.  In this interview, an analyst reveals the many challenges of getting the revenue assurance job done at a small GSM operator in Europe.
  • Revenue Assurance: History and New Beginnings in RA Maturity interview with Daniela Giacomantonio & Gadi Solotorevsky — The Roman Forum was the center of commercial life in ancient Rome.  Now, two millennia later, the Forum lives on in the exchange of ideas across countless professions and  media.  In this interview, two Revenue Assurance experts discuss both the new RA Maturity initiative of the TM Forum and the value of telco/solution vendor collaboration.
  • Migrating systems or launching LTE next year?  Don‘t forget transformation assurance & optimisation by Efrat Nissimov — System transformations and network migrations are major  revenue impacting events and they should raise a big red flag.  Why?  Because data integrity issues are bound to crop up as CSPs move vital data from a legacy system to something new.  It’s time for transformation assurance.
  • How can Cable/DSL Internet Providers Meet the Usage-Based Billing Mandate? interview with Ryan Guthrie — The popularity of YouTube, Netflix, and Hulu other video outlets has turned the tables on service profitability for cable/DSL service providers.  Many are moving to usage-based billing, but that largely unprepared for the revenue assurance aspects of this move.  This interview explains the technical challenge and points to solutions in billing, speed caps, and traffic revenue monitoring.
  • CABS Revenue Assurance: How Rural LECs can Recover $284 Million in Revenue Shortfalls interview with Kelly Cannon & Darrell Merschak — Independent rural LECs in the U.S. still rely on the AMA/EMI billing formats for CABS billing, even as that format has proven to be highly inaccurate as a source of inter-carrier records.  This interview includes an analysis and discussion of revenue recovery techniques ILECs can use by leveraging SS7 probes.  Also discussed are billing strategies, traffic dumping threats, and the possible fallout from the FCC’s bill-and-keep mandate.
  • Make Business Assurance Progress Every Day: How to Set Goals, Automate, and Energize Your Team interview with Kathleen Romano — Business assurance (BA) skills have wide applicability outside the revenue assurance and fraud mangement domains.  In this article, a telecom executive explains how she’s applying her BA skills in the Payments area.  In addition to discussing the key operational challenges in Payments, the interview also provides keen insights on setting goals in business assurance, leading a team, and making critical decisions.
  • LTE Rollout: Make it a Smashing Success with Risk Assessment, Controls, and Marketing Offer Analytics by Gadi Solotorevsky — LTE brings splendid new capabilities to mobile users.  But like 2G and 3G deployments before, operators can only make money if they successfuly plan, coordinate, deploy fast, and pay attention to pricing plans and the customer experience.  This article lays out a 3-phase tactical guide on  how revenue analytics professionals can add value in LTE service risk assessment, controls, and marketing offer analytics.
  • RA Prevention: How to Manage Revenue Risks and Communicate RA’s Value to Senior Execs by Shaul Moav — The era of revenue assurance prevention and risk assessment is here.  Several of the mature operators of the world have developed their own methodologies and tools.  Using firefighting and fire prevention as a metaphor, the article details a new commercial software approach explaining the goals, method of risk evaluation, and senior executive dashboards developed for the process.
  • Precision Clockworks: How Revenue Assurance Synchronizes with the Business at Swisscom interview with Marco Pollinger — An expert revenue assurance department is one whose work dovetails well with the lines of businesses it supports.  In this interview you’ll learn how Swisscom manages its revenue assurance function for maximum effect.  The article discusses: the operator’s innovative RA organization, the screening and RA approval of new services, its pre-production bill audits, and its coordination with corporate risk management.
  • Versatile, Portable & Corrections-Savvy: Quest for the Swiss Army Knife of Revenue Assurance Software by Mark Yelland — Revenue assurance maturity models are not cast in stone.  Since  best practices will change over time, it’s healthy to explore moving maturity models forward.  For example, great gains have been made in leakage detection, but RA corrections has been harder to master.  The author dreams about seven functions that should ideally come together in a single all-purpose revenue assurance software tool.
  • Bringing Strategic Planning & Value Engineering to Revenue Assurance interview with Maged Fawzy — Engineering and architectural techniques have a role in revenue assurance.  This interview with a top Egyptian RA consultant explains how continuous risk assessment and long range — yet flexible — RA planning can sharpen a carrier’s RA program and lead to better use of revenue assurance software and integration services.
  • Forensic Fossils: Is Your Revenue Assurance Shop Fit for Display at a Natural History Museum? interview with Jim Marsh — Without the continuous guiding light of seasoned revenue assurance leaders, even the best teams of RA professionals, technology, and business processes can fossilize and lose their vitality.
  • Revenue Assurance: The Magical Market Cap Multiplier by Van Howard & Curtis Mills — Many operators today consider revenue assurance yesterday’s opportunity.  But this article shows why significant revenue and cost leakage can still go undetected, even in companies with dedicated RA departments.  Also discussed are the benefits of a broader or more “forensic” approach to revenue assurance, an approach that boosts the bottom line regardless of the automated tools already in place.
  • From Risk to Robust: Turning the Big Picture Into a Real Agenda for Change in Telecoms by Eric Priezkalns — Inspired by a Financial Times article written by Nassim Taleb, author of “The Black Swan”, here is an insightful and entertaining primer on telecom risk management.  The article takes ten risk management lessons from Taleb and applies them specifically to the communications industry.  You’ll learn about the value of small scale trials, organization accountability, cures for a blame culture, incentives that work, the power of simplicity, and more.
  • Synthesizing the Telecom Business Assurance Practice With the Analytics World by Dan Baker — Business assurance is a wrapper term that allows you to draw a circle around various telecom assurance, control, and optimization activities.  This article maps business assurance as a subset of telecom analytics, constrasting it with marketing analytics while a diagram shows where biz assurance fits in the larger B/OSS world.
  • CABS Revenue Assurance Disputes: May the Carrier With the Best Data Win by Cheryl Smith Rardin & David West — Revenue assurance innovation is far easier when partners cooperate to make it happen.  This articles shows how a U.S. operator, software vendor, and consultant teamed to develop a breakthrough in Carrier Access Billing (CABS) assurance.  Learn about: the dispute resolution data gap that needed to be filled, the partnering strategy, the implementation challenges, and payback results.
  • Revenue Assurance vs.  Business Assurance: Who’s the Rightful King of Controls Software? interview with Sergio Luis Silvestre — Business controls software, originally developed for RA, is finding application in other areas of the business such as internal audit, collections, security and risk management.  This article argues that “business assurance” is the best term to describe this broader set of  controls software that can find a home in numerous departments or functions of a CSP’s business.
  • PwC on the Business of Revenue Assurance Consulting & Mentoring interview with Tim Banks & Dan Stevens — Revenue assurance consulting firms offer a broad range of services to clients these days.  The article explains the practice of mentoring RA mangers and providing a CFO with visibility on the status of an operator’s business controls.  Perspective is also offered on the value of RA software and the opportunity to broaden the RA practice scope.
  • Robots for Hire: Verifying Accuracy In the Age of Complex Mobile Billing/Charging interview with Xavier Lesage — As real-time charging and complex lifestyle calling plans gain credence across the globe in wireless, billing quality issues will rise in importance.  This article discusses a unique managed services approach to invoice testing and roaming fraud protection that checks results against advertised or published source data for the utmost accuracy.
  • Ericsson: Revenue Assurance Consulting With an NGN Flavor interview with Thomas Steagall — Helping operators detect billing and provisioning problem is merely table stakes in the RA services business these days.  The article discuss why operators need to ramp up their RA function with service experience and group-wide financial health monitoring.  Advise is also offered on: key RA maturity questions, risk-and-reward contracts, and how to extract greater value from software investments.
  • Do-It-Yourself RA for Small Operators and MVNOs interview with Mark Yelland — Budget-minded small operators and MVNOs are no longer hamstrung in RA capability anymore.  This article offers high-leverage strategies for operators who cannot afford expensive RA software tools.  With  data access, brains, and a DIY philosophy, any small operator can map a  path to greater RA savings, maturity, and program growth.
  • Revenue Assurance Maturity: Report From the Arena interview with Eric Nelson — Revenue assurance maturity can‘t be easily computed.  How do you  compare the KPIs of Comcast billing with that of mobile money RA in Western Africa?  Even still, this article offers some universal RA wisdom from a straight-shooting veteran of carriers large and small.  Topics discussed include: dashboard or process, COTS vs. inhouse solutions, and tips on gaining internal support for the RA practice.