Email a colleague    

October 2012

Clever Integration: The Sherlock Holmes Way to Tease Solutions from a Fog of Misaligned Data

Clever Integration: The Sherlock Holmes Way to Tease Solutions from a Fog of Misaligned Data

Big Data?  Forgive me for being cynical, but these days I’m suspicious of any noun with the “Big” adjective in front of it.

“Big” things often lead to big disappointments: Big Banks, Big Government, Big Debt, “Too Big to Fail” automobile companies, and “Big Transformation” projects that cost a ton of money but fail to make a telecom’s operating environment simpler or less costly to maintain.

If it was my choice, I’d substitute “Big Data” with something a little more descriptive like “Real-Time Data” or “Fine-Grained Data”?  In analytics terms, those are the chief benefits you get from these cheaper and faster boxes: the chance to act on data quicker and view it in far greater detail.

Now while Big Data enables many new capabilities, its value is greatly enhanced when you can combine it with “Big Thinking”.

Big Thinking is what Benedict Enweani and his UK-based software firm Ontology are all about.  And we’re pleased that Benedict has agreed to be interviewed in Black Swan.

First off, I must warn you to not be intimidated by that obscure word, “ontology”.  Not to worry.  Benedict does a great job of explaining his firm’s technology here.  The Ontology guys are brainy, down-to-earth people, not philosophy professors tossing $10 words around to put you to sleep.

I’m not going to steal Benedict’s thunder, but I will say that the analogy to Sherlock Holmes style thinking is very apropos.  You can apply deductive reasoning to investigate anything from mysterious murders to complex telecom systems.

As opposed to attacking integration issues with an army of Scotland Yard investigators though, Ontology uses a clever, repeatable method to orchestrate solutions at relatively low cost.  My hunch is you can apply this stuff to many business assurance problems.

Benedict, it would be great if you explained the origins of your firm.

Sure, Dan.  So, the genesis of Ontology, myself and my co-founder, Leo Zancani worked together at Orchestream, a mid-90s startup that focused on IP, VPNs, and internet services provisioning.  Orchestream was acquired in 2003 by Metasolv, then soon after by Oracle.

And the thing we noticed in each and every deployment was it would take 8 out of 10 days to align data coming out of billing, CRM, and inventory systems so that we could just create the IP VPNs for the customers and put them on to the network.

This data alignment issue wasn‘t simply an Orchestream problem.  In fact, every single deployment a service provider makes raised this issue.  And the problem stretches from the Tier 1 giants to very small operators.  BT, for instance, has about 3,000 different OSS systems for their core operations and even small service providers and ISPs have 10 to 20 different systems.

If you have a large array of clean systems, traditional integration technologies can actually bring data together quite nicely.  Likewise, if you only have a few systems and the data is dirty, you can manually clean up those systems and get a joined up view of your customers, bills, and equipment at relatively little cost.

But here’s the point: whether you have 10 systems with very dirty data or 100 systems with moderately dirty data, there is not a single technical solution you can turn to that will bring those data together because the many misalignments are so hard to pin down.

And in many cases, the inventory and CRM suppliers are more than happy to take on these data cleanup projects.  Some get rich from that type of work.

Wouldn‘t it be nice if telecoms were like where it is like one-to-one relationship between a book sold and something in physical inventory?  In telecom, things are never that simple.

Exactly, we’re now helping Level 3 Communications to align their inventory and CRM.  At face value, it sounds rather straightforward, but the problem is very complicated because there are something like 12 different systems where inventory data is stored.

The other complication we have in the telecom industry is that different internal consumers of data look at it in entirely different ways.  In service provisioning, it’s about configuring ports on a Cisco router.  In a Siebel CRM, you focus on a catalog of products with certain customer-restricted parameters.  In a billing system, you worry about a line item associated with a particular customer order.  In a performance management system, you’re collecting events and matching that with a service model and IP addresses.

So how do you begin to address this immense complexity?  How do you decide where to even start?

Well, suppose we wanted somebody with a fresh perspective to come in and solve the problem.  What if we went to the owner of a Mom and Pop shoe store and said, “Here’s our network operations center.  It’s a mess and I want you to go fix it.”

Five days later, the guy from the Mom and Pop store returns and says, “Yeah, I fixed it.” And you stand there incredulous and ask him what he did and he says:

“I started with the network and saw where the actual services were and wrote those down.  Then I went to the CRM system to look at the ”adds“ and ”deletes“ to determine what the network configuration should be.  Then I noticed how you had forgotten to disconnect a bunch of circuits, so I freed those up.  Finally, I updated the billing records to match the CRM.”

OK, what’s the point of my story?  It’s to say that even the toughest integration problems can be solved in step by step fashion.  Aligning systems is like solving a jigsaw puzzle where each piece you connect gives you a hint as to what to look for next.  You arrive at answers by conducting a series of searches from one system to the next, checking the inconsistencies and reconciling views.

But aren‘t you relying on an imprecise method of discovering where the errors lie?  In a traditional systems integration approach, you’re examining the systems as they were originally designed and conceived.  Isn’t that a more logical approach?

Traditional integration may be the more logical, but it is not the most efficient path -- as the history of telecom integration proves.  Integration is often an enormously expensive and risky business.

Consider this: the world’s biggest misaligned data system is not at BT, Verizon, or China Telecom.  The world’s biggest misaligned data system is the internet itself.  And yet the internet is very effective.  People will often examine seven or eight web sites before they decide to buy something.  The internet has become a key ordering channel for airlines, hotels, and many other industries, and yet nobody requires a fully integrated and joined view of the internet to get useful work done or to find answers.

The same goes for the systems at telecoms large and small.  The point is to deliver as much alignment as is required by the mission, not to align all systems and spend a fortune doing so.

What are some of the clues you use to connect the dots.  How do you how do you achieve accuracy with your system?

Well the first place to look is field names and id numbers.  You try to match these across multiple systems and see what you find.  An IP address is a unique identifier within a network.  And wherever you have an IP address, it implies the existence of an interface.  Likewise, an IP address implies the existence of a router or switch.  If an IP address can‘t be associated with a network element, that breaks the “ontology” or rules of existence for things and therefore indicates an error in the data.

Along the way, we can correct a slightly wrong identifier.  The whole point is to present the matches the system finds in a way that makes it easy for the network or system expert to make the final classification decisions.

In the area of network reconciliation, we can identify circuits that are likely to be stranded assets.  We may not be able to find a match in the network side, but we can investigate services being leased with no associated bill to a customer.

Other companies, such as Subex, are out there offering network inventory reconciliation solutions to identify stranded assets.  How would you distinguish your approach from theirs?

I know about TrueSource and it’s a fundamentally different methodology.  Their starting point involves building a very extensive model of the underlying systems, and then pulling data from those systems so you can look for inconsistencies

Rather than start with a model, Ontology’s approach is to recognize the structure that is implicit in the data systems that people already have.  Instead of pulling data from existing systems into another database, we pull it into a graph.

When you say “graph” I think of a probability graph with a normal distribution curves.  Is that what you mean?

No, I’m using graph in the sense of a map, a fully connected node and edge structure that stores relationships: one entity connects to another entity through this particular relationship.  And you can scan the nodes and examine all the relationships, so what you see is a kind of spider web called a “graph”.

What we are really storing is metadata — tags, pointers, and relationships among various databases and objects.

The Ontology Architecture

The Ontology Architecture

And as we examine the patterns in that graph, we look for consistencies, inconsistencies and when we find them, we classify things.  In short, our approach deduces the structure of network and service delivery systems without having to know very much about the kind of data we’re looking at.  And using this method we are able to deliver operational value in six to eight weeks, and the reason we can do that is we do not do a big integration at the beginning of the project.

Actually, Leo and I first got the idea for this approach from the intelligence community who pore through massive data sets trying to find terrorist cells.  In that scenario, you have no control over where and when the data will change.  Investigative work like that is a great example of teasing out the details of underlying systems using a centralized search method.

Benedict, this is highly interesting stuff.  It would be great if you could recap your perspective and value proposition.

Well, Dan, it’s no secret that the telecom business is getting more complex.  New services and new network technology are expanding.  In addition, many new partners are entering the scene, be they: virtual network operators, cloud providers, IT outsource partners, over-the-top content providers -- even managed service providers.

But no matter how complex back office systems and partner relationships get, a telecom’s ability to squeeze out greater profits and efficiency depends on knowing a lot about customer behavior and network utilization that only a finer grain level of inter-system analytics can deliver.

So there’s a tremendous need to join data across multiple misaligned systems for business intelligence , optimization, and assurance purposes.  But given the nature of today’s business, you need to do that at very low cost.  And that tends to rule out those expensive and risky full-scale integration projects that were the norm in the past.

We tackle the problem a different way.  We use orchestration and discovery techniques to align systems because they don‘t require you to change out or remodel your existing infrastructure.

What more can I say?  Fortunately we became profitable last year and doubled our revenue the last two.  So I encourage your readers to check us out.

Copyright 2012 Black Swan Telecom Journal

Benedict Enweani

Benedict Enweani

As CEO and co-founder of Ontology Systems, Benedict aims to make his company the leading innovator of semantic solutions in the data center and networking marketplace.  Before founding Ontology in 2005, Benedict was CTO at Corvil Networks and CTO at Orchestream (now Oracle).   Contact Benedict via

Black Swan Solution Guides & Papers

cSwans of a Feather

Related Articles

  • Subex’s New HyperSense AI-Platform Aims to Turn Telcos into Agile, Analytics-Driven Ubers and Rakutens interview with Rohit Maheshwari — An expert explains why telcos must swiftly adopt the AI/analytics methods of hyperscaler firms.  Giving telco examples where cross-company data sharing can make a difference, he explains the 12 key features of a new platform designed to deliver more agile and collaborative intelligence across a telco.
  • Subex’s COO: Machine Learning, Disruption & Adaptive Biz Models to Impact Telecoms in 2020 interview with Shankar Roddam — In our dynamic and digital-driven world, telecoms and enterprises face many business risks.  So what can you do to plot a successful future?  Hear 5 prophecies on 2020 telecom trends from the Chief Operating Officer of Subex.
  • TELUS Analytics Users Get Productivity Boost from Internal Team of Data Access & Showcasing Experts-for-Hire interview with Mange Kumarasamy — How a large organization with hundreds of analytics users gets help from an  internal Big Data team who sources multiple back-end databases; builds tailored reports; and drives campaigns that answer strategic questions for users.
  • Non-Verbal Speech Analytics: Monitoring Voice Calls in Real-Time for Customer Care, Sales, Retention & Onboarding interview with Yoav Degani — Non-verbal speech analysis studies the emotional context of voice qualities like intonation, tone, emphasis and rhythm.  A pioneer in voice analytics explains how its technology benefits customer care, sales, retention and onboarding.
  • Telecom CVM: From Scattered Campaigns to Unified & Consistent Communication with Customers interview with Cretièn Brandsma — Despite the many failures Customer Value Management has faced in telecom, CVM’s future is very hopeful.  A carrier expert explains why telecoms have faltered, how customer experience programs can be revitalized, and where telecoms should invest in better tactics and technology.
  • The Key to Driving 4G Profit: Sell Value, Not Bandwidth by Miri Duenias — Are you struggling to earn a profit on your 4G investments?  Many operators are failing today on the marketing side.  But aligning 4G products with a customer’s personal preferences and desires provides the necessary sizzle to boost sales and earn a handsome ROI.
  • Will Real-Time Decisioning Save Big Data Analytics from Overblown Hype? interview with Tom Erskine — Telecom analytics is more than just collecting and analyzing data.  It’s also about taking action — correct action — often in real-time and across a complex provisioning environment.  In this interview you’ll hear how next best actions are creating value in retention and upselling through a more flexible, business-process driven approach.
  • A Big Data Starter Kit in the Cloud: How Smaller Operators Can Get Rolling with Advanced Analytics interview with Ryan Guthrie — Medium to small operators know “big data” is out there alright, but technical staffing and cost issues have held them back from implementing it.  This interview discusses the advantages of moving advanced analytics to the cloud where operators can get up and running faster and at lower cost.
  • The Customer Engagement Era: How Personalization & Backend Integration Leads to a Richer Mobile Biz interview with Rita Tochner — How does a mobile operator move its subscribers to higher levels of spending and profit?  Fierce competition, social media scrutiny, and the high cost of new networks all conspire against these goals.  In this interview, however, you’ll learn how engaging better with customers, getting more personal, and being more sensitive to their individual needs is the path forward.
  • Telecoms Failing Badly in CAPEX: The Desperate Need for Asset Management & Financial Visibility interview with Ashwin Chalapathy — A 2012 PwC report put the telecom industry on the operating table, opened the patient up, and discovered a malignant cancer: poor network CAPEX management, a problem that puts telecoms in grave financial risk.  In this interview, a supplier of network analytics solutions provides greater detail on the problem and lays out its prescription for deeper asset management, capacity planning and data integrity checks.
  • Batting for More Churn Reduction and Revenue Assurance Home Runs interview with Peter Mueller — What’s it like to transform an IT shop to big data and cloud?  In this interview, the CTO of a boutique revenue assurance explains how his firm made the leap.  He shows how project-oriented programs and working with carrier customers to explore RA and churn reduction “hunches” is where much of the action is.
  • History Repeats: The Big Data Revolution in Telecom Service Assurance interview with Olav Tjelflaat — The lessons of telecom software history teach that new networks and unforeseen industry developments have an uncanny knack for disrupting business plans.  A service assurance incumbent reveals its strategy for becoming a leader in the emerging network analytics and assurance market.
  • From Alarms to Analytics: The Paradigm Shift in Service Assurance interview with Kelvin Hall — In a telecom world with millions of smart devices, the service assurance solutions of yesteryear are not getting the job done.  So alarm-heavy assurance is now shifting to big data solutions that deliver visual, multi-layered, and fine-grained views of network issues.  A data architect who works at large carriers provides an inside view of the key service provider problems driving this analytics shift.
  • The Shrink-Wrapped Search Engine: Why It’s Becoming a Vital Tool in Telecom Analytics interview with Tapan Bhatt — Google invented low cost, big data computing with its distributed search engine that lives in mammoth data centers populated with thousands of commodity CPUs and disks.  Now search engine technology is available as “shrink wrapped” enterprise software.  This article explains how this new technology is solving telecom analytics problems large and small.
  • Sharing Intelligence, Services, and Infrastructure across the Telecom Galaxy interview with Gary Zimmerman — The telecom industry is an industry of sharing.  In fact, the rise of mobile broadband is driving a greater reliance on real-time intelligence, services trading, and infrastructure exchange.  In this article, a leading info exchange provider explains the value of its services portfolio and points to other interoperability and sharing ideas under development.
  • Data Monetization: Why Selling Intelligence is a Hot New Revenue Stream for Mobile Carriers interview with Joe Levy — Data monetization is a revenue dream come true for mobile carriers: a highly profitable sideshow where the carrier analyzes and sells data it already collects for other purposes.  In this article you’ll learn how operators monetize their data through use cases in corporate advertising and media branding.
  • Harvesting Big Data Riches in Retailer Partnering, Actionable CEM & Network Optimization interview with Oded Ringer — In the analytics market there’s plenty of room for small solution firms to add value through a turnkey service or cloud/licensed solution.  But what about large services firms: where do they play?  In this article you’ll learn how a global services giant leverages data of different types to help telcos: monetize retail partnerships, optimize networks, and make CEM actionable.
  • Raising a Telco’s Value in the Digital Ecosystem: One Use Case at a Time interview with Jonathon Gordon — The speed of telecom innovation is forcing software vendors to radically adapt and transform their business models.  This article shows how a deep packet inspection company has  expanded into revenue generation, particularly  for mobile operators.  It offers a broad palette of value-adding use cases from video caching and parental controls to application-charging and DDoS security protection.
  • Radio Access Network Data: Why It’s Become An Immensely Useful Analytics Source interview with Neil Coleman — It’s hard to overstate the importance of Radio Access Network (RAN) analytics to a mobile operator’s business these days.  This article explains why the RAN data, which lives in the air interface between the base station and the handset --  can be used for a business benefit in network optimization and customer experience.
  • Back Office Streamlining to Enterprise Support: The Many Flavors of Wireline Analytics interview with Tom Nolting — Mobile analytics gets plenty of press coverage, but analytics is just as crucial for wireline operators.  In this article, a billing VP at a leading wireline operator discusses several diverse uses of analytics in billing, enterprise sales/retention, and network partner margin assurance.
  • Analytics Biology: The Power of Evolving to New Data Sources and Intelligence Gathering Methods interview with Paul Morrissey — Data warehouses create great value, yet it’s now time to let loose non-traditional big data platforms that create value in countless pockets of operational efficiency that have yet to be fully explored.  This article explains why telecoms must expand their analytics horizons and bring on all sorts of new data sources and novel intelligence gathering techniques.
  • B/OSS Mathematics: The Quest to Analyze Business Problems & Drive Operating Decisions interview with Matti Aksela — Analytics is the glory of mathematics brought to practical use.  And in telecom, analytics has merely stratched the surface of its full potential.  In this article, you’ll learn how machine learning is being combined with the power of CDR number crunching to optimize mobile top-ups, control churn — and in the future, help telecoms make critical network and operating decisions.
  • Leveraging the RA/FM Platform to Deliver Business Insights to Finance & Marketing by Amit Daniel — Carrier professionals using RA and fraud management tools are getting requests from internal customers who want the role of RA/FM platforms expanded to deliver up-to-date analytics data for finance and marketing purposes.  This article advocates a cross-product layer to serve such broader use cases.  The effect would be to transform the existing RA/FM platform into a combined business protection and business growth analytics engine.
  • A Mobile Marketer Service: Bridging Personalization & B/OSS Flowthrough interview with Efrat Nakibly — Marketing analytics is a prescriptive program for driving  actions such as sending a timely promotion to a mobile subscriber.  But completeness demands that you also be able to provision that treatment, qualify the promotion, and keep billing fully in the loop.  This article shows how a managed services program can deliver such an end-to-end process and manage customer life cycles on a one-to-one basis.
  • Science of Analytics: Bringing Prepaid Top Ups & Revenue Maximization under the Microscope interview with Derek Edwards — Prepaid subscribers are the customers that carriers know the least about.  The operator is not interacting with prepaid customers on a monthly basis.  You’re not sending a bill, nor do you have detailed profiles on these customers, especially in the developing world where customers are buying SIMs at a grocery store.  This interview explains how contextual marketing meets the unique analytics challenge of prepaid customers.
  • Connecting B/OSS Silos and Linking Revenue Analytics with the Customer Experience by Anssi Tauriainen — Customer experience analytics is a complex task that flexes B/OSS data to link the customer’s network experience and actions to improve it and drive greater revenue.  In this article, you’ll gain an understanding of how anayltics data needs to be managed across various customer life cycle stages and why it’s tailored for six specific user groups at the operator.
  • Profitable 3G: China’s Mobile Operators Monetize Networks with Retailers & Partners interview with Kevin Xu — Mobile operators are at the center of explosive growth in wireless services.  But to exploit this opportunity requires IT ingenuity and a broader view on how the mobile user can be served.  In this article you’ll learn the innovative techniques Chinese operators use to monetize 3G networks via analytics and partnerships with retailers, social networks, and advertisers.
  • Customer Analytics: Making the Strategic Leap From Hindsight to Foresight interview with Frank Bernhard — Are your company’s analytics programs scattered?  Is there a strategy in place for customer analytics?  This interview with a leading telecom analytics consultant explains why strategy and planning around the analytics function is crucial to getting your money’s worth.  Topics discussed include: hindsight vs. foresight; an advanced analytics program; and the interface sophistication required to support high end vs. low end analytics users.
  • Meeting the OTT Video Challenge: Real-Time, Fine-Grain Bandwidth Monitoring for Cable Operators interview with Mark Trudeau — Cable operators in North America are being overwhelmed by the surge in video and audio traffic.  In this article you’ll learn how Multi Service Operators (MSOs) are now monitoring their traffic to make critical decisions to protect QoS service and monetize bandwidth.  Also featured is expert perspective on trends in: network policy; bandwidth caps; and  customer care issues.
  • Analytics Meditations: The Power of Low-Cost Hardware and the Social Network Within interview with Ken King — Analytics didn‘t arrive yesterday.  Data warehousing and BI have been in the telecom vocabulary for twenty-five or more years.  In this interview, you’ll gain a perspective on why “big data” changes the game and why social network (or social circles) analysis promises the next level of insights.  Other interesting topics include: segmenting the analytics market, engaging with carrier clients, and upgrading from older- to newer-style methodologies.
  • LTE Analytics:  Learning New Rules in Real-Time Network Intelligence, Roaming and Customer Assurance interview with Martin Guilfoyle — LTE is telecom’s latest technology darling, and this article goes beyond the network jargon, to explain the momentous changes LTE brings.  The interview delves into the marriage of IMS, high QoS service delivery via IPX, real-time intelligence and roaming services, plus the new customer assurance hooks that LTE enables.
  • Shared Data Plans: The Challenge of Managing a Family of Pricing, Revenue Assurance, Fraud, and Network Policy Issues by Amit Daniel — Verizon Wireless‘ recent announcement of its move to shared data plans for families shook the mobile industry.  In this column, cVidya’s Amit Daniel shines a spotlight on the knowhow and analytics tools that operators now deperately need to offer the right  shared data price plans, ensure bandwidth throttling is handled correctly, and address new fraud concerns.
  • Analytics Guru: Are Telecoms Ready for the Biz Intelligence Explosion? interview with John Meyers — Business intelligence is evolving from the creation of dashboards and reports to taking action based on a deep knowledge of the environmental context.  The article explores the implications of “big data” in terms of IPTV, storage requirements, hardware, event collection, and deep packet inspection.
  • Social Networking for Telecoms: How To Enlist Friends and Family as Mobile Marketers interview with Simon Rees — Social Network Analysis (SNA) is about exploiting data on “friends and family” connections to combat churn and win new CSP business.  The article explores how an analysis of the ebb and flow of CDRs, phone calls, and messages, can identify key influencers and drive powerful marketing campaigns.
  • Making the Strategic Leap From Billing to Merchandising interview with Humera Malik — Today billing/charging technology has progressed to the point where the usage intelligence, the charges, the user behaviors, and the analytics can all come together in near real-time.  This article discusses the organizational and marketing strategies that enable a operator to create a true “merchandising” system that can revolutionize a CSP’s business.