Email a colleague    

October 2012

Clever Integration: The Sherlock Holmes Way to Tease Solutions from a Fog of Misaligned Data

Clever Integration: The Sherlock Holmes Way to Tease Solutions from a Fog of Misaligned Data

Big Data?  Forgive me for being cynical, but these days I’m suspicious of any noun with the “Big” adjective in front of it.

“Big” things often lead to big disappointments: Big Banks, Big Government, Big Debt, “Too Big to Fail” automobile companies, and “Big Transformation” projects that cost a ton of money but fail to make a telecom’s operating environment simpler or less costly to maintain.

If it was my choice, I’d substitute “Big Data” with something a little more descriptive like “Real-Time Data” or “Fine-Grained Data”?  In analytics terms, those are the chief benefits you get from these cheaper and faster boxes: the chance to act on data quicker and view it in far greater detail.

Now while Big Data enables many new capabilities, its value is greatly enhanced when you can combine it with “Big Thinking”.

Big Thinking is what Benedict Enweani and his UK-based software firm Ontology are all about.  And we’re pleased that Benedict has agreed to be interviewed in Black Swan.

First off, I must warn you to not be intimidated by that obscure word, “ontology”.  Not to worry.  Benedict does a great job of explaining his firm’s technology here.  The Ontology guys are brainy, down-to-earth people, not philosophy professors tossing $10 words around to put you to sleep.

I’m not going to steal Benedict’s thunder, but I will say that the analogy to Sherlock Holmes style thinking is very apropos.  You can apply deductive reasoning to investigate anything from mysterious murders to complex telecom systems.

As opposed to attacking integration issues with an army of Scotland Yard investigators though, Ontology uses a clever, repeatable method to orchestrate solutions at relatively low cost.  My hunch is you can apply this stuff to many business assurance problems.

Benedict, it would be great if you explained the origins of your firm.

Sure, Dan.  So, the genesis of Ontology, myself and my co-founder, Leo Zancani worked together at Orchestream, a mid-90s startup that focused on IP, VPNs, and internet services provisioning.  Orchestream was acquired in 2003 by Metasolv, then soon after by Oracle.

And the thing we noticed in each and every deployment was it would take 8 out of 10 days to align data coming out of billing, CRM, and inventory systems so that we could just create the IP VPNs for the customers and put them on to the network.

This data alignment issue wasn‘t simply an Orchestream problem.  In fact, every single deployment a service provider makes raised this issue.  And the problem stretches from the Tier 1 giants to very small operators.  BT, for instance, has about 3,000 different OSS systems for their core operations and even small service providers and ISPs have 10 to 20 different systems.

If you have a large array of clean systems, traditional integration technologies can actually bring data together quite nicely.  Likewise, if you only have a few systems and the data is dirty, you can manually clean up those systems and get a joined up view of your customers, bills, and equipment at relatively little cost.

But here’s the point: whether you have 10 systems with very dirty data or 100 systems with moderately dirty data, there is not a single technical solution you can turn to that will bring those data together because the many misalignments are so hard to pin down.

And in many cases, the inventory and CRM suppliers are more than happy to take on these data cleanup projects.  Some get rich from that type of work.

Wouldn‘t it be nice if telecoms were like Amazon.com where it is like one-to-one relationship between a book sold and something in physical inventory?  In telecom, things are never that simple.

Exactly, we’re now helping Level 3 Communications to align their inventory and CRM.  At face value, it sounds rather straightforward, but the problem is very complicated because there are something like 12 different systems where inventory data is stored.

The other complication we have in the telecom industry is that different internal consumers of data look at it in entirely different ways.  In service provisioning, it’s about configuring ports on a Cisco router.  In a Siebel CRM, you focus on a catalog of products with certain customer-restricted parameters.  In a billing system, you worry about a line item associated with a particular customer order.  In a performance management system, you’re collecting events and matching that with a service model and IP addresses.

So how do you begin to address this immense complexity?  How do you decide where to even start?

Well, suppose we wanted somebody with a fresh perspective to come in and solve the problem.  What if we went to the owner of a Mom and Pop shoe store and said, “Here’s our network operations center.  It’s a mess and I want you to go fix it.”

Five days later, the guy from the Mom and Pop store returns and says, “Yeah, I fixed it.” And you stand there incredulous and ask him what he did and he says:

“I started with the network and saw where the actual services were and wrote those down.  Then I went to the CRM system to look at the ”adds“ and ”deletes“ to determine what the network configuration should be.  Then I noticed how you had forgotten to disconnect a bunch of circuits, so I freed those up.  Finally, I updated the billing records to match the CRM.”

OK, what’s the point of my story?  It’s to say that even the toughest integration problems can be solved in step by step fashion.  Aligning systems is like solving a jigsaw puzzle where each piece you connect gives you a hint as to what to look for next.  You arrive at answers by conducting a series of searches from one system to the next, checking the inconsistencies and reconciling views.

But aren‘t you relying on an imprecise method of discovering where the errors lie?  In a traditional systems integration approach, you’re examining the systems as they were originally designed and conceived.  Isn’t that a more logical approach?

Traditional integration may be the more logical, but it is not the most efficient path -- as the history of telecom integration proves.  Integration is often an enormously expensive and risky business.

Consider this: the world’s biggest misaligned data system is not at BT, Verizon, or China Telecom.  The world’s biggest misaligned data system is the internet itself.  And yet the internet is very effective.  People will often examine seven or eight web sites before they decide to buy something.  The internet has become a key ordering channel for airlines, hotels, and many other industries, and yet nobody requires a fully integrated and joined view of the internet to get useful work done or to find answers.

The same goes for the systems at telecoms large and small.  The point is to deliver as much alignment as is required by the mission, not to align all systems and spend a fortune doing so.

What are some of the clues you use to connect the dots.  How do you how do you achieve accuracy with your system?

Well the first place to look is field names and id numbers.  You try to match these across multiple systems and see what you find.  An IP address is a unique identifier within a network.  And wherever you have an IP address, it implies the existence of an interface.  Likewise, an IP address implies the existence of a router or switch.  If an IP address can‘t be associated with a network element, that breaks the “ontology” or rules of existence for things and therefore indicates an error in the data.

Along the way, we can correct a slightly wrong identifier.  The whole point is to present the matches the system finds in a way that makes it easy for the network or system expert to make the final classification decisions.

In the area of network reconciliation, we can identify circuits that are likely to be stranded assets.  We may not be able to find a match in the network side, but we can investigate services being leased with no associated bill to a customer.

Other companies, such as Subex, are out there offering network inventory reconciliation solutions to identify stranded assets.  How would you distinguish your approach from theirs?

I know about TrueSource and it’s a fundamentally different methodology.  Their starting point involves building a very extensive model of the underlying systems, and then pulling data from those systems so you can look for inconsistencies

Rather than start with a model, Ontology’s approach is to recognize the structure that is implicit in the data systems that people already have.  Instead of pulling data from existing systems into another database, we pull it into a graph.

When you say “graph” I think of a probability graph with a normal distribution curves.  Is that what you mean?

No, I’m using graph in the sense of a map, a fully connected node and edge structure that stores relationships: one entity connects to another entity through this particular relationship.  And you can scan the nodes and examine all the relationships, so what you see is a kind of spider web called a “graph”.

What we are really storing is metadata — tags, pointers, and relationships among various databases and objects.

The Ontology Architecture

The Ontology Architecture

And as we examine the patterns in that graph, we look for consistencies, inconsistencies and when we find them, we classify things.  In short, our approach deduces the structure of network and service delivery systems without having to know very much about the kind of data we’re looking at.  And using this method we are able to deliver operational value in six to eight weeks, and the reason we can do that is we do not do a big integration at the beginning of the project.

Actually, Leo and I first got the idea for this approach from the intelligence community who pore through massive data sets trying to find terrorist cells.  In that scenario, you have no control over where and when the data will change.  Investigative work like that is a great example of teasing out the details of underlying systems using a centralized search method.

Benedict, this is highly interesting stuff.  It would be great if you could recap your perspective and value proposition.

Well, Dan, it’s no secret that the telecom business is getting more complex.  New services and new network technology are expanding.  In addition, many new partners are entering the scene, be they: virtual network operators, cloud providers, IT outsource partners, over-the-top content providers -- even managed service providers.

But no matter how complex back office systems and partner relationships get, a telecom’s ability to squeeze out greater profits and efficiency depends on knowing a lot about customer behavior and network utilization that only a finer grain level of inter-system analytics can deliver.

So there’s a tremendous need to join data across multiple misaligned systems for business intelligence , optimization, and assurance purposes.  But given the nature of today’s business, you need to do that at very low cost.  And that tends to rule out those expensive and risky full-scale integration projects that were the norm in the past.

We tackle the problem a different way.  We use orchestration and discovery techniques to align systems because they don‘t require you to change out or remodel your existing infrastructure.

What more can I say?  Fortunately we became profitable last year and doubled our revenue the last two.  So I encourage your readers to check us out.

Copyright 2012 Black Swan Telecom Journal

 

About the Expert

Benedict Enweani

Benedict Enweani

As CEO and co-founder of Ontology Systems, Benedict aims to make his company the leading innovator of semantic solutions in the data center and networking marketplace.  Before founding Ontology in 2005, Benedict was CTO at Corvil Networks and CTO at Orchestream (now Oracle).   Contact Benedict via

Related Stories

Related Articles

  • Multi-Vendor SDN/NFV Services: How Integrators Collaborate to Drive the Next Era of Network Automation interview with Bartosz Michalik & Marcin Paszkiewicz — What technical steps are required to orchestrate NFV/SDN and legacy multi-vendor networks?  This tutorial explains the role of key development standards like OpenDaylight and  integration challenges in the new era.
  • Monetizing NFV/SDN: How Policy Will Allow Virtual Networks to Play in the Big Leagues interview with Ari Banerjee — What’s it going to take to transform a bunch of VNFs into a large, synchronized network of VNFs that delivers a huge commercial advantage for operators?  A leading supplier of OSS/BSS solutions explains why closed loop automation and policy are key.
  • Pumping Crude Intelligence: How to Manage Telco Big Data before you Monetize It interview with Thomas Vasen — Mobile telecoms are eager to sell their time- and location-sensitive big data to marketers in other industries.  But a key bottleneck exists: finding a way to efficiently access and manage the huge data voluimes involved.  In this article, a supplier of mediation software explains his firm’s approach to tackling the problem.
  • When Big Data is Too Big: The Value of Real-Time Filtering and Formatting interview with Rick Aguirre — The volume of telecom network traffic is often so huge it outstrips the ability of even “big data” engines to analyze it fast enough.  In this article, you’ll learn about a business that filters and formats very large data sets and delivers the relevant data for applications like: data monetization, network optimization, network peering monitoring, and unstructured data storage.
  • How to Blaze an Access Path to Siloed Enterprise Data and Avoid the High Cost of Systems Integration by Dan Baker — Systems integration has always a big issue in the telecom back office.  And history tells us that it’s a messy and risky exercise when done on a large scale.  But now there’s an alternative to systems integration called “semantic search” and the concept is fully explained in a white paper.
  • Crusaders Clash: The Battle for Control of Telco 2.0 Service Delivery, Billing & Policy interview with Stephen Rickaby — Mobile is still reeling from the shock of being taken out of the driver’s seat in terms of services offered on the handset.  But  will telecoms make a services come-back?  This interview with an expert in the thick of Telco 2.0 transformation action discusses the strategic issues involved and also analyzes Oracle’s recent moves to acquire Acme Packet and Tekelec.
  • Clever Integration: The Sherlock Holmes Way to Tease Solutions from a Fog of Misaligned Data interview with Benedict Enweani — Tired of risky and expensive “big bang” systems integration projects that don‘t solve problems?  In this article you’ll learn how deductive reasoning can be used to weave through mountains of tangeled and misaligned data.  The approach uses a repeatable method to orchestrate solutions at relatively low cost and can be applied to many B/OSS and  assurance problems.
  • Telecom Mediation: Time to Move Back into the Limelight? by Dan Baker — While mediation technology remains crucial to assurance applications, solution vendors have been relatively quiet in recent years.  This article points to reasons why the mediation market may soon get more active.  Among the factors discussed are: consolidation, big data, group merger activity and the offload of mobile transactions to cheaper platforms.
  • Service Assurance Hits the Fashion Runway: The Power of Tailored Visuals interview with Richard Thomas — Some wonderful value is created by presenting data in a way that allows enterprises or internal departments to best extract meaning.  This point is a amplified through a Verizon case study where service assurance data was presented to a high-end fashion retailer during a multimedia event delivered live in over 200 global stores.  The data was presented in differently, keyed to the invidual needs of the retailer’s internal teams.
  • ILEC Business in the Balance: The Case for Probe-Based Traffic Monitoring interview with Darrell Merschak — As new, low-cost  and non-Bellcore switches penetrate the North American market, interexchange carrier are no longer delivering accurate billing usage data to rural ILECs and wireless providers.  This article explains the issue and proposes SS7 probes as a uniform source for establishing accurate usage data handshakes between carriers.
  • Putting a Database at the ‘Nexus’ of Service and Revenue Assurance interview with Michael Olbrich — Closing the B/OSS gap — getting network-facing OSS systems to communicate with customer-facing business systems is one of telecom’s greatest challenges.  This article shows the virtue of unifying B/OSS data and  processes under a single database.  Also discussed is the issue of vendor management and choosing trusted supplier to grow with.