Home
About Black Swan Editorial Mission Reports & Papers
Fraud Systems SIM Box Roaming Fraud PBX Fraud Subscription Databases
RA Practices Business Assurance RA Maturity RA Consulting Bill Testing
Wholesale Systems Net Inventory Margin Assure Partner Mgmt
Customer Behavior Machine Learning Network Analytics Marketing Analytics
Network Behavior Mobile & IoT Security Trusted Communication Signaling
Email a colleague    

April 2014

Pumping Crude Intelligence: How to Manage Telco Big Data before you Monetize It

interview with Thomas Vasen

Pumping Crude Intelligence: How to Manage Telco Big Data before you Monetize It

The geologists who work at big energy firms like Exxon and BP know with great certainty where huge reservoirs of natural gas and oil lie 5 miles below ground.

But it wasn‘t until advanced technologies came along -- like deep water drilling and fracking -- that these giant firms could economically bring that crude energy to the surface and put it to good use powering the planet.

Mobile telecoms face a similar challenge.  Their time- and location-sensitive big data can profitably be sold to marketers in other industries.  Solution firms like Zettics and comScore even offer a turnkey way to analyze and enrich that data so telcos don‘t even have to touch it.

But a key bottleneck still exists: finding a way to access and manage these huge data volumes effectively.  How do you economically pump the data in, store it, and filter it so it’s easier to work with?

Well, one company who has specialized in massaging telco data since 1999 is Sweden-based DigitalRoute.  The firm’s claim to fame is in BSS where it OEMs its mediation solution to billing heavy-weights like Netcracker and Comverse.

But now, the great excitement around mobile data monetizing has brought DigitalRoute full-square into the OSS side too.  So here to explain this trend and the many new data management challenges he’s seeing in the market is Thomas Vasen, the firm’s Product Management and Marketing VP.

Dan Baker: Thomas, it would be great if you could explain a little bit about DigitalRoute’s history in the mediation business.

Thomas Vasen: Sure, Dan.  We were actually quite fortunate.  In the late 90s when DigitalRoute was formed, we could ride the wave of commodity hardware and blade servers emerging back then.  And we’ve always designed our software to do end-to-end in-memory processing so that nothing touches disk while we process it.  We take whole files in at once to give us a performance advantage, and we feel that’s still an important edge for us.  We built one of the first truly convergent systems and can today handle batch and real-time data in a hybrid way.

Altogether we have over 300 active customers, most of whom have come to us through our OEM partners, though we serve some 60 operators direct.

What is it about your mediation approach that these OEMs like?  Are you low cost?  What’s the secret sauce?

It is more complicated than people guess.  We have focused on mediation and it requires expertise to do it correctly and efficiently.  Over the years we’ve added more power and speed as you’d expect, but one point we try to emphasize is being partner-friendly.

To start with, the software is fully re-brand-able.  Also, our partners build their own agents to take our technology and make it compatible with their proprietary solutions.  We have also built one-step-upgrade types of functionality where you can correct, upgrade, and even easily roll back features.  And our APIs are useful for embedding the software in a larger system.  The customer doesn‘t even recognize that mediation from us is a separate module.

I understand an OEM you’ve recently attracted is SAP.  What’s that all about?

Yes, we are the data access point and collection engine for SAP who takes telco data and sells it as intelligence to advertising firms and marketing agents for example.  The idea is to take raw geo-location and radio access network data and feed it enriched with demographics information from CRM, but anonymized of personal identifiers to SAP’s HANA databases.  There they analyze it for marketing purposes and detect interesting behavioral patterns.  Their service is called SAP Consumer Insight 365; it really unlocks and monetizes the network data.

Now the typical marketing agency who consumes this data might serve the leading department store in Sweden, as an example.  And that retailer wants to know what kind of people walk by their stores but never come inside.  That intelligence is of great value to them because if they cross-correlate demographics with location data, they can market to them much more effectively.

Now SAP is an IT giant with huge capabilities.  So why can‘t they do the data gathering and filtering themselves?

Basically they can‘t speak telco data at a deep level.  Telco has very specific data formats, different aggregation rules, especially when you get into the radio access network world.  It tricky to extract that.  SAP and other IT vendors lack the capacity to process it accurate and efficiently.

What we do is intelligently merge telco data streams and put everything together into something manageable.  This is why SAP is working with us.

What kinds of data do you capture?

Well, our claim to fame, of course, is CDR processing for BSS applications, but DPI is an important new data stream we work with today.  We don‘t capture DPI directly: we take in the records reported by the DPI engines or probes of Ericsson, Sandvine, Allot, Tek, Procera, and others.  We then cross reference to identify what kind of service the customer has or the URLs being served.

This DPI data is invaluable in measuring customer experience.  It’s not just about knowing what customers are doing, but also looking at dropped calls and sessions and figuring out what the customer did before and after the drop.

This data is all available but most mobile operators are not using it right now because the volumes are so massive, and they haven‘t learned how to manage it.

Why is that a problem?  I mean, commodity hardware is here and standards like Hadoop are making access to large data sets more affordable.

Hadoop’s vision is great, but the idea of plucking out data streams and saving them forever is not really working in the world of truly massive telco data.

We think the answer is to be real clever about the questions you ask of the data.

Most operators know exactly what they need to do with their network data.  It’s pretty structured.  The challenge is to filter it down to a reasonable size so it can be easily digested and worked with.  Accessing the raw data alone doesn‘t work well because you are dealing with insane data volumes.

You need to prep and strategically look at your data.  And it’s hard because in telecom the data off the switch is usually in binary form, not ASCII.  So you can‘t just dump it into Hadoop and expect magic to happen.  Data has to be converted to be able to correlate and unlock its value.

To give you an idea: a typical tier 2 operator in Europe takes in about 200 million records a day on the BSS side.  And a tier 1 operator in the U.S. might be dealing with 6 or 7 billion mediation records a day for BSS.  But that’s peanuts compared to the OSS side: an LTE network for a U.S. tier one is 2 million records a second!

And yet, DigitalRoute is able to manage that volume on a small set of standard hardware blades.

OK, we’ve talked about big data monetization, but what about the BSS side of the house?  Any action happening there?

As you know, Dan, the billing paradigm has shifted in a big way from minutes to bytes.  And all the BSS logic around charging based on distance, area codes, time of day, etc. is not needed anymore.

But where there’s a lot of action in understanding the data usage -- counting billions of small records.

We call this capability “service control”, which starts with identifying the data records and routing them to a bucket that either counts subscriber usage or content provider usage.  And our system allows you to create these buckets dynamically with an unlimited hierarchy.

Speaking of content, that’s a huge topic of interest.  In particular, there’s a big tussle over exactly who is going to pay for streaming digital video.

Yes, and a lot of early action there is happening in the U.S. market where some recent court rulings have relaxed the net neutrality laws somewhat.

And to me, these new regulations introduce a little common sense and give the service providers a little relief.  The problem is that video subscription services like Hulu and Netflix are consuming an enormous amount of bandwidth across high speed Internet modems.  Sandvine says that Netflix alone is eating up 30% or more of the North American data traffic.

So one remedy that’s emerging -- and AT&T was the first to announce it publicity — is to enable a reverse charging model similar to 1-800 and toll-free numbers.  The idea is that the company who owns the content pays for the delivery charges.

And this is really the way it should be.  After all, when you buy a book from Amazon, you also pay for the postage to have that book delivered to your home.

This trend is also playing into the cable industry too.  Very recently Netflix and Comcast came to an agreement where Netflix agrees to pay an undisclosed amount for delivering its content to Comcast subscribers.

So anyway you slice it: either tracking usage or the number of times a user passes a retail store, sounds like there’s plenty of opportunity for DigitalRoute and its OEMs to stay busy.

We certainly think so, Dan.

Copyright 2014 Black Swan Telecom Journal

 
Thomas Vasen

Thomas Vasen

Thomas Vasen is VP Product Management & Marketing at DigitalRoute.  He has over 17 years operational experience with product and service development in telecoms.  Most recently head of solution development at service assurance pioneer Polystar OSIX, before that he was an entrepreneur in a series of Voice over IP start-ups at operators in Europe.

At B2 Bredband AB, the largest ETTH broadband operator in Sweden, he was responsible for the setup and operation of the first primary line local-loop replacement service launched on SIP technology in the world.  Thomas has studied at the Erasmus University in Rotterdam and at the London School of Economics.   Contact Thomas via

Black Swan Solution Guides & Papers

cSwans of a Feather

  • Pumping Crude Intelligence: How to Manage Telco Big Data before you Monetize It — interview with Thomas Vasen — Mobile telecoms are eager to sell their time- and location-sensitive big data to marketers in other industries.  But a key bottleneck exists: finding a way to efficiently access and manage the huge data voluimes involved.  In this article, a supplier of mediation software explains his firm’s approach to tackling the problem.
  • When Big Data is Too Big: The Value of Real-Time Filtering and Formatting — interview with Rick Aguirre — The volume of telecom network traffic is often so huge it outstrips the ability of even “big data” engines to analyze it fast enough.  In this article, you’ll learn about a business that filters and formats very large data sets and delivers the relevant data for applications like: data monetization, network optimization, network peering monitoring, and unstructured data storage.
  • Crusaders Clash: The Battle for Control of Telco 2.0 Service Delivery, Billing & Policy — interview with Stephen Rickaby — Mobile is still reeling from the shock of being taken out of the driver’s seat in terms of services offered on the handset.  But  will telecoms make a services come-back?  This interview with an expert in the thick of Telco 2.0 transformation action discusses the strategic issues involved and also analyzes Oracle’s recent moves to acquire Acme Packet and Tekelec.
  • Telecom Mediation: Time to Move Back into the Limelight? — by Dan Baker — While mediation technology remains crucial to assurance applications, solution vendors have been relatively quiet in recent years.  This article points to reasons why the mediation market may soon get more active.  Among the factors discussed are: consolidation, big data, group merger activity and the offload of mobile transactions to cheaper platforms.
  • Putting a Database at the ‘Nexus’ of Service and Revenue Assurance — interview with Michael Olbrich — Closing the B/OSS gap — getting network-facing OSS systems to communicate with customer-facing business systems is one of telecom’s greatest challenges.  This article shows the virtue of unifying B/OSS data and  processes under a single database.  Also discussed is the issue of vendor management and choosing trusted supplier to grow with.

Related Articles

  • Tokopedia, Indonesia’s E-Commerce King, Partners with 11 Million Merchants; Adopts Multi-Cloud to Drive Innovation — interview with Warren Aw & Ryan de Melo — Indonesia’s Tokopedia, founded in 2009, has grown to become one of world’s leading e-commerce players.  Read about its success, technology direction, and multi-cloud connectivity adoption.
  • Bridge Alliance: Knocking Down Regional & Mobile Connectivity Barriers so Connected Car Markets Get Rolling in Asia — interview with Kwee Kchwee — The CEO of an Asian consortium of mobile operators explains how they  help simplify and harmonize their members‘ operations in support of multi-national corporations.  This integration is enabling two huge industries to come together in Asia: auto manufacturing and telco.
  • Epsilon’s Infiny NaaS Platform Brings Global Connection, Agility & Fast Provision for IoT, Clouds & Enterprises in Southeast Asia, China & Beyond — interview with Warren Aw — Network as a Service, powered by Software Defined Networks, are a faster, more agile, and more partner-friendly way of making data global connections.  A leading NaaS provider explains the benefits for cloud apps, enterprise IT, and IoT.
  • PCCW Global: On Leveraging Global IoT Connectivity to Create Mission Critical Use Cases for Enterprises — interview with Craig Price — A leading wholesale executive explains the business challenges of the current global IoT scene as it spans many spheres: technical, political, marketing, and enterprise customer value creation.
  • Senet’s Cloud & Shared Gateways Drive LoRaWAN IoT Adoption for Enterprise Businesses, Smart Cities & Telecoms — interview with Bruce Chatterley — An IoT netowork pioneer explains how LoRaWAN tech fits in the larger IoT ecosystem.  He gives use case examples, describes deployment restraints/costs, and shows how partnering, gateway sharing, and flexible deployment options are stimulating growth.
  • ARM Data Center Software’s Cloud-Based Network Inventory Links Network, Operations, Billing, Sales & CRM to One Database — interview with Joe McDermott & Frank McDermott — A firm offering a cloud-based network inventory system explains the virtues of: a single underlying database, flexible conversions, task-checking workflow, new software business models, views that identify stranded assets, and connecting to Microsoft’s cloud platform.
  • Pure Play NFV: Lessons Learned from Masergy’s Virtual Deployment for a Global Enterprise — interview with Prayson Pate — NFV is just getting off the ground, but one cloud provider to enterprises making a stir in virtual technology waters is Masergy.  Here are lessons learned from Masergy’s recent global deployment using a NFV pure play software approach.
  • The Digital Enabler: A Charging, Self-Care & Marketing Platform at the Core of the Mobile Business — interview with Jennifer Kyriakakis — The digital enabler is a central platform that ties together charging, self-care, and marketing.  The article explains why leading operators consider digital enablers pivotal to their digital strategies.
  • Delivering Service Assurance Excellence at a Reduced Operating Cost — interview with Gregg Hara — The great diversity and complexity of today’s networks make service assurance a big challenge.  But advances in off-the-shelf software now permit the configuring and visualizing of services across multiple technologies on a modest operating budget.
  • Are Cloud-Based Call Centers the Next Hot Product for the SMB Market? — interview with Doron Dovrat — Quality customer service can improve a company’s corporate identity and drive business growth.  But many SMBs are priced out of acquiring modern call center technology.  This article explains the benefits of affordable and flexible cloud-based call centers.
  • Flexing the OSS & Network to Support the Digital Ecosystem — interview with Ken Dilbeck — The need for telecoms to support a broader digital ecosystem requires an enormous change to OSS infrastructures and the way networks are being managed.  This interview sheds light on these challenges.
  • Crossing the Rubicon: Is it Time for Tier Ones to Move to a Real-Time Analytics BSS? — interview with Andy Tiller — Will tier one operators continue to maintain their quilt works of legacy and adjunct platforms — or will they radically transform their BSS architecture into a new  system designed to address the new telecom era?  An advocate for radical transformation discusses: real-time analytics, billing for enterprises, partnering mashups, and on-going transformation work at Telenor.
  • Paradigm Shift in OSS Software: Network Topology Views via Enterprise-Search — interview with Benedict Enweani — Enterprise-search is a wildly successful technology on the web, yet its influence has not yet rippled to the IT main stream.  But now a large Middle Eastern operator has deployed a major service assurance application using enterprise-search.  The interview discusses this multi-dimensional topology solution and compares it to traditional network inventory.
  • The Multi-Vendor MPLS: Enabling Tier 2 and 3 Telecoms to Offer World-Class Networks to SMBs — interview with Prabhu Ramachandran — MPLS is a networking technology that has caught fire in the last decade.  Yet the complexity of MPLS has relegated to being mostly a large carrier solution.  Now a developer of a multi-vendor MPLS solutions explains why the next wave of MPLS adoption will come from tier 2/3 carriers supporting SMB customers.
  • Enabling Telecoms & Utilities to Adapt to the Winds of Business Change — interview with Kirill Rechter — Billing is in the midst of momentous change.  Its value is no longer just around delivering multi-play services or sophisticated rating.  In this article you’ll learn how a billing/CRM supplier has adapted to the times by offering deeper value around the larger business issues of its telecom and utility clients.
  • Driving Customer Care Results & Cost Savings from Big Data Facts — interview with Brian Jurutka — Mobile broadband and today’s dizzying array of app and network technology present a big challenge to customer care.  In fact, care agents have a hard time staying one step ahead of customers who call to report problems.  But network analytics comes to the rescue with advanced mobile handset troubleshooting and an ability to put greater intelligence at the fingertips of highly trained reps.
  • Hadoop and M2M Meet Device and Network Management Systems — interview with Eric Wegner — Telecom big-data in networks is more than customer experience managment: it’s also about M2M plus network and element management systems.  This interview discusses the explosion in machine-to-machine devices, the virtues and drawbacks of Hadoop, and the network impact of shrink-wrapped search.
  • The Data Center & Cloud Infrastructure Boom: Is Your Sales/Engineering Team Equipped to Win? — by Dan Baker — The build-out of enterprise clouds and data centers is a golden opportunity for systems integrators, carriers, and cloud providers.  But the firms who win this business will have sales and engineering teams who can drive an effective and streamlined requirements-to-design-to-order process.  This white paper points to a solution — a collaborative solution designs system — and explains 8 key capabilities of an ideal platform.
  • Big Data: Is it Ready for Prime Time in Customer Experience Management? — interview with Thomas Sutter — Customer experience management is one of the most challenging of OSS domains and some suppliers are touting “big data” solutions as the silver bullet for CEM upgrades and consolidation.  This interview challenges the readiness of big data soluions to tackle OSS issues and deliver the cost savings.  The article also provides advice on managing technology risks, software vendor partnering, and the strategies of different OSS suppliers.
  • Calculated Risk: The Race to Deliver the Next Generation of LTE Service Management — interview with Edoardo Rizzi — LTE and the emerging heterogeneous networks are likely to shake up the service management and customer experience management worlds.  Learn about the many new network management challenges LTE presents, and how a small OSS software firm aims to beat the big established players to market with a bold new technology and strategy.
  • Decom Dilemma: Why Tearing Down Networks is Often Harder than Deploying Them — interview with Dan Hays — For every new 4G LTE and IP-based infrastructure deployed, there typically a legacy network that’s been rendered obsolete and needs to be decommissioned.  This article takes you through the many complexities of network decom, such as facilities planning, site lease terminations, green-safe equipment disposal, and tax relief programs.
  • Migration Success or Migraine Headache: Why Upfront Planning is Key to Network Decom — interview with Ron Angner — Shutting down old networks and migrating customers to new ones is among the most challenging activities a network operators does today.  This article provides advice on the many network issues surrounding migration and decommissioning.  Topics discussed include inventory reconciliation, LEC/CLEC coordination, and protection of customers in the midst of projects that require great program management skills.
  • Navigating the Telecom Solutions Wilderness: Advice from Some Veteran Mountaineers — interview with Al Brisard — Telecom solutions vendors struggle mightily to position their solutions and figure out what to offer next in a market where there’s considerable product and service crossover.  In this article, a veteran order management specialist firm lays out its strategy for mixing deep-bench functional expertise with process consulting, analytics, and custom API development.
  • Will Telecoms Sink Under the Weight of their Bloated and Out-of-Control Product Stacks? — interview with Simon Muderack — Telecoms pay daily for their lack of product integration as they constantly reinvent product wheels, lose customer intelligence, and waste time/money.  This article makes the case of an enterprise product catalog.  Drawing on central catalog cases at a few Tier 1 operators, the article explains the benefits: reducing billing and provisioning costs, promoting product reuse, and smoothing operations.
  • Virtual Operator Life: Enabling Multi-Level Resellers Through an Active Product Catalog — interview with Rob Hill — The value of product distribution via virtual operators is immense.  They enable a carrier to sell to markets it cannot profitably serve directly.  Yet the need for greater reseller flexibility in the bundling and pricing of increasingly complex IP and cloud services is now a major channel barrier.  This article explains what’s behind an innovative product catalog solution that doubles as a service creation environment for resellers in multiple tiers.
  • Telecom Blocking & Tackling: Executing the Fundamentals of the Order-to-Bill Process — interview with Ron Angner — Just as football teams need to be good at the basics of blocking and tackling, telecoms need to excel at their own fundamental skillset: the order-to-cash process.  In this article, a leading consulting firm explains its methodology for taking operators on the path towards order-to-cash excellence.  Issues discussed include: provisioning intervals; standardization and simplicity; the transition from legacy to improved process; and the major role that industry metrics play.
  • Wireline Act IV, Scene II: Packaging Network & SaaS Services Together to Serve SMBs — by John Frame — As revenue from telephony services has steadily declined, fixed network operators have scrambled to support VoIP, enhanced IP services, and now cloud applications.  This shift has also brought challenges to the provisioning software vendors who support the operators.  In this interview, a leading supplier explains how it’s transforming from plain ol‘ OSS software provider to packager of on-net and SaaS solutions from an array of third party cloud providers.
  • Telecom Merger Juggling Act: How to Convert the Back Office and Keep Customers and Investors Happy at the Same Time — interview with Curtis Mills — Billing and OSS conversions as the result of a merger are a risky activity as evidenced by famous cases at Fairpoint and Hawaiian Telcom.  This article offers advice on how to head off problems by monitoring key operations checkpoints, asking the right questions, and leading with a proven conversion methodology.
  • Is Order Management a Provisioning System or Your Best Salesperson? — by John Konczal — Order management as a differentiator is a very new concept to many CSP people, but it’s become a very real sales booster in many industries.  Using electronics retailer BestBuy as an example, the article points to several innovations that can — and are — being applied by CSPs today.  The article concludes with 8 key questions an operator should ask to measure advanced order management progress.
  • NEC Takes the Telecom Cloud from PowerPoint to Live Customers — interview with Shinya Kukita — In the cloud computing world, it’s a long road from technology success to telecom busness opportunity.  But this story about how NEC and Telefonica are partnering to offer cloud services to small and medium enterprises shows the experience of early cloud adoption.  Issues discussed in the article include: customer types, cloud application varieties, geographic region acceptance, and selling challenges.
  • Billing As Enabler for the Next Killer Business Model — interview with Scott Swartz — Facebook, cloud services, and Google Ads are examples of innovative business models that demand unique or non-standard billing techniques.  The article shows how flexible, change-on-the-fly, and metadata-driven billing architectures are enabling CSPs to offer truly ground breaking services.
  • Real-Time Provisioning of SIM Cards: A Boon to GSM Operators — interview with Simo Isomaki — Software-controlled SIM card configuration is revolutionizing the activation of GSM phones.  The article explains how dynamic SIM management decouples the selection of numbers/services and delivers new opportunities to market during the customer acquisition and intial provisoining phase.
  • A Cynic Converted: IN/Prepaid Platforms Are Now Pretty Cool — interview with Grant Lenahan — Service delivery platforms born in the IN era are often painted as inflexible and expensive to maintain.  Learn how modern SDPs with protocol mediation, high availability, and flexible Service Creation Environments are delivering value for operators such as Brazil’s Oi.
  • Achieving Revenue Maximization in the Telecom Contact Center — interview with Robert Lamb — Optimizing the contact center offers one of the greatest returns on investment for a CSP.  The director of AT&T’s contact center services business explains how telecoms can strike an “artful balance” between contact center investment and cost savings.  The discussion draws from AT&T’s consulting with world class customers like Ford, Dell, Discover Financial, DISH Network, and General Motors.
  • Mobile Broadband: The Customer Service Assurance Challenge — interview with Michele Campriani — iPhone and Android traffic is surging but operators struggle with network congestion and dropping ARPUs.  The answer?  Direct  resources and service quality measures to ensure VIPs are indeed getting the quality they expect.  Using real-life examples that cut to the chase of technical complexities, this article explains the chief causes of service quality degradation and describes efficient ways to deal with the problem.
  • Telco-in-a-Box: Are Telecoms Back in the B/OSS Business? — interview with Jim Dunlap — Most telecoms have long since folded their merchant B/OSS software/services businesses.  But now Cycle30, a subsidiary of Alaskan operator GCI, is offering a order-to-cash managed service for other operators and utilities.  The article discusses the company’s unique business model and contrasts it with billing service bureau and licensed software approaches.
  • Bricks, Mortar & Well-Trained Reps Make a Comeback in Customer Management — interview with Scott Kohlman — Greater industry competition, service complexity, and employee turnover have raised the bar in the customer support.  Indeed, complex services are putting an emphasis on quality care interactions in the store, on the web, and through the call center.  In this article you’ll learn about innovations in CRM, multi-tabbed agent portals,  call center agent training, customer treatment philosophies, and the impact of  self-service.
  • 21st Century Order Management: The Cross-Channel Sales Conversation — by John Konczal — Selling a mobile service is generally not a one-and-done transaction.  It often involves several interactions — across the web, call center, store, and even kiosks.  This article explains the power of a “cross-channel hub” which sits above all sales channels, interacts with them all, and allows a CSP to keep the sales conversation moving forward seamlessly.
  • Building a B/OSS Business Through Common Sense Customer Service — by David West — Delivering customer service excellence doesn‘t require mastering some secret technique.  The premise of this article is that plain dealing with customers and employees is all that’s needed for a winning formula.  The argument is spelling out in a simple 4 step methodology along with some practical examples.