Email a colleague    

September 2015

Delivering Service Assurance Excellence at a Reduced Operating Cost

Delivering Service Assurance Excellence at a Reduced Operating Cost

“What you really need is failure.  I’m not talking about failure on a small scale, but a really big failure.  Only after you have that experience can you begin to run your business creatively.”
Barry Diller, media mogul & billionaire, in Inc. Magazine

More than ever, an operator’s choice of network management software can truly spell success or failure.

When an operator lacks an advanced service assurance solution, it sets itself up for service quality failure, which lowers the customer experience — and may ultimately cause the business itself to fail.

And getting it right is a big challenge thanks to the great diversity and complexity of today’s networks.  A good assurance system must not only keep pace with business change, it must also be efficient.  Gone are the days when you could afford to hire a large team of engineers or integrators to keep the system accurate and up-to-date.

Well, one company delivering a service assurance system that fits today’s market need is Centina Systems, and their Vice President of Business Development and Marketing, Gregg Hara, now joins us.  Gregg explains the catalyst for starting Centina, the continuing network challenges they address, and innovative business model they’ve set up to put advanced service assurance within easier reach.

Dan Baker: Gregg, thanks for joining us.  Maybe we could begin by hearing a bit about Centina’s mission.

Gregg Hara: Thanks for the invitation, Dan.  The founders of Centina hail from the major equipment providers in communications — places like Nortel, Alcatel, Fujitsu and Cisco.  And their background was in building Element Management Systems (EMSs) for those hardware vendors, so these guys are pretty network-savvy, especially in transport and optical networks running the TL1 protocol.

Now having spent a good deal of time in the Network Operation Centers (NOCs) of telco customers, they noticed the immense problems the NOC users were having.  Here’s a quick rundown of the issues they noticed:

  • Fighting fires as opposed to doing prevention — What NOC people were basically doing was fighting fires — reacting to events, but doing little in the way of proactive maintenance.
  • Lack of automation to isolate problems and determine customer impact — Having to navigate through multiple systems made it impossible to quickly determine which customers were affected by problems.  The current systems could not even point the user in the right direction.  If they could, they’d be able to head off problems before they impact customers.
  • Users are overwhelmed by the workload — Changes need to be constantly made in the assurance system, yet those changes are very time consuming to implement because lots of scripting and programming is involved.  As a result, users can’t keep up and are forced to cut corners.

So seeing all these service assurance headaches, Centina’s founders figured there must be a better way.  That’s why they decide to go out on their own and build a better product.

What do you believe is the main source of problems in the legacy service assurance systems?

Well, to begin, there’s very little you get out of the box.  When you first install legacy products, they don’t do anything because they are merely toolkits.  You have to spend months and months — perhaps years — customizing them to monitor, correlate and do the tasks you want them to do.

Take for example integrating to all devices in the network.  Every provider usually has between 50 and 250 different device types, and if you want to monitor all of those different device types in one system, that’s a lot of hard work.

First, you have to build out probes to handle all the alarms and traps.  And if you want to monitor device performance, you have to define polling and write formulas for all the performance data.  And once you’ve established that baseline, you’ve also got to maintain it because engineering upgrades the network devices all the time.

This is very cumbersome, yet this is exactly what life is like in a legacy service assurance deployment.  People have dedicated their careers to becoming proficient in these toolkits, and it’s all about writing Perl script, coding and dealing in lots of technical detail.

Now this situation is only going to get worse as SDN and NFV services roll out because there will then be multiple points you have to monitor and correlate, from the application to the virtual network to the physical network.

How do you get around these network integration and maintenance headaches?

Well, at Centina we feel strongly that the only way an operator or MSO can manage service assurance successfully is to get some major help from their software provider.

A service provider shouldn’t have to manage the updating of network device integrations.  Seems like that should be the responsibility of the solution vendor they bought their tool from.

And that’s exactly what we’ve done.  Centina created what we call a Smart Plug-In for every vendor and model number that we have interfaced to at our customers.  Today, that’s over 1,000 different plug-ins across 135 different vendors.  This library works almost like an app on your phone.  The software goes out and discovers all the devices in your network and matches them up to the appropriate plug-in and just starts monitoring.

It will process alarms and handle every performance metric that the device supports.  Then it’s a simple matter of configuring which alarms and metrics you want to collect, how often you want to poll the performance data, and then define any thresholds.

Now when we sign up a new customer, they will invariably ask us to support some new devices not yet in our library.  It takes us between one and three weeks to build a brand new plug-in.

After that, when a new device release comes out, the operator is fully covered under our ongoing maintenance.

What about the service visibility you need to quickly isolate problems and find out what customers are affected?  And what about service level management?  To what degree is that concept real today?

Dan, if you talk to any service provider managing large networks, they’ll tell you that they have very little visibility into their end-to-end services — how the services are layered.  That’s very hard to deliver because these are multi-vendor, multi-technology environments.

Here’s the issue.  When you buy a high speed internet service from a Cox, Time Warner, Level 3, AT&T, or Verizon, you can get a pipe into the internet via Ethernet, but that Ethernet is running over some other technology.  It might be running over SONET or DWDM.  Many different layers may make up that service.  So when a network problem or performance issue occurs, it’s tough for a service provider to figure out where the root cause of the problem lies.

When you first get the alarm or a customer calls to complain about the issue, you need to figure out how the different layers are all being managed by different systems.  So the user is forced to swivel-chair through all these different systems to figure out what’s going.

What we’ve done at Centina is to give the user the ability to visualize end-to-end services, hierarchies, topological layers, plus provide various charts and alarm views.  And, since we have all this information in our system, it can automatically suppress all the noise in the network and highlight the root-cause of the problem to improve the reliability of the network.

It’s a big help because it pulls all the information you need to see all the layers visually.  For instance, it lays out in a tree the various Ethernet components, and mixed in there are the performance and alarm data at each level displayed in a vendor- and technology-independent way.

We have such a system up and running at WOW, Wide Open West, who is the 9th largest cable provider in the U.S. with about one million subscribers.

Gregg, what’s your solution look like from the user (operator) perspective?

Well, as I mentioned before, the catalyst for launching Centina in the first place was user problems in the NOC, so we’ve devoted a lot of time thinking how to improve the user experience.

Number one, everything is browser based, making it easy to learn and configure.  Being browser-based reduces on-going administration time and costs, too.

Now in a legacy assurance system you usually get a couple of alarm dashboards, but the ability to interact with the alarm system is very limited.  Users don’t have a lot of control over visualization and don’t have the ability to change the configuration of the system.

With our NetOmnia product, however, the user herself can create new views and filter the results without touching the underlying database or doing any scripting.  So if the user wants to look at performance in five separate regions or five separate technologies, it’s easy to create that view.

Say you want to automate action on an event.  Well, you can set up a little workflow that says, “If I get a critical alarm and it’s been active for more than 10 minutes and nobody has acknowledged it, then send an SMS to everybody in the NOC to let them know somebody needs to get on this problem fast.” A workflow like that only takes a couple of minutes to create.

So these are some highlights of our system.

Service providers and operators are looking at SDN and NFV to transform their business, so how has Centina prepared to meet the demands of virtualized networks?

Well, our solution can successfully support the current and evolving assurance needs of virtualized networks and we have been seeing an increasing interest in service assurance.  Through virtualization, service providers are not only decreasing their network’s maintenance and associated costs but are also increasing their service’s innovation capabilities and improving customer satisfaction — provided they have the right proactive, service assurance solutions in place to monitor network lifecycle events.  However, though the benefits of SDN and NFV are significant, service assurance is much more challenging and even more necessary than with traditional networks.

What is needed today is a comprehensive and adaptable service assurance solution that provides holistic service health and performance visualization across hybrid legacy and next-generation SDN/NFV networks.  Ultimately, having an end-to-end, strategic service assurance solution in place will help providers successfully transition to virtual networks.

Whereas most vendors are working towards supporting virtual services and presenting mostly slideware at this point, NetOmnia is already integrated with OpenStack and Open Daylight to support the assurance of SDN and NFV services.

Thanks, Gregg.  It’s nice to know that an affordable and full-featured assurance solution like this is available.  And for large telcos who are wedded to their current solution partner, awareness of these nice capabilities should help move some elephants.

Copyright 2015 Black Swan Telecom Journal

Gregg Hara

Gregg Hara

Gregg Hara is Vice President of Business Development and Marketing at Centina Systems.  He has over 20 years in the telecom and OSS software space and joined Centina in 2009.  He takes the initiative of defining new markets and establishing relationships with key service providers and vendors.  Prior to Centina Systems, Gregg was Senior Director of Sales for Sigma Systems and earlier, with TeleGea Systems.

Gregg started his career at GTE and was one of the original members of OSI, and then at Micromuse, in various roles of solution delivery, sales and business development of Network Management and Service Assurance solutions.

Gregg holds a bachelor’s degree in Electrical Enginerring from MIT in Cambridge, MA.

Black Swan Solution Guides & Papers

cSwans of a Feather

  • Delivering Service Assurance Excellence at a Reduced Operating Cost interview with Gregg Hara — The great diversity and complexity of today’s networks make service assurance a big challenge.  But advances in off-the-shelf software now permit the configuring and visualizing of services across multiple technologies on a modest operating budget.
  • Paradigm Shift in OSS Software: Network Topology Views via Enterprise-Search interview with Benedict Enweani — Enterprise-search is a wildly successful technology on the web, yet its influence has not yet rippled to the IT main stream.  But now a large Middle Eastern operator has deployed a major service assurance application using enterprise-search.  The interview discusses this multi-dimensional topology solution and compares it to traditional network inventory.
  • The Multi-Vendor MPLS: Enabling Tier 2 and 3 Telecoms to Offer World-Class Networks to SMBs interview with Prabhu Ramachandran — MPLS is a networking technology that has caught fire in the last decade.  Yet the complexity of MPLS has relegated to being mostly a large carrier solution.  Now a developer of a multi-vendor MPLS solutions explains why the next wave of MPLS adoption will come from tier 2/3 carriers supporting SMB customers.
  • Big Data: Is it Ready for Prime Time in Customer Experience Management? interview with Thomas Sutter — Customer experience management is one of the most challenging of OSS domains and some suppliers are touting “big data” solutions as the silver bullet for CEM upgrades and consolidation.  This interview challenges the readiness of big data soluions to tackle OSS issues and deliver the cost savings.  The article also provides advice on managing technology risks, software vendor partnering, and the strategies of different OSS suppliers.
  • Calculated Risk: The Race to Deliver the Next Generation of LTE Service Management interview with Edoardo Rizzi — LTE and the emerging heterogeneous networks are likely to shake up the service management and customer experience management worlds.  Learn about the many new network management challenges LTE presents, and how a small OSS software firm aims to beat the big established players to market with a bold new technology and strategy.
  • Mobile Broadband: The Customer Service Assurance Challenge interview with Michele Campriani — iPhone and Android traffic is surging but operators struggle with network congestion and dropping ARPUs.  The answer?  Direct  resources and service quality measures to ensure VIPs are indeed getting the quality they expect.  Using real-life examples that cut to the chase of technical complexities, this article explains the chief causes of service quality degradation and describes efficient ways to deal with the problem.

Related Articles

  • Black Swan Guide: Araxxe’s Revenue Assurance Consulting, Testing, and High Definition Billing Analysis Service by Dan Baker — How Araxxe’s end-to-end revenue assurance complements switch-to-bill RA  through telescope RA (external and partner data) and microscope RA (high-definition analysis of complex services like bundling and digital services).
  • Subex’s IDcentral Monetizes Telco & Enterprise Data to Deliver Digital ID & Risk Metric Services for Financing, KYC & More interview with Shankar Roddam — A new digital intelligence service that monetizes the idle data of telecoms and enterprises while also earning a good return for the owner of the data.
  • Opportunities & Obstacles: Consultant Luke Taylor Muses on the State of the Telecom Risk Assurance Business interview with Luke Taylor — A rambling discussion on the state of the risk assurance business with Luke Taylor, independent consultant in telecom revenue/fraud assurance and solution requirements and marketing.
  • LATRO’s Tips for Launching a Successful Revenue & Fraud Assurance Program for Mobile Money Operations in Developing Countries interview with Don Reinhart — A company building mobile money RA/FM tools and  managed services gives a concise, but detailed tutorial on how the Mobile Money Ecosystem works.  Revenue assurance pros will get tips on  what to look for in analytics/assurance tools, controls, and professional services.
  • A WeDo Conference Talk: Consulting & Analytics: Improving your Business Today, Enhancing it Tomorrow interview with Carla Cardoso & Bernado Lucas & Thomas Steagall — Leading risk management consultants explain their mission and walk-through RA, subscription fraud, and collections cases.  They also explain how analytics and machine learning can supplement process optimization.
  • PrologMobile’s Simple and Brilliant Plan to Save US MNOs Billions a Year in Recovered Phones & Retained Customers interview with Seth Heine — An expert in the mobile phone reverse supply chain explains how MNOs — via a neutral third party information exchange — can recover their original phones on the used market and save huge sums in multi-year customer retention.
  • WeDo Explores the IoT Ecosystem in Search of Tomorrow’s Pivotal Fraud & Business Assurance Solutions interview with Carlos Marques — A veteran product manager scans the IoT terrain, discusses key fraud and assurance challenges, and explains the preparatory steps WeDo is taking to become a key player in this emerging market.
  • New Report: Telecom Fraud & Business Assurance Solutions, Services & Strategies by Dan Baker & Luke Taylor & Colin Yates — TRI publishes a new market research report, Telecom Fraud & Business Assurance Solutions, Services & Strategies.  Free executive summary available.
  • Subex Juggles a Wide Variety of Business Assurance and Big Data Analytics Use Cases interview with Rohit Maheshwari — A expert in business assurance solutions explains top use cases such as: IoT security, big data analytics/AI, network asset optimization, multi-player gaming assurance, onboarding mobile subs, and AI customer analytics.
  • MTN Agility: Mastering Exponential Technologies in Revenue/Fraud Assurance and Beyond interview with Danie Maritz & Tony Sani & Luke Taylor — An in-depth look at RAFM operations and innovation at the MTN Group.  Topics discussed include RA/fraud control challenges, strategies, and MTN’s journey to exploit exponential tech (AI, robotics, and ML) in its RAFM program and support of internal non-telco businesses.
  • From Byzantine Software Contracts to Simple & Flexible RA Managed Services interview with Philippe Orsini — Is the way B2B/enterprise software is sold and delivered today progressive — or is it Byzantine in the age of cloud?  An expert lays out the case for managed services in RA and billing verification.
  • Premiere Experts Set to Speak at Summer RAG Conference in London, July 7th and 8th by Dan Baker — The Risk and Assurance Group (RAG) has announced that its 2016 summer conference will expand into a two-day event and feature many premiere experts. 
  • WeDo Hosts Revenue Assurance & Fraud Management Conference in Washington DC by Dan Baker — Black Swan is pleased to announce what looks to be a first class revenue assurance and fraud management conference being put on by WeDo Technologies, on October 1st and 2nd in beautiful Washington DC.
  • Test Call Generators: An Essential Test & Debugging Tool in Mobile Billing Assurance interview with Steffen Öftring — An “active” test call generator (TCG) can see problems that a “passive” revenue assurance system is blind to.  Here’s a discussion on the test call RA  process, over-the-air calls versus core call injection, and test call networks in global roaming RA.
  • The Revenue Assurance Game: How the Rules Change in the Era of IoT & Mobile Broadband interview with Rene Felber & Gadi Solotorevsky — Revenue assurance is perhaps the hardest of telecom functions to define because the term is used in so many different senses.  This discussion on the evolving role of revenue assurance was catalyzed by a survey of experts in the profession.
  • Day in the Life of a Revenue Assurance Analyst interview with Michael Lazarou — Revenue assurance is much more than a software category.  It’s individual analysts struggling to help their larger organizations get a handle on system errors and coordination problems.  In this interview, an analyst reveals the many challenges of getting the revenue assurance job done at a small GSM operator in Europe.
  • Revenue Assurance: History and New Beginnings in RA Maturity interview with Daniela Giacomantonio & Gadi Solotorevsky — The Roman Forum was the center of commercial life in ancient Rome.  Now, two millennia later, the Forum lives on in the exchange of ideas across countless professions and  media.  In this interview, two Revenue Assurance experts discuss both the new RA Maturity initiative of the TM Forum and the value of telco/solution vendor collaboration.
  • Migrating systems or launching LTE next year?  Don‘t forget transformation assurance & optimisation by Efrat Nissimov — System transformations and network migrations are major  revenue impacting events and they should raise a big red flag.  Why?  Because data integrity issues are bound to crop up as CSPs move vital data from a legacy system to something new.  It’s time for transformation assurance.
  • How can Cable/DSL Internet Providers Meet the Usage-Based Billing Mandate? interview with Ryan Guthrie — The popularity of YouTube, Netflix, and Hulu other video outlets has turned the tables on service profitability for cable/DSL service providers.  Many are moving to usage-based billing, but that largely unprepared for the revenue assurance aspects of this move.  This interview explains the technical challenge and points to solutions in billing, speed caps, and traffic revenue monitoring.
  • CABS Revenue Assurance: How Rural LECs can Recover $284 Million in Revenue Shortfalls interview with Kelly Cannon & Darrell Merschak — Independent rural LECs in the U.S. still rely on the AMA/EMI billing formats for CABS billing, even as that format has proven to be highly inaccurate as a source of inter-carrier records.  This interview includes an analysis and discussion of revenue recovery techniques ILECs can use by leveraging SS7 probes.  Also discussed are billing strategies, traffic dumping threats, and the possible fallout from the FCC’s bill-and-keep mandate.
  • Make Business Assurance Progress Every Day: How to Set Goals, Automate, and Energize Your Team interview with Kathleen Romano — Business assurance (BA) skills have wide applicability outside the revenue assurance and fraud mangement domains.  In this article, a telecom executive explains how she’s applying her BA skills in the Payments area.  In addition to discussing the key operational challenges in Payments, the interview also provides keen insights on setting goals in business assurance, leading a team, and making critical decisions.
  • LTE Rollout: Make it a Smashing Success with Risk Assessment, Controls, and Marketing Offer Analytics by Gadi Solotorevsky — LTE brings splendid new capabilities to mobile users.  But like 2G and 3G deployments before, operators can only make money if they successfuly plan, coordinate, deploy fast, and pay attention to pricing plans and the customer experience.  This article lays out a 3-phase tactical guide on  how revenue analytics professionals can add value in LTE service risk assessment, controls, and marketing offer analytics.
  • RA Prevention: How to Manage Revenue Risks and Communicate RA’s Value to Senior Execs by Shaul Moav — The era of revenue assurance prevention and risk assessment is here.  Several of the mature operators of the world have developed their own methodologies and tools.  Using firefighting and fire prevention as a metaphor, the article details a new commercial software approach explaining the goals, method of risk evaluation, and senior executive dashboards developed for the process.
  • Precision Clockworks: How Revenue Assurance Synchronizes with the Business at Swisscom interview with Marco Pollinger — An expert revenue assurance department is one whose work dovetails well with the lines of businesses it supports.  In this interview you’ll learn how Swisscom manages its revenue assurance function for maximum effect.  The article discusses: the operator’s innovative RA organization, the screening and RA approval of new services, its pre-production bill audits, and its coordination with corporate risk management.
  • Versatile, Portable & Corrections-Savvy: Quest for the Swiss Army Knife of Revenue Assurance Software by Mark Yelland — Revenue assurance maturity models are not cast in stone.  Since  best practices will change over time, it’s healthy to explore moving maturity models forward.  For example, great gains have been made in leakage detection, but RA corrections has been harder to master.  The author dreams about seven functions that should ideally come together in a single all-purpose revenue assurance software tool.
  • Bringing Strategic Planning & Value Engineering to Revenue Assurance interview with Maged Fawzy — Engineering and architectural techniques have a role in revenue assurance.  This interview with a top Egyptian RA consultant explains how continuous risk assessment and long range — yet flexible — RA planning can sharpen a carrier’s RA program and lead to better use of revenue assurance software and integration services.
  • Forensic Fossils: Is Your Revenue Assurance Shop Fit for Display at a Natural History Museum? interview with Jim Marsh — Without the continuous guiding light of seasoned revenue assurance leaders, even the best teams of RA professionals, technology, and business processes can fossilize and lose their vitality.
  • Revenue Assurance: The Magical Market Cap Multiplier by Van Howard & Curtis Mills — Many operators today consider revenue assurance yesterday’s opportunity.  But this article shows why significant revenue and cost leakage can still go undetected, even in companies with dedicated RA departments.  Also discussed are the benefits of a broader or more “forensic” approach to revenue assurance, an approach that boosts the bottom line regardless of the automated tools already in place.
  • From Risk to Robust: Turning the Big Picture Into a Real Agenda for Change in Telecoms by Eric Priezkalns — Inspired by a Financial Times article written by Nassim Taleb, author of “The Black Swan”, here is an insightful and entertaining primer on telecom risk management.  The article takes ten risk management lessons from Taleb and applies them specifically to the communications industry.  You’ll learn about the value of small scale trials, organization accountability, cures for a blame culture, incentives that work, the power of simplicity, and more.
  • Synthesizing the Telecom Business Assurance Practice With the Analytics World by Dan Baker — Business assurance is a wrapper term that allows you to draw a circle around various telecom assurance, control, and optimization activities.  This article maps business assurance as a subset of telecom analytics, constrasting it with marketing analytics while a diagram shows where biz assurance fits in the larger B/OSS world.
  • CABS Revenue Assurance Disputes: May the Carrier With the Best Data Win by Cheryl Smith Rardin & David West — Revenue assurance innovation is far easier when partners cooperate to make it happen.  This articles shows how a U.S. operator, software vendor, and consultant teamed to develop a breakthrough in Carrier Access Billing (CABS) assurance.  Learn about: the dispute resolution data gap that needed to be filled, the partnering strategy, the implementation challenges, and payback results.
  • Revenue Assurance vs.  Business Assurance: Who’s the Rightful King of Controls Software? interview with Sergio Luis Silvestre — Business controls software, originally developed for RA, is finding application in other areas of the business such as internal audit, collections, security and risk management.  This article argues that “business assurance” is the best term to describe this broader set of  controls software that can find a home in numerous departments or functions of a CSP’s business.
  • PwC on the Business of Revenue Assurance Consulting & Mentoring interview with Tim Banks & Dan Stevens — Revenue assurance consulting firms offer a broad range of services to clients these days.  The article explains the practice of mentoring RA mangers and providing a CFO with visibility on the status of an operator’s business controls.  Perspective is also offered on the value of RA software and the opportunity to broaden the RA practice scope.
  • Robots for Hire: Verifying Accuracy In the Age of Complex Mobile Billing/Charging interview with Xavier Lesage — As real-time charging and complex lifestyle calling plans gain credence across the globe in wireless, billing quality issues will rise in importance.  This article discusses a unique managed services approach to invoice testing and roaming fraud protection that checks results against advertised or published source data for the utmost accuracy.
  • Ericsson: Revenue Assurance Consulting With an NGN Flavor interview with Thomas Steagall — Helping operators detect billing and provisioning problem is merely table stakes in the RA services business these days.  The article discuss why operators need to ramp up their RA function with service experience and group-wide financial health monitoring.  Advise is also offered on: key RA maturity questions, risk-and-reward contracts, and how to extract greater value from software investments.
  • Do-It-Yourself RA for Small Operators and MVNOs interview with Mark Yelland — Budget-minded small operators and MVNOs are no longer hamstrung in RA capability anymore.  This article offers high-leverage strategies for operators who cannot afford expensive RA software tools.  With  data access, brains, and a DIY philosophy, any small operator can map a  path to greater RA savings, maturity, and program growth.
  • Revenue Assurance Maturity: Report From the Arena interview with Eric Nelson — Revenue assurance maturity can‘t be easily computed.  How do you  compare the KPIs of Comcast billing with that of mobile money RA in Western Africa?  Even still, this article offers some universal RA wisdom from a straight-shooting veteran of carriers large and small.  Topics discussed include: dashboard or process, COTS vs. inhouse solutions, and tips on gaining internal support for the RA practice.