Email a colleague    

April 2015

A Big Data Starter Kit in the Cloud: How Smaller Operators Can Get Rolling with Advanced Analytics

A Big Data Starter Kit in the Cloud: How Smaller Operators Can Get Rolling with Advanced Analytics

Up to now, most of the telecom big data action has been happening at the large carriers.

TRI’s 2014 study of telecom analytics solutions estimated the market at $1,990 million with 70% of the market serving the large, giant carriers.

Big Data in the Cloud Research

Medium to small operators know “big data” is out there alright, but most have taken only modest steps to use advanced analytics.  Technical staffing skills have prevented them from taking advantage of big data advanced analytics.  Plus many are not quite sure what practical benefits big data brings them.

Well, good news: cloud services are emerging to enable smaller operators to get started and avoid the cost of hiring a team of data scientists.

A pioneer in this big data cloud service for telecoms is New Jersey-based Advanced Technologies & Services (ATS), a revenue assurance and analytics specialty firm celebrating two decades of business this year.  Ryan Guthrie, the company’s VP of Solutions Engineering now joins us to explain what ATS’s service is all about.

Dan Baker: Ryan, it’s great to hear that somebody is looking out for the smaller operators who are often left in the dust by new technology and innovation.  What’s the biggest need for these operators as they start using analytics in the cloud?

Ryan Guthrie: Dan, believe it or not, one of the key things people struggle with is storage.  There’s a requirement to keep five years worth of call detail records (CDRs), but that ends up being a bit of a challenge.  Folks end up storing it on external hard drives under their desks and they waste a lot of time managing that and it’s hard to find data when you’re looking for it.

You’re better off storing the data in the cloud where it’s searchable, sortable, and they can very quickly access it.

That’s the service we provide.  We use Amazon Web Services and some Google services like BigQuery and BigTable.  These work similar to MySQL, but they are much faster.  Now a power user can run SQL queries on the data, or we can create canned reports for people who are looking for the same information on a daily/weekly basis.

So our market for this cloud service stretches from the Tier 1 carriers to the hundreds of small to medium sized operators in North America.  The flexibility of the cloud allows our pricing model to scale for carriers of all sizes.  That means cost is no longer a barrier for smaller operators to get started with big data.

What got you interested in offering this kind of service?  What’s your company’s background in big data?

A few years ago, we experimented with Hadoop technology, a way of processing data using scalable, commodity hardware.  We created our own Hadoop cluster in our shop.  Trouble is, it takes a lot of technical knowledge to manage Hadoop and we are telecom data experts, not Hadoop experts.

So, in search of an easier to maintain Hadoop environment, we found Amazon Web Services (AWS) and they had exactly the kind of service we needed to allow us to scale up.  So we run Hadoop using their backend and it’s seamless as to how many servers they are running.  You simply load the data up there and run queries on it.

If you want the results back the next couple of minutes, you can request more servers, more processors assigned to it.  If it’s a job you kick off at midnight and get a report the next morning, you can assign less processors to it and end up paying a little bit less.  So, it is pretty cool because you more or less pay as you go for processing power.

The other beauty, of course, is you don‘t have to worry about buying more servers or figuring out what to do when another hard drive goes down.

ATS Architecture for Big Data in the Cloud

On top of the cloud storage capability, what’s the most popular Big Data application you deliver?

On top of the cloud storage, we offer a variety of modules that can be easily bolted on. [See chart above for typical services] The most popular module is our KPI Dashboard with alarming capabilities.  We provide a complete turnkey solution that monitors call volumes and is built on a statistical model to alarm when volumes spike/dip outside of what’s expected.

At any time you can look and see; at 2 o‘clock on Thursday, what volume of traffic do we expect to see?  And if the traffic breaks the upper or lower thresholds, an alarm is triggered.  So if there was a big spike, maybe it’s fraud.  If there’s a big drop, then it may be that the switch went down or some records didn’t come off the switch.

To populate the dashboard, we typically get a copy of the CDRs right off the switch and it’s loaded into the system automatically as the data is sent.  The switch cuts the data and sends one copy of those records to the company’s mediation system and a copy gets sent to us.

What other applications are available?

Well, another popular one is retrieval of the phone records for law enforcement.  So we supply what we call our subpoena management module that maintains and finds the data needed to support Federal, State, and local law enforcement efforts when you need to retrieve phone calls and the like.

Some of the medium carriers get daily subpoena requests.  The smaller operators might get a request once a week, and having a cloud solution really simplifies this process.

We also have a margin analysis module that’s useful in wholesale, primarily monitoring international voice traffic.  Companies negotiate different deals over time and for one reason or another those routes start losing money.  If you’re not on top of it, it can become a big revenue leakage point.

What about security in the cloud?  I’m sure that’s a question you get asked quite often.

You are absolutely right.  One of the most common questions we get about storing data in the cloud is — will my data be safe?  The answer is yes, the data is just as safe, if not safer, than how they currently store data today.  Cloud services such as AWS understand how important security is to its clients and they do a great job at providing it.  If you look at who is using AWS to store data today it includes big name companies like Netflix, Comcast, Nokia, and even federal agencies like the CIA, FBI, and NASA.  You can be sure that if the CIA trusts the security of the cloud, you can too.

Ryan, it sounds like a very useful and much needed service to get the smaller operators rolling with a big data solution that’s practical and affordable.

Copyright 2015 Black Swan Telecom Journal

 
Ryan Guthrie

Ryan Guthrie

Ryan Guthrie is VP of Solutions Engineering at Advanced Technologies and Services, Inc. (ATS).  Ryan started with ATS in 2005 and has been involved in all aspects of the business from sales through implementation.  Ryan graduated from Boston College with a BS in Marketing and Information Systems.   Contact Ryan via

Black Swan Solution Guides & Papers

cSwans of a Feather

  • A Big Data Starter Kit in the Cloud: How Smaller Operators Can Get Rolling with Advanced Analytics interview with Ryan Guthrie — Medium to small operators know “big data” is out there alright, but technical staffing and cost issues have held them back from implementing it.  This interview discusses the advantages of moving advanced analytics to the cloud where operators can get up and running faster and at lower cost.
  • Telecoms Failing Badly in CAPEX: The Desperate Need for Asset Management & Financial Visibility interview with Ashwin Chalapathy — A 2012 PwC report put the telecom industry on the operating table, opened the patient up, and discovered a malignant cancer: poor network CAPEX management, a problem that puts telecoms in grave financial risk.  In this interview, a supplier of network analytics solutions provides greater detail on the problem and lays out its prescription for deeper asset management, capacity planning and data integrity checks.
  • History Repeats: The Big Data Revolution in Telecom Service Assurance interview with Olav Tjelflaat — The lessons of telecom software history teach that new networks and unforeseen industry developments have an uncanny knack for disrupting business plans.  A service assurance incumbent reveals its strategy for becoming a leader in the emerging network analytics and assurance market.
  • From Alarms to Analytics: The Paradigm Shift in Service Assurance interview with Kelvin Hall — In a telecom world with millions of smart devices, the service assurance solutions of yesteryear are not getting the job done.  So alarm-heavy assurance is now shifting to big data solutions that deliver visual, multi-layered, and fine-grained views of network issues.  A data architect who works at large carriers provides an inside view of the key service provider problems driving this analytics shift.
  • The Shrink-Wrapped Search Engine: Why It’s Becoming a Vital Tool in Telecom Analytics interview with Tapan Bhatt — Google invented low cost, big data computing with its distributed search engine that lives in mammoth data centers populated with thousands of commodity CPUs and disks.  Now search engine technology is available as “shrink wrapped” enterprise software.  This article explains how this new technology is solving telecom analytics problems large and small.
  • Harvesting Big Data Riches in Retailer Partnering, Actionable CEM & Network Optimization interview with Oded Ringer — In the analytics market there’s plenty of room for small solution firms to add value through a turnkey service or cloud/licensed solution.  But what about large services firms: where do they play?  In this article you’ll learn how a global services giant leverages data of different types to help telcos: monetize retail partnerships, optimize networks, and make CEM actionable.
  • Raising a Telco’s Value in the Digital Ecosystem: One Use Case at a Time interview with Jonathon Gordon — The speed of telecom innovation is forcing software vendors to radically adapt and transform their business models.  This article shows how a deep packet inspection company has  expanded into revenue generation, particularly  for mobile operators.  It offers a broad palette of value-adding use cases from video caching and parental controls to application-charging and DDoS security protection.
  • Radio Access Network Data: Why It’s Become An Immensely Useful Analytics Source interview with Neil Coleman — It’s hard to overstate the importance of Radio Access Network (RAN) analytics to a mobile operator’s business these days.  This article explains why the RAN data, which lives in the air interface between the base station and the handset --  can be used for a business benefit in network optimization and customer experience.
  • Analytics Biology: The Power of Evolving to New Data Sources and Intelligence Gathering Methods interview with Paul Morrissey — Data warehouses create great value, yet it’s now time to let loose non-traditional big data platforms that create value in countless pockets of operational efficiency that have yet to be fully explored.  This article explains why telecoms must expand their analytics horizons and bring on all sorts of new data sources and novel intelligence gathering techniques.
  • Connecting B/OSS Silos and Linking Revenue Analytics with the Customer Experience by Anssi Tauriainen — Customer experience analytics is a complex task that flexes B/OSS data to link the customer’s network experience and actions to improve it and drive greater revenue.  In this article, you’ll gain an understanding of how anayltics data needs to be managed across various customer life cycle stages and why it’s tailored for six specific user groups at the operator.
  • Meeting the OTT Video Challenge: Real-Time, Fine-Grain Bandwidth Monitoring for Cable Operators interview with Mark Trudeau — Cable operators in North America are being overwhelmed by the surge in video and audio traffic.  In this article you’ll learn how Multi Service Operators (MSOs) are now monitoring their traffic to make critical decisions to protect QoS service and monetize bandwidth.  Also featured is expert perspective on trends in: network policy; bandwidth caps; and  customer care issues.
  • LTE Analytics:  Learning New Rules in Real-Time Network Intelligence, Roaming and Customer Assurance interview with Martin Guilfoyle — LTE is telecom’s latest technology darling, and this article goes beyond the network jargon, to explain the momentous changes LTE brings.  The interview delves into the marriage of IMS, high QoS service delivery via IPX, real-time intelligence and roaming services, plus the new customer assurance hooks that LTE enables.

Related Articles

  • Tokopedia, Indonesia’s E-Commerce King, Partners with 11 Million Merchants; Adopts Multi-Cloud to Drive Innovation interview with Warren Aw & Ryan de Melo — Indonesia’s Tokopedia, founded in 2009, has grown to become one of world’s leading e-commerce players.  Read about its success, technology direction, and multi-cloud connectivity adoption.
  • Bridge Alliance: Knocking Down Regional & Mobile Connectivity Barriers so Connected Car Markets Get Rolling in Asia interview with Kwee Kchwee — The CEO of an Asian consortium of mobile operators explains how they  help simplify and harmonize their members‘ operations in support of multi-national corporations.  This integration is enabling two huge industries to come together in Asia: auto manufacturing and telco.
  • Epsilon’s Infiny NaaS Platform Brings Global Connection, Agility & Fast Provision for IoT, Clouds & Enterprises in Southeast Asia, China & Beyond interview with Warren Aw — Network as a Service, powered by Software Defined Networks, are a faster, more agile, and more partner-friendly way of making data global connections.  A leading NaaS provider explains the benefits for cloud apps, enterprise IT, and IoT.
  • PCCW Global: On Leveraging Global IoT Connectivity to Create Mission Critical Use Cases for Enterprises interview with Craig Price — A leading wholesale executive explains the business challenges of the current global IoT scene as it spans many spheres: technical, political, marketing, and enterprise customer value creation.
  • Senet’s Cloud & Shared Gateways Drive LoRaWAN IoT Adoption for Enterprise Businesses, Smart Cities & Telecoms interview with Bruce Chatterley — An IoT netowork pioneer explains how LoRaWAN tech fits in the larger IoT ecosystem.  He gives use case examples, describes deployment restraints/costs, and shows how partnering, gateway sharing, and flexible deployment options are stimulating growth.
  • ARM Data Center Software’s Cloud-Based Network Inventory Links Network, Operations, Billing, Sales & CRM to One Database interview with Joe McDermott & Frank McDermott — A firm offering a cloud-based network inventory system explains the virtues of: a single underlying database, flexible conversions, task-checking workflow, new software business models, views that identify stranded assets, and connecting to Microsoft’s cloud platform.
  • Pure Play NFV: Lessons Learned from Masergy’s Virtual Deployment for a Global Enterprise interview with Prayson Pate — NFV is just getting off the ground, but one cloud provider to enterprises making a stir in virtual technology waters is Masergy.  Here are lessons learned from Masergy’s recent global deployment using a NFV pure play software approach.
  • The Digital Enabler: A Charging, Self-Care & Marketing Platform at the Core of the Mobile Business interview with Jennifer Kyriakakis — The digital enabler is a central platform that ties together charging, self-care, and marketing.  The article explains why leading operators consider digital enablers pivotal to their digital strategies.
  • Delivering Service Assurance Excellence at a Reduced Operating Cost interview with Gregg Hara — The great diversity and complexity of today’s networks make service assurance a big challenge.  But advances in off-the-shelf software now permit the configuring and visualizing of services across multiple technologies on a modest operating budget.
  • Are Cloud-Based Call Centers the Next Hot Product for the SMB Market? interview with Doron Dovrat — Quality customer service can improve a company’s corporate identity and drive business growth.  But many SMBs are priced out of acquiring modern call center technology.  This article explains the benefits of affordable and flexible cloud-based call centers.
  • Flexing the OSS & Network to Support the Digital Ecosystem interview with Ken Dilbeck — The need for telecoms to support a broader digital ecosystem requires an enormous change to OSS infrastructures and the way networks are being managed.  This interview sheds light on these challenges.
  • Crossing the Rubicon: Is it Time for Tier Ones to Move to a Real-Time Analytics BSS? interview with Andy Tiller — Will tier one operators continue to maintain their quilt works of legacy and adjunct platforms — or will they radically transform their BSS architecture into a new  system designed to address the new telecom era?  An advocate for radical transformation discusses: real-time analytics, billing for enterprises, partnering mashups, and on-going transformation work at Telenor.
  • Paradigm Shift in OSS Software: Network Topology Views via Enterprise-Search interview with Benedict Enweani — Enterprise-search is a wildly successful technology on the web, yet its influence has not yet rippled to the IT main stream.  But now a large Middle Eastern operator has deployed a major service assurance application using enterprise-search.  The interview discusses this multi-dimensional topology solution and compares it to traditional network inventory.
  • The Multi-Vendor MPLS: Enabling Tier 2 and 3 Telecoms to Offer World-Class Networks to SMBs interview with Prabhu Ramachandran — MPLS is a networking technology that has caught fire in the last decade.  Yet the complexity of MPLS has relegated to being mostly a large carrier solution.  Now a developer of a multi-vendor MPLS solutions explains why the next wave of MPLS adoption will come from tier 2/3 carriers supporting SMB customers.
  • Enabling Telecoms & Utilities to Adapt to the Winds of Business Change interview with Kirill Rechter — Billing is in the midst of momentous change.  Its value is no longer just around delivering multi-play services or sophisticated rating.  In this article you’ll learn how a billing/CRM supplier has adapted to the times by offering deeper value around the larger business issues of its telecom and utility clients.
  • Driving Customer Care Results & Cost Savings from Big Data Facts interview with Brian Jurutka — Mobile broadband and today’s dizzying array of app and network technology present a big challenge to customer care.  In fact, care agents have a hard time staying one step ahead of customers who call to report problems.  But network analytics comes to the rescue with advanced mobile handset troubleshooting and an ability to put greater intelligence at the fingertips of highly trained reps.
  • Hadoop and M2M Meet Device and Network Management Systems interview with Eric Wegner — Telecom big-data in networks is more than customer experience managment: it’s also about M2M plus network and element management systems.  This interview discusses the explosion in machine-to-machine devices, the virtues and drawbacks of Hadoop, and the network impact of shrink-wrapped search.
  • The Data Center & Cloud Infrastructure Boom: Is Your Sales/Engineering Team Equipped to Win? by Dan Baker — The build-out of enterprise clouds and data centers is a golden opportunity for systems integrators, carriers, and cloud providers.  But the firms who win this business will have sales and engineering teams who can drive an effective and streamlined requirements-to-design-to-order process.  This white paper points to a solution — a collaborative solution designs system — and explains 8 key capabilities of an ideal platform.
  • Big Data: Is it Ready for Prime Time in Customer Experience Management? interview with Thomas Sutter — Customer experience management is one of the most challenging of OSS domains and some suppliers are touting “big data” solutions as the silver bullet for CEM upgrades and consolidation.  This interview challenges the readiness of big data soluions to tackle OSS issues and deliver the cost savings.  The article also provides advice on managing technology risks, software vendor partnering, and the strategies of different OSS suppliers.
  • Calculated Risk: The Race to Deliver the Next Generation of LTE Service Management interview with Edoardo Rizzi — LTE and the emerging heterogeneous networks are likely to shake up the service management and customer experience management worlds.  Learn about the many new network management challenges LTE presents, and how a small OSS software firm aims to beat the big established players to market with a bold new technology and strategy.
  • Decom Dilemma: Why Tearing Down Networks is Often Harder than Deploying Them interview with Dan Hays — For every new 4G LTE and IP-based infrastructure deployed, there typically a legacy network that’s been rendered obsolete and needs to be decommissioned.  This article takes you through the many complexities of network decom, such as facilities planning, site lease terminations, green-safe equipment disposal, and tax relief programs.
  • Migration Success or Migraine Headache: Why Upfront Planning is Key to Network Decom interview with Ron Angner — Shutting down old networks and migrating customers to new ones is among the most challenging activities a network operators does today.  This article provides advice on the many network issues surrounding migration and decommissioning.  Topics discussed include inventory reconciliation, LEC/CLEC coordination, and protection of customers in the midst of projects that require great program management skills.
  • Navigating the Telecom Solutions Wilderness: Advice from Some Veteran Mountaineers interview with Al Brisard — Telecom solutions vendors struggle mightily to position their solutions and figure out what to offer next in a market where there’s considerable product and service crossover.  In this article, a veteran order management specialist firm lays out its strategy for mixing deep-bench functional expertise with process consulting, analytics, and custom API development.
  • Will Telecoms Sink Under the Weight of their Bloated and Out-of-Control Product Stacks? interview with Simon Muderack — Telecoms pay daily for their lack of product integration as they constantly reinvent product wheels, lose customer intelligence, and waste time/money.  This article makes the case of an enterprise product catalog.  Drawing on central catalog cases at a few Tier 1 operators, the article explains the benefits: reducing billing and provisioning costs, promoting product reuse, and smoothing operations.
  • Virtual Operator Life: Enabling Multi-Level Resellers Through an Active Product Catalog interview with Rob Hill — The value of product distribution via virtual operators is immense.  They enable a carrier to sell to markets it cannot profitably serve directly.  Yet the need for greater reseller flexibility in the bundling and pricing of increasingly complex IP and cloud services is now a major channel barrier.  This article explains what’s behind an innovative product catalog solution that doubles as a service creation environment for resellers in multiple tiers.
  • Telecom Blocking & Tackling: Executing the Fundamentals of the Order-to-Bill Process interview with Ron Angner — Just as football teams need to be good at the basics of blocking and tackling, telecoms need to excel at their own fundamental skillset: the order-to-cash process.  In this article, a leading consulting firm explains its methodology for taking operators on the path towards order-to-cash excellence.  Issues discussed include: provisioning intervals; standardization and simplicity; the transition from legacy to improved process; and the major role that industry metrics play.
  • Wireline Act IV, Scene II: Packaging Network & SaaS Services Together to Serve SMBs by John Frame — As revenue from telephony services has steadily declined, fixed network operators have scrambled to support VoIP, enhanced IP services, and now cloud applications.  This shift has also brought challenges to the provisioning software vendors who support the operators.  In this interview, a leading supplier explains how it’s transforming from plain ol‘ OSS software provider to packager of on-net and SaaS solutions from an array of third party cloud providers.
  • Telecom Merger Juggling Act: How to Convert the Back Office and Keep Customers and Investors Happy at the Same Time interview with Curtis Mills — Billing and OSS conversions as the result of a merger are a risky activity as evidenced by famous cases at Fairpoint and Hawaiian Telcom.  This article offers advice on how to head off problems by monitoring key operations checkpoints, asking the right questions, and leading with a proven conversion methodology.
  • Is Order Management a Provisioning System or Your Best Salesperson? by John Konczal — Order management as a differentiator is a very new concept to many CSP people, but it’s become a very real sales booster in many industries.  Using electronics retailer BestBuy as an example, the article points to several innovations that can — and are — being applied by CSPs today.  The article concludes with 8 key questions an operator should ask to measure advanced order management progress.
  • NEC Takes the Telecom Cloud from PowerPoint to Live Customers interview with Shinya Kukita — In the cloud computing world, it’s a long road from technology success to telecom busness opportunity.  But this story about how NEC and Telefonica are partnering to offer cloud services to small and medium enterprises shows the experience of early cloud adoption.  Issues discussed in the article include: customer types, cloud application varieties, geographic region acceptance, and selling challenges.
  • Billing As Enabler for the Next Killer Business Model interview with Scott Swartz — Facebook, cloud services, and Google Ads are examples of innovative business models that demand unique or non-standard billing techniques.  The article shows how flexible, change-on-the-fly, and metadata-driven billing architectures are enabling CSPs to offer truly ground breaking services.
  • Real-Time Provisioning of SIM Cards: A Boon to GSM Operators interview with Simo Isomaki — Software-controlled SIM card configuration is revolutionizing the activation of GSM phones.  The article explains how dynamic SIM management decouples the selection of numbers/services and delivers new opportunities to market during the customer acquisition and intial provisoining phase.
  • A Cynic Converted: IN/Prepaid Platforms Are Now Pretty Cool interview with Grant Lenahan — Service delivery platforms born in the IN era are often painted as inflexible and expensive to maintain.  Learn how modern SDPs with protocol mediation, high availability, and flexible Service Creation Environments are delivering value for operators such as Brazil’s Oi.
  • Achieving Revenue Maximization in the Telecom Contact Center interview with Robert Lamb — Optimizing the contact center offers one of the greatest returns on investment for a CSP.  The director of AT&T’s contact center services business explains how telecoms can strike an “artful balance” between contact center investment and cost savings.  The discussion draws from AT&T’s consulting with world class customers like Ford, Dell, Discover Financial, DISH Network, and General Motors.
  • Mobile Broadband: The Customer Service Assurance Challenge interview with Michele Campriani — iPhone and Android traffic is surging but operators struggle with network congestion and dropping ARPUs.  The answer?  Direct  resources and service quality measures to ensure VIPs are indeed getting the quality they expect.  Using real-life examples that cut to the chase of technical complexities, this article explains the chief causes of service quality degradation and describes efficient ways to deal with the problem.
  • Telco-in-a-Box: Are Telecoms Back in the B/OSS Business? interview with Jim Dunlap — Most telecoms have long since folded their merchant B/OSS software/services businesses.  But now Cycle30, a subsidiary of Alaskan operator GCI, is offering a order-to-cash managed service for other operators and utilities.  The article discusses the company’s unique business model and contrasts it with billing service bureau and licensed software approaches.
  • Bricks, Mortar & Well-Trained Reps Make a Comeback in Customer Management interview with Scott Kohlman — Greater industry competition, service complexity, and employee turnover have raised the bar in the customer support.  Indeed, complex services are putting an emphasis on quality care interactions in the store, on the web, and through the call center.  In this article you’ll learn about innovations in CRM, multi-tabbed agent portals,  call center agent training, customer treatment philosophies, and the impact of  self-service.
  • 21st Century Order Management: The Cross-Channel Sales Conversation by John Konczal — Selling a mobile service is generally not a one-and-done transaction.  It often involves several interactions — across the web, call center, store, and even kiosks.  This article explains the power of a “cross-channel hub” which sits above all sales channels, interacts with them all, and allows a CSP to keep the sales conversation moving forward seamlessly.
  • Building a B/OSS Business Through Common Sense Customer Service by David West — Delivering customer service excellence doesn‘t require mastering some secret technique.  The premise of this article is that plain dealing with customers and employees is all that’s needed for a winning formula.  The argument is spelling out in a simple 4 step methodology along with some practical examples.