Email a colleague    

September 2014

Paradigm Shift in OSS Software: Network Topology Views via Enterprise-Search

Paradigm Shift in OSS Software: Network Topology Views via Enterprise-Search

“Throw a stone into the stream, and the circles that propagate themselves are a beautiful example of all influence.  Likewise, man is placed in the center of beings and a ray of relation passes from every other being to him.”   — Ralph Waldo Emerson, Nature 1836


Enterprise-search is the most frugal of big data’s creatures: it lives off the land and doesn‘t need to be fed.

It acquires its data from any number of sources: machine logs, tables, emails, twitter feeds, Excel, call center conversations — you name it.  It even bypasses APIs and the data trolls who force you to pay a king’s ransom to cross their bridges.

Search also discovers the relations in data without the heavy lifting and expense of architecting data schemas or constantly modifying the business logic inside the labyrinth of enterprise data gardens.

Linked-In is a perfect example of search’s power.  With a market cap of $28 billion, Linked-In makes its money by searching and analyzing the ever-changing relations (or links) between people and the subject matter that unites them.

And yet, as successful as enterprise-search has become on the web, its influence has not yet rippled into the IT main stream.  It’s still considered “experimental” technology: good for report generating and ad hoc analytics, but not for serious operational applications.

So this is precisely why Ontology’s recent announcement of the first successful “shrink-wrapped” enterprise-search OSS application for a Tier 1 operator is so significant.  It’s a clear sign that enterprise-search is now on the maturity fast track — and it can be at the core of genuine software, not just consulting services.

Here to discuss his company’s breakthrough and delve into a discussion on network inventory and the future of enterprise-search and topology visualization is Benedict Enweani, Ontology’s CEO.

Dan Baker: Benedict, first of all, congratulations.  The delivery of a large-scale enterprise-search application is exciting news for all of us big data enthusiasts.  Please tell us about it.

Benedict Enweani: Thanks, Dan.  About a year ago, we deployed the solution to a big Middle Eastern operator who has about 40 million subs.  We were initially hired to come in and create reports for them.  And they saw how we could generate across multi-technology and multi-vendor vendor views — see a full topology from the antenna and RAN all the way back through the various layers of microwave, backhaul, Ethernet and T1/E1 connections riding under that.

So they came back to us and said: “Why should we pay you all this money to get just reports from you.  Can‘t we just pay you to provide an operational system we can use for network operations and troubleshooting?”

Needless to say we were thrilled to get that offer.  So that was the impetus to package up our search technology and sell it as a solution.  In turn, it’s generating interest from other telecoms and managed service providers.

What did the operator consider most valuable in the solution?

In a word, the ability to see “multi-dimensional topologies”: multi-vendor, multi-technology, and multi-layer.  And when you get that view, it becomes much easier to do network change management, trouble shooting, and service impact analysis.

This Middle Eastern operator has 8,000 different configurations it needed to bring in.  And they didn‘t want to go into five different systems to find their answers.  So we delivered a tool intelligent enough to spot the context talked about in different configuration sources and build a structure that explains them and fits them in a data model.

Another value we brought in was an ability to resolve tough data integrity problems that require comparing multiple sources.

In the past we would simply extract our data into reports and special interfaces, but we realized that everyone has a need to visualize network topology so that’s what are our primary output is today.  And while topology sounds like a very network specific concern, it’s actually a data integration problem.  The information you need to understand a topology you can‘t actually discover using SNMP, in fact, getting an understanding of only a single system is not enough.  You need to know how things cross many different systems.  You need to study how everything interacts with everything else.

So the data sources we pulled in were many, for instance: GE Small World, Siebel CRM, Twitter feeds, Mobile network, Cisco configurations of fixed network, and addressable parts of the network from IBM.  Search semantically recognizes nodes, switches, cards, interfaces, and connections, and in this case across 10 different network domains.

In the topology you see a link between the radio antenna and base station controller.  You notice the base station serves six different cells, and there’s a microwave backhaul for the first couple of miles and that terminates on an optical network and runs into the BSC.  So all that detail is there.  And if you want to drill down and see how a particular switch is set up and what cards are in the chassis, you can do that.

Now a program like this is usually a candidate for a network inventory solution.  But that’s a very difficult road to take.  I have personally been involved in several network inventory projects, and often the network inventory system never gets fully functional.  What we accomplished for the Middle East operator took us 6 months.  But if it was a network inventory project, it would have taken three years.

So we figure, our technology is lower cost and a better choice than a network inventory solution in many cases.

“Better than network inventory” is a bold statement, Benedict, and it goes against the conventional wisdom of the past 15 years which says that network inventory is the Holy Grail of what you need to do in OSS.

Yes, I realize it’s a bold statement, so let me step back and explain my view of the OSS world -- its evolution and issues from an inventory point of view.

The first wave of inventory solutions was born in the TDM world where you simply documented facilities.  Then as IP emerged and greatly complicated things, folks like Cramer Systems and Granite came along with the idea that network inventory should be elevated to a kind of golden database and business process engine for provisioning the network.

However, history has shown there are big problems in keeping network inventory up to date.  And the complexity of accounting for many different types of services and equipment requires you to constantly create new SQL logic.  Then there’s the huge cost of manually or semi-automatically importing the data into inventory.  We’ve seen cases where it took 8 years to actually stand an inventory system up.  Plus, once you’re there, you need to ensure the inventory system is the exclusive way you the provision, else you can quickly lose the bubble.

I figure the third inventory wave was Intelliden’s idea of reconciling against the actual network itself instead of trying to meticulously replicate it in an off-line database.  Intelliden’s concept was an important breakthrough except that you could only see active elements in the network: it gave you no visibility over all the passive elements out there.  Plus there are issues keeping up with the changes to all the vendors‘ network equipment.

Network Data Alignment

So how does Ontology improve on the network inventory state-of-the-art?

Well we begin by recognizing that while getting detailed network views is key, going the extra step of automating provisioning may not be worth the effort, especially in large networks.

I worked for a company called Orchestream in the 90s and we built the IP network activation software. [Orchestream was bought by MetaSolv, then later acquired by Oracle.]

Now the task we set out for ourselves at Orchestream was a tough one because we needed to tightly integrate with all the NMSs and EMSs.  We found this effort can succeed in constrained domains, for instance a network with only Cisco and Juniper network gear.  But if you strayed off and bought something from another provider, then the cost of integrating to that equipment was just huge.

So this is why Ontology limits itself to getting the network view for service assurance and network planning, but we stop short of provisioning or writing anything to the network.  In fact, our Middle Eastern operator was very satisfied that we went halfway.  They use our solution for service assurance and generating the topological views, then make manual or semi-automated changes on their own using various NMSs and EMSs.

One of the biggest drawbacks of inventory solutions is that every time you run into a new source of data or change the infrastructure, you need to insert new logic in the inventory software’s model which is an expensive and error-prone task.

But with enterprise-search, if you add a new data source, you essentially create a new class and tell the system how objects of that class link to other things.  The search routine then automatically discovers and maps those objects into the topology.

For instance, to add a new type of antenna into your network you simply define how that antenna interacts with, say, your microwave links.  Then the next topology search would discover all those antenna nodes in your system.  Network nodes are constantly announcing how they relate to things they connect with.  For instance, there may be IP addresses in the antenna’s configuration, or maybe time slots associated with a SONET network within that equipment, so the system deduces that there’s a connection to an optical network.

Each data source updates on its own clock.  Faults are coming in all the time.  Network discovery engines are usually programmed to discover the network 1 to 4 times a day.  CRMs can update each transaction, however that normally creates a heavy load, so operators tend to batch that up once an hour.  Physical inventory updates don‘t usually happen faster than once a week.  And Twitter tweats are loading constantly.  Then using a technique we call “finger-printing” we detect abnormal behavior so if we normally receive an update from a source every hour, and we haven’t seen one in the last 3 hours, that raises an alert.

In what network areas you are applying your enterprise search technology to?

Actually there are four or five very specific areas that customers find most useful for this technology.  Let me walk you through them:

  • Change management -- You are going to make some network changes and you need to pick the most opportune time to make the change and know how the change will impact your SLAs with customers.  And will changes that others are making at the same time possibly create a catastrophe?
  • Network troubleshooting — If fifty 3G cells go down at one time, you ask yourself: what’s common for those 50 cells?  And the system comes back and tells you they all use the same transmission node.  So such root cause analysis can be done across a multi-layer system view.  Theoretically you can do this in an inventory system, but in the seven years that my partner Leo and I have been working on this project, we have actually never seen an inventory system that spans more than 2 technologies.
  • Service Impact Analysis — If a key DWM switch goes down, which antennas are affected?  Which service centers are affected?  And which of the actual enterprise customers are affected?
  • Enterprise Customer Topology — This is a very interesting one and a very large operator in the UK needed this capability.  Take a financial firm like Llyod’s Bank.  They have thousands of network delivery points and it’s very useful to search the carrier’s infrastructure and pull together a topology showing where service is running poorly and where there are issues.  For each enterprise client, we ended up saving the carrier many days of work trying to pull this same information from multiple sources.

So these four use cases are where the lion’s share of our ROI comes from.  So you are saving people time digging up vital information or decreasing their SLA violations.  Or you are decreasing the minutes of downtime, or maybe increasing service resilience.

And the beauty here is that an operator can achieve all these things without the expense of building out a new inventory system.

Benedict, thanks for this highly interesting briefing.  Good luck as you roll this solution out — and please keep us informed of your progress.

Copyright 2014 Black Swan Telecom Journal

 
Benedict Enweani

Benedict Enweani

As CEO and co-founder of Ontology Systems, Benedict aims to make his company the leading innovator of semantic solutions in the data center and networking marketplace.  Before founding Ontology in 2005, Benedict was CTO at Corvil Networks and CTO at Orchestream (now Oracle).   Contact Benedict via

Black Swan Solution Guides & Papers

cSwans of a Feather

  • Delivering Service Assurance Excellence at a Reduced Operating Cost interview with Gregg Hara — The great diversity and complexity of today’s networks make service assurance a big challenge.  But advances in off-the-shelf software now permit the configuring and visualizing of services across multiple technologies on a modest operating budget.
  • Paradigm Shift in OSS Software: Network Topology Views via Enterprise-Search interview with Benedict Enweani — Enterprise-search is a wildly successful technology on the web, yet its influence has not yet rippled to the IT main stream.  But now a large Middle Eastern operator has deployed a major service assurance application using enterprise-search.  The interview discusses this multi-dimensional topology solution and compares it to traditional network inventory.
  • The Multi-Vendor MPLS: Enabling Tier 2 and 3 Telecoms to Offer World-Class Networks to SMBs interview with Prabhu Ramachandran — MPLS is a networking technology that has caught fire in the last decade.  Yet the complexity of MPLS has relegated to being mostly a large carrier solution.  Now a developer of a multi-vendor MPLS solutions explains why the next wave of MPLS adoption will come from tier 2/3 carriers supporting SMB customers.
  • Big Data: Is it Ready for Prime Time in Customer Experience Management? interview with Thomas Sutter — Customer experience management is one of the most challenging of OSS domains and some suppliers are touting “big data” solutions as the silver bullet for CEM upgrades and consolidation.  This interview challenges the readiness of big data soluions to tackle OSS issues and deliver the cost savings.  The article also provides advice on managing technology risks, software vendor partnering, and the strategies of different OSS suppliers.
  • Calculated Risk: The Race to Deliver the Next Generation of LTE Service Management interview with Edoardo Rizzi — LTE and the emerging heterogeneous networks are likely to shake up the service management and customer experience management worlds.  Learn about the many new network management challenges LTE presents, and how a small OSS software firm aims to beat the big established players to market with a bold new technology and strategy.
  • Mobile Broadband: The Customer Service Assurance Challenge interview with Michele Campriani — iPhone and Android traffic is surging but operators struggle with network congestion and dropping ARPUs.  The answer?  Direct  resources and service quality measures to ensure VIPs are indeed getting the quality they expect.  Using real-life examples that cut to the chase of technical complexities, this article explains the chief causes of service quality degradation and describes efficient ways to deal with the problem.

Related Articles

  • Black Swan Guide: Araxxe’s Revenue Assurance Consulting, Testing, and High Definition Billing Analysis Service by Dan Baker — How Araxxe’s end-to-end revenue assurance complements switch-to-bill RA  through telescope RA (external and partner data) and microscope RA (high-definition analysis of complex services like bundling and digital services).
  • Subex’s IDcentral Monetizes Telco & Enterprise Data to Deliver Digital ID & Risk Metric Services for Financing, KYC & More interview with Shankar Roddam — A new digital intelligence service that monetizes the idle data of telecoms and enterprises while also earning a good return for the owner of the data.
  • Opportunities & Obstacles: Consultant Luke Taylor Muses on the State of the Telecom Risk Assurance Business interview with Luke Taylor — A rambling discussion on the state of the risk assurance business with Luke Taylor, independent consultant in telecom revenue/fraud assurance and solution requirements and marketing.
  • LATRO’s Tips for Launching a Successful Revenue & Fraud Assurance Program for Mobile Money Operations in Developing Countries interview with Don Reinhart — A company building mobile money RA/FM tools and  managed services gives a concise, but detailed tutorial on how the Mobile Money Ecosystem works.  Revenue assurance pros will get tips on  what to look for in analytics/assurance tools, controls, and professional services.
  • A WeDo Conference Talk: Consulting & Analytics: Improving your Business Today, Enhancing it Tomorrow interview with Carla Cardoso & Bernado Lucas & Thomas Steagall — Leading risk management consultants explain their mission and walk-through RA, subscription fraud, and collections cases.  They also explain how analytics and machine learning can supplement process optimization.
  • PrologMobile’s Simple and Brilliant Plan to Save US MNOs Billions a Year in Recovered Phones & Retained Customers interview with Seth Heine — An expert in the mobile phone reverse supply chain explains how MNOs — via a neutral third party information exchange — can recover their original phones on the used market and save huge sums in multi-year customer retention.
  • WeDo Explores the IoT Ecosystem in Search of Tomorrow’s Pivotal Fraud & Business Assurance Solutions interview with Carlos Marques — A veteran product manager scans the IoT terrain, discusses key fraud and assurance challenges, and explains the preparatory steps WeDo is taking to become a key player in this emerging market.
  • New Report: Telecom Fraud & Business Assurance Solutions, Services & Strategies by Dan Baker & Luke Taylor & Colin Yates — TRI publishes a new market research report, Telecom Fraud & Business Assurance Solutions, Services & Strategies.  Free executive summary available.
  • Subex Juggles a Wide Variety of Business Assurance and Big Data Analytics Use Cases interview with Rohit Maheshwari — A expert in business assurance solutions explains top use cases such as: IoT security, big data analytics/AI, network asset optimization, multi-player gaming assurance, onboarding mobile subs, and AI customer analytics.
  • MTN Agility: Mastering Exponential Technologies in Revenue/Fraud Assurance and Beyond interview with Danie Maritz & Tony Sani & Luke Taylor — An in-depth look at RAFM operations and innovation at the MTN Group.  Topics discussed include RA/fraud control challenges, strategies, and MTN’s journey to exploit exponential tech (AI, robotics, and ML) in its RAFM program and support of internal non-telco businesses.
  • From Byzantine Software Contracts to Simple & Flexible RA Managed Services interview with Philippe Orsini — Is the way B2B/enterprise software is sold and delivered today progressive — or is it Byzantine in the age of cloud?  An expert lays out the case for managed services in RA and billing verification.
  • Premiere Experts Set to Speak at Summer RAG Conference in London, July 7th and 8th by Dan Baker — The Risk and Assurance Group (RAG) has announced that its 2016 summer conference will expand into a two-day event and feature many premiere experts. 
  • WeDo Hosts Revenue Assurance & Fraud Management Conference in Washington DC by Dan Baker — Black Swan is pleased to announce what looks to be a first class revenue assurance and fraud management conference being put on by WeDo Technologies, on October 1st and 2nd in beautiful Washington DC.
  • Test Call Generators: An Essential Test & Debugging Tool in Mobile Billing Assurance interview with Steffen Öftring — An “active” test call generator (TCG) can see problems that a “passive” revenue assurance system is blind to.  Here’s a discussion on the test call RA  process, over-the-air calls versus core call injection, and test call networks in global roaming RA.
  • The Revenue Assurance Game: How the Rules Change in the Era of IoT & Mobile Broadband interview with Rene Felber & Gadi Solotorevsky — Revenue assurance is perhaps the hardest of telecom functions to define because the term is used in so many different senses.  This discussion on the evolving role of revenue assurance was catalyzed by a survey of experts in the profession.
  • Day in the Life of a Revenue Assurance Analyst interview with Michael Lazarou — Revenue assurance is much more than a software category.  It’s individual analysts struggling to help their larger organizations get a handle on system errors and coordination problems.  In this interview, an analyst reveals the many challenges of getting the revenue assurance job done at a small GSM operator in Europe.
  • Revenue Assurance: History and New Beginnings in RA Maturity interview with Daniela Giacomantonio & Gadi Solotorevsky — The Roman Forum was the center of commercial life in ancient Rome.  Now, two millennia later, the Forum lives on in the exchange of ideas across countless professions and  media.  In this interview, two Revenue Assurance experts discuss both the new RA Maturity initiative of the TM Forum and the value of telco/solution vendor collaboration.
  • Migrating systems or launching LTE next year?  Don‘t forget transformation assurance & optimisation by Efrat Nissimov — System transformations and network migrations are major  revenue impacting events and they should raise a big red flag.  Why?  Because data integrity issues are bound to crop up as CSPs move vital data from a legacy system to something new.  It’s time for transformation assurance.
  • How can Cable/DSL Internet Providers Meet the Usage-Based Billing Mandate? interview with Ryan Guthrie — The popularity of YouTube, Netflix, and Hulu other video outlets has turned the tables on service profitability for cable/DSL service providers.  Many are moving to usage-based billing, but that largely unprepared for the revenue assurance aspects of this move.  This interview explains the technical challenge and points to solutions in billing, speed caps, and traffic revenue monitoring.
  • CABS Revenue Assurance: How Rural LECs can Recover $284 Million in Revenue Shortfalls interview with Kelly Cannon & Darrell Merschak — Independent rural LECs in the U.S. still rely on the AMA/EMI billing formats for CABS billing, even as that format has proven to be highly inaccurate as a source of inter-carrier records.  This interview includes an analysis and discussion of revenue recovery techniques ILECs can use by leveraging SS7 probes.  Also discussed are billing strategies, traffic dumping threats, and the possible fallout from the FCC’s bill-and-keep mandate.
  • Make Business Assurance Progress Every Day: How to Set Goals, Automate, and Energize Your Team interview with Kathleen Romano — Business assurance (BA) skills have wide applicability outside the revenue assurance and fraud mangement domains.  In this article, a telecom executive explains how she’s applying her BA skills in the Payments area.  In addition to discussing the key operational challenges in Payments, the interview also provides keen insights on setting goals in business assurance, leading a team, and making critical decisions.
  • LTE Rollout: Make it a Smashing Success with Risk Assessment, Controls, and Marketing Offer Analytics by Gadi Solotorevsky — LTE brings splendid new capabilities to mobile users.  But like 2G and 3G deployments before, operators can only make money if they successfuly plan, coordinate, deploy fast, and pay attention to pricing plans and the customer experience.  This article lays out a 3-phase tactical guide on  how revenue analytics professionals can add value in LTE service risk assessment, controls, and marketing offer analytics.
  • RA Prevention: How to Manage Revenue Risks and Communicate RA’s Value to Senior Execs by Shaul Moav — The era of revenue assurance prevention and risk assessment is here.  Several of the mature operators of the world have developed their own methodologies and tools.  Using firefighting and fire prevention as a metaphor, the article details a new commercial software approach explaining the goals, method of risk evaluation, and senior executive dashboards developed for the process.
  • Precision Clockworks: How Revenue Assurance Synchronizes with the Business at Swisscom interview with Marco Pollinger — An expert revenue assurance department is one whose work dovetails well with the lines of businesses it supports.  In this interview you’ll learn how Swisscom manages its revenue assurance function for maximum effect.  The article discusses: the operator’s innovative RA organization, the screening and RA approval of new services, its pre-production bill audits, and its coordination with corporate risk management.
  • Versatile, Portable & Corrections-Savvy: Quest for the Swiss Army Knife of Revenue Assurance Software by Mark Yelland — Revenue assurance maturity models are not cast in stone.  Since  best practices will change over time, it’s healthy to explore moving maturity models forward.  For example, great gains have been made in leakage detection, but RA corrections has been harder to master.  The author dreams about seven functions that should ideally come together in a single all-purpose revenue assurance software tool.
  • Bringing Strategic Planning & Value Engineering to Revenue Assurance interview with Maged Fawzy — Engineering and architectural techniques have a role in revenue assurance.  This interview with a top Egyptian RA consultant explains how continuous risk assessment and long range — yet flexible — RA planning can sharpen a carrier’s RA program and lead to better use of revenue assurance software and integration services.
  • Forensic Fossils: Is Your Revenue Assurance Shop Fit for Display at a Natural History Museum? interview with Jim Marsh — Without the continuous guiding light of seasoned revenue assurance leaders, even the best teams of RA professionals, technology, and business processes can fossilize and lose their vitality.
  • Revenue Assurance: The Magical Market Cap Multiplier by Van Howard & Curtis Mills — Many operators today consider revenue assurance yesterday’s opportunity.  But this article shows why significant revenue and cost leakage can still go undetected, even in companies with dedicated RA departments.  Also discussed are the benefits of a broader or more “forensic” approach to revenue assurance, an approach that boosts the bottom line regardless of the automated tools already in place.
  • From Risk to Robust: Turning the Big Picture Into a Real Agenda for Change in Telecoms by Eric Priezkalns — Inspired by a Financial Times article written by Nassim Taleb, author of “The Black Swan”, here is an insightful and entertaining primer on telecom risk management.  The article takes ten risk management lessons from Taleb and applies them specifically to the communications industry.  You’ll learn about the value of small scale trials, organization accountability, cures for a blame culture, incentives that work, the power of simplicity, and more.
  • Synthesizing the Telecom Business Assurance Practice With the Analytics World by Dan Baker — Business assurance is a wrapper term that allows you to draw a circle around various telecom assurance, control, and optimization activities.  This article maps business assurance as a subset of telecom analytics, constrasting it with marketing analytics while a diagram shows where biz assurance fits in the larger B/OSS world.
  • CABS Revenue Assurance Disputes: May the Carrier With the Best Data Win by Cheryl Smith Rardin & David West — Revenue assurance innovation is far easier when partners cooperate to make it happen.  This articles shows how a U.S. operator, software vendor, and consultant teamed to develop a breakthrough in Carrier Access Billing (CABS) assurance.  Learn about: the dispute resolution data gap that needed to be filled, the partnering strategy, the implementation challenges, and payback results.
  • Revenue Assurance vs.  Business Assurance: Who’s the Rightful King of Controls Software? interview with Sergio Luis Silvestre — Business controls software, originally developed for RA, is finding application in other areas of the business such as internal audit, collections, security and risk management.  This article argues that “business assurance” is the best term to describe this broader set of  controls software that can find a home in numerous departments or functions of a CSP’s business.
  • PwC on the Business of Revenue Assurance Consulting & Mentoring interview with Tim Banks & Dan Stevens — Revenue assurance consulting firms offer a broad range of services to clients these days.  The article explains the practice of mentoring RA mangers and providing a CFO with visibility on the status of an operator’s business controls.  Perspective is also offered on the value of RA software and the opportunity to broaden the RA practice scope.
  • Robots for Hire: Verifying Accuracy In the Age of Complex Mobile Billing/Charging interview with Xavier Lesage — As real-time charging and complex lifestyle calling plans gain credence across the globe in wireless, billing quality issues will rise in importance.  This article discusses a unique managed services approach to invoice testing and roaming fraud protection that checks results against advertised or published source data for the utmost accuracy.
  • Ericsson: Revenue Assurance Consulting With an NGN Flavor interview with Thomas Steagall — Helping operators detect billing and provisioning problem is merely table stakes in the RA services business these days.  The article discuss why operators need to ramp up their RA function with service experience and group-wide financial health monitoring.  Advise is also offered on: key RA maturity questions, risk-and-reward contracts, and how to extract greater value from software investments.
  • Do-It-Yourself RA for Small Operators and MVNOs interview with Mark Yelland — Budget-minded small operators and MVNOs are no longer hamstrung in RA capability anymore.  This article offers high-leverage strategies for operators who cannot afford expensive RA software tools.  With  data access, brains, and a DIY philosophy, any small operator can map a  path to greater RA savings, maturity, and program growth.
  • Revenue Assurance Maturity: Report From the Arena interview with Eric Nelson — Revenue assurance maturity can‘t be easily computed.  How do you  compare the KPIs of Comcast billing with that of mobile money RA in Western Africa?  Even still, this article offers some universal RA wisdom from a straight-shooting veteran of carriers large and small.  Topics discussed include: dashboard or process, COTS vs. inhouse solutions, and tips on gaining internal support for the RA practice.