© 2022 Black Swan Telecom Journal | • | protecting and growing a robust communications business | • a service of |
Email a colleague |
July 2011
If you’ve read my other columns, you know that most of them focus on a particular industry theme or trend.
This one’s going to be different. This column is about the history of one company, Beck Computers, founded and run by Gary Beck. Yes, my interview covers fraud management software, which is Beck’s specialty, but this story is also about how a tiny (15-employee) software firm can succeed in the B/OSS market — and even serve Tier 1 carriers.
I should have run into Beck Computers years ago, particularly since they have some marquis accounts and their customers process hundreds of millions of CDRs a day. But somehow Beck evaded my radar until fraud expert, Jim Marsh, told me about the firm a few weeks ago.
The company began in 1983 when Gary teamed up with his Dad to form the company. Dad was an antique dealer. It’s questionable whether or not Dad actually ever turned on a computer in his life, but fear not: Dad did know a thing or two about business, and Gary was prepared to supply the computer expertise anyway. The company’s original products were outside telecom — in areas like inventory control, accounting, and logistics. In fact, Beck Computers is still serving some of the same customers today, although 95 percent of its business these days is creating and selling fraud management software and systems.
I’d planned for this interview with Gary to be short and sweet. But I didn’t count on a couple of things. Number one: Gary’s quite the talker; and two: he has a lot of interesting things to say. So my original plan was out the window. Suffice it to say that Gary and I spent an entertaining hour on the phone together. And that resulted in this double-sized blog, which will inspire the entrepreneurial spirit that shines — brightly or dimly — in telecom software firms large or small.
Dan Baker: TRI has been in business since 1994 and I thought that was a long time, but Beck Computers beats me by eleven years. So what’s your longevity secret, Gary? |
Gary Beck: First of all, Dan, we are not a sales organization selling software. We are a service company that truly partners with our customers. And I think this philosophy is what’s allowed us to survive.
Can you qualify that statement a bit? Lots of companies tell me they are partners with their customers. Why do you believe Beck’s any better? |
I think the difference is we treat another company’s requirements, their system, and their money as if it was our own. I don’t advise my customers to buy a bigger computer they don’t need. And I’m not always trying to sell them more software. We’ve created some great relationships and I make money from them over the long term.
We like to drop a black box solution into a customer’s organization and in that way we minimize the need for IT to waste precious resources managing it. If the computer causes any sort of trouble, they call us and we handle it, period. And that’s not the typical support you get elsewhere. We support our users and computers as mission critical systems. One of our customers has a system that has been up for four years and never been down. Our software and our hardware is designed to be highly available.
It’s pretty amazing that a tiny software firm like yours is handling fraud management for a carrier processing almost 200 million CDRs a day. Tell us how that started. |
Well, we had the good fortune of having our system installed at a Tier 2 operator, which was then acquired by a larger company. And now we handle the volumes of both companies combined.
Over the years, I’ve purposely over-engineered my systems to handle larger volumes than the customer initially anticipated. But mergers aren’t the only reason to over-engineer. Sometimes you don’t get the data because of problems with the network or the switch. And so, if you’re down for a few hours, you’ve got to catch up, but how do you catch up if you can’t process more than you’re sending?
Then there’s another issue most people try not to think about — and that’s hot hours. It’s a known fact that often about 17 percent of daily telecom traffic will actually come through in one hour. So to handle that volume and allow you to catch up, that’s how we design our systems.
But if you go to other suppliers you’re liable to end up buying two to three times as much equipment than originally proposed — “You need three times more disk“ or “Oh no, you need a bigger processor.” Many salesmen play that game because it’s their job to get the initial order. Well, I don’t play that game. Instead, I leave in some leeway: put in more processing power than I promise. And customers come back and say, “Gee, we’re now processing 50 million CDRs.“ And it’s at that time that I go back and say, “Guys, it’s now time to think about adding capacity because I only promised you 25 million. I’ve given you a lot of extra room, but now you need to consider something bigger.”
I understand that for a time you were actually replaced at Williams. |
Yes, a couple years into our work at WilTel, they told us we would be replaced by a solution from SOTAS (now out of business). SOTAS had absolutely beautiful marketing materials. And they made a lot of big promises. In fact, they were still developing their system when they put it in, but they promised it would be complete in a few months.
Now our systems were running in parallel for some time there. SOTAS actually upgraded twice during the implementation. And the amount of hardware they kept bringing in was crazy. We’re talking row upon row of servers, pre-processors, pre-post-processors, and all these things to get data to move through the system. They put in eleven 6-foot bays of equipment, promising WilTel that it would handle 25 million CDRs a day.
Well, it took ten months for them to decide to turn down our system. But after they did, it was only 6 months later that I got a sheepish call from our friends at WilTel saying, “Hey, Beck, how fast can you get a new system in here?“ It turns out WilTel spent over $2 million with SOTAS and the highest throughput they could achieve was 16 million CDRs. SOTAS was out the door when they asked for yet another $500,000 for more hardware. That request was the one that broke the camel’s back.
Now WilTel wanted us to deliver an entirely new system that could do 35 million CDRs a day. The day we rolled in that new system at WilTel was one I’ll always remember. My team and I from Beck were there to install the system that was shipped to WilTel in two big boxes. Inside were two bays of equipment, including a SAN with lots of disk on it, two fully clustered systems, and ancillary parts such as fiber switches for communicating with the disk, etc.
But when the shipment arrived, the WilTel folks said, “Hey wait a minute, Gary, you only have two boxes here. That can’t be enough.“ And I said, “No, that’s the whole system.” And they said, “Gary, you know we’re going to do extensive regression testing to ensure this can absolutely handle the volumes.“ Well, I assured them that we had delivered a system that did as we promised. And it was at this point that they walked me back to their data center — that was the first time I saw the massive eleven bays of SOTAS equipment.
WilTel tested our system extensively over the next two days and discovered it could process 60 million CDRs.
So there’s a simple lesson here. Salesmen will promise the moon just to get the deal. But it’s not the salesman’s job to deliver. Well, guess what. My company is going to deliver because it’s got my damn name on it. And it’s more than pride in what we do; I’m a small company. I can’t afford not to deliver what I promise.
That’s a great success story, Gary. And please tell us about your technology. How are you able to pack all that processing power in a small footprint? |
The simplest way to describe our technology, Dan, is to say we sell a black box. In that respect, we’re a lot like a cloud computing supplier because when you sign up for a SaaS or other cloud service, you don’t ask: is it running on Unix or a Mainframe? That’s not important. The only thing you worry about is performance — are you getting what you paid for?
Well, unfortunately, the real-time requirements of our fraud system mean we can’t rely on a relational database like everyone else. So we built our own memory models that store, manage and mediate the data internally. Our entire data structure — memory as well as disk — was designed around doing one thing: processing vast amounts of data very quickly and optimized for our technology on top.
But this computer merely sits in a corner and nobody in the IT shop even maintains it. What you need to do is post a big sign on the rack saying, “Black box application running here. Do Not Disturb.“ |
Exactly, but you’d be surprised how long it takes customers to come around to that way of thinking. They hardly know the system is there. Even when a disk dies, we ship them a new disk and they swap it in at no cost to them.
Remember these are extremely highly available, multi-clustered systems. You could almost take an axe to the back of it and it would still run. System A’s uptime is measured in years and the redundant system B is capable of processing the full load. So if either system completely fails, the other takes over.
But how many CLECs or other small carriers really need this power? |
Well, we can scale the solution up or down. It’s not one-size-fits-all.
I have a small wireless ISP customer in Texas. They run something like 100,000 call detail records a day — not a lot. But they were being hit by $10,000 a day in fraud, so that’s why they are on our system. It’s very small. They have one guy who checks the system a couple times a day. Well, that customer doesn’t need a big computer, so that’s why we’re hosting them. They are paying about $1,500 a month to be hosted on my system. That’s it. Plus, they never have to do anything special, just log in and FTP us their CDR files.
So we are flexible in the way we deliver the solution. Another big customer is running about 14 million CDRs and that’s fully hosted in my data center too. But I also have a customer in the UK running about 20 million CDRs and they have their own system.
OK, we’ve covered your hardware technology. What about the software side of your fraud management system? |
Dan, we do things differently in a number of areas, such as in-system-mediation and our customizable user interface, all designed to simplify the investigator’s job, but let me cover the two most important ones.
In our fraud system, for every line of telecom service, we create a three dimensional model of that line’s traffic automatically. And the beauty of this is you don’t have to set up the system and say: “Anybody who calls X number of times gets an alarm.“ Almost everyone designs their fraud system with thresholds like that, but when you think about it, it’s impractical because you’ll have customers who call a thousand times a day and others who call 10 times a day.
As I said, with us, every single phone number and phone line is profiled. That’s our unique Fingerprint Signature Profile. And by the way, that requires more computing power than other methods because we update the 3D profile constantly to keep it statistically current. And if you have tens of millions of lines in service, we do that for every line.
For example, we may find out that you typically make X number of calls during certain hours of the day and those call are X minutes long. Then the next time we see traffic, we compare that new call to see if it fits the pattern.
Now for a new customer, we tell them to send us up to 90 days worth of back data. And we will pre-process and run that data through to create the memory model. In that way, the day our fraud system is turned on, you’ve already got the memory models for every line of service. Interestingly enough, every time we‘ve first turned on our system for a customer (with their data) the demo was stopped because they immediately found fraud.
In addition to your automated method for finding fraud, how does the fraud investigator insert new rules into the system? |
Well, we allow the fraud investigator to create alarms and more rules in their system to alarm any combination of call data, destination, times, etc. We also create watch points, or simplified rules that you can implement fast without a lot of fuss.
Now with this design our systems generates more alarms than anybody, so I’m sure the next question you’re going to ask me is: “So how do you keep people from being inundated with alarms?“ That’s where our Automatic Intelligent Feedback and the way we roll alarms up to the line of service level comes in. Bottom line, the system learns and minimizes the work load of the investigator.
As we accumulate alarms up to the LOS level, the number of alarms contributes to the priority of those listed LOS’s. And as that rolls up, the phone numbers with the highest numbers, the highest valued lines, worst countries, most minutes, most international calls, and all the other rules — those are the ones at the top of the list. So as you work from the top down, within minutes you’ve handled everything that’s going to hurt you. In other words, we create such a tight net that we can identify exactly what the investigator should be looking at.
Another important feature I’ll highlight and it has to do with a customer’s traffic changes. In this lousy economy of ours, companies have been reducing the amount of outbound calls they make. Ah, but this would allow somebody to get in there and start using a company’s traffic and you would never detect the increase for a long time. So to combat that threat, our system calculates for attenuation. It will compensate for the lowering levels of traffic automatically.
Finally, the intelligence we’ve put into the system has allowed operators to run fraud management with far fewer people. In the WilTel days, fraud staffs at carriers were typically anywhere between 10 and 20 people. Today, even large operators need only 4 or 5 fraud investigators to run our system; small operators can get by with only one.
Gary, this is great stuff. One final question: what’s the biggest challenge the fraud investigators — the people who use your system — worry about these days? |
To be honest, the biggest problem they face is getting steady funding for their fraud department. Once they stop the fraud, after a while management says, “We don’t have any fraud. Why am I spending this kind of money on staffing if I don’t have fraud?“
To combat that misunderstanding, (besides management reports) we coined an analogy that our customers find useful when trying to convince management, and here it is: Fraud is a wind that always blows. If you put up a wall, you won’t feel the wind. But as soon as you take down that wall, it’s going to blow you over.
I remember one time the wind blew pretty fierce at one company. Due to a reduction in force, they had to reduce their staffing from 8 to 2 fraud investigators. Shortly thereafter, the company lost $2.2 million in fraud. The system even noted the events, but nobody took action on it because there was too much to do. In the end, they brought back half the team.
So remember that: Fraud is the wind that always blows.
This article first appeared in Billing and OSS World.
Copyright 2011 Black Swan Telecom Journal