This is a scene familiar to many a heroic Network Operations Director/VP – and I speak from experience.

Your boss, maybe the CTO, maybe the COO or CEO even, is setting your departments objectives for the coming year which translate into your own personal objectives and most likely the criteria for an annual bonus. Your boss has done a bit of reading on this, maybe spoken to the Head of Marketing, and decided that network availability should be 99%, dropped calls <2%, average network throughput per customer X and latency Y (fill them in yourselves).

There are three responses our heroic head of Network Ops can come up with.

The meek – ‘thank you sir/madam, I’ll do my best…………………’

The subjective – ‘yes, they seem about right’

The objective – ‘hang on a minute’

The objective response is perhaps the brave or foolhardy response career wise, but it is the right one, nonetheless. And the reason is simple enough. A critical question needs to be asked. ‘Who else in the organisation is sharing these network KPIs?’ 

Now just maybe you are working in an enlightened MNO/MVNO, and the business Key Performance Indicators have been holistically derived and there is evidence of a lot of joined up writing between the various directorates and departments. If you are, you have my congratulations, but my experience tells me that you are in a minority. During my time I have had many meetings and encounters with CTOs, and Network Ops Directors/VP  around the world and one consistent theme I have heard from so many is that network performance metrics are applied only to Tail End Charlie, the long-suffering head of Network Ops. Quite often this assignment, should they decide to accept it, is indeed mission impossible because the true network performance has been set in concrete long before the ops guys and girls get their hands on it.  Here is common mistake number one, network performance comes through design, procurement, and implementation. 

Ops can only maintain the network to the standard it has been designed and built to. Determining the right network performance indicators need to be the start of the network quality journey, not the end of it. Deciding them last means that the resulting measurements tell you how good the engineering design and build quality is and not a lot about the operational excellence of your Ops team.

I think that there are three steps to having a set of KPIs that work for a  Mobile Network Operator and Service Provider. One, get them right up front, two, make sure the whole company is signed up to them and thirdly, monitor and report them correctly, taking corrective action when necessary.

So, what does best practice look like? It does exist and I’ve seen it on my travels (but just not often enough). It should start with the Sales and Marketing department(s). This is step one. They should answer  questions like these:

• What quality of network do we need to be competitive in our marketplace? This applies to new customer acquisition as well as churn prevention.
• What price points do we need to hit for profitability? This determines the size and quality of network that the Service Provider can afford to build.
• What sort of customer experience are we looking for? Cheap and cheerful or Rolls Royce? What does “good enough” really look like?
• How many customers over what time are we shooting for and what will their various user profiles look like? What do those customers really focus on? This will help with the correct dimensioning of the network which is a crucial component of the end-user experience.
• Do we want in-building coverage as standard, and what sort of rural coverage is required and how extensive should the footprint be?
• What services do we want to offer customers now and in the future?
• This is then all baked into the company business plan and costed accordingly.

Using the wrong KPI’s can even damage your profits and business, as the outcome may be completely inappropriate!

I’ll talk more about setting the right targets later in this blog.

Another common mistake is to adopt what I have often heard  as being called industry best practice and/or a set of performance indicators derived from standards body recommendations. Whilst these are useful they are not a one-hat-fits-all set of KPIs that work for every MNO and in every market. I worked with an operator in a developing market who was the third entrant in the marketplace. The business wasn’t performing very well, and this was put down to the network performance and I was asked to come in and do a review. I discovered that they had adopted a gold standard set of network performance KPIs and they were indeed falling a long way short of achieving them. It was a tough operating environment, many base stations were off-grid, subject to power outages and theft and vandalism, and all the back haul needed to be self-provided. 

Even the world’s best Ops team were never going to meet the KPI targets that had been set for the Ops team. Here is the relevant point though. The market leader by some distance had network availability of 70%. The target for my client was 98%. Standard textbook target! Although they were well short of their own target they were still outperforming the market leader by some distance. This wasn’t translating into new customer sales though. The market leader was streets ahead in branding, channels, tariffing, and distribution. They were regarded as having poor customer service and people accepted that they were not the best network nor the cheapest, but they were still the dominant market leader. No amount of network performance KPI tinkering was ever going to solve their true market problem.

The lesson is to make sure your network performance KPI targets are right for your market and not out of a textbook. Marketing should have defined the product, and this should be the standard required for the business to succeed. Having the wrong KPI targets identified from the beginning is a common mistake, as is having too many performance indicators and what makes some of them ‘key’.

Working through the process for getting the network that the business needs, the Network Engineering department now need to design and dimension it. Here we often hit common mistake number two. The Engineering department gets down to designing the network in detail and before too long is attracting the attention of the CFO and Financial Controller because it looks like its going to cost a bit more to build than the business plan can take. There are three fixes. Downgrade the design (and tell Marketing, who immediately reduce their revenue targets), tell Procurement to get better pricing, and finally take out the luxury items like operating tools and systems and maybe a few less spares (but don’t tell Ops). 

After an often-tortuous process, a network design emerges, and mistake number two needs to be avoided at this stage before it’s too late. Can the Engineers demonstrate, at least in theory, that the network they have designed can meet the KPI targets – assuming they have been pre-set and agreed. In the case where they haven’t been  set and will be ordained  when the head of network ops arrives on the scene, it doesn’t matter anyway because it’s going to be his or her problem but in a best-in-class scenario, Engineering should be able to show all the link budgets and equipment MTBFs (Mean Time Between Failures) that demonstrate their design will deliver what’s needed. Include in this resilience and  N+1 of critical nodes. I can’t stress strongly enough, the foundations of network performance is by design, not by operation.

Let us move on the next step in the process and identify another common failing. Procurement. The US astronaut John Glenn, who sadly died recently, was often quoted as saying ‘I felt exactly how you would feel if you were getting ready to launch and knew you were sitting on top of 2 million parts — all built by the lowest bidder……’ 

Its procurement’s job to get best value for money. They are usually incentivised by how much money they can save in every product and service they buy. They’re generally quite good at it as well. What I rarely see is a procurement department incentivised equally to own part of the network performance – and they need to be. It’s too easy to cut essential operating functions from the procurement specification to reduce the bill but it is a false economy. We know of one Operator who thought that they could the halve costs of infrastructure compared to their competitor, until they discovered that they had built in no redundancy.

There are at least two other big departments at work in delivering the network. The Property Group who need to acquire the sites for base stations and the Implementation department who build the sites. You’ll will have guessed what comes next. How are they incentivised? Generally speaking, on speed and costs. The property people have a difficult job for sure, we all know that. Acquisition of sites is tough and getting tougher in some markets. Nonetheless, they also have a crucial role in helping network availably. In many MNOs I have worked with, the biggest cause of failure to hit network targets is inability to access sites quickly for repair. How may times have I seen a KPI specifying that faults must be attended to within four hours at a site with no weekend access? Lots is the answer. Next is climb to repair where permits and supervision is required and both take days to arrange. I do know that these are sometimes conditional restrictions in a site lease but let’s at least factor this into a network availability target.

Meanwhile, as soon as a site is acquired, the implementation teams need to get in and build and install the equipment. Speed is of the essence. Construction quality is often treated as nice to have but lack of it creates problems downstream. Inadequate antenna mountings, microwave dishes that go out of alignment in the wind, poor connectors, badly aligned antenna. Take your pick. Could the Ops team get a credit for every network outage caused by poor build quality? Nice to try! In many cases these activities are outsourced or contracted out to specialists in this field which is often a very sensible and pragmatic solution. In doing this though, make sure the contracts are written so as to achieve the required elements contributing to network performance targets and, crucially, make sure top class contract management is in place.

At this stage we are approaching the point of hand-over to Operations to bring the sites on stream. In realty they have little choice but to fire up the site because who wants that investment sitting there not earning money. In an ideal world there would be a comprehensive test of the site and some quality controls carried out on build and access. In a lot of networks this is done on a random basis – if at all. I have rarely seen a site sign-off that says this site will perform according to the Company agreed network performance targets. Isn’t that exactly what the testing regime should be all about before  a site is brought into service?

By now you may think I am letting the Ops team off lightly but they also have a big role to play. Operations is an overlooked science and requires mastery of logistics, mobilisation of staff and third parties, really good people, a great understanding of the network design (well documented in an ideal world) and a good understanding of all the interdependencies when things do go wrong. Yes, network faults happen but rarely is it due to misoperation but no excuses when it is. An essential part of operations is to identify immediately, and before the customer does, any network failure. This is largely the responsibility of a Network Operations Centre and in todays multi technology generation and sophisticated networks having the right and tools and systems is essential. I would say that this is part of the network engineering. Its very hard to retrospectively integrate tools and systems if they are not designed in from day one.

I do fully appreciate that in many business models some or all of the above functions are outsourced. Today we see virtual networks operators emerging with the complete design, build and operate of the network outsourced – often to the vendor supplying the equipment. In some cases they also supply the financing, so we move into a tricky area here. How much control can the Service Provider exert on a major vendor when that vendor is both gamekeeper and poacher? In scenarios where some or all of the network disciplines are outsourced, links with the design and deployment activities become more remote and more difficult to rectify. This makes the need for a common, and right set of KPIs even more important. The outsourced supplier(s) need to be tied into delivering the performance targets that the business needs.

I hope I have made the point that response number three from the Head of Network Ops I made right at the beginning, is the right one. Their job is to maintain the network to the quality level it has been designed and built to achieve. No one  can operate it at a higher level no matter what objectives are given to the Ops people. Setting network KPIs are crucial but they only work for the organisation when they are used correctly so let’s move on to what happens next and assume that we do have an across the board set of network performance indicators, and that we have selected the key ones and now we want to use them to monitor company (not just network ops) performance.

How we use the KPIs to improve business performance is important because surely that is the whole point of having KPIs. I started by saying the performance targets needed to be baked into the whole company ethos. This is true and for this to work it is necessary that each KPI has a detailed definition and explanation. I have seen the same network performance metric measured in five different ways in a single network and different results sent to different parts of the audience. I have even heard it said that Finance need network availability measured this way and Marketing want it measured a different way and the regulator wants is measure in yet another way. Quite extraordinary when you think about it. Get the definitions of each KPI well documented and the methods used for measuring them. It’s never too late to do this, even if you have been running a network for many years this is a good health check. This is important because not only does bad reporting of KPIs trip a business up in terms of its perceived performance versus the customer reported experience, it also can drive opex and capex spend in the wrong places and that is really not what any business wants.

Examples of this are applying the same KPIs to the busy hour as are used in the night-time low traffic periods. Including planned outages in the availability result when these are done out of hours. Reduced capacity outside the busy hour being included. Do you need the same level of network performance in all cells as opposed to the 20% of cells carrying 80% of your traffic ? Does deep rural coverage need the same performance level as city centres? All of these can result in a lowering of the overall 24-hour network availability, but do you really want to be spending money on the network to improve out of hour performance or very rural coverage when it is hardly noticeable to customers? There is a level of sophistication in the definition of KPIs that is essential to being able to make them work to identify the areas for  business improvement.

Once the KPIs are the right ones for the business and they are being measured accurately, the last element is the call to action when they are falling below design targets. This only happens when the KPIs are owned by the CEO/COO as they should be. It is no good leaving the KPIs to Ops to manage. This goes right back to where we started. The diligent Head of Network Operations will have persuaded his boss that that these are business targets, not just Ops targets, and that the business needs C level monitoring and ordering of improvement programmes when missed targets are identified.

Finally, KPIs need to be living metrics. They should evolve with new technology, with network growth and expansion, with new services and last but definitely not least, changing market and customer expectations. If you have the same basic set of network KPIs as you had 10 years ago, maybe with a few new data related ones thrown in, I suspect they are in need of a serious overhaul! Network KPIs need an annual review as part of the annual setting of the company operation plan. When was the last time your company did a comprehensive review of  your KPIs? Be honest!

At Azenby we can help. It does sometimes help to have an independent third eye looking at the network  KPIs being used in a business, how they are defined, monitored and used. We believe that network performance KPIs are a force for good and should be helping a mobile operator to produce the best network performance that it can, but we have also seen them damage a business because they are not the right ones, are badly monitored and poorly reported. We have 300-man years of experience in engineering and operating mobile networks and if you think you could do with an MOT of your network KPIs, why not get in touch and we’d like to talk it through  with you.