Hard to respect flawed Network Readiness Index

Tuanz and Vodafone remind us that New Zealand needs to focus on digital performance. See New Zealand tech slipping behind.

There is no room for complacency. Every other nation wants a competitive digital economy. They are all running towards the same goals.

Standing still is not an option. Any nation not keeping up with the pack risks becoming uncompetitive.

To make a case for more action, Tuanz draws on the Portulans Institute’s Network Readiness Index.

An alarming picture?

On the surface the index paints an alarming picture of New Zealand falling behind the rest of the world. That is until you look closer at the details and the league tables of digital performance.

There are places where the data and tables do not pass a basic sniff test. Many of the measures in the report are meaningless. In other cases, the writers infer a huge amount from the slimmest evidence.

New Zealand’s access story

Take “Access”. The report table shows New Zealand dropped from 16th in the world for access in 2020 to 42nd in the world in 2021. That’s a huge drop.

Keep in mind this is at a time service providers introduced the world-class 8 Gbps Hyperfibre. 2021 saw the Rural Connectivity Group plug rural broadband and mobile network gaps. Many New Zealanders moved to better broadband plans in the year.

By any standard New Zealand ‘access’ moved forward in 2021.

It’s possible the drop in the table is because 26 other countries moved forward faster than New Zealand.

Possible, but not likely.

Fractions of percentage points

Let’s dig deeper into the Portulans Institute’s “access” index. It notes “at least a 3G mobile network” covers a mere 99.46 percent of New Zealand’s population.

This puts us at 70 in the world in this category. It is our worst performing area in the access index.

If 3G covers 99.46 percent of New Zealanders, that means 0.54 percent are not covered. That’s a total of 28,000 people.

Compare with Australia

Australia ranks at number one in the world for “access”. That is the country where, if you are lucky, you may have a 25 Mbps fibre connection. Remember 85 percent of New Zealanders can get gigabit fibre or even Hyperfibre. Another 10 percent can get good fixed wireless broadband.

We have world class access, yet Portulans Institute ranks New Zealand 70 places behind Australia.

Australia is 44 in the world for the percentage of population covered by “at least a 3G mobile network”. Its 3G coverage figure is 99.87 percent. This implies there are 43 countries on that list have an even higher proportion with 3G coverage.

Feel the shame?

No doubt there will be people reading this who think the difference between 99.46 and 99.87 percent 3G coverage shames New Zealand.

In practice the gap between 99.46 and 99.87 percent doesn’t make any difference on a day to day basis.

And that’s before you think about the likely margin of error in those numbers. Hint: It’s likely to be more than one in a thousand.

The reality is that both countries have good 3G coverage scores. Neither is perfect, but New Zealand’s 99.46 percent is no handicap.

Singapore

It won’t surprise you to learn Singapore is top in this category with 100 percent coverage. The Portulans Institute doesn’t include the Vatican in its index. It it did, it would be another country with 100 percent coverage.

That’s because both countries are cities. Smothering a city in 3G is not the same as providing services throughout a country.

It’s a safe bet that Auckland and Melbourne are closer to 100 percent coverage than the countries they sit in.

Where is fibre?

The report does not even consider fast fibre networks. Which, are an important part of any modern access story. If the index included fibre, New Zealand’s position in the access table would be different.

Choosing to make a big deal out of a few thousandths difference in 3G coverage and ignoring fibre is a choice. It could be rational, but it feels arbitrary.

Many of the items the Portulans Institute measured have that arbitrary feel. Under “content” the report measures Github commits and Wikipedia edits. Sure, they measure something. Yet are they important components of a nation’s digital readiness?

Even the positives are weird

There’s a curious data point. New Zealand leapt 41 places up the table for “ICT regulatory environment” in 2021. Did something momentous happen in that area during the year? Did dozens of other countries unwrap their rules?

We could go on. There are questionable assumptions and measurement through the report. It’s not worthless, but nor is it a valuable pointer to relative performance.

There’s a good case for benchmarking our performance against other nations. But let’s not get excited about a poor global network readiness index report. It’s a distraction.

The argument for better digital performance stands on its own merit, not on some spurious set of figures made up by an otherwise unheard of think tank in Washington DC.