The NPD attempts to track market share for smartphones (amongst other things). And given that manufacturers don’t really want to tell anyone exactly how many devices were sold (heck, Samsung now just talks in percentages!), I think they do an admirable job. They are joined by other groups such as comScore and IDC, and each publishes quarterly estimates of how everyone is doing in the smartphone market.
For a while now I have stated my issues with Tablet market share numbers. First off, Samsung used inflated numbers for their Galaxy Tab … then they stopped reporting numbers at all. Other vendors have not been very forthcoming with sales figured. Only Apple is clear with their numbers, which means these reporting companies have to do some serious guesswork. Even Amazon’s Kindle Fire varies in estimated sales from ~700,000 to more than 2 million depending on who you ask (and if anyone thinks 700k is remotely correct … well, think again!).
I have grudgingly accepted that NPD and others would continue simply parroting the PR-based phony numbers from Android makers used to inflate market share – but I have always felt that the smartphone numbers were more reliable. I mean – these are activated devices, so we should be able to trust the numbers.
Also, what the numbers have been saying seemed very credible – RIM is flailing into irrelevance, Nokia has fallen off the map, Microsoft’s Windows Mobile died and Windows Phone is in a very slow ascendance, Palm is gone … and the world is using either Android or iOS devices. And based on the broader spread of carriers, price points, options and carrier push – more people are using Android than iOS. None of that strains credulity.
But now a report from BusinessInsider pulls apart ALL of the latest quarterly reports – from carriers as well as what NPD/IDC/comScore are saying … and they see a HUGE discrepancy in the US market numbers.
Here is a chunk of what they are saying:
NPD put out a press release saying Android was the best selling smartphone operating system in the first quarter, grabbing 61% of the market. Apple’s iOS was in second, according to NPD, with 29% of the market.
However, there’s a big problem with this data. It runs contrary to what the carriers themselves are actually reporting.
At AT&T, Apple activated 4.3 million iPhones, or 78% of all AT&T smartphones for the quarter. At Verizon it activated 3.2 million iPhones, or 51% of all Verizon smartphones for the quarter. At Sprint it activated 1.5 million iPhones, or 60% of all smartphones for the quarter, according to Macquarie’s Kevin Smithen.
Total it all up, and Apple had 63% of the smartphone market on those three big carriers.
They take it a step further, referencing a report stating that the ‘big 3’ account for 80% of the wireless market based on numbers from last August – which means for this quarter they would account for even MORE – and extrapolate that Apple would actually have accounted for a full 50% of the smartphone market, which means that Android would have lower share.
I really couldn’t care less about who is ‘winning’ the market share race – both Android and iOS are selling devices and apps and accessories as fast as they can make them, and each offers a great solution for many people … which is why I live on an Android phone and an Apple tablet (and both PC and Mac laptops, Kindle Fire and Nook Touch and so on).
But what really bugs me is the potentially HUGE inaccuracy of the data used to represent the massive US smartphone market. Think of this – depending on whether you trust NPD, comScore or IDC, you will find that Android holds between 47 and 61% of the US market. That is a pretty huge variance – even worse when you see that the iOS share for two surveys with about the same Android share varies by more than 10%.
But that is the most revealing fact – NPD, IDC and comScore ALL are surveys based on a very small population. The recent NPD survey was of just under 13,000 people. That seems like a decent amount – until you use their own numbers saying that Apple had 29% market share and Apple’s 35 million iPhones sold … which amounts to a quarter with 120 million smartphones sold and a survey fraction of 0.01% of units sold. Or even if you just use Apple’s numbers it is 0.03%.
Is it an inherent problem using such a small number? Not necessarily – a properly designed survey can use a very small sample size and provide accurate insights. However, two of the biggest factors in any sampling plan are bias and variability.
Bias means that your estimates are inaccurate because your sample selection isn’t representative.
Variability means that your method of selection or surveying techniques would not result in the same estimates if repeated multiple times, within a margin of error.
It is my assertion that ALL of the market share surveys suffer from BOTH problems, from the same cause. Pretty much all of the coverage of smartphones during the last few years has been about how dynamic and explosive the market is – iPhone and Android rise, RIM and Nokia plummet, Microsoft tumbles, Palm crumbles, and so on. The market is unrecognizable compared to just a few years ago.
And … the less predictable the subjects of a survey, the greater your sample size needs to be in order to get accurate results. What sort of numbers would we need from the NPD? I would estimate at least one million people randomly distributed geographically, economically and demographically. Practical? Not in the least.
Which is why I say that the hard numbers from carriers call BS on these surveys.
What is worse – in response to these reports, the NPD and others have called some of the numbers into question. They question the report estimating the ‘big 3’ owning 80-90% of the overall wireless market, instead saying it is lower … and actually closer to 60% than 80%. This is due to the growing pre-paid market.
But even then there are issues. Let us assume that the ‘big 3’ only control ~65% of the wireless market. Next … let us assume that EVERY pre-paid phone was an Android smartphone (bear with me). That would mean that Apple controlled 63% of 65% of the market. Which is ~41% – which is well outside of the reported error margin for the NPD report. And obviously most of the pre-paid market is NOT smartphones, so the balance would change … and maybe Apple DOES have 50% of the market.
Again – who cares? Well, as I said certainly not me. Whether or not Apple is 30%, 50% or 70% wouldn’t have made me choose an iPhone over the Droid 4. But what if you are developing apps … or making cases or other accessories? Suddenly this stuff matters. And what does it mean for the OTHER folks like Nokia? I mean, think about it – if we can’t trust the NPD saying that their estimate 3900 iPhones out of ~13000 total will scale to the general public, then why should we believe them when they try to scale less than 300 Windows Phone devices?
What do you think? Do these numbers mean anything? Should we care? How do you interpret this stuff?