There’s no reason to doubt Statistics Canada’s August jobs report

This post originally appeared at BusinessInCanada.com

In the wake of August’s jobs report, which showed that Canada lost 11,000 jobs over the course of the month, some economists are skeptical that these figures are accurate – and a number of media outlets are ON IT.

I, for one, find this lack of faith in Statistics Canada disturbing.

When asked if the agency was confidently standing behind August’s figures, Jason Gilmore, acting chief of Labour Analysis and Special Projects, replied, “Absolutely.”

Yes, StatsCan did err in the initial print of July’s jobs report. But officials have stressed that this was an isolated incident and I’m inclined to believe them, despite the concern some very well-respected economists voiced over these numbers.

“At first glance, the details of today’s report do not quite pass the smell test yet again,” writes Gluskin Sheff chief economist and strategist David Rosenberg.

“It looks very fishy to me,” wrote Scotiabank economists Derek Holt and Dov Zigler in their commentary on August’s Labour Force Survey.

The point made by these economists was that the “coincidental surge,” as Scotia’s duo put it, in self-employment nearly fully offset plummeting private payrolls. The increase in self-employment (86,900) and decrease in private-sector jobs (111,800) seen in August were both monthly records.

“What an utterly fascinating pair of coincidences,” wrote Holt and Zigler.

These Scotiabank economists, it should be noted, raised questions about the accuracy of July’s jobs figures before Statistics Canada revealed that they were erroneous–a remarkably prescient call. “What caught our attention was a series of remarkable coincidences that while not statistically impossible to explain collectively nevertheless arise with the regularity of asteroids hitting our planet,” they wrote.

But in a data release that’s as “notoriously volatile” as the Labour Force Survey, there will always be occasions in which the results raise a few eyebrows. But coincidences, like death and taxes, are a constant in life.

For instance, Abraham Lincoln and John F. Kennedy were elected President of the United States 100 years apart, were both killed with shots to the head by assassins who had 15 letters in their names, and were succeeded by men with the surname Johnson.

But if you picked any two pieces of economic data released by Statistics Canada during fiscal year 2012/2013 at random, there would be less than a 1 in 100,000 chance that both would contain errors.

That, according to C-3P0, is considerably lower than the probability of successfully navigating an asteroid field.

It would be more than a coincidence for consecutive Labour Force Surveys to be flawed; it would represent a systemic issue in how the data is collected and/or processed. The type of error StatsCan caught–and corrected–following the first print of July’s jobs numbers.

One would also assume that, in the wake of this public embarrassment, Statistics Canada’s analysts were working overtime to ensure that the data in August’s Labour Force Survey were collected, processed, and published accurately.

“These figures were double, triple, quadruple, quintuple-checked, and then some,” said StatsCan’s Gilmore. “They were evaluated by multiple people in multiple steps, and there has been no change to the system since correcting the issue that affected July’s results.”

Holt and Zigler posited that perhaps people who lost private-sector jobs declared themselves to be self-employed, which would have artificially boosted the headline figure for the monthly change in employment. That’s something that has a ring of truth to it. Even in anonymous surveys, people lie – and it’s hard for StatsCan to screen for that.

However, the overemphasis on seeming peculiarities in August’s Labour Force Survey only serves to undermine the public’s faith in Statistics Canada, and in this case, undeservedly so. And while this is a particularly egregious example of unbalanced coverage, it isn’t the first time the media has failed to explain Canadian jobs figures as accurately as possible. In fact, the same mistake is repeated month after month. Re-read the first sentence of this article, or the second sentence of the initial take on August’s Labour Force Survey.

Strictly speaking, I’m doing it wrong (and, for the sake of simplicity, will continue to going forward).

A more accurate way to report what we think we know about what happened to Canada’s labour market in August would look a little something like this:

“Canada’s national statistics agency says there’s a 68-percent chance that the economy added up to 17,900 jobs or lost up to 39,900 jobs, and a 95-percent chance employment grew by 46,800 or decreased by as much as 68,800 over the course of the month.”

If this doesn’t seem particularly useful or newsworthy, that’s because it isn’t. Perhaps the only justification for reporting the headline number on monthly net job growth as pseudo-gospel is the fact these releases do indeed move markets; in particular, they affect the level of the Canadian dollar.

So, if StatsCan’s August report is wrong (rather, if the figures in the report make it look like something’s wrong), it’s because one could argue that those reported figures are wrong, to a certain extent – they’re subject to the same substantial margin of error as they are every month.

“You have to keep in mind that this is a sample survey, and there is a certain margin of error attached to it,” said Vincent Ferrao, an analyst in StatsCan’s Labour Statistics Division, in an interview with the Business News Network. “The proper way to describe the situation this month is that overall employment was little changed.”

A month of little change in employment is more or less in line with what economists were expecting – the consensus estimate was for net job growth of 10,000 in August. And while CIBC’s chief economist Avery Shenfeld has described attempts to forecast monthly fluctuations in employment as a – and I quote – “crapshoot,” economists weren’t any more off-the-mark in their estimates for this jobs report than usual (see right).

The volatility of the Labour Force Survey and large standard of error imply that all one-month changes should be taken with a grain of salt; looking at broader trends provides more reliable and relevant information.

“More statistically precise inferences about the pace of employment growth can be obtained by combining information from several consecutive months,” wrote André Bernard and Guy Gellatly, members of StatsCan’s Analytical Studies Branch, in March. “The standard errors associated with the difference in the average monthly change in employment calculated over periods of three, six and twelve months are smaller than the standard error based solely on the current and previous reference months.”

That’s why Business in Canada includes a chart showing the six and 12-month moving averages for net monthly job growth following the publication of each Labour Force Survey, and prefers to use the longer-term average when discussing what’s going on in the labour market.

If we assume that the economy added as many jobs as economists anticipated in August, the six-month moving average for monthly net job growth would have perked up to 13,700 from 10,850 rather than declining to 10,200. But the 12-month moving average would have declined whether the estimated change in employment was 10,000 or -11,000:

And according to Gilmore, the large one-month changes in private and self-employment in August don’t really tell us anything new about job growth in these classes.

“Private sector employees increased in July, but we now see a large decline among that group,” he told us. “But the trend since September 2013 shows that this group is relatively unchanged.”

The same thing holds for self-employment, which has a relatively flat trend over the past eight to ten months despite tumbling in July and spiking in August.

“As a general commentary, throughout our entire time series since 1976, it has not been unusual to see a movement for any of our data (e.g., age-sex, industry, class of worker, provinces, etc.) move in the other direction the following month, to varying magnitudes of change,” writes Gilmore.

In sum, August’s Labour Force Survey shouldn’t change your perception of how the labour market has been doing lately (hint: badly).

And it shouldn’t change your perception about the quality of data released by Statistics Canada, an agency whose error rate is low–and falling.