UPDATE: Don’t be a Hoover

Last week I noted the jobs numbers and that Bush was “plus jobs” except for the seasonally-adjusted CES. I also noted that the archived press releases used a different number for January 2001 than the handy online database at the BLS did. (The press release numbers, in fact, would have meant that Bush was also “plus jobs” according to the seasonally-adjusted CES.)

Being unable to figure out the discrepancy myself and since no smart readers gave me a clue, I finally took the step of emailing the BLS. Shock of shock, but they responded within an hour or so.

Here’s the answer:

The discrepancy in the example you noted is due to the fact that the numbers reported in the press release (archived report) were pre-benchmark numbers, and the numbers in the database are post-benchmark numbers. I am providing you a link to the CES
Methodology page to help answer some of your questions.


And he kindly indicated that the most current numbers will always be in the database and that any analysis should use that data, not the archived press releases. Which makes sense.

A quick perusal of the Methodology page gives us:

(8) What is a benchmark?
The benchmark adjustment, a standard part of the payroll survey estimation process, is a once-a-year re-anchoring of the sample-based employment estimates to full population counts available principally through unemployment insurance (UI) tax records filed by employers with State Employment Security Agencies. By early October of each year, BLS completes preliminary tabulations of these universe counts for the first quarter of the year and routinely shares that information with the public at the time of the issuance of the September Employment Situation news release.

(9) What is the UI universe count?
The Bureau’s UI universe count is a quarterly tabulation, from administrative records, of the number of employees covered by unemployment insurance (UI) laws. UI universe counts, available on a lagged basis, contain individual employer records for over 8 million establishments and cover nearly 97 percent of total nonfarm employment; they thus provide a benchmark for the sample-based estimates. For the small segment of the population not covered by UI, BLS develops employment benchmarks from several alternative sources.

(10) Why are the payroll survey estimates benchmarked to UI universe counts?
The CES survey, like many other surveys, establishes benchmarks on a periodic basis in order to adjust its sample-based estimates to complete population counts available from administrative records.

Because of their much smaller size, sample surveys offer an ability to produce very timely estimates along with a greater ability to control the data quality of individual reports. There is a need, however, to recalibrate sample estimates periodically against full population counts. The use of a population count, or benchmark, allows a sample survey to adjust the results of estimation processes for new birth units in the population frame, and to adjust for sampling and other non-sampling errors.

(11) How does the benchmark revision affect the employment data for months prior to the benchmark month?
Following standard BLS methodology, the March UI-based benchmark employment level replaces the March sample-based employment estimate, and then the difference between the benchmark level and the sample-based estimate is wedged back to the previous benchmark level. For example, the benchmark revision that was released in February 2004 replaced the March 2003 estimate with the benchmark level, decreasing the employment level for that month by 122,000. To wedge this adjustment over the prior year, 1/12 of the difference was added to April 2002, 2/12s to May and so forth, through February 2003 which received 11/12s of the difference.

(12) How does the benchmark revision affect the employment data for months subsequent to the benchmark month?
Estimates for the period after the benchmark month (the post-benchmark period) are calculated for each month based on the new benchmark level, new net birth/death figures, and the annual sample update, which is implemented in November following the benchmark month.

(13) What are the causes of benchmark revisions?
In general, differences between universe counts and sample-based estimates result from both sampling and non-sampling error. Although sampling error is present in the payroll survey, as it is in all surveys, the CES sample is so large that sampling error is not usually an important factor in explaining the differences.

Nonsampling error arises in the survey estimates, and in the universe counts, from both the UI and the alternative sources used to establish the noncovered population benchmarks. Nonsampling error is a more significant cause of benchmark revisions. Sources of nonsampling error include coverage, response, and processing errors in both data series. Additionally, the survey is potentially subject to sample design and estimator biases.

Note in particular #11.

That means that later in 2001 a new benchmark was established using the information listed above, and the January numbers were adjusted by 10/12ths of the difference between the new benchmark and the old.

Giving us the discrepancy.

Since this benchmark adjustment meant the difference between Bush being “plus jobs” and “minus jobs” during his first term, it’s obvious that it’s close either way. The pro-Bush crowd will point out that things aren’t as bad as John Kerry and others claimed, and the anti-Bush crowd will respond with the obvious “yeah, but how many jobs did he CREATE?” even if the numbers next month manage to put Bush over the top.

Then, about the time the argument dies down or so, the benchmark will be adjusted again and the actual results may be totally different. Which of course could begin another round of arguments. Blah. Blah. Blah.

Suffice it to say that MO thinks things aren’t so bad. They could be a heck of a lot better, of course, but when I think back to my feelings in mid and late September of 2001, I realize that they could also be far, far worse. And I expected them to be.