numeric stddev_pop and var_pop are wrong

Started by Tom Lanealmost 19 years ago2 messagesbugs
Jump to latest
#1Tom Lane
tgl@sss.pgh.pa.us

When applied to "numeric" input, stddev_pop produces the same result as
stddev_samp, and var_pop produces the same result as var_samp. This
is because whoever wrote numeric_stddev_internal() forgot that the
divisor is different for the sample case. This mistake is in fact
visible in the regression test outputs for the function, so those
outputs weren't checked very carefully :-(

regards, tom lane

#2Neil Conway
neilc@samurai.com
In reply to: Tom Lane (#1)
Re: numeric stddev_pop and var_pop are wrong

On Mon, 2007-09-07 at 10:39 -0400, Tom Lane wrote:

When applied to "numeric" input, stddev_pop produces the same result as
stddev_samp, and var_pop produces the same result as var_samp. This
is because whoever wrote numeric_stddev_internal() forgot that the
divisor is different for the sample case. This mistake is in fact
visible in the regression test outputs for the function, so those
outputs weren't checked very carefully :-(

My fault, I'm afraid -- how embarrassing! I certainly tested the
function for numeric inputs while developing it. I can only guess that I
wrote it and tested it, then refactored and broke it, and blindly copied
the resulting regression outputs => expected...

Sorry,

Neil