BUG #3959: Huge calculation error

Started by Renaud Diezabout 18 years ago4 messagesbugs
Jump to latest
#1Renaud Diez
rdiez@salesprize.com

The following bug has been logged online:

Bug reference: 3959
Logged by: Renaud Diez
Email address: rdiez@salesprize.com
PostgreSQL version: 8.2
Operating system: Debian
Description: Huge calculation error
Details:

the basic mathematical expression like the following one doesn't compute the
correct result:

select 100*(1+(21/100));

return a result of 100 instead of 121.

#2Magnus Hagander
magnus@hagander.net
In reply to: Renaud Diez (#1)
Re: BUG #3959: Huge calculation error

Renaud Diez wrote:

The following bug has been logged online:

Bug reference: 3959
Logged by: Renaud Diez
Email address: rdiez@salesprize.com
PostgreSQL version: 8.2
Operating system: Debian
Description: Huge calculation error
Details:

the basic mathematical expression like the following one doesn't compute the
correct result:

select 100*(1+(21/100));

return a result of 100 instead of 121.

It does compute the correct result, because you're doing integer
calculations. 21/100 for integers is 0. To make it do float calc, you can do
select 100*(1+(21::float/100));

Or you can do
select 100*(1+(21.0/100));

which will force it to numeric.

//Magnus

#3Heikki Linnakangas
heikki.linnakangas@enterprisedb.com
In reply to: Renaud Diez (#1)
Re: BUG #3959: Huge calculation error

Renaud Diez wrote:

The following bug has been logged online:

Bug reference: 3959
Logged by: Renaud Diez
Email address: rdiez@salesprize.com
PostgreSQL version: 8.2
Operating system: Debian
Description: Huge calculation error
Details:

the basic mathematical expression like the following one doesn't compute the
correct result:

select 100*(1+(21/100));

return a result of 100 instead of 121.

That's because 21/100 = 0, in integer math. Try "SELECT
100*(1+(21/100.0))" to use floating points.

--
Heikki Linnakangas
EnterpriseDB http://www.enterprisedb.com

#4Rodriguez Fernando
rodriguez@ort.edu.uy
In reply to: Renaud Diez (#1)
Re: BUG #3959: Huge calculation error

Renaud Diez wrote:

The following bug has been logged online:

Bug reference: 3959
Logged by: Renaud Diez
Email address: rdiez@salesprize.com
PostgreSQL version: 8.2
Operating system: Debian
Description: Huge calculation error
Details:

the basic mathematical expression like the following one doesn't compute the
correct result:

select 100*(1+(21/100));

return a result of 100 instead of 121.

---------------------------(end of broadcast)---------------------------
TIP 6: explain analyze is your friend

Hola, en realidad no lo hace mal, no es lo que esprabas, lo que en
realidad necesitas es esto:
select 100*(1+(21.0/100))::float;
saludos Fernando