Same double precision operations, different results

Started by Daniel Vázquezalmost 12 years ago3 messages
#1Daniel Vázquez
daniel2d2art@gmail.com

Please can someone tell me why this behavior? Thx!!

I know it's calculating over coordinates on same points, but the operations
of values (all double precision) differs even forcing with cast

(*) Here some double precision calculations and obtaining values

# Select
# cos(radians(39.9826557)) as f1,
# cos(radians(39.9826557)) as f2,
# cos(radians(-0.04773120000004383) - radians(-0.04773120000004383)) as
f3,
# sin(radians(39.9826557)) as f4,
# sin(radians(39.9826557)) as f5;
f1 | f2 | f3 | f4 | f5

-------------------+-------------------+----+-------------------+-------------------
0.766238989559398 | 0.766238989559398 | 1 | 0.642555686986733 |
0.642555686986733
(1 row)

________________________________________

(*) Here above same obtaining values involved in simple operations

# select (0.766238989559398 * 0.766238989559398 * 1 + 0.642555686986733 *
0.642555686986733) calc;
calc
----------------------------------
0.999999999999999633651488135693
(1 row)

________________________________________

(*) Here above queries playing together ---> Result = 1 ????? NOOOO! It
should be 0.999999999999999633651488135693

# select (v.f1 * v.f2 * v.f3 + v.f4 * v.f5) calc from ( select
cos(radians(39.9826557)) as f1,
cos(radians(39.9826557)) as f2,
cos(radians(-0.04773120000004383) - radians(-0.04773120000004383)) as f3,
sin(radians(39.9826557)) as f4,
sin(radians(39.9826557)) as f5 ) as v;
calc
------
1
(1 row)

________________________________________

(*) Yep, this produces problems with acos. Here acos working directly with
the above values. OK this is like acos(0.999999999999999633651488135693)

# select acos(0.766238989559398 * 0.766238989559398 * 1 + 0.642555686986733
* 0.642555686986733);
acos
----------------------
2.58095682795179e-08
(1 row)

________________________________________

(*) Yep, you can see same result

# select acos(0.999999999999999633651488135693);
acos
----------------------
2.58095682795179e-08
(1 r

________________________________________

(*) ok acos values from -1 to 1 you can see acos(1) here

# select acos(1);
acos
------
0
(1 row)

________________________________________

(*) Why this calculation produces 1 and not 0.999999999999999633651488135693
?????????

# select (
cos(radians(39.9826557))
* cos(radians(39.9826557))
* cos(radians(-0.04773120000004383) - radians(-0.04773120000004383))
+ sin(radians(39.9826557))
* sin(radians(39.9826557))
) calc;
calc
------
1
(1 row)

________________________________________

(*) any way, the result is 1, then if acos(1) get results, this must get
same result .... NOPS!! "input is out of range"

# select acos(
# cos(radians(39.9826557))
# * cos(radians(39.9826557))
# * cos(radians(-0.04773120000004383) - radians(-0.04773120000004383))
# + sin(radians(39.9826557))
# * sin(radians(39.9826557))
# );
ERROR: input is out of range

________________________________________

(*) You are thinking: it's some cast problem ... NOPS!! "input is out of
range"

# select acos( cast (
cos(radians(39.9826557))
* cos(radians(39.9826557))
* cos(radians(-0.04773120000004383) - radians(-0.04773120000004383))
+ sin(radians(39.9826557))
* sin(radians(39.9826557)) as double precision )
);
ERROR: input is out of range

Really, I can get the reason!!! Some idea? Workaround? Thx

#2Tom Lane
tgl@sss.pgh.pa.us
In reply to: Daniel Vázquez (#1)
Re: Same double precision operations, different results

=?ISO-8859-1?Q?Daniel_V=E1zquez?= <daniel2d2art@gmail.com> writes:

Please can someone tell me why this behavior? Thx!!

You're confusing numeric and double precision; in particular this
calculation is *not* being done in float8, but numeric:

# select (0.766238989559398 * 0.766238989559398 * 1 + 0.642555686986733 *
0.642555686986733) calc;

(*) Why this calculation produces 1 and not 0.999999999999999633651488135693

Actually, it's not producing 1, but a smidgen more:

regression=# set extra_float_digits TO 3;
SET
regression=# select ( cast (
cos(radians(39.9826557))
* cos(radians(39.9826557))
* cos(radians(-0.04773120000004383) - radians(-0.04773120000004383))
+ sin(radians(39.9826557))
* sin(radians(39.9826557)) as double precision )
);
float8
---------------------
1.00000000000000022
(1 row)

You've got roundoff error either way, but this way happens to be in the
direction that makes acos() complain.

regards, tom lane

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

#3Daniel Vázquez
daniel2d2art@gmail.com
In reply to: Tom Lane (#2)
Re: Same double precision operations, different results

Thx Tom!!

Yep, I focused on all trigonometric functions take arguments and return
values of type double precision. Looking for the error I lost focus on
numeric values directly on the select and diff inner calculations.

I think best approach will be maintain double precision on trigonometric
calculations for faster and cast to numeric before acos operation.

Thank you man!

2014-02-13 18:26 GMT+01:00 Tom Lane <tgl@sss.pgh.pa.us>:

Show quoted text

=?ISO-8859-1?Q?Daniel_V=E1zquez?= <daniel2d2art@gmail.com> writes:

Please can someone tell me why this behavior? Thx!!

You're confusing numeric and double precision; in particular this
calculation is *not* being done in float8, but numeric:

# select (0.766238989559398 * 0.766238989559398 * 1 + 0.642555686986733 *
0.642555686986733) calc;

(*) Why this calculation produces 1 and not

0.999999999999999633651488135693

Actually, it's not producing 1, but a smidgen more:

regression=# set extra_float_digits TO 3;
SET
regression=# select ( cast (
cos(radians(39.9826557))
* cos(radians(39.9826557))
* cos(radians(-0.04773120000004383) - radians(-0.04773120000004383))
+ sin(radians(39.9826557))
* sin(radians(39.9826557)) as double precision )
);
float8
---------------------
1.00000000000000022
(1 row)

You've got roundoff error either way, but this way happens to be in the
direction that makes acos() complain.

regards, tom lane