float conversion / presentation problem ?
Hi folks,
i've noticed an problem with conversion from real to float -
this produces ugly values. (in fact it seems to be real->float8)
fxignal=# SELECT 12.345::real::float;
float8
------------------
12.3450002670288
Maybe its not really an conversion problem, but an bug in sprintf(),
(there were some other float related bugs known in glibc), as these
examples let me suspect:
fxignal=# SELECT 12.345::real::float::real;
float4
--------
12.345
fxignal=# SELECT ((12.345::real::float8)-(0.345::float8))::real;
float4
--------
12
I noticed this behaviour on postgresql-7.4.1 and 7.4.3 on glibc-2.3.2
BTW: real is an alias to float4 ?
regards,
--
---------------------------------------------------------------------
Enrico Weigelt == metux IT service
phone: +49 36207 519931 www: http://www.metux.de/
fax: +49 36207 519932 email: contact@metux.de
cellphone: +49 174 7066481
---------------------------------------------------------------------
-- DSL ab 0 Euro. -- statische IP -- UUCP -- Hosting -- Webshops --
---------------------------------------------------------------------
Enrico Weigelt <weigelt@metux.de> writes:
i've noticed an problem with conversion from real to float -
this produces ugly values. (in fact it seems to be real->float8)
You do realize that float4/real is only good to six decimal places
on most platforms?
You can take the issue up with the glibc boys if you like, but
I think you'll get laughed off.
regards, tom lane
Tom Lane wrote:
Enrico Weigelt <weigelt@metux.de> writes:
i've noticed an problem with conversion from real to float -
this produces ugly values. (in fact it seems to be real->float8)You do realize that float4/real is only good to six decimal places
on most platforms?You can take the issue up with the glibc boys if you like, but
I think you'll get laughed off.
and importantly, are stored as binary fractions. 0.10000 decimal is a
repeating fraction in binary, its like .1100110011.... x 2^-2 or something as a
FLOAT. Float4 has like 22 bits of significants for the mantissa, while Float8
has like 51 bits or something (this is from rusty memory, so I could have
slipped a few bits here).
John R Pierce <pierce@hogranch.com> writes:
Tom Lane wrote:
You do realize that float4/real is only good to six decimal places
on most platforms?
and importantly, are stored as binary fractions. 0.10000 decimal is a
repeating fraction in binary, its like .1100110011.... x 2^-2 or
something as a FLOAT.
Right, and conversely a fraction that's cut off at a certain number of
base-2 digits often corresponds to a repeating pattern of base-10
digits. I once had a clue about the exact laws for this, but it was
many years ago :-(
regards, tom lane
Enrico Weigelt wrote:
BTW: real is an alias to float4 ?
Yes.
-Neil
On Thu, Apr 07, 2005 at 01:02:50AM -0400, Tom Lane wrote:
John R Pierce <pierce@hogranch.com> writes:
Tom Lane wrote:
You do realize that float4/real is only good to six decimal
places on most platforms?and importantly, are stored as binary fractions. 0.10000 decimal
is a repeating fraction in binary, its like .1100110011.... x 2^-2
or something as a FLOAT.Right, and conversely a fraction that's cut off at a certain number
of base-2 digits often corresponds to a repeating pattern of base-10
digits. I once had a clue about the exact laws for this, but it was
many years ago :-(
As with base-10, any denominator that can't be expressed as a power of
the prime factors in the base results in a repeating number. So
basically any denominator that can't be expressed as 2^n turns into a
repeating binary doohicky.
Cheers,
D
--
David Fetter david@fetter.org http://fetter.org/
phone: +1 510 893 6100 mobile: +1 415 235 3778
Remember to vote!
* Tom Lane <tgl@sss.pgh.pa.us> wrote:
Enrico Weigelt <weigelt@metux.de> writes:
i've noticed an problem with conversion from real to float -
this produces ugly values. (in fact it seems to be real->float8)You do realize that float4/real is only good to six decimal places
on most platforms?
Thats okay for me. I need only four.
You can take the issue up with the glibc boys if you like, but
I think you'll get laughed off.
ehm, why ?
cu
--
---------------------------------------------------------------------
Enrico Weigelt == metux IT service
phone: +49 36207 519931 www: http://www.metux.de/
fax: +49 36207 519932 email: contact@metux.de
cellphone: +49 174 7066481
---------------------------------------------------------------------
-- DSL ab 0 Euro. -- statische IP -- UUCP -- Hosting -- Webshops --
---------------------------------------------------------------------