Bug in to_timestamp().

Started by amul sulalmost 10 years ago136 messageshackers
Jump to latest
#1amul sul
sul_amul@yahoo.co.in

Hi,

It's look like bug in to_timestamp() function when format string has more whitespaces compare to input string, see below:

Ex.1: Two white spaces before HH24 whereas one before input time string

postgres=# SELECT TO_TIMESTAMP('2016-06-13 15:43:36', 'YYYY/MM/DD HH24:MI:SS');
to_timestamp
------------------------
2016-06-13 05:43:36-07 <— incorrect time
(1 row)

Ex.2: One whitespace before YYYY format string

postgres=# SELECT TO_TIMESTAMP('2016/06/13 15:43:36', ' YYYY/MM/DD HH24:MI:SS');
to_timestamp
------------------------------
0016-06-13 15:43:36-07:52:58 <— incorrect year
(1 row)

If there are one or more consecutive whitespace in the format, we should skip those as long as we could get an actual field.
Thoughts?
Thanks & Regards,
Amul Sul

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

#2Tom Lane
tgl@sss.pgh.pa.us
In reply to: amul sul (#1)
Re: Bug in to_timestamp().

amul sul <sul_amul@yahoo.co.in> writes:

It's look like bug in to_timestamp() function when format string has more whitespaces compare to input string, see below:

No, I think this is a case of "input doesn't match the format string".

As a rule of thumb, using to_timestamp() for input that could be parsed
just fine by the standard timestamp input function is not a particularly
good idea. to_timestamp() is meant to deal with input that is in a
well-defined format that happens to not be parsable by timestamp_in.
This example doesn't meet either of those preconditions.

regards, tom lane

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

#3Robert Haas
robertmhaas@gmail.com
In reply to: Tom Lane (#2)
Re: Bug in to_timestamp().

On Mon, Jun 13, 2016 at 12:12 PM, Tom Lane <tgl@sss.pgh.pa.us> wrote:

amul sul <sul_amul@yahoo.co.in> writes:

It's look like bug in to_timestamp() function when format string has more whitespaces compare to input string, see below:

No, I think this is a case of "input doesn't match the format string".

As a rule of thumb, using to_timestamp() for input that could be parsed
just fine by the standard timestamp input function is not a particularly
good idea. to_timestamp() is meant to deal with input that is in a
well-defined format that happens to not be parsable by timestamp_in.
This example doesn't meet either of those preconditions.

I think a space in the format string should skip a whitespace
character in the input string, but not a non-whitespace character.
It's my understanding that these functions exist in no small part for
compatibility with Oracle, and Oracle declines to skip the digit '1'
on the basis of an extra space in the format string, which IMHO is the
behavior any reasonable user would expect.

--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

#4amul sul
sul_amul@yahoo.co.in
In reply to: Robert Haas (#3)
Re: Bug in to_timestamp().

On Monday, 13 June 2016 9:57 PM, Robert Haas <robertmhaas@gmail.com> wrote:

I think a space in the format string should skip a whitespace
character in the input string, but not a non-whitespace character.

+1, enough the benefit is we are giving correct output.

PFA patch proposing this fix.

Regards,
Amul Sul.

Attachments:

0001-RM37358-space-in-the-format-string-should-skip-a-whi.patchapplication/octet-streamDownload+24-13
#5Robert Haas
robertmhaas@gmail.com
In reply to: Robert Haas (#3)
Re: Bug in to_timestamp().

On Mon, Jun 13, 2016 at 12:25 PM, Robert Haas <robertmhaas@gmail.com> wrote:

On Mon, Jun 13, 2016 at 12:12 PM, Tom Lane <tgl@sss.pgh.pa.us> wrote:

amul sul <sul_amul@yahoo.co.in> writes:

It's look like bug in to_timestamp() function when format string has more whitespaces compare to input string, see below:

No, I think this is a case of "input doesn't match the format string".

As a rule of thumb, using to_timestamp() for input that could be parsed
just fine by the standard timestamp input function is not a particularly
good idea. to_timestamp() is meant to deal with input that is in a
well-defined format that happens to not be parsable by timestamp_in.
This example doesn't meet either of those preconditions.

I think a space in the format string should skip a whitespace
character in the input string, but not a non-whitespace character.
It's my understanding that these functions exist in no small part for
compatibility with Oracle, and Oracle declines to skip the digit '1'
on the basis of an extra space in the format string, which IMHO is the
behavior any reasonable user would expect.

So Amul and I are of one opinion and Tom is of another. Anyone else
have an opinion?

--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

#6David G. Johnston
david.g.johnston@gmail.com
In reply to: Robert Haas (#5)
Re: Bug in to_timestamp().

On Mon, Jun 20, 2016 at 8:19 AM, Robert Haas <robertmhaas@gmail.com> wrote:

On Mon, Jun 13, 2016 at 12:25 PM, Robert Haas <robertmhaas@gmail.com>
wrote:

On Mon, Jun 13, 2016 at 12:12 PM, Tom Lane <tgl@sss.pgh.pa.us> wrote:

amul sul <sul_amul@yahoo.co.in> writes:

It's look like bug in to_timestamp() function when format string has

more whitespaces compare to input string, see below:

No, I think this is a case of "input doesn't match the format string".

As a rule of thumb, using to_timestamp() for input that could be parsed
just fine by the standard timestamp input function is not a particularly
good idea. to_timestamp() is meant to deal with input that is in a
well-defined format that happens to not be parsable by timestamp_in.
This example doesn't meet either of those preconditions.

I think a space in the format string should skip a whitespace
character in the input string, but not a non-whitespace character.
It's my understanding that these functions exist in no small part for
compatibility with Oracle, and Oracle declines to skip the digit '1'
on the basis of an extra space in the format string, which IMHO is the
behavior any reasonable user would expect.

So Amul and I are of one opinion and Tom is of another. Anyone else
have an opinion?

​At least Tom's position has the benefit of being consistent with current
behavior. The current implementation doesn't actually care what literal
value you specify - any non-special character consumes a single token from
the input, period.

SELECT TO_TIMESTAMP('2016-06-13 15:43:36', 'YYYY/MM/DD--HH24:MI:SS');
SELECT TO_TIMESTAMP('2016-06-13 15:43:36', 'YYYY/MM/DD-HH24:MI:SS');

Both the above exhibit the same behavior as if you used a space instead of
the hyphen as the group separator.

The documentation should be updated to make this particular dynamic more
clear.

I don't see changing the general behavior of these "date formatting"
functions a worthwhile endeavor.​ Adding a less-liberal "parse_timestamp"
function I could get behind.

IOW, the user seems to be happy with the fact that the "/" in the date
matches his "-" but them complains that the space matches the number "1".
You don't get to have it both ways.

[re-reads the third usage note]

Or maybe you do. We already define space as a being a greedy operator
(when not used in conjunction with FX). A greedy space-space sequence
makes little sense on its face and if we are going to be helpful here we
should treat it as a single greedy space matcher.

Note that "returns an error because to_timestamp expects one space only" is
wrong - it errors because only a single space is captured and then the
attempt to parse ' JUN' using "MON" fails. The following query doesn't
fail though it exhibits the same space discrepancy (it just gives the same
"wrong" result).

SELECT TO_TIMESTAMP('2016-06-13 15:43:36', 'FXYYYY/MM/DD HH24:MI:SS');

Given that we already partially special-case the space expression I'd be
inclined to consider Robert's and Amul's position on the matter. I think
I'd redefine our treatment of space to be "zero or more" instead of "one or
more" and require that it only match a literal space in the input.

Having considered that, I'm not convinced its worth a compatibility break.
I'd much rather deprecate these <to_*> versions and write
slightly-less-liberal versions named <parse_*>.

In any case I'd called the present wording a bug. Instead:

A single space consumes a single token of input and then, unless the FX
modifier is present, consumes zero or more subsequent literal spaces.
Thus, using two spaces in a row without the FX modifier, while allowed, is
unlikely to give you a satisfactory result. The first space will consume
all available consecutive spaces so that the second space will be
guaranteed to consume a non-space token from the input.

David J.

#7Tom Lane
tgl@sss.pgh.pa.us
In reply to: Robert Haas (#5)
Re: Bug in to_timestamp().

Robert Haas <robertmhaas@gmail.com> writes:

On Mon, Jun 13, 2016 at 12:25 PM, Robert Haas <robertmhaas@gmail.com> wrote:

I think a space in the format string should skip a whitespace
character in the input string, but not a non-whitespace character.
It's my understanding that these functions exist in no small part for
compatibility with Oracle, and Oracle declines to skip the digit '1'
on the basis of an extra space in the format string, which IMHO is the
behavior any reasonable user would expect.

So Amul and I are of one opinion and Tom is of another. Anyone else
have an opinion?

I don't necessarily have an opinion yet. I would like to see more than
just an unsupported assertion about what Oracle's behavior is. Also,
how should FM mode affect this?

regards, tom lane

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

#8Laurenz Albe
laurenz.albe@cybertec.at
In reply to: Tom Lane (#7)
Re: Bug in to_timestamp().

Tom Lane wrote:

I don't necessarily have an opinion yet. I would like to see more than
just an unsupported assertion about what Oracle's behavior is. Also,
how should FM mode affect this?

I can supply what Oracle 12.1 does:

SQL> SELECT to_timestamp('2016-06-13 15:43:36', ' YYYY/MM/DD HH24:MI:SS') AS ts FROM dual;

TS
--------------------------------
2016-06-13 15:43:36.000000000 AD

SQL> SELECT to_timestamp('2016-06-13 15:43:36', 'YYYY/MM/DD HH24:MI:SS') AS ts FROM dual;

TS
--------------------------------
2016-06-13 15:43:36.000000000 AD

SQL> SELECT to_timestamp('2016-06-13 15:43:36', 'YYYY/MM/DD HH24:MI:SS') AS ts FROM dual;

TS
--------------------------------
2016-06-13 15:43:36.000000000 AD

(to_timestamp_tz behaves the same way.)

So Oracle seems to make no difference between one or more spaces.

Yours,
Laurenz Albe

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

#9Alex Ignatov
a.ignatov@postgrespro.ru
In reply to: Tom Lane (#7)
Re: Bug in to_timestamp().

On 20.06.2016 16:36, Tom Lane wrote:

Robert Haas <robertmhaas@gmail.com> writes:

On Mon, Jun 13, 2016 at 12:25 PM, Robert Haas <robertmhaas@gmail.com> wrote:

I think a space in the format string should skip a whitespace
character in the input string, but not a non-whitespace character.
It's my understanding that these functions exist in no small part for
compatibility with Oracle, and Oracle declines to skip the digit '1'
on the basis of an extra space in the format string, which IMHO is the
behavior any reasonable user would expect.

So Amul and I are of one opinion and Tom is of another. Anyone else
have an opinion?

I don't necessarily have an opinion yet. I would like to see more than
just an unsupported assertion about what Oracle's behavior is. Also,
how should FM mode affect this?

regards, tom lane

Oracle:
SQL> SELECT TO_TIMESTAMP('2016-06-13 99:99:99', 'YYYYMMDD HH24:MI:SS')
from dual;
SELECT TO_TIMESTAMP('2016-06-13 99:99:99', 'YYYYMMDD HH24:MI:SS') from dual
*
ERROR at line 1:
ORA-01843: not a valid month

PG:

postgres=# SELECT TO_TIMESTAMP('2016-06-13 99:99:99', 'YYYYMMDD
HH24:MI:SS');
to_timestamp
------------------------
2016-01-06 14:40:39+03
(1 row)

I know about:
"These functions interpret input liberally, with minimal error checking.
While they produce valid output, the conversion can yield unexpected
results" from docs but by providing illegal input parameters we have no
any exceptions or errors about that.
I think that to_timestamp() need to has more format checking than it has
now.

Alex Ignatov
Postgres Professional: http://www.postgrespro.com
The Russian Postgres Company

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

#10Alex Ignatov
a.ignatov@postgrespro.ru
In reply to: amul sul (#1)
Re: Bug in to_timestamp().

On 13.06.2016 18:52, amul sul wrote:

Hi,

It's look like bug in to_timestamp() function when format string has more whitespaces compare to input string, see below:

Ex.1: Two white spaces before HH24 whereas one before input time string

postgres=# SELECT TO_TIMESTAMP('2016-06-13 15:43:36', 'YYYY/MM/DD HH24:MI:SS');
to_timestamp
------------------------
2016-06-13 05:43:36-07 <— incorrect time
(1 row)

Ex.2: One whitespace before YYYY format string

postgres=# SELECT TO_TIMESTAMP('2016/06/13 15:43:36', ' YYYY/MM/DD HH24:MI:SS');
to_timestamp
------------------------------
0016-06-13 15:43:36-07:52:58 <— incorrect year
(1 row)

If there are one or more consecutive whitespace in the format, we should skip those as long as we could get an actual field.
Thoughts?
Thanks & Regards,
Amul Sul

From docs about to_timestamp() (
https://www.postgresql.org/docs/9.5/static/functions-formatting.html)
"These functions interpret input liberally, with minimal error checking.
While they produce valid output, the conversion can yield unexpected
results. For example, input to these functions is not restricted by
normal ranges, thus to_date('20096040','YYYYMMDD') returns 2014-01-17
rather than causing an error. Casting does not have this behavior."

And it wont stop on some simple whitespace. By using to_timestamp you
can get any output results by providing illegal input parameters values:

postgres=# SELECT TO_TIMESTAMP('2016-06-13 99:99:99', 'YYYYMMDD
HH24:MI:SS');
to_timestamp
------------------------
2016-01-06 14:40:39+03
(1 row)

Alex Ignatov
Postgres Professional: http://www.postgrespro.com
The Russian Postgres Company

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

#11amul sul
sul_amul@yahoo.co.in
In reply to: Alex Ignatov (#10)
Re: Bug in to_timestamp().

On Monday, 20 June 2016 8:53 PM, Alex Ignatov <a.ignatov@postgrespro.ru> wrote:

On 13.06.2016 18:52, amul sul wrote:

And it wont stop on some simple whitespace. By using to_timestamp you
can get any output results by providing illegal input parameters values:

postgres=# SELECT TO_TIMESTAMP('2016-06-13 99:99:99', 'YYYYMMDD
HH24:MI:SS');
to_timestamp
------------------------
2016-01-06 14:40:39+03

(1 row)

We do consume extra space from input string, but not if it is in format string, see below:

postgres=# SELECT TO_TIMESTAMP('2016-06-13 15:43:36', 'YYYY/MM/DD HH24:MI:SS');
to_timestamp
------------------------
2016-06-13 15:43:36-07
(1 row)

We should have same treatment for format string too.

Thoughts? Comments?

Regards,
Amul Sul

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

#12Bruce Momjian
bruce@momjian.us
In reply to: amul sul (#11)
Re: Bug in to_timestamp().

On Thu, Jun 23, 2016 at 07:41:26AM +0000, amul sul wrote:

On Monday, 20 June 2016 8:53 PM, Alex Ignatov <a.ignatov@postgrespro.ru> wrote:

On 13.06.2016 18:52, amul sul wrote:

And it wont stop on some simple whitespace. By using to_timestamp you
can get any output results by providing illegal input parameters values:

postgres=# SELECT TO_TIMESTAMP('2016-06-13 99:99:99', 'YYYYMMDD
HH24:MI:SS');
to_timestamp
------------------------
2016-01-06 14:40:39+03

(1 row)

We do consume extra space from input string, but not if it is in format string, see below:

postgres=# SELECT TO_TIMESTAMP('2016-06-13 15:43:36', 'YYYY/MM/DD HH24:MI:SS');
to_timestamp
------------------------
2016-06-13 15:43:36-07
(1 row)

We should have same treatment for format string too.

Thoughts? Comments?

Well, the user specifies the format string, while the input string comes
from the data, so I don't see having them behave the same as necessary.

--
Bruce Momjian <bruce@momjian.us> http://momjian.us
EnterpriseDB http://enterprisedb.com

+ As you are, so once was I. As I am, so you will be. +
+                     Ancient Roman grave inscription +

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

#13Alex Ignatov
a.ignatov@postgrespro.ru
In reply to: Bruce Momjian (#12)
Re: Bug in to_timestamp().

On 23.06.2016 16:30, Bruce Momjian wrote:

On Thu, Jun 23, 2016 at 07:41:26AM +0000, amul sul wrote:

On Monday, 20 June 2016 8:53 PM, Alex Ignatov <a.ignatov@postgrespro.ru> wrote:

On 13.06.2016 18:52, amul sul wrote:

And it wont stop on some simple whitespace. By using to_timestamp you
can get any output results by providing illegal input parameters values:
postgres=# SELECT TO_TIMESTAMP('2016-06-13 99:99:99', 'YYYYMMDD
HH24:MI:SS');
to_timestamp
------------------------
2016-01-06 14:40:39+03

(1 row)

We do consume extra space from input string, but not if it is in format string, see below:

postgres=# SELECT TO_TIMESTAMP('2016-06-13 15:43:36', 'YYYY/MM/DD HH24:MI:SS');
to_timestamp
------------------------
2016-06-13 15:43:36-07
(1 row)

We should have same treatment for format string too.

Thoughts? Comments?

Well, the user specifies the format string, while the input string comes
from the data, so I don't see having them behave the same as necessary.

To be honest they not just behave differently. to_timestamp is just
incorrectly handles input data and nothing else.There is no excuse for
such behavior:

postgres=# SELECT TO_TIMESTAMP('20:-16-06:13: 15_43:!36', 'YYYY/MM/DD
HH24:MI:SS');
to_timestamp
------------------------------
0018-08-05 13:15:43+02:30:17
(1 row)

Alex Ignatov
Postgres Professional: http://www.postgrespro.com
The Russian Postgres Company

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

#14David G. Johnston
david.g.johnston@gmail.com
In reply to: Alex Ignatov (#13)
Re: Bug in to_timestamp().

On Thu, Jun 23, 2016 at 12:16 PM, Alex Ignatov <a.ignatov@postgrespro.ru>
wrote:

On 23.06.2016 16:30, Bruce Momjian wrote:

On Thu, Jun 23, 2016 at 07:41:26AM +0000, amul sul wrote:

On Monday, 20 June 2016 8:53 PM, Alex Ignatov <a.ignatov@postgrespro.ru>
wrote:

On 13.06.2016 18:52, amul sul wrote:

And it wont stop on some simple whitespace. By using to_timestamp you
can get any output results by providing illegal input parameters values:
postgres=# SELECT TO_TIMESTAMP('2016-06-13 99:99:99', 'YYYYMMDD
HH24:MI:SS');
to_timestamp
------------------------
2016-01-06 14:40:39+03

(1 row)

We do consume extra space from input string, but not if it is in format
string, see below:

postgres=# SELECT TO_TIMESTAMP('2016-06-13 15:43:36', 'YYYY/MM/DD
HH24:MI:SS');
to_timestamp
------------------------
2016-06-13 15:43:36-07
(1 row)

We should have same treatment for format string too.

Thoughts? Comments?

Well, the user specifies the format string, while the input string comes
from the data, so I don't see having them behave the same as necessary.

To be honest they not just behave differently. to_timestamp is just
incorrectly handles input data and nothing else.There is no excuse for
such behavior:

postgres=# SELECT TO_TIMESTAMP('20:-16-06:13: 15_43:!36', 'YYYY/MM/DD
HH24:MI:SS');
to_timestamp
------------------------------
0018-08-05 13:15:43+02:30:17
(1 row)

T
​o be honest I don't see how this is relevant to quoted content. And
you've already made this point quite clearly - repeating it isn't
constructive. This behavior has existed for a long time and I don't see
that changing it is a worthwhile endeavor. I believe a new function is
required that has saner behavior. Otherwise given good input and a
well-formed parse string the function does exactly what it is designed to
do. Avoid giving it garbage and you will be fine. Maybe wrap the call to
the in a function that also checks for the expected layout and RAISE
EXCEPTION if it doesn't match.

​David J.

#15Alex Ignatov
a.ignatov@postgrespro.ru
In reply to: David G. Johnston (#14)
Re: Bug in to_timestamp().

On 23.06.2016 19:37, David G. Johnston wrote:

On Thu, Jun 23, 2016 at 12:16 PM, Alex Ignatov
<a.ignatov@postgrespro.ru <mailto:a.ignatov@postgrespro.ru>>wrote:

On 23.06.2016 16:30, Bruce Momjian wrote:

On Thu, Jun 23, 2016 at 07:41:26AM +0000, amul sul wrote:

On Monday, 20 June 2016 8:53 PM, Alex Ignatov
<a.ignatov@postgrespro.ru
<mailto:a.ignatov@postgrespro.ru>> wrote:

On 13.06.2016 18:52, amul sul wrote:

And it wont stop on some simple whitespace. By using
to_timestamp you
can get any output results by providing illegal input
parameters values:
postgres=# SELECT TO_TIMESTAMP('2016-06-13 99
<tel:2016-06-13%2099>:99:99', 'YYYYMMDD
HH24:MI:SS');
to_timestamp
------------------------
2016-01-06 14:40:39+03

(1 row)

We do consume extra space from input string, but not if it
is in format string, see below:

postgres=# SELECT TO_TIMESTAMP('2016-06-13 15:43:36',
'YYYY/MM/DD HH24:MI:SS');
to_timestamp
------------------------
2016-06-13 15:43:36-07
(1 row)

We should have same treatment for format string too.

Thoughts? Comments?

Well, the user specifies the format string, while the input
string comes
from the data, so I don't see having them behave the same as
necessary.

To be honest they not just behave differently. to_timestamp is
just incorrectly handles input data and nothing else.There is no
excuse for such behavior:

postgres=# SELECT TO_TIMESTAMP('20:-16-06:13: 15_43:!36',
'YYYY/MM/DD HH24:MI:SS');
to_timestamp
------------------------------
0018-08-05 13:15:43+02:30:17
(1 row)

T
​o be honest I don't see how this is relevant to quoted content. And
you've already made this point quite clearly - repeating it isn't
constructive. This behavior has existed for a long time and I don't
see that changing it is a worthwhile endeavor. I believe a new
function is required that has saner behavior. Otherwise given good
input and a well-formed parse string the function does exactly what it
is designed to do. Avoid giving it garbage and you will be fine.
Maybe wrap the call to the in a function that also checks for the
expected layout and RAISE EXCEPTION if it doesn't match.

​David J.

Arguing just like that one can say that we don't even need exception
like "division by zero". Just use well-formed numbers in denominator...
Input data sometimes can be generated automagically. Without exception
throwing debugging stored function containing to_timestamp can be painful.

#16David G. Johnston
david.g.johnston@gmail.com
In reply to: Alex Ignatov (#15)
Re: Bug in to_timestamp().

On Thursday, June 23, 2016, Alex Ignatov <a.ignatov@postgrespro.ru> wrote:

Arguing just like that one can say that we don't even need exception like
"division by zero". Just use well-formed numbers in denominator...
Input data sometimes can be generated automagically. Without exception
throwing debugging stored function containing to_timestamp can be painful.

to_timestamp with its present behavior is, IMO, a poorly designed function
that would never be accepted today. Concrete proposals for either fixing
or deprecating (or both) are welcome. Fixing it should not
cause unnecessary errors to be raised.

My main point is that I'm inclined to deprecate it.

My second point is if you are going to use this badly designed function you
need to protect yourself.

My understanding is that is not going to change for 9.6.

David J.

#17Robert Haas
robertmhaas@gmail.com
In reply to: David G. Johnston (#14)
Re: Bug in to_timestamp().

On Thu, Jun 23, 2016 at 12:37 PM, David G. Johnston
<david.g.johnston@gmail.com> wrote

To be honest I don't see how this is relevant to quoted content. And you've
already made this point quite clearly - repeating it isn't constructive.
This behavior has existed for a long time and I don't see that changing it
is a worthwhile endeavor. I believe a new function is required that has
saner behavior. Otherwise given good input and a well-formed parse string
the function does exactly what it is designed to do. Avoid giving it
garbage and you will be fine. Maybe wrap the call to the in a function that
also checks for the expected layout and RAISE EXCEPTION if it doesn't match.

Well, I think other people are allowed to disagree about whether
changing it is a worthwhile endeavor. I think there's an excellent
argument that the current behavior is stupid and broken and probably
nobody is intentionally relying on it.

Obviously, if somebody is relying on the existing behavior and we
change it and it breaks things, then that's bad, and a good argument
for worrying about backward-compatibility - e.g. by adding a new
function. But who would actually like the behavior that an extra
space in the format string causes non-whitespace characters to get
skipped? Next you'll be arguing that we can't fix a server crash
that's triggered by a certain query because somebody might be relying
on that query to restart the server...

--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

#18Robert Haas
robertmhaas@gmail.com
In reply to: David G. Johnston (#16)
Re: Bug in to_timestamp().

On Thu, Jun 23, 2016 at 1:12 PM, David G. Johnston
<david.g.johnston@gmail.com> wrote:

to_timestamp with its present behavior is, IMO, a poorly designed function
that would never be accepted today. Concrete proposals for either fixing or
deprecating (or both) are welcome. Fixing it should not cause unnecessary
errors to be raised.

Sheesh. Who put you in charge of this? You basically seem like you
are trying to shut up anyone who supports this change, and I don't
think that's right. Alex's opinion is just as valid as yours -
neither more nor less - and he has every right to express it without
being told by you that his emails are "not constructive".

My main point is that I'm inclined to deprecate it.

I can almost guarantee that would make a lot of users very unhappy.
This function is widely used.

My second point is if you are going to use this badly designed function you
need to protect yourself.

I agree that anyone using this function should test their format
strings carefully.

My understanding is that is not going to change for 9.6.

That's exactly what is under discussion here.

--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

#19Alvaro Herrera
alvherre@2ndquadrant.com
In reply to: David G. Johnston (#16)
Re: Bug in to_timestamp().

David G. Johnston wrote:

On Thursday, June 23, 2016, Alex Ignatov <a.ignatov@postgrespro.ru> wrote:

Arguing just like that one can say that we don't even need exception like
"division by zero". Just use well-formed numbers in denominator...
Input data sometimes can be generated automagically. Without exception
throwing debugging stored function containing to_timestamp can be painful.

to_timestamp with its present behavior is, IMO, a poorly designed function
that would never be accepted today.

I'm not sure about that.

to_timestamp was added to improve compatibility with Oracle, by commit
b866d2e2d794. I suppose the spec should follow what's documented here,

http://docs.oracle.com/cd/B19306_01/server.102/b14200/functions193.htm
http://docs.oracle.com/cd/B19306_01/server.102/b14200/sql_elements004.htm#i34924

and that wherever we deviate from that, is a bug that should be fixed.

--
�lvaro Herrera http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

#20Tom Lane
tgl@sss.pgh.pa.us
In reply to: Robert Haas (#18)
Re: Bug in to_timestamp().

Robert Haas <robertmhaas@gmail.com> writes:

On Thu, Jun 23, 2016 at 1:12 PM, David G. Johnston
<david.g.johnston@gmail.com> wrote:

My understanding is that is not going to change for 9.6.

That's exactly what is under discussion here.

I would definitely agree with David on that point. Making to_timestamp
noticeably better on this score seems like a nontrivial project, and
post-beta is not the time for that sort of thing, even if we had full
consensus on what to do. I'd suggest somebody work on a patch and put
it up for review in the next cycle.

Now, if you were to narrowly define the problem as "whether to skip
non-spaces for a space in the format", maybe that could be fixed
post-beta, but I think that's a wrongheaded approach. to_timestamp's
issues with input that doesn't match the format are far wider than that.
IMO we should try to resolve the whole problem with one coherent change,
not make incremental incompatible changes at the margins.

At the very least I'd want to see a thought-through proposal that
addresses all three of these interrelated points:

* what should a space in the format match
* what should a non-space, non-format-code character in the format match
* how should we handle fields that are not exactly the width suggested
by the format

regards, tom lane

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

#21Robert Haas
robertmhaas@gmail.com
In reply to: Tom Lane (#20)
#22David G. Johnston
david.g.johnston@gmail.com
In reply to: Robert Haas (#18)
#23Robert Haas
robertmhaas@gmail.com
In reply to: David G. Johnston (#22)
#24Tom Lane
tgl@sss.pgh.pa.us
In reply to: Robert Haas (#21)
#25Jim Nasby
Jim.Nasby@BlueTreble.com
In reply to: Alvaro Herrera (#19)
#26David G. Johnston
david.g.johnston@gmail.com
In reply to: Robert Haas (#23)
#27Alex Ignatov
a.ignatov@postgrespro.ru
In reply to: Tom Lane (#20)
#28Alex Ignatov
a.ignatov@postgrespro.ru
In reply to: Laurenz Albe (#8)
#29Steve Crawford
scrawford@pinpointresearch.com
In reply to: Alex Ignatov (#28)
#30Joshua D. Drake
jd@commandprompt.com
In reply to: Steve Crawford (#29)
#31Robert Haas
robertmhaas@gmail.com
In reply to: Steve Crawford (#29)
#32Tom Lane
tgl@sss.pgh.pa.us
In reply to: Robert Haas (#31)
#33Gavin Flower
GavinFlower@archidevsys.co.nz
In reply to: Robert Haas (#31)
#34Joshua D. Drake
jd@commandprompt.com
In reply to: Tom Lane (#32)
#35Steve Crawford
scrawford@pinpointresearch.com
In reply to: Joshua D. Drake (#34)
#36Robert Haas
robertmhaas@gmail.com
In reply to: Tom Lane (#32)
#37Bruce Momjian
bruce@momjian.us
In reply to: Robert Haas (#23)
#38Arthur Zakirov
a.zakirov@postgrespro.ru
In reply to: Tom Lane (#24)
#39Pavel Stehule
pavel.stehule@gmail.com
In reply to: Arthur Zakirov (#38)
#40Arthur Zakirov
a.zakirov@postgrespro.ru
In reply to: Pavel Stehule (#39)
#41amul sul
sul_amul@yahoo.co.in
In reply to: Arthur Zakirov (#40)
#42Robert Haas
robertmhaas@gmail.com
In reply to: amul sul (#41)
#43amul sul
sul_amul@yahoo.co.in
In reply to: Robert Haas (#42)
#44Arthur Zakirov
a.zakirov@postgrespro.ru
In reply to: Robert Haas (#42)
#45Arthur Zakirov
a.zakirov@postgrespro.ru
In reply to: Arthur Zakirov (#44)
#46amul sul
sul_amul@yahoo.co.in
In reply to: Arthur Zakirov (#45)
#47Robert Haas
robertmhaas@gmail.com
In reply to: amul sul (#46)
#48amul sul
sul_amul@yahoo.co.in
In reply to: Robert Haas (#47)
#49amul sul
sul_amul@yahoo.co.in
In reply to: Arthur Zakirov (#45)
#50Arthur Zakirov
a.zakirov@postgrespro.ru
In reply to: Robert Haas (#47)
#51amul sul
sul_amul@yahoo.co.in
In reply to: Arthur Zakirov (#50)
#52Arthur Zakirov
a.zakirov@postgrespro.ru
In reply to: Arthur Zakirov (#50)
#53amul sul
sul_amul@yahoo.co.in
In reply to: Arthur Zakirov (#52)
#54Arthur Zakirov
a.zakirov@postgrespro.ru
In reply to: Arthur Zakirov (#52)
#55Amul Sul
sulamul@gmail.com
In reply to: Arthur Zakirov (#54)
#56Arthur Zakirov
a.zakirov@postgrespro.ru
In reply to: Amul Sul (#55)
#57Amul Sul
sulamul@gmail.com
In reply to: Arthur Zakirov (#56)
#58Tom Lane
tgl@sss.pgh.pa.us
In reply to: Arthur Zakirov (#40)
#59Amul Sul
sulamul@gmail.com
In reply to: Tom Lane (#58)
#60Arthur Zakirov
a.zakirov@postgrespro.ru
In reply to: Amul Sul (#59)
#61Robert Haas
robertmhaas@gmail.com
In reply to: Amul Sul (#59)
#62Tom Lane
tgl@sss.pgh.pa.us
In reply to: Robert Haas (#61)
#63Amul Sul
sulamul@gmail.com
In reply to: Tom Lane (#62)
#64Michael Paquier
michael@paquier.xyz
In reply to: Amul Sul (#63)
#65Tom Lane
tgl@sss.pgh.pa.us
In reply to: Arthur Zakirov (#60)
#66Arthur Zakirov
a.zakirov@postgrespro.ru
In reply to: Tom Lane (#65)
#67Haribabu Kommi
kommi.haribabu@gmail.com
In reply to: Arthur Zakirov (#66)
#68Michael Paquier
michael@paquier.xyz
In reply to: Haribabu Kommi (#67)
#69Amul Sul
sulamul@gmail.com
In reply to: Michael Paquier (#68)
#70Arthur Zakirov
a.zakirov@postgrespro.ru
In reply to: Amul Sul (#69)
#71David Steele
david@pgmasters.net
In reply to: Arthur Zakirov (#70)
#72Tom Lane
tgl@sss.pgh.pa.us
In reply to: Arthur Zakirov (#66)
#73Arthur Zakirov
a.zakirov@postgrespro.ru
In reply to: Tom Lane (#72)
#74Tom Lane
tgl@sss.pgh.pa.us
In reply to: Arthur Zakirov (#73)
#75Arthur Zakirov
a.zakirov@postgrespro.ru
In reply to: Tom Lane (#74)
#76Michael Paquier
michael@paquier.xyz
In reply to: Arthur Zakirov (#75)
#77Robert Haas
robertmhaas@gmail.com
In reply to: Michael Paquier (#76)
#78Arthur Zakirov
a.zakirov@postgrespro.ru
In reply to: Robert Haas (#77)
#79Thomas Munro
thomas.munro@gmail.com
In reply to: Arthur Zakirov (#75)
#80Arthur Zakirov
a.zakirov@postgrespro.ru
In reply to: Thomas Munro (#79)
#81Dmitry Dolgov
9erthalion6@gmail.com
In reply to: Arthur Zakirov (#80)
#82Arthur Zakirov
a.zakirov@postgrespro.ru
In reply to: Dmitry Dolgov (#81)
#83Robert Haas
robertmhaas@gmail.com
In reply to: Dmitry Dolgov (#81)
#84Dmitry Dolgov
9erthalion6@gmail.com
In reply to: Robert Haas (#83)
#85Arthur Zakirov
a.zakirov@postgrespro.ru
In reply to: Dmitry Dolgov (#84)
#86Dmitry Dolgov
9erthalion6@gmail.com
In reply to: Arthur Zakirov (#85)
#87Dmitry Dolgov
9erthalion6@gmail.com
In reply to: Dmitry Dolgov (#86)
#88Arthur Zakirov
a.zakirov@postgrespro.ru
In reply to: Dmitry Dolgov (#87)
#89Dmitry Dolgov
9erthalion6@gmail.com
In reply to: Arthur Zakirov (#88)
#90Arthur Zakirov
a.zakirov@postgrespro.ru
In reply to: Dmitry Dolgov (#89)
#91Dmitry Dolgov
9erthalion6@gmail.com
In reply to: Arthur Zakirov (#90)
#92Dmitry Dolgov
9erthalion6@gmail.com
In reply to: Dmitry Dolgov (#91)
#93Arthur Zakirov
a.zakirov@postgrespro.ru
In reply to: Dmitry Dolgov (#92)
#94Dmitry Dolgov
9erthalion6@gmail.com
In reply to: Arthur Zakirov (#93)
#95Arthur Zakirov
a.zakirov@postgrespro.ru
In reply to: Dmitry Dolgov (#94)
#96Alexander Korotkov
aekorotkov@gmail.com
In reply to: Arthur Zakirov (#95)
#97Tom Lane
tgl@sss.pgh.pa.us
In reply to: Alexander Korotkov (#96)
#98Alexander Korotkov
aekorotkov@gmail.com
In reply to: Tom Lane (#97)
#99Alexander Korotkov
aekorotkov@gmail.com
In reply to: Alexander Korotkov (#98)
#100David G. Johnston
david.g.johnston@gmail.com
In reply to: Alexander Korotkov (#99)
#101Alexander Korotkov
aekorotkov@gmail.com
In reply to: David G. Johnston (#100)
#102Arthur Zakirov
a.zakirov@postgrespro.ru
In reply to: David G. Johnston (#100)
#103Arthur Zakirov
a.zakirov@postgrespro.ru
In reply to: Arthur Zakirov (#102)
#104Alexander Korotkov
aekorotkov@gmail.com
In reply to: Arthur Zakirov (#103)
#105Arthur Zakirov
a.zakirov@postgrespro.ru
In reply to: Alexander Korotkov (#104)
#106Alexander Korotkov
aekorotkov@gmail.com
In reply to: Arthur Zakirov (#105)
#107Alexander Korotkov
aekorotkov@gmail.com
In reply to: Alexander Korotkov (#106)
#108Arthur Zakirov
a.zakirov@postgrespro.ru
In reply to: Alexander Korotkov (#106)
#109Alexander Korotkov
aekorotkov@gmail.com
In reply to: Alexander Korotkov (#107)
#110Arthur Zakirov
a.zakirov@postgrespro.ru
In reply to: Alexander Korotkov (#109)
#111Alexander Korotkov
aekorotkov@gmail.com
In reply to: Arthur Zakirov (#110)
#112Liudmila Mantrova
l.mantrova@postgrespro.ru
In reply to: Alexander Korotkov (#109)
#113Alexander Korotkov
aekorotkov@gmail.com
In reply to: Liudmila Mantrova (#112)
#114David G. Johnston
david.g.johnston@gmail.com
In reply to: Alexander Korotkov (#111)
#115Alexander Korotkov
aekorotkov@gmail.com
In reply to: David G. Johnston (#114)
#116David G. Johnston
david.g.johnston@gmail.com
In reply to: Alexander Korotkov (#115)
#117Alexander Korotkov
aekorotkov@gmail.com
In reply to: David G. Johnston (#116)
#118Amul Sul
sulamul@gmail.com
In reply to: Alexander Korotkov (#117)
#119Alexander Korotkov
aekorotkov@gmail.com
In reply to: Amul Sul (#118)
#120Amul Sul
sulamul@gmail.com
In reply to: Alexander Korotkov (#119)
#121Alexander Korotkov
aekorotkov@gmail.com
In reply to: Amul Sul (#120)
#122Alexander Korotkov
aekorotkov@gmail.com
In reply to: Alexander Korotkov (#121)
#123Alexander Korotkov
aekorotkov@gmail.com
In reply to: Alexander Korotkov (#122)
#124Arthur Zakirov
a.zakirov@postgrespro.ru
In reply to: Alexander Korotkov (#123)
#125Alexander Korotkov
aekorotkov@gmail.com
In reply to: Arthur Zakirov (#124)
#126Tom Lane
tgl@sss.pgh.pa.us
In reply to: Alexander Korotkov (#125)
#127Prabhat Sahu
prabhat.sahu@enterprisedb.com
In reply to: Alexander Korotkov (#123)
#128Alexander Korotkov
aekorotkov@gmail.com
In reply to: Prabhat Sahu (#127)
#129Alexander Korotkov
aekorotkov@gmail.com
In reply to: Alexander Korotkov (#128)
#130Amul Sul
sulamul@gmail.com
In reply to: Alexander Korotkov (#129)
#131Amul Sul
sulamul@gmail.com
In reply to: Amul Sul (#130)
#132Alexander Korotkov
aekorotkov@gmail.com
In reply to: Amul Sul (#131)
#133Amul Sul
sulamul@gmail.com
In reply to: Alexander Korotkov (#132)
#134Alexander Korotkov
aekorotkov@gmail.com
In reply to: Amul Sul (#133)
#135Alexander Korotkov
aekorotkov@gmail.com
In reply to: Alexander Korotkov (#134)
#136Liudmila Mantrova
l.mantrova@postgrespro.ru
In reply to: Alexander Korotkov (#135)