MOVE LAST: why?

Tom Lane wrote:

2002-11-12 19:44 momjian

* doc/src/sgml/ref/move.sgml, src/backend/commands/portalcmds.c,
src/backend/executor/execMain.c, src/backend/parser/gram.y,
src/backend/parser/keywords.c, src/backend/tcop/utility.c,
src/include/commands/portalcmds.h, src/include/nodes/parsenodes.h:
Make MOVE/FETCH 0 actually move/fetch 0. Add MOVE LAST to move to
end of cursor.

Refresh my memory: what is the point of inventing an additional LAST
keyword, when the behavior is exactly the same as MOVE ALL ?

SQL compatibility, per Peter.

-- 
  Bruce Momjian                        |  http://candle.pha.pa.us
  pgman@candle.pha.pa.us               |  (610) 359-1001
  +  If your life is a hard drive,     |  13 Roberts Road
  +  Christ can be your backup.        |  Newtown Square, Pennsylvania 19073

Bruce Momjian <pgman@candle.pha.pa.us> writes:

Tom Lane wrote:

Refresh my memory: what is the point of inventing an additional LAST
keyword, when the behavior is exactly the same as MOVE ALL ?

SQL compatibility, per Peter.

Oh, I see. But then really it should be documented as a FETCH keyword,
not only a MOVE keyword. Will fix.

regards, tom lane

Tom Lane wrote:

Bruce Momjian <pgman@candle.pha.pa.us> writes:

Tom Lane wrote:

Refresh my memory: what is the point of inventing an additional LAST
keyword, when the behavior is exactly the same as MOVE ALL ?

SQL compatibility, per Peter.

Oh, I see. But then really it should be documented as a FETCH keyword,
not only a MOVE keyword. Will fix.

IIRC *FETCH LAST* doesn't mean *FETCH ALL*.

In addition *FETCH 0* seems to be changed to mean
*FETCH RELATIVE 0* currently. Is it reasonable ?
*FETCH n* never means *FETCH RELATIVE n*.

regards,
Hiroshi Inoue
http://w2422.nsk.ne.jp/~inoue/

Hiroshi Inoue <Inoue@tpf.co.jp> writes:

IIRC *FETCH LAST* doesn't mean *FETCH ALL*.

SQL92 says

ii) If the <fetch orientation> implicitly or explicitly spec-
ifies NEXT, specifies ABSOLUTE or RELATIVE with K greater
than N, or specifies LAST, then CR is positioned after the
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
last row.
^^^^^^^^

So as far as the ending cursor position is concerned, LAST agrees with
ALL. It looks to me like the SQL definition only contemplates returning
a single row, but it's less than clear *which* row they mean for LAST.

In addition *FETCH 0* seems to be changed to mean
*FETCH RELATIVE 0* currently. Is it reasonable ?

Sure. FETCH n in Postgres has always corresponded to FETCH RELATIVE n.

regards, tom lane

Tom Lane wrote:

Sure. FETCH n in Postgres has always corresponded to FETCH RELATIVE n.

IIRC in SQL standard FETCH retrieves rows one by one.

regards,
Hiroshi Inoue
http://w2422.nsk.ne.jp/~inoue/

Hiroshi Inoue <Inoue@tpf.co.jp> writes:

Tom Lane wrote:

Sure. FETCH n in Postgres has always corresponded to FETCH RELATIVE n.

IIRC in SQL standard FETCH retrieves rows one by one.

Yes, Postgres' idea of FETCH is only weakly related to the spec's idea.
But I believe you get similar results if you consider only the row last
returned by our FETCH.

regards, tom lane

Tom Lane wrote:

Bruce Momjian <pgman@candle.pha.pa.us> writes:

Tom Lane wrote:

Refresh my memory: what is the point of inventing an additional LAST
keyword, when the behavior is exactly the same as MOVE ALL ?

SQL compatibility, per Peter.

Oh, I see. But then really it should be documented as a FETCH keyword,
not only a MOVE keyword. Will fix.

Yes. SQL standard doesn't have move, but it has FETCH LAST, so we
borrowed it for MOVE.
-- 
  Bruce Momjian                        |  http://candle.pha.pa.us
  pgman@candle.pha.pa.us               |  (610) 359-1001
  +  If your life is a hard drive,     |  13 Roberts Road
  +  Christ can be your backup.        |  Newtown Square, Pennsylvania 19073

Tom Lane wrote:

Hiroshi Inoue <Inoue@tpf.co.jp> writes:

Tom Lane wrote:

Sure. FETCH n in Postgres has always corresponded to FETCH RELATIVE n.

IIRC in SQL standard FETCH retrieves rows one by one.

Yes, Postgres' idea of FETCH is only weakly related to the spec's idea.
But I believe you get similar results if you consider only the row last
returned by our FETCH.

FETCH n is a PostgreSQL's extention to retrieve multiple
rows by one FETCH not related to FETCH RELATIVE at all.

FETCH LAST should return the last one row.
FETCH RELATIVE m should return a row after skipping
m rows if we follow the SQL standard and so the current
implementation of FETCH RELATIVE is broken.

regards,
Hiroshi Inoue
http://w2422.nsk.ne.jp/~inoue/

Hiroshi Inoue <Inoue@tpf.co.jp> writes:

FETCH LAST should return the last one row.

That's not clear to me. Generally, I would think the cursor should
remain positioned on whatever row is returned, but the spec clearly says
that the final cursor position after FETCH LAST is *after* the last row.
Nor do I see where exactly it says that the last row is the one to
return in this case; the spec seems to treat LAST the same as PRIOR, so
that the *first* row encountered in the movement direction might be the
one to return. Can you disentangle the spec wording for me?

FETCH RELATIVE m should return a row after skipping
m rows if we follow the SQL standard and so the current
implementation of FETCH RELATIVE is broken.

No objection to that here. Are you volunteering to make it do that?

regards, tom lane

Tom Lane wrote:

Hiroshi Inoue <Inoue@tpf.co.jp> writes:

FETCH LAST should return the last one row.

That's not clear to me. Generally, I would think the cursor should
remain positioned on whatever row is returned, but the spec clearly says
that the final cursor position after FETCH LAST is *after* the last row.

In SQL99 the spec you referred to seems the ii) of b) which
begins with the word *Otherwise*. I see the following before it.

a) If K is greater than 0 (zero) and not greater than N, then CR
is positioned on the K-th row of Tt and the corresponding row
of T. That row becomes the current row of CR.

Then I'm also suspicious if MOVE LAST should mean MOVE ALL.

FETCH RELATIVE m should return a row after skipping
m rows if we follow the SQL standard and so the current
implementation of FETCH RELATIVE is broken.

No objection to that here. Are you volunteering to make it do that?

I'm suspicios if we should implement scrollable cursors
with the combination of the current MOVE and FETCH implemen-
tation. For example the backwards FETCH operation for group
nodes isn't implemented properly yet(maybe). Should we fix
it or take another way ?

regards,
Hiroshi Inoue
http://w2422.nsk.ne.jp/~inoue/

FETCH LAST should return the last one row.
FETCH RELATIVE m should return a row after skipping
m rows if we follow the SQL standard and so the current
implementation of FETCH RELATIVE is broken.

Yes, the syntax could probably be
FETCH [n] RELATIVE m
to keep the functionality to fetch n rows at once and not only one
after skipping m rows.

Andreas

I just received and applied the following ODBC patch from Wes Griffin.
It fixes a foreign key query I broke during my cleanups.

Can we make a new release of ODBC? Thanks.

-- 
  Bruce Momjian                        |  http://candle.pha.pa.us
  pgman@candle.pha.pa.us               |  (610) 359-1001
  +  If your life is a hard drive,     |  13 Roberts Road
  +  Christ can be your backup.        |  Newtown Square, Pennsylvania 19073

Hiroshi Inoue <Inoue@tpf.co.jp> writes:

I'm suspicios if we should implement scrollable cursors
with the combination of the current MOVE and FETCH implemen-
tation. For example the backwards FETCH operation for group
nodes isn't implemented properly yet(maybe).

Yeah, backwards scan is not implemented for quite a large number of plan
node types :-(. I am not sure that it is practical to fix them all.
I have been toying with the notion of making cursors on complex plans
safe for FETCH BACKWARD by sticking a MATERIAL node atop the plan, if
the top plan node isn't one that can handle backwards scan.

The trouble with this of course is that one of the main things people
like to use cursors for is huge result sets, and materializing those is
the last thing you want to do :-(

regards, tom lane

I said:

Yeah, backwards scan is not implemented for quite a large number of plan
node types :-(. I am not sure that it is practical to fix them all.
I have been toying with the notion of making cursors on complex plans
safe for FETCH BACKWARD by sticking a MATERIAL node atop the plan, if
the top plan node isn't one that can handle backwards scan.

I forgot to mention plan B: make use of ReScan. This could work like
so:

1. Cursor keeps track of row number (number of rows it's fetched).

2. To scan backwards when top plan type doesn't handle it, rewind all
the way with ReScan, then move forward the appropriate number of rows.

This would avoid any added overhead in the case where a backwards move
is never requested, and it also would support MOVE BACKWARD ALL quite
efficiently (much more so than now). On the other hand, it'd really
suck if the user asks for backwards scan from a point far into the
output.

Perhaps we could do something with a hybrid technique: don't materialize
the cursor output unless user actually asks for backwards scan. If he
does, then create a tuplestore and put the data into it (rescanning the
query output to do so), and finally supply the tuples from the tuplestore.

regards, tom lane

Tom Lane wrote:

Perhaps we could do something with a hybrid technique: don't materialize
the cursor output unless user actually asks for backwards scan.

Or we can check the existence of SCROLL keyword which is
currently ignored. In the first place SQL standard only
allows NEXT fetch unless SCROLL is specified.

regards,
Hiroshi Inoue
http://w2422.nsk.ne.jp/~inoue/

Tom Lane kirjutas N, 09.01.2003 kell 04:05:

I said:

Yeah, backwards scan is not implemented for quite a large number of plan
node types :-(. I am not sure that it is practical to fix them all.
I have been toying with the notion of making cursors on complex plans
safe for FETCH BACKWARD by sticking a MATERIAL node atop the plan,

How much work would it be do the MATERIAL node so that it is calculated
at the time of initial forward scan (i.e. FETCH/MOVE) ?

if the top plan node isn't one that can handle backwards scan.

I forgot to mention plan B: make use of ReScan. This could work like
so:

1. Cursor keeps track of row number (number of rows it's fetched).

2. To scan backwards when top plan type doesn't handle it, rewind all
the way with ReScan, then move forward the appropriate number of rows.

This would avoid any added overhead in the case where a backwards move
is never requested, and it also would support MOVE BACKWARD ALL quite
efficiently (much more so than now). On the other hand, it'd really
suck if the user asks for backwards scan from a point far into the
output.

Perhaps we could do something with a hybrid technique: don't materialize
the cursor output unless user actually asks for backwards scan. If he
does, then create a tuplestore and put the data into it (rescanning the
query output to do so), and finally supply the tuples from the tuplestore.

How hard would it be to save "snapshots" of scan state at certain
places, say at each 1000 tuples, so that a full re-scan is not
neccessary ?

regards, tom lane

---------------------------(end of broadcast)---------------------------
TIP 1: subscribe and unsubscribe commands go to majordomo@postgresql.org

--
Hannu Krosing <hannu@tm.ee>

Zeugswetter Andreas SB SD wrote:

FETCH LAST should return the last one row.
FETCH RELATIVE m should return a row after skipping
m rows if we follow the SQL standard and so the current
implementation of FETCH RELATIVE is broken.

Yes, the syntax could probably be
FETCH [n] RELATIVE m
to keep the functionality to fetch n rows at once and not only one
after skipping m rows.

What I've thought is
FETCH RELATIVE m [n].
Either is OK to me.

regards,
Hiroshi Inoue
http://w2422.nsk.ne.jp/~inoue/

Bruce Momjian wrote:

I just received and applied the following ODBC patch from Wes Griffin.
It fixes a foreign key query I broke during my cleanups.

Can we make a new release of ODBC? Thanks.

Sorry I'm not ready.

regards,
Hiroshi Inoue
http://w2422.nsk.ne.jp/~inoue/

Tom Lane wrote:

Hiroshi Inoue <Inoue@tpf.co.jp> writes:

I'm suspicios if we should implement scrollable cursors
with the combination of the current MOVE and FETCH implemen-
tation. For example the backwards FETCH operation for group
nodes isn't implemented properly yet(maybe).

Yeah, backwards scan is not implemented for quite a large number of plan
node types :-(. I am not sure that it is practical to fix them all.
I have been toying with the notion of making cursors on complex plans
safe for FETCH BACKWARD by sticking a MATERIAL node atop the plan, if
the top plan node isn't one that can handle backwards scan.

The trouble with this of course is that one of the main things people
like to use cursors for is huge result sets, and materializing those is
the last thing you want to do :-(

Honestly I'm not so enthusiastic about scrollable cursors.
Even though PostgreSQL provides an efficient scrollable
cursor, I would use it little unless it could survive
across transactions.

Anyway it's too bad that FETCH LAST means FETCH ALL.

regards,
Hiroshi Inoue
http://w2422.nsk.ne.jp/~inoue/