BUG #11986: psql uses pager inside Emacs shell buffer (not a terminal)

On Mon, Nov 17, 2014 at 09:34:34AM +0000, eda@waniasset.com wrote:

The following bug has been logged on the website:

Bug reference: 11986
Logged by: Ed Avis
Email address: eda@waniasset.com
PostgreSQL version: 9.3.5
Operating system: Linux (Fedora 20)
Description:

GNU Emacs lets you run some commands inside Emacs - start it and do M-x
shell (that is, Alt-x, then type shell, then press enter).

This 'shell buffer' is interactive but it is not really a terminal, since it
doesn't support the normal terminal movement commands. If you try to use a
pager program such as 'more' or 'less' it tends to get in the way. Better
to print all of the output and let Emacs handle scrolling through it.

It would be friendler for psql to be aware of this and not use a pager if
running inside an Emacs shell buffer, or other dumb terminal: check if $TERM
is set to 'dumb'.

This is an interesting idea. What are the odds that a "dumb" terminal
is inside Emacs? Is there another environment variable set by Emacs
that we can check? 'set' should show you all the defined shell
variables.

--
Bruce Momjian <bruce@momjian.us> http://momjian.us
EnterpriseDB http://enterprisedb.com

+ Everyone has their own god. +

--
Sent via pgsql-bugs mailing list (pgsql-bugs@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-bugs

Bruce Momjian <bruce@momjian.us> writes:

On Mon, Nov 17, 2014 at 09:34:34AM +0000, eda@waniasset.com wrote:

It would be friendler for psql to be aware of this and not use a pager if
running inside an Emacs shell buffer, or other dumb terminal: check if $TERM
is set to 'dumb'.

This is an interesting idea. What are the odds that a "dumb" terminal
is inside Emacs? Is there another environment variable set by Emacs
that we can check?

A truly dumb terminal would be where you'd need a pager the most,
so I'm not enamored of this specific test proposal. If there's some
more-specific way to detect that stdout is an Emacs buffer, I'd be for it.

A quick check here shows a couple of different possibilities in the
environment seen in a shell buffer:

EMACS=t
INSIDE_EMACS=23.1.1,comint

I am not real sure of the properties of either of these variables
though, in particular not whether they are present in all versions
of Emacs. (The value of the second one seems to be an Emacs version
identifier, fwiw.) Another question is whether any effort gets made
to unset them in processes that are descendants of an Emacs but are
not connected to a shell-buffer pty.

regards, tom lane

--
Sent via pgsql-bugs mailing list (pgsql-bugs@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-bugs

On 2 December 2014 at 21:24, Tom Lane <tgl@sss.pgh.pa.us> wrote:

Bruce Momjian <bruce@momjian.us> writes:

On Mon, Nov 17, 2014 at 09:34:34AM +0000, eda@waniasset.com wrote:

It would be friendler for psql to be aware of this and not use a pager

if

running inside an Emacs shell buffer, or other dumb terminal: check if

$TERM

is set to 'dumb'.

This is an interesting idea. What are the odds that a "dumb" terminal
is inside Emacs? Is there another environment variable set by Emacs
that we can check?

A truly dumb terminal would be where you'd need a pager the most,
so I'm not enamored of this specific test proposal. If there's some
more-specific way to detect that stdout is an Emacs buffer, I'd be for it.

A quick check here shows a couple of different possibilities in the
environment seen in a shell buffer:

EMACS=t
INSIDE_EMACS=23.1.1,comint

I am not real sure of the properties of either of these variables
though, in particular not whether they are present in all versions
of Emacs. (The value of the second one seems to be an Emacs version
identifier, fwiw.) Another question is whether any effort gets made
to unset them in processes that are descendants of an Emacs but are
not connected to a shell-buffer pty.

regards, tom lane

Entertainingly, I just noticed this as an issue with a script I was running
today; when I ran it inside a M-x shell buffer, a psql session stopped,
waiting for input, which was rather annoying.

I wound up setting --pset pager=off, as an extra option to psql. That's
a strategy one might use to work around this.

It sure would be nice if there was an environment variable that
received an "EMACS_BUFFER=t" value or such; that ought to be
a better indicator that psql could let Emacs handle buffering for it.

I do see much the same options as you, albeit with a different version
number.

https://www.gnu.org/software/emacs/manual/html_node/emacs/Interactive-Shell.html

Disappointingly, eshell (a shell implemented in Emacs Lisp) doesn't set
either of those
environment variables.

However, from the documentation, it appears that INSIDE_EMACS is intended
to
provide the sort of functionality that the original poster was after.

BTW, "comint" is intended to indicate "command interpreter inside a
buffer", so
that value is pretty apropos.
--
When confronted by a difficult problem, solve it by reducing it to the
question, "How would the Lone Ranger handle this?"

A truly dumb terminal would be where you'd need a pager the most

That is true but how many are using teletypes these days? If the terminal type is 'dumb' that indicates for sure that some other program is processing the output. Nobody is running psql inside a terminal which is truly bereft of cursor movement commands - except perhaps on Windows, which can be detected separately.

______________________________________________________________________
This email has been scanned by the Symantec Email Security.cloud service.
For more information please visit http://www.symanteccloud.com
______________________________________________________________________

--
Sent via pgsql-bugs mailing list (pgsql-bugs@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-bugs

On Wed, Dec 3, 2014 at 03:48:45AM +0000, Ed Avis wrote:

A truly dumb terminal would be where you'd need a pager the most

That is true but how many are using teletypes these days? If the
terminal type is 'dumb' that indicates for sure that some other
program is processing the output. Nobody is running psql inside a
terminal which is truly bereft of cursor movement commands - except
perhaps on Windows, which can be detected separately.

Many shell scripts run from cron have a 'dumb' terminal value.

--
Bruce Momjian <bruce@momjian.us> http://momjian.us
EnterpriseDB http://enterprisedb.com

+ Everyone has their own god. +

--
Sent via pgsql-bugs mailing list (pgsql-bugs@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-bugs

'bruce@momjian.us' wrote:

On Wed, Dec 3, 2014 at 03:48:45AM +0000, Ed Avis wrote:

A truly dumb terminal would be where you'd need a pager the most

That is true but how many are using teletypes these days? If the
terminal type is 'dumb' that indicates for sure that some other
program is processing the output. Nobody is running psql inside a
terminal which is truly bereft of cursor movement commands - except
perhaps on Windows, which can be detected separately.

Many shell scripts run from cron have a 'dumb' terminal value.

Surely you don't want pagers running there either ...

--
�lvaro Herrera http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

--
Sent via pgsql-bugs mailing list (pgsql-bugs@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-bugs

On Fri, Dec 5, 2014 at 03:33:43PM -0300, Alvaro Herrera wrote:

'bruce@momjian.us' wrote:

On Wed, Dec 3, 2014 at 03:48:45AM +0000, Ed Avis wrote:

A truly dumb terminal would be where you'd need a pager the most

That is true but how many are using teletypes these days? If the
terminal type is 'dumb' that indicates for sure that some other
program is processing the output. Nobody is running psql inside a
terminal which is truly bereft of cursor movement commands - except
perhaps on Windows, which can be detected separately.

Many shell scripts run from cron have a 'dumb' terminal value.

Surely you don't want pagers running there either ...

Well, that's an interesting point. Does anyone object to turning off
'pager' for a 'dumb' terminal?

--
Bruce Momjian <bruce@momjian.us> http://momjian.us
EnterpriseDB http://enterprisedb.com

+ Everyone has their own god. +

--
Sent via pgsql-bugs mailing list (pgsql-bugs@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-bugs

Many shell scripts run from cron have a 'dumb' terminal value.

And these scripts should output directly and not run a pager -- so matching what is being suggested for 'dumb'.

In fact they already run without pager because their stdout is not a tty. What is being discussed here is what psql should do when it is writing to a tty, so interactive in some sense, but with TERM=dumb. Decades ago that would have meant a teleprinter terminal (or, perhaps, some ancient serial terminal or Minix 1.0). Nowadays it means running inside emacs or some other management program, so there is no need to run a pager.

______________________________________________________________________
This email has been scanned by the Symantec Email Security.cloud service.
For more information please visit http://www.symanteccloud.com
______________________________________________________________________

--
Sent via pgsql-bugs mailing list (pgsql-bugs@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-bugs

Ed Avis <eda@waniasset.com> writes:

Many shell scripts run from cron have a 'dumb' terminal value.

And these scripts should output directly and not run a pager -- so matching what is being suggested for 'dumb'.

In fact they already run without pager because their stdout is not a
tty.

Right. It only matters when we think stdout is a tty.

What is being discussed here is what psql should do when it is writing to a tty, so interactive in some sense, but with TERM=dumb. Decades ago that would have meant a teleprinter terminal (or, perhaps, some ancient serial terminal or Minix 1.0). Nowadays it means running inside emacs or some other management program, so there is no need to run a pager.

I am pretty hesitant about extending the emacs example to apply to *every*
context where TERM=dumb, especially given the complete lack of field
complaints in this area. I'm good with putting in a special hack for when
we're sure we're inside emacs, but I don't think an adequate case has been
made for assuming TERM=dumb means the opposite of what it appears to mean
on its face. Especially not if there won't be a way for the user to
override the choice.

regards, tom lane

--
Sent via pgsql-bugs mailing list (pgsql-bugs@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-bugs

OK, I think Tom Lane's position is reasonable. There could be other 'dumb' terminals out there we have not considered, and some of them might even run psql. I will check for a way to detect running inside emacs shell buffers.

______________________________________________________________________
This email has been scanned by the Symantec Email Security.cloud service.
For more information please visit http://www.symanteccloud.com
______________________________________________________________________

--
Sent via pgsql-bugs mailing list (pgsql-bugs@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-bugs

On Fri, Dec 5, 2014 at 8:08 PM, Ed Avis <eda@waniasset.com> wrote:

OK, I think Tom Lane's position is reasonable. There could be other 'dumb'
terminals out there we have not considered, and some of them might even run
psql. I will check for a way to detect running inside emacs shell buffers.

Although I am an emacs user, and have been bitten by the unwanted pager
problem inside shell buffers, I would vote against making psql detect
running inside emacs and change behaviour, specially considering it may
open the gates to also detect vi, netbeans, eclipse or whatever program can
run it as a subprocess, specially when any reasonable shell used inside the
buffer supports things like {alias psqlemacs='psql --pset=pager=off'} or
functions or scripts which easily avoid the problem.

Francisco Olarte.

Tom Lane <tgl <at> sss.pgh.pa.us> writes:

A quick check here shows a couple of different possibilities in the
environment seen in a shell buffer:

EMACS=t
INSIDE_EMACS=23.1.1,comint

Apparently EMACS=t is deprecated and INSIDE_EMACS is the one to use.
However, Emacs being Emacs, it has more than one way to run interactive
subprocesses. This bug report was about 'shell' mode, where a text editing
buffer provides a simple dumb terminal without cursor movement escape
sequences (and sets TERM=dumb). But there is also 'term', providing a
reasonably good terminal emulator inside Emacs. Inside that, a pager does
work (even sophisticated ones such as 'less') and it makes sense to use a
pager since you don't have a scrollable text buffer with all output, but
just a fixed-size terminal window.

'term' mode sets the environment variables to something like

TERM=eterm-color
INSIDE_EMACS=24.4.1,term:0.96

So if INSIDE_EMACS is to be used (rather than checking for TERM=dumb) then
perhaps it should look for the 'comint' string.

I appreciate the concern that this could open a Pandora's box of special
case checks for all sorts of terminal types, but this is only the psql
interactive shell we are talking about here, not the core database code.
And all that is being controlled is the relatively minor choice of whether
to run a pager program, not anything that affects database operations.
psql already contains lots of features which exist to make interactive use
more comfortable even at the expense of some complexity.

Let me see if I can make psql check INSIDE_EMACS on startup to see if it
is running inside 'comint' and if so turn off the pager (by default).

--
Ed Avis <eda@waniasset.com>

--
Sent via pgsql-bugs mailing list (pgsql-bugs@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-bugs

On Mon, Dec 8, 2014 at 12:39 PM, Ed Avis <eda@waniasset.com> wrote:

.....

I appreciate the concern that this could open a Pandora's box of special

case checks for all sorts of terminal types, but this is only the psql
interactive shell we are talking about here, not the core database code.
And all that is being controlled is the relatively minor choice of whether
to run a pager program, not anything that affects database operations.
psql already contains lots of features which exist to make interactive use
more comfortable even at the expense of some complexity.

I've been making some tests with emacs shell mode, and it seems less does
not work ( expected ) and more works partially ( it seems in shell mode
emacs does whole lines only, so \040 does not work ). The ONLY problem I
see with psql is that the more/less usage as default pager seems to be
undocumented ( I mean, I have not bien able to find it without resorting to
the source code ). My problems with emacs seem to be caused by my actions,
that is, invoking a program which needs a minimally smart terminal inside a
really dumb one, and are easily solved by not doing it ( by the existing
mechanism of psetting pager to off, which I think is easy enough for
someone who uses psql inside emacs shell buffers, as he must be familiar to
the problems with any terminal controlling program ) . Psql source is hairy
enough cattering for windows/*ix stuff, and anyone can write a wrapper to
test for the INSIDE_EMACS and invoke psql with the appropiate pset option.

And the psql interactive shell is not a joke, it is a big chunk of the
reasons why I use postgres.

Just my opinion / vote.

Regards.
Francisco Olarte.

Francisco Olarte wrote:

I've been making some tests with emacs shell mode, and it seems less does
not work ( expected ) and more works partially ( it seems in shell mode
emacs does whole lines only, so \040 does not work ). The ONLY problem I
see with psql is that the more/less usage as default pager seems to be
undocumented ( I mean, I have not bien able to find it without resorting to
the source code ).

Do we really hardcode more/less? I thought we just used whatever $PAGER
is set to. I just tried
PAGER=most psql

and indeed it uses most rather than less.

BTW another thing you could try is
export PAGER=cat
from within Emacs.

As for Win32 users,
export PAGER="format C:"
might be the thing to do ...

--
�lvaro Herrera http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

--
Sent via pgsql-bugs mailing list (pgsql-bugs@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-bugs

Alvaro Herrera <alvherre@2ndquadrant.com> writes:

Do we really hardcode more/less? I thought we just used whatever $PAGER
is set to.

Yeah, if PAGER is set. If not, we fall back to some wired-in default.
But I think the proposal here is to let an emacs-related environment
variable override whether we use a pager at all.

BTW another thing you could try is
export PAGER=cat
from within Emacs.

True. That would have the advantage of fixing more stuff than just psql,
so there's some attraction to doing that instead of hacking psql.

regards, tom lane

--
Sent via pgsql-bugs mailing list (pgsql-bugs@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-bugs

On Mon, Dec 8, 2014 at 11:39:12AM +0000, Ed Avis wrote:

Tom Lane <tgl <at> sss.pgh.pa.us> writes:

A quick check here shows a couple of different possibilities in the
environment seen in a shell buffer:

EMACS=t
INSIDE_EMACS=23.1.1,comint

Apparently EMACS=t is deprecated and INSIDE_EMACS is the one to use.
However, Emacs being Emacs, it has more than one way to run interactive
subprocesses. This bug report was about 'shell' mode, where a text editing
buffer provides a simple dumb terminal without cursor movement escape
sequences (and sets TERM=dumb). But there is also 'term', providing a
reasonably good terminal emulator inside Emacs. Inside that, a pager does
work (even sophisticated ones such as 'less') and it makes sense to use a
pager since you don't have a scrollable text buffer with all output, but
just a fixed-size terminal window.

'term' mode sets the environment variables to something like

TERM=eterm-color
INSIDE_EMACS=24.4.1,term:0.96

So if INSIDE_EMACS is to be used (rather than checking for TERM=dumb) then
perhaps it should look for the 'comint' string.

I appreciate the concern that this could open a Pandora's box of special
case checks for all sorts of terminal types, but this is only the psql
interactive shell we are talking about here, not the core database code.
And all that is being controlled is the relatively minor choice of whether
to run a pager program, not anything that affects database operations.
psql already contains lots of features which exist to make interactive use
more comfortable even at the expense of some complexity.

Based on your report, I think checking for INSIDE_EMACS being set _and_
TERM=dumb would be the best bet. Do you want me to write a patch for
you to test?

--
Bruce Momjian <bruce@momjian.us> http://momjian.us
EnterpriseDB http://enterprisedb.com

+ Everyone has their own god. +

--
Sent via pgsql-bugs mailing list (pgsql-bugs@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-bugs

On 9 December 2014 at 15:24, Bruce Momjian <bruce@momjian.us> wrote:

On Mon, Dec 8, 2014 at 11:39:12AM +0000, Ed Avis wrote:

Tom Lane <tgl <at> sss.pgh.pa.us> writes:

A quick check here shows a couple of different possibilities in the
environment seen in a shell buffer:

EMACS=t
INSIDE_EMACS=23.1.1,comint

Apparently EMACS=t is deprecated and INSIDE_EMACS is the one to use.
However, Emacs being Emacs, it has more than one way to run interactive
subprocesses. This bug report was about 'shell' mode, where a text

editing

buffer provides a simple dumb terminal without cursor movement escape
sequences (and sets TERM=dumb). But there is also 'term', providing a
reasonably good terminal emulator inside Emacs. Inside that, a pager

does

work (even sophisticated ones such as 'less') and it makes sense to use

a

pager since you don't have a scrollable text buffer with all output, but
just a fixed-size terminal window.

'term' mode sets the environment variables to something like

TERM=eterm-color
INSIDE_EMACS=24.4.1,term:0.96

So if INSIDE_EMACS is to be used (rather than checking for TERM=dumb)

then

perhaps it should look for the 'comint' string.

I appreciate the concern that this could open a Pandora's box of special
case checks for all sorts of terminal types, but this is only the psql
interactive shell we are talking about here, not the core database code.
And all that is being controlled is the relatively minor choice of

whether

to run a pager program, not anything that affects database operations.
psql already contains lots of features which exist to make interactive

use

more comfortable even at the expense of some complexity.

Based on your report, I think checking for INSIDE_EMACS being set _and_
TERM=dumb would be the best bet. Do you want me to write a patch for
you to test?

I'd be really inclined to search INSIDE_EMACS for "comint", as well.

That indicates that the comint mode is being used. <
http://www.emacswiki.org/emacs/ComintMode&gt;

That should indicate a scenario where pager should be suppressed.
TERM=dumb would tend to agree with that; I'm not sure it is helpful to
involve $TERM in the logic.

--
When confronted by a difficult problem, solve it by reducing it to the
question, "How would the Lone Ranger handle this?"

On Tue, Dec 9, 2014 at 03:53:52PM -0500, Christopher Browne wrote:

Based on your report, I think checking for INSIDE_EMACS being set _and_
TERM=dumb would be the best bet.� Do you want me to write a patch for
you to test?

I'd be really inclined to search INSIDE_EMACS for "comint", as well.

That indicates that the comint mode is being used.� <http://www.emacswiki.org/
emacs/ComintMode>

That should indicate a scenario where pager should be suppressed.��� TERM=dumb
would tend to agree with that; I'm not sure it is helpful to involve $TERM in
the logic.

I don't understand. Is COMINT an environment variable name? I don't see
it in INSIDE_EMACS:

TERM=term-color
INSIDE_EMACS=24.4.1,term:0.96

--
Bruce Momjian <bruce@momjian.us> http://momjian.us
EnterpriseDB http://enterprisedb.com

+ Everyone has their own god. +

--
Sent via pgsql-bugs mailing list (pgsql-bugs@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-bugs

On 9 December 2014 at 15:56, Bruce Momjian <bruce@momjian.us> wrote:

On Tue, Dec 9, 2014 at 03:53:52PM -0500, Christopher Browne wrote:

Based on your report, I think checking for INSIDE_EMACS being set _and_
TERM=dumb would be the best bet. Do you want me to write a patch for
you to test?

I'd be really inclined to search INSIDE_EMACS for "comint", as well.

That indicates that the comint mode is being used. <

http://www.emacswiki.org/

emacs/ComintMode>

That should indicate a scenario where pager should be suppressed.

TERM=dumb

would tend to agree with that; I'm not sure it is helpful to involve

$TERM in

the logic.

I don't understand. Is COMINT an environment variable name? I don't see
it in INSIDE_EMACS:

TERM=term-color
INSIDE_EMACS=24.4.1,term:0.96

In that case, you're running a shell in a buffer that is emulating a
terminal
(a not-dumb one), and I think you'll find that more, less, and most will
work fine
as pagers just as they would in an xterm.

I duplicate that by running M-x term.

I shouldn't think you'd be getting complaints from psql about the terminal
not being "fully functional" in your case. (I don't, running things in M-x
term)

Your case is, in effect, one where it would be fine for psql to NOT suppress
the pager. It's a perfectly good test case, but it's not the one that the
original poster was concerned about.

Try M-x shell; when I do so, I get...

$ echo $TERM
dumb
$ echo $INSIDE_EMACS
24.3.1,comint

And "psql -l" kind of blows up on me, as was the point of the thread...

$ psql -l
Line style (linestyle) is ascii.
WARNING: terminal is not fully functional
- (press RETURN)
[If I press return, I get a list of databases]
[Then, I get...]
(END)

And to get out of (END), I need to type "q", which gets captured by the dumb
terminal.

M-x term provides a shell with rather different behaviour.

It's a good idea to test the patch against both ways of running psql to make
sure things "play well" both ways.
--
When confronted by a difficult problem, solve it by reducing it to the
question, "How would the Lone Ranger handle this?"