very long record lines in expanded psql output

Hi

čt 5. 8. 2021 v 12:36 odesílatel Platon Pronko <platon7pronko@gmail.com>
napsal:

In expanded mode psql calculates the width of the longest field in all the
output rows,
and prints the header line according to it. This often results in record
header wrapping
over several lines (see example below).

This huge record header is printed the same way before each record, even
if all the fields
in current record are small and fit into terminal width. This often leads
to situations
when record header occupies the entirety of the screen, for all records,
even though there are
only a few records with huge fields in a record set.

Maybe we can avoid making the header line longer than terminal width for
\pset border 0
and \pset border 1? We already have terminal width calculated. Please see
attached a patch
with the proposed implementation.

Example output before the modification, in a terminal with a 100-column
width:

$ psql template1 -c "\x on" -c "\pset border 1;" -c "select n, repeat('x',
n) as long_column_name from unnest(array[42,210]) as n"
Expanded display is on.
Border style is 1.
─[ RECORD 1
]────┬──────────────────────────────────────────────────────────────────────────────────

────────────────────────────────────────────────────────────────────────────────────────────────────
─────────────────────────────
n │ 42
long_column_name │ xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
─[ RECORD 2
]────┼──────────────────────────────────────────────────────────────────────────────────

────────────────────────────────────────────────────────────────────────────────────────────────────
─────────────────────────────
n │ 210
long_column_name │
xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx

xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxxxxxxxxxxxx

And here's the same command after the patch:

$ psql template1 -c "\x on" -c "\pset border 1;" -c "select n, repeat('x',
n) as long_column_name from unnest(array[42,210]) as n"
Expanded display is on.
Border style is 1.
─[ RECORD 1
]────┬──────────────────────────────────────────────────────────────────────────────────
n │ 42
long_column_name │ xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
─[ RECORD 2
]────┼──────────────────────────────────────────────────────────────────────────────────
n │ 210
long_column_name │
xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx

xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxxxxxxxxxxxx

the length of lines should be consistent.

Your proposal breaks pspg

https://github.com/okbob/pspg

It can be separate option, but it should not be default

Pavel

Best regards,

Hi!

pspg looks nice :)

Your proposal breaks pspg

https://github.com/okbob/pspg

I tried the following (after the patch):

./psql template1 -c "select n, repeat('x', n) as long_column_name from unnest(array[42,210]) as n" | pspg

And it seems that pspg handled the situation without any issue. Also I tried inserting newlines into the output, and pspg still performed without issue. Can you give an example of scenario which should break after changing the record header length?

Best regards,
Platon Pronko

čt 5. 8. 2021 v 13:14 odesílatel Platon Pronko <platon7pronko@gmail.com>
napsal:

Hi!

pspg looks nice :)

Your proposal breaks pspg

https://github.com/okbob/pspg

I tried the following (after the patch):

./psql template1 -c "select n, repeat('x', n) as long_column_name from
unnest(array[42,210]) as n" | pspg

And it seems that pspg handled the situation without any issue. Also I
tried inserting newlines into the output, and pspg still performed without
issue. Can you give an example of scenario which should break after
changing the record header length?

I think so there can be two basic problems:

a) with line cursor

b) format detection - I try to detect the header line - and I expect this
line has the most length of all lines. I use a line with the same length as
the column's type info (data, border).

Did you test the wrapped format? It is working

\pset format wrapped
\x
select * from pg_class;

Regards

Pavel

Hi!

a) with line cursor

Tried in different configurations, seems that line cursor works fine.

b) format detection - I try to detect the header line - and I expect this
line has the most length of all lines. I use a line with the same length as
the column's type info (data, border).

I looked at pspg source, and here's what I found (please correct me if some/all
of my assumptions are wrong):

1. Input handling and format detection happens in table.c readfile().

2. There's some variables in DataDesc - maxx and maxbytes - that store the
longest line seen so far. But they are re-updated on each new row, so the fact
that header is shorter shouldn't affect them.

3. Expanded header detection is handled by is_expanded_header function. It has
ei_minx and ei_maxx return pointers, but when it is used from readfile() these
pointers are set to NULL in both cases - so header length is simply ignored.

Did you test the wrapped format? It is working

\pset format wrapped
\x
select * from pg_class;

There's no difference in outputs in wrapped format, so pspg behavior is also unaffected.

By the way, it seems that pspg recommends setting \pset border 2 anyway, so in vast
majority of cases there should be no possibility of difference at all - proposed patch
doesn't change output for \pset border 2 (because there's no sane way of making it
look okay in presence of long fields anyway).

Best regards,
Platon Pronko

čt 5. 8. 2021 v 15:30 odesílatel Platon Pronko <platon7pronko@gmail.com>
napsal:

Hi!

a) with line cursor

Tried in different configurations, seems that line cursor works fine.

b) format detection - I try to detect the header line - and I expect this
line has the most length of all lines. I use a line with the same length

as

the column's type info (data, border).

I looked at pspg source, and here's what I found (please correct me if
some/all
of my assumptions are wrong):

1. Input handling and format detection happens in table.c readfile().

2. There's some variables in DataDesc - maxx and maxbytes - that store the
longest line seen so far. But they are re-updated on each new row, so the
fact
that header is shorter shouldn't affect them.

3. Expanded header detection is handled by is_expanded_header function. It
has
ei_minx and ei_maxx return pointers, but when it is used from readfile()
these
pointers are set to NULL in both cases - so header length is simply
ignored.

The problem can be in translate_headline in bad desc->headline_size

Did you test the wrapped format? It is working

\pset format wrapped
\x
select * from pg_class;

There's no difference in outputs in wrapped format, so pspg behavior is
also unaffected.

By the way, it seems that pspg recommends setting \pset border 2 anyway,
so in vast
majority of cases there should be no possibility of difference at all -
proposed patch
doesn't change output for \pset border 2 (because there's no sane way of
making it
look okay in presence of long fields anyway).

Although pspg prefers border 2, it is working for all configurations now.

I think your proposal is not good, because it is breaking consistency and
if people want similar output, then they can use a wrapped format.

Regards

Pavel

Hi!

My apologies - I was using pspg incorrectly. If it's called via the pipe
then all the column wrapping is ignored, and that's why I couldn't reproduce
the issues. If instead pspg is used via "\setenv PAGER pspg", then I indeed can't
scroll the table horizontally more than 1 character, and that's definitely
broken.

I'll be using "\pset format wrapped" from now on, it's good enough. My only
issue would be with copying the long cell contents, but that's an uncommon
case and can be worked around without too much trouble.

Best regards,
Platon Pronko

čt 5. 8. 2021 v 16:32 odesílatel Platon Pronko <platon7pronko@gmail.com>
napsal:

Hi!

My apologies - I was using pspg incorrectly. If it's called via the pipe
then all the column wrapping is ignored, and that's why I couldn't
reproduce
the issues. If instead pspg is used via "\setenv PAGER pspg", then I
indeed can't
scroll the table horizontally more than 1 character, and that's definitely
broken.

I'll be using "\pset format wrapped" from now on, it's good enough. My only
issue would be with copying the long cell contents, but that's an uncommon
case and can be worked around without too much trouble.

pspg supports copy cell to clipboard - you can copy complete cell, although
some parts are invisible

https://github.com/okbob/pspg#export--clipboard

but I am not sure if it is working well in extended mode. This mode is not
extra tested in pspg

Regards

Pavel

Hi!

pspg supports copy cell to clipboard - you can copy complete cell

but I am not sure if it is working well in extended mode. This mode is not
extra tested in pspg

Thanks for the tip!

I tried it out, in non-extended mode copy indeed works well, in extended mode vertical cursor
selects both columns (and copies both), and in extended+wrapped mode copy includes wrapping
symbol.

Best regards,
Platon Pronko

čt 5. 8. 2021 v 16:50 odesílatel Platon Pronko <platon7pronko@gmail.com>
napsal:

Hi!

pspg supports copy cell to clipboard - you can copy complete cell

but I am not sure if it is working well in extended mode. This mode is

not

extra tested in pspg

Thanks for the tip!

I tried it out, in non-extended mode copy indeed works well, in extended
mode vertical cursor
selects both columns (and copies both), and in extended+wrapped mode copy
includes wrapping
symbol.

Yes, this is probably not finished yet. It was a surprise for me, so I am
able to use the clipboard from the terminal application, so support for
copy is relatively new feature. I'll check it when I have time.

Pavel

On 8/5/21 6:56 AM, Pavel Stehule wrote:

Hi

čt 5. 8. 2021 v 12:36 odesílatel Platon Pronko
<platon7pronko@gmail.com <mailto:platon7pronko@gmail.com>> napsal:

In expanded mode psql calculates the width of the longest field in
all the output rows,
and prints the header line according to it. This often results in
record header wrapping
over several lines (see example below).

This huge record header is printed the same way before each
record, even if all the fields
in current record are small and fit into terminal width. This
often leads to situations
when record header occupies the entirety of the screen, for all
records, even though there are
only a few records with huge fields in a record set.

Maybe we can avoid making the header line longer than terminal
width for \pset border 0
and \pset border 1? We already have terminal width calculated.
Please see attached a patch
with the proposed implementation.

Example output before the modification, in a terminal with a
100-column width:

$ psql template1 -c "\x on" -c "\pset border 1;" -c "select n,
repeat('x', n) as long_column_name from unnest(array[42,210]) as n"
Expanded display is on.
Border style is 1.
─[ RECORD 1
]────┬──────────────────────────────────────────────────────────────────────────────────
────────────────────────────────────────────────────────────────────────────────────────────────────
─────────────────────────────
n                │ 42
long_column_name │ xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
─[ RECORD 2
]────┼──────────────────────────────────────────────────────────────────────────────────
────────────────────────────────────────────────────────────────────────────────────────────────────
─────────────────────────────
n                │ 210
long_column_name │
xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxxxxxxxxxxxx

And here's the same command after the patch:

$ psql template1 -c "\x on" -c "\pset border 1;" -c "select n,
repeat('x', n) as long_column_name from unnest(array[42,210]) as n"
Expanded display is on.
Border style is 1.
─[ RECORD 1
]────┬──────────────────────────────────────────────────────────────────────────────────
n                │ 42
long_column_name │ xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
─[ RECORD 2
]────┼──────────────────────────────────────────────────────────────────────────────────
n                │ 210
long_column_name │
xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxxxxxxxxxxxx

the length of lines should be consistent.

Your proposal breaks pspg

https://github.com/okbob/pspg <https://github.com/okbob/pspg&gt;

It can be separate option, but it should not be default

I also find this annoying and would be happy to be rid of it.  I could
see setting an option like

\pset xheader_width column|page|nnn

cheers

andrew

--
Andrew Dunstan
EDB: https://www.enterprisedb.com

Hi!

I also find this annoying and would be happy to be rid of it.

Have you tried "\pset format wrapped"? Pavel suggested it, and it solved most of the problem for me, for example.

Best regards,
Platon Pronko

On 8/5/21 12:04 PM, Platon Pronko wrote:

Hi!

I also find this annoying and would be happy to be rid of it.

Have you tried "\pset format wrapped"? Pavel suggested it, and it
solved most of the problem for me, for example.

Yes, but it changes the data line output. Ideally, you should be able
to  modify these independently.

cheers

andrew

--
Andrew Dunstan
EDB: https://www.enterprisedb.com

Hi!

I also find this annoying and would be happy to be rid of it.

Have you tried "\pset format wrapped"? Pavel suggested it, and it
solved most of the problem for me, for example.

Yes, but it changes the data line output. Ideally, you should be able
to  modify these independently.

I agree, and I think this can be implemented, but I'm a bit afraid of
introducing an additional psql option (there's already quite a lot of them).
I suspect primary PostgreSQL maintainers won't be happy with such an approach.

Best regards,
Platon Pronko

čt 5. 8. 2021 v 18:48 odesílatel Platon Pronko <platon7pronko@gmail.com>
napsal:

Hi!

I also find this annoying and would be happy to be rid of it.

Have you tried "\pset format wrapped"? Pavel suggested it, and it
solved most of the problem for me, for example.

Yes, but it changes the data line output. Ideally, you should be able
to modify these independently.

I agree, and I think this can be implemented, but I'm a bit afraid of
introducing an additional psql option (there's already quite a lot of
them).
I suspect primary PostgreSQL maintainers won't be happy with such an
approach.

it can be a fully new format - designed for simple copy from terminal. Some
like

====== record: 10 ======
------------ proname ------------
left
------------ prosrc ------------
$lalalal
ewqrwqerw
ewqrwqerqrewq
$
===============

Regards

Pavel

Hi!

I also find this annoying and would be happy to be rid of it.

Have you tried "\pset format wrapped"? Pavel suggested it, and it
solved most of the problem for me, for example.

Yes, but it changes the data line output. Ideally, you should be able
to modify these independently.

I agree, and I think this can be implemented, but I'm a bit afraid of
introducing an additional psql option (there's already quite a lot of
them).
I suspect primary PostgreSQL maintainers won't be happy with such an
approach.

it can be a fully new format - designed for simple copy from terminal. Some
like

====== record: 10 ======
------------ proname ------------
left
------------ prosrc ------------
$lalalal
ewqrwqerw
ewqrwqerqrewq
$
===============

I don't think that would be a good idea. If somebody really just needs to copy,
then wrapping the query in "copy (...) to stdout" already works nicely,
no need to create a special mode just for that.
I think the question was more about being able to copy in an ad-hoc way,
in the middle of scrolling trough "normal" output.

Best regards,
Platon Pronko

čt 5. 8. 2021 v 20:30 odesílatel Platon Pronko <platon7pronko@gmail.com>
napsal:

Hi!

I also find this annoying and would be happy to be rid of it.

Have you tried "\pset format wrapped"? Pavel suggested it, and it
solved most of the problem for me, for example.

Yes, but it changes the data line output. Ideally, you should be able
to modify these independently.

I agree, and I think this can be implemented, but I'm a bit afraid of
introducing an additional psql option (there's already quite a lot of
them).
I suspect primary PostgreSQL maintainers won't be happy with such an
approach.

it can be a fully new format - designed for simple copy from terminal.

Some

like

====== record: 10 ======
------------ proname ------------
left
------------ prosrc ------------
$lalalal
ewqrwqerw
ewqrwqerqrewq
$
===============

I don't think that would be a good idea. If somebody really just needs to
copy,
then wrapping the query in "copy (...) to stdout" already works nicely,
no need to create a special mode just for that.

It is working well, but it is copy to file, not copy to clipboard.

I think the question was more about being able to copy in an ad-hoc way,

in the middle of scrolling trough "normal" output.

I understand your motivation, but I don't feel a significant benefit to
increase the complexity of the main format (more, when it breaks pspg,
without possibility to fix it). but it can depend on usual data, and it is
true, so when I use pspg, I don't use extended mode too often.

Hi!

I don't think that would be a good idea. If somebody really just needs to
copy,
then wrapping the query in "copy (...) to stdout" already works nicely,
no need to create a special mode just for that.

It is working well, but it is copy to file, not copy to clipboard.

Maybe I misunderstood - are you suggesting this a special mode in pspg? I thought
that you were suggesting to implement a mode in psql, something like
"\pset format_for_clipboard".

I think the question was more about being able to copy in an ad-hoc way,
in the middle of scrolling trough "normal" output.

I understand your motivation, but I don't feel a significant benefit to
increase the complexity of the main format (more, when it breaks pspg,
without possibility to fix it). but it can depend on usual data, and it is
true, so when I use pspg, I don't use extended mode too often.

To clarify: I'm not suggesting my patch anymore - it definitely breaks pspg and this
is completely unacceptable.
We're just discussing the possible alternate solutions, I think.

Best regards,
Platon Pronko

čt 5. 8. 2021 v 20:48 odesílatel Platon Pronko <platon7pronko@gmail.com>
napsal:

Hi!

I don't think that would be a good idea. If somebody really just needs

to

copy,
then wrapping the query in "copy (...) to stdout" already works nicely,
no need to create a special mode just for that.

It is working well, but it is copy to file, not copy to clipboard.

Maybe I misunderstood - are you suggesting this a special mode in pspg? I
thought
that you were suggesting to implement a mode in psql, something like
"\pset format_for_clipboard".

no, it was proposed for psql

I think the question was more about being able to copy in an ad-hoc way,
in the middle of scrolling trough "normal" output.

I understand your motivation, but I don't feel a significant benefit to
increase the complexity of the main format (more, when it breaks pspg,
without possibility to fix it). but it can depend on usual data, and it

is

true, so when I use pspg, I don't use extended mode too often.

To clarify: I'm not suggesting my patch anymore - it definitely breaks
pspg and this
is completely unacceptable.
We're just discussing the possible alternate solutions, I think.

yes

Pavel

it can be a fully new format - designed for simple copy from terminal. Some
like

====== record: 10 ======
------------ proname ------------
left
------------ prosrc ------------
$lalalal
ewqrwqerw
ewqrwqerqrewq
$
===============

no, it was proposed for psql

How is this better than "copy (...) to stdout"? E.g.:

$ copy (select * from pg_class limit 1) to stdout;
2619 pg_statistic 11 12016 0 10 2 2619 0 18 402 18 2840 t f p r 31 0 f f f f f t n f0 478 1 {postgres=arwdDxt/postgres} \N \N

You can still copy the necessary parts (using mouse selection) -
seems that it achieves the the same goal.

Best regards,
Platon Pronko

čt 5. 8. 2021 v 20:56 odesílatel Platon Pronko <platon7pronko@gmail.com>
napsal:

it can be a fully new format - designed for simple copy from terminal.

Some

like

====== record: 10 ======
------------ proname ------------
left
------------ prosrc ------------
$lalalal
ewqrwqerw
ewqrwqerqrewq
$
===============

no, it was proposed for psql

How is this better than "copy (...) to stdout"? E.g.:

$ copy (select * from pg_class limit 1) to stdout;
2619 pg_statistic 11 12016 0 10 2 2619 0
18 402 18 2840 t f p r 31
0 f f f f f t n f0
478 1 {postgres=arwdDxt/postgres} \N \N

You can still copy the necessary parts (using mouse selection) -
seems that it achieves the the same goal.

I think copy format can be pretty unclean when the line will be longer with
multiline values or with a lot of values. When the length of the line will
be longer than the terminal width, then searching the wanted column can be
pretty difficult.

When you know, so any value is on separate line or lines, then selection is
just primitive, and there is good enough information to find wanted data
quickly.

For this case I am working with hypothetical longer records with possible
multiline fields - just something that does chaos on the terminal, and you
want to select some part of data by mouse from the terminal screen.

Regards

Pavel

Platon Pronko wrote:

Maybe we can avoid making the header line longer than terminal width
for \pset border 0 and \pset border 1? We already have terminal
width calculated. Please see attached a patch with the proposed
implementation.

+1 for doing something against these long lines.

Rather than padding up to the terminal's width, we could simply end
the line after the "-[RECORD N]-" marker, with the idea that the
rest of it does not add much to readability anyway.
And when writing into a file as opposed to a terminal, getting rid of
these lines is useful too.

In that case, your example could be displayed like this:

=> \pset expanded on
=> \pset border 1

=> select n, repeat('x', n) as long_column_name from unnest(array[42,210])
as n;

-[ RECORD 1 ]-
n | 42
long_column_name | xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
-[ RECORD 2 ]-
n | 210
long_column_name |
xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx

=> \pset border 0
=> select n, repeat('x', n) as long_column_name from unnest(array[42,210])
as n;

* Record 1
n 42
long_column_name xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
* Record 2
n 210
long_column_name
xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx

Best regards,
--
Daniel Vérité
PostgreSQL-powered mailer: https://www.manitou-mail.org
Twitter: @DanielVerite

On 8/5/21 2:25 PM, Pavel Stehule wrote:

čt 5. 8. 2021 v 18:48 odesílatel Platon Pronko
<platon7pronko@gmail.com <mailto:platon7pronko@gmail.com>> napsal:

Hi!

I also find this annoying and would be happy to be rid of it.

Have you tried "\pset format wrapped"? Pavel suggested it, and it
solved most of the problem for me, for example.

Yes, but it changes the data line output. Ideally, you should be

able

to  modify these independently.

I agree, and I think this can be implemented, but I'm a bit afraid of
introducing an additional psql option (there's already quite a lot
of them).
I suspect primary PostgreSQL maintainers won't be happy with such
an approach.

it can be a fully new format - designed for simple copy from terminal.
Some like

====== record: 10 ======
------------ proname ------------
left
------------ prosrc  ------------
$lalalal
ewqrwqerw
ewqrwqerqrewq
$
===============

I think that's really a different idea. The original proposal was simply
to deal with insanely long header lines. This seems like massive scope
creep.

cheers

andrew

--
Andrew Dunstan
EDB: https://www.enterprisedb.com

On 8/5/21 12:48 PM, Platon Pronko wrote:

Hi!

I also find this annoying and would be happy to be rid of it.

Have you tried "\pset format wrapped"? Pavel suggested it, and it
solved most of the problem for me, for example.

Yes, but it changes the data line output. Ideally, you should be able
to  modify these independently.

I agree, and I think this can be implemented, but I'm a bit afraid of
introducing an additional psql option (there's already quite a lot of
them).
I suspect primary PostgreSQL maintainers won't be happy with such an
approach.

I think I qualify as one of those ... :-)

cheers

andrew

--
Andrew Dunstan
EDB: https://www.enterprisedb.com

Hi!

I also find this annoying and would be happy to be rid of it.

Have you tried "\pset format wrapped"? Pavel suggested it, and it
solved most of the problem for me, for example.

Yes, but it changes the data line output. Ideally, you should be able
to  modify these independently.

I agree, and I think this can be implemented, but I'm a bit afraid of
introducing an additional psql option (there's already quite a lot of
them).
I suspect primary PostgreSQL maintainers won't be happy with such an
approach.

I think I qualify as one of those ... :-)

Sorry, I'm new here, don't know who's who :)

I'll start working on a new patch then. A couple questions about specifics:

1. Can we add "expanded" in the option name, like "xheader_expanded_width"?
I think adjusting the header row width doesn't make sense on any other modes,
and placing that in the option name makes intent a bit clearer.

2. What was "column" option in your original suggestion supposed to do?
("\pset xheader_width column|page|nnn")

3. Should we bother with using this option when in "\pset border 2" mode?
I can do it for consistency, but it will still look bad.

Best regards,
Platon Pronko

On 8/7/21 10:56 AM, Platon Pronko wrote:

Hi!

I also find this annoying and would be happy to be rid of it.

Have you tried "\pset format wrapped"? Pavel suggested it, and it
solved most of the problem for me, for example.

Yes, but it changes the data line output. Ideally, you should be able
to  modify these independently.

I agree, and I think this can be implemented, but I'm a bit afraid of
introducing an additional psql option (there's already quite a lot of
them).
I suspect primary PostgreSQL maintainers won't be happy with such an
approach.

I think I qualify as one of those ... :-)

Sorry, I'm new here, don't know who's who :)

No problem. Welcome! We're always very glad to see new contributors.

I'll start working on a new patch then. A couple questions about
specifics:

1. Can we add "expanded" in the option name, like
"xheader_expanded_width"?
I think adjusting the header row width doesn't make sense on any other
modes,
and placing that in the option name makes intent a bit clearer.

"xheader" was meant to be shorthand for "expanded output header"

2. What was "column" option in your original suggestion supposed to do?
("\pset xheader_width column|page|nnn")

It's meant to say don't print anything past the column spec, e.g.:

-[ RECORD 1 ]----+
n | 42
long_column_name | xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
-[ RECORD 2 ]----+
n | 210
long_column_name | xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx

3. Should we bother with using this option when in "\pset border 2" mode?
I can do it for consistency, but it will still look bad.

Probably not, but since I never use it I'll let others who do weigh in
on the subject.

cheers

andrew

--
Andrew Dunstan
EDB: https://www.enterprisedb.com

čt 5. 8. 2021 v 17:13 odesílatel Pavel Stehule <pavel.stehule@gmail.com>
napsal:

čt 5. 8. 2021 v 16:50 odesílatel Platon Pronko <platon7pronko@gmail.com>
napsal:

Hi!

pspg supports copy cell to clipboard - you can copy complete cell

but I am not sure if it is working well in extended mode. This mode is

not

extra tested in pspg

Thanks for the tip!

I tried it out, in non-extended mode copy indeed works well, in extended
mode vertical cursor
selects both columns (and copies both), and in extended+wrapped mode copy
includes wrapping
symbol.

Yes, this is probably not finished yet. It was a surprise for me, so I am
able to use the clipboard from the terminal application, so support for
copy is relatively new feature. I'll check it when I have time.

I fixed this issue. Now, the export should work in wrapped mode too (master
branch).

Regards

Pavel

Hi!

Please find attached the patch implementing the proposed changes.
I hope I didn't forget anything (docs, tab completion, comments, etc),
if I did forget something please tell me and I'll fix it.

Best regards,
Platon Pronko

Hi!

Apparently I did forget something, and that's the patch itself :)
Thanks for Justin Pryzby for pointing this out.

Attaching the patch now.

Best regards,
Platon Pronko

On 8/23/21 2:00 PM, Platon Pronko wrote:

Hi!

Apparently I did forget something, and that's the patch itself :)
Thanks for Justin Pryzby for pointing this out.

Attaching the patch now.

Please add it to the commitfest, the next one starts in about a week.

cheers

andrew

--
Andrew Dunstan
EDB: https://www.enterprisedb.com

Hi!

Done, here's the link: https://commitfest.postgresql.org/34/3295/

Best regards,
Platon Pronko

On 8/23/21 2:00 PM, Platon Pronko wrote:

Hi!

Apparently I did forget something, and that's the patch itself :)
Thanks for Justin Pryzby for pointing this out.

Attaching the patch now.

(Please avoid top-posting on PostgreSQL lists)

This patch seems basically sound. A couple of things:

1. It's not following project indentation style (BSD brace placement)

2. It would possibly be better to pass the relevant parts of the options
to print_aligned_vertical_line() rather than the whole options
structure. It feels odd to pass both that and opt_border.

cheers

andrew

--
Andrew Dunstan
EDB: https://www.enterprisedb.com

On 2021-09-23 22:28, Andrew Dunstan wrote:

2. It would possibly be better to pass the relevant parts of the options
to print_aligned_vertical_line() rather than the whole options
structure. It feels odd to pass both that and opt_border.

What do you think about doing it the other way around - passing only whole
options structure? That way we will roll 4 parameters (opt_border, printTextFormat,
and two xheader ones) into only one argument.
This increases code coupling a bit, but I'm not sure if that's relevant here.

Best regards,
Platon Pronko

On 9/24/21 12:49 AM, Platon Pronko wrote:

On 2021-09-23 22:28, Andrew Dunstan wrote:

2. It would possibly be better to pass the relevant parts of the options
to print_aligned_vertical_line() rather than the whole options
structure. It feels odd to pass both that and opt_border.

What do you think about doing it the other way around - passing only
whole
options structure? That way we will roll 4 parameters (opt_border,
printTextFormat,
and two xheader ones) into only one argument.
This increases code coupling a bit, but I'm not sure if that's
relevant here.

Sure, as long as it doesn't result in duplicated computation.

cheers

andrew

--
Andrew Dunstan
EDB: https://www.enterprisedb.com

On 2021-09-24 14:42, Andrew Dunstan wrote:

On 9/24/21 12:49 AM, Platon Pronko wrote:

On 2021-09-23 22:28, Andrew Dunstan wrote:

2. It would possibly be better to pass the relevant parts of the options
to print_aligned_vertical_line() rather than the whole options
structure. It feels odd to pass both that and opt_border.

What do you think about doing it the other way around - passing only
whole
options structure? That way we will roll 4 parameters (opt_border,
printTextFormat,
and two xheader ones) into only one argument.
This increases code coupling a bit, but I'm not sure if that's
relevant here.

Sure, as long as it doesn't result in duplicated computation.

Hi!

Please find attached the updated patch - with fixed braces and
adjusted parameters to print_aligned_vertical_line().

Best regards,
Platon Pronko

On Sun, Oct 3, 2021 at 5:57 AM Platon Pronko <platon7pronko@gmail.com>
wrote:

On 2021-09-24 14:42, Andrew Dunstan wrote:

On 9/24/21 12:49 AM, Platon Pronko wrote:

On 2021-09-23 22:28, Andrew Dunstan wrote:

2. It would possibly be better to pass the relevant parts of the

options

to print_aligned_vertical_line() rather than the whole options
structure. It feels odd to pass both that and opt_border.

What do you think about doing it the other way around - passing only
whole
options structure? That way we will roll 4 parameters (opt_border,
printTextFormat,
and two xheader ones) into only one argument.
This increases code coupling a bit, but I'm not sure if that's
relevant here.

Sure, as long as it doesn't result in duplicated computation.

Hi!

Please find attached the updated patch - with fixed braces and
adjusted parameters to print_aligned_vertical_line().

Best regards,
Platon Pronko

Hi,

+ pg_log_error("\\pset: allowed xheader_width values are full
(default), column, page, or an number specifying exact width.");

an number specifying -> a number specifying

Cheers

Hi,

+ pg_log_error("\\pset: allowed xheader_width values are full
(default), column, page, or an number specifying exact width.");

an number specifying -> a number specifying

Cheers

Fixed, attaching the updated patch. Thank you!

Best regards,
Platon Pronko

On 2021-10-03 Su 16:03, Platon Pronko wrote:

Hi,

+               pg_log_error("\\pset: allowed xheader_width values
are full
(default), column, page, or an number specifying exact width.");

  an number specifying -> a number specifying

Cheers

Fixed, attaching the updated patch. Thank you!

Committed. There were a couple of bits missing, which I filled in, and I
changed the behaviour when the option is given without an argument, so
that instead of resetting it the current value is shown, similarly to
how most pset options work.

cheers

andrew

--
Andrew Dunstan
EDB: https://www.enterprisedb.com

Re: Andrew Dunstan

+�������������� pg_log_error("\\pset: allowed xheader_width values are full
(default), column, page, or an number specifying exact width.");

Committed. There were a couple of bits missing, which I filled in, and I
changed the behaviour when the option is given without an argument, so
that instead of resetting it the current value is shown, similarly to
how most pset options work.

Thanks for implementing this! This has been #1 on my PG annoyances
list since forever. I had even tried to patch it 10� years ago, but
ever managed to get that patch to submittable quality.

Now, is there a chance that we could make this the default? Having
psql print a whole screen of ------------ minus characters doesn't
seem very useful as default behavior.

I see upthread that Pavel was objecting to that because pspg doesn't
understand it. But since the expanded format is a one-column output,
and the only thing pspg needs to know is the maxium line length, I'd
think pspg could just look at each line, and update the maximum as it
reads the input anyway.

Can we set 'page' as default?

Christoph

út 30. 8. 2022 v 15:49 odesílatel Christoph Berg <myon@debian.org> napsal:

Re: Andrew Dunstan

+ pg_log_error("\\pset: allowed xheader_width values

are full

(default), column, page, or an number specifying exact width.");

Committed. There were a couple of bits missing, which I filled in, and I
changed the behaviour when the option is given without an argument, so
that instead of resetting it the current value is shown, similarly to
how most pset options work.

Thanks for implementing this! This has been #1 on my PG annoyances
list since forever. I had even tried to patch it 10½ years ago, but
ever managed to get that patch to submittable quality.

Now, is there a chance that we could make this the default? Having
psql print a whole screen of ------------ minus characters doesn't
seem very useful as default behavior.

I see upthread that Pavel was objecting to that because pspg doesn't
understand it. But since the expanded format is a one-column output,
and the only thing pspg needs to know is the maxium line length, I'd
think pspg could just look at each line, and update the maximum as it
reads the input anyway.

pspg requires all lines to have the same width. It can do some corrections
- but it is hard to detect wanted differences or just plain text format.

can be nice to have the first invisible row with some information about
used formatting. pspg does some heuristic but this code is not nice and it
is fragile.

Regards

Pavel

Re: Pavel Stehule

pspg requires all lines to have the same width. It can do some corrections
- but it is hard to detect wanted differences or just plain text format.

can be nice to have the first invisible row with some information about
used formatting. pspg does some heuristic but this code is not nice and it
is fragile.

I like pspg and use it myself, but I don't think a tool that does the
right thing by hiding a full screen of ---- from the user should
hinder making the same progress in psql with a simple pager.

Christoph

út 30. 8. 2022 v 16:36 odesílatel Christoph Berg <myon@debian.org> napsal:

Re: Pavel Stehule

pspg requires all lines to have the same width. It can do some

corrections

- but it is hard to detect wanted differences or just plain text format.

can be nice to have the first invisible row with some information about
used formatting. pspg does some heuristic but this code is not nice and

it

is fragile.

I like pspg and use it myself, but I don't think a tool that does the
right thing by hiding a full screen of ---- from the user should
hinder making the same progress in psql with a simple pager.

ASCII allows to set some metadata, that should be invisible in all
correctly implemented pagers.

út 30. 8. 2022 v 16:49 odesílatel Pavel Stehule <pavel.stehule@gmail.com>
napsal:

út 30. 8. 2022 v 16:36 odesílatel Christoph Berg <myon@debian.org> napsal:

Re: Pavel Stehule

pspg requires all lines to have the same width. It can do some

corrections

- but it is hard to detect wanted differences or just plain text format.

can be nice to have the first invisible row with some information about
used formatting. pspg does some heuristic but this code is not nice and

it

is fragile.

I like pspg and use it myself, but I don't think a tool that does the
right thing by hiding a full screen of ---- from the user should
hinder making the same progress in psql with a simple pager.

ASCII allows to set some metadata, that should be invisible in all
correctly implemented pagers.

or these parameters can be sent by pager's command line or via some
environment variable. Currently there are only two pagers on the world that
support tabular format, and both are created against psql (pspg and ov), so
we can define our own protocol. Surely - pspg will have heuristic forever,
because I want to support psql, mysql and many others. But it can be fine
to switch to some more robust mode. It can be interesting for continuous
load via pipe.

Regards

Pavel

On 2022-08-30 Tu 10:55, Pavel Stehule wrote:

út 30. 8. 2022 v 16:49 odesílatel Pavel Stehule
<pavel.stehule@gmail.com> napsal:

út 30. 8. 2022 v 16:36 odesílatel Christoph Berg <myon@debian.org>
napsal:

Re: Pavel Stehule

pspg requires all lines to have the same width. It can do

some corrections

- but it is hard to detect wanted differences or just plain

text format.

can be nice to have the first invisible row with some

information about

used formatting. pspg does some heuristic but this code is

not nice and it

is fragile.

I like pspg and use it myself, but I don't think a tool that
does the
right thing by hiding a full screen of ---- from the user should
hinder making the same progress in psql with a simple pager.

ASCII allows to set some metadata, that should be invisible in all
correctly implemented pagers.

or these parameters can be sent by pager's command line or via some
environment variable. Currently there are only two pagers on the world
that support tabular format, and both are created against psql (pspg
and ov), so we can define our own protocol. Surely - pspg will have
heuristic forever, because I want to support psql, mysql and many
others. But it can be fine to switch to some more robust mode. It can
be interesting for continuous load via pipe.  

I'm somewhat sympathetic to Christoph's position.

Surely pspg could itself issue

   \pset xheader_width full

at the start of a session.

cheers

andrew

--

Andrew Dunstan
EDB: https://www.enterprisedb.com

On 2022-Jul-25, Andrew Dunstan wrote:

Committed. There were a couple of bits missing, which I filled in, and I
changed the behaviour when the option is given without an argument, so
that instead of resetting it the current value is shown, similarly to
how most pset options work.

I was translating the new messages introduced by this commit, and
decided to unify them. While doing so, I notice that the feature
misbehaves when you give it a string value that doesn't match any valid
value: it just resets the value to 0, which is entirely the wrong thing.
For other settings, we first verify that the given value is valid, and
only then we change the setting.

So I came up with the attached. While at it, I noticed that we have
other uses of atoi() there, most notably pager_min_lines. My first
impulse was to just to do the same as for xheader_width, namely to test
whether the atoid result is zero, and not change in that case. But then
I noticed we're already using variables.c facilities, so I decided to do
likewise; this results in simpler code. So we're now stricter, because
variables.c rejects trailing junk after the number. (123asd was
previously parsed as 123, now it's an error.)

There remain atoi uses in this code, and aside from the atoi()-induced
behavior of making any non-number input set it to 0, it does stuff like

=# \pset border 65538asd
Border style is 2.

because border style is short int, so this value overflows it. Same
happens with 'columns'. However, this is all very old code and nobody
seems to have complained.

--
Álvaro Herrera 48°01'N 7°57'E — https://www.EnterpriseDB.com/
"¿Cómo puedes confiar en algo que pagas y que no ves,
y no confiar en algo que te dan y te lo muestran?" (Germán Poo)