Patch for - Allow server logs to be remotely read

Uh, I just added /contrib/adminpack a few weeks ago to CVS, which does
this, and more. Sorry I forgot to mark the TODO item as completed.

---------------------------------------------------------------------------

Dhanaraj M wrote:

The patch is attached for the following TODO item:

*Allow server logs to be remotely read.

Steps:

1. When the server starts (**pg_ctl start)**, the path of the postmaster file is stored
2. The user can access the logfile using the following 2 functions:
* pg_file_readlog(int linenumber) - Retrieves the string for the given line number
* **pg_file_countlog() - Retrieves the number of lines existing in the logfile*

I have implemented this based on the suggestions given in the hackers mailing list.
If you know a better way, please share it with me. Waiting for your reply.

Thanks
Dhanaraj

---------------------------(end of broadcast)---------------------------
TIP 5: don't forget to increase your free space map settings

--
Bruce Momjian http://candle.pha.pa.us
EnterpriseDB http://www.enterprisedb.com

+ If your life is a hard drive, Christ can be your backup. +

Bruce Momjian wrote:

Uh, I just added /contrib/adminpack a few weeks ago to CVS, which does
this, and more. Sorry I forgot to mark the TODO item as completed.

Huh, how do you read files with adminpack? I only see

Datum pg_file_write(PG_FUNCTION_ARGS);
Datum pg_file_rename(PG_FUNCTION_ARGS);
Datum pg_file_unlink(PG_FUNCTION_ARGS);
Datum pg_logdir_ls(PG_FUNCTION_ARGS);

--
Alvaro Herrera http://www.CommandPrompt.com/
The PostgreSQL Company - Command Prompt, Inc.

Alvaro Herrera wrote:

Bruce Momjian wrote:

Uh, I just added /contrib/adminpack a few weeks ago to CVS, which does
this, and more. Sorry I forgot to mark the TODO item as completed.

Huh, how do you read files with adminpack? I only see

Datum pg_file_write(PG_FUNCTION_ARGS);
Datum pg_file_rename(PG_FUNCTION_ARGS);
Datum pg_file_unlink(PG_FUNCTION_ARGS);
Datum pg_logdir_ls(PG_FUNCTION_ARGS);

Uh, README.adminpack shows:

int8 pg_catalog.pg_file_write(fname text, data text, append bool)
int8 pg_catalog.pg_file_read(fname text, data text, append bool)
bool pg_catalog.pg_file_rename(oldname text, newname text)
bool pg_catalog.pg_file_rename(oldname text, newname text, archivname text)
bool pg_catalog.pg_file_unlink(fname text)
bigint pg_catalog.pg_file_size(text)
int4 pg_catalog.pg_logfile_rotate()
setof record pg_catalog.pg_logdir_ls()

Is that wrong? Yes. Looking at the C file, I see what you mean. Let
me update the README.adminpack file. read_file is already in the
backend code, and was in 8.1.X too.

test=> \df *read*
List of functions
Schema | Name | Result data type | Argument data types
------------+--------------+------------------+----------------------
pg_catalog | loread | bytea | integer, integer
pg_catalog | pg_read_file | text | text, bigint, bigint
(2 rows)

The unlink part is part of the adminpack.

--
Bruce Momjian http://candle.pha.pa.us
EnterpriseDB http://www.enterprisedb.com

+ If your life is a hard drive, Christ can be your backup. +

Bruce Momjian wrote:

Alvaro Herrera wrote:

Bruce Momjian wrote:

Uh, I just added /contrib/adminpack a few weeks ago to CVS, which does
this, and more. Sorry I forgot to mark the TODO item as completed.

Huh, how do you read files with adminpack? I only see

Datum pg_file_write(PG_FUNCTION_ARGS);
Datum pg_file_rename(PG_FUNCTION_ARGS);
Datum pg_file_unlink(PG_FUNCTION_ARGS);
Datum pg_logdir_ls(PG_FUNCTION_ARGS);

Uh, README.adminpack shows:

int8 pg_catalog.pg_file_write(fname text, data text, append bool)
int8 pg_catalog.pg_file_read(fname text, data text, append bool)
bool pg_catalog.pg_file_rename(oldname text, newname text)
bool pg_catalog.pg_file_rename(oldname text, newname text, archivname text)
bool pg_catalog.pg_file_unlink(fname text)
bigint pg_catalog.pg_file_size(text)
int4 pg_catalog.pg_logfile_rotate()
setof record pg_catalog.pg_logdir_ls()

Is that wrong? Yes. Looking at the C file, I see what you mean. Let
me update the README.adminpack file. read_file is already in the
backend code, and was in 8.1.X too.

test=> \df *read*
List of functions
Schema | Name | Result data type | Argument data types
------------+--------------+------------------+----------------------
pg_catalog | loread | bytea | integer, integer
pg_catalog | pg_read_file | text | text, bigint, bigint
(2 rows)

The unlink part is part of the adminpack.

I see it now. They do some renaming of functions to match the use by
pgadmin, etc:

CREATE FUNCTION pg_catalog.pg_file_read(text, bigint, bigint)
RETURNS text
AS 'pg_read_file'
LANGUAGE INTERNAL VOLATILE STRICT;

'pg_file_read' becomes 'pg_read_file.' Anyway, the read part was
already in the backend, and the unlink part is new in adminpack.

--
Bruce Momjian http://candle.pha.pa.us
EnterpriseDB http://www.enterprisedb.com

+ If your life is a hard drive, Christ can be your backup. +

Bruce Momjian <pgman@candle.pha.pa.us> writes:

Is that wrong? Yes. Looking at the C file, I see what you mean. Let
me update the README.adminpack file. read_file is already in the
backend code, and was in 8.1.X too.

I wonder if we should take pg_read_file (and the rest of genfile.c)
back out of the backend and stick them into contrib/adminpack. The
argument for having them in the backend was always pretty weak to me.
In particular, a DBA who doesn't want them in his system for security
reasons has no simple way to get rid of them if they're in core, but
not installing a contrib module is easy enough.

regards, tom lane

Tom Lane wrote:

Bruce Momjian <pgman@candle.pha.pa.us> writes:

Is that wrong? Yes. Looking at the C file, I see what you mean. Let
me update the README.adminpack file. read_file is already in the
backend code, and was in 8.1.X too.

I wonder if we should take pg_read_file (and the rest of genfile.c)
back out of the backend and stick them into contrib/adminpack. The
argument for having them in the backend was always pretty weak to me.
In particular, a DBA who doesn't want them in his system for security
reasons has no simple way to get rid of them if they're in core, but
not installing a contrib module is easy enough.

I thought about that but what we have in the backend now is read-only
which basically could be done using COPY, so I don't see any security
value to moving them out. They are super-user only just like COPY.

--
Bruce Momjian http://candle.pha.pa.us
EnterpriseDB http://www.enterprisedb.com

+ If your life is a hard drive, Christ can be your backup. +

Bruce Momjian <pgman@candle.pha.pa.us> writes:

Tom Lane wrote:

I wonder if we should take pg_read_file (and the rest of genfile.c)
back out of the backend and stick them into contrib/adminpack.

I thought about that but what we have in the backend now is read-only
which basically could be done using COPY, so I don't see any security
value to moving them out. They are super-user only just like COPY.

The you-can-do-it-with-COPY argument doesn't apply to pg_ls_dir, nor to
pg_stat_file, and I find it unconvincing even for pg_read_file. COPY
isn't at all friendly for trying to read binary files, for instance.
Even for plain ASCII text you'd have to try to find a delimiter
character not present anywhere in the file, and backslashes in the file
would get corrupted.

But the basic point here is that someone who wants filesystem access
from the database is going to install adminpack anyway. Why should
someone who *doesn't* want filesystem access from the database be
forced to have some capabilities of that type installed anyway?

regards, tom lane

Tom Lane wrote:

Bruce Momjian <pgman@candle.pha.pa.us> writes:

Tom Lane wrote:

I wonder if we should take pg_read_file (and the rest of genfile.c)
back out of the backend and stick them into contrib/adminpack.

I thought about that but what we have in the backend now is read-only
which basically could be done using COPY, so I don't see any security
value to moving them out. They are super-user only just like COPY.

The you-can-do-it-with-COPY argument doesn't apply to pg_ls_dir, nor to
pg_stat_file, and I find it unconvincing even for pg_read_file. COPY
isn't at all friendly for trying to read binary files, for instance.
Even for plain ASCII text you'd have to try to find a delimiter
character not present anywhere in the file, and backslashes in the file
would get corrupted.

But the basic point here is that someone who wants filesystem access
from the database is going to install adminpack anyway. Why should
someone who *doesn't* want filesystem access from the database be
forced to have some capabilities of that type installed anyway?

Remember we went around and around on this with the pgAdmin guys, so you
are going to have to get their input. Also consider that pgAdmin might
be doing remote administration on a database it can't load shared
objects into, so having the read stuff always be there might help them.

I don't see anyone complaining about our read-only file access in the
backend, so I don't see a readon to remove it.

--
Bruce Momjian http://candle.pha.pa.us
EnterpriseDB http://www.enterprisedb.com

+ If your life is a hard drive, Christ can be your backup. +

Bruce Momjian wrote:

Tom Lane wrote:

Bruce Momjian <pgman@candle.pha.pa.us> writes:

Tom Lane wrote:

I wonder if we should take pg_read_file (and the rest of genfile.c)
back out of the backend and stick them into contrib/adminpack.

I thought about that but what we have in the backend now is read-only
which basically could be done using COPY, so I don't see any security
value to moving them out. They are super-user only just like COPY.

The you-can-do-it-with-COPY argument doesn't apply to pg_ls_dir, nor to
pg_stat_file, and I find it unconvincing even for pg_read_file. COPY
isn't at all friendly for trying to read binary files, for instance.
Even for plain ASCII text you'd have to try to find a delimiter
character not present anywhere in the file, and backslashes in the file
would get corrupted.

But the basic point here is that someone who wants filesystem access
from the database is going to install adminpack anyway. Why should
someone who *doesn't* want filesystem access from the database be
forced to have some capabilities of that type installed anyway?

Remember we went around and around on this with the pgAdmin guys, so you
are going to have to get their input. Also consider that pgAdmin might
be doing remote administration on a database it can't load shared
objects into, so having the read stuff always be there might help them.

I don't see anyone complaining about our read-only file access in the
backend, so I don't see a readon to remove it.

For the record, I am not happy with forcing this either. Providing it in
an optional module seems perfectly reasonable to me. I suppose an
alternative would be to turn the capability on or off via a (yet
another) switch.

cheers

andrew

Bruce Momjian wrote:

Uh, I just added /contrib/adminpack a few weeks ago to CVS, which does
this, and more. Sorry I forgot to mark the TODO item as completed.

---------------------------------------------------------------------------

1. int8 pg_catalog.pg_file_read(fname text, data text, append bool)

Though the above pg_file_read provides an interface to read the files,
the admin has to know the complete file path in order to read from them.
This can be simplified. As I have implemented. it does not take the file
name as a parameter.
It automatically stored the postmaster file path and reads whenever
required.

2. As suggested in the mailing list by Tom lane, this feature is
implemented in contrib pkg.
Hence, this feature is not forced on all.

Alvaro Herrera wrote:

Bruce Momjian wrote:

Uh, I just added /contrib/adminpack a few weeks ago to CVS, which does
this, and more. Sorry I forgot to mark the TODO item as completed.

Huh, how do you read files with adminpack?

try
select * from pg_logdir_ls() as (filetime timestamp, filename text)
and read the file you need.

Regards,
Andreas

Tom Lane wrote:

Bruce Momjian <pgman@candle.pha.pa.us> writes:

Tom Lane wrote:

I wonder if we should take pg_read_file (and the rest of genfile.c)
back out of the backend and stick them into contrib/adminpack.

I thought about that but what we have in the backend now is read-only
which basically could be done using COPY, so I don't see any security
value to moving them out. They are super-user only just like COPY.

The you-can-do-it-with-COPY argument doesn't apply to pg_ls_dir, nor to
pg_stat_file, and I find it unconvincing even for pg_read_file. COPY
isn't at all friendly for trying to read binary files, for instance.

pg_file_read returns text which isn't binary-friendly either.

Regards,
Andreas