Adminpack removal

I agree that removing adminpack was a bit of a surprise for me as
well. At first I assumed that it was just moved into the core to
accompany the file and directory *reading* functions, until I found
the release notes mentioning that now one of the users of adminpack
does not need it and so it is dropped.

The easy and currently supported-in-core way to do file manipulation
is using pl/pythonu or pl/perlu but I agree that it is an overkill if
all you need is a little bit of file manipulation.

Best Regards
Hannu

On Thu, 27 Jun 2024, 07:34 Philippe BEAUDOIN, <phb.emaj@free.fr> wrote:

Hi,

I have just tested PG17 beta1 with the E-Maj solution I maintain. The
only issue I found is the removal of the adminpack contrib.

In the emaj extension, which is the heart of the solution, and which is
written in plpgsql, the code just uses the pg_file_unlink() function to
automatically remove files produced by COPY TO statements when no rows
have been written. In some specific use cases, it avoids the user to get
a few interesting files among numerous empty files in a directory. I
have found a workaround. That's a little bit ugly, but it works. So this
is not blocking for me.

FYI, the project's repo is on github (https://github.com/dalibo/emaj),
which was supposed to be scanned to detect potential adminpack usages.

The extension at first glance doesn't currently seem to depend on
adminpack: it is not included in the control file as dependency, and
has not been included as a dependency since the creation of that file.

Where else would you expect us to search for dependencies?

Kind regards,

Matthias van de Meent

Le 27/06/2024 à 10:38, Matthias van de Meent a écrit :

On Thu, 27 Jun 2024, 07:34 Philippe BEAUDOIN, <phb.emaj@free.fr> wrote:

Hi,

I have just tested PG17 beta1 with the E-Maj solution I maintain. The
only issue I found is the removal of the adminpack contrib.

In the emaj extension, which is the heart of the solution, and which is
written in plpgsql, the code just uses the pg_file_unlink() function to
automatically remove files produced by COPY TO statements when no rows
have been written. In some specific use cases, it avoids the user to get
a few interesting files among numerous empty files in a directory. I
have found a workaround. That's a little bit ugly, but it works. So this
is not blocking for me.

FYI, the project's repo is on github (https://github.com/dalibo/emaj),
which was supposed to be scanned to detect potential adminpack usages.

The extension at first glance doesn't currently seem to depend on
adminpack: it is not included in the control file as dependency, and
has not been included as a dependency since the creation of that file.

You are right. Even before the adminpack usage removal, the extension
was not listed as prerequisite in the control file. In fact, I
introduced a new E-Maj feature in the version of last automn, that used
the adminpack extension in one specific case. But the user may not
install adminpack. In such a case, the feature was limited and a warning
message told the user why it reached the limitation. I was waiting for
some feedbacks before possibly adding adminpack as a real prerequisite.

Where else would you expect us to search for dependencies?

The word "adminpack" can be found in the sql source file
(sql/emaj--4.4.0.sql), and in 2 documentation source files (in
docs/en/*.rst).

The pg_file_unlink() function name can be found in the same sql source file.

But, I understand that looking for simple strings in all types of files
in a lot of repo is costly and may report a lot of noise.

More broadly, my feeling is that just looking at public repositories is
not enough. The Postgres features usage can be found in:

- public tools, visible in repo (in github, gitlab and some other
platforms) ;

- softwares from commercial vendors, so in close source ;

- and a huge number of applications developed in all organizations, and
that are not public.

So just looking in public repo covers probably less than 1% of the code.
However, this may give a first idea, especialy if a feature use is
already detected.

In this "adminpack" case, it may be interesting to distinguish the
pg_logdir_ls() function, which covers a very specific administration
feature, and the other functions, which are of a general interest. It
wouldn't be surprising that pg_logdir_ls() be really obsolete now that
it is not used by pgAdmin anymore, and thus could be removed if nobody
complains about that. May be the others functions could be directly
integrated into the core (or left in adminpack, with the pgAdmin
reference removed from the documentation).

Kind Regards.

Philippe.

On 28 Jun 2024, at 09:06, Philippe BEAUDOIN <phb.emaj@free.fr> wrote:

So just looking in public repo covers probably less than 1% of the code. However, this may give a first idea, especialy if a feature use is already detected.

Searching for anything on Github is essentially a dead end since it reports so
many duplicates in forks etc. That being said, I did a lot of searching and
browsing to find users [0]/messages/by-id/B07CC211-DE35-4AC5-BD4E-0C6466700B06@yesql.se, but came up empty (apart from forks which already
maintain their own copy). A more targeted search is the Debian Code search
which at the time of removal (and well before then) showed zero occurrences of
adminpack functions in any packaged software, and no extensions which had
adminpack as a dependency. While not an exhaustive search by any means, it
does provide a good hint.

Since you list no other extensions using adminpack to support keeping it, I
assume you also didn't find any when searching?

--
Daniel Gustafsson

[0]: /messages/by-id/B07CC211-DE35-4AC5-BD4E-0C6466700B06@yesql.se

Le 01/07/2024 à 10:07, Daniel Gustafsson a écrit :

On 28 Jun 2024, at 09:06, Philippe BEAUDOIN <phb.emaj@free.fr> wrote:
So just looking in public repo covers probably less than 1% of the code. However, this may give a first idea, especialy if a feature use is already detected.

Searching for anything on Github is essentially a dead end since it reports so
many duplicates in forks etc. That being said, I did a lot of searching and
browsing to find users [0], but came up empty (apart from forks which already
maintain their own copy). A more targeted search is the Debian Code search
which at the time of removal (and well before then) showed zero occurrences of
adminpack functions in any packaged software, and no extensions which had
adminpack as a dependency. While not an exhaustive search by any means, it
does provide a good hint.

Since you list no other extensions using adminpack to support keeping it, I
assume you also didn't find any when searching?

I just said that there are much much more code in private repos (so not
analyzable) than in the public ones.