make_ctags: use -I option to ignore pg_node_attr macro

Hello

On 2022-Oct-07, Yugo NAGATA wrote:

I found that tag files generated by src/tools/make_ctags
doesn't include some keywords, that were field names of node
structs, for example norm_select in RestrictInfo. Such fields
are defined with pg_node_attr macro that was introduced
recently, like:

Selectivity norm_selec pg_node_attr(equal_ignore);

In this case, pg_node_attr is mistakenly interpreted to be
the name of the field. So, I propose to use -I option of ctags
to ignore the marco name. Attached is a patch to do it.

I've wondered if there's anybody that uses this script. I suppose if
you're reporting this problem then it has at least one user, and
therefore worth fixing.

If we do patch it, how about doing some more invasive surgery and
bringing it to the XXI century? I think the `find` line is a bit lame:

# this is outputting the tags into the file 'tags', and appending
find `pwd`/ -type f -name '*.[chyl]' -print |
-	xargs ctags -a -f tags "$FLAGS"
+	xargs ctags -a -f tags "$FLAGS" -I "$IGNORE_LIST"

especially because it includes everything in tmp_install, which pollutes
the tag list.

In my own tags script I just call "ctags -R", and I feed cscope with
these find lines

(find $SRCDIR \( -name tmp_install -prune -o -name tmp_check -prune \) -o \( -name "*.[chly]" -o -iname "*makefile*" -o -name "*.mk" -o -name "*.in" -o -name "*.sh" -o -name "*.sgml" -o -name "*.sql" -o -name "*.p[lm]" \) -type f -print ; \
find $BUILDDIR \( -name tmp_install -prune \) -o \( -name \*.h -a -type f \) -print )

which seems to give decent results. (Nowadays I wonder if it'd be
better to exclude the "*_d.h" files from the builddir.)
(I wonder why don't I have a prune for .git ...)

--
Álvaro Herrera 48°01'N 7°57'E — https://www.EnterpriseDB.com/
¡Ay, ay, ay! Con lo mucho que yo lo quería (bis)
se fue de mi vera ... se fue para siempre, pa toíta ... pa toíta la vida
¡Ay Camarón! ¡Ay Camarón! (Paco de Lucía)

Hello

On Mon, 10 Oct 2022 12:04:22 +0200
Alvaro Herrera <alvherre@alvh.no-ip.org> wrote:

Hello

On 2022-Oct-07, Yugo NAGATA wrote:

I found that tag files generated by src/tools/make_ctags
doesn't include some keywords, that were field names of node
structs, for example norm_select in RestrictInfo. Such fields
are defined with pg_node_attr macro that was introduced
recently, like:

Selectivity norm_selec pg_node_attr(equal_ignore);

In this case, pg_node_attr is mistakenly interpreted to be
the name of the field. So, I propose to use -I option of ctags
to ignore the marco name. Attached is a patch to do it.

I've wondered if there's anybody that uses this script. I suppose if
you're reporting this problem then it has at least one user, and
therefore worth fixing.

Yeah, I am a make_ctags user, there may be few users though....

If we do patch it, how about doing some more invasive surgery and
bringing it to the XXI century? I think the `find` line is a bit lame:

# this is outputting the tags into the file 'tags', and appending
find `pwd`/ -type f -name '*.[chyl]' -print |
-	xargs ctags -a -f tags "$FLAGS"
+	xargs ctags -a -f tags "$FLAGS" -I "$IGNORE_LIST"

especially because it includes everything in tmp_install, which pollutes
the tag list.

In my own tags script I just call "ctags -R", and I feed cscope with
these find lines

(find $SRCDIR \( -name tmp_install -prune -o -name tmp_check -prune \) -o \( -name "*.[chly]" -o -iname "*makefile*" -o -name "*.mk" -o -name "*.in" -o -name "*.sh" -o -name "*.sgml" -o -name "*.sql" -o -name "*.p[lm]" \) -type f -print ; \
find $BUILDDIR \( -name tmp_install -prune \) -o \( -name \*.h -a -type f \) -print )

Thank you for your comments.

I updated the patch to ignore the code under tmp_install and add
some file types like sql, p[lm], and so on. .sgml or .sh is not
included because they don't seem to be beneficial for ctags.

Regards,
Yugo Nagata

--
Yugo NAGATA <nagata@sraoss.co.jp>

Hi,

I found that tag files generated by src/tools/make_ctags
doesn't include some keywords, that were field names of node
structs, for example norm_select in RestrictInfo. Such fields
are defined with pg_node_attr macro that was introduced
recently, like:

Selectivity norm_selec pg_node_attr(equal_ignore);

In this case, pg_node_attr is mistakenly interpreted to be
the name of the field. So, I propose to use -I option of ctags
to ignore the marco name. Attached is a patch to do it.

I found the same issue with make_etags too.

I updated the patch to ignore the code under tmp_install and add
some file types like sql, p[lm], and so on. .sgml or .sh is not
included because they don't seem to be beneficial for ctags.

I tried to apply the v2 patch approach to make_etags but noticed that
make_ctags and make_etags have quite a few duplicate codes, that would
bring maintenance headache. I think we could merge make_etags into
make_ctags, then add "-e" option (or whatever) to make_ctags so that
it generates tags files for emacs if the option is specified.

Patch attahced.

Best reagards,
--
Tatsuo Ishii
SRA OSS LLC
English: http://www.sraoss.co.jp/index_en/
Japanese:http://www.sraoss.co.jp

On 2022-Oct-12, Tatsuo Ishii wrote:

I tried to apply the v2 patch approach to make_etags but noticed that
make_ctags and make_etags have quite a few duplicate codes, that would
bring maintenance headache. I think we could merge make_etags into
make_ctags, then add "-e" option (or whatever) to make_ctags so that
it generates tags files for emacs if the option is specified.

If we're going to do this, then I suggest that make_etags should become
a symlink to make_ctags, and behave as if -e is given when called under
that name.

+tags_file=tags

+rm -f ./$tags_file

I think $tags_file should include the leading ./ bit, to reduce
confusion.

However ... hmm ...

find . \( -name 'CVS' -prune \) -o \( -name .git -prune \) -o -type d -print |
while read DIR
-do	[ "$DIR" != "." ] && ln -f -s `echo "$DIR" | sed 's;/[^/]*;/..;g'`/tags "$DIR"/tags
+do	[ "$DIR" != "." ] && ln -f -s `echo "$DIR" | sed 's;/[^/]*;/..;g'`/$tags_file "$DIR"/$tags_file
done

... does this create a tags symlink on each directory? This seems
strange to me, but I admit the hack I keep in my .vim/vimrc looks even
more strange:

: let $CSCOPE_DB=substitute(getcwd(), "^\\(.*/pgsql/source/ [^/]*\\)/.*", "\\1", "")
: let &tags=substitute(getcwd(), "^\\(.*/pgsql/source/[^/]*\\)/.*", "\\1", "") . "/tags"

Not sure which is worse. Having dozens of identically named symlinks
doesn't strike me as a great arrangement though. I would definitely not
use make_ctags if this is unavoidable. I see both scripts do likewise
currently.

(I keep all my build trees under /pgsql/build [a symlink to
~/Code/pgsql/source], and all source trees under /pgsql/source, so this
is an easy conversion to make most of the time.)

--
Álvaro Herrera PostgreSQL Developer — https://www.EnterpriseDB.com/
"World domination is proceeding according to plan" (Andrew Morton)

I tried to apply the v2 patch approach to make_etags but noticed that
make_ctags and make_etags have quite a few duplicate codes, that would
bring maintenance headache. I think we could merge make_etags into
make_ctags, then add "-e" option (or whatever) to make_ctags so that
it generates tags files for emacs if the option is specified.

If we're going to do this, then I suggest that make_etags should become
a symlink to make_ctags, and behave as if -e is given when called under
that name.

What I had in my mind was making make_etags a script just exec
make_ctags (with -e option). But I don't have strong
preference. Symlink is ok for me too.

+tags_file=tags

+rm -f ./$tags_file

I think $tags_file should include the leading ./ bit, to reduce
confusion.

Ok.

However ... hmm ...

find . \( -name 'CVS' -prune \) -o \( -name .git -prune \) -o -type d -print |
while read DIR
-do	[ "$DIR" != "." ] && ln -f -s `echo "$DIR" | sed 's;/[^/]*;/..;g'`/tags "$DIR"/tags
+do	[ "$DIR" != "." ] && ln -f -s `echo "$DIR" | sed 's;/[^/]*;/..;g'`/$tags_file "$DIR"/$tags_file
done

... does this create a tags symlink on each directory? This seems
strange to me,

I don't know the original author's intention for this but I think it
makes use of the tag file in emacs a little bit easier. Emacs
confirms for the first time the default location of tags file under
the same directory where the source file resides. I can just hit
return key if there's a symlink of tags. If we do not create the
symlink, we have to specify the directory where the tags file was
originally created, which is a little bit annoying.

but I admit the hack I keep in my .vim/vimrc looks even
more strange:

: let $CSCOPE_DB=substitute(getcwd(), "^\\(.*/pgsql/source/ [^/]*\\)/.*", "\\1", "")
: let &tags=substitute(getcwd(), "^\\(.*/pgsql/source/[^/]*\\)/.*", "\\1", "") . "/tags"

Not sure which is worse. Having dozens of identically named symlinks
doesn't strike me as a great arrangement though. I would definitely not
use make_ctags if this is unavoidable. I see both scripts do likewise
currently.

Well, I often visit different tags file for different development
projects (for example Pgpool-II) and I don't want to have fixed
location of tags file in emacs setting. For this reason make_etags's
approach is acceptable for me.

Best reagards,
--
Tatsuo Ishii
SRA OSS LLC
English: http://www.sraoss.co.jp/index_en/
Japanese:http://www.sraoss.co.jp

Tatsuo Ishii <ishii@sraoss.co.jp> writes:

If we're going to do this, then I suggest that make_etags should become
a symlink to make_ctags, and behave as if -e is given when called under
that name.

What I had in my mind was making make_etags a script just exec
make_ctags (with -e option). But I don't have strong
preference. Symlink is ok for me too.

I don't think it's possible to store a symlink in git, so
having one file exec the other sounds saner to me too.

regards, tom lane

On 2022-Oct-12, Tom Lane wrote:

Tatsuo Ishii <ishii@sraoss.co.jp> writes:

If we're going to do this, then I suggest that make_etags should become
a symlink to make_ctags, and behave as if -e is given when called under
that name.

What I had in my mind was making make_etags a script just exec
make_ctags (with -e option). But I don't have strong
preference. Symlink is ok for me too.

I don't think it's possible to store a symlink in git, so
having one file exec the other sounds saner to me too.

I tried before my reply and it seems to work, but perhaps it requires
too new a git version. It may also be a problem under Windows. Having
one exec the other sounds perfect.

--
Álvaro Herrera PostgreSQL Developer — https://www.EnterpriseDB.com/

On Wed, Oct 12, 2022 at 10:09:06AM -0400, Tom Lane wrote:

I don't think it's possible to store a symlink in git, so
having one file exec the other sounds saner to me too.

Git handles symlink just fine, but those will obviously create extra burden for
Windows users (git will just create a text file containing the target path I
think), so agreed (even if I doubt that any Windows user will run those
scripts anyway).

On 2022-Oct-12, Tatsuo Ishii wrote:

find . \( -name 'CVS' -prune \) -o \( -name .git -prune \) -o -type d -print |
while read DIR
-do	[ "$DIR" != "." ] && ln -f -s `echo "$DIR" | sed 's;/[^/]*;/..;g'`/tags "$DIR"/tags
+do	[ "$DIR" != "." ] && ln -f -s `echo "$DIR" | sed 's;/[^/]*;/..;g'`/$tags_file "$DIR"/$tags_file
done

... does this create a tags symlink on each directory? This seems
strange to me,

I don't know the original author's intention for this but I think it
makes use of the tag file in emacs a little bit easier. Emacs
confirms for the first time the default location of tags file under
the same directory where the source file resides. I can just hit
return key if there's a symlink of tags. If we do not create the
symlink, we have to specify the directory where the tags file was
originally created, which is a little bit annoying.

OK, that sounds good then. I would make a feature request to have a
switch that supresses creation of these links, then.

Well, I often visit different tags file for different development
projects (for example Pgpool-II) and I don't want to have fixed
location of tags file in emacs setting. For this reason make_etags's
approach is acceptable for me.

Makes sense.

--
Álvaro Herrera 48°01'N 7°57'E — https://www.EnterpriseDB.com/
Thou shalt study thy libraries and strive not to reinvent them without
cause, that thy code may be short and readable and thy days pleasant
and productive. (7th Commandment for C Programmers)

On Wed, Oct 12, 2022 at 10:26:03PM +0900, Tatsuo Ishii wrote:

However ... hmm ...

find . \( -name 'CVS' -prune \) -o \( -name .git -prune \) -o -type d -print |
while read DIR
-do	[ "$DIR" != "." ] && ln -f -s `echo "$DIR" | sed 's;/[^/]*;/..;g'`/tags "$DIR"/tags
+do	[ "$DIR" != "." ] && ln -f -s `echo "$DIR" | sed 's;/[^/]*;/..;g'`/$tags_file "$DIR"/$tags_file
done

... does this create a tags symlink on each directory? This seems
strange to me,

I don't know the original author's intention for this but I think it
makes use of the tag file in emacs a little bit easier. Emacs
confirms for the first time the default location of tags file under
the same directory where the source file resides. I can just hit
return key if there's a symlink of tags. If we do not create the
symlink, we have to specify the directory where the tags file was
originally created, which is a little bit annoying.

Yes, that is exactly the intent of why it uses symlinks in every
directory.

--
Bruce Momjian <bruce@momjian.us> https://momjian.us
EDB https://enterprisedb.com

Indecision is a decision. Inaction is an action. Mark Batterson

On 10.10.22 12:04, Alvaro Herrera wrote:

In my own tags script I just call "ctags -R", and I feed cscope with
these find lines

(find $SRCDIR \( -name tmp_install -prune -o -name tmp_check -prune \) -o \( -name "*.[chly]" -o -iname "*makefile*" -o -name "*.mk" -o -name "*.in" -o -name "*.sh" -o -name "*.sgml" -o -name "*.sql" -o -name "*.p[lm]" \) -type f -print ; \
find $BUILDDIR \( -name tmp_install -prune \) -o \( -name \*.h -a -type f \) -print )

which seems to give decent results. (Nowadays I wonder if it'd be
better to exclude the "*_d.h" files from the builddir.)
(I wonder why don't I have a prune for .git ...)

Or use git ls-files.

OK, that sounds good then. I would make a feature request to have a
switch that supresses creation of these links, then.

Ok, I have added "-n" option to make_ctags so that it skips to create
the links.

Also I have changed make_etags so that it exec make_ctags, which seems
to be the consensus.

Best reagards,
--
Tatsuo Ishii
SRA OSS LLC
English: http://www.sraoss.co.jp/index_en/
Japanese:http://www.sraoss.co.jp

On Thu, 13 Oct 2022 15:35:09 +0900 (JST)
Tatsuo Ishii <ishii@sraoss.co.jp> wrote:

OK, that sounds good then. I would make a feature request to have a
switch that supresses creation of these links, then.

Ok, I have added "-n" option to make_ctags so that it skips to create
the links.

Also I have changed make_etags so that it exec make_ctags, which seems
to be the consensus.

Thank you for following up my patch.
I fixed the patch to allow use both -e and -n options together.

Regards,
Yugo Nagata

Best reagards,
--
Tatsuo Ishii
SRA OSS LLC
English: http://www.sraoss.co.jp/index_en/
Japanese:http://www.sraoss.co.jp

--
Yugo NAGATA <nagata@sraoss.co.jp>

On Thu, 13 Oct 2022 15:35:09 +0900 (JST)
Tatsuo Ishii <ishii@sraoss.co.jp> wrote:

OK, that sounds good then. I would make a feature request to have a
switch that supresses creation of these links, then.

Ok, I have added "-n" option to make_ctags so that it skips to create
the links.

Also I have changed make_etags so that it exec make_ctags, which seems
to be the consensus.

Thank you for following up my patch.
I fixed the patch to allow use both -e and -n options together.

Thanks. I have made mostly cosmetic changes so that it is more
consistent with existing scripts.

I would like to push v6 patch if there's no objection.

Best reagards,
--
Tatsuo Ishii
SRA OSS LLC
English: http://www.sraoss.co.jp/index_en/
Japanese:http://www.sraoss.co.jp

Hi,

On Sat, 15 Oct 2022 10:40:29 +0900 (JST)
Tatsuo Ishii <ishii@sraoss.co.jp> wrote:

On Thu, 13 Oct 2022 15:35:09 +0900 (JST)
Tatsuo Ishii <ishii@sraoss.co.jp> wrote:

OK, that sounds good then. I would make a feature request to have a
switch that supresses creation of these links, then.

Ok, I have added "-n" option to make_ctags so that it skips to create
the links.

Also I have changed make_etags so that it exec make_ctags, which seems
to be the consensus.

Thank you for following up my patch.
I fixed the patch to allow use both -e and -n options together.

Thanks. I have made mostly cosmetic changes so that it is more
consistent with existing scripts.

I would like to push v6 patch if there's no objection.

I am fine with this patch.

By the way, in passing, how about adding "tags" and "TAGS" to
.gitignore file?

Best reagards,
--
Tatsuo Ishii
SRA OSS LLC
English: http://www.sraoss.co.jp/index_en/
Japanese:http://www.sraoss.co.jp

--
Yugo NAGATA <nagata@sraoss.co.jp>

Hi,

On Sat, 15 Oct 2022 10:40:29 +0900 (JST)
Tatsuo Ishii <ishii@sraoss.co.jp> wrote:

On Thu, 13 Oct 2022 15:35:09 +0900 (JST)
Tatsuo Ishii <ishii@sraoss.co.jp> wrote:

OK, that sounds good then. I would make a feature request to have a
switch that supresses creation of these links, then.

Ok, I have added "-n" option to make_ctags so that it skips to create
the links.

Also I have changed make_etags so that it exec make_ctags, which seems
to be the consensus.

Thank you for following up my patch.
I fixed the patch to allow use both -e and -n options together.

Thanks. I have made mostly cosmetic changes so that it is more
consistent with existing scripts.

I would like to push v6 patch if there's no objection.

I am fine with this patch.

Thanks. the v6 patch pushed to master branch.

By the way, in passing, how about adding "tags" and "TAGS" to
.gitignore file?

Sounds like a good idea.

Best reagards,
--
Tatsuo Ishii
SRA OSS LLC
English: http://www.sraoss.co.jp/index_en/
Japanese:http://www.sraoss.co.jp

On Wed, 19 Oct 2022 13:25:17 +0900 (JST)
Tatsuo Ishii <ishii@sraoss.co.jp> wrote:

Hi,

On Sat, 15 Oct 2022 10:40:29 +0900 (JST)
Tatsuo Ishii <ishii@sraoss.co.jp> wrote:

On Thu, 13 Oct 2022 15:35:09 +0900 (JST)
Tatsuo Ishii <ishii@sraoss.co.jp> wrote:

OK, that sounds good then. I would make a feature request to have a
switch that supresses creation of these links, then.

Ok, I have added "-n" option to make_ctags so that it skips to create
the links.

Also I have changed make_etags so that it exec make_ctags, which seems
to be the consensus.

Thank you for following up my patch.
I fixed the patch to allow use both -e and -n options together.

Thanks. I have made mostly cosmetic changes so that it is more
consistent with existing scripts.

I would like to push v6 patch if there's no objection.

I am fine with this patch.

Thanks. the v6 patch pushed to master branch.

Thanks!

By the way, in passing, how about adding "tags" and "TAGS" to
.gitignore file?

Sounds like a good idea.

Ok, the patch is attached.

Regards,
Yugo Nagata

--
Yugo NAGATA <nagata@sraoss.co.jp>

By the way, in passing, how about adding "tags" and "TAGS" to
.gitignore file?

Sounds like a good idea.

Ok, the patch is attached.

I have search the mail archive and found this:

/messages/by-id/CAFcNs+rG-DASXzHcecYKvAj+rmxi8CpMAgbpGpEK-mjC96F=Lg@mail.gmail.com

It seems the consensus was to avoid to put this sort of things into
.gitignore in the PostgreSQL source tree. Rather, put into personal
.gitignore or whatever so that developers don't need to care about
other's preference.

Best reagards,
--
Tatsuo Ishii
SRA OSS LLC
English: http://www.sraoss.co.jp/index_en/
Japanese:http://www.sraoss.co.jp

On Wed, 19 Oct 2022 17:17:17 +0900 (JST)
Tatsuo Ishii <ishii@sraoss.co.jp> wrote:

By the way, in passing, how about adding "tags" and "TAGS" to
.gitignore file?

Sounds like a good idea.

Ok, the patch is attached.

I have search the mail archive and found this:

/messages/by-id/CAFcNs+rG-DASXzHcecYKvAj+rmxi8CpMAgbpGpEK-mjC96F=Lg@mail.gmail.com

It seems the consensus was to avoid to put this sort of things into
.gitignore in the PostgreSQL source tree. Rather, put into personal
.gitignore or whatever so that developers don't need to care about
other's preference.

Ok, I understand. Thanks!

By the way, after executing both make_etags and make_ctags, trying tag jump
in my vim causes the following error even though there are correct tags files.

E431: Format error in tags file "backend/access/heap/TAGS"

Removing all TAGS files as below can resolve this error.
$ find . -name TAGS | xargs rm

So, should we have one more option of make_{ce}tags script to clean up
existing tags/TAGS files?

Best reagards,
--
Tatsuo Ishii
SRA OSS LLC
English: http://www.sraoss.co.jp/index_en/
Japanese:http://www.sraoss.co.jp

--
Yugo NAGATA <nagata@sraoss.co.jp>

By the way, after executing both make_etags and make_ctags, trying tag jump
in my vim causes the following error even though there are correct tags files.

E431: Format error in tags file "backend/access/heap/TAGS"

Removing all TAGS files as below can resolve this error.
$ find . -name TAGS | xargs rm

So, should we have one more option of make_{ce}tags script to clean up
existing tags/TAGS files?

Not sure. Before the commit make_ctags did not do such a thing but we
never heard any complain like yours. Also I believe vi/vim users never
invoke make_etags (same thing can be said to emacs users). So why
should we bother?

Best reagards,
--
Tatsuo Ishii
SRA OSS LLC
English: http://www.sraoss.co.jp/index_en/
Japanese:http://www.sraoss.co.jp

On Wed, 19 Oct 2022 18:11:13 +0900 (JST)
Tatsuo Ishii <ishii@sraoss.co.jp> wrote:

By the way, after executing both make_etags and make_ctags, trying tag jump
in my vim causes the following error even though there are correct tags files.

E431: Format error in tags file "backend/access/heap/TAGS"

Removing all TAGS files as below can resolve this error.
$ find . -name TAGS | xargs rm

So, should we have one more option of make_{ce}tags script to clean up
existing tags/TAGS files?

Not sure. Before the commit make_ctags did not do such a thing but we
never heard any complain like yours. Also I believe vi/vim users never
invoke make_etags (same thing can be said to emacs users). So why
should we bother?

Indeed, it was my first use of make_etags (or make_ctags -e) and it was
just for testing the patch. Similarly, someone who runs mistakenly this
command might want this option. However, as you say, there've been no
complain about this, so I don't feel it necessary so much. Maybe, users
of this command would be able to remove tags by their selves easily.

Regards,
Yugo Nagata

--
Yugo NAGATA <nagata@sraoss.co.jp>

On 2022/10/19 13:25, Tatsuo Ishii wrote:

Thanks. the v6 patch pushed to master branch.

Since this commit, make_etags has started failing to generate
tags files with the following error messages, on my MacOS.

$ src/tools/make_etags
/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/ctags: illegal option -- e
usage: ctags [-BFadtuwvx] [-f tagsfile] file ...
sort: No such file or directory

In my MacOS, non-Exuberant ctags is installed and doesn't support
-e option. But the commit changed make_etags so that it always
calls ctags with -e option via make_ctags. This seems the cause of
the above failure.

IS_EXUBERANT=""
ctags --version 2>&1 | grep Exuberant && IS_EXUBERANT="Y"

make_ctags has the above code and seems to support non-Exuberant ctags.
If so, we should revert the changes of make_etags by the commit and
make make_etags work with that ctags? Or, we should support
only Exuberant-type ctags (btw, I'm ok with this) and get rid of
something like the above code?

Regards,

--
Fujii Masao
Advanced Computing Technology Center
Research and Development Headquarters
NTT DATA CORPORATION

On 2022/10/19 13:25, Tatsuo Ishii wrote:

Thanks. the v6 patch pushed to master branch.

Since this commit, make_etags has started failing to generate
tags files with the following error messages, on my MacOS.

$ src/tools/make_etags
/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/ctags:
illegal option -- e
usage: ctags [-BFadtuwvx] [-f tagsfile] file ...
sort: No such file or directory

In my MacOS, non-Exuberant ctags is installed and doesn't support
-e option. But the commit changed make_etags so that it always
calls ctags with -e option via make_ctags. This seems the cause of
the above failure.

IS_EXUBERANT=""
ctags --version 2>&1 | grep Exuberant && IS_EXUBERANT="Y"

make_ctags has the above code and seems to support non-Exuberant
ctags.
If so, we should revert the changes of make_etags by the commit and
make make_etags work with that ctags? Or, we should support
only Exuberant-type ctags (btw, I'm ok with this) and get rid of
something like the above code?

Thanks for the report. I will look into this.

Best reagards,
--
Tatsuo Ishii
SRA OSS LLC
English: http://www.sraoss.co.jp/index_en/
Japanese:http://www.sraoss.co.jp

Since this commit, make_etags has started failing to generate
tags files with the following error messages, on my MacOS.

$ src/tools/make_etags
/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/ctags:
illegal option -- e
usage: ctags [-BFadtuwvx] [-f tagsfile] file ...
sort: No such file or directory

In my MacOS, non-Exuberant ctags is installed and doesn't support
-e option. But the commit changed make_etags so that it always
calls ctags with -e option via make_ctags. This seems the cause of
the above failure.

IS_EXUBERANT=""
ctags --version 2>&1 | grep Exuberant && IS_EXUBERANT="Y"

make_ctags has the above code and seems to support non-Exuberant
ctags.
If so, we should revert the changes of make_etags by the commit and
make make_etags work with that ctags? Or, we should support
only Exuberant-type ctags (btw, I'm ok with this) and get rid of
something like the above code?

Thanks for the report. I will look into this.

Previous make_etags relied on etags command:

#!/bin/sh

# src/tools/make_etags

command -v etags >/dev/null || \
{ echo "'etags' program not found" 1>&2; exit 1; }
:
:

My Mac (M1 Mac running macOS 12.6) does not have etags. Thus before
the commit make_etags on Mac failed anyway. Do we want make_etags to
restore the previous behavior? i.e. 'etags' program not found

If so, we should revert the changes of make_etags by the commit and
make make_etags work with that ctags?

I think ctags on Mac cannot produce tags file for emacs.

Best reagards,
--
Tatsuo Ishii
SRA OSS LLC
English: http://www.sraoss.co.jp/index_en/
Japanese:http://www.sraoss.co.jp

On Tue, 07 Feb 2023 17:19:37 +0900 (JST)
Tatsuo Ishii <ishii@sraoss.co.jp> wrote:

Since this commit, make_etags has started failing to generate
tags files with the following error messages, on my MacOS.

$ src/tools/make_etags
/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/ctags:
illegal option -- e
usage: ctags [-BFadtuwvx] [-f tagsfile] file ...
sort: No such file or directory

In my MacOS, non-Exuberant ctags is installed and doesn't support
-e option. But the commit changed make_etags so that it always
calls ctags with -e option via make_ctags. This seems the cause of
the above failure.

IS_EXUBERANT=""
ctags --version 2>&1 | grep Exuberant && IS_EXUBERANT="Y"

make_ctags has the above code and seems to support non-Exuberant
ctags.
If so, we should revert the changes of make_etags by the commit and
make make_etags work with that ctags? Or, we should support
only Exuberant-type ctags (btw, I'm ok with this) and get rid of
something like the above code?

Thanks for the report. I will look into this.

Previous make_etags relied on etags command:

#!/bin/sh

# src/tools/make_etags

command -v etags >/dev/null || \
{ echo "'etags' program not found" 1>&2; exit 1; }
:
:

My Mac (M1 Mac running macOS 12.6) does not have etags. Thus before
the commit make_etags on Mac failed anyway. Do we want make_etags to
restore the previous behavior? i.e. 'etags' program not found

If so, we should revert the changes of make_etags by the commit and
make make_etags work with that ctags?

I think ctags on Mac cannot produce tags file for emacs.

Does is make sense to change make_etags as the attached patch does?
This allows make_etags to use etags if Exuberant-type ctags is not
available. This allows users to use make_etags if hey has either
Exuberant-type ctags or etags.

Regards,
Yugo Nagata

Best reagards,
--
Tatsuo Ishii
SRA OSS LLC
English: http://www.sraoss.co.jp/index_en/
Japanese:http://www.sraoss.co.jp

--
Yugo NAGATA <nagata@sraoss.co.jp>

Does is make sense to change make_etags as the attached patch does?
This allows make_etags to use etags if Exuberant-type ctags is not
available. This allows users to use make_etags if hey has either
Exuberant-type ctags or etags.

The patch drops support for "-n" option :-<

Attached is the patch by fixing make_ctags (make_etags is not
touched).

If Exuberant-type ctags is available, use it (not changed).
If Exuberant-type ctags is not available, try old ctags (not changed).
If the old ctags does not support "-e" option, try etags (new).
If etags is not available, give up (new).

Best reagards,
--
Tatsuo Ishii
SRA OSS LLC
English: http://www.sraoss.co.jp/index_en/
Japanese:http://www.sraoss.co.jp

On Tue, 07 Feb 2023 21:29:04 +0900 (JST)
Tatsuo Ishii <ishii@sraoss.co.jp> wrote:

Does is make sense to change make_etags as the attached patch does?
This allows make_etags to use etags if Exuberant-type ctags is not
available. This allows users to use make_etags if hey has either
Exuberant-type ctags or etags.

The patch drops support for "-n" option :-<

Attached is the patch by fixing make_ctags (make_etags is not
touched).

If Exuberant-type ctags is available, use it (not changed).
If Exuberant-type ctags is not available, try old ctags (not changed).
If the old ctags does not support "-e" option, try etags (new).

I am not sure if this is good way to check if ctags supports "-e" or not.

+ then ctags --version 2>&1 | grep -- -e >/dev/null

Perhaps, "--help" might be intended rather than "--version" to check
supported options? Even so, ctags would have other option whose name contains
"-e" than Emacs support, so this check could end in a wrong result. Therefore,
it seems to me that it is better to check immediately if etags is available
in case that we don't have Exuberant-type ctags.

Regards,
Yugo Nagata

If etags is not available, give up (new).

Best reagards,
--
Tatsuo Ishii
SRA OSS LLC
English: http://www.sraoss.co.jp/index_en/
Japanese:http://www.sraoss.co.jp

--
Yugo NAGATA <nagata@sraoss.co.jp>

The patch drops support for "-n" option :-<

Attached is the patch by fixing make_ctags (make_etags is not
touched).

If Exuberant-type ctags is available, use it (not changed).
If Exuberant-type ctags is not available, try old ctags (not changed).
If the old ctags does not support "-e" option, try etags (new).

I am not sure if this is good way to check if ctags supports "-e" or not.

+ then ctags --version 2>&1 | grep -- -e >/dev/null

Perhaps, "--help" might be intended rather than "--version" to check
supported options?

Yeah, that was my mistake.

Even so, ctags would have other option whose name contains
"-e" than Emacs support, so this check could end in a wrong result. Therefore,
it seems to me that it is better to check immediately if etags is available
in case that we don't have Exuberant-type ctags.

That makes sense.

Best reagards,
--
Tatsuo Ishii
SRA OSS LLC
English: http://www.sraoss.co.jp/index_en/
Japanese:http://www.sraoss.co.jp

I am not sure if this is good way to check if ctags supports "-e" or not.

+ then ctags --version 2>&1 | grep -- -e >/dev/null

Perhaps, "--help" might be intended rather than "--version" to check
supported options?

Yeah, that was my mistake.

Even so, ctags would have other option whose name contains
"-e" than Emacs support, so this check could end in a wrong result. Therefore,
it seems to me that it is better to check immediately if etags is available
in case that we don't have Exuberant-type ctags.

That makes sense.

Attached is the v2 patch.

Best reagards,
--
Tatsuo Ishii
SRA OSS LLC
English: http://www.sraoss.co.jp/index_en/
Japanese:http://www.sraoss.co.jp

On 2023/02/08 9:49, Tatsuo Ishii wrote:

I am not sure if this is good way to check if ctags supports "-e" or not.

+ then ctags --version 2>&1 | grep -- -e >/dev/null

Perhaps, "--help" might be intended rather than "--version" to check
supported options?

Yeah, that was my mistake.

Even so, ctags would have other option whose name contains
"-e" than Emacs support, so this check could end in a wrong result. Therefore,
it seems to me that it is better to check immediately if etags is available
in case that we don't have Exuberant-type ctags.

That makes sense.

Attached is the v2 patch.

Thanks for the patch!

With the patch, I got the following error when executing make_etags..

$ ./src/tools/make_etags
etags: invalid option -- 'e'
Try 'etags --help' for a complete list of options.
sort: No such file or directory

+		if [ $? != 0 -a -z "$ETAGS_EXISTS" ]
+		then	echo "'ctags' does not support emacs mode and etags does not exist" 1>&2; exit 1
+		fi

This code can be reached after "rm -f ./$TAGS_FILE" is executed in make_ctags.
But we should check whether the required program has been already installed
and exit immediately if not, before modifying anything?

This is the comment for the commit d1e2a380cb. I found that make_etags with
an invalid option reported the following usage message mentioning make_ctags
(not make_etags). Isn't this confusing?

$ ./src/tools/make_etags -a
Usage: /.../make_ctags [-e][-n]

Regards,

--
Fujii Masao
Advanced Computing Technology Center
Research and Development Headquarters
NTT DATA CORPORATION

Attached is the v2 patch.

Thanks for the patch!

With the patch, I got the following error when executing make_etags..

$ ./src/tools/make_etags
etags: invalid option -- 'e'
Try 'etags --help' for a complete list of options.
sort: No such file or directory

Oops. Thank you for pointing it out. BTW, just out of curiosity, do
you have etags on you Mac? Mine doesn't have etags. That's why I
missed the error.

+		if [ $? != 0 -a -z "$ETAGS_EXISTS" ]
+ then echo "'ctags' does not support emacs mode and etags does not
exist" 1>&2; exit 1
+		fi

This code can be reached after "rm -f ./$TAGS_FILE" is executed in
make_ctags.
But we should check whether the required program has been already
installed
and exit immediately if not, before modifying anything?

Agreed.

This is the comment for the commit d1e2a380cb. I found that make_etags
with
an invalid option reported the following usage message mentioning
make_ctags
(not make_etags). Isn't this confusing?

$ ./src/tools/make_etags -a
Usage: /.../make_ctags [-e][-n]

That's hard to fix without some code duplication. We decided that
make_etags is not a symlink to make_ctags, rather execs make_ctags. Of
course we could let make_etags perform the same option check, but I
doubt it's worth the trouble.

Anyway, attached is the v3 patch.

Best reagards,
--
Tatsuo Ishii
SRA OSS LLC
English: http://www.sraoss.co.jp/index_en/
Japanese:http://www.sraoss.co.jp

On 2023/02/08 20:17, Tatsuo Ishii wrote:

Attached is the v2 patch.

Thanks for the patch!

With the patch, I got the following error when executing make_etags..

$ ./src/tools/make_etags
etags: invalid option -- 'e'
Try 'etags --help' for a complete list of options.
sort: No such file or directory

Oops. Thank you for pointing it out. BTW, just out of curiosity, do
you have etags on you Mac?

Yes.

$ etags --version
etags (GNU Emacs 28.2)
Copyright (C) 2022 Free Software Foundation, Inc.
This program is distributed under the terms in ETAGS.README

This is the comment for the commit d1e2a380cb. I found that make_etags
with
an invalid option reported the following usage message mentioning
make_ctags
(not make_etags). Isn't this confusing?

$ ./src/tools/make_etags -a
Usage: /.../make_ctags [-e][-n]

That's hard to fix without some code duplication. We decided that
make_etags is not a symlink to make_ctags, rather execs make_ctags. Of
course we could let make_etags perform the same option check, but I
doubt it's worth the trouble.

How about just applying the following into make_etags?

+if [ $# -gt 1 ] || ( [ $# -eq 1 ] && [ $1 != "-n" ] )
+then	echo "Usage: $0 [-n]"
+	exit 1
+fi

Anyway, attached is the v3 patch.

Thanks for updating the patch!

With the patch, make_etags caused the following error messages
on my MacOS.

$ ./src/tools/make_etags
No such file or directory
No such file or directory

To fix this error, probaby we should get rid of double-quotes
from "$FLAGS" "$IGNORE_IDENTIFIES" in the following command.

-	xargs ctags $MODE -a -f $TAGS_FILE "$FLAGS" "$IGNORE_IDENTIFIES"
+	xargs $PROG $MODE $TAGS_OPT $TAGS_FILE "$FLAGS" "$IGNORE_IDENTIFIES"

+	else	ctags --help 2>&1 | grep -- -e >/dev/null
+		# Note that "ctags --help" does not always work. Even certain ctags does not have the option.

This code seems to assume that there is non-Exuberant ctags
supporting -e option. But does such ctags really exist?

I fixed the above issues and refactored the code.
Attached is the updated version of the patch. Thought?

Regards,

--
Fujii Masao
Advanced Computing Technology Center
Research and Development Headquarters
NTT DATA CORPORATION

Oops. Thank you for pointing it out. BTW, just out of curiosity, do
you have etags on you Mac?

Yes.

$ etags --version
etags (GNU Emacs 28.2)
Copyright (C) 2022 Free Software Foundation, Inc.
This program is distributed under the terms in ETAGS.README

Ok. Probably that was installed with emacs. For some reason I don't
know, I don't have etags even I already installed emacs. So I decided
to test using my old Ubuntu 18 vm, which has old ctags and etags.

How about just applying the following into make_etags?

+if [ $# -gt 1 ] || ( [ $# -eq 1 ] && [ $1 != "-n" ] )
+then	echo "Usage: $0 [-n]"
+	exit 1
+fi

Ok from me. Looks simple enough.

With the patch, make_etags caused the following error messages
on my MacOS.

$ ./src/tools/make_etags
No such file or directory
No such file or directory

To fix this error, probaby we should get rid of double-quotes
from "$FLAGS" "$IGNORE_IDENTIFIES" in the following command.

-	xargs ctags $MODE -a -f $TAGS_FILE "$FLAGS" "$IGNORE_IDENTIFIES"
+ xargs $PROG $MODE $TAGS_OPT $TAGS_FILE "$FLAGS" "$IGNORE_IDENTIFIES"

Oh, I see.

+	else	ctags --help 2>&1 | grep -- -e >/dev/null
+ # Note that "ctags --help" does not always work. Even certain ctags
does not have the option.

This code seems to assume that there is non-Exuberant ctags
supporting -e option. But does such ctags really exist?

Good question. I vaguely recalled so、but I haven't been able to find
any evidence for it.

I fixed the above issues and refactored the code.
Attached is the updated version of the patch. Thought?

Thank you! Looks good to me.

Best reagards,
--
Tatsuo Ishii
SRA OSS LLC
English: http://www.sraoss.co.jp/index_en/
Japanese:http://www.sraoss.co.jp

I fixed the above issues and refactored the code.
Attached is the updated version of the patch. Thought?

Thank you! Looks good to me.

Fix pushed. Thank you!

Best reagards,
--
Tatsuo Ishii
SRA OSS LLC
English: http://www.sraoss.co.jp/index_en/
Japanese:http://www.sraoss.co.jp

Hi,

On Tue, Feb 14, 2023 at 8:15 PM Tatsuo Ishii <ishii@sraoss.co.jp> wrote:

I fixed the above issues and refactored the code.
Attached is the updated version of the patch. Thought?

Thank you! Looks good to me.

Fix pushed. Thank you!

In my Mac environment where non-Exuberant ctags and emacs 28.2 are
installed, the generated etags file cannot be loaded by emacs due to
file format error. The generated TAGS file is:

% head -10 TAGS

) /
sizeof(BlockNumber)sizeof(BlockNumber)117,3750
my
@newa newa395,10443

variadic array[1,2]:array[1,2]56,1803

variadic array[]::inarray[]::i72,2331

variadic array[array64,2111

variadic array[array68,2222

variadic array[array76,2441
(2 * (2 53,1353
my $fn fn387,10147
startblock 101,4876

Since the etags files consist of multiple sections[1]https://en.wikipedia.org/wiki/Ctags#Etags_2 we cannot sort
the generated etags file. With the attached patch, make_etags (with
non-Exuberant ctags) generates a correct format etags file and it
works:

% head -10 TAGS

/Users/masahiko/pgsql/source/postgresql/GNUmakefile,1187
subdir 7,56
top_builddir 8,65
docs:docs13,167
world-contrib-recurse:world-contrib-recurse19,273
world-bin-contrib-recurse:world-bin-contrib-recurse24,394
html man:html man26,444
install-docs:install-docs29,474
install-world-contrib-recurse:install-world-contrib-recurse35,604

BTW regarding the following comment, as far as I can read the
Wikipedia page for ctags[1]https://en.wikipedia.org/wiki/Ctags#Etags_2, Exuberant ctags file doesn't have a
header section.

# Exuberant tags has a header that we cannot sort in with the other entries
# so we skip the sort step
# Why are we sorting this? I guess some tag implementation need this,
# particularly for append mode. bjm 2012-02-24
if [ ! "$IS_EXUBERANT" ]

Instead, the page says that sorting non-Exuberant tags file allows for
faster searching on of the tags file. I've fixed the comment
accordingly too.

Regards,

[1]: https://en.wikipedia.org/wiki/Ctags#Etags_2

--
Masahiko Sawada
Amazon Web Services: https://aws.amazon.com

Hi Sawada-san,

In my Mac environment where non-Exuberant ctags and emacs 28.2 are
installed, the generated etags file cannot be loaded by emacs due to
file format error. The generated TAGS file is:

% head -10 TAGS

) /
sizeof(BlockNumber)sizeof(BlockNumber)117,3750
my
@newa newa395,10443

variadic array[1,2]:array[1,2]56,1803

variadic array[]::inarray[]::i72,2331

variadic array[array64,2111

variadic array[array68,2222

variadic array[array76,2441
(2 * (2 53,1353
my $fn fn387,10147
startblock 101,4876

Since the etags files consist of multiple sections[1] we cannot sort
the generated etags file. With the attached patch, make_etags (with
non-Exuberant ctags) generates a correct format etags file and it
works:

% head -10 TAGS

/Users/masahiko/pgsql/source/postgresql/GNUmakefile,1187
subdir 7,56
top_builddir 8,65
docs:docs13,167
world-contrib-recurse:world-contrib-recurse19,273
world-bin-contrib-recurse:world-bin-contrib-recurse24,394
html man:html man26,444
install-docs:install-docs29,474
install-world-contrib-recurse:install-world-contrib-recurse35,604

BTW regarding the following comment, as far as I can read the
Wikipedia page for ctags[1], Exuberant ctags file doesn't have a
header section.

# Exuberant tags has a header that we cannot sort in with the other entries
# so we skip the sort step
# Why are we sorting this? I guess some tag implementation need this,
# particularly for append mode. bjm 2012-02-24
if [ ! "$IS_EXUBERANT" ]

Instead, the page says that sorting non-Exuberant tags file allows for
faster searching on of the tags file. I've fixed the comment
accordingly too.

Regards,

[1] https://en.wikipedia.org/wiki/Ctags#Etags_2

Sorry for late reply and thanks for the patch!

I have confirmed the error with make_etags on my Mac (emacs 28.1 +
non-Exuberant ctags), and the error is fixed by your patch. Also I
have confirmed the patch does not affect make_etags on my Linux (emacs
26.3 + Exuberant ctags).

I will push the fix to REL_15_STABLE and master branch if there's no
objection.

Best reagards,
--
Tatsuo Ishii
SRA OSS LLC
English: http://www.sraoss.co.jp/index_en/
Japanese:http://www.sraoss.co.jp

Hi Sawada-san,

In my Mac environment where non-Exuberant ctags and emacs 28.2 are
installed, the generated etags file cannot be loaded by emacs due to
file format error. The generated TAGS file is:

% head -10 TAGS

) /
sizeof(BlockNumber)sizeof(BlockNumber)117,3750
my
@newa newa395,10443

variadic array[1,2]:array[1,2]56,1803

variadic array[]::inarray[]::i72,2331

variadic array[array64,2111

variadic array[array68,2222

variadic array[array76,2441
(2 * (2 53,1353
my $fn fn387,10147
startblock 101,4876

Since the etags files consist of multiple sections[1] we cannot sort
the generated etags file. With the attached patch, make_etags (with
non-Exuberant ctags) generates a correct format etags file and it
works:

% head -10 TAGS

/Users/masahiko/pgsql/source/postgresql/GNUmakefile,1187
subdir 7,56
top_builddir 8,65
docs:docs13,167
world-contrib-recurse:world-contrib-recurse19,273
world-bin-contrib-recurse:world-bin-contrib-recurse24,394
html man:html man26,444
install-docs:install-docs29,474
install-world-contrib-recurse:install-world-contrib-recurse35,604

BTW regarding the following comment, as far as I can read the
Wikipedia page for ctags[1], Exuberant ctags file doesn't have a
header section.

# Exuberant tags has a header that we cannot sort in with the other entries
# so we skip the sort step
# Why are we sorting this? I guess some tag implementation need this,
# particularly for append mode. bjm 2012-02-24
if [ ! "$IS_EXUBERANT" ]

Instead, the page says that sorting non-Exuberant tags file allows for
faster searching on of the tags file. I've fixed the comment
accordingly too.

Regards,

[1] https://en.wikipedia.org/wiki/Ctags#Etags_2

Sorry for late reply and thanks for the patch!

I have confirmed the error with make_etags on my Mac (emacs 28.1 +
non-Exuberant ctags), and the error is fixed by your patch. Also I
have confirmed the patch does not affect make_etags on my Linux (emacs
26.3 + Exuberant ctags).

I will push the fix to REL_15_STABLE and master branch if there's no
objection.

Fix pushded. Thanks!

Best reagards,
--
Tatsuo Ishii
SRA OSS LLC
English: http://www.sraoss.co.jp/index_en/
Japanese:http://www.sraoss.co.jp

On Wed, Jun 14, 2023 at 11:16 AM Tatsuo Ishii <ishii@sraoss.co.jp> wrote:

Hi Sawada-san,

In my Mac environment where non-Exuberant ctags and emacs 28.2 are
installed, the generated etags file cannot be loaded by emacs due to
file format error. The generated TAGS file is:

% head -10 TAGS

) /
sizeof(BlockNumber)sizeof(BlockNumber)117,3750
my
@newa newa395,10443

variadic array[1,2]:array[1,2]56,1803

variadic array[]::inarray[]::i72,2331

variadic array[array64,2111

variadic array[array68,2222

variadic array[array76,2441
(2 * (2 53,1353
my $fn fn387,10147
startblock 101,4876

Since the etags files consist of multiple sections[1] we cannot sort
the generated etags file. With the attached patch, make_etags (with
non-Exuberant ctags) generates a correct format etags file and it
works:

% head -10 TAGS

/Users/masahiko/pgsql/source/postgresql/GNUmakefile,1187
subdir 7,56
top_builddir 8,65
docs:docs13,167
world-contrib-recurse:world-contrib-recurse19,273
world-bin-contrib-recurse:world-bin-contrib-recurse24,394
html man:html man26,444
install-docs:install-docs29,474
install-world-contrib-recurse:install-world-contrib-recurse35,604

BTW regarding the following comment, as far as I can read the
Wikipedia page for ctags[1], Exuberant ctags file doesn't have a
header section.

# Exuberant tags has a header that we cannot sort in with the other entries
# so we skip the sort step
# Why are we sorting this? I guess some tag implementation need this,
# particularly for append mode. bjm 2012-02-24
if [ ! "$IS_EXUBERANT" ]

Instead, the page says that sorting non-Exuberant tags file allows for
faster searching on of the tags file. I've fixed the comment
accordingly too.

Regards,

[1] https://en.wikipedia.org/wiki/Ctags#Etags_2

Sorry for late reply and thanks for the patch!

I have confirmed the error with make_etags on my Mac (emacs 28.1 +
non-Exuberant ctags), and the error is fixed by your patch. Also I
have confirmed the patch does not affect make_etags on my Linux (emacs
26.3 + Exuberant ctags).

I will push the fix to REL_15_STABLE and master branch if there's no
objection.

Fix pushded. Thanks!

Thank you!

Regards,

--
Masahiko Sawada
Amazon Web Services: https://aws.amazon.com