make_ctags: use -I option to ignore pg_node_attr macro

Hello

On 2022-Oct-07, Yugo NAGATA wrote:

I found that tag files generated by src/tools/make_ctags
doesn't include some keywords, that were field names of node
structs, for example norm_select in RestrictInfo. Such fields
are defined with pg_node_attr macro that was introduced
recently, like:

Selectivity norm_selec pg_node_attr(equal_ignore);

In this case, pg_node_attr is mistakenly interpreted to be
the name of the field. So, I propose to use -I option of ctags
to ignore the marco name. Attached is a patch to do it.

I've wondered if there's anybody that uses this script. I suppose if
you're reporting this problem then it has at least one user, and
therefore worth fixing.

If we do patch it, how about doing some more invasive surgery and
bringing it to the XXI century? I think the `find` line is a bit lame:

# this is outputting the tags into the file 'tags', and appending
find `pwd`/ -type f -name '*.[chyl]' -print |
-	xargs ctags -a -f tags "$FLAGS"
+	xargs ctags -a -f tags "$FLAGS" -I "$IGNORE_LIST"

especially because it includes everything in tmp_install, which pollutes
the tag list.

In my own tags script I just call "ctags -R", and I feed cscope with
these find lines

(find $SRCDIR \( -name tmp_install -prune -o -name tmp_check -prune \) -o \( -name "*.[chly]" -o -iname "*makefile*" -o -name "*.mk" -o -name "*.in" -o -name "*.sh" -o -name "*.sgml" -o -name "*.sql" -o -name "*.p[lm]" \) -type f -print ; \
find $BUILDDIR \( -name tmp_install -prune \) -o \( -name \*.h -a -type f \) -print )

which seems to give decent results. (Nowadays I wonder if it'd be
better to exclude the "*_d.h" files from the builddir.)
(I wonder why don't I have a prune for .git ...)

--
Álvaro Herrera 48°01'N 7°57'E — https://www.EnterpriseDB.com/
¡Ay, ay, ay! Con lo mucho que yo lo quería (bis)
se fue de mi vera ... se fue para siempre, pa toíta ... pa toíta la vida
¡Ay Camarón! ¡Ay Camarón! (Paco de Lucía)

Hello

On Mon, 10 Oct 2022 12:04:22 +0200
Alvaro Herrera <alvherre@alvh.no-ip.org> wrote:

Hello

On 2022-Oct-07, Yugo NAGATA wrote:

I found that tag files generated by src/tools/make_ctags
doesn't include some keywords, that were field names of node
structs, for example norm_select in RestrictInfo. Such fields
are defined with pg_node_attr macro that was introduced
recently, like:

Selectivity norm_selec pg_node_attr(equal_ignore);

In this case, pg_node_attr is mistakenly interpreted to be
the name of the field. So, I propose to use -I option of ctags
to ignore the marco name. Attached is a patch to do it.

I've wondered if there's anybody that uses this script. I suppose if
you're reporting this problem then it has at least one user, and
therefore worth fixing.

Yeah, I am a make_ctags user, there may be few users though....

If we do patch it, how about doing some more invasive surgery and
bringing it to the XXI century? I think the `find` line is a bit lame:

# this is outputting the tags into the file 'tags', and appending
find `pwd`/ -type f -name '*.[chyl]' -print |
-	xargs ctags -a -f tags "$FLAGS"
+	xargs ctags -a -f tags "$FLAGS" -I "$IGNORE_LIST"

especially because it includes everything in tmp_install, which pollutes
the tag list.

In my own tags script I just call "ctags -R", and I feed cscope with
these find lines

(find $SRCDIR \( -name tmp_install -prune -o -name tmp_check -prune \) -o \( -name "*.[chly]" -o -iname "*makefile*" -o -name "*.mk" -o -name "*.in" -o -name "*.sh" -o -name "*.sgml" -o -name "*.sql" -o -name "*.p[lm]" \) -type f -print ; \
find $BUILDDIR \( -name tmp_install -prune \) -o \( -name \*.h -a -type f \) -print )

Thank you for your comments.

I updated the patch to ignore the code under tmp_install and add
some file types like sql, p[lm], and so on. .sgml or .sh is not
included because they don't seem to be beneficial for ctags.

Regards,
Yugo Nagata

--
Yugo NAGATA <nagata@sraoss.co.jp>

Hi,

I found that tag files generated by src/tools/make_ctags
doesn't include some keywords, that were field names of node
structs, for example norm_select in RestrictInfo. Such fields
are defined with pg_node_attr macro that was introduced
recently, like:

Selectivity norm_selec pg_node_attr(equal_ignore);

In this case, pg_node_attr is mistakenly interpreted to be
the name of the field. So, I propose to use -I option of ctags
to ignore the marco name. Attached is a patch to do it.

I found the same issue with make_etags too.

I updated the patch to ignore the code under tmp_install and add
some file types like sql, p[lm], and so on. .sgml or .sh is not
included because they don't seem to be beneficial for ctags.

I tried to apply the v2 patch approach to make_etags but noticed that
make_ctags and make_etags have quite a few duplicate codes, that would
bring maintenance headache. I think we could merge make_etags into
make_ctags, then add "-e" option (or whatever) to make_ctags so that
it generates tags files for emacs if the option is specified.

Patch attahced.

Best reagards,
--
Tatsuo Ishii
SRA OSS LLC
English: http://www.sraoss.co.jp/index_en/
Japanese:http://www.sraoss.co.jp

On 2022-Oct-12, Tatsuo Ishii wrote:

I tried to apply the v2 patch approach to make_etags but noticed that
make_ctags and make_etags have quite a few duplicate codes, that would
bring maintenance headache. I think we could merge make_etags into
make_ctags, then add "-e" option (or whatever) to make_ctags so that
it generates tags files for emacs if the option is specified.

If we're going to do this, then I suggest that make_etags should become
a symlink to make_ctags, and behave as if -e is given when called under
that name.

+tags_file=tags

+rm -f ./$tags_file

I think $tags_file should include the leading ./ bit, to reduce
confusion.

However ... hmm ...

find . \( -name 'CVS' -prune \) -o \( -name .git -prune \) -o -type d -print |
while read DIR
-do	[ "$DIR" != "." ] && ln -f -s `echo "$DIR" | sed 's;/[^/]*;/..;g'`/tags "$DIR"/tags
+do	[ "$DIR" != "." ] && ln -f -s `echo "$DIR" | sed 's;/[^/]*;/..;g'`/$tags_file "$DIR"/$tags_file
done

... does this create a tags symlink on each directory? This seems
strange to me, but I admit the hack I keep in my .vim/vimrc looks even
more strange:

: let $CSCOPE_DB=substitute(getcwd(), "^\\(.*/pgsql/source/ [^/]*\\)/.*", "\\1", "")
: let &tags=substitute(getcwd(), "^\\(.*/pgsql/source/[^/]*\\)/.*", "\\1", "") . "/tags"

Not sure which is worse. Having dozens of identically named symlinks
doesn't strike me as a great arrangement though. I would definitely not
use make_ctags if this is unavoidable. I see both scripts do likewise
currently.

(I keep all my build trees under /pgsql/build [a symlink to
~/Code/pgsql/source], and all source trees under /pgsql/source, so this
is an easy conversion to make most of the time.)

--
Álvaro Herrera PostgreSQL Developer — https://www.EnterpriseDB.com/
"World domination is proceeding according to plan" (Andrew Morton)

I tried to apply the v2 patch approach to make_etags but noticed that
make_ctags and make_etags have quite a few duplicate codes, that would
bring maintenance headache. I think we could merge make_etags into
make_ctags, then add "-e" option (or whatever) to make_ctags so that
it generates tags files for emacs if the option is specified.

If we're going to do this, then I suggest that make_etags should become
a symlink to make_ctags, and behave as if -e is given when called under
that name.

What I had in my mind was making make_etags a script just exec
make_ctags (with -e option). But I don't have strong
preference. Symlink is ok for me too.

+tags_file=tags

+rm -f ./$tags_file

I think $tags_file should include the leading ./ bit, to reduce
confusion.

Ok.

However ... hmm ...

find . \( -name 'CVS' -prune \) -o \( -name .git -prune \) -o -type d -print |
while read DIR
-do	[ "$DIR" != "." ] && ln -f -s `echo "$DIR" | sed 's;/[^/]*;/..;g'`/tags "$DIR"/tags
+do	[ "$DIR" != "." ] && ln -f -s `echo "$DIR" | sed 's;/[^/]*;/..;g'`/$tags_file "$DIR"/$tags_file
done

... does this create a tags symlink on each directory? This seems
strange to me,

I don't know the original author's intention for this but I think it
makes use of the tag file in emacs a little bit easier. Emacs
confirms for the first time the default location of tags file under
the same directory where the source file resides. I can just hit
return key if there's a symlink of tags. If we do not create the
symlink, we have to specify the directory where the tags file was
originally created, which is a little bit annoying.

but I admit the hack I keep in my .vim/vimrc looks even
more strange:

: let $CSCOPE_DB=substitute(getcwd(), "^\\(.*/pgsql/source/ [^/]*\\)/.*", "\\1", "")
: let &tags=substitute(getcwd(), "^\\(.*/pgsql/source/[^/]*\\)/.*", "\\1", "") . "/tags"

Not sure which is worse. Having dozens of identically named symlinks
doesn't strike me as a great arrangement though. I would definitely not
use make_ctags if this is unavoidable. I see both scripts do likewise
currently.

Well, I often visit different tags file for different development
projects (for example Pgpool-II) and I don't want to have fixed
location of tags file in emacs setting. For this reason make_etags's
approach is acceptable for me.

Best reagards,
--
Tatsuo Ishii
SRA OSS LLC
English: http://www.sraoss.co.jp/index_en/
Japanese:http://www.sraoss.co.jp

Tatsuo Ishii <ishii@sraoss.co.jp> writes:

If we're going to do this, then I suggest that make_etags should become
a symlink to make_ctags, and behave as if -e is given when called under
that name.

What I had in my mind was making make_etags a script just exec
make_ctags (with -e option). But I don't have strong
preference. Symlink is ok for me too.

I don't think it's possible to store a symlink in git, so
having one file exec the other sounds saner to me too.

regards, tom lane

On 2022-Oct-12, Tom Lane wrote:

Tatsuo Ishii <ishii@sraoss.co.jp> writes:

If we're going to do this, then I suggest that make_etags should become
a symlink to make_ctags, and behave as if -e is given when called under
that name.

What I had in my mind was making make_etags a script just exec
make_ctags (with -e option). But I don't have strong
preference. Symlink is ok for me too.

I don't think it's possible to store a symlink in git, so
having one file exec the other sounds saner to me too.

I tried before my reply and it seems to work, but perhaps it requires
too new a git version. It may also be a problem under Windows. Having
one exec the other sounds perfect.

--
Álvaro Herrera PostgreSQL Developer — https://www.EnterpriseDB.com/

On Wed, Oct 12, 2022 at 10:09:06AM -0400, Tom Lane wrote:

I don't think it's possible to store a symlink in git, so
having one file exec the other sounds saner to me too.

Git handles symlink just fine, but those will obviously create extra burden for
Windows users (git will just create a text file containing the target path I
think), so agreed (even if I doubt that any Windows user will run those
scripts anyway).

On 2022-Oct-12, Tatsuo Ishii wrote:

find . \( -name 'CVS' -prune \) -o \( -name .git -prune \) -o -type d -print |
while read DIR
-do	[ "$DIR" != "." ] && ln -f -s `echo "$DIR" | sed 's;/[^/]*;/..;g'`/tags "$DIR"/tags
+do	[ "$DIR" != "." ] && ln -f -s `echo "$DIR" | sed 's;/[^/]*;/..;g'`/$tags_file "$DIR"/$tags_file
done

... does this create a tags symlink on each directory? This seems
strange to me,

I don't know the original author's intention for this but I think it
makes use of the tag file in emacs a little bit easier. Emacs
confirms for the first time the default location of tags file under
the same directory where the source file resides. I can just hit
return key if there's a symlink of tags. If we do not create the
symlink, we have to specify the directory where the tags file was
originally created, which is a little bit annoying.

OK, that sounds good then. I would make a feature request to have a
switch that supresses creation of these links, then.

Well, I often visit different tags file for different development
projects (for example Pgpool-II) and I don't want to have fixed
location of tags file in emacs setting. For this reason make_etags's
approach is acceptable for me.

Makes sense.

--
Álvaro Herrera 48°01'N 7°57'E — https://www.EnterpriseDB.com/
Thou shalt study thy libraries and strive not to reinvent them without
cause, that thy code may be short and readable and thy days pleasant
and productive. (7th Commandment for C Programmers)

On Wed, Oct 12, 2022 at 10:26:03PM +0900, Tatsuo Ishii wrote:

However ... hmm ...

find . \( -name 'CVS' -prune \) -o \( -name .git -prune \) -o -type d -print |
while read DIR
-do	[ "$DIR" != "." ] && ln -f -s `echo "$DIR" | sed 's;/[^/]*;/..;g'`/tags "$DIR"/tags
+do	[ "$DIR" != "." ] && ln -f -s `echo "$DIR" | sed 's;/[^/]*;/..;g'`/$tags_file "$DIR"/$tags_file
done

... does this create a tags symlink on each directory? This seems
strange to me,

I don't know the original author's intention for this but I think it
makes use of the tag file in emacs a little bit easier. Emacs
confirms for the first time the default location of tags file under
the same directory where the source file resides. I can just hit
return key if there's a symlink of tags. If we do not create the
symlink, we have to specify the directory where the tags file was
originally created, which is a little bit annoying.

Yes, that is exactly the intent of why it uses symlinks in every
directory.

--
Bruce Momjian <bruce@momjian.us> https://momjian.us
EDB https://enterprisedb.com

Indecision is a decision. Inaction is an action. Mark Batterson

On 10.10.22 12:04, Alvaro Herrera wrote:

In my own tags script I just call "ctags -R", and I feed cscope with
these find lines

(find $SRCDIR \( -name tmp_install -prune -o -name tmp_check -prune \) -o \( -name "*.[chly]" -o -iname "*makefile*" -o -name "*.mk" -o -name "*.in" -o -name "*.sh" -o -name "*.sgml" -o -name "*.sql" -o -name "*.p[lm]" \) -type f -print ; \
find $BUILDDIR \( -name tmp_install -prune \) -o \( -name \*.h -a -type f \) -print )

which seems to give decent results. (Nowadays I wonder if it'd be
better to exclude the "*_d.h" files from the builddir.)
(I wonder why don't I have a prune for .git ...)

Or use git ls-files.

OK, that sounds good then. I would make a feature request to have a
switch that supresses creation of these links, then.

Ok, I have added "-n" option to make_ctags so that it skips to create
the links.

Also I have changed make_etags so that it exec make_ctags, which seems
to be the consensus.

Best reagards,
--
Tatsuo Ishii
SRA OSS LLC
English: http://www.sraoss.co.jp/index_en/
Japanese:http://www.sraoss.co.jp

On Thu, 13 Oct 2022 15:35:09 +0900 (JST)
Tatsuo Ishii <ishii@sraoss.co.jp> wrote:

OK, that sounds good then. I would make a feature request to have a
switch that supresses creation of these links, then.

Ok, I have added "-n" option to make_ctags so that it skips to create
the links.

Also I have changed make_etags so that it exec make_ctags, which seems
to be the consensus.

Thank you for following up my patch.
I fixed the patch to allow use both -e and -n options together.

Regards,
Yugo Nagata

Best reagards,
--
Tatsuo Ishii
SRA OSS LLC
English: http://www.sraoss.co.jp/index_en/
Japanese:http://www.sraoss.co.jp

--
Yugo NAGATA <nagata@sraoss.co.jp>

On Thu, 13 Oct 2022 15:35:09 +0900 (JST)
Tatsuo Ishii <ishii@sraoss.co.jp> wrote:

OK, that sounds good then. I would make a feature request to have a
switch that supresses creation of these links, then.

Ok, I have added "-n" option to make_ctags so that it skips to create
the links.

Also I have changed make_etags so that it exec make_ctags, which seems
to be the consensus.

Thank you for following up my patch.
I fixed the patch to allow use both -e and -n options together.

Thanks. I have made mostly cosmetic changes so that it is more
consistent with existing scripts.

I would like to push v6 patch if there's no objection.

Best reagards,
--
Tatsuo Ishii
SRA OSS LLC
English: http://www.sraoss.co.jp/index_en/
Japanese:http://www.sraoss.co.jp

Hi,

On Sat, 15 Oct 2022 10:40:29 +0900 (JST)
Tatsuo Ishii <ishii@sraoss.co.jp> wrote:

On Thu, 13 Oct 2022 15:35:09 +0900 (JST)
Tatsuo Ishii <ishii@sraoss.co.jp> wrote:

OK, that sounds good then. I would make a feature request to have a
switch that supresses creation of these links, then.

Ok, I have added "-n" option to make_ctags so that it skips to create
the links.

Also I have changed make_etags so that it exec make_ctags, which seems
to be the consensus.

Thank you for following up my patch.
I fixed the patch to allow use both -e and -n options together.

Thanks. I have made mostly cosmetic changes so that it is more
consistent with existing scripts.

I would like to push v6 patch if there's no objection.

I am fine with this patch.

By the way, in passing, how about adding "tags" and "TAGS" to
.gitignore file?

Best reagards,
--
Tatsuo Ishii
SRA OSS LLC
English: http://www.sraoss.co.jp/index_en/
Japanese:http://www.sraoss.co.jp

--
Yugo NAGATA <nagata@sraoss.co.jp>

Hi,

On Sat, 15 Oct 2022 10:40:29 +0900 (JST)
Tatsuo Ishii <ishii@sraoss.co.jp> wrote:

On Thu, 13 Oct 2022 15:35:09 +0900 (JST)
Tatsuo Ishii <ishii@sraoss.co.jp> wrote:

OK, that sounds good then. I would make a feature request to have a
switch that supresses creation of these links, then.

Ok, I have added "-n" option to make_ctags so that it skips to create
the links.

Also I have changed make_etags so that it exec make_ctags, which seems
to be the consensus.

Thank you for following up my patch.
I fixed the patch to allow use both -e and -n options together.

Thanks. I have made mostly cosmetic changes so that it is more
consistent with existing scripts.

I would like to push v6 patch if there's no objection.

I am fine with this patch.

Thanks. the v6 patch pushed to master branch.

By the way, in passing, how about adding "tags" and "TAGS" to
.gitignore file?

Sounds like a good idea.

Best reagards,
--
Tatsuo Ishii
SRA OSS LLC
English: http://www.sraoss.co.jp/index_en/
Japanese:http://www.sraoss.co.jp

On Wed, 19 Oct 2022 13:25:17 +0900 (JST)
Tatsuo Ishii <ishii@sraoss.co.jp> wrote:

Hi,

On Sat, 15 Oct 2022 10:40:29 +0900 (JST)
Tatsuo Ishii <ishii@sraoss.co.jp> wrote:

On Thu, 13 Oct 2022 15:35:09 +0900 (JST)
Tatsuo Ishii <ishii@sraoss.co.jp> wrote:

OK, that sounds good then. I would make a feature request to have a
switch that supresses creation of these links, then.

Ok, I have added "-n" option to make_ctags so that it skips to create
the links.

Also I have changed make_etags so that it exec make_ctags, which seems
to be the consensus.

Thank you for following up my patch.
I fixed the patch to allow use both -e and -n options together.

Thanks. I have made mostly cosmetic changes so that it is more
consistent with existing scripts.

I would like to push v6 patch if there's no objection.

I am fine with this patch.

Thanks. the v6 patch pushed to master branch.

Thanks!

By the way, in passing, how about adding "tags" and "TAGS" to
.gitignore file?

Sounds like a good idea.

Ok, the patch is attached.

Regards,
Yugo Nagata

--
Yugo NAGATA <nagata@sraoss.co.jp>

By the way, in passing, how about adding "tags" and "TAGS" to
.gitignore file?

Sounds like a good idea.

Ok, the patch is attached.

I have search the mail archive and found this:

/messages/by-id/CAFcNs+rG-DASXzHcecYKvAj+rmxi8CpMAgbpGpEK-mjC96F=Lg@mail.gmail.com

It seems the consensus was to avoid to put this sort of things into
.gitignore in the PostgreSQL source tree. Rather, put into personal
.gitignore or whatever so that developers don't need to care about
other's preference.

Best reagards,
--
Tatsuo Ishii
SRA OSS LLC
English: http://www.sraoss.co.jp/index_en/
Japanese:http://www.sraoss.co.jp

On Wed, 19 Oct 2022 17:17:17 +0900 (JST)
Tatsuo Ishii <ishii@sraoss.co.jp> wrote:

By the way, in passing, how about adding "tags" and "TAGS" to
.gitignore file?

Sounds like a good idea.

Ok, the patch is attached.

I have search the mail archive and found this:

/messages/by-id/CAFcNs+rG-DASXzHcecYKvAj+rmxi8CpMAgbpGpEK-mjC96F=Lg@mail.gmail.com

It seems the consensus was to avoid to put this sort of things into
.gitignore in the PostgreSQL source tree. Rather, put into personal
.gitignore or whatever so that developers don't need to care about
other's preference.

Ok, I understand. Thanks!

By the way, after executing both make_etags and make_ctags, trying tag jump
in my vim causes the following error even though there are correct tags files.

E431: Format error in tags file "backend/access/heap/TAGS"

Removing all TAGS files as below can resolve this error.
$ find . -name TAGS | xargs rm

So, should we have one more option of make_{ce}tags script to clean up
existing tags/TAGS files?

Best reagards,
--
Tatsuo Ishii
SRA OSS LLC
English: http://www.sraoss.co.jp/index_en/
Japanese:http://www.sraoss.co.jp

--
Yugo NAGATA <nagata@sraoss.co.jp>