Official devcontainer config

On Thu, 1 Aug 2024 at 16:56, Junwang Zhao <zhjwpku@gmail.com> wrote:

I post my daily used devcontainer config[2] , Jelte(ccd)
suggested that it might be a good idea we integrate the
config into postgres repo so that the barrier to entry for
new developers will be much lower.

In my experience adding a devcontainer config has definitely made it
easier for people to contribute to Citus. So thank you for working on
this! This is not a full review, but an initial pass.

After diving into the container, the postCreateCommand.sh
will be automatically called, it will do some configurations
like git, perf, .vscode, core_pattern, etc. It also downloads
michaelpq's pg_plugins and FlameGraph.

I think the .git settings don't fit well here, they are mostly aliases
which are very much based on personal preference and not related to
Postgres development. It seems better recommend users to use the
devcontainer dotfiles support for this:
https://code.visualstudio.com/docs/devcontainers/containers#_personalizing-with-dotfile-repositories

- pgindent

It might make sense to install Tristan (ccd) his Postgres Hacker
extension for vscode to make this a bit more userfriendly:
https://marketplace.visualstudio.com/items?itemName=tristan957.postgres-hacker

On 2024-08-01 Th 10:56 AM, Junwang Zhao wrote:

Stack Overflow 2024 developer survey[1] said VSCode
is the most used development environment.

In a PostgreSQL Hacker Mentoring discussion, we talked
about how to use vscode to debug and running postgres,
Andrey(ccd) has tons of tips for new developers, and
I post my daily used devcontainer config[2] , Jelte(ccd)
suggested that it might be a good idea we integrate the
config into postgres repo so that the barrier to entry for
new developers will be much lower.

**Note**

This is not intended to change the workflow of experienced
hackers, it is just hoping to make the life easier for
beginner developers.

**How to use**

Open VSCode Command Palette(cmd/ctrl + shift + p),
search devcontainer, then choose something like
`Dev containers: Rebuild and Reopen in Container`, you are
good to go.

**About the patch**

devcontainer.json:

The .devcontainer/devcontainer.json is the entry point for
VSCode to *open folder to develop in a container*, it will build
the docker image for the first time you open in container,
this will take some time.

There are some parameters(runArgs) for running the container,
we need some settings and privileges to run perf or generate
core dumps.

It has a mount point mapping the hosts $HOME/freedom
to container's /opt/freedom, I chose the name *freedom*
because the container is kind of a jail.

It also installed some vscode extensions and did some
customizations.

After diving into the container, the postCreateCommand.sh
will be automatically called, it will do some configurations
like git, perf, .vscode, core_pattern, etc. It also downloads
michaelpq's pg_plugins and FlameGraph.

Dockerfile:

It is based on debian bookworm, it installed dependencies
to build postgres, also IPC::Run to run TAP tests I guess.

It also has a .gdbinit to break elog.c:errfinish for elevel 21,
this will make the debugging easier why error is logged.

gdbpg.py is adapted from https://github.com/tvondra/gdbpg,
I think putting it here will make it evolve as time goes.

tasks.json:

This is kind of like a bookkeeping for developers, it has
the following commands:

- meson debug setup
- meson release setup
- ninja build
- regression tests
- ninja install
- init cluster
- start cluster
- stop cluster
- install pg_bsd_indent
- pgindent
- apply patch
- generate flamegraph

launch.json:

It has one configuration that makes it possible to attach
to one process(e.g. postgres backend) and debug
with vscode.

PFA and please give it a try if you are a VSCode user.

[1]: https://survey.stackoverflow.co/2024/technology#1-integrated-development-environment
[2]: https://github.com/atomicdb/devcontainer/tree/main/postgres

Not totally opposed, and I will probably give it a try very soon, but
I'm wondering if this really needs to go in the core repo. We've
generally shied away from doing much in the way of editor / devenv
support, trying to be fairly agnostic. It's true we carry .dir-locals.el
and .editorconfig, so that's not entirely true, but those are really
just about supporting our indentation etc. standards.

Also, might it not be better for this to be carried in a separate repo
maintained by people using vscode? I don't know how may committers do.
Maybe lots do, and I'm just a dinosaur. If vscode really needs
.devcontainer to live in the code root, maybe it could be symlinked.
Another reason not carry the code ourselves is that it will make it less
convenient for people who want to customize it.

Without having tried it, looks like a nice effort though. Well done.

cheers

andrew

--
Andrew Dunstan
EDB: https://www.enterprisedb.com

On 01.08.24 23:38, Andrew Dunstan wrote:

Not totally opposed, and I will probably give it a try very soon, but
I'm wondering if this really needs to go in the core repo. We've
generally shied away from doing much in the way of editor / devenv
support, trying to be fairly agnostic. It's true we carry .dir-locals.el
and .editorconfig, so that's not entirely true, but those are really
just about supporting our indentation etc. standards.

Yeah, the editor support in the tree ought to be minimal and factual,
based on coding standards and widely recognized best practices, not a
collection of one person's favorite aliases and scripts. If the scripts
are good, let's look at them and maybe put them under src/tools/ for
everyone to use. But a lot of this looks like it will requite active
maintenance if output formats or node formats or build targets etc.
change. And other things require specific local paths. That's fine for
a local script or something, but not for a mainline tool that the
community will need to maintain.

I suggest to start with a very minimal configuration. What are the
settings that absolute everyone will need, maybe to set indentation
style or something.

On 2024-08-02 Fr 2:45 PM, Peter Eisentraut wrote:

On 01.08.24 23:38, Andrew Dunstan wrote:

Not totally opposed, and I will probably give it a try very soon, but
I'm wondering if this really needs to go in the core repo. We've
generally shied away from doing much in the way of editor / devenv
support, trying to be fairly agnostic. It's true we carry
.dir-locals.el and .editorconfig, so that's not entirely true, but
those are really just about supporting our indentation etc. standards.

Yeah, the editor support in the tree ought to be minimal and factual,
based on coding standards and widely recognized best practices, not a
collection of one person's favorite aliases and scripts.  If the
scripts are good, let's look at them and maybe put them under
src/tools/ for everyone to use.  But a lot of this looks like it will
requite active maintenance if output formats or node formats or build
targets etc. change.  And other things require specific local paths. 
That's fine for a local script or something, but not for a mainline
tool that the community will need to maintain.

I suggest to start with a very minimal configuration. What are the
settings that absolute everyone will need, maybe to set indentation
style or something.

I believe you can get VS Code to support editorconfig, so from that POV
maybe we don't need to do anything.

I did try yesterday with the code from the OP's patch symlinked into my
repo, but got an error with the Docker build, which kinda reinforces
your point.

Your point about "one person's preferences" is well taken - some of the
git aliases supplied  clash with mine.

cheers

andrew

--
Andrew Dunstan
EDB: https://www.enterprisedb.com

On Fri, Aug 2, 2024 at 12:29 AM Jelte Fennema-Nio <postgres@jeltef.nl> wrote:

On Thu, 1 Aug 2024 at 16:56, Junwang Zhao <zhjwpku@gmail.com> wrote:

I post my daily used devcontainer config[2] , Jelte(ccd)
suggested that it might be a good idea we integrate the
config into postgres repo so that the barrier to entry for
new developers will be much lower.

In my experience adding a devcontainer config has definitely made it
easier for people to contribute to Citus. So thank you for working on
this! This is not a full review, but an initial pass.

After diving into the container, the postCreateCommand.sh
will be automatically called, it will do some configurations
like git, perf, .vscode, core_pattern, etc. It also downloads
michaelpq's pg_plugins and FlameGraph.

I think the .git settings don't fit well here, they are mostly aliases
which are very much based on personal preference and not related to
Postgres development. It seems better recommend users to use the
devcontainer dotfiles support for this:
https://code.visualstudio.com/docs/devcontainers/containers#_personalizing-with-dotfile-repositories

- pgindent

It might make sense to install Tristan (ccd) his Postgres Hacker
extension for vscode to make this a bit more userfriendly:
https://marketplace.visualstudio.com/items?itemName=tristan957.postgres-hacker

Good to know, I will try this later.

--
Regards
Junwang Zhao

On Fri, Aug 2, 2024 at 5:38 AM Andrew Dunstan <andrew@dunslane.net> wrote:

On 2024-08-01 Th 10:56 AM, Junwang Zhao wrote:

Stack Overflow 2024 developer survey[1] said VSCode
is the most used development environment.

In a PostgreSQL Hacker Mentoring discussion, we talked
about how to use vscode to debug and running postgres,
Andrey(ccd) has tons of tips for new developers, and
I post my daily used devcontainer config[2] , Jelte(ccd)
suggested that it might be a good idea we integrate the
config into postgres repo so that the barrier to entry for
new developers will be much lower.

**Note**

This is not intended to change the workflow of experienced
hackers, it is just hoping to make the life easier for
beginner developers.

**How to use**

Open VSCode Command Palette(cmd/ctrl + shift + p),
search devcontainer, then choose something like
`Dev containers: Rebuild and Reopen in Container`, you are
good to go.

**About the patch**

devcontainer.json:

The .devcontainer/devcontainer.json is the entry point for
VSCode to *open folder to develop in a container*, it will build
the docker image for the first time you open in container,
this will take some time.

There are some parameters(runArgs) for running the container,
we need some settings and privileges to run perf or generate
core dumps.

It has a mount point mapping the hosts $HOME/freedom
to container's /opt/freedom, I chose the name *freedom*
because the container is kind of a jail.

It also installed some vscode extensions and did some
customizations.

After diving into the container, the postCreateCommand.sh
will be automatically called, it will do some configurations
like git, perf, .vscode, core_pattern, etc. It also downloads
michaelpq's pg_plugins and FlameGraph.

Dockerfile:

It is based on debian bookworm, it installed dependencies
to build postgres, also IPC::Run to run TAP tests I guess.

It also has a .gdbinit to break elog.c:errfinish for elevel 21,
this will make the debugging easier why error is logged.

gdbpg.py is adapted from https://github.com/tvondra/gdbpg,
I think putting it here will make it evolve as time goes.

tasks.json:

This is kind of like a bookkeeping for developers, it has
the following commands:

- meson debug setup
- meson release setup
- ninja build
- regression tests
- ninja install
- init cluster
- start cluster
- stop cluster
- install pg_bsd_indent
- pgindent
- apply patch
- generate flamegraph

launch.json:

It has one configuration that makes it possible to attach
to one process(e.g. postgres backend) and debug
with vscode.

PFA and please give it a try if you are a VSCode user.

[1]: https://survey.stackoverflow.co/2024/technology#1-integrated-development-environment
[2]: https://github.com/atomicdb/devcontainer/tree/main/postgres

Not totally opposed, and I will probably give it a try very soon, but
I'm wondering if this really needs to go in the core repo. We've
generally shied away from doing much in the way of editor / devenv
support, trying to be fairly agnostic. It's true we carry .dir-locals.el
and .editorconfig, so that's not entirely true, but those are really
just about supporting our indentation etc. standards.

Also, might it not be better for this to be carried in a separate repo
maintained by people using vscode? I don't know how may committers do.
Maybe lots do, and I'm just a dinosaur. If vscode really needs
.devcontainer to live in the code root, maybe it could be symlinked.
Another reason not carry the code ourselves is that it will make it less
convenient for people who want to customize it.

Agree, if finally we make this into a separate repo, that can also
benefit new developers.

Without having tried it, looks like a nice effort though. Well done.

cheers

andrew

--
Andrew Dunstan
EDB: https://www.enterprisedb.com

--
Regards
Junwang Zhao

On Sat, Aug 3, 2024 at 2:45 AM Peter Eisentraut <peter@eisentraut.org> wrote:

On 01.08.24 23:38, Andrew Dunstan wrote:

Not totally opposed, and I will probably give it a try very soon, but
I'm wondering if this really needs to go in the core repo. We've
generally shied away from doing much in the way of editor / devenv
support, trying to be fairly agnostic. It's true we carry .dir-locals.el
and .editorconfig, so that's not entirely true, but those are really
just about supporting our indentation etc. standards.

Yeah, the editor support in the tree ought to be minimal and factual,
based on coding standards and widely recognized best practices, not a
collection of one person's favorite aliases and scripts. If the scripts
are good, let's look at them and maybe put them under src/tools/ for
everyone to use. But a lot of this looks like it will requite active
maintenance if output formats or node formats or build targets etc.
change. And other things require specific local paths. That's fine for
a local script or something, but not for a mainline tool that the
community will need to maintain.

Yeah, personal favorite aliases and scripts are not good, that
also concerns me, I will delete those parts in future patches.

I suggest to start with a very minimal configuration. What are the
settings that absolute everyone will need, maybe to set indentation
style or something.

Yeah, reasonable, I will discuss it with Andrey after he tries .devcontainer.

--
Regards
Junwang Zhao

On Sat, Aug 3, 2024 at 7:30 PM Andrew Dunstan <andrew@dunslane.net> wrote:

On 2024-08-02 Fr 2:45 PM, Peter Eisentraut wrote:

On 01.08.24 23:38, Andrew Dunstan wrote:

Not totally opposed, and I will probably give it a try very soon, but
I'm wondering if this really needs to go in the core repo. We've
generally shied away from doing much in the way of editor / devenv
support, trying to be fairly agnostic. It's true we carry
.dir-locals.el and .editorconfig, so that's not entirely true, but
those are really just about supporting our indentation etc. standards.

Yeah, the editor support in the tree ought to be minimal and factual,
based on coding standards and widely recognized best practices, not a
collection of one person's favorite aliases and scripts. If the
scripts are good, let's look at them and maybe put them under
src/tools/ for everyone to use. But a lot of this looks like it will
requite active maintenance if output formats or node formats or build
targets etc. change. And other things require specific local paths.
That's fine for a local script or something, but not for a mainline
tool that the community will need to maintain.

I suggest to start with a very minimal configuration. What are the
settings that absolute everyone will need, maybe to set indentation
style or something.

I believe you can get VS Code to support editorconfig, so from that POV
maybe we don't need to do anything.

I did try yesterday with the code from the OP's patch symlinked into my
repo, but got an error with the Docker build, which kinda reinforces
your point.

The reason symlink does not work is that configure_vscode needs to copy
launch.json and tasks.json into .vscode, it has to be in the
WORKDIR/.devcontainer.

Your point about "one person's preferences" is well taken - some of the
git aliases supplied clash with mine.

Yeah, I will remove that.

cheers

andrew

--
Andrew Dunstan
EDB: https://www.enterprisedb.com

--
Regards
Junwang Zhao

On 2024-08-03 Sa 10:13 PM, Junwang Zhao wrote:

On Sat, Aug 3, 2024 at 7:30 PM Andrew Dunstan <andrew@dunslane.net> wrote:

On 2024-08-02 Fr 2:45 PM, Peter Eisentraut wrote:

On 01.08.24 23:38, Andrew Dunstan wrote:

Not totally opposed, and I will probably give it a try very soon, but
I'm wondering if this really needs to go in the core repo. We've
generally shied away from doing much in the way of editor / devenv
support, trying to be fairly agnostic. It's true we carry
.dir-locals.el and .editorconfig, so that's not entirely true, but
those are really just about supporting our indentation etc. standards.

Yeah, the editor support in the tree ought to be minimal and factual,
based on coding standards and widely recognized best practices, not a
collection of one person's favorite aliases and scripts. If the
scripts are good, let's look at them and maybe put them under
src/tools/ for everyone to use. But a lot of this looks like it will
requite active maintenance if output formats or node formats or build
targets etc. change. And other things require specific local paths.
That's fine for a local script or something, but not for a mainline
tool that the community will need to maintain.

I suggest to start with a very minimal configuration. What are the
settings that absolute everyone will need, maybe to set indentation
style or something.

I believe you can get VS Code to support editorconfig, so from that POV
maybe we don't need to do anything.

I did try yesterday with the code from the OP's patch symlinked into my
repo, but got an error with the Docker build, which kinda reinforces
your point.

The reason symlink does not work is that configure_vscode needs to copy
launch.json and tasks.json into .vscode, it has to be in the
WORKDIR/.devcontainer.

That's kind of awful. Anyway, I think we don't need to do anything about
ignoring those. The user should simply add entries for them to
.git/info/exclude or their local global exclude file (I have
core.excludesfile = /home/andrew/.gitignore set.)

I was eventually able to get it to work without using a symlink.

cheers

andrew

--
Andrew Dunstan
EDB: https://www.enterprisedb.com

Hi hackers,

On Sun, Aug 4, 2024 at 10:12 PM Andrew Dunstan <andrew@dunslane.net> wrote:

On 2024-08-03 Sa 10:13 PM, Junwang Zhao wrote:

On Sat, Aug 3, 2024 at 7:30 PM Andrew Dunstan <andrew@dunslane.net> wrote:

On 2024-08-02 Fr 2:45 PM, Peter Eisentraut wrote:

On 01.08.24 23:38, Andrew Dunstan wrote:

Not totally opposed, and I will probably give it a try very soon, but
I'm wondering if this really needs to go in the core repo. We've
generally shied away from doing much in the way of editor / devenv
support, trying to be fairly agnostic. It's true we carry
.dir-locals.el and .editorconfig, so that's not entirely true, but
those are really just about supporting our indentation etc. standards.

Yeah, the editor support in the tree ought to be minimal and factual,
based on coding standards and widely recognized best practices, not a
collection of one person's favorite aliases and scripts. If the
scripts are good, let's look at them and maybe put them under
src/tools/ for everyone to use. But a lot of this looks like it will
requite active maintenance if output formats or node formats or build
targets etc. change. And other things require specific local paths.
That's fine for a local script or something, but not for a mainline
tool that the community will need to maintain.

I suggest to start with a very minimal configuration. What are the
settings that absolute everyone will need, maybe to set indentation
style or something.

I believe you can get VS Code to support editorconfig, so from that POV
maybe we don't need to do anything.

I did try yesterday with the code from the OP's patch symlinked into my
repo, but got an error with the Docker build, which kinda reinforces
your point.

The reason symlink does not work is that configure_vscode needs to copy
launch.json and tasks.json into .vscode, it has to be in the
WORKDIR/.devcontainer.

That's kind of awful. Anyway, I think we don't need to do anything about
ignoring those. The user should simply add entries for them to
.git/info/exclude or their local global exclude file (I have
core.excludesfile = /home/andrew/.gitignore set.)

I was eventually able to get it to work without using a symlink.

cheers

andrew

--
Andrew Dunstan
EDB: https://www.enterprisedb.com

I did some work on the **devcontainer config** today, hoping this can
address some of the concerns raised on this thread.

1. I created a separate git repo named **Unofficial devcontainer
config for Postgres**[1]https://github.com/pghacking/.devcontainer.
2. The usage is quite simple, clone the repo directly into postgres's tree root,
and reopen in DevContainer (See REAME in [1]https://github.com/pghacking/.devcontainer if you are interested).
3. Using Tristan's vscode code extension postgres-hacker as the formatter,
pgindent will be called on file save, I found this quite convenient.
4. Remove the git settings, Dev container copies the global
~/.gitconfig into the container
by default, so normally it is not necessary to add extra git aliases.

What I haven't addressed is that the repo still uses specific local
paths, I think
this is ok since the code is not going into the core.

One thing I want to ask is, is there any objection to adding the
.devcontainer and .vscode to the .gitignore file?

There are *.vcproj and pgsql.sln in .gitignore, so I guess it's ok to add
.devcontainer and .vscode?

[1]: https://github.com/pghacking/.devcontainer

--
Regards
Junwang Zhao

On 24.08.24 14:49, Junwang Zhao wrote:

What I haven't addressed is that the repo still uses specific local
paths, I think
this is ok since the code is not going into the core.

I'm not among the target users of this, but I imagine that that would
significantly reduce the utility of this for everyone besides you?

One thing I want to ask is, is there any objection to adding the
.devcontainer and .vscode to the .gitignore file?

The standing policy is that files related to IDEs and editors should not
be in our .gitignore, but you can put them into your personal ignore
file somewhere.

There are *.vcproj and pgsql.sln in .gitignore, so I guess it's ok to add
.devcontainer and .vscode?

Those are files generated by the build process, so it is appropriate to
have them there. But in fact, they should have been removed now that
the MSVC build system is done.

On Sat, Aug 24, 2024 at 9:47 PM Peter Eisentraut <peter@eisentraut.org> wrote:

On 24.08.24 14:49, Junwang Zhao wrote:

What I haven't addressed is that the repo still uses specific local
paths, I think
this is ok since the code is not going into the core.

I'm not among the target users of this, but I imagine that that would
significantly reduce the utility of this for everyone besides you?

Yeah, the reason why I started this thread is that we(at least Jelta and I)
think it may make some potential new contributors' lives easier.

One thing I want to ask is, is there any objection to adding the
.devcontainer and .vscode to the .gitignore file?

The standing policy is that files related to IDEs and editors should not
be in our .gitignore, but you can put them into your personal ignore
file somewhere.

Sure, I can put them in global ignore file in various ways.

I just saw the policy in the comment, so I'm ok with it.

There are *.vcproj and pgsql.sln in .gitignore, so I guess it's ok to add
.devcontainer and .vscode?

Those are files generated by the build process, so it is appropriate to
have them there. But in fact, they should have been removed now that
the MSVC build system is done.

--
Regards
Junwang Zhao

On Sat Aug 3, 2024 at 9:00 PM CDT, Junwang Zhao wrote:

On Fri, Aug 2, 2024 at 12:29 AM Jelte Fennema-Nio <postgres@jeltef.nl> wrote:

It might make sense to install Tristan (ccd) his Postgres Hacker
extension for vscode to make this a bit more userfriendly:
https://marketplace.visualstudio.com/items?itemName=tristan957.postgres-hacker

Good to know, I will try this later.

I am open to feature requests and contributions. Here is the repo:

https://git.sr.ht/~tristan957/vscode-postgres-hacker

--
Tristan Partin
Neon (https://neon.tech)