Mostly Harmless: Welcoming our C++ friends

Kurt Harriman wrote:

Sometimes people would like to call C++ code in the PostgreSQL
backend environment... for example, in user-defined functions,
triggers, access methods. And there is sometimes a need for
C++ code to call back into PostgreSQL's C functions, such as
the SPI interface.

Have you considered writing a procedural language plugin for C++?
PostgreSQL supports a lot of extension languages, and none of them
require the amount of backend changes that you outline here, because the
PL plugin serves as glue.

A seriously substantial portion of the diff for this patch all is
supporting trivial renaming, like changing everything that uses:

-       TypeName   *typename = (TypeName *) cmd->def;
+       TypeName   *typeName = (TypeName *) cmd->def;

Is that really necessary? After going through a few pages of diff code
where supporting this trivial bit was the only change, my eyes glazed
over. Minimizing diff size makes it much more likely somebody will
complete a review of the functional parts of your submission before
getting bored.

If it is needed, I'd suggest you'd get a warmer reception here submitting
two diffs, one that just did the renaming and a second that actually had
the functional bits in it. Then it would be possible to casually scan the
renaming one for a second to see it was trivial and boring, followed by a
review of the functional one that was focused on its real changes.

--
* Greg Smith gsmith@gregsmith.com http://www.gregsmith.com Baltimore, MD

Hi Greg,

Actually I did email this to the list with 4 separate diffs in
4 separate attached files. I don't know why it appears all
massed together at http://archives.postgresql.org/pgsql-hackers.
I'll try resubmitting them separately. The first diff consists
of just the renaming, which is intentionally trivial and boring.
The second diff adds a few extern "C" {...} declarations - also
trivial and boring, but small. The interesting part is in the
third diff.

Regards,
... kurt

Greg Smith wrote:

On Fri, 5 Dec 2008, Greg Smith wrote:

If it is needed, I'd suggest you'd get a warmer reception here submitting two
diffs, one that just did the renaming and a second that actually had the
functional bits in it.

You can just ignore this late night bit of idiocy, or mock me for it as
you see fit. Note to other reviewers: if your e-mail client is the sort
that bunches a series of text attachments all together, make sure to
scroll completely past the first patch in the diff before you pay
attention to the rest of it. I'm going to bed.

--
* Greg Smith gsmith@gregsmith.com http://www.gregsmith.com Baltimore, MD

(Re-sending just the first of four patches: c++reserved)

These patches are based on CVS head in which the latest commit was
user: petere
date: Thu Dec 04 17:51:28 2008 +0000
summary: Default values for function arguments

1. c++reserved

User-defined functions and extensions may need to access
backend data structures such as parse trees. A few of the
relevant header files contain field or parameter names
which happen to be C++ reserved words. This makes them
unusable from C++ because the compiler chokes on the
reserved word. It has been suggested that the C++ user
could surround these #includes with #defines to substitute
innocuous words for the reserved words; but that would be
unbearably kludgy, error prone and unmaintainable. A polite
host does not demand such things of a guest.

Fortunately, there are not many instances which are likely
to be encountered by our C++ guests, and these can easily
be changed. In memnodes.h, parsenodes.h, and primnodes.h,
this patch changes the following field names:

typename => typeName
typeid => typeOid
using => usingClause
delete => delete_context

Also, the patch changes a few parameter names in function
prototypes in makefuncs.h, parse_type.h, and builtins.h:

typename => typeName
typeid => typeOid
namespace => qualifier

There's no need to ask PostgreSQL developers to remember to
avoid C++ reserved words, because C++ users who are affected
by such occurrences can be asked to submit a corrective patch.

(Re-sending just the second of four patches: c++bookends)

These patches are based on CVS head in which the latest commit was
user: petere
date: Thu Dec 04 17:51:28 2008 +0000
summary: Default values for function arguments

2. c++bookends

C++ code can call C functions and share global variables with C,
provided those declarations are surrounded by "bookends":

extern "C" {
...
};

Header files can be made bilingual, to declare interfaces which
look the same to both C and C++ callers. This is done by
placing C++ bookends within the header file, guarded by #ifdefs

#ifdef __cplusplus
extern "C" {
#endif
...
#ifdef __cplusplus
}; /* extern "C" */
#endif

This way the C++ caller can just #include the header file without
worrying whether the interface is implemented in C or C++.

Usually, extension modules written in C++ will put bookends around
all of their PostgreSQL #includes.

However, "postgres.h" usually stands alone as the first #include,
followed by some system #includes, and then the rest of the
PostgreSQL #includes. It is much nicer if a C++ file has just one
pair of bookends around its main block of PostgreSQL #includes.
This patch gives postgres.h its own internal bookends, making it
bilingual, so that its #include can continue to stand alone at the
head of each file.

Just a few additional header files are mentioned in the PostgreSQL
Reference Manual for add-on developers to use: fmgr.h, funcapi.h,
and spi.h. This patch adds bookends within those three files for
the benefit of beginners writing very simple extensions in C++.
Documentation and learning are simplified because C example code
can be compiled as C or C++ without change.

(Re-sending just the third of four patches: c++configure)

These patches are based on CVS head in which the latest commit was
user: petere
date: Thu Dec 04 17:51:28 2008 +0000
summary: Default values for function arguments

3. c++configure

This patch adds C++ support to the PostgreSQL build system.

After you have applied the patch, cd to the top of the source
tree (to the directory containing the file 'configure.in') and
execute these two commands to regenerate the 'configure' script
and some related files:
autoconf
autoheader

Much as it already does for the C compiler, the 'configure' script
will try to find the C++ compiler and set up appropriate command
line options. If 'configure' finds a C++ compiler, it will set up
src/Makefile.global to define the following makefile variables:

CXX = command for invoking C++ compiler
CXXCPP = command for invoking C++ preprocessor
CXXFLAGS = C++ compiler options
GXX = 'yes' if the C++ compiler is gcc/g++

Implicit rules are defined so that gmake will automatically invoke
the C++ compiler using the above variables given a source file name
suffixed with '.cpp' or '.cc'. So, to add a file named marvin.cpp
to the build, just add 'marvin.o' to the OBJS list in the Makefile.

To C++-compile a file with '.c' suffix, the Makefile should list
the .o file in both OBJS and CXXOBJS.

The pg_config utility can be used to display the CXX and CXXFLAGS.

Most C++ code typically uses some C++ features whose implementation
makes use of the compiler's runtime library: exceptions, static
constructors, new/delete, STL containers, stream I/O, etc. Specify
the following 'configure' option to link the C++ runtime library
into the postgres backend:

--enable-cplusplus

If --enable-cplusplus is specified, the makefile variable
'enable_cplusplus' will be set to 'yes', and pg_config.h will
#define ENABLE_CPLUSPLUS.

To ensure that the C++ runtime library is properly initialized,
on some platforms it is necessary for the main() function to be
compiled as C++. Therefore, if --enable-cplusplus is configured,
src/backend/main/main.c will be compiled as C++. This is
handled by the following lines in src/backend/main/Makefile:

ifeq ($(enable_cplusplus),yes)
CXXOBJS = main.o
endif

Fortunately, main.c can be compiled as either C or C++ with no
difficulty after applying the c++reserved and c++bookends
patches. To make main.c bilingual, all that was needed was
a pair of bookends around its #includes.

Limitations:

- I haven't added support for profiling and code coverage for
C++. Automatic dependency generation is supported, however.

- This ought to work on platforms which use GCC, and maybe some
others. The only one I have tested is x86_32 Linux with GCC
4.1.2. Hopefully some interested hackers will try it on
platforms to which they have access, and post the results.

(Re-sending just the fourth of four patches: c++exception)

These patches are based on CVS head in which the latest commit was
user: petere
date: Thu Dec 04 17:51:28 2008 +0000
summary: Default values for function arguments

4. c++exception

When C code calls C++ code, all C++ exceptions need to be caught
and fully contained within the C++ code. Exceptions should never
be thrown outward across the C/C++ frontier.

If an exception is not caught within C++ code, and the search for
a matching 'catch' bumps into a C stack frame, the result may be
platform dependent. On my platform (Linux/GCC), if this happens
in the postgres backend, the process terminates silently as if
abort() had been called.

With this patch, if --enable-cplusplus is configured,
PostgresMain defines a handler to intercept any uncaught C++
exception and convert it to a conventional PostgreSQL error of
FATAL severity. This allows the backend to clean up and report
the error before terminating.

On Fri, 5 Dec 2008, Kurt Harriman wrote:

Actually I did email this to the list with 4 separate diffs in
4 separate attached files. I don't know why it appears all
massed together at http://archives.postgresql.org/pgsql-hackers.

Thanks for being so attentive. Your e-mail was fine, the archives just
mash multiple text-based attachments tagged as Text/PLAIN together like
that, as does my mail reader. 1522 lines of only renaming in part 1, no
wonder I gave up.

I see you've already started re-sending the individual parts as separate
messages; that's nice for initial casual browsing of them, but will get
boring for everybody if you do that every time. If you've got more than
one text file to attach, for future updates you might consider a tar
archive of them to keep them running together in the archives. Once
you're putting stuff into an archive, might as well compress it too,
particularly for a patch of this size.

--
* Greg Smith gsmith@gregsmith.com http://www.gregsmith.com Baltimore, MD

On Friday 05 December 2008 03:55, Peter Eisentraut wrote:

Kurt Harriman wrote:

Sometimes people would like to call C++ code in the PostgreSQL
backend environment... for example, in user-defined functions,
triggers, access methods. And there is sometimes a need for
C++ code to call back into PostgreSQL's C functions, such as
the SPI interface.

Have you considered writing a procedural language plugin for C++?
PostgreSQL supports a lot of extension languages, and none of them
require the amount of backend changes that you outline here, because the
PL plugin serves as glue.

I think this patch is great, although I haven't had time to test it yet. The
only real "backend change" is the exception-handling clause; and the fact
that the backend will also be linked against the C++ runtime library.
Everything else is routine stuff that an experienced C++ developer would end
up catching while trying to get his build-system for a new project running;
but it could certainly scare away someone with less experience. Better to
deal with this way ahead of time and test it on a few platforms.

--
David Lee Lambert ... Software Developer
Cell phone: +1 586-873-8813 ; alt. email <dlambert@bmtcarhaul.com> or
<lamber45@msu.edu>
GPG key at http://www.lmert.com/keyring.txt

Greg Smith <gsmith@gregsmith.com> writes:

You can just ignore this late night bit of idiocy, or mock me for it as you see
fit. Note to other reviewers: if your e-mail client is the sort that bunches
a series of text attachments all together, make sure to scroll completely past
the first patch in the diff before you pay attention to the rest of it. I'm
going to bed.

Your email client is doing the right thing. The attachments had the following
header on them which controls this:

Content-Disposition: inline;

I wonder how many email clients let the poster control this header though :(
If you post with content-disposition set to "attachment" instead of "inline"
it should appear as a separate file you can save.

Regarding the patches, we could apply the trivial stuff right around the time
of the pgindent run, after all the major patches are drained from the queue so
it doesn't cause extra conflicts. It would still cause any other pending
patches for 8.5 to bitrot but from the sounds of things shouldn't be too hard
to fix up.

It seems to me we ought to do this regardless of whether we apply the
functional changes.

--
Gregory Stark
EnterpriseDB http://www.enterprisedb.com
Ask me about EnterpriseDB's On-Demand Production Tuning

Hi Peter,

Have you considered writing a procedural language plugin for C++?

C++ can masquerade as C, so I don't think it needs a separate
plugin. Just tell PostgreSQL that your user-defined function
is C even though you secretly know it is C++.

This series of patches is meant to address some of the
mechanics of getting C++ code compiled and linked for the
PostgreSQL backend environment.

At present the build system has no support for languages other
than C. To interface PostgreSQL to a C++ tool or library, it's
necessary to either hack the PostgreSQL build system, or bypass
it and set up your own build system. Either alternative is
likely to be non-portable and difficult for others to understand
or use. This presents a serious obstacle to contributing the
code to the PostgreSQL community or sharing it with others.

Because C++ is so similar to C, the PostgreSQL build system can
easily provide equal support to both languages. C++ users can
then integrate their code easily and portably, and others can
share the code with no need to wrestle with jury-rigged
makefiles. Nobody should have to figure out autoconf, m4,
and gmake unless they want to.

The 'c++configure' patch therefore addresses the necessity to
find the host platform's C++ compiler; invoke it with appropriate
options; link with the C++ runtime library; and initialize the
C++ environment.

Another obstacle which would not be addressed by a procedural
language plugin is the problem of access to the backend's APIs
and data structures. C++ can use C declarations directly with
no extra wrappers or translation layers such as other languages
usually need. However, C++ cannot parse a C header file in
which a C++ reserved word is used as an identifier. The
'c++reserved' patch renames some fields in a very few header
files so C++ code can interface with the PostgreSQL backend
environment to the extent needed for implementing a procedural
language, data type, etc. Although tedious, such renaming is
by far the most reliable, maintainable and efficient means of
exposing the PostgreSQL runtime facilities to C++. As a
straightforward renaming, it is a safe change: its completeness
and much of its correctness are checked by the C compiler.

PostgreSQL supports a lot of extension languages, and none of
them require the amount of backend changes that you outline here,
because the PL plugin serves as glue.

C++ doesn't need glue like those other languages, but it does need
just a little help so that it can be used for the same kinds of
jobs that C is used for.

Those other extension languages, such as plpgsql or plpython,
serve a different audience than C or C++. They offer quick
development, ease of use, and high-level expressiveness where
performance is not the primary concern.

C or C++ are chosen when high performance is needed with precise
control over data representation and the ability to interoperate
directly with almost any language / library / system call / network
protocol / etc - notably, PostgreSQL's own tree structures and
data types.

Thanks for your comments; I hope I've responded adequately.
In any case, I welcome further dialogue on these or other topics.

Regards,
... kurt

Peter Eisentraut wrote:

Kurt Harriman <harriman@acm.org> writes:

Hi Peter,

Have you considered writing a procedural language plugin for C++?

C++ can masquerade as C, so I don't think it needs a separate
plugin. Just tell PostgreSQL that your user-defined function
is C even though you secretly know it is C++.

Well one thing that might be useful for a c++ procedural language would be
catching C++ exceptions and translating them into ereports which could then be
caught in Postgres.

That's actually what I thought you had done but I just reread your mail and
realized you only handled unhandled exceptions which cause the backend to die.

The other way around could be useful too -- catching ereports/elogs within a
backend API call from C++ code and throwing a C++ exception. I'm not sure if
that's doable though.

--
Gregory Stark
EnterpriseDB http://www.enterprisedb.com
Ask me about EnterpriseDB's RemoteDBA services!

Kurt Harriman wrote:

Have you considered writing a procedural language plugin for C++?

C++ can masquerade as C, so I don't think it needs a separate
plugin. Just tell PostgreSQL that your user-defined function
is C even though you secretly know it is C++.

FYI, we have received patches morally equivalent to yours many times
over the years, and they have all been rejected. You might want to
review the archives about that.

Kurt Harriman <harriman@acm.org> writes:

[ make the backend C++-compilable ]

This has been proposed before, and rejected before, and I don't believe
the arguments have changed in the least.

regards, tom lane

Gregory Stark <stark@enterprisedb.com> writes:

Well one thing that might be useful for a c++ procedural language would be
catching C++ exceptions and translating them into ereports which could then be
caught in Postgres.

That's actually what I thought you had done but I just reread your mail and
realized you only handled unhandled exceptions which cause the backend to die.

Well, that's too bad, because fixing the error-handling impedance
mismatch is the one thing that's been missing from every previous
proposal, and it's the one thing that might actually make C++ in the
backend useful rather than a toy.

It's possible/likely that this would be easier to do in the context of
a PL; that would at least provide a single point at which to catch
C++ exceptions and turn them into elogs. The hard part is turning
elogs into exceptions so that errors thrown by core backend functions
that're called by the C++ code will behave as a C++ programmer would
expect.

For comparison look at the way that errors are handled in pl/perl etc.
The relatively narrow "SPI" API for calling back into the main backend
makes it somewhat sane to convert elogs into Perl errors, though it's
less efficient than one could wish. I don't know how to scale that
solution up to the point where you could call any random internal
backend function, as Kurt seems to be hoping for.

regards, tom lane

Peter Eisentraut wrote:

FYI, we have received patches morally equivalent to yours many times
over the years, and they have all been rejected. You might want to
review the archives about that.

Hi Peter,

I went back as far as 2005 in the archives, and found only this thread
covering similar territory:

* SPI-header-files safe for C++-compiler
http://groups.google.com/group/pgsql.patches/browse_frm/thread/6dab9a8134cce102/3c91e40a9e615362?lnk=gst&amp;q=rejected+c%2B%2B#3c91e40a9e615362
or http://tinyurl.com/6bjcdq
June 2007

That patch appears approximately equivalent to the first two of my
patches, c++reserved and c++bookends. The opposing arguments in
that thread seem strained, in my opinion, conveying more heat than
light. "Hey you kids, get off my lawn!" The conclusion of the
deliberative process wasn't set down in that thread; but evidently
the arguments in favor were not sufficiently persuasive: in the
end, the patch was not applied.

The foremost opposing argument seems to have been that there
should be no attempt to alleviate the existing reserved word
problem without automatic enforcement to guarantee that never
in the future can new occurrences be introduced.

But can we not separate the two problems of (1) actual identifiers
which prevent C++ compilation today, vs. (2) hypothetical code which
someone might submit in the future? The first problem is immediate;
the second would only be troublesome if the hypothetical identifier
makes it all the way through beta testing into a release.

Post #21 in the thread, by Tom Lane on July 4 2007 at 8:05 am,
suggests an automated check for non-C++-compilable header files,
and highlights the practical difficulties caused by lack of C++
support in the build system. To invoke the C++ compiler at
present, typically one would use a hard-wired compiler name
with hard-wired flags and paths. My third patch - c++configure -
begins to address the need for a portable way to build C++ code,
compatible with the way we build C code.

The notion of converting all of PostgreSQL to C++ was touched upon.
Little would be gained, at the cost of much tedium, so I advise
against it. I wouldn't want to redo the old stuff unless there's a
clear benefit. My proposal aims to make C++ a practical choice for
adding *new* things to PostgreSQL.

A topic of great interest is the relationship between C++ exceptions
and the PostgreSQL backend's error handling based on setjmp/longjmp.
My fourth patch - c++exception - adds a backstop to limit the damage
in case a C++ exception is thrown from anywhere in the backend and
not caught. The patch converts an uncaught exception to a PostgreSQL
FATAL error, so the process can clean itself up and report its failure
rather than just silently disappearing. If C++ code doesn't catch its
exceptions, that is a programming error, similar to a segment
violation or null pointer dereference, and worthy of termination.

These four patches aren't meant to create a palace of luxury for
C++ programmers. More could be done: more sophisticated error
handling could be provided; new/delete could be hooked up to
palloc/pfree; templates and class libraries could be written.
But C++ programmers can do these things for themselves, if we
give them a fighting chance: just take away the roadblocks
(primarily the reserved words) and make it easy to compile and
link.

Regards,
... kurt

PS. A few other threads had (at least somewhat) relevant discussion.
They're listed below. I didn't find any other patches. I'd appreciate
any links or pointers to any other threads which I should look at.

* STL problem in stored procedures
http://groups.google.com/group/pgsql.general/browse_frm/thread/ee352086139df2bf/400e8133b3e87d74?tvc=1&amp;q=stl+problem+in+stored+procedures#400e8133b3e87d74
http://tinyurl.com/5hhf2v
October 2005

* C++ -> C : Module for converting the WHERE clause to the canonical form with PostgreSQL
http://groups.google.com/group/pgsql.hackers/browse_frm/thread/6cb99c3521318653/d6f2b9509cda35c5?lnk=gst&amp;q=tom+lane+c%2B%2B#d6f2b9509cda35c5
or http://tinyurl.com/6atqmq
January 2006

* PG Extensions: Must be statically linked?
http://groups.google.com/group/pgsql.hackers/browse_frm/thread/89d3650c52430186/c63c94680b399827?lnk=gst&amp;q=pg+extensions+must+be+statically+linked#c63c94680b399827
or http://tinyurl.com/6q5jdz
March 2006

* Writing triggers in C++
http://groups.google.com/group/pgsql.hackers/browse_frm/thread/2a95d656b8add4dd/ded7ff4ce06ae456?lnk=gst&amp;q=writing+triggers+in+c%2B%2B#ded7ff4ce06ae456
or http://tinyurl.com/6kx8ba
February 2007

Tom Lane wrote:

Kurt Harriman <harriman@acm.org> writes:

[ make the backend C++-compilable ]

This has been proposed before, and rejected before, and I don't believe
the arguments have changed in the least.

Hi Tom,

Of the series of four patches, the first two (c++reserved and c++bookends)
appear much the same as Jacob Rief's patch submitted in June 2007.
http://tinyurl.com/6bjcdq
(Incidentally, I only just now found that thread. I hadn't seen it
earlier because I hadn't searched the now-defunct pgsql-patches list.)

The third patch (c++configure) addresses a problem which I have not
seen discussed before: There needs to be a portable way to compile
and link C++ code.

As it stands, the third patch depends upon the first two, because the
third one can optionally compile main.c as C++. Since main.c includes
some header files in which C++ reserved words are used as identifiers,
it cannot be compiled as C++ without fixing at least a subset of those
identifiers.

However, if it is decided that the identifiers cannot be changed, then
I could revise the c++configure and c++exception patches to remove the
dependency.

Of course it can be expected that, once or twice a year, some
starry-eyed newcomer will repeat the plea for the reserved words
to be fixed, until you succumb or add it to the FAQ.

If a C++ programmer needs ereport(ERROR)s to be recast as C++
exceptions, I propose they can handle that in their own code
without special provisions being made in PostgreSQL code.
Therefore, I claim it does not need to be addressed in this
series of patches. It is a separate issue.

However, PostgreSQL code should provide a last-resort exception handler
as a backstop against C++ programming errors. That is the purpose of
the fourth patch (c++exception). C++ programmers should catch their
own exceptions, but if they let one get away, PostgresMain should try
to make sure the shared segment isn't left in a corrupt state. In
other words, PostgresMain's defense against uncaught C++ exceptions
should be approximately equivalent to its existing defense against
SIGSEGVs, null pointer dereferencing errors, and similar faults in C.

Regards,
... kurt

On Friday 05 December 2008 10:45, Kurt Harriman wrote:

Tom Lane wrote:

Kurt Harriman <harriman@acm.org> writes:

[ make the backend C++-compilable ]

I tested applying this patch to CVS HEAD today and compiling
with --enable-cplusplus with gcc 4.2:

$ ldd postmaster
...
libstdc++.so.6 => /usr/lib/libstdc++.so.6 (0xb7bf9000)
...

Then I ran pgbench and played with a table with a UUID column. Performance
was great.

(I first mistakenly applied it against a not-up-to-date source-tree ---
something from mid-September --- and that ended up not compiling.)

I still have not tried this with my own C++ code, but it seems to have less
impact on the build process than some might have feared.

--
David Lee Lambert ... Software Developer
Cell phone: +1 586-873-8813 ; alt. email <dlambert@bmtcarhaul.com> or
<lamber45@msu.edu>
GPG key at http://www.lmert.com/keyring.txt