pgsql: Add documentation for the JIT feature.

Started by Andres Freundalmost 8 years ago27 messages
#1Andres Freund
andres@anarazel.de

Add documentation for the JIT feature.

As promised in earlier commits, this adds documentation about the new
build options, the new GUCs, about the planner logic when JIT is used,
and the benefits of JIT in general.

Also adds a more implementation oriented README.

I'm sure we're going to want to expand this further, but I think this
is a reasonable start.

Author: Andres Freund, with contributions by Thomas Munro
Reviewed-By: Thomas Munro
Discussion: /messages/by-id/20170901064131.tazjxwus3k2w3ybh@alap3.anarazel.de

Branch
------
master

Details
-------
https://git.postgresql.org/pg/commitdiff/e6c039d13e16a3a2dec5ba479d9d1fb3229c03a3

Modified Files
--------------
doc/src/sgml/acronyms.sgml | 10 ++
doc/src/sgml/config.sgml | 183 ++++++++++++++++++++++++-
doc/src/sgml/filelist.sgml | 1 +
doc/src/sgml/func.sgml | 8 ++
doc/src/sgml/installation.sgml | 53 ++++++++
doc/src/sgml/jit.sgml | 299 +++++++++++++++++++++++++++++++++++++++++
doc/src/sgml/postgres.sgml | 1 +
doc/src/sgml/storage.sgml | 2 +-
src/backend/jit/README | 289 +++++++++++++++++++++++++++++++++++++++
9 files changed, 844 insertions(+), 2 deletions(-)

#2Simon Riggs
simon@2ndquadrant.com
In reply to: Andres Freund (#1)
Re: pgsql: Add documentation for the JIT feature.

On 28 March 2018 at 22:23, Andres Freund <andres@anarazel.de> wrote:

Add documentation for the JIT feature.

Very nice feature and most welcome but we should call it something
other than just "JIT"

JIT means Just In Time, which could be applied to many concepts and
has been in use for many years in a range of concepts. particularly in
manufacturing/logistics and project management.

The feature is JIT compilation, not just "JIT". If we had replication
with eager consensus, it would be silly to call the feature simply
"eager" with no other qualifier.

Thanks

--
Simon Riggs http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

#3Tom Lane
tgl@sss.pgh.pa.us
In reply to: Andres Freund (#1)
Re: pgsql: Add documentation for the JIT feature.

Andres Freund <andres@anarazel.de> writes:

Add documentation for the JIT feature.

guaibausaurus complains that this broke tarball building:

make -C postgresql-11devel/doc/src/sgml/ INSTALL
make[1]: Entering directory '/home/pgsql/buildfarm/buildroot/HEAD/pgsql.build/postgresql-11devel/doc/src/sgml'
/usr/bin/xsltproc --path . --stringparam pg.version '11devel' --xinclude standalone-profile.xsl standalone-install.xml >INSTALL.xml
/usr/bin/xmllint --noout --valid INSTALL.xml
INSTALL.xml:721: element xref: validity error : IDREF attribute linkend references an unknown ID "jit"
Makefile:115: recipe for target 'INSTALL.html' failed
make[1]: *** [INSTALL.html] Error 4

I'd guess you put a link into installation.sgml that can't be there.

regards, tom lane

#4Alvaro Herrera
alvherre@alvh.no-ip.org
In reply to: Simon Riggs (#2)
Re: pgsql: Add documentation for the JIT feature.

Moving thread to pgsql-hackers.

Simon Riggs wrote:

On 28 March 2018 at 22:23, Andres Freund <andres@anarazel.de> wrote:

Add documentation for the JIT feature.

Very nice feature and most welcome but we should call it something
other than just "JIT"

JIT means Just In Time, which could be applied to many concepts and
has been in use for many years in a range of concepts. particularly in
manufacturing/logistics and project management.

I agree. In some email threads Andres has been using "JIT" as a verb,
too, such as "JITing expressions" and such; that's a bit shocking, in a
way. Honestly I don't care in a pgsql-hackers thread, I mean we all
understand what it means, but in user-facing docs and things we should
use complete words, "JIT-compile", "JIT-compilation", "JIT-compiling"
and so on.

--
�lvaro Herrera https://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

#5John Naylor
jcnaylor@gmail.com
In reply to: Alvaro Herrera (#4)
Re: pgsql: Add documentation for the JIT feature.

I agree. In some email threads Andres has been using "JIT" as a verb,
too, such as "JITing expressions" and such; that's a bit shocking, in a
way. Honestly I don't care in a pgsql-hackers thread, I mean we all
understand what it means, but in user-facing docs and things we should
use complete words, "JIT-compile", "JIT-compilation", "JIT-compiling"
and so on.

Earlier today, I did some web searches to determine how people spell
"JITed" (Andres' spelling), and also found JITted, JIT-ed, JIT'd, and
jitted. No one agrees on that, but it seems very common to use "JIT"
as a verb. See the LLVM docs:

https://llvm.org/docs/DebuggingJITedCode.html

-John Naylor

#6Alvaro Herrera
alvherre@alvh.no-ip.org
In reply to: John Naylor (#5)
Re: pgsql: Add documentation for the JIT feature.

John Naylor wrote:

I agree. In some email threads Andres has been using "JIT" as a verb,
too, such as "JITing expressions" and such; that's a bit shocking, in a
way. Honestly I don't care in a pgsql-hackers thread, I mean we all
understand what it means, but in user-facing docs and things we should
use complete words, "JIT-compile", "JIT-compilation", "JIT-compiling"
and so on.

Earlier today, I did some web searches to determine how people spell
"JITed" (Andres' spelling), and also found JITted, JIT-ed, JIT'd, and
jitted. No one agrees on that, but it seems very common to use "JIT"
as a verb.

Yes: among compiler writers, people who are swimming in jitted bytes all
day long -- a tough bunch if I've seen any. (Not that us here are
sparrows, mind.)

I meant that our docs are for normal people, not *them*.

(I too was thinking about the double 't' there while drafting the above
but decided to leave that concern out.)

--
�lvaro Herrera https://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

#7Tom Lane
tgl@sss.pgh.pa.us
In reply to: Alvaro Herrera (#4)
Re: pgsql: Add documentation for the JIT feature.

Alvaro Herrera <alvherre@alvh.no-ip.org> writes:

Simon Riggs wrote:

JIT means Just In Time, which could be applied to many concepts and
has been in use for many years in a range of concepts. particularly in
manufacturing/logistics and project management.

I agree. In some email threads Andres has been using "JIT" as a verb,
too, such as "JITing expressions" and such; that's a bit shocking, in a
way. Honestly I don't care in a pgsql-hackers thread, I mean we all
understand what it means, but in user-facing docs and things we should
use complete words, "JIT-compile", "JIT-compilation", "JIT-compiling"
and so on.

I'd go a little further and drop "JIT" from user-facing documentation
altogether. Instead refer to the feature as "compilation of expressions"
or some such. JIT is just jargon. Plus, the timing of the compilation is
actually the least important property for our purpose.

regards, tom lane

#8Stephen Frost
sfrost@snowman.net
In reply to: Tom Lane (#7)
Re: pgsql: Add documentation for the JIT feature.

Greetings Tom, all,

* Tom Lane (tgl@sss.pgh.pa.us) wrote:

Alvaro Herrera <alvherre@alvh.no-ip.org> writes:

Simon Riggs wrote:

JIT means Just In Time, which could be applied to many concepts and
has been in use for many years in a range of concepts. particularly in
manufacturing/logistics and project management.

I agree. In some email threads Andres has been using "JIT" as a verb,
too, such as "JITing expressions" and such; that's a bit shocking, in a
way. Honestly I don't care in a pgsql-hackers thread, I mean we all
understand what it means, but in user-facing docs and things we should
use complete words, "JIT-compile", "JIT-compilation", "JIT-compiling"
and so on.

I'd go a little further and drop "JIT" from user-facing documentation
altogether. Instead refer to the feature as "compilation of expressions"
or some such. JIT is just jargon. Plus, the timing of the compilation is
actually the least important property for our purpose.

Agreed.

Thanks!

Stephen

#9David Steele
david@pgmasters.net
In reply to: Stephen Frost (#8)
Re: pgsql: Add documentation for the JIT feature.

On 3/29/18 9:51 AM, Stephen Frost wrote:

Greetings Tom, all,

* Tom Lane (tgl@sss.pgh.pa.us) wrote:

Alvaro Herrera <alvherre@alvh.no-ip.org> writes:

Simon Riggs wrote:

JIT means Just In Time, which could be applied to many concepts and
has been in use for many years in a range of concepts. particularly in
manufacturing/logistics and project management.

I agree. In some email threads Andres has been using "JIT" as a verb,
too, such as "JITing expressions" and such; that's a bit shocking, in a
way. Honestly I don't care in a pgsql-hackers thread, I mean we all
understand what it means, but in user-facing docs and things we should
use complete words, "JIT-compile", "JIT-compilation", "JIT-compiling"
and so on.

I'd go a little further and drop "JIT" from user-facing documentation
altogether. Instead refer to the feature as "compilation of expressions"
or some such. JIT is just jargon. Plus, the timing of the compilation is
actually the least important property for our purpose.

Agreed.

+1. Or simply "expression compilation".

--
-David
david@pgmasters.net

#10Robert Haas
robertmhaas@gmail.com
In reply to: Tom Lane (#7)
Re: pgsql: Add documentation for the JIT feature.

On Thu, Mar 29, 2018 at 9:44 AM, Tom Lane <tgl@sss.pgh.pa.us> wrote:

I'd go a little further and drop "JIT" from user-facing documentation
altogether. Instead refer to the feature as "compilation of expressions"
or some such. JIT is just jargon. Plus, the timing of the compilation is
actually the least important property for our purpose.

I agree that talking about JIT compilation (or just-in-time
compilation) would be better than talking just about JIT, but refusing
to mention JIT seems counter-productive to me. There are a lot of
people who know what just-in-time compilation is and will not realize
that "compilation of expressions" refers to any such technology. If
you don't know what it is, you can Google it. Just typing "jit" into
Google produces a stupid urban dictionary hit and then this:
https://en.wikipedia.org/wiki/Just-in-time_compilation -- and that
contains useful information that you'll never find if you search for
"compilation of expressions".

Also, in a way, you could argue that v10 already did "compilation of
expressions". It didn't compile them to machine language, true, but
it translated them into a form which is faster to execute, and which
is at least arguably a form of bytecode. It's not going to be clear,
even to an expert, that "compilation of expressions" means something
other than that, but if you say JIT, then all of a sudden people know
what we're talking about.

I agree that JIT is jargon, but it's pretty commonly-used jargon.

--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

#11Andres Freund
andres@anarazel.de
In reply to: Robert Haas (#10)
Re: pgsql: Add documentation for the JIT feature.

On 2018-03-29 13:26:31 -0400, Robert Haas wrote:

On Thu, Mar 29, 2018 at 9:44 AM, Tom Lane <tgl@sss.pgh.pa.us> wrote:

I'd go a little further and drop "JIT" from user-facing documentation
altogether. Instead refer to the feature as "compilation of expressions"
or some such. JIT is just jargon. Plus, the timing of the compilation is
actually the least important property for our purpose.

I agree that talking about JIT compilation (or just-in-time
compilation) would be better than talking just about JIT, but refusing
to mention JIT seems counter-productive to me. There are a lot of
people who know what just-in-time compilation is and will not realize
that "compilation of expressions" refers to any such technology. If
you don't know what it is, you can Google it. Just typing "jit" into
Google produces a stupid urban dictionary hit and then this:
https://en.wikipedia.org/wiki/Just-in-time_compilation -- and that
contains useful information that you'll never find if you search for
"compilation of expressions".

Also, in a way, you could argue that v10 already did "compilation of
expressions". It didn't compile them to machine language, true, but
it translated them into a form which is faster to execute, and which
is at least arguably a form of bytecode. It's not going to be clear,
even to an expert, that "compilation of expressions" means something
other than that, but if you say JIT, then all of a sudden people know
what we're talking about.

I agree that JIT is jargon, but it's pretty commonly-used jargon.

Precisely this. I'm very strongly against just saying "expression
compilation", it's just too imprecise. As Robert mentions it could refer
to what we do in v10, it could refer to ahead of time compilation of PL
functions, and it doesn't include compiling tuple deforming. Nor will
it describe compiling sorting, copy or whatnot.

I'm very open however to replacing JITing with JIT compiling and smilar
substitutions.

Greetings,

Andres Freund

#12Andres Freund
andres@anarazel.de
In reply to: Tom Lane (#3)
Re: pgsql: Add documentation for the JIT feature.

On 2018-03-29 01:10:46 -0400, Tom Lane wrote:

Andres Freund <andres@anarazel.de> writes:

Add documentation for the JIT feature.

guaibausaurus complains that this broke tarball building:

make -C postgresql-11devel/doc/src/sgml/ INSTALL
make[1]: Entering directory '/home/pgsql/buildfarm/buildroot/HEAD/pgsql.build/postgresql-11devel/doc/src/sgml'
/usr/bin/xsltproc --path . --stringparam pg.version '11devel' --xinclude standalone-profile.xsl standalone-install.xml >INSTALL.xml
/usr/bin/xmllint --noout --valid INSTALL.xml
INSTALL.xml:721: element xref: validity error : IDREF attribute linkend references an unknown ID "jit"
Makefile:115: recipe for target 'INSTALL.html' failed
make[1]: *** [INSTALL.html] Error 4

I'd guess you put a link into installation.sgml that can't be there.

Thanks for noticing, and thanks Bruce for fixing.

Could we add INSTALL to the all target in src/docs/sgml? It's a bit
awkward that one can build the docs and miss such a mistake. It's fast
to build in comparison to the rest of the docs, so that doesn't seem
like a high price?

Greetings,

Andres Freund

#13Alvaro Herrera
alvherre@alvh.no-ip.org
In reply to: Andres Freund (#12)
Re: pgsql: Add documentation for the JIT feature.

Andres Freund wrote:

Could we add INSTALL to the all target in src/docs/sgml? It's a bit
awkward that one can build the docs and miss such a mistake. It's fast
to build in comparison to the rest of the docs, so that doesn't seem
like a high price?

Maybe it can be tested for in "make check"? That's probably even
faster.

--
�lvaro Herrera https://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

#14Andres Freund
andres@anarazel.de
In reply to: Alvaro Herrera (#13)
Re: pgsql: Add documentation for the JIT feature.

Hi,

On 2018-03-29 15:20:58 -0300, Alvaro Herrera wrote:

Andres Freund wrote:

Could we add INSTALL to the all target in src/docs/sgml? It's a bit
awkward that one can build the docs and miss such a mistake. It's fast
to build in comparison to the rest of the docs, so that doesn't seem
like a high price?

Maybe it can be tested for in "make check"? That's probably even
faster.

Hm, what's wrong just doing it in the normal build? It's a desired build
artifact, so I really don't see any argument for not building it by
default? Don't quite see what the advantage of doing it during make
check would be?

Greetings,

Andres Freund

#15Alvaro Herrera
alvherre@alvh.no-ip.org
In reply to: Andres Freund (#14)
Re: pgsql: Add documentation for the JIT feature.

Andres Freund wrote:

Hi,

On 2018-03-29 15:20:58 -0300, Alvaro Herrera wrote:

Andres Freund wrote:

Could we add INSTALL to the all target in src/docs/sgml? It's a bit
awkward that one can build the docs and miss such a mistake. It's fast
to build in comparison to the rest of the docs, so that doesn't seem
like a high price?

Maybe it can be tested for in "make check"? That's probably even
faster.

Hm, what's wrong just doing it in the normal build? It's a desired build
artifact, so I really don't see any argument for not building it by
default? Don't quite see what the advantage of doing it during make
check would be?

I meant running something that would check that the file compiles,
without actually producing the output. For the regular docs, there's a
couple of orders of magnitude of difference in time to do the check vs.
the actual build.

It takes 1.2 second to build for me FWIW, so it's not like it's a huge
time waste anyhow. Probably more than what will ever be saved has
already been spent in this conversation :-)

--
�lvaro Herrera https://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

#16Stephen Frost
sfrost@snowman.net
In reply to: Andres Freund (#11)
Re: pgsql: Add documentation for the JIT feature.

Greetings,

* Andres Freund (andres@anarazel.de) wrote:

On 2018-03-29 13:26:31 -0400, Robert Haas wrote:

On Thu, Mar 29, 2018 at 9:44 AM, Tom Lane <tgl@sss.pgh.pa.us> wrote:

I'd go a little further and drop "JIT" from user-facing documentation
altogether. Instead refer to the feature as "compilation of expressions"
or some such. JIT is just jargon. Plus, the timing of the compilation is
actually the least important property for our purpose.

I agree that talking about JIT compilation (or just-in-time
compilation) would be better than talking just about JIT, but refusing
to mention JIT seems counter-productive to me. There are a lot of
people who know what just-in-time compilation is and will not realize
that "compilation of expressions" refers to any such technology. If
you don't know what it is, you can Google it. Just typing "jit" into
Google produces a stupid urban dictionary hit and then this:
https://en.wikipedia.org/wiki/Just-in-time_compilation -- and that
contains useful information that you'll never find if you search for
"compilation of expressions".

Also, in a way, you could argue that v10 already did "compilation of
expressions". It didn't compile them to machine language, true, but
it translated them into a form which is faster to execute, and which
is at least arguably a form of bytecode. It's not going to be clear,
even to an expert, that "compilation of expressions" means something
other than that, but if you say JIT, then all of a sudden people know
what we're talking about.

I agree that JIT is jargon, but it's pretty commonly-used jargon.

Precisely this. I'm very strongly against just saying "expression
compilation", it's just too imprecise. As Robert mentions it could refer
to what we do in v10, it could refer to ahead of time compilation of PL
functions, and it doesn't include compiling tuple deforming. Nor will
it describe compiling sorting, copy or whatnot.

I'm very open however to replacing JITing with JIT compiling and smilar
substitutions.

What we've done elsewhere when there's been similar jargon is to say
something along the lines of:

"compiling of routines (also known as Just-In-Time or JIT compilation)"

Thanks!

Stephen

#17Robert Haas
robertmhaas@gmail.com
In reply to: Stephen Frost (#16)
Re: pgsql: Add documentation for the JIT feature.

On Thu, Mar 29, 2018 at 3:00 PM, Stephen Frost <sfrost@snowman.net> wrote:

What we've done elsewhere when there's been similar jargon is to say
something along the lines of:

"compiling of routines (also known as Just-In-Time or JIT compilation)"

That strike me as roughly analogous to saying:

"hiding rows users don't have permission to see (also known as
row-level security)"

IOW, it's just substituting a generic description for a term of art
that has a real meaning. I can't see that as an improvement. It
would be possible to compile routines without involving JIT; as I
mentioned earlier, you could compile them to bytecode. Or, you could
compile them to machine code at definition time as, I am told by
Thomas, some database products used to do, I think by generating C
code and then shelling out to "cc". Just-in-time compilation clearly
conveys that we are generating machine code at runtime; "compiling of
routines" does not.

Besides, if we did that, what are we going to call the GUCs?
compiling_of_routines_above_cost? compiling_of_routines_provider =
'llvmjit'? Ugh.

This technique has bene called JIT since about 1983, and is well-known
under that name, mostly because of Java. Calling it some made-up name
we create ourselves is not going to make this more comprehensible to
users.

--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

#18Andres Freund
andres@anarazel.de
In reply to: Stephen Frost (#16)
Re: pgsql: Add documentation for the JIT feature.

On 2018-03-29 15:00:36 -0400, Stephen Frost wrote:

Greetings,

* Andres Freund (andres@anarazel.de) wrote:

On 2018-03-29 13:26:31 -0400, Robert Haas wrote:

On Thu, Mar 29, 2018 at 9:44 AM, Tom Lane <tgl@sss.pgh.pa.us> wrote:

I'd go a little further and drop "JIT" from user-facing documentation
altogether. Instead refer to the feature as "compilation of expressions"
or some such. JIT is just jargon. Plus, the timing of the compilation is
actually the least important property for our purpose.

I agree that talking about JIT compilation (or just-in-time
compilation) would be better than talking just about JIT, but refusing
to mention JIT seems counter-productive to me. There are a lot of
people who know what just-in-time compilation is and will not realize
that "compilation of expressions" refers to any such technology. If
you don't know what it is, you can Google it. Just typing "jit" into
Google produces a stupid urban dictionary hit and then this:
https://en.wikipedia.org/wiki/Just-in-time_compilation -- and that
contains useful information that you'll never find if you search for
"compilation of expressions".

Also, in a way, you could argue that v10 already did "compilation of
expressions". It didn't compile them to machine language, true, but
it translated them into a form which is faster to execute, and which
is at least arguably a form of bytecode. It's not going to be clear,
even to an expert, that "compilation of expressions" means something
other than that, but if you say JIT, then all of a sudden people know
what we're talking about.

I agree that JIT is jargon, but it's pretty commonly-used jargon.

Precisely this. I'm very strongly against just saying "expression
compilation", it's just too imprecise. As Robert mentions it could refer
to what we do in v10, it could refer to ahead of time compilation of PL
functions, and it doesn't include compiling tuple deforming. Nor will
it describe compiling sorting, copy or whatnot.

I'm very open however to replacing JITing with JIT compiling and smilar
substitutions.

What we've done elsewhere when there's been similar jargon is to say
something along the lines of:

"compiling of routines (also known as Just-In-Time or JIT compilation)"

There's an entire section explaining what JIT is and what we currently
can JIT. We can't just say "compiling of routines (also ...)", because
a) we don't do that. b) even if you take expressions being JIT compiled
as that, we also JIT compile tuple deforming. And hopefully will JIT
further things in the not too far away future.

The general config.sgml references and such read
Sets the planner's cutoff above which JIT compilation is used as part
of query execution (see <xref linkend="jit"/>). Performing
<acronym>JIT</acronym> costs time but can accelerate query execution.
it's not used in a standalone manner without references.

and the jit specific section starts with:
<para>
Just-in-time compilation (<acronym>JIT</acronym>) is the process of turning
some form of interpreted program evaluation into a native program, and
doing so at runtime.

For example, instead of using a facility that can evaluate arbitrary SQL
expressions to evaluate an SQL predicate like <literal>WHERE a.col =
3</literal>, it is possible to generate a function than can be natively
executed by the CPU that just handles that expression, yielding a speedup.
</para>

and continues a bit later with
<sect2 id="jit-accelerated-operations">
<title><acronym>JIT</acronym> Accelerated Operations</title>
<para>
Currently <productname>PostgreSQL</productname>'s <acronym>JIT</acronym>
implementation has support for accelerating expression evaluation and
tuple deforming. Several other operations could be accelerated in the
future.
</para>
<para>
Expression evaluation is used to evaluate <literal>WHERE</literal>
clauses, target lists, aggregates and projections. It can be accelerated
by generating code specific to each case.
</para>
<para>
Tuple deforming is the process of transforming an on-disk tuple (see <xref
linkend="heaptuple"/>) into its in-memory representation. It can be
accelerated by creating a function specific to the table layout and the
number of columns to be extracted.
</para>
</sect2>

I think there's a few references to standalone JIT that deserve to be
replaced with JIT compilation and similar.

Greetings,

Andres Freund

#19David G. Johnston
david.g.johnston@gmail.com
In reply to: Andres Freund (#11)
Re: pgsql: Add documentation for the JIT feature.

On Thursday, March 29, 2018, Andres Freund <andres@anarazel.de> wrote:

On 2018-03-29 13:26:31 -0400, Robert Haas wrote:

Also, in a way, you could argue that v10 already did "compilation of
expressions". It didn't compile them to machine language, true, but
it translated them into a form which is faster to execute, and which
is at least arguably a form of bytecode. It's not going to be clear,
even to an expert, that "compilation of expressions" means something
other than that, but if you say JIT, then all of a sudden people know
what we're talking about.

I agree that JIT is jargon, but it's pretty commonly-used jargon.

Precisely this. I'm very strongly against just saying "expression
compilation", it's just too imprecise. As Robert mentions it could refer
to what we do in v10, it could refer to ahead of time compilation of PL
functions, and it doesn't include compiling tuple deforming. Nor will
it describe compiling sorting, copy or whatnot.

I'm very open however to replacing JITing with JIT compiling and smilar
substitutions.

FWIW "JIT expression compilation" is comprehensible and precise to me.

David J.

In reply to: Robert Haas (#17)
Re: pgsql: Add documentation for the JIT feature.

On Thu, Mar 29, 2018 at 12:22 PM, Robert Haas <robertmhaas@gmail.com> wrote:

This technique has bene called JIT since about 1983, and is well-known
under that name, mostly because of Java. Calling it some made-up name
we create ourselves is not going to make this more comprehensible to
users.

+1. You make a good point about Java. Furthermore, when people talk
about just in time compilation in database systems, which is far from
a novel thing, they generally use the word JIT.

Why confuse things?

--
Peter Geoghegan

#21David Steele
david@pgmasters.net
In reply to: Andres Freund (#18)
Re: pgsql: Add documentation for the JIT feature.

On 3/29/18 3:27 PM, Andres Freund wrote:

There's an entire section explaining what JIT is and what we currently
can JIT. We can't just say "compiling of routines (also ...)", because
a) we don't do that. b) even if you take expressions being JIT compiled
as that, we also JIT compile tuple deforming. And hopefully will JIT
further things in the not too far away future.

OK, I'm convinced.

--
-David
david@pgmasters.net

#22Andres Freund
andres@anarazel.de
In reply to: Alvaro Herrera (#4)
Re: pgsql: Add documentation for the JIT feature.

Hi,

On 2018-03-29 09:59:55 -0300, Alvaro Herrera wrote:

I agree. In some email threads Andres has been using "JIT" as a verb,
too, such as "JITing expressions" and such; that's a bit shocking, in a
way. Honestly I don't care in a pgsql-hackers thread, I mean we all
understand what it means, but in user-facing docs and things we should
use complete words, "JIT-compile", "JIT-compilation", "JIT-compiling"
and so on.

FWIW, I don't think "JITing" or "JITed" appears anywhere in user facing
docs?

Greetings,

Andres Freund

#23Peter Eisentraut
peter.eisentraut@2ndquadrant.com
In reply to: Alvaro Herrera (#15)
Re: pgsql: Add documentation for the JIT feature.

On 3/29/18 14:43, Alvaro Herrera wrote:

Hm, what's wrong just doing it in the normal build? It's a desired build
artifact, so I really don't see any argument for not building it by
default? Don't quite see what the advantage of doing it during make
check would be?

I meant running something that would check that the file compiles,
without actually producing the output. For the regular docs, there's a
couple of orders of magnitude of difference in time to do the check vs.
the actual build.

Or we do both.

--
Peter Eisentraut http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

#24Tom Lane
tgl@sss.pgh.pa.us
In reply to: Peter Eisentraut (#23)
Re: pgsql: Add documentation for the JIT feature.

Peter Eisentraut <peter.eisentraut@2ndquadrant.com> writes:

On 3/29/18 14:43, Alvaro Herrera wrote:

Hm, what's wrong just doing it in the normal build? It's a desired build
artifact, so I really don't see any argument for not building it by
default? Don't quite see what the advantage of doing it during make
check would be?

I meant running something that would check that the file compiles,
without actually producing the output. For the regular docs, there's a
couple of orders of magnitude of difference in time to do the check vs.
the actual build.

Or we do both.

I'm OK with adding INSTALL to the default build target in doc/src/sgml;
the incremental cost isn't large and we now realize there'd be useful
error detection. I'm *not* OK with expanding the scope of "make check"
to include building the documentation. It's never had anything to do
with docs before and I see no reason to start now. Personally, when
I'm working on a patch, the doc updates if any are a completely separate
matter. I don't want to waste cycles on testing docs when I'm trying
to test code, any more than I would like the reverse (ie forcing a docs
build to build code too).

regards, tom lane

#25Andres Freund
andres@anarazel.de
In reply to: Tom Lane (#24)
Re: pgsql: Add documentation for the JIT feature.

On March 31, 2018 8:43:37 AM PDT, Tom Lane <tgl@sss.pgh.pa.us> wrote:

Peter Eisentraut <peter.eisentraut@2ndquadrant.com> writes:

On 3/29/18 14:43, Alvaro Herrera wrote:

I'm *not* OK with expanding the scope of "make check"
to include building the documentation. It's never had anything to do
with docs before and I see no reason to start now. Personally, when
I'm working on a patch, the doc updates if any are a completely
separate
matter. I don't want to waste cycles on testing docs when I'm trying
to test code, any more than I would like the reverse (ie forcing a docs
build to build code too).

They're a local check target in the docs directory. But it just checks postgres.xml, not additional targets. Don't think anybody proposed to add the doc check to the top-level check target.

Andres
--
Sent from my Android device with K-9 Mail. Please excuse my brevity.

#26Tom Lane
tgl@sss.pgh.pa.us
In reply to: Andres Freund (#25)
Re: pgsql: Add documentation for the JIT feature.

Andres Freund <andres@anarazel.de> writes:

On March 31, 2018 8:43:37 AM PDT, Tom Lane <tgl@sss.pgh.pa.us> wrote:

I don't want to waste cycles on testing docs when I'm trying
to test code, any more than I would like the reverse (ie forcing a docs
build to build code too).

They're a local check target in the docs directory. But it just checks
postgres.xml, not additional targets. Don't think anybody proposed to
add the doc check to the top-level check target.

Ah, didn't realize that "make check" in the docs isn't connected up to
the main check target. As long as that's true, no objection.

regards, tom lane

#27Alvaro Herrera
alvherre@alvh.no-ip.org
In reply to: Tom Lane (#26)
Re: pgsql: Add documentation for the JIT feature.

Tom Lane wrote:

Andres Freund <andres@anarazel.de> writes:

On March 31, 2018 8:43:37 AM PDT, Tom Lane <tgl@sss.pgh.pa.us> wrote:

I don't want to waste cycles on testing docs when I'm trying
to test code, any more than I would like the reverse (ie forcing a docs
build to build code too).

They're a local check target in the docs directory. But it just checks
postgres.xml, not additional targets. Don't think anybody proposed to
add the doc check to the top-level check target.

Ah, didn't realize that "make check" in the docs isn't connected up to
the main check target. As long as that's true, no objection.

I'm not proposing to add doc/src/sgml's check target to top-level check,
though perhaps it wouldn't be such a terrible idea -- seems quick
enough, at under half a second:

$ time LC_ALL=C make -C doc/src/sgml/ check
make: Entering directory '/home/alvherre/Code/pgsql/build/master/doc/src/sgml'
/usr/bin/xmllint --path . --noout --valid /pgsql/source/master/doc/src/sgml/postgres.sgml
make: Leaving directory '/home/alvherre/Code/pgsql/build/master/doc/src/sgml'

real 0m0,494s
user 0m0,461s
sys 0m0,033s

--
�lvaro Herrera https://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services