gettimeofday is at the end of its usefulness?
A recent question from Tim Kane prompted me to measure the overhead
costs of EXPLAIN ANALYZE, which I'd not checked in awhile. Things
are far worse than I thought. On my current server (by no means
lavish hardware: Xeon E5-2609 @2.40GHz) a simple seqscan can run
at something like 110 nsec per row:
regression=# create table foo as select x as f1 from generate_series(1,1000000) x;
SELECT 1000000
regression=# vacuum foo;
VACUUM
regression=# explain analyze select * from foo;
QUERY PLAN
---------------------------------------------------------------------------------------------------------------
Seq Scan on foo (cost=0.00..14425.00 rows=1000000 width=4) (actual time=0.053..111.720 rows=1000000 loops=1)
Planning time: 0.222 ms
Execution time: 166.682 ms
(3 rows)
(and, btw, this is a debug build --- without assert and memory
context checks it'd be faster.)
The problem with this number is that a simple test program shows that
gettimeofday() requires about 40 nsec on this hardware. That means
that two-thirds of the above timing measurement is overhead.
To add insult to injury, gettimeofday's output cannot be more precise than
1 microsecond, making its use for measuring sub-microsecond intervals at
best stochastic.
I looked around a bit and found that recent versions of POSIX have a
function clock_gettime() that's a bit more modern than gettimeofday():
at least, the output struct provides nsec rather than usec available
precision. I benchmarked this, with the CLOCK_REALTIME selector, and
found that it also requires about 40nsec, while the output is actually
good to perhaps 10nsec precision. (I base this on seeing no duplicate
readings in a tight loop, so that the value is certainly getting advanced
more often than once every 40 nsec.)
There's also a CLOCK_REALTIME_COARSE selector, which is noticeably faster
--- about 10nsec for me --- but the output appears to only advance once
every millisecond, so it's probably useless for our purposes. The other
selectors mentioned in the Linux man page are considerably slower than
CLOCK_REALTIME for me, suggesting that they actually call into the kernel.I also tried a loop around a bare "rdtsc" assembly instruction, finding
that that instruction takes about 10nsec. That would be a nice
improvement over gettimeofday, except that using that directly would
involve dealing with cross-CPU skew, which seems like no fun at all.
And I don't really want to get into finding equivalents for non-Intel
architectures, either.
Anyway it looks like clock_gettime() might be worth using on Linux
just for the more precise output. It doesn't seem to exist on OS X
though, and I have no idea about elsewhere.
I'm curious if anybody has ideas about other things we might do for
portable high-precision timing.
regards, tom lane
Attachments:
testclock.ctext/x-c; charset=us-ascii; name=testclock.cDownload
On Tue, May 13, 2014 at 3:58 PM, Tom Lane <tgl@sss.pgh.pa.us> wrote:
There's also a CLOCK_REALTIME_COARSE selector, which is noticeably faster --- about 10nsec for me --- but the output appears to only advance once every millisecond, so it's probably useless for our purposes. The other selectors mentioned in the Linux man page are considerably slower than CLOCK_REALTIME for me, suggesting that they actually call into the kernel.
What Linux kernel version is in use here? Apparently, as I think
you've stated another way, more recent versions have VDSO for this,
which can make a big difference. This article seems like a sensible
guide to all of this:
https://access.redhat.com/site/documentation/en-US/Red_Hat_Enterprise_MRG/1.3/html/Realtime_Tuning_Guide/sect-Realtime_Tuning_Guide-General_System_Tuning-gettimeofday_speedup.html
CLOCK_REALTIME_COARSE seemingly influences precision in a way that
allows user space applications to decide on their precision/cost
trade-off, rather than being forced to use the system default (that
procfs surfaces) through gettimeofday():
http://lwn.net/Articles/342018/
I can see a benefit in exposing this trade-off to Postgres code
directly. I still think that a correlated reference period will prove
useful, and while there are a number of ways to amortize the cost of
repeatedly (coarsely) getting the wall time in the ordinary course of
choosing victim buffers, it would be nice to do this too.
--
Peter Geoghegan
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
Peter Geoghegan <pg@heroku.com> writes:
On Tue, May 13, 2014 at 3:58 PM, Tom Lane <tgl@sss.pgh.pa.us> wrote:
There's also a CLOCK_REALTIME_COARSE selector, which is noticeably faster --- about 10nsec for me --- but the output appears to only advance once every millisecond, so it's probably useless for our purposes. The other selectors mentioned in the Linux man page are considerably slower than CLOCK_REALTIME for me, suggesting that they actually call into the kernel.
What Linux kernel version is in use here?
Ah, sorry, I should have specified. This is RHEL6.5, current kernel
version 2.6.32-431.17.1.el6.x86_64.
Apparently, as I think
you've stated another way, more recent versions have VDSO for this,
which can make a big difference. This article seems like a sensible
guide to all of this:
https://access.redhat.com/site/documentation/en-US/Red_Hat_Enterprise_MRG/1.3/html/Realtime_Tuning_Guide/sect-Realtime_Tuning_Guide-General_System_Tuning-gettimeofday_speedup.html
This appears to be talking about RHEL5, which is quite a bit older
(and, I'd guess, trailing edge for anybody who might deploy PG 9.5).
I did confirm that /proc/sys/kernel/vsyscall64 exists and has a default
setting of 1 on RHEL6. Setting it to 0 causes gettimeofday to take
150ns, which probably represents the time for a trivial kernel call.
The MRG extension described on the linked page doesn't seem to be
implemented in stock RHEL6 (setting vsyscall64 to 2 is allowed but
doesn't change behavior compared to 1). However, if I'm reading it
right, all that does is make gettimeofday behave like
clock_gettime(CLOCK_REALTIME_COARSE).
CLOCK_REALTIME_COARSE seemingly influences precision in a way that
allows user space applications to decide on their precision/cost
trade-off, rather than being forced to use the system default (that
procfs surfaces) through gettimeofday():
http://lwn.net/Articles/342018/
Yeah, I think these are the same implementations exposed to apps in two
different ways, one being a system-wide switch affecting gettimeofday()
while the other allows the app source code to say which one it wants.
regards, tom lane
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
On Tue, May 13, 2014 at 11:58 PM, Tom Lane <tgl@sss.pgh.pa.us> wrote:
I also tried a loop around a bare "rdtsc" assembly instruction, finding
that that instruction takes about 10nsec. That would be a nice
improvement over gettimeofday, except that using that directly would
involve dealing with cross-CPU skew, which seems like no fun at all.
And I don't really want to get into finding equivalents for non-Intel
architectures, either.
I always assumed the kernel used rdtsc to implement some of the high
performance timers. It can save the current time in a mapped page when
it schedules a process and then in the vdso syscall (ie in user-space)
it can use rdtsc to calculate the offset needed to adjust that
timestamp to the current time. This seems consistent with your
calculations that showed the 40ns overhead with +/- 10ns precision.
I actually think it would be more interesting if we could measure the
overhead and adjust for it. I don't think people are really concerned
with how long EXPLAIN ANALYZE takes to run if they could get accurate
numbers out of it.
Other profiling tools I poked at in the past ran a tight loop around
the profiling code to estimate the time it actually took and then
subtracted that from all the measurements. I think that might work for
the actual clock_gettime overhead. If we did that then we could call
it twice and measure the time spent in the rest of the EXPLAIN ANALYZE
code and subtract that plus the time for the two clock_gettimes from
the run-time...
--
greg
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
Greg Stark <stark@mit.edu> writes:
I actually think it would be more interesting if we could measure the
overhead and adjust for it.
Actually, that's quite a good thought. The overhead should be a pretty
stable number on any given machine, so in theory we could do this to
high precision. And the numbers I just showed say that on current
x86_64 platforms, the *best we could possibly hope for* in terms of
direct overhead reduction is about 4x. Which is something, but it
hardly makes the problem vanish.
I have a vague feeling that we discussed subtract-the-overhead once before
and thought it wasn't necessary yet. Maybe it's time.
But we also need to be using something that gives better than 1usec
resolution. So I'm still thinking we should use clock_gettime() where
available, and look for alternative APIs where not.
regards, tom lane
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
On Tuesday, May 13, 2014, Tom Lane <tgl@sss.pgh.pa.us> wrote:
A recent question from Tim Kane prompted me to measure the overhead
costs of EXPLAIN ANALYZE, which I'd not checked in awhile. Things
are far worse than I thought. On my current server (by no means
lavish hardware: Xeon E5-2609 @2.40GHz) a simple seqscan can run
at something like 110 nsec per row:regression=# create table foo as select x as f1 from
generate_series(1,1000000) x;
SELECT 1000000
regression=# vacuum foo;
VACUUM
regression=# explain analyze select * from foo;
QUERY PLAN---------------------------------------------------------------------------------------------------------------
Seq Scan on foo (cost=0.00..14425.00 rows=1000000 width=4) (actual
time=0.053..111.720 rows=1000000 loops=1)
Planning time: 0.222 ms
Execution time: 166.682 ms
(3 rows)(and, btw, this is a debug build --- without assert and memory
context checks it'd be faster.)The problem with this number is that a simple test program shows that
gettimeofday() requires about 40 nsec on this hardware. That means
that two-thirds of the above timing measurement is overhead.
I'm all for finding something better if we can, but in the mean time this
is certainly not unexpected, and isn't it exactly what "explain
(analyze,timing off)" was invented for?
Cheers,
Jeff
On Wed, May 14, 2014 at 6:34 AM, Greg Stark <stark@mit.edu> wrote:
I always assumed the kernel used rdtsc to implement some of the high
performance timers. It can save the current time in a mapped page when
it schedules a process and then in the vdso syscall (ie in user-space)
it can use rdtsc to calculate the offset needed to adjust that
timestamp to the current time. This seems consistent with your
calculations that showed the 40ns overhead with +/- 10ns precision.
Both gettimeofday and clock_gettime do exactly that. [1]https://github.com/torvalds/linux/blob/master/arch/x86/vdso/vclock_gettime.c#L223
clock_gettime(CLOCK_MONOTONIC) is the mode of operation we would want
to use here.
I actually think it would be more interesting if we could measure the
overhead and adjust for it. I don't think people are really concerned
with how long EXPLAIN ANALYZE takes to run if they could get accurate
numbers out of it.
Measuring would also be a good idea so we can automatically turn on
performance counters like IO timing when we know it's not obscenely
expensive.
However, subtracting the overhead will still skew the numbers somewhat
by giving more breathing time for memory and IO prefetching
mechanisms. Another option to consider would be to add a sampling
based mechanism for low overhead time attribution. It would be even
better if we could distinguish between time spent waiting on locks vs.
waiting on IO vs. waiting to be scheduled vs. actually executing.
[1]: https://github.com/torvalds/linux/blob/master/arch/x86/vdso/vclock_gettime.c#L223
Regards,
Ants Aasma
--
Cybertec Schönig & Schönig GmbH
Gröhrmühlgasse 26
A-2700 Wiener Neustadt
Web: http://www.postgresql-support.de
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
On Tue, May 13, 2014 at 11:34 PM, Greg Stark <stark@mit.edu> wrote:
I always assumed the kernel used rdtsc to implement some of the high
performance timers. It can save the current time in a mapped page when
it schedules a process and then in the vdso syscall (ie in user-space)
it can use rdtsc to calculate the offset needed to adjust that
timestamp to the current time. This seems consistent with your
calculations that showed the 40ns overhead with +/- 10ns precision.
Crazy idea: Instead of trying to time precisely the amount of time we
spend in each node, configure a very-high frequency timer interrupt
(or background thread?) that does:
SomeGlobalVariablePointingToTheCurrentNode->profiling_counter++;
--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
I posted this on this mailing list before at Jane Street we have developed
very fast code to get timing information based on TSC if available. It's
all ocaml but well documented and mostly just calls to c functions so
should be easy to port to C and we release it under a very liberal license
so it should be no problem to take the ideas:
https://github.com/janestreet/core/blob/master/lib/time_stamp_counter.mli
Hope this is useful.
Bene
On Wed, May 14, 2014 at 12:41 PM, Robert Haas <robertmhaas@gmail.com> wrote:
Show quoted text
On Tue, May 13, 2014 at 11:34 PM, Greg Stark <stark@mit.edu> wrote:
I always assumed the kernel used rdtsc to implement some of the high
performance timers. It can save the current time in a mapped page when
it schedules a process and then in the vdso syscall (ie in user-space)
it can use rdtsc to calculate the offset needed to adjust that
timestamp to the current time. This seems consistent with your
calculations that showed the 40ns overhead with +/- 10ns precision.Crazy idea: Instead of trying to time precisely the amount of time we
spend in each node, configure a very-high frequency timer interrupt
(or background thread?) that does:SomeGlobalVariablePointingToTheCurrentNode->profiling_counter++;
--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
On Thu, May 15, 2014 at 8:19 AM, Benedikt Grundmann <
bgrundmann@janestreet.com> wrote:
I posted this on this mailing list before at Jane Street we have developed
very fast code to get timing information based on TSC if available. It's
all ocaml but well documented and mostly just calls to c functions so
should be easy to port to C and we release it under a very liberal license
so it should be no problem to take the ideas:https://github.com/janestreet/core/blob/master/lib/time_stamp_counter.mli
Hope this is useful.
Bene
Also I'm sorry for top posting. hackers is the only mailing list I'm on
that requires this and some others require top posting so this runs counter
my habits and I only realized after sending...
Show quoted text
On Wed, May 14, 2014 at 12:41 PM, Robert Haas <robertmhaas@gmail.com>wrote:
On Tue, May 13, 2014 at 11:34 PM, Greg Stark <stark@mit.edu> wrote:
I always assumed the kernel used rdtsc to implement some of the high
performance timers. It can save the current time in a mapped page when
it schedules a process and then in the vdso syscall (ie in user-space)
it can use rdtsc to calculate the offset needed to adjust that
timestamp to the current time. This seems consistent with your
calculations that showed the 40ns overhead with +/- 10ns precision.Crazy idea: Instead of trying to time precisely the amount of time we
spend in each node, configure a very-high frequency timer interrupt
(or background thread?) that does:SomeGlobalVariablePointingToTheCurrentNode->profiling_counter++;
--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
On Thu, May 15, 2014 at 8:19 AM, Benedikt Grundmann
<bgrundmann@janestreet.com> wrote:
I posted this on this mailing list before at Jane Street we have developed
very fast code to get timing information based on TSC if available. It's
all ocaml but well documented and mostly just calls to c functions so should
be easy to port to C and we release it under a very liberal license so it
should be no problem to take the ideas:
What OS do you run it on though? How fast is your implementation
compared to the kernel implementation of clock_gettime()?
Are you sure your implementation is actually faster? And are you sure
you're protected against clocks going backwards? I think you should
put some i/o in the loop in the test and start several threads running
it to make it more likely the thread is rescheduled to a different
processor during the test. It suspect you'll find the rdtsc goes
backwards sometimes or produces crazy results when switching
processors.
--
greg
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
On Thu, May 15, 2014 at 11:31 AM, Greg Stark <stark@mit.edu> wrote:
On Thu, May 15, 2014 at 8:19 AM, Benedikt Grundmann
<bgrundmann@janestreet.com> wrote:I posted this on this mailing list before at Jane Street we have
developed
very fast code to get timing information based on TSC if available. It's
all ocaml but well documented and mostly just calls to c functions soshould
be easy to port to C and we release it under a very liberal license so it
should be no problem to take the ideas:What OS do you run it on though? How fast is your implementation
compared to the kernel implementation of clock_gettime()?Are you sure your implementation is actually faster? And are you sure
you're protected against clocks going backwards? I think you should
put some i/o in the loop in the test and start several threads running
it to make it more likely the thread is rescheduled to a different
processor during the test. It suspect you'll find the rdtsc goes
backwards sometimes or produces crazy results when switching
processors.
There are benchmarks in the link I posted (obtained by a micro benchmarking
library we developed / use internally which takes great care to obtain
reliable numbers) . We use posix threads extensively. We internally spend
a lot of time setting up ntp and monitoring systems so that clock backwards
never happens (so with other words I wouldn't be surprised if the library
does NOT work correctly when it does -- our protection is outside). I do
not believe we have seen the tdtsc going backwards on thread context switch
you mention (and as said we use lots of threads). OS? Centos 6.5
primarily.
--
Show quoted text
greg
On 2014-05-15 12:04:25 +0100, Benedikt Grundmann wrote:
On Thu, May 15, 2014 at 11:31 AM, Greg Stark <stark@mit.edu> wrote:
On Thu, May 15, 2014 at 8:19 AM, Benedikt Grundmann
<bgrundmann@janestreet.com> wrote:I posted this on this mailing list before at Jane Street we have
developed
very fast code to get timing information based on TSC if available. It's
all ocaml but well documented and mostly just calls to c functions soshould
be easy to port to C and we release it under a very liberal license so it
should be no problem to take the ideas:What OS do you run it on though? How fast is your implementation
compared to the kernel implementation of clock_gettime()?Are you sure your implementation is actually faster? And are you sure
you're protected against clocks going backwards? I think you should
put some i/o in the loop in the test and start several threads running
it to make it more likely the thread is rescheduled to a different
processor during the test. It suspect you'll find the rdtsc goes
backwards sometimes or produces crazy results when switching
processors.There are benchmarks in the link I posted (obtained by a micro benchmarking
library we developed / use internally which takes great care to obtain
reliable numbers) . We use posix threads extensively. We internally spend
a lot of time setting up ntp and monitoring systems so that clock backwards
never happens (so with other words I wouldn't be surprised if the library
does NOT work correctly when it does -- our protection is outside). I do
not believe we have seen the tdtsc going backwards on thread context switch
you mention (and as said we use lots of threads). OS? Centos 6.5
primarily.
Did you test it on server with more one socket (i.e. not just multiple
cores, but distinct cpu cases)? That's where you expect to see
differences in TSC to have funny effects.
Greetings,
Andres Freund
--
Andres Freund http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Training & Services
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
Hi,
On 2014-05-13 18:58:11 -0400, Tom Lane wrote:
Anyway it looks like clock_gettime() might be worth using on Linux
just for the more precise output. It doesn't seem to exist on OS X
though, and I have no idea about elsewhere.
Agreed that using clock_gettime() would be a good idea. I'd say we
should have a wrapper around it that is able to provide nanosecond
precision. If only gettimeofday() (and whatever windows is using) is
available, we can dynamically fall back to that.
I'm curious if anybody has ideas about other things we might do for
portable high-precision timing.
It's far from a solve-it-all, but can we perhaps try to coalesce
repeated time measurements? We'll very frequently do a
InstrStopNode();
/* minimal amount of work */
InstrStartNode();
which will measure the time twice. I think there's a fair number of
scenarios where once would be enough. I'll freely admit that I haven't
looked enough to determine how we could do that API wise.
Greetings,
Andres Freund
--
Andres Freund http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Training & Services
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
On Tue, May 13, 2014 at 06:58:11PM -0400, Tom Lane wrote:
A recent question from Tim Kane prompted me to measure the overhead
costs of EXPLAIN ANALYZE, which I'd not checked in awhile. Things
are far worse than I thought. On my current server (by no means
lavish hardware: Xeon E5-2609 @2.40GHz) a simple seqscan can run
at something like 110 nsec per row:
I assume you ran pg_test_timing too:
Testing timing overhead for 3 seconds.
Per loop time including overhead: 41.70 nsec
Histogram of timing durations:
< usec % of total count
1 95.83035 68935459
2 4.16923 2999133
4 0.00037 268
8 0.00004 31
16 0.00000 1
32 0.00000 1
My overhead of 41.70 nsec matches yours.
--
Bruce Momjian <bruce@momjian.us> http://momjian.us
EnterpriseDB http://enterprisedb.com
+ Everyone has their own god. +
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
On 15 May 2014 at 19:56, Bruce Momjian <bruce@momjian.us> wrote:
On Tue, May 13, 2014 at 06:58:11PM -0400, Tom Lane wrote:
A recent question from Tim Kane prompted me to measure the overhead
costs of EXPLAIN ANALYZE, which I'd not checked in awhile. Things
are far worse than I thought. On my current server (by no means
lavish hardware: Xeon E5-2609 @2.40GHz) a simple seqscan can run
at something like 110 nsec per row:I assume you ran pg_test_timing too:
Testing timing overhead for 3 seconds.
Per loop time including overhead: 41.70 nsec
Histogram of timing durations:
< usec % of total count
1 95.83035 68935459
2 4.16923 2999133
4 0.00037 268
8 0.00004 31
16 0.00000 1
32 0.00000 1My overhead of 41.70 nsec matches yours.
Did this idea die, or is it still worth considering?
Thom
Thom Brown <thom@linux.com> writes:
On 15 May 2014 at 19:56, Bruce Momjian <bruce@momjian.us> wrote:
On Tue, May 13, 2014 at 06:58:11PM -0400, Tom Lane wrote:
A recent question from Tim Kane prompted me to measure the overhead
costs of EXPLAIN ANALYZE, which I'd not checked in awhile. Things
are far worse than I thought. On my current server (by no means
lavish hardware: Xeon E5-2609 @2.40GHz) a simple seqscan can run
at something like 110 nsec per row:
Did this idea die, or is it still worth considering?
We still have a problem, for sure. I'm not sure that there was any
consensus on what to do about it. Using clock_gettime(CLOCK_REALTIME)
if available would be a straightforward change that should ameliorate
gettimeofday()'s 1-usec-precision-limit problem; but it doesn't do
anything to fix the excessive-overhead problem. The ideas about the
latter were all over the map, and none of them looked easy.
If you're feeling motivated to work on this area, feel free.
regards, tom lane
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
On 6/8/16 9:56 AM, Tom Lane wrote:
Thom Brown <thom@linux.com> writes:
On 15 May 2014 at 19:56, Bruce Momjian <bruce@momjian.us> wrote:
On Tue, May 13, 2014 at 06:58:11PM -0400, Tom Lane wrote:
A recent question from Tim Kane prompted me to measure the overhead
costs of EXPLAIN ANALYZE, which I'd not checked in awhile. Things
are far worse than I thought. On my current server (by no means
lavish hardware: Xeon E5-2609 @2.40GHz) a simple seqscan can run
at something like 110 nsec per row:Did this idea die, or is it still worth considering?
We still have a problem, for sure. I'm not sure that there was any
consensus on what to do about it. Using clock_gettime(CLOCK_REALTIME)
if available would be a straightforward change that should ameliorate
gettimeofday()'s 1-usec-precision-limit problem; but it doesn't do
anything to fix the excessive-overhead problem. The ideas about the
latter were all over the map, and none of them looked easy.If you're feeling motivated to work on this area, feel free.
Semi-related: someone (Robert I think) recently mentioned investigating
"vectorized" executor nodes, where multiple tuples would be processed in
one shot. If we had that presumably the explain penalty would be a moot
point.
--
Jim Nasby, Data Architect, Blue Treble Consulting, Austin TX
Experts in Analytics, Data Architecture and PostgreSQL
Data in Trouble? Get it in Treble! http://BlueTreble.com
855-TREBLE2 (855-873-2532) mobile: 512-569-9461
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
On Tue, Jun 14, 2016 at 4:27 PM, Jim Nasby <Jim.Nasby@bluetreble.com> wrote:
Semi-related: someone (Robert I think) recently mentioned investigating
"vectorized" executor nodes, where multiple tuples would be processed in one
shot. If we had that presumably the explain penalty would be a moot point.
Yeah, both Andres and I are interested in that, and I think he's
actively working on it. It would be quite neat if this had the effect
of reducing EXPLAIN ANALYZE's overhead to something trivial.
--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
On 2016-07-07 14:43:31 -0400, Robert Haas wrote:
On Tue, Jun 14, 2016 at 4:27 PM, Jim Nasby <Jim.Nasby@bluetreble.com> wrote:
Semi-related: someone (Robert I think) recently mentioned investigating
"vectorized" executor nodes, where multiple tuples would be processed in one
shot. If we had that presumably the explain penalty would be a moot point.Yeah, both Andres and I are interested in that, and I think he's
actively working on it. It would be quite neat if this had the effect
of reducing EXPLAIN ANALYZE's overhead to something trivial.
I am, and it does reduce the overhead. Depends on the type of plan
though. Index nestloops e.g. don't benefit on the inner side.
Andres
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
