Proposed changes to DTrace probe implementation

Started by Robert Loralmost 18 years ago15 messages
#1Robert Lor
Robert.Lor@Sun.COM
2 attachment(s)

Motivation:
To enable probes to work with Mac OS X Leopard and other OSes that will
support DTrace in the future. Bug filed on this issue
http://archives.postgresql.org/pgsql-bugs/2007-10/msg00201.php

The problem:
There are two different mechanisms to implementing application level
probes in DTrace as explained here
http://blogs.sun.com/ahl/entry/user_land_tracing_gets_better. I
originally chose the first mechanism (using DTRACE_PROBEn macros) for
simplicity, but Leopard's DTrace implementation only supports the second
mechanism (using macros from dtrace generated header file). The second
mechanism was introduced as an enhancement, and one of the benefits is
type checking.

Proposed changes:
* Switch from using DTRACE_PROBEn macros to the dynamically generated
macros (remove pg_trace.h)
* Use "dtrace -h" to create a header file that contains the dynamically
generated macros to be used in the source code instead of the
DTRACE_PROBEn macros.
* Create a new header file called probes_null.h to map the generated
macros to no-ops
* This is unrelated to making DTrace work on Leopard, but I'd like to
propose that we split the probes into generic database and Postgres
specific providers, called "database" and "postgresql" respectively.
Other databases will be adding DTrace probes as well, and we want to use
"database" as the provider for probes that are common to all databases.
This way scripts that use the common provider will work on databases
that implement the common probes.

With the proposed changes, the steps for adding new probes are slightly
different, but the ways the probes are used will not change.

Steps for adding new probes now:
1) Add probe name to probes.d
2) Add probe to source code using one of PG_TRACEn macros
3) Recompile

Steps for adding probes with proposed changes:
1) Add probe name to probes.d (probes.h will be created at build time)
2) Add null macros to probes_null.h
3) Add probe to source code using the dynamically generated macro from
probes.h
4) Recompile

Preliminary patch is attached. There is one known issue with the patch.
When compiling outside of the source tree, probes.d needs to be
symlinked to the source tree. I haven't figured out how to copy the file
to the build tree yet. I'm sure this is trivial for you gurus out there!

Comments?

Regards,
-Robert

Attachments:

dtrace.patchtext/plain; name=dtrace.patch; x-mac-creator=0; x-mac-type=0Download
? src/include/probes_null.h
Index: src/Makefile
===================================================================
RCS file: /projects/cvsroot/pgsql/src/Makefile,v
retrieving revision 1.42
diff -r1.42 Makefile
16a17,19
> ifeq ($(enable_dtrace), yes)
> 	$(DTRACE) -h -s $(top_builddir)/src/backend/utils/probes.d -o $(top_builddir)/src/include/probes.h
> endif
Index: src/backend/Makefile
===================================================================
RCS file: /projects/cvsroot/pgsql/src/backend/Makefile,v
retrieving revision 1.125
diff -r1.125 Makefile
22a23
> ifeq ($(PORTNAME), solaris)
25a27
> endif
143a146
> ifeq ($(PORTNAME), solaris)
145a149
> endif
Index: src/backend/access/transam/xact.c
===================================================================
RCS file: /projects/cvsroot/pgsql/src/backend/access/transam/xact.c,v
retrieving revision 1.257
diff -r1.257 xact.c
1482c1482
< 	PG_TRACE1(transaction__start, vxid.localTransactionId);
---
> 	DATABASE_TRANSACTION_START(vxid.localTransactionId);
1607c1607
< 	PG_TRACE1(transaction__commit, MyProc->lxid);
---
> 	DATABASE_TRANSACTION_COMMIT(MyProc->lxid);
1993c1993
< 	PG_TRACE1(transaction__abort, MyProc->lxid);
---
> 	DATABASE_TRANSACTION_ABORT(MyProc->lxid);
Index: src/backend/storage/lmgr/lock.c
===================================================================
RCS file: /projects/cvsroot/pgsql/src/backend/storage/lmgr/lock.c,v
retrieving revision 1.181
diff -r1.181 lock.c
790c790
< 		PG_TRACE2(lock__startwait, locktag->locktag_field2, lockmode);
---
> 		POSTGRESQL_LOCK_STARTWAIT(locktag->locktag_field2, lockmode);
794c794
< 		PG_TRACE2(lock__endwait, locktag->locktag_field2, lockmode);
---
> 		POSTGRESQL_LOCK_ENDWAIT(locktag->locktag_field2, lockmode);
Index: src/backend/storage/lmgr/lwlock.c
===================================================================
RCS file: /projects/cvsroot/pgsql/src/backend/storage/lmgr/lwlock.c,v
retrieving revision 1.50
diff -r1.50 lwlock.c
450c450
< 		PG_TRACE2(lwlock__startwait, lockid, mode);
---
> 		POSTGRESQL_LWLOCK_STARTWAIT(lockid, mode);
461c461
< 		PG_TRACE2(lwlock__endwait, lockid, mode);
---
> 		POSTGRESQL_LWLOCK_ENDWAIT(lockid, mode);
472c472
< 	PG_TRACE2(lwlock__acquire, lockid, mode);
---
> 	POSTGRESQL_LWLOCK_ACQUIRE(lockid, mode);
543c543
< 		PG_TRACE2(lwlock__condacquire__fail, lockid, mode);
---
> 		POSTGRESQL_LWLOCK_CONDACQUIRE_FAIL(lockid, mode);
549c549
< 		PG_TRACE2(lwlock__condacquire, lockid, mode);
---
> 		POSTGRESQL_LWLOCK_CONDACQUIRE(lockid, mode);
634c634
< 	PG_TRACE1(lwlock__release, lockid);
---
> 	POSTGRESQL_LWLOCK_RELEASE(lockid);
Index: src/backend/utils/probes.d
===================================================================
RCS file: /projects/cvsroot/pgsql/src/backend/utils/probes.d,v
retrieving revision 1.2
diff -r1.2 probes.d
10c10
< provider postgresql {
---
> provider database {
14a15,19
> 
> };
> 
> provider postgresql {
> 
Index: src/include/c.h
===================================================================
RCS file: /projects/cvsroot/pgsql/src/include/c.h,v
retrieving revision 1.222
diff -r1.222 c.h
60c60,64
< #include "pg_trace.h"
---
> #ifdef ENABLE_DTRACE
> #include "probes.h"
> #else
> #include "probes_null.h"
> #endif
probes_null.htext/plain; name=probes_null.hDownload
#2Peter Eisentraut
peter_e@gmx.net
In reply to: Robert Lor (#1)
Re: Proposed changes to DTrace probe implementation

Am Freitag, 22. Februar 2008 schrieb Robert Lor:

Preliminary patch is attached.

Could you please use diff -c or -u for the patch? It's quite hard to read
this way.

--
Peter Eisentraut
http://developer.postgresql.org/~petere/

#3Robert Lor
Robert.Lor@Sun.COM
In reply to: Peter Eisentraut (#2)
Re: Proposed changes to DTrace probe implementation

Peter,

Peter Eisentraut <peter_e@gmx.net> wrote:

Could you please use diff -c or -u for the patch? It's quite hard to
read this way.

Attached is the patch with diff -c.

Thanks,
-Robert

#4Robert Lor
Robert.Lor@Sun.COM
In reply to: Robert Lor (#3)
1 attachment(s)
Re: Proposed changes to DTrace probe implementation

Robert Lor <Robert.Lor@Sun.COM> wrote:

Peter,

Peter Eisentraut <peter_e@gmx.net> wrote:

Could you please use diff -c or -u for the patch? It's quite hard

to

read this way.

Attached is the patch with diff -c.

Oops, here's the real attachment!

Regards,
-Robert

Attachments:

dtrace.patchapplication/octet-stream; NAME=dtrace.patchDownload
? src/include/probes.h
? src/include/probes_null.h
Index: src/Makefile
===================================================================
RCS file: /projects/cvsroot/pgsql/src/Makefile,v
retrieving revision 1.42
diff -c -r1.42 Makefile
*** src/Makefile	21 Aug 2007 01:11:12 -0000	1.42
--- src/Makefile	22 Feb 2008 18:33:58 -0000
***************
*** 14,19 ****
--- 14,22 ----
  
  
  all install installdirs uninstall distprep:
+ ifeq ($(enable_dtrace), yes)
+ 	$(DTRACE) -h -s $(top_builddir)/src/backend/utils/probes.d -o $(top_builddir)/src/include/probes.h
+ endif
  	$(MAKE) -C port $@
  	$(MAKE) -C timezone $@
  	$(MAKE) -C backend $@
Index: src/backend/Makefile
===================================================================
RCS file: /projects/cvsroot/pgsql/src/backend/Makefile,v
retrieving revision 1.125
diff -c -r1.125 Makefile
*** src/backend/Makefile	1 Jan 2008 19:45:45 -0000	1.125
--- src/backend/Makefile	22 Feb 2008 18:33:58 -0000
***************
*** 20,28 ****
--- 20,30 ----
  
  SUBSYSOBJS = $(DIRS:%=%/SUBSYS.o)
  
+ ifeq ($(PORTNAME), solaris)
  ifeq ($(enable_dtrace), yes)
  LOCALOBJS += utils/probes.o
  endif
+ endif
  
  OBJS = $(SUBSYSOBJS) $(LOCALOBJS) $(top_builddir)/src/port/libpgport_srv.a
  
***************
*** 141,148 ****
--- 143,152 ----
  	    $(LN_S) ../../../$(subdir)/utils/fmgroids.h .
  
  
+ ifeq ($(PORTNAME), solaris)
  utils/probes.o: utils/probes.d $(SUBSYSOBJS)
  	$(DTRACE) $(DTRACEFLAGS) -G -s $^ -o $@
+ endif
  
  
  ##########################################################################
Index: src/backend/access/transam/xact.c
===================================================================
RCS file: /projects/cvsroot/pgsql/src/backend/access/transam/xact.c,v
retrieving revision 1.257
diff -c -r1.257 xact.c
*** src/backend/access/transam/xact.c	15 Jan 2008 18:56:59 -0000	1.257
--- src/backend/access/transam/xact.c	22 Feb 2008 18:33:58 -0000
***************
*** 1479,1485 ****
  	Assert(MyProc->backendId == vxid.backendId);
  	MyProc->lxid = vxid.localTransactionId;
  
! 	PG_TRACE1(transaction__start, vxid.localTransactionId);
  
  	/*
  	 * set transaction_timestamp() (a/k/a now()).  We want this to be the same
--- 1479,1485 ----
  	Assert(MyProc->backendId == vxid.backendId);
  	MyProc->lxid = vxid.localTransactionId;
  
! 	DATABASE_TRANSACTION_START(vxid.localTransactionId);
  
  	/*
  	 * set transaction_timestamp() (a/k/a now()).  We want this to be the same
***************
*** 1604,1610 ****
  	 */
  	latestXid = RecordTransactionCommit();
  
! 	PG_TRACE1(transaction__commit, MyProc->lxid);
  
  	/*
  	 * Let others know about no transaction in progress by me. Note that this
--- 1604,1610 ----
  	 */
  	latestXid = RecordTransactionCommit();
  
! 	DATABASE_TRANSACTION_COMMIT(MyProc->lxid);
  
  	/*
  	 * Let others know about no transaction in progress by me. Note that this
***************
*** 1990,1996 ****
  	 */
  	latestXid = RecordTransactionAbort(false);
  
! 	PG_TRACE1(transaction__abort, MyProc->lxid);
  
  	/*
  	 * Let others know about no transaction in progress by me. Note that this
--- 1990,1996 ----
  	 */
  	latestXid = RecordTransactionAbort(false);
  
! 	DATABASE_TRANSACTION_ABORT(MyProc->lxid);
  
  	/*
  	 * Let others know about no transaction in progress by me. Note that this
Index: src/backend/storage/lmgr/lock.c
===================================================================
RCS file: /projects/cvsroot/pgsql/src/backend/storage/lmgr/lock.c,v
retrieving revision 1.181
diff -c -r1.181 lock.c
*** src/backend/storage/lmgr/lock.c	2 Feb 2008 22:26:17 -0000	1.181
--- src/backend/storage/lmgr/lock.c	22 Feb 2008 18:33:59 -0000
***************
*** 787,797 ****
  		 * Sleep till someone wakes me up.
  		 */
  
! 		PG_TRACE2(lock__startwait, locktag->locktag_field2, lockmode);
  
  		WaitOnLock(locallock, owner);
  
! 		PG_TRACE2(lock__endwait, locktag->locktag_field2, lockmode);
  
  		/*
  		 * NOTE: do not do any material change of state between here and
--- 787,797 ----
  		 * Sleep till someone wakes me up.
  		 */
  
! 		POSTGRESQL_LOCK_STARTWAIT(locktag->locktag_field2, lockmode);
  
  		WaitOnLock(locallock, owner);
  
! 		POSTGRESQL_LOCK_ENDWAIT(locktag->locktag_field2, lockmode);
  
  		/*
  		 * NOTE: do not do any material change of state between here and
Index: src/backend/storage/lmgr/lwlock.c
===================================================================
RCS file: /projects/cvsroot/pgsql/src/backend/storage/lmgr/lwlock.c,v
retrieving revision 1.50
diff -c -r1.50 lwlock.c
*** src/backend/storage/lmgr/lwlock.c	1 Jan 2008 19:45:52 -0000	1.50
--- src/backend/storage/lmgr/lwlock.c	22 Feb 2008 18:33:59 -0000
***************
*** 447,453 ****
  		block_counts[lockid]++;
  #endif
  
! 		PG_TRACE2(lwlock__startwait, lockid, mode);
  
  		for (;;)
  		{
--- 447,453 ----
  		block_counts[lockid]++;
  #endif
  
! 		POSTGRESQL_LWLOCK_STARTWAIT(lockid, mode);
  
  		for (;;)
  		{
***************
*** 458,464 ****
  			extraWaits++;
  		}
  
! 		PG_TRACE2(lwlock__endwait, lockid, mode);
  
  		LOG_LWDEBUG("LWLockAcquire", lockid, "awakened");
  
--- 458,464 ----
  			extraWaits++;
  		}
  
! 		POSTGRESQL_LWLOCK_ENDWAIT(lockid, mode);
  
  		LOG_LWDEBUG("LWLockAcquire", lockid, "awakened");
  
***************
*** 469,475 ****
  	/* We are done updating shared state of the lock itself. */
  	SpinLockRelease(&lock->mutex);
  
! 	PG_TRACE2(lwlock__acquire, lockid, mode);
  
  	/* Add lock to list of locks held by this backend */
  	held_lwlocks[num_held_lwlocks++] = lockid;
--- 469,475 ----
  	/* We are done updating shared state of the lock itself. */
  	SpinLockRelease(&lock->mutex);
  
! 	POSTGRESQL_LWLOCK_ACQUIRE(lockid, mode);
  
  	/* Add lock to list of locks held by this backend */
  	held_lwlocks[num_held_lwlocks++] = lockid;
***************
*** 540,552 ****
  		/* Failed to get lock, so release interrupt holdoff */
  		RESUME_INTERRUPTS();
  		LOG_LWDEBUG("LWLockConditionalAcquire", lockid, "failed");
! 		PG_TRACE2(lwlock__condacquire__fail, lockid, mode);
  	}
  	else
  	{
  		/* Add lock to list of locks held by this backend */
  		held_lwlocks[num_held_lwlocks++] = lockid;
! 		PG_TRACE2(lwlock__condacquire, lockid, mode);
  	}
  
  	return !mustwait;
--- 540,552 ----
  		/* Failed to get lock, so release interrupt holdoff */
  		RESUME_INTERRUPTS();
  		LOG_LWDEBUG("LWLockConditionalAcquire", lockid, "failed");
! 		POSTGRESQL_LWLOCK_CONDACQUIRE_FAIL(lockid, mode);
  	}
  	else
  	{
  		/* Add lock to list of locks held by this backend */
  		held_lwlocks[num_held_lwlocks++] = lockid;
! 		POSTGRESQL_LWLOCK_CONDACQUIRE(lockid, mode);
  	}
  
  	return !mustwait;
***************
*** 631,637 ****
  	/* We are done updating shared state of the lock itself. */
  	SpinLockRelease(&lock->mutex);
  
! 	PG_TRACE1(lwlock__release, lockid);
  
  	/*
  	 * Awaken any waiters I removed from the queue.
--- 631,637 ----
  	/* We are done updating shared state of the lock itself. */
  	SpinLockRelease(&lock->mutex);
  
! 	POSTGRESQL_LWLOCK_RELEASE(lockid);
  
  	/*
  	 * Awaken any waiters I removed from the queue.
Index: src/backend/utils/probes.d
===================================================================
RCS file: /projects/cvsroot/pgsql/src/backend/utils/probes.d,v
retrieving revision 1.2
diff -c -r1.2 probes.d
*** src/backend/utils/probes.d	2 Jan 2008 02:42:06 -0000	1.2
--- src/backend/utils/probes.d	22 Feb 2008 18:33:59 -0000
***************
*** 7,17 ****
   * ----------
   */
  
! provider postgresql {
  
  probe transaction__start(int);
  probe transaction__commit(int);
  probe transaction__abort(int);
  probe lwlock__acquire(int, int);
  probe lwlock__release(int);
  probe lwlock__startwait(int, int);
--- 7,22 ----
   * ----------
   */
  
! provider database {
  
  probe transaction__start(int);
  probe transaction__commit(int);
  probe transaction__abort(int);
+ 
+ };
+ 
+ provider postgresql {
+ 
  probe lwlock__acquire(int, int);
  probe lwlock__release(int);
  probe lwlock__startwait(int, int);
Index: src/include/c.h
===================================================================
RCS file: /projects/cvsroot/pgsql/src/include/c.h,v
retrieving revision 1.222
diff -c -r1.222 c.h
*** src/include/c.h	1 Jan 2008 19:45:56 -0000	1.222
--- src/include/c.h	22 Feb 2008 18:33:59 -0000
***************
*** 57,63 ****
  #include "pg_config_os.h"		/* must be before any system header files */
  #endif
  #include "postgres_ext.h"
! #include "pg_trace.h"
  
  #if _MSC_VER >= 1400
  #define errcode __msvc_errcode
--- 57,67 ----
  #include "pg_config_os.h"		/* must be before any system header files */
  #endif
  #include "postgres_ext.h"
! #ifdef ENABLE_DTRACE
! #include "probes.h"
! #else
! #include "probes_null.h"
! #endif
  
  #if _MSC_VER >= 1400
  #define errcode __msvc_errcode
#5Robert Lor
Robert.Lor@Sun.COM
In reply to: Robert Lor (#1)
Re: Proposed changes to DTrace probe implementation

Robert Lor wrote:

Proposed changes:
* Switch from using DTRACE_PROBEn macros to the dynamically generated
macros (remove pg_trace.h)
* Use "dtrace -h" to create a header file that contains the
dynamically generated macros to be used in the source code instead of
the DTRACE_PROBEn macros.
* Create a new header file called probes_null.h to map the generated
macros to no-ops
* This is unrelated to making DTrace work on Leopard, but I'd like to
propose that we split the probes into generic database and Postgres
specific providers, called "database" and "postgresql" respectively.
Other databases will be adding DTrace probes as well, and we want to
use "database" as the provider for probes that are common to all
databases. This way scripts that use the common provider will work on
databases that implement the common probes.

Since I haven't heard any objections, I assume the proposed changes are
acceptable. I'll go a head and submit the patch.

Regards,
-Robert

#6Tom Lane
tgl@sss.pgh.pa.us
In reply to: Robert Lor (#5)
Re: Proposed changes to DTrace probe implementation

Robert Lor <Robert.Lor@Sun.COM> writes:

* This is unrelated to making DTrace work on Leopard, but I'd like to
propose that we split the probes into generic database and Postgres
specific providers, called "database" and "postgresql" respectively.
Other databases will be adding DTrace probes as well, and we want to
use "database" as the provider for probes that are common to all
databases.

I'm unconvinced that there will be any probes that are common to all
databases. I'd skip this part...

regards, tom lane

#7Robert Lor
Robert.Lor@Sun.COM
In reply to: Tom Lane (#6)
Re: Proposed changes to DTrace probe implementation

Tom Lane wrote:

I'm unconvinced that there will be any probes that are common to all
databases. I'd skip this part...

Any reason why we can't consider probes like transaction-start,
transaction-commit, or transaction-abort as common probes that can also
be used in other (maybe no all) databases? We are only talking about
the probe definition here as shown below, not how they will be implemented.

probe transaction__start(int);
probe transaction__commit(int);
probe transaction__abort(int);

Regards,
-Robert

#8Tom Lane
tgl@sss.pgh.pa.us
In reply to: Robert Lor (#7)
Re: Proposed changes to DTrace probe implementation

Robert Lor <Robert.Lor@Sun.COM> writes:

Tom Lane wrote:

I'm unconvinced that there will be any probes that are common to all
databases. I'd skip this part...

Any reason why we can't consider probes like transaction-start,
transaction-commit, or transaction-abort as common probes that can also
be used in other (maybe no all) databases?

I'm unimpressed; it's not at all clear that you'd be measuring quite the
same thing in, say, mysql as in postgres.

Possibly I have a different view of the uses of dtrace than you do, but
most of the events I'd be interested in probing are probably pretty
Postgres-specific. I think distinguishing half a dozen of them on the
assumption that there should be (exact) matches to that probe point in
most databases is misleading and a source of useless extra notation.

regards, tom lane

#9Robert Lor
Robert.Lor@Sun.COM
In reply to: Tom Lane (#8)
Re: Proposed changes to DTrace probe implementation

Tom Lane wrote:

I'm unimpressed; it's not at all clear that you'd be measuring quite the
same thing in, say, mysql as in postgres.

I think it depends on the probe, but for transaction rates like start or
commit, don't you think it's pretty black & white as long as the probes
are placed in the correct locations.

Possibly I have a different view of the uses of dtrace than you do, but
most of the events I'd be interested in probing are probably pretty
Postgres-specific.

I agree that most probes, especially low level ones, will be Postgres
specific.

I think distinguishing half a dozen of them on the
assumption that there should be (exact) matches to that probe point in
most databases is misleading and a source of useless extra notation.

This is just forward looking on my part since other OS databases will
most likely be adding Dtrace probes as well. I'll put them back
together in one provider for now!

Regards,
-Robert

#10Gregory Stark
stark@enterprisedb.com
In reply to: Tom Lane (#8)
Re: Proposed changes to DTrace probe implementation

"Tom Lane" <tgl@sss.pgh.pa.us> writes:

Possibly I have a different view of the uses of dtrace than you do, but
most of the events I'd be interested in probing are probably pretty
Postgres-specific.

I think both types of probes are useful to different people.

One of the really neat things about dtrace, imho, is that it lets you
correlate data from different levels of abstraction. You can find out how many
physical i/o's happen per i/o syscall. And how many i/o syscalls per database
transaction. And how many database transactions per application request. Etc.

Perhaps looking at the standard database SNMP MIB counters would give us a
place to start for upward facing events people want to trace for databases in
general.

--
Gregory Stark
EnterpriseDB http://www.enterprisedb.com
Ask me about EnterpriseDB's On-Demand Production Tuning

#11Robert Lor
Robert.Lor@Sun.COM
In reply to: Gregory Stark (#10)
Re: Proposed changes to DTrace probe implementation

Gregory Stark wrote:

I think both types of probes are useful to different people.

I think certain higher level probes can be really useful to DBAs.

Perhaps looking at the standard database SNMP MIB counters would give us a
place to start for upward facing events people want to trace for databases in
general.

Great idea. I found this link for public RDBMS MIB
http://www.mnlab.cs.depaul.edu/cgi-bin/sbrowser.cgi?HOST=&amp;OID=RDBMS-MIB!rdbmsMIB

The stats in rdbmsSrvInfoTable is quite useful, and it's one of the
tables that Oracle implements in their SNMP support.
http://download-east.oracle.com/docs/cd/B14099_19/manage.1012/b16244/appdx_d_rdbms.htm

Regards,
-Robert

#12Tom Lane
tgl@sss.pgh.pa.us
In reply to: Robert Lor (#9)
Re: Proposed changes to DTrace probe implementation

Robert Lor <Robert.Lor@Sun.COM> writes:

Tom Lane wrote:

I'm unimpressed; it's not at all clear that you'd be measuring quite the
same thing in, say, mysql as in postgres.

I think it depends on the probe, but for transaction rates like start or
commit, don't you think it's pretty black & white as long as the probes
are placed in the correct locations.

I'm not so sure. For instance, from what I understand about mysql
(admittedly not a lot) their notion of transaction is rather
storage-engine-specific. It wouldn't be hard to come up with situations
where they show many more or fewer "transactions" than we do.

The concern I've got about this is basically that it would encourage
plastering the same label on subtly different counts, leading to
confusion and perhaps mistaken conclusions. I would prefer to see any
common probes be reverse-engineered *after the fact*, ie, after you've
already instrumented several DB's you're in a better position to figure
out what's common and what's not. I distrust preconceived notions about
that.

regards, tom lane

#13Robert Lor
Robert.Lor@Sun.COM
In reply to: Tom Lane (#12)
Re: Proposed changes to DTrace probe implementation

Tom Lane wrote:

The concern I've got about this is basically that it would encourage
plastering the same label on subtly different counts, leading to
confusion and perhaps mistaken conclusions. I would prefer to see any
common probes be reverse-engineered *after the fact*, ie, after you've
already instrumented several DB's you're in a better position to figure
out what's common and what's not. I distrust preconceived notions about
that.

Your point is well taken, and we can revisit this later!

Regards,
-Robert

#14Magnus Hagander
magnus@hagander.net
In reply to: Robert Lor (#11)
Re: Proposed changes to DTrace probe implementation

On Tue, Feb 26, 2008 at 03:48:28PM -0600, Robert Lor wrote:

Gregory Stark wrote:

I think both types of probes are useful to different people.

I think certain higher level probes can be really useful to DBAs.

Perhaps looking at the standard database SNMP MIB counters would give us a
place to start for upward facing events people want to trace for databases
in
general.

Great idea. I found this link for public RDBMS MIB
http://www.mnlab.cs.depaul.edu/cgi-bin/sbrowser.cgi?HOST=&amp;OID=RDBMS-MIB!rdbmsMIB

The stats in rdbmsSrvInfoTable is quite useful, and it's one of the
tables that Oracle implements in their SNMP support.
http://download-east.oracle.com/docs/cd/B14099_19/manage.1012/b16244/appdx_d_rdbms.htm

Incidentally, most of that's already supported by the pg snmp provider,
through the stats system.

//Magnus

#15Paul van den Bogaard
Paul.Vandenbogaard@Sun.COM
In reply to: Magnus Hagander (#14)
Re: Proposed changes to DTrace probe implementation

but putting these and other counters in context is what could be
missing. Correlating a given (set of) stats with others (possible
outside of the application domain) is one of the assets offered by
DTrace. Besides the generic transaction begin/start/end it could
also be helpful to see the number of parses,binds,executes per
transaction, user, connection etc.
And yes, I feel Tom is right in fearing that these things can be used
in "creative" ways. However is this not true for most benchmarks/
results when ones objective is to "show" how perfect/better/whatever
product/platform A behaves/is compared to B, C, etc...

One benefit for generalizing a subset of the DTrace probes is the
possibility of creating generic DTrace scripts that can be used for
many database installations. DTrace is great, programming DTrace
scripts is fun (my view, and sure I am biased as a Sun employee :-),
but it is not likely to be something a dba would like to master. The
availability of "generic" scripts does add value.

BTW I wonder if we could somehow combine DTrace as a contextual tool
with the counters provided through the stats interface. Any insight/
ideas?

--Paul.

On 27-feb-2008, at 10:28, Magnus Hagander wrote:

On Tue, Feb 26, 2008 at 03:48:28PM -0600, Robert Lor wrote:

Gregory Stark wrote:

I think both types of probes are useful to different people.

I think certain higher level probes can be really useful to DBAs.

Perhaps looking at the standard database SNMP MIB counters would
give us a
place to start for upward facing events people want to trace for
databases
in
general.

Great idea. I found this link for public RDBMS MIB
http://www.mnlab.cs.depaul.edu/cgi-bin/sbrowser.cgi?
HOST=&OID=RDBMS-MIB!rdbmsMIB

The stats in rdbmsSrvInfoTable is quite useful, and it's one of the
tables that Oracle implements in their SNMP support.
http://download-east.oracle.com/docs/cd/B14099_19/manage.1012/
b16244/appdx_d_rdbms.htm

Incidentally, most of that's already supported by the pg snmp
provider,
through the stats system.

//Magnus

---------------------------(end of
broadcast)---------------------------
TIP 3: Have you checked our extensive FAQ?

http://www.postgresql.org/docs/faq

------------------------------------------------------------------------
---------------------
Paul van den Bogaard
Paul.vandenBogaard@sun.com
ISV-E -- ISV Engineering, Opensource Engineering group

Sun Microsystems, Inc phone: +31
334 515 918
Saturnus 1
extentsion: x (70)15918
3824 ME Amersfoort mobile: +31
651 913 354
The Netherlands
fax: +31 334 515 001