Testing with concurrent sessions

On Jan 1, 2010, at 6:01 PM, Kevin Grittner wrote:

It's very soon going to be critical that I be able to test particular
interleavings of statements in particular concurrent transaction sets
to be able to make meaningful progress on the serializable
transaction work. It would be wonderful if some of these scripts
could be integrated into the PostgreSQL 'make check' scripts,
although that's not an absolute requirement. I'm not really
concerned about performance tests for a while, just testing the
behavior of particular interleavings of statements in multiple
sessions. If psql isn't expected to support that soon, any
suggestions? Is pgTAP suited to this?

We've discussed it a bit in the past with regard to testing replication and such. I think the consensus was, failing support for concurrent sessions in psql, to use a Perl script to control multiple psql sessions and perhaps use Test::More to do the testing. Although pgTAP might make sense, too, if the tests ought to run in the database.

Best,

David

"David E. Wheeler" <david@kineticode.com> wrote:

I think the consensus was, failing support for concurrent sessions
in psql, to use a Perl script to control multiple psql sessions
and perhaps use Test::More to do the testing.

Are there any examples of that? While I can hack my way through
regular expressions when I need them, perl as a language is
something I don't know at all; with an example I might be able to
come up to speed quickly, though.

Although pgTAP might make sense, too, if the
tests ought to run in the database.

I need to run statements against a database; I don't particularly
need any special features of psql for this. Can anyone confirm that
pgTAP can let you interleave specific statements against specific
connections in a specific sequence? (The answer to that didn't leap
out at me in a quick scan of the docs.)

-Kevin

On mån, 2010-01-04 at 17:10 -0600, Kevin Grittner wrote:

"David E. Wheeler" <david@kineticode.com> wrote:

I think the consensus was, failing support for concurrent sessions
in psql, to use a Perl script to control multiple psql sessions
and perhaps use Test::More to do the testing.

Are there any examples of that? While I can hack my way through
regular expressions when I need them, perl as a language is
something I don't know at all; with an example I might be able to
come up to speed quickly, though.

If you're not deep into Perl, perhaps ignore the Test::More comment for
the moment and just use DBI to connect to several database sessions,
execute your queries and check if the results are what you want. Once
you have gotten somewhere with that, wrapping a test harness around it
is something others will be able to help with.

Although pgTAP might make sense, too, if the
tests ought to run in the database.

I need to run statements against a database; I don't particularly
need any special features of psql for this. Can anyone confirm that
pgTAP can let you interleave specific statements against specific
connections in a specific sequence? (The answer to that didn't leap
out at me in a quick scan of the docs.)

pgTAP isn't really going to help you here, as it runs with *one*
database session, and its main functionality is to format the result of
SQL functions into TAP output, which is not very much like what you
ought to be doing.

On Jan 4, 2010, at 3:29 PM, Peter Eisentraut wrote:

If you're not deep into Perl, perhaps ignore the Test::More comment for
the moment and just use DBI to connect to several database sessions,
execute your queries and check if the results are what you want. Once
you have gotten somewhere with that, wrapping a test harness around it
is something others will be able to help with.

Last I heard, Andrew was willing to require Test::More for testing, so that a Perl script could handle multiple psql connections (perhaps forked) and output test results based on them. But he wasn't as interested in requiring DBI and DBD::Pg, neither of which are in the Perl core and are more of a PITA to install (not huge, but the barrier might as well stay low).

pgTAP isn't really going to help you here, as it runs with *one*
database session, and its main functionality is to format the result of
SQL functions into TAP output, which is not very much like what you
ought to be doing.

Right, exactly.

Best,

David

Hi,

Kevin Grittner wrote:

It's very soon going to be critical that I be able to test particular
interleavings of statements in particular concurrent transaction sets
to be able to make meaningful progress on the serializable
transaction work.

I've something in place for Postgres-R, as I also need to test
concurrent transactions there. It's based on python/twisted and is able
to start multiple Postgres instances (as required for testing
replication) and query them concurrently (as you seem to need as well).
It uses an asynchronous event loop (from twisted) and basically controls
processes, issues queries and checks results and ordering constraints
(e.g. transaction X must commit and return a result before transaction Y).

I'm still under the impression that this testing framework needs
cleanup. However, others already showed interest as well...

Regards

Markus Wanner

Markus Wanner <markus@bluegap.ch> wrote:

Kevin Grittner wrote:

It's very soon going to be critical that I be able to test
particular interleavings of statements in particular concurrent
transaction sets to be able to make meaningful progress on the
serializable transaction work.

I've something in place for Postgres-R, as I also need to test
concurrent transactions there. It's based on python/twisted and is
able to start multiple Postgres instances (as required for testing
replication) and query them concurrently (as you seem to need as
well). It uses an asynchronous event loop (from twisted) and
basically controls processes, issues queries and checks results
and ordering constraints (e.g. transaction X must commit and
return a result before transaction Y).

Where would I find this (and any related documentation)?

-Kevin

Hi,

Kevin Grittner wrote:

Where would I find this (and any related documentation)?

Sorry, if that didn't get clear. I'm trying to put together something I
can release real soon now (tm). I'll keep you informed.

Regards

Markus Wanner

"David E. Wheeler" <david@kineticode.com> wrote:

Last I heard, Andrew was willing to require Test::More for
testing, so that a Perl script could handle multiple psql
connections (perhaps forked) and output test results based on
them. But he wasn't as interested in requiring DBI and DBD::Pg,
neither of which are in the Perl core and are more of a PITA to
install (not huge, but the barrier might as well stay low).

OK, I've gotten familiar with Perl as a programming language and
tinkered with Test::More. What's not clear to me yet is what would
be considered good technique for launching several psql sessions
from that environment, interleaving commands to each of them, and
checking results from each of them as the test plan progresses. Any
code snippets or URLs to help me understand that are welcome. (It
seems clear enough with DBI, but I'm trying to avoid that per the
above.)

-Kevin

On ons, 2010-01-06 at 15:52 -0600, Kevin Grittner wrote:

"David E. Wheeler" <david@kineticode.com> wrote:

Last I heard, Andrew was willing to require Test::More for
testing, so that a Perl script could handle multiple psql
connections (perhaps forked) and output test results based on
them. But he wasn't as interested in requiring DBI and DBD::Pg,
neither of which are in the Perl core and are more of a PITA to
install (not huge, but the barrier might as well stay low).

OK, I've gotten familiar with Perl as a programming language and
tinkered with Test::More. What's not clear to me yet is what would
be considered good technique for launching several psql sessions
from that environment, interleaving commands to each of them, and
checking results from each of them as the test plan progresses. Any
code snippets or URLs to help me understand that are welcome. (It
seems clear enough with DBI, but I'm trying to avoid that per the
above.)

Then I don't see much of a point in using Perl. You might as well fire
up a few psqls from a shell script.

On Jan 6, 2010, at 1:52 PM, Kevin Grittner wrote:

Last I heard, Andrew was willing to require Test::More for
testing, so that a Perl script could handle multiple psql
connections (perhaps forked) and output test results based on
them. But he wasn't as interested in requiring DBI and DBD::Pg,
neither of which are in the Perl core and are more of a PITA to
install (not huge, but the barrier might as well stay low).

OK, I've gotten familiar with Perl as a programming language and
tinkered with Test::More. What's not clear to me yet is what would
be considered good technique for launching several psql sessions
from that environment, interleaving commands to each of them, and
checking results from each of them as the test plan progresses. Any
code snippets or URLs to help me understand that are welcome. (It
seems clear enough with DBI, but I'm trying to avoid that per the
above.)

Probably the simplest way is to use the core IPC::Open3 module:

http://search.cpan.org/perldoc?IPC::Open3

IPC::Run might be easier to use if it's available, but it's not in Perl core, so YMMV. Really it's up to andrew what modules he requires test servers to have.

Best,

David

On Jan 6, 2010, at 2:08 PM, Peter Eisentraut wrote:

Then I don't see much of a point in using Perl. You might as well fire
up a few psqls from a shell script

If you're more comfortable with shell, then yes. Although then it won't run on Windows, will it?

Best,

David

On 2010-01-07 00:08 +0200, Peter Eisentraut wrote:

On ons, 2010-01-06 at 15:52 -0600, Kevin Grittner wrote:

"David E. Wheeler"<david@kineticode.com> wrote:

Last I heard, Andrew was willing to require Test::More for
testing, so that a Perl script could handle multiple psql
connections (perhaps forked) and output test results based on
them. But he wasn't as interested in requiring DBI and DBD::Pg,
neither of which are in the Perl core and are more of a PITA to
install (not huge, but the barrier might as well stay low).

OK, I've gotten familiar with Perl as a programming language and
tinkered with Test::More. What's not clear to me yet is what would
be considered good technique for launching several psql sessions
from that environment, interleaving commands to each of them, and
checking results from each of them as the test plan progresses. Any
code snippets or URLs to help me understand that are welcome. (It
seems clear enough with DBI, but I'm trying to avoid that per the
above.)

Then I don't see much of a point in using Perl. You might as well fire
up a few psqls from a shell script.

I don't see how that would work, but I might have misunderstood what
we're reaching for here. What I think would be most useful would be to
interleave statements between transactions, not just randomly fire psql
sessions and hope for race conditions.

Regards,
Marko Tiikkaja

Marko Tiikkaja <marko.tiikkaja@cs.helsinki.fi> wrote:

On 2010-01-07 00:08 +0200, Peter Eisentraut wrote:

Then I don't see much of a point in using Perl. You might as
well fire up a few psqls from a shell script.

I don't see how that would work, but I might have misunderstood
what we're reaching for here. What I think would be most useful
would be to interleave statements between transactions, not just
randomly fire psql sessions and hope for race conditions.

Yeah, I want to test specific interleavings of statements on
concurrent connections. There may *also* be some tests which throw
a lot at the server concurrently in a more random fashion, but it is
important to be able to have some very controlled tests where we
don't count on randomly creating the desired conflicts.

It would be valuable to be able to include some of these tests with
controlled and predicatable statement interleavings in the "make
check" tests.

-Kevin

Marko Tiikkaja wrote:

On 2010-01-07 00:08 +0200, Peter Eisentraut wrote:

Then I don't see much of a point in using Perl. You might as well fire
up a few psqls from a shell script.

I don't see how that would work, but I might have misunderstood what
we're reaching for here. What I think would be most useful would be
to interleave statements between transactions, not just randomly
fire psql sessions and hope for race conditions.

Open a few psql with -f pointing to a pipe, and from the shell write
into the pipe? I don't think it's straightforward, but it should be
possible.

--
Alvaro Herrera http://www.CommandPrompt.com/
PostgreSQL Replication, Consulting, Custom Development, 24x7 support

Alvaro Herrera <alvherre@commandprompt.com> wrote:

Open a few psql with -f pointing to a pipe, and from the shell
write into the pipe? I don't think it's straightforward, but it
should be possible.

I'll play with it and see what I can do.

Thanks,

-Kevin

On Wed, Jan 6, 2010 at 4:52 PM, Kevin Grittner
<Kevin.Grittner@wicourts.gov> wrote:

"David E. Wheeler" <david@kineticode.com> wrote:

Last I heard, Andrew was willing to require Test::More for
testing, so that a Perl script could handle multiple psql
connections (perhaps forked) and output test results based on
them. But he wasn't as interested in requiring DBI and DBD::Pg,
neither of which are in the Perl core and are more of a PITA to
install (not huge, but the barrier might as well stay low).

OK, I've gotten familiar with Perl as a programming language and
tinkered with Test::More.  What's not clear to me yet is what would
be considered good technique for launching several psql sessions
from that environment, interleaving commands to each of them, and
checking results from each of them as the test plan progresses.  Any
code snippets or URLs to help me understand that are welcome.  (It
seems clear enough with DBI, but I'm trying to avoid that per the
above.)

Doing this without DBI is going to be ten times harder than doing it
with DBI. Are we really sure that's not a viable option?

...Robert

Robert Haas wrote:

On Wed, Jan 6, 2010 at 4:52 PM, Kevin Grittner
<Kevin.Grittner@wicourts.gov> wrote:

"David E. Wheeler" <david@kineticode.com> wrote:

Last I heard, Andrew was willing to require Test::More for
testing, so that a Perl script could handle multiple psql
connections (perhaps forked) and output test results based on
them. But he wasn't as interested in requiring DBI and DBD::Pg,
neither of which are in the Perl core and are more of a PITA to
install (not huge, but the barrier might as well stay low).

OK, I've gotten familiar with Perl as a programming language and
tinkered with Test::More. What's not clear to me yet is what would
be considered good technique for launching several psql sessions
from that environment, interleaving commands to each of them, and
checking results from each of them as the test plan progresses. Any
code snippets or URLs to help me understand that are welcome. (It
seems clear enough with DBI, but I'm trying to avoid that per the
above.)

Doing this without DBI is going to be ten times harder than doing it
with DBI. Are we really sure that's not a viable option?

In the buildfarm? Yes, I think so. The philosophy of the buildfarm is
that it should do what you would do yourself by hand.

And adding DBI as a requirement for running a buildfarm member would be
a significant extra barrier to entry, ISTM. (I am very fond of DBI, and
use it frequently, BTW)

I'm persuadable on most things, but this one would take a bit of doing.

A parallel psql seems to me a better way to go. We talked about that a
while ago, but I don't recall what happened to it.

cheers

andrew

On Wed, Jan 6, 2010 at 8:40 PM, Andrew Dunstan <andrew@dunslane.net> wrote:

Doing this without DBI is going to be ten times harder than doing it
with DBI.  Are we really sure that's not a viable option?

In the buildfarm? Yes, I think so. The philosophy of the buildfarm is that
it should do what you would do yourself by hand.

It just seems crazy to me to try to test anything without proper
language bindings. Opening a psql session and parsing the results
seems extraordinarily painful. I wonder if it would make sense write
a small wrapper program that uses libpq and dumps out the results in a
format that is easy for Perl to parse.

Another idea would be to make a set of Perl libpq bindings that is
simpler than DBD::Pg and don't go through DBI. If we put those in the
main source tree (perhaps as a contrib module) they would be available
wherever we need them.

A parallel psql seems to me a better way to go. We talked about that a while
ago, but I don't recall what happened to it.

That seems like a dead-end to me. It's hard for me to imagine it's
ever going to be more than a toy.

...Robert

Robert Haas <robertmhaas@gmail.com> writes:

It just seems crazy to me to try to test anything without proper
language bindings. Opening a psql session and parsing the results
seems extraordinarily painful. I wonder if it would make sense write
a small wrapper program that uses libpq and dumps out the results in a
format that is easy for Perl to parse.

Another idea would be to make a set of Perl libpq bindings that is
simpler than DBD::Pg and don't go through DBI. If we put those in the
main source tree (perhaps as a contrib module) they would be available
wherever we need them.

We have not yet fully accepted the notion that you must have Perl to
build (and, in fact, I am right now trying to verify that you don't).
I don't think that requiring Perl to test is going to fly.

A parallel psql seems to me a better way to go. We talked about that a while
ago, but I don't recall what happened to it.

That seems like a dead-end to me. It's hard for me to imagine it's
ever going to be more than a toy.

Well, the argument there is that it might be useful for actual use,
not only testing.

regards, tom lane

Andrew Dunstan wrote:

Robert Haas wrote:

Doing this without DBI is going to be ten times harder than doing
it with DBI. Are we really sure that's not a viable option?

In the buildfarm? Yes, I think so. The philosophy of the buildfarm
is that it should do what you would do yourself by hand.

And adding DBI as a requirement for running a buildfarm member
would be a significant extra barrier to entry, ISTM. (I am very
fond of DBI, and use it frequently, BTW)

I'm persuadable on most things, but this one would take a bit of
doing.

As far as I've been able to determine so far, to call psql in a
relatively portable way would require something like this:

http://perldoc.perl.org/perlfork.html

Is that really better than DBI?

Don't we need some way to routinely test multi-session issues?

Other ideas?

-Kevin

Tom Lane wrote:

We have not yet fully accepted the notion that you must have Perl
to build (and, in fact, I am right now trying to verify that you
don't). I don't think that requiring Perl to test is going to fly.

Well, if that's the consensus, I have to choose between trying to
implement multi-session psql and using testing which can't carry over
to long-term regular use. Are we anywhere close to an agreement on
what the multi-session psql implementation would look like? (If not
I can put something forward.)

-Kevin

On Wed, Jan 6, 2010 at 9:26 PM, Tom Lane <tgl@sss.pgh.pa.us> wrote:

Robert Haas <robertmhaas@gmail.com> writes:

It just seems crazy to me to try to test anything without proper
language bindings.  Opening a psql session and parsing the results
seems extraordinarily painful.  I wonder if it would make sense write
a small wrapper program that uses libpq and dumps out the results in a
format that is easy for Perl to parse.

Another idea would be to make a set of Perl libpq bindings that is
simpler than DBD::Pg and don't go through DBI.  If we put those in the
main source tree (perhaps as a contrib module) they would be available
wherever we need them.

We have not yet fully accepted the notion that you must have Perl to
build (and, in fact, I am right now trying to verify that you don't).
I don't think that requiring Perl to test is going to fly.

I suppose that depends on the context. I'm not exactly sure what
Kevin's goal is here. For basic regression tests, yeah, we'd probably
like to keep that Perl-free. For more complex testing, I think using
Perl makes sense. Or to put the shoe on the other foot, if we DON'T
allow the use of Perl for more complex testing, then we're probably
not going to have any more complex tests. If we use a hypothetical
concurrent psql implementation to run the tests, how will we analyze
the results? It's no secret that the current regression tests are
fairly limited, in part because the only thing we can do with them is
diff the output against one or two "known good" results.

...Robert

Robert Haas wrote:

I'm not exactly sure what Kevin's goal is here.

I think it would be insane to try to do something like serializable
isolation mode without having regression tests. I would want more
tests than could reasonably go into the 'make check' suite to support
development, but it would be very good to have some go in there.

For basic regression tests, yeah, we'd probably like to keep that
Perl-free.

OK. Is parallel psql the only reasonable option?

For more complex testing, I think using Perl makes sense. Or to put
the shoe on the other foot, if we DON'T allow the use of Perl for
more complex testing, then we're probably not going to have any
more complex tests.

Do you envision some test suite committed to CVS beyond the 'make
check' tests, for "on demand" testing at a more rigorous level?
Am I missing something that's already there?

-Kevin

Kevin Grittner escribi�:

Well, if that's the consensus, I have to choose between trying to
implement multi-session psql and using testing which can't carry over
to long-term regular use. Are we anywhere close to an agreement on
what the multi-session psql implementation would look like? (If not
I can put something forward.)

See
http://archives.postgresql.org/message-id/8204.1207689056@sss.pgh.pa.us
and followups.

--
Alvaro Herrera http://www.CommandPrompt.com/
The PostgreSQL Company - Command Prompt, Inc.

On Wed, Jan 6, 2010 at 10:00 PM, Kevin Grittner
<Kevin.Grittner@wicourts.gov> wrote:

For basic regression tests, yeah, we'd probably like to keep that
Perl-free.

OK.  Is parallel psql the only reasonable option?

It seems so, assuming you're willing to concede that it is a
reasonable option in the first place.

For more complex testing, I think using Perl makes sense. Or to put
the shoe on the other foot, if we DON'T allow the use of Perl for
more complex testing, then we're probably not going to have any
more complex tests.

Do you envision some test suite committed to CVS beyond the 'make
check' tests, for "on demand" testing at a more rigorous level?
Am I missing something that's already there?

Personally, I tend to think that to test this well you are going to
need a test suite written in a scripting language. Whether or not
that gets committed to CVS is a political question, but I would be in
favor of it (assuming it's good, of course). Maybe you will find that
you can do it all with concurrent psql, but (1) I'm not convinced and
(2) if that's your plan, does that mean you're going to do nothing
until someone implements concurrent psql?

...Robert

Robert Haas wrote:

It just seems crazy to me to try to test anything without proper
language bindings. Opening a psql session and parsing the results
seems extraordinarily painful.

I've written a Python based program that spawns a captive psql and talks
to it--twice for different people--that ultimately uses the same sort of
open3() spawning David mentioned is available via IPC::Open3. You can
throw together a prototype that works well enough for some purposes in a
couple of hours. I don't know that it would ever reach really robust
though.

The main problem with that whole approach is that you have to be
extremely careful in how you deal with the situation where the captive
program is spewing an unknown amount of information back at you. How do
you know when it's done? Easy for the child and its master to deadlock
if you're not careful. In the past I worked around that issue by just
waiting for the process to end and then returning everything it had
written until that time. I don't know that this would be flexible
enough for what's needed for concurrent testing, where people are
probably going to want more of a "send a command, get some lines back
again" approach that keeps the session open.

If I thought a captive psql would work well in this context I'd have
written a prototype already. I'm not sure if it's actually possible to
do this well enough to meet expectations. Parsing psql output is
completely viable for trivial purposes though, and if the requirements
were constrained enough it might work well enough for simple concurrent
testing. While both concurrent psql and the libpq shim you suggested
would take more work, I feel a bit more confident those would result in
something that really worked as expected on every platform when finished.

--
Greg Smith 2ndQuadrant Baltimore, MD
PostgreSQL Training, Services and Support
greg@2ndQuadrant.com www.2ndQuadrant.com

On ons, 2010-01-06 at 14:10 -0800, David E. Wheeler wrote:

On Jan 6, 2010, at 2:08 PM, Peter Eisentraut wrote:

Then I don't see much of a point in using Perl. You might as well fire
up a few psqls from a shell script

If you're more comfortable with shell, then yes. Although then it won't run on Windows, will it?

Well, I'm not saying that this is necessarily the better alternative.
But you might have to compromise somewhere. Otherwise this project will
take two years to complete and will be an unmaintainable mess (see
pg_regress).

On ons, 2010-01-06 at 20:49 -0500, Robert Haas wrote:

Another idea would be to make a set of Perl libpq bindings that is
simpler than DBD::Pg and don't go through DBI. If we put those in the
main source tree (perhaps as a contrib module) they would be available
wherever we need them.

http://anoncvs.postgresql.org/cvsweb.cgi/pgsql/src/interfaces/perl5/?hideattic=0

On 7/01/2010 9:15 AM, Robert Haas wrote:

On Wed, Jan 6, 2010 at 4:52 PM, Kevin Grittner
<Kevin.Grittner@wicourts.gov> wrote:

"David E. Wheeler"<david@kineticode.com> wrote:

Last I heard, Andrew was willing to require Test::More for
testing, so that a Perl script could handle multiple psql
connections (perhaps forked) and output test results based on
them. But he wasn't as interested in requiring DBI and DBD::Pg,
neither of which are in the Perl core and are more of a PITA to
install (not huge, but the barrier might as well stay low).

OK, I've gotten familiar with Perl as a programming language and
tinkered with Test::More. What's not clear to me yet is what would
be considered good technique for launching several psql sessions
from that environment, interleaving commands to each of them, and
checking results from each of them as the test plan progresses. Any
code snippets or URLs to help me understand that are welcome. (It
seems clear enough with DBI, but I'm trying to avoid that per the
above.)

Doing this without DBI is going to be ten times harder than doing it
with DBI. Are we really sure that's not a viable option?

At this point, I'm personally wondering if it's worth putting together a
simple (ish) C program that reads a file describing command
interleavings on n connections. It fires up one thread per connection
required, then begins queuing commands up for the threads to execute in
per-thread fifo queues. The input file may contain synchronization
points where two or more explicitly specified threads (or just all
threads) must finish all their queued work before they may be given more.

Yes, it requires wrangling low-level threading ( pthreads, or the
practically equivalent for simple purposes but differently spelled win32
threading ) so it's not going to be beautiful. But it'd permit a
declarative form for tests and a single, probably fairly maintainable,
test runner.

I reluctantly suspect that XML would be a good way to describe the tests
- first a block declaring your connections and their conn strings, then
a sequence of statements (each of which is associated with a named
connection) and synchronization points. Though, come to think of it, a
custom plaintext format would be pretty trivial too.

CONN conn1: dbname=regress, user=regress
CONN conn2: dbname=regress, user=regress
STMT conn1: SELECT blah blah;
STMT conn2: UPDATE blah blah;
SYNC conn1, conn2

etc. Or alternately one-file-per-connection (which would be handy if one
connection has *lots* of commands and others only occasional ones) - the
only trouble there being how to conveniently specify synchronization points.

Anyway: If Java were acceptable I'd put one together now - but somehow I
don't think requiring Java would be popular if Perl is an issue ;-) My
C/pthreads is more than a little bit rusty (ie: practially nonexistent)
and mostly confined to exception-controlled C++ code with RAII for lock
management. If C++ is OK, I can write and post a tool for evaluation,
but if it must be plain C ... well, I'll avoid scarring you with the
sight of what I'd produce.

I suspect that a custom tool for this job could actually be fairly
simple. A lot simpler than a proper, clean and usable parallel psql, anyway.

--
Craig Ringer

-----BEGIN PGP SIGNED MESSAGE-----
Hash: RIPEMD160

Doing this without DBI is going to be ten times harder than doing it
with DBI. Are we really sure that's not a viable option?

In the buildfarm? Yes, I think so. The philosophy of the buildfarm is
that it should do what you would do yourself by hand.

And adding DBI as a requirement for running a buildfarm member would be
a significant extra barrier to entry, ISTM. (I am very fond of DBI, and
use it frequently, BTW)

What about something less than a requirement then? If you have it great,
you can run these extra tests. If you don't have it, no harm, no foul.

We could even bundle DBI and DBD::Pg to ensure that the minimum versions
are there. All the prerequisites should be in place for 99% of the machines:
a C compiler and Perl are the biggies, and I can't see any buildfarm members
running without those. :)

- --
Greg Sabino Mullane greg@turnstep.com
PGP Key: 0x14964AC8 201001071014
http://biglumber.com/x/web?pk=2529DF6AB8F79407E94445B4BC9B906714964AC8
-----BEGIN PGP SIGNATURE-----

iEYEAREDAAYFAktF+ucACgkQvJuQZxSWSsjYOgCglyLIyGCr60og+iQSnyRgkowd
+lYAnRDjPe/XxC7gb9OBPdpZlqU0wncK
=kPIR
-----END PGP SIGNATURE-----

On 2010-01-07 11:50 +0200, Craig Ringer wrote:

On 7/01/2010 9:15 AM, Robert Haas wrote:

Doing this without DBI is going to be ten times harder than doing it
with DBI. Are we really sure that's not a viable option?

At this point, I'm personally wondering if it's worth putting together a
simple (ish) C program that reads a file describing command
interleavings on n connections. It fires up one thread per connection
required, then begins queuing commands up for the threads to execute in
per-thread fifo queues. The input file may contain synchronization
points where two or more explicitly specified threads (or just all
threads) must finish all their queued work before they may be given more.

CONN conn1: dbname=regress, user=regress
CONN conn2: dbname=regress, user=regress
STMT conn1: SELECT blah blah;
STMT conn2: UPDATE blah blah;
SYNC conn1, conn2

etc. Or alternately one-file-per-connection (which would be handy if one
connection has *lots* of commands and others only occasional ones) - the
only trouble there being how to conveniently specify synchronization points.

I had a similar syntax in mind, but instead of using threads, just
execute the file in order using asynchronous connections.

Regards,
Marko Tiikkaja

On 2010-01-07 18:13 +0200, Marko Tiikkaja wrote:

I had a similar syntax in mind, but instead of using threads, just
execute the file in order using asynchronous connections.

I completely failed to make the point here which was to somehow mark
which statements will (or, should) block. So here we go:

A=> BEGIN;
B=> BEGIN;
A=> UPDATE foo ..;
&B=> UPDATE foo ..; -- this will block
A=> COMMIT;
B=> SELECT * FROM foo;
B=> COMMIT;

[expected output here]

Regards,
Marko Tiikkaja

On Jan 6, 2010, at 6:26 PM, Tom Lane wrote:

We have not yet fully accepted the notion that you must have Perl to
build (and, in fact, I am right now trying to verify that you don't).
I don't think that requiring Perl to test is going to fly.

I believe that the build farm already requires Perl, regardless of whether the PostgreSQL build itself requires it.

Best,

David

"Greg Sabino Mullane" <greg@turnstep.com> writes:

We could even bundle DBI and DBD::Pg to ensure that the minimum versions
are there.

As a packager, my reaction to that is "over my dead body". We have
enough trouble keeping our own software up to date, and pretty much
every external component that we've started to bundle has been a
disaster from a maintenance standpoint. (Examples: the zic database
is constant work and yet almost never up to date; the snowball stemmer
never gets updated.)

regards, tom lane

David E. Wheeler wrote:

On Jan 6, 2010, at 6:26 PM, Tom Lane wrote:

We have not yet fully accepted the notion that you must have Perl to
build (and, in fact, I am right now trying to verify that you don't).
I don't think that requiring Perl to test is going to fly.

I believe that the build farm already requires Perl, regardless of whether the PostgreSQL build itself requires it.

Unless I am mistaken, Perl is required in any case to build from CVS,
although not from a tarball.

DBI/DBD::pg is quite another issue, however. I have been deliberately
very conservative about what modules to require for the buildfarm, and
we have similarly (and I think wisely) been conservative about what
modules to require for Perl programs in the build process.

Using DBI/DBD::Pg would raise another issue - what version of libpq
would it be using? Not the one in the build being tested, that's for
sure. If you really want to use Perl then either a Pure Perl DBI driver
(which Greg has talked about) or a thin veneer over libpq such as we
used to have in contrib seems a safer way to go.

cheers

On Thu, Jan 7, 2010 at 11:46 AM, Andrew Dunstan <andrew@dunslane.net> wrote:

Using DBI/DBD::Pg would raise another issue - what version of libpq would it
be using? Not the one in the build being tested, that's for sure. If you
really want to use Perl then either a Pure Perl DBI driver (which Greg has
talked about) or a thin veneer over libpq such as we used to have in contrib
seems a safer way to go.

I completely agree. As between those two options, count me as +1 for
the thin veneer.

...Robert

On Wed, Jan 06, 2010 at 08:40:28PM -0500, Andrew Dunstan wrote:

A parallel psql seems to me a better way to go. We talked about that
a while ago, but I don't recall what happened to it.

Greg Stark had a patch a couple of years ago. Dunno what happened to
it since then.

Cheers,
David.
--
David Fetter <david@fetter.org> http://fetter.org/
Phone: +1 415 235 3778 AIM: dfetter666 Yahoo!: dfetter
Skype: davidfetter XMPP: david.fetter@gmail.com
iCal: webcal://www.tripit.com/feed/ical/people/david74/tripit.ics

Remember to vote!
Consider donating to Postgres: http://www.postgresql.org/about/donate

On Jan 6, 2010, at 6:31 PM, Kevin Grittner wrote:

As far as I've been able to determine so far, to call psql in a
relatively portable way would require something like this:

http://perldoc.perl.org/perlfork.html

Here's an example using IPC::Open3:

#!/usr/local/bin/perl -w

use strict;
use warnings;

use IPC::Open3;
use Symbol 'gensym';
use constant EOC => "__DONE__\n";

my ($in1, $out1, $err1) = (gensym, gensym, gensym);
my ($in2, $out2, $err2) = (gensym, gensym, gensym);

my $pid1 = open3 $in1, $out1, $err1, 'bash';
my $pid2 = open3 $in2, $out2, $err2, 'bash';

print $in1 "cd ~/dev/postgresql\n";
print $in1 "ls doc\n";
print $in1 "echo ", EOC;
while ((my $line = <$out1>)) {
last if $line eq EOC;
print "LS: $line";
}

print "#### Finished file listing\n\n";

print $in2 "cd ~/dev/postgresql\n";
print $in2 "head -4 README\n";
print $in2 "echo ", EOC;
while (defined( my $line = <$out2> )) {
last if $line eq EOC;
print "HEAD: $line";
}

print "#### Finished reading README\n\n";

print $in1 "exit\n";
print $in2 "exit\n";
waitpid $pid2, 0;

print "#### All done!\n";

With that, I get:

LS: KNOWN_BUGS
LS: MISSING_FEATURES
LS: Makefile
LS: README.mb.big5
LS: README.mb.jp
LS: TODO
LS: bug.template
LS: src
#### Finished file listing

HEAD: PostgreSQL Database Management System
HEAD: =====================================
HEAD:
HEAD: This directory contains the source code distribution of the PostgreSQL
#### Finished reading README

#### All done!

I could easily write a very simple module to abstract all that stuff for you, then you could just do something like:

my $psql1 = Shell::Pipe->new(qw(psql -U postgres));
my $psql2 = Shell::Pipe->new(qw(psql -U postgres));

$psql1->print('SELECT * from pg_class');
while (my $line = $psql1->readln) { print "Output: $line\n" }
$psql1->close;

All I'd need is some more reliable way than "echo "DONE__\n" to be able to tell when a particular command has finished outputting.

Thoughts?

Best,

David

Andrew Dunstan <andrew@dunslane.net> writes:

Unless I am mistaken, Perl is required in any case to build from CVS,
although not from a tarball.

Right, but to my mind "building from a tarball" needs to include the
ability to run the regression tests on what you built. So injecting
Perl into that is moving the goalposts on build requirements.

regards, tom lane

On Jan 7, 2010, at 9:08 AM, Tom Lane wrote:

Right, but to my mind "building from a tarball" needs to include the
ability to run the regression tests on what you built. So injecting
Perl into that is moving the goalposts on build requirements.

In that case, there's nothing for it except concurrent psql. Or else some sort of shell environment that's available on all platforms. do we require bash on Windows? Oh, wait, the Windows build requires Perl…

Best,

David

"David E. Wheeler" <david@kineticode.com> writes:

On Jan 7, 2010, at 9:08 AM, Tom Lane wrote:

Right, but to my mind "building from a tarball" needs to include the
ability to run the regression tests on what you built. So injecting
Perl into that is moving the goalposts on build requirements.

In that case, there's nothing for it except concurrent psql.

Unless we are prepared to define concurrency testing as something
separate from the basic regression tests. Which is kind of annoying but
perhaps less so than the alternatives. It certainly seems to me to
be the kind of thing you wouldn't need to test in order to have
confidence in a local build.

regards, tom lane

On Jan 7, 2010, at 9:19 AM, Tom Lane wrote:

In that case, there's nothing for it except concurrent psql.

Unless we are prepared to define concurrency testing as something
separate from the basic regression tests. Which is kind of annoying but
perhaps less so than the alternatives. It certainly seems to me to
be the kind of thing you wouldn't need to test in order to have
confidence in a local build.

I was rather assuming that was what we were talking about here, since we have in the past discussed testing things like dump and restore, which would require something like Perl to handle multiple processes, and wouldn't work very well for a regular release.

I think if we have the ability to add tests that are not distributed, it gives us a lot more freedom and opportunity to test things that are not currently well-tested.

Best,

David

-----BEGIN PGP SIGNED MESSAGE-----
Hash: RIPEMD160

Unless we are prepared to define concurrency testing as something
separate from the basic regression tests. Which is kind of annoying but
perhaps less so than the alternatives. It certainly seems to me to
be the kind of thing you wouldn't need to test in order to have
confidence in a local build.

I thought we were leaning towards something separate.

- --
Greg Sabino Mullane greg@turnstep.com
PGP Key: 0x14964AC8 201001071234
http://biglumber.com/x/web?pk=2529DF6AB8F79407E94445B4BC9B906714964AC8
-----BEGIN PGP SIGNATURE-----

iEYEAREDAAYFAktGGxYACgkQvJuQZxSWSsgG0gCfZ4eTpXd/97p3zSJpLqGhKMh6
8nMAoJ2lQUaCWNVeSPDU8fq7VnkO0s4C
=xBOo
-----END PGP SIGNATURE-----

-----BEGIN PGP SIGNED MESSAGE-----
Hash: RIPEMD160

Using DBI/DBD::Pg would raise another issue - what version of libpq
would it be using? Not the one in the build being tested, that's for
sure.

Er...why not? That's what psql uses. As for those advocating using a
custom C program written using libpq - that's basically what
DBI/DBD::Pg ends up being! Only with a shiny Perl outside and years
of real-world testing and usage.

If you really want to use Perl then either a Pure Perl DBI driver
(which Greg has talked about) or a thin veneer over libpq such as we
used to have in contrib seems a safer way to go.

I'm still *very* interested in making a libpq-less pure perl driver,
if anyone feels like funding it, let me know! :)

- --
Greg Sabino Mullane greg@turnstep.com
PGP Key: 0x14964AC8 201001071236
http://biglumber.com/x/web?pk=2529DF6AB8F79407E94445B4BC9B906714964AC8
-----BEGIN PGP SIGNATURE-----

iEYEAREDAAYFAktGHF0ACgkQvJuQZxSWSsiFmACfUJbRDUJGvDTJNjgj/dyQKVCA
tZwAn2fiXKNWbWzYXobrHZjeE8aSSiVv
=sGzK
-----END PGP SIGNATURE-----

-----BEGIN PGP SIGNED MESSAGE-----
Hash: RIPEMD160

We could even bundle DBI and DBD::Pg to ensure that the minimum versions
are there.

As a packager, my reaction to that is "over my dead body". We have
enough trouble keeping our own software up to date, and pretty much
every external component that we've started to bundle has been a
disaster from a maintenance standpoint. (Examples: the zic database
is constant work and yet almost never up to date; the snowball stemmer
never gets updated.)

As a counterargument, I'll point out that this won't be as critical
as zic, especially if we're talking about an additional/optional
set of tests. Also, Tim Bunce and I are right here, so the maintenance
should not be that bad (and I'd hazard that a lot more people in
the community know Perl/DBI than zic or stemmers).

- --
Greg Sabino Mullane greg@turnstep.com
PGP Key: 0x14964AC8 201001071315
http://biglumber.com/x/web?pk=2529DF6AB8F79407E94445B4BC9B906714964AC8
-----BEGIN PGP SIGNATURE-----

iEYEAREDAAYFAktGJNQACgkQvJuQZxSWSshoHwCg9urTTT19m55soiIjuYKKuB5L
PjYAoJbDAe7j4xsxsSFfVEkYDFpTjKE9
=48oW
-----END PGP SIGNATURE-----

Alvaro Herrera <alvherre@commandprompt.com> wrote:

Kevin Grittner escribiￜ:

Are we anywhere close to an agreement on what the multi-session
psql implementation would look like?

See

http://archives.postgresql.org/message-id/8204.1207689056@sss.pgh.pa.us

and followups.

Thanks, I had missed or forgotten this thread. It seems like it
drifted more-or-less to a consensus. Does everyone agree that this
thread represents the will of the community?

I see the thread started at the point where Greg Stark had a patch,
but by the end it was a "start over, stealing code as appropriate"
situation. Has anything happened with that? Any plans?

-Kevin

On Jan 7, 2010, at 12:39 PM, Greg Sabino Mullane wrote:

I'm still *very* interested in making a libpq-less pure perl driver,
if anyone feels like funding it, let me know! :)

You mean this one:

http://search.cpan.org/~arc/DBD-PgPP-0.07/lib/DBD/PgPP.pm

?

Cheers,
M

A.M. wrote:

On Jan 7, 2010, at 12:39 PM, Greg Sabino Mullane wrote:

I'm still *very* interested in making a libpq-less pure perl driver,
if anyone feels like funding it, let me know! :)

You mean this one:

http://search.cpan.org/~arc/DBD-PgPP-0.07/lib/DBD/PgPP.pm

?

It has a list of limitations as long as your arm.

cheers

andrew

On 8/01/2010 1:39 AM, Greg Sabino Mullane wrote:

-----BEGIN PGP SIGNED MESSAGE-----
Hash: RIPEMD160

Using DBI/DBD::Pg would raise another issue - what version of libpq
would it be using? Not the one in the build being tested, that's for
sure.

Er...why not? That's what psql uses.

Because you'd have to build DBD::Pg against the new libpq, as you do
psql. That means you need DBD::Pg sources and the build environment for
Perl (headers etc) not just a working Perl runtime. Big difference.

There's no guarantee the user's machine has the same major version of
libpq and thus no guarantee that DBD::Pq can be redirected to use your
custom libpq by LD_LIBRARY_PATH. It might also override the library
search path with rpath linking. Building your own would be pretty much
unavoidable unless you're prepared to either require the user to provide
a matching version of DBD::Pg or have the tests running with whatever
random version happens to be lying around.

Using whatever DBD::Pg version happens to be present on the machine
would be a bit of a nightmare for reproducibility of test results, and
would be really unattractive for use in the standard tests. "make check
fails on my some-random-distro" would become painfully common on the
GENERAL list...

Is bundling a Perl module in the source tree and building it as part of
the Pg build a reasonable choice? Personally, I don't think so.

Additionally, a dedicated testing tool like some folks have been talking
about would be really handy for users who want to test their schema.
I've had to write my own (in Java, or I'd be offering it) for this
purpose, as psql is completely unsuitable for concurrent-run testing and
I needed to show that my locking was safe and deadlock-free in some of
the more complex stored procs I have.

--
Craig Ringer

-----BEGIN PGP SIGNED MESSAGE-----
Hash: RIPEMD160

Using DBI/DBD::Pg would raise another issue - what version of libpq
would it be using? Not the one in the build being tested, that's for
sure.

Er...why not? That's what psql uses.

Because you'd have to build DBD::Pg against the new libpq, as you do
psql. That means you need DBD::Pg sources and the build environment for
Perl (headers etc) not just a working Perl runtime. Big difference.

Yes, but that is what I was envisioning. As you point out, that's the
only sane way to make sure we have a good version of DBD::Pg with
which to test. As a side effect, it put libpq through some extra
paces as well. :)

Is bundling a Perl module in the source tree and building it as part of
the Pg build a reasonable choice? Personally, I don't think so.

*shrug* It's different, but it's the best solution to the problem at
hand. It wouldn't be built as part of Pg, only as part of the tests.

Additionally, a dedicated testing tool like some folks have been talking
about would be really handy for users who want to test their schema.
I've had to write my own (in Java, or I'd be offering it) for this
purpose, as psql is completely unsuitable for concurrent-run testing and
I needed to show that my locking was safe and deadlock-free in some of
the more complex stored procs I have.

Sure, but it's the difference between waiting for someone to write something
(and then dealing with the invevitable bugs, tweaks, and enhancements), or
using a solid, known quantity (DBI + Test::More).

- --
Greg Sabino Mullane greg@turnstep.com
PGP Key: 0x14964AC8 201001102316
http://biglumber.com/x/web?pk=2529DF6AB8F79407E94445B4BC9B906714964AC8
-----BEGIN PGP SIGNATURE-----

iEYEAREDAAYFAktKplIACgkQvJuQZxSWSsj7TQCeMOXWS+uLIZE9QbeBWPxYv/rg
HhEAn0QZUzE2/8uyg5Oi+K8qL/oTeDSO
=R8Az
-----END PGP SIGNATURE-----

On Mon, Jan 11, 2010 at 04:17:42AM -0000, Greg Sabino Mullane wrote:

Because you'd have to build DBD::Pg against the new libpq, as you do
psql. That means you need DBD::Pg sources and the build environment for
Perl (headers etc) not just a working Perl runtime. Big difference.

Yes, but that is what I was envisioning. As you point out, that's the
only sane way to make sure we have a good version of DBD::Pg with
which to test. As a side effect, it put libpq through some extra
paces as well. :)

Is there a reason why you're suggesting using DBI? There is also the Pg
perl module which works as well and is one tenth of the size. It also
doesn't have external dependancies. It's just a plain wrapper around
libpq, which for the purposes of testing may be better.

http://search.cpan.org/~mergl/pgsql_perl5-1.9.0/Pg.pm

Have a nice day,
--
Martijn van Oosterhout <kleptog@svana.org> http://svana.org/kleptog/

-----BEGIN PGP SIGNED MESSAGE-----
Hash: RIPEMD160

Is there a reason why you're suggesting using DBI? There is also the Pg
perl module which works as well and is one tenth of the size. It also
doesn't have external dependancies. It's just a plain wrapper around
libpq, which for the purposes of testing may be better.

http://search.cpan.org/~mergl/pgsql_perl5-1.9.0/Pg.pm

Works as well? Did you take a look at that link? The last update was
early 2000, which should give you an indication of just how dead
it is.

- --
Greg Sabino Mullane greg@turnstep.com
PGP Key: 0x14964AC8 201001111241
http://biglumber.com/x/web?pk=2529DF6AB8F79407E94445B4BC9B906714964AC8

-----BEGIN PGP SIGNATURE-----

iEYEAREDAAYFAktLYtwACgkQvJuQZxSWSsjNJgCg1IZBNIZsUMZ2V83/NjgJnrGO
3NEAoK/w0byO45zmni/i9lnhliD4UpkU
=ndmb
-----END PGP SIGNATURE-----

On Mon, Jan 11, 2010 at 05:42:08PM -0000, Greg Sabino Mullane wrote:

Is there a reason why you're suggesting using DBI? There is also the Pg
perl module which works as well and is one tenth of the size. It also
doesn't have external dependancies. It's just a plain wrapper around
libpq, which for the purposes of testing may be better.

http://search.cpan.org/~mergl/pgsql_perl5-1.9.0/Pg.pm

Works as well? Did you take a look at that link? The last update was
early 2000, which should give you an indication of just how dead
it is.

Dead or not, it still works, even against 8.4. I have many programs
that use it. It's simply a wrapper around the libpq interface and as
long as the libpq interface remains stable (which we go to great pains
to do), so will this module.

Given the talk of importing some perl module into the postgresql tree
it just seemed more logical to me to take something that was close to
libpq and had no external dependancies than taking a module with an
external dependancy (namely DBI).

Have a nice day,
--
Martijn van Oosterhout <kleptog@svana.org> http://svana.org/kleptog/

Hi,

Markus Wanner wrote:

Sorry, if that didn't get clear. I'm trying to put together something I
can release real soon now (tm). I'll keep you informed.

Okay, here we go: dtester version 0.0.

This emerged out of Postgres-R, where I don't just need to test multiple
client connections, but multiple postmasters interacting with each
other. None the less, it may be suitable for other needs as well,
especially testing with concurrent sessions.

I've decided to release this as a separate project named dtester, as
proposed by Michael Tan (thanks for your inspiration).

It's certainly missing lots of things, mainly documentation. However,
I've attached a patch which integrates nicely into the Postgres
Makefiles, so you just need to say: make dcheck.

That very same patch includes a test case with three concurrent
transactions with circular dependencies, where the current SERIALIZABLE
isolation level fails to provide serializability.

Installing dtester itself is as simple as 'python setup.py install' in
the extracted archive's directory.

Go try it, read the code and simply ask, if you get stuck. I'll try to
come up with some more documentation and such...

Regards

Markus Wanner

Markus Wanner <markus@bluegap.ch> wrote:

Okay, here we go: dtester version 0.0.

C'mon, you could have tried to inspire a *bit* more confidence by
calling it version 0.1 or something! ;-)

It's certainly missing lots of things, mainly documentation.
However, I've attached a patch which integrates nicely into the
Postgres Makefiles, so you just need to say: make dcheck.

That sounds very cool.

That very same patch includes a test case with three concurrent
transactions with circular dependencies, where the current
SERIALIZABLE isolation level fails to provide serializability.

Fantastic! I'll expand that a bit....

Installing dtester itself is as simple as 'python setup.py
install' in the extracted archive's directory.

Go try it, read the code and simply ask, if you get stuck. I'll
try to come up with some more documentation and such...

I'm reading through it all now. Expect feedback soon!

And THANK YOU VERY MUCH!

-Kevin

Hi,

Kevin Grittner wrote:

C'mon, you could have tried to inspire a *bit* more confidence by
calling it version 0.1 or something! ;-)

LOL

As David used to say: JFDI

I found that I just needed to ask for python-twisted.

Oh, sorry, yes, requirements: python, twisted.

I must admit that I haven't ever tested on python 2.6 before. I'll try
that (especially as it's the staircase to 3.0, IIUC).

Two more things: the concurrent update test (in the patch part) is
complete, while the second one is just a skeleton, ATM. (Just does a
concurrent COMMIT without actually doing anything).

Second: at the very end of pg_dtester.py, you find the line:
reporter = StreamReporter()

Try a CursesReporter() instead, it gives much nicer output!

Regards

Markus Wanner

Markus Wanner <markus@bluegap.ch> wrote:

I must admit that I haven't ever tested on python 2.6 before. I'll
try that (especially as it's the staircase to 3.0, IIUC).

I don't use python much, so I can't comment on that. I do see that
my system has these two versions on it, with a symlink that makes
2.6 the default.

Python 2.6.2 (release26-maint, Apr 19 2009, 01:56:41)
[GCC 4.3.3] on linux2

Python 3.0.1+ (r301:69556, Apr 15 2009, 15:59:22)
[GCC 4.3.3] on linux2

I haven't quite gotten it to work yet; I'll start over with 3.0 and
see how it goes. I'll also attach the results of the 2.6 attempt.

Try a CursesReporter() instead, it gives much nicer output!

Thanks, I'll try it.

A few other issues in testing so far:

(1) I see that a 'make dcheck' does a 'make install'. That's not
right. For one thing I usually install in a location where I need
to sudo to install; but more importantly, I want to do all checks
*before* I install. It's easy enough to work around that for now,
but I don't think it's acceptable long-term.

(2) After a 'make dcheck' failure, the cluster created for the
testing is left running.

(3) If the install could check dependencies, report problems, and
refuse to install without required packages, that would be less
confusing for python novices (like me).

Perhaps some of these problems will go away with python 3.0, but I
figured I should pass the info along.

Thanks again for this. It should help me a lot.

-Kevin

I wrote:

I'll also attach the results of the 2.6 attempt.

Let's try that again.

-Kevin

Hi,

Quoting "Kevin Grittner" <Kevin.Grittner@wicourts.gov>:

I haven't quite gotten it to work yet; I'll start over with 3.0 and
see how it goes.

Let's stick to 2.x versions, first...

I'll also attach the results of the 2.6 attempt.

Thanks, that looks already pretty promising. ;-)

A few other issues in testing so far:

(1) I see that a 'make dcheck' does a 'make install'. That's not
right. For one thing I usually install in a location where I need
to sudo to install; but more importantly, I want to do all checks
*before* I install. It's easy enough to work around that for now,
but I don't think it's acceptable long-term.

It does: "temp_install: creating temporary installation" means it's
running make install in the background.

(2) After a 'make dcheck' failure, the cluster created for the
testing is left running.

That counts as a bug. I also get that from time to time (and with
Postgres-R testing on 3+ instances, it's even more annoying).

Note that the error just before that is, that a psql process it starts
cannot connect to its postmaster ("startup of test test-conn-0A
failed, skipping.") Please check the log
(src/test/regress/dtester.log) for why that failed in the first place.
Can you connect manually to the database (that's still running after a
make dcheck)?

(3) If the install could check dependencies, report problems, and
refuse to install without required packages, that would be less
confusing for python novices (like me).

I'm not exactly a distutils hacker... Anybody else got any clue here?

Perhaps some of these problems will go away with python 3.0, but I
figured I should pass the info along.

I'd rather suspect that more of them will arise.

Regards

Markus

"Markus Wanner" <markus@bluegap.ch> wrote:

Quoting "Kevin Grittner" <Kevin.Grittner@wicourts.gov>:

I haven't quite gotten it to work yet; I'll start over with 3.0
and see how it goes.

Let's stick to 2.x versions, first...

OK

It does: "temp_install: creating temporary installation" means
it's running make install in the background.

OK, sorry for misreading that.

(2) After a 'make dcheck' failure, the cluster created for the
testing is left running.

That counts as a bug. I also get that from time to time (and with
Postgres-R testing on 3+ instances, it's even more annoying).

Note that the error just before that is, that a psql process it
starts cannot connect to its postmaster ("startup of test
test-conn-0A failed, skipping.") Please check the log
(src/test/regress/dtester.log) for why that failed in the first
place.

Not sure what's relevant there. Entire file tarball attached.

Can you connect manually to the database (that's still running
after a make dcheck)?

Yes I can. Any queries you'd like me to run in there?

-Kevin

Markus Wanner <markus@bluegap.ch> wrote:

Second: at the very end of pg_dtester.py, you find the line:
reporter = StreamReporter()

Try a CursesReporter() instead, it gives much nicer output!

When I try to do that, Kate complains (I'm even copying their typo):

You are trying to save a python file as non ASCII, without
specifiying a correct source encoding line for encoding "utf-8"

It offers these options:

Insert: # -*- coding: utf-8 -*-
Save Nevertheless
Cancel

Should that coding line be in there?

-Kevin

-----BEGIN PGP SIGNED MESSAGE-----
Hash: RIPEMD160

Dead or not, it still works, even against 8.4. I have many programs
that use it. It's simply a wrapper around the libpq interface and as
long as the libpq interface remains stable (which we go to great pains
to do), so will this module.

Well, I stand corrected. Good to know.

Given the talk of importing some perl module into the postgresql tree
it just seemed more logical to me to take something that was close to
libpq and had no external dependancies than taking a module with an
external dependancy (namely DBI).

Yes, I could see that. Actually, I just came across another one
by Hiroyuki OYAMA and Aaron Crane. This was last updated January 10, 2010! :

http://search.cpan.org/~arc/DBD-PgPP-0.08/

Still requires DBI of course, but no Perl library or compiling required
as DBD::Pg does. So we've not got three valid options. :)

- --
Greg Sabino Mullane greg@turnstep.com
PGP Key: 0x14964AC8 201001151129
http://biglumber.com/x/web?pk=2529DF6AB8F79407E94445B4BC9B906714964AC8
-----BEGIN PGP SIGNATURE-----

iEYEAREDAAYFAktQmI0ACgkQvJuQZxSWSsgNugCgjwkT9QwGpvhcIXCNYhRcTwSW
JZcAnjvrsjwpO/QvJ1LzU+cUZ4UqajxV
=bu4q
-----END PGP SIGNATURE-----

Hi,

Kevin Grittner wrote:

Not sure what's relevant there. Entire file tarball attached.

Due to reasons mentioned in this thread as well, I've decided to use
psql to connect to the database. dtester is parsing its output and
checks that against expectations. Hawever, that has its own pitfalls, so
in the end I'm almost about to change back to using libpq or
implementing the bare minimum protocol (that might have its own merits
within the twisted world, if implemented in the required async fashion).

Strangely, your log has escape codes in it, which I'm assuming makes the
parsing choke. Is that something special to your installation? My psql
never colors its outputs...

However, the quickest way forward probably is to filter out escape
sequences. Turning off tty is not really an option, because dtester
doesn't have a chance to capture all necessary events, in that mode.

Yes I can. Any queries you'd like me to run in there?

It looks like psql can connect, too. It's just the parsing of outputs
which fails.

Regards

Markus

Hi,

Kevin Grittner wrote:

You are trying to save a python file as non ASCII, without
specifiying a correct source encoding line for encoding "utf-8"

I wasn't aware I had non-ascii characters in there. Inserting an
encoding line seems fine. I'll fix that for the upcoming version 0.1.

Regards

Markus

Markus Wanner <markus@bluegap.ch> wrote:

I wasn't aware I had non-ascii characters in there. Inserting an
encoding line seems fine. I'll fix that for the upcoming version
0.1.

Yeah, I couldn't find any, either. I just tried creating a minimal
python file in Kate, and it gave me that even though I *know* it was
all ASCII characters right off my keyboard. I guess Kate is being
overly picky. On the other hand, if it silences an annoying message
sometimes, maybe that's reason enough to add it.

-Kevin

Markus Wanner <markus@bluegap.ch> wrote:

Strangely, your log has escape codes in it, which I'm assuming
makes the parsing choke. Is that something special to your
installation? My psql never colors its outputs...

I haven't configured anything like that intentionally. I don't
*see* any colors when I use psql. Can you think of anywhere I
should check something which might be causing this?

-Kevin

Hi,

Kevin Grittner wrote:

I haven't configured anything like that intentionally. I don't
*see* any colors when I use psql. Can you think of anywhere I
should check something which might be causing this?

No idea ATM.

However, just to make sure that has absolutely nothing to do with the
curses reporter I've written: is that dtester.log you just sent the log
from a run with the StreamReporter or the CursesReporter? (Should not
have any influence for the log, but you never know).

Please recheck with the StreamReporter and try to grep the lines
starting with "[psql0]", "[psql1]" and "[psql2]". Dtester simply logs
all and any output of all 3rd party processes started.

Alternatively, you may want to filter out all lines that start with
"[postmaster0]", that might already reduce what we can consider noise in
this case.

Regards

Markus Wanner

Markus Wanner <markus@bluegap.ch> wrote:

Strangely, your log has escape codes in it, which I'm assuming
makes the parsing choke. Is that something special to your
installation?

My pager is "less"; could that cause it? Could the twisted
environment look like one where the pager should kick in?

-Kevin

Hi,

Kevin Grittner wrote:

My pager is "less"; could that cause it? Could the twisted
environment look like one where the pager should kick in?

Yes, that could be it. At least it fails here, too, if I set PAGER=less.
Try:

PAGER=more make dcheck

So, the solution probably lies in adjusting the environment, before
starting psql. (Maybe even dropping all existing environment variables
for better control of the situation). Will add that for dtester 0.1.

(Also note that I plan to move most of what's currently in the patch to
the dtester package itself. However, that requires it to be (even more)
generic.)

Thank you for testing the tester ;-)

Regards

Markus

Markus Wanner wrote:

Hi,

Kevin Grittner wrote:

My pager is "less"; could that cause it? Could the twisted
environment look like one where the pager should kick in?

Yes, that could be it. At least it fails here, too, if I set
PAGER=less. Try:

PAGER=more make dcheck

Surely for automated use you want the psql pager off altogether. "psql
--pset pager=off" or some such invocation should do it.

cheers

andrew

Markus Wanner <markus@bluegap.ch> wrote:

So, the solution probably lies in adjusting the environment,
before starting psql. (Maybe even dropping all existing
environment variables for better control of the situation). Will
add that for dtester 0.1.

Based on Andrew's suggestion, I changed line 276 to:

args=['psql', '-A', '--pset=pager=off',

I now get 5 of 6 tests succeeded (83.3%), processed in 18.5 seconds.

I'm not clear on what you want to see from the run or whether it
might be better sent off-list.

Also, in looking closer at how you have the tests defined, it
doesn't look to me like you're carefully interleaving specific
sequences of statements on specific connections so much as opening
multiple connections and then for each statement saying "run this on
all connections." That's certainly a valid test to include, but I
need the more controlled format, too. It does appear that that's
pretty straightforward to code; you just haven't chosen to do so in
the particular tests here, correct?

-Kevin

"Kevin Grittner" <Kevin.Grittner@wicourts.gov> wrote:

Also, in looking closer at how you have the tests defined, it
doesn't look to me like you're carefully interleaving specific
sequences of statements on specific connections so much as opening
multiple connections and then for each statement saying "run this
on all connections."

I take it back; you've got both.

I do want to expand the tests quite a bit -- do I work them all into
this same file, or how would I proceed? I think I'll need about 20
more tests, but I don't want to get in the way of your work on the
framework which runs them.

-Kevin

Markus Wanner <markus@bluegap.ch> wrote:

Please recheck with the StreamReporter and try to grep the lines
starting with "[psql0]", "[psql1]" and "[psql2]". Dtester simply
logs all and any output of all 3rd party processes started.

For me, all psql output seems to be [psql0]; no [psql1] or [psql2].

Bug?

-Kevin

Markus Wanner <markus@bluegap.ch> wrote:

Go try it, read the code and simply ask, if you get stuck. I'll
try to come up with some more documentation and such...

I'm a little unclear about the differences between "uses",
"depends", and "onlyAfter". Here's what they *sound* like they
mean, to me; although I don't think the code isn't entirely
consistent with this interpretation.

"uses" means that the referenced task has complimentary setUp and
tearDown methods, and the dependent task may only run after a
successful invocation of the referenced task's setUp method, and the
referenced task will wait for completion of all dependent tasks
before invoking tearDown.

"depends" means that the tearDown method of the referenced task
doesn't undo the work of its setUp, at least for purposes of the
dependent task. The dependent task can only start after successful
completion of the referenced class's work (*just* setUp, or all the
way to tearDown?), but the referenced task doesn't need to wait for
the dependent task.

"onlyAfter" means that the dependent task must wait for completion
of the referenced task, but doesn't care whether or not the
referenced class completed successfully.

How close am I?

-Kevin

Hi,

Kevin Grittner wrote:

I'm a little unclear about the differences between "uses",
"depends", and "onlyAfter". Here's what they *sound* like they
mean, to me; although I don't think the code isn't entirely
consistent with this interpretation.

Wow, you are way ahead of me. I intended to write some documentation
about that, but...

I differentiate tests and test suites. Tests mainly have a run method,
while test suites have setUp and tearDown ones.

"uses" means that the referenced task has complimentary setUp and
tearDown methods, and the dependent task may only run after a
successful invocation of the referenced task's setUp method, and the
referenced task will wait for completion of all dependent tasks
before invoking tearDown.

Absolutely correct (may I just copy that para for documentation)? ;-)

Two additional things: tests and test suites may have requirements (in
the form of interfaces). The used test suites are passed to the
dependent task and it may call the referenced tasks's methods, for
example to get the database directory or to run a certain SQL command.

Second, if the referenced task fails, any running dependent task is
getting aborted as well. That might be obvious, though.

"depends" means that the tearDown method of the referenced task
doesn't undo the work of its setUp, at least for purposes of the
dependent task. The dependent task can only start after successful
completion of the referenced class's work (*just* setUp, or all the
way to tearDown?), but the referenced task doesn't need to wait for
the dependent task.

Hm.. no, not quite. The fact that not all suites clean up after them has
nothing to do with how they are referenced ("uses" or "depends"). So
far, it's entirely up to the test suite. I dislike that, but it works.
(I've been thinking about some separate resource allocation handling and
what not, but..)

The only difference between "depends" and "uses" is the requirements
fulfilling. "uses" does that, while "depends" only adds the timing and
functional dependencies, but doesn't pass the referenced task as an
argument to the dependent task.

"onlyAfter" means that the dependent task must wait for completion
of the referenced task, but doesn't care whether or not the
referenced class completed successfully.

That's how I think it *should* be. ATM "onlyAfter" requires successful
completion of the dependent task.

I'd like to change that to support "onlyAfter", "onlyAfterSuccessOf" and
"onlyAfterFailureOf". Plus "onlyBefore" for convenience.

This is all work in progress and I'm open to suggestions and requests.

Thank you for thinking through all of this. I'm sure you understand now,
why it's not a version 0.1, yet :-)

Regards

Markus

Hi,

Kevin Grittner wrote:

Based on Andrew's suggestion, I changed line 276 to:

args=['psql', '-A', '--pset=pager=off',

That looks like a correct fix for psql, yes. Thanks for pointing that
out Andrew.

Other processes might be confused by (or at least act differently with)
a PAGER env variable, so that still needs to be cleared in general.

I now get 5 of 6 tests succeeded (83.3%), processed in 18.5 seconds.

That's perfect. The one test that fails is expected to fail (another
thing dtester doesn't support, yet). The serialization code you write
should finally make that test pass ;-)

I do want to expand the tests quite a bit -- do I work them all into
this same file, or how would I proceed? I think I'll need about 20
more tests, but I don't want to get in the way of your work on the
framework which runs them.

Well, first of all, another piece of the missing manual: there are
BaseTest and SyncTest classes. Those based on BaseTest runs within the
event loop of the twisted framework, thus need to be written in the very
same asynchronous fashion. Mostly calling async methods that return a
Deferred object, on which you may addCallback() or addErrback(). See the
fine twisted documentation, especially the part about "Low-Level
Networking and Event Loop" here:

http://twistedmatrix.com/documents/current/core/howto/index.html

The SyncTest is based on BaseTest, but a new thread is created to run
its run method, passing back its results to the main event loop when
done. That allows you to call blocking methods without having to care
about blocking the entire event loop.

However, it makes interacting between the two models a bit complicated.
To call an async function from a SyncTest, you need to call the syncCall
method. The separate thread then waits for some callback in the main
event loop.

Both have their own set of caveats, IMO.

I'm not sure about how to organize the tests and ongoing development of
the framework. I've already broken the Postgres-R tests with dtester-0.0.

Maybe we put up a git branch with the dtester patches included? So
whenever I want to change the framework, I can check if and how it
affects your tests.

Regards

Markus

Markus Wanner wrote:

I do want to expand the tests quite a bit -- do I work them all into
this same file, or how would I proceed? I think I'll need about 20
more tests, but I don't want to get in the way of your work on the
framework which runs them.

Well, first of all, another piece of the missing manual: there are
BaseTest and SyncTest classes. Those based on BaseTest runs within the
event loop of the twisted framework, thus need to be written in the very
same asynchronous fashion. Mostly calling async methods that return a
Deferred object, on which you may addCallback() or addErrback(). See the
fine twisted documentation, especially the part about "Low-Level
Networking and Event Loop" here:

http://twistedmatrix.com/documents/current/core/howto/index.html

I'm not sure about how to organize the tests and ongoing development of
the framework. I've already broken the Postgres-R tests with dtester-0.0.

Hi,

sorry to butt in to the conversation, but I have spent some time
wrapping/refining the concepts in dtester, and the results are here:

http://git.wulczer.org/?p=twisted-psql.git;a=summary

It reqires Twisted and has been tested on Python 2.5 (should work on
2.6, no idea about 3.0). The program you use to run it - trial - should
come with your distro's Twisted packages. The tests don't start a server
or anything, so you need to have a PG instance running. To try it:

git clone git://wulczer.org/twisted-psql.git
cd twisted-psql # this is important, or Python won't find the modules
$EDITOR config.py # set the path to psql and connection details for PG
trial test.test_serialization_error
trial test.test_true_serialization

Both tests should pass, the latter being marked as an expectedFailure.
You can then look at test/*.py to see my (puny) attempt at having some
abstraction layer over the asynchronocity of the tests.

I borrowed the idea of wrapping a psql in a Twisted protocol and added a
Deferred interface around it, which made it possible to run tests with
trial: the Twisted unit testing framework.

As a developer of a failry large Python system based on Twisted, that
sports hundreds of trial-based tests, I very strongly recommend trial
for asynchronous unit testing. It handles lots of boring details, is
well maintained and Twisted itself is simply designed to do asynchronous
programming. As an added bonus, the runnning and reporting
infrastructure is already there, you just write the tests.

My code is very rough and lacks good error reporting, for instance
failed tests will probably result in a "test hung" and the need to
Ctrl+C, but that can be easily improved. A thing that would help
tremendously would be a real Twisted protocol that talks to PG on the
protocol level, not by parsing psql output (which is very clumsy and
error prone IMHO).

I found one such project:
http://www.jamwt.com/pgasync/
but it had some issues with committing (all my test programs were
exiting before PG got the final COMMIT, which resulted in the
impossibility to do anything) and it does too much things that Python PG
drivers like to do (like declaring a CURSOR for each query, bleah). A
good implementation would hugely improve the quality and robustness of
any such testsuite.

Cheers,
Jan
--
Jan Urbanski
GPG key ID: E583D7D2

ouden estin

Markus Wanner wrote:
Kevin Grittner wrote:

I differentiate tests and test suites. Tests mainly have a run
method, while test suites have setUp and tearDown ones.

I hadn't caught on to that distinction yet. That should help.

"uses" means that the referenced task has complimentary setUp and
tearDown methods, and the dependent task may only run after a
successful invocation of the referenced task's setUp method, and
the referenced task will wait for completion of all dependent
tasks before invoking tearDown.

Absolutely correct (may I just copy that para for documentation)?
;-)

Use any of my language that you like, but I'm not generally known for
my word-smithing ability, so use at your own risk. ;-)

Two additional things: tests and test suites may have requirements
(in the form of interfaces). The used test suites are passed to the
dependent task and it may call the referenced tasks's methods, for
example to get the database directory or to run a certain SQL
command.

Makes sense.

Second, if the referenced task fails, any running dependent task is
getting aborted as well. That might be obvious, though.

I figured that, although it's good to have it confirmed.

"depends" means that the tearDown method of the referenced task
doesn't undo the work of its setUp, at least for purposes of the
dependent task. The dependent task can only start after successful
completion of the referenced class's work (*just* setUp, or all
the way to tearDown?), but the referenced task doesn't need to
wait for the dependent task.

Hm.. no, not quite. The fact that not all suites clean up after
them has nothing to do with how they are referenced ("uses" or
"depends"). So far, it's entirely up to the test suite. I dislike
that, but it works. (I've been thinking about some separate
resource allocation handling and what not, but..)

The only difference between "depends" and "uses" is the
requirements fulfilling. "uses" does that, while "depends" only
adds the timing and functional dependencies, but doesn't pass the
referenced task as an argument to the dependent task.

OK, that accounts for most of the differences between what they
sounded like to me and what I saw in the code. That's workable, now
that I understand it.

"onlyAfter" means that the dependent task must wait for completion
of the referenced task, but doesn't care whether or not the
referenced class completed successfully.

That's how I think it *should* be. ATM "onlyAfter" requires
successful completion of the dependent task.

That accounts for the rest of the differences.

I'd like to change that to support "onlyAfter",
"onlyAfterSuccessOf" and "onlyAfterFailureOf". Plus "onlyBefore"
for convenience.

onlyAfterSuccessOf would be the same as depends with an empty
tearDown method? So it would effectively be syntactic sugar, for
convenience?

An onlyBefore reference from a to b would be semantically identical
to an onlyAfter reference from b to a? If so, that one seems to me
like it would muddy the waters more than it would help.

Thank you for thinking through all of this. I'm sure you understand
now, why it's not a version 0.1, yet :-)

Thank you for putting it together! I was afraid I was going to have
to go off-task on serializable implementation to write something so I
could test it. I'm more than happy to help stabilize your tool
instead!

-Kevin

Markus Wanner wrote:
Kevin Grittner wrote:

args=['psql', '-A', '--pset=pager=off',

That looks like a correct fix for psql, yes.

Other processes might be confused by (or at least act differently
with) a PAGER env variable, so that still needs to be cleared in
general.

I see your point. Even with a general solution, probably best to
leave the pset there for psql, though.

[discussion of BaseTest vs SyncTest]

Both have their own set of caveats, IMO.

I'll look those over. Any caveats beyond what you already mentioned
of which I should be particularly careful?

Maybe we put up a git branch with the dtester patches included? So
whenever I want to change the framework, I can check if and how it
affects your tests.

I strongly encourage you to set that up on git.postgresql.org. If
for some reason that's not practicable, I can give you write
permissions to my repository there and set up a dtester branch for
this. I've barely scratched the surface on git in the last few
weeks, and already I'm a big fan. I was never convinced that
subversion was an improvement over cvs -- they each had advantages
over the other which seemed a wash for me -- but git takes everything
to a whole new level.

-Kevin

Hi,

Quoting "Kevin Grittner" <Kevin.Grittner@wicourts.gov>:

I strongly encourage you to set that up on git.postgresql.org.

I'm about to provide git repositories for Postgres-R anyway, so I've
setup two projects on git.postgres-r.org:

dtester: that's the driver/harness code
postgres-dtests: a Postgres clone with the dtester patch applied - this
is based on the Postgres git repository, so you can easily switch
between Postgres branches.

I'd like to clean postgres-dtests in the sense that all tests included
there are expected to succeed on Postgres HEAD. Those (like the
initial SSI ones) that are expected to fail should get marked as such
in there.

If you want to add SSI specific tests, which your are expecting to
succeed on your branch, I'd recommend you create your own branch
(or merge into your branch from postgres-dtests). Git makes that simple
enough.

I've barely scratched the surface on git in the last few
weeks, and already I'm a big fan. I was never convinced that
subversion was an improvement over cvs -- they each had advantages
over the other which seemed a wash for me -- but git takes everything
to a whole new level.

I agree, and would like to extend that to DVCSes in general. Having
started with monotone, I'm used to a different level of convenience,
especially WRT network exchange and merging. To be fair, I'm excited
about how fast git is (especially WRT network exchange, where monotone
just plain sucks).

I see your point. Even with a general solution, probably best to
leave the pset there for psql, though.

Agreed, that's fixed in the new git repository.

I'll look those over. Any caveats beyond what you already mentioned
of which I should be particularly careful?

Uh.. no, there's no difference other than that. It's a paradigm
question. Ones like it that way, others the other. And then there are
applications that are a better fit than others...

Now, I tend towards the event based approach, because it basically
relieves you from having to deal with concurrent threads and all their
issues. You need to get a single ordering of events anyway, if you want
to check ordering constraints.

Regards

Markus

Hi,

Jan Urbański wrote:

sorry to butt in to the conversation, but I have spent some time
wrapping/refining the concepts in dtester, and the results are here:

http://git.wulczer.org/?p=twisted-psql.git;a=summary

That seems to cover the concurrent psql part of dtester. But I don't see
how that's wrapping or refining dtester.

I borrowed the idea of wrapping a psql in a Twisted protocol and added a
Deferred interface around it, which made it possible to run tests with
trial: the Twisted unit testing framework.

dtester works pretty much the same way, except that it doesn't use
trial. But the Deferred interface is about the same.

As a developer of a failry large Python system based on Twisted, that
sports hundreds of trial-based tests, I very strongly recommend trial
for asynchronous unit testing. It handles lots of boring details, is
well maintained and Twisted itself is simply designed to do asynchronous
programming. As an added bonus, the runnning and reporting
infrastructure is already there, you just write the tests.

I'm using trial for two other projects as well and I've started with it
for Postgres. IMO trial is very good for unit tests, but fails horribly
for anything that involves complex setups and watch-guarding of multiple
processes. That's where dtester shines.

My code is very rough and lacks good error reporting, for instance
failed tests will probably result in a "test hung" and the need to
Ctrl+C, but that can be easily improved.

Don't underestimate that. There are lots of sources of errors. Not
having a sentinel over the postmaster itself, unit tests from trial
simply fail to notice errors there. (And coding for Postgres, these are
the ones you are interested in). Now combine all of that and error
handling and (useful) reporting is *the* major challenge in automating
testing.

A thing that would help
tremendously would be a real Twisted protocol that talks to PG on the
protocol level, not by parsing psql output (which is very clumsy and
error prone IMHO).

I agree. (Well, now I do).

I found one such project:
http://www.jamwt.com/pgasync/
but it had some issues with committing

Yeah, I didn't like that one, either.

Regards

Markus

[off-list to avoid distracting from the 9.0 wrap-up effort]

Markus Wanner <markus@bluegap.ch> wrote:

Quoting "Kevin Grittner" <Kevin.Grittner@wicourts.gov>:

I strongly encourage you to set that up on git.postgresql.org.

I'm about to provide git repositories for Postgres-R anyway, so
I've setup two projects on git.postgres-r.org:

dtester: that's the driver/harness code
postgres-dtests: a Postgres clone with the dtester patch applied -
this is based on the Postgres git repository, so you can easily
switch between Postgres branches.

I just got to the point of having what appears to be a working but
poorly optimized version of serializable transactions, so it is
critical that I create a good set of tests to confirm correct
behavior and monitor for regressions as I optimize. I see that
you've been working on dtester recently -- should I grab what you've
got here, stick with 0.1, or do you want to package something? If I
should pull from your git, any hints on the best git statements to
merge that in are welcome, I'm still rather new to git, and I tend
not to get things right on a first try without some hints. :-/

Thanks again for this tool!

-Kevin

I wrote:

[off-list to avoid distracting from the 9.0 wrap-up effort]

Arg. I really didn't mean that to go to the list. :-(

-Kevin

Hi,

Kevin Grittner wrote:

I just got to the point of having what appears to be a working but
poorly optimized version of serializable transactions,

Cool.

so it is
critical that I create a good set of tests to confirm correct
behavior and monitor for regressions as I optimize. I see that
you've been working on dtester recently -- should I grab what you've
got here, stick with 0.1, or do you want to package something?

I'm preparing for a 0.1 release, yes. However, I'm trying to keep up
with the postgres-dtests branch, so that should continue to work with
the latest dtester code.

If I
should pull from your git, any hints on the best git statements to
merge that in are welcome, I'm still rather new to git, and I tend
not to get things right on a first try without some hints. :-/

I'd recommend merging postgres-dtests into your branch, so you'll get
all of the adjustments from there, if I change dtester itself.

Once 0.1 is out, I'm *trying* to remain backwards compatible in future
releases. However, that holds me from releasing 0.1 in the first place :-)

Independent of versions and releases: if you build your tests on top of
dtester, I'm more than willing to support you with that. If you keep
your code in a git repository, I could even provide patches, in case I
need (or want) to change the dtester interface.

Regards

Markus Wanner