Does derby have an embedded Mode like Derby ?

On Tue, Aug 04, 2009 at 01:37:34PM +0100, Paul Taylor wrote:

Does postgres have an embedded mode to allow a database to be embedded
with Java application without requiring seperate db manager, like the
derby database does ?

No, and I think the consensus is that this would be bad. See:

http://wiki.postgresql.org/wiki/FAQ#Can_PostgreSQL_be_embedded.3F

--
Sam http://samason.me.uk/

Sam Mason wrote:

On Tue, Aug 04, 2009 at 01:37:34PM +0100, Paul Taylor wrote:

Does postgres have an embedded mode to allow a database to be embedded
with Java application without requiring seperate db manager, like the
derby database does ?

No, and I think the consensus is that this would be bad. See:

http://wiki.postgresql.org/wiki/FAQ#Can_PostgreSQL_be_embedded.3F

Thats a shame, I wanted to write junit test for a java program that
queried a database, requiring a full database to be available for unit
tests is not really an environment I want to have.

Paul

In response to Paul Taylor <ijabz@fastmail.fm>:

Sam Mason wrote:

On Tue, Aug 04, 2009 at 01:37:34PM +0100, Paul Taylor wrote:

Does postgres have an embedded mode to allow a database to be embedded
with Java application without requiring seperate db manager, like the
derby database does ?

No, and I think the consensus is that this would be bad. See:

http://wiki.postgresql.org/wiki/FAQ#Can_PostgreSQL_be_embedded.3F

Thats a shame, I wanted to write junit test for a java program that
queried a database, requiring a full database to be available for unit
tests is not really an environment I want to have.

Has it occurred to you that testing a DB client when there's no
DB isn't really a very accurate or realistic test?

--
Bill Moran
http://www.potentialtech.com
http://people.collaborativefusion.com/~wmoran/

Bill Moran wrote:

In response to Paul Taylor <ijabz@fastmail.fm>:

Sam Mason wrote:

On Tue, Aug 04, 2009 at 01:37:34PM +0100, Paul Taylor wrote:

Does postgres have an embedded mode to allow a database to be embedded
with Java application without requiring seperate db manager, like the
derby database does ?

No, and I think the consensus is that this would be bad. See:

http://wiki.postgresql.org/wiki/FAQ#Can_PostgreSQL_be_embedded.3F

Thats a shame, I wanted to write junit test for a java program that
queried a database, requiring a full database to be available for unit
tests is not really an environment I want to have.

Has it occurred to you that testing a DB client when there's no
DB isn't really a very accurate or realistic test?

I am testing the code that extracts information from a read only
database. These are UNIT tests so only interested in getting the right
results given a particular set of data, anything else is a distraction.

Paul

Paul Taylor, 04.08.2009 15:48:

Thats a shame, I wanted to write junit test for a java program that
queried a database, requiring a full database to be available for unit
tests is not really an environment I want to have.

Well if you want to test the database, you will need a full database.

But you can easily setup and start a Postgres server through a batch file/shell script.
I have a small batch file that simply uses the binary ZIP distribution to run initdb and create the initial databases. We are using this to bundle Postgres e.g. with a Tomcat web application.

You don't really need to run an installer and/or create registry entries (for windows). This would then resemble more the Derby network server setup.

Thomas

In response to Paul Taylor <ijabz@fastmail.fm>:

Bill Moran wrote:

In response to Paul Taylor <ijabz@fastmail.fm>:

Sam Mason wrote:

On Tue, Aug 04, 2009 at 01:37:34PM +0100, Paul Taylor wrote:

Does postgres have an embedded mode to allow a database to be embedded
with Java application without requiring seperate db manager, like the
derby database does ?

No, and I think the consensus is that this would be bad. See:

http://wiki.postgresql.org/wiki/FAQ#Can_PostgreSQL_be_embedded.3F

Thats a shame, I wanted to write junit test for a java program that
queried a database, requiring a full database to be available for unit
tests is not really an environment I want to have.

Has it occurred to you that testing a DB client when there's no
DB isn't really a very accurate or realistic test?

I am testing the code that extracts information from a read only
database. These are UNIT tests so only interested in getting the right
results given a particular set of data, anything else is a distraction.

Then replace the DB client class with a class that returns fabricated
data for the purpose of your test.

--
Bill Moran
http://www.potentialtech.com
http://people.collaborativefusion.com/~wmoran/

Paul Taylor, 04.08.2009 15:48:

Thats a shame, I wanted to write junit test for a java program that
queried a database, requiring a full database to be available for unit
tests is not really an environment I want to have.

Well if you want to test the database, you will need a full database.

But you can easily setup and start a Postgres server through a batch
file/shell script. I have a small batch file that simply uses the binary
ZIP distribution to run initdb and create the initial databases. We are
using this to bundle Postgres e.g. with a Tomcat web application.
You don't really need to run an installer and/or create registry entries
(for windows). This would then resemble more the Derby network server
setup.

Thomas

Of course you can always use Derby for testing the Postgres later. I have
found HSQLDB to be closer to Postgres than Derby is. I do seem to recall the
Netbeans group recommends using sever mode during development then switch to
imbedded for distribution. I have done it both ways with equal success and
similar efforts.

Bayless

Bayless Kirtley wrote:

Paul Taylor, 04.08.2009 15:48:

Thats a shame, I wanted to write junit test for a java program that
queried a database, requiring a full database to be available for
unit tests is not really an environment I want to have.

Well if you want to test the database, you will need a full database.

But you can easily setup and start a Postgres server through a batch
file/shell script. I have a small batch file that simply uses the
binary ZIP distribution to run initdb and create the initial
databases. We are using this to bundle Postgres e.g. with a Tomcat
web application.
You don't really need to run an installer and/or create registry
entries (for windows). This would then resemble more the Derby
network server setup.

Thomas

Of course you can always use Derby for testing the Postgres later. I
have found HSQLDB to be closer to Postgres than Derby is. I do seem to
recall the Netbeans group recommends using sever mode during
development then switch to imbedded for distribution. I have done it
both ways with equal success and similar efforts.

Bayless

Thanks I am trying to use Derby but fallen foul of Postgres array
datatype which Derby doesn't support, Ive just had a quick look at
HSQLDB and it doesn't appear to support it either.

Paul

Thomas Kellerer wrote:

Paul Taylor, 04.08.2009 15:48:

Thats a shame, I wanted to write junit test for a java program that
queried a database, requiring a full database to be available for
unit tests is not really an environment I want to have.

Well if you want to test the database, you will need a full database.

It was a simple question, does Postgres have an embedded mode (which is
still a full database) but unfortunately it does not.

But you can easily setup and start a Postgres server through a batch
file/shell script. I have a small batch file that simply uses the
binary ZIP distribution to run initdb and create the initial
databases. We are using this to bundle Postgres e.g. with a Tomcat web
application.
You don't really need to run an installer and/or create registry
entries (for windows). This would then resemble more the Derby network
server setup.

Yeah, but this is messy and low because you have to wait the database to
be fully initilized before running the test. And for unit tests you
should be start the database for each test, but wouldnt be possible with
your method it would be to slow. Furthermore this is a Windows only
solution, there would be additional work to do to setup for Mac or Linux

thanks anyway

Paul

Bill Moran wrote:

In response to Paul Taylor <ijabz@fastmail.fm>:

Bill Moran wrote:

In response to Paul Taylor <ijabz@fastmail.fm>:

Sam Mason wrote:

On Tue, Aug 04, 2009 at 01:37:34PM +0100, Paul Taylor wrote:

Does postgres have an embedded mode to allow a database to be embedded
with Java application without requiring seperate db manager, like the
derby database does ?

No, and I think the consensus is that this would be bad. See:

http://wiki.postgresql.org/wiki/FAQ#Can_PostgreSQL_be_embedded.3F

Thats a shame, I wanted to write junit test for a java program that
queried a database, requiring a full database to be available for unit
tests is not really an environment I want to have.

Has it occurred to you that testing a DB client when there's no
DB isn't really a very accurate or realistic test?

I am testing the code that extracts information from a read only
database. These are UNIT tests so only interested in getting the right
results given a particular set of data, anything else is a distraction.

Then replace the DB client class with a class that returns fabricated
data for the purpose of your test.

Won't work because I am writing SQL and I want to test the SQL is correct

Paul Taylor wrote on 04.08.2009 17:04:

Well if you want to test the database, you will need a full database.

It was a simple question, does Postgres have an embedded mode (which is
still a full database) but unfortunately it does not.

I'm just trying to help get your test environment into proper shape. If you are not open for other
solutions, then ignore the rest of this email

Yeah, but this is messy and low because you have to wait the database to
be fully initilized before running the test.

Depends on what you mean with "initialize". You only need to run initdb once. After that you can
e.g. use DbUnit to populate the tables needed for testing.

If you do that, you can simply have an empty database cluster directory that you unzip before
running the tests suite.

And for unit tests you
should be start the database for each test, but wouldnt be possible with
your method it would be to slow.

Well if you use something like DbUnit, you can simply leave the postmaster running and re-initialize
the tables for each test (something you would need to do with Derby as well).

Furthermore this is a Windows only
solution, there would be additional work to do to setup for Mac or Linux

No I don't think it's windows only. The batch file simply calls the already existings commandline
programs from the Postgres bin directory. They names are identical across all platforms. The only
thing would be the zipped database cluster as that is platform specific.

Btw: I find the H2 database a lot closer to Postgres than HSQLDB. The author is actively trying to
add compatibility levels for various DBMS. It might be worth looking into that as well.

But if you e.g. have triggers or stored procedures you'll have to use "the real thing".

We have e.g. unit test for one of our application that we ran on MySQL just for the fun of it, and
it failed miserable due to the different locking behaviour between Postgres and MySQL. I would
expect similar problems with heavy transactional tests with any embedded engine as well.

Thomas

On Tue, Aug 4, 2009 at 8:37 AM, Paul Taylor<ijabz@fastmail.fm> wrote:

Bill Moran wrote:

In response to Paul Taylor <ijabz@fastmail.fm>:

Bill Moran wrote:

In response to Paul Taylor <ijabz@fastmail.fm>:

Sam Mason wrote:

On Tue, Aug 04, 2009 at 01:37:34PM +0100, Paul Taylor wrote:

Does postgres have an embedded mode to allow a database to be
embedded with Java application without requiring seperate db manager, like
the derby database does ?

No, and I think the consensus is that this would be bad.  See:

 http://wiki.postgresql.org/wiki/FAQ#Can_PostgreSQL_be_embedded.3F

Thats a shame, I wanted to write junit test for a java program that
queried a database, requiring a full database to be available for unit tests
is not really an environment I want to have.

Has it occurred to you that testing a DB client when there's no
DB isn't really a very accurate or realistic test?

I am testing the code that extracts information from a read only
database. These are UNIT tests so only interested in getting the right
results given a particular set of data, anything else is a distraction.

Then replace the DB client class with a class that returns fabricated
data for the purpose of your test.

Won't work because I am writing SQL and I want to test the SQL is correct

Can you do what you need to in a virtual machine? I use several
virtual machines in my development environment. One for cranking out
code. One as a "clean" test environment.

On Tue, Aug 04, 2009 at 04:04:42PM +0100, Paul Taylor wrote:

You don't really need to run an installer and/or create registry
entries (for windows). This would then resemble more the Derby
network server setup.

Yeah, but this is messy and low because you have to wait the
database to be fully initilized before running the test. And for
unit tests you should be start the database for each test, but
wouldnt be possible with your method it would be to slow.

One could drop and re-create the databases as needed. No
need to re-initdb the cluster or even stop the PostgreSQL
process. That wouldn't be slow.

Karsten
--
GPG key ID E4071346 @ wwwkeys.pgp.net
E167 67FD A291 2BEA 73BD 4537 78B9 A9F9 E407 1346

Thats a shame, I wanted to write junit test for a java program that queried a database, requiring a full database to be available for unit tests is not really an >environment I want to have.

Why not? if you doesn't test the real thing you test is meaningless.
you should test the real database your are going to use with a setup
as similar to the deployment setup you are going to use in production.

In response to Paul Taylor <ijabz@fastmail.fm>:

Bill Moran wrote:

Then replace the DB client class with a class that returns fabricated
data for the purpose of your test.

Won't work because I am writing SQL and I want to test the SQL is correct

Well, be warned that not all alternatives to PostgreSQL will have the
same SQL compliance as Postgres ... so substituting another db backend
is going to be less than reliable.

I hope you don't take these next comments as an attack or anything, but
I think your whole approach to testing is flawed. The fact that tests
are complicated to set up and take a while to run is just life. I mean,
who cares if they take a while to run? Computers don't need to sleep, so
have them run overnight. And any test that's actually comprehensive is
going to take effort to write anyway. Thing is, you'll only be setting
up the PG startup/teardown stuff once, then any test that needs it can
call those functions/scripts/whatever. If you need more computers or a
different OS, then virtualize. All the tools are there for you.

--
Bill Moran
http://www.potentialtech.com
http://people.collaborativefusion.com/~wmoran/

Bill Moran wrote:

In response to Paul Taylor <ijabz@fastmail.fm>:

Bill Moran wrote:

Then replace the DB client class with a class that returns fabricated
data for the purpose of your test.

Won't work because I am writing SQL and I want to test the SQL is correct

Well, be warned that not all alternatives to PostgreSQL will have the
same SQL compliance as Postgres ... so substituting another db backend
is going to be less than reliable.

I hope you don't take these next comments as an attack or anything, but
I think your whole approach to testing is flawed. The fact that tests
are complicated to set up and take a while to run is just life. I mean,
who cares if they take a while to run? Computers don't need to sleep, so
have them run overnight. And any test that's actually comprehensive is
going to take effort to write anyway. Thing is, you'll only be setting
up the PG startup/teardown stuff once, then any test that needs it can
call those functions/scripts/whatever. If you need more computers or a
different OS, then virtualize. All the tools are there for you.

No, I don't take it personally but I think you are missing the point
these are UNIT test not INTEGRATION tests. As such they need to be run
everytime a developer wants to commit some changes to the codebase and
therefore should be quick and unobstrusive to run.

Paul

In response to Paul Taylor <ijabz@fastmail.fm>:

Bill Moran wrote:

In response to Paul Taylor <ijabz@fastmail.fm>:

Bill Moran wrote:

Then replace the DB client class with a class that returns fabricated
data for the purpose of your test.

Won't work because I am writing SQL and I want to test the SQL is correct

Well, be warned that not all alternatives to PostgreSQL will have the
same SQL compliance as Postgres ... so substituting another db backend
is going to be less than reliable.

I hope you don't take these next comments as an attack or anything, but
I think your whole approach to testing is flawed. The fact that tests
are complicated to set up and take a while to run is just life. I mean,
who cares if they take a while to run? Computers don't need to sleep, so
have them run overnight. And any test that's actually comprehensive is
going to take effort to write anyway. Thing is, you'll only be setting
up the PG startup/teardown stuff once, then any test that needs it can
call those functions/scripts/whatever. If you need more computers or a
different OS, then virtualize. All the tools are there for you.

No, I don't take it personally but I think you are missing the point
these are UNIT test not INTEGRATION tests.

I don't feel like I've missed the point, but I wonder if your definition
of "unit" test includes something that I'm not familiar with. We do
extensive unit tests that require a DB backend to work. I mean, if
you're unit testing the DB client class, it needs a back end to talk
to.

As such they need to be run
everytime a developer wants to commit some changes to the codebase and
therefore should be quick and unobstrusive to run.

And I don't see how that precludes having a DB there that the developer
can connect to for testing. We have a DB server specifically for this
purpose so that devs can use it to test out their changes before
checking code in. More specifically, we have one virtual machine that
serves as a dedicated DB server for developer testing.

I mean, if it won't work for you, OK. I'm just offering suggestions, and
honestly am confused as to what your actual holdup is.

--
Bill Moran
http://www.potentialtech.com
http://people.collaborativefusion.com/~wmoran/

Paul Taylor wrote:

I am testing the code that extracts information from a read only
database. These are UNIT tests so only interested in getting the right
results given a particular set of data, anything else is a distraction.

I don't understand your test environment. If all your code is doing is
extracting info from a database, why would you want to have database
management embedded in your test? Simply have a test database already
configured and running, then run your JUnit tests against that existing
database. That simple approach would obviously work across all platforms.

The only reason I can see for doing DDL in a unit test would be if the
end product will be doing such DDL.

--
Guy Rouillier

----- Original Message -----
From: "Paul Taylor" <ijabz@fastmail.fm>
To: <pgsql-general@postgresql.org>
Sent: Tuesday, August 04, 2009 9:56 AM
Subject: Re: [GENERAL] Does derby have an embedded Mode like Derby ?

Of course you can always use Derby for testing the Postgres later. I have
found HSQLDB to be closer to Postgres than Derby is. I do seem to recall
the Netbeans group recommends using sever mode during development then
switch to imbedded for distribution. I have done it both ways with equal
success and similar efforts.

Bayless

Thanks I am trying to use Derby but fallen foul of Postgres array
datatype which Derby doesn't support, Ive just had a quick look at HSQLDB
and it doesn't appear to support it either.

Paul

Yes, I'm afraid you're gonna be out of luck on finding the array type in any
of
the smaller embedded databases. Honestly, the beg project I've been on for a
year or so has used Postgres right through full development and testing.
It's
not hard to start and stop the database whenever I need to although I
usually
just leave it running.

Bayless