Google SoC--Idea Request

-----Original Message-----
From: "Jonah H. Harris"<jonah.harris@gmail.com>
Sent: 15/04/06 20:06:27
To: "Pgsql Hackers"<pgsql-hackers@postgresql.org>
Subject: [HACKERS] Google SoC--Idea Request

As such, we need to quickly put together a list of oh, 15-20 midlevel
project ideas.

There's a couple of listen/notify todos iirc that would be nice to get done - one to allow a message to be sent with the notify, and one to move from a table based design to shared mem/disk.

Regards, Dave

-----Unmodified Original Message-----
Hey everyone,

I know we started a discussion a month or so ago regarding ideas for
SoC projects. However, after reading through the thread, I didn't see
us nail down any actual items.

As such, we need to quickly put together a list of oh, 15-20 midlevel
project ideas. I'm sure we can pull some off the TODO list, but we
should also look at project ideas for porting some of the most used
third-party OSS software to PostgreSQL too (portals, CMS systems,
accounting systems, etc.).

All ideas welcome!

--
Jonah H. Harris, Database Internals Architect
EnterpriseDB Corporation
732.331.1324

---------------------------(end of broadcast)---------------------------
TIP 2: Don't 'kill -9' the postmaster

On Sat, 2006-04-15 at 21:24 +0100, Dave Page wrote:

one to allow a message to be sent with the notify, and one to move
from a table based design to shared mem/disk.

Doing the latter is a precondition for implementing the former in a
reasonable way, I believe.

BTW, these two web log entries summarizing Mono and Mozilla's
experiences with SoC might make interesting reading:

http://weblogs.mozillazine.org/gerv/archives/2006/03/summer_of_code_six_months_on.html
http://tirania.org/blog/archive/2006/Apr-13.html

we should also look at project ideas for porting some of the most used
third-party OSS software to PostgreSQL too (portals, CMS systems,
accounting systems, etc.).

Given the above, I would be wary of such projects bit-rotting. If the
upstream project hasn't bothered to add PostgreSQL support, there might
be a good reason why: writing truly database-agnostic applications is
not always easy (or even desirable).

-Neil

On 4/15/06, Neil Conway <neilc@samurai.com> wrote:

Doing the latter is a precondition for implementing the former in a
reasonable way, I believe.

BTW, these two web log entries summarizing Mono and Mozilla's
experiences with SoC might make interesting reading:

Thanks for the reading material. I don't think our project is exactly
the same, but it's good information to keep in mind.

Given the above, I would be wary of such projects bit-rotting. If the
upstream project hasn't bothered to add PostgreSQL support, there might
be a good reason why: writing truly database-agnostic applications is
not always easy (or even desirable).

This isn't always the case. In a lot of cases, the developers just
wanted to take the easy route and used MySQL... they have a lot of
people asking for PostgreSQL support but they don't have the expertise
to add it themselves.

--
Jonah H. Harris, Database Internals Architect
EnterpriseDB Corporation
732.331.1324

On Saturday 15 April 2006 19:25, Jonah H. Harris wrote:

On 4/15/06, Neil Conway <neilc@samurai.com> wrote:

Doing the latter is a precondition for implementing the former in a
reasonable way, I believe.

BTW, these two web log entries summarizing Mono and Mozilla's
experiences with SoC might make interesting reading:

Thanks for the reading material. I don't think our project is exactly
the same, but it's good information to keep in mind.

Agreed. I sent some ideas to Josh, was thinking he might be posting a list
soon. I kept it aimed at a few ideas I have had/seen that need an initial
push to get going but beyond that could be (and likely would be) community
maintained. Example? Extendning the build farm code to test external pl
langs or database drivers or patches other modules. We've talked about it,
and if someone had the time to make the push, I believe this would be
community maintained going forward.

Given the above, I would be wary of such projects bit-rotting. If the
upstream project hasn't bothered to add PostgreSQL support, there might
be a good reason why: writing truly database-agnostic applications is
not always easy (or even desirable).

This isn't always the case. In a lot of cases, the developers just
wanted to take the easy route and used MySQL... they have a lot of
people asking for PostgreSQL support but they don't have the expertise
to add it themselves.

I think more importantly is that the time needed to do an initial port is
often much greater than it is to maintain a port.

--
Robert Treat
Build A Brighter Lamp :: Linux Apache {middleware} PostgreSQL

-----Original Message-----
From: "Jonah H. Harris"<jonah.harris@gmail.com>
Sent: 15/04/06 20:06:27
To: "Pgsql Hackers"<pgsql-hackers@postgresql.org>
Subject: [HACKERS] Google SoC--Idea Request

As such, we need to quickly put together a list of oh, 15-20 midlevel
project ideas.

Another thought - a nice C++ project, requiring minimal previous knowledge of existing code would be to add a query builder to pgAdmin.

Regards, Dave

-----Unmodified Original Message-----
Hey everyone,

I know we started a discussion a month or so ago regarding ideas for
SoC projects. However, after reading through the thread, I didn't see
us nail down any actual items.

As such, we need to quickly put together a list of oh, 15-20 midlevel
project ideas. I'm sure we can pull some off the TODO list, but we
should also look at project ideas for porting some of the most used
third-party OSS software to PostgreSQL too (portals, CMS systems,
accounting systems, etc.).

All ideas welcome!

--
Jonah H. Harris, Database Internals Architect
EnterpriseDB Corporation
732.331.1324

---------------------------(end of broadcast)---------------------------
TIP 2: Don't 'kill -9' the postmaster

OpenMFG has done some work on getting PostgreSQL working with the Drupal CMS and the Mantis bugtracker (and also integrating those two, btw). We're in contact with the respective projects about getting our patches worked in, but if anyone's keeping a tally, just wanted you to be aware.

Regards,
Ned

Jonah H. Harris wrote:

* Jonah H. Harris (jonah.harris@gmail.com) wrote:

I know we started a discussion a month or so ago regarding ideas for
SoC projects. However, after reading through the thread, I didn't see
us nail down any actual items.

I got an email already for a good idea, actually, which is to work on
having pg_hba.conf modifiable from SQL. The only problem with that is
that it really needs to be done in an acceptable way which requires
probably as much design work as actual programming. Another idea along
those same lines would be having .k5login-style support for Kerberos.
We'd need a conf-flag for that for backwards compatibility (once the
.k5login-style support exists we should clean up our Kerberos
credentials matching to, for example, not accept 'sfrost/root' for
'sfrost' or 'sfrost@ABC.COM' for 'sfrost@XYZ.com').

It'd also be nice to support SASL, and better hashes than md5.

Thanks,

Stephen

On Sat, Apr 15, 2006 at 03:05:20PM -0400, Jonah H. Harris wrote:

All ideas welcome!

I know it's not directly PostgreSQL related, but I'd love to see the
dbt* code improved. Items on my wish-list:

- make it easy to run the test framework and clients on a seperate
machine from the database server
- keep results in a database
- provide a front-end to allow users to schedule tests in a queue
- add support for windows, at least for the database (theoretically
possible to run that way now, but you have to do everything by hand)

Another idea: afaik, spikesource is still offering a bounty for
improvements to OSS test suites, something that'd fit well with SoC.
--
Jim C. Nasby, Sr. Engineering Consultant jnasby@pervasive.com
Pervasive Software http://pervasive.com work: 512-231-6117
vcard: http://jim.nasby.net/pervasive.vcf cell: 512-569-9461

Jim C. Nasby wrote:

On Sat, Apr 15, 2006 at 03:05:20PM -0400, Jonah H. Harris wrote:

All ideas welcome!

I know it's not directly PostgreSQL related, but I'd love to see the
dbt* code improved. Items on my wish-list:

- make it easy to run the test framework and clients on a seperate
machine from the database server
- keep results in a database
- provide a front-end to allow users to schedule tests in a queue
- add support for windows, at least for the database (theoretically
possible to run that way now, but you have to do everything by hand)

Another idea: afaik, spikesource is still offering a bounty for
improvements to OSS test suites, something that'd fit well with SoC.

I second this. :) There are also the TPC-App (Java) fair-use
implementation that I've started and the TPC-E (next gen OLTP) that I
would like to start.

Mark

On Tue, Apr 18, 2006 at 11:27:40AM -0700, Mark Wong wrote:

Jim C. Nasby wrote:

On Sat, Apr 15, 2006 at 03:05:20PM -0400, Jonah H. Harris wrote:

All ideas welcome!

I know it's not directly PostgreSQL related, but I'd love to see the
dbt* code improved. Items on my wish-list:

- make it easy to run the test framework and clients on a seperate
machine from the database server
- keep results in a database
- provide a front-end to allow users to schedule tests in a queue
- add support for windows, at least for the database (theoretically
possible to run that way now, but you have to do everything by hand)

Another idea: afaik, spikesource is still offering a bounty for
improvements to OSS test suites, something that'd fit well with SoC.

I second this. :) There are also the TPC-App (Java) fair-use
implementation that I've started and the TPC-E (next gen OLTP) that I
would like to start.

Maybe before starting on TPC-E it makes sense to try and get a common
framework for all the different tests built? AFAIK most of the
benchmarks all use a fairly standard client-server infrastructure, so we
should hopefully be able to share that between the different tests...
--
Jim C. Nasby, Sr. Engineering Consultant jnasby@pervasive.com
Pervasive Software http://pervasive.com work: 512-231-6117
vcard: http://jim.nasby.net/pervasive.vcf cell: 512-569-9461

On 4/18/06, Jim C. Nasby <jnasby@pervasive.com> wrote:

On Tue, Apr 18, 2006 at 11:27:40AM -0700, Mark Wong wrote:

Jim C. Nasby wrote:

On Sat, Apr 15, 2006 at 03:05:20PM -0400, Jonah H. Harris wrote:

All ideas welcome!

I know it's not directly PostgreSQL related, but I'd love to see the
dbt* code improved. Items on my wish-list:

- make it easy to run the test framework and clients on a seperate
machine from the database server
- keep results in a database
- provide a front-end to allow users to schedule tests in a queue
- add support for windows, at least for the database (theoretically
possible to run that way now, but you have to do everything by hand)

Another idea: afaik, spikesource is still offering a bounty for
improvements to OSS test suites, something that'd fit well with SoC.

I second this. :) There are also the TPC-App (Java) fair-use
implementation that I've started and the TPC-E (next gen OLTP) that I
would like to start.

Maybe before starting on TPC-E it makes sense to try and get a common
framework for all the different tests built? AFAIK most of the
benchmarks all use a fairly standard client-server infrastructure, so we
should hopefully be able to share that between the different tests...

I agree with Jim. A framework would really help out here. All of the
tests are basically the same and would benefit from a framework.

However, Mark, do you think Java is a reliable benchmarking platform?
At EnterpriseDB, we've tried several Java benchmarks and could never
get as repeatable or reliable of a benchmark as DBT2 gives you.

--
Jonah H. Harris, Database Internals Architect
EnterpriseDB Corporation
732.331.1324

Jonah H. Harris wrote:

On 4/18/06, Jim C. Nasby <jnasby@pervasive.com> wrote:

On Tue, Apr 18, 2006 at 11:27:40AM -0700, Mark Wong wrote:

Jim C. Nasby wrote:

On Sat, Apr 15, 2006 at 03:05:20PM -0400, Jonah H. Harris wrote:

All ideas welcome!

I know it's not directly PostgreSQL related, but I'd love to see the
dbt* code improved. Items on my wish-list:

- make it easy to run the test framework and clients on a seperate
machine from the database server
- keep results in a database
- provide a front-end to allow users to schedule tests in a queue
- add support for windows, at least for the database (theoretically
possible to run that way now, but you have to do everything by hand)

Another idea: afaik, spikesource is still offering a bounty for
improvements to OSS test suites, something that'd fit well with SoC.

I second this. :) There are also the TPC-App (Java) fair-use
implementation that I've started and the TPC-E (next gen OLTP) that I
would like to start.

Maybe before starting on TPC-E it makes sense to try and get a common
framework for all the different tests built? AFAIK most of the
benchmarks all use a fairly standard client-server infrastructure, so we
should hopefully be able to share that between the different tests...

I agree with Jim. A framework would really help out here. All of the
tests are basically the same and would benefit from a framework.

This has crossed my mind before. I haven't been able to come up with
something that I've felt good about on my own though.

However, Mark, do you think Java is a reliable benchmarking platform?
At EnterpriseDB, we've tried several Java benchmarks and could never
get as repeatable or reliable of a benchmark as DBT2 gives you.

I don't have much experience here yet. I've only got a portion of the
TPC-App implemented, although probably enough now to see how repeatable
it is thus far. Do you want to give my DBT4 kit a shot? :) I'm curious
to what platforms you've tried Java on as I've heard the Linux
implementations aren't as good as their Windows counterparts. I'm not
sure how true that is today though.

Mark

Proposed item: Improve PL/PHP support, especially installation on non-
Linux platforms. PL/PHP does not currently work on OS X (not sure
about Windows, but I doubt it).

Alvaro indicated he would be willing to provide direction on this
with testing support from me. He also said there are several other
possible PL/PHP issues that would warrant a SoC project.

John DeSoi, Ph.D.
http://pgedit.com/
Power Tools for PostgreSQL

On 4/19/06, John DeSoi <desoi@pgedit.com> wrote:

Alvaro indicated he would be willing to provide direction on this
with testing support from me. He also said there are several other
possible PL/PHP issues that would warrant a SoC project.

Cool... let's get 'em all listed here so we can move forward.

--
Jonah H. Harris, Database Internals Architect
EnterpriseDB Corporation
732.331.1324

John DeSoi wrote:

Proposed item: Improve PL/PHP support, especially installation on
non-Linux platforms. PL/PHP does not currently work on OS X (not sure
about Windows, but I doubt it).

It definitely does NOT work on Windows. MacOSX is just a matter of us
having some time.

Alvaro indicated he would be willing to provide direction on this with
testing support from me. He also said there are several other possible
PL/PHP issues that would warrant a SoC project.

Well my number one issue is the build process which needs to be cleaned
up but there are other more technical issues to be resolved as well.

Joshua D. Drake

John DeSoi, Ph.D.
http://pgedit.com/
Power Tools for PostgreSQL

---------------------------(end of broadcast)---------------------------
TIP 3: Have you checked our extensive FAQ?

http://www.postgresql.org/docs/faq

--

=== The PostgreSQL Company: Command Prompt, Inc. ===
Sales/Support: +1.503.667.4564 || 24x7/Emergency: +1.800.492.2240
Providing the most comprehensive PostgreSQL solutions since 1997
http://www.commandprompt.com/

Joshua D. Drake wrote:

John DeSoi wrote:

Proposed item: Improve PL/PHP support, especially installation on
non-Linux platforms. PL/PHP does not currently work on OS X (not sure
about Windows, but I doubt it).

It definitely does NOT work on Windows. MacOSX is just a matter of us
having some time.

Alvaro indicated he would be willing to provide direction on this with
testing support from me. He also said there are several other possible
PL/PHP issues that would warrant a SoC project.

Well my number one issue is the build process which needs to be cleaned
up but there are other more technical issues to be resolved as well.

Yeah, there are also a number of possible improvements documented as
tickets in the Trac site and others that currently exist only as very
vague noise in my head.

--
Alvaro Herrera http://www.CommandPrompt.com/
The PostgreSQL Company - Command Prompt, Inc.

On Wednesday 19 April 2006 12:09, Jonah H. Harris wrote:

On 4/19/06, John DeSoi <desoi@pgedit.com> wrote:

Alvaro indicated he would be willing to provide direction on this
with testing support from me. He also said there are several other
possible PL/PHP issues that would warrant a SoC project.

Cool... let's get 'em all listed here so we can move forward.

I think Martin Oosterhout's nearby email on coverity bug reports might make a
good SoC project, but should it also be added to the TODO list?

--
Robert Treat
Build A Brighter Lamp :: Linux Apache {middleware} PostgreSQL

On Thu, Apr 20, 2006 at 08:51:25AM -0400, Robert Treat wrote:

On Wednesday 19 April 2006 12:09, Jonah H. Harris wrote:

On 4/19/06, John DeSoi <desoi@pgedit.com> wrote:

Alvaro indicated he would be willing to provide direction on this
with testing support from me. He also said there are several other
possible PL/PHP issues that would warrant a SoC project.

Cool... let's get 'em all listed here so we can move forward.

I think Martin Oosterhout's nearby email on coverity bug reports might make a
good SoC project, but should it also be added to the TODO list?

Nice idea, though it would be much more useful if the reports could be
exported en-masse. There's an export function but it only exports the
user comments, not the error itself. So unless people signup there's no
easy way to get the info to people. :(

In any case, after you weed out the false-positives and exclude ECPG
you're only talking about less than 50 issues that may need to be
addressed. Hardly a project that will take any amount of time.

Have a nice day,
--
Martijn van Oosterhout <kleptog@svana.org> http://svana.org/kleptog/

Martijn van Oosterhout <kleptog@svana.org> writes:

On Thu, Apr 20, 2006 at 08:51:25AM -0400, Robert Treat wrote:

I think Martin Oosterhout's nearby email on coverity bug reports might make a
good SoC project, but should it also be added to the TODO list?

...
In any case, after you weed out the false-positives and exclude ECPG
you're only talking about less than 50 issues that may need to be
addressed. Hardly a project that will take any amount of time.

Nor one we'd be willing to wait till the summer to address, if any of
the bugs are real.

regards, tom lane