OLAP

Started by Alban Hertroysover 12 years ago9 messagesgeneral
Jump to latest
#1Alban Hertroys
haramrae@gmail.com

Hi all,

At work we have a system that's built on top of a proprietary OLAP database
system (Rocket Gentia). We're looking into replacing that system with
something that's still supported and in such a way that we can also access
the data from our reporting software (WebFOCUS by information builders).

Because I'm always advocating PG, I was asked whether PG would be suitable
for this, but I'm not really familiar enough with OLAP databases to be able
to comment on that.

I got three prerequisites for a solution, namely:
1. It must contain correct information,
2. It must be fast and
3. It must be easy to maintain the data and the models; that's a task for a
3rd party back-end application, but it would be helpful to be able to name
something to the higher-ups.

Next to that, because we're also going to access the system using our
reporting software (which is read-only access), it would be best if the
entire data model and all the business rules are stored inside the database
so that we're looking at the data in the same way that the "back-end" sees
it.

For size, we're looking at about 20 years of sales and shipment data all
over the world (although mostly in Europe) for about 5mln sold products per
year.

I suspect there might be some "middleware" that handles the models and
dimensions and stuff and manages triggers on relational tables in PG or a
setup like that.
I've seen an old reference to "Cybertec OLAP", but they don't seem to carry
a product like that if I watch their site.

I'm looking for suggestions for something that would be able to do this.

Cheers,
Alban.
--
If you can't see the forest for the trees,
Cut the trees and you'll see there is no forest.

#2Paul Jungwirth
pj@illuminatedcomputing.com
In reply to: Alban Hertroys (#1)
Re: OLAP

Hi Alban,

I think Postgres works great for OLAP work, and Amazon's Red Shift is
even based on Postgres. 100 million sales should be not problem at
all. My understanding is Greenplum also builds on top of Postgres, so
if you ever do outgrow your Postgres installation, that would be an
easy migration path. One Postgres OLAP tool to consider is Pentaho.
That will save you lots of time around ETL, ad-hoc reporting, and
other standard OLAP functionality.

Good luck!
Paul

On Tue, Aug 27, 2013 at 8:12 AM, Alban Hertroys <haramrae@gmail.com> wrote:

Hi all,

At work we have a system that's built on top of a proprietary OLAP database
system (Rocket Gentia). We're looking into replacing that system with
something that's still supported and in such a way that we can also access
the data from our reporting software (WebFOCUS by information builders).

Because I'm always advocating PG, I was asked whether PG would be suitable
for this, but I'm not really familiar enough with OLAP databases to be able
to comment on that.

I got three prerequisites for a solution, namely:
1. It must contain correct information,
2. It must be fast and
3. It must be easy to maintain the data and the models; that's a task for a
3rd party back-end application, but it would be helpful to be able to name
something to the higher-ups.

Next to that, because we're also going to access the system using our
reporting software (which is read-only access), it would be best if the
entire data model and all the business rules are stored inside the database
so that we're looking at the data in the same way that the "back-end" sees
it.

For size, we're looking at about 20 years of sales and shipment data all
over the world (although mostly in Europe) for about 5mln sold products per
year.

I suspect there might be some "middleware" that handles the models and
dimensions and stuff and manages triggers on relational tables in PG or a
setup like that.
I've seen an old reference to "Cybertec OLAP", but they don't seem to carry
a product like that if I watch their site.

I'm looking for suggestions for something that would be able to do this.

Cheers,
Alban.
--
If you can't see the forest for the trees,
Cut the trees and you'll see there is no forest.

--
_________________________________
Pulchritudo splendor veritatis.

--
Sent via pgsql-general mailing list (pgsql-general@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-general

#3Alban Hertroys
haramrae@gmail.com
In reply to: Paul Jungwirth (#2)
Re: OLAP

On Aug 27, 2013, at 19:07, Paul Jungwirth <pj@illuminatedcomputing.com> wrote:

Hi Alban,

I think Postgres works great for OLAP work

What do you base that on?
I don't really doubt that the database layer is up to the task, I'm much more worried about maintaining the model and the cube data and all that typical OLAP stuff that I've mostly just heard about.

, and Amazon's Red Shift is
even based on Postgres. 100 million sales should be not problem at
all. My understanding is Greenplum also builds on top of Postgres, so
if you ever do outgrow your Postgres installation, that would be an
easy migration path.

What's the benefit of GreenPlum for OLAP? Isn't it a columnar database? And a pretty old fork of Postgres at that?
GreenPlum has a pretty steep price-tag too.

I didn't really look into Red Shift, perhaps I should…

Anyway, I'm not at all sure I want to use some product that's heavily modified from Postgres. If it is, it has to be really really good.

One Postgres OLAP tool to consider is Pentaho.
That will save you lots of time around ETL, ad-hoc reporting, and
other standard OLAP functionality.

How is Pentaho an OLAP tool? Aren't you mixing up a few things?
We already use Pentaho for ETL, so I'm a bit familiar with it. Why do you consider it suitable for managing an OLAP database?

How would Pentaho manage cube rollup triggers, business models, dimensions and stuff like that?
We don't want to hand code those, that's far too error-prone and far too much work to keep track of. That stuff needs to be automated, preferably similar to what we're used to from Gentia (well, not me - I can't make heads or tails of Gentia, but the person who asked me about PG's suitability has been developing with it for years). That's what we're comparing to.

Unfortunately, I can't find any decent information about Gentia for reference. Apparently these days they're all about NoSQL databases and such. That's not what we have - I guess the clunky GUI is a hint that it's something of the past...

BTW, please don't top-post.

On Tue, Aug 27, 2013 at 8:12 AM, Alban Hertroys <haramrae@gmail.com> wrote:

Hi all,

At work we have a system that's built on top of a proprietary OLAP database
system (Rocket Gentia). We're looking into replacing that system with
something that's still supported and in such a way that we can also access
the data from our reporting software (WebFOCUS by information builders).

Because I'm always advocating PG, I was asked whether PG would be suitable
for this, but I'm not really familiar enough with OLAP databases to be able
to comment on that.

I got three prerequisites for a solution, namely:
1. It must contain correct information,
2. It must be fast and
3. It must be easy to maintain the data and the models; that's a task for a
3rd party back-end application, but it would be helpful to be able to name
something to the higher-ups.

Next to that, because we're also going to access the system using our
reporting software (which is read-only access), it would be best if the
entire data model and all the business rules are stored inside the database
so that we're looking at the data in the same way that the "back-end" sees
it.

For size, we're looking at about 20 years of sales and shipment data all
over the world (although mostly in Europe) for about 5mln sold products per
year.

I suspect there might be some "middleware" that handles the models and
dimensions and stuff and manages triggers on relational tables in PG or a
setup like that.
I've seen an old reference to "Cybertec OLAP", but they don't seem to carry
a product like that if I watch their site.

I'm looking for suggestions for something that would be able to do this.

Cheers,
Alban.
--
If you can't see the forest for the trees,
Cut the trees and you'll see there is no forest.

--
_________________________________
Pulchritudo splendor veritatis.

Alban Hertroys
--
If you can't see the forest for the trees,
cut the trees and you'll find there is no forest.

--
Sent via pgsql-general mailing list (pgsql-general@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-general

#4Jerry Sievers
gsievers19@comcast.net
In reply to: Alban Hertroys (#3)
Re: OLAP

Alban Hertroys <haramrae@gmail.com> writes:

On Aug 27, 2013, at 19:07, Paul Jungwirth <pj@illuminatedcomputing.com> wrote:

Hi Alban,

I think Postgres works great for OLAP work

What do you base that on?
I don't really doubt that the database layer is up to the task, I'm much more worried about maintaining the model and the cube data and all that typical OLAP stuff that I've mostly just heard about.

, and Amazon's Red Shift is
even based on Postgres. 100 million sales should be not problem at
all. My understanding is Greenplum also builds on top of Postgres, so
if you ever do outgrow your Postgres installation, that would be an
easy migration path.

What's the benefit of GreenPlum for OLAP? Isn't it a columnar database? And a pretty old fork of Postgres at that?
GreenPlum has a pretty steep price-tag too.

Vertica is another case of an analytics focused platform that dirived
from Postgres, version 8.0 IIRC.

It was, by the time I first looked at it back about 4 years ago, only
superficially resembling Postgres. Performance was absolutely
shocking in terms of how quickly it processed queries over certain
kinds of data... and for which the set of expected queries to be run
over same was identifiable in advance.

Sample queries are given to a moddeler which in turn creates a set of
"projections" which are physical manifestations of the backend storage
intended to optimize for this specialized workload.

Vertica and I presume Green Plumb are *not* well suited for an OLTP
role so it takes a fair amount of learning to make good use of them.

Just FWIW.

I didn't really look into Red Shift, perhaps I should…

Anyway, I'm not at all sure I want to use some product that's heavily modified from Postgres. If it is, it has to be really really good.

One Postgres OLAP tool to consider is Pentaho.
That will save you lots of time around ETL, ad-hoc reporting, and
other standard OLAP functionality.

How is Pentaho an OLAP tool? Aren't you mixing up a few things?
We already use Pentaho for ETL, so I'm a bit familiar with it. Why do you consider it suitable for managing an OLAP database?

How would Pentaho manage cube rollup triggers, business models, dimensions and stuff like that?
We don't want to hand code those, that's far too error-prone and far too much work to keep track of. That stuff needs to be automated, preferably similar to what we're used to from Gentia (well, not me - I can't make heads or tails of Gentia, but the person who asked me about PG's suitability has been developing with it for years). That's what we're comparing to.

Unfortunately, I can't find any decent information about Gentia for reference. Apparently these days they're all about NoSQL databases and such. That's not what we have - I guess the clunky GUI is a hint that it's something of the past...

BTW, please don't top-post.

On Tue, Aug 27, 2013 at 8:12 AM, Alban Hertroys <haramrae@gmail.com> wrote:

Hi all,

At work we have a system that's built on top of a proprietary OLAP database
system (Rocket Gentia). We're looking into replacing that system with
something that's still supported and in such a way that we can also access
the data from our reporting software (WebFOCUS by information builders).

Because I'm always advocating PG, I was asked whether PG would be suitable
for this, but I'm not really familiar enough with OLAP databases to be able
to comment on that.

I got three prerequisites for a solution, namely:
1. It must contain correct information,
2. It must be fast and
3. It must be easy to maintain the data and the models; that's a task for a
3rd party back-end application, but it would be helpful to be able to name
something to the higher-ups.

Next to that, because we're also going to access the system using our
reporting software (which is read-only access), it would be best if the
entire data model and all the business rules are stored inside the database
so that we're looking at the data in the same way that the "back-end" sees
it.

For size, we're looking at about 20 years of sales and shipment data all
over the world (although mostly in Europe) for about 5mln sold products per
year.

I suspect there might be some "middleware" that handles the models and
dimensions and stuff and manages triggers on relational tables in PG or a
setup like that.
I've seen an old reference to "Cybertec OLAP", but they don't seem to carry
a product like that if I watch their site.

I'm looking for suggestions for something that would be able to do this.

Cheers,
Alban.
--
If you can't see the forest for the trees,
Cut the trees and you'll see there is no forest.

--
_________________________________
Pulchritudo splendor veritatis.

Alban Hertroys
--
If you can't see the forest for the trees,
cut the trees and you'll find there is no forest.

--
Sent via pgsql-general mailing list (pgsql-general@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-general

--
Jerry Sievers
Postgres DBA/Development Consulting
e: postgres.consulting@comcast.net
p: 312.241.7800

--
Sent via pgsql-general mailing list (pgsql-general@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-general

#5Pavel Stehule
pavel.stehule@gmail.com
In reply to: Jerry Sievers (#4)
Re: OLAP

Dne 28. 8. 2013 0:05 "Jerry Sievers" <gsievers19@comcast.net> napsal(a):

Alban Hertroys <haramrae@gmail.com> writes:

On Aug 27, 2013, at 19:07, Paul Jungwirth <pj@illuminatedcomputing.com>

wrote:

Hi Alban,

I think Postgres works great for OLAP work

What do you base that on?
I don't really doubt that the database layer is up to the task, I'm

much more worried about maintaining the model and the cube data and all
that typical OLAP stuff that I've mostly just heard about.

, and Amazon's Red Shift is
even based on Postgres. 100 million sales should be not problem at
all. My understanding is Greenplum also builds on top of Postgres, so
if you ever do outgrow your Postgres installation, that would be an
easy migration path.

What's the benefit of GreenPlum for OLAP? Isn't it a columnar database?

And a pretty old fork of Postgres at that?

GreenPlum has a pretty steep price-tag too.

Vertica is another case of an analytics focused platform that dirived
from Postgres, version 8.0 IIR

vertica use a similar interface, but internally use nothing from pg. it was
written from zero.

It was, by the time I first looked at it back about 4 years ago, only
superficially resembling Postgres. Performance was absolutely
shocking in terms of how quickly it processed queries over certain
kinds of data... and for which the set of expected queries to be run
over same was identifiable in advance.

Sample queries are given to a moddeler which in turn creates a set of
"projections" which are physical manifestations of the backend storage
intended to optimize for this specialized workload.

Vertica and I presume Green Plumb are *not* well suited for an OLTP
role so it takes a fair amount of learning to make good use of them.

Just FWIW.

I didn't really look into Red Shift, perhaps I should…

Anyway, I'm not at all sure I want to use some product that's heavily

modified from Postgres. If it is, it has to be really really good.

One Postgres OLAP tool to consider is Pentaho.
That will save you lots of time around ETL, ad-hoc reporting, and
other standard OLAP functionality.

How is Pentaho an OLAP tool? Aren't you mixing up a few things?
We already use Pentaho for ETL, so I'm a bit familiar with it. Why do

you consider it suitable for managing an OLAP database?

How would Pentaho manage cube rollup triggers, business models,

dimensions and stuff like that?

We don't want to hand code those, that's far too error-prone and far

too much work to keep track of. That stuff needs to be automated,
preferably similar to what we're used to from Gentia (well, not me - I
can't make heads or tails of Gentia, but the person who asked me about PG's
suitability has been developing with it for years). That's what we're
comparing to.

Unfortunately, I can't find any decent information about Gentia for

reference. Apparently these days they're all about NoSQL databases and
such. That's not what we have - I guess the clunky GUI is a hint that it's
something of the past...

BTW, please don't top-post.

On Tue, Aug 27, 2013 at 8:12 AM, Alban Hertroys <haramrae@gmail.com>

wrote:

Hi all,

At work we have a system that's built on top of a proprietary OLAP

database

system (Rocket Gentia). We're looking into replacing that system with
something that's still supported and in such a way that we can also

access

the data from our reporting software (WebFOCUS by information

builders).

Because I'm always advocating PG, I was asked whether PG would be

suitable

for this, but I'm not really familiar enough with OLAP databases to

be able

to comment on that.

I got three prerequisites for a solution, namely:
1. It must contain correct information,
2. It must be fast and
3. It must be easy to maintain the data and the models; that's a task

for a

3rd party back-end application, but it would be helpful to be able to

name

something to the higher-ups.

Next to that, because we're also going to access the system using our
reporting software (which is read-only access), it would be best if

the

entire data model and all the business rules are stored inside the

database

so that we're looking at the data in the same way that the "back-end"

sees

it.

For size, we're looking at about 20 years of sales and shipment data

all

over the world (although mostly in Europe) for about 5mln sold

products per

year.

I suspect there might be some "middleware" that handles the models and
dimensions and stuff and manages triggers on relational tables in PG

or a

setup like that.
I've seen an old reference to "Cybertec OLAP", but they don't seem to

carry

a product like that if I watch their site.

I'm looking for suggestions for something that would be able to do

this.

Show quoted text

Cheers,
Alban.
--
If you can't see the forest for the trees,
Cut the trees and you'll see there is no forest.

--
_________________________________
Pulchritudo splendor veritatis.

Alban Hertroys
--
If you can't see the forest for the trees,
cut the trees and you'll find there is no forest.

--
Sent via pgsql-general mailing list (pgsql-general@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-general

--
Jerry Sievers
Postgres DBA/Development Consulting
e: postgres.consulting@comcast.net
p: 312.241.7800

--
Sent via pgsql-general mailing list (pgsql-general@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-general

#6Carlos Saritama
cssaritama@gmail.com
In reply to: Pavel Stehule (#5)
Re: OLAP

according to what you write pentaho best fits your needs

On Tue, Aug 27, 2013 at 5:52 PM, Pavel Stehule <pavel.stehule@gmail.com>wrote:

Show quoted text

Dne 28. 8. 2013 0:05 "Jerry Sievers" <gsievers19@comcast.net> napsal(a):

Alban Hertroys <haramrae@gmail.com> writes:

On Aug 27, 2013, at 19:07, Paul Jungwirth <pj@illuminatedcomputing.com>

wrote:

Hi Alban,

I think Postgres works great for OLAP work

What do you base that on?
I don't really doubt that the database layer is up to the task, I'm

much more worried about maintaining the model and the cube data and all
that typical OLAP stuff that I've mostly just heard about.

, and Amazon's Red Shift is
even based on Postgres. 100 million sales should be not problem at
all. My understanding is Greenplum also builds on top of Postgres, so
if you ever do outgrow your Postgres installation, that would be an
easy migration path.

What's the benefit of GreenPlum for OLAP? Isn't it a columnar

database? And a pretty old fork of Postgres at that?

GreenPlum has a pretty steep price-tag too.

Vertica is another case of an analytics focused platform that dirived
from Postgres, version 8.0 IIR

vertica use a similar interface, but internally use nothing from pg. it
was written from zero.

It was, by the time I first looked at it back about 4 years ago, only
superficially resembling Postgres. Performance was absolutely
shocking in terms of how quickly it processed queries over certain
kinds of data... and for which the set of expected queries to be run
over same was identifiable in advance.

Sample queries are given to a moddeler which in turn creates a set of
"projections" which are physical manifestations of the backend storage
intended to optimize for this specialized workload.

Vertica and I presume Green Plumb are *not* well suited for an OLTP
role so it takes a fair amount of learning to make good use of them.

Just FWIW.

I didn't really look into Red Shift, perhaps I should…

Anyway, I'm not at all sure I want to use some product that's heavily

modified from Postgres. If it is, it has to be really really good.

One Postgres OLAP tool to consider is Pentaho.
That will save you lots of time around ETL, ad-hoc reporting, and
other standard OLAP functionality.

How is Pentaho an OLAP tool? Aren't you mixing up a few things?
We already use Pentaho for ETL, so I'm a bit familiar with it. Why do

you consider it suitable for managing an OLAP database?

How would Pentaho manage cube rollup triggers, business models,

dimensions and stuff like that?

We don't want to hand code those, that's far too error-prone and far

too much work to keep track of. That stuff needs to be automated,
preferably similar to what we're used to from Gentia (well, not me - I
can't make heads or tails of Gentia, but the person who asked me about PG's
suitability has been developing with it for years). That's what we're
comparing to.

Unfortunately, I can't find any decent information about Gentia for

reference. Apparently these days they're all about NoSQL databases and
such. That's not what we have - I guess the clunky GUI is a hint that it's
something of the past...

BTW, please don't top-post.

On Tue, Aug 27, 2013 at 8:12 AM, Alban Hertroys <haramrae@gmail.com>

wrote:

Hi all,

At work we have a system that's built on top of a proprietary OLAP

database

system (Rocket Gentia). We're looking into replacing that system with
something that's still supported and in such a way that we can also

access

the data from our reporting software (WebFOCUS by information

builders).

Because I'm always advocating PG, I was asked whether PG would be

suitable

for this, but I'm not really familiar enough with OLAP databases to

be able

to comment on that.

I got three prerequisites for a solution, namely:
1. It must contain correct information,
2. It must be fast and
3. It must be easy to maintain the data and the models; that's a

task for a

3rd party back-end application, but it would be helpful to be able

to name

something to the higher-ups.

Next to that, because we're also going to access the system using our
reporting software (which is read-only access), it would be best if

the

entire data model and all the business rules are stored inside the

database

so that we're looking at the data in the same way that the

"back-end" sees

it.

For size, we're looking at about 20 years of sales and shipment data

all

over the world (although mostly in Europe) for about 5mln sold

products per

year.

I suspect there might be some "middleware" that handles the models

and

dimensions and stuff and manages triggers on relational tables in PG

or a

setup like that.
I've seen an old reference to "Cybertec OLAP", but they don't seem

to carry

a product like that if I watch their site.

I'm looking for suggestions for something that would be able to do

this.

Cheers,
Alban.
--
If you can't see the forest for the trees,
Cut the trees and you'll see there is no forest.

--
_________________________________
Pulchritudo splendor veritatis.

Alban Hertroys
--
If you can't see the forest for the trees,
cut the trees and you'll find there is no forest.

--
Sent via pgsql-general mailing list (pgsql-general@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-general

--
Jerry Sievers
Postgres DBA/Development Consulting
e: postgres.consulting@comcast.net
p: 312.241.7800

--
Sent via pgsql-general mailing list (pgsql-general@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-general

#7Carlos Saritama
cssaritama@gmail.com
In reply to: Carlos Saritama (#6)
Re: OLAP

Checkout the Saiku, the future of Open Source Interactive OLAP(
http://analytical-labs.com )

On Tue, Aug 27, 2013 at 8:34 PM, Carlos Saritama <cssaritama@gmail.com>wrote:

Show quoted text

according to what you write pentaho best fits your needs

On Tue, Aug 27, 2013 at 5:52 PM, Pavel Stehule <pavel.stehule@gmail.com>wrote:

Dne 28. 8. 2013 0:05 "Jerry Sievers" <gsievers19@comcast.net> napsal(a):

Alban Hertroys <haramrae@gmail.com> writes:

On Aug 27, 2013, at 19:07, Paul Jungwirth <

pj@illuminatedcomputing.com> wrote:

Hi Alban,

I think Postgres works great for OLAP work

What do you base that on?
I don't really doubt that the database layer is up to the task, I'm

much more worried about maintaining the model and the cube data and all
that typical OLAP stuff that I've mostly just heard about.

, and Amazon's Red Shift is
even based on Postgres. 100 million sales should be not problem at
all. My understanding is Greenplum also builds on top of Postgres, so
if you ever do outgrow your Postgres installation, that would be an
easy migration path.

What's the benefit of GreenPlum for OLAP? Isn't it a columnar

database? And a pretty old fork of Postgres at that?

GreenPlum has a pretty steep price-tag too.

Vertica is another case of an analytics focused platform that dirived
from Postgres, version 8.0 IIR

vertica use a similar interface, but internally use nothing from pg. it
was written from zero.

It was, by the time I first looked at it back about 4 years ago, only
superficially resembling Postgres. Performance was absolutely
shocking in terms of how quickly it processed queries over certain
kinds of data... and for which the set of expected queries to be run
over same was identifiable in advance.

Sample queries are given to a moddeler which in turn creates a set of
"projections" which are physical manifestations of the backend storage
intended to optimize for this specialized workload.

Vertica and I presume Green Plumb are *not* well suited for an OLTP
role so it takes a fair amount of learning to make good use of them.

Just FWIW.

I didn't really look into Red Shift, perhaps I should…

Anyway, I'm not at all sure I want to use some product that's heavily

modified from Postgres. If it is, it has to be really really good.

One Postgres OLAP tool to consider is Pentaho.
That will save you lots of time around ETL, ad-hoc reporting, and
other standard OLAP functionality.

How is Pentaho an OLAP tool? Aren't you mixing up a few things?
We already use Pentaho for ETL, so I'm a bit familiar with it. Why do

you consider it suitable for managing an OLAP database?

How would Pentaho manage cube rollup triggers, business models,

dimensions and stuff like that?

We don't want to hand code those, that's far too error-prone and far

too much work to keep track of. That stuff needs to be automated,
preferably similar to what we're used to from Gentia (well, not me - I
can't make heads or tails of Gentia, but the person who asked me about PG's
suitability has been developing with it for years). That's what we're
comparing to.

Unfortunately, I can't find any decent information about Gentia for

reference. Apparently these days they're all about NoSQL databases and
such. That's not what we have - I guess the clunky GUI is a hint that it's
something of the past...

BTW, please don't top-post.

On Tue, Aug 27, 2013 at 8:12 AM, Alban Hertroys <haramrae@gmail.com>

wrote:

Hi all,

At work we have a system that's built on top of a proprietary OLAP

database

system (Rocket Gentia). We're looking into replacing that system

with

something that's still supported and in such a way that we can also

access

the data from our reporting software (WebFOCUS by information

builders).

Because I'm always advocating PG, I was asked whether PG would be

suitable

for this, but I'm not really familiar enough with OLAP databases to

be able

to comment on that.

I got three prerequisites for a solution, namely:
1. It must contain correct information,
2. It must be fast and
3. It must be easy to maintain the data and the models; that's a

task for a

3rd party back-end application, but it would be helpful to be able

to name

something to the higher-ups.

Next to that, because we're also going to access the system using

our

reporting software (which is read-only access), it would be best if

the

entire data model and all the business rules are stored inside the

database

so that we're looking at the data in the same way that the

"back-end" sees

it.

For size, we're looking at about 20 years of sales and shipment

data all

over the world (although mostly in Europe) for about 5mln sold

products per

year.

I suspect there might be some "middleware" that handles the models

and

dimensions and stuff and manages triggers on relational tables in

PG or a

setup like that.
I've seen an old reference to "Cybertec OLAP", but they don't seem

to carry

a product like that if I watch their site.

I'm looking for suggestions for something that would be able to do

this.

Cheers,
Alban.
--
If you can't see the forest for the trees,
Cut the trees and you'll see there is no forest.

--
_________________________________
Pulchritudo splendor veritatis.

Alban Hertroys
--
If you can't see the forest for the trees,
cut the trees and you'll find there is no forest.

--
Sent via pgsql-general mailing list (pgsql-general@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-general

--
Jerry Sievers
Postgres DBA/Development Consulting
e: postgres.consulting@comcast.net
p: 312.241.7800

--
Sent via pgsql-general mailing list (pgsql-general@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-general

#8Jayadevan M
maymala.jayadevan@gmail.com
In reply to: Alban Hertroys (#3)
Re: OLAP

Alban Hertroys-4 wrote

How is Pentaho an OLAP tool? Aren't you mixing up a few things?
We already use Pentaho for ETL, so I'm a bit familiar with it. Why do you
consider it suitable for managing an OLAP database?

How would Pentaho manage cube rollup triggers, business models, dimensions
and stuff like that?
We don't want to hand code those, that's far too error-prone and far too
much work to keep track of. That stuff needs to be automated, preferably
similar to what we're used to from Gentia (well, not me - I can't make
heads or tails of Gentia, but the person who asked me about PG's
suitability has been developing with it for years). That's what we're
comparing to.

Pentaho, Jasper and Actuate are the common OLAP tools. Pentahos' ETL tool
(Kettle) is more popular than their Reporting solution. Similarly, Jasper's
reporting tool is more popular than their ETL (they integrate talend). The
OLAP functionality in both come from Mondrian, as far as I know.
The cubes, dimensions etc can be defined using a UI tool and uploaded to the
reporting "server" - either Jasper or Pentaho. These typically support MDX,
in addition to standard SQL.
http://en.wikipedia.org/wiki/MultiDimensional_eXpressions

--
View this message in context: http://postgresql.1045698.n5.nabble.com/OLAP-tp5768698p5768782.html
Sent from the PostgreSQL - general mailing list archive at Nabble.com.

--
Sent via pgsql-general mailing list (pgsql-general@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-general

#9bricklen
bricklen@gmail.com
In reply to: Jerry Sievers (#4)
Re: OLAP

On Tue, Aug 27, 2013 at 3:04 PM, Jerry Sievers <gsievers19@comcast.net>wrote:

Vertica is another case of an analytics focused platform that dirived
from Postgres, version 8.0 IIRC.

FWIW,
You might be thinking of Greenplum, and I think it forked from pg 8.1.
Vertica was built from scratch, it is not using Postgres as the basis for
the db. This is according to the engineers that I have spoken to in the
past at Vertica. I believe there was some code ported over for their
command line client "vsql", but I could be wrong about that. At the very
least, vsql looks and acts like psql from a user's point of view.

I agree absolutely with you about the dazzling speed of certain types of
queries -- one of my tests was a simple aggregate query (a bunch of SUM()
and GROUP BY's) over about 500M rows and it returned results in a handful
of seconds. The same query in our Postgresql server with 2x the number of
disks took handfuls of minutes IIRC. On the flip side, I found some of the
complex Window functions performed on par or better in Postgresql than in
Vertica.