Autonomous database is coming to Postgres?

Started by tsunakawa.takay@fujitsu.comover 5 years ago3 messages
#1tsunakawa.takay@fujitsu.com
tsunakawa.takay@fujitsu.com

Hello,

I'm not sure if I should have posted this to pgsql-advocacy, but this is being developed so I posted here.

Does anyone know if this development come to open source Postgres, or only to the cloud services of Microsoft and Google?

(I wonder this will become another reason that Postgres won't incorporate optimizer hint feature.)

Data systems that learn to be better
http://news.mit.edu/2020/mit-data-systems-learn-be-better-tsunami-bao-0810

[Quote]
--------------------------------------------------
As a first step toward this vision, Kraska and colleagues developed Tsunami and Bao. Tsunami uses machine learning to automatically re-organize a dataset’s storage layout based on the types of queries that its users make. Tests show that it can run queries up to 10 times faster than state-of-the-art systems. What’s more, its datasets can be organized via a series of "learned indexes" that are up to 100 times smaller than the indexes used in traditional systems.

Bao, meanwhile, focuses on improving the efficiency of query optimization through machine learning.
...
Traditional query optimizers take years to build, are very hard to maintain, and, most importantly, do not learn from their mistakes. Bao is the first learning-based approach to query optimization that has been fully integrated into the popular database management system PostgreSQL. Lead author Ryan Marcus, a postdoc in Kraska’s group, says that Bao produces query plans that run up to 50 percent faster than those created by the PostgreSQL optimizer, meaning that it could help to significantly reduce the cost of cloud services, like Amazon’s Redshift, that are based on PostgreSQL.

Kraska says that in contrast to other learning-based approaches to query optimization, Bao learns much faster and can outperform open-source and commercial optimizers with as little as one hour of training time.In the future, his team aims to integrate Bao into cloud systems to improve resource utilization in environments where disk, RAM, and CPU time are scarce resources.
...
The work was done as part of the Data System and AI Lab (DSAIL@CSAIL), which is sponsored by Intel, Google, Microsoft, and the U.S. National Science Foundation.
--------------------------------------------------

Regards
Takayuki Tsunakawa

#2Bruce Momjian
bruce@momjian.us
In reply to: tsunakawa.takay@fujitsu.com (#1)
Re: Autonomous database is coming to Postgres?

On Thu, Aug 13, 2020 at 03:26:33AM +0000, tsunakawa.takay@fujitsu.com wrote:

Hello,

I'm not sure if I should have posted this to pgsql-advocacy, but this is being developed so I posted here.

Does anyone know if this development come to open source Postgres, or only to the cloud services of Microsoft and Google?

(I wonder this will become another reason that Postgres won't incorporate optimizer hint feature.)

Data systems that learn to be better
http://news.mit.edu/2020/mit-data-systems-learn-be-better-tsunami-bao-0810

It seems interesting, but I don't know anyone working on this.

--
Bruce Momjian <bruce@momjian.us> https://momjian.us
EnterpriseDB https://enterprisedb.com

The usefulness of a cup is in its emptiness, Bruce Lee

#3Dmitry Dolgov
9erthalion6@gmail.com
In reply to: Bruce Momjian (#2)
Re: Autonomous database is coming to Postgres?

On Fri, Aug 14, 2020 at 08:55:53AM -0400, Bruce Momjian wrote:

On Thu, Aug 13, 2020 at 03:26:33AM +0000, tsunakawa.takay@fujitsu.com wrote:
Hello,

I'm not sure if I should have posted this to pgsql-advocacy, but this is being developed so I posted here.
Does anyone know if this development come to open source Postgres, or only to the cloud services of Microsoft and Google?
(I wonder this will become another reason that Postgres won't incorporate optimizer hint feature.)

Data systems that learn to be better
http://news.mit.edu/2020/mit-data-systems-learn-be-better-tsunami-bao-0810

It seems interesting, but I don't know anyone working on this.

Tim Kraska mentioned in twitter plans about releasing BAO as an open
source project (PostgreSQL extension I guess?), but there seems to be no
interaction with the community.