nested hstore patch

Started by Teodor Sigaevover 12 years ago53 messageshackers
Jump to latest
#1Teodor Sigaev
teodor@sigaev.ru

Hi!

Attatched patch adds nesting feature, types (string, boll and numeric values),
arrays and scalar to hstore type.

All new features are described in PGConf.EU talk
http://www.sai.msu.su/~megera/postgres/talks/hstore-dublin-2013.pdf (since PGCon
some features was added).

Patch includes:
1 implementaion SRF_RETURN_NEXT_NULL()
2 contrib/hstore changes
3 docs of new hstore module (many thanks to David E. Wheeler
<david.wheeler@pgexperts.com>)

In current state patch is in WIP status, for short period I plan to move support
of binary nested structure to core to share binary representation for hstore and
json types.

--
Teodor Sigaev E-mail: teodor@sigaev.ru
WWW: http://www.sigaev.ru/

Attachments:

nested_hstore-0.40.patch.gzapplication/x-tar; name=nested_hstore-0.40.patch.gzDownload
#2Andrew Dunstan
andrew@dunslane.net
In reply to: Teodor Sigaev (#1)
Re: nested hstore patch

On 11/12/2013 01:35 PM, Teodor Sigaev wrote:

Hi!

Attatched patch adds nesting feature, types (string, boll and numeric
values), arrays and scalar to hstore type.

All new features are described in PGConf.EU talk
http://www.sai.msu.su/~megera/postgres/talks/hstore-dublin-2013.pdf
(since PGCon some features was added).

Patch includes:
1 implementaion SRF_RETURN_NEXT_NULL()
2 contrib/hstore changes
3 docs of new hstore module (many thanks to David E. Wheeler
<david.wheeler@pgexperts.com>)

In current state patch is in WIP status, for short period I plan to
move support of binary nested structure to core to share binary
representation for hstore and json types.

Thanks, Teodor.

As soon as we have that shared binary representation available, I will
be working on adapting it to JSON.

cheers

andrew

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

#3Peter Eisentraut
peter_e@gmx.net
In reply to: Teodor Sigaev (#1)
Re: nested hstore patch

On 11/12/13, 1:35 PM, Teodor Sigaev wrote:

Attatched patch adds nesting feature, types (string, boll and numeric
values), arrays and scalar to hstore type.

Could you check your email client for next time? It's sending
Content-Type: application/x-tar for a *.patch.gz file.

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

#4Hannu Krosing
hannu@tm.ee
In reply to: Andrew Dunstan (#2)
Re: nested hstore patch

On 11/13/2013 01:37 AM, Andrew Dunstan wrote:

On 11/12/2013 01:35 PM, Teodor Sigaev wrote:

Hi!

Attatched patch adds nesting feature, types (string, boll and numeric
values), arrays and scalar to hstore type.

All new features are described in PGConf.EU talk
http://www.sai.msu.su/~megera/postgres/talks/hstore-dublin-2013.pdf
(since PGCon some features was added).

Patch includes:
1 implementaion SRF_RETURN_NEXT_NULL()
2 contrib/hstore changes
3 docs of new hstore module (many thanks to David E. Wheeler
<david.wheeler@pgexperts.com>)

In current state patch is in WIP status, for short period I plan to
move support of binary nested structure to core to share binary
representation for hstore and json types.

Thanks, Teodor.

As soon as we have that shared binary representation available, I will
be working on adapting it to JSON.

As I remember from earlier discussions, current json has some
artefacts that some people want to preserve and which are incompatible
with hstore approach where you have actual object behind the serialisation.

I remember strong voices in support of *not* normalising json, so that
things like

{"a":1,"a":true, "a":"b", "a":none}

would go through the system unaltered, for claimed standard usage of
json as
"processing instructions". That is as source code which can possibly
converted
to JavaScript Object and not something that would come out of
serialising of
any existing JavaScript Object.

I suggest we add another type, maybe jsobj, which has input and output
as standard
"JSON" but which is defined from the start to be equivalent of existing
object
and not "preservable source code" to such object.

Cheers

--
Hannu Krosing
PostgreSQL Consultant
Performance, Scalability and High Availability
2ndQuadrant Nordic OÜ

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

#5David E. Wheeler
david@kineticode.com
In reply to: Hannu Krosing (#4)
Re: nested hstore patch

On Nov 13, 2013, at 3:59 PM, Hannu Krosing <hannu@2ndquadrant.com> wrote:

I remember strong voices in support of *not* normalising json, so that
things like

{"a":1,"a":true, "a":"b", "a":none}

would go through the system unaltered, for claimed standard usage of
json as
"processing instructions". That is as source code which can possibly
converted
to JavaScript Object and not something that would come out of
serialising of
any existing JavaScript Object.

My recollection from PGCon was that there was consensus to normalize on the way in -- or at least, if we switched to a binary representation as proposed by Oleg & Teodor, it was not worth the hassle to try to keep it.

I suggest we add another type, maybe jsobj, which has input and output
as standard
"JSON" but which is defined from the start to be equivalent of existing
object
and not "preservable source code" to such object.

-1 Let's try to keep this simple. See also VARCHAR and VARCHAR2 on Oracle.

Best,

David

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

#6Hannu Krosing
hannu@tm.ee
In reply to: David E. Wheeler (#5)
Re: nested hstore patch

On 11/14/2013 01:32 AM, David E. Wheeler wrote:

On Nov 13, 2013, at 3:59 PM, Hannu Krosing <hannu@2ndquadrant.com> wrote:

I remember strong voices in support of *not* normalising json, so that
things like

{"a":1,"a":true, "a":"b", "a":none}

would go through the system unaltered, for claimed standard usage of
json as
"processing instructions". That is as source code which can possibly
converted
to JavaScript Object and not something that would come out of
serialising of
any existing JavaScript Object.

My recollection from PGCon was that there was consensus to normalize on
the way in --

Great news! I remember advocating this approach in the mailing lists
but having been out-voted based on "current real-world usage out there" :)

or at least, if we switched to a binary representation as proposed by
Oleg & Teodor, it was not worth the hassle to try to keep it.

Very much agree. For the source code approach I'd recommend
text type with maybe a check that it is possible to convert it to json.

--
Hannu Krosing
PostgreSQL Consultant
Performance, Scalability and High Availability
2ndQuadrant Nordic O�

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

#7Andrew Dunstan
andrew@dunslane.net
In reply to: Hannu Krosing (#6)
Re: nested hstore patch

On 11/14/2013 03:21 AM, Hannu Krosing wrote:

On 11/14/2013 01:32 AM, David E. Wheeler wrote:

On Nov 13, 2013, at 3:59 PM, Hannu Krosing <hannu@2ndquadrant.com> wrote:

I remember strong voices in support of *not* normalising json, so that
things like

{"a":1,"a":true, "a":"b", "a":none}

would go through the system unaltered, for claimed standard usage of
json as
"processing instructions". That is as source code which can possibly
converted
to JavaScript Object and not something that would come out of
serialising of
any existing JavaScript Object.

My recollection from PGCon was that there was consensus to normalize on
the way in --

Great news! I remember advocating this approach in the mailing lists
but having been out-voted based on "current real-world usage out there" :)

or at least, if we switched to a binary representation as proposed by
Oleg & Teodor, it was not worth the hassle to try to keep it.

Very much agree. For the source code approach I'd recommend
text type with maybe a check that it is possible to convert it to json.

I don't think you and David are saying the same thing. AIUI he wants one
JSON type and is prepared to discard text preservation (duplicate keys
and key order). You want two json types, one of which would feature text
preservation.

Correct me if I'm wrong.

cheers

andrew

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

#8Hannu Krosing
hannu@tm.ee
In reply to: Andrew Dunstan (#7)
Re: nested hstore patch

On 11/14/2013 01:47 PM, Andrew Dunstan wrote:

On 11/14/2013 03:21 AM, Hannu Krosing wrote:

On 11/14/2013 01:32 AM, David E. Wheeler wrote:

On Nov 13, 2013, at 3:59 PM, Hannu Krosing <hannu@2ndquadrant.com>
wrote:

I remember strong voices in support of *not* normalising json, so that
things like

{"a":1,"a":true, "a":"b", "a":none}

would go through the system unaltered, for claimed standard usage of
json as
"processing instructions". That is as source code which can possibly
converted
to JavaScript Object and not something that would come out of
serialising of
any existing JavaScript Object.

My recollection from PGCon was that there was consensus to normalize on
the way in --

Great news! I remember advocating this approach in the mailing lists
but having been out-voted based on "current real-world usage out
there" :)

or at least, if we switched to a binary representation as proposed by
Oleg & Teodor, it was not worth the hassle to try to keep it.

Very much agree. For the source code approach I'd recommend
text type with maybe a check that it is possible to convert it to json.

I don't think you and David are saying the same thing. AIUI he wants
one JSON
type and is prepared to discard text preservation (duplicate keys and
key order).
You want two json types, one of which would feature text preservation.

I actually *want* the same thing that David wants, but I think that
Merlin has
valid concerns about backwards compatibility.

If we have let this behaviour in, it is not nice to break several uses
of it now.

If we could somehow turn "old json" into a text domain with json syntax
check
(which it really is up to 9.3) via pg_upgrade that would be great.

It would be the required for pg_dump to have some swicth to output
different typename in CREATE TABLE and similar.

Correct me if I'm wrong.

cheers

andrew

--
Hannu Krosing
PostgreSQL Consultant
Performance, Scalability and High Availability
2ndQuadrant Nordic O�

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

#9Andres Freund
andres@anarazel.de
In reply to: Teodor Sigaev (#1)
Re: nested hstore patch

Hi,

On 2013-11-12 22:35:31 +0400, Teodor Sigaev wrote:

Attatched patch adds nesting feature, types (string, boll and numeric
values), arrays and scalar to hstore type.

I took a quick peek at this:
* You cannot simply catch and ignore errors by doing
+	PG_TRY();
+	{
+		n = DatumGetNumeric(DirectFunctionCall3(numeric_in, CStringGetDatum(s->val), 0, -1));
+	}
+	PG_CATCH();
+	{
+		n = NULL;
+	}
+	PG_END_TRY();
 That skips cleanup and might ignore some errors (think memory allocation
 failures). But why do you even want to silently ignore errors there?
* Shouldn't the checks for v->size be done before filling the
  datastructures in makeHStoreValueArray() and makeHStoreValuePairs()?
* could you make ORDER_PAIRS() a function instead of a macro? It's
  pretty long and there's no reason not to use a function.
* You call numeric_recv via recvHStoreValue via recvHStore without
  checks on the input length. That seems - without having checked it in
  detail - a good way to read unrelated memory. Generally ISTM the input
  needs to be more carefully checked in the whole recv function.
* There's quite some new new, completely uncommented, code. Especially
  in hstore_op.c.
* the _PG_init you added should probably do a EmitWarningsOnPlaceholders().
* why does hstore need it's own atoi?
* shouldn't all the function prototypes be marked as externs?
* Lots of trailing whitespaces, quite some long lines, cuddly braces,
  ...
* I think hstore_compat.c's header should be updated.

Greetings,

Andres Freund

--
Andres Freund http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Training & Services

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

#10Bruce Momjian
bruce@momjian.us
In reply to: Hannu Krosing (#8)
Re: nested hstore patch

On Thu, Nov 14, 2013 at 05:50:02PM +0100, Hannu Krosing wrote:

If we could somehow turn "old json" into a text domain with json syntax
check
(which it really is up to 9.3) via pg_upgrade that would be great.

It would be the required for pg_dump to have some swicth to output
different typename in CREATE TABLE and similar.

I don't think pg_upgrade isn't in a position to handle this. I think it
would require a script to be run after pg_upgrade completes.

--
Bruce Momjian <bruce@momjian.us> http://momjian.us
EnterpriseDB http://enterprisedb.com

+ Everyone has their own god. +

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

#11Robert Haas
robertmhaas@gmail.com
In reply to: Hannu Krosing (#4)
Re: nested hstore patch

On Wed, Nov 13, 2013 at 6:59 PM, Hannu Krosing <hannu@2ndquadrant.com> wrote:

I remember strong voices in support of *not* normalising json, so that
things like

{"a":1,"a":true, "a":"b", "a":none}

would go through the system unaltered, for claimed standard usage of
json as
"processing instructions". That is as source code which can possibly
converted
to JavaScript Object and not something that would come out of
serialising of
any existing JavaScript Object.

Yeah, as the guy who wrote the original version of the JSON type,
which works just exactly like the XML type does, I stronly object to
changing the behavior. And doubly so now that it's released, as we
would be breaking backward compatibility.

I suggest we add another type, maybe jsobj, which has input and output
as standard
"JSON" but which is defined from the start to be equivalent of existing
object
and not "preservable source code" to such object.

I think this was the consensus solution when this was last discussed,
and I support it. There is similar space for a binary XML data type
if someone feels like implementing it. I think the names that were
proposed previously were something like jsonb and xmlb.

--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

#12Bruce Momjian
bruce@momjian.us
In reply to: Robert Haas (#11)
Re: nested hstore patch

On Tue, Nov 19, 2013 at 10:51:06AM -0500, Robert Haas wrote:

I think this was the consensus solution when this was last discussed,
and I support it. There is similar space for a binary XML data type
if someone feels like implementing it. I think the names that were
proposed previously were something like jsonb and xmlb.

The natural name is OBJSON, meaning object JSON, because as PostgreSQL
people, we have to double-use letters wherever possible. ;-)

--
Bruce Momjian <bruce@momjian.us> http://momjian.us
EnterpriseDB http://enterprisedb.com

+ Everyone has their own god. +

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

#13Andrew Dunstan
andrew@dunslane.net
In reply to: Robert Haas (#11)
Re: nested hstore patch

On 11/19/2013 10:51 AM, Robert Haas wrote:

I suggest we add another type, maybe jsobj, which has input and output
as standard
"JSON" but which is defined from the start to be equivalent of existing
object
and not "preservable source code" to such object.

I think this was the consensus solution when this was last discussed,
and I support it. There is similar space for a binary XML data type
if someone feels like implementing it. I think the names that were
proposed previously were something like jsonb and xmlb.

I think that's the consensus position on a strategy.

JSONB seems to be the current winner min the name stakes.

cheers

andrew

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

#14Robert Haas
robertmhaas@gmail.com
In reply to: Bruce Momjian (#12)
Re: nested hstore patch

On Tue, Nov 19, 2013 at 10:55 AM, Bruce Momjian <bruce@momjian.us> wrote:

On Tue, Nov 19, 2013 at 10:51:06AM -0500, Robert Haas wrote:

I think this was the consensus solution when this was last discussed,
and I support it. There is similar space for a binary XML data type
if someone feels like implementing it. I think the names that were
proposed previously were something like jsonb and xmlb.

The natural name is OBJSON, meaning object JSON, because as PostgreSQL
people, we have to double-use letters wherever possible. ;-)

Personally, I think the patch author should just run ps auxww | md5 |
sed 's/^[^0-9]//' | cut -c1-8 and call the new data type by the
resulting name.

--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

#15Andrew Dunstan
andrew@dunslane.net
In reply to: Robert Haas (#14)
Re: nested hstore patch

On 11/19/2013 11:00 AM, Robert Haas wrote:

On Tue, Nov 19, 2013 at 10:55 AM, Bruce Momjian <bruce@momjian.us> wrote:

On Tue, Nov 19, 2013 at 10:51:06AM -0500, Robert Haas wrote:

I think this was the consensus solution when this was last discussed,
and I support it. There is similar space for a binary XML data type
if someone feels like implementing it. I think the names that were
proposed previously were something like jsonb and xmlb.

The natural name is OBJSON, meaning object JSON, because as PostgreSQL
people, we have to double-use letters wherever possible. ;-)

Personally, I think the patch author should just run ps auxww | md5 |
sed 's/^[^0-9]//' | cut -c1-8 and call the new data type by the
resulting name.

My personal vote goes for "marmaduke". I've been dying to call some
builtin object that for ages.

cheers

andrew

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

#16Peter Eisentraut
peter_e@gmx.net
In reply to: Teodor Sigaev (#1)
Re: nested hstore patch

On 11/12/13, 1:35 PM, Teodor Sigaev wrote:

Hi!

Attatched patch adds nesting feature, types (string, boll and numeric
values), arrays and scalar to hstore type.

Documentation doesn't build:

openjade:hstore.sgml:206:16:E: document type does not allow element "VARLISTENTRY" here; assuming missing "VARIABLELIST" start-tag

Compiler warnings:

hstore_io.c: In function 'array_to_hstore':
hstore_io.c:1736:29: error: 'result' may be used uninitialized in this function [-Werror=maybe-uninitialized]

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

#17David E. Wheeler
david@kineticode.com
In reply to: Peter Eisentraut (#16)
Re: nested hstore patch

On Nov 20, 2013, at 6:19 AM, Peter Eisentraut <peter_e@gmx.net> wrote:

openjade:hstore.sgml:206:16:E: document type does not allow element "VARLISTENTRY" here; assuming missing "VARIABLELIST" start-tag

Thanks, I fixed this one.

David

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

#18David E. Wheeler
david@kineticode.com
In reply to: Teodor Sigaev (#1)
Re: nested hstore patch

On Nov 12, 2013, at 10:35 AM, Teodor Sigaev <teodor@sigaev.ru> wrote:

Hi!

Attatched patch adds nesting feature, types (string, boll and numeric values), arrays and scalar to hstore type.

My apologies for not getting to this sooner, work has been a bit nutty. The truth is that I reviewed this patch quite a bit a month back, mostly so I could write documentation, the results of which are included in this patch. And I'm super excited for what's to come in the next iteration, as I hear that Teodor and Andrew are hard at work adding jsonb as a binary-compatible JSON data type.

Meanwhile, for this version, a quick overview of what has changed since 9.2.

Contents & Purpose
==================

Improved Data Type Support
--------------------------

* Added data type support for values. Previously they could only be strings or NULL, but with this patch they can also be numbers or booleans.

* Added array support. Values can be arrays of other values. The format for arrays is a bracketed, comma-delimited list.

* Added nesting support. hstore values can themselves be hstores. Nested hstores are wrapped in braces, but the root-level hstore is not (for compatibility with the format of previous versions of hstore).

* An hstore value is no longer required to be an hstore object. It can now be any scalar value.

These three items make the basic format feature-complete with JSON. Here's an example where the values are scalars:

=% SELECT 'foo'::hstore, '"hi \"bob\""'::hstore, '1.0'::hstore, 'true'::hstore, NULL::hstore;
hstore | hstore | hstore | hstore | hstore
--------+--------------+--------+--------+--------
"foo" | "hi \"bob\"" | 1.0 | t |

And here are a couple of arrays with strings, numbers, booleans, and NULLs:

SELECT '[k,v]'::hstore, '[1.0, "hi there", false, null]'::hstore;
hstore | hstore
------------+----------------------------
["k", "v"] | [1.0, "hi there", f, NULL]

Here's a complicated example formatted with `hstore.pretty_print` enabled.

=% SET hstore.pretty_print=true;
=% SELECT '{
"type" => "Feature",
"bbox" => [-180.0, -90.0, 180.0, 90.0],
"geometry" => {
"type" => "Polygon",
"coordinates" => [[
[-180.0, 10.0], [20.0, 90.0], [180.0, -5.0], [-30.0, -90.0]
]]
}
}'::hstore;
hstore
--------------------------
"bbox"=> +
[ +
-180.0, +
-90.0, +
180.0, +
90.0 +
], +
"type"=>"Feature", +
"geometry"=> +
{ +
"type"=>"Polygon", +
"coordinates"=> +
[ +
[ +
[ +
-180.0, +
10.0 +
], +
[ +
20.0, +
90.0 +
], +
[ +
180.0, +
-5.0 +
], +
[ +
-30.0, +
-90.0 +
] +
] +
] +
}

So, exact feature parity with the JSON data type.

* hstore.pretty_print is a new GUC, specifically to allow an HSTORE value to be pretty-printed. There is also a function to pretty-print, so we might be able to just do away with the GUC.

Interface
---------

* New operators:
  + `hstore -> int`:     Get string value at array index (starting at 0)
  + `hstore ^> text`:    Get numeric value for key
  + `hstore ^> int`:     Get numeric value at array index
  + `hstore ?> text`:    Get boolean value for key
  + `hstore ?> int`:     Get boolean value at array index
  + `hstore #> text[]`:  Get string value for key path
  + `hstore #^> text[]`: Get numeric value for key path
  + `hstore #?> text[]`: Get boolean value for key path
  + `hstore %> text`:    Get hstore value for key
  + `hstore %> int`:     Get hstore value at array index
  + `hstore #%> text[]`: Get hstore value for key path
  + `hstore ? int`:      Does hstore contain array index
  + `hstore #? text[]`:  Does hstore contain key path
  + `hstore - int`:      Delete index from left operand
  + `hstore #- text[]`:  Delete key path from left operand
* New functions:
  + `hstore(text)`:             Make a text scalar hstore
  + `hstore(numeric)`:          Make a numeric scalar hstore
  + `hstore(boolean)`:          Make a boolean scalar hstore
  + `hstore(text, hstore)`:     Make a nested hstore
  + `hstore(text, numeric)`:    Make an hstore with a key and numeric value
  + `hstore(text, boolean)`:    Make an hstore with a key and boolean value
  + `array_to_hstore(anyarray): Make an array hstore from an SQL array
  + `hvals(hstore)`             Get values as a set of hstore values
  + `json_to_hstore(json)`      Convert JSON to hstore
  + `each_hstore(hstore)`       Get set of hstore key/value pairs
  + `hstore_typeof(hstore)`     Return text name for the hstore type (hash, array, text, numeric, etc.)
  + `replace(hstore,text[],hstore)`:     Replace value at specified path
  + `concat_path(hstore,text[],hstore)`: Concatenate hstore value at specified path
  + `hstore_print(hstore, params)`:      Format hstore as text

The hstore_print() function has a number of optional boolean parameters to affect how the resulting text is formatted. They all default to false:

- pretty_print
- array_curly_braces: use {} instead of [] for arrays
- root_hash_decorated: Use {} for the root hash
- json: Format as JSON
- loose: Try to parse numbers and booleans from text values

Other Changes
-------------

* New casts: JSON and HSTORE can be cast to each other. I don't think they're implicit, though the forthcoming jsonb data type might support explicit casting to and from hstore, since internally they will be identical.

* The internal representation has been changed, but should be backward (and pg_upgrade) compatible, just as Andrew Gierth's change from 8.4 to 9.0 was. One can do an in-place update to rewrite all records at once. Of course, nested and/or non-hash hstore values dumped from 9.4 will not be able to be loaded into 9.3.

* GIN indexing is now supported. This is actually pretty amazing. For an hstore value, even hash keys are considered values, as far as the index is concerned. This makes it efficient to find hstore values that contain a key. I wrote an example in this blog post:

http://theory.so/pg/2013/10/25/indexing-nested-hstore/

Submission review
=================
* Is the patch in a patch format which has context? Yes.
* Does it apply cleanly to the current git master? It did for me, though I think Peter has found an issue or two since.
* Does it include reasonable tests, necessary doc patches, etc? Yes.

Usability review
================

* Does the patch actually implement what it says it does? Yes.
* Do we want that? OH yes.
* Do we already have it? No.
* Does it follow SQL spec, or the community-agreed behavior? Yes, though want jsonb, too.
* Does it include pg_dump support? Yes
* Are there dangers? Could break backward compatibility, though I don't think it does.
* Have all the bases been covered? I think so

Feature test
============

* Does the feature work as advertised? Yes.
* Are there corner cases the author has failed to consider? All I noticed were promptly fixed.
* Are there any assertion failures or crashes? No

Performance review
==================

* Does the patch slow down simple tests? No
* If it claims to improve performance, does it? Yes, with GIN index support. Loading hstore values is slower than loading JSON, but everything else is faster than JSON.
* Does it slow down other things? No.

Coding review
=============

* Does it follow the project guidelines? Yes.
* Are there portability issues? Unknown
* Will it work on Windows/BSD etc? Tested on OS X only.
* Are the comments sufficient and accurate? Yes.
* Does it do what it says, correctly? As best I can tell, yes.
* Does it produce compiler warnings? No.
* Can you make it crash? No, but I did find a bug or two that was promptly fixed.

Architecture review
===================

* Is everything done in a way that fits together coherently with other features/modules? yes.
* Are there interdependencies that can cause problems? No

Conclusion
==========

I love where nested hstore is going, especially since it will be used for jsonb, too. The nesting, data type, and GIN index support is really great, and the new constructors provide a nice SQL API that make it easy to use. I think that the next version of this patch will be full of win for the project.

This was considered a WIP patch, since the jsonb support is still forthcoming, so it's appropriate to leave it marked “Returned with feedback”. As Andrew is doing much of that work, the code itself will get a much closer examination from him. But for the hstore feature itself, I think the current interface and features are ready to go.

Best,

David

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

#19Andres Freund
andres@anarazel.de
In reply to: David E. Wheeler (#18)
Re: nested hstore patch

On 2013-12-20 15:16:30 -0800, David E. Wheeler wrote:

But for the hstore feature itself, I think the current interface and features are ready to go.

I think this patch needs significant amount of work because it can be
considered ready for committer. I found the list of issues in
http://archives.postgresql.org/message-id/20131118163633.GE20305%40awork2.anarazel.de
within 10 minutes, indicating that it clearly cannot be ready yet.

Greetings,

Andres Freund

--
Andres Freund http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Training & Services

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

#20Robert Haas
robertmhaas@gmail.com
In reply to: David E. Wheeler (#18)
Re: nested hstore patch

On Fri, Dec 20, 2013 at 6:16 PM, David E. Wheeler <david@justatheory.com> wrote:

* New operators:
+ `hstore -> int`:     Get string value at array index (starting at 0)
+ `hstore ^> text`:    Get numeric value for key
+ `hstore ^> int`:     Get numeric value at array index
+ `hstore ?> text`:    Get boolean value for key
+ `hstore ?> int`:     Get boolean value at array index
+ `hstore #> text[]`:  Get string value for key path
+ `hstore #^> text[]`: Get numeric value for key path
+ `hstore #?> text[]`: Get boolean value for key path
+ `hstore %> text`:    Get hstore value for key
+ `hstore %> int`:     Get hstore value at array index
+ `hstore #%> text[]`: Get hstore value for key path
+ `hstore ? int`:      Does hstore contain array index
+ `hstore #? text[]`:  Does hstore contain key path
+ `hstore - int`:      Delete index from left operand
+ `hstore #- text[]`:  Delete key path from left operand

Although in some ways there's a certain elegance to this, it also
sorta looks like punctuation soup. I can't help wondering whether
we'd be better off sticking to function names.

--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

#21Hannu Krosing
hannu@tm.ee
In reply to: Robert Haas (#20)
#22Pavel Stehule
pavel.stehule@gmail.com
In reply to: Hannu Krosing (#21)
#23Jonathan S. Katz
jonathan.katz@excoventures.com
In reply to: Robert Haas (#20)
#24Oleg Bartunov
oleg@sai.msu.su
In reply to: Andres Freund (#9)
#25Erik Rijkers
er@xs4all.nl
In reply to: Oleg Bartunov (#24)
#26Jim Nasby
Jim.Nasby@BlueTreble.com
In reply to: Pavel Stehule (#22)
#27Andrew Dunstan
andrew@dunslane.net
In reply to: Oleg Bartunov (#24)
#28Josh Berkus
josh@agliodbs.com
In reply to: Teodor Sigaev (#1)
#29Andrew Dunstan
andrew@dunslane.net
In reply to: Josh Berkus (#28)
#30Oleg Bartunov
oleg@sai.msu.su
In reply to: Andrew Dunstan (#29)
#31Robert Haas
robertmhaas@gmail.com
In reply to: Andrew Dunstan (#29)
#32Andrew Dunstan
andrew@dunslane.net
In reply to: Robert Haas (#31)
#33Peter Eisentraut
peter_e@gmx.net
In reply to: Andrew Dunstan (#29)
#34Erik Rijkers
er@xs4all.nl
In reply to: Peter Eisentraut (#33)
#35Andrew Dunstan
andrew@dunslane.net
In reply to: Erik Rijkers (#34)
#36David E. Wheeler
david@kineticode.com
In reply to: Andrew Dunstan (#35)
#37Erik Rijkers
er@xs4all.nl
In reply to: Andrew Dunstan (#35)
#38Erik Rijkers
er@xs4all.nl
In reply to: Erik Rijkers (#37)
#39Oleg Bartunov
oleg@sai.msu.su
In reply to: Erik Rijkers (#38)
#40Andrew Dunstan
andrew@dunslane.net
In reply to: Erik Rijkers (#38)
#41Alvaro Herrera
alvherre@2ndquadrant.com
In reply to: Andrew Dunstan (#40)
#42Oleg Bartunov
oleg@sai.msu.su
In reply to: Andrew Dunstan (#40)
#43Andrew Dunstan
andrew@dunslane.net
In reply to: Alvaro Herrera (#41)
#44Andrew Dunstan
andrew@dunslane.net
In reply to: Oleg Bartunov (#42)
#45Erik Rijkers
er@xs4all.nl
In reply to: Andrew Dunstan (#44)
#46Oleg Bartunov
oleg@sai.msu.su
In reply to: Erik Rijkers (#45)
#47Erik Rijkers
er@xs4all.nl
In reply to: Andrew Dunstan (#40)
#48Oleg Bartunov
oleg@sai.msu.su
In reply to: Erik Rijkers (#47)
#49Erik Rijkers
er@xs4all.nl
In reply to: Oleg Bartunov (#48)
#50Oleg Bartunov
oleg@sai.msu.su
In reply to: Erik Rijkers (#49)
#51Erik Rijkers
er@xs4all.nl
In reply to: Oleg Bartunov (#50)
#52Andrew Dunstan
andrew@dunslane.net
In reply to: Erik Rijkers (#51)
#53Andrew Dunstan
andrew@dunslane.net
In reply to: Andrew Dunstan (#52)