Function array_agg(array)
Started by Ali Akbarover 11 years ago40 messages
Greetings,
While looking for easier items in PostgreSQL Wiki's Todo List (this will be
my 3rd patch), i found this TODO:
Add a built-in array_agg(anyarray) or similar, that can aggregate
1-dimensional arrays into a 2-dimensional array.
I've stumbled by this lack of array_agg(anyarray) sometimes ago in my work,
so i decided to explore this.
Currently, if we try array_agg(anyarray), PostgreSQL behaves like this:
# select array_agg('{1,2}'::int[]);
ERROR: could not find array type for data type integer[]
Reading implementation of array_agg, it looks like the array_agg function
is generic, and can process any input. The error comes from PostgreSQL not
finding array type for int[] (_int4 in pg_proc).
In PostgreSQL, any array is multidimensional, array type for any array is
the same:
- the type of {1,2} is int[]
- {{1,2}, {3,4}} is int[]
- {{{1},{2}, {3} ,{4}}} is still int[]
So, can't we just set the typarray of array types to its self oid? (patch
attached). So far:
- the array_agg is working and returning correct types:
backend> select array_agg('{1,2}'::int[]);
1: array_agg (typeid = 1007, len = -1, typmod = -1, byval = f)
----
1: array_agg = "{"{1,2}"}" (typeid = 1007, len = -1, typmod = -1,
byval = f)
----
select array_agg('{''a'',''b''}'::varchar[]);
1: array_agg (typeid = 1015, len = -1, typmod = -1, byval = f)
----
1: array_agg = "{"{'a','b'}"}" (typeid = 1015, len = -1, typmod =
-1, byval = f)
----
- Regression tests passed except for the pg_type sanity check while
checking typelem relation with typarray:
SELECT p1.oid, p1.typname as basetype, p2.typname as arraytype,
p2.typelem, p2.typlen
FROM pg_type p1 LEFT JOIN pg_type p2 ON (p1.typarray = p2.oid)
WHERE p1.typarray <> 0 AND
(p2.oid IS NULL OR p2.typelem <> p1.oid OR p2.typlen <> -1);
! oid | basetype | arraytype | typelem | typlen
! ------+----------------+----------------+---------+--------
! 143 | _xml | _xml | 142 | -1
! 199 | _json | _json | 114 | -1
! 629 | _line | _line | 628 | -1
! 719 | _circle | _circle | 718 | -1
... (cut)
Aside from the sanity check complaints, I don't see any problems in the
resulting array operations.
So, back to the question: Can't we just set the typarray of array types to
its self oid?
Regards,
--
Ali Akbar
Attachments:
array_agg_anyarray-1.patchtext/x-diff; charset=US-ASCII; name=array_agg_anyarray-1.patchDownload
diff --git a/src/include/catalog/pg_type.h b/src/include/catalog/pg_type.h
index b7d9256..7934874 100644
--- a/src/include/catalog/pg_type.h
+++ b/src/include/catalog/pg_type.h
@@ -357,8 +357,8 @@ DATA(insert OID = 114 ( json PGNSP PGUID -1 f b U f t \054 0 0 199 json_in j
DATA(insert OID = 142 ( xml PGNSP PGUID -1 f b U f t \054 0 0 143 xml_in xml_out xml_recv xml_send - - - i x f 0 -1 0 0 _null_ _null_ _null_ ));
DESCR("XML content");
#define XMLOID 142
-DATA(insert OID = 143 ( _xml PGNSP PGUID -1 f b A f t \054 0 142 0 array_in array_out array_recv array_send - - array_typanalyze i x f 0 -1 0 0 _null_ _null_ _null_ ));
-DATA(insert OID = 199 ( _json PGNSP PGUID -1 f b A f t \054 0 114 0 array_in array_out array_recv array_send - - array_typanalyze i x f 0 -1 0 0 _null_ _null_ _null_ ));
+DATA(insert OID = 143 ( _xml PGNSP PGUID -1 f b A f t \054 0 142 143 array_in array_out array_recv array_send - - array_typanalyze i x f 0 -1 0 0 _null_ _null_ _null_ ));
+DATA(insert OID = 199 ( _json PGNSP PGUID -1 f b A f t \054 0 114 199 array_in array_out array_recv array_send - - array_typanalyze i x f 0 -1 0 0 _null_ _null_ _null_ ));
DATA(insert OID = 194 ( pg_node_tree PGNSP PGUID -1 f b S f t \054 0 0 0 pg_node_tree_in pg_node_tree_out pg_node_tree_recv pg_node_tree_send - - - i x f 0 -1 0 100 _null_ _null_ _null_ ));
DESCR("string representing an internal node tree");
@@ -395,7 +395,7 @@ DESCR("geometric polygon '(pt1,...)'");
DATA(insert OID = 628 ( line PGNSP PGUID 24 f b G f t \054 0 701 629 line_in line_out line_recv line_send - - - d p f 0 -1 0 0 _null_ _null_ _null_ ));
DESCR("geometric line");
#define LINEOID 628
-DATA(insert OID = 629 ( _line PGNSP PGUID -1 f b A f t \054 0 628 0 array_in array_out array_recv array_send - - array_typanalyze d x f 0 -1 0 0 _null_ _null_ _null_ ));
+DATA(insert OID = 629 ( _line PGNSP PGUID -1 f b A f t \054 0 628 629 array_in array_out array_recv array_send - - array_typanalyze d x f 0 -1 0 0 _null_ _null_ _null_ ));
/* OIDS 700 - 799 */
@@ -421,11 +421,11 @@ DESCR("");
DATA(insert OID = 718 ( circle PGNSP PGUID 24 f b G f t \054 0 0 719 circle_in circle_out circle_recv circle_send - - - d p f 0 -1 0 0 _null_ _null_ _null_ ));
DESCR("geometric circle '(center,radius)'");
#define CIRCLEOID 718
-DATA(insert OID = 719 ( _circle PGNSP PGUID -1 f b A f t \054 0 718 0 array_in array_out array_recv array_send - - array_typanalyze d x f 0 -1 0 0 _null_ _null_ _null_ ));
+DATA(insert OID = 719 ( _circle PGNSP PGUID -1 f b A f t \054 0 718 719 array_in array_out array_recv array_send - - array_typanalyze d x f 0 -1 0 0 _null_ _null_ _null_ ));
DATA(insert OID = 790 ( money PGNSP PGUID 8 FLOAT8PASSBYVAL b N f t \054 0 0 791 cash_in cash_out cash_recv cash_send - - - d p f 0 -1 0 0 _null_ _null_ _null_ ));
DESCR("monetary amounts, $d,ddd.cc");
#define CASHOID 790
-DATA(insert OID = 791 ( _money PGNSP PGUID -1 f b A f t \054 0 790 0 array_in array_out array_recv array_send - - array_typanalyze d x f 0 -1 0 0 _null_ _null_ _null_ ));
+DATA(insert OID = 791 ( _money PGNSP PGUID -1 f b A f t \054 0 790 791 array_in array_out array_recv array_send - - array_typanalyze d x f 0 -1 0 0 _null_ _null_ _null_ ));
/* OIDS 800 - 899 */
DATA(insert OID = 829 ( macaddr PGNSP PGUID 6 f b U f t \054 0 0 1040 macaddr_in macaddr_out macaddr_recv macaddr_send - - - i p f 0 -1 0 0 _null_ _null_ _null_ ));
@@ -441,46 +441,46 @@ DESCR("network IP address/netmask, network address");
/* OIDS 900 - 999 */
/* OIDS 1000 - 1099 */
-DATA(insert OID = 1000 ( _bool PGNSP PGUID -1 f b A f t \054 0 16 0 array_in array_out array_recv array_send - - array_typanalyze i x f 0 -1 0 0 _null_ _null_ _null_ ));
-DATA(insert OID = 1001 ( _bytea PGNSP PGUID -1 f b A f t \054 0 17 0 array_in array_out array_recv array_send - - array_typanalyze i x f 0 -1 0 0 _null_ _null_ _null_ ));
-DATA(insert OID = 1002 ( _char PGNSP PGUID -1 f b A f t \054 0 18 0 array_in array_out array_recv array_send - - array_typanalyze i x f 0 -1 0 0 _null_ _null_ _null_ ));
-DATA(insert OID = 1003 ( _name PGNSP PGUID -1 f b A f t \054 0 19 0 array_in array_out array_recv array_send - - array_typanalyze i x f 0 -1 0 0 _null_ _null_ _null_ ));
-DATA(insert OID = 1005 ( _int2 PGNSP PGUID -1 f b A f t \054 0 21 0 array_in array_out array_recv array_send - - array_typanalyze i x f 0 -1 0 0 _null_ _null_ _null_ ));
+DATA(insert OID = 1000 ( _bool PGNSP PGUID -1 f b A f t \054 0 16 1000 array_in array_out array_recv array_send - - array_typanalyze i x f 0 -1 0 0 _null_ _null_ _null_ ));
+DATA(insert OID = 1001 ( _bytea PGNSP PGUID -1 f b A f t \054 0 17 1001 array_in array_out array_recv array_send - - array_typanalyze i x f 0 -1 0 0 _null_ _null_ _null_ ));
+DATA(insert OID = 1002 ( _char PGNSP PGUID -1 f b A f t \054 0 18 1002 array_in array_out array_recv array_send - - array_typanalyze i x f 0 -1 0 0 _null_ _null_ _null_ ));
+DATA(insert OID = 1003 ( _name PGNSP PGUID -1 f b A f t \054 0 19 1003 array_in array_out array_recv array_send - - array_typanalyze i x f 0 -1 0 0 _null_ _null_ _null_ ));
+DATA(insert OID = 1005 ( _int2 PGNSP PGUID -1 f b A f t \054 0 21 1005 array_in array_out array_recv array_send - - array_typanalyze i x f 0 -1 0 0 _null_ _null_ _null_ ));
#define INT2ARRAYOID 1005
-DATA(insert OID = 1006 ( _int2vector PGNSP PGUID -1 f b A f t \054 0 22 0 array_in array_out array_recv array_send - - array_typanalyze i x f 0 -1 0 0 _null_ _null_ _null_ ));
-DATA(insert OID = 1007 ( _int4 PGNSP PGUID -1 f b A f t \054 0 23 0 array_in array_out array_recv array_send - - array_typanalyze i x f 0 -1 0 0 _null_ _null_ _null_ ));
+DATA(insert OID = 1006 ( _int2vector PGNSP PGUID -1 f b A f t \054 0 22 1006 array_in array_out array_recv array_send - - array_typanalyze i x f 0 -1 0 0 _null_ _null_ _null_ ));
+DATA(insert OID = 1007 ( _int4 PGNSP PGUID -1 f b A f t \054 0 23 1007 array_in array_out array_recv array_send - - array_typanalyze i x f 0 -1 0 0 _null_ _null_ _null_ ));
#define INT4ARRAYOID 1007
-DATA(insert OID = 1008 ( _regproc PGNSP PGUID -1 f b A f t \054 0 24 0 array_in array_out array_recv array_send - - array_typanalyze i x f 0 -1 0 0 _null_ _null_ _null_ ));
-DATA(insert OID = 1009 ( _text PGNSP PGUID -1 f b A f t \054 0 25 0 array_in array_out array_recv array_send - - array_typanalyze i x f 0 -1 0 100 _null_ _null_ _null_ ));
+DATA(insert OID = 1008 ( _regproc PGNSP PGUID -1 f b A f t \054 0 24 1008 array_in array_out array_recv array_send - - array_typanalyze i x f 0 -1 0 0 _null_ _null_ _null_ ));
+DATA(insert OID = 1009 ( _text PGNSP PGUID -1 f b A f t \054 0 25 1009 array_in array_out array_recv array_send - - array_typanalyze i x f 0 -1 0 100 _null_ _null_ _null_ ));
#define TEXTARRAYOID 1009
-DATA(insert OID = 1028 ( _oid PGNSP PGUID -1 f b A f t \054 0 26 0 array_in array_out array_recv array_send - - array_typanalyze i x f 0 -1 0 0 _null_ _null_ _null_ ));
+DATA(insert OID = 1028 ( _oid PGNSP PGUID -1 f b A f t \054 0 26 1028 array_in array_out array_recv array_send - - array_typanalyze i x f 0 -1 0 0 _null_ _null_ _null_ ));
#define OIDARRAYOID 1028
-DATA(insert OID = 1010 ( _tid PGNSP PGUID -1 f b A f t \054 0 27 0 array_in array_out array_recv array_send - - array_typanalyze i x f 0 -1 0 0 _null_ _null_ _null_ ));
-DATA(insert OID = 1011 ( _xid PGNSP PGUID -1 f b A f t \054 0 28 0 array_in array_out array_recv array_send - - array_typanalyze i x f 0 -1 0 0 _null_ _null_ _null_ ));
-DATA(insert OID = 1012 ( _cid PGNSP PGUID -1 f b A f t \054 0 29 0 array_in array_out array_recv array_send - - array_typanalyze i x f 0 -1 0 0 _null_ _null_ _null_ ));
-DATA(insert OID = 1013 ( _oidvector PGNSP PGUID -1 f b A f t \054 0 30 0 array_in array_out array_recv array_send - - array_typanalyze i x f 0 -1 0 0 _null_ _null_ _null_ ));
-DATA(insert OID = 1014 ( _bpchar PGNSP PGUID -1 f b A f t \054 0 1042 0 array_in array_out array_recv array_send bpchartypmodin bpchartypmodout array_typanalyze i x f 0 -1 0 100 _null_ _null_ _null_ ));
-DATA(insert OID = 1015 ( _varchar PGNSP PGUID -1 f b A f t \054 0 1043 0 array_in array_out array_recv array_send varchartypmodin varchartypmodout array_typanalyze i x f 0 -1 0 100 _null_ _null_ _null_ ));
-DATA(insert OID = 1016 ( _int8 PGNSP PGUID -1 f b A f t \054 0 20 0 array_in array_out array_recv array_send - - array_typanalyze d x f 0 -1 0 0 _null_ _null_ _null_ ));
-DATA(insert OID = 1017 ( _point PGNSP PGUID -1 f b A f t \054 0 600 0 array_in array_out array_recv array_send - - array_typanalyze d x f 0 -1 0 0 _null_ _null_ _null_ ));
-DATA(insert OID = 1018 ( _lseg PGNSP PGUID -1 f b A f t \054 0 601 0 array_in array_out array_recv array_send - - array_typanalyze d x f 0 -1 0 0 _null_ _null_ _null_ ));
-DATA(insert OID = 1019 ( _path PGNSP PGUID -1 f b A f t \054 0 602 0 array_in array_out array_recv array_send - - array_typanalyze d x f 0 -1 0 0 _null_ _null_ _null_ ));
-DATA(insert OID = 1020 ( _box PGNSP PGUID -1 f b A f t \073 0 603 0 array_in array_out array_recv array_send - - array_typanalyze d x f 0 -1 0 0 _null_ _null_ _null_ ));
-DATA(insert OID = 1021 ( _float4 PGNSP PGUID -1 f b A f t \054 0 700 0 array_in array_out array_recv array_send - - array_typanalyze i x f 0 -1 0 0 _null_ _null_ _null_ ));
+DATA(insert OID = 1010 ( _tid PGNSP PGUID -1 f b A f t \054 0 27 1010 array_in array_out array_recv array_send - - array_typanalyze i x f 0 -1 0 0 _null_ _null_ _null_ ));
+DATA(insert OID = 1011 ( _xid PGNSP PGUID -1 f b A f t \054 0 28 1011 array_in array_out array_recv array_send - - array_typanalyze i x f 0 -1 0 0 _null_ _null_ _null_ ));
+DATA(insert OID = 1012 ( _cid PGNSP PGUID -1 f b A f t \054 0 29 1012 array_in array_out array_recv array_send - - array_typanalyze i x f 0 -1 0 0 _null_ _null_ _null_ ));
+DATA(insert OID = 1013 ( _oidvector PGNSP PGUID -1 f b A f t \054 0 30 1013 array_in array_out array_recv array_send - - array_typanalyze i x f 0 -1 0 0 _null_ _null_ _null_ ));
+DATA(insert OID = 1014 ( _bpchar PGNSP PGUID -1 f b A f t \054 0 1042 1014 array_in array_out array_recv array_send bpchartypmodin bpchartypmodout array_typanalyze i x f 0 -1 0 100 _null_ _null_ _null_ ));
+DATA(insert OID = 1015 ( _varchar PGNSP PGUID -1 f b A f t \054 0 1043 1015 array_in array_out array_recv array_send varchartypmodin varchartypmodout array_typanalyze i x f 0 -1 0 100 _null_ _null_ _null_ ));
+DATA(insert OID = 1016 ( _int8 PGNSP PGUID -1 f b A f t \054 0 20 1016 array_in array_out array_recv array_send - - array_typanalyze d x f 0 -1 0 0 _null_ _null_ _null_ ));
+DATA(insert OID = 1017 ( _point PGNSP PGUID -1 f b A f t \054 0 600 1017 array_in array_out array_recv array_send - - array_typanalyze d x f 0 -1 0 0 _null_ _null_ _null_ ));
+DATA(insert OID = 1018 ( _lseg PGNSP PGUID -1 f b A f t \054 0 601 1018 array_in array_out array_recv array_send - - array_typanalyze d x f 0 -1 0 0 _null_ _null_ _null_ ));
+DATA(insert OID = 1019 ( _path PGNSP PGUID -1 f b A f t \054 0 602 1019 array_in array_out array_recv array_send - - array_typanalyze d x f 0 -1 0 0 _null_ _null_ _null_ ));
+DATA(insert OID = 1020 ( _box PGNSP PGUID -1 f b A f t \073 0 603 1020 array_in array_out array_recv array_send - - array_typanalyze d x f 0 -1 0 0 _null_ _null_ _null_ ));
+DATA(insert OID = 1021 ( _float4 PGNSP PGUID -1 f b A f t \054 0 700 1021 array_in array_out array_recv array_send - - array_typanalyze i x f 0 -1 0 0 _null_ _null_ _null_ ));
#define FLOAT4ARRAYOID 1021
-DATA(insert OID = 1022 ( _float8 PGNSP PGUID -1 f b A f t \054 0 701 0 array_in array_out array_recv array_send - - array_typanalyze d x f 0 -1 0 0 _null_ _null_ _null_ ));
-DATA(insert OID = 1023 ( _abstime PGNSP PGUID -1 f b A f t \054 0 702 0 array_in array_out array_recv array_send - - array_typanalyze i x f 0 -1 0 0 _null_ _null_ _null_ ));
-DATA(insert OID = 1024 ( _reltime PGNSP PGUID -1 f b A f t \054 0 703 0 array_in array_out array_recv array_send - - array_typanalyze i x f 0 -1 0 0 _null_ _null_ _null_ ));
-DATA(insert OID = 1025 ( _tinterval PGNSP PGUID -1 f b A f t \054 0 704 0 array_in array_out array_recv array_send - - array_typanalyze i x f 0 -1 0 0 _null_ _null_ _null_ ));
-DATA(insert OID = 1027 ( _polygon PGNSP PGUID -1 f b A f t \054 0 604 0 array_in array_out array_recv array_send - - array_typanalyze d x f 0 -1 0 0 _null_ _null_ _null_ ));
+DATA(insert OID = 1022 ( _float8 PGNSP PGUID -1 f b A f t \054 0 701 1022 array_in array_out array_recv array_send - - array_typanalyze d x f 0 -1 0 0 _null_ _null_ _null_ ));
+DATA(insert OID = 1023 ( _abstime PGNSP PGUID -1 f b A f t \054 0 702 1023 array_in array_out array_recv array_send - - array_typanalyze i x f 0 -1 0 0 _null_ _null_ _null_ ));
+DATA(insert OID = 1024 ( _reltime PGNSP PGUID -1 f b A f t \054 0 703 1024 array_in array_out array_recv array_send - - array_typanalyze i x f 0 -1 0 0 _null_ _null_ _null_ ));
+DATA(insert OID = 1025 ( _tinterval PGNSP PGUID -1 f b A f t \054 0 704 1025 array_in array_out array_recv array_send - - array_typanalyze i x f 0 -1 0 0 _null_ _null_ _null_ ));
+DATA(insert OID = 1027 ( _polygon PGNSP PGUID -1 f b A f t \054 0 604 1027 array_in array_out array_recv array_send - - array_typanalyze d x f 0 -1 0 0 _null_ _null_ _null_ ));
DATA(insert OID = 1033 ( aclitem PGNSP PGUID 12 f b U f t \054 0 0 1034 aclitemin aclitemout - - - - - i p f 0 -1 0 0 _null_ _null_ _null_ ));
DESCR("access control list");
#define ACLITEMOID 1033
-DATA(insert OID = 1034 ( _aclitem PGNSP PGUID -1 f b A f t \054 0 1033 0 array_in array_out array_recv array_send - - array_typanalyze i x f 0 -1 0 0 _null_ _null_ _null_ ));
-DATA(insert OID = 1040 ( _macaddr PGNSP PGUID -1 f b A f t \054 0 829 0 array_in array_out array_recv array_send - - array_typanalyze i x f 0 -1 0 0 _null_ _null_ _null_ ));
-DATA(insert OID = 1041 ( _inet PGNSP PGUID -1 f b A f t \054 0 869 0 array_in array_out array_recv array_send - - array_typanalyze i x f 0 -1 0 0 _null_ _null_ _null_ ));
-DATA(insert OID = 651 ( _cidr PGNSP PGUID -1 f b A f t \054 0 650 0 array_in array_out array_recv array_send - - array_typanalyze i x f 0 -1 0 0 _null_ _null_ _null_ ));
-DATA(insert OID = 1263 ( _cstring PGNSP PGUID -1 f b A f t \054 0 2275 0 array_in array_out array_recv array_send - - array_typanalyze i x f 0 -1 0 0 _null_ _null_ _null_ ));
+DATA(insert OID = 1034 ( _aclitem PGNSP PGUID -1 f b A f t \054 0 1033 1034 array_in array_out array_recv array_send - - array_typanalyze i x f 0 -1 0 0 _null_ _null_ _null_ ));
+DATA(insert OID = 1040 ( _macaddr PGNSP PGUID -1 f b A f t \054 0 829 1040 array_in array_out array_recv array_send - - array_typanalyze i x f 0 -1 0 0 _null_ _null_ _null_ ));
+DATA(insert OID = 1041 ( _inet PGNSP PGUID -1 f b A f t \054 0 869 1041 array_in array_out array_recv array_send - - array_typanalyze i x f 0 -1 0 0 _null_ _null_ _null_ ));
+DATA(insert OID = 651 ( _cidr PGNSP PGUID -1 f b A f t \054 0 650 651 array_in array_out array_recv array_send - - array_typanalyze i x f 0 -1 0 0 _null_ _null_ _null_ ));
+DATA(insert OID = 1263 ( _cstring PGNSP PGUID -1 f b A f t \054 0 2275 1263 array_in array_out array_recv array_send - - array_typanalyze i x f 0 -1 0 0 _null_ _null_ _null_ ));
#define CSTRINGARRAYOID 1263
DATA(insert OID = 1042 ( bpchar PGNSP PGUID -1 f b S f t \054 0 0 1014 bpcharin bpcharout bpcharrecv bpcharsend bpchartypmodin bpchartypmodout - i x f 0 -1 0 100 _null_ _null_ _null_ ));
@@ -501,34 +501,34 @@ DESCR("time of day");
DATA(insert OID = 1114 ( timestamp PGNSP PGUID 8 FLOAT8PASSBYVAL b D f t \054 0 0 1115 timestamp_in timestamp_out timestamp_recv timestamp_send timestamptypmodin timestamptypmodout - d p f 0 -1 0 0 _null_ _null_ _null_ ));
DESCR("date and time");
#define TIMESTAMPOID 1114
-DATA(insert OID = 1115 ( _timestamp PGNSP PGUID -1 f b A f t \054 0 1114 0 array_in array_out array_recv array_send timestamptypmodin timestamptypmodout array_typanalyze d x f 0 -1 0 0 _null_ _null_ _null_ ));
-DATA(insert OID = 1182 ( _date PGNSP PGUID -1 f b A f t \054 0 1082 0 array_in array_out array_recv array_send - - array_typanalyze i x f 0 -1 0 0 _null_ _null_ _null_ ));
-DATA(insert OID = 1183 ( _time PGNSP PGUID -1 f b A f t \054 0 1083 0 array_in array_out array_recv array_send timetypmodin timetypmodout array_typanalyze d x f 0 -1 0 0 _null_ _null_ _null_ ));
+DATA(insert OID = 1115 ( _timestamp PGNSP PGUID -1 f b A f t \054 0 1114 1115 array_in array_out array_recv array_send timestamptypmodin timestamptypmodout array_typanalyze d x f 0 -1 0 0 _null_ _null_ _null_ ));
+DATA(insert OID = 1182 ( _date PGNSP PGUID -1 f b A f t \054 0 1082 1182 array_in array_out array_recv array_send - - array_typanalyze i x f 0 -1 0 0 _null_ _null_ _null_ ));
+DATA(insert OID = 1183 ( _time PGNSP PGUID -1 f b A f t \054 0 1083 1183 array_in array_out array_recv array_send timetypmodin timetypmodout array_typanalyze d x f 0 -1 0 0 _null_ _null_ _null_ ));
DATA(insert OID = 1184 ( timestamptz PGNSP PGUID 8 FLOAT8PASSBYVAL b D t t \054 0 0 1185 timestamptz_in timestamptz_out timestamptz_recv timestamptz_send timestamptztypmodin timestamptztypmodout - d p f 0 -1 0 0 _null_ _null_ _null_ ));
DESCR("date and time with time zone");
#define TIMESTAMPTZOID 1184
-DATA(insert OID = 1185 ( _timestamptz PGNSP PGUID -1 f b A f t \054 0 1184 0 array_in array_out array_recv array_send timestamptztypmodin timestamptztypmodout array_typanalyze d x f 0 -1 0 0 _null_ _null_ _null_ ));
+DATA(insert OID = 1185 ( _timestamptz PGNSP PGUID -1 f b A f t \054 0 1184 1185 array_in array_out array_recv array_send timestamptztypmodin timestamptztypmodout array_typanalyze d x f 0 -1 0 0 _null_ _null_ _null_ ));
DATA(insert OID = 1186 ( interval PGNSP PGUID 16 f b T t t \054 0 0 1187 interval_in interval_out interval_recv interval_send intervaltypmodin intervaltypmodout - d p f 0 -1 0 0 _null_ _null_ _null_ ));
DESCR("@ <number> <units>, time interval");
#define INTERVALOID 1186
-DATA(insert OID = 1187 ( _interval PGNSP PGUID -1 f b A f t \054 0 1186 0 array_in array_out array_recv array_send intervaltypmodin intervaltypmodout array_typanalyze d x f 0 -1 0 0 _null_ _null_ _null_ ));
+DATA(insert OID = 1187 ( _interval PGNSP PGUID -1 f b A f t \054 0 1186 1187 array_in array_out array_recv array_send intervaltypmodin intervaltypmodout array_typanalyze d x f 0 -1 0 0 _null_ _null_ _null_ ));
/* OIDS 1200 - 1299 */
-DATA(insert OID = 1231 ( _numeric PGNSP PGUID -1 f b A f t \054 0 1700 0 array_in array_out array_recv array_send numerictypmodin numerictypmodout array_typanalyze i x f 0 -1 0 0 _null_ _null_ _null_ ));
+DATA(insert OID = 1231 ( _numeric PGNSP PGUID -1 f b A f t \054 0 1700 1231 array_in array_out array_recv array_send numerictypmodin numerictypmodout array_typanalyze i x f 0 -1 0 0 _null_ _null_ _null_ ));
DATA(insert OID = 1266 ( timetz PGNSP PGUID 12 f b D f t \054 0 0 1270 timetz_in timetz_out timetz_recv timetz_send timetztypmodin timetztypmodout - d p f 0 -1 0 0 _null_ _null_ _null_ ));
DESCR("time of day with time zone");
#define TIMETZOID 1266
-DATA(insert OID = 1270 ( _timetz PGNSP PGUID -1 f b A f t \054 0 1266 0 array_in array_out array_recv array_send timetztypmodin timetztypmodout array_typanalyze d x f 0 -1 0 0 _null_ _null_ _null_ ));
+DATA(insert OID = 1270 ( _timetz PGNSP PGUID -1 f b A f t \054 0 1266 1270 array_in array_out array_recv array_send timetztypmodin timetztypmodout array_typanalyze d x f 0 -1 0 0 _null_ _null_ _null_ ));
/* OIDS 1500 - 1599 */
DATA(insert OID = 1560 ( bit PGNSP PGUID -1 f b V f t \054 0 0 1561 bit_in bit_out bit_recv bit_send bittypmodin bittypmodout - i x f 0 -1 0 0 _null_ _null_ _null_ ));
DESCR("fixed-length bit string");
#define BITOID 1560
-DATA(insert OID = 1561 ( _bit PGNSP PGUID -1 f b A f t \054 0 1560 0 array_in array_out array_recv array_send bittypmodin bittypmodout array_typanalyze i x f 0 -1 0 0 _null_ _null_ _null_ ));
+DATA(insert OID = 1561 ( _bit PGNSP PGUID -1 f b A f t \054 0 1560 1561 array_in array_out array_recv array_send bittypmodin bittypmodout array_typanalyze i x f 0 -1 0 0 _null_ _null_ _null_ ));
DATA(insert OID = 1562 ( varbit PGNSP PGUID -1 f b V t t \054 0 0 1563 varbit_in varbit_out varbit_recv varbit_send varbittypmodin varbittypmodout - i x f 0 -1 0 0 _null_ _null_ _null_ ));
DESCR("variable-length bit string");
#define VARBITOID 1562
-DATA(insert OID = 1563 ( _varbit PGNSP PGUID -1 f b A f t \054 0 1562 0 array_in array_out array_recv array_send varbittypmodin varbittypmodout array_typanalyze i x f 0 -1 0 0 _null_ _null_ _null_ ));
+DATA(insert OID = 1563 ( _varbit PGNSP PGUID -1 f b A f t \054 0 1562 1563 array_in array_out array_recv array_send varbittypmodin varbittypmodout array_typanalyze i x f 0 -1 0 0 _null_ _null_ _null_ ));
/* OIDS 1600 - 1699 */
@@ -542,7 +542,7 @@ DESCR("reference to cursor (portal name)");
#define REFCURSOROID 1790
/* OIDS 2200 - 2299 */
-DATA(insert OID = 2201 ( _refcursor PGNSP PGUID -1 f b A f t \054 0 1790 0 array_in array_out array_recv array_send - - array_typanalyze i x f 0 -1 0 0 _null_ _null_ _null_ ));
+DATA(insert OID = 2201 ( _refcursor PGNSP PGUID -1 f b A f t \054 0 1790 2201 array_in array_out array_recv array_send - - array_typanalyze i x f 0 -1 0 0 _null_ _null_ _null_ ));
DATA(insert OID = 2202 ( regprocedure PGNSP PGUID 4 t b N f t \054 0 0 2207 regprocedurein regprocedureout regprocedurerecv regproceduresend - - - i p f 0 -1 0 0 _null_ _null_ _null_ ));
DESCR("registered procedure (with args)");
@@ -564,24 +564,24 @@ DATA(insert OID = 2206 ( regtype PGNSP PGUID 4 t b N f t \054 0 0 2211 regty
DESCR("registered type");
#define REGTYPEOID 2206
-DATA(insert OID = 2207 ( _regprocedure PGNSP PGUID -1 f b A f t \054 0 2202 0 array_in array_out array_recv array_send - - array_typanalyze i x f 0 -1 0 0 _null_ _null_ _null_ ));
-DATA(insert OID = 2208 ( _regoper PGNSP PGUID -1 f b A f t \054 0 2203 0 array_in array_out array_recv array_send - - array_typanalyze i x f 0 -1 0 0 _null_ _null_ _null_ ));
-DATA(insert OID = 2209 ( _regoperator PGNSP PGUID -1 f b A f t \054 0 2204 0 array_in array_out array_recv array_send - - array_typanalyze i x f 0 -1 0 0 _null_ _null_ _null_ ));
-DATA(insert OID = 2210 ( _regclass PGNSP PGUID -1 f b A f t \054 0 2205 0 array_in array_out array_recv array_send - - array_typanalyze i x f 0 -1 0 0 _null_ _null_ _null_ ));
-DATA(insert OID = 2211 ( _regtype PGNSP PGUID -1 f b A f t \054 0 2206 0 array_in array_out array_recv array_send - - array_typanalyze i x f 0 -1 0 0 _null_ _null_ _null_ ));
+DATA(insert OID = 2207 ( _regprocedure PGNSP PGUID -1 f b A f t \054 0 2202 2207 array_in array_out array_recv array_send - - array_typanalyze i x f 0 -1 0 0 _null_ _null_ _null_ ));
+DATA(insert OID = 2208 ( _regoper PGNSP PGUID -1 f b A f t \054 0 2203 2208 array_in array_out array_recv array_send - - array_typanalyze i x f 0 -1 0 0 _null_ _null_ _null_ ));
+DATA(insert OID = 2209 ( _regoperator PGNSP PGUID -1 f b A f t \054 0 2204 2209 array_in array_out array_recv array_send - - array_typanalyze i x f 0 -1 0 0 _null_ _null_ _null_ ));
+DATA(insert OID = 2210 ( _regclass PGNSP PGUID -1 f b A f t \054 0 2205 2210 array_in array_out array_recv array_send - - array_typanalyze i x f 0 -1 0 0 _null_ _null_ _null_ ));
+DATA(insert OID = 2211 ( _regtype PGNSP PGUID -1 f b A f t \054 0 2206 2211 array_in array_out array_recv array_send - - array_typanalyze i x f 0 -1 0 0 _null_ _null_ _null_ ));
#define REGTYPEARRAYOID 2211
/* uuid */
DATA(insert OID = 2950 ( uuid PGNSP PGUID 16 f b U f t \054 0 0 2951 uuid_in uuid_out uuid_recv uuid_send - - - c p f 0 -1 0 0 _null_ _null_ _null_ ));
DESCR("UUID datatype");
#define UUIDOID 2950
-DATA(insert OID = 2951 ( _uuid PGNSP PGUID -1 f b A f t \054 0 2950 0 array_in array_out array_recv array_send - - array_typanalyze i x f 0 -1 0 0 _null_ _null_ _null_ ));
+DATA(insert OID = 2951 ( _uuid PGNSP PGUID -1 f b A f t \054 0 2950 2951 array_in array_out array_recv array_send - - array_typanalyze i x f 0 -1 0 0 _null_ _null_ _null_ ));
/* pg_lsn */
DATA(insert OID = 3220 ( pg_lsn PGNSP PGUID 8 FLOAT8PASSBYVAL b U f t \054 0 0 3221 pg_lsn_in pg_lsn_out pg_lsn_recv pg_lsn_send - - - d p f 0 -1 0 0 _null_ _null_ _null_ ));
DESCR("PostgreSQL LSN datatype");
#define LSNOID 3220
-DATA(insert OID = 3221 ( _pg_lsn PGNSP PGUID -1 f b A f t \054 0 3220 0 array_in array_out array_recv array_send - - array_typanalyze d x f 0 -1 0 0 _null_ _null_ _null_ ));
+DATA(insert OID = 3221 ( _pg_lsn PGNSP PGUID -1 f b A f t \054 0 3220 3221 array_in array_out array_recv array_send - - array_typanalyze d x f 0 -1 0 0 _null_ _null_ _null_ ));
/* text search */
DATA(insert OID = 3614 ( tsvector PGNSP PGUID -1 f b U f t \054 0 0 3643 tsvectorin tsvectorout tsvectorrecv tsvectorsend - - ts_typanalyze i x f 0 -1 0 0 _null_ _null_ _null_ ));
@@ -600,42 +600,42 @@ DATA(insert OID = 3769 ( regdictionary PGNSP PGUID 4 t b N f t \054 0 0 3770 reg
DESCR("registered text search dictionary");
#define REGDICTIONARYOID 3769
-DATA(insert OID = 3643 ( _tsvector PGNSP PGUID -1 f b A f t \054 0 3614 0 array_in array_out array_recv array_send - - array_typanalyze i x f 0 -1 0 0 _null_ _null_ _null_ ));
-DATA(insert OID = 3644 ( _gtsvector PGNSP PGUID -1 f b A f t \054 0 3642 0 array_in array_out array_recv array_send - - array_typanalyze i x f 0 -1 0 0 _null_ _null_ _null_ ));
-DATA(insert OID = 3645 ( _tsquery PGNSP PGUID -1 f b A f t \054 0 3615 0 array_in array_out array_recv array_send - - array_typanalyze i x f 0 -1 0 0 _null_ _null_ _null_ ));
-DATA(insert OID = 3735 ( _regconfig PGNSP PGUID -1 f b A f t \054 0 3734 0 array_in array_out array_recv array_send - - array_typanalyze i x f 0 -1 0 0 _null_ _null_ _null_ ));
-DATA(insert OID = 3770 ( _regdictionary PGNSP PGUID -1 f b A f t \054 0 3769 0 array_in array_out array_recv array_send - - array_typanalyze i x f 0 -1 0 0 _null_ _null_ _null_ ));
+DATA(insert OID = 3643 ( _tsvector PGNSP PGUID -1 f b A f t \054 0 3614 3643 array_in array_out array_recv array_send - - array_typanalyze i x f 0 -1 0 0 _null_ _null_ _null_ ));
+DATA(insert OID = 3644 ( _gtsvector PGNSP PGUID -1 f b A f t \054 0 3642 3644 array_in array_out array_recv array_send - - array_typanalyze i x f 0 -1 0 0 _null_ _null_ _null_ ));
+DATA(insert OID = 3645 ( _tsquery PGNSP PGUID -1 f b A f t \054 0 3615 3645 array_in array_out array_recv array_send - - array_typanalyze i x f 0 -1 0 0 _null_ _null_ _null_ ));
+DATA(insert OID = 3735 ( _regconfig PGNSP PGUID -1 f b A f t \054 0 3734 3735 array_in array_out array_recv array_send - - array_typanalyze i x f 0 -1 0 0 _null_ _null_ _null_ ));
+DATA(insert OID = 3770 ( _regdictionary PGNSP PGUID -1 f b A f t \054 0 3769 3770 array_in array_out array_recv array_send - - array_typanalyze i x f 0 -1 0 0 _null_ _null_ _null_ ));
/* jsonb */
DATA(insert OID = 3802 ( jsonb PGNSP PGUID -1 f b U f t \054 0 0 3807 jsonb_in jsonb_out jsonb_recv jsonb_send - - - i x f 0 -1 0 0 _null_ _null_ _null_ ));
DESCR("Binary JSON");
#define JSONBOID 3802
-DATA(insert OID = 3807 ( _jsonb PGNSP PGUID -1 f b A f t \054 0 3802 0 array_in array_out array_recv array_send - - array_typanalyze i x f 0 -1 0 0 _null_ _null_ _null_ ));
+DATA(insert OID = 3807 ( _jsonb PGNSP PGUID -1 f b A f t \054 0 3802 3807 array_in array_out array_recv array_send - - array_typanalyze i x f 0 -1 0 0 _null_ _null_ _null_ ));
DATA(insert OID = 2970 ( txid_snapshot PGNSP PGUID -1 f b U f t \054 0 0 2949 txid_snapshot_in txid_snapshot_out txid_snapshot_recv txid_snapshot_send - - - d x f 0 -1 0 0 _null_ _null_ _null_ ));
DESCR("txid snapshot");
-DATA(insert OID = 2949 ( _txid_snapshot PGNSP PGUID -1 f b A f t \054 0 2970 0 array_in array_out array_recv array_send - - array_typanalyze d x f 0 -1 0 0 _null_ _null_ _null_ ));
+DATA(insert OID = 2949 ( _txid_snapshot PGNSP PGUID -1 f b A f t \054 0 2970 2949 array_in array_out array_recv array_send - - array_typanalyze d x f 0 -1 0 0 _null_ _null_ _null_ ));
/* range types */
DATA(insert OID = 3904 ( int4range PGNSP PGUID -1 f r R f t \054 0 0 3905 range_in range_out range_recv range_send - - range_typanalyze i x f 0 -1 0 0 _null_ _null_ _null_ ));
DESCR("range of integers");
#define INT4RANGEOID 3904
-DATA(insert OID = 3905 ( _int4range PGNSP PGUID -1 f b A f t \054 0 3904 0 array_in array_out array_recv array_send - - array_typanalyze i x f 0 -1 0 0 _null_ _null_ _null_ ));
+DATA(insert OID = 3905 ( _int4range PGNSP PGUID -1 f b A f t \054 0 3904 3905 array_in array_out array_recv array_send - - array_typanalyze i x f 0 -1 0 0 _null_ _null_ _null_ ));
DATA(insert OID = 3906 ( numrange PGNSP PGUID -1 f r R f t \054 0 0 3907 range_in range_out range_recv range_send - - range_typanalyze i x f 0 -1 0 0 _null_ _null_ _null_ ));
DESCR("range of numerics");
-DATA(insert OID = 3907 ( _numrange PGNSP PGUID -1 f b A f t \054 0 3906 0 array_in array_out array_recv array_send - - array_typanalyze i x f 0 -1 0 0 _null_ _null_ _null_ ));
+DATA(insert OID = 3907 ( _numrange PGNSP PGUID -1 f b A f t \054 0 3906 3907 array_in array_out array_recv array_send - - array_typanalyze i x f 0 -1 0 0 _null_ _null_ _null_ ));
DATA(insert OID = 3908 ( tsrange PGNSP PGUID -1 f r R f t \054 0 0 3909 range_in range_out range_recv range_send - - range_typanalyze d x f 0 -1 0 0 _null_ _null_ _null_ ));
DESCR("range of timestamps without time zone");
-DATA(insert OID = 3909 ( _tsrange PGNSP PGUID -1 f b A f t \054 0 3908 0 array_in array_out array_recv array_send - - array_typanalyze d x f 0 -1 0 0 _null_ _null_ _null_ ));
+DATA(insert OID = 3909 ( _tsrange PGNSP PGUID -1 f b A f t \054 0 3908 3909 array_in array_out array_recv array_send - - array_typanalyze d x f 0 -1 0 0 _null_ _null_ _null_ ));
DATA(insert OID = 3910 ( tstzrange PGNSP PGUID -1 f r R f t \054 0 0 3911 range_in range_out range_recv range_send - - range_typanalyze d x f 0 -1 0 0 _null_ _null_ _null_ ));
DESCR("range of timestamps with time zone");
-DATA(insert OID = 3911 ( _tstzrange PGNSP PGUID -1 f b A f t \054 0 3910 0 array_in array_out array_recv array_send - - array_typanalyze d x f 0 -1 0 0 _null_ _null_ _null_ ));
+DATA(insert OID = 3911 ( _tstzrange PGNSP PGUID -1 f b A f t \054 0 3910 3911 array_in array_out array_recv array_send - - array_typanalyze d x f 0 -1 0 0 _null_ _null_ _null_ ));
DATA(insert OID = 3912 ( daterange PGNSP PGUID -1 f r R f t \054 0 0 3913 range_in range_out range_recv range_send - - range_typanalyze i x f 0 -1 0 0 _null_ _null_ _null_ ));
DESCR("range of dates");
-DATA(insert OID = 3913 ( _daterange PGNSP PGUID -1 f b A f t \054 0 3912 0 array_in array_out array_recv array_send - - array_typanalyze i x f 0 -1 0 0 _null_ _null_ _null_ ));
+DATA(insert OID = 3913 ( _daterange PGNSP PGUID -1 f b A f t \054 0 3912 3913 array_in array_out array_recv array_send - - array_typanalyze i x f 0 -1 0 0 _null_ _null_ _null_ ));
DATA(insert OID = 3926 ( int8range PGNSP PGUID -1 f r R f t \054 0 0 3927 range_in range_out range_recv range_send - - range_typanalyze d x f 0 -1 0 0 _null_ _null_ _null_ ));
DESCR("range of bigints");
-DATA(insert OID = 3927 ( _int8range PGNSP PGUID -1 f b A f t \054 0 3926 0 array_in array_out array_recv array_send - - array_typanalyze d x f 0 -1 0 0 _null_ _null_ _null_ ));
+DATA(insert OID = 3927 ( _int8range PGNSP PGUID -1 f b A f t \054 0 3926 3927 array_in array_out array_recv array_send - - array_typanalyze d x f 0 -1 0 0 _null_ _null_ _null_ ));
/*
* pseudo-types
@@ -652,7 +652,7 @@ DATA(insert OID = 3927 ( _int8range PGNSP PGUID -1 f b A f t \054 0 3926 0 arr
*/
DATA(insert OID = 2249 ( record PGNSP PGUID -1 f p P f t \054 0 0 2287 record_in record_out record_recv record_send - - - d x f 0 -1 0 0 _null_ _null_ _null_ ));
#define RECORDOID 2249
-DATA(insert OID = 2287 ( _record PGNSP PGUID -1 f p P f t \054 0 2249 0 array_in array_out array_recv array_send - - array_typanalyze d x f 0 -1 0 0 _null_ _null_ _null_ ));
+DATA(insert OID = 2287 ( _record PGNSP PGUID -1 f p P f t \054 0 2249 2287 array_in array_out array_recv array_send - - array_typanalyze d x f 0 -1 0 0 _null_ _null_ _null_ ));
#define RECORDARRAYOID 2287
DATA(insert OID = 2275 ( cstring PGNSP PGUID -2 f p P f t \054 0 0 1263 cstring_in cstring_out cstring_recv cstring_send - - - c p f 0 -1 0 0 _null_ _null_ _null_ ));
#define CSTRINGOID 2275
Re: Function array_agg(array)
Ali Akbar <the.apaan@gmail.com> writes:
So, can't we just set the typarray of array types to its self oid?
Seems dangerous as heck; certainly it would have side-effects far more
wide-ranging than just making this particular function work.
A safer answer is to split array_agg into two functions,
array_agg(anynonarray) -> anyarray
array_agg(anyarray) -> anyarray
I rather imagine you should do that anyway, because I really doubt
that this hack is operating quite as intended. I suspect you are
producing arrays containing arrays as elements, not true 2-D arrays.
That's not a direction we want to go in I think; certainly there are
no other operations that produce such things.
regards, tom lane
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
Re: Function array_agg(array)
2014-10-11 22:28 GMT+07:00 Tom Lane <tgl@sss.pgh.pa.us>:
Seems dangerous as heck; certainly it would have side-effects far more
wide-ranging than just making this particular function work.A safer answer is to split array_agg into two functions,
array_agg(anynonarray) -> anyarray
array_agg(anyarray) -> anyarrayI rather imagine you should do that anyway, because I really doubt
that this hack is operating quite as intended. I suspect you are
producing arrays containing arrays as elements, not true 2-D arrays.
That's not a direction we want to go in I think; certainly there are
no other operations that produce such things.
Thanks for the review. Yes, it looks like the patch produced array as the
elements. So, all array operations behaves wierdly.
In this quick & dirty patch, I am trying to implement the
array_agg(anyarray), introducing two new functions:
- array_agg_anyarray_transfn
- array_agg_anyarray_finalfn
At first, i want to use accumArrayResult and makeMdArrayResult, but it's
complicated to work with multi-dimensional arrays with those two functions.
So i combined array_cat with those function.
Currently, it cannot handle NULL arrays:
backend> select array_agg(a) from (values(null::int[])) a(a);
1: array_agg (typeid = 1007, len = -1, typmod = -1, byval = f)
----
ERROR: cannot aggregate null arrays
Regards,
--
Ali Akbar
Attachments:
array_agg_anyarray-2.patchtext/x-diff; charset=US-ASCII; name=array_agg_anyarray-2.patchDownload
*** a/src/backend/utils/adt/array_userfuncs.c
--- b/src/backend/utils/adt/array_userfuncs.c
***************
*** 16,21 ****
--- 16,51 ----
#include "utils/builtins.h"
#include "utils/lsyscache.h"
+ #include "utils/memutils.h"
+
+ /*-------------------------------------------------------------------------
+ * ArrayAggAnyArrayState:
+ * aggregate state for array_agg(anyarray)
+ *-------------------------------------------------------------------------
+ */
+ typedef struct
+ {
+ MemoryContext mcontext; /* where all the temp stuff is kept */
+ char *data; /* array of accumulated data */
+ bits8 *nullbitmap; /* bitmap of is-null flags for data */
+
+ int abytes; /* allocated length of above arrays */
+ int aitems; /* allocated length of above arrays */
+ int nbytes; /* number of used bytes in above arrays */
+ int nitems; /* number of elements in above arrays */
+ int narray; /* number of array accumulated */
+ Oid element_type; /* data type of the Datums */
+ int16 typlen; /* needed info about datatype */
+ bool typbyval;
+ char typalign;
+
+ int ndims; /* element dimensions */
+ int *dims;
+ int *lbs;
+
+ bool hasnull; /* any element has null */
+ } ArrayAggAnyArrayState;
+
/*-----------------------------------------------------------------------------
* array_push :
***************
*** 544,546 **** array_agg_finalfn(PG_FUNCTION_ARGS)
--- 574,814 ----
PG_RETURN_DATUM(result);
}
+
+ /*
+ * ARRAY_AGG(anyarray) aggregate function
+ */
+ Datum
+ array_agg_anyarray_transfn(PG_FUNCTION_ARGS)
+ {
+ MemoryContext aggcontext,
+ arr_context,
+ oldcontext;
+ ArrayAggAnyArrayState *astate;
+
+ Oid arg_typeid = get_fn_expr_argtype(fcinfo->flinfo, 1);
+ Oid arg_elemtype = get_element_type(arg_typeid);
+ ArrayType *arg;
+ int *dims,
+ *lbs,
+ ndims,
+ nitems,
+ ndatabytes;
+ char *data;
+
+ int i;
+
+ if (arg_elemtype == InvalidOid)
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("could not determine input data type")));
+
+ if (!AggCheckCallContext(fcinfo, &aggcontext))
+ elog(ERROR, "array_agg_anyarray_transfn called in non-aggregate context");
+
+ if (PG_ARGISNULL(1))
+ elog(ERROR, "cannot aggregate null arrays");
+
+ astate = PG_ARGISNULL(0) ? NULL : (ArrayAggAnyArrayState *) PG_GETARG_POINTER(0);
+
+ if (astate == NULL)
+ {
+ arr_context = AllocSetContextCreate(aggcontext,
+ "array_agg_anyarray_transfn",
+ ALLOCSET_DEFAULT_MINSIZE,
+ ALLOCSET_DEFAULT_INITSIZE,
+ ALLOCSET_DEFAULT_MAXSIZE);
+ oldcontext = MemoryContextSwitchTo(arr_context);
+ astate = (ArrayAggAnyArrayState *) palloc(sizeof(ArrayAggAnyArrayState));
+
+ astate->mcontext = arr_context;
+ astate->abytes = 0;
+ astate->aitems = 0;
+ astate->data = NULL;
+ astate->nullbitmap = NULL;
+ astate->nitems = 0;
+ astate->narray = 0;
+ astate->element_type = arg_elemtype;
+ get_typlenbyvalalign(arg_elemtype,
+ &astate->typlen,
+ &astate->typbyval,
+ &astate->typalign);
+ }
+ else
+ {
+ oldcontext = MemoryContextSwitchTo(astate->mcontext);
+ Assert(astate->element_type == arg_elemtype);
+ }
+
+ arg = PG_GETARG_ARRAYTYPE_P(1);
+
+ ndims = ARR_NDIM(arg);
+ dims = ARR_DIMS(arg);
+ lbs = ARR_LBOUND(arg);
+ data = ARR_DATA_PTR(arg);
+ nitems = ArrayGetNItems(ndims, dims);
+
+ ndatabytes = ARR_SIZE(arg) - ARR_DATA_OFFSET(arg);
+
+ if (astate->data == NULL)
+ {
+ if (ndims + 1 > MAXDIM)
+ ereport(ERROR,
+ (errcode(ERRCODE_PROGRAM_LIMIT_EXCEEDED),
+ errmsg("number of array dimensions (%d) exceeds the maximum allowed (%d)",
+ ndims + 1, MAXDIM)));
+
+ astate->ndims = ndims;
+ astate->dims = (int *) palloc(ndims * sizeof(int));
+ astate->lbs = (int *) palloc(ndims * sizeof(int));
+ memcpy(astate->dims, dims, ndims * sizeof(int));
+ memcpy(astate->lbs, lbs, ndims * sizeof(int));
+
+ astate->abytes = 64 * (ndatabytes == 0 ? 1 : ndatabytes);
+ astate->aitems = 64 * nitems;
+ astate->data = (char *) palloc(astate->abytes);
+
+ memcpy(astate->data, data, ndatabytes);
+ astate->nbytes = ndatabytes;
+ astate->nitems = nitems;
+
+ if (ARR_HASNULL(arg))
+ {
+ astate->hasnull = true;
+ astate->nullbitmap = (bits8 *) palloc((astate->aitems + 7) / 8);
+ array_bitmap_copy(astate->nullbitmap, 0,
+ ARR_NULLBITMAP(arg), 0,
+ nitems);
+ }
+ else
+ astate->hasnull = false;
+ }
+ else
+ {
+ if (astate->ndims != ndims)
+ ereport(ERROR,
+ (errcode(ERRCODE_ARRAY_SUBSCRIPT_ERROR),
+ errmsg("cannot aggregate incompatible arrays"),
+ errdetail("Arrays of %d and %d dimensions are not "
+ "compatible for concatenation.",
+ astate->ndims, ndims)));
+
+ for (i = 0; i < ndims; i++)
+ if (astate->dims[i] != dims[i] || astate->lbs[i] != lbs[i])
+ ereport(ERROR,
+ (errcode(ERRCODE_ARRAY_SUBSCRIPT_ERROR),
+ errmsg("cannot aggregate incompatible arrays"),
+ errdetail("Arrays with differing element dimensions are "
+ "not compatible for concatenation.")));
+
+ if (astate->nbytes + ndatabytes >= astate->abytes)
+ {
+ astate->nbytes *= 2;
+ astate->data = (char *)
+ repalloc(astate->data, astate->nbytes);
+ }
+ if (astate->nitems + nitems >= astate->aitems)
+ {
+ astate->aitems *= 2;
+ astate->nullbitmap = (bits8 *)
+ repalloc(astate->nullbitmap, (astate->aitems + 7) / 8);
+ }
+
+ memcpy(astate->data + astate->nbytes, data, ndatabytes);
+ astate->nbytes += ndatabytes;
+
+ if (ARR_HASNULL(arg) || astate->hasnull)
+ {
+ if (!astate->hasnull)
+ {
+ astate->hasnull = true;
+ astate->nullbitmap = (bits8 *) palloc((astate->aitems + 7) / 8);
+ array_bitmap_copy(astate->nullbitmap, 0,
+ NULL, 0,
+ astate->nitems);
+ }
+ array_bitmap_copy(astate->nullbitmap, astate->nitems,
+ ARR_NULLBITMAP(arg), 0,
+ nitems);
+ astate->nitems += nitems;
+ }
+ }
+ astate->narray += 1;
+
+ MemoryContextSwitchTo(oldcontext);
+
+ /*
+ * The transition type for array_agg() is declared to be "internal", which
+ * is a pass-by-value type the same size as a pointer. So we can safely
+ * pass the pointer through nodeAgg.c's machinations.
+ */
+ PG_RETURN_POINTER(astate);
+ }
+
+ Datum
+ array_agg_anyarray_finalfn(PG_FUNCTION_ARGS)
+ {
+ MemoryContext oldcontext;
+
+ ArrayType *result;
+ ArrayAggAnyArrayState *astate;
+
+ int dataoffset,
+ nbytes,
+ ndims;
+
+ /*
+ * Test for null before Asserting we are in right context. This is to
+ * avoid possible Assert failure in 8.4beta installations, where it is
+ * possible for users to create NULL constants of type internal.
+ */
+ if (PG_ARGISNULL(0))
+ PG_RETURN_NULL(); /* returns null iff no input values */
+
+ /* cannot be called directly because of internal-type argument */
+ Assert(AggCheckCallContext(fcinfo, NULL));
+
+ oldcontext = MemoryContextSwitchTo(CurrentMemoryContext);
+
+ astate = (ArrayAggAnyArrayState *) PG_GETARG_POINTER(0);
+
+ ndims = astate->ndims + 1;
+ nbytes = astate->nbytes;
+ /* compute required space */
+ if (astate->hasnull)
+ {
+ dataoffset = ARR_OVERHEAD_WITHNULLS(ndims, astate->nitems);
+ nbytes += dataoffset;
+ }
+ else
+ {
+ dataoffset = 0;
+ nbytes += ARR_OVERHEAD_NONULLS(ndims);
+ }
+
+ result = (ArrayType *) palloc0(nbytes);
+ SET_VARSIZE(result, nbytes);
+ result->ndim = astate->ndims + 1;
+ result->dataoffset = dataoffset;
+ result->elemtype = astate->element_type;
+
+ ARR_DIMS(result)[0] = astate->narray;
+ ARR_LBOUND(result)[0] = 1;
+ memcpy(&(ARR_DIMS(result)[1]), astate->dims, (ndims - 1) * sizeof(int));
+ memcpy(&(ARR_LBOUND(result)[1]), astate->lbs, (ndims - 1) * sizeof(int));
+
+ memcpy(ARR_DATA_PTR(result), astate->data, astate->nbytes);
+ if (astate->hasnull)
+ array_bitmap_copy(ARR_NULLBITMAP(result), 0,
+ astate->nullbitmap, 0,
+ astate->nitems);
+
+ MemoryContextSwitchTo(oldcontext);
+
+ /* we cannot release ArrayAggAnyArrayState because sometimes
+ * aggregate final functions are re-executed. Rather, it is nodeAgg.c's
+ * responsibility to reset aggcontext when it's safe to do so
+ */
+
+ PG_RETURN_DATUM(PointerGetDatum(result));
+ }
*** a/src/include/catalog/pg_aggregate.h
--- b/src/include/catalog/pg_aggregate.h
***************
*** 275,280 **** DATA(insert ( 2901 n 0 xmlconcat2 - - - - f f 0 142 0 0 0 _null_
--- 275,281 ----
/* array */
DATA(insert ( 2335 n 0 array_agg_transfn array_agg_finalfn - - - t f 0 2281 0 0 0 _null_ _null_ ));
+ DATA(insert ( 6005 n 0 array_agg_anyarray_transfn array_agg_anyarray_finalfn - - - t f 0 2281 0 0 0 _null_ _null_ ));
/* text */
DATA(insert ( 3538 n 0 string_agg_transfn string_agg_finalfn - - - f f 0 2281 0 0 0 _null_ _null_ ));
*** a/src/include/catalog/pg_proc.h
--- b/src/include/catalog/pg_proc.h
***************
*** 879,889 **** DATA(insert OID = 3167 ( array_remove PGNSP PGUID 12 1 0 0 0 f f f f f f i 2
DESCR("remove any occurrences of an element from an array");
DATA(insert OID = 3168 ( array_replace PGNSP PGUID 12 1 0 0 0 f f f f f f i 3 0 2277 "2277 2283 2283" _null_ _null_ _null_ _null_ array_replace _null_ _null_ _null_ ));
DESCR("replace any occurrences of an element in an array");
! DATA(insert OID = 2333 ( array_agg_transfn PGNSP PGUID 12 1 0 0 0 f f f f f f i 2 0 2281 "2281 2283" _null_ _null_ _null_ _null_ array_agg_transfn _null_ _null_ _null_ ));
DESCR("aggregate transition function");
! DATA(insert OID = 2334 ( array_agg_finalfn PGNSP PGUID 12 1 0 0 0 f f f f f f i 2 0 2277 "2281 2283" _null_ _null_ _null_ _null_ array_agg_finalfn _null_ _null_ _null_ ));
DESCR("aggregate final function");
! DATA(insert OID = 2335 ( array_agg PGNSP PGUID 12 1 0 0 0 t f f f f f i 1 0 2277 "2283" _null_ _null_ _null_ _null_ aggregate_dummy _null_ _null_ _null_ ));
DESCR("concatenate aggregate input into an array");
DATA(insert OID = 3218 ( width_bucket PGNSP PGUID 12 1 0 0 0 f f f f t f i 2 0 23 "2283 2277" _null_ _null_ _null_ _null_ width_bucket_array _null_ _null_ _null_ ));
DESCR("bucket number of operand given a sorted array of bucket lower bounds");
--- 879,895 ----
DESCR("remove any occurrences of an element from an array");
DATA(insert OID = 3168 ( array_replace PGNSP PGUID 12 1 0 0 0 f f f f f f i 3 0 2277 "2277 2283 2283" _null_ _null_ _null_ _null_ array_replace _null_ _null_ _null_ ));
DESCR("replace any occurrences of an element in an array");
! DATA(insert OID = 2333 ( array_agg_transfn PGNSP PGUID 12 1 0 0 0 f f f f f f i 2 0 2281 "2281 2776" _null_ _null_ _null_ _null_ array_agg_transfn _null_ _null_ _null_ ));
DESCR("aggregate transition function");
! DATA(insert OID = 2334 ( array_agg_finalfn PGNSP PGUID 12 1 0 0 0 f f f f f f i 2 0 2277 "2281 2776" _null_ _null_ _null_ _null_ array_agg_finalfn _null_ _null_ _null_ ));
DESCR("aggregate final function");
! DATA(insert OID = 2335 ( array_agg PGNSP PGUID 12 1 0 0 0 t f f f f f i 1 0 2277 "2776" _null_ _null_ _null_ _null_ aggregate_dummy _null_ _null_ _null_ ));
! DESCR("concatenate aggregate input into an array");
! DATA(insert OID = 6003 ( array_agg_anyarray_transfn PGNSP PGUID 12 1 0 0 0 f f f f f f i 2 0 2281 "2281 2277" _null_ _null_ _null_ _null_ array_agg_anyarray_transfn _null_ _null_ _null_ ));
! DESCR("aggregate transition function");
! DATA(insert OID = 6004 ( array_agg_anyarray_finalfn PGNSP PGUID 12 1 0 0 0 f f f f f f i 2 0 2277 "2281 2277" _null_ _null_ _null_ _null_ array_agg_anyarray_finalfn _null_ _null_ _null_ ));
! DESCR("aggregate final function");
! DATA(insert OID = 6005 ( array_agg PGNSP PGUID 12 1 0 0 0 t f f f f f i 1 0 2277 "2277" _null_ _null_ _null_ _null_ aggregate_dummy _null_ _null_ _null_ ));
DESCR("concatenate aggregate input into an array");
DATA(insert OID = 3218 ( width_bucket PGNSP PGUID 12 1 0 0 0 f f f f t f i 2 0 23 "2283 2277" _null_ _null_ _null_ _null_ width_bucket_array _null_ _null_ _null_ ));
DESCR("bucket number of operand given a sorted array of bucket lower bounds");
*** a/src/include/utils/array.h
--- b/src/include/utils/array.h
***************
*** 293,298 **** extern ArrayType *create_singleton_array(FunctionCallInfo fcinfo,
--- 293,301 ----
extern Datum array_agg_transfn(PG_FUNCTION_ARGS);
extern Datum array_agg_finalfn(PG_FUNCTION_ARGS);
+ extern Datum array_agg_anyarray_transfn(PG_FUNCTION_ARGS);
+ extern Datum array_agg_anyarray_finalfn(PG_FUNCTION_ARGS);
+
/*
* prototypes for functions defined in array_typanalyze.c
*/
Re: Function array_agg(array)
2014-10-12 19:37 GMT+07:00 Ali Akbar <the.apaan@gmail.com>:
Currently, it cannot handle NULL arrays:
backend> select array_agg(a) from (values(null::int[])) a(a);
1: array_agg (typeid = 1007, len = -1, typmod = -1, byval = f)
----
ERROR: cannot aggregate null arrays
While thinking about the function behavior if its input is NULL array (e.g:
NULL:int[]), i've found:
- currentpatch doesn't handle empty array correctly:
- when there is only one array to aggregate, the resulting array is
wrong
- when the first array is empty array, and the second array is also
empty array, it segfaulted
- if we see NULL array as NULL values, the resulting array cannot be
differentiated from array of null ints:
- SELECT array_agg(NULL::int[]) FROM generate_series(1,2); ---> {NULL,
NULL} with type int[]
- SELECT array_agg(NULL::int) FROM generate_series(1,2); --> {NULL,
NULL} with type int[]
Also i've found that handling NULL array is listed as BUG in TODO. The
discussion in the thread is still not finished, with last email from Tom
Lane (/messages/by-id/18866.1226025853@sss.pgh.pa.us):
array_lower raise exception if array is empty (there are no dimensions
to inquire about)
array_upper raise exception if array is empty (there are no dimensions
to inquire about)Well, these beg the question: is an empty array zero-dimensional, or
is it a one-dimensional array of no elements, or perhaps both of those
as well as higher-dimensional cases where any axis has zero elements,
or ???It's really all kind of messy ... we need to trade off simplicity of
definition, ease of use, backwards compatibility, and standards
compliance (though the standard has only 1-D arrays so it's of just
limited help here).
So, is there any idea how we will handle NULL and empty array in
array_agg(anyarray)?
I propose we just reject those input because the output will make no sense:
- array_agg(NULL::int[]) --> the result will be indistinguished from
array_agg of NULL ints.
- array_agg('{}'::int[]) --> how we determine the dimension of the result?
is it 0? Or the result will be just an empty array {} ?
Regards,
--
Ali Akbar
Re: Function array_agg(array)
So, is there any idea how we will handle NULL and empty array in
array_agg(anyarray)?
I propose we just reject those input because the output will make no sense:
- array_agg(NULL::int[]) --> the result will be indistinguished from
array_agg of NULL ints.
- array_agg('{}'::int[]) --> how we determine the dimension of the result?
is it 0? Or the result will be just an empty array {} ?
This updated patch rejects NULL and {} arrays as noted above.
Regards,
--
Ali Akbar
Attachments:
array_agg_anyarray-3.patchtext/x-patch; charset=US-ASCII; name=array_agg_anyarray-3.patchDownload
*** a/src/backend/utils/adt/array_userfuncs.c
--- b/src/backend/utils/adt/array_userfuncs.c
***************
*** 16,21 ****
--- 16,51 ----
#include "utils/builtins.h"
#include "utils/lsyscache.h"
+ #include "utils/memutils.h"
+
+ /*-------------------------------------------------------------------------
+ * ArrayAggAnyArrayState:
+ * aggregate state for array_agg(anyarray)
+ *-------------------------------------------------------------------------
+ */
+ typedef struct
+ {
+ MemoryContext mcontext; /* where all the temp stuff is kept */
+ char *data; /* array of accumulated data */
+ bits8 *nullbitmap; /* bitmap of is-null flags for data */
+
+ int abytes; /* allocated length of above arrays */
+ int aitems; /* allocated length of above arrays */
+ int nbytes; /* number of used bytes in above arrays */
+ int nitems; /* number of elements in above arrays */
+ int narray; /* number of array accumulated */
+ Oid element_type; /* data type of the Datums */
+ int16 typlen; /* needed info about datatype */
+ bool typbyval;
+ char typalign;
+
+ int ndims; /* element dimensions */
+ int *dims;
+ int *lbs;
+
+ bool hasnull; /* any element has null */
+ } ArrayAggAnyArrayState;
+
/*-----------------------------------------------------------------------------
* array_push :
***************
*** 544,546 **** array_agg_finalfn(PG_FUNCTION_ARGS)
--- 574,821 ----
PG_RETURN_DATUM(result);
}
+
+ /*
+ * ARRAY_AGG(anyarray) aggregate function
+ */
+ Datum
+ array_agg_anyarray_transfn(PG_FUNCTION_ARGS)
+ {
+ MemoryContext aggcontext,
+ arr_context,
+ oldcontext;
+ ArrayAggAnyArrayState *astate;
+
+ Oid arg_typeid = get_fn_expr_argtype(fcinfo->flinfo, 1);
+ Oid arg_elemtype = get_element_type(arg_typeid);
+ ArrayType *arg;
+ int *dims,
+ *lbs,
+ ndims,
+ nitems,
+ ndatabytes;
+ char *data;
+
+ int i;
+
+ if (arg_elemtype == InvalidOid)
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("could not determine input data type")));
+
+ if (!AggCheckCallContext(fcinfo, &aggcontext))
+ elog(ERROR, "array_agg_anyarray_transfn called in non-aggregate context");
+
+ if (PG_ARGISNULL(1))
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("cannot aggregate null arrays")));
+
+ astate = PG_ARGISNULL(0) ? NULL : (ArrayAggAnyArrayState *) PG_GETARG_POINTER(0);
+
+ if (astate == NULL)
+ {
+ arr_context = AllocSetContextCreate(aggcontext,
+ "array_agg_anyarray_transfn",
+ ALLOCSET_DEFAULT_MINSIZE,
+ ALLOCSET_DEFAULT_INITSIZE,
+ ALLOCSET_DEFAULT_MAXSIZE);
+ oldcontext = MemoryContextSwitchTo(arr_context);
+ astate = (ArrayAggAnyArrayState *) palloc(sizeof(ArrayAggAnyArrayState));
+
+ astate->mcontext = arr_context;
+ astate->abytes = 0;
+ astate->aitems = 0;
+ astate->data = NULL;
+ astate->nullbitmap = NULL;
+ astate->nitems = 0;
+ astate->narray = 0;
+ astate->element_type = arg_elemtype;
+ get_typlenbyvalalign(arg_elemtype,
+ &astate->typlen,
+ &astate->typbyval,
+ &astate->typalign);
+ }
+ else
+ {
+ oldcontext = MemoryContextSwitchTo(astate->mcontext);
+ Assert(astate->element_type == arg_elemtype);
+ }
+
+ arg = PG_GETARG_ARRAYTYPE_P(1);
+
+ ndims = ARR_NDIM(arg);
+ dims = ARR_DIMS(arg);
+ lbs = ARR_LBOUND(arg);
+ data = ARR_DATA_PTR(arg);
+ nitems = ArrayGetNItems(ndims, dims);
+
+ ndatabytes = ARR_SIZE(arg) - ARR_DATA_OFFSET(arg);
+
+ if (astate->data == NULL)
+ {
+ if (ndims == 0)
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("cannot aggregate empty arrays")));
+
+ if (ndims + 1 > MAXDIM)
+ ereport(ERROR,
+ (errcode(ERRCODE_PROGRAM_LIMIT_EXCEEDED),
+ errmsg("number of array dimensions (%d) exceeds the maximum allowed (%d)",
+ ndims + 1, MAXDIM)));
+
+ astate->ndims = ndims;
+ astate->dims = (int *) palloc(ndims * sizeof(int));
+ astate->lbs = (int *) palloc(ndims * sizeof(int));
+ memcpy(astate->dims, dims, ndims * sizeof(int));
+ memcpy(astate->lbs, lbs, ndims * sizeof(int));
+
+ astate->abytes = 64 * (ndatabytes == 0 ? 1 : ndatabytes);
+ astate->aitems = 64 * nitems;
+ astate->data = (char *) palloc(astate->abytes);
+
+ memcpy(astate->data, data, ndatabytes);
+ astate->nbytes = ndatabytes;
+ astate->nitems = nitems;
+
+ if (ARR_HASNULL(arg))
+ {
+ astate->hasnull = true;
+ astate->nullbitmap = (bits8 *) palloc((astate->aitems + 7) / 8);
+ array_bitmap_copy(astate->nullbitmap, 0,
+ ARR_NULLBITMAP(arg), 0,
+ nitems);
+ }
+ else
+ astate->hasnull = false;
+ }
+ else
+ {
+ if (astate->ndims != ndims)
+ ereport(ERROR,
+ (errcode(ERRCODE_ARRAY_SUBSCRIPT_ERROR),
+ errmsg("cannot aggregate incompatible arrays"),
+ errdetail("Arrays of %d and %d dimensions are not "
+ "compatible for concatenation.",
+ astate->ndims, ndims)));
+
+ for (i = 0; i < ndims; i++)
+ if (astate->dims[i] != dims[i] || astate->lbs[i] != lbs[i])
+ ereport(ERROR,
+ (errcode(ERRCODE_ARRAY_SUBSCRIPT_ERROR),
+ errmsg("cannot aggregate incompatible arrays"),
+ errdetail("Arrays with differing element dimensions are "
+ "not compatible for concatenation.")));
+
+ if (astate->nbytes + ndatabytes >= astate->abytes)
+ {
+ astate->nbytes *= 2;
+ astate->data = (char *)
+ repalloc(astate->data, astate->nbytes);
+ }
+ if (astate->nitems + nitems >= astate->aitems)
+ {
+ astate->aitems *= 2;
+ astate->nullbitmap = (bits8 *)
+ repalloc(astate->nullbitmap, (astate->aitems + 7) / 8);
+ }
+
+ memcpy(astate->data + astate->nbytes, data, ndatabytes);
+ astate->nbytes += ndatabytes;
+
+ if (ARR_HASNULL(arg) || astate->hasnull)
+ {
+ if (!astate->hasnull)
+ {
+ astate->hasnull = true;
+ astate->nullbitmap = (bits8 *) palloc((astate->aitems + 7) / 8);
+ array_bitmap_copy(astate->nullbitmap, 0,
+ NULL, 0,
+ astate->nitems);
+ }
+ array_bitmap_copy(astate->nullbitmap, astate->nitems,
+ ARR_NULLBITMAP(arg), 0,
+ nitems);
+ astate->nitems += nitems;
+ }
+ }
+ astate->narray += 1;
+
+ MemoryContextSwitchTo(oldcontext);
+
+ /*
+ * The transition type for array_agg() is declared to be "internal", which
+ * is a pass-by-value type the same size as a pointer. So we can safely
+ * pass the pointer through nodeAgg.c's machinations.
+ */
+ PG_RETURN_POINTER(astate);
+ }
+
+ Datum
+ array_agg_anyarray_finalfn(PG_FUNCTION_ARGS)
+ {
+ MemoryContext oldcontext;
+
+ ArrayType *result;
+ ArrayAggAnyArrayState *astate;
+
+ int dataoffset,
+ nbytes,
+ ndims;
+
+ /*
+ * Test for null before Asserting we are in right context. This is to
+ * avoid possible Assert failure in 8.4beta installations, where it is
+ * possible for users to create NULL constants of type internal.
+ */
+ if (PG_ARGISNULL(0))
+ PG_RETURN_NULL(); /* returns null iff no input values */
+
+ /* cannot be called directly because of internal-type argument */
+ Assert(AggCheckCallContext(fcinfo, NULL));
+
+ oldcontext = MemoryContextSwitchTo(CurrentMemoryContext);
+
+ astate = (ArrayAggAnyArrayState *) PG_GETARG_POINTER(0);
+
+ ndims = astate->ndims + 1;
+ nbytes = astate->nbytes;
+ /* compute required space */
+ if (astate->hasnull)
+ {
+ dataoffset = ARR_OVERHEAD_WITHNULLS(ndims, astate->nitems);
+ nbytes += dataoffset;
+ }
+ else
+ {
+ dataoffset = 0;
+ nbytes += ARR_OVERHEAD_NONULLS(ndims);
+ }
+
+ result = (ArrayType *) palloc0(nbytes);
+ SET_VARSIZE(result, nbytes);
+ result->ndim = astate->ndims + 1;
+ result->dataoffset = dataoffset;
+ result->elemtype = astate->element_type;
+
+ ARR_DIMS(result)[0] = astate->narray;
+ ARR_LBOUND(result)[0] = 1;
+ memcpy(&(ARR_DIMS(result)[1]), astate->dims, (ndims - 1) * sizeof(int));
+ memcpy(&(ARR_LBOUND(result)[1]), astate->lbs, (ndims - 1) * sizeof(int));
+
+ memcpy(ARR_DATA_PTR(result), astate->data, astate->nbytes);
+ if (astate->hasnull)
+ array_bitmap_copy(ARR_NULLBITMAP(result), 0,
+ astate->nullbitmap, 0,
+ astate->nitems);
+
+ MemoryContextSwitchTo(oldcontext);
+
+ /* we cannot release ArrayAggAnyArrayState because sometimes
+ * aggregate final functions are re-executed. Rather, it is nodeAgg.c's
+ * responsibility to reset aggcontext when it's safe to do so
+ */
+
+ PG_RETURN_DATUM(PointerGetDatum(result));
+ }
*** a/src/include/catalog/pg_aggregate.h
--- b/src/include/catalog/pg_aggregate.h
***************
*** 275,280 **** DATA(insert ( 2901 n 0 xmlconcat2 - - - - f f 0 142 0 0 0 _null_
--- 275,281 ----
/* array */
DATA(insert ( 2335 n 0 array_agg_transfn array_agg_finalfn - - - t f 0 2281 0 0 0 _null_ _null_ ));
+ DATA(insert ( 6005 n 0 array_agg_anyarray_transfn array_agg_anyarray_finalfn - - - t f 0 2281 0 0 0 _null_ _null_ ));
/* text */
DATA(insert ( 3538 n 0 string_agg_transfn string_agg_finalfn - - - f f 0 2281 0 0 0 _null_ _null_ ));
*** a/src/include/catalog/pg_proc.h
--- b/src/include/catalog/pg_proc.h
***************
*** 879,889 **** DATA(insert OID = 3167 ( array_remove PGNSP PGUID 12 1 0 0 0 f f f f f f i 2
DESCR("remove any occurrences of an element from an array");
DATA(insert OID = 3168 ( array_replace PGNSP PGUID 12 1 0 0 0 f f f f f f i 3 0 2277 "2277 2283 2283" _null_ _null_ _null_ _null_ array_replace _null_ _null_ _null_ ));
DESCR("replace any occurrences of an element in an array");
! DATA(insert OID = 2333 ( array_agg_transfn PGNSP PGUID 12 1 0 0 0 f f f f f f i 2 0 2281 "2281 2283" _null_ _null_ _null_ _null_ array_agg_transfn _null_ _null_ _null_ ));
DESCR("aggregate transition function");
! DATA(insert OID = 2334 ( array_agg_finalfn PGNSP PGUID 12 1 0 0 0 f f f f f f i 2 0 2277 "2281 2283" _null_ _null_ _null_ _null_ array_agg_finalfn _null_ _null_ _null_ ));
DESCR("aggregate final function");
! DATA(insert OID = 2335 ( array_agg PGNSP PGUID 12 1 0 0 0 t f f f f f i 1 0 2277 "2283" _null_ _null_ _null_ _null_ aggregate_dummy _null_ _null_ _null_ ));
DESCR("concatenate aggregate input into an array");
DATA(insert OID = 3218 ( width_bucket PGNSP PGUID 12 1 0 0 0 f f f f t f i 2 0 23 "2283 2277" _null_ _null_ _null_ _null_ width_bucket_array _null_ _null_ _null_ ));
DESCR("bucket number of operand given a sorted array of bucket lower bounds");
--- 879,895 ----
DESCR("remove any occurrences of an element from an array");
DATA(insert OID = 3168 ( array_replace PGNSP PGUID 12 1 0 0 0 f f f f f f i 3 0 2277 "2277 2283 2283" _null_ _null_ _null_ _null_ array_replace _null_ _null_ _null_ ));
DESCR("replace any occurrences of an element in an array");
! DATA(insert OID = 2333 ( array_agg_transfn PGNSP PGUID 12 1 0 0 0 f f f f f f i 2 0 2281 "2281 2776" _null_ _null_ _null_ _null_ array_agg_transfn _null_ _null_ _null_ ));
DESCR("aggregate transition function");
! DATA(insert OID = 2334 ( array_agg_finalfn PGNSP PGUID 12 1 0 0 0 f f f f f f i 2 0 2277 "2281 2776" _null_ _null_ _null_ _null_ array_agg_finalfn _null_ _null_ _null_ ));
DESCR("aggregate final function");
! DATA(insert OID = 2335 ( array_agg PGNSP PGUID 12 1 0 0 0 t f f f f f i 1 0 2277 "2776" _null_ _null_ _null_ _null_ aggregate_dummy _null_ _null_ _null_ ));
! DESCR("concatenate aggregate input into an array");
! DATA(insert OID = 6003 ( array_agg_anyarray_transfn PGNSP PGUID 12 1 0 0 0 f f f f f f i 2 0 2281 "2281 2277" _null_ _null_ _null_ _null_ array_agg_anyarray_transfn _null_ _null_ _null_ ));
! DESCR("aggregate transition function");
! DATA(insert OID = 6004 ( array_agg_anyarray_finalfn PGNSP PGUID 12 1 0 0 0 f f f f f f i 2 0 2277 "2281 2277" _null_ _null_ _null_ _null_ array_agg_anyarray_finalfn _null_ _null_ _null_ ));
! DESCR("aggregate final function");
! DATA(insert OID = 6005 ( array_agg PGNSP PGUID 12 1 0 0 0 t f f f f f i 1 0 2277 "2277" _null_ _null_ _null_ _null_ aggregate_dummy _null_ _null_ _null_ ));
DESCR("concatenate aggregate input into an array");
DATA(insert OID = 3218 ( width_bucket PGNSP PGUID 12 1 0 0 0 f f f f t f i 2 0 23 "2283 2277" _null_ _null_ _null_ _null_ width_bucket_array _null_ _null_ _null_ ));
DESCR("bucket number of operand given a sorted array of bucket lower bounds");
*** a/src/include/utils/array.h
--- b/src/include/utils/array.h
***************
*** 293,298 **** extern ArrayType *create_singleton_array(FunctionCallInfo fcinfo,
--- 293,301 ----
extern Datum array_agg_transfn(PG_FUNCTION_ARGS);
extern Datum array_agg_finalfn(PG_FUNCTION_ARGS);
+ extern Datum array_agg_anyarray_transfn(PG_FUNCTION_ARGS);
+ extern Datum array_agg_anyarray_finalfn(PG_FUNCTION_ARGS);
+
/*
* prototypes for functions defined in array_typanalyze.c
*/
Re: Function array_agg(array)
Hi
2014-10-19 8:02 GMT+02:00 Ali Akbar <the.apaan@gmail.com>:
So, is there any idea how we will handle NULL and empty array in
array_agg(anyarray)?
I propose we just reject those input because the output will make no
sense:
- array_agg(NULL::int[]) --> the result will be indistinguished from
array_agg of NULL ints.
- array_agg('{}'::int[]) --> how we determine the dimension of the
result? is it 0? Or the result will be just an empty array {} ?This updated patch rejects NULL and {} arrays as noted above.
I agree with your proposal. I have a few comments to design:
1. patch doesn't hold documentation and regress tests, please append it.
2. this functionality (multidimensional aggregation) can be interesting
more times, so maybe some interface like array builder should be preferred.
3. array_agg was consistent with array(subselect), so it should be fixed too
postgres=# select array_agg(a) from test;
array_agg
-----------------------
{{1,2,3,4},{1,2,3,4}}
(1 row)
postgres=# select array(select a from test);
ERROR: could not find array type for data type integer[]
4. why you use a magic constant (64) there?
+ astate->abytes = 64 * (ndatabytes == 0 ? 1 : ndatabytes);
+ astate->aitems = 64 * nitems;+ astate->nullbitmap = (bits8 *)
+ repalloc(astate->nullbitmap, (astate->aitems + 7) / 8);Regards
Pavel
Show quoted text
Regards,
--
Ali Akbar--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
Re: Function array_agg(array)
Thanks for the review
2014-10-22 20:51 GMT+07:00 Pavel Stehule <pavel.stehule@gmail.com>:
I agree with your proposal. I have a few comments to design:
1. patch doesn't hold documentation and regress tests, please append it.
OK, i'll add the documentation and regression test
2. this functionality (multidimensional aggregation) can be interesting
more times, so maybe some interface like array builder should be preferred.
We already have accumArrayResult and makeArrayResult/makeMdArrayResult,
haven't we?
Actually array_agg(anyarray) can be implemented by using accumArrayResult
and makeMdArrayResult, but in the process we will deconstruct the array
data and null bit-flags into ArrayBuildState->dvalues and dnulls. And we
will reconstruct it through makeMdArrayResult. I think this way it's not
efficient, so i decided to reimplement it and memcpy the array data and
null flags as-is.
In other places, i think it's clearer if we just use accumArrayResult and
makeArrayResult/makeMdArrayResult as API for building (multidimensional)
arrays.
3. array_agg was consistent with array(subselect), so it should be fixed
toopostgres=# select array_agg(a) from test;
array_agg
-----------------------
{{1,2,3,4},{1,2,3,4}}
(1 row)postgres=# select array(select a from test);
ERROR: could not find array type for data type integer[]
I'm pretty new in postgresql development. Can you point out where is
array(subselect) implemented?
4. why you use a magic constant (64) there?
+ astate->abytes = 64 * (ndatabytes == 0 ? 1 : ndatabytes); + astate->aitems = 64 * nitems;+ astate->nullbitmap = (bits8 *) + repalloc(astate->nullbitmap, (astate->aitems + 7) / 8);
just follow the arbitrary size choosen in accumArrayResult (arrayfuncs.c):
astate->alen = 64; /* arbitrary starting array size */
it can be any number not too small and too big. Too small, and we will
realloc shortly. Too big, we will end up wasting memory.
Regards,
--
Ali Akbar
Re: Function array_agg(array)
2014-10-22 16:58 GMT+02:00 Ali Akbar <the.apaan@gmail.com>:
Thanks for the review
2014-10-22 20:51 GMT+07:00 Pavel Stehule <pavel.stehule@gmail.com>:
I agree with your proposal. I have a few comments to design:
1. patch doesn't hold documentation and regress tests, please append it.
OK, i'll add the documentation and regression test
2. this functionality (multidimensional aggregation) can be interesting
more times, so maybe some interface like array builder should be preferred.We already have accumArrayResult and makeArrayResult/makeMdArrayResult,
haven't we?Actually array_agg(anyarray) can be implemented by using accumArrayResult
and makeMdArrayResult, but in the process we will deconstruct the array
data and null bit-flags into ArrayBuildState->dvalues and dnulls. And we
will reconstruct it through makeMdArrayResult. I think this way it's not
efficient, so i decided to reimplement it and memcpy the array data and
null flags as-is.
aha, so isn't better to fix a performance for accumArrayResult ?
In other places, i think it's clearer if we just use accumArrayResult and
makeArrayResult/makeMdArrayResult as API for building (multidimensional)
arrays.3. array_agg was consistent with array(subselect), so it should be fixed
toopostgres=# select array_agg(a) from test;
array_agg
-----------------------
{{1,2,3,4},{1,2,3,4}}
(1 row)postgres=# select array(select a from test);
ERROR: could not find array type for data type integer[]I'm pretty new in postgresql development. Can you point out where is
array(subselect) implemented?
where you can start?
postgres=# explain select array(select a from test);
ERROR: 42704: could not find array type for data type integer[]
LOCATION: exprType, nodeFuncs.c:116
look to code ... your magic keyword is a ARRAY_SUBLINK .. search in
postgresql sources this keyword
4. why you use a magic constant (64) there?
+ astate->abytes = 64 * (ndatabytes == 0 ? 1 : ndatabytes); + astate->aitems = 64 * nitems;+ astate->nullbitmap = (bits8 *) + repalloc(astate->nullbitmap, (astate->aitems + 7) / 8);just follow the arbitrary size choosen in accumArrayResult (arrayfuncs.c):
astate->alen = 64; /* arbitrary starting array size */it can be any number not too small and too big. Too small, and we will
realloc shortly. Too big, we will end up wasting memory.
you can try to alloc 1KB instead as start -- it is used more times in
Postgres. Then a overhead is max 1KB per agg call - what is acceptable.
You take this value from accumArrayResult, but it is targeted for shorted
scalars - you should to expect so any array will be much larger.
Show quoted text
Regards,
--
Ali Akbar
Re: Function array_agg(array)
2014-10-22 22:48 GMT+07:00 Pavel Stehule <pavel.stehule@gmail.com>:
2014-10-22 16:58 GMT+02:00 Ali Akbar <the.apaan@gmail.com>:
Thanks for the review
2014-10-22 20:51 GMT+07:00 Pavel Stehule <pavel.stehule@gmail.com>:
I agree with your proposal. I have a few comments to design:
1. patch doesn't hold documentation and regress tests, please append it.
OK, i'll add the documentation and regression test
2. this functionality (multidimensional aggregation) can be interesting
more times, so maybe some interface like array builder should be preferred.We already have accumArrayResult and makeArrayResult/makeMdArrayResult,
haven't we?Actually array_agg(anyarray) can be implemented by using accumArrayResult
and makeMdArrayResult, but in the process we will deconstruct the array
data and null bit-flags into ArrayBuildState->dvalues and dnulls. And we
will reconstruct it through makeMdArrayResult. I think this way it's not
efficient, so i decided to reimplement it and memcpy the array data and
null flags as-is.aha, so isn't better to fix a performance for accumArrayResult ?
Ok, i'll go this route. While reading the code of array(subselect) as you
pointed below, i think the easiest way to implement support for both
array_agg(anyarray) and array(subselect) is to change accumArrayResult so
it operates both with scalar datum(s) and array datums, with performance
optimization for the latter.
In other places, i think it's clearer if we just use accumArrayResult and
makeArrayResult/makeMdArrayResult as API for building (multidimensional)
arrays.3. array_agg was consistent with array(subselect), so it should be fixed
toopostgres=# select array_agg(a) from test;
array_agg
-----------------------
{{1,2,3,4},{1,2,3,4}}
(1 row)postgres=# select array(select a from test);
ERROR: could not find array type for data type integer[]I'm pretty new in postgresql development. Can you point out where is
array(subselect) implemented?where you can start?
postgres=# explain select array(select a from test);
ERROR: 42704: could not find array type for data type integer[]
LOCATION: exprType, nodeFuncs.c:116look to code ... your magic keyword is a ARRAY_SUBLINK .. search in
postgresql sources this keyword
Found it. Thanks.
On a side note in postgres array type for integer[] is still integer[], but
in pg_type, integer[] has no array type. I propose we consider to change it
so array type of anyarray is itself (not in this patch, of course, because
it is a big change). Consider the following code in nodeFuncs.c:109
if (sublink->subLinkType == ARRAY_SUBLINK)
{
type = get_array_type(type);
if (!OidIsValid(type))
ereport(ERROR,
(errcode(ERRCODE_UNDEFINED_OBJECT),
errmsg("could not find array type for data type %s",
format_type_be(exprType((Node *) tent->expr)))));
}
to implement array(subselect) you pointed above, we must specially checks
for array types. Quick search on get_array_type returns 10 places.
I'll think more about this later. For this patch, i'll go without changes
in pg_type.h.
4. why you use a magic constant (64) there?
+ astate->abytes = 64 * (ndatabytes == 0 ? 1 : ndatabytes); + astate->aitems = 64 * nitems;+ astate->nullbitmap = (bits8 *) + repalloc(astate->nullbitmap, (astate->aitems + 7) / 8);just follow the arbitrary size choosen in accumArrayResult (arrayfuncs.c):
astate->alen = 64; /* arbitrary starting array size */it can be any number not too small and too big. Too small, and we will
realloc shortly. Too big, we will end up wasting memory.you can try to alloc 1KB instead as start -- it is used more times in
Postgres. Then a overhead is max 1KB per agg call - what is acceptable.You take this value from accumArrayResult, but it is targeted for shorted
scalars - you should to expect so any array will be much larger.
Ok.
Regards,
--
Ali Akbar
Re: Function array_agg(array)
2014-10-23 4:00 GMT+02:00 Ali Akbar <the.apaan@gmail.com>:
2014-10-22 22:48 GMT+07:00 Pavel Stehule <pavel.stehule@gmail.com>:
2014-10-22 16:58 GMT+02:00 Ali Akbar <the.apaan@gmail.com>:
Thanks for the review
2014-10-22 20:51 GMT+07:00 Pavel Stehule <pavel.stehule@gmail.com>:
I agree with your proposal. I have a few comments to design:
1. patch doesn't hold documentation and regress tests, please append it.
OK, i'll add the documentation and regression test
2. this functionality (multidimensional aggregation) can be interesting
more times, so maybe some interface like array builder should be preferred.We already have accumArrayResult and makeArrayResult/makeMdArrayResult,
haven't we?Actually array_agg(anyarray) can be implemented by using
accumArrayResult and makeMdArrayResult, but in the process we will
deconstruct the array data and null bit-flags into ArrayBuildState->dvalues
and dnulls. And we will reconstruct it through makeMdArrayResult. I think
this way it's not efficient, so i decided to reimplement it and memcpy the
array data and null flags as-is.aha, so isn't better to fix a performance for accumArrayResult ?
Ok, i'll go this route. While reading the code of array(subselect) as you
pointed below, i think the easiest way to implement support for both
array_agg(anyarray) and array(subselect) is to change accumArrayResult so
it operates both with scalar datum(s) and array datums, with performance
optimization for the latter.In other places, i think it's clearer if we just use accumArrayResult and
makeArrayResult/makeMdArrayResult as API for building (multidimensional)
arrays.3. array_agg was consistent with array(subselect), so it should be
fixed toopostgres=# select array_agg(a) from test;
array_agg
-----------------------
{{1,2,3,4},{1,2,3,4}}
(1 row)postgres=# select array(select a from test);
ERROR: could not find array type for data type integer[]I'm pretty new in postgresql development. Can you point out where is
array(subselect) implemented?where you can start?
postgres=# explain select array(select a from test);
ERROR: 42704: could not find array type for data type integer[]
LOCATION: exprType, nodeFuncs.c:116look to code ... your magic keyword is a ARRAY_SUBLINK .. search in
postgresql sources this keywordFound it. Thanks.
On a side note in postgres array type for integer[] is still integer[],
but in pg_type, integer[] has no array type. I propose we consider to
change it so array type of anyarray is itself (not in this patch, of
course, because it is a big change). Consider the following code in
nodeFuncs.c:109
yes, it is true - this is really big change and maybe needs separate
discuss - ***if we allow cycle there. I am not sure about possible side
effects***.
Maybe this change is not necessary, you can fix a check only ... "if type
is not array or if get_array_type is null raise a error"
if (sublink->subLinkType == ARRAY_SUBLINK)
{
type = get_array_type(type);
if (!OidIsValid(type))
ereport(ERROR,
(errcode(ERRCODE_UNDEFINED_OBJECT),
errmsg("could not find array type for data type %s",
format_type_be(exprType((Node *) tent->expr)))));
}to implement array(subselect) you pointed above, we must specially checks
for array types. Quick search on get_array_type returns 10 places.
attention: probably we don't would to allow arrays everywhere.
Show quoted text
I'll think more about this later. For this patch, i'll go without changes
in pg_type.h.4. why you use a magic constant (64) there?
+ astate->abytes = 64 * (ndatabytes == 0 ? 1 : ndatabytes); + astate->aitems = 64 * nitems;+ astate->nullbitmap = (bits8 *) + repalloc(astate->nullbitmap, (astate->aitems + 7) / 8);just follow the arbitrary size choosen in accumArrayResult
(arrayfuncs.c):
astate->alen = 64; /* arbitrary starting array size */it can be any number not too small and too big. Too small, and we will
realloc shortly. Too big, we will end up wasting memory.you can try to alloc 1KB instead as start -- it is used more times in
Postgres. Then a overhead is max 1KB per agg call - what is acceptable.You take this value from accumArrayResult, but it is targeted for shorted
scalars - you should to expect so any array will be much larger.Ok.
Regards,
--
Ali Akbar
Re: Function array_agg(array)
Updated patch attached.
2014-10-22 20:51 GMT+07:00 Pavel Stehule <pavel.stehule@gmail.com>:
I agree with your proposal. I have a few comments to design:
1. patch doesn't hold documentation and regress tests, please append
it.
i've added some regression tests in arrays.sql and aggregate.sql.
I've only found the documentation of array_agg. Where is the doc for
array(subselect) defined?
2. this functionality (multidimensional aggregation) can be interesting
more times, so maybe some interface like array builder should be preferred.
We already have accumArrayResult and makeArrayResult/makeMdArrayResult,
haven't we?Actually array_agg(anyarray) can be implemented by using
accumArrayResult and makeMdArrayResult, but in the process we will
deconstruct the array data and null bit-flags into ArrayBuildState->dvalues
and dnulls. And we will reconstruct it through makeMdArrayResult. I think
this way it's not efficient, so i decided to reimplement it and memcpy the
array data and null flags as-is.aha, so isn't better to fix a performance for accumArrayResult ?
Ok, i'll go this route. While reading the code of array(subselect) as you
pointed below, i think the easiest way to implement support for both
array_agg(anyarray) and array(subselect) is to change accumArrayResult so
it operates both with scalar datum(s) and array datums, with performance
optimization for the latter.
implemented it by modifying ArrayBuildState to ArrayBuildStateArray and
ArrayBuildStateScalar (do you have any idea for better struct naming?)
In other places, i think it's clearer if we just use accumArrayResult and
makeArrayResult/makeMdArrayResult as API for building (multidimensional)
arrays.3. array_agg was consistent with array(subselect), so it should be
fixed toopostgres=# select array_agg(a) from test;
array_agg
-----------------------
{{1,2,3,4},{1,2,3,4}}
(1 row)postgres=# select array(select a from test);
ERROR: could not find array type for data type integer[]I'm pretty new in postgresql development. Can you point out where is
array(subselect) implemented?where you can start?
postgres=# explain select array(select a from test);
ERROR: 42704: could not find array type for data type integer[]
LOCATION: exprType, nodeFuncs.c:116look to code ... your magic keyword is a ARRAY_SUBLINK .. search in
postgresql sources this keywordattention: probably we don't would to allow arrays everywhere.
I've changed the places where i think it's appropriate.
4. why you use a magic constant (64) there?
+ astate->abytes = 64 * (ndatabytes == 0 ? 1 : ndatabytes); + astate->aitems = 64 * nitems;+ astate->nullbitmap = (bits8 *) + repalloc(astate->nullbitmap, (astate->aitems + 7) / 8);just follow the arbitrary size choosen in accumArrayResult
(arrayfuncs.c):
astate->alen = 64; /* arbitrary starting array size */it can be any number not too small and too big. Too small, and we will
realloc shortly. Too big, we will end up wasting memory.you can try to alloc 1KB instead as start -- it is used more times in
Postgres. Then a overhead is max 1KB per agg call - what is acceptable.You take this value from accumArrayResult, but it is targeted for
shorted scalars - you should to expect so any array will be much larger.
this patch allocates 1KB (1024 bytes) if the ndatabytes is < 512bytes. If
it is larger, it allocates 4 * size. For nullbitmap, it allocates 4 *
number of items in array.
Regards,
--
Ali Akbar
Attachments:
array_agg_anyarray-4.patchtext/x-patch; charset=US-ASCII; name=array_agg_anyarray-4.patchDownload
*** a/doc/src/sgml/func.sgml
--- b/doc/src/sgml/func.sgml
***************
*** 12046,12051 **** NULL baz</literallayout>(3 rows)</entry>
--- 12046,12067 ----
<row>
<entry>
<indexterm>
+ <primary>array_agg</primary>
+ </indexterm>
+ <function>array_agg(<replaceable class="parameter">anyarray</replaceable>)</function>
+ </entry>
+ <entry>
+ any
+ </entry>
+ <entry>
+ the same array type as input type
+ </entry>
+ <entry>input arrays, aggregated into higher-order multidimesional array. Rejects NULL and empty array as input.</entry>
+ </row>
+
+ <row>
+ <entry>
+ <indexterm>
<primary>average</primary>
</indexterm>
<indexterm>
*** a/src/backend/nodes/nodeFuncs.c
--- b/src/backend/nodes/nodeFuncs.c
***************
*** 108,119 **** exprType(const Node *expr)
type = exprType((Node *) tent->expr);
if (sublink->subLinkType == ARRAY_SUBLINK)
{
! type = get_array_type(type);
! if (!OidIsValid(type))
! ereport(ERROR,
! (errcode(ERRCODE_UNDEFINED_OBJECT),
! errmsg("could not find array type for data type %s",
! format_type_be(exprType((Node *) tent->expr)))));
}
}
else if (sublink->subLinkType == MULTIEXPR_SUBLINK)
--- 108,123 ----
type = exprType((Node *) tent->expr);
if (sublink->subLinkType == ARRAY_SUBLINK)
{
! if (!OidIsValid(get_element_type(type)))
! {
! /* not array, so check for its array type */
! type = get_array_type(type);
! if (!OidIsValid(type))
! ereport(ERROR,
! (errcode(ERRCODE_UNDEFINED_OBJECT),
! errmsg("could not find array type for data type %s",
! format_type_be(exprType((Node *) tent->expr)))));
! }
}
}
else if (sublink->subLinkType == MULTIEXPR_SUBLINK)
***************
*** 139,150 **** exprType(const Node *expr)
type = subplan->firstColType;
if (subplan->subLinkType == ARRAY_SUBLINK)
{
! type = get_array_type(type);
! if (!OidIsValid(type))
! ereport(ERROR,
! (errcode(ERRCODE_UNDEFINED_OBJECT),
! errmsg("could not find array type for data type %s",
! format_type_be(subplan->firstColType))));
}
}
else if (subplan->subLinkType == MULTIEXPR_SUBLINK)
--- 143,158 ----
type = subplan->firstColType;
if (subplan->subLinkType == ARRAY_SUBLINK)
{
! if (!OidIsValid(get_element_type(type)))
! {
! /* not array, so check for its array type */
! type = get_array_type(type);
! if (!OidIsValid(type))
! ereport(ERROR,
! (errcode(ERRCODE_UNDEFINED_OBJECT),
! errmsg("could not find array type for data type %s",
! format_type_be(subplan->firstColType))));
! }
}
}
else if (subplan->subLinkType == MULTIEXPR_SUBLINK)
*** a/src/backend/optimizer/plan/subselect.c
--- b/src/backend/optimizer/plan/subselect.c
***************
*** 668,677 **** build_subplan(PlannerInfo *root, Plan *plan, PlannerInfo *subroot,
Assert(!te->resjunk);
Assert(testexpr == NULL);
! arraytype = get_array_type(exprType((Node *) te->expr));
! if (!OidIsValid(arraytype))
! elog(ERROR, "could not find array type for datatype %s",
! format_type_be(exprType((Node *) te->expr)));
prm = generate_new_param(root,
arraytype,
exprTypmod((Node *) te->expr),
--- 668,683 ----
Assert(!te->resjunk);
Assert(testexpr == NULL);
!
! arraytype = exprType((Node *) te->expr);
! if (!OidIsValid(get_element_type(arraytype)))
! {
! /* not array, so get the array type */
! arraytype = get_array_type(exprType((Node *) te->expr));
! if (!OidIsValid(arraytype))
! elog(ERROR, "could not find array type for datatype %s",
! format_type_be(exprType((Node *) te->expr)));
! }
prm = generate_new_param(root,
arraytype,
exprTypmod((Node *) te->expr),
*** a/src/backend/utils/adt/array_userfuncs.c
--- b/src/backend/utils/adt/array_userfuncs.c
***************
*** 16,22 ****
#include "utils/builtins.h"
#include "utils/lsyscache.h"
-
/*-----------------------------------------------------------------------------
* array_push :
* push an element onto either end of a one-dimensional array
--- 16,21 ----
*** a/src/backend/utils/adt/arrayfuncs.c
--- b/src/backend/utils/adt/arrayfuncs.c
***************
*** 145,151 **** static int width_bucket_array_variable(Datum operand,
Oid collation,
TypeCacheEntry *typentry);
-
/*
* array_in :
* converts an array from the external format in "string" to
--- 145,150 ----
*** a/src/include/catalog/pg_aggregate.h
--- b/src/include/catalog/pg_aggregate.h
***************
*** 275,280 **** DATA(insert ( 2901 n 0 xmlconcat2 - - - - f f 0 142 0 0 0 _null_
--- 275,281 ----
/* array */
DATA(insert ( 2335 n 0 array_agg_transfn array_agg_finalfn - - - t f 0 2281 0 0 0 _null_ _null_ ));
+ DATA(insert ( 6005 n 0 array_agg_anyarray_transfn array_agg_anyarray_finalfn - - - t f 0 2281 0 0 0 _null_ _null_ ));
/* text */
DATA(insert ( 3538 n 0 string_agg_transfn string_agg_finalfn - - - f f 0 2281 0 0 0 _null_ _null_ ));
*** a/src/include/catalog/pg_proc.h
--- b/src/include/catalog/pg_proc.h
***************
*** 879,889 **** DATA(insert OID = 3167 ( array_remove PGNSP PGUID 12 1 0 0 0 f f f f f f i 2
DESCR("remove any occurrences of an element from an array");
DATA(insert OID = 3168 ( array_replace PGNSP PGUID 12 1 0 0 0 f f f f f f i 3 0 2277 "2277 2283 2283" _null_ _null_ _null_ _null_ array_replace _null_ _null_ _null_ ));
DESCR("replace any occurrences of an element in an array");
! DATA(insert OID = 2333 ( array_agg_transfn PGNSP PGUID 12 1 0 0 0 f f f f f f i 2 0 2281 "2281 2283" _null_ _null_ _null_ _null_ array_agg_transfn _null_ _null_ _null_ ));
DESCR("aggregate transition function");
! DATA(insert OID = 2334 ( array_agg_finalfn PGNSP PGUID 12 1 0 0 0 f f f f f f i 2 0 2277 "2281 2283" _null_ _null_ _null_ _null_ array_agg_finalfn _null_ _null_ _null_ ));
DESCR("aggregate final function");
! DATA(insert OID = 2335 ( array_agg PGNSP PGUID 12 1 0 0 0 t f f f f f i 1 0 2277 "2283" _null_ _null_ _null_ _null_ aggregate_dummy _null_ _null_ _null_ ));
DESCR("concatenate aggregate input into an array");
DATA(insert OID = 3218 ( width_bucket PGNSP PGUID 12 1 0 0 0 f f f f t f i 2 0 23 "2283 2277" _null_ _null_ _null_ _null_ width_bucket_array _null_ _null_ _null_ ));
DESCR("bucket number of operand given a sorted array of bucket lower bounds");
--- 879,895 ----
DESCR("remove any occurrences of an element from an array");
DATA(insert OID = 3168 ( array_replace PGNSP PGUID 12 1 0 0 0 f f f f f f i 3 0 2277 "2277 2283 2283" _null_ _null_ _null_ _null_ array_replace _null_ _null_ _null_ ));
DESCR("replace any occurrences of an element in an array");
! DATA(insert OID = 2333 ( array_agg_transfn PGNSP PGUID 12 1 0 0 0 f f f f f f i 2 0 2281 "2281 2776" _null_ _null_ _null_ _null_ array_agg_transfn _null_ _null_ _null_ ));
DESCR("aggregate transition function");
! DATA(insert OID = 2334 ( array_agg_finalfn PGNSP PGUID 12 1 0 0 0 f f f f f f i 2 0 2277 "2281 2776" _null_ _null_ _null_ _null_ array_agg_finalfn _null_ _null_ _null_ ));
DESCR("aggregate final function");
! DATA(insert OID = 2335 ( array_agg PGNSP PGUID 12 1 0 0 0 t f f f f f i 1 0 2277 "2776" _null_ _null_ _null_ _null_ aggregate_dummy _null_ _null_ _null_ ));
! DESCR("concatenate aggregate input into an array");
! DATA(insert OID = 6003 ( array_agg_anyarray_transfn PGNSP PGUID 12 1 0 0 0 f f f f f f i 2 0 2281 "2281 2277" _null_ _null_ _null_ _null_ array_agg_anyarray_transfn _null_ _null_ _null_ ));
! DESCR("aggregate transition function");
! DATA(insert OID = 6004 ( array_agg_anyarray_finalfn PGNSP PGUID 12 1 0 0 0 f f f f f f i 2 0 2277 "2281 2277" _null_ _null_ _null_ _null_ array_agg_anyarray_finalfn _null_ _null_ _null_ ));
! DESCR("aggregate final function");
! DATA(insert OID = 6005 ( array_agg PGNSP PGUID 12 1 0 0 0 t f f f f f i 1 0 2277 "2277" _null_ _null_ _null_ _null_ aggregate_dummy _null_ _null_ _null_ ));
DESCR("concatenate aggregate input into an array");
DATA(insert OID = 3218 ( width_bucket PGNSP PGUID 12 1 0 0 0 f f f f t f i 2 0 23 "2283 2277" _null_ _null_ _null_ _null_ width_bucket_array _null_ _null_ _null_ ));
DESCR("bucket number of operand given a sorted array of bucket lower bounds");
*** a/src/include/utils/array.h
--- b/src/include/utils/array.h
***************
*** 76,94 **** typedef struct
/*
* working state for accumArrayResult() and friends
*/
typedef struct ArrayBuildState
{
MemoryContext mcontext; /* where all the temp stuff is kept */
Datum *dvalues; /* array of accumulated Datums */
bool *dnulls; /* array of is-null flags for Datums */
int alen; /* allocated length of above arrays */
int nelems; /* number of valid entries in above arrays */
! Oid element_type; /* data type of the Datums */
! int16 typlen; /* needed info about datatype */
! bool typbyval;
! char typalign;
! } ArrayBuildState;
/*
* structure to cache type metadata needed for array manipulation
--- 76,132 ----
/*
* working state for accumArrayResult() and friends
+ *
+ * is_array_accum: whether accumulating array values.
+ * (if true must be casted to ArrayBuildStateArray, else
+ * cast to ArrayBuildStateScalar)
*/
typedef struct ArrayBuildState
{
+ bool is_array_accum;
MemoryContext mcontext; /* where all the temp stuff is kept */
+ Oid element_type; /* data type of the Datums */
+ int16 typlen; /* needed info about datatype */
+ bool typbyval;
+ char typalign;
+ } ArrayBuildState;
+
+ /*
+ * array build state for array accumulation of scalar datums
+ */
+ typedef struct ArrayBuildStateScalar
+ {
+ ArrayBuildState astate;
+
Datum *dvalues; /* array of accumulated Datums */
bool *dnulls; /* array of is-null flags for Datums */
int alen; /* allocated length of above arrays */
int nelems; /* number of valid entries in above arrays */
! } ArrayBuildStateScalar;
!
!
! /*
! * array build state for array accumulation of array datums
! */
! typedef struct
! {
! ArrayBuildState astate;
!
! char *data; /* array of accumulated data */
! bits8 *nullbitmap; /* bitmap of is-null flags for data */
!
! int abytes; /* allocated length of above arrays */
! int aitems; /* allocated length of above arrays */
! int nbytes; /* number of used bytes in above arrays */
! int nitems; /* number of elements in above arrays */
! int narray; /* number of array accumulated */
!
! int ndims; /* element dimensions */
! int *dims;
! int *lbs;
!
! bool hasnull; /* any element has null */
! } ArrayBuildStateArray;
/*
* structure to cache type metadata needed for array manipulation
***************
*** 260,266 **** extern Datum makeArrayResult(ArrayBuildState *astate,
MemoryContext rcontext);
extern Datum makeMdArrayResult(ArrayBuildState *astate, int ndims,
int *dims, int *lbs, MemoryContext rcontext, bool release);
!
extern ArrayIterator array_create_iterator(ArrayType *arr, int slice_ndim);
extern bool array_iterate(ArrayIterator iterator, Datum *value, bool *isnull);
extern void array_free_iterator(ArrayIterator iterator);
--- 298,305 ----
MemoryContext rcontext);
extern Datum makeMdArrayResult(ArrayBuildState *astate, int ndims,
int *dims, int *lbs, MemoryContext rcontext, bool release);
! extern Datum makeArrayResultArray(ArrayBuildStateArray *astate,
! MemoryContext rcontext, bool release);
extern ArrayIterator array_create_iterator(ArrayType *arr, int slice_ndim);
extern bool array_iterate(ArrayIterator iterator, Datum *value, bool *isnull);
extern void array_free_iterator(ArrayIterator iterator);
***************
*** 293,298 **** extern ArrayType *create_singleton_array(FunctionCallInfo fcinfo,
--- 332,340 ----
extern Datum array_agg_transfn(PG_FUNCTION_ARGS);
extern Datum array_agg_finalfn(PG_FUNCTION_ARGS);
+ extern Datum array_agg_anyarray_transfn(PG_FUNCTION_ARGS);
+ extern Datum array_agg_anyarray_finalfn(PG_FUNCTION_ARGS);
+
/*
* prototypes for functions defined in array_typanalyze.c
*/
*** a/src/test/regress/expected/aggregates.out
--- b/src/test/regress/expected/aggregates.out
***************
*** 914,919 **** select array_agg(distinct a order by a desc nulls last)
--- 914,946 ----
{3,2,1,NULL}
(1 row)
+ -- array_agg(anyarray)
+ select array_agg(ar)
+ from (values ('{1,2}'::int[]), ('{3,4}'::int[])) v(ar);
+ array_agg
+ ---------------
+ {{1,2},{3,4}}
+ (1 row)
+
+ select array_agg(distinct ar order by ar desc)
+ from (select array[i / 2] from generate_series(1,10) a(i)) b(ar);
+ array_agg
+ ---------------------------
+ {{5},{4},{3},{2},{1},{0}}
+ (1 row)
+
+ select array_agg(ar)
+ from (select array_agg(array[i, i+1, i-1])
+ from generate_series(1,2) a(i)) b(ar);
+ array_agg
+ ---------------------
+ {{{1,2,0},{2,3,1}}}
+ (1 row)
+
+ select array_agg('{}'::int[]) from generate_series(1,2);
+ ERROR: cannot accumulate empty arrays
+ select array_agg(null::int[]) from generate_series(1,2);
+ ERROR: cannot accumulate null arrays
-- multi-arg aggs, strict/nonstrict, distinct/order by
select aggfstr(a,b,c)
from (values (1,3,'foo'),(0,null,null),(2,2,'bar'),(3,1,'baz')) v(a,b,c);
*** a/src/test/regress/expected/arrays.out
--- b/src/test/regress/expected/arrays.out
***************
*** 1521,1526 **** select array_agg(unique1) from tenk1 where unique1 < -15;
--- 1521,1543 ----
(1 row)
+ select array(select unique1 from tenk1 where unique1 < 15 order by unique1);
+ array
+ --------------------------------------
+ {0,1,2,3,4,5,6,7,8,9,10,11,12,13,14}
+ (1 row)
+
+ select array(select array[i,i/2] from generate_series(1,5) a(i));
+ array
+ ---------------------------------
+ {{1,0},{2,1},{3,1},{4,2},{5,2}}
+ (1 row)
+
+ -- cannot accumulate null arrays and empty arrays
+ select array(select null::int[]);
+ ERROR: cannot accumulate null arrays
+ select array(select '{}'::int[]);
+ ERROR: cannot accumulate empty arrays
select unnest(array[1,2,3]);
unnest
--------
*** a/src/test/regress/sql/aggregates.sql
--- b/src/test/regress/sql/aggregates.sql
***************
*** 322,327 **** select array_agg(distinct a order by a desc)
--- 322,339 ----
select array_agg(distinct a order by a desc nulls last)
from (values (1),(2),(1),(3),(null),(2)) v(a);
+ -- array_agg(anyarray)
+ select array_agg(ar)
+ from (values ('{1,2}'::int[]), ('{3,4}'::int[])) v(ar);
+ select array_agg(distinct ar order by ar desc)
+ from (select array[i / 2] from generate_series(1,10) a(i)) b(ar);
+ select array_agg(ar)
+ from (select array_agg(array[i, i+1, i-1])
+ from generate_series(1,2) a(i)) b(ar);
+
+ select array_agg('{}'::int[]) from generate_series(1,2);
+ select array_agg(null::int[]) from generate_series(1,2);
+
-- multi-arg aggs, strict/nonstrict, distinct/order by
select aggfstr(a,b,c)
*** a/src/test/regress/sql/arrays.sql
--- b/src/test/regress/sql/arrays.sql
***************
*** 432,437 **** select array_agg(ten) from (select ten from tenk1 where unique1 < 15 order by un
--- 432,443 ----
select array_agg(nullif(ten, 4)) from (select ten from tenk1 where unique1 < 15 order by unique1) ss;
select array_agg(unique1) from tenk1 where unique1 < -15;
+ select array(select unique1 from tenk1 where unique1 < 15 order by unique1);
+ select array(select array[i,i/2] from generate_series(1,5) a(i));
+ -- cannot accumulate null arrays and empty arrays
+ select array(select null::int[]);
+ select array(select '{}'::int[]);
+
select unnest(array[1,2,3]);
select * from unnest(array[1,2,3]);
select unnest(array[1,2,3,4.5]::float8[]);
Re: Function array_agg(array)
Hi
it looks well
doc:
http://www.postgresql.org/docs/9.4/static/sql-expressions.html#SQL-SYNTAX-ARRAY-CONSTRUCTORS
it should be fixed too
Regards
Pavel
2014-10-24 10:24 GMT+02:00 Ali Akbar <the.apaan@gmail.com>:
Show quoted text
Updated patch attached.
2014-10-22 20:51 GMT+07:00 Pavel Stehule <pavel.stehule@gmail.com>:
I agree with your proposal. I have a few comments to design:
1. patch doesn't hold documentation and regress tests, please append
it.i've added some regression tests in arrays.sql and aggregate.sql.
I've only found the documentation of array_agg. Where is the doc for
array(subselect) defined?2. this functionality (multidimensional aggregation) can be interesting
more times, so maybe some interface like array builder should be preferred.
We already have accumArrayResult and
makeArrayResult/makeMdArrayResult, haven't we?Actually array_agg(anyarray) can be implemented by using
accumArrayResult and makeMdArrayResult, but in the process we will
deconstruct the array data and null bit-flags into ArrayBuildState->dvalues
and dnulls. And we will reconstruct it through makeMdArrayResult. I think
this way it's not efficient, so i decided to reimplement it and memcpy the
array data and null flags as-is.aha, so isn't better to fix a performance for accumArrayResult ?
Ok, i'll go this route. While reading the code of array(subselect) as
you pointed below, i think the easiest way to implement support for both
array_agg(anyarray) and array(subselect) is to change accumArrayResult so
it operates both with scalar datum(s) and array datums, with performance
optimization for the latter.implemented it by modifying ArrayBuildState to ArrayBuildStateArray and
ArrayBuildStateScalar (do you have any idea for better struct naming?)In other places, i think it's clearer if we just use accumArrayResult and
makeArrayResult/makeMdArrayResult as API for building (multidimensional)
arrays.3. array_agg was consistent with array(subselect), so it should be
fixed toopostgres=# select array_agg(a) from test;
array_agg
-----------------------
{{1,2,3,4},{1,2,3,4}}
(1 row)postgres=# select array(select a from test);
ERROR: could not find array type for data type integer[]I'm pretty new in postgresql development. Can you point out where is
array(subselect) implemented?where you can start?
postgres=# explain select array(select a from test);
ERROR: 42704: could not find array type for data type integer[]
LOCATION: exprType, nodeFuncs.c:116look to code ... your magic keyword is a ARRAY_SUBLINK .. search in
postgresql sources this keywordattention: probably we don't would to allow arrays everywhere.
I've changed the places where i think it's appropriate.
4. why you use a magic constant (64) there?
+ astate->abytes = 64 * (ndatabytes == 0 ? 1 : ndatabytes); + astate->aitems = 64 * nitems;+ astate->nullbitmap = (bits8 *) + repalloc(astate->nullbitmap, (astate->aitems + 7) / 8);just follow the arbitrary size choosen in accumArrayResult
(arrayfuncs.c):
astate->alen = 64; /* arbitrary starting array size */it can be any number not too small and too big. Too small, and we will
realloc shortly. Too big, we will end up wasting memory.you can try to alloc 1KB instead as start -- it is used more times in
Postgres. Then a overhead is max 1KB per agg call - what is acceptable.You take this value from accumArrayResult, but it is targeted for
shorted scalars - you should to expect so any array will be much larger.this patch allocates 1KB (1024 bytes) if the ndatabytes is < 512bytes. If
it is larger, it allocates 4 * size. For nullbitmap, it allocates 4 *
number of items in array.Regards,
--
Ali Akbar
Re: Function array_agg(array)
2014-10-24 15:48 GMT+07:00 Pavel Stehule <pavel.stehule@gmail.com>:
Hi
it looks well
doc:
http://www.postgresql.org/docs/9.4/static/sql-expressions.html#SQL-SYNTAX-ARRAY-CONSTRUCTORS
it should be fixed tooRegards
Pavel
doc updated with additional example for array(subselect). patch attached.
Regards,
--
Ali Akbar
Attachments:
array_agg_anyarray-5.patchtext/x-patch; charset=US-ASCII; name=array_agg_anyarray-5.patchDownload
*** a/doc/src/sgml/func.sgml
--- b/doc/src/sgml/func.sgml
***************
*** 12046,12051 **** NULL baz</literallayout>(3 rows)</entry>
--- 12046,12067 ----
<row>
<entry>
<indexterm>
+ <primary>array_agg</primary>
+ </indexterm>
+ <function>array_agg(<replaceable class="parameter">anyarray</replaceable>)</function>
+ </entry>
+ <entry>
+ any
+ </entry>
+ <entry>
+ the same array type as input type
+ </entry>
+ <entry>input arrays, aggregated into higher-order multidimesional array. Rejects NULL and empty array as input.</entry>
+ </row>
+
+ <row>
+ <entry>
+ <indexterm>
<primary>average</primary>
</indexterm>
<indexterm>
*** a/doc/src/sgml/syntax.sgml
--- b/doc/src/sgml/syntax.sgml
***************
*** 2238,2243 **** SELECT ARRAY(SELECT oid FROM pg_proc WHERE proname LIKE 'bytea%');
--- 2238,2248 ----
array
-----------------------------------------------------------------------
{2011,1954,1948,1952,1951,1244,1950,2005,1949,1953,2006,31,2412,2413}
+
+ SELECT ARRAY(SELECT array(select i) FROM generate_series(1,5) a(i));
+ array
+ -----------------------
+ {{1},{2},{3},{4},{5}}
(1 row)
</programlisting>
The subquery must return a single column. The resulting
*** a/src/backend/nodes/nodeFuncs.c
--- b/src/backend/nodes/nodeFuncs.c
***************
*** 108,119 **** exprType(const Node *expr)
type = exprType((Node *) tent->expr);
if (sublink->subLinkType == ARRAY_SUBLINK)
{
! type = get_array_type(type);
! if (!OidIsValid(type))
! ereport(ERROR,
! (errcode(ERRCODE_UNDEFINED_OBJECT),
! errmsg("could not find array type for data type %s",
! format_type_be(exprType((Node *) tent->expr)))));
}
}
else if (sublink->subLinkType == MULTIEXPR_SUBLINK)
--- 108,123 ----
type = exprType((Node *) tent->expr);
if (sublink->subLinkType == ARRAY_SUBLINK)
{
! if (!OidIsValid(get_element_type(type)))
! {
! /* not array, so check for its array type */
! type = get_array_type(type);
! if (!OidIsValid(type))
! ereport(ERROR,
! (errcode(ERRCODE_UNDEFINED_OBJECT),
! errmsg("could not find array type for data type %s",
! format_type_be(exprType((Node *) tent->expr)))));
! }
}
}
else if (sublink->subLinkType == MULTIEXPR_SUBLINK)
***************
*** 139,150 **** exprType(const Node *expr)
type = subplan->firstColType;
if (subplan->subLinkType == ARRAY_SUBLINK)
{
! type = get_array_type(type);
! if (!OidIsValid(type))
! ereport(ERROR,
! (errcode(ERRCODE_UNDEFINED_OBJECT),
! errmsg("could not find array type for data type %s",
! format_type_be(subplan->firstColType))));
}
}
else if (subplan->subLinkType == MULTIEXPR_SUBLINK)
--- 143,158 ----
type = subplan->firstColType;
if (subplan->subLinkType == ARRAY_SUBLINK)
{
! if (!OidIsValid(get_element_type(type)))
! {
! /* not array, so check for its array type */
! type = get_array_type(type);
! if (!OidIsValid(type))
! ereport(ERROR,
! (errcode(ERRCODE_UNDEFINED_OBJECT),
! errmsg("could not find array type for data type %s",
! format_type_be(subplan->firstColType))));
! }
}
}
else if (subplan->subLinkType == MULTIEXPR_SUBLINK)
*** a/src/backend/optimizer/plan/subselect.c
--- b/src/backend/optimizer/plan/subselect.c
***************
*** 668,677 **** build_subplan(PlannerInfo *root, Plan *plan, PlannerInfo *subroot,
Assert(!te->resjunk);
Assert(testexpr == NULL);
! arraytype = get_array_type(exprType((Node *) te->expr));
! if (!OidIsValid(arraytype))
! elog(ERROR, "could not find array type for datatype %s",
! format_type_be(exprType((Node *) te->expr)));
prm = generate_new_param(root,
arraytype,
exprTypmod((Node *) te->expr),
--- 668,683 ----
Assert(!te->resjunk);
Assert(testexpr == NULL);
!
! arraytype = exprType((Node *) te->expr);
! if (!OidIsValid(get_element_type(arraytype)))
! {
! /* not array, so get the array type */
! arraytype = get_array_type(exprType((Node *) te->expr));
! if (!OidIsValid(arraytype))
! elog(ERROR, "could not find array type for datatype %s",
! format_type_be(exprType((Node *) te->expr)));
! }
prm = generate_new_param(root,
arraytype,
exprTypmod((Node *) te->expr),
*** a/src/backend/utils/adt/array_userfuncs.c
--- b/src/backend/utils/adt/array_userfuncs.c
***************
*** 16,22 ****
#include "utils/builtins.h"
#include "utils/lsyscache.h"
-
/*-----------------------------------------------------------------------------
* array_push :
* push an element onto either end of a one-dimensional array
--- 16,21 ----
*** a/src/backend/utils/adt/arrayfuncs.c
--- b/src/backend/utils/adt/arrayfuncs.c
***************
*** 145,151 **** static int width_bucket_array_variable(Datum operand,
Oid collation,
TypeCacheEntry *typentry);
-
/*
* array_in :
* converts an array from the external format in "string" to
--- 145,150 ----
*** a/src/include/catalog/pg_aggregate.h
--- b/src/include/catalog/pg_aggregate.h
***************
*** 275,280 **** DATA(insert ( 2901 n 0 xmlconcat2 - - - - f f 0 142 0 0 0 _null_
--- 275,281 ----
/* array */
DATA(insert ( 2335 n 0 array_agg_transfn array_agg_finalfn - - - t f 0 2281 0 0 0 _null_ _null_ ));
+ DATA(insert ( 6005 n 0 array_agg_anyarray_transfn array_agg_anyarray_finalfn - - - t f 0 2281 0 0 0 _null_ _null_ ));
/* text */
DATA(insert ( 3538 n 0 string_agg_transfn string_agg_finalfn - - - f f 0 2281 0 0 0 _null_ _null_ ));
*** a/src/include/catalog/pg_proc.h
--- b/src/include/catalog/pg_proc.h
***************
*** 879,889 **** DATA(insert OID = 3167 ( array_remove PGNSP PGUID 12 1 0 0 0 f f f f f f i 2
DESCR("remove any occurrences of an element from an array");
DATA(insert OID = 3168 ( array_replace PGNSP PGUID 12 1 0 0 0 f f f f f f i 3 0 2277 "2277 2283 2283" _null_ _null_ _null_ _null_ array_replace _null_ _null_ _null_ ));
DESCR("replace any occurrences of an element in an array");
! DATA(insert OID = 2333 ( array_agg_transfn PGNSP PGUID 12 1 0 0 0 f f f f f f i 2 0 2281 "2281 2283" _null_ _null_ _null_ _null_ array_agg_transfn _null_ _null_ _null_ ));
DESCR("aggregate transition function");
! DATA(insert OID = 2334 ( array_agg_finalfn PGNSP PGUID 12 1 0 0 0 f f f f f f i 2 0 2277 "2281 2283" _null_ _null_ _null_ _null_ array_agg_finalfn _null_ _null_ _null_ ));
DESCR("aggregate final function");
! DATA(insert OID = 2335 ( array_agg PGNSP PGUID 12 1 0 0 0 t f f f f f i 1 0 2277 "2283" _null_ _null_ _null_ _null_ aggregate_dummy _null_ _null_ _null_ ));
DESCR("concatenate aggregate input into an array");
DATA(insert OID = 3218 ( width_bucket PGNSP PGUID 12 1 0 0 0 f f f f t f i 2 0 23 "2283 2277" _null_ _null_ _null_ _null_ width_bucket_array _null_ _null_ _null_ ));
DESCR("bucket number of operand given a sorted array of bucket lower bounds");
--- 879,895 ----
DESCR("remove any occurrences of an element from an array");
DATA(insert OID = 3168 ( array_replace PGNSP PGUID 12 1 0 0 0 f f f f f f i 3 0 2277 "2277 2283 2283" _null_ _null_ _null_ _null_ array_replace _null_ _null_ _null_ ));
DESCR("replace any occurrences of an element in an array");
! DATA(insert OID = 2333 ( array_agg_transfn PGNSP PGUID 12 1 0 0 0 f f f f f f i 2 0 2281 "2281 2776" _null_ _null_ _null_ _null_ array_agg_transfn _null_ _null_ _null_ ));
DESCR("aggregate transition function");
! DATA(insert OID = 2334 ( array_agg_finalfn PGNSP PGUID 12 1 0 0 0 f f f f f f i 2 0 2277 "2281 2776" _null_ _null_ _null_ _null_ array_agg_finalfn _null_ _null_ _null_ ));
DESCR("aggregate final function");
! DATA(insert OID = 2335 ( array_agg PGNSP PGUID 12 1 0 0 0 t f f f f f i 1 0 2277 "2776" _null_ _null_ _null_ _null_ aggregate_dummy _null_ _null_ _null_ ));
! DESCR("concatenate aggregate input into an array");
! DATA(insert OID = 6003 ( array_agg_anyarray_transfn PGNSP PGUID 12 1 0 0 0 f f f f f f i 2 0 2281 "2281 2277" _null_ _null_ _null_ _null_ array_agg_anyarray_transfn _null_ _null_ _null_ ));
! DESCR("aggregate transition function");
! DATA(insert OID = 6004 ( array_agg_anyarray_finalfn PGNSP PGUID 12 1 0 0 0 f f f f f f i 2 0 2277 "2281 2277" _null_ _null_ _null_ _null_ array_agg_anyarray_finalfn _null_ _null_ _null_ ));
! DESCR("aggregate final function");
! DATA(insert OID = 6005 ( array_agg PGNSP PGUID 12 1 0 0 0 t f f f f f i 1 0 2277 "2277" _null_ _null_ _null_ _null_ aggregate_dummy _null_ _null_ _null_ ));
DESCR("concatenate aggregate input into an array");
DATA(insert OID = 3218 ( width_bucket PGNSP PGUID 12 1 0 0 0 f f f f t f i 2 0 23 "2283 2277" _null_ _null_ _null_ _null_ width_bucket_array _null_ _null_ _null_ ));
DESCR("bucket number of operand given a sorted array of bucket lower bounds");
*** a/src/include/utils/array.h
--- b/src/include/utils/array.h
***************
*** 76,94 **** typedef struct
/*
* working state for accumArrayResult() and friends
*/
typedef struct ArrayBuildState
{
MemoryContext mcontext; /* where all the temp stuff is kept */
Datum *dvalues; /* array of accumulated Datums */
bool *dnulls; /* array of is-null flags for Datums */
int alen; /* allocated length of above arrays */
int nelems; /* number of valid entries in above arrays */
! Oid element_type; /* data type of the Datums */
! int16 typlen; /* needed info about datatype */
! bool typbyval;
! char typalign;
! } ArrayBuildState;
/*
* structure to cache type metadata needed for array manipulation
--- 76,132 ----
/*
* working state for accumArrayResult() and friends
+ *
+ * is_array_accum: whether accumulating array values.
+ * (if true must be casted to ArrayBuildStateArray, else
+ * cast to ArrayBuildStateScalar)
*/
typedef struct ArrayBuildState
{
+ bool is_array_accum;
MemoryContext mcontext; /* where all the temp stuff is kept */
+ Oid element_type; /* data type of the Datums */
+ int16 typlen; /* needed info about datatype */
+ bool typbyval;
+ char typalign;
+ } ArrayBuildState;
+
+ /*
+ * array build state for array accumulation of scalar datums
+ */
+ typedef struct ArrayBuildStateScalar
+ {
+ ArrayBuildState astate;
+
Datum *dvalues; /* array of accumulated Datums */
bool *dnulls; /* array of is-null flags for Datums */
int alen; /* allocated length of above arrays */
int nelems; /* number of valid entries in above arrays */
! } ArrayBuildStateScalar;
!
!
! /*
! * array build state for array accumulation of array datums
! */
! typedef struct
! {
! ArrayBuildState astate;
!
! char *data; /* array of accumulated data */
! bits8 *nullbitmap; /* bitmap of is-null flags for data */
!
! int abytes; /* allocated length of above arrays */
! int aitems; /* allocated length of above arrays */
! int nbytes; /* number of used bytes in above arrays */
! int nitems; /* number of elements in above arrays */
! int narray; /* number of array accumulated */
!
! int ndims; /* element dimensions */
! int *dims;
! int *lbs;
!
! bool hasnull; /* any element has null */
! } ArrayBuildStateArray;
/*
* structure to cache type metadata needed for array manipulation
***************
*** 260,266 **** extern Datum makeArrayResult(ArrayBuildState *astate,
MemoryContext rcontext);
extern Datum makeMdArrayResult(ArrayBuildState *astate, int ndims,
int *dims, int *lbs, MemoryContext rcontext, bool release);
!
extern ArrayIterator array_create_iterator(ArrayType *arr, int slice_ndim);
extern bool array_iterate(ArrayIterator iterator, Datum *value, bool *isnull);
extern void array_free_iterator(ArrayIterator iterator);
--- 298,305 ----
MemoryContext rcontext);
extern Datum makeMdArrayResult(ArrayBuildState *astate, int ndims,
int *dims, int *lbs, MemoryContext rcontext, bool release);
! extern Datum makeArrayResultArray(ArrayBuildStateArray *astate,
! MemoryContext rcontext, bool release);
extern ArrayIterator array_create_iterator(ArrayType *arr, int slice_ndim);
extern bool array_iterate(ArrayIterator iterator, Datum *value, bool *isnull);
extern void array_free_iterator(ArrayIterator iterator);
***************
*** 293,298 **** extern ArrayType *create_singleton_array(FunctionCallInfo fcinfo,
--- 332,340 ----
extern Datum array_agg_transfn(PG_FUNCTION_ARGS);
extern Datum array_agg_finalfn(PG_FUNCTION_ARGS);
+ extern Datum array_agg_anyarray_transfn(PG_FUNCTION_ARGS);
+ extern Datum array_agg_anyarray_finalfn(PG_FUNCTION_ARGS);
+
/*
* prototypes for functions defined in array_typanalyze.c
*/
*** a/src/test/regress/expected/aggregates.out
--- b/src/test/regress/expected/aggregates.out
***************
*** 914,919 **** select array_agg(distinct a order by a desc nulls last)
--- 914,946 ----
{3,2,1,NULL}
(1 row)
+ -- array_agg(anyarray)
+ select array_agg(ar)
+ from (values ('{1,2}'::int[]), ('{3,4}'::int[])) v(ar);
+ array_agg
+ ---------------
+ {{1,2},{3,4}}
+ (1 row)
+
+ select array_agg(distinct ar order by ar desc)
+ from (select array[i / 2] from generate_series(1,10) a(i)) b(ar);
+ array_agg
+ ---------------------------
+ {{5},{4},{3},{2},{1},{0}}
+ (1 row)
+
+ select array_agg(ar)
+ from (select array_agg(array[i, i+1, i-1])
+ from generate_series(1,2) a(i)) b(ar);
+ array_agg
+ ---------------------
+ {{{1,2,0},{2,3,1}}}
+ (1 row)
+
+ select array_agg('{}'::int[]) from generate_series(1,2);
+ ERROR: cannot accumulate empty arrays
+ select array_agg(null::int[]) from generate_series(1,2);
+ ERROR: cannot accumulate null arrays
-- multi-arg aggs, strict/nonstrict, distinct/order by
select aggfstr(a,b,c)
from (values (1,3,'foo'),(0,null,null),(2,2,'bar'),(3,1,'baz')) v(a,b,c);
*** a/src/test/regress/expected/arrays.out
--- b/src/test/regress/expected/arrays.out
***************
*** 1521,1526 **** select array_agg(unique1) from tenk1 where unique1 < -15;
--- 1521,1543 ----
(1 row)
+ select array(select unique1 from tenk1 where unique1 < 15 order by unique1);
+ array
+ --------------------------------------
+ {0,1,2,3,4,5,6,7,8,9,10,11,12,13,14}
+ (1 row)
+
+ select array(select array[i,i/2] from generate_series(1,5) a(i));
+ array
+ ---------------------------------
+ {{1,0},{2,1},{3,1},{4,2},{5,2}}
+ (1 row)
+
+ -- cannot accumulate null arrays and empty arrays
+ select array(select null::int[]);
+ ERROR: cannot accumulate null arrays
+ select array(select '{}'::int[]);
+ ERROR: cannot accumulate empty arrays
select unnest(array[1,2,3]);
unnest
--------
*** a/src/test/regress/sql/aggregates.sql
--- b/src/test/regress/sql/aggregates.sql
***************
*** 322,327 **** select array_agg(distinct a order by a desc)
--- 322,339 ----
select array_agg(distinct a order by a desc nulls last)
from (values (1),(2),(1),(3),(null),(2)) v(a);
+ -- array_agg(anyarray)
+ select array_agg(ar)
+ from (values ('{1,2}'::int[]), ('{3,4}'::int[])) v(ar);
+ select array_agg(distinct ar order by ar desc)
+ from (select array[i / 2] from generate_series(1,10) a(i)) b(ar);
+ select array_agg(ar)
+ from (select array_agg(array[i, i+1, i-1])
+ from generate_series(1,2) a(i)) b(ar);
+
+ select array_agg('{}'::int[]) from generate_series(1,2);
+ select array_agg(null::int[]) from generate_series(1,2);
+
-- multi-arg aggs, strict/nonstrict, distinct/order by
select aggfstr(a,b,c)
*** a/src/test/regress/sql/arrays.sql
--- b/src/test/regress/sql/arrays.sql
***************
*** 432,437 **** select array_agg(ten) from (select ten from tenk1 where unique1 < 15 order by un
--- 432,443 ----
select array_agg(nullif(ten, 4)) from (select ten from tenk1 where unique1 < 15 order by unique1) ss;
select array_agg(unique1) from tenk1 where unique1 < -15;
+ select array(select unique1 from tenk1 where unique1 < 15 order by unique1);
+ select array(select array[i,i/2] from generate_series(1,5) a(i));
+ -- cannot accumulate null arrays and empty arrays
+ select array(select null::int[]);
+ select array(select '{}'::int[]);
+
select unnest(array[1,2,3]);
select * from unnest(array[1,2,3]);
select unnest(array[1,2,3,4.5]::float8[]);
Re: Function array_agg(array)
Hi
some in last patch is wrong, I cannot to compile it:
arrayfuncs.c: In function ‘accumArrayResult’:
arrayfuncs.c:4603:9: error: ‘ArrayBuildState’ has no member named ‘alen’
astate->alen = 64; /* arbitrary starting array size */
^
arrayfuncs.c:4604:9: error: ‘ArrayBuildState’ has no member named ‘dvalues’
astate->dvalues = (Datum *) palloc(astate->alen * sizeof(Datum));
^
arrayfuncs.c:4604:44: error: ‘ArrayBuildState’ has no member named ‘alen’
astate->dvalues = (Datum *) palloc(astate->alen * sizeof(Datum));
^
arrayfuncs.c:4605:9: error: ‘ArrayBuildState’ has no member named ‘dnulls’
astate->dnulls = (bool *) palloc(astate->alen * sizeof(bool));
^
arrayfuncs.c:4605:42: error: ‘ArrayBuildState’ has no member named ‘alen’
astate->dnulls = (bool *) palloc(astate->alen * sizeof(bool));
^
arrayfuncs.c:4606:9: error: ‘ArrayBuildState’ has no member named ‘nelems’
astate->nelems = 0;
^
arrayfuncs.c:4618:13: error: ‘ArrayBuildState’ has no member named ‘nelems’
if (astate->nelems >= astate->alen)
^
arrayfuncs.c:4618:31: error: ‘ArrayBuildState’ has no member named ‘alen’
if (astate->nelems >= astate->alen)
^
arrayfuncs.c:4620:10: error: ‘ArrayBuildState’ has no member named ‘alen’
astate->alen *= 2;
2014-10-24 11:24 GMT+02:00 Ali Akbar <the.apaan@gmail.com>:
Show quoted text
2014-10-24 15:48 GMT+07:00 Pavel Stehule <pavel.stehule@gmail.com>:
Hi
it looks well
doc:
http://www.postgresql.org/docs/9.4/static/sql-expressions.html#SQL-SYNTAX-ARRAY-CONSTRUCTORS
it should be fixed tooRegards
Pavel
doc updated with additional example for array(subselect). patch attached.
Regards,
--
Ali Akbar
Re: Function array_agg(array)
2014-10-24 16:26 GMT+07:00 Pavel Stehule <pavel.stehule@gmail.com>:
Hi
some in last patch is wrong, I cannot to compile it:
arrayfuncs.c: In function ‘accumArrayResult’:
arrayfuncs.c:4603:9: error: ‘ArrayBuildState’ has no member named ‘alen’
astate->alen = 64; /* arbitrary starting array size */
^
arrayfuncs.c:4604:9: error: ‘ArrayBuildState’ has no member named ‘dvalues’
astate->dvalues = (Datum *) palloc(astate->alen * sizeof(Datum));
^
arrayfuncs.c:4604:44: error: ‘ArrayBuildState’ has no member named ‘alen’
astate->dvalues = (Datum *) palloc(astate->alen * sizeof(Datum));
^
arrayfuncs.c:4605:9: error: ‘ArrayBuildState’ has no member named ‘dnulls’
astate->dnulls = (bool *) palloc(astate->alen * sizeof(bool));
^
arrayfuncs.c:4605:42: error: ‘ArrayBuildState’ has no member named ‘alen’
astate->dnulls = (bool *) palloc(astate->alen * sizeof(bool));
^
arrayfuncs.c:4606:9: error: ‘ArrayBuildState’ has no member named ‘nelems’
astate->nelems = 0;
^
arrayfuncs.c:4618:13: error: ‘ArrayBuildState’ has no member named ‘nelems’
if (astate->nelems >= astate->alen)
^
arrayfuncs.c:4618:31: error: ‘ArrayBuildState’ has no member named ‘alen’
if (astate->nelems >= astate->alen)
^
arrayfuncs.c:4620:10: error: ‘ArrayBuildState’ has no member named ‘alen’
astate->alen *= 2;
Sorry, correct patch attached.
This patch is in patience format (git --patience ..). In previous patches,
i use context format (git --patience ... | filterdiff --format=context),
but it turns out that some modification is lost.
--
Ali Akbar
Attachments:
array_agg_anyarray-6.patchtext/x-patch; charset=US-ASCII; name=array_agg_anyarray-6.patchDownload
diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml
index 7e5bcd9..f59738a 100644
--- a/doc/src/sgml/func.sgml
+++ b/doc/src/sgml/func.sgml
@@ -12046,6 +12046,22 @@ NULL baz</literallayout>(3 rows)</entry>
<row>
<entry>
<indexterm>
+ <primary>array_agg</primary>
+ </indexterm>
+ <function>array_agg(<replaceable class="parameter">anyarray</replaceable>)</function>
+ </entry>
+ <entry>
+ any
+ </entry>
+ <entry>
+ the same array type as input type
+ </entry>
+ <entry>input arrays, aggregated into higher-order multidimesional array. Rejects NULL and empty array as input.</entry>
+ </row>
+
+ <row>
+ <entry>
+ <indexterm>
<primary>average</primary>
</indexterm>
<indexterm>
diff --git a/doc/src/sgml/syntax.sgml b/doc/src/sgml/syntax.sgml
index 2f0680f..8c182a4 100644
--- a/doc/src/sgml/syntax.sgml
+++ b/doc/src/sgml/syntax.sgml
@@ -2238,6 +2238,11 @@ SELECT ARRAY(SELECT oid FROM pg_proc WHERE proname LIKE 'bytea%');
array
-----------------------------------------------------------------------
{2011,1954,1948,1952,1951,1244,1950,2005,1949,1953,2006,31,2412,2413}
+
+SELECT ARRAY(SELECT array(select i) FROM generate_series(1,5) a(i));
+ array
+-----------------------
+ {{1},{2},{3},{4},{5}}
(1 row)
</programlisting>
The subquery must return a single column. The resulting
diff --git a/src/backend/nodes/nodeFuncs.c b/src/backend/nodes/nodeFuncs.c
index 41e973b..0261fcb 100644
--- a/src/backend/nodes/nodeFuncs.c
+++ b/src/backend/nodes/nodeFuncs.c
@@ -108,12 +108,16 @@ exprType(const Node *expr)
type = exprType((Node *) tent->expr);
if (sublink->subLinkType == ARRAY_SUBLINK)
{
- type = get_array_type(type);
- if (!OidIsValid(type))
- ereport(ERROR,
- (errcode(ERRCODE_UNDEFINED_OBJECT),
- errmsg("could not find array type for data type %s",
- format_type_be(exprType((Node *) tent->expr)))));
+ if (!OidIsValid(get_element_type(type)))
+ {
+ /* not array, so check for its array type */
+ type = get_array_type(type);
+ if (!OidIsValid(type))
+ ereport(ERROR,
+ (errcode(ERRCODE_UNDEFINED_OBJECT),
+ errmsg("could not find array type for data type %s",
+ format_type_be(exprType((Node *) tent->expr)))));
+ }
}
}
else if (sublink->subLinkType == MULTIEXPR_SUBLINK)
@@ -139,12 +143,16 @@ exprType(const Node *expr)
type = subplan->firstColType;
if (subplan->subLinkType == ARRAY_SUBLINK)
{
- type = get_array_type(type);
- if (!OidIsValid(type))
- ereport(ERROR,
- (errcode(ERRCODE_UNDEFINED_OBJECT),
- errmsg("could not find array type for data type %s",
- format_type_be(subplan->firstColType))));
+ if (!OidIsValid(get_element_type(type)))
+ {
+ /* not array, so check for its array type */
+ type = get_array_type(type);
+ if (!OidIsValid(type))
+ ereport(ERROR,
+ (errcode(ERRCODE_UNDEFINED_OBJECT),
+ errmsg("could not find array type for data type %s",
+ format_type_be(subplan->firstColType))));
+ }
}
}
else if (subplan->subLinkType == MULTIEXPR_SUBLINK)
diff --git a/src/backend/optimizer/plan/subselect.c b/src/backend/optimizer/plan/subselect.c
index 3e7dc85..8fc8b49 100644
--- a/src/backend/optimizer/plan/subselect.c
+++ b/src/backend/optimizer/plan/subselect.c
@@ -668,10 +668,16 @@ build_subplan(PlannerInfo *root, Plan *plan, PlannerInfo *subroot,
Assert(!te->resjunk);
Assert(testexpr == NULL);
- arraytype = get_array_type(exprType((Node *) te->expr));
- if (!OidIsValid(arraytype))
- elog(ERROR, "could not find array type for datatype %s",
- format_type_be(exprType((Node *) te->expr)));
+
+ arraytype = exprType((Node *) te->expr);
+ if (!OidIsValid(get_element_type(arraytype)))
+ {
+ /* not array, so get the array type */
+ arraytype = get_array_type(exprType((Node *) te->expr));
+ if (!OidIsValid(arraytype))
+ elog(ERROR, "could not find array type for datatype %s",
+ format_type_be(exprType((Node *) te->expr)));
+ }
prm = generate_new_param(root,
arraytype,
exprTypmod((Node *) te->expr),
diff --git a/src/backend/utils/adt/array_userfuncs.c b/src/backend/utils/adt/array_userfuncs.c
index 831466d..9fdb25b 100644
--- a/src/backend/utils/adt/array_userfuncs.c
+++ b/src/backend/utils/adt/array_userfuncs.c
@@ -16,7 +16,6 @@
#include "utils/builtins.h"
#include "utils/lsyscache.h"
-
/*-----------------------------------------------------------------------------
* array_push :
* push an element onto either end of a one-dimensional array
@@ -513,8 +512,6 @@ array_agg_finalfn(PG_FUNCTION_ARGS)
{
Datum result;
ArrayBuildState *state;
- int dims[1];
- int lbs[1];
/*
* Test for null before Asserting we are in right context. This is to
@@ -529,18 +526,43 @@ array_agg_finalfn(PG_FUNCTION_ARGS)
state = (ArrayBuildState *) PG_GETARG_POINTER(0);
- dims[0] = state->nelems;
- lbs[0] = 1;
-
/*
* Make the result. We cannot release the ArrayBuildState because
* sometimes aggregate final functions are re-executed. Rather, it is
* nodeAgg.c's responsibility to reset the aggcontext when it's safe to do
* so.
*/
- result = makeMdArrayResult(state, 1, dims, lbs,
- CurrentMemoryContext,
- false);
+ if (!state->is_array_accum)
+ {
+ int dims[1];
+ int lbs[1];
+
+ dims[0] = ((ArrayBuildStateScalar *) state)->nelems;
+ lbs[0] = 1;
+
+ result = makeMdArrayResult(state, 1, dims, lbs,
+ CurrentMemoryContext,
+ false);
+ }
+ else
+ result = makeArrayResultArray((ArrayBuildStateArray *) state,
+ CurrentMemoryContext,
+ false);
PG_RETURN_DATUM(result);
}
+
+/*
+ * ARRAY_AGG(anyarray) aggregate function
+ */
+Datum
+array_agg_anyarray_transfn(PG_FUNCTION_ARGS)
+{
+ return array_agg_transfn(fcinfo);
+}
+
+Datum
+array_agg_anyarray_finalfn(PG_FUNCTION_ARGS)
+{
+ return array_agg_finalfn(fcinfo);
+}
diff --git a/src/backend/utils/adt/arrayfuncs.c b/src/backend/utils/adt/arrayfuncs.c
index 6c8b41d..582912e 100644
--- a/src/backend/utils/adt/arrayfuncs.c
+++ b/src/backend/utils/adt/arrayfuncs.c
@@ -145,7 +145,6 @@ static int width_bucket_array_variable(Datum operand,
Oid collation,
TypeCacheEntry *typentry);
-
/*
* array_in :
* converts an array from the external format in "string" to
@@ -4588,9 +4587,13 @@ accumArrayResult(ArrayBuildState *astate,
MemoryContext arr_context,
oldcontext;
+ ArrayBuildStateScalar *astate_scalar = NULL; /* for scalar datum accumulation */
+ ArrayBuildStateArray *astate_array = NULL; /* for array datum accumulation */
+
if (astate == NULL)
{
/* First time through --- initialize */
+ Oid subelement_type = get_element_type(element_type);
/* Make a temporary context to hold all the junk */
arr_context = AllocSetContextCreate(rcontext,
@@ -4599,60 +4602,224 @@ accumArrayResult(ArrayBuildState *astate,
ALLOCSET_DEFAULT_INITSIZE,
ALLOCSET_DEFAULT_MAXSIZE);
oldcontext = MemoryContextSwitchTo(arr_context);
- astate = (ArrayBuildState *) palloc(sizeof(ArrayBuildState));
- astate->mcontext = arr_context;
- astate->alen = 64; /* arbitrary starting array size */
- astate->dvalues = (Datum *) palloc(astate->alen * sizeof(Datum));
- astate->dnulls = (bool *) palloc(astate->alen * sizeof(bool));
- astate->nelems = 0;
- astate->element_type = element_type;
- get_typlenbyvalalign(element_type,
- &astate->typlen,
- &astate->typbyval,
- &astate->typalign);
+
+ if (subelement_type == InvalidOid)
+ {
+ /* scalar accumulate */
+ astate_scalar = (ArrayBuildStateScalar *) palloc(sizeof(ArrayBuildStateScalar));
+ astate = (ArrayBuildState *) astate_scalar;
+ astate->is_array_accum = false;
+ astate->mcontext = arr_context;
+
+ astate_scalar->alen = 64; /* arbitrary starting array size */
+ astate_scalar->dvalues = (Datum *) palloc(astate_scalar->alen * sizeof(Datum));
+ astate_scalar->dnulls = (bool *) palloc(astate_scalar->alen * sizeof(bool));
+ astate_scalar->nelems = 0;
+
+ astate->element_type = element_type;
+ get_typlenbyvalalign(element_type,
+ &astate->typlen,
+ &astate->typbyval,
+ &astate->typalign);
+ }
+ else
+ {
+ /* array accumulate */
+ astate_array = (ArrayBuildStateArray *) palloc(sizeof(ArrayBuildStateArray));
+ astate = (ArrayBuildState *) astate_array;
+ astate->is_array_accum = true;
+ astate->mcontext = arr_context;
+
+ astate_array->abytes = 0;
+ astate_array->aitems = 0;
+ astate_array->data = NULL;
+ astate_array->nullbitmap = NULL;
+ astate_array->nitems = 0;
+ astate_array->narray = 0;
+
+ astate->element_type = subelement_type;
+ get_typlenbyvalalign(subelement_type,
+ &astate->typlen,
+ &astate->typbyval,
+ &astate->typalign);
+ }
}
else
{
oldcontext = MemoryContextSwitchTo(astate->mcontext);
- Assert(astate->element_type == element_type);
- /* enlarge dvalues[]/dnulls[] if needed */
- if (astate->nelems >= astate->alen)
+ if (!astate->is_array_accum)
{
- astate->alen *= 2;
- astate->dvalues = (Datum *)
- repalloc(astate->dvalues, astate->alen * sizeof(Datum));
- astate->dnulls = (bool *)
- repalloc(astate->dnulls, astate->alen * sizeof(bool));
+ Assert(astate->element_type == element_type);
+ astate_scalar = (ArrayBuildStateScalar *) astate;
+
+ /* enlarge dvalues[]/dnulls[] if needed */
+ if (astate_scalar->nelems >= astate_scalar->alen)
+ {
+ astate_scalar->alen *= 2;
+ astate_scalar->dvalues = (Datum *)
+ repalloc(astate_scalar->dvalues, astate_scalar->alen * sizeof(Datum));
+ astate_scalar->dnulls = (bool *)
+ repalloc(astate_scalar->dnulls, astate_scalar->alen * sizeof(bool));
+ }
+ }
+ else
+ {
+ Assert(astate->element_type == get_element_type(element_type));
+ astate_array = (ArrayBuildStateArray *) astate;
+ /*
+ * array accumulate alloc & realloc placed below because
+ * we need to know its length first
+ */
}
}
- /*
- * Ensure pass-by-ref stuff is copied into mcontext; and detoast it too if
- * it's varlena. (You might think that detoasting is not needed here
- * because construct_md_array can detoast the array elements later.
- * However, we must not let construct_md_array modify the ArrayBuildState
- * because that would mean array_agg_finalfn damages its input, which is
- * verboten. Also, this way frequently saves one copying step.)
- */
- if (!disnull && !astate->typbyval)
+ if (!astate->is_array_accum)
{
- if (astate->typlen == -1)
- dvalue = PointerGetDatum(PG_DETOAST_DATUM_COPY(dvalue));
- else
- dvalue = datumCopy(dvalue, astate->typbyval, astate->typlen);
+ /*
+ * Ensure pass-by-ref stuff is copied into mcontext; and detoast it too if
+ * it's varlena. (You might think that detoasting is not needed here
+ * because construct_md_array can detoast the array elements later.
+ * However, we must not let construct_md_array modify the ArrayBuildState
+ * because that would mean array_agg_finalfn damages its input, which is
+ * verboten. Also, this way frequently saves one copying step.)
+ */
+ if (!disnull && !astate->typbyval)
+ {
+ if (astate->typlen == -1)
+ dvalue = PointerGetDatum(PG_DETOAST_DATUM_COPY(dvalue));
+ else
+ dvalue = datumCopy(dvalue, astate->typbyval, astate->typlen);
+ }
+
+ astate_scalar->dvalues[astate_scalar->nelems] = dvalue;
+ astate_scalar->dnulls[astate_scalar->nelems] = disnull;
+ astate_scalar->nelems++;
}
+ else
+ {
+ ArrayType *arg;
+ int *dims,
+ *lbs,
+ ndims,
+ nitems,
+ ndatabytes;
+ char *data;
+ int i;
- astate->dvalues[astate->nelems] = dvalue;
- astate->dnulls[astate->nelems] = disnull;
- astate->nelems++;
+ if (disnull)
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("cannot accumulate null arrays")));
+
+ arg = DatumGetArrayTypeP(dvalue);
+
+ ndims = ARR_NDIM(arg);
+ dims = ARR_DIMS(arg);
+ lbs = ARR_LBOUND(arg);
+ data = ARR_DATA_PTR(arg);
+ nitems = ArrayGetNItems(ndims, dims);
+
+ ndatabytes = ARR_SIZE(arg) - ARR_DATA_OFFSET(arg);
+
+ if (astate_array->data == NULL)
+ {
+ /* first allocation */
+ if (ndims == 0)
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("cannot accumulate empty arrays")));
+
+ if (ndims + 1 > MAXDIM)
+ ereport(ERROR,
+ (errcode(ERRCODE_PROGRAM_LIMIT_EXCEEDED),
+ errmsg("number of array dimensions (%d) exceeds the maximum allowed (%d)",
+ ndims + 1, MAXDIM)));
+
+ astate_array->ndims = ndims;
+ astate_array->dims = (int *) palloc(ndims * sizeof(int));
+ astate_array->lbs = (int *) palloc(ndims * sizeof(int));
+ memcpy(astate_array->dims, dims, ndims * sizeof(int));
+ memcpy(astate_array->lbs, lbs, ndims * sizeof(int));
+
+ astate_array->abytes = ndatabytes >= 512 ? 4 * ndatabytes : 1024;
+ astate_array->aitems = 4 * nitems;
+ astate_array->data = (char *) palloc(astate_array->abytes);
+
+ memcpy(astate_array->data, data, ndatabytes);
+ astate_array->nbytes = ndatabytes;
+ astate_array->nitems = nitems;
+ if (ARR_HASNULL(arg))
+ {
+ astate_array->hasnull = true;
+ astate_array->nullbitmap = (bits8 *) palloc((astate_array->aitems + 7) / 8);
+ array_bitmap_copy(astate_array->nullbitmap, 0,
+ ARR_NULLBITMAP(arg), 0,
+ nitems);
+ }
+ else
+ astate_array->hasnull = false;
+ }
+ else
+ {
+ if (astate_array->ndims != ndims)
+ ereport(ERROR,
+ (errcode(ERRCODE_ARRAY_SUBSCRIPT_ERROR),
+ errmsg("cannot aggregate incompatible arrays"),
+ errdetail("Arrays of %d and %d dimensions are not "
+ "compatible for concatenation.",
+ astate_array->ndims, ndims)));
+
+ for (i = 0; i < ndims; i++)
+ if (astate_array->dims[i] != dims[i] || astate_array->lbs[i] != lbs[i])
+ ereport(ERROR,
+ (errcode(ERRCODE_ARRAY_SUBSCRIPT_ERROR),
+ errmsg("cannot aggregate incompatible arrays"),
+ errdetail("Arrays with differing element dimensions are "
+ "not compatible for concatenation.")));
+
+ if (astate_array->nbytes + ndatabytes >= astate_array->abytes)
+ {
+ astate_array->nbytes *= 2;
+ astate_array->data = (char *)
+ repalloc(astate_array->data, astate_array->nbytes);
+ }
+ if (astate_array->nitems + nitems >= astate_array->aitems)
+ {
+ astate_array->aitems *= 2;
+ astate_array->nullbitmap = (bits8 *)
+ repalloc(astate_array->nullbitmap, (astate_array->aitems + 7) / 8);
+ }
+
+ memcpy(astate_array->data + astate_array->nbytes, data, ndatabytes);
+ astate_array->nbytes += ndatabytes;
+
+ if (ARR_HASNULL(arg) || astate_array->hasnull)
+ {
+ if (!astate_array->hasnull)
+ {
+ astate_array->hasnull = true;
+ astate_array->nullbitmap = (bits8 *) palloc((astate_array->aitems + 7) / 8);
+ array_bitmap_copy(astate_array->nullbitmap, 0,
+ NULL, 0,
+ astate_array->nitems);
+ }
+ array_bitmap_copy(astate_array->nullbitmap, astate_array->nitems,
+ ARR_NULLBITMAP(arg), 0,
+ nitems);
+ astate_array->nitems += nitems;
+ }
+ }
+ astate_array->narray += 1;
+ }
MemoryContextSwitchTo(oldcontext);
return astate;
}
/*
- * makeArrayResult - produce 1-D final result of accumArrayResult
+ * makeArrayResult - produce 1-D final result of scalar accumArrayResult
+ * - produce N+1-D final result of array accumArrayResult
*
* astate is working state (not NULL)
* rcontext is where to construct result
@@ -4661,17 +4828,22 @@ Datum
makeArrayResult(ArrayBuildState *astate,
MemoryContext rcontext)
{
- int dims[1];
- int lbs[1];
+ if (!astate->is_array_accum)
+ {
+ int dims[1];
+ int lbs[1];
- dims[0] = astate->nelems;
- lbs[0] = 1;
+ dims[0] = ((ArrayBuildStateScalar *) astate)->nelems;
+ lbs[0] = 1;
- return makeMdArrayResult(astate, 1, dims, lbs, rcontext, true);
+ return makeMdArrayResult(astate, 1, dims, lbs, rcontext, true);
+ }
+ else
+ return makeArrayResultArray((ArrayBuildStateArray *) astate, rcontext, true);
}
/*
- * makeMdArrayResult - produce multi-D final result of accumArrayResult
+ * makeMdArrayResult - produce multi-D final result of scalar accumArrayResult
*
* beware: no check that specified dimensions match the number of values
* accumulated.
@@ -4690,12 +4862,17 @@ makeMdArrayResult(ArrayBuildState *astate,
{
ArrayType *result;
MemoryContext oldcontext;
+ ArrayBuildStateScalar *astate_scalar;
+
+ Assert(!astate->is_array_accum);
+
+ astate_scalar = (ArrayBuildStateScalar *) astate;
/* Build the final array result in rcontext */
oldcontext = MemoryContextSwitchTo(rcontext);
- result = construct_md_array(astate->dvalues,
- astate->dnulls,
+ result = construct_md_array(astate_scalar->dvalues,
+ astate_scalar->dnulls,
ndims,
dims,
lbs,
@@ -4713,6 +4890,68 @@ makeMdArrayResult(ArrayBuildState *astate,
return PointerGetDatum(result);
}
+/*
+ * makeMdArrayResultArray - produce N+1-D final result of array accumArrayResult
+ *
+ * astate is working state (not NULL)
+ * rcontext is where to construct result
+ * release is true if okay to release working state
+ */
+Datum
+makeArrayResultArray(ArrayBuildStateArray *astate,
+ MemoryContext rcontext,
+ bool release)
+{
+ MemoryContext oldcontext;
+
+ ArrayType *result;
+
+ int dataoffset,
+ nbytes,
+ ndims;
+
+ oldcontext = MemoryContextSwitchTo(rcontext);
+
+ ndims = astate->ndims + 1;
+ nbytes = astate->nbytes;
+ /* compute required space */
+ if (astate->hasnull)
+ {
+ dataoffset = ARR_OVERHEAD_WITHNULLS(ndims, astate->nitems);
+ nbytes += dataoffset;
+ }
+ else
+ {
+ dataoffset = 0;
+ nbytes += ARR_OVERHEAD_NONULLS(ndims);
+ }
+
+ result = (ArrayType *) palloc0(nbytes);
+ SET_VARSIZE(result, nbytes);
+ result->ndim = astate->ndims + 1;
+ result->dataoffset = dataoffset;
+ result->elemtype = astate->astate.element_type;
+
+ ARR_DIMS(result)[0] = astate->narray;
+ ARR_LBOUND(result)[0] = 1;
+ memcpy(&(ARR_DIMS(result)[1]), astate->dims, (ndims - 1) * sizeof(int));
+ memcpy(&(ARR_LBOUND(result)[1]), astate->lbs, (ndims - 1) * sizeof(int));
+
+ memcpy(ARR_DATA_PTR(result), astate->data, astate->nbytes);
+ if (astate->hasnull)
+ array_bitmap_copy(ARR_NULLBITMAP(result), 0,
+ astate->nullbitmap, 0,
+ astate->nitems);
+
+ MemoryContextSwitchTo(oldcontext);
+
+ /* Clean up all the junk */
+ if (release)
+ MemoryContextDelete(astate->astate.mcontext);
+
+ PG_RETURN_DATUM(PointerGetDatum(result));
+}
+
Datum
array_larger(PG_FUNCTION_ARGS)
{
diff --git a/src/include/catalog/pg_aggregate.h b/src/include/catalog/pg_aggregate.h
index 3ba9e5e..2005199 100644
--- a/src/include/catalog/pg_aggregate.h
+++ b/src/include/catalog/pg_aggregate.h
@@ -275,6 +275,7 @@ DATA(insert ( 2901 n 0 xmlconcat2 - - - - f f 0 142 0 0 0 _null_
/* array */
DATA(insert ( 2335 n 0 array_agg_transfn array_agg_finalfn - - - t f 0 2281 0 0 0 _null_ _null_ ));
+DATA(insert ( 6005 n 0 array_agg_anyarray_transfn array_agg_anyarray_finalfn - - - t f 0 2281 0 0 0 _null_ _null_ ));
/* text */
DATA(insert ( 3538 n 0 string_agg_transfn string_agg_finalfn - - - f f 0 2281 0 0 0 _null_ _null_ ));
diff --git a/src/include/catalog/pg_proc.h b/src/include/catalog/pg_proc.h
index b6dc1b8..9273c1f 100644
--- a/src/include/catalog/pg_proc.h
+++ b/src/include/catalog/pg_proc.h
@@ -879,11 +879,17 @@ DATA(insert OID = 3167 ( array_remove PGNSP PGUID 12 1 0 0 0 f f f f f f i 2
DESCR("remove any occurrences of an element from an array");
DATA(insert OID = 3168 ( array_replace PGNSP PGUID 12 1 0 0 0 f f f f f f i 3 0 2277 "2277 2283 2283" _null_ _null_ _null_ _null_ array_replace _null_ _null_ _null_ ));
DESCR("replace any occurrences of an element in an array");
-DATA(insert OID = 2333 ( array_agg_transfn PGNSP PGUID 12 1 0 0 0 f f f f f f i 2 0 2281 "2281 2283" _null_ _null_ _null_ _null_ array_agg_transfn _null_ _null_ _null_ ));
+DATA(insert OID = 2333 ( array_agg_transfn PGNSP PGUID 12 1 0 0 0 f f f f f f i 2 0 2281 "2281 2776" _null_ _null_ _null_ _null_ array_agg_transfn _null_ _null_ _null_ ));
DESCR("aggregate transition function");
-DATA(insert OID = 2334 ( array_agg_finalfn PGNSP PGUID 12 1 0 0 0 f f f f f f i 2 0 2277 "2281 2283" _null_ _null_ _null_ _null_ array_agg_finalfn _null_ _null_ _null_ ));
+DATA(insert OID = 2334 ( array_agg_finalfn PGNSP PGUID 12 1 0 0 0 f f f f f f i 2 0 2277 "2281 2776" _null_ _null_ _null_ _null_ array_agg_finalfn _null_ _null_ _null_ ));
DESCR("aggregate final function");
-DATA(insert OID = 2335 ( array_agg PGNSP PGUID 12 1 0 0 0 t f f f f f i 1 0 2277 "2283" _null_ _null_ _null_ _null_ aggregate_dummy _null_ _null_ _null_ ));
+DATA(insert OID = 2335 ( array_agg PGNSP PGUID 12 1 0 0 0 t f f f f f i 1 0 2277 "2776" _null_ _null_ _null_ _null_ aggregate_dummy _null_ _null_ _null_ ));
+DESCR("concatenate aggregate input into an array");
+DATA(insert OID = 6003 ( array_agg_anyarray_transfn PGNSP PGUID 12 1 0 0 0 f f f f f f i 2 0 2281 "2281 2277" _null_ _null_ _null_ _null_ array_agg_anyarray_transfn _null_ _null_ _null_ ));
+DESCR("aggregate transition function");
+DATA(insert OID = 6004 ( array_agg_anyarray_finalfn PGNSP PGUID 12 1 0 0 0 f f f f f f i 2 0 2277 "2281 2277" _null_ _null_ _null_ _null_ array_agg_anyarray_finalfn _null_ _null_ _null_ ));
+DESCR("aggregate final function");
+DATA(insert OID = 6005 ( array_agg PGNSP PGUID 12 1 0 0 0 t f f f f f i 1 0 2277 "2277" _null_ _null_ _null_ _null_ aggregate_dummy _null_ _null_ _null_ ));
DESCR("concatenate aggregate input into an array");
DATA(insert OID = 3218 ( width_bucket PGNSP PGUID 12 1 0 0 0 f f f f t f i 2 0 23 "2283 2277" _null_ _null_ _null_ _null_ width_bucket_array _null_ _null_ _null_ ));
DESCR("bucket number of operand given a sorted array of bucket lower bounds");
diff --git a/src/include/utils/array.h b/src/include/utils/array.h
index e744314..c6fbc7f 100644
--- a/src/include/utils/array.h
+++ b/src/include/utils/array.h
@@ -76,14 +76,15 @@ typedef struct
/*
* working state for accumArrayResult() and friends
+ *
+ * is_array_accum: whether accumulating array values.
+ * (if true must be casted to ArrayBuildStateArray, else
+ * cast to ArrayBuildStateScalar)
*/
typedef struct ArrayBuildState
{
+ bool is_array_accum;
MemoryContext mcontext; /* where all the temp stuff is kept */
- Datum *dvalues; /* array of accumulated Datums */
- bool *dnulls; /* array of is-null flags for Datums */
- int alen; /* allocated length of above arrays */
- int nelems; /* number of valid entries in above arrays */
Oid element_type; /* data type of the Datums */
int16 typlen; /* needed info about datatype */
bool typbyval;
@@ -91,6 +92,43 @@ typedef struct ArrayBuildState
} ArrayBuildState;
/*
+ * array build state for array accumulation of scalar datums
+ */
+typedef struct ArrayBuildStateScalar
+{
+ ArrayBuildState astate;
+
+ Datum *dvalues; /* array of accumulated Datums */
+ bool *dnulls; /* array of is-null flags for Datums */
+ int alen; /* allocated length of above arrays */
+ int nelems; /* number of valid entries in above arrays */
+} ArrayBuildStateScalar;
+
+
+/*
+ * array build state for array accumulation of array datums
+ */
+typedef struct
+{
+ ArrayBuildState astate;
+
+ char *data; /* array of accumulated data */
+ bits8 *nullbitmap; /* bitmap of is-null flags for data */
+
+ int abytes; /* allocated length of above arrays */
+ int aitems; /* allocated length of above arrays */
+ int nbytes; /* number of used bytes in above arrays */
+ int nitems; /* number of elements in above arrays */
+ int narray; /* number of array accumulated */
+
+ int ndims; /* element dimensions */
+ int *dims;
+ int *lbs;
+
+ bool hasnull; /* any element has null */
+} ArrayBuildStateArray;
+
+/*
* structure to cache type metadata needed for array manipulation
*/
typedef struct ArrayMetaState
@@ -260,7 +298,8 @@ extern Datum makeArrayResult(ArrayBuildState *astate,
MemoryContext rcontext);
extern Datum makeMdArrayResult(ArrayBuildState *astate, int ndims,
int *dims, int *lbs, MemoryContext rcontext, bool release);
-
+extern Datum makeArrayResultArray(ArrayBuildStateArray *astate,
+ MemoryContext rcontext, bool release);
extern ArrayIterator array_create_iterator(ArrayType *arr, int slice_ndim);
extern bool array_iterate(ArrayIterator iterator, Datum *value, bool *isnull);
extern void array_free_iterator(ArrayIterator iterator);
@@ -293,6 +332,9 @@ extern ArrayType *create_singleton_array(FunctionCallInfo fcinfo,
extern Datum array_agg_transfn(PG_FUNCTION_ARGS);
extern Datum array_agg_finalfn(PG_FUNCTION_ARGS);
+extern Datum array_agg_anyarray_transfn(PG_FUNCTION_ARGS);
+extern Datum array_agg_anyarray_finalfn(PG_FUNCTION_ARGS);
+
/*
* prototypes for functions defined in array_typanalyze.c
*/
diff --git a/src/test/regress/expected/aggregates.out b/src/test/regress/expected/aggregates.out
index 58df854..eed1d5b 100644
--- a/src/test/regress/expected/aggregates.out
+++ b/src/test/regress/expected/aggregates.out
@@ -914,6 +914,33 @@ select array_agg(distinct a order by a desc nulls last)
{3,2,1,NULL}
(1 row)
+-- array_agg(anyarray)
+select array_agg(ar)
+ from (values ('{1,2}'::int[]), ('{3,4}'::int[])) v(ar);
+ array_agg
+---------------
+ {{1,2},{3,4}}
+(1 row)
+
+select array_agg(distinct ar order by ar desc)
+ from (select array[i / 2] from generate_series(1,10) a(i)) b(ar);
+ array_agg
+---------------------------
+ {{5},{4},{3},{2},{1},{0}}
+(1 row)
+
+select array_agg(ar)
+ from (select array_agg(array[i, i+1, i-1])
+ from generate_series(1,2) a(i)) b(ar);
+ array_agg
+---------------------
+ {{{1,2,0},{2,3,1}}}
+(1 row)
+
+select array_agg('{}'::int[]) from generate_series(1,2);
+ERROR: cannot accumulate empty arrays
+select array_agg(null::int[]) from generate_series(1,2);
+ERROR: cannot accumulate null arrays
-- multi-arg aggs, strict/nonstrict, distinct/order by
select aggfstr(a,b,c)
from (values (1,3,'foo'),(0,null,null),(2,2,'bar'),(3,1,'baz')) v(a,b,c);
diff --git a/src/test/regress/expected/arrays.out b/src/test/regress/expected/arrays.out
index 46eff67..e80ebec 100644
--- a/src/test/regress/expected/arrays.out
+++ b/src/test/regress/expected/arrays.out
@@ -1521,6 +1521,23 @@ select array_agg(unique1) from tenk1 where unique1 < -15;
(1 row)
+select array(select unique1 from tenk1 where unique1 < 15 order by unique1);
+ array
+--------------------------------------
+ {0,1,2,3,4,5,6,7,8,9,10,11,12,13,14}
+(1 row)
+
+select array(select array[i,i/2] from generate_series(1,5) a(i));
+ array
+---------------------------------
+ {{1,0},{2,1},{3,1},{4,2},{5,2}}
+(1 row)
+
+-- cannot accumulate null arrays and empty arrays
+select array(select null::int[]);
+ERROR: cannot accumulate null arrays
+select array(select '{}'::int[]);
+ERROR: cannot accumulate empty arrays
select unnest(array[1,2,3]);
unnest
--------
diff --git a/src/test/regress/sql/aggregates.sql b/src/test/regress/sql/aggregates.sql
index 8096a6f..2296337 100644
--- a/src/test/regress/sql/aggregates.sql
+++ b/src/test/regress/sql/aggregates.sql
@@ -322,6 +322,18 @@ select array_agg(distinct a order by a desc)
select array_agg(distinct a order by a desc nulls last)
from (values (1),(2),(1),(3),(null),(2)) v(a);
+-- array_agg(anyarray)
+select array_agg(ar)
+ from (values ('{1,2}'::int[]), ('{3,4}'::int[])) v(ar);
+select array_agg(distinct ar order by ar desc)
+ from (select array[i / 2] from generate_series(1,10) a(i)) b(ar);
+select array_agg(ar)
+ from (select array_agg(array[i, i+1, i-1])
+ from generate_series(1,2) a(i)) b(ar);
+
+select array_agg('{}'::int[]) from generate_series(1,2);
+select array_agg(null::int[]) from generate_series(1,2);
+
-- multi-arg aggs, strict/nonstrict, distinct/order by
select aggfstr(a,b,c)
diff --git a/src/test/regress/sql/arrays.sql b/src/test/regress/sql/arrays.sql
index fa8a20a..cb00f5f 100644
--- a/src/test/regress/sql/arrays.sql
+++ b/src/test/regress/sql/arrays.sql
@@ -432,6 +432,12 @@ select array_agg(ten) from (select ten from tenk1 where unique1 < 15 order by un
select array_agg(nullif(ten, 4)) from (select ten from tenk1 where unique1 < 15 order by unique1) ss;
select array_agg(unique1) from tenk1 where unique1 < -15;
+select array(select unique1 from tenk1 where unique1 < 15 order by unique1);
+select array(select array[i,i/2] from generate_series(1,5) a(i));
+-- cannot accumulate null arrays and empty arrays
+select array(select null::int[]);
+select array(select '{}'::int[]);
+
select unnest(array[1,2,3]);
select * from unnest(array[1,2,3]);
select unnest(array[1,2,3,4.5]::float8[]);
Re: Function array_agg(array)
On Fri, Oct 24, 2014 at 11:43 AM, Ali Akbar <the.apaan@gmail.com> wrote:
2014-10-24 16:26 GMT+07:00 Pavel Stehule <pavel.stehule@gmail.com>:
Hi
some in last patch is wrong, I cannot to compile it:
arrayfuncs.c: In function 'accumArrayResult':
arrayfuncs.c:4603:9: error: 'ArrayBuildState' has no member named 'alen'
astate->alen = 64; /* arbitrary starting array size */
^
arrayfuncs.c:4604:9: error: 'ArrayBuildState' has no member named
'dvalues'
astate->dvalues = (Datum *) palloc(astate->alen * sizeof(Datum));
^
arrayfuncs.c:4604:44: error: 'ArrayBuildState' has no member named 'alen'
astate->dvalues = (Datum *) palloc(astate->alen * sizeof(Datum));
^
arrayfuncs.c:4605:9: error: 'ArrayBuildState' has no member named 'dnulls'
astate->dnulls = (bool *) palloc(astate->alen * sizeof(bool));
^
arrayfuncs.c:4605:42: error: 'ArrayBuildState' has no member named 'alen'
astate->dnulls = (bool *) palloc(astate->alen * sizeof(bool));
^
arrayfuncs.c:4606:9: error: 'ArrayBuildState' has no member named 'nelems'
astate->nelems = 0;
^
arrayfuncs.c:4618:13: error: 'ArrayBuildState' has no member named
'nelems'
if (astate->nelems >= astate->alen)
^
arrayfuncs.c:4618:31: error: 'ArrayBuildState' has no member named 'alen'
if (astate->nelems >= astate->alen)
^
arrayfuncs.c:4620:10: error: 'ArrayBuildState' has no member named 'alen'
astate->alen *= 2;Sorry, correct patch attached.
This patch is in patience format (git --patience ..). In previous patches,
i use context format (git --patience ... | filterdiff --format=context),
but it turns out that some modification is lost.
That's not surprising, sometimes filterdiff misses the shot.
--
Michael
Re: Function array_agg(array)
2014-10-24 11:43 GMT+02:00 Ali Akbar <the.apaan@gmail.com>:
2014-10-24 16:26 GMT+07:00 Pavel Stehule <pavel.stehule@gmail.com>:
Hi
some in last patch is wrong, I cannot to compile it:
arrayfuncs.c: In function ‘accumArrayResult’:
arrayfuncs.c:4603:9: error: ‘ArrayBuildState’ has no member named ‘alen’
astate->alen = 64; /* arbitrary starting array size */
^
arrayfuncs.c:4604:9: error: ‘ArrayBuildState’ has no member named
‘dvalues’
astate->dvalues = (Datum *) palloc(astate->alen * sizeof(Datum));
^
arrayfuncs.c:4604:44: error: ‘ArrayBuildState’ has no member named ‘alen’
astate->dvalues = (Datum *) palloc(astate->alen * sizeof(Datum));
^
arrayfuncs.c:4605:9: error: ‘ArrayBuildState’ has no member named ‘dnulls’
astate->dnulls = (bool *) palloc(astate->alen * sizeof(bool));
^
arrayfuncs.c:4605:42: error: ‘ArrayBuildState’ has no member named ‘alen’
astate->dnulls = (bool *) palloc(astate->alen * sizeof(bool));
^
arrayfuncs.c:4606:9: error: ‘ArrayBuildState’ has no member named ‘nelems’
astate->nelems = 0;
^
arrayfuncs.c:4618:13: error: ‘ArrayBuildState’ has no member named
‘nelems’
if (astate->nelems >= astate->alen)
^
arrayfuncs.c:4618:31: error: ‘ArrayBuildState’ has no member named ‘alen’
if (astate->nelems >= astate->alen)
^
arrayfuncs.c:4620:10: error: ‘ArrayBuildState’ has no member named ‘alen’
astate->alen *= 2;Sorry, correct patch attached.
This patch is in patience format (git --patience ..). In previous patches,
i use context format (git --patience ... | filterdiff --format=context),
but it turns out that some modification is lost.
last version is compileable, but some is still broken
postgres=# select array_agg(array[i, i+1, i-1])
from generate_series(1,2) a(i);
ERROR: could not find array type for data type integer[]
but array(subselect) works
postgres=# select array(select a from xx);
array
-------------------
{{1,2,3},{1,2,3}}
(1 row)
Regards
Pavel
Show quoted text
--
Ali Akbar
Re: Function array_agg(array)
Hi
I did some performance tests and it is interesting:
it is about 15% faster than original implementation.
Regards
Pavel
2014-10-24 13:58 GMT+02:00 Pavel Stehule <pavel.stehule@gmail.com>:
Show quoted text
2014-10-24 11:43 GMT+02:00 Ali Akbar <the.apaan@gmail.com>:
2014-10-24 16:26 GMT+07:00 Pavel Stehule <pavel.stehule@gmail.com>:
Hi
some in last patch is wrong, I cannot to compile it:
arrayfuncs.c: In function ‘accumArrayResult’:
arrayfuncs.c:4603:9: error: ‘ArrayBuildState’ has no member named ‘alen’
astate->alen = 64; /* arbitrary starting array size */
^
arrayfuncs.c:4604:9: error: ‘ArrayBuildState’ has no member named
‘dvalues’
astate->dvalues = (Datum *) palloc(astate->alen * sizeof(Datum));
^
arrayfuncs.c:4604:44: error: ‘ArrayBuildState’ has no member named ‘alen’
astate->dvalues = (Datum *) palloc(astate->alen * sizeof(Datum));
^
arrayfuncs.c:4605:9: error: ‘ArrayBuildState’ has no member named
‘dnulls’
astate->dnulls = (bool *) palloc(astate->alen * sizeof(bool));
^
arrayfuncs.c:4605:42: error: ‘ArrayBuildState’ has no member named ‘alen’
astate->dnulls = (bool *) palloc(astate->alen * sizeof(bool));
^
arrayfuncs.c:4606:9: error: ‘ArrayBuildState’ has no member named
‘nelems’
astate->nelems = 0;
^
arrayfuncs.c:4618:13: error: ‘ArrayBuildState’ has no member named
‘nelems’
if (astate->nelems >= astate->alen)
^
arrayfuncs.c:4618:31: error: ‘ArrayBuildState’ has no member named ‘alen’
if (astate->nelems >= astate->alen)
^
arrayfuncs.c:4620:10: error: ‘ArrayBuildState’ has no member named ‘alen’
astate->alen *= 2;Sorry, correct patch attached.
This patch is in patience format (git --patience ..). In previous
patches, i use context format (git --patience ... | filterdiff
--format=context), but it turns out that some modification is lost.last version is compileable, but some is still broken
postgres=# select array_agg(array[i, i+1, i-1])
from generate_series(1,2) a(i);
ERROR: could not find array type for data type integer[]but array(subselect) works
postgres=# select array(select a from xx);
array
-------------------
{{1,2,3},{1,2,3}}
(1 row)Regards
Pavel
--
Ali Akbar
Re: Function array_agg(array)
Michael Paquier wrote:
That's not surprising, sometimes filterdiff misses the shot.
Really? Wow, that's bad news. I've been using it to submit patches
from time to time ...
--
�lvaro Herrera http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Training & Services
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
Re: Function array_agg(array)
2014-10-24 13:58 GMT+02:00 Pavel Stehule <pavel.stehule@gmail.com>:
2014-10-24 11:43 GMT+02:00 Ali Akbar <the.apaan@gmail.com>:
2014-10-24 16:26 GMT+07:00 Pavel Stehule <pavel.stehule@gmail.com>:
Hi
some in last patch is wrong, I cannot to compile it:
arrayfuncs.c: In function ‘accumArrayResult’:
arrayfuncs.c:4603:9: error: ‘ArrayBuildState’ has no member named ‘alen’
astate->alen = 64; /* arbitrary starting array size */
^
arrayfuncs.c:4604:9: error: ‘ArrayBuildState’ has no member named
‘dvalues’
astate->dvalues = (Datum *) palloc(astate->alen * sizeof(Datum));
^
arrayfuncs.c:4604:44: error: ‘ArrayBuildState’ has no member named ‘alen’
astate->dvalues = (Datum *) palloc(astate->alen * sizeof(Datum));
^
arrayfuncs.c:4605:9: error: ‘ArrayBuildState’ has no member named
‘dnulls’
astate->dnulls = (bool *) palloc(astate->alen * sizeof(bool));
^
arrayfuncs.c:4605:42: error: ‘ArrayBuildState’ has no member named ‘alen’
astate->dnulls = (bool *) palloc(astate->alen * sizeof(bool));
^
arrayfuncs.c:4606:9: error: ‘ArrayBuildState’ has no member named
‘nelems’
astate->nelems = 0;
^
arrayfuncs.c:4618:13: error: ‘ArrayBuildState’ has no member named
‘nelems’
if (astate->nelems >= astate->alen)
^
arrayfuncs.c:4618:31: error: ‘ArrayBuildState’ has no member named ‘alen’
if (astate->nelems >= astate->alen)
^
arrayfuncs.c:4620:10: error: ‘ArrayBuildState’ has no member named ‘alen’
astate->alen *= 2;Sorry, correct patch attached.
This patch is in patience format (git --patience ..). In previous
patches, i use context format (git --patience ... | filterdiff
--format=context), but it turns out that some modification is lost.last version is compileable, but some is still broken
postgres=# select array_agg(array[i, i+1, i-1])
from generate_series(1,2) a(i);
ERROR: could not find array type for data type integer[]
I am sorry, it works - I had a problem with broken database
I fixed small issue in regress tests and I enhanced tests for varlena types
and null values.
Regards
Pavel
Show quoted text
but array(subselect) works
postgres=# select array(select a from xx);
array
-------------------
{{1,2,3},{1,2,3}}
(1 row)Regards
Pavel
--
Ali Akbar
Attachments:
array_agg_anyarray-7.patchtext/x-patch; charset=US-ASCII; name=array_agg_anyarray-7.patchDownload
diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml
new file mode 100644
index 7e5bcd9..f59738a
*** a/doc/src/sgml/func.sgml
--- b/doc/src/sgml/func.sgml
*************** NULL baz</literallayout>(3 rows)</entry>
*** 12046,12051 ****
--- 12046,12067 ----
<row>
<entry>
<indexterm>
+ <primary>array_agg</primary>
+ </indexterm>
+ <function>array_agg(<replaceable class="parameter">anyarray</replaceable>)</function>
+ </entry>
+ <entry>
+ any
+ </entry>
+ <entry>
+ the same array type as input type
+ </entry>
+ <entry>input arrays, aggregated into higher-order multidimesional array. Rejects NULL and empty array as input.</entry>
+ </row>
+
+ <row>
+ <entry>
+ <indexterm>
<primary>average</primary>
</indexterm>
<indexterm>
diff --git a/doc/src/sgml/syntax.sgml b/doc/src/sgml/syntax.sgml
new file mode 100644
index 2f0680f..8c182a4
*** a/doc/src/sgml/syntax.sgml
--- b/doc/src/sgml/syntax.sgml
*************** SELECT ARRAY(SELECT oid FROM pg_proc WHE
*** 2238,2243 ****
--- 2238,2248 ----
array
-----------------------------------------------------------------------
{2011,1954,1948,1952,1951,1244,1950,2005,1949,1953,2006,31,2412,2413}
+
+ SELECT ARRAY(SELECT array(select i) FROM generate_series(1,5) a(i));
+ array
+ -----------------------
+ {{1},{2},{3},{4},{5}}
(1 row)
</programlisting>
The subquery must return a single column. The resulting
diff --git a/src/backend/nodes/nodeFuncs.c b/src/backend/nodes/nodeFuncs.c
new file mode 100644
index 41e973b..0261fcb
*** a/src/backend/nodes/nodeFuncs.c
--- b/src/backend/nodes/nodeFuncs.c
*************** exprType(const Node *expr)
*** 108,119 ****
type = exprType((Node *) tent->expr);
if (sublink->subLinkType == ARRAY_SUBLINK)
{
! type = get_array_type(type);
! if (!OidIsValid(type))
! ereport(ERROR,
! (errcode(ERRCODE_UNDEFINED_OBJECT),
! errmsg("could not find array type for data type %s",
! format_type_be(exprType((Node *) tent->expr)))));
}
}
else if (sublink->subLinkType == MULTIEXPR_SUBLINK)
--- 108,123 ----
type = exprType((Node *) tent->expr);
if (sublink->subLinkType == ARRAY_SUBLINK)
{
! if (!OidIsValid(get_element_type(type)))
! {
! /* not array, so check for its array type */
! type = get_array_type(type);
! if (!OidIsValid(type))
! ereport(ERROR,
! (errcode(ERRCODE_UNDEFINED_OBJECT),
! errmsg("could not find array type for data type %s",
! format_type_be(exprType((Node *) tent->expr)))));
! }
}
}
else if (sublink->subLinkType == MULTIEXPR_SUBLINK)
*************** exprType(const Node *expr)
*** 139,150 ****
type = subplan->firstColType;
if (subplan->subLinkType == ARRAY_SUBLINK)
{
! type = get_array_type(type);
! if (!OidIsValid(type))
! ereport(ERROR,
! (errcode(ERRCODE_UNDEFINED_OBJECT),
! errmsg("could not find array type for data type %s",
! format_type_be(subplan->firstColType))));
}
}
else if (subplan->subLinkType == MULTIEXPR_SUBLINK)
--- 143,158 ----
type = subplan->firstColType;
if (subplan->subLinkType == ARRAY_SUBLINK)
{
! if (!OidIsValid(get_element_type(type)))
! {
! /* not array, so check for its array type */
! type = get_array_type(type);
! if (!OidIsValid(type))
! ereport(ERROR,
! (errcode(ERRCODE_UNDEFINED_OBJECT),
! errmsg("could not find array type for data type %s",
! format_type_be(subplan->firstColType))));
! }
}
}
else if (subplan->subLinkType == MULTIEXPR_SUBLINK)
diff --git a/src/backend/optimizer/plan/subselect.c b/src/backend/optimizer/plan/subselect.c
new file mode 100644
index 3e7dc85..8fc8b49
*** a/src/backend/optimizer/plan/subselect.c
--- b/src/backend/optimizer/plan/subselect.c
*************** build_subplan(PlannerInfo *root, Plan *p
*** 668,677 ****
Assert(!te->resjunk);
Assert(testexpr == NULL);
! arraytype = get_array_type(exprType((Node *) te->expr));
! if (!OidIsValid(arraytype))
! elog(ERROR, "could not find array type for datatype %s",
! format_type_be(exprType((Node *) te->expr)));
prm = generate_new_param(root,
arraytype,
exprTypmod((Node *) te->expr),
--- 668,683 ----
Assert(!te->resjunk);
Assert(testexpr == NULL);
!
! arraytype = exprType((Node *) te->expr);
! if (!OidIsValid(get_element_type(arraytype)))
! {
! /* not array, so get the array type */
! arraytype = get_array_type(exprType((Node *) te->expr));
! if (!OidIsValid(arraytype))
! elog(ERROR, "could not find array type for datatype %s",
! format_type_be(exprType((Node *) te->expr)));
! }
prm = generate_new_param(root,
arraytype,
exprTypmod((Node *) te->expr),
diff --git a/src/backend/utils/adt/array_userfuncs.c b/src/backend/utils/adt/array_userfuncs.c
new file mode 100644
index 831466d..9fdb25b
*** a/src/backend/utils/adt/array_userfuncs.c
--- b/src/backend/utils/adt/array_userfuncs.c
***************
*** 16,22 ****
#include "utils/builtins.h"
#include "utils/lsyscache.h"
-
/*-----------------------------------------------------------------------------
* array_push :
* push an element onto either end of a one-dimensional array
--- 16,21 ----
*************** array_agg_finalfn(PG_FUNCTION_ARGS)
*** 513,520 ****
{
Datum result;
ArrayBuildState *state;
- int dims[1];
- int lbs[1];
/*
* Test for null before Asserting we are in right context. This is to
--- 512,517 ----
*************** array_agg_finalfn(PG_FUNCTION_ARGS)
*** 529,546 ****
state = (ArrayBuildState *) PG_GETARG_POINTER(0);
- dims[0] = state->nelems;
- lbs[0] = 1;
-
/*
* Make the result. We cannot release the ArrayBuildState because
* sometimes aggregate final functions are re-executed. Rather, it is
* nodeAgg.c's responsibility to reset the aggcontext when it's safe to do
* so.
*/
! result = makeMdArrayResult(state, 1, dims, lbs,
! CurrentMemoryContext,
! false);
PG_RETURN_DATUM(result);
}
--- 526,568 ----
state = (ArrayBuildState *) PG_GETARG_POINTER(0);
/*
* Make the result. We cannot release the ArrayBuildState because
* sometimes aggregate final functions are re-executed. Rather, it is
* nodeAgg.c's responsibility to reset the aggcontext when it's safe to do
* so.
*/
! if (!state->is_array_accum)
! {
! int dims[1];
! int lbs[1];
!
! dims[0] = ((ArrayBuildStateScalar *) state)->nelems;
! lbs[0] = 1;
!
! result = makeMdArrayResult(state, 1, dims, lbs,
! CurrentMemoryContext,
! false);
! }
! else
! result = makeArrayResultArray((ArrayBuildStateArray *) state,
! CurrentMemoryContext,
! false);
PG_RETURN_DATUM(result);
}
+
+ /*
+ * ARRAY_AGG(anyarray) aggregate function
+ */
+ Datum
+ array_agg_anyarray_transfn(PG_FUNCTION_ARGS)
+ {
+ return array_agg_transfn(fcinfo);
+ }
+
+ Datum
+ array_agg_anyarray_finalfn(PG_FUNCTION_ARGS)
+ {
+ return array_agg_finalfn(fcinfo);
+ }
diff --git a/src/backend/utils/adt/arrayfuncs.c b/src/backend/utils/adt/arrayfuncs.c
new file mode 100644
index 6c8b41d..582912e
*** a/src/backend/utils/adt/arrayfuncs.c
--- b/src/backend/utils/adt/arrayfuncs.c
*************** static int width_bucket_array_variable(D
*** 145,151 ****
Oid collation,
TypeCacheEntry *typentry);
-
/*
* array_in :
* converts an array from the external format in "string" to
--- 145,150 ----
*************** accumArrayResult(ArrayBuildState *astate
*** 4588,4596 ****
--- 4587,4599 ----
MemoryContext arr_context,
oldcontext;
+ ArrayBuildStateScalar *astate_scalar = NULL; /* for scalar datum accumulation */
+ ArrayBuildStateArray *astate_array = NULL; /* for array datum accumulation */
+
if (astate == NULL)
{
/* First time through --- initialize */
+ Oid subelement_type = get_element_type(element_type);
/* Make a temporary context to hold all the junk */
arr_context = AllocSetContextCreate(rcontext,
*************** accumArrayResult(ArrayBuildState *astate
*** 4599,4658 ****
ALLOCSET_DEFAULT_INITSIZE,
ALLOCSET_DEFAULT_MAXSIZE);
oldcontext = MemoryContextSwitchTo(arr_context);
! astate = (ArrayBuildState *) palloc(sizeof(ArrayBuildState));
! astate->mcontext = arr_context;
! astate->alen = 64; /* arbitrary starting array size */
! astate->dvalues = (Datum *) palloc(astate->alen * sizeof(Datum));
! astate->dnulls = (bool *) palloc(astate->alen * sizeof(bool));
! astate->nelems = 0;
! astate->element_type = element_type;
! get_typlenbyvalalign(element_type,
! &astate->typlen,
! &astate->typbyval,
! &astate->typalign);
}
else
{
oldcontext = MemoryContextSwitchTo(astate->mcontext);
! Assert(astate->element_type == element_type);
! /* enlarge dvalues[]/dnulls[] if needed */
! if (astate->nelems >= astate->alen)
{
! astate->alen *= 2;
! astate->dvalues = (Datum *)
! repalloc(astate->dvalues, astate->alen * sizeof(Datum));
! astate->dnulls = (bool *)
! repalloc(astate->dnulls, astate->alen * sizeof(bool));
}
}
! /*
! * Ensure pass-by-ref stuff is copied into mcontext; and detoast it too if
! * it's varlena. (You might think that detoasting is not needed here
! * because construct_md_array can detoast the array elements later.
! * However, we must not let construct_md_array modify the ArrayBuildState
! * because that would mean array_agg_finalfn damages its input, which is
! * verboten. Also, this way frequently saves one copying step.)
! */
! if (!disnull && !astate->typbyval)
{
! if (astate->typlen == -1)
! dvalue = PointerGetDatum(PG_DETOAST_DATUM_COPY(dvalue));
! else
! dvalue = datumCopy(dvalue, astate->typbyval, astate->typlen);
}
! astate->dvalues[astate->nelems] = dvalue;
! astate->dnulls[astate->nelems] = disnull;
! astate->nelems++;
MemoryContextSwitchTo(oldcontext);
return astate;
}
/*
! * makeArrayResult - produce 1-D final result of accumArrayResult
*
* astate is working state (not NULL)
* rcontext is where to construct result
--- 4602,4825 ----
ALLOCSET_DEFAULT_INITSIZE,
ALLOCSET_DEFAULT_MAXSIZE);
oldcontext = MemoryContextSwitchTo(arr_context);
!
! if (subelement_type == InvalidOid)
! {
! /* scalar accumulate */
! astate_scalar = (ArrayBuildStateScalar *) palloc(sizeof(ArrayBuildStateScalar));
! astate = (ArrayBuildState *) astate_scalar;
! astate->is_array_accum = false;
! astate->mcontext = arr_context;
!
! astate_scalar->alen = 64; /* arbitrary starting array size */
! astate_scalar->dvalues = (Datum *) palloc(astate_scalar->alen * sizeof(Datum));
! astate_scalar->dnulls = (bool *) palloc(astate_scalar->alen * sizeof(bool));
! astate_scalar->nelems = 0;
!
! astate->element_type = element_type;
! get_typlenbyvalalign(element_type,
! &astate->typlen,
! &astate->typbyval,
! &astate->typalign);
! }
! else
! {
! /* array accumulate */
! astate_array = (ArrayBuildStateArray *) palloc(sizeof(ArrayBuildStateArray));
! astate = (ArrayBuildState *) astate_array;
! astate->is_array_accum = true;
! astate->mcontext = arr_context;
!
! astate_array->abytes = 0;
! astate_array->aitems = 0;
! astate_array->data = NULL;
! astate_array->nullbitmap = NULL;
! astate_array->nitems = 0;
! astate_array->narray = 0;
!
! astate->element_type = subelement_type;
! get_typlenbyvalalign(subelement_type,
! &astate->typlen,
! &astate->typbyval,
! &astate->typalign);
! }
}
else
{
oldcontext = MemoryContextSwitchTo(astate->mcontext);
! if (!astate->is_array_accum)
{
! Assert(astate->element_type == element_type);
! astate_scalar = (ArrayBuildStateScalar *) astate;
!
! /* enlarge dvalues[]/dnulls[] if needed */
! if (astate_scalar->nelems >= astate_scalar->alen)
! {
! astate_scalar->alen *= 2;
! astate_scalar->dvalues = (Datum *)
! repalloc(astate_scalar->dvalues, astate_scalar->alen * sizeof(Datum));
! astate_scalar->dnulls = (bool *)
! repalloc(astate_scalar->dnulls, astate_scalar->alen * sizeof(bool));
! }
! }
! else
! {
! Assert(astate->element_type == get_element_type(element_type));
! astate_array = (ArrayBuildStateArray *) astate;
! /*
! * array accumulate alloc & realloc placed below because
! * we need to know its length first
! */
}
}
! if (!astate->is_array_accum)
{
! /*
! * Ensure pass-by-ref stuff is copied into mcontext; and detoast it too if
! * it's varlena. (You might think that detoasting is not needed here
! * because construct_md_array can detoast the array elements later.
! * However, we must not let construct_md_array modify the ArrayBuildState
! * because that would mean array_agg_finalfn damages its input, which is
! * verboten. Also, this way frequently saves one copying step.)
! */
! if (!disnull && !astate->typbyval)
! {
! if (astate->typlen == -1)
! dvalue = PointerGetDatum(PG_DETOAST_DATUM_COPY(dvalue));
! else
! dvalue = datumCopy(dvalue, astate->typbyval, astate->typlen);
! }
!
! astate_scalar->dvalues[astate_scalar->nelems] = dvalue;
! astate_scalar->dnulls[astate_scalar->nelems] = disnull;
! astate_scalar->nelems++;
}
+ else
+ {
+ ArrayType *arg;
+ int *dims,
+ *lbs,
+ ndims,
+ nitems,
+ ndatabytes;
+ char *data;
+ int i;
! if (disnull)
! ereport(ERROR,
! (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
! errmsg("cannot accumulate null arrays")));
!
! arg = DatumGetArrayTypeP(dvalue);
!
! ndims = ARR_NDIM(arg);
! dims = ARR_DIMS(arg);
! lbs = ARR_LBOUND(arg);
! data = ARR_DATA_PTR(arg);
! nitems = ArrayGetNItems(ndims, dims);
+ ndatabytes = ARR_SIZE(arg) - ARR_DATA_OFFSET(arg);
+
+ if (astate_array->data == NULL)
+ {
+ /* first allocation */
+ if (ndims == 0)
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("cannot accumulate empty arrays")));
+
+ if (ndims + 1 > MAXDIM)
+ ereport(ERROR,
+ (errcode(ERRCODE_PROGRAM_LIMIT_EXCEEDED),
+ errmsg("number of array dimensions (%d) exceeds the maximum allowed (%d)",
+ ndims + 1, MAXDIM)));
+
+ astate_array->ndims = ndims;
+ astate_array->dims = (int *) palloc(ndims * sizeof(int));
+ astate_array->lbs = (int *) palloc(ndims * sizeof(int));
+ memcpy(astate_array->dims, dims, ndims * sizeof(int));
+ memcpy(astate_array->lbs, lbs, ndims * sizeof(int));
+
+ astate_array->abytes = ndatabytes >= 512 ? 4 * ndatabytes : 1024;
+ astate_array->aitems = 4 * nitems;
+ astate_array->data = (char *) palloc(astate_array->abytes);
+
+ memcpy(astate_array->data, data, ndatabytes);
+ astate_array->nbytes = ndatabytes;
+ astate_array->nitems = nitems;
+
+ if (ARR_HASNULL(arg))
+ {
+ astate_array->hasnull = true;
+ astate_array->nullbitmap = (bits8 *) palloc((astate_array->aitems + 7) / 8);
+ array_bitmap_copy(astate_array->nullbitmap, 0,
+ ARR_NULLBITMAP(arg), 0,
+ nitems);
+ }
+ else
+ astate_array->hasnull = false;
+ }
+ else
+ {
+ if (astate_array->ndims != ndims)
+ ereport(ERROR,
+ (errcode(ERRCODE_ARRAY_SUBSCRIPT_ERROR),
+ errmsg("cannot aggregate incompatible arrays"),
+ errdetail("Arrays of %d and %d dimensions are not "
+ "compatible for concatenation.",
+ astate_array->ndims, ndims)));
+
+ for (i = 0; i < ndims; i++)
+ if (astate_array->dims[i] != dims[i] || astate_array->lbs[i] != lbs[i])
+ ereport(ERROR,
+ (errcode(ERRCODE_ARRAY_SUBSCRIPT_ERROR),
+ errmsg("cannot aggregate incompatible arrays"),
+ errdetail("Arrays with differing element dimensions are "
+ "not compatible for concatenation.")));
+
+ if (astate_array->nbytes + ndatabytes >= astate_array->abytes)
+ {
+ astate_array->nbytes *= 2;
+ astate_array->data = (char *)
+ repalloc(astate_array->data, astate_array->nbytes);
+ }
+ if (astate_array->nitems + nitems >= astate_array->aitems)
+ {
+ astate_array->aitems *= 2;
+ astate_array->nullbitmap = (bits8 *)
+ repalloc(astate_array->nullbitmap, (astate_array->aitems + 7) / 8);
+ }
+
+ memcpy(astate_array->data + astate_array->nbytes, data, ndatabytes);
+ astate_array->nbytes += ndatabytes;
+
+ if (ARR_HASNULL(arg) || astate_array->hasnull)
+ {
+ if (!astate_array->hasnull)
+ {
+ astate_array->hasnull = true;
+ astate_array->nullbitmap = (bits8 *) palloc((astate_array->aitems + 7) / 8);
+ array_bitmap_copy(astate_array->nullbitmap, 0,
+ NULL, 0,
+ astate_array->nitems);
+ }
+ array_bitmap_copy(astate_array->nullbitmap, astate_array->nitems,
+ ARR_NULLBITMAP(arg), 0,
+ nitems);
+ astate_array->nitems += nitems;
+ }
+ }
+ astate_array->narray += 1;
+ }
MemoryContextSwitchTo(oldcontext);
return astate;
}
/*
! * makeArrayResult - produce 1-D final result of scalar accumArrayResult
! * - produce N+1-D final result of array accumArrayResult
*
* astate is working state (not NULL)
* rcontext is where to construct result
*************** Datum
*** 4661,4677 ****
makeArrayResult(ArrayBuildState *astate,
MemoryContext rcontext)
{
! int dims[1];
! int lbs[1];
! dims[0] = astate->nelems;
! lbs[0] = 1;
! return makeMdArrayResult(astate, 1, dims, lbs, rcontext, true);
}
/*
! * makeMdArrayResult - produce multi-D final result of accumArrayResult
*
* beware: no check that specified dimensions match the number of values
* accumulated.
--- 4828,4849 ----
makeArrayResult(ArrayBuildState *astate,
MemoryContext rcontext)
{
! if (!astate->is_array_accum)
! {
! int dims[1];
! int lbs[1];
! dims[0] = ((ArrayBuildStateScalar *) astate)->nelems;
! lbs[0] = 1;
! return makeMdArrayResult(astate, 1, dims, lbs, rcontext, true);
! }
! else
! return makeArrayResultArray((ArrayBuildStateArray *) astate, rcontext, true);
}
/*
! * makeMdArrayResult - produce multi-D final result of scalar accumArrayResult
*
* beware: no check that specified dimensions match the number of values
* accumulated.
*************** makeMdArrayResult(ArrayBuildState *astat
*** 4690,4701 ****
{
ArrayType *result;
MemoryContext oldcontext;
/* Build the final array result in rcontext */
oldcontext = MemoryContextSwitchTo(rcontext);
! result = construct_md_array(astate->dvalues,
! astate->dnulls,
ndims,
dims,
lbs,
--- 4862,4878 ----
{
ArrayType *result;
MemoryContext oldcontext;
+ ArrayBuildStateScalar *astate_scalar;
+
+ Assert(!astate->is_array_accum);
+
+ astate_scalar = (ArrayBuildStateScalar *) astate;
/* Build the final array result in rcontext */
oldcontext = MemoryContextSwitchTo(rcontext);
! result = construct_md_array(astate_scalar->dvalues,
! astate_scalar->dnulls,
ndims,
dims,
lbs,
*************** makeMdArrayResult(ArrayBuildState *astat
*** 4713,4718 ****
--- 4890,4957 ----
return PointerGetDatum(result);
}
+ /*
+ * makeMdArrayResultArray - produce N+1-D final result of array accumArrayResult
+ *
+ * astate is working state (not NULL)
+ * rcontext is where to construct result
+ * release is true if okay to release working state
+ */
+ Datum
+ makeArrayResultArray(ArrayBuildStateArray *astate,
+ MemoryContext rcontext,
+ bool release)
+ {
+ MemoryContext oldcontext;
+
+ ArrayType *result;
+
+ int dataoffset,
+ nbytes,
+ ndims;
+
+ oldcontext = MemoryContextSwitchTo(rcontext);
+
+ ndims = astate->ndims + 1;
+ nbytes = astate->nbytes;
+ /* compute required space */
+ if (astate->hasnull)
+ {
+ dataoffset = ARR_OVERHEAD_WITHNULLS(ndims, astate->nitems);
+ nbytes += dataoffset;
+ }
+ else
+ {
+ dataoffset = 0;
+ nbytes += ARR_OVERHEAD_NONULLS(ndims);
+ }
+
+ result = (ArrayType *) palloc0(nbytes);
+ SET_VARSIZE(result, nbytes);
+ result->ndim = astate->ndims + 1;
+ result->dataoffset = dataoffset;
+ result->elemtype = astate->astate.element_type;
+
+ ARR_DIMS(result)[0] = astate->narray;
+ ARR_LBOUND(result)[0] = 1;
+ memcpy(&(ARR_DIMS(result)[1]), astate->dims, (ndims - 1) * sizeof(int));
+ memcpy(&(ARR_LBOUND(result)[1]), astate->lbs, (ndims - 1) * sizeof(int));
+
+ memcpy(ARR_DATA_PTR(result), astate->data, astate->nbytes);
+ if (astate->hasnull)
+ array_bitmap_copy(ARR_NULLBITMAP(result), 0,
+ astate->nullbitmap, 0,
+ astate->nitems);
+
+ MemoryContextSwitchTo(oldcontext);
+
+ /* Clean up all the junk */
+ if (release)
+ MemoryContextDelete(astate->astate.mcontext);
+
+ PG_RETURN_DATUM(PointerGetDatum(result));
+ }
+
Datum
array_larger(PG_FUNCTION_ARGS)
{
diff --git a/src/include/catalog/pg_aggregate.h b/src/include/catalog/pg_aggregate.h
new file mode 100644
index 3ba9e5e..2005199
*** a/src/include/catalog/pg_aggregate.h
--- b/src/include/catalog/pg_aggregate.h
*************** DATA(insert ( 2901 n 0 xmlconcat2 -
*** 275,280 ****
--- 275,281 ----
/* array */
DATA(insert ( 2335 n 0 array_agg_transfn array_agg_finalfn - - - t f 0 2281 0 0 0 _null_ _null_ ));
+ DATA(insert ( 6005 n 0 array_agg_anyarray_transfn array_agg_anyarray_finalfn - - - t f 0 2281 0 0 0 _null_ _null_ ));
/* text */
DATA(insert ( 3538 n 0 string_agg_transfn string_agg_finalfn - - - f f 0 2281 0 0 0 _null_ _null_ ));
diff --git a/src/include/catalog/pg_proc.h b/src/include/catalog/pg_proc.h
new file mode 100644
index b6dc1b8..9273c1f
*** a/src/include/catalog/pg_proc.h
--- b/src/include/catalog/pg_proc.h
*************** DATA(insert OID = 3167 ( array_remove
*** 879,889 ****
DESCR("remove any occurrences of an element from an array");
DATA(insert OID = 3168 ( array_replace PGNSP PGUID 12 1 0 0 0 f f f f f f i 3 0 2277 "2277 2283 2283" _null_ _null_ _null_ _null_ array_replace _null_ _null_ _null_ ));
DESCR("replace any occurrences of an element in an array");
! DATA(insert OID = 2333 ( array_agg_transfn PGNSP PGUID 12 1 0 0 0 f f f f f f i 2 0 2281 "2281 2283" _null_ _null_ _null_ _null_ array_agg_transfn _null_ _null_ _null_ ));
DESCR("aggregate transition function");
! DATA(insert OID = 2334 ( array_agg_finalfn PGNSP PGUID 12 1 0 0 0 f f f f f f i 2 0 2277 "2281 2283" _null_ _null_ _null_ _null_ array_agg_finalfn _null_ _null_ _null_ ));
DESCR("aggregate final function");
! DATA(insert OID = 2335 ( array_agg PGNSP PGUID 12 1 0 0 0 t f f f f f i 1 0 2277 "2283" _null_ _null_ _null_ _null_ aggregate_dummy _null_ _null_ _null_ ));
DESCR("concatenate aggregate input into an array");
DATA(insert OID = 3218 ( width_bucket PGNSP PGUID 12 1 0 0 0 f f f f t f i 2 0 23 "2283 2277" _null_ _null_ _null_ _null_ width_bucket_array _null_ _null_ _null_ ));
DESCR("bucket number of operand given a sorted array of bucket lower bounds");
--- 879,895 ----
DESCR("remove any occurrences of an element from an array");
DATA(insert OID = 3168 ( array_replace PGNSP PGUID 12 1 0 0 0 f f f f f f i 3 0 2277 "2277 2283 2283" _null_ _null_ _null_ _null_ array_replace _null_ _null_ _null_ ));
DESCR("replace any occurrences of an element in an array");
! DATA(insert OID = 2333 ( array_agg_transfn PGNSP PGUID 12 1 0 0 0 f f f f f f i 2 0 2281 "2281 2776" _null_ _null_ _null_ _null_ array_agg_transfn _null_ _null_ _null_ ));
DESCR("aggregate transition function");
! DATA(insert OID = 2334 ( array_agg_finalfn PGNSP PGUID 12 1 0 0 0 f f f f f f i 2 0 2277 "2281 2776" _null_ _null_ _null_ _null_ array_agg_finalfn _null_ _null_ _null_ ));
DESCR("aggregate final function");
! DATA(insert OID = 2335 ( array_agg PGNSP PGUID 12 1 0 0 0 t f f f f f i 1 0 2277 "2776" _null_ _null_ _null_ _null_ aggregate_dummy _null_ _null_ _null_ ));
! DESCR("concatenate aggregate input into an array");
! DATA(insert OID = 6003 ( array_agg_anyarray_transfn PGNSP PGUID 12 1 0 0 0 f f f f f f i 2 0 2281 "2281 2277" _null_ _null_ _null_ _null_ array_agg_anyarray_transfn _null_ _null_ _null_ ));
! DESCR("aggregate transition function");
! DATA(insert OID = 6004 ( array_agg_anyarray_finalfn PGNSP PGUID 12 1 0 0 0 f f f f f f i 2 0 2277 "2281 2277" _null_ _null_ _null_ _null_ array_agg_anyarray_finalfn _null_ _null_ _null_ ));
! DESCR("aggregate final function");
! DATA(insert OID = 6005 ( array_agg PGNSP PGUID 12 1 0 0 0 t f f f f f i 1 0 2277 "2277" _null_ _null_ _null_ _null_ aggregate_dummy _null_ _null_ _null_ ));
DESCR("concatenate aggregate input into an array");
DATA(insert OID = 3218 ( width_bucket PGNSP PGUID 12 1 0 0 0 f f f f t f i 2 0 23 "2283 2277" _null_ _null_ _null_ _null_ width_bucket_array _null_ _null_ _null_ ));
DESCR("bucket number of operand given a sorted array of bucket lower bounds");
diff --git a/src/include/utils/array.h b/src/include/utils/array.h
new file mode 100644
index e744314..c6fbc7f
*** a/src/include/utils/array.h
--- b/src/include/utils/array.h
*************** typedef struct
*** 76,89 ****
/*
* working state for accumArrayResult() and friends
*/
typedef struct ArrayBuildState
{
MemoryContext mcontext; /* where all the temp stuff is kept */
- Datum *dvalues; /* array of accumulated Datums */
- bool *dnulls; /* array of is-null flags for Datums */
- int alen; /* allocated length of above arrays */
- int nelems; /* number of valid entries in above arrays */
Oid element_type; /* data type of the Datums */
int16 typlen; /* needed info about datatype */
bool typbyval;
--- 76,90 ----
/*
* working state for accumArrayResult() and friends
+ *
+ * is_array_accum: whether accumulating array values.
+ * (if true must be casted to ArrayBuildStateArray, else
+ * cast to ArrayBuildStateScalar)
*/
typedef struct ArrayBuildState
{
+ bool is_array_accum;
MemoryContext mcontext; /* where all the temp stuff is kept */
Oid element_type; /* data type of the Datums */
int16 typlen; /* needed info about datatype */
bool typbyval;
*************** typedef struct ArrayBuildState
*** 91,96 ****
--- 92,134 ----
} ArrayBuildState;
/*
+ * array build state for array accumulation of scalar datums
+ */
+ typedef struct ArrayBuildStateScalar
+ {
+ ArrayBuildState astate;
+
+ Datum *dvalues; /* array of accumulated Datums */
+ bool *dnulls; /* array of is-null flags for Datums */
+ int alen; /* allocated length of above arrays */
+ int nelems; /* number of valid entries in above arrays */
+ } ArrayBuildStateScalar;
+
+
+ /*
+ * array build state for array accumulation of array datums
+ */
+ typedef struct
+ {
+ ArrayBuildState astate;
+
+ char *data; /* array of accumulated data */
+ bits8 *nullbitmap; /* bitmap of is-null flags for data */
+
+ int abytes; /* allocated length of above arrays */
+ int aitems; /* allocated length of above arrays */
+ int nbytes; /* number of used bytes in above arrays */
+ int nitems; /* number of elements in above arrays */
+ int narray; /* number of array accumulated */
+
+ int ndims; /* element dimensions */
+ int *dims;
+ int *lbs;
+
+ bool hasnull; /* any element has null */
+ } ArrayBuildStateArray;
+
+ /*
* structure to cache type metadata needed for array manipulation
*/
typedef struct ArrayMetaState
*************** extern Datum makeArrayResult(ArrayBuildS
*** 260,266 ****
MemoryContext rcontext);
extern Datum makeMdArrayResult(ArrayBuildState *astate, int ndims,
int *dims, int *lbs, MemoryContext rcontext, bool release);
!
extern ArrayIterator array_create_iterator(ArrayType *arr, int slice_ndim);
extern bool array_iterate(ArrayIterator iterator, Datum *value, bool *isnull);
extern void array_free_iterator(ArrayIterator iterator);
--- 298,305 ----
MemoryContext rcontext);
extern Datum makeMdArrayResult(ArrayBuildState *astate, int ndims,
int *dims, int *lbs, MemoryContext rcontext, bool release);
! extern Datum makeArrayResultArray(ArrayBuildStateArray *astate,
! MemoryContext rcontext, bool release);
extern ArrayIterator array_create_iterator(ArrayType *arr, int slice_ndim);
extern bool array_iterate(ArrayIterator iterator, Datum *value, bool *isnull);
extern void array_free_iterator(ArrayIterator iterator);
*************** extern ArrayType *create_singleton_array
*** 293,298 ****
--- 332,340 ----
extern Datum array_agg_transfn(PG_FUNCTION_ARGS);
extern Datum array_agg_finalfn(PG_FUNCTION_ARGS);
+ extern Datum array_agg_anyarray_transfn(PG_FUNCTION_ARGS);
+ extern Datum array_agg_anyarray_finalfn(PG_FUNCTION_ARGS);
+
/*
* prototypes for functions defined in array_typanalyze.c
*/
diff --git a/src/test/regress/expected/aggregates.out b/src/test/regress/expected/aggregates.out
new file mode 100644
index 58df854..7568724
*** a/src/test/regress/expected/aggregates.out
--- b/src/test/regress/expected/aggregates.out
*************** select array_agg(distinct a order by a d
*** 914,919 ****
--- 914,984 ----
{3,2,1,NULL}
(1 row)
+ -- array_agg(anyarray)
+ select array_agg(ar)
+ from (values ('{1,2}'::int[]), ('{3,4}'::int[])) v(ar);
+ array_agg
+ ---------------
+ {{1,2},{3,4}}
+ (1 row)
+
+ select array_agg(distinct ar order by ar desc)
+ from (select array[i / 2] from generate_series(1,10) a(i)) b(ar);
+ array_agg
+ ---------------------------
+ {{5},{4},{3},{2},{1},{0}}
+ (1 row)
+
+ select array_agg(ar)
+ from (select array_agg(array[i, i+1, i-1])
+ from generate_series(1,2) a(i)) b(ar);
+ array_agg
+ ---------------------
+ {{{1,2,0},{2,3,1}}}
+ (1 row)
+
+ -- array_agg(anyarray), varlena types
+ select array_agg(array[1.2,1.3,1.4]) from generate_series(1,3);
+ array_agg
+ ---------------------------------------------
+ {{1.2,1.3,1.4},{1.2,1.3,1.4},{1.2,1.3,1.4}}
+ (1 row)
+
+ select array_agg(array['Hello','Hohoho','Hi']) from generate_series(1,3);
+ array_agg
+ ---------------------------------------------------------
+ {{Hello,Hohoho,Hi},{Hello,Hohoho,Hi},{Hello,Hohoho,Hi}}
+ (1 row)
+
+ -- array_agg(anyarray), arrays with nulls
+ select array_agg(array[i, null, i+1, null, i+2]) from generate_series(1,3) g(i);
+ array_agg
+ ---------------------------------------------------------
+ {{1,NULL,2,NULL,3},{2,NULL,3,NULL,4},{3,NULL,4,NULL,5}}
+ (1 row)
+
+ select array_agg(array[1.1+ i, null, 1.1+i+1, null, 1.1+i+2]) from generate_series(1,3) g(i);
+ array_agg
+ ---------------------------------------------------------------------------
+ {{2.1,NULL,3.1,NULL,4.1},{3.1,NULL,4.1,NULL,5.1},{4.1,NULL,5.1,NULL,6.1}}
+ (1 row)
+
+ select array_agg(array[null, 'Hello','Hohoho', null,'Hi']) from generate_series(1,3);
+ array_agg
+ ---------------------------------------------------------------------------------------
+ {{NULL,Hello,Hohoho,NULL,Hi},{NULL,Hello,Hohoho,NULL,Hi},{NULL,Hello,Hohoho,NULL,Hi}}
+ (1 row)
+
+ select array_agg(array[[null, 'Hello', null, 'Hi'],['Hello', null, 'Hi', null]]) from generate_series(1,2);
+ array_agg
+ -------------------------------------------------------------------------------------------
+ {{{NULL,Hello,NULL,Hi},{Hello,NULL,Hi,NULL}},{{NULL,Hello,NULL,Hi},{Hello,NULL,Hi,NULL}}}
+ (1 row)
+
+ select array_agg('{}'::int[]) from generate_series(1,2);
+ ERROR: cannot accumulate empty arrays
+ select array_agg(null::int[]) from generate_series(1,2);
+ ERROR: cannot accumulate null arrays
-- multi-arg aggs, strict/nonstrict, distinct/order by
select aggfstr(a,b,c)
from (values (1,3,'foo'),(0,null,null),(2,2,'bar'),(3,1,'baz')) v(a,b,c);
diff --git a/src/test/regress/expected/arrays.out b/src/test/regress/expected/arrays.out
new file mode 100644
index 46eff67..e80ebec
*** a/src/test/regress/expected/arrays.out
--- b/src/test/regress/expected/arrays.out
*************** select array_agg(unique1) from tenk1 whe
*** 1521,1526 ****
--- 1521,1543 ----
(1 row)
+ select array(select unique1 from tenk1 where unique1 < 15 order by unique1);
+ array
+ --------------------------------------
+ {0,1,2,3,4,5,6,7,8,9,10,11,12,13,14}
+ (1 row)
+
+ select array(select array[i,i/2] from generate_series(1,5) a(i));
+ array
+ ---------------------------------
+ {{1,0},{2,1},{3,1},{4,2},{5,2}}
+ (1 row)
+
+ -- cannot accumulate null arrays and empty arrays
+ select array(select null::int[]);
+ ERROR: cannot accumulate null arrays
+ select array(select '{}'::int[]);
+ ERROR: cannot accumulate empty arrays
select unnest(array[1,2,3]);
unnest
--------
diff --git a/src/test/regress/sql/aggregates.sql b/src/test/regress/sql/aggregates.sql
new file mode 100644
index 8096a6f..165f248
*** a/src/test/regress/sql/aggregates.sql
--- b/src/test/regress/sql/aggregates.sql
*************** select array_agg(distinct a order by a d
*** 322,327 ****
--- 322,350 ----
select array_agg(distinct a order by a desc nulls last)
from (values (1),(2),(1),(3),(null),(2)) v(a);
+ -- array_agg(anyarray)
+ select array_agg(ar)
+ from (values ('{1,2}'::int[]), ('{3,4}'::int[])) v(ar);
+ select array_agg(distinct ar order by ar desc)
+ from (select array[i / 2] from generate_series(1,10) a(i)) b(ar);
+ select array_agg(ar)
+ from (select array_agg(array[i, i+1, i-1])
+ from generate_series(1,2) a(i)) b(ar);
+
+ -- array_agg(anyarray), varlena types
+ select array_agg(array[1.2,1.3,1.4]) from generate_series(1,3);
+ select array_agg(array['Hello','Hohoho','Hi']) from generate_series(1,3);
+
+ -- array_agg(anyarray), arrays with nulls
+ select array_agg(array[i, null, i+1, null, i+2]) from generate_series(1,3) g(i);
+ select array_agg(array[1.1+ i, null, 1.1+i+1, null, 1.1+i+2]) from generate_series(1,3) g(i);
+ select array_agg(array[null, 'Hello','Hohoho', null,'Hi']) from generate_series(1,3);
+
+ select array_agg(array[[null, 'Hello', null, 'Hi'],['Hello', null, 'Hi', null]]) from generate_series(1,2);
+
+ select array_agg('{}'::int[]) from generate_series(1,2);
+ select array_agg(null::int[]) from generate_series(1,2);
+
-- multi-arg aggs, strict/nonstrict, distinct/order by
select aggfstr(a,b,c)
diff --git a/src/test/regress/sql/arrays.sql b/src/test/regress/sql/arrays.sql
new file mode 100644
index fa8a20a..cb00f5f
*** a/src/test/regress/sql/arrays.sql
--- b/src/test/regress/sql/arrays.sql
*************** select array_agg(ten) from (select ten f
*** 432,437 ****
--- 432,443 ----
select array_agg(nullif(ten, 4)) from (select ten from tenk1 where unique1 < 15 order by unique1) ss;
select array_agg(unique1) from tenk1 where unique1 < -15;
+ select array(select unique1 from tenk1 where unique1 < 15 order by unique1);
+ select array(select array[i,i/2] from generate_series(1,5) a(i));
+ -- cannot accumulate null arrays and empty arrays
+ select array(select null::int[]);
+ select array(select '{}'::int[]);
+
select unnest(array[1,2,3]);
select * from unnest(array[1,2,3]);
select unnest(array[1,2,3,4.5]::float8[]);
Re: Function array_agg(array)
Hi Ali
I checked a code. I am thinking so code organization is not good.
accumArrayResult is too long now. makeMdArrayResult will not work, when
arrays was joined (it is not consistent now). I don't like a usage of
state->is_array_accum in array_userfunc.c -- it is signal of wrong
wrapping.
next question: there is function array_append(anyarray, anyelement). Isn't
time to define array_append(anyarray, anyarray) now?
Regards
Pavel
2014-10-24 15:05 GMT+02:00 Pavel Stehule <pavel.stehule@gmail.com>:
Show quoted text
2014-10-24 13:58 GMT+02:00 Pavel Stehule <pavel.stehule@gmail.com>:
2014-10-24 11:43 GMT+02:00 Ali Akbar <the.apaan@gmail.com>:
2014-10-24 16:26 GMT+07:00 Pavel Stehule <pavel.stehule@gmail.com>:
Hi
some in last patch is wrong, I cannot to compile it:
arrayfuncs.c: In function ‘accumArrayResult’:
arrayfuncs.c:4603:9: error: ‘ArrayBuildState’ has no member named ‘alen’
astate->alen = 64; /* arbitrary starting array size */
^
arrayfuncs.c:4604:9: error: ‘ArrayBuildState’ has no member named
‘dvalues’
astate->dvalues = (Datum *) palloc(astate->alen * sizeof(Datum));
^
arrayfuncs.c:4604:44: error: ‘ArrayBuildState’ has no member named
‘alen’
astate->dvalues = (Datum *) palloc(astate->alen * sizeof(Datum));
^
arrayfuncs.c:4605:9: error: ‘ArrayBuildState’ has no member named
‘dnulls’
astate->dnulls = (bool *) palloc(astate->alen * sizeof(bool));
^
arrayfuncs.c:4605:42: error: ‘ArrayBuildState’ has no member named
‘alen’
astate->dnulls = (bool *) palloc(astate->alen * sizeof(bool));
^
arrayfuncs.c:4606:9: error: ‘ArrayBuildState’ has no member named
‘nelems’
astate->nelems = 0;
^
arrayfuncs.c:4618:13: error: ‘ArrayBuildState’ has no member named
‘nelems’
if (astate->nelems >= astate->alen)
^
arrayfuncs.c:4618:31: error: ‘ArrayBuildState’ has no member named
‘alen’
if (astate->nelems >= astate->alen)
^
arrayfuncs.c:4620:10: error: ‘ArrayBuildState’ has no member named
‘alen’
astate->alen *= 2;Sorry, correct patch attached.
This patch is in patience format (git --patience ..). In previous
patches, i use context format (git --patience ... | filterdiff
--format=context), but it turns out that some modification is lost.last version is compileable, but some is still broken
postgres=# select array_agg(array[i, i+1, i-1])
from generate_series(1,2) a(i);
ERROR: could not find array type for data type integer[]I am sorry, it works - I had a problem with broken database
I fixed small issue in regress tests and I enhanced tests for varlena
types and null values.Regards
Pavel
but array(subselect) works
postgres=# select array(select a from xx);
array
-------------------
{{1,2,3},{1,2,3}}
(1 row)Regards
Pavel
--
Ali Akbar
Re: Function array_agg(array)
I fixed small issue in regress tests and I enhanced tests for varlena
types and null values.
Thanks.
it is about 15% faster than original implementation.
15% faster than array_agg(scalar)? I haven't verify the performance, but
because the internal array data and null bitmap is copied as-is, that will
be faster.
2014-10-25 1:51 GMT+07:00 Pavel Stehule <pavel.stehule@gmail.com>:
Hi Ali
I checked a code. I am thinking so code organization is not good.
accumArrayResult is too long now. makeMdArrayResult will not work, when
arrays was joined (it is not consistent now). I don't like a usage of
state->is_array_accum in array_userfunc.c -- it is signal of wrong
wrapping.
Yes, i was thinking the same. Attached WIP patch to reorganizate the code.
makeMdArrayResult works now, with supplied arguments act as override from
default values calculated from ArrayBuildStateArray.
In array_userfunc.c, because we don't want to override ndims, dims and lbs,
i copied array_agg_finalfn and only change the call to makeMdArrayResult
(we don't uses makeArrayResult because we want to set release to false).
Another alternative is to create new makeArrayResult-like function that has
parameter bool release.
next question: there is function array_append(anyarray, anyelement). Isn't
time to define array_append(anyarray, anyarray) now?
There is array_cat(anyarray, anyarray):
/*-----------------------------------------------------------------------------
* array_cat :
* concatenate two nD arrays to form an nD array, or
* push an (n-1)D array onto the end of an nD array
*----------------------------------------------------------------------------
*/
Regards,
--
Ali Akbar
Attachments:
array_agg_anyarray-8.patchtext/x-patch; charset=US-ASCII; name=array_agg_anyarray-8.patchDownload
diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml
index 7e5bcd9..f59738a 100644
--- a/doc/src/sgml/func.sgml
+++ b/doc/src/sgml/func.sgml
@@ -12046,6 +12046,22 @@ NULL baz</literallayout>(3 rows)</entry>
<row>
<entry>
<indexterm>
+ <primary>array_agg</primary>
+ </indexterm>
+ <function>array_agg(<replaceable class="parameter">anyarray</replaceable>)</function>
+ </entry>
+ <entry>
+ any
+ </entry>
+ <entry>
+ the same array type as input type
+ </entry>
+ <entry>input arrays, aggregated into higher-order multidimesional array. Rejects NULL and empty array as input.</entry>
+ </row>
+
+ <row>
+ <entry>
+ <indexterm>
<primary>average</primary>
</indexterm>
<indexterm>
diff --git a/doc/src/sgml/syntax.sgml b/doc/src/sgml/syntax.sgml
index 2f0680f..8c182a4 100644
--- a/doc/src/sgml/syntax.sgml
+++ b/doc/src/sgml/syntax.sgml
@@ -2238,6 +2238,11 @@ SELECT ARRAY(SELECT oid FROM pg_proc WHERE proname LIKE 'bytea%');
array
-----------------------------------------------------------------------
{2011,1954,1948,1952,1951,1244,1950,2005,1949,1953,2006,31,2412,2413}
+
+SELECT ARRAY(SELECT array(select i) FROM generate_series(1,5) a(i));
+ array
+-----------------------
+ {{1},{2},{3},{4},{5}}
(1 row)
</programlisting>
The subquery must return a single column. The resulting
diff --git a/src/backend/nodes/nodeFuncs.c b/src/backend/nodes/nodeFuncs.c
index 41e973b..0261fcb 100644
--- a/src/backend/nodes/nodeFuncs.c
+++ b/src/backend/nodes/nodeFuncs.c
@@ -108,12 +108,16 @@ exprType(const Node *expr)
type = exprType((Node *) tent->expr);
if (sublink->subLinkType == ARRAY_SUBLINK)
{
- type = get_array_type(type);
- if (!OidIsValid(type))
- ereport(ERROR,
- (errcode(ERRCODE_UNDEFINED_OBJECT),
- errmsg("could not find array type for data type %s",
- format_type_be(exprType((Node *) tent->expr)))));
+ if (!OidIsValid(get_element_type(type)))
+ {
+ /* not array, so check for its array type */
+ type = get_array_type(type);
+ if (!OidIsValid(type))
+ ereport(ERROR,
+ (errcode(ERRCODE_UNDEFINED_OBJECT),
+ errmsg("could not find array type for data type %s",
+ format_type_be(exprType((Node *) tent->expr)))));
+ }
}
}
else if (sublink->subLinkType == MULTIEXPR_SUBLINK)
@@ -139,12 +143,16 @@ exprType(const Node *expr)
type = subplan->firstColType;
if (subplan->subLinkType == ARRAY_SUBLINK)
{
- type = get_array_type(type);
- if (!OidIsValid(type))
- ereport(ERROR,
- (errcode(ERRCODE_UNDEFINED_OBJECT),
- errmsg("could not find array type for data type %s",
- format_type_be(subplan->firstColType))));
+ if (!OidIsValid(get_element_type(type)))
+ {
+ /* not array, so check for its array type */
+ type = get_array_type(type);
+ if (!OidIsValid(type))
+ ereport(ERROR,
+ (errcode(ERRCODE_UNDEFINED_OBJECT),
+ errmsg("could not find array type for data type %s",
+ format_type_be(subplan->firstColType))));
+ }
}
}
else if (subplan->subLinkType == MULTIEXPR_SUBLINK)
diff --git a/src/backend/optimizer/plan/subselect.c b/src/backend/optimizer/plan/subselect.c
index 3e7dc85..8fc8b49 100644
--- a/src/backend/optimizer/plan/subselect.c
+++ b/src/backend/optimizer/plan/subselect.c
@@ -668,10 +668,16 @@ build_subplan(PlannerInfo *root, Plan *plan, PlannerInfo *subroot,
Assert(!te->resjunk);
Assert(testexpr == NULL);
- arraytype = get_array_type(exprType((Node *) te->expr));
- if (!OidIsValid(arraytype))
- elog(ERROR, "could not find array type for datatype %s",
- format_type_be(exprType((Node *) te->expr)));
+
+ arraytype = exprType((Node *) te->expr);
+ if (!OidIsValid(get_element_type(arraytype)))
+ {
+ /* not array, so get the array type */
+ arraytype = get_array_type(exprType((Node *) te->expr));
+ if (!OidIsValid(arraytype))
+ elog(ERROR, "could not find array type for datatype %s",
+ format_type_be(exprType((Node *) te->expr)));
+ }
prm = generate_new_param(root,
arraytype,
exprTypmod((Node *) te->expr),
diff --git a/src/backend/utils/adt/array_userfuncs.c b/src/backend/utils/adt/array_userfuncs.c
index 831466d..8ad623c 100644
--- a/src/backend/utils/adt/array_userfuncs.c
+++ b/src/backend/utils/adt/array_userfuncs.c
@@ -16,7 +16,6 @@
#include "utils/builtins.h"
#include "utils/lsyscache.h"
-
/*-----------------------------------------------------------------------------
* array_push :
* push an element onto either end of a one-dimensional array
@@ -529,16 +528,57 @@ array_agg_finalfn(PG_FUNCTION_ARGS)
state = (ArrayBuildState *) PG_GETARG_POINTER(0);
- dims[0] = state->nelems;
+ /*
+ * Make the result. We cannot release the ArrayBuildState because
+ * sometimes aggregate final functions are re-executed. Rather, it is
+ * nodeAgg.c's responsibility to reset the aggcontext when it's safe to do
+ * so.
+ */
+ dims[0] = ((ArrayBuildStateScalar *) state)->nelems;
lbs[0] = 1;
+ result = makeMdArrayResult(state, 1, dims, lbs,
+ CurrentMemoryContext,
+ false);
+
+ PG_RETURN_DATUM(result);
+}
+
+/*
+ * ARRAY_AGG(anyarray) aggregate function
+ */
+Datum
+array_agg_anyarray_transfn(PG_FUNCTION_ARGS)
+{
+ return array_agg_transfn(fcinfo);
+}
+
+Datum
+array_agg_anyarray_finalfn(PG_FUNCTION_ARGS)
+{
+ Datum result;
+ ArrayBuildState *state;
+
+ /*
+ * Test for null before Asserting we are in right context. This is to
+ * avoid possible Assert failure in 8.4beta installations, where it is
+ * possible for users to create NULL constants of type internal.
+ */
+ if (PG_ARGISNULL(0))
+ PG_RETURN_NULL(); /* returns null iff no input values */
+
+ /* cannot be called directly because of internal-type argument */
+ Assert(AggCheckCallContext(fcinfo, NULL));
+
+ state = (ArrayBuildState *) PG_GETARG_POINTER(0);
+
/*
* Make the result. We cannot release the ArrayBuildState because
* sometimes aggregate final functions are re-executed. Rather, it is
* nodeAgg.c's responsibility to reset the aggcontext when it's safe to do
* so.
*/
- result = makeMdArrayResult(state, 1, dims, lbs,
+ result = makeMdArrayResult(state, -1, NULL, NULL,
CurrentMemoryContext,
false);
diff --git a/src/backend/utils/adt/arrayfuncs.c b/src/backend/utils/adt/arrayfuncs.c
index 6c8b41d..ba51792 100644
--- a/src/backend/utils/adt/arrayfuncs.c
+++ b/src/backend/utils/adt/arrayfuncs.c
@@ -145,6 +145,19 @@ static int width_bucket_array_variable(Datum operand,
Oid collation,
TypeCacheEntry *typentry);
+/* internal functions of accumArrayResult */
+static ArrayBuildStateScalar *initializeArrayBuildStateScalar(Oid element_type,
+ MemoryContext arr_context);
+static ArrayBuildStateArray *initializeArrayBuildStateArray(Oid subelement_type,
+ MemoryContext arr_context);
+static void appendScalarDatum(ArrayBuildStateScalar *astate_scalar,
+ Datum dvalue, bool disnull);
+static void appendArrayDatum(ArrayBuildStateArray *astate_array,
+ Datum dvalue, bool disnull);
+
+/* internal function of makeMdArrayResult */
+static ArrayType *makeMdArrayResultArray(ArrayBuildStateArray *astate,
+ int ndims, int *dims, int *lbs);
/*
* array_in :
@@ -4588,9 +4601,13 @@ accumArrayResult(ArrayBuildState *astate,
MemoryContext arr_context,
oldcontext;
+ ArrayBuildStateScalar *astate_scalar = NULL; /* for scalar datum accumulation */
+ ArrayBuildStateArray *astate_array = NULL; /* for array datum accumulation */
+
if (astate == NULL)
{
/* First time through --- initialize */
+ Oid subelement_type = get_element_type(element_type);
/* Make a temporary context to hold all the junk */
arr_context = AllocSetContextCreate(rcontext,
@@ -4599,33 +4616,127 @@ accumArrayResult(ArrayBuildState *astate,
ALLOCSET_DEFAULT_INITSIZE,
ALLOCSET_DEFAULT_MAXSIZE);
oldcontext = MemoryContextSwitchTo(arr_context);
- astate = (ArrayBuildState *) palloc(sizeof(ArrayBuildState));
- astate->mcontext = arr_context;
- astate->alen = 64; /* arbitrary starting array size */
- astate->dvalues = (Datum *) palloc(astate->alen * sizeof(Datum));
- astate->dnulls = (bool *) palloc(astate->alen * sizeof(bool));
- astate->nelems = 0;
- astate->element_type = element_type;
- get_typlenbyvalalign(element_type,
- &astate->typlen,
- &astate->typbyval,
- &astate->typalign);
+
+ if (!OidIsValid(subelement_type))
+ {
+ astate_scalar = initializeArrayBuildStateScalar(element_type, arr_context);
+ astate = (ArrayBuildState *) astate_scalar;
+
+ appendScalarDatum(astate_scalar, dvalue, disnull);
+ }
+ else
+ {
+ astate_array = initializeArrayBuildStateArray(subelement_type, arr_context);
+ astate = (ArrayBuildState *) astate_array;
+
+ appendArrayDatum(astate_array, dvalue, disnull);
+ }
}
else
{
oldcontext = MemoryContextSwitchTo(astate->mcontext);
- Assert(astate->element_type == element_type);
- /* enlarge dvalues[]/dnulls[] if needed */
- if (astate->nelems >= astate->alen)
+ if (!astate->is_array_accum)
{
- astate->alen *= 2;
- astate->dvalues = (Datum *)
- repalloc(astate->dvalues, astate->alen * sizeof(Datum));
- astate->dnulls = (bool *)
- repalloc(astate->dnulls, astate->alen * sizeof(bool));
+ Assert(astate->element_type == element_type);
+ astate_scalar = (ArrayBuildStateScalar *) astate;
+
+ /* enlarge dvalues[]/dnulls[] if needed */
+ if (astate_scalar->nelems >= astate_scalar->alen)
+ {
+ astate_scalar->alen *= 2;
+ astate_scalar->dvalues = (Datum *)
+ repalloc(astate_scalar->dvalues, astate_scalar->alen * sizeof(Datum));
+ astate_scalar->dnulls = (bool *)
+ repalloc(astate_scalar->dnulls, astate_scalar->alen * sizeof(bool));
+ }
+
+ appendScalarDatum(astate_scalar, dvalue, disnull);
+ }
+ else
+ {
+ Assert(astate->element_type == get_element_type(element_type));
+ astate_array = (ArrayBuildStateArray *) astate;
+ /*
+ * alloc & realloc placed in appendArrayDatum because
+ * we need to know the dvalue length first
+ */
+ appendArrayDatum(astate_array, dvalue, disnull);
}
}
+ MemoryContextSwitchTo(oldcontext);
+
+ return astate;
+}
+
+/*
+ * Initialize ArrayBuildState for scalar accumulation (internal function of
+ * accumArrayResult)
+ */
+ArrayBuildStateScalar *
+initializeArrayBuildStateScalar(Oid element_type,
+ MemoryContext arr_context)
+{
+ ArrayBuildStateScalar *astate_scalar;
+
+ /* scalar accumulate */
+ astate_scalar = (ArrayBuildStateScalar *) palloc(sizeof(ArrayBuildStateScalar));
+ astate_scalar->astate.is_array_accum = false;
+ astate_scalar->astate.mcontext = arr_context;
+
+ astate_scalar->alen = 64; /* arbitrary starting array size */
+ astate_scalar->dvalues = (Datum *) palloc(astate_scalar->alen * sizeof(Datum));
+ astate_scalar->dnulls = (bool *) palloc(astate_scalar->alen * sizeof(bool));
+ astate_scalar->nelems = 0;
+
+ astate_scalar->astate.element_type = element_type;
+ get_typlenbyvalalign(element_type,
+ &astate_scalar->astate.typlen,
+ &astate_scalar->astate.typbyval,
+ &astate_scalar->astate.typalign);
+
+ return astate_scalar;
+}
+
+/*
+ * Initialize ArrayBuildState for array accum (internal of accumArrayResult)
+ */
+ArrayBuildStateArray *
+initializeArrayBuildStateArray(Oid subelement_type,
+ MemoryContext arr_context)
+{
+ ArrayBuildStateArray *astate_array;
+
+ /* array accumulate */
+ astate_array = (ArrayBuildStateArray *) palloc(sizeof(ArrayBuildStateArray));
+ astate_array->astate.is_array_accum = true;
+ astate_array->astate.mcontext = arr_context;
+
+ astate_array->abytes = 0;
+ astate_array->aitems = 0;
+ astate_array->data = NULL;
+ astate_array->nbytes = 0;
+ astate_array->hasnull = false;
+ astate_array->nullbitmap = NULL;
+ astate_array->nitems = 0;
+ astate_array->narray = 0;
+
+ astate_array->astate.element_type = subelement_type;
+ get_typlenbyvalalign(subelement_type,
+ &astate_array->astate.typlen,
+ &astate_array->astate.typbyval,
+ &astate_array->astate.typalign);
+
+ return astate_array;
+}
+
+/*
+ * Append a datum for scalar accumulation (internal of accumArrayResult)
+ */
+void
+appendScalarDatum(ArrayBuildStateScalar *astate_scalar,
+ Datum dvalue, bool disnull)
+{
/*
* Ensure pass-by-ref stuff is copied into mcontext; and detoast it too if
* it's varlena. (You might think that detoasting is not needed here
@@ -4634,25 +4745,129 @@ accumArrayResult(ArrayBuildState *astate,
* because that would mean array_agg_finalfn damages its input, which is
* verboten. Also, this way frequently saves one copying step.)
*/
- if (!disnull && !astate->typbyval)
+ if (!disnull && !astate_scalar->astate.typbyval)
{
- if (astate->typlen == -1)
+ if (astate_scalar->astate.typlen == -1)
dvalue = PointerGetDatum(PG_DETOAST_DATUM_COPY(dvalue));
else
- dvalue = datumCopy(dvalue, astate->typbyval, astate->typlen);
+ dvalue = datumCopy(dvalue, astate_scalar->astate.typbyval,
+ astate_scalar->astate.typlen);
}
- astate->dvalues[astate->nelems] = dvalue;
- astate->dnulls[astate->nelems] = disnull;
- astate->nelems++;
+ astate_scalar->dvalues[astate_scalar->nelems] = dvalue;
+ astate_scalar->dnulls[astate_scalar->nelems] = disnull;
+ astate_scalar->nelems++;
+}
- MemoryContextSwitchTo(oldcontext);
+/*
+ * Append a datum for array accum (internal of accumArrayResult)
+ */
+void
+appendArrayDatum(ArrayBuildStateArray *astate_array,
+ Datum dvalue, bool disnull)
+{
+ ArrayType *arg;
+ int *dims,
+ *lbs,
+ ndims,
+ nitems,
+ ndatabytes;
+ char *data;
+ int i;
- return astate;
+ if (disnull)
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("cannot accumulate null arrays")));
+
+ arg = DatumGetArrayTypeP(dvalue);
+
+ ndims = ARR_NDIM(arg);
+ dims = ARR_DIMS(arg);
+ lbs = ARR_LBOUND(arg);
+ data = ARR_DATA_PTR(arg);
+ nitems = ArrayGetNItems(ndims, dims);
+ ndatabytes = ARR_SIZE(arg) - ARR_DATA_OFFSET(arg);
+
+ if (astate_array->data == NULL)
+ {
+ /* first allocation */
+ if (ndims == 0)
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("cannot accumulate empty arrays")));
+
+ if (ndims + 1 > MAXDIM)
+ ereport(ERROR,
+ (errcode(ERRCODE_PROGRAM_LIMIT_EXCEEDED),
+ errmsg("number of array dimensions (%d) exceeds the maximum allowed (%d)",
+ ndims + 1, MAXDIM)));
+
+ astate_array->ndims = ndims;
+ astate_array->dims = (int *) palloc(ndims * sizeof(int));
+ astate_array->lbs = (int *) palloc(ndims * sizeof(int));
+ memcpy(astate_array->dims, dims, ndims * sizeof(int));
+ memcpy(astate_array->lbs, lbs, ndims * sizeof(int));
+
+ astate_array->abytes = ndatabytes >= 512 ? 4 * ndatabytes : 1024;
+ astate_array->data = (char *) palloc(astate_array->abytes);
+ astate_array->aitems = 4 * nitems;
+ }
+ else
+ {
+ if (astate_array->ndims != ndims)
+ ereport(ERROR,
+ (errcode(ERRCODE_ARRAY_SUBSCRIPT_ERROR),
+ errmsg("cannot accumulate incompatible arrays"),
+ errdetail("Arrays of %d and %d dimensions are not "
+ "compatible for concatenation.",
+ astate_array->ndims, ndims)));
+
+ for (i = 0; i < ndims; i++)
+ if (astate_array->dims[i] != dims[i] || astate_array->lbs[i] != lbs[i])
+ ereport(ERROR,
+ (errcode(ERRCODE_ARRAY_SUBSCRIPT_ERROR),
+ errmsg("cannot accumulate incompatible arrays"),
+ errdetail("Arrays with differing element dimensions are "
+ "not compatible for concatenation.")));
+
+ if (astate_array->nbytes + ndatabytes >= astate_array->abytes)
+ {
+ astate_array->abytes *= 2;
+ astate_array->data = (char *)
+ repalloc(astate_array->data, astate_array->abytes);
+ }
+ if (astate_array->nitems + nitems >= astate_array->aitems)
+ {
+ astate_array->aitems *= 2;
+ astate_array->nullbitmap = (bits8 *)
+ repalloc(astate_array->nullbitmap, (astate_array->aitems + 7) / 8);
+ }
+ }
+ memcpy(astate_array->data + astate_array->nbytes, data, ndatabytes);
+ astate_array->nbytes += ndatabytes;
+
+ if (ARR_HASNULL(arg) || astate_array->hasnull)
+ {
+ if (!astate_array->hasnull)
+ {
+ astate_array->hasnull = true;
+ astate_array->nullbitmap = (bits8 *) palloc((astate_array->aitems + 7) / 8);
+ array_bitmap_copy(astate_array->nullbitmap, 0,
+ NULL, 0,
+ astate_array->nitems);
+ }
+ array_bitmap_copy(astate_array->nullbitmap, astate_array->nitems,
+ ARR_NULLBITMAP(arg), 0,
+ nitems);
+ astate_array->nitems += nitems;
+ }
+ astate_array->narray += 1;
}
/*
- * makeArrayResult - produce 1-D final result of accumArrayResult
+ * makeArrayResult - produce 1-D final result of scalar accumArrayResult
+ * - produce N+1-D final result of array accumArrayResult
*
* astate is working state (not NULL)
* rcontext is where to construct result
@@ -4661,17 +4876,22 @@ Datum
makeArrayResult(ArrayBuildState *astate,
MemoryContext rcontext)
{
- int dims[1];
- int lbs[1];
+ if (!astate->is_array_accum)
+ {
+ int dims[1];
+ int lbs[1];
- dims[0] = astate->nelems;
- lbs[0] = 1;
+ dims[0] = ((ArrayBuildStateScalar *) astate)->nelems;
+ lbs[0] = 1;
- return makeMdArrayResult(astate, 1, dims, lbs, rcontext, true);
+ return makeMdArrayResult(astate, 1, dims, lbs, rcontext, true);
+ }
+ else
+ return makeMdArrayResult(astate, -1, NULL, NULL, rcontext, true);
}
/*
- * makeMdArrayResult - produce multi-D final result of accumArrayResult
+ * makeMdArrayResult - produce multi-D final result of scalar accumArrayResult
*
* beware: no check that specified dimensions match the number of values
* accumulated.
@@ -4679,6 +4899,11 @@ makeArrayResult(ArrayBuildState *astate,
* astate is working state (not NULL)
* rcontext is where to construct result
* release is true if okay to release working state
+ *
+ * for array accumulation:
+ * ndims == -1: default (calculated from astate)
+ * dims == NULL: default dims (calculated from astate)
+ * lbs == NULL: default (calculated from astate)
*/
Datum
makeMdArrayResult(ArrayBuildState *astate,
@@ -4694,8 +4919,13 @@ makeMdArrayResult(ArrayBuildState *astate,
/* Build the final array result in rcontext */
oldcontext = MemoryContextSwitchTo(rcontext);
- result = construct_md_array(astate->dvalues,
- astate->dnulls,
+ if (!astate->is_array_accum)
+ {
+ ArrayBuildStateScalar *astate_scalar;
+ astate_scalar = (ArrayBuildStateScalar *) astate;
+
+ result = construct_md_array(astate_scalar->dvalues,
+ astate_scalar->dnulls,
ndims,
dims,
lbs,
@@ -4703,6 +4933,16 @@ makeMdArrayResult(ArrayBuildState *astate,
astate->typlen,
astate->typbyval,
astate->typalign);
+ }
+ else
+ {
+ ArrayBuildStateArray *astate_array;
+ astate_array = (ArrayBuildStateArray *) astate;
+ result = makeMdArrayResultArray(astate_array,
+ ndims,
+ dims,
+ lbs);
+ }
MemoryContextSwitchTo(oldcontext);
@@ -4713,6 +4953,75 @@ makeMdArrayResult(ArrayBuildState *astate,
return PointerGetDatum(result);
}
+/*
+ * makeMdArrayResultArray - internal of makeArrayResult & makeMdArrayResult
+ *
+ * beware: no check that specified dimensions match the values
+ * accumulated.
+ *
+ * astate is working state (not NULL)
+ * ndims > -1 if wants to override default (calculated from astate)
+ * dims not NULL if wants to override default
+ * lbs not NULL if wants to override default
+ */
+ArrayType *
+makeMdArrayResultArray(ArrayBuildStateArray *astate,
+ int ndims,
+ int *dims,
+ int *lbs)
+{
+ ArrayType *result;
+
+ int dataoffset,
+ nbytes;
+
+ if (ndims == -1)
+ ndims = astate->ndims + 1;
+
+ nbytes = astate->nbytes;
+ /* compute required space */
+ if (astate->hasnull)
+ {
+ dataoffset = ARR_OVERHEAD_WITHNULLS(ndims, astate->nitems);
+ nbytes += dataoffset;
+ }
+ else
+ {
+ dataoffset = 0;
+ nbytes += ARR_OVERHEAD_NONULLS(ndims);
+ }
+
+ result = (ArrayType *) palloc0(nbytes);
+ SET_VARSIZE(result, nbytes);
+ result->ndim = astate->ndims + 1;
+ result->dataoffset = dataoffset;
+ result->elemtype = astate->astate.element_type;
+
+ if (dims != NULL)
+ memcpy(ARR_DIMS(result), dims, ndims * sizeof(int));
+ else
+ {
+ ARR_DIMS(result)[0] = astate->narray;
+ memcpy(&(ARR_DIMS(result)[1]), astate->dims, (ndims - 1) * sizeof(int));
+ }
+ if (lbs != NULL)
+ memcpy(ARR_LBOUND(result), lbs, ndims * sizeof(int));
+ else
+ {
+ ARR_LBOUND(result)[0] = 1;
+ memcpy(&(ARR_LBOUND(result)[1]), astate->lbs, (ndims - 1) * sizeof(int));
+ }
+
+ memcpy(ARR_DATA_PTR(result), astate->data, astate->nbytes);
+
+ if (astate->hasnull)
+ array_bitmap_copy(ARR_NULLBITMAP(result), 0,
+ astate->nullbitmap, 0,
+ astate->nitems);
+
+ return result;
+}
+
Datum
array_larger(PG_FUNCTION_ARGS)
{
diff --git a/src/include/catalog/pg_aggregate.h b/src/include/catalog/pg_aggregate.h
index 3ba9e5e..2005199 100644
--- a/src/include/catalog/pg_aggregate.h
+++ b/src/include/catalog/pg_aggregate.h
@@ -275,6 +275,7 @@ DATA(insert ( 2901 n 0 xmlconcat2 - - - - f f 0 142 0 0 0 _null_
/* array */
DATA(insert ( 2335 n 0 array_agg_transfn array_agg_finalfn - - - t f 0 2281 0 0 0 _null_ _null_ ));
+DATA(insert ( 6005 n 0 array_agg_anyarray_transfn array_agg_anyarray_finalfn - - - t f 0 2281 0 0 0 _null_ _null_ ));
/* text */
DATA(insert ( 3538 n 0 string_agg_transfn string_agg_finalfn - - - f f 0 2281 0 0 0 _null_ _null_ ));
diff --git a/src/include/catalog/pg_proc.h b/src/include/catalog/pg_proc.h
index b6dc1b8..9273c1f 100644
--- a/src/include/catalog/pg_proc.h
+++ b/src/include/catalog/pg_proc.h
@@ -879,11 +879,17 @@ DATA(insert OID = 3167 ( array_remove PGNSP PGUID 12 1 0 0 0 f f f f f f i 2
DESCR("remove any occurrences of an element from an array");
DATA(insert OID = 3168 ( array_replace PGNSP PGUID 12 1 0 0 0 f f f f f f i 3 0 2277 "2277 2283 2283" _null_ _null_ _null_ _null_ array_replace _null_ _null_ _null_ ));
DESCR("replace any occurrences of an element in an array");
-DATA(insert OID = 2333 ( array_agg_transfn PGNSP PGUID 12 1 0 0 0 f f f f f f i 2 0 2281 "2281 2283" _null_ _null_ _null_ _null_ array_agg_transfn _null_ _null_ _null_ ));
+DATA(insert OID = 2333 ( array_agg_transfn PGNSP PGUID 12 1 0 0 0 f f f f f f i 2 0 2281 "2281 2776" _null_ _null_ _null_ _null_ array_agg_transfn _null_ _null_ _null_ ));
DESCR("aggregate transition function");
-DATA(insert OID = 2334 ( array_agg_finalfn PGNSP PGUID 12 1 0 0 0 f f f f f f i 2 0 2277 "2281 2283" _null_ _null_ _null_ _null_ array_agg_finalfn _null_ _null_ _null_ ));
+DATA(insert OID = 2334 ( array_agg_finalfn PGNSP PGUID 12 1 0 0 0 f f f f f f i 2 0 2277 "2281 2776" _null_ _null_ _null_ _null_ array_agg_finalfn _null_ _null_ _null_ ));
DESCR("aggregate final function");
-DATA(insert OID = 2335 ( array_agg PGNSP PGUID 12 1 0 0 0 t f f f f f i 1 0 2277 "2283" _null_ _null_ _null_ _null_ aggregate_dummy _null_ _null_ _null_ ));
+DATA(insert OID = 2335 ( array_agg PGNSP PGUID 12 1 0 0 0 t f f f f f i 1 0 2277 "2776" _null_ _null_ _null_ _null_ aggregate_dummy _null_ _null_ _null_ ));
+DESCR("concatenate aggregate input into an array");
+DATA(insert OID = 6003 ( array_agg_anyarray_transfn PGNSP PGUID 12 1 0 0 0 f f f f f f i 2 0 2281 "2281 2277" _null_ _null_ _null_ _null_ array_agg_anyarray_transfn _null_ _null_ _null_ ));
+DESCR("aggregate transition function");
+DATA(insert OID = 6004 ( array_agg_anyarray_finalfn PGNSP PGUID 12 1 0 0 0 f f f f f f i 2 0 2277 "2281 2277" _null_ _null_ _null_ _null_ array_agg_anyarray_finalfn _null_ _null_ _null_ ));
+DESCR("aggregate final function");
+DATA(insert OID = 6005 ( array_agg PGNSP PGUID 12 1 0 0 0 t f f f f f i 1 0 2277 "2277" _null_ _null_ _null_ _null_ aggregate_dummy _null_ _null_ _null_ ));
DESCR("concatenate aggregate input into an array");
DATA(insert OID = 3218 ( width_bucket PGNSP PGUID 12 1 0 0 0 f f f f t f i 2 0 23 "2283 2277" _null_ _null_ _null_ _null_ width_bucket_array _null_ _null_ _null_ ));
DESCR("bucket number of operand given a sorted array of bucket lower bounds");
diff --git a/src/include/utils/array.h b/src/include/utils/array.h
index e744314..8dddd02 100644
--- a/src/include/utils/array.h
+++ b/src/include/utils/array.h
@@ -76,14 +76,15 @@ typedef struct
/*
* working state for accumArrayResult() and friends
+ *
+ * is_array_accum: whether accumulating array values.
+ * (if true must be casted to ArrayBuildStateArray, else
+ * cast to ArrayBuildStateScalar)
*/
typedef struct ArrayBuildState
{
+ bool is_array_accum;
MemoryContext mcontext; /* where all the temp stuff is kept */
- Datum *dvalues; /* array of accumulated Datums */
- bool *dnulls; /* array of is-null flags for Datums */
- int alen; /* allocated length of above arrays */
- int nelems; /* number of valid entries in above arrays */
Oid element_type; /* data type of the Datums */
int16 typlen; /* needed info about datatype */
bool typbyval;
@@ -91,6 +92,43 @@ typedef struct ArrayBuildState
} ArrayBuildState;
/*
+ * array build state for array accumulation of scalar datums
+ */
+typedef struct ArrayBuildStateScalar
+{
+ ArrayBuildState astate;
+
+ Datum *dvalues; /* array of accumulated Datums */
+ bool *dnulls; /* array of is-null flags for Datums */
+ int alen; /* allocated length of above arrays */
+ int nelems; /* number of valid entries in above arrays */
+} ArrayBuildStateScalar;
+
+
+/*
+ * array build state for array accumulation of array datums
+ */
+typedef struct
+{
+ ArrayBuildState astate;
+
+ char *data; /* array of accumulated data */
+ bits8 *nullbitmap; /* bitmap of is-null flags for data */
+
+ int abytes; /* allocated length of above arrays */
+ int aitems; /* allocated length of above arrays */
+ int nbytes; /* number of used bytes in above arrays */
+ int nitems; /* number of elements in above arrays */
+ int narray; /* number of array accumulated */
+
+ int ndims; /* element dimensions */
+ int *dims;
+ int *lbs;
+
+ bool hasnull; /* any element has null */
+} ArrayBuildStateArray;
+
+/*
* structure to cache type metadata needed for array manipulation
*/
typedef struct ArrayMetaState
@@ -260,7 +298,6 @@ extern Datum makeArrayResult(ArrayBuildState *astate,
MemoryContext rcontext);
extern Datum makeMdArrayResult(ArrayBuildState *astate, int ndims,
int *dims, int *lbs, MemoryContext rcontext, bool release);
-
extern ArrayIterator array_create_iterator(ArrayType *arr, int slice_ndim);
extern bool array_iterate(ArrayIterator iterator, Datum *value, bool *isnull);
extern void array_free_iterator(ArrayIterator iterator);
@@ -293,6 +330,9 @@ extern ArrayType *create_singleton_array(FunctionCallInfo fcinfo,
extern Datum array_agg_transfn(PG_FUNCTION_ARGS);
extern Datum array_agg_finalfn(PG_FUNCTION_ARGS);
+extern Datum array_agg_anyarray_transfn(PG_FUNCTION_ARGS);
+extern Datum array_agg_anyarray_finalfn(PG_FUNCTION_ARGS);
+
/*
* prototypes for functions defined in array_typanalyze.c
*/
diff --git a/src/test/regress/expected/aggregates.out b/src/test/regress/expected/aggregates.out
index 58df854..607aeea 100644
--- a/src/test/regress/expected/aggregates.out
+++ b/src/test/regress/expected/aggregates.out
@@ -914,6 +914,76 @@ select array_agg(distinct a order by a desc nulls last)
{3,2,1,NULL}
(1 row)
+-- array_agg(anyarray)
+select array_agg(ar)
+ from (values ('{1,2}'::int[]), ('{3,4}'::int[])) v(ar);
+ array_agg
+---------------
+ {{1,2},{3,4}}
+(1 row)
+
+select array_agg(distinct ar order by ar desc)
+ from (select array[i / 2] from generate_series(1,10) a(i)) b(ar);
+ array_agg
+---------------------------
+ {{5},{4},{3},{2},{1},{0}}
+(1 row)
+
+select array_agg(ar)
+ from (select array_agg(array[i, i+1, i-1])
+ from generate_series(1,2) a(i)) b(ar);
+ array_agg
+---------------------
+ {{{1,2,0},{2,3,1}}}
+(1 row)
+
+-- array_agg(anyarray), varlena types
+select array_agg(array[1.2,1.3,1.4]) from generate_series(1,3);
+ array_agg
+---------------------------------------------
+ {{1.2,1.3,1.4},{1.2,1.3,1.4},{1.2,1.3,1.4}}
+(1 row)
+
+select array_agg(array['Hello','Hohoho','Hi']) from generate_series(1,3);
+ array_agg
+---------------------------------------------------------
+ {{Hello,Hohoho,Hi},{Hello,Hohoho,Hi},{Hello,Hohoho,Hi}}
+(1 row)
+
+-- array_agg(anyarray), arrays with nulls
+select array_agg(array[i, null, i+1, null, i+2]) from generate_series(1,3) g(i);
+ array_agg
+---------------------------------------------------------
+ {{1,NULL,2,NULL,3},{2,NULL,3,NULL,4},{3,NULL,4,NULL,5}}
+(1 row)
+
+select array_agg(array[1.1+ i, null, 1.1+i+1, null, 1.1+i+2]) from generate_series(1,3) g(i);
+ array_agg
+---------------------------------------------------------------------------
+ {{2.1,NULL,3.1,NULL,4.1},{3.1,NULL,4.1,NULL,5.1},{4.1,NULL,5.1,NULL,6.1}}
+(1 row)
+
+select array_agg(array[null, 'Hello','Hohoho', null,'Hi']) from generate_series(1,3);
+ array_agg
+---------------------------------------------------------------------------------------
+ {{NULL,Hello,Hohoho,NULL,Hi},{NULL,Hello,Hohoho,NULL,Hi},{NULL,Hello,Hohoho,NULL,Hi}}
+(1 row)
+
+select array_agg(array[[null, 'Hello', null, 'Hi'],['Hello', null, 'Hi', null]]) from generate_series(1,2);
+ array_agg
+-------------------------------------------------------------------------------------------
+ {{{NULL,Hello,NULL,Hi},{Hello,NULL,Hi,NULL}},{{NULL,Hello,NULL,Hi},{Hello,NULL,Hi,NULL}}}
+(1 row)
+
+-- errors
+select array_agg('{}'::int[]) from generate_series(1,2);
+ERROR: cannot accumulate empty arrays
+select array_agg(null::int[]) from generate_series(1,2);
+ERROR: cannot accumulate null arrays
+select array_agg(ar)
+ from (values ('{1,2}'::int[]), ('{3}'::int[])) v(ar);
+ERROR: cannot accumulate incompatible arrays
+DETAIL: Arrays with differing element dimensions are not compatible for concatenation.
-- multi-arg aggs, strict/nonstrict, distinct/order by
select aggfstr(a,b,c)
from (values (1,3,'foo'),(0,null,null),(2,2,'bar'),(3,1,'baz')) v(a,b,c);
diff --git a/src/test/regress/expected/arrays.out b/src/test/regress/expected/arrays.out
index 46eff67..e80ebec 100644
--- a/src/test/regress/expected/arrays.out
+++ b/src/test/regress/expected/arrays.out
@@ -1521,6 +1521,23 @@ select array_agg(unique1) from tenk1 where unique1 < -15;
(1 row)
+select array(select unique1 from tenk1 where unique1 < 15 order by unique1);
+ array
+--------------------------------------
+ {0,1,2,3,4,5,6,7,8,9,10,11,12,13,14}
+(1 row)
+
+select array(select array[i,i/2] from generate_series(1,5) a(i));
+ array
+---------------------------------
+ {{1,0},{2,1},{3,1},{4,2},{5,2}}
+(1 row)
+
+-- cannot accumulate null arrays and empty arrays
+select array(select null::int[]);
+ERROR: cannot accumulate null arrays
+select array(select '{}'::int[]);
+ERROR: cannot accumulate empty arrays
select unnest(array[1,2,3]);
unnest
--------
diff --git a/src/test/regress/sql/aggregates.sql b/src/test/regress/sql/aggregates.sql
index 8096a6f..a70419f 100644
--- a/src/test/regress/sql/aggregates.sql
+++ b/src/test/regress/sql/aggregates.sql
@@ -322,6 +322,32 @@ select array_agg(distinct a order by a desc)
select array_agg(distinct a order by a desc nulls last)
from (values (1),(2),(1),(3),(null),(2)) v(a);
+-- array_agg(anyarray)
+select array_agg(ar)
+ from (values ('{1,2}'::int[]), ('{3,4}'::int[])) v(ar);
+select array_agg(distinct ar order by ar desc)
+ from (select array[i / 2] from generate_series(1,10) a(i)) b(ar);
+select array_agg(ar)
+ from (select array_agg(array[i, i+1, i-1])
+ from generate_series(1,2) a(i)) b(ar);
+
+-- array_agg(anyarray), varlena types
+select array_agg(array[1.2,1.3,1.4]) from generate_series(1,3);
+select array_agg(array['Hello','Hohoho','Hi']) from generate_series(1,3);
+
+-- array_agg(anyarray), arrays with nulls
+select array_agg(array[i, null, i+1, null, i+2]) from generate_series(1,3) g(i);
+select array_agg(array[1.1+ i, null, 1.1+i+1, null, 1.1+i+2]) from generate_series(1,3) g(i);
+select array_agg(array[null, 'Hello','Hohoho', null,'Hi']) from generate_series(1,3);
+
+select array_agg(array[[null, 'Hello', null, 'Hi'],['Hello', null, 'Hi', null]]) from generate_series(1,2);
+
+-- errors
+select array_agg('{}'::int[]) from generate_series(1,2);
+select array_agg(null::int[]) from generate_series(1,2);
+select array_agg(ar)
+ from (values ('{1,2}'::int[]), ('{3}'::int[])) v(ar);
+
-- multi-arg aggs, strict/nonstrict, distinct/order by
select aggfstr(a,b,c)
diff --git a/src/test/regress/sql/arrays.sql b/src/test/regress/sql/arrays.sql
index fa8a20a..cb00f5f 100644
--- a/src/test/regress/sql/arrays.sql
+++ b/src/test/regress/sql/arrays.sql
@@ -432,6 +432,12 @@ select array_agg(ten) from (select ten from tenk1 where unique1 < 15 order by un
select array_agg(nullif(ten, 4)) from (select ten from tenk1 where unique1 < 15 order by unique1) ss;
select array_agg(unique1) from tenk1 where unique1 < -15;
+select array(select unique1 from tenk1 where unique1 < 15 order by unique1);
+select array(select array[i,i/2] from generate_series(1,5) a(i));
+-- cannot accumulate null arrays and empty arrays
+select array(select null::int[]);
+select array(select '{}'::int[]);
+
select unnest(array[1,2,3]);
select * from unnest(array[1,2,3]);
select unnest(array[1,2,3,4.5]::float8[]);
Re: Function array_agg(array)
2014-10-25 10:29 GMT+07:00 Ali Akbar <the.apaan@gmail.com>:
I fixed small issue in regress tests and I enhanced tests for varlena
types and null values.
Thanks.
it is about 15% faster than original implementation.
15% faster than array_agg(scalar)? I haven't verify the performance, but
because the internal array data and null bitmap is copied as-is, that will
be faster.2014-10-25 1:51 GMT+07:00 Pavel Stehule <pavel.stehule@gmail.com>:
Hi Ali
I checked a code. I am thinking so code organization is not good.
accumArrayResult is too long now. makeMdArrayResult will not work, when
arrays was joined (it is not consistent now). I don't like a usage of
state->is_array_accum in array_userfunc.c -- it is signal of wrong
wrapping.Yes, i was thinking the same. Attached WIP patch to reorganizate the code.
makeMdArrayResult works now, with supplied arguments act as override from
default values calculated from ArrayBuildStateArray.In array_userfunc.c, because we don't want to override ndims, dims and
lbs, i copied array_agg_finalfn and only change the call to
makeMdArrayResult (we don't uses makeArrayResult because we want to set
release to false). Another alternative is to create new
makeArrayResult-like function that has parameter bool release.
adding makeArrayResult1 (do you have better name for this?), that accepts
bool release parameter. array_agg_finalfn becomes more clean, and no
duplicate code in array_agg_anyarray_finalfn.
next question: there is function array_append(anyarray, anyelement). Isn't
time to define array_append(anyarray, anyarray) now?
There is array_cat(anyarray, anyarray):
/*-----------------------------------------------------------------------------
* array_cat :
* concatenate two nD arrays to form an nD array, or
* push an (n-1)D array onto the end of an nD array*----------------------------------------------------------------------------
*/
Regards,
--
Ali Akbar
Attachments:
array_agg_anyarray-9.patchtext/x-patch; charset=US-ASCII; name=array_agg_anyarray-9.patchDownload
diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml
index 7e5bcd9..f59738a 100644
--- a/doc/src/sgml/func.sgml
+++ b/doc/src/sgml/func.sgml
@@ -12046,6 +12046,22 @@ NULL baz</literallayout>(3 rows)</entry>
<row>
<entry>
<indexterm>
+ <primary>array_agg</primary>
+ </indexterm>
+ <function>array_agg(<replaceable class="parameter">anyarray</replaceable>)</function>
+ </entry>
+ <entry>
+ any
+ </entry>
+ <entry>
+ the same array type as input type
+ </entry>
+ <entry>input arrays, aggregated into higher-order multidimesional array. Rejects NULL and empty array as input.</entry>
+ </row>
+
+ <row>
+ <entry>
+ <indexterm>
<primary>average</primary>
</indexterm>
<indexterm>
diff --git a/doc/src/sgml/syntax.sgml b/doc/src/sgml/syntax.sgml
index 2f0680f..8c182a4 100644
--- a/doc/src/sgml/syntax.sgml
+++ b/doc/src/sgml/syntax.sgml
@@ -2238,6 +2238,11 @@ SELECT ARRAY(SELECT oid FROM pg_proc WHERE proname LIKE 'bytea%');
array
-----------------------------------------------------------------------
{2011,1954,1948,1952,1951,1244,1950,2005,1949,1953,2006,31,2412,2413}
+
+SELECT ARRAY(SELECT array(select i) FROM generate_series(1,5) a(i));
+ array
+-----------------------
+ {{1},{2},{3},{4},{5}}
(1 row)
</programlisting>
The subquery must return a single column. The resulting
diff --git a/src/backend/nodes/nodeFuncs.c b/src/backend/nodes/nodeFuncs.c
index 41e973b..0261fcb 100644
--- a/src/backend/nodes/nodeFuncs.c
+++ b/src/backend/nodes/nodeFuncs.c
@@ -108,12 +108,16 @@ exprType(const Node *expr)
type = exprType((Node *) tent->expr);
if (sublink->subLinkType == ARRAY_SUBLINK)
{
- type = get_array_type(type);
- if (!OidIsValid(type))
- ereport(ERROR,
- (errcode(ERRCODE_UNDEFINED_OBJECT),
- errmsg("could not find array type for data type %s",
- format_type_be(exprType((Node *) tent->expr)))));
+ if (!OidIsValid(get_element_type(type)))
+ {
+ /* not array, so check for its array type */
+ type = get_array_type(type);
+ if (!OidIsValid(type))
+ ereport(ERROR,
+ (errcode(ERRCODE_UNDEFINED_OBJECT),
+ errmsg("could not find array type for data type %s",
+ format_type_be(exprType((Node *) tent->expr)))));
+ }
}
}
else if (sublink->subLinkType == MULTIEXPR_SUBLINK)
@@ -139,12 +143,16 @@ exprType(const Node *expr)
type = subplan->firstColType;
if (subplan->subLinkType == ARRAY_SUBLINK)
{
- type = get_array_type(type);
- if (!OidIsValid(type))
- ereport(ERROR,
- (errcode(ERRCODE_UNDEFINED_OBJECT),
- errmsg("could not find array type for data type %s",
- format_type_be(subplan->firstColType))));
+ if (!OidIsValid(get_element_type(type)))
+ {
+ /* not array, so check for its array type */
+ type = get_array_type(type);
+ if (!OidIsValid(type))
+ ereport(ERROR,
+ (errcode(ERRCODE_UNDEFINED_OBJECT),
+ errmsg("could not find array type for data type %s",
+ format_type_be(subplan->firstColType))));
+ }
}
}
else if (subplan->subLinkType == MULTIEXPR_SUBLINK)
diff --git a/src/backend/optimizer/plan/subselect.c b/src/backend/optimizer/plan/subselect.c
index 3e7dc85..8fc8b49 100644
--- a/src/backend/optimizer/plan/subselect.c
+++ b/src/backend/optimizer/plan/subselect.c
@@ -668,10 +668,16 @@ build_subplan(PlannerInfo *root, Plan *plan, PlannerInfo *subroot,
Assert(!te->resjunk);
Assert(testexpr == NULL);
- arraytype = get_array_type(exprType((Node *) te->expr));
- if (!OidIsValid(arraytype))
- elog(ERROR, "could not find array type for datatype %s",
- format_type_be(exprType((Node *) te->expr)));
+
+ arraytype = exprType((Node *) te->expr);
+ if (!OidIsValid(get_element_type(arraytype)))
+ {
+ /* not array, so get the array type */
+ arraytype = get_array_type(exprType((Node *) te->expr));
+ if (!OidIsValid(arraytype))
+ elog(ERROR, "could not find array type for datatype %s",
+ format_type_be(exprType((Node *) te->expr)));
+ }
prm = generate_new_param(root,
arraytype,
exprTypmod((Node *) te->expr),
diff --git a/src/backend/utils/adt/array_userfuncs.c b/src/backend/utils/adt/array_userfuncs.c
index 831466d..b55f3f0 100644
--- a/src/backend/utils/adt/array_userfuncs.c
+++ b/src/backend/utils/adt/array_userfuncs.c
@@ -16,7 +16,6 @@
#include "utils/builtins.h"
#include "utils/lsyscache.h"
-
/*-----------------------------------------------------------------------------
* array_push :
* push an element onto either end of a one-dimensional array
@@ -513,8 +512,6 @@ array_agg_finalfn(PG_FUNCTION_ARGS)
{
Datum result;
ArrayBuildState *state;
- int dims[1];
- int lbs[1];
/*
* Test for null before Asserting we are in right context. This is to
@@ -529,18 +526,28 @@ array_agg_finalfn(PG_FUNCTION_ARGS)
state = (ArrayBuildState *) PG_GETARG_POINTER(0);
- dims[0] = state->nelems;
- lbs[0] = 1;
-
/*
* Make the result. We cannot release the ArrayBuildState because
* sometimes aggregate final functions are re-executed. Rather, it is
* nodeAgg.c's responsibility to reset the aggcontext when it's safe to do
* so.
*/
- result = makeMdArrayResult(state, 1, dims, lbs,
- CurrentMemoryContext,
- false);
+ result = makeArrayResult1(state, CurrentMemoryContext, false);
PG_RETURN_DATUM(result);
}
+
+/*
+ * ARRAY_AGG(anyarray) aggregate function
+ */
+Datum
+array_agg_anyarray_transfn(PG_FUNCTION_ARGS)
+{
+ return array_agg_transfn(fcinfo);
+}
+
+Datum
+array_agg_anyarray_finalfn(PG_FUNCTION_ARGS)
+{
+ return array_agg_finalfn(fcinfo);
+}
diff --git a/src/backend/utils/adt/arrayfuncs.c b/src/backend/utils/adt/arrayfuncs.c
index 6c8b41d..df2a2d6 100644
--- a/src/backend/utils/adt/arrayfuncs.c
+++ b/src/backend/utils/adt/arrayfuncs.c
@@ -145,6 +145,19 @@ static int width_bucket_array_variable(Datum operand,
Oid collation,
TypeCacheEntry *typentry);
+/* internal functions of accumArrayResult */
+static ArrayBuildStateScalar *initializeArrayBuildStateScalar(Oid element_type,
+ MemoryContext arr_context);
+static ArrayBuildStateArray *initializeArrayBuildStateArray(Oid subelement_type,
+ MemoryContext arr_context);
+static void appendScalarDatum(ArrayBuildStateScalar *astate_scalar,
+ Datum dvalue, bool disnull);
+static void appendArrayDatum(ArrayBuildStateArray *astate_array,
+ Datum dvalue, bool disnull);
+
+/* internal function of makeMdArrayResult */
+static ArrayType *makeMdArrayResultArray(ArrayBuildStateArray *astate,
+ int ndims, int *dims, int *lbs);
/*
* array_in :
@@ -4588,9 +4601,13 @@ accumArrayResult(ArrayBuildState *astate,
MemoryContext arr_context,
oldcontext;
+ ArrayBuildStateScalar *astate_scalar = NULL; /* for scalar datum accumulation */
+ ArrayBuildStateArray *astate_array = NULL; /* for array datum accumulation */
+
if (astate == NULL)
{
/* First time through --- initialize */
+ Oid subelement_type = get_element_type(element_type);
/* Make a temporary context to hold all the junk */
arr_context = AllocSetContextCreate(rcontext,
@@ -4599,33 +4616,127 @@ accumArrayResult(ArrayBuildState *astate,
ALLOCSET_DEFAULT_INITSIZE,
ALLOCSET_DEFAULT_MAXSIZE);
oldcontext = MemoryContextSwitchTo(arr_context);
- astate = (ArrayBuildState *) palloc(sizeof(ArrayBuildState));
- astate->mcontext = arr_context;
- astate->alen = 64; /* arbitrary starting array size */
- astate->dvalues = (Datum *) palloc(astate->alen * sizeof(Datum));
- astate->dnulls = (bool *) palloc(astate->alen * sizeof(bool));
- astate->nelems = 0;
- astate->element_type = element_type;
- get_typlenbyvalalign(element_type,
- &astate->typlen,
- &astate->typbyval,
- &astate->typalign);
+
+ if (!OidIsValid(subelement_type))
+ {
+ astate_scalar = initializeArrayBuildStateScalar(element_type, arr_context);
+ astate = (ArrayBuildState *) astate_scalar;
+
+ appendScalarDatum(astate_scalar, dvalue, disnull);
+ }
+ else
+ {
+ astate_array = initializeArrayBuildStateArray(subelement_type, arr_context);
+ astate = (ArrayBuildState *) astate_array;
+
+ appendArrayDatum(astate_array, dvalue, disnull);
+ }
}
else
{
oldcontext = MemoryContextSwitchTo(astate->mcontext);
- Assert(astate->element_type == element_type);
- /* enlarge dvalues[]/dnulls[] if needed */
- if (astate->nelems >= astate->alen)
+ if (!astate->is_array_accum)
{
- astate->alen *= 2;
- astate->dvalues = (Datum *)
- repalloc(astate->dvalues, astate->alen * sizeof(Datum));
- astate->dnulls = (bool *)
- repalloc(astate->dnulls, astate->alen * sizeof(bool));
+ Assert(astate->element_type == element_type);
+ astate_scalar = (ArrayBuildStateScalar *) astate;
+
+ /* enlarge dvalues[]/dnulls[] if needed */
+ if (astate_scalar->nelems >= astate_scalar->alen)
+ {
+ astate_scalar->alen *= 2;
+ astate_scalar->dvalues = (Datum *)
+ repalloc(astate_scalar->dvalues, astate_scalar->alen * sizeof(Datum));
+ astate_scalar->dnulls = (bool *)
+ repalloc(astate_scalar->dnulls, astate_scalar->alen * sizeof(bool));
+ }
+
+ appendScalarDatum(astate_scalar, dvalue, disnull);
+ }
+ else
+ {
+ Assert(astate->element_type == get_element_type(element_type));
+ astate_array = (ArrayBuildStateArray *) astate;
+ /*
+ * alloc & realloc placed in appendArrayDatum because
+ * we need to know the dvalue length first
+ */
+ appendArrayDatum(astate_array, dvalue, disnull);
}
}
+ MemoryContextSwitchTo(oldcontext);
+
+ return astate;
+}
+
+/*
+ * Initialize ArrayBuildState for scalar accumulation (internal function of
+ * accumArrayResult)
+ */
+ArrayBuildStateScalar *
+initializeArrayBuildStateScalar(Oid element_type,
+ MemoryContext arr_context)
+{
+ ArrayBuildStateScalar *astate_scalar;
+
+ /* scalar accumulate */
+ astate_scalar = (ArrayBuildStateScalar *) palloc(sizeof(ArrayBuildStateScalar));
+ astate_scalar->astate.is_array_accum = false;
+ astate_scalar->astate.mcontext = arr_context;
+
+ astate_scalar->alen = 64; /* arbitrary starting array size */
+ astate_scalar->dvalues = (Datum *) palloc(astate_scalar->alen * sizeof(Datum));
+ astate_scalar->dnulls = (bool *) palloc(astate_scalar->alen * sizeof(bool));
+ astate_scalar->nelems = 0;
+
+ astate_scalar->astate.element_type = element_type;
+ get_typlenbyvalalign(element_type,
+ &astate_scalar->astate.typlen,
+ &astate_scalar->astate.typbyval,
+ &astate_scalar->astate.typalign);
+
+ return astate_scalar;
+}
+
+/*
+ * Initialize ArrayBuildState for array accum (internal of accumArrayResult)
+ */
+ArrayBuildStateArray *
+initializeArrayBuildStateArray(Oid subelement_type,
+ MemoryContext arr_context)
+{
+ ArrayBuildStateArray *astate_array;
+
+ /* array accumulate */
+ astate_array = (ArrayBuildStateArray *) palloc(sizeof(ArrayBuildStateArray));
+ astate_array->astate.is_array_accum = true;
+ astate_array->astate.mcontext = arr_context;
+
+ astate_array->abytes = 0;
+ astate_array->aitems = 0;
+ astate_array->data = NULL;
+ astate_array->nbytes = 0;
+ astate_array->hasnull = false;
+ astate_array->nullbitmap = NULL;
+ astate_array->nitems = 0;
+ astate_array->narray = 0;
+
+ astate_array->astate.element_type = subelement_type;
+ get_typlenbyvalalign(subelement_type,
+ &astate_array->astate.typlen,
+ &astate_array->astate.typbyval,
+ &astate_array->astate.typalign);
+
+ return astate_array;
+}
+
+/*
+ * Append a datum for scalar accumulation (internal of accumArrayResult)
+ */
+void
+appendScalarDatum(ArrayBuildStateScalar *astate_scalar,
+ Datum dvalue, bool disnull)
+{
/*
* Ensure pass-by-ref stuff is copied into mcontext; and detoast it too if
* it's varlena. (You might think that detoasting is not needed here
@@ -4634,25 +4745,129 @@ accumArrayResult(ArrayBuildState *astate,
* because that would mean array_agg_finalfn damages its input, which is
* verboten. Also, this way frequently saves one copying step.)
*/
- if (!disnull && !astate->typbyval)
+ if (!disnull && !astate_scalar->astate.typbyval)
{
- if (astate->typlen == -1)
+ if (astate_scalar->astate.typlen == -1)
dvalue = PointerGetDatum(PG_DETOAST_DATUM_COPY(dvalue));
else
- dvalue = datumCopy(dvalue, astate->typbyval, astate->typlen);
+ dvalue = datumCopy(dvalue, astate_scalar->astate.typbyval,
+ astate_scalar->astate.typlen);
}
- astate->dvalues[astate->nelems] = dvalue;
- astate->dnulls[astate->nelems] = disnull;
- astate->nelems++;
+ astate_scalar->dvalues[astate_scalar->nelems] = dvalue;
+ astate_scalar->dnulls[astate_scalar->nelems] = disnull;
+ astate_scalar->nelems++;
+}
- MemoryContextSwitchTo(oldcontext);
+/*
+ * Append a datum for array accum (internal of accumArrayResult)
+ */
+void
+appendArrayDatum(ArrayBuildStateArray *astate_array,
+ Datum dvalue, bool disnull)
+{
+ ArrayType *arg;
+ int *dims,
+ *lbs,
+ ndims,
+ nitems,
+ ndatabytes;
+ char *data;
+ int i;
- return astate;
+ if (disnull)
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("cannot accumulate null arrays")));
+
+ arg = DatumGetArrayTypeP(dvalue);
+
+ ndims = ARR_NDIM(arg);
+ dims = ARR_DIMS(arg);
+ lbs = ARR_LBOUND(arg);
+ data = ARR_DATA_PTR(arg);
+ nitems = ArrayGetNItems(ndims, dims);
+ ndatabytes = ARR_SIZE(arg) - ARR_DATA_OFFSET(arg);
+
+ if (astate_array->data == NULL)
+ {
+ /* first allocation */
+ if (ndims == 0)
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("cannot accumulate empty arrays")));
+
+ if (ndims + 1 > MAXDIM)
+ ereport(ERROR,
+ (errcode(ERRCODE_PROGRAM_LIMIT_EXCEEDED),
+ errmsg("number of array dimensions (%d) exceeds the maximum allowed (%d)",
+ ndims + 1, MAXDIM)));
+
+ astate_array->ndims = ndims;
+ astate_array->dims = (int *) palloc(ndims * sizeof(int));
+ astate_array->lbs = (int *) palloc(ndims * sizeof(int));
+ memcpy(astate_array->dims, dims, ndims * sizeof(int));
+ memcpy(astate_array->lbs, lbs, ndims * sizeof(int));
+
+ astate_array->abytes = ndatabytes >= 512 ? 4 * ndatabytes : 1024;
+ astate_array->data = (char *) palloc(astate_array->abytes);
+ astate_array->aitems = 4 * nitems;
+ }
+ else
+ {
+ if (astate_array->ndims != ndims)
+ ereport(ERROR,
+ (errcode(ERRCODE_ARRAY_SUBSCRIPT_ERROR),
+ errmsg("cannot accumulate incompatible arrays"),
+ errdetail("Arrays of %d and %d dimensions are not "
+ "compatible for concatenation.",
+ astate_array->ndims, ndims)));
+
+ for (i = 0; i < ndims; i++)
+ if (astate_array->dims[i] != dims[i] || astate_array->lbs[i] != lbs[i])
+ ereport(ERROR,
+ (errcode(ERRCODE_ARRAY_SUBSCRIPT_ERROR),
+ errmsg("cannot accumulate incompatible arrays"),
+ errdetail("Arrays with differing element dimensions are "
+ "not compatible for concatenation.")));
+
+ if (astate_array->nbytes + ndatabytes >= astate_array->abytes)
+ {
+ astate_array->abytes *= 2;
+ astate_array->data = (char *)
+ repalloc(astate_array->data, astate_array->abytes);
+ }
+ if (astate_array->nitems + nitems >= astate_array->aitems)
+ {
+ astate_array->aitems *= 2;
+ astate_array->nullbitmap = (bits8 *)
+ repalloc(astate_array->nullbitmap, (astate_array->aitems + 7) / 8);
+ }
+ }
+ memcpy(astate_array->data + astate_array->nbytes, data, ndatabytes);
+ astate_array->nbytes += ndatabytes;
+
+ if (astate_array->hasnull || ARR_HASNULL(arg))
+ {
+ if (!astate_array->hasnull)
+ {
+ astate_array->hasnull = true;
+ astate_array->nullbitmap = (bits8 *) palloc((astate_array->aitems + 7) / 8);
+ array_bitmap_copy(astate_array->nullbitmap, 0,
+ NULL, 0,
+ astate_array->nitems);
+ }
+ array_bitmap_copy(astate_array->nullbitmap, astate_array->nitems,
+ ARR_NULLBITMAP(arg), 0,
+ nitems);
+ astate_array->nitems += nitems;
+ }
+ astate_array->narray += 1;
}
/*
- * makeArrayResult - produce 1-D final result of accumArrayResult
+ * makeArrayResult - produce 1-D final result of scalar accumArrayResult
+ * - produce N+1-D final result of array accumArrayResult
*
* astate is working state (not NULL)
* rcontext is where to construct result
@@ -4661,17 +4876,36 @@ Datum
makeArrayResult(ArrayBuildState *astate,
MemoryContext rcontext)
{
- int dims[1];
- int lbs[1];
+ return makeArrayResult1(astate, rcontext, true);
+}
- dims[0] = astate->nelems;
- lbs[0] = 1;
+/*
+ * makeArrayResult1 - same as makeArrayResult, with option not to
+ * release working state
+ *
+ * release is true if okay to release working state
+ */
+Datum
+makeArrayResult1(ArrayBuildState *astate,
+ MemoryContext rcontext,
+ bool release)
+{
+ if (!astate->is_array_accum)
+ {
+ int dims[1];
+ int lbs[1];
+
+ dims[0] = ((ArrayBuildStateScalar *) astate)->nelems;
+ lbs[0] = 1;
- return makeMdArrayResult(astate, 1, dims, lbs, rcontext, true);
+ return makeMdArrayResult(astate, 1, dims, lbs, rcontext, release);
+ }
+ else
+ return makeMdArrayResult(astate, -1, NULL, NULL, rcontext, release);
}
/*
- * makeMdArrayResult - produce multi-D final result of accumArrayResult
+ * makeMdArrayResult - produce multi-D final result of scalar accumArrayResult
*
* beware: no check that specified dimensions match the number of values
* accumulated.
@@ -4679,6 +4913,11 @@ makeArrayResult(ArrayBuildState *astate,
* astate is working state (not NULL)
* rcontext is where to construct result
* release is true if okay to release working state
+ *
+ * for array accumulation:
+ * ndims == -1: default (calculated from astate)
+ * dims == NULL: default dims (calculated from astate)
+ * lbs == NULL: default (calculated from astate)
*/
Datum
makeMdArrayResult(ArrayBuildState *astate,
@@ -4694,8 +4933,13 @@ makeMdArrayResult(ArrayBuildState *astate,
/* Build the final array result in rcontext */
oldcontext = MemoryContextSwitchTo(rcontext);
- result = construct_md_array(astate->dvalues,
- astate->dnulls,
+ if (!astate->is_array_accum)
+ {
+ ArrayBuildStateScalar *astate_scalar;
+ astate_scalar = (ArrayBuildStateScalar *) astate;
+
+ result = construct_md_array(astate_scalar->dvalues,
+ astate_scalar->dnulls,
ndims,
dims,
lbs,
@@ -4703,6 +4947,16 @@ makeMdArrayResult(ArrayBuildState *astate,
astate->typlen,
astate->typbyval,
astate->typalign);
+ }
+ else
+ {
+ ArrayBuildStateArray *astate_array;
+ astate_array = (ArrayBuildStateArray *) astate;
+ result = makeMdArrayResultArray(astate_array,
+ ndims,
+ dims,
+ lbs);
+ }
MemoryContextSwitchTo(oldcontext);
@@ -4713,6 +4967,75 @@ makeMdArrayResult(ArrayBuildState *astate,
return PointerGetDatum(result);
}
+/*
+ * makeMdArrayResultArray - internal of makeArrayResult & makeMdArrayResult
+ *
+ * beware: no check that specified dimensions match the values
+ * accumulated.
+ *
+ * astate is working state (not NULL)
+ * ndims > -1 if wants to override default (calculated from astate)
+ * dims not NULL if wants to override default
+ * lbs not NULL if wants to override default
+ */
+ArrayType *
+makeMdArrayResultArray(ArrayBuildStateArray *astate,
+ int ndims,
+ int *dims,
+ int *lbs)
+{
+ ArrayType *result;
+
+ int dataoffset,
+ nbytes;
+
+ if (ndims == -1)
+ ndims = astate->ndims + 1;
+
+ nbytes = astate->nbytes;
+ /* compute required space */
+ if (astate->hasnull)
+ {
+ dataoffset = ARR_OVERHEAD_WITHNULLS(ndims, astate->nitems);
+ nbytes += dataoffset;
+ }
+ else
+ {
+ dataoffset = 0;
+ nbytes += ARR_OVERHEAD_NONULLS(ndims);
+ }
+
+ result = (ArrayType *) palloc0(nbytes);
+ SET_VARSIZE(result, nbytes);
+ result->ndim = astate->ndims + 1;
+ result->dataoffset = dataoffset;
+ result->elemtype = astate->astate.element_type;
+
+ if (dims != NULL)
+ memcpy(ARR_DIMS(result), dims, ndims * sizeof(int));
+ else
+ {
+ ARR_DIMS(result)[0] = astate->narray;
+ memcpy(&(ARR_DIMS(result)[1]), astate->dims, (ndims - 1) * sizeof(int));
+ }
+ if (lbs != NULL)
+ memcpy(ARR_LBOUND(result), lbs, ndims * sizeof(int));
+ else
+ {
+ ARR_LBOUND(result)[0] = 1;
+ memcpy(&(ARR_LBOUND(result)[1]), astate->lbs, (ndims - 1) * sizeof(int));
+ }
+
+ memcpy(ARR_DATA_PTR(result), astate->data, astate->nbytes);
+
+ if (astate->hasnull)
+ array_bitmap_copy(ARR_NULLBITMAP(result), 0,
+ astate->nullbitmap, 0,
+ astate->nitems);
+
+ return result;
+}
+
Datum
array_larger(PG_FUNCTION_ARGS)
{
diff --git a/src/include/catalog/pg_aggregate.h b/src/include/catalog/pg_aggregate.h
index 3ba9e5e..2005199 100644
--- a/src/include/catalog/pg_aggregate.h
+++ b/src/include/catalog/pg_aggregate.h
@@ -275,6 +275,7 @@ DATA(insert ( 2901 n 0 xmlconcat2 - - - - f f 0 142 0 0 0 _null_
/* array */
DATA(insert ( 2335 n 0 array_agg_transfn array_agg_finalfn - - - t f 0 2281 0 0 0 _null_ _null_ ));
+DATA(insert ( 6005 n 0 array_agg_anyarray_transfn array_agg_anyarray_finalfn - - - t f 0 2281 0 0 0 _null_ _null_ ));
/* text */
DATA(insert ( 3538 n 0 string_agg_transfn string_agg_finalfn - - - f f 0 2281 0 0 0 _null_ _null_ ));
diff --git a/src/include/catalog/pg_proc.h b/src/include/catalog/pg_proc.h
index b6dc1b8..9273c1f 100644
--- a/src/include/catalog/pg_proc.h
+++ b/src/include/catalog/pg_proc.h
@@ -879,11 +879,17 @@ DATA(insert OID = 3167 ( array_remove PGNSP PGUID 12 1 0 0 0 f f f f f f i 2
DESCR("remove any occurrences of an element from an array");
DATA(insert OID = 3168 ( array_replace PGNSP PGUID 12 1 0 0 0 f f f f f f i 3 0 2277 "2277 2283 2283" _null_ _null_ _null_ _null_ array_replace _null_ _null_ _null_ ));
DESCR("replace any occurrences of an element in an array");
-DATA(insert OID = 2333 ( array_agg_transfn PGNSP PGUID 12 1 0 0 0 f f f f f f i 2 0 2281 "2281 2283" _null_ _null_ _null_ _null_ array_agg_transfn _null_ _null_ _null_ ));
+DATA(insert OID = 2333 ( array_agg_transfn PGNSP PGUID 12 1 0 0 0 f f f f f f i 2 0 2281 "2281 2776" _null_ _null_ _null_ _null_ array_agg_transfn _null_ _null_ _null_ ));
DESCR("aggregate transition function");
-DATA(insert OID = 2334 ( array_agg_finalfn PGNSP PGUID 12 1 0 0 0 f f f f f f i 2 0 2277 "2281 2283" _null_ _null_ _null_ _null_ array_agg_finalfn _null_ _null_ _null_ ));
+DATA(insert OID = 2334 ( array_agg_finalfn PGNSP PGUID 12 1 0 0 0 f f f f f f i 2 0 2277 "2281 2776" _null_ _null_ _null_ _null_ array_agg_finalfn _null_ _null_ _null_ ));
DESCR("aggregate final function");
-DATA(insert OID = 2335 ( array_agg PGNSP PGUID 12 1 0 0 0 t f f f f f i 1 0 2277 "2283" _null_ _null_ _null_ _null_ aggregate_dummy _null_ _null_ _null_ ));
+DATA(insert OID = 2335 ( array_agg PGNSP PGUID 12 1 0 0 0 t f f f f f i 1 0 2277 "2776" _null_ _null_ _null_ _null_ aggregate_dummy _null_ _null_ _null_ ));
+DESCR("concatenate aggregate input into an array");
+DATA(insert OID = 6003 ( array_agg_anyarray_transfn PGNSP PGUID 12 1 0 0 0 f f f f f f i 2 0 2281 "2281 2277" _null_ _null_ _null_ _null_ array_agg_anyarray_transfn _null_ _null_ _null_ ));
+DESCR("aggregate transition function");
+DATA(insert OID = 6004 ( array_agg_anyarray_finalfn PGNSP PGUID 12 1 0 0 0 f f f f f f i 2 0 2277 "2281 2277" _null_ _null_ _null_ _null_ array_agg_anyarray_finalfn _null_ _null_ _null_ ));
+DESCR("aggregate final function");
+DATA(insert OID = 6005 ( array_agg PGNSP PGUID 12 1 0 0 0 t f f f f f i 1 0 2277 "2277" _null_ _null_ _null_ _null_ aggregate_dummy _null_ _null_ _null_ ));
DESCR("concatenate aggregate input into an array");
DATA(insert OID = 3218 ( width_bucket PGNSP PGUID 12 1 0 0 0 f f f f t f i 2 0 23 "2283 2277" _null_ _null_ _null_ _null_ width_bucket_array _null_ _null_ _null_ ));
DESCR("bucket number of operand given a sorted array of bucket lower bounds");
diff --git a/src/include/utils/array.h b/src/include/utils/array.h
index e744314..daaae88 100644
--- a/src/include/utils/array.h
+++ b/src/include/utils/array.h
@@ -76,14 +76,15 @@ typedef struct
/*
* working state for accumArrayResult() and friends
+ *
+ * is_array_accum: whether accumulating array values.
+ * (if true must be casted to ArrayBuildStateArray, else
+ * cast to ArrayBuildStateScalar)
*/
typedef struct ArrayBuildState
{
+ bool is_array_accum;
MemoryContext mcontext; /* where all the temp stuff is kept */
- Datum *dvalues; /* array of accumulated Datums */
- bool *dnulls; /* array of is-null flags for Datums */
- int alen; /* allocated length of above arrays */
- int nelems; /* number of valid entries in above arrays */
Oid element_type; /* data type of the Datums */
int16 typlen; /* needed info about datatype */
bool typbyval;
@@ -91,6 +92,43 @@ typedef struct ArrayBuildState
} ArrayBuildState;
/*
+ * array build state for array accumulation of scalar datums
+ */
+typedef struct ArrayBuildStateScalar
+{
+ ArrayBuildState astate;
+
+ Datum *dvalues; /* array of accumulated Datums */
+ bool *dnulls; /* array of is-null flags for Datums */
+ int alen; /* allocated length of above arrays */
+ int nelems; /* number of valid entries in above arrays */
+} ArrayBuildStateScalar;
+
+
+/*
+ * array build state for array accumulation of array datums
+ */
+typedef struct
+{
+ ArrayBuildState astate;
+
+ char *data; /* array of accumulated data */
+ bits8 *nullbitmap; /* bitmap of is-null flags for data */
+
+ int abytes; /* allocated length of above arrays */
+ int aitems; /* allocated length of above arrays */
+ int nbytes; /* number of used bytes in above arrays */
+ int nitems; /* number of elements in above arrays */
+ int narray; /* number of array accumulated */
+
+ int ndims; /* element dimensions */
+ int *dims;
+ int *lbs;
+
+ bool hasnull; /* any element has null */
+} ArrayBuildStateArray;
+
+/*
* structure to cache type metadata needed for array manipulation
*/
typedef struct ArrayMetaState
@@ -258,9 +296,10 @@ extern ArrayBuildState *accumArrayResult(ArrayBuildState *astate,
MemoryContext rcontext);
extern Datum makeArrayResult(ArrayBuildState *astate,
MemoryContext rcontext);
+extern Datum makeArrayResult1(ArrayBuildState *astate,
+ MemoryContext rcontext, bool release);
extern Datum makeMdArrayResult(ArrayBuildState *astate, int ndims,
int *dims, int *lbs, MemoryContext rcontext, bool release);
-
extern ArrayIterator array_create_iterator(ArrayType *arr, int slice_ndim);
extern bool array_iterate(ArrayIterator iterator, Datum *value, bool *isnull);
extern void array_free_iterator(ArrayIterator iterator);
@@ -293,6 +332,9 @@ extern ArrayType *create_singleton_array(FunctionCallInfo fcinfo,
extern Datum array_agg_transfn(PG_FUNCTION_ARGS);
extern Datum array_agg_finalfn(PG_FUNCTION_ARGS);
+extern Datum array_agg_anyarray_transfn(PG_FUNCTION_ARGS);
+extern Datum array_agg_anyarray_finalfn(PG_FUNCTION_ARGS);
+
/*
* prototypes for functions defined in array_typanalyze.c
*/
diff --git a/src/test/regress/expected/aggregates.out b/src/test/regress/expected/aggregates.out
index 58df854..607aeea 100644
--- a/src/test/regress/expected/aggregates.out
+++ b/src/test/regress/expected/aggregates.out
@@ -914,6 +914,76 @@ select array_agg(distinct a order by a desc nulls last)
{3,2,1,NULL}
(1 row)
+-- array_agg(anyarray)
+select array_agg(ar)
+ from (values ('{1,2}'::int[]), ('{3,4}'::int[])) v(ar);
+ array_agg
+---------------
+ {{1,2},{3,4}}
+(1 row)
+
+select array_agg(distinct ar order by ar desc)
+ from (select array[i / 2] from generate_series(1,10) a(i)) b(ar);
+ array_agg
+---------------------------
+ {{5},{4},{3},{2},{1},{0}}
+(1 row)
+
+select array_agg(ar)
+ from (select array_agg(array[i, i+1, i-1])
+ from generate_series(1,2) a(i)) b(ar);
+ array_agg
+---------------------
+ {{{1,2,0},{2,3,1}}}
+(1 row)
+
+-- array_agg(anyarray), varlena types
+select array_agg(array[1.2,1.3,1.4]) from generate_series(1,3);
+ array_agg
+---------------------------------------------
+ {{1.2,1.3,1.4},{1.2,1.3,1.4},{1.2,1.3,1.4}}
+(1 row)
+
+select array_agg(array['Hello','Hohoho','Hi']) from generate_series(1,3);
+ array_agg
+---------------------------------------------------------
+ {{Hello,Hohoho,Hi},{Hello,Hohoho,Hi},{Hello,Hohoho,Hi}}
+(1 row)
+
+-- array_agg(anyarray), arrays with nulls
+select array_agg(array[i, null, i+1, null, i+2]) from generate_series(1,3) g(i);
+ array_agg
+---------------------------------------------------------
+ {{1,NULL,2,NULL,3},{2,NULL,3,NULL,4},{3,NULL,4,NULL,5}}
+(1 row)
+
+select array_agg(array[1.1+ i, null, 1.1+i+1, null, 1.1+i+2]) from generate_series(1,3) g(i);
+ array_agg
+---------------------------------------------------------------------------
+ {{2.1,NULL,3.1,NULL,4.1},{3.1,NULL,4.1,NULL,5.1},{4.1,NULL,5.1,NULL,6.1}}
+(1 row)
+
+select array_agg(array[null, 'Hello','Hohoho', null,'Hi']) from generate_series(1,3);
+ array_agg
+---------------------------------------------------------------------------------------
+ {{NULL,Hello,Hohoho,NULL,Hi},{NULL,Hello,Hohoho,NULL,Hi},{NULL,Hello,Hohoho,NULL,Hi}}
+(1 row)
+
+select array_agg(array[[null, 'Hello', null, 'Hi'],['Hello', null, 'Hi', null]]) from generate_series(1,2);
+ array_agg
+-------------------------------------------------------------------------------------------
+ {{{NULL,Hello,NULL,Hi},{Hello,NULL,Hi,NULL}},{{NULL,Hello,NULL,Hi},{Hello,NULL,Hi,NULL}}}
+(1 row)
+
+-- errors
+select array_agg('{}'::int[]) from generate_series(1,2);
+ERROR: cannot accumulate empty arrays
+select array_agg(null::int[]) from generate_series(1,2);
+ERROR: cannot accumulate null arrays
+select array_agg(ar)
+ from (values ('{1,2}'::int[]), ('{3}'::int[])) v(ar);
+ERROR: cannot accumulate incompatible arrays
+DETAIL: Arrays with differing element dimensions are not compatible for concatenation.
-- multi-arg aggs, strict/nonstrict, distinct/order by
select aggfstr(a,b,c)
from (values (1,3,'foo'),(0,null,null),(2,2,'bar'),(3,1,'baz')) v(a,b,c);
diff --git a/src/test/regress/expected/arrays.out b/src/test/regress/expected/arrays.out
index 46eff67..e80ebec 100644
--- a/src/test/regress/expected/arrays.out
+++ b/src/test/regress/expected/arrays.out
@@ -1521,6 +1521,23 @@ select array_agg(unique1) from tenk1 where unique1 < -15;
(1 row)
+select array(select unique1 from tenk1 where unique1 < 15 order by unique1);
+ array
+--------------------------------------
+ {0,1,2,3,4,5,6,7,8,9,10,11,12,13,14}
+(1 row)
+
+select array(select array[i,i/2] from generate_series(1,5) a(i));
+ array
+---------------------------------
+ {{1,0},{2,1},{3,1},{4,2},{5,2}}
+(1 row)
+
+-- cannot accumulate null arrays and empty arrays
+select array(select null::int[]);
+ERROR: cannot accumulate null arrays
+select array(select '{}'::int[]);
+ERROR: cannot accumulate empty arrays
select unnest(array[1,2,3]);
unnest
--------
diff --git a/src/test/regress/sql/aggregates.sql b/src/test/regress/sql/aggregates.sql
index 8096a6f..a70419f 100644
--- a/src/test/regress/sql/aggregates.sql
+++ b/src/test/regress/sql/aggregates.sql
@@ -322,6 +322,32 @@ select array_agg(distinct a order by a desc)
select array_agg(distinct a order by a desc nulls last)
from (values (1),(2),(1),(3),(null),(2)) v(a);
+-- array_agg(anyarray)
+select array_agg(ar)
+ from (values ('{1,2}'::int[]), ('{3,4}'::int[])) v(ar);
+select array_agg(distinct ar order by ar desc)
+ from (select array[i / 2] from generate_series(1,10) a(i)) b(ar);
+select array_agg(ar)
+ from (select array_agg(array[i, i+1, i-1])
+ from generate_series(1,2) a(i)) b(ar);
+
+-- array_agg(anyarray), varlena types
+select array_agg(array[1.2,1.3,1.4]) from generate_series(1,3);
+select array_agg(array['Hello','Hohoho','Hi']) from generate_series(1,3);
+
+-- array_agg(anyarray), arrays with nulls
+select array_agg(array[i, null, i+1, null, i+2]) from generate_series(1,3) g(i);
+select array_agg(array[1.1+ i, null, 1.1+i+1, null, 1.1+i+2]) from generate_series(1,3) g(i);
+select array_agg(array[null, 'Hello','Hohoho', null,'Hi']) from generate_series(1,3);
+
+select array_agg(array[[null, 'Hello', null, 'Hi'],['Hello', null, 'Hi', null]]) from generate_series(1,2);
+
+-- errors
+select array_agg('{}'::int[]) from generate_series(1,2);
+select array_agg(null::int[]) from generate_series(1,2);
+select array_agg(ar)
+ from (values ('{1,2}'::int[]), ('{3}'::int[])) v(ar);
+
-- multi-arg aggs, strict/nonstrict, distinct/order by
select aggfstr(a,b,c)
diff --git a/src/test/regress/sql/arrays.sql b/src/test/regress/sql/arrays.sql
index fa8a20a..cb00f5f 100644
--- a/src/test/regress/sql/arrays.sql
+++ b/src/test/regress/sql/arrays.sql
@@ -432,6 +432,12 @@ select array_agg(ten) from (select ten from tenk1 where unique1 < 15 order by un
select array_agg(nullif(ten, 4)) from (select ten from tenk1 where unique1 < 15 order by unique1) ss;
select array_agg(unique1) from tenk1 where unique1 < -15;
+select array(select unique1 from tenk1 where unique1 < 15 order by unique1);
+select array(select array[i,i/2] from generate_series(1,5) a(i));
+-- cannot accumulate null arrays and empty arrays
+select array(select null::int[]);
+select array(select '{}'::int[]);
+
select unnest(array[1,2,3]);
select * from unnest(array[1,2,3]);
select unnest(array[1,2,3,4.5]::float8[]);
Re: Function array_agg(array)
2014-10-25 8:19 GMT+02:00 Ali Akbar <the.apaan@gmail.com>:
2014-10-25 10:29 GMT+07:00 Ali Akbar <the.apaan@gmail.com>:
I fixed small issue in regress tests and I enhanced tests for varlena
types and null values.
Thanks.
it is about 15% faster than original implementation.
15% faster than array_agg(scalar)? I haven't verify the performance, but
because the internal array data and null bitmap is copied as-is, that will
be faster.2014-10-25 1:51 GMT+07:00 Pavel Stehule <pavel.stehule@gmail.com>:
Hi Ali
I checked a code. I am thinking so code organization is not good.
accumArrayResult is too long now. makeMdArrayResult will not work, when
arrays was joined (it is not consistent now). I don't like a usage of
state->is_array_accum in array_userfunc.c -- it is signal of wrong
wrapping.Yes, i was thinking the same. Attached WIP patch to reorganizate the
code. makeMdArrayResult works now, with supplied arguments act as override
from default values calculated from ArrayBuildStateArray.In array_userfunc.c, because we don't want to override ndims, dims and
lbs, i copied array_agg_finalfn and only change the call to
makeMdArrayResult (we don't uses makeArrayResult because we want to set
release to false). Another alternative is to create new
makeArrayResult-like function that has parameter bool release.adding makeArrayResult1 (do you have better name for this?), that accepts
bool release parameter. array_agg_finalfn becomes more clean, and no
duplicate code in array_agg_anyarray_finalfn.
makeArrayResult1 - I have no better name now
I found one next minor detail.
you reuse a array_agg_transfn function. Inside is a message
"array_agg_transfn called in non-aggregate context". It is not correct for
array_agg_anyarray_transfn
probably specification dim and lbs in array_agg_finalfn is useless, when
you expect a natural result. A automatic similar to
array_agg_anyarray_finalfn should work there too.
makeMdArrayResultArray and appendArrayDatum should be marked as "static"
Show quoted text
next question: there is function array_append(anyarray, anyelement).
Isn't time to define array_append(anyarray, anyarray) now?
There is array_cat(anyarray, anyarray):
/*-----------------------------------------------------------------------------
* array_cat :
* concatenate two nD arrays to form an nD array, or
* push an (n-1)D array onto the end of an nD array*----------------------------------------------------------------------------
*/Regards,
--
Ali Akbar
Re: Function array_agg(array)
2014-10-25 8:33 GMT+02:00 Pavel Stehule <pavel.stehule@gmail.com>:
2014-10-25 8:19 GMT+02:00 Ali Akbar <the.apaan@gmail.com>:
2014-10-25 10:29 GMT+07:00 Ali Akbar <the.apaan@gmail.com>:
I fixed small issue in regress tests and I enhanced tests for varlena
types and null values.
Thanks.
it is about 15% faster than original implementation.
15% faster than array_agg(scalar)? I haven't verify the performance, but
because the internal array data and null bitmap is copied as-is, that will
be faster.2014-10-25 1:51 GMT+07:00 Pavel Stehule <pavel.stehule@gmail.com>:
Hi Ali
I checked a code. I am thinking so code organization is not good.
accumArrayResult is too long now. makeMdArrayResult will not work, when
arrays was joined (it is not consistent now). I don't like a usage of
state->is_array_accum in array_userfunc.c -- it is signal of wrong
wrapping.Yes, i was thinking the same. Attached WIP patch to reorganizate the
code. makeMdArrayResult works now, with supplied arguments act as override
from default values calculated from ArrayBuildStateArray.In array_userfunc.c, because we don't want to override ndims, dims and
lbs, i copied array_agg_finalfn and only change the call to
makeMdArrayResult (we don't uses makeArrayResult because we want to set
release to false). Another alternative is to create new
makeArrayResult-like function that has parameter bool release.adding makeArrayResult1 (do you have better name for this?), that accepts
bool release parameter. array_agg_finalfn becomes more clean, and no
duplicate code in array_agg_anyarray_finalfn.makeArrayResult1 - I have no better name now
I found one next minor detail.
you reuse a array_agg_transfn function. Inside is a message
"array_agg_transfn called in non-aggregate context". It is not correct for
array_agg_anyarray_transfnprobably specification dim and lbs in array_agg_finalfn is useless, when
you expect a natural result. A automatic similar to
array_agg_anyarray_finalfn should work there too.makeMdArrayResultArray and appendArrayDatum should be marked as "static"
I am thinking so change of makeArrayResult is wrong. It is developed for 1D
array. When we expect somewhere ND result now, then we should to use
makeMdArrayResult there. So makeArrayResult should to return 1D array in
all cases. Else a difference between makeArrayResult and makeMdArrayResult
hasn't big change and we have a problem with naming.
Regards
Pavel
Show quoted text
next question: there is function array_append(anyarray, anyelement).
Isn't time to define array_append(anyarray, anyarray) now?
There is array_cat(anyarray, anyarray):
/*-----------------------------------------------------------------------------
* array_cat :
* concatenate two nD arrays to form an nD array, or
* push an (n-1)D array onto the end of an nD array*----------------------------------------------------------------------------
*/Regards,
--
Ali Akbar
Re: Function array_agg(array)
makeArrayResult1 - I have no better name now
I found one next minor detail.
you reuse a array_agg_transfn function. Inside is a message
"array_agg_transfn called in non-aggregate context". It is not correct for
array_agg_anyarray_transfn
Fixed.
probably specification dim and lbs in array_agg_finalfn is useless, when
you expect a natural result. A automatic similar to
array_agg_anyarray_finalfn should work there too.makeMdArrayResultArray and appendArrayDatum should be marked as "static"
ok, marked all of that as static.
I am thinking so change of makeArrayResult is wrong. It is developed for
1D array. When we expect somewhere ND result now, then we should to use
makeMdArrayResult there. So makeArrayResult should to return 1D array in
all cases. Else a difference between makeArrayResult and makeMdArrayResult
hasn't big change and we have a problem with naming.
I'm thinking it like this:
- if we want to accumulate array normally, use accumArrayResult and
makeArrayResult. If we accumulate scalar the result will be 1D array. if we
accumulate array, the resulting dimension is incremented by 1.
- if we want, somehow to affect the normal behavior, and change the
dimensions, use makeMdArrayResult.
Searching through the postgres' code, other than internal use in
arrayfuncs.c, makeMdArrayResult is used only
in src/pl/plperl/plperl.c:plperl_array_to_datum, while converting perl
array to postgres array.
So if somehow we will accumulate array other than in array_agg, i think the
most natural way is using accumArrayResult and then makeArrayResult.
CMIIW
Regards,
--
Ali Akbar
Attachments:
array_agg_anyarray-10.patchtext/x-patch; charset=US-ASCII; name=array_agg_anyarray-10.patchDownload
diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml
index 7e5bcd9..f59738a 100644
--- a/doc/src/sgml/func.sgml
+++ b/doc/src/sgml/func.sgml
@@ -12046,6 +12046,22 @@ NULL baz</literallayout>(3 rows)</entry>
<row>
<entry>
<indexterm>
+ <primary>array_agg</primary>
+ </indexterm>
+ <function>array_agg(<replaceable class="parameter">anyarray</replaceable>)</function>
+ </entry>
+ <entry>
+ any
+ </entry>
+ <entry>
+ the same array type as input type
+ </entry>
+ <entry>input arrays, aggregated into higher-order multidimesional array. Rejects NULL and empty array as input.</entry>
+ </row>
+
+ <row>
+ <entry>
+ <indexterm>
<primary>average</primary>
</indexterm>
<indexterm>
diff --git a/doc/src/sgml/syntax.sgml b/doc/src/sgml/syntax.sgml
index 2f0680f..8c182a4 100644
--- a/doc/src/sgml/syntax.sgml
+++ b/doc/src/sgml/syntax.sgml
@@ -2238,6 +2238,11 @@ SELECT ARRAY(SELECT oid FROM pg_proc WHERE proname LIKE 'bytea%');
array
-----------------------------------------------------------------------
{2011,1954,1948,1952,1951,1244,1950,2005,1949,1953,2006,31,2412,2413}
+
+SELECT ARRAY(SELECT array(select i) FROM generate_series(1,5) a(i));
+ array
+-----------------------
+ {{1},{2},{3},{4},{5}}
(1 row)
</programlisting>
The subquery must return a single column. The resulting
diff --git a/src/backend/nodes/nodeFuncs.c b/src/backend/nodes/nodeFuncs.c
index 41e973b..0261fcb 100644
--- a/src/backend/nodes/nodeFuncs.c
+++ b/src/backend/nodes/nodeFuncs.c
@@ -108,12 +108,16 @@ exprType(const Node *expr)
type = exprType((Node *) tent->expr);
if (sublink->subLinkType == ARRAY_SUBLINK)
{
- type = get_array_type(type);
- if (!OidIsValid(type))
- ereport(ERROR,
- (errcode(ERRCODE_UNDEFINED_OBJECT),
- errmsg("could not find array type for data type %s",
- format_type_be(exprType((Node *) tent->expr)))));
+ if (!OidIsValid(get_element_type(type)))
+ {
+ /* not array, so check for its array type */
+ type = get_array_type(type);
+ if (!OidIsValid(type))
+ ereport(ERROR,
+ (errcode(ERRCODE_UNDEFINED_OBJECT),
+ errmsg("could not find array type for data type %s",
+ format_type_be(exprType((Node *) tent->expr)))));
+ }
}
}
else if (sublink->subLinkType == MULTIEXPR_SUBLINK)
@@ -139,12 +143,16 @@ exprType(const Node *expr)
type = subplan->firstColType;
if (subplan->subLinkType == ARRAY_SUBLINK)
{
- type = get_array_type(type);
- if (!OidIsValid(type))
- ereport(ERROR,
- (errcode(ERRCODE_UNDEFINED_OBJECT),
- errmsg("could not find array type for data type %s",
- format_type_be(subplan->firstColType))));
+ if (!OidIsValid(get_element_type(type)))
+ {
+ /* not array, so check for its array type */
+ type = get_array_type(type);
+ if (!OidIsValid(type))
+ ereport(ERROR,
+ (errcode(ERRCODE_UNDEFINED_OBJECT),
+ errmsg("could not find array type for data type %s",
+ format_type_be(subplan->firstColType))));
+ }
}
}
else if (subplan->subLinkType == MULTIEXPR_SUBLINK)
diff --git a/src/backend/optimizer/plan/subselect.c b/src/backend/optimizer/plan/subselect.c
index 3e7dc85..8fc8b49 100644
--- a/src/backend/optimizer/plan/subselect.c
+++ b/src/backend/optimizer/plan/subselect.c
@@ -668,10 +668,16 @@ build_subplan(PlannerInfo *root, Plan *plan, PlannerInfo *subroot,
Assert(!te->resjunk);
Assert(testexpr == NULL);
- arraytype = get_array_type(exprType((Node *) te->expr));
- if (!OidIsValid(arraytype))
- elog(ERROR, "could not find array type for datatype %s",
- format_type_be(exprType((Node *) te->expr)));
+
+ arraytype = exprType((Node *) te->expr);
+ if (!OidIsValid(get_element_type(arraytype)))
+ {
+ /* not array, so get the array type */
+ arraytype = get_array_type(exprType((Node *) te->expr));
+ if (!OidIsValid(arraytype))
+ elog(ERROR, "could not find array type for datatype %s",
+ format_type_be(exprType((Node *) te->expr)));
+ }
prm = generate_new_param(root,
arraytype,
exprTypmod((Node *) te->expr),
diff --git a/src/backend/utils/adt/array_userfuncs.c b/src/backend/utils/adt/array_userfuncs.c
index 831466d..7fbf39d 100644
--- a/src/backend/utils/adt/array_userfuncs.c
+++ b/src/backend/utils/adt/array_userfuncs.c
@@ -16,6 +16,8 @@
#include "utils/builtins.h"
#include "utils/lsyscache.h"
+static Datum array_agg_transfn_internal(PG_FUNCTION_ARGS,
+ const char* fname);
/*-----------------------------------------------------------------------------
* array_push :
@@ -476,6 +478,19 @@ create_singleton_array(FunctionCallInfo fcinfo,
Datum
array_agg_transfn(PG_FUNCTION_ARGS)
{
+ return array_agg_transfn_internal(fcinfo, "array_agg_transfn");
+}
+
+/*
+ * internal function of ARRAY_AGG
+ * called by array_agg_transfn and array_agg_transfn_array
+ *
+ * fname - function name (for error reporting)
+ */
+static Datum
+array_agg_transfn_internal(PG_FUNCTION_ARGS,
+ const char* fname)
+{
Oid arg1_typeid = get_fn_expr_argtype(fcinfo->flinfo, 1);
MemoryContext aggcontext;
ArrayBuildState *state;
@@ -489,7 +504,7 @@ array_agg_transfn(PG_FUNCTION_ARGS)
if (!AggCheckCallContext(fcinfo, &aggcontext))
{
/* cannot be called directly because of internal-type argument */
- elog(ERROR, "array_agg_transfn called in non-aggregate context");
+ elog(ERROR, "%s called in non-aggregate context", fname);
}
state = PG_ARGISNULL(0) ? NULL : (ArrayBuildState *) PG_GETARG_POINTER(0);
@@ -513,8 +528,6 @@ array_agg_finalfn(PG_FUNCTION_ARGS)
{
Datum result;
ArrayBuildState *state;
- int dims[1];
- int lbs[1];
/*
* Test for null before Asserting we are in right context. This is to
@@ -529,18 +542,28 @@ array_agg_finalfn(PG_FUNCTION_ARGS)
state = (ArrayBuildState *) PG_GETARG_POINTER(0);
- dims[0] = state->nelems;
- lbs[0] = 1;
-
/*
* Make the result. We cannot release the ArrayBuildState because
* sometimes aggregate final functions are re-executed. Rather, it is
* nodeAgg.c's responsibility to reset the aggcontext when it's safe to do
* so.
*/
- result = makeMdArrayResult(state, 1, dims, lbs,
- CurrentMemoryContext,
- false);
+ result = makeArrayResult1(state, CurrentMemoryContext, false);
PG_RETURN_DATUM(result);
}
+
+/*
+ * ARRAY_AGG(anyarray) aggregate function
+ */
+Datum
+array_agg_anyarray_transfn(PG_FUNCTION_ARGS)
+{
+ return array_agg_transfn_internal(fcinfo, "array_agg_anyarray_transfn");
+}
+
+Datum
+array_agg_anyarray_finalfn(PG_FUNCTION_ARGS)
+{
+ return array_agg_finalfn(fcinfo);
+}
diff --git a/src/backend/utils/adt/arrayfuncs.c b/src/backend/utils/adt/arrayfuncs.c
index 6c8b41d..24de430 100644
--- a/src/backend/utils/adt/arrayfuncs.c
+++ b/src/backend/utils/adt/arrayfuncs.c
@@ -145,6 +145,19 @@ static int width_bucket_array_variable(Datum operand,
Oid collation,
TypeCacheEntry *typentry);
+/* internal functions of accumArrayResult */
+static ArrayBuildStateScalar *initializeArrayBuildStateScalar(Oid element_type,
+ MemoryContext arr_context);
+static ArrayBuildStateArray *initializeArrayBuildStateArray(Oid subelement_type,
+ MemoryContext arr_context);
+static void appendScalarDatum(ArrayBuildStateScalar *astate_scalar,
+ Datum dvalue, bool disnull);
+static void appendArrayDatum(ArrayBuildStateArray *astate_array,
+ Datum dvalue, bool disnull);
+
+/* internal function of makeMdArrayResult */
+static ArrayType *makeMdArrayResultArray(ArrayBuildStateArray *astate,
+ int ndims, int *dims, int *lbs);
/*
* array_in :
@@ -4588,9 +4601,13 @@ accumArrayResult(ArrayBuildState *astate,
MemoryContext arr_context,
oldcontext;
+ ArrayBuildStateScalar *astate_scalar = NULL; /* for scalar datum accumulation */
+ ArrayBuildStateArray *astate_array = NULL; /* for array datum accumulation */
+
if (astate == NULL)
{
/* First time through --- initialize */
+ Oid subelement_type = get_element_type(element_type);
/* Make a temporary context to hold all the junk */
arr_context = AllocSetContextCreate(rcontext,
@@ -4599,33 +4616,127 @@ accumArrayResult(ArrayBuildState *astate,
ALLOCSET_DEFAULT_INITSIZE,
ALLOCSET_DEFAULT_MAXSIZE);
oldcontext = MemoryContextSwitchTo(arr_context);
- astate = (ArrayBuildState *) palloc(sizeof(ArrayBuildState));
- astate->mcontext = arr_context;
- astate->alen = 64; /* arbitrary starting array size */
- astate->dvalues = (Datum *) palloc(astate->alen * sizeof(Datum));
- astate->dnulls = (bool *) palloc(astate->alen * sizeof(bool));
- astate->nelems = 0;
- astate->element_type = element_type;
- get_typlenbyvalalign(element_type,
- &astate->typlen,
- &astate->typbyval,
- &astate->typalign);
+
+ if (!OidIsValid(subelement_type))
+ {
+ astate_scalar = initializeArrayBuildStateScalar(element_type, arr_context);
+ astate = (ArrayBuildState *) astate_scalar;
+
+ appendScalarDatum(astate_scalar, dvalue, disnull);
+ }
+ else
+ {
+ astate_array = initializeArrayBuildStateArray(subelement_type, arr_context);
+ astate = (ArrayBuildState *) astate_array;
+
+ appendArrayDatum(astate_array, dvalue, disnull);
+ }
}
else
{
oldcontext = MemoryContextSwitchTo(astate->mcontext);
- Assert(astate->element_type == element_type);
- /* enlarge dvalues[]/dnulls[] if needed */
- if (astate->nelems >= astate->alen)
+ if (!astate->is_array_accum)
{
- astate->alen *= 2;
- astate->dvalues = (Datum *)
- repalloc(astate->dvalues, astate->alen * sizeof(Datum));
- astate->dnulls = (bool *)
- repalloc(astate->dnulls, astate->alen * sizeof(bool));
+ Assert(astate->element_type == element_type);
+ astate_scalar = (ArrayBuildStateScalar *) astate;
+
+ /* enlarge dvalues[]/dnulls[] if needed */
+ if (astate_scalar->nelems >= astate_scalar->alen)
+ {
+ astate_scalar->alen *= 2;
+ astate_scalar->dvalues = (Datum *)
+ repalloc(astate_scalar->dvalues, astate_scalar->alen * sizeof(Datum));
+ astate_scalar->dnulls = (bool *)
+ repalloc(astate_scalar->dnulls, astate_scalar->alen * sizeof(bool));
+ }
+
+ appendScalarDatum(astate_scalar, dvalue, disnull);
+ }
+ else
+ {
+ Assert(astate->element_type == get_element_type(element_type));
+ astate_array = (ArrayBuildStateArray *) astate;
+ /*
+ * alloc & realloc placed in appendArrayDatum because
+ * we need to know the dvalue length first
+ */
+ appendArrayDatum(astate_array, dvalue, disnull);
}
}
+ MemoryContextSwitchTo(oldcontext);
+
+ return astate;
+}
+
+/*
+ * Initialize ArrayBuildState for scalar accumulation (internal function of
+ * accumArrayResult)
+ */
+static ArrayBuildStateScalar *
+initializeArrayBuildStateScalar(Oid element_type,
+ MemoryContext arr_context)
+{
+ ArrayBuildStateScalar *astate_scalar;
+
+ /* scalar accumulate */
+ astate_scalar = (ArrayBuildStateScalar *) palloc(sizeof(ArrayBuildStateScalar));
+ astate_scalar->astate.is_array_accum = false;
+ astate_scalar->astate.mcontext = arr_context;
+
+ astate_scalar->alen = 64; /* arbitrary starting array size */
+ astate_scalar->dvalues = (Datum *) palloc(astate_scalar->alen * sizeof(Datum));
+ astate_scalar->dnulls = (bool *) palloc(astate_scalar->alen * sizeof(bool));
+ astate_scalar->nelems = 0;
+
+ astate_scalar->astate.element_type = element_type;
+ get_typlenbyvalalign(element_type,
+ &astate_scalar->astate.typlen,
+ &astate_scalar->astate.typbyval,
+ &astate_scalar->astate.typalign);
+
+ return astate_scalar;
+}
+
+/*
+ * Initialize ArrayBuildState for array accum (internal of accumArrayResult)
+ */
+static ArrayBuildStateArray *
+initializeArrayBuildStateArray(Oid subelement_type,
+ MemoryContext arr_context)
+{
+ ArrayBuildStateArray *astate_array;
+
+ /* array accumulate */
+ astate_array = (ArrayBuildStateArray *) palloc(sizeof(ArrayBuildStateArray));
+ astate_array->astate.is_array_accum = true;
+ astate_array->astate.mcontext = arr_context;
+
+ astate_array->abytes = 0;
+ astate_array->aitems = 0;
+ astate_array->data = NULL;
+ astate_array->nbytes = 0;
+ astate_array->hasnull = false;
+ astate_array->nullbitmap = NULL;
+ astate_array->nitems = 0;
+ astate_array->narray = 0;
+
+ astate_array->astate.element_type = subelement_type;
+ get_typlenbyvalalign(subelement_type,
+ &astate_array->astate.typlen,
+ &astate_array->astate.typbyval,
+ &astate_array->astate.typalign);
+
+ return astate_array;
+}
+
+/*
+ * Append a datum for scalar accumulation (internal of accumArrayResult)
+ */
+static void
+appendScalarDatum(ArrayBuildStateScalar *astate_scalar,
+ Datum dvalue, bool disnull)
+{
/*
* Ensure pass-by-ref stuff is copied into mcontext; and detoast it too if
* it's varlena. (You might think that detoasting is not needed here
@@ -4634,25 +4745,129 @@ accumArrayResult(ArrayBuildState *astate,
* because that would mean array_agg_finalfn damages its input, which is
* verboten. Also, this way frequently saves one copying step.)
*/
- if (!disnull && !astate->typbyval)
+ if (!disnull && !astate_scalar->astate.typbyval)
{
- if (astate->typlen == -1)
+ if (astate_scalar->astate.typlen == -1)
dvalue = PointerGetDatum(PG_DETOAST_DATUM_COPY(dvalue));
else
- dvalue = datumCopy(dvalue, astate->typbyval, astate->typlen);
+ dvalue = datumCopy(dvalue, astate_scalar->astate.typbyval,
+ astate_scalar->astate.typlen);
}
- astate->dvalues[astate->nelems] = dvalue;
- astate->dnulls[astate->nelems] = disnull;
- astate->nelems++;
+ astate_scalar->dvalues[astate_scalar->nelems] = dvalue;
+ astate_scalar->dnulls[astate_scalar->nelems] = disnull;
+ astate_scalar->nelems++;
+}
- MemoryContextSwitchTo(oldcontext);
+/*
+ * Append a datum for array accum (internal of accumArrayResult)
+ */
+static void
+appendArrayDatum(ArrayBuildStateArray *astate_array,
+ Datum dvalue, bool disnull)
+{
+ ArrayType *arg;
+ int *dims,
+ *lbs,
+ ndims,
+ nitems,
+ ndatabytes;
+ char *data;
+ int i;
- return astate;
+ if (disnull)
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("cannot accumulate null arrays")));
+
+ arg = DatumGetArrayTypeP(dvalue);
+
+ ndims = ARR_NDIM(arg);
+ dims = ARR_DIMS(arg);
+ lbs = ARR_LBOUND(arg);
+ data = ARR_DATA_PTR(arg);
+ nitems = ArrayGetNItems(ndims, dims);
+ ndatabytes = ARR_SIZE(arg) - ARR_DATA_OFFSET(arg);
+
+ if (astate_array->data == NULL)
+ {
+ /* first allocation */
+ if (ndims == 0)
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("cannot accumulate empty arrays")));
+
+ if (ndims + 1 > MAXDIM)
+ ereport(ERROR,
+ (errcode(ERRCODE_PROGRAM_LIMIT_EXCEEDED),
+ errmsg("number of array dimensions (%d) exceeds the maximum allowed (%d)",
+ ndims + 1, MAXDIM)));
+
+ astate_array->ndims = ndims;
+ astate_array->dims = (int *) palloc(ndims * sizeof(int));
+ astate_array->lbs = (int *) palloc(ndims * sizeof(int));
+ memcpy(astate_array->dims, dims, ndims * sizeof(int));
+ memcpy(astate_array->lbs, lbs, ndims * sizeof(int));
+
+ astate_array->abytes = ndatabytes >= 512 ? 4 * ndatabytes : 1024;
+ astate_array->data = (char *) palloc(astate_array->abytes);
+ astate_array->aitems = 4 * nitems;
+ }
+ else
+ {
+ if (astate_array->ndims != ndims)
+ ereport(ERROR,
+ (errcode(ERRCODE_ARRAY_SUBSCRIPT_ERROR),
+ errmsg("cannot accumulate incompatible arrays"),
+ errdetail("Arrays of %d and %d dimensions are not "
+ "compatible for concatenation.",
+ astate_array->ndims, ndims)));
+
+ for (i = 0; i < ndims; i++)
+ if (astate_array->dims[i] != dims[i] || astate_array->lbs[i] != lbs[i])
+ ereport(ERROR,
+ (errcode(ERRCODE_ARRAY_SUBSCRIPT_ERROR),
+ errmsg("cannot accumulate incompatible arrays"),
+ errdetail("Arrays with differing element dimensions are "
+ "not compatible for concatenation.")));
+
+ if (astate_array->nbytes + ndatabytes >= astate_array->abytes)
+ {
+ astate_array->abytes *= 2;
+ astate_array->data = (char *)
+ repalloc(astate_array->data, astate_array->abytes);
+ }
+ if (astate_array->nitems + nitems >= astate_array->aitems)
+ {
+ astate_array->aitems *= 2;
+ astate_array->nullbitmap = (bits8 *)
+ repalloc(astate_array->nullbitmap, (astate_array->aitems + 7) / 8);
+ }
+ }
+ memcpy(astate_array->data + astate_array->nbytes, data, ndatabytes);
+ astate_array->nbytes += ndatabytes;
+
+ if (astate_array->hasnull || ARR_HASNULL(arg))
+ {
+ if (!astate_array->hasnull)
+ {
+ astate_array->hasnull = true;
+ astate_array->nullbitmap = (bits8 *) palloc((astate_array->aitems + 7) / 8);
+ array_bitmap_copy(astate_array->nullbitmap, 0,
+ NULL, 0,
+ astate_array->nitems);
+ }
+ array_bitmap_copy(astate_array->nullbitmap, astate_array->nitems,
+ ARR_NULLBITMAP(arg), 0,
+ nitems);
+ astate_array->nitems += nitems;
+ }
+ astate_array->narray += 1;
}
/*
- * makeArrayResult - produce 1-D final result of accumArrayResult
+ * makeArrayResult - produce 1-D final result of scalar accumArrayResult
+ * - produce N+1-D final result of array accumArrayResult
*
* astate is working state (not NULL)
* rcontext is where to construct result
@@ -4661,17 +4876,36 @@ Datum
makeArrayResult(ArrayBuildState *astate,
MemoryContext rcontext)
{
- int dims[1];
- int lbs[1];
+ return makeArrayResult1(astate, rcontext, true);
+}
- dims[0] = astate->nelems;
- lbs[0] = 1;
+/*
+ * makeArrayResult1 - same as makeArrayResult, with option not to
+ * release working state
+ *
+ * release is true if okay to release working state
+ */
+Datum
+makeArrayResult1(ArrayBuildState *astate,
+ MemoryContext rcontext,
+ bool release)
+{
+ if (!astate->is_array_accum)
+ {
+ int dims[1];
+ int lbs[1];
+
+ dims[0] = ((ArrayBuildStateScalar *) astate)->nelems;
+ lbs[0] = 1;
- return makeMdArrayResult(astate, 1, dims, lbs, rcontext, true);
+ return makeMdArrayResult(astate, 1, dims, lbs, rcontext, release);
+ }
+ else
+ return makeMdArrayResult(astate, -1, NULL, NULL, rcontext, release);
}
/*
- * makeMdArrayResult - produce multi-D final result of accumArrayResult
+ * makeMdArrayResult - produce multi-D final result of scalar accumArrayResult
*
* beware: no check that specified dimensions match the number of values
* accumulated.
@@ -4679,6 +4913,11 @@ makeArrayResult(ArrayBuildState *astate,
* astate is working state (not NULL)
* rcontext is where to construct result
* release is true if okay to release working state
+ *
+ * for array accumulation:
+ * ndims == -1: default (calculated from astate)
+ * dims == NULL: default dims (calculated from astate)
+ * lbs == NULL: default (calculated from astate)
*/
Datum
makeMdArrayResult(ArrayBuildState *astate,
@@ -4694,8 +4933,13 @@ makeMdArrayResult(ArrayBuildState *astate,
/* Build the final array result in rcontext */
oldcontext = MemoryContextSwitchTo(rcontext);
- result = construct_md_array(astate->dvalues,
- astate->dnulls,
+ if (!astate->is_array_accum)
+ {
+ ArrayBuildStateScalar *astate_scalar;
+ astate_scalar = (ArrayBuildStateScalar *) astate;
+
+ result = construct_md_array(astate_scalar->dvalues,
+ astate_scalar->dnulls,
ndims,
dims,
lbs,
@@ -4703,6 +4947,16 @@ makeMdArrayResult(ArrayBuildState *astate,
astate->typlen,
astate->typbyval,
astate->typalign);
+ }
+ else
+ {
+ ArrayBuildStateArray *astate_array;
+ astate_array = (ArrayBuildStateArray *) astate;
+ result = makeMdArrayResultArray(astate_array,
+ ndims,
+ dims,
+ lbs);
+ }
MemoryContextSwitchTo(oldcontext);
@@ -4713,6 +4967,75 @@ makeMdArrayResult(ArrayBuildState *astate,
return PointerGetDatum(result);
}
+/*
+ * makeMdArrayResultArray - internal of makeArrayResult & makeMdArrayResult
+ *
+ * beware: no check that specified dimensions match the values
+ * accumulated.
+ *
+ * astate is working state (not NULL)
+ * ndims > -1 if wants to override default (calculated from astate)
+ * dims not NULL if wants to override default
+ * lbs not NULL if wants to override default
+ */
+static ArrayType *
+makeMdArrayResultArray(ArrayBuildStateArray *astate,
+ int ndims,
+ int *dims,
+ int *lbs)
+{
+ ArrayType *result;
+
+ int dataoffset,
+ nbytes;
+
+ if (ndims == -1)
+ ndims = astate->ndims + 1;
+
+ nbytes = astate->nbytes;
+ /* compute required space */
+ if (astate->hasnull)
+ {
+ dataoffset = ARR_OVERHEAD_WITHNULLS(ndims, astate->nitems);
+ nbytes += dataoffset;
+ }
+ else
+ {
+ dataoffset = 0;
+ nbytes += ARR_OVERHEAD_NONULLS(ndims);
+ }
+
+ result = (ArrayType *) palloc0(nbytes);
+ SET_VARSIZE(result, nbytes);
+ result->ndim = astate->ndims + 1;
+ result->dataoffset = dataoffset;
+ result->elemtype = astate->astate.element_type;
+
+ if (dims != NULL)
+ memcpy(ARR_DIMS(result), dims, ndims * sizeof(int));
+ else
+ {
+ ARR_DIMS(result)[0] = astate->narray;
+ memcpy(&(ARR_DIMS(result)[1]), astate->dims, (ndims - 1) * sizeof(int));
+ }
+ if (lbs != NULL)
+ memcpy(ARR_LBOUND(result), lbs, ndims * sizeof(int));
+ else
+ {
+ ARR_LBOUND(result)[0] = 1;
+ memcpy(&(ARR_LBOUND(result)[1]), astate->lbs, (ndims - 1) * sizeof(int));
+ }
+
+ memcpy(ARR_DATA_PTR(result), astate->data, astate->nbytes);
+
+ if (astate->hasnull)
+ array_bitmap_copy(ARR_NULLBITMAP(result), 0,
+ astate->nullbitmap, 0,
+ astate->nitems);
+
+ return result;
+}
+
Datum
array_larger(PG_FUNCTION_ARGS)
{
diff --git a/src/include/catalog/pg_aggregate.h b/src/include/catalog/pg_aggregate.h
index 3ba9e5e..2005199 100644
--- a/src/include/catalog/pg_aggregate.h
+++ b/src/include/catalog/pg_aggregate.h
@@ -275,6 +275,7 @@ DATA(insert ( 2901 n 0 xmlconcat2 - - - - f f 0 142 0 0 0 _null_
/* array */
DATA(insert ( 2335 n 0 array_agg_transfn array_agg_finalfn - - - t f 0 2281 0 0 0 _null_ _null_ ));
+DATA(insert ( 6005 n 0 array_agg_anyarray_transfn array_agg_anyarray_finalfn - - - t f 0 2281 0 0 0 _null_ _null_ ));
/* text */
DATA(insert ( 3538 n 0 string_agg_transfn string_agg_finalfn - - - f f 0 2281 0 0 0 _null_ _null_ ));
diff --git a/src/include/catalog/pg_proc.h b/src/include/catalog/pg_proc.h
index b6dc1b8..9273c1f 100644
--- a/src/include/catalog/pg_proc.h
+++ b/src/include/catalog/pg_proc.h
@@ -879,11 +879,17 @@ DATA(insert OID = 3167 ( array_remove PGNSP PGUID 12 1 0 0 0 f f f f f f i 2
DESCR("remove any occurrences of an element from an array");
DATA(insert OID = 3168 ( array_replace PGNSP PGUID 12 1 0 0 0 f f f f f f i 3 0 2277 "2277 2283 2283" _null_ _null_ _null_ _null_ array_replace _null_ _null_ _null_ ));
DESCR("replace any occurrences of an element in an array");
-DATA(insert OID = 2333 ( array_agg_transfn PGNSP PGUID 12 1 0 0 0 f f f f f f i 2 0 2281 "2281 2283" _null_ _null_ _null_ _null_ array_agg_transfn _null_ _null_ _null_ ));
+DATA(insert OID = 2333 ( array_agg_transfn PGNSP PGUID 12 1 0 0 0 f f f f f f i 2 0 2281 "2281 2776" _null_ _null_ _null_ _null_ array_agg_transfn _null_ _null_ _null_ ));
DESCR("aggregate transition function");
-DATA(insert OID = 2334 ( array_agg_finalfn PGNSP PGUID 12 1 0 0 0 f f f f f f i 2 0 2277 "2281 2283" _null_ _null_ _null_ _null_ array_agg_finalfn _null_ _null_ _null_ ));
+DATA(insert OID = 2334 ( array_agg_finalfn PGNSP PGUID 12 1 0 0 0 f f f f f f i 2 0 2277 "2281 2776" _null_ _null_ _null_ _null_ array_agg_finalfn _null_ _null_ _null_ ));
DESCR("aggregate final function");
-DATA(insert OID = 2335 ( array_agg PGNSP PGUID 12 1 0 0 0 t f f f f f i 1 0 2277 "2283" _null_ _null_ _null_ _null_ aggregate_dummy _null_ _null_ _null_ ));
+DATA(insert OID = 2335 ( array_agg PGNSP PGUID 12 1 0 0 0 t f f f f f i 1 0 2277 "2776" _null_ _null_ _null_ _null_ aggregate_dummy _null_ _null_ _null_ ));
+DESCR("concatenate aggregate input into an array");
+DATA(insert OID = 6003 ( array_agg_anyarray_transfn PGNSP PGUID 12 1 0 0 0 f f f f f f i 2 0 2281 "2281 2277" _null_ _null_ _null_ _null_ array_agg_anyarray_transfn _null_ _null_ _null_ ));
+DESCR("aggregate transition function");
+DATA(insert OID = 6004 ( array_agg_anyarray_finalfn PGNSP PGUID 12 1 0 0 0 f f f f f f i 2 0 2277 "2281 2277" _null_ _null_ _null_ _null_ array_agg_anyarray_finalfn _null_ _null_ _null_ ));
+DESCR("aggregate final function");
+DATA(insert OID = 6005 ( array_agg PGNSP PGUID 12 1 0 0 0 t f f f f f i 1 0 2277 "2277" _null_ _null_ _null_ _null_ aggregate_dummy _null_ _null_ _null_ ));
DESCR("concatenate aggregate input into an array");
DATA(insert OID = 3218 ( width_bucket PGNSP PGUID 12 1 0 0 0 f f f f t f i 2 0 23 "2283 2277" _null_ _null_ _null_ _null_ width_bucket_array _null_ _null_ _null_ ));
DESCR("bucket number of operand given a sorted array of bucket lower bounds");
diff --git a/src/include/utils/array.h b/src/include/utils/array.h
index e744314..daaae88 100644
--- a/src/include/utils/array.h
+++ b/src/include/utils/array.h
@@ -76,14 +76,15 @@ typedef struct
/*
* working state for accumArrayResult() and friends
+ *
+ * is_array_accum: whether accumulating array values.
+ * (if true must be casted to ArrayBuildStateArray, else
+ * cast to ArrayBuildStateScalar)
*/
typedef struct ArrayBuildState
{
+ bool is_array_accum;
MemoryContext mcontext; /* where all the temp stuff is kept */
- Datum *dvalues; /* array of accumulated Datums */
- bool *dnulls; /* array of is-null flags for Datums */
- int alen; /* allocated length of above arrays */
- int nelems; /* number of valid entries in above arrays */
Oid element_type; /* data type of the Datums */
int16 typlen; /* needed info about datatype */
bool typbyval;
@@ -91,6 +92,43 @@ typedef struct ArrayBuildState
} ArrayBuildState;
/*
+ * array build state for array accumulation of scalar datums
+ */
+typedef struct ArrayBuildStateScalar
+{
+ ArrayBuildState astate;
+
+ Datum *dvalues; /* array of accumulated Datums */
+ bool *dnulls; /* array of is-null flags for Datums */
+ int alen; /* allocated length of above arrays */
+ int nelems; /* number of valid entries in above arrays */
+} ArrayBuildStateScalar;
+
+
+/*
+ * array build state for array accumulation of array datums
+ */
+typedef struct
+{
+ ArrayBuildState astate;
+
+ char *data; /* array of accumulated data */
+ bits8 *nullbitmap; /* bitmap of is-null flags for data */
+
+ int abytes; /* allocated length of above arrays */
+ int aitems; /* allocated length of above arrays */
+ int nbytes; /* number of used bytes in above arrays */
+ int nitems; /* number of elements in above arrays */
+ int narray; /* number of array accumulated */
+
+ int ndims; /* element dimensions */
+ int *dims;
+ int *lbs;
+
+ bool hasnull; /* any element has null */
+} ArrayBuildStateArray;
+
+/*
* structure to cache type metadata needed for array manipulation
*/
typedef struct ArrayMetaState
@@ -258,9 +296,10 @@ extern ArrayBuildState *accumArrayResult(ArrayBuildState *astate,
MemoryContext rcontext);
extern Datum makeArrayResult(ArrayBuildState *astate,
MemoryContext rcontext);
+extern Datum makeArrayResult1(ArrayBuildState *astate,
+ MemoryContext rcontext, bool release);
extern Datum makeMdArrayResult(ArrayBuildState *astate, int ndims,
int *dims, int *lbs, MemoryContext rcontext, bool release);
-
extern ArrayIterator array_create_iterator(ArrayType *arr, int slice_ndim);
extern bool array_iterate(ArrayIterator iterator, Datum *value, bool *isnull);
extern void array_free_iterator(ArrayIterator iterator);
@@ -293,6 +332,9 @@ extern ArrayType *create_singleton_array(FunctionCallInfo fcinfo,
extern Datum array_agg_transfn(PG_FUNCTION_ARGS);
extern Datum array_agg_finalfn(PG_FUNCTION_ARGS);
+extern Datum array_agg_anyarray_transfn(PG_FUNCTION_ARGS);
+extern Datum array_agg_anyarray_finalfn(PG_FUNCTION_ARGS);
+
/*
* prototypes for functions defined in array_typanalyze.c
*/
diff --git a/src/test/regress/expected/aggregates.out b/src/test/regress/expected/aggregates.out
index 58df854..607aeea 100644
--- a/src/test/regress/expected/aggregates.out
+++ b/src/test/regress/expected/aggregates.out
@@ -914,6 +914,76 @@ select array_agg(distinct a order by a desc nulls last)
{3,2,1,NULL}
(1 row)
+-- array_agg(anyarray)
+select array_agg(ar)
+ from (values ('{1,2}'::int[]), ('{3,4}'::int[])) v(ar);
+ array_agg
+---------------
+ {{1,2},{3,4}}
+(1 row)
+
+select array_agg(distinct ar order by ar desc)
+ from (select array[i / 2] from generate_series(1,10) a(i)) b(ar);
+ array_agg
+---------------------------
+ {{5},{4},{3},{2},{1},{0}}
+(1 row)
+
+select array_agg(ar)
+ from (select array_agg(array[i, i+1, i-1])
+ from generate_series(1,2) a(i)) b(ar);
+ array_agg
+---------------------
+ {{{1,2,0},{2,3,1}}}
+(1 row)
+
+-- array_agg(anyarray), varlena types
+select array_agg(array[1.2,1.3,1.4]) from generate_series(1,3);
+ array_agg
+---------------------------------------------
+ {{1.2,1.3,1.4},{1.2,1.3,1.4},{1.2,1.3,1.4}}
+(1 row)
+
+select array_agg(array['Hello','Hohoho','Hi']) from generate_series(1,3);
+ array_agg
+---------------------------------------------------------
+ {{Hello,Hohoho,Hi},{Hello,Hohoho,Hi},{Hello,Hohoho,Hi}}
+(1 row)
+
+-- array_agg(anyarray), arrays with nulls
+select array_agg(array[i, null, i+1, null, i+2]) from generate_series(1,3) g(i);
+ array_agg
+---------------------------------------------------------
+ {{1,NULL,2,NULL,3},{2,NULL,3,NULL,4},{3,NULL,4,NULL,5}}
+(1 row)
+
+select array_agg(array[1.1+ i, null, 1.1+i+1, null, 1.1+i+2]) from generate_series(1,3) g(i);
+ array_agg
+---------------------------------------------------------------------------
+ {{2.1,NULL,3.1,NULL,4.1},{3.1,NULL,4.1,NULL,5.1},{4.1,NULL,5.1,NULL,6.1}}
+(1 row)
+
+select array_agg(array[null, 'Hello','Hohoho', null,'Hi']) from generate_series(1,3);
+ array_agg
+---------------------------------------------------------------------------------------
+ {{NULL,Hello,Hohoho,NULL,Hi},{NULL,Hello,Hohoho,NULL,Hi},{NULL,Hello,Hohoho,NULL,Hi}}
+(1 row)
+
+select array_agg(array[[null, 'Hello', null, 'Hi'],['Hello', null, 'Hi', null]]) from generate_series(1,2);
+ array_agg
+-------------------------------------------------------------------------------------------
+ {{{NULL,Hello,NULL,Hi},{Hello,NULL,Hi,NULL}},{{NULL,Hello,NULL,Hi},{Hello,NULL,Hi,NULL}}}
+(1 row)
+
+-- errors
+select array_agg('{}'::int[]) from generate_series(1,2);
+ERROR: cannot accumulate empty arrays
+select array_agg(null::int[]) from generate_series(1,2);
+ERROR: cannot accumulate null arrays
+select array_agg(ar)
+ from (values ('{1,2}'::int[]), ('{3}'::int[])) v(ar);
+ERROR: cannot accumulate incompatible arrays
+DETAIL: Arrays with differing element dimensions are not compatible for concatenation.
-- multi-arg aggs, strict/nonstrict, distinct/order by
select aggfstr(a,b,c)
from (values (1,3,'foo'),(0,null,null),(2,2,'bar'),(3,1,'baz')) v(a,b,c);
diff --git a/src/test/regress/expected/arrays.out b/src/test/regress/expected/arrays.out
index 46eff67..e80ebec 100644
--- a/src/test/regress/expected/arrays.out
+++ b/src/test/regress/expected/arrays.out
@@ -1521,6 +1521,23 @@ select array_agg(unique1) from tenk1 where unique1 < -15;
(1 row)
+select array(select unique1 from tenk1 where unique1 < 15 order by unique1);
+ array
+--------------------------------------
+ {0,1,2,3,4,5,6,7,8,9,10,11,12,13,14}
+(1 row)
+
+select array(select array[i,i/2] from generate_series(1,5) a(i));
+ array
+---------------------------------
+ {{1,0},{2,1},{3,1},{4,2},{5,2}}
+(1 row)
+
+-- cannot accumulate null arrays and empty arrays
+select array(select null::int[]);
+ERROR: cannot accumulate null arrays
+select array(select '{}'::int[]);
+ERROR: cannot accumulate empty arrays
select unnest(array[1,2,3]);
unnest
--------
diff --git a/src/test/regress/sql/aggregates.sql b/src/test/regress/sql/aggregates.sql
index 8096a6f..a70419f 100644
--- a/src/test/regress/sql/aggregates.sql
+++ b/src/test/regress/sql/aggregates.sql
@@ -322,6 +322,32 @@ select array_agg(distinct a order by a desc)
select array_agg(distinct a order by a desc nulls last)
from (values (1),(2),(1),(3),(null),(2)) v(a);
+-- array_agg(anyarray)
+select array_agg(ar)
+ from (values ('{1,2}'::int[]), ('{3,4}'::int[])) v(ar);
+select array_agg(distinct ar order by ar desc)
+ from (select array[i / 2] from generate_series(1,10) a(i)) b(ar);
+select array_agg(ar)
+ from (select array_agg(array[i, i+1, i-1])
+ from generate_series(1,2) a(i)) b(ar);
+
+-- array_agg(anyarray), varlena types
+select array_agg(array[1.2,1.3,1.4]) from generate_series(1,3);
+select array_agg(array['Hello','Hohoho','Hi']) from generate_series(1,3);
+
+-- array_agg(anyarray), arrays with nulls
+select array_agg(array[i, null, i+1, null, i+2]) from generate_series(1,3) g(i);
+select array_agg(array[1.1+ i, null, 1.1+i+1, null, 1.1+i+2]) from generate_series(1,3) g(i);
+select array_agg(array[null, 'Hello','Hohoho', null,'Hi']) from generate_series(1,3);
+
+select array_agg(array[[null, 'Hello', null, 'Hi'],['Hello', null, 'Hi', null]]) from generate_series(1,2);
+
+-- errors
+select array_agg('{}'::int[]) from generate_series(1,2);
+select array_agg(null::int[]) from generate_series(1,2);
+select array_agg(ar)
+ from (values ('{1,2}'::int[]), ('{3}'::int[])) v(ar);
+
-- multi-arg aggs, strict/nonstrict, distinct/order by
select aggfstr(a,b,c)
diff --git a/src/test/regress/sql/arrays.sql b/src/test/regress/sql/arrays.sql
index fa8a20a..cb00f5f 100644
--- a/src/test/regress/sql/arrays.sql
+++ b/src/test/regress/sql/arrays.sql
@@ -432,6 +432,12 @@ select array_agg(ten) from (select ten from tenk1 where unique1 < 15 order by un
select array_agg(nullif(ten, 4)) from (select ten from tenk1 where unique1 < 15 order by unique1) ss;
select array_agg(unique1) from tenk1 where unique1 < -15;
+select array(select unique1 from tenk1 where unique1 < 15 order by unique1);
+select array(select array[i,i/2] from generate_series(1,5) a(i));
+-- cannot accumulate null arrays and empty arrays
+select array(select null::int[]);
+select array(select '{}'::int[]);
+
select unnest(array[1,2,3]);
select * from unnest(array[1,2,3]);
select unnest(array[1,2,3,4.5]::float8[]);
Re: Function array_agg(array)
2014-10-25 10:16 GMT+02:00 Ali Akbar <the.apaan@gmail.com>:
makeArrayResult1 - I have no better name now
I found one next minor detail.
you reuse a array_agg_transfn function. Inside is a message
"array_agg_transfn called in non-aggregate context". It is not correct for
array_agg_anyarray_transfnFixed.
probably specification dim and lbs in array_agg_finalfn is useless, when
you expect a natural result. A automatic similar to
array_agg_anyarray_finalfn should work there too.makeMdArrayResultArray and appendArrayDatum should be marked as "static"
ok, marked all of that as static.
I am thinking so change of makeArrayResult is wrong. It is developed for
1D array. When we expect somewhere ND result now, then we should to use
makeMdArrayResult there. So makeArrayResult should to return 1D array in
all cases. Else a difference between makeArrayResult and makeMdArrayResult
hasn't big change and we have a problem with naming.I'm thinking it like this:
- if we want to accumulate array normally, use accumArrayResult and
makeArrayResult. If we accumulate scalar the result will be 1D array. if we
accumulate array, the resulting dimension is incremented by 1.
- if we want, somehow to affect the normal behavior, and change the
dimensions, use makeMdArrayResult.Searching through the postgres' code, other than internal use in
arrayfuncs.c, makeMdArrayResult is used only
in src/pl/plperl/plperl.c:plperl_array_to_datum, while converting perl
array to postgres array.So if somehow we will accumulate array other than in array_agg, i think
the most natural way is using accumArrayResult and then makeArrayResult.
ok, there is more variants and I can't to decide. But I am not satisfied
with this API. We do some wrong in structure. makeMdArrayResult is now
ugly.
Show quoted text
CMIIW
Regards,
--
Ali Akbar
Re: Function array_agg(array)
2014-10-25 15:43 GMT+07:00 Pavel Stehule <pavel.stehule@gmail.com>:
2014-10-25 10:16 GMT+02:00 Ali Akbar <the.apaan@gmail.com>:
makeArrayResult1 - I have no better name now
I found one next minor detail.
you reuse a array_agg_transfn function. Inside is a message
"array_agg_transfn called in non-aggregate context". It is not correct for
array_agg_anyarray_transfnFixed.
probably specification dim and lbs in array_agg_finalfn is useless, when
you expect a natural result. A automatic similar to
array_agg_anyarray_finalfn should work there too.makeMdArrayResultArray and appendArrayDatum should be marked as
"static"ok, marked all of that as static.
I am thinking so change of makeArrayResult is wrong. It is developed for
1D array. When we expect somewhere ND result now, then we should to use
makeMdArrayResult there. So makeArrayResult should to return 1D array in
all cases. Else a difference between makeArrayResult and makeMdArrayResult
hasn't big change and we have a problem with naming.I'm thinking it like this:
- if we want to accumulate array normally, use accumArrayResult and
makeArrayResult. If we accumulate scalar the result will be 1D array. if we
accumulate array, the resulting dimension is incremented by 1.
- if we want, somehow to affect the normal behavior, and change the
dimensions, use makeMdArrayResult.Searching through the postgres' code, other than internal use in
arrayfuncs.c, makeMdArrayResult is used only
in src/pl/plperl/plperl.c:plperl_array_to_datum, while converting perl
array to postgres array.So if somehow we will accumulate array other than in array_agg, i think
the most natural way is using accumArrayResult and then makeArrayResult.ok, there is more variants and I can't to decide. But I am not satisfied
with this API. We do some wrong in structure. makeMdArrayResult is now
ugly.
One approach that i can think is we cleanly separate the structures and
API. We don't touch existing ArrayBuildState, accumArrayResult,
makeArrayResult and makeMdArrayResult, and we create new functions:
ArrayBuildStateArray, accumArrayResultArray, makeArrayResultArray and
makeArrayResultArrayCustom.
But if we do that, the array(subselect) implementation will not be as
simple as current patch. We must also change all the ARRAY_SUBLINK-related
accumArrayResult call.
Regarding current patch implementation, the specific typedefs are declared
as:
+typedef struct ArrayBuildStateScalar
+{
+ ArrayBuildState astate;
...
+} ArrayBuildStateArray;so it necessities rather ugly code like this:
+ result->elemtype = astate->astate.element_type;
Can we use C11 feature of unnamed struct like this? :
+typedef struct ArrayBuildStateScalar
+{
+ ArrayBuildState;
...
+} ArrayBuildStateArray;so the code can be a little less ugly by doing it like this:
+ result->elemtype = astate->element_type;
I don't know whether all currently supported compilers implements this
feature..
Can you suggest a better structure for this?
If we can't settle on a better structure, i think i'll reimplement
array_agg without the performance improvement, using deconstruct_array and
unchanged accumArrayResult & makeMdArrayResult.
Thanks,
--
Ali Akbar
Re: Function array_agg(array)
2014-10-25 12:20 GMT+02:00 Ali Akbar <the.apaan@gmail.com>:
2014-10-25 15:43 GMT+07:00 Pavel Stehule <pavel.stehule@gmail.com>:
2014-10-25 10:16 GMT+02:00 Ali Akbar <the.apaan@gmail.com>:
makeArrayResult1 - I have no better name now
I found one next minor detail.
you reuse a array_agg_transfn function. Inside is a message
"array_agg_transfn called in non-aggregate context". It is not correct for
array_agg_anyarray_transfnFixed.
probably specification dim and lbs in array_agg_finalfn is useless,
when you expect a natural result. A automatic similar to
array_agg_anyarray_finalfn should work there too.makeMdArrayResultArray and appendArrayDatum should be marked as
"static"ok, marked all of that as static.
I am thinking so change of makeArrayResult is wrong. It is developed
for 1D array. When we expect somewhere ND result now, then we should to use
makeMdArrayResult there. So makeArrayResult should to return 1D array in
all cases. Else a difference between makeArrayResult and makeMdArrayResult
hasn't big change and we have a problem with naming.I'm thinking it like this:
- if we want to accumulate array normally, use accumArrayResult and
makeArrayResult. If we accumulate scalar the result will be 1D array. if we
accumulate array, the resulting dimension is incremented by 1.
- if we want, somehow to affect the normal behavior, and change the
dimensions, use makeMdArrayResult.Searching through the postgres' code, other than internal use in
arrayfuncs.c, makeMdArrayResult is used only
in src/pl/plperl/plperl.c:plperl_array_to_datum, while converting perl
array to postgres array.So if somehow we will accumulate array other than in array_agg, i think
the most natural way is using accumArrayResult and then makeArrayResult.ok, there is more variants and I can't to decide. But I am not satisfied
with this API. We do some wrong in structure. makeMdArrayResult is now
ugly.One approach that i can think is we cleanly separate the structures and
API. We don't touch existing ArrayBuildState, accumArrayResult,
makeArrayResult and makeMdArrayResult, and we create new functions:
ArrayBuildStateArray, accumArrayResultArray, makeArrayResultArray and
makeArrayResultArrayCustom.
yes, I am thinking about separatate path, that will be joined in
constructMdArray
In reality, there are two different array builders - with different API and
better to respect it.
But if we do that, the array(subselect) implementation will not be as
simple as current patch. We must also change all the ARRAY_SUBLINK-related
accumArrayResult call.Regarding current patch implementation, the specific typedefs are declared as: +typedef struct ArrayBuildStateScalar +{ + ArrayBuildState astate; ... +} ArrayBuildStateArray;so it necessities rather ugly code like this:
+ result->elemtype = astate->astate.element_type;Can we use C11 feature of unnamed struct like this? :
no, what I know a C11 is prohibited
+typedef struct ArrayBuildStateScalar +{ + ArrayBuildState; ... +} ArrayBuildStateArray;so the code can be a little less ugly by doing it like this:
+ result->elemtype = astate->element_type;I don't know whether all currently supported compilers implements this
feature..Can you suggest a better structure for this?
If we can't settle on a better structure, i think i'll reimplement
array_agg without the performance improvement, using deconstruct_array and
unchanged accumArrayResult & makeMdArrayResult.
you can check it? We can test, how performance lost we get. As second
benefit we can get numbers for introduction new optimized array builder
Regards
Pavel
Show quoted text
Thanks,
--
Ali Akbar
Re: Function array_agg(array)
you can check it? We can test, how performance lost we get. As second
benefit we can get numbers for introduction new optimized array builder
array_agg(anyarray) with deconstruct_array, unchanged accumArrayResult and
makeMdArrayResult:
INSERT 0 1
Time: 852,527 ms
INSERT 0 1
Time: 844,275 ms
INSERT 0 1
Time: 858,855 ms
INSERT 0 1
Time: 861,072 ms
INSERT 0 1
Time: 952,006 ms
INSERT 0 1
Time: 953,918 ms
INSERT 0 1
Time: 926,945 ms
INSERT 0 1
Time: 923,692 ms
INSERT 0 1
Time: 940,916 ms
INSERT 0 1
Time: 948,700 ms
INSERT 0 1
Time: 933,333 ms
INSERT 0 1
Time: 948,869 ms
INSERT 0 1
Time: 847,113 ms
INSERT 0 1
Time: 908,572 ms
Total: 12776.83
Avg: 912,63
with last patch (v10):
INSERT 0 1
Time: 643,339 ms
INSERT 0 1
Time: 608,010 ms
INSERT 0 1
Time: 610,465 ms
INSERT 0 1
Time: 613,931 ms
INSERT 0 1
Time: 616,466 ms
INSERT 0 1
Time: 634,754 ms
INSERT 0 1
Time: 683,566 ms
INSERT 0 1
Time: 656,665 ms
INSERT 0 1
Time: 630,096 ms
INSERT 0 1
Time: 607,564 ms
INSERT 0 1
Time: 610,353 ms
INSERT 0 1
Time: 626,816 ms
INSERT 0 1
Time: 610,450 ms
INSERT 0 1
Time: 614,342 ms
Total: 8842,7
Avg: 631,6
It's 30% faster (i tried varlena element - text). I tried several times and
it's consistent in +/- 30%.
quick & dirty non-optimized patch and the test script attached.
Regards,
--
Ali Akbar
Attachments:
array_agg_anyarray_notoptimized_dirty.patchtext/x-patch; charset=US-ASCII; name=array_agg_anyarray_notoptimized_dirty.patchDownload
diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml
index 7e5bcd9..f59738a 100644
--- a/doc/src/sgml/func.sgml
+++ b/doc/src/sgml/func.sgml
@@ -12046,6 +12046,22 @@ NULL baz</literallayout>(3 rows)</entry>
<row>
<entry>
<indexterm>
+ <primary>array_agg</primary>
+ </indexterm>
+ <function>array_agg(<replaceable class="parameter">anyarray</replaceable>)</function>
+ </entry>
+ <entry>
+ any
+ </entry>
+ <entry>
+ the same array type as input type
+ </entry>
+ <entry>input arrays, aggregated into higher-order multidimesional array. Rejects NULL and empty array as input.</entry>
+ </row>
+
+ <row>
+ <entry>
+ <indexterm>
<primary>average</primary>
</indexterm>
<indexterm>
diff --git a/doc/src/sgml/syntax.sgml b/doc/src/sgml/syntax.sgml
index 2f0680f..8c182a4 100644
--- a/doc/src/sgml/syntax.sgml
+++ b/doc/src/sgml/syntax.sgml
@@ -2238,6 +2238,11 @@ SELECT ARRAY(SELECT oid FROM pg_proc WHERE proname LIKE 'bytea%');
array
-----------------------------------------------------------------------
{2011,1954,1948,1952,1951,1244,1950,2005,1949,1953,2006,31,2412,2413}
+
+SELECT ARRAY(SELECT array(select i) FROM generate_series(1,5) a(i));
+ array
+-----------------------
+ {{1},{2},{3},{4},{5}}
(1 row)
</programlisting>
The subquery must return a single column. The resulting
diff --git a/src/backend/nodes/nodeFuncs.c b/src/backend/nodes/nodeFuncs.c
index 41e973b..0261fcb 100644
--- a/src/backend/nodes/nodeFuncs.c
+++ b/src/backend/nodes/nodeFuncs.c
@@ -108,12 +108,16 @@ exprType(const Node *expr)
type = exprType((Node *) tent->expr);
if (sublink->subLinkType == ARRAY_SUBLINK)
{
- type = get_array_type(type);
- if (!OidIsValid(type))
- ereport(ERROR,
- (errcode(ERRCODE_UNDEFINED_OBJECT),
- errmsg("could not find array type for data type %s",
- format_type_be(exprType((Node *) tent->expr)))));
+ if (!OidIsValid(get_element_type(type)))
+ {
+ /* not array, so check for its array type */
+ type = get_array_type(type);
+ if (!OidIsValid(type))
+ ereport(ERROR,
+ (errcode(ERRCODE_UNDEFINED_OBJECT),
+ errmsg("could not find array type for data type %s",
+ format_type_be(exprType((Node *) tent->expr)))));
+ }
}
}
else if (sublink->subLinkType == MULTIEXPR_SUBLINK)
@@ -139,12 +143,16 @@ exprType(const Node *expr)
type = subplan->firstColType;
if (subplan->subLinkType == ARRAY_SUBLINK)
{
- type = get_array_type(type);
- if (!OidIsValid(type))
- ereport(ERROR,
- (errcode(ERRCODE_UNDEFINED_OBJECT),
- errmsg("could not find array type for data type %s",
- format_type_be(subplan->firstColType))));
+ if (!OidIsValid(get_element_type(type)))
+ {
+ /* not array, so check for its array type */
+ type = get_array_type(type);
+ if (!OidIsValid(type))
+ ereport(ERROR,
+ (errcode(ERRCODE_UNDEFINED_OBJECT),
+ errmsg("could not find array type for data type %s",
+ format_type_be(subplan->firstColType))));
+ }
}
}
else if (subplan->subLinkType == MULTIEXPR_SUBLINK)
diff --git a/src/backend/optimizer/plan/subselect.c b/src/backend/optimizer/plan/subselect.c
index 3e7dc85..8fc8b49 100644
--- a/src/backend/optimizer/plan/subselect.c
+++ b/src/backend/optimizer/plan/subselect.c
@@ -668,10 +668,16 @@ build_subplan(PlannerInfo *root, Plan *plan, PlannerInfo *subroot,
Assert(!te->resjunk);
Assert(testexpr == NULL);
- arraytype = get_array_type(exprType((Node *) te->expr));
- if (!OidIsValid(arraytype))
- elog(ERROR, "could not find array type for datatype %s",
- format_type_be(exprType((Node *) te->expr)));
+
+ arraytype = exprType((Node *) te->expr);
+ if (!OidIsValid(get_element_type(arraytype)))
+ {
+ /* not array, so get the array type */
+ arraytype = get_array_type(exprType((Node *) te->expr));
+ if (!OidIsValid(arraytype))
+ elog(ERROR, "could not find array type for datatype %s",
+ format_type_be(exprType((Node *) te->expr)));
+ }
prm = generate_new_param(root,
arraytype,
exprTypmod((Node *) te->expr),
diff --git a/src/backend/utils/adt/array_userfuncs.c b/src/backend/utils/adt/array_userfuncs.c
index 831466d..35f427d 100644
--- a/src/backend/utils/adt/array_userfuncs.c
+++ b/src/backend/utils/adt/array_userfuncs.c
@@ -16,6 +16,8 @@
#include "utils/builtins.h"
#include "utils/lsyscache.h"
+static Datum array_agg_transfn_internal(PG_FUNCTION_ARGS,
+ const char* fname);
/*-----------------------------------------------------------------------------
* array_push :
@@ -476,6 +478,19 @@ create_singleton_array(FunctionCallInfo fcinfo,
Datum
array_agg_transfn(PG_FUNCTION_ARGS)
{
+ return array_agg_transfn_internal(fcinfo, "array_agg_transfn");
+}
+
+/*
+ * internal function of ARRAY_AGG
+ * called by array_agg_transfn and array_agg_transfn_array
+ *
+ * fname - function name (for error reporting)
+ */
+static Datum
+array_agg_transfn_internal(PG_FUNCTION_ARGS,
+ const char* fname)
+{
Oid arg1_typeid = get_fn_expr_argtype(fcinfo->flinfo, 1);
MemoryContext aggcontext;
ArrayBuildState *state;
@@ -489,7 +504,7 @@ array_agg_transfn(PG_FUNCTION_ARGS)
if (!AggCheckCallContext(fcinfo, &aggcontext))
{
/* cannot be called directly because of internal-type argument */
- elog(ERROR, "array_agg_transfn called in non-aggregate context");
+ elog(ERROR, "%s called in non-aggregate context", fname);
}
state = PG_ARGISNULL(0) ? NULL : (ArrayBuildState *) PG_GETARG_POINTER(0);
@@ -513,8 +528,8 @@ array_agg_finalfn(PG_FUNCTION_ARGS)
{
Datum result;
ArrayBuildState *state;
- int dims[1];
- int lbs[1];
+ int dims[1];
+ int lbs[1];
/*
* Test for null before Asserting we are in right context. This is to
@@ -529,8 +544,131 @@ array_agg_finalfn(PG_FUNCTION_ARGS)
state = (ArrayBuildState *) PG_GETARG_POINTER(0);
+ /*
+ * Make the result. We cannot release the ArrayBuildState because
+ * sometimes aggregate final functions are re-executed. Rather, it is
+ * nodeAgg.c's responsibility to reset the aggcontext when it's safe to do
+ * so.
+ */
dims[0] = state->nelems;
lbs[0] = 1;
+ result = makeMdArrayResult(state,
+ 1, dims, lbs,
+ CurrentMemoryContext, false
+ );
+
+ PG_RETURN_DATUM(result);
+}
+
+typedef struct ArrayBuildStateA {
+ ArrayBuildState * astate;
+ int ndims;
+ int *dims;
+ int *lbs;
+} ArrayBuildStateA;
+
+/*
+ * ARRAY_AGG(anyarray) aggregate function
+ */
+Datum
+array_agg_anyarray_transfn(PG_FUNCTION_ARGS)
+{
+ Oid arg1_typeid = get_fn_expr_argtype(fcinfo->flinfo, 1);
+ MemoryContext aggcontext,oldcontext;
+ ArrayBuildStateA *state;
+ ArrayType *elem;
+
+ int16 typlen;
+ bool typbyval;
+ char typalign;
+
+ Datum *elements;
+ bool *nulls;
+ int nelem;
+
+ int i,j;
+
+ if (arg1_typeid == InvalidOid)
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("could not determine input data type")));
+
+ if (!AggCheckCallContext(fcinfo, &aggcontext))
+ {
+ /* cannot be called directly because of internal-type argument */
+ elog(ERROR, "array_agg_anyarray_transfn called in non-aggregate context");
+ }
+
+ state = PG_ARGISNULL(0) ? NULL : (ArrayBuildStateA *) PG_GETARG_POINTER(0);
+
+ elem = PG_ARGISNULL(1) ? NULL : PG_GETARG_ARRAYTYPE_P(1);
+
+ oldcontext = MemoryContextSwitchTo(aggcontext);
+ if (state == NULL)
+ {
+ state = (ArrayBuildStateA *) palloc(sizeof(ArrayBuildStateA));
+ state->astate = NULL;
+ state->ndims = ARR_NDIM(elem) + 1;
+ state->dims = (int *) palloc(state->ndims * sizeof(int));
+ state->lbs = (int *) palloc(state->ndims * sizeof(int));
+
+ memcpy(&(state->dims[1]), ARR_DIMS(elem), (state->ndims - 1) * sizeof(int));
+ memcpy(&(state->lbs[1]), ARR_LBOUND(elem), (state->ndims - 1) * sizeof(int));
+
+ state->lbs[0] = 1;
+ state->dims[0] = 1;
+ }
+ else
+ {
+ //TODO: check for correct dimensions
+ state->dims[0] += 1;
+ }
+
+ arg1_typeid = ARR_ELEMTYPE(elem);
+ get_typlenbyvalalign(arg1_typeid,
+ &typlen,
+ &typbyval,
+ &typalign);
+
+ deconstruct_array(elem, arg1_typeid, typlen, typbyval, typalign,
+ &elements, &nulls, &nelem);
+
+ for (i = 0; i < nelem; i++)
+ {
+ state->astate = accumArrayResult(state->astate,
+ elements[i],
+ nulls[i],
+ arg1_typeid,
+ aggcontext);
+ }
+ MemoryContextSwitchTo(oldcontext);
+ /*
+ * The transition type for array_agg() is declared to be "internal", which
+ * is a pass-by-value type the same size as a pointer. So we can safely
+ * pass the ArrayBuildState pointer through nodeAgg.c's machinations.
+ */
+ PG_RETURN_POINTER(state);
+}
+
+Datum
+array_agg_anyarray_finalfn(PG_FUNCTION_ARGS)
+{
+ Datum result;
+ ArrayBuildStateA *state;
+ int i;
+
+ /*
+ * Test for null before Asserting we are in right context. This is to
+ * avoid possible Assert failure in 8.4beta installations, where it is
+ * possible for users to create NULL constants of type internal.
+ */
+ if (PG_ARGISNULL(0))
+ PG_RETURN_NULL(); /* returns null iff no input values */
+
+ /* cannot be called directly because of internal-type argument */
+ Assert(AggCheckCallContext(fcinfo, NULL));
+
+ state = (ArrayBuildStateA *) PG_GETARG_POINTER(0);
/*
* Make the result. We cannot release the ArrayBuildState because
@@ -538,9 +676,11 @@ array_agg_finalfn(PG_FUNCTION_ARGS)
* nodeAgg.c's responsibility to reset the aggcontext when it's safe to do
* so.
*/
- result = makeMdArrayResult(state, 1, dims, lbs,
- CurrentMemoryContext,
- false);
+
+ result = makeMdArrayResult(state->astate,
+ state->ndims, state->dims, state->lbs,
+ CurrentMemoryContext, false
+ );
PG_RETURN_DATUM(result);
}
diff --git a/src/backend/utils/adt/arrayutils.c b/src/backend/utils/adt/arrayutils.c
index 477ccad..af5622a 100644
--- a/src/backend/utils/adt/arrayutils.c
+++ b/src/backend/utils/adt/arrayutils.c
@@ -106,8 +106,8 @@ ArrayGetNItems(int ndim, const int *dims)
if ((Size) ret > MaxArraySize)
ereport(ERROR,
(errcode(ERRCODE_PROGRAM_LIMIT_EXCEEDED),
- errmsg("array size exceeds the maximum allowed (%d)",
- (int) MaxArraySize)));
+ errmsg("array size (%d) exceeds the maximum allowed (%d)",
+ (int) ret, (int) MaxArraySize)));
return (int) ret;
}
diff --git a/src/include/catalog/pg_aggregate.h b/src/include/catalog/pg_aggregate.h
index 3ba9e5e..2005199 100644
--- a/src/include/catalog/pg_aggregate.h
+++ b/src/include/catalog/pg_aggregate.h
@@ -275,6 +275,7 @@ DATA(insert ( 2901 n 0 xmlconcat2 - - - - f f 0 142 0 0 0 _null_
/* array */
DATA(insert ( 2335 n 0 array_agg_transfn array_agg_finalfn - - - t f 0 2281 0 0 0 _null_ _null_ ));
+DATA(insert ( 6005 n 0 array_agg_anyarray_transfn array_agg_anyarray_finalfn - - - t f 0 2281 0 0 0 _null_ _null_ ));
/* text */
DATA(insert ( 3538 n 0 string_agg_transfn string_agg_finalfn - - - f f 0 2281 0 0 0 _null_ _null_ ));
diff --git a/src/include/catalog/pg_proc.h b/src/include/catalog/pg_proc.h
index b6dc1b8..9273c1f 100644
--- a/src/include/catalog/pg_proc.h
+++ b/src/include/catalog/pg_proc.h
@@ -879,11 +879,17 @@ DATA(insert OID = 3167 ( array_remove PGNSP PGUID 12 1 0 0 0 f f f f f f i 2
DESCR("remove any occurrences of an element from an array");
DATA(insert OID = 3168 ( array_replace PGNSP PGUID 12 1 0 0 0 f f f f f f i 3 0 2277 "2277 2283 2283" _null_ _null_ _null_ _null_ array_replace _null_ _null_ _null_ ));
DESCR("replace any occurrences of an element in an array");
-DATA(insert OID = 2333 ( array_agg_transfn PGNSP PGUID 12 1 0 0 0 f f f f f f i 2 0 2281 "2281 2283" _null_ _null_ _null_ _null_ array_agg_transfn _null_ _null_ _null_ ));
+DATA(insert OID = 2333 ( array_agg_transfn PGNSP PGUID 12 1 0 0 0 f f f f f f i 2 0 2281 "2281 2776" _null_ _null_ _null_ _null_ array_agg_transfn _null_ _null_ _null_ ));
DESCR("aggregate transition function");
-DATA(insert OID = 2334 ( array_agg_finalfn PGNSP PGUID 12 1 0 0 0 f f f f f f i 2 0 2277 "2281 2283" _null_ _null_ _null_ _null_ array_agg_finalfn _null_ _null_ _null_ ));
+DATA(insert OID = 2334 ( array_agg_finalfn PGNSP PGUID 12 1 0 0 0 f f f f f f i 2 0 2277 "2281 2776" _null_ _null_ _null_ _null_ array_agg_finalfn _null_ _null_ _null_ ));
DESCR("aggregate final function");
-DATA(insert OID = 2335 ( array_agg PGNSP PGUID 12 1 0 0 0 t f f f f f i 1 0 2277 "2283" _null_ _null_ _null_ _null_ aggregate_dummy _null_ _null_ _null_ ));
+DATA(insert OID = 2335 ( array_agg PGNSP PGUID 12 1 0 0 0 t f f f f f i 1 0 2277 "2776" _null_ _null_ _null_ _null_ aggregate_dummy _null_ _null_ _null_ ));
+DESCR("concatenate aggregate input into an array");
+DATA(insert OID = 6003 ( array_agg_anyarray_transfn PGNSP PGUID 12 1 0 0 0 f f f f f f i 2 0 2281 "2281 2277" _null_ _null_ _null_ _null_ array_agg_anyarray_transfn _null_ _null_ _null_ ));
+DESCR("aggregate transition function");
+DATA(insert OID = 6004 ( array_agg_anyarray_finalfn PGNSP PGUID 12 1 0 0 0 f f f f f f i 2 0 2277 "2281 2277" _null_ _null_ _null_ _null_ array_agg_anyarray_finalfn _null_ _null_ _null_ ));
+DESCR("aggregate final function");
+DATA(insert OID = 6005 ( array_agg PGNSP PGUID 12 1 0 0 0 t f f f f f i 1 0 2277 "2277" _null_ _null_ _null_ _null_ aggregate_dummy _null_ _null_ _null_ ));
DESCR("concatenate aggregate input into an array");
DATA(insert OID = 3218 ( width_bucket PGNSP PGUID 12 1 0 0 0 f f f f t f i 2 0 23 "2283 2277" _null_ _null_ _null_ _null_ width_bucket_array _null_ _null_ _null_ ));
DESCR("bucket number of operand given a sorted array of bucket lower bounds");
diff --git a/src/include/utils/array.h b/src/include/utils/array.h
index e744314..b4dd433 100644
--- a/src/include/utils/array.h
+++ b/src/include/utils/array.h
@@ -292,6 +292,8 @@ extern ArrayType *create_singleton_array(FunctionCallInfo fcinfo,
extern Datum array_agg_transfn(PG_FUNCTION_ARGS);
extern Datum array_agg_finalfn(PG_FUNCTION_ARGS);
+extern Datum array_agg_anyarray_transfn(PG_FUNCTION_ARGS);
+extern Datum array_agg_anyarray_finalfn(PG_FUNCTION_ARGS);
/*
* prototypes for functions defined in array_typanalyze.c
diff --git a/src/test/regress/expected/aggregates.out b/src/test/regress/expected/aggregates.out
index 58df854..607aeea 100644
--- a/src/test/regress/expected/aggregates.out
+++ b/src/test/regress/expected/aggregates.out
@@ -914,6 +914,76 @@ select array_agg(distinct a order by a desc nulls last)
{3,2,1,NULL}
(1 row)
+-- array_agg(anyarray)
+select array_agg(ar)
+ from (values ('{1,2}'::int[]), ('{3,4}'::int[])) v(ar);
+ array_agg
+---------------
+ {{1,2},{3,4}}
+(1 row)
+
+select array_agg(distinct ar order by ar desc)
+ from (select array[i / 2] from generate_series(1,10) a(i)) b(ar);
+ array_agg
+---------------------------
+ {{5},{4},{3},{2},{1},{0}}
+(1 row)
+
+select array_agg(ar)
+ from (select array_agg(array[i, i+1, i-1])
+ from generate_series(1,2) a(i)) b(ar);
+ array_agg
+---------------------
+ {{{1,2,0},{2,3,1}}}
+(1 row)
+
+-- array_agg(anyarray), varlena types
+select array_agg(array[1.2,1.3,1.4]) from generate_series(1,3);
+ array_agg
+---------------------------------------------
+ {{1.2,1.3,1.4},{1.2,1.3,1.4},{1.2,1.3,1.4}}
+(1 row)
+
+select array_agg(array['Hello','Hohoho','Hi']) from generate_series(1,3);
+ array_agg
+---------------------------------------------------------
+ {{Hello,Hohoho,Hi},{Hello,Hohoho,Hi},{Hello,Hohoho,Hi}}
+(1 row)
+
+-- array_agg(anyarray), arrays with nulls
+select array_agg(array[i, null, i+1, null, i+2]) from generate_series(1,3) g(i);
+ array_agg
+---------------------------------------------------------
+ {{1,NULL,2,NULL,3},{2,NULL,3,NULL,4},{3,NULL,4,NULL,5}}
+(1 row)
+
+select array_agg(array[1.1+ i, null, 1.1+i+1, null, 1.1+i+2]) from generate_series(1,3) g(i);
+ array_agg
+---------------------------------------------------------------------------
+ {{2.1,NULL,3.1,NULL,4.1},{3.1,NULL,4.1,NULL,5.1},{4.1,NULL,5.1,NULL,6.1}}
+(1 row)
+
+select array_agg(array[null, 'Hello','Hohoho', null,'Hi']) from generate_series(1,3);
+ array_agg
+---------------------------------------------------------------------------------------
+ {{NULL,Hello,Hohoho,NULL,Hi},{NULL,Hello,Hohoho,NULL,Hi},{NULL,Hello,Hohoho,NULL,Hi}}
+(1 row)
+
+select array_agg(array[[null, 'Hello', null, 'Hi'],['Hello', null, 'Hi', null]]) from generate_series(1,2);
+ array_agg
+-------------------------------------------------------------------------------------------
+ {{{NULL,Hello,NULL,Hi},{Hello,NULL,Hi,NULL}},{{NULL,Hello,NULL,Hi},{Hello,NULL,Hi,NULL}}}
+(1 row)
+
+-- errors
+select array_agg('{}'::int[]) from generate_series(1,2);
+ERROR: cannot accumulate empty arrays
+select array_agg(null::int[]) from generate_series(1,2);
+ERROR: cannot accumulate null arrays
+select array_agg(ar)
+ from (values ('{1,2}'::int[]), ('{3}'::int[])) v(ar);
+ERROR: cannot accumulate incompatible arrays
+DETAIL: Arrays with differing element dimensions are not compatible for concatenation.
-- multi-arg aggs, strict/nonstrict, distinct/order by
select aggfstr(a,b,c)
from (values (1,3,'foo'),(0,null,null),(2,2,'bar'),(3,1,'baz')) v(a,b,c);
diff --git a/src/test/regress/expected/arrays.out b/src/test/regress/expected/arrays.out
index 46eff67..e80ebec 100644
--- a/src/test/regress/expected/arrays.out
+++ b/src/test/regress/expected/arrays.out
@@ -1521,6 +1521,23 @@ select array_agg(unique1) from tenk1 where unique1 < -15;
(1 row)
+select array(select unique1 from tenk1 where unique1 < 15 order by unique1);
+ array
+--------------------------------------
+ {0,1,2,3,4,5,6,7,8,9,10,11,12,13,14}
+(1 row)
+
+select array(select array[i,i/2] from generate_series(1,5) a(i));
+ array
+---------------------------------
+ {{1,0},{2,1},{3,1},{4,2},{5,2}}
+(1 row)
+
+-- cannot accumulate null arrays and empty arrays
+select array(select null::int[]);
+ERROR: cannot accumulate null arrays
+select array(select '{}'::int[]);
+ERROR: cannot accumulate empty arrays
select unnest(array[1,2,3]);
unnest
--------
diff --git a/src/test/regress/sql/aggregates.sql b/src/test/regress/sql/aggregates.sql
index 8096a6f..a70419f 100644
--- a/src/test/regress/sql/aggregates.sql
+++ b/src/test/regress/sql/aggregates.sql
@@ -322,6 +322,32 @@ select array_agg(distinct a order by a desc)
select array_agg(distinct a order by a desc nulls last)
from (values (1),(2),(1),(3),(null),(2)) v(a);
+-- array_agg(anyarray)
+select array_agg(ar)
+ from (values ('{1,2}'::int[]), ('{3,4}'::int[])) v(ar);
+select array_agg(distinct ar order by ar desc)
+ from (select array[i / 2] from generate_series(1,10) a(i)) b(ar);
+select array_agg(ar)
+ from (select array_agg(array[i, i+1, i-1])
+ from generate_series(1,2) a(i)) b(ar);
+
+-- array_agg(anyarray), varlena types
+select array_agg(array[1.2,1.3,1.4]) from generate_series(1,3);
+select array_agg(array['Hello','Hohoho','Hi']) from generate_series(1,3);
+
+-- array_agg(anyarray), arrays with nulls
+select array_agg(array[i, null, i+1, null, i+2]) from generate_series(1,3) g(i);
+select array_agg(array[1.1+ i, null, 1.1+i+1, null, 1.1+i+2]) from generate_series(1,3) g(i);
+select array_agg(array[null, 'Hello','Hohoho', null,'Hi']) from generate_series(1,3);
+
+select array_agg(array[[null, 'Hello', null, 'Hi'],['Hello', null, 'Hi', null]]) from generate_series(1,2);
+
+-- errors
+select array_agg('{}'::int[]) from generate_series(1,2);
+select array_agg(null::int[]) from generate_series(1,2);
+select array_agg(ar)
+ from (values ('{1,2}'::int[]), ('{3}'::int[])) v(ar);
+
-- multi-arg aggs, strict/nonstrict, distinct/order by
select aggfstr(a,b,c)
diff --git a/src/test/regress/sql/arrays.sql b/src/test/regress/sql/arrays.sql
index fa8a20a..cb00f5f 100644
--- a/src/test/regress/sql/arrays.sql
+++ b/src/test/regress/sql/arrays.sql
@@ -432,6 +432,12 @@ select array_agg(ten) from (select ten from tenk1 where unique1 < 15 order by un
select array_agg(nullif(ten, 4)) from (select ten from tenk1 where unique1 < 15 order by unique1) ss;
select array_agg(unique1) from tenk1 where unique1 < -15;
+select array(select unique1 from tenk1 where unique1 < 15 order by unique1);
+select array(select array[i,i/2] from generate_series(1,5) a(i));
+-- cannot accumulate null arrays and empty arrays
+select array(select null::int[]);
+select array(select '{}'::int[]);
+
select unnest(array[1,2,3]);
select * from unnest(array[1,2,3]);
select unnest(array[1,2,3,4.5]::float8[]);
Re: Function array_agg(array)
Hi
My idea is using new ArrayBuilder optimized for building multidimensional
arrays with own State type. I think so casting to ArrayBuildState is base
of our problems, so I don't would to do. Code in array_agg_* is simple,
little bit more complex code is in nodeSubplan.c. Some schematic changes
are in attachments.
Regards
Pavel
2014-10-25 15:58 GMT+02:00 Ali Akbar <the.apaan@gmail.com>:
Show quoted text
you can check it? We can test, how performance lost we get. As second
benefit we can get numbers for introduction new optimized array builder
array_agg(anyarray) with deconstruct_array, unchanged accumArrayResult and
makeMdArrayResult:INSERT 0 1
Time: 852,527 ms
INSERT 0 1
Time: 844,275 ms
INSERT 0 1
Time: 858,855 ms
INSERT 0 1
Time: 861,072 ms
INSERT 0 1
Time: 952,006 ms
INSERT 0 1
Time: 953,918 ms
INSERT 0 1
Time: 926,945 ms
INSERT 0 1
Time: 923,692 ms
INSERT 0 1
Time: 940,916 ms
INSERT 0 1
Time: 948,700 ms
INSERT 0 1
Time: 933,333 ms
INSERT 0 1
Time: 948,869 ms
INSERT 0 1
Time: 847,113 ms
INSERT 0 1
Time: 908,572 msTotal: 12776.83
Avg: 912,63
with last patch (v10):
INSERT 0 1
Time: 643,339 ms
INSERT 0 1
Time: 608,010 ms
INSERT 0 1
Time: 610,465 ms
INSERT 0 1
Time: 613,931 ms
INSERT 0 1
Time: 616,466 ms
INSERT 0 1
Time: 634,754 ms
INSERT 0 1
Time: 683,566 ms
INSERT 0 1
Time: 656,665 ms
INSERT 0 1
Time: 630,096 ms
INSERT 0 1
Time: 607,564 ms
INSERT 0 1
Time: 610,353 ms
INSERT 0 1
Time: 626,816 ms
INSERT 0 1
Time: 610,450 ms
INSERT 0 1
Time: 614,342 msTotal: 8842,7
Avg: 631,6
It's 30% faster (i tried varlena element - text). I tried several times
and it's consistent in +/- 30%.quick & dirty non-optimized patch and the test script attached.
Regards,
--
Ali Akbar
Attachments:
design.gifimage/gif; name=design.gifDownload
GIF89a0 � � ��!� � , 0 �
# 0# ( 1 )"-%<&1 )5 +8"
'+4
8 = ##''++22 ###+++ -;#1?66 33/<<$333996;;;(B3R 6X 9\ ;`$2B(7H*;M,>P.@T0CW1EY3H]"Bl$Fq%Hu&Jx(N}6Kb8Ne;Sl>Wq?XrE
J N O S![$`&d(k+r-z1@@&DD)MM.SS2YY5]]8aa;ii?CCCIIFKKKSSSXXT[[[@YtC]zEa~mmBqqDzzIccchhclllsss{{{)Q�-W�-X�.Y�0\�1_�1a�3d�Gc�He�Jh�Lj�On�Ro�Po�Qq�Tu�Vx�Xz�Ds�T{�Y}�aZ�\��]��Y��a��`��a��c��e��g��i��j��m��n��q���4�6� 8� :� >� ?� @� A�"D�#F�$H�%K�&M�(O�(P�(Q�*U�+V�,X�-[�.]�0_�0`�0a�2e��M��P��S��V��Y��]��c��f��h��s��l��n��q��t��y��~����������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������� ��� � � H����*\�����#J�H����3j������ C� I��I��R�\�����,O��I����8s�������@?� J�h��H�*]��� nP�J�J����X�j������`�� K�,8;M
]�6e��p���K�i��x������_�g�� <�nD�+^��+Z��"nL������K�s��'S�������A� (�*���� p����XNHC`���N��{v���`@�U�[Z���-5�p�n��z ���G��=�y��� N������:�~�J��������8un�Q�� MT��1A�@�}���� �d�@�*�Y\�u��j���7#���P�H��,����.���*����2�H��<������Zk{�@ -��bB���6Y�� ��mc��$�d�OBu�}�L���H*�����@������f�o�9��Z��d��L �����@�[: U�V��-�� ��X����S� -�d�� ��r��� -�� �Pi�i_�m���
��^q@6P�y��Ps���r��4P�,�V�r����M6��U��tZ���2K�n��V���Q����"����J%6�n�����*�������|�8�T��E�g� �oiX8���a�r-QaC��J�*7�\l-����\ K%pV�l��X����������\P�_�1�{������(��DMR F'M��%��X*N�3�Tw���Jg�5EHo�uFL���]�m��hTv�l'v[m��v�t�m��v��6[y � >}.8Nn��{u��r�S@�����;dN��Q���+���J!y:����/�P��}wn����0��^P��n�����o8�/���o���'���@�����+<��2�o�B �X/��1 | 3~��` �=vL"����o������������ � �� ���w0����7� ��{��1��=b OP�2V n/���A �q���" " �2(������s �A/�0 �S���@0/� ^2� � �0x���
ExBe� }?�����-a2J���|� �<�~�c� ���6���pdcx�������������$� I��)�}MAc ���
Z��#��M������W� �a-�$���
���� ��I\�����$���I " >�d-��� ��\%��� D�~� ��$��G�'�� �w�,R~s�#A��m�d ��� ���r�f}j�/�7
L��_��<�AxlX�1�W�� �(@ ���1 {m@2�� #x+�� ���j���� 2���x`��
���-�Q�V\� ��
�3�c|�H8=��b�at��9= t|�d�"�I��u���C1�p�lz3 ���S?5@` �`H8�p<D��S�
�)<�R��f5���)��=U�Z������{�� �J��Zq}f��"�aNb���� ��]�/�C�rP�=���[e �wK �o�m�X ��! |�� �������eD� +[ �"� ?P�0` �5" ?�&Z���#vQ��T ��� 9k!����Zw�8�+v�kH� 2�=Ac1�yXVs 0��h��I$@ �U@ � ?(`~���@���r~���� q��N"�Ar�� l�~H���ByL�a`� (�$z)���w� !�w�\���L�� <� B~ ���������#7���U�r 2��#)��.]���& /���n!� �� �X����l� ;D
�(]����r@�rI���L�5����p&���l<����\�����=���67��� ����h��G>\���v@ �����}gn����6M�R��c ��E����� �Y�U�������w��^��n�N�
0��bt�� ����a���������i�1;k�`����m�Y�l�.0�085����N���Qe����������e�${l�N��F]�u��� /d����y��{tI|��| ���3��Dh��9 ��# �|W/�|���!:/�U
��*�| zo !
�x���� ��2P:�t�����W�d�5�,�M������E) �<5��#N���� �$��
��= x��I8� ���ny= "���* ,��� ��D'Y��=����_ �I�����0��� @�f�����x���l����6�� �����p��!�g g �o�H|�sx�d��� R���zR���>�����s1�W�S���1@� �����Dc
!?C��q���#��b��r�� %�� $�!�������$�����!�?}F ��� �?�W���|�����s ���@nJ�z�1���� 1 j�
D� ��'=^pf��: [�n��m�l ��Du��Q��>'� ��m�m1 �GP4WV��q�q�� �C'dA302T��a�fA|��^�H�@�� m`�� O9� ]&�72A�s����0 >Q�L�R�3��0���q��C8�=)<7��cQA��]4 �=I4<5�r<�xB1�R10 ����#]� n`�*$90�s�<OTq��R� :E���E<!%��#|&�����c��J��J�$K�% ��`��I��a�����4a��6 ��H�2~�4M�dM��z�> �c�D~��al7 ���@�H?���y'f`��L�1~�M�����Z�Po! ��� 1zXpxr��?�p��O�<�WWr�f#V��G���o(��#y�����TK%�_� ����� AE � y48q�MQij���`N��P�R9�TY�Vy�X��N9Z��^��`��������#Z���l9?j�Fo�?q Gm)?uiTw�F��pAmn����#q�%A�K�p&� �����������0��c���&+P��Y+@k�F��)��ar"A�KSh�qo � ���#�i��rQ��8�9��I� �� *�� �k�iP{�����9��n�k�������N3�i����9��� ����9� ����� �@j��4U��q 0�(�,W�
W�6�Y���p��k�3ay����� ��U�<�o��Z�,�RX@ V��\C�c��l
Am�i8 |:�����#��iA���?r2?�#Gj�H��J��E��GJg�
Uq��2.����!��:�� ����������=+�q*$P]T<��Q*�<��!��zu���E��d�GH<>:��B�B%�r.�R��z,��,_a.��R�$�0��j'��.�
,��������,��3Q/S"��
�b/TA,h�X@ P�*��*@/�� N�� ������m
1 �f:�W
;tA,�B0�Ck�q�,gCw�A?<� E>��*pU��<'D -� � @V���� 8TO����<�P ` G�����^�D��STE��E� �jB�c��8Fj�"��KQ��0����a �� �
"k����(�P" ��
��Z:+������qQ8���<+=��>�@;��I��z 2� nH���n#9W'pP���c�"�BL��R3�-�S�Q�Q�)� ]�3k05���9:����t*�N� �d��tC{=��O/G �#P�SP��c �P
U ��NWQ���Q���R�CRU<��R+�Q~(u��P�6U<;� ��SXg�c�"1�I��&���a���1�2�^z�Z�I\:L[B���g�d�k<�VeurLH<qE�O�G|����C����C�d��"��y��l�����5��Am�Q�2q��k��e��{R�VQ���#�"�f�s!�������H!�����Vl 3 7�i�I������� �M���z�!��fn�B�`��s� +.,$�Km(\�
��}� i� l7���K� �K&��
|7 �%L�%Q�0�l3� �Ga��N�5<m(����Z|�W� q�g����
�K *� �Y�o "�0�` �$� G[m},��n� "a,`�� b�� "���1`��GL�-8�#�j
I�w ^0� �&��^��� ~z����1��� �&��� ��'11��\����� l�@�<������ ��3�� � ul� �E�SQ � ���� �� �!��� �0 �� �� �� �q �A�X!�A1�
' b Q��� ���� �!3�s��� �=������
q�6�&A�� �B(z%Rr.SB)Yb&\2%_�(+ db)h�&l��&pr rR�u���!�
W���
��*!-�O2���*�"&��`����]�V}���0>� �}�?qf_���\�U�F�X�]~���;>m}n<8�T��k=<R���� ���>a�2� Z��}��}�c�5j����j�s�s��*<~�=T���+��j���*0 ����7�G0P� :>T
�V�C��<�@4A���F��}A�0 ���fy]f�� �A�5 ]t� ���M��wM�v�{���we������}�Mv�H�����_��$��oK�La�����-�uJg��d�
n���j��>�����3�������vU��}������<�P���=�_D� ��a`�� �|��L��&� ��D���L� �)i����*~���*P���
:�}o��8^���1��}�� ;A~���F>��V*��a�<�MO~�<�T ��� ���dj]~F��,�nn8�^��Q1 t�ki.Y��o��^�c�->�������3�������^p�� 1@�g��4i���
�v�
�>��^��~�������������� ��>��^����^l��U��aV����^R�m ����������� ������T�0k<�L����.�nV� f��>�����������^����i�V�@�^� ���`3�s�hn �q����� �p
���
�
��
@�� � �N �
�n � �� �
��
��.�QU� "O�KP -O�&��
�@�0�#P ��� �N
�>)��-/�%� �~�� �L �0� �����0�� �U? �n %�
�N0�<����B_���BR�F�@�Q��3 P���@�=�������
"���~��O����K���?��`�P��� ��a@�@30l���Yd�w���>�p�~/8��O��� �/ �_ ��
�P�2 $���~ �#U��j�����G��O�������� �O��o����A
�N
��p��?
�o
�� ��@��������N`
] T� ����R� d`
(R��@�*�A�N2tQb � F D���� T< $HQ���
?5|1����lR�S�N�=w��T�P�E� E�T�R�M�>�U�T�U�^��U�V���~V�X�a��E;� �m���W�\�u����6���� �k00E;�� F�X�b�� ?�,��d��#_6�6�f��= Z4���M�F�Z�j�U+������c��m���[w���� \�p���'��|�����?� ]�t����g]�v��� ^<���������z������\�X������_�~��U�P6�$�@D0A � �0B '������B
7��C?�C�@$�DOD1E�DTNE_�1FOd1�o�1G w����x2H!�$1 a+2I%�dr�#_k2J)����'+�2K-����+-�2L1�$s�/�+3M5�d��3�j3N9����7��3O=���N<�4PA����A�E4� 5KQG �tBF���RK/�o��0��SO��T�OG%���B��TUWe��P[�5VY�BuV[o��V\w��T]{6�H�Xc%�Xe��3Yf��6Lg����(��6[m��v[o���[p�%�Dq�E7]
�U�]wd�]y��U{��7_}���_�8`�&�`�F8a�f�a� �8b�'��b�/�8c�7��c�?9d�G&�d�OF9e�Wf�e�M�7f���wf�o6�U�a���:h��&�h��F:i��f�i���:j�����j��;���k��;l��&�l��F;m��^��6{.������nn��{n��������;p�|p� G�o� _���g<r�%�|r�+��qn������:t�G'�t���zM�Og�u�_�=v����j�\}v�w�������tm^q�@>��`�m�o�yn�/���}�% 4|F�
��� �P~� � :� n�������{�� _yn�w ��}��x���F�������hh 5��?���~����f Z<�( � ��q����(A
�t�s���<���wH��&�A�h��B
��i�b}?��y��[ m����C����0t��E@��iT��WD��n���[@��Ex�b�/���,��R\"���F'��g(l�
����Q# \ "��������\������ &c����m���6�@��M0 �p�&i�I���hT �w�D
��$��.���mL >S��G�
2�wi�� ��Ba���D ��G����&4e�h"�i�C]YGL�q3i� �7�&N��s��T�9����0�=�8���n
������L�e��O���� z,�i��R ��������� �7Q�V4 ^���>��|p��A�hGA:R��4�'%)JM�R����*�iP
Q��t\
��U�`� ��2��0����^��AQ�����A]D�� �1U�V��W�jV��U�v��_kX�:���S8���$��V����o�k\�:W����w�k^�ZWg��HP��T���pT����B �e`�Wm� {� ��SeC |�.v���T�A� ������ ���0�[�~A��
C �1���6 ����d\T�U�0�B� ��,UW ��f���� ���a\����_g�������}+3��^����Rl�KV�F�����V�{U`�[�o~l��2���]� <` X�m D ` �a�6�X`a�� l5C 1� T 6 $�� �eXk�� ��~���I���@d��I@" ;��D0�VT��Zip�'Gy`�P$���[���g�wH���|�������^�� c5�_hC� �
]U�P�@�m�l8������
m�A b@U�����%���� �R`���,�,V3���V� �@2(�3f�$ �! � d ����� �F ���$Z ��*`k}D z�V2@j���= =�8�I@��l ����[���e�y��^kT�j�`��X+�7b���������1 5�� 3XO���_��(j������5���O�/�*�����},U#��p �c �p1�g�������5[�BO���yUs ]7����
�
�*� t7�"�m8�[��N�^%�m[�K�8���z��u�w��x}F �</�CIh.0(�7���������]�s�z~��U���9@$�ZmE" fP+� �m ;" 9`���}��u��Z �w��o�A �V}o����6E��v� l�6�Y �G9��_�����@� �c�!Ho�����(}� / �K� �[ 1�����
�� �=F��
���� ��{�{��w����]-�j�m��-(������G�j���p5��#q� bX�����s����B?���:��S
���eX+T+D0� A�@��4���s
P�1@ �R�L 6�)��1 ��h� \��s0��!dB���$,00�&�B�0
�)��v��w��0d
5����xh�1d���0��3\2l�:�4�9,0;����C ���P�?4D���
��xBa@��H��I������B�L��M�:d�7�.��pxyDJ ���/e��� h�|:���?S��RFH��pp���/K�;Lt+��D�����+a ��k�����b<��� ��c�7Ol�1�����6��S�*��� X��"-�R.��*�s���B-/� �w�R���r9{���2��j�����* 0� w 4��/��,��G�K�}/�4:�
���DKE�
��� �*a,
�:�c+g�6i�= �E8�e=�@�CX+E ��$�m#�I��C0�?��3< �\S+h�� X+��<�Z6��I�����C <�?ml�z8 o��G
��p 68*���0�� GT$+��KY�*�\?�RI�`Il4��d+a+ ��+cd/g�1�K���*�
5��Y��(��� 6?��K�R� ������
��� �5�������pPCS���� ��A�-XDd �_0����p ��.R4�T�����8��b��R0@�_����5x73���*���p�Q��^��1� e��d�,�M���`��8�
P����h\+ �1 3�
0�
��S+I6I8�g�J'C6�� C� +��� @w�=��� ��<h+��3�������2dS���m+H�5�����2(
3�(�:1p�t{�?�� � <�7���XsAs��h�>�
q�������L��7 _��2$��-�����/6u��r96�4�"��������zS��@����bS��/�<�������6�
���@ �m�6��6oC� ��q+�s�<u v�����X�V�7$xR��7 �3`U2���<�I1b;� �2� ����6K�Ts�0B(� ��2�2X�=�=��(�<��Re�j�g5���?(�P g����
��������@T ���/M d ��(O[!H������W@�~�J������(L�D���?�\�zu����v5
�x= ���0���U{ .�� t
;x��%��5��E��U��e��u������Y�u����%YhX��>������R�������q�
X�-�oA@�0H��e�od�1���������rX�y u}
b������(Z� ���B�m����a��y�y���U[�:�����
7H[[I�0i�%��_����W��X� ������v�(��/�A�`;0!t���.x��
� �����%��5�
��
b�Ra� ��1������x"��]m(��{����
�=���
�}
�4�������*�E v=����
^,�����e]����]��8\����^�����K�=��d��� �\rbr��!\: ����
w� ���v�]��Z y���*��u �����*4dX�����\�R.�
�ap�X 5 d�E_8:��p@-�"���3e�-���
������B��2,��(�".�2��b,8 ������� ��������x_���?j�Ez%Uz V��Eb$�q$HB��9�@Z$�y� h���9���i�7>cWR�B� n0cn@����5�$���)���cX�B~$0`Y���� [���p���^�H\�(��_���_���6�Dp =5?T�������'��&iz�\��:���A^ �8�#�U
�B��|��
�a C�HeVl�7���XV��5 <��08*�{?�4B{N�2��b4/p4H�4��4���������p��pX�5��������_����.���,@��a'PA��������f���8�����8��� ���9���2�1( � Nr~��JN����S�^�a���d�9�
:,6:��� 9�L�x.��J,���Z�(���'��������W�%�1�e���x(
��g&�U,�mT�4� �SAU? �*�F ���,����Nv
�%���f�T��@��>���0�����k��k}"�m��n
60�|HE�k1��a`35�[) e��E�����6
��`h�� d�0@ ���a�������� ���_PZ�.��* � � �]z���Xn��l�`��&�r17����i��u��F
����=����w��P�^8���nu�m���������F�����z n�������q[�����������>D� �P�s
�r��@nd
�vp�6/�|������a����Q��iqw��y��V�w l���{�� ���f`(����o�X@q�v�����q ��*��+�*!'��}
�Nm/
��t �������"c�! �� n �� �9��1 ���i�����Y�8�J^sn��
��6 npd ~dB/���r�r,��J��I���P�]������M_���y#F��9�"���.��0� (�WG#5R�ih#d����K��_v���r� �x\I��P�r(�RO�'��Lg�^���
9��v�vi�B�_tOep.)���3� f���b��v6]������ y.�%�G��)_Xe��:�<Is�*!�� ��p�� ����B�~
/LnIv������Aj�u��g �^��S�������o� ���E0�r��k���`�� �:���`@�? /$���
��uQ��� ������ 0�����y��B,��,Tx
yx�8�1���Z��/��Wen�3�����
5@��X�4�� �e�
w{�
y
��-4y� � h���{�����@����O� �
�{?�A������O�v��������z �"��z�t�G
�� w��6 ��t�`�pz*�����b��00��W��6Xv����T��_
�vm���z`���w���g
0���&~kA��
q���mHq���n�j�����7��G��� ���w��o
d��@�R�~���rx4� �V,h� ��
2l��!��'RLh' ��7r���#��"G�,i�$J��~��A,e�z�bPp�&��:w�������B�-j�(��Js�q�kLHtv*R�j�*��Z�re(����b��l�k�
_�N��� �a����k�.��H�����o>0�(���(��UL�Xj��)�*D���� L�\�r�� _ �H�3j�`��v �m��|�������m��7��9�/ Tx�vV��x85�K6-�`�WX������L����`��� �
U�'�@�*����ph �z%0@P|"}� tJ|�7�P��Di �7P�9 A�� E!�W=� �Fn*���2keT�8����@0�����=��#�;7�^A��F"�M:�$�\��@6pe�h�%��]@��PC 3x�l���/�(#�9��f$�u�y'� �gQy��\�]"��IA�(1Q�)��"���BIz�B@ �lRB ���Iz�!4J �NZ)+�@�id����@� ���q� H��$Tr^�%p���jX@ ��R`%�b�����j��Jk�{k����D#lb
(
4�B_��.����nH|�{ �:��@�z�%�^7�* 4��(O,@R4:(0�'�2�0���j-'��{�.p
�
�R�� <a�)"@If�B��,� 3�50�@�M�_� Q(��)����O��~��k���J=5�U�6��V�t��]{�5���5�e�}6IX���H\��6�qw 5�u�}7�j���aS�MA��=8� ���%��������K>9rzI9��k�9��{ ���}>:���~:�?�.z����:��������~;��������;�� ?<�y��W��+�<���~|��K?=���}��k�=����=R��?>�� �Q���>��/��Q��??���o����?����T�(��C! �@�P( | #���K�� �FA}d�� ���6������ S����,|! c����6�� o�� �0�<�!!�� ��������!q�N|b���)R�yR�"��+j��^� �(�1�.�d<#'g�4���g[��(�z�q�v�c)��=��]u�# �?
������E2���|$$#)�IR����$&3��Mr����$(C)�Q����<%*S��U�����|%,c)�Y�����%.s��]�����%0���C�����1����!s��|&L �i.S���� ���mr����&8�)�q����<':���u����t'8��'k^S~�|'>���}������?��.z��}�(B���2���� ����:���(>�i>�b�� �(H?
Qv�O ��7P�Rm���+�FKa����t�6u)Ne������:�)O����u�F��F�w��2��N}�:G����i�P�*V��T�Y��Y�*X� jTT���-��Q��S�h�*X�*W����,*Na�ll��&��ml�Yla����������\�Mj
� �^��M�n���6!�[p��-�hI���6��-�a�M�n�����6)kY��a���F`[��j��n d��V �
���4H����,X�j���=�P{�M����x��K����nu|�Mo9����n�
�Fa�[�&V���/~�)�����
� [��nC�[H,7�0ZCX����7��X����6���Mv��(����[j|8 �
�|G �m v�� - ��] ��nv��������y��������<C�������]'a���sy�Z�&/ �S���27�� 3w7 �@�����.{9�� ���L�.o8�t�p���/�y�d����HC����� jq GW�Xp@
^�],@ ��}t�'
]m����\���Y���� ��yXY$v��l�[k-�Z���B j��m\���7@-iJ6����.&0�^� ��E-�\�j�" ���6���hO;��_��ZhC�9��D3���nx���#����|��&�NI�S�S�3���8�Z�li�� ?x �
�:�A g���� @b=����l !�u�����2|�78�a"gY�pA �� c��3�q��yn���L&���T}�s���g��^����u�w���R^�� �5��o�����.��u����`.y���
.L� (3/L0�3l�)�����*�)��� ?�l� �������u,��}��(������<��y�ALj�+��C�����w� ��-�8]/{2�~���:�}zN�� �t*zBo� Z'����R
���-p�W�1�����o��ns���s�L�����U��������z6��_���r��~��a ��?����������? � � &�* ;�������^\����_9U�\9`A�D ( �=]NY�Ca`C� �����n It`} � F� � �� ���W�T ��H >���] �a� a���(!����>!�D!M��`������v�����!��!��bT�Q R�:1\_ �C � ����3�C�p� R;� � >�p�/�p�K(b�aJ����C2
C -�� � �")��)�"*��*�"+��+�",��,��,:��� ��.�"/��/�"0�b#L(�� �2&�2.#36�3>#4Fc4NH�5^#x� ��l\�7*c �2|�2����=��$�#;��;�#<���<�#=��=�#>��>��=:�9�N1��1~# �@/�B@5"�/� �A/�@ �B54$7F�DVC8� �#2f�/(Cn��0�A �@ �0�@ �H26�/T�I�$2� ����H�dN*CA dB.�@(�
@@8TC6�3�$0�CC��S�� ��G�#�B/�@K*G*c6�2�� ��0Db/<%9��_��<2C;�%;��:��$�%^��^��^�e_��_�%?&a�=>[�N@��@z#W��/�$K�Y*d8x$2r�B2�#Gb&H�d `�r$c1Xfe� $#K*�F��j��c�0 KC2~%0�N~�n&c2P@ 2�f ��q �0��2<�D�S~e2��X�e8�A $�ez�8$&^��a~'xf^�'y��$�#1Vb 5�5� t� 1P@ �C�c ��X^�/�@���G*����A/�X�@ (�h"�1X�{R&3�A ��k�f2*h�Y:@|�g}&cn�&X��2�� gn ����e�� 7j����"�X 2"�D:��J�9��:�'���gh �#h� �# @���!@�$)�Nb����{�#�F#f�)�z�P:@Q����)�F#w��w�gl�#�� ;z�D�:F���$t�\�:
�N� @; i��@ �A�j #��!��#�!�@ ��$@B �� @�:*�H�^ ���D�:2B � ;�@ $���*;&A��:��:J� t ;������J�bbS�.k92��>kvB��n����`�#��@;z@ 8�$DB |@;>#����#$4�:v������j �N�l ;>B � ��"L� ���6)�j+;.i��+�� �� ���$���N����#@ lk�����X�y ��$�������o����ig*ci��2��2���}&M�l $2���P�(�2��p�$�.ek"cL�m:�g�h8��2����t��~H(WnI�����A�����#m& �&�2�/�A~����� �7����;^+<�m�n��=N)��x�#�Z�zF���!��l>�,4���*���2 �1d#��j�&�&�2�Gr�m�f����Dg ��/���� x$t��X��X�p@�h/0#1��h�K&$�^��B�$"#�����@�J�����i���n/��#���3T�0 k��O8��
� 4�F4$��i:��:c��$2�}�n5xe �f�^��^��S��$������ef#��K:@1 #���0|�d���� ���
4E Hd����H���N��:p/ � ��'e�02�� @��B8 ��:��~�#�������fj @�;6��B<�A H�;��p� t��@��c �:� �c �*�~@t� n���t1 "�#b "���I��l��C/@@��X�/4z��b��q�o$���6��*r�J2��$2L/�����"��c�V���� �:>) �� �@� �c���Bq'� A ��-�� @����:�j��$�$dj.����i��#��# �4��$d1 �� �����:�A l���##����7�30��0���� �$T�� A x�<f� <���^*z����!���������@8�!�i"?�D�o�:+%/�E3����[^��c��k� !��)W����B�$���\�:��;�4;�� �! B�zk�"s'��������>s2��4Ws<@JC��#8��$,i��4S��!���R�$t� X�uU�%=Ru@c��v ���/��N��o<�$��D��]�5�ft%_�&��[�3(�+;�2;��:"A+ ���N����;>)�tM[��: B� � #0����$L����\���:����cOBR�6f+�k7vl�4���l�0����L�fs�g�v�>��������j�c'�yj�AF d[w�� L���5�F4^�u$?+_��E��& q$l���qa���}�7?�����u����B�M;p����y��D��E�7��~G��g�2��Y�� C� 8���� y�i�+�#3�^;x3�5��-yV8Y�8��8x����8�������� ��p�����
<�%��j@NH@�g��c�+�H,y�;9�Wy>H�G�����_y�k9N C"��[��/� �9Z�b����A��%"� �����@����
4t� ��p��:F<���y�(G9:���;�\��kN/�y�_:�_��SN �������{7��,���zFld� ���D��������r����z>@��s���F=�:��F������� ��?{F���*;�SN�C;�c��P�i�9P@��{>d���]x���;���{��;]�����9���X����N��8��������O8�|_8]�A������r������O<�#c������o<��8]0t�7�� C�C|��<���� �s����`z<]����O�;�D�DW-�R#�gc� 7�7�����|��<�/=�����E�����K����/�H�1�$�� K��O"�Bjd8�#1�d�F�T:���d8��j��n�$N2e���e� e8���*�S2���UZe8`�VreHndG��~r%�N$�#�����<��c�n L��`�S{ln�� 8v17��o�I�;��<O@�A��������92��B�$��0L����
�q"gf �jh��& ��l�f�"�Wvdn��R�&pZo���� �3g������t �?/�V��\~x��j �Gc�0��@�� �n�H�$ @��4P� @&M"@� f" �HO�?��H��H =�@�q� Hs`�����/�pr�]�����Ngv�:�hQ�G��b��iS�O�F%���jU�W�f�zUjW�_��+PlY�g�������v���;�n��U�;a�\c���B����Q�&F��T &xF� ���"���m���w�]eq�[Q������ @!��pl��;&@���/g�;� � V�y<F�6���#@\d� x��}�8�h�C�g�gy�/S�T�a�� =&����K����C�I�xXI��&�a��H��?$I@ I6����P`#'9$� j���0�o#C�/�
���{z����� d���F���QG o��,s �Q�!����#�L27����'��R��l���)��������J)��2L/���� ����l��7���M
���5�L�E��a��$������,T� M�*C-J����z����b`cs�
P��9�I2I-u�1MMUL0IE5K3%
JN� ��|n���|�!/�_�
V�a�������6T�4IAu�uWi���Vef�vZk��V[j�u(E��Q\C���� �)�
q��Q�Q�\�+�LT��+
~i�Q�����m��V ����������� �Z��*q�%���Y.6&`5@��7��B�3�����fS����<
�&�f���� Tp���5s�.Xc��
�����FI��,�)� ��I����e6";!�'A{� �0� ���>;!3��x�#��0@�;� �� >H�=��c�� �?&���0�3 gc &�v~����� n����Z�HsTp�,dfX���7���z4 `�)�����4�I��+5�ER�a#.^9z_�w�f����p��Q�>&_6��J�.bz���_�#�aPc��= c�p��t�/�34c(�.L��X�F � b# QD��H� J��� ���H�A8����%VK:����r��F �����5 Dp�$�%e��Xx��D�N� ���m���Kl" ���$��IN:`�Gl��}��. #`�%`�O��8�D���b�p��uu\�nT� ��)�����G��J_8�����UC�1F���*)cp�R(0�5�4�� � �Q��.�� h�0 �4���� B���fnpC X�_$` ^`CHi�GN �@F%/I2�����)&2~1 ��i��� 8~y����0��& ����p�x��I,H���Fl��� ��� ���@L�C ��$0��� N� ��E� �'hn/���ur��A�F���x&$x�I �E�@�!�W2A K&�@#Q�� �cH���\g�:
�Fp��P~��_�)�����4F�J���]t�*�N5���mf l�&J��T���+ D�Mb+�4D�@��gu�[m��I��)�����S����g� ��p��M1*U�JU�*6��Mlc��4�Fj4�J���Y�n����ST�#q@�Ps<�^�Z���Rv~ �j��X+��I�5�m�����
U�m�R����7XhS[q����~V*��i��Y���
���mW~qSq�a���we�@���+�}�p_b\��I��uo��D����O9VPZ��V7Q�m�9( �^����5���{`E
�[�S��� �0Q&,� �e�����/�aW8�$&q��A���DJ�����E�=��8c � �1yG�c!�(@�qWlz�G*���1FG$dP��D�rx�z��� ^Z����sB
i��j2�v� 0 i�q�����b8j��������!�BB ��# ?�F��/�
�%��cf���������
�Vt�g���6Jt=�;>����*G��� 7������� �������.u��0�[� v����������s~������18�,;�5�K��Xy!���v� ,���n����������VK��r�+ �v�Y��n����������`j�e4z7���*��F�&/=�p�/��
w��! q�O�����/~�P���xYZm(|�����s�ha$I�)�Q���;p��5���q�s����=���� t�}��p 6|��i��/*��G��kv9������� ���u������d����>v��}�g�o{�� w�� �s�{���v��}�w���k��X���0��6v�t���P#O���q ����[g�t,��E�W�D���A z��^�d�Ex��[0��Ra�� o�<*�N�ai�{����S=M������ ��7����|��"���f�s���P��icG;�)��:�{ ~�����?�9v�����w���� ��a��7�S^O��k���h7Tl������
0U�/Vz�
��� �!���� `��� <O
� 0��N�DV/���+�oH�O l���v2/w�Od�7:�.dCb>#:��Y��hI 9t�L^)KPR���� `�r� � j ^a� �\!�n� k��P
�a@ln& �x� a ���b! ��f` Vo���p� �0
��
�0 !�
��� � �����
u�\! Za���@�:�\`�A&`6�
�p �! �����P��dr������$���b�^�*c|��@b ��.��' |! V�|AzPd��_n�����` b�6p�CF��c@C|�R�e�����7�1.�J�@7�1 �F���bK�dz .p#�J����h~!b^�6�z�� ��eb` �� �&�;t1R����� , �@xa� ��`� br&QR%Y�%k&�x! ��h� p�%��'��eAw! �����(Q���(u&e��h2+�2 �r%����'}(�R�h�,i�������&��%�� �� �h.-a���
��B��|���� �b�^��x��".��� ~��.4�: �Y>R�'~!4ES r~�$ ��=fK|�|@I:����6�b4��4*�:��2/�7� e�KN!��
�f>S4E�~�"�8� &�{.2d�G~�$y* ���
`� �P �����x�
� ��<[��~�����;��=�s�` f!*e�?�3@��,���S<��<�r
kR>�3?�o>�S�h���=�Ai�-��a=�S?[�� /-1=�0 S���r�)���b
���D�;�%���/ C0� \C1� �3>rg,3�7�3H�" ��dN:z��|�.@�
~4!m�T� 1�46f�.r7��2,�2�6��8�C8�c6�#:�"#��:�C;�"0���S��� @�� ��|nh� L��RR�3 P�d2�
R�OQ� �Y�O�vhz��dt� �����-���D�������\e�dK ` �j�"�T�Q����:�>#0��UC����/�����t z��b5��� 0 )l��x�,pI����Gy�W�$X0Y�5��dY
E��Q�UZ{�ZK�_ V����J��f�������,]�W{�^��e���_/8V`df@aF �)b�� �bh�a��:�|�s.� K� 7z! V�.\&�b�d}�D�t6�@l#i�fh(�h�B����a�Y�[��`����uk?O`i�L9Vla�K��� Lb�b3�|Pc4����yn3�j�~�����b�z� {|A{�{6�+f��c42 \s:��}�' �j�Gc�6 0� ���}�Ij���Q�6��`}k�n���u�v�\��Vae���,��h���P�6 -�4i���a3 F�w��
�A�#i<�#k��D �Ji���T� rId2�zj��ri����`
�b<��� ��b�I3������d���46_������`� ��
��8�x��� ��!8�%x��`�5�������������W.���v+��U.��#r�|TUHX{���j^��gpK_������1H�����5(6om���mX� �Wk8^Y���5� ��q
��8 |���M��X��o(�f�~�P�x�o��������8��J��Pjt��� �s��5 Lw���L��T�x��X�
��!����� ������@�
�]�,� ���� N�'�G��`���C�0��� �\�v�4��@- lI�FX��$ �� 2����ay��v��3������k������ v���~�F����Y(�v��u��, ��������
`v4�8��}- �b z������������� �����}-��� B��^���/�,������T�M���8�C����V�9�������M�� �1���l�=���(�y����l� �K���ag(�
Vl�,:����b�l�y��R�u���� Z�x������"����������`����������_����6OZ��Z�V�
�������}�Gm$b��(��z�Z�( �����a�v� �;��_P� �@�����(6P&���, ��!������,�%������
8� ����Y�D�:{�dm��"�/��� V ��q;�%�t��}[�a�U�#s���:� �{�\�)����z�g �Uk����\;� ������{��`��
���@� ��������;��-*f�����f`������oN�!�
�^ ��
���
�e�k��:� +j{������ �� �9���=���D���
L*�����T���k-��^��t�o�b!
D����_��2��6 ��|�� �E ��M*���!���s �``l��>&1�ZSi��������kEq������ `�a���L�dr� ��E��.�2�� ������]QR��/��� ]�k��-��|�)��u���� H*�k�R w0`�Q�! � S� �A��DK��inR}�in)s
�+w��Q]��������� �kn��3��84(� +�2 h��c}��/�A����D��1}�$���=�� K���z����
T��>]v[��
�<Su],����]> ���� ���u��iN���kN�u����m.�����=�����1~���� �+(����u �����o���!��Q>� ^�W��1|�
��3��I4r�y��q���b������L>kg�_<�R��A)��l���19z�:v>kI)�i��h��^N#"��.��#=��1���4=��b��:r#;*�I����7$�����{f ��:���4]�% �q�uC��1�r.T�^8�������)���B~]* � vYWv�/`�.�_ �m�z�q."5�z�wKR�3 ��s�Gy��b/��~.0WJ�w�2�t�'��;t�71S���6�b���7+r�X �cE��������eFN|�C��P�_�2T 6k�-�� �1�p8��8 bE�p������� ���S�p � ��
�����������K�$��!� +J ��25@P�+@��'M[Q���a�&������>cJ�hI`{Z��5�V�v�y�
6���d��=V��zh� �6���t����7���|��� 8�X5�~��8����;~������+[����L��C��:�
�V?�M���3�~}�+��c{���[����
x�����S1<�����G� ;:l�'�K���:�����v�80�� ���l����� �K������?\���c����`g� ��l����n�AP����G�o�UO����� ~�l�!Hb��X����b�X)�W^���;�cs��W�a�P�q>Id�_����&&��+2 eV0~� ���������{E�A#�Z�I�~HF���hf��t,�y�� b�Nbj V�p]rid|���F��� #v �h��.�h��>
i��NJi��^�i�u �� �v�i��:ra9���9�Cho{��%���=���k�����p����
�����;j f>j�Hl��> m�D�CA������[�����O9+hKn����n]^d��m�+��I_>3�o�������*�X=+��o]��7�0�)#_�? q���3������q � ��c/��Lr�&{��
��
��|����p>K��s�:G��������, �b����J/���]�
6=4�~��p�1P�u�^'7 +� �|]��|�����
w�r���
P f��s���_e��W��Mx�K�}w`�� �����w_�#F>5?�y�cI�b��N�9��{���^$ z��;�b&��3��z���-���\{������E "�������[>�X�|���W� 1��0����h)�W����F/���~c��A/��UO�n��~���>h�`��:��
�
p�,���C-�~�i�(��� c+8s?�
�R���;�b�$,� O���p�,l� _��p�4��
o�B9]01��>X�_� ��!d2��\& p�YB��(Jq�T�����(���h)� ������ ���
���� QW�G��(fq�t�� ��Gn1�� ���� c0N��F]�
�X���HJr���d �(=2z�"���;~A�6D�� J��TP�r,���+_ �XZ��3�/b ���E nPAEtIJ-�rW�PN��JY*s��l& i��w��*`��0� 6� c �#�y�a��?!K2�� lB��@ H��
�`���@J����� V �RB �0�*N�O\�3�$E �
X��3�)f ,�a$ hAKx�� %�D@1B&�NH@�@
�����`:B'���x p�J�����F�IBA�ci�@�fP����d�/�@
t���<�8y�*0�,���%@Ox�j��G;��b��8
h���R`
-D'�0�pT�|Ma%H����:-�&�����eE P� |� ;-a J�� ���X+�0���$`@h�������u&4��d���������(*H��U�}�W�P�
���������%���KD�����fY1�4`��D�:BP ��D � ������gA [��������M�� �B *q�'�v�� �( �Z� �E(& S�"
#��i0K| X�hQ+cb &��A0 ���0+8�X�"��%S9�a� ���2X�q �
c�eV�6 # bL����, U�������/_���iG9�1�_�A
`��
�!S@+X�
g
�� c�0�A�xX�X�V����D+��d ���/Thh Z��@m��� L��P�� kcVX�� �� K��s ��S��E�P���u�"��IG��v
@jI8��2
�
�V�jS���06;� �0!��6v�W��w���� 5&N��S��,.� O��� �)D��QA��t����� nT���# 3
;�n
�{����2m������u +m��b x
���[
�"�#�0lY� �B��qa1�q�o; u�S�l) �#�1 c>� ���/ Qao�G �a�����sV(��g&�u�0�Y�����TZG}���:��������Z���'��t�}�lo� ����� <^�������O��s�;b�����}���d���x� ~W�/�� ��'~��Y���a�hh~�����?���~�� ��O����~��o��_�����p���}����~���<��`�>��/~��o��+�w�_���o��C����������������%������
?����h�?��o������L����������������� �'�h��A��� 8 ���b��H�8���x������ ����!H��7�%���w�)�����-����1H��7�5���w�9�����=����AH�s7�E��[w�I�����M�V��QH��4�U��;t�Y�����]����aH��3�e���s�i�����m�q��qH�]3�u��Ks�y��9��}�%���H�3�����r�������������H��2�����r�������������H���{���rx�����v����z���H�~8�����x�������������H��8�����x�������������H��8�����x�����x?������H��7���� 2���������D�(��H��h���������������)�I�i��� �� ��
�� �)�I�i������� �� �!)�#I�%i�'��)��+��-��/ �1)�3I�5i�7��9��;��=��? �A)�CI�Ei�G��I��K��5��O�#���(��S��Ui����YI�[�����_��a����@g��i��k���eB��h�@�ui�w��y��{��}�� ��)��I��i����� |� q��s���)��I��i�������py�bi �y��P��I���
������������������� ��i�������������������� ��i�����Y��I&�������)��I��i��������� �i�� ��)��I�����>�I$�Y����� ��y�F��^������y��97����� ���yk�9&�)� ��y��r�����*�j�J$��J���
7
� *�#��:$��$�����p��v�����2J��9�6Z�1��;��=z�>��?��&��%��I�����J��Nz@ �0�D�#(��Yj�-�6/��_:�� � �� ��
�i�1��� �<*�A*�p:�oj�B��_���y �p g \P�X� )�
h���h����J�b* v���|���*��J�� Wp ��0 T
���J��J�}������`���
�J������
�j������P ����������*���@
�0�
ty �P X ��
��
�
��
�������k"m��`�����^�����
t������J�zY���
�:
t�c�������
�v����
[�{ � +����:�|��K���h ���y���
�� �@�� �p��J gZ��
u)�.���� ���!��C��>�5 ���z�� ��8������ � �L��PK�00�� 6��J��O��_�+ �� p�u����b��d�S[� ��s ����^ �#[��z��z�z�����-��R� ��� tY�h����k��j��-{J����FK5��~I
��
k���[��:��������@ \`]��yy���R� ��
�P���;��k��[����J���������{����$K
v���K����,K �� Y����
� ��{�=����,�
������4������0@ `����Z�T��w� W��<��P�_������ L
L���0 (0 o+��� |��;��� ��{��` |�
��,�w�
�� �������������[P�* 4��<�B �q�����[b�FB��������)
�z 9+�Q�4U���i�����) X���p`�il����gl�eL�V���[JS\$b| �Z '�
k�'
�
Wp�p��R*��
����
����P ���
�
t9
���\ ��� �L��`�O��|���Q�&���\���z��x�@����A�� A X`��P �L�`�4b\��)�p������.���@ 0� `�P������ ����� ����u�����x�
z�!@;�w 0��.P �k�i���L�� �u)�t�
�Z���M,�wi�&`�pv����i������`����R{�u��=� [�t�
��������+��������z�� �B�� *�C=��I�D��G}�������������B�
��@ ��,���k��� �fK�(pw�
� �� ���X �vY�"-�[�w\��;�_ ������ L W +k�|���+�?�3H+�n��B��=����� ����O-!���X�����p�����
����i� h� K�� ;�ty�sL `�-��0 �m����"���M��l����������X��������v��:C����������MM�����}�P �������&@ �� �p��
:�����&� �����=��*�� � �X0 ��]P�]� �L � g0��� �
� �n�wy�]@ ����` ������������� &p����P�I|����>���
����������=^������C���{�Hn��}�IN�_ ��y�J������J�C}����uY�Y��v��_~�EM�H>���3G^��L�?���I�C�uE�#l^��p OK��;
��~�{I
����PGl��. ����� M��>�*��8��� ����A��� �<���y���b�9��y�Y^���e��c�� J��~2�~�� ��
��^�������� #���n���n2�.�:���~��������n�t��� "x.�����.�u����%�m���n)������ ��<���������� ��c��"�^����~��>�B�������N��@��m�e ��|� �� ��������� �|
��/�?�~�����(�������3(E@�4�!�|
//��g��:����?O�C�<Ho�mQ��������MS�{Q�.VO����������M �����3�?�� _����[�;���s�����v��^a�����2����������X�I/�P����������/��.x����ix�����7����� �/�:1��� �5����?�:�����q��a����1��c5~��{���b�����s~��{���@����p����j`W��^`W�� �p[���p�~��{�U��T������i� A �]�?�mHd8 j�
$X��A� .d���C�%N�XQ�>�5n����F�!E�$Y�����T�d���K�1e��Ys��v6u����&J��
%Z��Q�I�.e��)�b��N�Z�*�xA�n�j��W�`��%[�,D;���e���[�q���[��]�y��g�lH{v&
&\��a��/f���c��%Ov���_������f��EOL���i��U�f������)��]��m�� ?� ��V����&^\s�����
� 2�q����:��1x��6F�� ����A����g���a����^��|��K"���{
�PF������|Y��
@PA�V�������a�^T( 6�aK<0��@��b��PFE] �� P�d"|�-5�E�����1�E-`p ��B& �
x��p$$P�
����pb�� �R�5��I�=4�TsM��+������38��!��bcb�Rf.#t`�p�A�J ��"�p�
���HE�$/ `c-)Pk���t��FU���=�T�C? Xa-P7
g�6�)�b~ # �za�B��e� ��t�_�r� ~!�;��O/2�;�Mlsc��m �v�o' w\n���\p�W�n����9�|w^;��K5$3�>A5�-���B ��tYV���b� c�Af:6J�Q��9���}U�T �Nh4� ���_v���}������&����.� g�K�2�k�zh� ��^;��i���W.�^^�� �)T�!�!�e�^kk^�H_$�8
PT���\�����Y��d8;c�x�J/U
�
�(`� �����*}��e��e��mV��d� �����y��������m:?�������K
p6&�U�>���
7�<�v�uW p0���-<;�+�i�� �p�����T�0����_b��Ow}�J p� & �q���
&��n?y�6�a>�a�(^�q:0��Iv�V~::���K�8 "�����! 4��JFt�;O���A�Fu�� ������CD(\][XX�L��S*�K�\�e)*�������s����\CL�eD!Q�G\"�� � ��a��� ,����Dv��F �C���k��IX���.������C9�P/�{i��'��? 3� t� z�@ �0�C x�C 9� �0e��$r0E?0�`A ���q�!�������$&�@�a9�� ��N�r���A6�<� �L� $)HBr\,�� 3�Q4i��6��7�%��g [hN��.�`��h����@� � 1 8�&;�� <�0P a � �2 � D �0`>���xD= 3J,�H ��@O���H#�� ! ��Dj��d����fgfzS��S��Q':}���������lr�P@ �Gu�$���<����?!���(`��hB'�DE|5 �h('I �3�Om���8 �&�����T�v���L&6��S���3�%�Ht�<� ���k����A@�� ����pT��
86 ��k�����#`������;s��B&�" �M: ``la����t���*a��U�*4���@�0Z� �|9 t00��� ��w0O�&$�t[|�@^�ka5#������@�8"rX�0�b��� z�6�s�z� d��� �m.Y�����#�T���������y\ �*C<a��_�ST� ���#�K�N ���8�WqF��� A,�u0� LW����B���AB� $ y��-�� ��`� @Z�� �x��@{d�*_��W,h�i=�������^2%�������������Dkp�����X� �ZN�(f 8 ���<K0gA�K�S�&[G�(��~��z�p��)kX8�g$T3Gn�E�^�?y� �E0��bb��8��� ���$6��l���������m- ��_� Gk���"4�&IoZu7�2�>�
��(����#�Er�R��c ����H8 �/���/�`����0l��
[mg��27 e���n!��A�p�-��b�]Kl�� M�x�{�lh��� I�6��i�� m0A� <�eo�T����� � �0&���e���[ ��K_i:����3���-����/��0���G���@�_�]�� ���q�~ 7� y��� @a-��;�����
C���7�(!7�� 5�Be-S]����5�L��O ����HV�
w�������9��@�N@7��O-� ����qS�g� ^��q���7@�������S�7���� ��������;|������ ��V �z4��w�����tq+cH�c�� � ���v�z~�}@L:�v&����7���BQ��� 5��x�0P�X�1��I[� p���7���k�p@ �@LQ�6��M�3�3���-'
�� �%��<�$o�� �X���� �IB"l '��202 ��<@ !$ 3x�I` , 4��� ��;�2h�(� �-,D������ �� P �:���� ���;��z��X_x�����C���@�Fs�b �����[���d
)�>r�
�.�B� ��p&��"3� ��$�$?( H($I�����
8�^�%Y�� ��?`�
@��Bn�?p�',��D�$�P
2��(�I0 x�T��Q4 H�T� +� �1���IP� @ �E�2�?�_�3�B�����I���=��<�~$�_��1�9���BQ��D8�!�q�D0�D����P�]�H��
��DPT�C����h� � [R �I��^ZE���.�"����32� ���+�H2�
��&3x� ;��*��$
��h��.�����x|�#��L)�b�
�II�(�4)~�I����8� � .IX�Ipi�6_���G��K����; � -�4�������2�
.+ ��� H� ���
��j�L%C��G�'"�, ������$h�}i*�����P( ��?�<�� � ��%��B�B+�BC=�"U�����t20i��<�i����GG��|�x3P���K����<�^X��L?)L�lHE��<1���p�����2� H [�(��N���QB���?���
��jG�����'�h$��� R)�4F�p%���'��F �B�:(�4�����2�'Q��( 5�IhE�
-�N�x��B������Q�P
� ��<O!E����� ���D�x�I ���&��
�M�"���
�%�\4��B �M$�G�,N��3XS2�� 8����WbE��^z���M��R<���"�&��2�2� �h*O�=` ( =�1
3Fp� X��J <��2 (�� c��f�s�0��z�������N�TU� � � R[ ��8��Y�`s' � �hG�$�4 ���C�
+�$A,(W]UV�V�`��K r�Uo5�\E'^��3��P�d}>8�M��)&�j��x(DkeU}������V��U\�V!�U<��#
asW�]�d; yM�v��g��j�W��/�m��
X�5O����PLk���=Y�[��(
�X|��X�(��2�X��X� ���q >�=�(��������-d�}}Y�uq� ���Y��)����8��H� ���m�"�Z���� B229�8��H����]���z���=����I�Y��Y� [��A�%�-��V�}��0� ��E��}\��\��\��\��\��\��\��\��\H�Dx\Ap��������������M��5]��]��]��������]�m��]]���������=��������[ =\� �z��6�^��^��
�����^�u/�^� _��lUU��Y����m_�u�_��������|_����
`����VbP���_.`�� ������zX�5`�`k=����i ��X �`����`��� �`.�~���
�����W~af��U` ��$�a �a��0����1p b"�&/p�;�a�S�.b'~����~Tb�3�z�b,�bz���C*N9��b1 c8y/��;$� c6n����3�=d[7�c;
�Ib4<7��;�c@6
�H`��b��@NdE��a �3��b!^dJ�d�@a��d�b���dO�d����=f�6`PNeUf�1��B.e4;�U�eZvV <M����e^��^����emB���ecNe�m9a^#
>fg��a��`�e��X
~flN�HU4[���l�D&G�6o��#gu�~5gj��<^gy�crvg9.,b(�y��6^n.�sn��g�fcd��f���nh'^h6Khz@��h,n��~�5�g� ���h�i� i�.��m;�]�^i��ni�~i�E���h���;������[�X�EX�
\���%���f�!L����i� ����� j�������f���:�e�l��k��j����>��(j�h����F�w(��k��ks"�� >1Z�d����"� #���� l�l�������h���7 ���k�k�� �Fk�n�����~������l����>���l��
�& b��{��8��k� ��h��@9�����1h�/�#M���:I��^��hm��i����p�N ����� q�������� �����X�&I�&$�
�b1��i�g� `�C��!/Y� Y ��)�x������.��,�h�0�� J��N��.�/
) ��/��m�A �g� L . O�
Gl2r���5���3�kU��Ib�;� >���:�@ � �C�!a�!e�
P�l��(�� _�W���[�]�y�Z?���i,e@G!��j��H�5
G�P�F�0 ��?��
�"����E 3il� �B�t��#.N�����\�2� _D�dE�: ���Y� ��� K�(0 ��# ^lG'�1�xX��y ,��d�q���@q�Q�5�0��{�o�����!�`@����l��������Y f1�D ��7cv`����y����:�8m�P��8���'��W���DK ���� h�0����r���� ���%8 �����I��� H��A���y���� ��B3sw�`���[?�f��m�0�e��Wq������!���dP �a� x�����I���+��e����,����>��X�d n�j`��7���q;� y����8��J#k�$��HM�L%�t�7�������� j����*�L�|N*���J�*��_{01�HbM�|��{�0[mzYj ��[�^_ `W���56� 0�yb��+Q�@ �a � �IvN�@c�v�S���|��+ ���Is����_�����a�:��)�� ��4 �����p���G��J��X��G(� `I|z��x��: �PZz�U)� �� ���H��x�����uF I4�u�P�0�o ��"L�p!�� �A�(q"��':��q#���ft )rd� &�h�� C;�^��)S�2e�f�T�s���0o���ShP�>�`S���Mq�����P�Oe�!�5�=;���
+����fs�V�������� Bd"���� `�(��P�x��9���W2}�c�$=�LJ`� < 6\��G�Y��[���:���d�E���IBfo��=�YV������u��^��#xq9���� W!:��'���=�J�]^
OS������$�kF 7V���o�����w��?��������� ��[B��A�v7�9*�WaH�a�!Dr��/ct�H^<X�I*�2x�=E �/��"�1�'�O��8R~����?Y�#��Gnd��=�����E8� ��X%Ab��� ���_��K����$��c��3����p�Yg�W�9�"���������Z���1c���]�B�����e��R���mD��u��I��yw�/�M�K; ���
*�(��9�Nlh� ���� :���Nm��N���� ����h���� ��$��!�NG;"z������2���` k���
Ov��;�����������s����lD�m��Q�M��T8Qy1@/���{5�� �n���
�.K-�k�E�:��0-��2�c`�p@8��0m�!�|�X�U��I���� �f� RV Q�WfhA ���B_�0�Yy� $@ �uX�=I ��1<����pR�Z��L��Z�/��R�K��03 S*� �2� �cx � 8���*��q8���/b F���:m �1���
���/��P�K|? �������Ts�
2�4 �� �z +�"f�/Ya�
���/��p�2+|!@�/!�9yAs8 � ����mvAY��n�@�������� ���N��q�L ��� �@@,�Xx� ��+ D�Xv��0��ei
�� A=�����}������d9���p�R�� �
���Q�TP�/an'�IFLN���` ��
{�����De�PF��:��*W����`�+_T#8� �Q�b�.&�[c5~���r0��0��0�/!d5� /��&��Yz�� ^��o? �,4��M��GP[ ��������ZFC�?�A aX4�#@
,9H��P�I���H0� 6��?� ._��`����H��ayK ����Ei`��W�9���{H%-'!�
��,��
c�0�\'1 ��H@p��,��&�C���I�r�%��KJ<N'��p� �`\qb����D� {�R���' � 0����){���B,��CY�+V���9�)~�
�����ya�.;^�|�a��/���&�_`5�����/��
�Z"AJE � _��#V) H a�e�I��l@ L�Z�����,&t�W ��}��1�9 �r�l^F-���$�"W��5,��AN�wN0�1�`%!�I�5�X�$� ���� '����T&2 ub��
�h�,������G�%VOut&'}X&�0� %�
�P��I�(H �T 6 0�l���I��7����s��i�^R
*���
���Kv���w�?����/y�W�
��(p5��� �?�`@aq d s��bf��(�6\�F����KXV�� ��@
a����H�p�������=��i��_Hl���F2:�c^���8 �=�= �+c9o�-n ����>�pC��%�JT�"a���Q�x��J��L���w�W&7U{��]��r�� �A ]1��b���C �a�T��(������?���5��:!�� 5��a��0�� @ A ��>t ���VK���r �|;��Km�F� ���29�@
Z��%@ �Dg�*
%@Z2j��� ����1�_#� �����>��dO"�_���(;�`�����5������D��B� ���_��_�����/���8N"�_x�:��A���f �9$�3�9�,�6�Y&�����"����&��Q�k�� 8�["��q�� 7Y.mYf�&���% |�%h p�t�����!�&� 0� ��%`b�\����� 2X����&���6 !�`a�n�
a�� �RX� \=�[G��p���l�����B@�\��].�
R$:����l��7�)�
&�/x!�!OyP���DENF�i�X�U�7e���={��Nz�B�6�d�)7A����_� �qO~!��@r�3�ap��U=�E��������3#�g�(p�}���'��r �3�H ��=M����H��E C� tTp ��q s��C8���C ���I|\�|d����|������U��i� � �
Cx0<G=�K�y\Z�2�<��(l�
�A�xh�r�y��� N��}����B��<�~(G�A"G���p�`��`B�@�m�����}!A a� a�t��� ��Z������������K � IrD�Ar��2`V� D8����� <�#Z�H��\��8� >�
JP�"
b! �q�p� #J�&[��A��b.�D=h�.>���a
�0���`F�AG�Gpq!,��&vI���T�/b�Bh_6*��iD#�A8
�8�c9��9��8��:�c9�c;�c<��:��<��=�c=��<��;��;�c@��@��8$<�#$�D/���V��+>#4�"1�Y� �
� 7j$C8�F*W"zdH�Y2���ul��@�^b` �����L ��M*2T$N�����i��@�"�d4:" rH=LOO�$1��0�SJ%����e\3��Jn"0(�}��T��S�����X��u� I e3.��eDl�~�H�eH"W]VI�$^��p� ���C�dVJ�%v�}�A��F��brH:fd��L��
fD��%B����db�\zfu�bh�&C�\#C��*VGJ��[��%�&"^ei�"� �frDen��@�H�CmR�kvl���sPNo��72�p�s�����'�d[n�qZb �9@�J�Zfx��^�'o�ZH�p����>X�� r����#��}v�P�gi� `fDtBG{V�{Z"�!�`a���0hG���Ff= �F���ff��8��qxAUJh�9h�fDr��b� G��|g�2�#�P �^���
[�:���h]��Fl�st�[��#� �h��y��z��*�A�g���jw��v��C F�wB�E
C����
P@ ����&��f�����fA`�eeQ���j��e$�B �H�*i���Y~DF fkZ)
�������i�ZI9< �*�:�H^ZFP�Hdju���F�T�Z�ni�}jc� ;$>�j�����j�������� ���>6��bg�t�
��F�4��ZI�dx���> ;�7H��Rk�Z��bk�j��rk�z���k��k.�@�N 6h�7��Al*u���H���"���� d����>k����l�
����nA,p+
LC����A�+t��6aGL ��+�8�Ix��n�Bk��l�������j����*���C���>G����j�^�q���M���k�z�6�l�
��
m-\������.0�� j~aXu����*��\�o�+�rm�zm�f�,h����,Aiv�
�^G��m�C�����m�����m��������C�n�
��.�8��}��*��2��b�h+�B����r�r%G�B��f&���n���8�mudn��,�z������.7�n�����.������n�����n�����n�����.���Nk�A�f-PnIX.�����������n�j����2N��n��n��o���6@6`����r�&��� �:��&�ro�������2�~I�n���o0���`k l����b��Z��6���&��ok0�������*� ������ ���2���j_��C�BW����
so�����S+�p ��� �pc�W� #��nC����&�p[�U���P���0������JC \�-�k,,��^A-p����1
�q�n> �q�o��h�ZliR�r �DkJ�"� ��� ��6L+��4�@ l��� ��&r #;27�@|���B L��nr'72��&��,�B �17�r �r,�2*��*�2$K2%[�.`r���'��(Sk)�2�^Av8�K+$SC� H_,r#�+7`��3K�T�4(1�21�Z�s.�N1�������L�=�D�����.�|A4P�#%�������|��Hm� �/7l� �q4�+l�fAFO�+H��Z���l�J+5@���BHK�HK� 8�'��)s� �4���2 /K�47��7g�FsCGk����D������J�47�B �Qg3F����B �-HO�6� 8�@.\�6Lul4@Vs�VKC ��p���}�#�"sv� P�0ON/�@D 0�/���P�� �}W�%�� bG���UL � �8����P����B ����i�v� ������?�C���JM�� A�d�32�����f��P������}4���&��+���IL�Z�PW�6�� @�tK+/ H+
\A�j+v� D�I��.���){��w,�2yc�y�7w[w�R7y��v��6x7xs�x�7�NuUK�U����B ��Fg7�@6-�O�QS��>�,����@�x>���j�R*�^0O���=�I� �%�"�D`3U8 P@1 ����@�M��0�Al��\OM�E|AU@�H��Ne�����
�����K$�c��$��e�1�1�/�� ��xQ�x�)�`KpS6&�W50�N$���q�uC dA�Vr�mct�b��z3gC��W�Tw�.��m�?��oy��)s�����2z�;��S2S��^z�:7Dz.Lz�gk�'-�#:7<8-���j�6�@XK�T����������q�B��t�5w���K���)��NvHU��G�%t�tK� ��U��h�SQe�� 0 C��
N���W��;�I�R�� �� P��B�� �O����08� � H�a���� ����y�S�;���.�� tAFc�6ttuo8E� ���<�'���. 8�� �|���+��Vg�-�
��-l��������|��<��|�':��<��|�vAt�|��U�:87��s���|���x�T���s8Uc|:_���B ��K���|���� S��"�
1CP �8J����#�z���}L� &�����_>��W�q��G~O�=U��/��p�G5���@��|��D�t�����f�R���u�� ot�g:�Wz����O�w���>Q[�.$�V7?�&���7�+��s���|�
��?yO?�'�P�N��W�9?��[�:�K�e/ ����Ox�� �N5�� �0��! ��p�� l 7��@���!� ��
�1\57|�[`�71V� 7R�C� [ 7#�B�2�TF���_l& ,�s` `>5��nX� ����8��
����W//� f00*\�;��l/�O����o�^�{�������y� 6|8 ;;�7v�rd��ne 0kcm�8w�LM��[R���%@�m�F�0�Z�6^�i�}7c��17�L�sn��cm�=�����7w�zt���W�>�yv�~� ���v��������_� ���3$]�:���6���������?����o��|
� pA|�? �����*C��P&��Rl� [-�I<��O| ��ZlQ��Y-�S�Q�m���[�� ����"�I�b0�I�(��C���b�����;�K���.� �>0+$F� ���L2� P� �l��^�,D ����>���-f�F~�3H#]:D3,���~i�� ���L5���*c@f�7���KQM��M��<�U���R�� #V�� ���relW@}���^{}lX`%+V�\����,�eL�Z�c�Z"��� ���k� `�K1���v�
��R��o�t�L��xQ�Ho�jG{���~���Od�� �4Z9T���]����+gI�p�� ����/���/t��TF����2�d�K���`���|a �b��>�zq6V� �TP�]ye���� �U����|�m�/~�V��y-�����5����k�e������V���a���0��n�����s�. �(���9��������b�^h����B���fJP�.�!�j"4�$���� �2k�.G*��1
0V( � ��w&�)����:k�c$�w ����������AX��������Q�����o�0�1�R#�_�#��f���5��O�#�z�e 0^Z|& ��)�� @�v�I#�PF `���$�,- �J���>�K/���LD %���B8E
XA�V��,�!���Cl��:]/1�����W����_0@���6��y �a^z��8i�|� ��� �$c& ��CV0��('�� H�$��CA!�P�2*A�eT��L4R:�-�0F f0���!� �>2.d#r���aTc3��2���
�KIF��e�du����njz2�� 5�����m�I�n�+ &1 A\,���� c ��������o&Q�db�b��@X<�/� ��D�KP�L����!�X� ����]7�� �y�!������� ���KP ~QO{
�dt���������!*�cJ` ���B��^���2���,q���C�z�X�|� ��� H���Ne���2���R�&� `� K��� xo�PL[����T(��~���` ���LF��}���
�K��p�!h��L0g81��3a�[�@�` �
*�4`y�z D�(Qj�rI|�UF5����E����{<�E��drW-� �A �M1A���?B �a. �&����L�& 5x��� >"��e�-q@$_� �:� r�-�Y� ���['N�V}��|U�r+�00ve,���:2���+]<�8�er'C��0Y!�����#��d\{���o�T ��I���ev��/���MTQ�
����8���Jo��;� �7N���'�[a � �8��. ��)��� _^�_�6�]L����b?���z:\c��Y�.�a�&|�t7�=S1��/&P�� co� K7��}���|���>>��A���'��u�v��d3����0��@�)�Y��(1�fa-[k�{�Kw� �S!9�����|-_P
���k9��r���w�N��� �W� #���iO������I�D��% ����'=�����}�[#�Pk[������q]�Z������ �b����u
j e?�������
�]�����r�`4�������p(��v��A 08�{���]o��c�`ps�� �$b/�(�|3m���C
��0TmpG��������;����Hb-��\bZ�!�E�|1����'W9�� �Tz���!��0x�+Ob=�q>6�X�� r��^�!�e�1����XQ��cD.zF���6O 1����#����J�y�������-G(@��� /����R�� ��Kg�������}�/b��(o���W�����z �KO�����+���� ���}������8d�������8*���G�������8��O�\z&��-�_��&�w�y �z��/��x�qe+�^����!y�U�D?u_�{_�� �C.���/����RN|�u�������{���z�'� n�/�~a��_�T�/�o��� ��� ;� � � �*,�P�b �B3� K� ���(���N �M�@����/NO�n���L����
��o������bW����H��/ n�
�
��y8������l���!u�
���fE�.�y<P �lC �k�0 ���0�XO�j�
�,�����O�JO-., ���|oV ���p���/\I��� �*�� �x� ����8�z$��t�/�M
�-�b l)91��uH�C�aS���
��l��h@�cq�����#C(�J����
�/r���������e� ��� ��
���������!� C%��a
q�����Xg���J�����J�q 3� ��0� Y������ �o�n,0� �/F=�n ai��0"�Eq���:L ����S�@�ZBM82�
@���q
a� b ��bR&� ��$k�
;�#
w�P
o��� ����"� f�%g�)�2Fj(�e(� ��V� ��0 zA ��n� 1b ����()��)��-�*�� 6����!�C!0IU��.�r�l��f�����R��R)a�-�)�2����2#S2]��/;, ��B�,/�0�O��3��1K�1'�4O5_�2-�6�0�N%��rQ-C�X��-G��J35{�7STV���9R����X�6��l�����9�:��:j:�S:�3:��tS�x8�S<��!�s��1�����z�r9m���� ���
�O� ?C>�s8�3���<�@'�<��2#
o�=�(�>exj�2��O��1h�BC@�L@ DC��������c���0 �1Gj�� ��1��R�0 �@G1Fe�Fm�f! d�1V`,2�tFk�1b! ��ft�>��
j��4 ��6���vA����G]`�A&`��I��10TL��L����M��Ly�C��CE�P U5�L,�� `�8W4� � 2� ��1� ��D0�6"�R-�h4�� ��Iq�S�!�Y� �tU��D��Uo�T CC{�>�S��D��<�R/� 2tC�4 ��1��1u��P�UZ��D� ����A��T� S�@f�2�Yi��\755�u��1` 9x�Xo#\��]�
�F�t _c_m7r�D� M1k�u0� �5YC��X�W���Y� Z��c;�L�U�f��C��l[u�d�`�j�D��a��Fw(3Z� ^�#��D�� ��T#7l�1`Vg���T3J�V�vg��V�Bu5bvY-�fq6Y3�
�Ubu�bCa#colc=�l��.@���9�Q[ �z
�1y�
�����tawhf�u���`o��`C^Q�V��` o
��o�v�` f!`W�r1Ws��i#j �`l-v�6r�o��X��fO�b�Vm��l�w��v���C Mn�� ��$�wpTB7Dd ��> y�7y!z��3���n7w�WZww�\�92,05�0�� z�
�����d]GcS\ 7��`��^�F�r \a5r�SR�z�7���{XD
Xw����c �x��NH `7�F|g�C{�� �@������]�sDXw-��������Vd�x��6�4m�-�����~a
� V�����
��x��g��d���J�~r"� `�����x��cU
o0� 6d�~+�2�d��f L&|�u�6��8�����X�����X'Z"�Nt>���F
N4|��Nvo(@ (����wb2��J;���;��_lg��*
�c �4�qp�q��%\ �I(���p�s���t���b��,0��'��*\J!T�'�J&X���Cg^*������Mb�I��Y��p���<��)�
�1�@��9x���@}�*1G��g�-7��:y�V�=�����i&F�
��" b�
�b���.|�������d"F�!�H~�yA8���C
v�J (�d"�Gk� =^�� B^ @�y��Y;|�������T Z)BQ���G�-��S�GY&����T* �i���>���d9� k&v�>���a�g�� ��e�'Z&,: ���db �X@���0�&>�.����:>���HBuT���Y_`:; -�jn6d ��CC����p�b%�
�U��}N����>��+�",��!4�+�a���,�B�x* [%0�8[.�$�+��5H#q%H�K�B(��(�� "��a�b(� �)>������ ��9�� M��w����M� �')?dBL��[l@��' 3/U�](��������,��%�9FH���i��(�;{��@����!-��� ���K@�������%B������2�+�� �9�b������Y�[��k�N|�Zz�?�S��D�K��P�� �$�D����]��P�9��K�la�7/�N"y|�r����ug(`���9���l��7�<J~���C��m�&
�;�mw���c� 5�<I�@���h���|ob��� ����;�MS��� T��C� ���e�����\��|<�\2��C������]��\�0�{��c 2! C�����
&&�
]_�f07 ����C�-|6�9;���-=������� �c Dg}.E]<$���c���T�^�L�~]��]� ��]Zw�`��� Y���[�����}�����[T}����{7;���\�/=,i%�����������������(%����Z0Q�]�^H����� r����n�% "��3^����������E� 6��O�0����
� �) n�N�P��k> za���=�B^Z^}m^�3> �,�V@�%��ke��q�����A��� ���>V�% |]��>�� ���
� ��>V&]=����2��>;j� ��+��2���~ �%�cP=���QR�\S�?�����b�`�� ��& ����%� _<"��N� �o&�+/��p �����y��%�"��c �`R�/���������<N����`F6|c@�Uo��$����_�� l�/�`��/��D�����?������ �~�?J*
�� ��N��9�K�������~7��_�~���#�� �?I� ��+�����*\�����#J�H����3j������ C� ����r }QG�����b��I����4_���.� �:�
J����H��*]Z��0�6�{���U�8�j�z�"�^�z��J����h��]�1 _ �cK����xc�M.F�r+�
L����f�����c���#�K9�� (!K��������A�Q�[��+��y�k>7`Q��M�����f���q*��{W �k8\`q~�+_����^�q��s���#��������� ��_ A�K m�����m������DF�
�a��_W����9*��� 6� h��0oC�XX e�y!�
(e(��$���
�a4�up�h�� "FLaa���8�(�ymPx�;jP`��K���d��D6��� ���y!��!B���xI&V>` ���d��N*�yQ;A�WfR\�%�;�3��x�I�zj��0����F���fj�E�������F*���Vj���f����v�����*���&�+�i23�@L�Ej�d�(:v����������+���k���&����6���v��V=��QF�����2%����1 �����������������v��;�S�� �1�*�����`�$��l��'�������;m@���� �9LV�7u1 CD��$�l��('���A�1�N� �r-�eq��� zQ����,��D���8���o�LQ9l\�q����Sg_)��C#���`��+�TkdN�c�sQ9��m�kY}5g�8�� �*���|����d�-Q=���i����y ���3}��}=����g�y��8 Q;&�������
*�����A.h+��6�Aon����8��D�qE�� �EjL��a��r������G/���'��/3@�P�����x�d����n������>���C��1P����� ������M�d;P,!�{�����K nXA���v # ��2�@� 5tb��j�5M � @�
`��ab8@Tp���0�3�a��P�e����� ���� �8`C�� /@� � `��A��J
-B �v lba�@%z� |�`a #��x�a9a �z#u%D�Cm������c�j�!z��)�������x���_ �
]�`�`) N|�X��H��h�K@'X�� ���E(O��U���$0]A� $��)@�
,Uc�
h#8I OX� P��� �p��p@6Y�� �� �'���]�2��,&�p��a�a�!9x�r �
1(�
�0 %R?�D
���-
���D �]������V�'����0E ����+R �D�0�]� ��B ��� �bW�� 'x%������(x�Ei�tW�0)9�I]%
K�L�����TW9�)� ��w �`(�
jP����F��/����f(��Q��fG
��d�*�����BW� @AX����"���Z���@����@A
R�5�M0'Nu*,�u����u����* �dE*y�QQ.��5��'HQ��nS�X��V�'����� ���w(�j(��`���n�\�7� ��B�� �
,T�����J�R�R
e�pUV4@��� ���^����z�c!{����W9��|���^���� �j����0�:d�hk�8&���jM���q�6�@r�X�`��y�8���1dO�ap0�A���i�� G)�]uE�W?�U��)��N@^Wd�LX@ ������'ve�)�` %|e0ky�%��`�S� ���f���/�y����JA`�N`�Am�)*<�%����
����j�]s��L V������u�iVt��@��>\s(c��
�
��"C��E\L� L 1 �m�9 ����� ���
;X�^�;��X�B
`��Xc�0�� 18A
@���+�6�=xM��$�9�X�/�}�v;���JvD��q��H ��0��@�,�D P �o*���7��@F8�Q�r[<"���=�1x{�����/F� �#��G8�!q�� �x�;
�����,�
�A�sb���@'L�i��fu:X� .a��3��^�*� �Mp��L-���LK<��%�;��y]
`;�w�n��1�p���S��A2X�$�S��:�A�<
'�%h#%�+TX���uUN�����}t ��+O,� A%L���B0�'� p���!(� �UxbA u�y�o��e��A �����,@ * �����9��s�n��_"���*NAU�� ����PSZ�3���`@je��*x���%��D��]�b����h�h��w���{����U�i�LU��Sd��7���f��`��xm�5�}� �-�� �W0]D�� �� %��8e�XFQ�ua��
�`�R.����
��u/�J��
�0 f�
�EfBH @QG�Y�� e�+��N���%ph���`�L����
�5?��f�" � !�.x5,��d 0��, �w��
�� y�+��W�,=��� H�T����V�;x������ %�x�%���vp�T��`�$g��yES�g�[�������]Kw ������T�tF�0
� �p
� M
zH^[6
��t�` ��W5��x���T^7��{�� �p �N�p
��+�� �` � ����(��%��bl����8����b�iF��x�C1t�a�R �,��.�+),V��Q������^c���.������rt=��C���!A��a�9���� y� ��!����$���b���4)*�, �: 6&Y�>97�9��D ?3��H�A�CY�Ny2=��R�K�M��X�@G9�\ U�W�+.��d�,Q��\����e���r��h)�jl��v /[�zys�uy���,p��>���������2��I��y�`�9��i�`��y�����������������Y��Y�)��I�����i�������9��Y��y�����9��9���������v �����:O�[�c� ����A�����9�)y���i����9�N��y��9"������������� �9��y�D�����9 ��� �u��Z��1�y�
����� ��*w��Z�U3������� ��T�� :�$��"Z�(����*�� ��.�
�2Z��I�6�����:�����>�� �BZ��I�F��h��J��I��N��y�RZ�� �V�����Z�� ��^�A�bZ�G�f�����j�� ��n��F�rZ� �v����z�����~�y��Z�cB����M�����7����"��Z� B�����D����8��������Z������ D����Z�������Z�
�����������B���g���Z�WA����p����������:��Z�#A���� �����&��������Z�^�����tA����|����
��Z�
����G���*������� ��;��Z�+������ ������+�[��z�������� +� ��:�$��&{����*+�,�����0��2;��Z�6+�8�����<��>����B+�D[�~z�H��J��v��N+�P�n:�T��V{�f��Z+�\��^��`��b;�VZ�f+�h��N���l��n��F�r+�t[�>z�x��z��6��~+���.:�����{�(���K���� ������;�Z��k������� ���� �����[�z��;������������7������{�����������������;�?������������;�����������[�G�������z��������
��;��{�OZ>���������;��[��{������������� �<�\�|���
�� ���� �<�\�|������ �� �� �"<�$\�&|�(��*��,��.��0 �2<�4\�6|�8��:��<��>��@ �B<�D\�F|�H��J��L��N��P �R<�T\�V|�X��Z��\��^��` �b<�d\�f|�h��j��l��n��p �r<�t\�v|�x��z,� !� STARDIV 5.0 ST m ;Re: Function array_agg(array)
2014-10-27 1:38 GMT+07:00 Pavel Stehule <pavel.stehule@gmail.com>:
Hi
My idea is using new ArrayBuilder optimized for building multidimensional
arrays with own State type. I think so casting to ArrayBuildState is base
of our problems, so I don't would to do. Code in array_agg_* is simple,
little bit more complex code is in nodeSubplan.c. Some schematic changes
are in attachments.
Thanks! The structure looks clear, and thanks for the example on
nodeSubplan.c. I will restructure the v10 of the patch to this structure.
Regards,
--
Ali Akbar
Re: Function array_agg(array)
2014-10-27 9:11 GMT+07:00 Ali Akbar <the.apaan@gmail.com>:
2014-10-27 1:38 GMT+07:00 Pavel Stehule <pavel.stehule@gmail.com>:
Hi
My idea is using new ArrayBuilder optimized for building multidimensional
arrays with own State type. I think so casting to ArrayBuildState is base
of our problems, so I don't would to do. Code in array_agg_* is simple,
little bit more complex code is in nodeSubplan.c. Some schematic changes
are in attachments.Thanks! The structure looks clear, and thanks for the example on
nodeSubplan.c. I will restructure the v10 of the patch to this structure.
Patch attached.
Regards,
--
Ali Akbar
Attachments:
array_agg_anyarray-11.patchtext/x-patch; charset=US-ASCII; name=array_agg_anyarray-11.patchDownload
diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml
index 7e5bcd9..f59738a 100644
--- a/doc/src/sgml/func.sgml
+++ b/doc/src/sgml/func.sgml
@@ -12046,6 +12046,22 @@ NULL baz</literallayout>(3 rows)</entry>
<row>
<entry>
<indexterm>
+ <primary>array_agg</primary>
+ </indexterm>
+ <function>array_agg(<replaceable class="parameter">anyarray</replaceable>)</function>
+ </entry>
+ <entry>
+ any
+ </entry>
+ <entry>
+ the same array type as input type
+ </entry>
+ <entry>input arrays, aggregated into higher-order multidimesional array. Rejects NULL and empty array as input.</entry>
+ </row>
+
+ <row>
+ <entry>
+ <indexterm>
<primary>average</primary>
</indexterm>
<indexterm>
diff --git a/doc/src/sgml/syntax.sgml b/doc/src/sgml/syntax.sgml
index 2f0680f..8c182a4 100644
--- a/doc/src/sgml/syntax.sgml
+++ b/doc/src/sgml/syntax.sgml
@@ -2238,6 +2238,11 @@ SELECT ARRAY(SELECT oid FROM pg_proc WHERE proname LIKE 'bytea%');
array
-----------------------------------------------------------------------
{2011,1954,1948,1952,1951,1244,1950,2005,1949,1953,2006,31,2412,2413}
+
+SELECT ARRAY(SELECT array(select i) FROM generate_series(1,5) a(i));
+ array
+-----------------------
+ {{1},{2},{3},{4},{5}}
(1 row)
</programlisting>
The subquery must return a single column. The resulting
diff --git a/src/backend/executor/nodeSubplan.c b/src/backend/executor/nodeSubplan.c
index 401bad4..eb4de3b 100644
--- a/src/backend/executor/nodeSubplan.c
+++ b/src/backend/executor/nodeSubplan.c
@@ -231,7 +231,10 @@ ExecScanSubPlan(SubPlanState *node,
bool found = false; /* TRUE if got at least one subplan tuple */
ListCell *pvar;
ListCell *l;
+ bool use_md_array_builder;
+ Oid md_array_element_type;
ArrayBuildState *astate = NULL;
+ MdArrayBuildState *mdastate = NULL;
/*
* MULTIEXPR subplans, when "executed", just return NULL; but first we
@@ -366,8 +369,25 @@ ExecScanSubPlan(SubPlanState *node,
/* stash away current value */
Assert(subplan->firstColType == tdesc->attrs[0]->atttypid);
dvalue = slot_getattr(slot, 1, &disnull);
- astate = accumArrayResult(astate, dvalue, disnull,
- subplan->firstColType, oldcontext);
+ /*
+ * use a fast array multidimensional builder when input is a array
+ * only check on first iteration. On subsequent, use the cached values
+ */
+ if (astate == NULL && mdastate == NULL)
+ {
+ md_array_element_type = get_element_type(subplan->firstColType);
+ use_md_array_builder = OidIsValid(md_array_element_type);
+ }
+
+ if (use_md_array_builder)
+ mdastate = accumMdArray(mdastate,
+ disnull? NULL :
+ DatumGetArrayTypeP(dvalue),
+ disnull, md_array_element_type,
+ oldcontext);
+ else
+ astate = accumArrayResult(astate, dvalue, disnull,
+ subplan->firstColType, oldcontext);
/* keep scanning subplan to collect all values */
continue;
}
@@ -439,6 +459,8 @@ ExecScanSubPlan(SubPlanState *node,
/* We return the result in the caller's context */
if (astate != NULL)
result = makeArrayResult(astate, oldcontext);
+ else if (mdastate != NULL)
+ result = makeMdArray(mdastate, oldcontext);
else
result = PointerGetDatum(construct_empty_array(subplan->firstColType));
}
@@ -951,7 +973,10 @@ ExecSetParamPlan(SubPlanState *node, ExprContext *econtext)
ListCell *pvar;
ListCell *l;
bool found = false;
+ bool use_md_array_builder;
+ Oid md_array_element_type;
ArrayBuildState *astate = NULL;
+ MdArrayBuildState *mdastate = NULL;
if (subLinkType == ANY_SUBLINK ||
subLinkType == ALL_SUBLINK)
@@ -1018,8 +1043,25 @@ ExecSetParamPlan(SubPlanState *node, ExprContext *econtext)
/* stash away current value */
Assert(subplan->firstColType == tdesc->attrs[0]->atttypid);
dvalue = slot_getattr(slot, 1, &disnull);
- astate = accumArrayResult(astate, dvalue, disnull,
- subplan->firstColType, oldcontext);
+ /*
+ * use a fast array multidimensional builder when input is a array
+ * only check on first iteration. On subsequent, use the cached values
+ */
+ if (astate == NULL && mdastate == NULL)
+ {
+ md_array_element_type = get_element_type(subplan->firstColType);
+ use_md_array_builder = OidIsValid(md_array_element_type);
+ }
+
+ if (use_md_array_builder)
+ mdastate = accumMdArray(mdastate,
+ disnull? NULL :
+ DatumGetArrayTypeP(dvalue),
+ disnull, md_array_element_type,
+ oldcontext);
+ else
+ astate = accumArrayResult(astate, dvalue, disnull,
+ subplan->firstColType, oldcontext);
/* keep scanning subplan to collect all values */
continue;
}
@@ -1075,6 +1117,9 @@ ExecSetParamPlan(SubPlanState *node, ExprContext *econtext)
if (astate != NULL)
node->curArray = makeArrayResult(astate,
econtext->ecxt_per_query_memory);
+ else if (mdastate != NULL)
+ node->curArray = makeMdArray(mdastate,
+ econtext->ecxt_per_query_memory);
else
{
MemoryContextSwitchTo(econtext->ecxt_per_query_memory);
diff --git a/src/backend/nodes/nodeFuncs.c b/src/backend/nodes/nodeFuncs.c
index 41e973b..0261fcb 100644
--- a/src/backend/nodes/nodeFuncs.c
+++ b/src/backend/nodes/nodeFuncs.c
@@ -108,12 +108,16 @@ exprType(const Node *expr)
type = exprType((Node *) tent->expr);
if (sublink->subLinkType == ARRAY_SUBLINK)
{
- type = get_array_type(type);
- if (!OidIsValid(type))
- ereport(ERROR,
- (errcode(ERRCODE_UNDEFINED_OBJECT),
- errmsg("could not find array type for data type %s",
- format_type_be(exprType((Node *) tent->expr)))));
+ if (!OidIsValid(get_element_type(type)))
+ {
+ /* not array, so check for its array type */
+ type = get_array_type(type);
+ if (!OidIsValid(type))
+ ereport(ERROR,
+ (errcode(ERRCODE_UNDEFINED_OBJECT),
+ errmsg("could not find array type for data type %s",
+ format_type_be(exprType((Node *) tent->expr)))));
+ }
}
}
else if (sublink->subLinkType == MULTIEXPR_SUBLINK)
@@ -139,12 +143,16 @@ exprType(const Node *expr)
type = subplan->firstColType;
if (subplan->subLinkType == ARRAY_SUBLINK)
{
- type = get_array_type(type);
- if (!OidIsValid(type))
- ereport(ERROR,
- (errcode(ERRCODE_UNDEFINED_OBJECT),
- errmsg("could not find array type for data type %s",
- format_type_be(subplan->firstColType))));
+ if (!OidIsValid(get_element_type(type)))
+ {
+ /* not array, so check for its array type */
+ type = get_array_type(type);
+ if (!OidIsValid(type))
+ ereport(ERROR,
+ (errcode(ERRCODE_UNDEFINED_OBJECT),
+ errmsg("could not find array type for data type %s",
+ format_type_be(subplan->firstColType))));
+ }
}
}
else if (subplan->subLinkType == MULTIEXPR_SUBLINK)
diff --git a/src/backend/optimizer/plan/subselect.c b/src/backend/optimizer/plan/subselect.c
index 3e7dc85..8fc8b49 100644
--- a/src/backend/optimizer/plan/subselect.c
+++ b/src/backend/optimizer/plan/subselect.c
@@ -668,10 +668,16 @@ build_subplan(PlannerInfo *root, Plan *plan, PlannerInfo *subroot,
Assert(!te->resjunk);
Assert(testexpr == NULL);
- arraytype = get_array_type(exprType((Node *) te->expr));
- if (!OidIsValid(arraytype))
- elog(ERROR, "could not find array type for datatype %s",
- format_type_be(exprType((Node *) te->expr)));
+
+ arraytype = exprType((Node *) te->expr);
+ if (!OidIsValid(get_element_type(arraytype)))
+ {
+ /* not array, so get the array type */
+ arraytype = get_array_type(exprType((Node *) te->expr));
+ if (!OidIsValid(arraytype))
+ elog(ERROR, "could not find array type for datatype %s",
+ format_type_be(exprType((Node *) te->expr)));
+ }
prm = generate_new_param(root,
arraytype,
exprTypmod((Node *) te->expr),
diff --git a/src/backend/utils/adt/array_userfuncs.c b/src/backend/utils/adt/array_userfuncs.c
index 831466d..dcb624c 100644
--- a/src/backend/utils/adt/array_userfuncs.c
+++ b/src/backend/utils/adt/array_userfuncs.c
@@ -16,6 +16,8 @@
#include "utils/builtins.h"
#include "utils/lsyscache.h"
+static Datum array_agg_transfn_internal(PG_FUNCTION_ARGS,
+ const char* fname);
/*-----------------------------------------------------------------------------
* array_push :
@@ -513,8 +515,8 @@ array_agg_finalfn(PG_FUNCTION_ARGS)
{
Datum result;
ArrayBuildState *state;
- int dims[1];
- int lbs[1];
+ int dims[0];
+ int lbs[0];
/*
* Test for null before Asserting we are in right context. This is to
@@ -529,8 +531,76 @@ array_agg_finalfn(PG_FUNCTION_ARGS)
state = (ArrayBuildState *) PG_GETARG_POINTER(0);
+ /*
+ * Make the result. We cannot release the ArrayBuildState because
+ * sometimes aggregate final functions are re-executed. Rather, it is
+ * nodeAgg.c's responsibility to reset the aggcontext when it's safe to do
+ * so.
+ */
dims[0] = state->nelems;
lbs[0] = 1;
+ result = makeMdArrayResult(state, 1, dims, lbs, CurrentMemoryContext, false);
+
+ PG_RETURN_DATUM(result);
+}
+
+/*
+ * ARRAY_AGG(anyarray) aggregate function
+ */
+Datum
+array_agg_anyarray_transfn(PG_FUNCTION_ARGS)
+{
+ Oid arg1_typeid = get_fn_expr_argtype(fcinfo->flinfo, 1);
+ Oid element_type = get_element_type(arg1_typeid);
+ MemoryContext aggcontext;
+ MdArrayBuildState *state;
+ Datum elem;
+
+ if (arg1_typeid == InvalidOid)
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("could not determine input data type")));
+
+ if (!AggCheckCallContext(fcinfo, &aggcontext))
+ {
+ /* cannot be called directly because of internal-type argument */
+ elog(ERROR, "array_agg_anyarray_transfn called in non-aggregate context");
+ }
+
+ state = PG_ARGISNULL(0) ? NULL : (MdArrayBuildState *) PG_GETARG_POINTER(0);
+ elem = PG_ARGISNULL(1) ? (Datum) 0 : PG_GETARG_ARRAYTYPE_P(1);
+ state = accumMdArray(state,
+ elem,
+ PG_ARGISNULL(1),
+ element_type,
+ aggcontext);
+
+ /*
+ * The transition type for array_agg() is declared to be "internal", which
+ * is a pass-by-value type the same size as a pointer. So we can safely
+ * pass the ArrayBuildState pointer through nodeAgg.c's machinations.
+ */
+ PG_RETURN_POINTER(state);
+}
+
+Datum
+array_agg_anyarray_finalfn(PG_FUNCTION_ARGS)
+{
+ Datum result;
+ MdArrayBuildState *state;
+
+ /*
+ * Test for null before Asserting we are in right context. This is to
+ * avoid possible Assert failure in 8.4beta installations, where it is
+ * possible for users to create NULL constants of type internal.
+ */
+ if (PG_ARGISNULL(0))
+ PG_RETURN_NULL(); /* returns null iff no input values */
+
+ /* cannot be called directly because of internal-type argument */
+ Assert(AggCheckCallContext(fcinfo, NULL));
+
+ state = (MdArrayBuildState *) PG_GETARG_POINTER(0);
/*
* Make the result. We cannot release the ArrayBuildState because
@@ -538,9 +608,7 @@ array_agg_finalfn(PG_FUNCTION_ARGS)
* nodeAgg.c's responsibility to reset the aggcontext when it's safe to do
* so.
*/
- result = makeMdArrayResult(state, 1, dims, lbs,
- CurrentMemoryContext,
- false);
+ result = makeMdCustomArray(state, -1, NULL, NULL, CurrentMemoryContext, false);
PG_RETURN_DATUM(result);
}
diff --git a/src/backend/utils/adt/arrayfuncs.c b/src/backend/utils/adt/arrayfuncs.c
index 6c8b41d..1c922d5 100644
--- a/src/backend/utils/adt/arrayfuncs.c
+++ b/src/backend/utils/adt/arrayfuncs.c
@@ -4576,6 +4576,9 @@ array_insert_slice(ArrayType *destArray,
/*
* accumArrayResult - accumulate one (more) Datum for an array result
*
+ * Beware: use this only for scalar datums. For Array datums, use
+ * accumMdArray and friends.
+ *
* astate is working state (NULL on first call)
* rcontext is where to keep working state
*/
@@ -4588,6 +4591,9 @@ accumArrayResult(ArrayBuildState *astate,
MemoryContext arr_context,
oldcontext;
+ /* element_type must not be an array! */
+ Assert(!OidIsValid(get_element_type(element_type)));
+
if (astate == NULL)
{
/* First time through --- initialize */
@@ -4713,6 +4719,269 @@ makeMdArrayResult(ArrayBuildState *astate,
return PointerGetDatum(result);
}
+/*
+ * accumMdArray - accumulate one (more) array Datum for accumulation
+ *
+ * astate is working state (NULL on first call)
+ * element_type is the type that array hold
+ * rcontext is where to keep working state
+ */
+MdArrayBuildState *
+accumMdArray(MdArrayBuildState *astate,
+ ArrayType *arg, bool disnull,
+ Oid element_type,
+ MemoryContext rcontext)
+{
+ return accumMdCustomArray(astate, arg, disnull, element_type,
+ true, rcontext);
+}
+
+/*
+ * accumMdCustomArray - accumMdArray with option to not check
+ * dims and lbs
+ *
+ * astate is working state (NULL on first call)
+ * element_type is the type that array hold
+ * check_dims_and_lbs is the option to check dims and lbs
+ * rcontext is where to keep working state
+ */
+MdArrayBuildState *
+accumMdCustomArray(MdArrayBuildState *astate,
+ ArrayType *arg, bool disnull,
+ Oid element_type,
+ bool check_dims_and_lbs,
+ MemoryContext rcontext)
+{
+ MemoryContext arr_context,
+ oldcontext;
+
+ int *dims,
+ *lbs,
+ ndims,
+ nitems,
+ ndatabytes;
+ char *data;
+ int i;
+
+ /* element_type must not be an array! */
+ Assert(!OidIsValid(get_element_type(element_type)));
+
+ if (disnull)
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("cannot accumulate null arrays")));
+
+ ndims = ARR_NDIM(arg);
+ dims = ARR_DIMS(arg);
+ lbs = ARR_LBOUND(arg);
+ data = ARR_DATA_PTR(arg);
+ nitems = ArrayGetNItems(ndims, dims);
+ ndatabytes = ARR_SIZE(arg) - ARR_DATA_OFFSET(arg);
+
+ if (ndims == 0)
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("cannot accumulate empty arrays")));
+
+ if (ndims + 1 > MAXDIM)
+ ereport(ERROR,
+ (errcode(ERRCODE_PROGRAM_LIMIT_EXCEEDED),
+ errmsg("number of array dimensions (%d) exceeds the maximum allowed (%d)",
+ ndims + 1, MAXDIM)));
+
+ if (astate == NULL)
+ {
+ /* Make a temporary context to hold all the junk */
+ arr_context = AllocSetContextCreate(rcontext,
+ "accumArrayResult",
+ ALLOCSET_DEFAULT_MINSIZE,
+ ALLOCSET_DEFAULT_INITSIZE,
+ ALLOCSET_DEFAULT_MAXSIZE);
+ oldcontext = MemoryContextSwitchTo(arr_context);
+
+ /* array accumulate */
+ astate = (MdArrayBuildState *) palloc(sizeof(MdArrayBuildState));
+ astate->mcontext = arr_context;
+
+ /*
+ * the resulting dimensions is n + 1, with first dimension is the number of
+ * array accumulated
+ */
+ astate->ndims = ndims + 1;
+ astate->dims = (int *) palloc((astate->ndims) * sizeof(int));
+ astate->lbs = (int *) palloc((astate->ndims) * sizeof(int));
+ astate->dims[0] = 0;
+ memcpy(&astate->dims[1], dims, ndims * sizeof(int));
+ astate->lbs[0] = 1;
+ memcpy(&astate->lbs[1], lbs, ndims * sizeof(int));
+
+ astate->aitems = 0;
+ astate->nbytes = 0;
+ astate->hasnull = false;
+ astate->nullbitmap = NULL;
+ astate->nitems = 0;
+
+ astate->abytes = ndatabytes >= 512 ? 4 * ndatabytes : 1024;
+ astate->data = (char *) palloc(astate->abytes);
+
+ astate->element_type = element_type;
+ get_typlenbyvalalign(element_type,
+ &astate->typlen,
+ &astate->typbyval,
+ &astate->typalign);
+ }
+ else
+ {
+ oldcontext = MemoryContextSwitchTo(astate->mcontext);
+
+ Assert(astate->element_type == element_type);
+
+ if (check_dims_and_lbs)
+ {
+ if (astate->ndims != ndims + 1)
+ ereport(ERROR,
+ (errcode(ERRCODE_ARRAY_SUBSCRIPT_ERROR),
+ errmsg("cannot accumulate incompatible arrays"),
+ errdetail("Arrays of %d and %d dimensions are not "
+ "compatible for accumulation.",
+ astate->ndims - 1, ndims)));
+
+ for (i = 0; i < ndims; i++)
+ if (astate->dims[i+1] != dims[i] || astate->lbs[i+1] != lbs[i])
+ ereport(ERROR,
+ (errcode(ERRCODE_ARRAY_SUBSCRIPT_ERROR),
+ errmsg("cannot accumulate incompatible arrays"),
+ errdetail("Arrays with differing element dimensions are "
+ "not compatible for concatenation.")));
+ }
+
+ /* grow array if needed */
+ if (astate->nbytes + ndatabytes >= astate->abytes)
+ {
+ astate->abytes *= 2;
+ astate->data = (char *)
+ repalloc(astate->data, astate->abytes);
+ }
+ if (astate->hasnull && astate->nitems + nitems >= astate->aitems)
+ {
+ astate->aitems *= 2;
+ astate->nullbitmap = (bits8 *)
+ repalloc(astate->nullbitmap, (astate->aitems + 7) / 8);
+ }
+ }
+
+ memcpy(astate->data + astate->nbytes, data, ndatabytes);
+ astate->nbytes += ndatabytes;
+
+ if (astate->hasnull || ARR_HASNULL(arg))
+ {
+ if (!astate->hasnull)
+ {
+ /* arbitrary number if nitems below 32 */
+ astate->aitems = astate->nitems < 32 ? 64 : astate->nitems * 8;
+ astate->hasnull = true;
+ astate->nullbitmap = (bits8 *) palloc((astate->aitems + 7) / 8);
+ array_bitmap_copy(astate->nullbitmap, 0,
+ NULL, 0,
+ astate->nitems);
+ }
+ array_bitmap_copy(astate->nullbitmap, astate->nitems,
+ ARR_NULLBITMAP(arg), 0,
+ nitems);
+ astate->nitems += nitems;
+ }
+ astate->dims[0] += 1;
+
+ MemoryContextSwitchTo(oldcontext);
+
+ return astate;
+}
+
+/*
+ * makeMdArray - produce N+1-D final result of accumMdArray
+ *
+ * astate is working state (not NULL)
+ * rcontext is where to construct result
+ */
+Datum
+makeMdArray(MdArrayBuildState *astate,
+ MemoryContext rcontext)
+{
+ return makeMdCustomArray(astate, -1, NULL, NULL, rcontext, true);
+}
+
+/*
+ * makeMdCustomArray - produce custom-D final result of accumMdArray
+ *
+ * beware: no check that specified dimensions match the number of values
+ * accumulated.
+ *
+ * astate is working state (not NULL)
+ * rcontext is where to construct result
+ * release is true if okay to release working state
+ * ndims = -1 will produce N+1-D array, with dims and lbs value ignored
+ */
+Datum
+makeMdCustomArray(MdArrayBuildState *astate,
+ int ndims,
+ int *dims,
+ int *lbs,
+ MemoryContext rcontext,
+ bool release)
+{
+ ArrayType *result;
+ MemoryContext oldcontext;
+
+ int dataoffset,
+ nbytes;
+
+ /* Build the final array result in rcontext */
+ oldcontext = MemoryContextSwitchTo(rcontext);
+
+ if (ndims == -1)
+ {
+ ndims = astate->ndims;
+ dims = astate->dims;
+ lbs = astate->lbs;
+ }
+
+ nbytes = astate->nbytes;
+ /* compute required space */
+ if (astate->hasnull)
+ {
+ dataoffset = ARR_OVERHEAD_WITHNULLS(ndims, astate->nitems);
+ nbytes += dataoffset;
+ }
+ else
+ {
+ dataoffset = 0;
+ nbytes += ARR_OVERHEAD_NONULLS(ndims);
+ }
+
+ result = (ArrayType *) palloc0(nbytes);
+ SET_VARSIZE(result, nbytes);
+ result->ndim = ndims;
+ result->dataoffset = dataoffset;
+ result->elemtype = astate->element_type;
+
+ memcpy(ARR_DIMS(result), dims, ndims * sizeof(int));
+ memcpy(ARR_LBOUND(result), lbs, ndims * sizeof(int));
+ memcpy(ARR_DATA_PTR(result), astate->data, astate->nbytes);
+
+ if (astate->hasnull)
+ array_bitmap_copy(ARR_NULLBITMAP(result), 0,
+ astate->nullbitmap, 0,
+ astate->nitems);
+
+ MemoryContextSwitchTo(oldcontext);
+
+ /* Clean up all the junk */
+ if (release)
+ MemoryContextDelete(astate->mcontext);
+
+ return PointerGetDatum(result);
+}
+
Datum
array_larger(PG_FUNCTION_ARGS)
{
diff --git a/src/include/catalog/pg_aggregate.h b/src/include/catalog/pg_aggregate.h
index 3ba9e5e..2005199 100644
--- a/src/include/catalog/pg_aggregate.h
+++ b/src/include/catalog/pg_aggregate.h
@@ -275,6 +275,7 @@ DATA(insert ( 2901 n 0 xmlconcat2 - - - - f f 0 142 0 0 0 _null_
/* array */
DATA(insert ( 2335 n 0 array_agg_transfn array_agg_finalfn - - - t f 0 2281 0 0 0 _null_ _null_ ));
+DATA(insert ( 6005 n 0 array_agg_anyarray_transfn array_agg_anyarray_finalfn - - - t f 0 2281 0 0 0 _null_ _null_ ));
/* text */
DATA(insert ( 3538 n 0 string_agg_transfn string_agg_finalfn - - - f f 0 2281 0 0 0 _null_ _null_ ));
diff --git a/src/include/catalog/pg_proc.h b/src/include/catalog/pg_proc.h
index b6dc1b8..9273c1f 100644
--- a/src/include/catalog/pg_proc.h
+++ b/src/include/catalog/pg_proc.h
@@ -879,11 +879,17 @@ DATA(insert OID = 3167 ( array_remove PGNSP PGUID 12 1 0 0 0 f f f f f f i 2
DESCR("remove any occurrences of an element from an array");
DATA(insert OID = 3168 ( array_replace PGNSP PGUID 12 1 0 0 0 f f f f f f i 3 0 2277 "2277 2283 2283" _null_ _null_ _null_ _null_ array_replace _null_ _null_ _null_ ));
DESCR("replace any occurrences of an element in an array");
-DATA(insert OID = 2333 ( array_agg_transfn PGNSP PGUID 12 1 0 0 0 f f f f f f i 2 0 2281 "2281 2283" _null_ _null_ _null_ _null_ array_agg_transfn _null_ _null_ _null_ ));
+DATA(insert OID = 2333 ( array_agg_transfn PGNSP PGUID 12 1 0 0 0 f f f f f f i 2 0 2281 "2281 2776" _null_ _null_ _null_ _null_ array_agg_transfn _null_ _null_ _null_ ));
DESCR("aggregate transition function");
-DATA(insert OID = 2334 ( array_agg_finalfn PGNSP PGUID 12 1 0 0 0 f f f f f f i 2 0 2277 "2281 2283" _null_ _null_ _null_ _null_ array_agg_finalfn _null_ _null_ _null_ ));
+DATA(insert OID = 2334 ( array_agg_finalfn PGNSP PGUID 12 1 0 0 0 f f f f f f i 2 0 2277 "2281 2776" _null_ _null_ _null_ _null_ array_agg_finalfn _null_ _null_ _null_ ));
DESCR("aggregate final function");
-DATA(insert OID = 2335 ( array_agg PGNSP PGUID 12 1 0 0 0 t f f f f f i 1 0 2277 "2283" _null_ _null_ _null_ _null_ aggregate_dummy _null_ _null_ _null_ ));
+DATA(insert OID = 2335 ( array_agg PGNSP PGUID 12 1 0 0 0 t f f f f f i 1 0 2277 "2776" _null_ _null_ _null_ _null_ aggregate_dummy _null_ _null_ _null_ ));
+DESCR("concatenate aggregate input into an array");
+DATA(insert OID = 6003 ( array_agg_anyarray_transfn PGNSP PGUID 12 1 0 0 0 f f f f f f i 2 0 2281 "2281 2277" _null_ _null_ _null_ _null_ array_agg_anyarray_transfn _null_ _null_ _null_ ));
+DESCR("aggregate transition function");
+DATA(insert OID = 6004 ( array_agg_anyarray_finalfn PGNSP PGUID 12 1 0 0 0 f f f f f f i 2 0 2277 "2281 2277" _null_ _null_ _null_ _null_ array_agg_anyarray_finalfn _null_ _null_ _null_ ));
+DESCR("aggregate final function");
+DATA(insert OID = 6005 ( array_agg PGNSP PGUID 12 1 0 0 0 t f f f f f i 1 0 2277 "2277" _null_ _null_ _null_ _null_ aggregate_dummy _null_ _null_ _null_ ));
DESCR("concatenate aggregate input into an array");
DATA(insert OID = 3218 ( width_bucket PGNSP PGUID 12 1 0 0 0 f f f f t f i 2 0 23 "2283 2277" _null_ _null_ _null_ _null_ width_bucket_array _null_ _null_ _null_ ));
DESCR("bucket number of operand given a sorted array of bucket lower bounds");
diff --git a/src/include/utils/array.h b/src/include/utils/array.h
index e744314..b8f0fe1 100644
--- a/src/include/utils/array.h
+++ b/src/include/utils/array.h
@@ -75,7 +75,8 @@ typedef struct
} ArrayType;
/*
- * working state for accumArrayResult() and friends
+ * working state for array accumulation of scalar datums
+ * with accumArrayResult() and friends
*/
typedef struct ArrayBuildState
{
@@ -91,6 +92,34 @@ typedef struct ArrayBuildState
} ArrayBuildState;
/*
+ * array build state for array accumulation of array datums
+ * with accumMdArray() and friends
+ */
+typedef struct
+{
+ MemoryContext mcontext; /* where all the temp stuff is kept */
+
+ char *data; /* array of accumulated data */
+ bits8 *nullbitmap; /* bitmap of is-null flags for data */
+
+ int abytes; /* allocated length of above arrays */
+ int aitems; /* allocated length of above arrays */
+ int nbytes; /* number of used bytes in above arrays */
+ int nitems; /* number of elements in above arrays */
+
+ int ndims; /* element dimensions */
+ int *dims;
+ int *lbs;
+
+ bool hasnull; /* any element has null */
+
+ Oid element_type; /* data type of the Datums */
+ int16 typlen; /* needed info about datatype */
+ bool typbyval;
+ char typalign;
+} MdArrayBuildState;
+
+/*
* structure to cache type metadata needed for array manipulation
*/
typedef struct ArrayMetaState
@@ -260,6 +289,19 @@ extern Datum makeArrayResult(ArrayBuildState *astate,
MemoryContext rcontext);
extern Datum makeMdArrayResult(ArrayBuildState *astate, int ndims,
int *dims, int *lbs, MemoryContext rcontext, bool release);
+extern MdArrayBuildState *accumMdArray(MdArrayBuildState *astate,
+ ArrayType *arg, bool disnull,
+ Oid element_type,
+ MemoryContext rcontext);
+extern MdArrayBuildState *accumMdCustomArray(MdArrayBuildState *astate,
+ ArrayType *arg, bool disnull,
+ Oid element_type,
+ bool check_dims_and_lbs,
+ MemoryContext rcontext);
+extern Datum makeMdArray(MdArrayBuildState *astate,
+ MemoryContext rcontext);
+extern Datum makeMdCustomArray(MdArrayBuildState *astate, int ndims,
+ int *dims, int *lbs, MemoryContext rcontext, bool release);
extern ArrayIterator array_create_iterator(ArrayType *arr, int slice_ndim);
extern bool array_iterate(ArrayIterator iterator, Datum *value, bool *isnull);
@@ -292,6 +334,8 @@ extern ArrayType *create_singleton_array(FunctionCallInfo fcinfo,
extern Datum array_agg_transfn(PG_FUNCTION_ARGS);
extern Datum array_agg_finalfn(PG_FUNCTION_ARGS);
+extern Datum array_agg_anyarray_transfn(PG_FUNCTION_ARGS);
+extern Datum array_agg_anyarray_finalfn(PG_FUNCTION_ARGS);
/*
* prototypes for functions defined in array_typanalyze.c
diff --git a/src/test/regress/expected/aggregates.out b/src/test/regress/expected/aggregates.out
index 58df854..607aeea 100644
--- a/src/test/regress/expected/aggregates.out
+++ b/src/test/regress/expected/aggregates.out
@@ -914,6 +914,76 @@ select array_agg(distinct a order by a desc nulls last)
{3,2,1,NULL}
(1 row)
+-- array_agg(anyarray)
+select array_agg(ar)
+ from (values ('{1,2}'::int[]), ('{3,4}'::int[])) v(ar);
+ array_agg
+---------------
+ {{1,2},{3,4}}
+(1 row)
+
+select array_agg(distinct ar order by ar desc)
+ from (select array[i / 2] from generate_series(1,10) a(i)) b(ar);
+ array_agg
+---------------------------
+ {{5},{4},{3},{2},{1},{0}}
+(1 row)
+
+select array_agg(ar)
+ from (select array_agg(array[i, i+1, i-1])
+ from generate_series(1,2) a(i)) b(ar);
+ array_agg
+---------------------
+ {{{1,2,0},{2,3,1}}}
+(1 row)
+
+-- array_agg(anyarray), varlena types
+select array_agg(array[1.2,1.3,1.4]) from generate_series(1,3);
+ array_agg
+---------------------------------------------
+ {{1.2,1.3,1.4},{1.2,1.3,1.4},{1.2,1.3,1.4}}
+(1 row)
+
+select array_agg(array['Hello','Hohoho','Hi']) from generate_series(1,3);
+ array_agg
+---------------------------------------------------------
+ {{Hello,Hohoho,Hi},{Hello,Hohoho,Hi},{Hello,Hohoho,Hi}}
+(1 row)
+
+-- array_agg(anyarray), arrays with nulls
+select array_agg(array[i, null, i+1, null, i+2]) from generate_series(1,3) g(i);
+ array_agg
+---------------------------------------------------------
+ {{1,NULL,2,NULL,3},{2,NULL,3,NULL,4},{3,NULL,4,NULL,5}}
+(1 row)
+
+select array_agg(array[1.1+ i, null, 1.1+i+1, null, 1.1+i+2]) from generate_series(1,3) g(i);
+ array_agg
+---------------------------------------------------------------------------
+ {{2.1,NULL,3.1,NULL,4.1},{3.1,NULL,4.1,NULL,5.1},{4.1,NULL,5.1,NULL,6.1}}
+(1 row)
+
+select array_agg(array[null, 'Hello','Hohoho', null,'Hi']) from generate_series(1,3);
+ array_agg
+---------------------------------------------------------------------------------------
+ {{NULL,Hello,Hohoho,NULL,Hi},{NULL,Hello,Hohoho,NULL,Hi},{NULL,Hello,Hohoho,NULL,Hi}}
+(1 row)
+
+select array_agg(array[[null, 'Hello', null, 'Hi'],['Hello', null, 'Hi', null]]) from generate_series(1,2);
+ array_agg
+-------------------------------------------------------------------------------------------
+ {{{NULL,Hello,NULL,Hi},{Hello,NULL,Hi,NULL}},{{NULL,Hello,NULL,Hi},{Hello,NULL,Hi,NULL}}}
+(1 row)
+
+-- errors
+select array_agg('{}'::int[]) from generate_series(1,2);
+ERROR: cannot accumulate empty arrays
+select array_agg(null::int[]) from generate_series(1,2);
+ERROR: cannot accumulate null arrays
+select array_agg(ar)
+ from (values ('{1,2}'::int[]), ('{3}'::int[])) v(ar);
+ERROR: cannot accumulate incompatible arrays
+DETAIL: Arrays with differing element dimensions are not compatible for concatenation.
-- multi-arg aggs, strict/nonstrict, distinct/order by
select aggfstr(a,b,c)
from (values (1,3,'foo'),(0,null,null),(2,2,'bar'),(3,1,'baz')) v(a,b,c);
diff --git a/src/test/regress/expected/arrays.out b/src/test/regress/expected/arrays.out
index 46eff67..e80ebec 100644
--- a/src/test/regress/expected/arrays.out
+++ b/src/test/regress/expected/arrays.out
@@ -1521,6 +1521,23 @@ select array_agg(unique1) from tenk1 where unique1 < -15;
(1 row)
+select array(select unique1 from tenk1 where unique1 < 15 order by unique1);
+ array
+--------------------------------------
+ {0,1,2,3,4,5,6,7,8,9,10,11,12,13,14}
+(1 row)
+
+select array(select array[i,i/2] from generate_series(1,5) a(i));
+ array
+---------------------------------
+ {{1,0},{2,1},{3,1},{4,2},{5,2}}
+(1 row)
+
+-- cannot accumulate null arrays and empty arrays
+select array(select null::int[]);
+ERROR: cannot accumulate null arrays
+select array(select '{}'::int[]);
+ERROR: cannot accumulate empty arrays
select unnest(array[1,2,3]);
unnest
--------
diff --git a/src/test/regress/sql/aggregates.sql b/src/test/regress/sql/aggregates.sql
index 8096a6f..a70419f 100644
--- a/src/test/regress/sql/aggregates.sql
+++ b/src/test/regress/sql/aggregates.sql
@@ -322,6 +322,32 @@ select array_agg(distinct a order by a desc)
select array_agg(distinct a order by a desc nulls last)
from (values (1),(2),(1),(3),(null),(2)) v(a);
+-- array_agg(anyarray)
+select array_agg(ar)
+ from (values ('{1,2}'::int[]), ('{3,4}'::int[])) v(ar);
+select array_agg(distinct ar order by ar desc)
+ from (select array[i / 2] from generate_series(1,10) a(i)) b(ar);
+select array_agg(ar)
+ from (select array_agg(array[i, i+1, i-1])
+ from generate_series(1,2) a(i)) b(ar);
+
+-- array_agg(anyarray), varlena types
+select array_agg(array[1.2,1.3,1.4]) from generate_series(1,3);
+select array_agg(array['Hello','Hohoho','Hi']) from generate_series(1,3);
+
+-- array_agg(anyarray), arrays with nulls
+select array_agg(array[i, null, i+1, null, i+2]) from generate_series(1,3) g(i);
+select array_agg(array[1.1+ i, null, 1.1+i+1, null, 1.1+i+2]) from generate_series(1,3) g(i);
+select array_agg(array[null, 'Hello','Hohoho', null,'Hi']) from generate_series(1,3);
+
+select array_agg(array[[null, 'Hello', null, 'Hi'],['Hello', null, 'Hi', null]]) from generate_series(1,2);
+
+-- errors
+select array_agg('{}'::int[]) from generate_series(1,2);
+select array_agg(null::int[]) from generate_series(1,2);
+select array_agg(ar)
+ from (values ('{1,2}'::int[]), ('{3}'::int[])) v(ar);
+
-- multi-arg aggs, strict/nonstrict, distinct/order by
select aggfstr(a,b,c)
diff --git a/src/test/regress/sql/arrays.sql b/src/test/regress/sql/arrays.sql
index fa8a20a..cb00f5f 100644
--- a/src/test/regress/sql/arrays.sql
+++ b/src/test/regress/sql/arrays.sql
@@ -432,6 +432,12 @@ select array_agg(ten) from (select ten from tenk1 where unique1 < 15 order by un
select array_agg(nullif(ten, 4)) from (select ten from tenk1 where unique1 < 15 order by unique1) ss;
select array_agg(unique1) from tenk1 where unique1 < -15;
+select array(select unique1 from tenk1 where unique1 < 15 order by unique1);
+select array(select array[i,i/2] from generate_series(1,5) a(i));
+-- cannot accumulate null arrays and empty arrays
+select array(select null::int[]);
+select array(select '{}'::int[]);
+
select unnest(array[1,2,3]);
select * from unnest(array[1,2,3]);
select unnest(array[1,2,3,4.5]::float8[]);
Re: Function array_agg(array)
Hi
I did some minor changes in code
* move tests of old or new builder style for array sublink out of main
cycles
* some API simplification of new builder - we should not to create
identical API, mainly it has no sense
Regards
Pavel Stehule
2014-10-27 8:12 GMT+01:00 Ali Akbar <the.apaan@gmail.com>:
Show quoted text
2014-10-27 9:11 GMT+07:00 Ali Akbar <the.apaan@gmail.com>:
2014-10-27 1:38 GMT+07:00 Pavel Stehule <pavel.stehule@gmail.com>:
Hi
My idea is using new ArrayBuilder optimized for building
multidimensional arrays with own State type. I think so casting to
ArrayBuildState is base of our problems, so I don't would to do. Code in
array_agg_* is simple, little bit more complex code is in nodeSubplan.c.
Some schematic changes are in attachments.Thanks! The structure looks clear, and thanks for the example on
nodeSubplan.c. I will restructure the v10 of the patch to this structure.Patch attached.
Regards,
--
Ali Akbar
Attachments:
array_agg_anyarray-12.patchtext/x-patch; charset=US-ASCII; name=array_agg_anyarray-12.patchDownload
commit f09f41f5fe780bed4cf25949961a4e68e6402ff0
Author: Pavel Stehule <pavel.stehule@gooddata.com>
Date: Mon Oct 27 10:03:07 2014 +0100
initial
diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml
index 7e5bcd9..f59738a 100644
--- a/doc/src/sgml/func.sgml
+++ b/doc/src/sgml/func.sgml
@@ -12046,6 +12046,22 @@ NULL baz</literallayout>(3 rows)</entry>
<row>
<entry>
<indexterm>
+ <primary>array_agg</primary>
+ </indexterm>
+ <function>array_agg(<replaceable class="parameter">anyarray</replaceable>)</function>
+ </entry>
+ <entry>
+ any
+ </entry>
+ <entry>
+ the same array type as input type
+ </entry>
+ <entry>input arrays, aggregated into higher-order multidimesional array. Rejects NULL and empty array as input.</entry>
+ </row>
+
+ <row>
+ <entry>
+ <indexterm>
<primary>average</primary>
</indexterm>
<indexterm>
diff --git a/doc/src/sgml/syntax.sgml b/doc/src/sgml/syntax.sgml
index 2f0680f..8c182a4 100644
--- a/doc/src/sgml/syntax.sgml
+++ b/doc/src/sgml/syntax.sgml
@@ -2238,6 +2238,11 @@ SELECT ARRAY(SELECT oid FROM pg_proc WHERE proname LIKE 'bytea%');
array
-----------------------------------------------------------------------
{2011,1954,1948,1952,1951,1244,1950,2005,1949,1953,2006,31,2412,2413}
+
+SELECT ARRAY(SELECT array(select i) FROM generate_series(1,5) a(i));
+ array
+-----------------------
+ {{1},{2},{3},{4},{5}}
(1 row)
</programlisting>
The subquery must return a single column. The resulting
diff --git a/src/backend/executor/nodeSubplan.c b/src/backend/executor/nodeSubplan.c
index 401bad4..21b8de1 100644
--- a/src/backend/executor/nodeSubplan.c
+++ b/src/backend/executor/nodeSubplan.c
@@ -231,7 +231,10 @@ ExecScanSubPlan(SubPlanState *node,
bool found = false; /* TRUE if got at least one subplan tuple */
ListCell *pvar;
ListCell *l;
+ bool use_md_array_builder = false;
+ Oid md_array_element_type = InvalidOid;
ArrayBuildState *astate = NULL;
+ MdArrayBuildState *mdastate = NULL;
/*
* MULTIEXPR subplans, when "executed", just return NULL; but first we
@@ -260,6 +263,16 @@ ExecScanSubPlan(SubPlanState *node,
}
/*
+ * use a fast array multidimensional builder when input is a array
+ * only check on first iteration. On subsequent, use the cached values
+ */
+ if (subLinkType == ARRAY_SUBLINK)
+ {
+ md_array_element_type = get_element_type(subplan->firstColType);
+ use_md_array_builder = OidIsValid(md_array_element_type);
+ }
+
+ /*
* We are probably in a short-lived expression-evaluation context. Switch
* to the per-query context for manipulating the child plan's chgParam,
* calling ExecProcNode on it, etc.
@@ -366,8 +379,16 @@ ExecScanSubPlan(SubPlanState *node,
/* stash away current value */
Assert(subplan->firstColType == tdesc->attrs[0]->atttypid);
dvalue = slot_getattr(slot, 1, &disnull);
- astate = accumArrayResult(astate, dvalue, disnull,
- subplan->firstColType, oldcontext);
+
+ if (use_md_array_builder)
+ mdastate = accumMdArray(mdastate,
+ disnull? NULL :
+ DatumGetArrayTypeP(dvalue),
+ disnull, md_array_element_type,
+ oldcontext);
+ else
+ astate = accumArrayResult(astate, dvalue, disnull,
+ subplan->firstColType, oldcontext);
/* keep scanning subplan to collect all values */
continue;
}
@@ -439,6 +460,8 @@ ExecScanSubPlan(SubPlanState *node,
/* We return the result in the caller's context */
if (astate != NULL)
result = makeArrayResult(astate, oldcontext);
+ else if (mdastate != NULL)
+ result = PointerGetDatum(makeMdArray(mdastate, oldcontext, true));
else
result = PointerGetDatum(construct_empty_array(subplan->firstColType));
}
@@ -951,7 +974,10 @@ ExecSetParamPlan(SubPlanState *node, ExprContext *econtext)
ListCell *pvar;
ListCell *l;
bool found = false;
+ bool use_md_array_builder = false;
+ Oid md_array_element_type = InvalidOid;
ArrayBuildState *astate = NULL;
+ MdArrayBuildState *mdastate = NULL;
if (subLinkType == ANY_SUBLINK ||
subLinkType == ALL_SUBLINK)
@@ -960,6 +986,16 @@ ExecSetParamPlan(SubPlanState *node, ExprContext *econtext)
elog(ERROR, "CTE subplans should not be executed via ExecSetParamPlan");
/*
+ * use a fast array multidimensional builder when input is a array
+ * only check on first iteration. On subsequent, use the cached values
+ */
+ if (subLinkType == ARRAY_SUBLINK)
+ {
+ md_array_element_type = get_element_type(subplan->firstColType);
+ use_md_array_builder = OidIsValid(md_array_element_type);
+ }
+
+ /*
* Must switch to per-query memory context.
*/
oldcontext = MemoryContextSwitchTo(econtext->ecxt_per_query_memory);
@@ -1018,8 +1054,16 @@ ExecSetParamPlan(SubPlanState *node, ExprContext *econtext)
/* stash away current value */
Assert(subplan->firstColType == tdesc->attrs[0]->atttypid);
dvalue = slot_getattr(slot, 1, &disnull);
- astate = accumArrayResult(astate, dvalue, disnull,
- subplan->firstColType, oldcontext);
+
+ if (use_md_array_builder)
+ mdastate = accumMdArray(mdastate,
+ disnull? NULL :
+ DatumGetArrayTypeP(dvalue),
+ disnull, md_array_element_type,
+ oldcontext);
+ else
+ astate = accumArrayResult(astate, dvalue, disnull,
+ subplan->firstColType, oldcontext);
/* keep scanning subplan to collect all values */
continue;
}
@@ -1075,6 +1119,9 @@ ExecSetParamPlan(SubPlanState *node, ExprContext *econtext)
if (astate != NULL)
node->curArray = makeArrayResult(astate,
econtext->ecxt_per_query_memory);
+ else if (mdastate != NULL)
+ node->curArray = PointerGetDatum(makeMdArray(mdastate,
+ econtext->ecxt_per_query_memory, true));
else
{
MemoryContextSwitchTo(econtext->ecxt_per_query_memory);
diff --git a/src/backend/nodes/nodeFuncs.c b/src/backend/nodes/nodeFuncs.c
index 41e973b..0261fcb 100644
--- a/src/backend/nodes/nodeFuncs.c
+++ b/src/backend/nodes/nodeFuncs.c
@@ -108,12 +108,16 @@ exprType(const Node *expr)
type = exprType((Node *) tent->expr);
if (sublink->subLinkType == ARRAY_SUBLINK)
{
- type = get_array_type(type);
- if (!OidIsValid(type))
- ereport(ERROR,
- (errcode(ERRCODE_UNDEFINED_OBJECT),
- errmsg("could not find array type for data type %s",
- format_type_be(exprType((Node *) tent->expr)))));
+ if (!OidIsValid(get_element_type(type)))
+ {
+ /* not array, so check for its array type */
+ type = get_array_type(type);
+ if (!OidIsValid(type))
+ ereport(ERROR,
+ (errcode(ERRCODE_UNDEFINED_OBJECT),
+ errmsg("could not find array type for data type %s",
+ format_type_be(exprType((Node *) tent->expr)))));
+ }
}
}
else if (sublink->subLinkType == MULTIEXPR_SUBLINK)
@@ -139,12 +143,16 @@ exprType(const Node *expr)
type = subplan->firstColType;
if (subplan->subLinkType == ARRAY_SUBLINK)
{
- type = get_array_type(type);
- if (!OidIsValid(type))
- ereport(ERROR,
- (errcode(ERRCODE_UNDEFINED_OBJECT),
- errmsg("could not find array type for data type %s",
- format_type_be(subplan->firstColType))));
+ if (!OidIsValid(get_element_type(type)))
+ {
+ /* not array, so check for its array type */
+ type = get_array_type(type);
+ if (!OidIsValid(type))
+ ereport(ERROR,
+ (errcode(ERRCODE_UNDEFINED_OBJECT),
+ errmsg("could not find array type for data type %s",
+ format_type_be(subplan->firstColType))));
+ }
}
}
else if (subplan->subLinkType == MULTIEXPR_SUBLINK)
diff --git a/src/backend/optimizer/plan/subselect.c b/src/backend/optimizer/plan/subselect.c
index 3e7dc85..8fc8b49 100644
--- a/src/backend/optimizer/plan/subselect.c
+++ b/src/backend/optimizer/plan/subselect.c
@@ -668,10 +668,16 @@ build_subplan(PlannerInfo *root, Plan *plan, PlannerInfo *subroot,
Assert(!te->resjunk);
Assert(testexpr == NULL);
- arraytype = get_array_type(exprType((Node *) te->expr));
- if (!OidIsValid(arraytype))
- elog(ERROR, "could not find array type for datatype %s",
- format_type_be(exprType((Node *) te->expr)));
+
+ arraytype = exprType((Node *) te->expr);
+ if (!OidIsValid(get_element_type(arraytype)))
+ {
+ /* not array, so get the array type */
+ arraytype = get_array_type(exprType((Node *) te->expr));
+ if (!OidIsValid(arraytype))
+ elog(ERROR, "could not find array type for datatype %s",
+ format_type_be(exprType((Node *) te->expr)));
+ }
prm = generate_new_param(root,
arraytype,
exprTypmod((Node *) te->expr),
diff --git a/src/backend/utils/adt/array_userfuncs.c b/src/backend/utils/adt/array_userfuncs.c
index 831466d..8066d70 100644
--- a/src/backend/utils/adt/array_userfuncs.c
+++ b/src/backend/utils/adt/array_userfuncs.c
@@ -513,8 +513,8 @@ array_agg_finalfn(PG_FUNCTION_ARGS)
{
Datum result;
ArrayBuildState *state;
- int dims[1];
- int lbs[1];
+ int dims[0];
+ int lbs[0];
/*
* Test for null before Asserting we are in right context. This is to
@@ -529,8 +529,76 @@ array_agg_finalfn(PG_FUNCTION_ARGS)
state = (ArrayBuildState *) PG_GETARG_POINTER(0);
+ /*
+ * Make the result. We cannot release the ArrayBuildState because
+ * sometimes aggregate final functions are re-executed. Rather, it is
+ * nodeAgg.c's responsibility to reset the aggcontext when it's safe to do
+ * so.
+ */
dims[0] = state->nelems;
lbs[0] = 1;
+ result = makeMdArrayResult(state, 1, dims, lbs, CurrentMemoryContext, false);
+
+ PG_RETURN_DATUM(result);
+}
+
+/*
+ * ARRAY_AGG(anyarray) aggregate function
+ */
+Datum
+array_agg_anyarray_transfn(PG_FUNCTION_ARGS)
+{
+ Oid arg1_typeid = get_fn_expr_argtype(fcinfo->flinfo, 1);
+ Oid element_type = get_element_type(arg1_typeid);
+ MemoryContext aggcontext;
+ MdArrayBuildState *state;
+ ArrayType *elem;
+
+ if (arg1_typeid == InvalidOid)
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("could not determine input data type")));
+
+ if (!AggCheckCallContext(fcinfo, &aggcontext))
+ {
+ /* cannot be called directly because of internal-type argument */
+ elog(ERROR, "array_agg_anyarray_transfn called in non-aggregate context");
+ }
+
+ state = PG_ARGISNULL(0) ? NULL : (MdArrayBuildState *) PG_GETARG_POINTER(0);
+ elem = PG_ARGISNULL(1) ? NULL : PG_GETARG_ARRAYTYPE_P(1);
+ state = accumMdArray(state,
+ elem,
+ PG_ARGISNULL(1),
+ element_type,
+ aggcontext);
+
+ /*
+ * The transition type for array_agg() is declared to be "internal", which
+ * is a pass-by-value type the same size as a pointer. So we can safely
+ * pass the ArrayBuildState pointer through nodeAgg.c's machinations.
+ */
+ PG_RETURN_POINTER(state);
+}
+
+Datum
+array_agg_anyarray_finalfn(PG_FUNCTION_ARGS)
+{
+ ArrayType *result;
+ MdArrayBuildState *state;
+
+ /*
+ * Test for null before Asserting we are in right context. This is to
+ * avoid possible Assert failure in 8.4beta installations, where it is
+ * possible for users to create NULL constants of type internal.
+ */
+ if (PG_ARGISNULL(0))
+ PG_RETURN_NULL(); /* returns null iff no input values */
+
+ /* cannot be called directly because of internal-type argument */
+ Assert(AggCheckCallContext(fcinfo, NULL));
+
+ state = (MdArrayBuildState *) PG_GETARG_POINTER(0);
/*
* Make the result. We cannot release the ArrayBuildState because
@@ -538,9 +606,7 @@ array_agg_finalfn(PG_FUNCTION_ARGS)
* nodeAgg.c's responsibility to reset the aggcontext when it's safe to do
* so.
*/
- result = makeMdArrayResult(state, 1, dims, lbs,
- CurrentMemoryContext,
- false);
+ result = makeMdArray(state, CurrentMemoryContext, false);
- PG_RETURN_DATUM(result);
+ PG_RETURN_ARRAYTYPE_P(result);
}
diff --git a/src/backend/utils/adt/arrayfuncs.c b/src/backend/utils/adt/arrayfuncs.c
index 6c8b41d..06f92d8 100644
--- a/src/backend/utils/adt/arrayfuncs.c
+++ b/src/backend/utils/adt/arrayfuncs.c
@@ -4576,6 +4576,9 @@ array_insert_slice(ArrayType *destArray,
/*
* accumArrayResult - accumulate one (more) Datum for an array result
*
+ * Beware: use this only for scalar datums. For Array datums, use
+ * accumMdArray and friends.
+ *
* astate is working state (NULL on first call)
* rcontext is where to keep working state
*/
@@ -4588,6 +4591,9 @@ accumArrayResult(ArrayBuildState *astate,
MemoryContext arr_context,
oldcontext;
+ /* element_type must not be an array! */
+ Assert(!OidIsValid(get_element_type(element_type)));
+
if (astate == NULL)
{
/* First time through --- initialize */
@@ -4713,6 +4719,222 @@ makeMdArrayResult(ArrayBuildState *astate,
return PointerGetDatum(result);
}
+/*
+ * accumMdArray - accumulate one (more) array Datum for accumulation
+ *
+ * astate is working state (NULL on first call)
+ * element_type is the type that array hold
+ * rcontext is where to keep working state
+ */
+MdArrayBuildState *
+accumMdArray(MdArrayBuildState *astate,
+ ArrayType *arg, bool disnull,
+ Oid element_type,
+ MemoryContext rcontext)
+{
+ MemoryContext arr_context,
+ oldcontext;
+
+ int *dims,
+ *lbs,
+ ndims,
+ nitems,
+ ndatabytes;
+ char *data;
+ int i;
+
+ /* element_type must not be an array! */
+ Assert(!OidIsValid(get_element_type(element_type)));
+
+ if (disnull)
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("cannot accumulate null arrays")));
+
+ ndims = ARR_NDIM(arg);
+ dims = ARR_DIMS(arg);
+ lbs = ARR_LBOUND(arg);
+ data = ARR_DATA_PTR(arg);
+ nitems = ArrayGetNItems(ndims, dims);
+ ndatabytes = ARR_SIZE(arg) - ARR_DATA_OFFSET(arg);
+
+ if (ndims == 0)
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("cannot accumulate empty arrays")));
+
+ if (ndims + 1 > MAXDIM)
+ ereport(ERROR,
+ (errcode(ERRCODE_PROGRAM_LIMIT_EXCEEDED),
+ errmsg("number of array dimensions (%d) exceeds the maximum allowed (%d)",
+ ndims + 1, MAXDIM)));
+
+ if (astate == NULL)
+ {
+ /* Make a temporary context to hold all the junk */
+ arr_context = AllocSetContextCreate(rcontext,
+ "accumArrayResult",
+ ALLOCSET_DEFAULT_MINSIZE,
+ ALLOCSET_DEFAULT_INITSIZE,
+ ALLOCSET_DEFAULT_MAXSIZE);
+ oldcontext = MemoryContextSwitchTo(arr_context);
+
+ /* array accumulate */
+ astate = (MdArrayBuildState *) palloc(sizeof(MdArrayBuildState));
+ astate->mcontext = arr_context;
+
+ /*
+ * the resulting dimensions is n + 1, with first dimension is the number of
+ * array accumulated
+ */
+ astate->ndims = ndims + 1;
+ astate->dims = (int *) palloc((astate->ndims) * sizeof(int));
+ astate->lbs = (int *) palloc((astate->ndims) * sizeof(int));
+ astate->dims[0] = 0;
+ memcpy(&astate->dims[1], dims, ndims * sizeof(int));
+ astate->lbs[0] = 1;
+ memcpy(&astate->lbs[1], lbs, ndims * sizeof(int));
+
+ astate->aitems = 0;
+ astate->nbytes = 0;
+ astate->hasnull = false;
+ astate->nullbitmap = NULL;
+ astate->nitems = 0;
+
+ astate->abytes = ndatabytes >= 512 ? 4 * ndatabytes : 1024;
+ astate->data = (char *) palloc(astate->abytes);
+
+ astate->element_type = element_type;
+ get_typlenbyvalalign(element_type,
+ &astate->typlen,
+ &astate->typbyval,
+ &astate->typalign);
+ }
+ else
+ {
+ oldcontext = MemoryContextSwitchTo(astate->mcontext);
+
+ Assert(astate->element_type == element_type);
+
+ if (astate->ndims != ndims + 1)
+ ereport(ERROR,
+ (errcode(ERRCODE_ARRAY_SUBSCRIPT_ERROR),
+ errmsg("cannot accumulate incompatible arrays"),
+ errdetail("Arrays of %d and %d dimensions are not "
+ "compatible for accumulation.",
+ astate->ndims - 1, ndims)));
+
+ for (i = 0; i < ndims; i++)
+ if (astate->dims[i+1] != dims[i] || astate->lbs[i+1] != lbs[i])
+ ereport(ERROR,
+ (errcode(ERRCODE_ARRAY_SUBSCRIPT_ERROR),
+ errmsg("cannot accumulate incompatible arrays"),
+ errdetail("Arrays with differing element dimensions are "
+ "not compatible for concatenation.")));
+
+ /* grow array if needed */
+ if (astate->nbytes + ndatabytes >= astate->abytes)
+ {
+ astate->abytes *= 2;
+ astate->data = (char *)
+ repalloc(astate->data, astate->abytes);
+ }
+ if (astate->hasnull && astate->nitems + nitems >= astate->aitems)
+ {
+ astate->aitems *= 2;
+ astate->nullbitmap = (bits8 *)
+ repalloc(astate->nullbitmap, (astate->aitems + 7) / 8);
+ }
+ }
+
+ memcpy(astate->data + astate->nbytes, data, ndatabytes);
+ astate->nbytes += ndatabytes;
+
+ if (astate->hasnull || ARR_HASNULL(arg))
+ {
+ if (!astate->hasnull)
+ {
+ /* arbitrary number if nitems below 32 */
+ astate->aitems = astate->nitems < 32 ? 64 : astate->nitems * 8;
+ astate->hasnull = true;
+ astate->nullbitmap = (bits8 *) palloc((astate->aitems + 7) / 8);
+ array_bitmap_copy(astate->nullbitmap, 0,
+ NULL, 0,
+ astate->nitems);
+ }
+ array_bitmap_copy(astate->nullbitmap, astate->nitems,
+ ARR_NULLBITMAP(arg), 0,
+ nitems);
+ astate->nitems += nitems;
+ }
+ astate->dims[0] += 1;
+
+ MemoryContextSwitchTo(oldcontext);
+
+ return astate;
+}
+
+/*
+ * makeMdCustomArray - produce custom-D final result of accumMdArray
+ *
+ * beware: no check that specified dimensions match the number of values
+ * accumulated.
+ *
+ * astate is working state (not NULL)
+ * rcontext is where to construct result
+ * release is true if okay to release working state
+ */
+Datum
+makeMdArray(MdArrayBuildState *astate,
+ MemoryContext rcontext,
+ bool release)
+{
+ ArrayType *result;
+ MemoryContext oldcontext;
+
+ int dataoffset,
+ nbytes;
+
+ /* Build the final array result in rcontext */
+ oldcontext = MemoryContextSwitchTo(rcontext);
+
+ nbytes = astate->nbytes;
+ /* compute required space */
+ if (astate->hasnull)
+ {
+ dataoffset = ARR_OVERHEAD_WITHNULLS(astate->ndims, astate->nitems);
+ nbytes += dataoffset;
+ }
+ else
+ {
+ dataoffset = 0;
+ nbytes += ARR_OVERHEAD_NONULLS(astate->ndims);
+ }
+
+ result = (ArrayType *) palloc0(nbytes);
+ SET_VARSIZE(result, nbytes);
+ result->ndim = astate->ndims;
+ result->dataoffset = dataoffset;
+ result->elemtype = astate->element_type;
+
+ memcpy(ARR_DIMS(result), astate->dims, astate->ndims * sizeof(int));
+ memcpy(ARR_LBOUND(result), astate->lbs, astate->ndims * sizeof(int));
+ memcpy(ARR_DATA_PTR(result), astate->data, astate->nbytes);
+
+ if (astate->hasnull)
+ array_bitmap_copy(ARR_NULLBITMAP(result), 0,
+ astate->nullbitmap, 0,
+ astate->nitems);
+
+ MemoryContextSwitchTo(oldcontext);
+
+ /* Clean up all the junk */
+ if (release)
+ MemoryContextDelete(astate->mcontext);
+
+ return PointerGetDatum(result);
+}
+
Datum
array_larger(PG_FUNCTION_ARGS)
{
diff --git a/src/include/catalog/pg_aggregate.h b/src/include/catalog/pg_aggregate.h
index 3ba9e5e..2005199 100644
--- a/src/include/catalog/pg_aggregate.h
+++ b/src/include/catalog/pg_aggregate.h
@@ -275,6 +275,7 @@ DATA(insert ( 2901 n 0 xmlconcat2 - - - - f f 0 142 0 0 0 _null_
/* array */
DATA(insert ( 2335 n 0 array_agg_transfn array_agg_finalfn - - - t f 0 2281 0 0 0 _null_ _null_ ));
+DATA(insert ( 6005 n 0 array_agg_anyarray_transfn array_agg_anyarray_finalfn - - - t f 0 2281 0 0 0 _null_ _null_ ));
/* text */
DATA(insert ( 3538 n 0 string_agg_transfn string_agg_finalfn - - - f f 0 2281 0 0 0 _null_ _null_ ));
diff --git a/src/include/catalog/pg_proc.h b/src/include/catalog/pg_proc.h
index b6dc1b8..9273c1f 100644
--- a/src/include/catalog/pg_proc.h
+++ b/src/include/catalog/pg_proc.h
@@ -879,11 +879,17 @@ DATA(insert OID = 3167 ( array_remove PGNSP PGUID 12 1 0 0 0 f f f f f f i 2
DESCR("remove any occurrences of an element from an array");
DATA(insert OID = 3168 ( array_replace PGNSP PGUID 12 1 0 0 0 f f f f f f i 3 0 2277 "2277 2283 2283" _null_ _null_ _null_ _null_ array_replace _null_ _null_ _null_ ));
DESCR("replace any occurrences of an element in an array");
-DATA(insert OID = 2333 ( array_agg_transfn PGNSP PGUID 12 1 0 0 0 f f f f f f i 2 0 2281 "2281 2283" _null_ _null_ _null_ _null_ array_agg_transfn _null_ _null_ _null_ ));
+DATA(insert OID = 2333 ( array_agg_transfn PGNSP PGUID 12 1 0 0 0 f f f f f f i 2 0 2281 "2281 2776" _null_ _null_ _null_ _null_ array_agg_transfn _null_ _null_ _null_ ));
DESCR("aggregate transition function");
-DATA(insert OID = 2334 ( array_agg_finalfn PGNSP PGUID 12 1 0 0 0 f f f f f f i 2 0 2277 "2281 2283" _null_ _null_ _null_ _null_ array_agg_finalfn _null_ _null_ _null_ ));
+DATA(insert OID = 2334 ( array_agg_finalfn PGNSP PGUID 12 1 0 0 0 f f f f f f i 2 0 2277 "2281 2776" _null_ _null_ _null_ _null_ array_agg_finalfn _null_ _null_ _null_ ));
DESCR("aggregate final function");
-DATA(insert OID = 2335 ( array_agg PGNSP PGUID 12 1 0 0 0 t f f f f f i 1 0 2277 "2283" _null_ _null_ _null_ _null_ aggregate_dummy _null_ _null_ _null_ ));
+DATA(insert OID = 2335 ( array_agg PGNSP PGUID 12 1 0 0 0 t f f f f f i 1 0 2277 "2776" _null_ _null_ _null_ _null_ aggregate_dummy _null_ _null_ _null_ ));
+DESCR("concatenate aggregate input into an array");
+DATA(insert OID = 6003 ( array_agg_anyarray_transfn PGNSP PGUID 12 1 0 0 0 f f f f f f i 2 0 2281 "2281 2277" _null_ _null_ _null_ _null_ array_agg_anyarray_transfn _null_ _null_ _null_ ));
+DESCR("aggregate transition function");
+DATA(insert OID = 6004 ( array_agg_anyarray_finalfn PGNSP PGUID 12 1 0 0 0 f f f f f f i 2 0 2277 "2281 2277" _null_ _null_ _null_ _null_ array_agg_anyarray_finalfn _null_ _null_ _null_ ));
+DESCR("aggregate final function");
+DATA(insert OID = 6005 ( array_agg PGNSP PGUID 12 1 0 0 0 t f f f f f i 1 0 2277 "2277" _null_ _null_ _null_ _null_ aggregate_dummy _null_ _null_ _null_ ));
DESCR("concatenate aggregate input into an array");
DATA(insert OID = 3218 ( width_bucket PGNSP PGUID 12 1 0 0 0 f f f f t f i 2 0 23 "2283 2277" _null_ _null_ _null_ _null_ width_bucket_array _null_ _null_ _null_ ));
DESCR("bucket number of operand given a sorted array of bucket lower bounds");
diff --git a/src/include/utils/array.h b/src/include/utils/array.h
index e744314..c6b5539 100644
--- a/src/include/utils/array.h
+++ b/src/include/utils/array.h
@@ -75,7 +75,8 @@ typedef struct
} ArrayType;
/*
- * working state for accumArrayResult() and friends
+ * working state for array accumulation of scalar datums
+ * with accumArrayResult() and friends
*/
typedef struct ArrayBuildState
{
@@ -91,6 +92,34 @@ typedef struct ArrayBuildState
} ArrayBuildState;
/*
+ * array build state for array accumulation of array datums
+ * with accumMdArray() and friends
+ */
+typedef struct
+{
+ MemoryContext mcontext; /* where all the temp stuff is kept */
+
+ char *data; /* array of accumulated data */
+ bits8 *nullbitmap; /* bitmap of is-null flags for data */
+
+ int abytes; /* allocated length of above arrays */
+ int aitems; /* allocated length of above arrays */
+ int nbytes; /* number of used bytes in above arrays */
+ int nitems; /* number of elements in above arrays */
+
+ int ndims; /* element dimensions */
+ int *dims;
+ int *lbs;
+
+ bool hasnull; /* any element has null */
+
+ Oid element_type; /* data type of the Datums */
+ int16 typlen; /* needed info about datatype */
+ bool typbyval;
+ char typalign;
+} MdArrayBuildState;
+
+/*
* structure to cache type metadata needed for array manipulation
*/
typedef struct ArrayMetaState
@@ -260,6 +289,13 @@ extern Datum makeArrayResult(ArrayBuildState *astate,
MemoryContext rcontext);
extern Datum makeMdArrayResult(ArrayBuildState *astate, int ndims,
int *dims, int *lbs, MemoryContext rcontext, bool release);
+extern MdArrayBuildState *accumMdArray(MdArrayBuildState *astate,
+ ArrayType *arg, bool disnull,
+ Oid element_type,
+ MemoryContext rcontext);
+extern ArrayType *makeMdArray(MdArrayBuildState *astate,
+ MemoryContext rcontext,
+ bool release);
extern ArrayIterator array_create_iterator(ArrayType *arr, int slice_ndim);
extern bool array_iterate(ArrayIterator iterator, Datum *value, bool *isnull);
@@ -292,6 +328,8 @@ extern ArrayType *create_singleton_array(FunctionCallInfo fcinfo,
extern Datum array_agg_transfn(PG_FUNCTION_ARGS);
extern Datum array_agg_finalfn(PG_FUNCTION_ARGS);
+extern Datum array_agg_anyarray_transfn(PG_FUNCTION_ARGS);
+extern Datum array_agg_anyarray_finalfn(PG_FUNCTION_ARGS);
/*
* prototypes for functions defined in array_typanalyze.c
diff --git a/src/test/regress/expected/aggregates.out b/src/test/regress/expected/aggregates.out
index 58df854..607aeea 100644
--- a/src/test/regress/expected/aggregates.out
+++ b/src/test/regress/expected/aggregates.out
@@ -914,6 +914,76 @@ select array_agg(distinct a order by a desc nulls last)
{3,2,1,NULL}
(1 row)
+-- array_agg(anyarray)
+select array_agg(ar)
+ from (values ('{1,2}'::int[]), ('{3,4}'::int[])) v(ar);
+ array_agg
+---------------
+ {{1,2},{3,4}}
+(1 row)
+
+select array_agg(distinct ar order by ar desc)
+ from (select array[i / 2] from generate_series(1,10) a(i)) b(ar);
+ array_agg
+---------------------------
+ {{5},{4},{3},{2},{1},{0}}
+(1 row)
+
+select array_agg(ar)
+ from (select array_agg(array[i, i+1, i-1])
+ from generate_series(1,2) a(i)) b(ar);
+ array_agg
+---------------------
+ {{{1,2,0},{2,3,1}}}
+(1 row)
+
+-- array_agg(anyarray), varlena types
+select array_agg(array[1.2,1.3,1.4]) from generate_series(1,3);
+ array_agg
+---------------------------------------------
+ {{1.2,1.3,1.4},{1.2,1.3,1.4},{1.2,1.3,1.4}}
+(1 row)
+
+select array_agg(array['Hello','Hohoho','Hi']) from generate_series(1,3);
+ array_agg
+---------------------------------------------------------
+ {{Hello,Hohoho,Hi},{Hello,Hohoho,Hi},{Hello,Hohoho,Hi}}
+(1 row)
+
+-- array_agg(anyarray), arrays with nulls
+select array_agg(array[i, null, i+1, null, i+2]) from generate_series(1,3) g(i);
+ array_agg
+---------------------------------------------------------
+ {{1,NULL,2,NULL,3},{2,NULL,3,NULL,4},{3,NULL,4,NULL,5}}
+(1 row)
+
+select array_agg(array[1.1+ i, null, 1.1+i+1, null, 1.1+i+2]) from generate_series(1,3) g(i);
+ array_agg
+---------------------------------------------------------------------------
+ {{2.1,NULL,3.1,NULL,4.1},{3.1,NULL,4.1,NULL,5.1},{4.1,NULL,5.1,NULL,6.1}}
+(1 row)
+
+select array_agg(array[null, 'Hello','Hohoho', null,'Hi']) from generate_series(1,3);
+ array_agg
+---------------------------------------------------------------------------------------
+ {{NULL,Hello,Hohoho,NULL,Hi},{NULL,Hello,Hohoho,NULL,Hi},{NULL,Hello,Hohoho,NULL,Hi}}
+(1 row)
+
+select array_agg(array[[null, 'Hello', null, 'Hi'],['Hello', null, 'Hi', null]]) from generate_series(1,2);
+ array_agg
+-------------------------------------------------------------------------------------------
+ {{{NULL,Hello,NULL,Hi},{Hello,NULL,Hi,NULL}},{{NULL,Hello,NULL,Hi},{Hello,NULL,Hi,NULL}}}
+(1 row)
+
+-- errors
+select array_agg('{}'::int[]) from generate_series(1,2);
+ERROR: cannot accumulate empty arrays
+select array_agg(null::int[]) from generate_series(1,2);
+ERROR: cannot accumulate null arrays
+select array_agg(ar)
+ from (values ('{1,2}'::int[]), ('{3}'::int[])) v(ar);
+ERROR: cannot accumulate incompatible arrays
+DETAIL: Arrays with differing element dimensions are not compatible for concatenation.
-- multi-arg aggs, strict/nonstrict, distinct/order by
select aggfstr(a,b,c)
from (values (1,3,'foo'),(0,null,null),(2,2,'bar'),(3,1,'baz')) v(a,b,c);
diff --git a/src/test/regress/expected/arrays.out b/src/test/regress/expected/arrays.out
index 46eff67..e80ebec 100644
--- a/src/test/regress/expected/arrays.out
+++ b/src/test/regress/expected/arrays.out
@@ -1521,6 +1521,23 @@ select array_agg(unique1) from tenk1 where unique1 < -15;
(1 row)
+select array(select unique1 from tenk1 where unique1 < 15 order by unique1);
+ array
+--------------------------------------
+ {0,1,2,3,4,5,6,7,8,9,10,11,12,13,14}
+(1 row)
+
+select array(select array[i,i/2] from generate_series(1,5) a(i));
+ array
+---------------------------------
+ {{1,0},{2,1},{3,1},{4,2},{5,2}}
+(1 row)
+
+-- cannot accumulate null arrays and empty arrays
+select array(select null::int[]);
+ERROR: cannot accumulate null arrays
+select array(select '{}'::int[]);
+ERROR: cannot accumulate empty arrays
select unnest(array[1,2,3]);
unnest
--------
diff --git a/src/test/regress/sql/aggregates.sql b/src/test/regress/sql/aggregates.sql
index 8096a6f..a70419f 100644
--- a/src/test/regress/sql/aggregates.sql
+++ b/src/test/regress/sql/aggregates.sql
@@ -322,6 +322,32 @@ select array_agg(distinct a order by a desc)
select array_agg(distinct a order by a desc nulls last)
from (values (1),(2),(1),(3),(null),(2)) v(a);
+-- array_agg(anyarray)
+select array_agg(ar)
+ from (values ('{1,2}'::int[]), ('{3,4}'::int[])) v(ar);
+select array_agg(distinct ar order by ar desc)
+ from (select array[i / 2] from generate_series(1,10) a(i)) b(ar);
+select array_agg(ar)
+ from (select array_agg(array[i, i+1, i-1])
+ from generate_series(1,2) a(i)) b(ar);
+
+-- array_agg(anyarray), varlena types
+select array_agg(array[1.2,1.3,1.4]) from generate_series(1,3);
+select array_agg(array['Hello','Hohoho','Hi']) from generate_series(1,3);
+
+-- array_agg(anyarray), arrays with nulls
+select array_agg(array[i, null, i+1, null, i+2]) from generate_series(1,3) g(i);
+select array_agg(array[1.1+ i, null, 1.1+i+1, null, 1.1+i+2]) from generate_series(1,3) g(i);
+select array_agg(array[null, 'Hello','Hohoho', null,'Hi']) from generate_series(1,3);
+
+select array_agg(array[[null, 'Hello', null, 'Hi'],['Hello', null, 'Hi', null]]) from generate_series(1,2);
+
+-- errors
+select array_agg('{}'::int[]) from generate_series(1,2);
+select array_agg(null::int[]) from generate_series(1,2);
+select array_agg(ar)
+ from (values ('{1,2}'::int[]), ('{3}'::int[])) v(ar);
+
-- multi-arg aggs, strict/nonstrict, distinct/order by
select aggfstr(a,b,c)
diff --git a/src/test/regress/sql/arrays.sql b/src/test/regress/sql/arrays.sql
index fa8a20a..cb00f5f 100644
--- a/src/test/regress/sql/arrays.sql
+++ b/src/test/regress/sql/arrays.sql
@@ -432,6 +432,12 @@ select array_agg(ten) from (select ten from tenk1 where unique1 < 15 order by un
select array_agg(nullif(ten, 4)) from (select ten from tenk1 where unique1 < 15 order by unique1) ss;
select array_agg(unique1) from tenk1 where unique1 < -15;
+select array(select unique1 from tenk1 where unique1 < 15 order by unique1);
+select array(select array[i,i/2] from generate_series(1,5) a(i));
+-- cannot accumulate null arrays and empty arrays
+select array(select null::int[]);
+select array(select '{}'::int[]);
+
select unnest(array[1,2,3]);
select * from unnest(array[1,2,3]);
select unnest(array[1,2,3,4.5]::float8[]);
Re: Function array_agg(array)
2014-10-27 16:15 GMT+07:00 Pavel Stehule <pavel.stehule@gmail.com>:
Hi
I did some minor changes in code
* move tests of old or new builder style for array sublink out of main
cycles
* some API simplification of new builder - we should not to create
identical API, mainly it has no sense
minor changes in the patch:
* remove array_agg_finalfn_internal declaration in top of array_userfuncs.c
* fix comment of makeMdArray
* fix return of makeMdArray
* remove unnecesary changes to array_agg_finalfn
Regards,
--
Ali Akbar
Attachments:
array_agg_anyarray-13.patchtext/x-patch; charset=US-ASCII; name=array_agg_anyarray-13.patchDownload
diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml
index 7e5bcd9..f59738a 100644
--- a/doc/src/sgml/func.sgml
+++ b/doc/src/sgml/func.sgml
@@ -12046,6 +12046,22 @@ NULL baz</literallayout>(3 rows)</entry>
<row>
<entry>
<indexterm>
+ <primary>array_agg</primary>
+ </indexterm>
+ <function>array_agg(<replaceable class="parameter">anyarray</replaceable>)</function>
+ </entry>
+ <entry>
+ any
+ </entry>
+ <entry>
+ the same array type as input type
+ </entry>
+ <entry>input arrays, aggregated into higher-order multidimesional array. Rejects NULL and empty array as input.</entry>
+ </row>
+
+ <row>
+ <entry>
+ <indexterm>
<primary>average</primary>
</indexterm>
<indexterm>
diff --git a/doc/src/sgml/syntax.sgml b/doc/src/sgml/syntax.sgml
index 2f0680f..8c182a4 100644
--- a/doc/src/sgml/syntax.sgml
+++ b/doc/src/sgml/syntax.sgml
@@ -2238,6 +2238,11 @@ SELECT ARRAY(SELECT oid FROM pg_proc WHERE proname LIKE 'bytea%');
array
-----------------------------------------------------------------------
{2011,1954,1948,1952,1951,1244,1950,2005,1949,1953,2006,31,2412,2413}
+
+SELECT ARRAY(SELECT array(select i) FROM generate_series(1,5) a(i));
+ array
+-----------------------
+ {{1},{2},{3},{4},{5}}
(1 row)
</programlisting>
The subquery must return a single column. The resulting
diff --git a/src/backend/executor/nodeSubplan.c b/src/backend/executor/nodeSubplan.c
index 401bad4..21b8de1 100644
--- a/src/backend/executor/nodeSubplan.c
+++ b/src/backend/executor/nodeSubplan.c
@@ -231,7 +231,10 @@ ExecScanSubPlan(SubPlanState *node,
bool found = false; /* TRUE if got at least one subplan tuple */
ListCell *pvar;
ListCell *l;
+ bool use_md_array_builder = false;
+ Oid md_array_element_type = InvalidOid;
ArrayBuildState *astate = NULL;
+ MdArrayBuildState *mdastate = NULL;
/*
* MULTIEXPR subplans, when "executed", just return NULL; but first we
@@ -260,6 +263,16 @@ ExecScanSubPlan(SubPlanState *node,
}
/*
+ * use a fast array multidimensional builder when input is a array
+ * only check on first iteration. On subsequent, use the cached values
+ */
+ if (subLinkType == ARRAY_SUBLINK)
+ {
+ md_array_element_type = get_element_type(subplan->firstColType);
+ use_md_array_builder = OidIsValid(md_array_element_type);
+ }
+
+ /*
* We are probably in a short-lived expression-evaluation context. Switch
* to the per-query context for manipulating the child plan's chgParam,
* calling ExecProcNode on it, etc.
@@ -366,8 +379,16 @@ ExecScanSubPlan(SubPlanState *node,
/* stash away current value */
Assert(subplan->firstColType == tdesc->attrs[0]->atttypid);
dvalue = slot_getattr(slot, 1, &disnull);
- astate = accumArrayResult(astate, dvalue, disnull,
- subplan->firstColType, oldcontext);
+
+ if (use_md_array_builder)
+ mdastate = accumMdArray(mdastate,
+ disnull? NULL :
+ DatumGetArrayTypeP(dvalue),
+ disnull, md_array_element_type,
+ oldcontext);
+ else
+ astate = accumArrayResult(astate, dvalue, disnull,
+ subplan->firstColType, oldcontext);
/* keep scanning subplan to collect all values */
continue;
}
@@ -439,6 +460,8 @@ ExecScanSubPlan(SubPlanState *node,
/* We return the result in the caller's context */
if (astate != NULL)
result = makeArrayResult(astate, oldcontext);
+ else if (mdastate != NULL)
+ result = PointerGetDatum(makeMdArray(mdastate, oldcontext, true));
else
result = PointerGetDatum(construct_empty_array(subplan->firstColType));
}
@@ -951,7 +974,10 @@ ExecSetParamPlan(SubPlanState *node, ExprContext *econtext)
ListCell *pvar;
ListCell *l;
bool found = false;
+ bool use_md_array_builder = false;
+ Oid md_array_element_type = InvalidOid;
ArrayBuildState *astate = NULL;
+ MdArrayBuildState *mdastate = NULL;
if (subLinkType == ANY_SUBLINK ||
subLinkType == ALL_SUBLINK)
@@ -960,6 +986,16 @@ ExecSetParamPlan(SubPlanState *node, ExprContext *econtext)
elog(ERROR, "CTE subplans should not be executed via ExecSetParamPlan");
/*
+ * use a fast array multidimensional builder when input is a array
+ * only check on first iteration. On subsequent, use the cached values
+ */
+ if (subLinkType == ARRAY_SUBLINK)
+ {
+ md_array_element_type = get_element_type(subplan->firstColType);
+ use_md_array_builder = OidIsValid(md_array_element_type);
+ }
+
+ /*
* Must switch to per-query memory context.
*/
oldcontext = MemoryContextSwitchTo(econtext->ecxt_per_query_memory);
@@ -1018,8 +1054,16 @@ ExecSetParamPlan(SubPlanState *node, ExprContext *econtext)
/* stash away current value */
Assert(subplan->firstColType == tdesc->attrs[0]->atttypid);
dvalue = slot_getattr(slot, 1, &disnull);
- astate = accumArrayResult(astate, dvalue, disnull,
- subplan->firstColType, oldcontext);
+
+ if (use_md_array_builder)
+ mdastate = accumMdArray(mdastate,
+ disnull? NULL :
+ DatumGetArrayTypeP(dvalue),
+ disnull, md_array_element_type,
+ oldcontext);
+ else
+ astate = accumArrayResult(astate, dvalue, disnull,
+ subplan->firstColType, oldcontext);
/* keep scanning subplan to collect all values */
continue;
}
@@ -1075,6 +1119,9 @@ ExecSetParamPlan(SubPlanState *node, ExprContext *econtext)
if (astate != NULL)
node->curArray = makeArrayResult(astate,
econtext->ecxt_per_query_memory);
+ else if (mdastate != NULL)
+ node->curArray = PointerGetDatum(makeMdArray(mdastate,
+ econtext->ecxt_per_query_memory, true));
else
{
MemoryContextSwitchTo(econtext->ecxt_per_query_memory);
diff --git a/src/backend/nodes/nodeFuncs.c b/src/backend/nodes/nodeFuncs.c
index 41e973b..0261fcb 100644
--- a/src/backend/nodes/nodeFuncs.c
+++ b/src/backend/nodes/nodeFuncs.c
@@ -108,12 +108,16 @@ exprType(const Node *expr)
type = exprType((Node *) tent->expr);
if (sublink->subLinkType == ARRAY_SUBLINK)
{
- type = get_array_type(type);
- if (!OidIsValid(type))
- ereport(ERROR,
- (errcode(ERRCODE_UNDEFINED_OBJECT),
- errmsg("could not find array type for data type %s",
- format_type_be(exprType((Node *) tent->expr)))));
+ if (!OidIsValid(get_element_type(type)))
+ {
+ /* not array, so check for its array type */
+ type = get_array_type(type);
+ if (!OidIsValid(type))
+ ereport(ERROR,
+ (errcode(ERRCODE_UNDEFINED_OBJECT),
+ errmsg("could not find array type for data type %s",
+ format_type_be(exprType((Node *) tent->expr)))));
+ }
}
}
else if (sublink->subLinkType == MULTIEXPR_SUBLINK)
@@ -139,12 +143,16 @@ exprType(const Node *expr)
type = subplan->firstColType;
if (subplan->subLinkType == ARRAY_SUBLINK)
{
- type = get_array_type(type);
- if (!OidIsValid(type))
- ereport(ERROR,
- (errcode(ERRCODE_UNDEFINED_OBJECT),
- errmsg("could not find array type for data type %s",
- format_type_be(subplan->firstColType))));
+ if (!OidIsValid(get_element_type(type)))
+ {
+ /* not array, so check for its array type */
+ type = get_array_type(type);
+ if (!OidIsValid(type))
+ ereport(ERROR,
+ (errcode(ERRCODE_UNDEFINED_OBJECT),
+ errmsg("could not find array type for data type %s",
+ format_type_be(subplan->firstColType))));
+ }
}
}
else if (subplan->subLinkType == MULTIEXPR_SUBLINK)
diff --git a/src/backend/optimizer/plan/subselect.c b/src/backend/optimizer/plan/subselect.c
index 3e7dc85..8fc8b49 100644
--- a/src/backend/optimizer/plan/subselect.c
+++ b/src/backend/optimizer/plan/subselect.c
@@ -668,10 +668,16 @@ build_subplan(PlannerInfo *root, Plan *plan, PlannerInfo *subroot,
Assert(!te->resjunk);
Assert(testexpr == NULL);
- arraytype = get_array_type(exprType((Node *) te->expr));
- if (!OidIsValid(arraytype))
- elog(ERROR, "could not find array type for datatype %s",
- format_type_be(exprType((Node *) te->expr)));
+
+ arraytype = exprType((Node *) te->expr);
+ if (!OidIsValid(get_element_type(arraytype)))
+ {
+ /* not array, so get the array type */
+ arraytype = get_array_type(exprType((Node *) te->expr));
+ if (!OidIsValid(arraytype))
+ elog(ERROR, "could not find array type for datatype %s",
+ format_type_be(exprType((Node *) te->expr)));
+ }
prm = generate_new_param(root,
arraytype,
exprTypmod((Node *) te->expr),
diff --git a/src/backend/utils/adt/array_userfuncs.c b/src/backend/utils/adt/array_userfuncs.c
index 831466d..d56f872 100644
--- a/src/backend/utils/adt/array_userfuncs.c
+++ b/src/backend/utils/adt/array_userfuncs.c
@@ -544,3 +544,72 @@ array_agg_finalfn(PG_FUNCTION_ARGS)
PG_RETURN_DATUM(result);
}
+
+/*
+ * ARRAY_AGG(anyarray) aggregate function
+ */
+Datum
+array_agg_anyarray_transfn(PG_FUNCTION_ARGS)
+{
+ Oid arg1_typeid = get_fn_expr_argtype(fcinfo->flinfo, 1);
+ Oid element_type = get_element_type(arg1_typeid);
+ MemoryContext aggcontext;
+ MdArrayBuildState *state;
+ ArrayType *elem;
+
+ if (arg1_typeid == InvalidOid)
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("could not determine input data type")));
+
+ if (!AggCheckCallContext(fcinfo, &aggcontext))
+ {
+ /* cannot be called directly because of internal-type argument */
+ elog(ERROR, "array_agg_anyarray_transfn called in non-aggregate context");
+ }
+
+ state = PG_ARGISNULL(0) ? NULL : (MdArrayBuildState *) PG_GETARG_POINTER(0);
+ elem = PG_ARGISNULL(1) ? NULL : PG_GETARG_ARRAYTYPE_P(1);
+ state = accumMdArray(state,
+ elem,
+ PG_ARGISNULL(1),
+ element_type,
+ aggcontext);
+
+ /*
+ * The transition type for array_agg() is declared to be "internal", which
+ * is a pass-by-value type the same size as a pointer. So we can safely
+ * pass the ArrayBuildState pointer through nodeAgg.c's machinations.
+ */
+ PG_RETURN_POINTER(state);
+}
+
+Datum
+array_agg_anyarray_finalfn(PG_FUNCTION_ARGS)
+{
+ ArrayType *result;
+ MdArrayBuildState *state;
+
+ /*
+ * Test for null before Asserting we are in right context. This is to
+ * avoid possible Assert failure in 8.4beta installations, where it is
+ * possible for users to create NULL constants of type internal.
+ */
+ if (PG_ARGISNULL(0))
+ PG_RETURN_NULL(); /* returns null iff no input values */
+
+ /* cannot be called directly because of internal-type argument */
+ Assert(AggCheckCallContext(fcinfo, NULL));
+
+ state = (MdArrayBuildState *) PG_GETARG_POINTER(0);
+
+ /*
+ * Make the result. We cannot release the ArrayBuildState because
+ * sometimes aggregate final functions are re-executed. Rather, it is
+ * nodeAgg.c's responsibility to reset the aggcontext when it's safe to do
+ * so.
+ */
+ result = makeMdArray(state, CurrentMemoryContext, false);
+
+ PG_RETURN_ARRAYTYPE_P(result);
+}
diff --git a/src/backend/utils/adt/arrayfuncs.c b/src/backend/utils/adt/arrayfuncs.c
index 6c8b41d..10ceaa7 100644
--- a/src/backend/utils/adt/arrayfuncs.c
+++ b/src/backend/utils/adt/arrayfuncs.c
@@ -4576,6 +4576,9 @@ array_insert_slice(ArrayType *destArray,
/*
* accumArrayResult - accumulate one (more) Datum for an array result
*
+ * Beware: use this only for scalar datums. For Array datums, use
+ * accumMdArray and friends.
+ *
* astate is working state (NULL on first call)
* rcontext is where to keep working state
*/
@@ -4588,6 +4591,9 @@ accumArrayResult(ArrayBuildState *astate,
MemoryContext arr_context,
oldcontext;
+ /* element_type must not be an array! */
+ Assert(!OidIsValid(get_element_type(element_type)));
+
if (astate == NULL)
{
/* First time through --- initialize */
@@ -4713,6 +4719,219 @@ makeMdArrayResult(ArrayBuildState *astate,
return PointerGetDatum(result);
}
+/*
+ * accumMdArray - accumulate one (more) array Datum for accumulation
+ *
+ * astate is working state (NULL on first call)
+ * element_type is the type that array hold
+ * rcontext is where to keep working state
+ */
+MdArrayBuildState *
+accumMdArray(MdArrayBuildState *astate,
+ ArrayType *arg, bool disnull,
+ Oid element_type,
+ MemoryContext rcontext)
+{
+ MemoryContext arr_context,
+ oldcontext;
+
+ int *dims,
+ *lbs,
+ ndims,
+ nitems,
+ ndatabytes;
+ char *data;
+ int i;
+
+ /* element_type must not be an array! */
+ Assert(!OidIsValid(get_element_type(element_type)));
+
+ if (disnull)
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("cannot accumulate null arrays")));
+
+ ndims = ARR_NDIM(arg);
+ dims = ARR_DIMS(arg);
+ lbs = ARR_LBOUND(arg);
+ data = ARR_DATA_PTR(arg);
+ nitems = ArrayGetNItems(ndims, dims);
+ ndatabytes = ARR_SIZE(arg) - ARR_DATA_OFFSET(arg);
+
+ if (ndims == 0)
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("cannot accumulate empty arrays")));
+
+ if (ndims + 1 > MAXDIM)
+ ereport(ERROR,
+ (errcode(ERRCODE_PROGRAM_LIMIT_EXCEEDED),
+ errmsg("number of array dimensions (%d) exceeds the maximum allowed (%d)",
+ ndims + 1, MAXDIM)));
+
+ if (astate == NULL)
+ {
+ /* Make a temporary context to hold all the junk */
+ arr_context = AllocSetContextCreate(rcontext,
+ "accumArrayResult",
+ ALLOCSET_DEFAULT_MINSIZE,
+ ALLOCSET_DEFAULT_INITSIZE,
+ ALLOCSET_DEFAULT_MAXSIZE);
+ oldcontext = MemoryContextSwitchTo(arr_context);
+
+ /* array accumulate */
+ astate = (MdArrayBuildState *) palloc(sizeof(MdArrayBuildState));
+ astate->mcontext = arr_context;
+
+ /*
+ * the resulting dimensions is n + 1, with first dimension is the number of
+ * array accumulated
+ */
+ astate->ndims = ndims + 1;
+ astate->dims = (int *) palloc((astate->ndims) * sizeof(int));
+ astate->lbs = (int *) palloc((astate->ndims) * sizeof(int));
+ astate->dims[0] = 0;
+ memcpy(&astate->dims[1], dims, ndims * sizeof(int));
+ astate->lbs[0] = 1;
+ memcpy(&astate->lbs[1], lbs, ndims * sizeof(int));
+
+ astate->aitems = 0;
+ astate->nbytes = 0;
+ astate->hasnull = false;
+ astate->nullbitmap = NULL;
+ astate->nitems = 0;
+
+ astate->abytes = ndatabytes >= 512 ? 4 * ndatabytes : 1024;
+ astate->data = (char *) palloc(astate->abytes);
+
+ astate->element_type = element_type;
+ get_typlenbyvalalign(element_type,
+ &astate->typlen,
+ &astate->typbyval,
+ &astate->typalign);
+ }
+ else
+ {
+ oldcontext = MemoryContextSwitchTo(astate->mcontext);
+
+ Assert(astate->element_type == element_type);
+
+ if (astate->ndims != ndims + 1)
+ ereport(ERROR,
+ (errcode(ERRCODE_ARRAY_SUBSCRIPT_ERROR),
+ errmsg("cannot accumulate incompatible arrays"),
+ errdetail("Arrays of %d and %d dimensions are not "
+ "compatible for accumulation.",
+ astate->ndims - 1, ndims)));
+
+ for (i = 0; i < ndims; i++)
+ if (astate->dims[i+1] != dims[i] || astate->lbs[i+1] != lbs[i])
+ ereport(ERROR,
+ (errcode(ERRCODE_ARRAY_SUBSCRIPT_ERROR),
+ errmsg("cannot accumulate incompatible arrays"),
+ errdetail("Arrays with differing element dimensions are "
+ "not compatible for concatenation.")));
+
+ /* grow array if needed */
+ if (astate->nbytes + ndatabytes >= astate->abytes)
+ {
+ astate->abytes *= 2;
+ astate->data = (char *)
+ repalloc(astate->data, astate->abytes);
+ }
+ if (astate->hasnull && astate->nitems + nitems >= astate->aitems)
+ {
+ astate->aitems *= 2;
+ astate->nullbitmap = (bits8 *)
+ repalloc(astate->nullbitmap, (astate->aitems + 7) / 8);
+ }
+ }
+
+ memcpy(astate->data + astate->nbytes, data, ndatabytes);
+ astate->nbytes += ndatabytes;
+
+ if (astate->hasnull || ARR_HASNULL(arg))
+ {
+ if (!astate->hasnull)
+ {
+ /* arbitrary number if nitems below 32 */
+ astate->aitems = astate->nitems < 32 ? 64 : astate->nitems * 8;
+ astate->hasnull = true;
+ astate->nullbitmap = (bits8 *) palloc((astate->aitems + 7) / 8);
+ array_bitmap_copy(astate->nullbitmap, 0,
+ NULL, 0,
+ astate->nitems);
+ }
+ array_bitmap_copy(astate->nullbitmap, astate->nitems,
+ ARR_NULLBITMAP(arg), 0,
+ nitems);
+ astate->nitems += nitems;
+ }
+ astate->dims[0] += 1;
+
+ MemoryContextSwitchTo(oldcontext);
+
+ return astate;
+}
+
+/*
+ * makeMdArray - produce N+1-D final result of accumMdArray
+ *
+ * astate is working state (not NULL)
+ * rcontext is where to construct result
+ * release is true if okay to release working state
+ */
+ArrayType *
+makeMdArray(MdArrayBuildState *astate,
+ MemoryContext rcontext,
+ bool release)
+{
+ ArrayType *result;
+ MemoryContext oldcontext;
+
+ int dataoffset,
+ nbytes;
+
+ /* Build the final array result in rcontext */
+ oldcontext = MemoryContextSwitchTo(rcontext);
+
+ nbytes = astate->nbytes;
+ /* compute required space */
+ if (astate->hasnull)
+ {
+ dataoffset = ARR_OVERHEAD_WITHNULLS(astate->ndims, astate->nitems);
+ nbytes += dataoffset;
+ }
+ else
+ {
+ dataoffset = 0;
+ nbytes += ARR_OVERHEAD_NONULLS(astate->ndims);
+ }
+
+ result = (ArrayType *) palloc0(nbytes);
+ SET_VARSIZE(result, nbytes);
+ result->ndim = astate->ndims;
+ result->dataoffset = dataoffset;
+ result->elemtype = astate->element_type;
+
+ memcpy(ARR_DIMS(result), astate->dims, astate->ndims * sizeof(int));
+ memcpy(ARR_LBOUND(result), astate->lbs, astate->ndims * sizeof(int));
+ memcpy(ARR_DATA_PTR(result), astate->data, astate->nbytes);
+
+ if (astate->hasnull)
+ array_bitmap_copy(ARR_NULLBITMAP(result), 0,
+ astate->nullbitmap, 0,
+ astate->nitems);
+
+ MemoryContextSwitchTo(oldcontext);
+
+ /* Clean up all the junk */
+ if (release)
+ MemoryContextDelete(astate->mcontext);
+
+ return result;
+}
+
Datum
array_larger(PG_FUNCTION_ARGS)
{
diff --git a/src/include/catalog/pg_aggregate.h b/src/include/catalog/pg_aggregate.h
index 3ba9e5e..2005199 100644
--- a/src/include/catalog/pg_aggregate.h
+++ b/src/include/catalog/pg_aggregate.h
@@ -275,6 +275,7 @@ DATA(insert ( 2901 n 0 xmlconcat2 - - - - f f 0 142 0 0 0 _null_
/* array */
DATA(insert ( 2335 n 0 array_agg_transfn array_agg_finalfn - - - t f 0 2281 0 0 0 _null_ _null_ ));
+DATA(insert ( 6005 n 0 array_agg_anyarray_transfn array_agg_anyarray_finalfn - - - t f 0 2281 0 0 0 _null_ _null_ ));
/* text */
DATA(insert ( 3538 n 0 string_agg_transfn string_agg_finalfn - - - f f 0 2281 0 0 0 _null_ _null_ ));
diff --git a/src/include/catalog/pg_proc.h b/src/include/catalog/pg_proc.h
index b6dc1b8..9273c1f 100644
--- a/src/include/catalog/pg_proc.h
+++ b/src/include/catalog/pg_proc.h
@@ -879,11 +879,17 @@ DATA(insert OID = 3167 ( array_remove PGNSP PGUID 12 1 0 0 0 f f f f f f i 2
DESCR("remove any occurrences of an element from an array");
DATA(insert OID = 3168 ( array_replace PGNSP PGUID 12 1 0 0 0 f f f f f f i 3 0 2277 "2277 2283 2283" _null_ _null_ _null_ _null_ array_replace _null_ _null_ _null_ ));
DESCR("replace any occurrences of an element in an array");
-DATA(insert OID = 2333 ( array_agg_transfn PGNSP PGUID 12 1 0 0 0 f f f f f f i 2 0 2281 "2281 2283" _null_ _null_ _null_ _null_ array_agg_transfn _null_ _null_ _null_ ));
+DATA(insert OID = 2333 ( array_agg_transfn PGNSP PGUID 12 1 0 0 0 f f f f f f i 2 0 2281 "2281 2776" _null_ _null_ _null_ _null_ array_agg_transfn _null_ _null_ _null_ ));
DESCR("aggregate transition function");
-DATA(insert OID = 2334 ( array_agg_finalfn PGNSP PGUID 12 1 0 0 0 f f f f f f i 2 0 2277 "2281 2283" _null_ _null_ _null_ _null_ array_agg_finalfn _null_ _null_ _null_ ));
+DATA(insert OID = 2334 ( array_agg_finalfn PGNSP PGUID 12 1 0 0 0 f f f f f f i 2 0 2277 "2281 2776" _null_ _null_ _null_ _null_ array_agg_finalfn _null_ _null_ _null_ ));
DESCR("aggregate final function");
-DATA(insert OID = 2335 ( array_agg PGNSP PGUID 12 1 0 0 0 t f f f f f i 1 0 2277 "2283" _null_ _null_ _null_ _null_ aggregate_dummy _null_ _null_ _null_ ));
+DATA(insert OID = 2335 ( array_agg PGNSP PGUID 12 1 0 0 0 t f f f f f i 1 0 2277 "2776" _null_ _null_ _null_ _null_ aggregate_dummy _null_ _null_ _null_ ));
+DESCR("concatenate aggregate input into an array");
+DATA(insert OID = 6003 ( array_agg_anyarray_transfn PGNSP PGUID 12 1 0 0 0 f f f f f f i 2 0 2281 "2281 2277" _null_ _null_ _null_ _null_ array_agg_anyarray_transfn _null_ _null_ _null_ ));
+DESCR("aggregate transition function");
+DATA(insert OID = 6004 ( array_agg_anyarray_finalfn PGNSP PGUID 12 1 0 0 0 f f f f f f i 2 0 2277 "2281 2277" _null_ _null_ _null_ _null_ array_agg_anyarray_finalfn _null_ _null_ _null_ ));
+DESCR("aggregate final function");
+DATA(insert OID = 6005 ( array_agg PGNSP PGUID 12 1 0 0 0 t f f f f f i 1 0 2277 "2277" _null_ _null_ _null_ _null_ aggregate_dummy _null_ _null_ _null_ ));
DESCR("concatenate aggregate input into an array");
DATA(insert OID = 3218 ( width_bucket PGNSP PGUID 12 1 0 0 0 f f f f t f i 2 0 23 "2283 2277" _null_ _null_ _null_ _null_ width_bucket_array _null_ _null_ _null_ ));
DESCR("bucket number of operand given a sorted array of bucket lower bounds");
diff --git a/src/include/utils/array.h b/src/include/utils/array.h
index e744314..c6b5539 100644
--- a/src/include/utils/array.h
+++ b/src/include/utils/array.h
@@ -75,7 +75,8 @@ typedef struct
} ArrayType;
/*
- * working state for accumArrayResult() and friends
+ * working state for array accumulation of scalar datums
+ * with accumArrayResult() and friends
*/
typedef struct ArrayBuildState
{
@@ -91,6 +92,34 @@ typedef struct ArrayBuildState
} ArrayBuildState;
/*
+ * array build state for array accumulation of array datums
+ * with accumMdArray() and friends
+ */
+typedef struct
+{
+ MemoryContext mcontext; /* where all the temp stuff is kept */
+
+ char *data; /* array of accumulated data */
+ bits8 *nullbitmap; /* bitmap of is-null flags for data */
+
+ int abytes; /* allocated length of above arrays */
+ int aitems; /* allocated length of above arrays */
+ int nbytes; /* number of used bytes in above arrays */
+ int nitems; /* number of elements in above arrays */
+
+ int ndims; /* element dimensions */
+ int *dims;
+ int *lbs;
+
+ bool hasnull; /* any element has null */
+
+ Oid element_type; /* data type of the Datums */
+ int16 typlen; /* needed info about datatype */
+ bool typbyval;
+ char typalign;
+} MdArrayBuildState;
+
+/*
* structure to cache type metadata needed for array manipulation
*/
typedef struct ArrayMetaState
@@ -260,6 +289,13 @@ extern Datum makeArrayResult(ArrayBuildState *astate,
MemoryContext rcontext);
extern Datum makeMdArrayResult(ArrayBuildState *astate, int ndims,
int *dims, int *lbs, MemoryContext rcontext, bool release);
+extern MdArrayBuildState *accumMdArray(MdArrayBuildState *astate,
+ ArrayType *arg, bool disnull,
+ Oid element_type,
+ MemoryContext rcontext);
+extern ArrayType *makeMdArray(MdArrayBuildState *astate,
+ MemoryContext rcontext,
+ bool release);
extern ArrayIterator array_create_iterator(ArrayType *arr, int slice_ndim);
extern bool array_iterate(ArrayIterator iterator, Datum *value, bool *isnull);
@@ -292,6 +328,8 @@ extern ArrayType *create_singleton_array(FunctionCallInfo fcinfo,
extern Datum array_agg_transfn(PG_FUNCTION_ARGS);
extern Datum array_agg_finalfn(PG_FUNCTION_ARGS);
+extern Datum array_agg_anyarray_transfn(PG_FUNCTION_ARGS);
+extern Datum array_agg_anyarray_finalfn(PG_FUNCTION_ARGS);
/*
* prototypes for functions defined in array_typanalyze.c
diff --git a/src/test/regress/expected/aggregates.out b/src/test/regress/expected/aggregates.out
index 58df854..607aeea 100644
--- a/src/test/regress/expected/aggregates.out
+++ b/src/test/regress/expected/aggregates.out
@@ -914,6 +914,76 @@ select array_agg(distinct a order by a desc nulls last)
{3,2,1,NULL}
(1 row)
+-- array_agg(anyarray)
+select array_agg(ar)
+ from (values ('{1,2}'::int[]), ('{3,4}'::int[])) v(ar);
+ array_agg
+---------------
+ {{1,2},{3,4}}
+(1 row)
+
+select array_agg(distinct ar order by ar desc)
+ from (select array[i / 2] from generate_series(1,10) a(i)) b(ar);
+ array_agg
+---------------------------
+ {{5},{4},{3},{2},{1},{0}}
+(1 row)
+
+select array_agg(ar)
+ from (select array_agg(array[i, i+1, i-1])
+ from generate_series(1,2) a(i)) b(ar);
+ array_agg
+---------------------
+ {{{1,2,0},{2,3,1}}}
+(1 row)
+
+-- array_agg(anyarray), varlena types
+select array_agg(array[1.2,1.3,1.4]) from generate_series(1,3);
+ array_agg
+---------------------------------------------
+ {{1.2,1.3,1.4},{1.2,1.3,1.4},{1.2,1.3,1.4}}
+(1 row)
+
+select array_agg(array['Hello','Hohoho','Hi']) from generate_series(1,3);
+ array_agg
+---------------------------------------------------------
+ {{Hello,Hohoho,Hi},{Hello,Hohoho,Hi},{Hello,Hohoho,Hi}}
+(1 row)
+
+-- array_agg(anyarray), arrays with nulls
+select array_agg(array[i, null, i+1, null, i+2]) from generate_series(1,3) g(i);
+ array_agg
+---------------------------------------------------------
+ {{1,NULL,2,NULL,3},{2,NULL,3,NULL,4},{3,NULL,4,NULL,5}}
+(1 row)
+
+select array_agg(array[1.1+ i, null, 1.1+i+1, null, 1.1+i+2]) from generate_series(1,3) g(i);
+ array_agg
+---------------------------------------------------------------------------
+ {{2.1,NULL,3.1,NULL,4.1},{3.1,NULL,4.1,NULL,5.1},{4.1,NULL,5.1,NULL,6.1}}
+(1 row)
+
+select array_agg(array[null, 'Hello','Hohoho', null,'Hi']) from generate_series(1,3);
+ array_agg
+---------------------------------------------------------------------------------------
+ {{NULL,Hello,Hohoho,NULL,Hi},{NULL,Hello,Hohoho,NULL,Hi},{NULL,Hello,Hohoho,NULL,Hi}}
+(1 row)
+
+select array_agg(array[[null, 'Hello', null, 'Hi'],['Hello', null, 'Hi', null]]) from generate_series(1,2);
+ array_agg
+-------------------------------------------------------------------------------------------
+ {{{NULL,Hello,NULL,Hi},{Hello,NULL,Hi,NULL}},{{NULL,Hello,NULL,Hi},{Hello,NULL,Hi,NULL}}}
+(1 row)
+
+-- errors
+select array_agg('{}'::int[]) from generate_series(1,2);
+ERROR: cannot accumulate empty arrays
+select array_agg(null::int[]) from generate_series(1,2);
+ERROR: cannot accumulate null arrays
+select array_agg(ar)
+ from (values ('{1,2}'::int[]), ('{3}'::int[])) v(ar);
+ERROR: cannot accumulate incompatible arrays
+DETAIL: Arrays with differing element dimensions are not compatible for concatenation.
-- multi-arg aggs, strict/nonstrict, distinct/order by
select aggfstr(a,b,c)
from (values (1,3,'foo'),(0,null,null),(2,2,'bar'),(3,1,'baz')) v(a,b,c);
diff --git a/src/test/regress/expected/arrays.out b/src/test/regress/expected/arrays.out
index 46eff67..e80ebec 100644
--- a/src/test/regress/expected/arrays.out
+++ b/src/test/regress/expected/arrays.out
@@ -1521,6 +1521,23 @@ select array_agg(unique1) from tenk1 where unique1 < -15;
(1 row)
+select array(select unique1 from tenk1 where unique1 < 15 order by unique1);
+ array
+--------------------------------------
+ {0,1,2,3,4,5,6,7,8,9,10,11,12,13,14}
+(1 row)
+
+select array(select array[i,i/2] from generate_series(1,5) a(i));
+ array
+---------------------------------
+ {{1,0},{2,1},{3,1},{4,2},{5,2}}
+(1 row)
+
+-- cannot accumulate null arrays and empty arrays
+select array(select null::int[]);
+ERROR: cannot accumulate null arrays
+select array(select '{}'::int[]);
+ERROR: cannot accumulate empty arrays
select unnest(array[1,2,3]);
unnest
--------
diff --git a/src/test/regress/sql/aggregates.sql b/src/test/regress/sql/aggregates.sql
index 8096a6f..a70419f 100644
--- a/src/test/regress/sql/aggregates.sql
+++ b/src/test/regress/sql/aggregates.sql
@@ -322,6 +322,32 @@ select array_agg(distinct a order by a desc)
select array_agg(distinct a order by a desc nulls last)
from (values (1),(2),(1),(3),(null),(2)) v(a);
+-- array_agg(anyarray)
+select array_agg(ar)
+ from (values ('{1,2}'::int[]), ('{3,4}'::int[])) v(ar);
+select array_agg(distinct ar order by ar desc)
+ from (select array[i / 2] from generate_series(1,10) a(i)) b(ar);
+select array_agg(ar)
+ from (select array_agg(array[i, i+1, i-1])
+ from generate_series(1,2) a(i)) b(ar);
+
+-- array_agg(anyarray), varlena types
+select array_agg(array[1.2,1.3,1.4]) from generate_series(1,3);
+select array_agg(array['Hello','Hohoho','Hi']) from generate_series(1,3);
+
+-- array_agg(anyarray), arrays with nulls
+select array_agg(array[i, null, i+1, null, i+2]) from generate_series(1,3) g(i);
+select array_agg(array[1.1+ i, null, 1.1+i+1, null, 1.1+i+2]) from generate_series(1,3) g(i);
+select array_agg(array[null, 'Hello','Hohoho', null,'Hi']) from generate_series(1,3);
+
+select array_agg(array[[null, 'Hello', null, 'Hi'],['Hello', null, 'Hi', null]]) from generate_series(1,2);
+
+-- errors
+select array_agg('{}'::int[]) from generate_series(1,2);
+select array_agg(null::int[]) from generate_series(1,2);
+select array_agg(ar)
+ from (values ('{1,2}'::int[]), ('{3}'::int[])) v(ar);
+
-- multi-arg aggs, strict/nonstrict, distinct/order by
select aggfstr(a,b,c)
diff --git a/src/test/regress/sql/arrays.sql b/src/test/regress/sql/arrays.sql
index fa8a20a..cb00f5f 100644
--- a/src/test/regress/sql/arrays.sql
+++ b/src/test/regress/sql/arrays.sql
@@ -432,6 +432,12 @@ select array_agg(ten) from (select ten from tenk1 where unique1 < 15 order by un
select array_agg(nullif(ten, 4)) from (select ten from tenk1 where unique1 < 15 order by unique1) ss;
select array_agg(unique1) from tenk1 where unique1 < -15;
+select array(select unique1 from tenk1 where unique1 < 15 order by unique1);
+select array(select array[i,i/2] from generate_series(1,5) a(i));
+-- cannot accumulate null arrays and empty arrays
+select array(select null::int[]);
+select array(select '{}'::int[]);
+
select unnest(array[1,2,3]);
select * from unnest(array[1,2,3]);
select unnest(array[1,2,3,4.5]::float8[]);
Re: Function array_agg(array)
2014-10-27 11:20 GMT+01:00 Ali Akbar <the.apaan@gmail.com>:
2014-10-27 16:15 GMT+07:00 Pavel Stehule <pavel.stehule@gmail.com>:
Hi
I did some minor changes in code
* move tests of old or new builder style for array sublink out of main
cycles
* some API simplification of new builder - we should not to create
identical API, mainly it has no senseminor changes in the patch:
* remove array_agg_finalfn_internal declaration in top of array_userfuncs.c
* fix comment of makeMdArray
* fix return of makeMdArray
* remove unnecesary changes to array_agg_finalfn
super
I tested last version and I have not any objections.
1. We would to have this feature - it is long time number of our ToDo List
2. Proposal and design of multidimensional aggregation is clean and nobody
has objection here.
3. There is zero impact on current implementation. From performance reasons
it uses new array optimized aggregator - 30% faster for this purpose than
current (scalar) array aggregator
4. Code is clean and respect PostgreSQL coding rules
5. There are documentation and necessary regress tests
6. Patching and compilation is clean without warnings.
This patch is ready for commit
Thank you for patch
Regards
Pavel
Show quoted text
Regards,
--
Ali Akbar
Re: Function array_agg(array)
super
I tested last version and I have not any objections.
1. We would to have this feature - it is long time number of our ToDo List
2. Proposal and design of multidimensional aggregation is clean and nobody
has objection here.3. There is zero impact on current implementation. From performance
reasons it uses new array optimized aggregator - 30% faster for this
purpose than current (scalar) array aggregator4. Code is clean and respect PostgreSQL coding rules
5. There are documentation and necessary regress tests
6. Patching and compilation is clean without warnings.
This patch is ready for commit
Thank you for patch
Thank you for the thorough review process.
Regards,
--
Ali Akbar
Re: Function array_agg(array)
Pavel Stehule <pavel.stehule@gmail.com> writes:
2014-10-27 11:20 GMT+01:00 Ali Akbar <the.apaan@gmail.com>:
[ array_agg_anyarray-13.patch ]
This patch is ready for commit
I've committed this after some significant modifications.
I did not like the API chosen, specifically the business about callers
needing to deal with both accumArrayResult and accumMdArray, because that
seemed pretty messy and error-prone. There was in fact at least one
calling-logic error, namely that the empty array created by nodeSubplan.c
when there were zero inputs would have the wrong element type for the
array-input case.
It seemed to me that the best fix for that would be to push the empty
array creation special case into the accumArrayResult function family.
After some thought about how to do that, I settled on adding an "init"
function that can create an ArrayBuildState with no data yet put into
it; the main purpose is just to save the info about the input datatype.
Then, if makeArrayResult receives the ArrayBuildState with still no
data in it, it can create the appropriate empty array. (This turned
out to be a better idea than I first realized, as both xml.c and plperl.c
had coding patterns that could be made significantly less ugly with a
convention that the ArrayBuildState is always there.)
After doing that, I adjusted your new functions to have similar APIs,
and then added a switching layer on top for use by callers that want
to support both the scalar and array cases. This made the adjustments
for array subplans more or less one-liner changes in the relevant
places. We could possibly have unified the array_agg support functions
as well, but I didn't see much point in that given that we need two
sets of pg_proc entries to make type resolution work properly.
There were a number of other smaller issues too, like not being cautious
about memory allocation (the submitted code could both fail to allocate
enough, and compute an allocation request that would overflow an int).
regards, tom lane
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
Re: Function array_agg(array)
2014-11-26 0:38 GMT+07:00 Tom Lane <tgl@sss.pgh.pa.us>:
Pavel Stehule <pavel.stehule@gmail.com> writes:
2014-10-27 11:20 GMT+01:00 Ali Akbar <the.apaan@gmail.com>:
[ array_agg_anyarray-13.patch ]
This patch is ready for commit
I've committed this after some significant modifications.
I did not like the API chosen, specifically the business about callers
needing to deal with both accumArrayResult and accumMdArray, because that
seemed pretty messy and error-prone.
Thanks!
When in the reviewing process, we tried to implement in existing API, and
it was messy, so the last patch is with two API. We didn't think what you
eventually did. 3 API: existing for scalar, a new API for array, _and_ a
new API for both. Great!!
Just curious, in accumArrayResultArr, while enlarging array and
nullbitmaps, why it's implemented with:
astate->abytes = Max(astate->abytes * 2,
astate->nbytes + ndatabytes);
and
astate->aitems = Max(astate->aitems * 2, newnitems);
won't it be more consistent if it's implemented just like in the first
allocation?:
while (astate->aitems <= newnitems)
astate->aitems *= 2;
Anyway, thanks for the big modifications. I learned a lot from that.
Regards,
--
Ali Akbar
Re: Function array_agg(array)
Ali Akbar <the.apaan@gmail.com> writes:
Just curious, in accumArrayResultArr, while enlarging array and
nullbitmaps, why it's implemented with:
astate->abytes = Max(astate->abytes * 2,
astate->nbytes + ndatabytes);
and
astate->aitems = Max(astate->aitems * 2, newnitems);
won't it be more consistent if it's implemented just like in the first
allocation?:
while (astate->aitems <= newnitems)
astate->aitems *= 2;
The idea was to try to force the initial allocation to be a power of 2,
while not insisting on that for later enlargements. I can't point to any
hard reasons for doing it that way, but it seemed like a good idea.
Power-of-2 allocations are good up to a certain point but after that they
tend to get wasteful ...
regards, tom lane
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers