PL/Python adding support for multi-dimensional arrays
Started by Alexey Grishchenkoover 9 years ago26 messages
Hi
Current implementation of PL/Python does not allow the use of
multi-dimensional arrays, for both input and output parameters. This forces
end users to introduce workarounds like casting arrays to text before
passing them to the functions and parsing them after, which is an
error-prone approach
This patch adds support for multi-dimensional arrays as both input and
output parameters for PL/Python functions. The number of dimensions
supported is limited by Postgres MAXDIM macrovariable, by default equal to
6. Both input and output multi-dimensional arrays should have fixed
dimension sizes, i.e. 2-d arrays should represent MxN matrix, 3-d arrays
represent MxNxK cube, etc.
This patch does not support multi-dimensional arrays of composite types, as
composite types in Python might be represented as iterators and there is no
obvious way to find out when the nested array stops and composite type
structure starts. For example, if we have a composite type of (int, text),
we can try to return "[ [ [1,'a'], [2,'b'] ], [ [3,'c'], [4,'d'] ] ]", and
it is hard to find out that the first two lists are lists, and the third
one represents structure. Things are getting even more complex when you
have arrays as members of composite type. This is why I think this
limitation is reasonable.
Given the function:
CREATE FUNCTION test_type_conversion_array_int4(x int4[]) RETURNS int4[] AS
$$
plpy.info(x, type(x))
return x
$$ LANGUAGE plpythonu;
Before patch:
# SELECT * FROM test_type_conversion_array_int4(ARRAY[[1,2,3],[4,5,6]]);
ERROR: cannot convert multidimensional array to Python list
DETAIL: PL/Python only supports one-dimensional arrays.
CONTEXT: PL/Python function "test_type_conversion_array_int4"
After patch:
# SELECT * FROM test_type_conversion_array_int4(ARRAY[[1,2,3],[4,5,6]]);
INFO: ([[1, 2, 3], [4, 5, 6]], <type 'list'>)
test_type_conversion_array_int4
---------------------------------
{{1,2,3},{4,5,6}}
(1 row)
--
Best regards,
Alexey Grishchenko
Attachments:
0001-PL-Python-adding-support-for-multi-dimensional-arrays.patchapplication/octet-stream; name=0001-PL-Python-adding-support-for-multi-dimensional-arrays.patchDownload
From f45ac553c262c33e7f3be4d8c1616fff55e2bcbe Mon Sep 17 00:00:00 2001
From: Alexey Grishchenko <agrishchenko@pivotal.io>
Date: Wed, 3 Aug 2016 12:33:14 +0100
Subject: [PATCH] PL/Python adding support for multi-dimensional arrays
This patch adds support for multi-dimensional arrays as both input
and output parameters for PL/Python functions. The number of dimensions
is limited by Postgres MAXDIM macrovariable, by default equal to 6.
Both input and output multi-dimensional arrays should have fixed
dimension sizes, i.e. 2-d arrays should represent MxN matrix, 3-d
arrays representing MxNxK cube, etc.
Patch does not support multi-dimensional arrays of composite types,
as composite types in Python might be represented as iterators and
there is no way to find out when the composite type structure start,
so composite types can be used only in a single-dimensional arrays.
---
src/pl/plpython/expected/plpython_types.out | 30 +++-
src/pl/plpython/expected/plpython_types_3.out | 30 +++-
src/pl/plpython/plpy_typeio.c | 243 +++++++++++++++++++++-----
src/pl/plpython/sql/plpython_types.sql | 7 +
4 files changed, 261 insertions(+), 49 deletions(-)
diff --git a/src/pl/plpython/expected/plpython_types.out b/src/pl/plpython/expected/plpython_types.out
index f0b6abd..1a06620 100644
--- a/src/pl/plpython/expected/plpython_types.out
+++ b/src/pl/plpython/expected/plpython_types.out
@@ -537,9 +537,19 @@ INFO: (None, <type 'NoneType'>)
(1 row)
SELECT * FROM test_type_conversion_array_int4(ARRAY[[1,2,3],[4,5,6]]);
-ERROR: cannot convert multidimensional array to Python list
-DETAIL: PL/Python only supports one-dimensional arrays.
-CONTEXT: PL/Python function "test_type_conversion_array_int4"
+INFO: ([[1, 2, 3], [4, 5, 6]], <type 'list'>)
+ test_type_conversion_array_int4
+---------------------------------
+ {{1,2,3},{4,5,6}}
+(1 row)
+
+SELECT * FROM test_type_conversion_array_int4(ARRAY[[[1,2,NULL],[NULL,5,6]],[[NULL,8,9],[10,11,12]]]);
+INFO: ([[[1, 2, None], [None, 5, 6]], [[None, 8, 9], [10, 11, 12]]], <type 'list'>)
+ test_type_conversion_array_int4
+---------------------------------------------------
+ {{{1,2,NULL},{NULL,5,6}},{{NULL,8,9},{10,11,12}}}
+(1 row)
+
CREATE FUNCTION test_type_conversion_array_text(x text[]) RETURNS text[] AS $$
plpy.info(x, type(x))
return x
@@ -551,6 +561,13 @@ INFO: (['foo', 'bar'], <type 'list'>)
{foo,bar}
(1 row)
+SELECT * FROM test_type_conversion_array_text(ARRAY[['foo', 'bar'],['foo2', 'bar2']]);
+INFO: ([['foo', 'bar'], ['foo2', 'bar2']], <type 'list'>)
+ test_type_conversion_array_text
+---------------------------------
+ {{foo,bar},{foo2,bar2}}
+(1 row)
+
CREATE FUNCTION test_type_conversion_array_bytea(x bytea[]) RETURNS bytea[] AS $$
plpy.info(x, type(x))
return x
@@ -578,6 +595,13 @@ SELECT * FROM test_type_conversion_array_mixed2();
ERROR: invalid input syntax for integer: "abc"
CONTEXT: while creating return value
PL/Python function "test_type_conversion_array_mixed2"
+CREATE FUNCTION test_type_conversion_mdarray_malformed() RETURNS int[] AS $$
+return [[1,2,3],[4,5]]
+$$ LANGUAGE plpythonu;
+SELECT * FROM test_type_conversion_mdarray_malformed();
+ERROR: Multidimensional arrays must have array expressions with matching dimensions. PL/Python function return value has sequence length 2 while expected 3
+CONTEXT: while creating return value
+PL/Python function "test_type_conversion_mdarray_malformed"
CREATE FUNCTION test_type_conversion_array_record() RETURNS type_record[] AS $$
return [{'first': 'one', 'second': 42}, {'first': 'two', 'second': 11}]
$$ LANGUAGE plpythonu;
diff --git a/src/pl/plpython/expected/plpython_types_3.out b/src/pl/plpython/expected/plpython_types_3.out
index 56b78e1..3ef2862 100644
--- a/src/pl/plpython/expected/plpython_types_3.out
+++ b/src/pl/plpython/expected/plpython_types_3.out
@@ -537,9 +537,19 @@ INFO: (None, <class 'NoneType'>)
(1 row)
SELECT * FROM test_type_conversion_array_int4(ARRAY[[1,2,3],[4,5,6]]);
-ERROR: cannot convert multidimensional array to Python list
-DETAIL: PL/Python only supports one-dimensional arrays.
-CONTEXT: PL/Python function "test_type_conversion_array_int4"
+INFO: ([[1, 2, 3], [4, 5, 6]], <class 'list'>)
+ test_type_conversion_array_int4
+---------------------------------
+ {{1,2,3},{4,5,6}}
+(1 row)
+
+SELECT * FROM test_type_conversion_array_int4(ARRAY[[[1,2,NULL],[NULL,5,6]],[[NULL,8,9],[10,11,12]]]);
+INFO: ([[[1, 2, None], [None, 5, 6]], [[None, 8, 9], [10, 11, 12]]], <class 'list'>)
+ test_type_conversion_array_int4
+---------------------------------------------------
+ {{{1,2,NULL},{NULL,5,6}},{{NULL,8,9},{10,11,12}}}
+(1 row)
+
CREATE FUNCTION test_type_conversion_array_text(x text[]) RETURNS text[] AS $$
plpy.info(x, type(x))
return x
@@ -551,6 +561,13 @@ INFO: (['foo', 'bar'], <class 'list'>)
{foo,bar}
(1 row)
+SELECT * FROM test_type_conversion_array_text(ARRAY[['foo', 'bar'],['foo2', 'bar2']]);
+INFO: ([['foo', 'bar'], ['foo2', 'bar2']], <class 'list'>)
+ test_type_conversion_array_text
+---------------------------------
+ {{foo,bar},{foo2,bar2}}
+(1 row)
+
CREATE FUNCTION test_type_conversion_array_bytea(x bytea[]) RETURNS bytea[] AS $$
plpy.info(x, type(x))
return x
@@ -578,6 +595,13 @@ SELECT * FROM test_type_conversion_array_mixed2();
ERROR: invalid input syntax for integer: "abc"
CONTEXT: while creating return value
PL/Python function "test_type_conversion_array_mixed2"
+CREATE FUNCTION test_type_conversion_mdarray_malformed() RETURNS int[] AS $$
+return [[1,2,3],[4,5]]
+$$ LANGUAGE plpython3u;
+SELECT * FROM test_type_conversion_mdarray_malformed();
+ERROR: Multidimensional arrays must have array expressions with matching dimensions. PL/Python function return value has sequence length 2 while expected 3
+CONTEXT: while creating return value
+PL/Python function "test_type_conversion_mdarray_malformed"
CREATE FUNCTION test_type_conversion_array_record() RETURNS type_record[] AS $$
return [{'first': 'one', 'second': 42}, {'first': 'two', 'second': 11}]
$$ LANGUAGE plpython3u;
diff --git a/src/pl/plpython/plpy_typeio.c b/src/pl/plpython/plpy_typeio.c
index 7ad7a44..3fdc60d 100644
--- a/src/pl/plpython/plpy_typeio.c
+++ b/src/pl/plpython/plpy_typeio.c
@@ -631,46 +631,111 @@ PLyList_FromArray(PLyDatumToOb *arg, Datum d)
{
ArrayType *array = DatumGetArrayTypeP(d);
PLyDatumToOb *elm = arg->elm;
- PyObject *list;
- int length;
- int lbound;
+ int ndim;
+ int *dims;
+ int *lb;
+ char *dataptr;
+ bits8 *bitmap;
+ int bitmask;
int i;
+ int dim;
+ int indx[MAXDIM];
+ PyObject *lists[MAXDIM];
if (ARR_NDIM(array) == 0)
return PyList_New(0);
- if (ARR_NDIM(array) != 1)
- ereport(ERROR,
- (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
- errmsg("cannot convert multidimensional array to Python list"),
- errdetail("PL/Python only supports one-dimensional arrays.")));
+ /* Array dimensions and left bounds */
+ ndim = ARR_NDIM(array);
+ dims = ARR_DIMS(array);
+ lb = ARR_LBOUND(array);
- length = ARR_DIMS(array)[0];
- lbound = ARR_LBOUND(array)[0];
- list = PyList_New(length);
- if (list == NULL)
- PLy_elog(ERROR, "could not create new Python list");
+ /* Internal array representation pointers */
+ dataptr = ARR_DATA_PTR(array);
+ bitmap = ARR_NULLBITMAP(array);
+ bitmask = 1;
- for (i = 0; i < length; i++)
+ /* Iterators initialization */
+ for (i = 0; i < ndim; i++) {
+ indx[i] = lb[i];
+ lists[i] = NULL;
+ }
+ lists[0] = PyList_New(dims[0]);
+
+ /* We need this incref to keep the pointer valid after the array traversal
+ * terminates, as this traversal does DECREF for all the lists in array */
+ Py_INCREF(lists[0]);
+
+ /* In this cycle we are going over array dimensions. Postgres offers you an
+ * option to iterate over all the multi-dimensional array elemens in order.
+ * For 3-dimesnional array the order of iteration would be following - first
+ * you start with [0,0,0] elements through [0,0,k], then [0,1,0] till [0,1,k]
+ * till [0,m,k], then [1,0,0] till [1,0,k] till [1,m,k], and so on.
+ * In Python, each 1-d array is a separate list object, so 3-d array of
+ * [n,m,k] element is a list of n m-element arrays, each element of which is
+ * k-element array. In this cycle we traverse from outter dimensions to
+ * inner ones, creating nested Python lists during traversal */
+ dim = 0;
+ while (dim >= 0)
{
- Datum elem;
- bool isnull;
- int offset;
-
- offset = lbound + i;
- elem = array_ref(array, 1, &offset, arg->typlen,
- elm->typlen, elm->typbyval, elm->typalign,
- &isnull);
- if (isnull)
+ /* If we finished up iterating over current dimension - go one level up */
+ if (indx[dim] > dims[dim])
{
- Py_INCREF(Py_None);
- PyList_SET_ITEM(list, i, Py_None);
+ Py_DECREF(lists[dim]);
+ indx[dim] = 0;
+ dim -= 1;
+ }
+ /* If we are processing inner dimension - create one more list */
+ else if (dim < ndim - 1)
+ {
+ lists[dim+1] = PyList_New(dims[dim+1]);
+
+ /* We need this INCREF as we keep array pointer on our side,
+ * while PyList_SET_ITEM steals the reference */
+ Py_INCREF(lists[dim+1]);
+
+ PyList_SET_ITEM(lists[dim], indx[dim] - lb[dim], lists[dim+1]);
+ indx[dim] += 1;
+ dim += 1;
+ indx[dim] = lb[dim];
+ }
+ /* If we are iterating over the outter dimension, fill the list with
+ * values from the original Postgres array */
+ else if (dim == ndim - 1)
+ {
+ for (indx[dim] = lb[dim]; indx[dim] <= dims[dim]; indx[dim]++)
+ {
+ /* checking for NULL */
+ if (bitmap && (*bitmap & bitmask) == 0)
+ {
+ Py_INCREF(Py_None);
+ PyList_SET_ITEM(lists[dim], indx[dim] - lb[dim], Py_None);
+ }
+ else
+ {
+ Datum itemvalue;
+
+ itemvalue = fetch_att(dataptr, elm->typbyval, elm->typlen);
+ PyList_SET_ITEM(lists[dim], indx[dim] - lb[dim], elm->func(elm, itemvalue));
+ dataptr = att_addlength_pointer(dataptr, elm->typlen, dataptr);
+ dataptr = (char *) att_align_nominal(dataptr, elm->typalign);
+ }
+
+ /* advance bitmap pointer if any */
+ if (bitmap)
+ {
+ bitmask <<= 1;
+ if (bitmask == 0x100 /* (1<<8) */)
+ {
+ bitmap++;
+ bitmask = 1;
+ }
+ }
+ }
}
- else
- PyList_SET_ITEM(list, i, elm->func(elm, elem));
}
- return list;
+ return lists[0];
}
/*
@@ -866,39 +931,131 @@ static Datum
PLySequence_ToArray(PLyObToDatum *arg, int32 typmod, PyObject *plrv)
{
ArrayType *array;
- Datum rv;
int i;
Datum *elems;
bool *nulls;
int len;
- int lbs;
+ int ndim;
+ int dims[MAXDIM];
+ int lbs[MAXDIM];
+ int indx[MAXDIM];
+ PyObject *stack[MAXDIM];
+ int dim;
+ int idxelem;
+ Datum rv;
Assert(plrv != Py_None);
if (!PySequence_Check(plrv))
PLy_elog(ERROR, "return value of function with array return type is not a Python sequence");
- len = PySequence_Length(plrv);
+ ndim = 1;
+ dims[0] = PySequence_Length(plrv);
+ len = dims[0];
+ stack[0] = plrv;
+
+ /* We don't want to create multi-dimensional arrays when we have an empty sequence,
+ * when we need to parse sequence of composite objects, and when we have strings */
+ if (len > 0 && !type_is_rowtype(get_base_element_type(arg->typoid)) &&
+ !PyString_Check(plrv) && !PyBytes_Check(plrv) && !PyUnicode_Check(plrv))
+ {
+ PyObject *pyptr = PySequence_GetItem(plrv, 0);
+
+ /* We want to iterate through all iterable objects except by strings on nested levels */
+ while (pyptr != NULL && PySequence_Check(pyptr) &&
+ !(PyString_Check(pyptr) || PyBytes_Check(pyptr) || PyUnicode_Check(pyptr)))
+ {
+ dims[ndim] = PySequence_Length(pyptr);
+ if (dims[ndim] < 0)
+ PLy_elog(ERROR, "Cannot determine sequence length for function return value");
+ len *= dims[ndim];
+ stack[ndim] = pyptr;
+ ndim += 1;
+ if (dims[ndim - 1] == 0) {
+ pyptr = NULL;
+ break;
+ }
+ pyptr = PySequence_GetItem(pyptr, 0);
+ }
+
+ /* Pyptr points to element of n-dimensional array, we don't need its reference */
+ Py_XDECREF(pyptr);
+ }
+
+ /* We need this incref to keep the pointer valid after the array traversal
+ * terminates, as this traversal does DECREF for all the lists in array, and
+ * stack[0] corresponds to function return value */
+ Py_INCREF(stack[0]);
+
elems = palloc(sizeof(*elems) * len);
nulls = palloc(sizeof(*nulls) * len);
- for (i = 0; i < len; i++)
- {
- PyObject *obj = PySequence_GetItem(plrv, i);
+ for (i = 0; i < ndim; i++) {
+ indx[i] = 0;
+ lbs[i] = 1;
+ }
- if (obj == Py_None)
- nulls[i] = true;
- else
+ /* In this cycle we are going over nested Python lists, fetching elements
+ * from the deepest level and putting them into a linear array for Postgres
+ * to interpret them as n-dimensional array. This is a cycle implementation
+ * of DFS (recursive traversal of nested arrays here), keeping the stack in
+ * "stack" variable */
+ dim = 0;
+ idxelem = 0;
+ while (dim >= 0)
+ {
+ /* If we finished up iterating over current list - go one level up */
+ if (indx[dim] == dims[dim])
+ {
+ Py_DECREF(stack[dim]);
+ indx[dim] = 0;
+ dim -= 1;
+ }
+ /* If we are processing inner list - create one more list */
+ else if (dim < ndim - 1)
{
- nulls[i] = false;
- elems[i] = arg->elm->func(arg->elm, -1, obj);
+ stack[dim+1] = PySequence_GetItem(stack[dim], indx[dim]);
+ if (PySequence_Length(stack[dim+1]) != dims[dim+1])
+ PLy_elog(ERROR, "Multidimensional arrays must have array expressions with matching dimensions. "
+ "PL/Python function return value has sequence length %d while expected %d",
+ (int)PySequence_Length(stack[dim+1]), dims[dim+1]);
+ indx[dim] += 1;
+ dim += 1;
+ }
+ /* If we are iterating over the outter list, fill the output array */
+ else if (dim == ndim - 1)
+ {
+ for (indx[dim] = 0; indx[dim] < dims[dim]; indx[dim]++)
+ {
+ PyObject *obj = PySequence_GetItem(stack[dim], indx[dim]);
+
+ if (obj == Py_None)
+ nulls[idxelem] = true;
+ else
+ {
+ nulls[idxelem] = false;
+
+ /*
+ * We don't support arrays of row types yet, so the first argument
+ * can be NULL.
+ */
+ elems[idxelem] = arg->elm->func(arg->elm, -1, obj);
+ }
+ Py_XDECREF(obj);
+ idxelem += 1;
+ }
}
- Py_XDECREF(obj);
}
- lbs = 1;
- array = construct_md_array(elems, nulls, 1, &len, &lbs,
- get_base_element_type(arg->typoid), arg->elm->typlen, arg->elm->typbyval, arg->elm->typalign);
+ array = construct_md_array(elems,
+ nulls,
+ ndim,
+ dims,
+ lbs,
+ get_base_element_type(arg->typoid),
+ arg->elm->typlen,
+ arg->elm->typbyval,
+ arg->elm->typalign);
/*
* If the result type is a domain of array, the resulting array must be
diff --git a/src/pl/plpython/sql/plpython_types.sql b/src/pl/plpython/sql/plpython_types.sql
index 19d920d..f03a8b7 100644
--- a/src/pl/plpython/sql/plpython_types.sql
+++ b/src/pl/plpython/sql/plpython_types.sql
@@ -237,6 +237,7 @@ SELECT * FROM test_type_conversion_array_int4(ARRAY[NULL,1]);
SELECT * FROM test_type_conversion_array_int4(ARRAY[]::integer[]);
SELECT * FROM test_type_conversion_array_int4(NULL);
SELECT * FROM test_type_conversion_array_int4(ARRAY[[1,2,3],[4,5,6]]);
+SELECT * FROM test_type_conversion_array_int4(ARRAY[[[1,2,NULL],[NULL,5,6]],[[NULL,8,9],[10,11,12]]]);
CREATE FUNCTION test_type_conversion_array_text(x text[]) RETURNS text[] AS $$
@@ -245,6 +246,7 @@ return x
$$ LANGUAGE plpythonu;
SELECT * FROM test_type_conversion_array_text(ARRAY['foo', 'bar']);
+SELECT * FROM test_type_conversion_array_text(ARRAY[['foo', 'bar'],['foo2', 'bar2']]);
CREATE FUNCTION test_type_conversion_array_bytea(x bytea[]) RETURNS bytea[] AS $$
@@ -268,6 +270,11 @@ $$ LANGUAGE plpythonu;
SELECT * FROM test_type_conversion_array_mixed2();
+CREATE FUNCTION test_type_conversion_mdarray_malformed() RETURNS int[] AS $$
+return [[1,2,3],[4,5]]
+$$ LANGUAGE plpythonu;
+
+SELECT * FROM test_type_conversion_mdarray_malformed();
CREATE FUNCTION test_type_conversion_array_record() RETURNS type_record[] AS $$
return [{'first': 'one', 'second': 42}, {'first': 'two', 'second': 11}]
--
2.7.4 (Apple Git-66)
Re: PL/Python adding support for multi-dimensional arrays
On Wed, Aug 3, 2016 at 12:49 PM, Alexey Grishchenko <agrishchenko@pivotal.io
wrote:
Hi
Current implementation of PL/Python does not allow the use of
multi-dimensional arrays, for both input and output parameters. This forces
end users to introduce workarounds like casting arrays to text before
passing them to the functions and parsing them after, which is an
error-prone approachThis patch adds support for multi-dimensional arrays as both input and
output parameters for PL/Python functions. The number of dimensions
supported is limited by Postgres MAXDIM macrovariable, by default equal to
6. Both input and output multi-dimensional arrays should have fixed
dimension sizes, i.e. 2-d arrays should represent MxN matrix, 3-d arrays
represent MxNxK cube, etc.This patch does not support multi-dimensional arrays of composite types,
as composite types in Python might be represented as iterators and there is
no obvious way to find out when the nested array stops and composite type
structure starts. For example, if we have a composite type of (int, text),
we can try to return "[ [ [1,'a'], [2,'b'] ], [ [3,'c'], [4,'d'] ] ]", and
it is hard to find out that the first two lists are lists, and the third
one represents structure. Things are getting even more complex when you
have arrays as members of composite type. This is why I think this
limitation is reasonable.Given the function:
CREATE FUNCTION test_type_conversion_array_int4(x int4[]) RETURNS int4[]
AS $$
plpy.info(x, type(x))
return x
$$ LANGUAGE plpythonu;Before patch:
# SELECT * FROM test_type_conversion_array_int4(ARRAY[[1,2,3],[4,5,6]]);
ERROR: cannot convert multidimensional array to Python list
DETAIL: PL/Python only supports one-dimensional arrays.
CONTEXT: PL/Python function "test_type_conversion_array_int4"After patch:
# SELECT * FROM test_type_conversion_array_int4(ARRAY[[1,2,3],[4,5,6]]);
INFO: ([[1, 2, 3], [4, 5, 6]], <type 'list'>)
test_type_conversion_array_int4
---------------------------------
{{1,2,3},{4,5,6}}
(1 row)--
Best regards,
Alexey Grishchenko
Also this patch incorporates the fix for
/messages/by-id/CAH38_tkwA5qgLV8zPN1OpPzhtkNKQb30n3xq-2NR9jUfv3qwHA@mail.gmail.com,
as they touch the same piece of code - array manipulation in PL/Python
--
Best regards,
Alexey Grishchenko
Re: PL/Python adding support for multi-dimensional arrays
Hi
2016-08-03 13:54 GMT+02:00 Alexey Grishchenko <agrishchenko@pivotal.io>:
On Wed, Aug 3, 2016 at 12:49 PM, Alexey Grishchenko <
agrishchenko@pivotal.io> wrote:Hi
Current implementation of PL/Python does not allow the use of
multi-dimensional arrays, for both input and output parameters. This forces
end users to introduce workarounds like casting arrays to text before
passing them to the functions and parsing them after, which is an
error-prone approachThis patch adds support for multi-dimensional arrays as both input and
output parameters for PL/Python functions. The number of dimensions
supported is limited by Postgres MAXDIM macrovariable, by default equal to
6. Both input and output multi-dimensional arrays should have fixed
dimension sizes, i.e. 2-d arrays should represent MxN matrix, 3-d arrays
represent MxNxK cube, etc.This patch does not support multi-dimensional arrays of composite types,
as composite types in Python might be represented as iterators and there is
no obvious way to find out when the nested array stops and composite type
structure starts. For example, if we have a composite type of (int, text),
we can try to return "[ [ [1,'a'], [2,'b'] ], [ [3,'c'], [4,'d'] ] ]", and
it is hard to find out that the first two lists are lists, and the third
one represents structure. Things are getting even more complex when you
have arrays as members of composite type. This is why I think this
limitation is reasonable.Given the function:
CREATE FUNCTION test_type_conversion_array_int4(x int4[]) RETURNS int4[]
AS $$
plpy.info(x, type(x))
return x
$$ LANGUAGE plpythonu;Before patch:
# SELECT * FROM test_type_conversion_array_int4(ARRAY[[1,2,3],[4,5,6]]);
ERROR: cannot convert multidimensional array to Python list
DETAIL: PL/Python only supports one-dimensional arrays.
CONTEXT: PL/Python function "test_type_conversion_array_int4"After patch:
# SELECT * FROM test_type_conversion_array_int4(ARRAY[[1,2,3],[4,5,6]]);
INFO: ([[1, 2, 3], [4, 5, 6]], <type 'list'>)
test_type_conversion_array_int4
---------------------------------
{{1,2,3},{4,5,6}}
(1 row)--
Best regards,
Alexey GrishchenkoAlso this patch incorporates the fix for https://www.postgresql.
org/message-id/CAH38_tkwA5qgLV8zPN1OpPzhtkNKQb30n3x
q-2NR9jUfv3qwHA%40mail.gmail.com, as they touch the same piece of code -
array manipulation in PL/Python
I am sending review of this patch:
1. The implemented functionality is clearly benefit - passing MD arrays,
pretty faster passing bigger arrays
2. I was able to use this patch cleanly without any errors or warnings
3. There is no any error or warning
4. All tests passed - I tested Python 2.7 and Python 3.5
5. The code is well commented and clean
6. For this new functionality the documentation is not necessary
7. I invite more regress tests for both directions (Python <-> Postgres)
for more than two dimensions
My only one objection is not enough regress tests - after fixing this patch
will be ready for commiters.
Good work, Alexey
Thank you
Regards
Pavel
Show quoted text
--
Best regards,
Alexey Grishchenko
Re: PL/Python adding support for multi-dimensional arrays
On 10 August 2016 at 01:53, Pavel Stehule <pavel.stehule@gmail.com> wrote:
Hi
2016-08-03 13:54 GMT+02:00 Alexey Grishchenko <agrishchenko@pivotal.io>:
On Wed, Aug 3, 2016 at 12:49 PM, Alexey Grishchenko <
agrishchenko@pivotal.io> wrote:Hi
Current implementation of PL/Python does not allow the use of
multi-dimensional arrays, for both input and output parameters. This forces
end users to introduce workarounds like casting arrays to text before
passing them to the functions and parsing them after, which is an
error-prone approachThis patch adds support for multi-dimensional arrays as both input and
output parameters for PL/Python functions. The number of dimensions
supported is limited by Postgres MAXDIM macrovariable, by default equal to
6. Both input and output multi-dimensional arrays should have fixed
dimension sizes, i.e. 2-d arrays should represent MxN matrix, 3-d arrays
represent MxNxK cube, etc.This patch does not support multi-dimensional arrays of composite types,
as composite types in Python might be represented as iterators and there is
no obvious way to find out when the nested array stops and composite type
structure starts. For example, if we have a composite type of (int, text),
we can try to return "[ [ [1,'a'], [2,'b'] ], [ [3,'c'], [4,'d'] ] ]", and
it is hard to find out that the first two lists are lists, and the third
one represents structure. Things are getting even more complex when you
have arrays as members of composite type. This is why I think this
limitation is reasonable.Given the function:
CREATE FUNCTION test_type_conversion_array_int4(x int4[]) RETURNS
int4[] AS $$
plpy.info(x, type(x))
return x
$$ LANGUAGE plpythonu;Before patch:
# SELECT * FROM test_type_conversion_array_int4(ARRAY[[1,2,3],[4,5,6]]);
ERROR: cannot convert multidimensional array to Python list
DETAIL: PL/Python only supports one-dimensional arrays.
CONTEXT: PL/Python function "test_type_conversion_array_int4"After patch:
# SELECT * FROM test_type_conversion_array_int4(ARRAY[[1,2,3],[4,5,6]]);
INFO: ([[1, 2, 3], [4, 5, 6]], <type 'list'>)
test_type_conversion_array_int4
---------------------------------
{{1,2,3},{4,5,6}}
(1 row)--
Best regards,
Alexey GrishchenkoAlso this patch incorporates the fix for https://www.postgresql.org
/message-id/CAH38_tkwA5qgLV8zPN1OpPzhtkNKQb30n3xq-
2NR9jUfv3qwHA%40mail.gmail.com, as they touch the same piece of code -
array manipulation in PL/PythonI am sending review of this patch:
1. The implemented functionality is clearly benefit - passing MD arrays,
pretty faster passing bigger arrays
2. I was able to use this patch cleanly without any errors or warnings
3. There is no any error or warning
4. All tests passed - I tested Python 2.7 and Python 3.5
5. The code is well commented and clean
6. For this new functionality the documentation is not necessary7. I invite more regress tests for both directions (Python <-> Postgres)
for more than two dimensionsMy only one objection is not enough regress tests - after fixing this
patch will be ready for commiters.Good work, Alexey
Thank you
Regards
Pavel
--
Best regards,
Alexey Grishchenko
Pavel,
I will pick this up.
Dave Cramer
davec@postgresintl.com
www.postgresintl.com
Re: PL/Python adding support for multi-dimensional arrays
On 18 September 2016 at 09:27, Dave Cramer <pg@fastcrypt.com> wrote:
On 10 August 2016 at 01:53, Pavel Stehule <pavel.stehule@gmail.com> wrote:
Hi
2016-08-03 13:54 GMT+02:00 Alexey Grishchenko <agrishchenko@pivotal.io>:
On Wed, Aug 3, 2016 at 12:49 PM, Alexey Grishchenko <
agrishchenko@pivotal.io> wrote:Hi
Current implementation of PL/Python does not allow the use of
multi-dimensional arrays, for both input and output parameters. This forces
end users to introduce workarounds like casting arrays to text before
passing them to the functions and parsing them after, which is an
error-prone approachThis patch adds support for multi-dimensional arrays as both input and
output parameters for PL/Python functions. The number of dimensions
supported is limited by Postgres MAXDIM macrovariable, by default equal to
6. Both input and output multi-dimensional arrays should have fixed
dimension sizes, i.e. 2-d arrays should represent MxN matrix, 3-d arrays
represent MxNxK cube, etc.This patch does not support multi-dimensional arrays of composite
types, as composite types in Python might be represented as iterators and
there is no obvious way to find out when the nested array stops and
composite type structure starts. For example, if we have a composite type
of (int, text), we can try to return "[ [ [1,'a'], [2,'b'] ], [ [3,'c'],
[4,'d'] ] ]", and it is hard to find out that the first two lists are
lists, and the third one represents structure. Things are getting even more
complex when you have arrays as members of composite type. This is why I
think this limitation is reasonable.Given the function:
CREATE FUNCTION test_type_conversion_array_int4(x int4[]) RETURNS
int4[] AS $$
plpy.info(x, type(x))
return x
$$ LANGUAGE plpythonu;Before patch:
# SELECT * FROM test_type_conversion_array_int
4(ARRAY[[1,2,3],[4,5,6]]);
ERROR: cannot convert multidimensional array to Python list
DETAIL: PL/Python only supports one-dimensional arrays.
CONTEXT: PL/Python function "test_type_conversion_array_int4"After patch:
# SELECT * FROM test_type_conversion_array_int
4(ARRAY[[1,2,3],[4,5,6]]);
INFO: ([[1, 2, 3], [4, 5, 6]], <type 'list'>)
test_type_conversion_array_int4
---------------------------------
{{1,2,3},{4,5,6}}
(1 row)--
Best regards,
Alexey GrishchenkoAlso this patch incorporates the fix for https://www.postgresql.org
/message-id/CAH38_tkwA5qgLV8zPN1OpPzhtkNKQb30n3xq-2NR9jUfv3q
wHA%40mail.gmail.com, as they touch the same piece of code - array
manipulation in PL/PythonI am sending review of this patch:
1. The implemented functionality is clearly benefit - passing MD arrays,
pretty faster passing bigger arrays
2. I was able to use this patch cleanly without any errors or warnings
3. There is no any error or warning
4. All tests passed - I tested Python 2.7 and Python 3.5
5. The code is well commented and clean
6. For this new functionality the documentation is not necessary7. I invite more regress tests for both directions (Python <-> Postgres)
for more than two dimensionsMy only one objection is not enough regress tests - after fixing this
patch will be ready for commiters.Good work, Alexey
Thank you
Regards
Pavel
--
Best regards,
Alexey GrishchenkoPavel,
I will pick this up.
Pavel,
Please see attached patch which provides more test cases
I just realized this patch contains the original patch as well. What is the
protocol for sending in subsequent patches ?
Show quoted text
Dave Cramer
davec@postgresintl.com
www.postgresintl.com
Attachments:
0002-PL-Python-adding-support-for-multi-dimensional-arrays.patchapplication/octet-stream; name=0002-PL-Python-adding-support-for-multi-dimensional-arrays.patchDownload
diff --git a/src/pl/plpython/expected/plpython_types.out b/src/pl/plpython/expected/plpython_types.out
index f0b6abd..f94d2c3 100644
--- a/src/pl/plpython/expected/plpython_types.out
+++ b/src/pl/plpython/expected/plpython_types.out
@@ -537,9 +537,128 @@ INFO: (None, <type 'NoneType'>)
(1 row)
SELECT * FROM test_type_conversion_array_int4(ARRAY[[1,2,3],[4,5,6]]);
-ERROR: cannot convert multidimensional array to Python list
-DETAIL: PL/Python only supports one-dimensional arrays.
-CONTEXT: PL/Python function "test_type_conversion_array_int4"
+INFO: ([[1, 2, 3], [4, 5, 6]], <type 'list'>)
+ test_type_conversion_array_int4
+---------------------------------
+ {{1,2,3},{4,5,6}}
+(1 row)
+
+SELECT * FROM test_type_conversion_array_int4(ARRAY[[[1,2,NULL],[NULL,5,6]],[[NULL,8,9],[10,11,12]]]);
+INFO: ([[[1, 2, None], [None, 5, 6]], [[None, 8, 9], [10, 11, 12]]], <type 'list'>)
+ test_type_conversion_array_int4
+---------------------------------------------------
+ {{{1,2,NULL},{NULL,5,6}},{{NULL,8,9},{10,11,12}}}
+(1 row)
+
+CREATE FUNCTION test_type_conversion_array_int8(x int8[]) RETURNS int8[] AS $$
+plpy.info(x, type(x))
+return x
+$$ LANGUAGE plpythonu;
+SELECT * FROM test_type_conversion_array_int8(ARRAY[[[1,2,NULL],[NULL,5,6]],[[NULL,8,9],[10,11,12]]]::int8[]);
+INFO: ([[[1L, 2L, None], [None, 5L, 6L]], [[None, 8L, 9L], [10L, 11L, 12L]]], <type 'list'>)
+ test_type_conversion_array_int8
+---------------------------------------------------
+ {{{1,2,NULL},{NULL,5,6}},{{NULL,8,9},{10,11,12}}}
+(1 row)
+
+CREATE FUNCTION test_type_conversion_array_float4(x float4[]) RETURNS float4[] AS $$
+plpy.info(x, type(x))
+return x
+$$ LANGUAGE plpythonu;
+SELECT * FROM test_type_conversion_array_float4(ARRAY[[[1.2,2.3,NULL],[NULL,5.7,6.8]],[[NULL,8.9,9.345],[10.123,11.456,12.6768]]]::float4[]);
+INFO: ([[[1.2000000476837158, 2.299999952316284, None], [None, 5.699999809265137, 6.800000190734863]], [[None, 8.899999618530273, 9.345000267028809], [10.123000144958496, 11.456000328063965, 12.676799774169922]]], <type 'list'>)
+ test_type_conversion_array_float4
+------------------------------------------------------------------------------
+ {{{1.2,2.3,NULL},{NULL,5.7,6.8}},{{NULL,8.9,9.345},{10.123,11.456,12.6768}}}
+(1 row)
+
+CREATE FUNCTION test_type_conversion_array_float8(x float8[]) RETURNS float8[] AS $$
+plpy.info(x, type(x))
+return x
+$$ LANGUAGE plpythonu;
+SELECT * FROM test_type_conversion_array_float8(ARRAY[[[1.2,2.3,NULL],[NULL,5.7,6.8]],[[NULL,8.9,9.345],[10.123,11.456,12.6768]]]::float8[]);
+INFO: ([[[1.2, 2.3, None], [None, 5.7, 6.8]], [[None, 8.9, 9.345], [10.123, 11.456, 12.6768]]], <type 'list'>)
+ test_type_conversion_array_float8
+------------------------------------------------------------------------------
+ {{{1.2,2.3,NULL},{NULL,5.7,6.8}},{{NULL,8.9,9.345},{10.123,11.456,12.6768}}}
+(1 row)
+
+CREATE FUNCTION test_type_conversion_array_date(x date[]) RETURNS date[] AS $$
+plpy.info(x, type(x))
+return x
+$$ LANGUAGE plpythonu;
+SELECT * FROM test_type_conversion_array_date(ARRAY[[['2016-09-21','2016-09-22',NULL],[NULL,'2016-10-21','2016-10-22']],
+ [[NULL,'2016-11-21','2016-10-21'],['2015-09-21','2015-09-22','2014-09-21']]]::date[]);
+INFO: ([[['09-21-2016', '09-22-2016', None], [None, '10-21-2016', '10-22-2016']], [[None, '11-21-2016', '10-21-2016'], ['09-21-2015', '09-22-2015', '09-21-2014']]], <type 'list'>)
+ test_type_conversion_array_date
+---------------------------------------------------------------------------------------------------------------------------------
+ {{{09-21-2016,09-22-2016,NULL},{NULL,10-21-2016,10-22-2016}},{{NULL,11-21-2016,10-21-2016},{09-21-2015,09-22-2015,09-21-2014}}}
+(1 row)
+
+CREATE FUNCTION test_type_conversion_array_timestamp(x timestamp[]) RETURNS timestamp[] AS $$
+plpy.info(x, type(x))
+return x
+$$ LANGUAGE plpythonu;
+SELECT * FROM test_type_conversion_array_timestamp(ARRAY[[['2016-09-21 15:34:24.078792-04','2016-10-22 11:34:24.078795-04',NULL],
+ [NULL,'2016-10-21 11:34:25.078792-04','2016-10-21 11:34:24.098792-04']],
+ [[NULL,'2016-01-21 11:34:24.078792-04','2016-11-21 11:34:24.108792-04'],
+ ['2015-09-21 11:34:24.079792-04','2014-09-21 11:34:24.078792-04','2013-09-21 11:34:24.078792-04']]]::timestamp[]);
+INFO: ([[['Wed Sep 21 15:34:24.078792 2016', 'Sat Oct 22 11:34:24.078795 2016', None], [None, 'Fri Oct 21 11:34:25.078792 2016', 'Fri Oct 21 11:34:24.098792 2016']], [[None, 'Thu Jan 21 11:34:24.078792 2016', 'Mon Nov 21 11:34:24.108792 2016'], ['Mon Sep 21 11:34:24.079792 2015', 'Sun Sep 21 11:34:24.078792 2014', 'Sat Sep 21 11:34:24.078792 2013']]], <type 'list'>)
+ test_type_conversion_array_timestamp
+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
+ {{{"Wed Sep 21 15:34:24.078792 2016","Sat Oct 22 11:34:24.078795 2016",NULL},{NULL,"Fri Oct 21 11:34:25.078792 2016","Fri Oct 21 11:34:24.098792 2016"}},{{NULL,"Thu Jan 21 11:34:24.078792 2016","Mon Nov 21 11:34:24.108792 2016"},{"Mon Sep 21 11:34:24.079792 2015","Sun Sep 21 11:34:24.078792 2014","Sat Sep 21 11:34:24.078792 2013"}}}
+(1 row)
+
+CREATE OR REPLACE FUNCTION pyreturnmultidemint4(h int4, i int4, j int4, k int4 ) RETURNS int4[] AS $BODY$
+m = [[[[x for x in range(h)] for y in range(i)] for z in range(j)] for w in range(k)]
+plpy.info(m, type(m))
+return m
+$BODY$ LANGUAGE plpythonu;
+select pyreturnmultidemint4(8,5,3,2);
+INFO: ([[[[0, 1, 2, 3, 4, 5, 6, 7], [0, 1, 2, 3, 4, 5, 6, 7], [0, 1, 2, 3, 4, 5, 6, 7], [0, 1, 2, 3, 4, 5, 6, 7], [0, 1, 2, 3, 4, 5, 6, 7]], [[0, 1, 2, 3, 4, 5, 6, 7], [0, 1, 2, 3, 4, 5, 6, 7], [0, 1, 2, 3, 4, 5, 6, 7], [0, 1, 2, 3, 4, 5, 6, 7], [0, 1, 2, 3, 4, 5, 6, 7]], [[0, 1, 2, 3, 4, 5, 6, 7], [0, 1, 2, 3, 4, 5, 6, 7], [0, 1, 2, 3, 4, 5, 6, 7], [0, 1, 2, 3, 4, 5, 6, 7], [0, 1, 2, 3, 4, 5, 6, 7]]], [[[0, 1, 2, 3, 4, 5, 6, 7], [0, 1, 2, 3, 4, 5, 6, 7], [0, 1, 2, 3, 4, 5, 6, 7], [0, 1, 2, 3, 4, 5, 6, 7], [0, 1, 2, 3, 4, 5, 6, 7]], [[0, 1, 2, 3, 4, 5, 6, 7], [0, 1, 2, 3, 4, 5, 6, 7], [0, 1, 2, 3, 4, 5, 6, 7], [0, 1, 2, 3, 4, 5, 6, 7], [0, 1, 2, 3, 4, 5, 6, 7]], [[0, 1, 2, 3, 4, 5, 6, 7], [0, 1, 2, 3, 4, 5, 6, 7], [0, 1, 2, 3, 4, 5, 6, 7], [0, 1, 2, 3, 4, 5, 6, 7], [0, 1, 2, 3, 4, 5, 6, 7]]]], <type 'list'>)
+ pyreturnmultidemint4
+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
+ {{{{0,1,2,3,4,5,6,7},{0,1,2,3,4,5,6,7},{0,1,2,3,4,5,6,7},{0,1,2,3,4,5,6,7},{0,1,2,3,4,5,6,7}},{{0,1,2,3,4,5,6,7},{0,1,2,3,4,5,6,7},{0,1,2,3,4,5,6,7},{0,1,2,3,4,5,6,7},{0,1,2,3,4,5,6,7}},{{0,1,2,3,4,5,6,7},{0,1,2,3,4,5,6,7},{0,1,2,3,4,5,6,7},{0,1,2,3,4,5,6,7},{0,1,2,3,4,5,6,7}}},{{{0,1,2,3,4,5,6,7},{0,1,2,3,4,5,6,7},{0,1,2,3,4,5,6,7},{0,1,2,3,4,5,6,7},{0,1,2,3,4,5,6,7}},{{0,1,2,3,4,5,6,7},{0,1,2,3,4,5,6,7},{0,1,2,3,4,5,6,7},{0,1,2,3,4,5,6,7},{0,1,2,3,4,5,6,7}},{{0,1,2,3,4,5,6,7},{0,1,2,3,4,5,6,7},{0,1,2,3,4,5,6,7},{0,1,2,3,4,5,6,7},{0,1,2,3,4,5,6,7}}}}
+(1 row)
+
+CREATE OR REPLACE FUNCTION pyreturnmultidemint8(h int4, i int4, j int4, k int4 ) RETURNS int8[] AS $BODY$
+m = [[[[x for x in range(h)] for y in range(i)] for z in range(j)] for w in range(k)]
+plpy.info(m, type(m))
+return m
+$BODY$ LANGUAGE plpythonu;
+select pyreturnmultidemint8(5,5,3,2);
+INFO: ([[[[0, 1, 2, 3, 4], [0, 1, 2, 3, 4], [0, 1, 2, 3, 4], [0, 1, 2, 3, 4], [0, 1, 2, 3, 4]], [[0, 1, 2, 3, 4], [0, 1, 2, 3, 4], [0, 1, 2, 3, 4], [0, 1, 2, 3, 4], [0, 1, 2, 3, 4]], [[0, 1, 2, 3, 4], [0, 1, 2, 3, 4], [0, 1, 2, 3, 4], [0, 1, 2, 3, 4], [0, 1, 2, 3, 4]]], [[[0, 1, 2, 3, 4], [0, 1, 2, 3, 4], [0, 1, 2, 3, 4], [0, 1, 2, 3, 4], [0, 1, 2, 3, 4]], [[0, 1, 2, 3, 4], [0, 1, 2, 3, 4], [0, 1, 2, 3, 4], [0, 1, 2, 3, 4], [0, 1, 2, 3, 4]], [[0, 1, 2, 3, 4], [0, 1, 2, 3, 4], [0, 1, 2, 3, 4], [0, 1, 2, 3, 4], [0, 1, 2, 3, 4]]]], <type 'list'>)
+ pyreturnmultidemint8
+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
+ {{{{0,1,2,3,4},{0,1,2,3,4},{0,1,2,3,4},{0,1,2,3,4},{0,1,2,3,4}},{{0,1,2,3,4},{0,1,2,3,4},{0,1,2,3,4},{0,1,2,3,4},{0,1,2,3,4}},{{0,1,2,3,4},{0,1,2,3,4},{0,1,2,3,4},{0,1,2,3,4},{0,1,2,3,4}}},{{{0,1,2,3,4},{0,1,2,3,4},{0,1,2,3,4},{0,1,2,3,4},{0,1,2,3,4}},{{0,1,2,3,4},{0,1,2,3,4},{0,1,2,3,4},{0,1,2,3,4},{0,1,2,3,4}},{{0,1,2,3,4},{0,1,2,3,4},{0,1,2,3,4},{0,1,2,3,4},{0,1,2,3,4}}}}
+(1 row)
+
+CREATE OR REPLACE FUNCTION pyreturnmultidemfloat4(h int4, i int4, j int4, k int4 ) RETURNS float4[] AS $BODY$
+
+m = [[[[x for x in range(h)] for y in range(i)] for z in range(j)] for w in range(k)]
+plpy.info(m, type(m))
+return m
+$BODY$ LANGUAGE plpythonu;
+select pyreturnmultidemfloat4(6,5,3,2);
+INFO: ([[[[0, 1, 2, 3, 4, 5], [0, 1, 2, 3, 4, 5], [0, 1, 2, 3, 4, 5], [0, 1, 2, 3, 4, 5], [0, 1, 2, 3, 4, 5]], [[0, 1, 2, 3, 4, 5], [0, 1, 2, 3, 4, 5], [0, 1, 2, 3, 4, 5], [0, 1, 2, 3, 4, 5], [0, 1, 2, 3, 4, 5]], [[0, 1, 2, 3, 4, 5], [0, 1, 2, 3, 4, 5], [0, 1, 2, 3, 4, 5], [0, 1, 2, 3, 4, 5], [0, 1, 2, 3, 4, 5]]], [[[0, 1, 2, 3, 4, 5], [0, 1, 2, 3, 4, 5], [0, 1, 2, 3, 4, 5], [0, 1, 2, 3, 4, 5], [0, 1, 2, 3, 4, 5]], [[0, 1, 2, 3, 4, 5], [0, 1, 2, 3, 4, 5], [0, 1, 2, 3, 4, 5], [0, 1, 2, 3, 4, 5], [0, 1, 2, 3, 4, 5]], [[0, 1, 2, 3, 4, 5], [0, 1, 2, 3, 4, 5], [0, 1, 2, 3, 4, 5], [0, 1, 2, 3, 4, 5], [0, 1, 2, 3, 4, 5]]]], <type 'list'>)
+ pyreturnmultidemfloat4
+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
+ {{{{0,1,2,3,4,5},{0,1,2,3,4,5},{0,1,2,3,4,5},{0,1,2,3,4,5},{0,1,2,3,4,5}},{{0,1,2,3,4,5},{0,1,2,3,4,5},{0,1,2,3,4,5},{0,1,2,3,4,5},{0,1,2,3,4,5}},{{0,1,2,3,4,5},{0,1,2,3,4,5},{0,1,2,3,4,5},{0,1,2,3,4,5},{0,1,2,3,4,5}}},{{{0,1,2,3,4,5},{0,1,2,3,4,5},{0,1,2,3,4,5},{0,1,2,3,4,5},{0,1,2,3,4,5}},{{0,1,2,3,4,5},{0,1,2,3,4,5},{0,1,2,3,4,5},{0,1,2,3,4,5},{0,1,2,3,4,5}},{{0,1,2,3,4,5},{0,1,2,3,4,5},{0,1,2,3,4,5},{0,1,2,3,4,5},{0,1,2,3,4,5}}}}
+(1 row)
+
+CREATE OR REPLACE FUNCTION pyreturnmultidemfloat8(h int4, i int4, j int4, k int4 ) RETURNS float8[] AS $BODY$
+
+m = [[[[x for x in range(h)] for y in range(i)] for z in range(j)] for w in range(k)]
+plpy.info(m, type(m))
+return m
+$BODY$ LANGUAGE plpythonu;
+select pyreturnmultidemfloat8(7,5,3,2);
+INFO: ([[[[0, 1, 2, 3, 4, 5, 6], [0, 1, 2, 3, 4, 5, 6], [0, 1, 2, 3, 4, 5, 6], [0, 1, 2, 3, 4, 5, 6], [0, 1, 2, 3, 4, 5, 6]], [[0, 1, 2, 3, 4, 5, 6], [0, 1, 2, 3, 4, 5, 6], [0, 1, 2, 3, 4, 5, 6], [0, 1, 2, 3, 4, 5, 6], [0, 1, 2, 3, 4, 5, 6]], [[0, 1, 2, 3, 4, 5, 6], [0, 1, 2, 3, 4, 5, 6], [0, 1, 2, 3, 4, 5, 6], [0, 1, 2, 3, 4, 5, 6], [0, 1, 2, 3, 4, 5, 6]]], [[[0, 1, 2, 3, 4, 5, 6], [0, 1, 2, 3, 4, 5, 6], [0, 1, 2, 3, 4, 5, 6], [0, 1, 2, 3, 4, 5, 6], [0, 1, 2, 3, 4, 5, 6]], [[0, 1, 2, 3, 4, 5, 6], [0, 1, 2, 3, 4, 5, 6], [0, 1, 2, 3, 4, 5, 6], [0, 1, 2, 3, 4, 5, 6], [0, 1, 2, 3, 4, 5, 6]], [[0, 1, 2, 3, 4, 5, 6], [0, 1, 2, 3, 4, 5, 6], [0, 1, 2, 3, 4, 5, 6], [0, 1, 2, 3, 4, 5, 6], [0, 1, 2, 3, 4, 5, 6]]]], <type 'list'>)
+ pyreturnmultidemfloat8
+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
+ {{{{0,1,2,3,4,5,6},{0,1,2,3,4,5,6},{0,1,2,3,4,5,6},{0,1,2,3,4,5,6},{0,1,2,3,4,5,6}},{{0,1,2,3,4,5,6},{0,1,2,3,4,5,6},{0,1,2,3,4,5,6},{0,1,2,3,4,5,6},{0,1,2,3,4,5,6}},{{0,1,2,3,4,5,6},{0,1,2,3,4,5,6},{0,1,2,3,4,5,6},{0,1,2,3,4,5,6},{0,1,2,3,4,5,6}}},{{{0,1,2,3,4,5,6},{0,1,2,3,4,5,6},{0,1,2,3,4,5,6},{0,1,2,3,4,5,6},{0,1,2,3,4,5,6}},{{0,1,2,3,4,5,6},{0,1,2,3,4,5,6},{0,1,2,3,4,5,6},{0,1,2,3,4,5,6},{0,1,2,3,4,5,6}},{{0,1,2,3,4,5,6},{0,1,2,3,4,5,6},{0,1,2,3,4,5,6},{0,1,2,3,4,5,6},{0,1,2,3,4,5,6}}}}
+(1 row)
+
CREATE FUNCTION test_type_conversion_array_text(x text[]) RETURNS text[] AS $$
plpy.info(x, type(x))
return x
@@ -551,6 +670,13 @@ INFO: (['foo', 'bar'], <type 'list'>)
{foo,bar}
(1 row)
+SELECT * FROM test_type_conversion_array_text(ARRAY[['foo', 'bar'],['foo2', 'bar2']]);
+INFO: ([['foo', 'bar'], ['foo2', 'bar2']], <type 'list'>)
+ test_type_conversion_array_text
+---------------------------------
+ {{foo,bar},{foo2,bar2}}
+(1 row)
+
CREATE FUNCTION test_type_conversion_array_bytea(x bytea[]) RETURNS bytea[] AS $$
plpy.info(x, type(x))
return x
@@ -578,6 +704,13 @@ SELECT * FROM test_type_conversion_array_mixed2();
ERROR: invalid input syntax for integer: "abc"
CONTEXT: while creating return value
PL/Python function "test_type_conversion_array_mixed2"
+CREATE FUNCTION test_type_conversion_mdarray_malformed() RETURNS int[] AS $$
+return [[1,2,3],[4,5]]
+$$ LANGUAGE plpythonu;
+SELECT * FROM test_type_conversion_mdarray_malformed();
+ERROR: Multidimensional arrays must have array expressions with matching dimensions. PL/Python function return value has sequence length 2 while expected 3
+CONTEXT: while creating return value
+PL/Python function "test_type_conversion_mdarray_malformed"
CREATE FUNCTION test_type_conversion_array_record() RETURNS type_record[] AS $$
return [{'first': 'one', 'second': 42}, {'first': 'two', 'second': 11}]
$$ LANGUAGE plpythonu;
diff --git a/src/pl/plpython/expected/plpython_types_3.out b/src/pl/plpython/expected/plpython_types_3.out
index 56b78e1..3ef2862 100644
--- a/src/pl/plpython/expected/plpython_types_3.out
+++ b/src/pl/plpython/expected/plpython_types_3.out
@@ -537,9 +537,19 @@ INFO: (None, <class 'NoneType'>)
(1 row)
SELECT * FROM test_type_conversion_array_int4(ARRAY[[1,2,3],[4,5,6]]);
-ERROR: cannot convert multidimensional array to Python list
-DETAIL: PL/Python only supports one-dimensional arrays.
-CONTEXT: PL/Python function "test_type_conversion_array_int4"
+INFO: ([[1, 2, 3], [4, 5, 6]], <class 'list'>)
+ test_type_conversion_array_int4
+---------------------------------
+ {{1,2,3},{4,5,6}}
+(1 row)
+
+SELECT * FROM test_type_conversion_array_int4(ARRAY[[[1,2,NULL],[NULL,5,6]],[[NULL,8,9],[10,11,12]]]);
+INFO: ([[[1, 2, None], [None, 5, 6]], [[None, 8, 9], [10, 11, 12]]], <class 'list'>)
+ test_type_conversion_array_int4
+---------------------------------------------------
+ {{{1,2,NULL},{NULL,5,6}},{{NULL,8,9},{10,11,12}}}
+(1 row)
+
CREATE FUNCTION test_type_conversion_array_text(x text[]) RETURNS text[] AS $$
plpy.info(x, type(x))
return x
@@ -551,6 +561,13 @@ INFO: (['foo', 'bar'], <class 'list'>)
{foo,bar}
(1 row)
+SELECT * FROM test_type_conversion_array_text(ARRAY[['foo', 'bar'],['foo2', 'bar2']]);
+INFO: ([['foo', 'bar'], ['foo2', 'bar2']], <class 'list'>)
+ test_type_conversion_array_text
+---------------------------------
+ {{foo,bar},{foo2,bar2}}
+(1 row)
+
CREATE FUNCTION test_type_conversion_array_bytea(x bytea[]) RETURNS bytea[] AS $$
plpy.info(x, type(x))
return x
@@ -578,6 +595,13 @@ SELECT * FROM test_type_conversion_array_mixed2();
ERROR: invalid input syntax for integer: "abc"
CONTEXT: while creating return value
PL/Python function "test_type_conversion_array_mixed2"
+CREATE FUNCTION test_type_conversion_mdarray_malformed() RETURNS int[] AS $$
+return [[1,2,3],[4,5]]
+$$ LANGUAGE plpython3u;
+SELECT * FROM test_type_conversion_mdarray_malformed();
+ERROR: Multidimensional arrays must have array expressions with matching dimensions. PL/Python function return value has sequence length 2 while expected 3
+CONTEXT: while creating return value
+PL/Python function "test_type_conversion_mdarray_malformed"
CREATE FUNCTION test_type_conversion_array_record() RETURNS type_record[] AS $$
return [{'first': 'one', 'second': 42}, {'first': 'two', 'second': 11}]
$$ LANGUAGE plpython3u;
diff --git a/src/pl/plpython/plpy_typeio.c b/src/pl/plpython/plpy_typeio.c
index 70f2e6d..9436412 100644
--- a/src/pl/plpython/plpy_typeio.c
+++ b/src/pl/plpython/plpy_typeio.c
@@ -631,46 +631,111 @@ PLyList_FromArray(PLyDatumToOb *arg, Datum d)
{
ArrayType *array = DatumGetArrayTypeP(d);
PLyDatumToOb *elm = arg->elm;
- PyObject *list;
- int length;
- int lbound;
+ int ndim;
+ int *dims;
+ int *lb;
+ char *dataptr;
+ bits8 *bitmap;
+ int bitmask;
int i;
+ int dim;
+ int indx[MAXDIM];
+ PyObject *lists[MAXDIM];
if (ARR_NDIM(array) == 0)
return PyList_New(0);
- if (ARR_NDIM(array) != 1)
- ereport(ERROR,
- (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
- errmsg("cannot convert multidimensional array to Python list"),
- errdetail("PL/Python only supports one-dimensional arrays.")));
+ /* Array dimensions and left bounds */
+ ndim = ARR_NDIM(array);
+ dims = ARR_DIMS(array);
+ lb = ARR_LBOUND(array);
- length = ARR_DIMS(array)[0];
- lbound = ARR_LBOUND(array)[0];
- list = PyList_New(length);
- if (list == NULL)
- PLy_elog(ERROR, "could not create new Python list");
+ /* Internal array representation pointers */
+ dataptr = ARR_DATA_PTR(array);
+ bitmap = ARR_NULLBITMAP(array);
+ bitmask = 1;
- for (i = 0; i < length; i++)
+ /* Iterators initialization */
+ for (i = 0; i < ndim; i++) {
+ indx[i] = lb[i];
+ lists[i] = NULL;
+ }
+ lists[0] = PyList_New(dims[0]);
+
+ /* We need this incref to keep the pointer valid after the array traversal
+ * terminates, as this traversal does DECREF for all the lists in array */
+ Py_INCREF(lists[0]);
+
+ /* In this cycle we are going over array dimensions. Postgres offers you an
+ * option to iterate over all the multi-dimensional array elemens in order.
+ * For 3-dimesnional array the order of iteration would be following - first
+ * you start with [0,0,0] elements through [0,0,k], then [0,1,0] till [0,1,k]
+ * till [0,m,k], then [1,0,0] till [1,0,k] till [1,m,k], and so on.
+ * In Python, each 1-d array is a separate list object, so 3-d array of
+ * [n,m,k] element is a list of n m-element arrays, each element of which is
+ * k-element array. In this cycle we traverse from outter dimensions to
+ * inner ones, creating nested Python lists during traversal */
+ dim = 0;
+ while (dim >= 0)
{
- Datum elem;
- bool isnull;
- int offset;
-
- offset = lbound + i;
- elem = array_ref(array, 1, &offset, arg->typlen,
- elm->typlen, elm->typbyval, elm->typalign,
- &isnull);
- if (isnull)
+ /* If we finished up iterating over current dimension - go one level up */
+ if (indx[dim] > dims[dim])
{
- Py_INCREF(Py_None);
- PyList_SET_ITEM(list, i, Py_None);
+ Py_DECREF(lists[dim]);
+ indx[dim] = 0;
+ dim -= 1;
+ }
+ /* If we are processing inner dimension - create one more list */
+ else if (dim < ndim - 1)
+ {
+ lists[dim+1] = PyList_New(dims[dim+1]);
+
+ /* We need this INCREF as we keep array pointer on our side,
+ * while PyList_SET_ITEM steals the reference */
+ Py_INCREF(lists[dim+1]);
+
+ PyList_SET_ITEM(lists[dim], indx[dim] - lb[dim], lists[dim+1]);
+ indx[dim] += 1;
+ dim += 1;
+ indx[dim] = lb[dim];
+ }
+ /* If we are iterating over the outter dimension, fill the list with
+ * values from the original Postgres array */
+ else if (dim == ndim - 1)
+ {
+ for (indx[dim] = lb[dim]; indx[dim] <= dims[dim]; indx[dim]++)
+ {
+ /* checking for NULL */
+ if (bitmap && (*bitmap & bitmask) == 0)
+ {
+ Py_INCREF(Py_None);
+ PyList_SET_ITEM(lists[dim], indx[dim] - lb[dim], Py_None);
+ }
+ else
+ {
+ Datum itemvalue;
+
+ itemvalue = fetch_att(dataptr, elm->typbyval, elm->typlen);
+ PyList_SET_ITEM(lists[dim], indx[dim] - lb[dim], elm->func(elm, itemvalue));
+ dataptr = att_addlength_pointer(dataptr, elm->typlen, dataptr);
+ dataptr = (char *) att_align_nominal(dataptr, elm->typalign);
+ }
+
+ /* advance bitmap pointer if any */
+ if (bitmap)
+ {
+ bitmask <<= 1;
+ if (bitmask == 0x100 /* (1<<8) */)
+ {
+ bitmap++;
+ bitmask = 1;
+ }
+ }
+ }
}
- else
- PyList_SET_ITEM(list, i, elm->func(elm, elem));
}
- return list;
+ return lists[0];
}
/*
@@ -864,39 +929,131 @@ static Datum
PLySequence_ToArray(PLyObToDatum *arg, int32 typmod, PyObject *plrv)
{
ArrayType *array;
- Datum rv;
int i;
Datum *elems;
bool *nulls;
int len;
- int lbs;
+ int ndim;
+ int dims[MAXDIM];
+ int lbs[MAXDIM];
+ int indx[MAXDIM];
+ PyObject *stack[MAXDIM];
+ int dim;
+ int idxelem;
+ Datum rv;
Assert(plrv != Py_None);
if (!PySequence_Check(plrv))
PLy_elog(ERROR, "return value of function with array return type is not a Python sequence");
- len = PySequence_Length(plrv);
+ ndim = 1;
+ dims[0] = PySequence_Length(plrv);
+ len = dims[0];
+ stack[0] = plrv;
+
+ /* We don't want to create multi-dimensional arrays when we have an empty sequence,
+ * when we need to parse sequence of composite objects, and when we have strings */
+ if (len > 0 && !type_is_rowtype(get_base_element_type(arg->typoid)) &&
+ !PyString_Check(plrv) && !PyBytes_Check(plrv) && !PyUnicode_Check(plrv))
+ {
+ PyObject *pyptr = PySequence_GetItem(plrv, 0);
+
+ /* We want to iterate through all iterable objects except by strings on nested levels */
+ while (pyptr != NULL && PySequence_Check(pyptr) &&
+ !(PyString_Check(pyptr) || PyBytes_Check(pyptr) || PyUnicode_Check(pyptr)))
+ {
+ dims[ndim] = PySequence_Length(pyptr);
+ if (dims[ndim] < 0)
+ PLy_elog(ERROR, "Cannot determine sequence length for function return value");
+ len *= dims[ndim];
+ stack[ndim] = pyptr;
+ ndim += 1;
+ if (dims[ndim - 1] == 0) {
+ pyptr = NULL;
+ break;
+ }
+ pyptr = PySequence_GetItem(pyptr, 0);
+ }
+
+ /* Pyptr points to element of n-dimensional array, we don't need its reference */
+ Py_XDECREF(pyptr);
+ }
+
+ /* We need this incref to keep the pointer valid after the array traversal
+ * terminates, as this traversal does DECREF for all the lists in array, and
+ * stack[0] corresponds to function return value */
+ Py_INCREF(stack[0]);
+
elems = palloc(sizeof(*elems) * len);
nulls = palloc(sizeof(*nulls) * len);
- for (i = 0; i < len; i++)
- {
- PyObject *obj = PySequence_GetItem(plrv, i);
+ for (i = 0; i < ndim; i++) {
+ indx[i] = 0;
+ lbs[i] = 1;
+ }
- if (obj == Py_None)
- nulls[i] = true;
- else
+ /* In this cycle we are going over nested Python lists, fetching elements
+ * from the deepest level and putting them into a linear array for Postgres
+ * to interpret them as n-dimensional array. This is a cycle implementation
+ * of DFS (recursive traversal of nested arrays here), keeping the stack in
+ * "stack" variable */
+ dim = 0;
+ idxelem = 0;
+ while (dim >= 0)
+ {
+ /* If we finished up iterating over current list - go one level up */
+ if (indx[dim] == dims[dim])
+ {
+ Py_DECREF(stack[dim]);
+ indx[dim] = 0;
+ dim -= 1;
+ }
+ /* If we are processing inner list - create one more list */
+ else if (dim < ndim - 1)
{
- nulls[i] = false;
- elems[i] = arg->elm->func(arg->elm, -1, obj);
+ stack[dim+1] = PySequence_GetItem(stack[dim], indx[dim]);
+ if (PySequence_Length(stack[dim+1]) != dims[dim+1])
+ PLy_elog(ERROR, "Multidimensional arrays must have array expressions with matching dimensions. "
+ "PL/Python function return value has sequence length %d while expected %d",
+ (int)PySequence_Length(stack[dim+1]), dims[dim+1]);
+ indx[dim] += 1;
+ dim += 1;
+ }
+ /* If we are iterating over the outter list, fill the output array */
+ else if (dim == ndim - 1)
+ {
+ for (indx[dim] = 0; indx[dim] < dims[dim]; indx[dim]++)
+ {
+ PyObject *obj = PySequence_GetItem(stack[dim], indx[dim]);
+
+ if (obj == Py_None)
+ nulls[idxelem] = true;
+ else
+ {
+ nulls[idxelem] = false;
+
+ /*
+ * We don't support arrays of row types yet, so the first argument
+ * can be NULL.
+ */
+ elems[idxelem] = arg->elm->func(arg->elm, -1, obj);
+ }
+ Py_XDECREF(obj);
+ idxelem += 1;
+ }
}
- Py_XDECREF(obj);
}
- lbs = 1;
- array = construct_md_array(elems, nulls, 1, &len, &lbs,
- get_base_element_type(arg->typoid), arg->elm->typlen, arg->elm->typbyval, arg->elm->typalign);
+ array = construct_md_array(elems,
+ nulls,
+ ndim,
+ dims,
+ lbs,
+ get_base_element_type(arg->typoid),
+ arg->elm->typlen,
+ arg->elm->typbyval,
+ arg->elm->typalign);
/*
* If the result type is a domain of array, the resulting array must be
diff --git a/src/pl/plpython/sql/plpython_types.sql b/src/pl/plpython/sql/plpython_types.sql
index 19d920d..33abfac 100644
--- a/src/pl/plpython/sql/plpython_types.sql
+++ b/src/pl/plpython/sql/plpython_types.sql
@@ -237,7 +237,81 @@ SELECT * FROM test_type_conversion_array_int4(ARRAY[NULL,1]);
SELECT * FROM test_type_conversion_array_int4(ARRAY[]::integer[]);
SELECT * FROM test_type_conversion_array_int4(NULL);
SELECT * FROM test_type_conversion_array_int4(ARRAY[[1,2,3],[4,5,6]]);
+SELECT * FROM test_type_conversion_array_int4(ARRAY[[[1,2,NULL],[NULL,5,6]],[[NULL,8,9],[10,11,12]]]);
+CREATE FUNCTION test_type_conversion_array_int8(x int8[]) RETURNS int8[] AS $$
+plpy.info(x, type(x))
+return x
+$$ LANGUAGE plpythonu;
+
+SELECT * FROM test_type_conversion_array_int8(ARRAY[[[1,2,NULL],[NULL,5,6]],[[NULL,8,9],[10,11,12]]]::int8[]);
+
+CREATE FUNCTION test_type_conversion_array_float4(x float4[]) RETURNS float4[] AS $$
+plpy.info(x, type(x))
+return x
+$$ LANGUAGE plpythonu;
+
+SELECT * FROM test_type_conversion_array_float4(ARRAY[[[1.2,2.3,NULL],[NULL,5.7,6.8]],[[NULL,8.9,9.345],[10.123,11.456,12.6768]]]::float4[]);
+
+CREATE FUNCTION test_type_conversion_array_float8(x float8[]) RETURNS float8[] AS $$
+plpy.info(x, type(x))
+return x
+$$ LANGUAGE plpythonu;
+
+SELECT * FROM test_type_conversion_array_float8(ARRAY[[[1.2,2.3,NULL],[NULL,5.7,6.8]],[[NULL,8.9,9.345],[10.123,11.456,12.6768]]]::float8[]);
+
+CREATE FUNCTION test_type_conversion_array_date(x date[]) RETURNS date[] AS $$
+plpy.info(x, type(x))
+return x
+$$ LANGUAGE plpythonu;
+
+SELECT * FROM test_type_conversion_array_date(ARRAY[[['2016-09-21','2016-09-22',NULL],[NULL,'2016-10-21','2016-10-22']],
+ [[NULL,'2016-11-21','2016-10-21'],['2015-09-21','2015-09-22','2014-09-21']]]::date[]);
+
+CREATE FUNCTION test_type_conversion_array_timestamp(x timestamp[]) RETURNS timestamp[] AS $$
+plpy.info(x, type(x))
+return x
+$$ LANGUAGE plpythonu;
+
+SELECT * FROM test_type_conversion_array_timestamp(ARRAY[[['2016-09-21 15:34:24.078792-04','2016-10-22 11:34:24.078795-04',NULL],
+ [NULL,'2016-10-21 11:34:25.078792-04','2016-10-21 11:34:24.098792-04']],
+ [[NULL,'2016-01-21 11:34:24.078792-04','2016-11-21 11:34:24.108792-04'],
+ ['2015-09-21 11:34:24.079792-04','2014-09-21 11:34:24.078792-04','2013-09-21 11:34:24.078792-04']]]::timestamp[]);
+
+
+CREATE OR REPLACE FUNCTION pyreturnmultidemint4(h int4, i int4, j int4, k int4 ) RETURNS int4[] AS $BODY$
+m = [[[[x for x in range(h)] for y in range(i)] for z in range(j)] for w in range(k)]
+plpy.info(m, type(m))
+return m
+$BODY$ LANGUAGE plpythonu;
+
+select pyreturnmultidemint4(8,5,3,2);
+
+CREATE OR REPLACE FUNCTION pyreturnmultidemint8(h int4, i int4, j int4, k int4 ) RETURNS int8[] AS $BODY$
+m = [[[[x for x in range(h)] for y in range(i)] for z in range(j)] for w in range(k)]
+plpy.info(m, type(m))
+return m
+$BODY$ LANGUAGE plpythonu;
+
+select pyreturnmultidemint8(5,5,3,2);
+
+CREATE OR REPLACE FUNCTION pyreturnmultidemfloat4(h int4, i int4, j int4, k int4 ) RETURNS float4[] AS $BODY$
+
+m = [[[[x for x in range(h)] for y in range(i)] for z in range(j)] for w in range(k)]
+plpy.info(m, type(m))
+return m
+$BODY$ LANGUAGE plpythonu;
+
+select pyreturnmultidemfloat4(6,5,3,2);
+
+CREATE OR REPLACE FUNCTION pyreturnmultidemfloat8(h int4, i int4, j int4, k int4 ) RETURNS float8[] AS $BODY$
+
+m = [[[[x for x in range(h)] for y in range(i)] for z in range(j)] for w in range(k)]
+plpy.info(m, type(m))
+return m
+$BODY$ LANGUAGE plpythonu;
+
+select pyreturnmultidemfloat8(7,5,3,2);
CREATE FUNCTION test_type_conversion_array_text(x text[]) RETURNS text[] AS $$
plpy.info(x, type(x))
@@ -245,6 +319,7 @@ return x
$$ LANGUAGE plpythonu;
SELECT * FROM test_type_conversion_array_text(ARRAY['foo', 'bar']);
+SELECT * FROM test_type_conversion_array_text(ARRAY[['foo', 'bar'],['foo2', 'bar2']]);
CREATE FUNCTION test_type_conversion_array_bytea(x bytea[]) RETURNS bytea[] AS $$
@@ -268,6 +343,11 @@ $$ LANGUAGE plpythonu;
SELECT * FROM test_type_conversion_array_mixed2();
+CREATE FUNCTION test_type_conversion_mdarray_malformed() RETURNS int[] AS $$
+return [[1,2,3],[4,5]]
+$$ LANGUAGE plpythonu;
+
+SELECT * FROM test_type_conversion_mdarray_malformed();
CREATE FUNCTION test_type_conversion_array_record() RETURNS type_record[] AS $$
return [{'first': 'one', 'second': 42}, {'first': 'two', 'second': 11}]
Re: PL/Python adding support for multi-dimensional arrays
Hi
2016-09-21 19:53 GMT+02:00 Dave Cramer <pg@fastcrypt.com>:
On 18 September 2016 at 09:27, Dave Cramer <pg@fastcrypt.com> wrote:
On 10 August 2016 at 01:53, Pavel Stehule <pavel.stehule@gmail.com>
wrote:Hi
2016-08-03 13:54 GMT+02:00 Alexey Grishchenko <agrishchenko@pivotal.io>:
On Wed, Aug 3, 2016 at 12:49 PM, Alexey Grishchenko <
agrishchenko@pivotal.io> wrote:Hi
Current implementation of PL/Python does not allow the use of
multi-dimensional arrays, for both input and output parameters. This forces
end users to introduce workarounds like casting arrays to text before
passing them to the functions and parsing them after, which is an
error-prone approachThis patch adds support for multi-dimensional arrays as both input and
output parameters for PL/Python functions. The number of dimensions
supported is limited by Postgres MAXDIM macrovariable, by default equal to
6. Both input and output multi-dimensional arrays should have fixed
dimension sizes, i.e. 2-d arrays should represent MxN matrix, 3-d arrays
represent MxNxK cube, etc.This patch does not support multi-dimensional arrays of composite
types, as composite types in Python might be represented as iterators and
there is no obvious way to find out when the nested array stops and
composite type structure starts. For example, if we have a composite type
of (int, text), we can try to return "[ [ [1,'a'], [2,'b'] ], [ [3,'c'],
[4,'d'] ] ]", and it is hard to find out that the first two lists are
lists, and the third one represents structure. Things are getting even more
complex when you have arrays as members of composite type. This is why I
think this limitation is reasonable.Given the function:
CREATE FUNCTION test_type_conversion_array_int4(x int4[]) RETURNS
int4[] AS $$
plpy.info(x, type(x))
return x
$$ LANGUAGE plpythonu;Before patch:
# SELECT * FROM test_type_conversion_array_int
4(ARRAY[[1,2,3],[4,5,6]]);
ERROR: cannot convert multidimensional array to Python list
DETAIL: PL/Python only supports one-dimensional arrays.
CONTEXT: PL/Python function "test_type_conversion_array_int4"After patch:
# SELECT * FROM test_type_conversion_array_int
4(ARRAY[[1,2,3],[4,5,6]]);
INFO: ([[1, 2, 3], [4, 5, 6]], <type 'list'>)
test_type_conversion_array_int4
---------------------------------
{{1,2,3},{4,5,6}}
(1 row)--
Best regards,
Alexey GrishchenkoAlso this patch incorporates the fix for https://www.postgresql.org
/message-id/CAH38_tkwA5qgLV8zPN1OpPzhtkNKQb30n3xq-2NR9jUfv3q
wHA%40mail.gmail.com, as they touch the same piece of code - array
manipulation in PL/PythonI am sending review of this patch:
1. The implemented functionality is clearly benefit - passing MD arrays,
pretty faster passing bigger arrays
2. I was able to use this patch cleanly without any errors or warnings
3. There is no any error or warning
4. All tests passed - I tested Python 2.7 and Python 3.5
5. The code is well commented and clean
6. For this new functionality the documentation is not necessary7. I invite more regress tests for both directions (Python <-> Postgres)
for more than two dimensionsMy only one objection is not enough regress tests - after fixing this
patch will be ready for commiters.
Now, the tests are enough - so I'll mark this patch as ready for commiters.
I had to fix tests - there was lot of white spaces, and the result for
python3 was missing
Regards
Pavel
Show quoted text
Good work, Alexey
Thank you
Regards
Pavel
--
Best regards,
Alexey GrishchenkoPavel,
I will pick this up.
Pavel,
Please see attached patch which provides more test cases
I just realized this patch contains the original patch as well. What is
the protocol for sending in subsequent patches ?Dave Cramer
davec@postgresintl.com
www.postgresintl.com
Attachments:
PL-Python-adding-support-for-multi-dimensional-arrays-20160922.patchtext/x-patch; charset=US-ASCII; name=PL-Python-adding-support-for-multi-dimensional-arrays-20160922.patchDownload
diff --git a/src/pl/plpython/expected/plpython_types.out b/src/pl/plpython/expected/plpython_types.out
index f0b6abd..296ef8b 100644
--- a/src/pl/plpython/expected/plpython_types.out
+++ b/src/pl/plpython/expected/plpython_types.out
@@ -537,9 +537,126 @@ INFO: (None, <type 'NoneType'>)
(1 row)
SELECT * FROM test_type_conversion_array_int4(ARRAY[[1,2,3],[4,5,6]]);
-ERROR: cannot convert multidimensional array to Python list
-DETAIL: PL/Python only supports one-dimensional arrays.
-CONTEXT: PL/Python function "test_type_conversion_array_int4"
+INFO: ([[1, 2, 3], [4, 5, 6]], <type 'list'>)
+ test_type_conversion_array_int4
+---------------------------------
+ {{1,2,3},{4,5,6}}
+(1 row)
+
+SELECT * FROM test_type_conversion_array_int4(ARRAY[[[1,2,NULL],[NULL,5,6]],[[NULL,8,9],[10,11,12]]]);
+INFO: ([[[1, 2, None], [None, 5, 6]], [[None, 8, 9], [10, 11, 12]]], <type 'list'>)
+ test_type_conversion_array_int4
+---------------------------------------------------
+ {{{1,2,NULL},{NULL,5,6}},{{NULL,8,9},{10,11,12}}}
+(1 row)
+
+CREATE FUNCTION test_type_conversion_array_int8(x int8[]) RETURNS int8[] AS $$
+plpy.info(x, type(x))
+return x
+$$ LANGUAGE plpythonu;
+SELECT * FROM test_type_conversion_array_int8(ARRAY[[[1,2,NULL],[NULL,5,6]],[[NULL,8,9],[10,11,12]]]::int8[]);
+INFO: ([[[1L, 2L, None], [None, 5L, 6L]], [[None, 8L, 9L], [10L, 11L, 12L]]], <type 'list'>)
+ test_type_conversion_array_int8
+---------------------------------------------------
+ {{{1,2,NULL},{NULL,5,6}},{{NULL,8,9},{10,11,12}}}
+(1 row)
+
+CREATE FUNCTION test_type_conversion_array_float4(x float4[]) RETURNS float4[] AS $$
+plpy.info(x, type(x))
+return x
+$$ LANGUAGE plpythonu;
+SELECT * FROM test_type_conversion_array_float4(ARRAY[[[1.2,2.3,NULL],[NULL,5.7,6.8]],[[NULL,8.9,9.345],[10.123,11.456,12.6768]]]::float4[]);
+INFO: ([[[1.2000000476837158, 2.299999952316284, None], [None, 5.699999809265137, 6.800000190734863]], [[None, 8.899999618530273, 9.345000267028809], [10.123000144958496, 11.456000328063965, 12.676799774169922]]], <type 'list'>)
+ test_type_conversion_array_float4
+------------------------------------------------------------------------------
+ {{{1.2,2.3,NULL},{NULL,5.7,6.8}},{{NULL,8.9,9.345},{10.123,11.456,12.6768}}}
+(1 row)
+
+CREATE FUNCTION test_type_conversion_array_float8(x float8[]) RETURNS float8[] AS $$
+plpy.info(x, type(x))
+return x
+$$ LANGUAGE plpythonu;
+SELECT * FROM test_type_conversion_array_float8(ARRAY[[[1.2,2.3,NULL],[NULL,5.7,6.8]],[[NULL,8.9,9.345],[10.123,11.456,12.6768]]]::float8[]);
+INFO: ([[[1.2, 2.3, None], [None, 5.7, 6.8]], [[None, 8.9, 9.345], [10.123, 11.456, 12.6768]]], <type 'list'>)
+ test_type_conversion_array_float8
+------------------------------------------------------------------------------
+ {{{1.2,2.3,NULL},{NULL,5.7,6.8}},{{NULL,8.9,9.345},{10.123,11.456,12.6768}}}
+(1 row)
+
+CREATE FUNCTION test_type_conversion_array_date(x date[]) RETURNS date[] AS $$
+plpy.info(x, type(x))
+return x
+$$ LANGUAGE plpythonu;
+SELECT * FROM test_type_conversion_array_date(ARRAY[[['2016-09-21','2016-09-22',NULL],[NULL,'2016-10-21','2016-10-22']],
+ [[NULL,'2016-11-21','2016-10-21'],['2015-09-21','2015-09-22','2014-09-21']]]::date[]);
+INFO: ([[['09-21-2016', '09-22-2016', None], [None, '10-21-2016', '10-22-2016']], [[None, '11-21-2016', '10-21-2016'], ['09-21-2015', '09-22-2015', '09-21-2014']]], <type 'list'>)
+ test_type_conversion_array_date
+---------------------------------------------------------------------------------------------------------------------------------
+ {{{09-21-2016,09-22-2016,NULL},{NULL,10-21-2016,10-22-2016}},{{NULL,11-21-2016,10-21-2016},{09-21-2015,09-22-2015,09-21-2014}}}
+(1 row)
+
+CREATE FUNCTION test_type_conversion_array_timestamp(x timestamp[]) RETURNS timestamp[] AS $$
+plpy.info(x, type(x))
+return x
+$$ LANGUAGE plpythonu;
+SELECT * FROM test_type_conversion_array_timestamp(ARRAY[[['2016-09-21 15:34:24.078792-04','2016-10-22 11:34:24.078795-04',NULL],
+ [NULL,'2016-10-21 11:34:25.078792-04','2016-10-21 11:34:24.098792-04']],
+ [[NULL,'2016-01-21 11:34:24.078792-04','2016-11-21 11:34:24.108792-04'],
+ ['2015-09-21 11:34:24.079792-04','2014-09-21 11:34:24.078792-04','2013-09-21 11:34:24.078792-04']]]::timestamp[]);
+INFO: ([[['Wed Sep 21 15:34:24.078792 2016', 'Sat Oct 22 11:34:24.078795 2016', None], [None, 'Fri Oct 21 11:34:25.078792 2016', 'Fri Oct 21 11:34:24.098792 2016']], [[None, 'Thu Jan 21 11:34:24.078792 2016', 'Mon Nov 21 11:34:24.108792 2016'], ['Mon Sep 21 11:34:24.079792 2015', 'Sun Sep 21 11:34:24.078792 2014', 'Sat Sep 21 11:34:24.078792 2013']]], <type 'list'>)
+ test_type_conversion_array_timestamp
+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
+ {{{"Wed Sep 21 15:34:24.078792 2016","Sat Oct 22 11:34:24.078795 2016",NULL},{NULL,"Fri Oct 21 11:34:25.078792 2016","Fri Oct 21 11:34:24.098792 2016"}},{{NULL,"Thu Jan 21 11:34:24.078792 2016","Mon Nov 21 11:34:24.108792 2016"},{"Mon Sep 21 11:34:24.079792 2015","Sun Sep 21 11:34:24.078792 2014","Sat Sep 21 11:34:24.078792 2013"}}}
+(1 row)
+
+CREATE OR REPLACE FUNCTION pyreturnmultidemint4(h int4, i int4, j int4, k int4 ) RETURNS int4[] AS $BODY$
+m = [[[[x for x in range(h)] for y in range(i)] for z in range(j)] for w in range(k)]
+plpy.info(m, type(m))
+return m
+$BODY$ LANGUAGE plpythonu;
+select pyreturnmultidemint4(8,5,3,2);
+INFO: ([[[[0, 1, 2, 3, 4, 5, 6, 7], [0, 1, 2, 3, 4, 5, 6, 7], [0, 1, 2, 3, 4, 5, 6, 7], [0, 1, 2, 3, 4, 5, 6, 7], [0, 1, 2, 3, 4, 5, 6, 7]], [[0, 1, 2, 3, 4, 5, 6, 7], [0, 1, 2, 3, 4, 5, 6, 7], [0, 1, 2, 3, 4, 5, 6, 7], [0, 1, 2, 3, 4, 5, 6, 7], [0, 1, 2, 3, 4, 5, 6, 7]], [[0, 1, 2, 3, 4, 5, 6, 7], [0, 1, 2, 3, 4, 5, 6, 7], [0, 1, 2, 3, 4, 5, 6, 7], [0, 1, 2, 3, 4, 5, 6, 7], [0, 1, 2, 3, 4, 5, 6, 7]]], [[[0, 1, 2, 3, 4, 5, 6, 7], [0, 1, 2, 3, 4, 5, 6, 7], [0, 1, 2, 3, 4, 5, 6, 7], [0, 1, 2, 3, 4, 5, 6, 7], [0, 1, 2, 3, 4, 5, 6, 7]], [[0, 1, 2, 3, 4, 5, 6, 7], [0, 1, 2, 3, 4, 5, 6, 7], [0, 1, 2, 3, 4, 5, 6, 7], [0, 1, 2, 3, 4, 5, 6, 7], [0, 1, 2, 3, 4, 5, 6, 7]], [[0, 1, 2, 3, 4, 5, 6, 7], [0, 1, 2, 3, 4, 5, 6, 7], [0, 1, 2, 3, 4, 5, 6, 7], [0, 1, 2, 3, 4, 5, 6, 7], [0, 1, 2, 3, 4, 5, 6, 7]]]], <type 'list'>)
+ pyreturnmultidemint4
+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
+ {{{{0,1,2,3,4,5,6,7},{0,1,2,3,4,5,6,7},{0,1,2,3,4,5,6,7},{0,1,2,3,4,5,6,7},{0,1,2,3,4,5,6,7}},{{0,1,2,3,4,5,6,7},{0,1,2,3,4,5,6,7},{0,1,2,3,4,5,6,7},{0,1,2,3,4,5,6,7},{0,1,2,3,4,5,6,7}},{{0,1,2,3,4,5,6,7},{0,1,2,3,4,5,6,7},{0,1,2,3,4,5,6,7},{0,1,2,3,4,5,6,7},{0,1,2,3,4,5,6,7}}},{{{0,1,2,3,4,5,6,7},{0,1,2,3,4,5,6,7},{0,1,2,3,4,5,6,7},{0,1,2,3,4,5,6,7},{0,1,2,3,4,5,6,7}},{{0,1,2,3,4,5,6,7},{0,1,2,3,4,5,6,7},{0,1,2,3,4,5,6,7},{0,1,2,3,4,5,6,7},{0,1,2,3,4,5,6,7}},{{0,1,2,3,4,5,6,7},{0,1,2,3,4,5,6,7},{0,1,2,3,4,5,6,7},{0,1,2,3,4,5,6,7},{0,1,2,3,4,5,6,7}}}}
+(1 row)
+
+CREATE OR REPLACE FUNCTION pyreturnmultidemint8(h int4, i int4, j int4, k int4 ) RETURNS int8[] AS $BODY$
+m = [[[[x for x in range(h)] for y in range(i)] for z in range(j)] for w in range(k)]
+plpy.info(m, type(m))
+return m
+$BODY$ LANGUAGE plpythonu;
+select pyreturnmultidemint8(5,5,3,2);
+INFO: ([[[[0, 1, 2, 3, 4], [0, 1, 2, 3, 4], [0, 1, 2, 3, 4], [0, 1, 2, 3, 4], [0, 1, 2, 3, 4]], [[0, 1, 2, 3, 4], [0, 1, 2, 3, 4], [0, 1, 2, 3, 4], [0, 1, 2, 3, 4], [0, 1, 2, 3, 4]], [[0, 1, 2, 3, 4], [0, 1, 2, 3, 4], [0, 1, 2, 3, 4], [0, 1, 2, 3, 4], [0, 1, 2, 3, 4]]], [[[0, 1, 2, 3, 4], [0, 1, 2, 3, 4], [0, 1, 2, 3, 4], [0, 1, 2, 3, 4], [0, 1, 2, 3, 4]], [[0, 1, 2, 3, 4], [0, 1, 2, 3, 4], [0, 1, 2, 3, 4], [0, 1, 2, 3, 4], [0, 1, 2, 3, 4]], [[0, 1, 2, 3, 4], [0, 1, 2, 3, 4], [0, 1, 2, 3, 4], [0, 1, 2, 3, 4], [0, 1, 2, 3, 4]]]], <type 'list'>)
+ pyreturnmultidemint8
+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
+ {{{{0,1,2,3,4},{0,1,2,3,4},{0,1,2,3,4},{0,1,2,3,4},{0,1,2,3,4}},{{0,1,2,3,4},{0,1,2,3,4},{0,1,2,3,4},{0,1,2,3,4},{0,1,2,3,4}},{{0,1,2,3,4},{0,1,2,3,4},{0,1,2,3,4},{0,1,2,3,4},{0,1,2,3,4}}},{{{0,1,2,3,4},{0,1,2,3,4},{0,1,2,3,4},{0,1,2,3,4},{0,1,2,3,4}},{{0,1,2,3,4},{0,1,2,3,4},{0,1,2,3,4},{0,1,2,3,4},{0,1,2,3,4}},{{0,1,2,3,4},{0,1,2,3,4},{0,1,2,3,4},{0,1,2,3,4},{0,1,2,3,4}}}}
+(1 row)
+
+CREATE OR REPLACE FUNCTION pyreturnmultidemfloat4(h int4, i int4, j int4, k int4 ) RETURNS float4[] AS $BODY$
+m = [[[[x for x in range(h)] for y in range(i)] for z in range(j)] for w in range(k)]
+plpy.info(m, type(m))
+return m
+$BODY$ LANGUAGE plpythonu;
+select pyreturnmultidemfloat4(6,5,3,2);
+INFO: ([[[[0, 1, 2, 3, 4, 5], [0, 1, 2, 3, 4, 5], [0, 1, 2, 3, 4, 5], [0, 1, 2, 3, 4, 5], [0, 1, 2, 3, 4, 5]], [[0, 1, 2, 3, 4, 5], [0, 1, 2, 3, 4, 5], [0, 1, 2, 3, 4, 5], [0, 1, 2, 3, 4, 5], [0, 1, 2, 3, 4, 5]], [[0, 1, 2, 3, 4, 5], [0, 1, 2, 3, 4, 5], [0, 1, 2, 3, 4, 5], [0, 1, 2, 3, 4, 5], [0, 1, 2, 3, 4, 5]]], [[[0, 1, 2, 3, 4, 5], [0, 1, 2, 3, 4, 5], [0, 1, 2, 3, 4, 5], [0, 1, 2, 3, 4, 5], [0, 1, 2, 3, 4, 5]], [[0, 1, 2, 3, 4, 5], [0, 1, 2, 3, 4, 5], [0, 1, 2, 3, 4, 5], [0, 1, 2, 3, 4, 5], [0, 1, 2, 3, 4, 5]], [[0, 1, 2, 3, 4, 5], [0, 1, 2, 3, 4, 5], [0, 1, 2, 3, 4, 5], [0, 1, 2, 3, 4, 5], [0, 1, 2, 3, 4, 5]]]], <type 'list'>)
+ pyreturnmultidemfloat4
+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
+ {{{{0,1,2,3,4,5},{0,1,2,3,4,5},{0,1,2,3,4,5},{0,1,2,3,4,5},{0,1,2,3,4,5}},{{0,1,2,3,4,5},{0,1,2,3,4,5},{0,1,2,3,4,5},{0,1,2,3,4,5},{0,1,2,3,4,5}},{{0,1,2,3,4,5},{0,1,2,3,4,5},{0,1,2,3,4,5},{0,1,2,3,4,5},{0,1,2,3,4,5}}},{{{0,1,2,3,4,5},{0,1,2,3,4,5},{0,1,2,3,4,5},{0,1,2,3,4,5},{0,1,2,3,4,5}},{{0,1,2,3,4,5},{0,1,2,3,4,5},{0,1,2,3,4,5},{0,1,2,3,4,5},{0,1,2,3,4,5}},{{0,1,2,3,4,5},{0,1,2,3,4,5},{0,1,2,3,4,5},{0,1,2,3,4,5},{0,1,2,3,4,5}}}}
+(1 row)
+
+CREATE OR REPLACE FUNCTION pyreturnmultidemfloat8(h int4, i int4, j int4, k int4 ) RETURNS float8[] AS $BODY$
+m = [[[[x for x in range(h)] for y in range(i)] for z in range(j)] for w in range(k)]
+plpy.info(m, type(m))
+return m
+$BODY$ LANGUAGE plpythonu;
+select pyreturnmultidemfloat8(7,5,3,2);
+INFO: ([[[[0, 1, 2, 3, 4, 5, 6], [0, 1, 2, 3, 4, 5, 6], [0, 1, 2, 3, 4, 5, 6], [0, 1, 2, 3, 4, 5, 6], [0, 1, 2, 3, 4, 5, 6]], [[0, 1, 2, 3, 4, 5, 6], [0, 1, 2, 3, 4, 5, 6], [0, 1, 2, 3, 4, 5, 6], [0, 1, 2, 3, 4, 5, 6], [0, 1, 2, 3, 4, 5, 6]], [[0, 1, 2, 3, 4, 5, 6], [0, 1, 2, 3, 4, 5, 6], [0, 1, 2, 3, 4, 5, 6], [0, 1, 2, 3, 4, 5, 6], [0, 1, 2, 3, 4, 5, 6]]], [[[0, 1, 2, 3, 4, 5, 6], [0, 1, 2, 3, 4, 5, 6], [0, 1, 2, 3, 4, 5, 6], [0, 1, 2, 3, 4, 5, 6], [0, 1, 2, 3, 4, 5, 6]], [[0, 1, 2, 3, 4, 5, 6], [0, 1, 2, 3, 4, 5, 6], [0, 1, 2, 3, 4, 5, 6], [0, 1, 2, 3, 4, 5, 6], [0, 1, 2, 3, 4, 5, 6]], [[0, 1, 2, 3, 4, 5, 6], [0, 1, 2, 3, 4, 5, 6], [0, 1, 2, 3, 4, 5, 6], [0, 1, 2, 3, 4, 5, 6], [0, 1, 2, 3, 4, 5, 6]]]], <type 'list'>)
+ pyreturnmultidemfloat8
+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
+ {{{{0,1,2,3,4,5,6},{0,1,2,3,4,5,6},{0,1,2,3,4,5,6},{0,1,2,3,4,5,6},{0,1,2,3,4,5,6}},{{0,1,2,3,4,5,6},{0,1,2,3,4,5,6},{0,1,2,3,4,5,6},{0,1,2,3,4,5,6},{0,1,2,3,4,5,6}},{{0,1,2,3,4,5,6},{0,1,2,3,4,5,6},{0,1,2,3,4,5,6},{0,1,2,3,4,5,6},{0,1,2,3,4,5,6}}},{{{0,1,2,3,4,5,6},{0,1,2,3,4,5,6},{0,1,2,3,4,5,6},{0,1,2,3,4,5,6},{0,1,2,3,4,5,6}},{{0,1,2,3,4,5,6},{0,1,2,3,4,5,6},{0,1,2,3,4,5,6},{0,1,2,3,4,5,6},{0,1,2,3,4,5,6}},{{0,1,2,3,4,5,6},{0,1,2,3,4,5,6},{0,1,2,3,4,5,6},{0,1,2,3,4,5,6},{0,1,2,3,4,5,6}}}}
+(1 row)
+
CREATE FUNCTION test_type_conversion_array_text(x text[]) RETURNS text[] AS $$
plpy.info(x, type(x))
return x
@@ -551,6 +668,13 @@ INFO: (['foo', 'bar'], <type 'list'>)
{foo,bar}
(1 row)
+SELECT * FROM test_type_conversion_array_text(ARRAY[['foo', 'bar'],['foo2', 'bar2']]);
+INFO: ([['foo', 'bar'], ['foo2', 'bar2']], <type 'list'>)
+ test_type_conversion_array_text
+---------------------------------
+ {{foo,bar},{foo2,bar2}}
+(1 row)
+
CREATE FUNCTION test_type_conversion_array_bytea(x bytea[]) RETURNS bytea[] AS $$
plpy.info(x, type(x))
return x
@@ -578,6 +702,13 @@ SELECT * FROM test_type_conversion_array_mixed2();
ERROR: invalid input syntax for integer: "abc"
CONTEXT: while creating return value
PL/Python function "test_type_conversion_array_mixed2"
+CREATE FUNCTION test_type_conversion_mdarray_malformed() RETURNS int[] AS $$
+return [[1,2,3],[4,5]]
+$$ LANGUAGE plpythonu;
+SELECT * FROM test_type_conversion_mdarray_malformed();
+ERROR: Multidimensional arrays must have array expressions with matching dimensions. PL/Python function return value has sequence length 2 while expected 3
+CONTEXT: while creating return value
+PL/Python function "test_type_conversion_mdarray_malformed"
CREATE FUNCTION test_type_conversion_array_record() RETURNS type_record[] AS $$
return [{'first': 'one', 'second': 42}, {'first': 'two', 'second': 11}]
$$ LANGUAGE plpythonu;
diff --git a/src/pl/plpython/expected/plpython_types_3.out b/src/pl/plpython/expected/plpython_types_3.out
index 56b78e1..b244d2a 100644
--- a/src/pl/plpython/expected/plpython_types_3.out
+++ b/src/pl/plpython/expected/plpython_types_3.out
@@ -537,9 +537,126 @@ INFO: (None, <class 'NoneType'>)
(1 row)
SELECT * FROM test_type_conversion_array_int4(ARRAY[[1,2,3],[4,5,6]]);
-ERROR: cannot convert multidimensional array to Python list
-DETAIL: PL/Python only supports one-dimensional arrays.
-CONTEXT: PL/Python function "test_type_conversion_array_int4"
+INFO: ([[1, 2, 3], [4, 5, 6]], <class 'list'>)
+ test_type_conversion_array_int4
+---------------------------------
+ {{1,2,3},{4,5,6}}
+(1 row)
+
+SELECT * FROM test_type_conversion_array_int4(ARRAY[[[1,2,NULL],[NULL,5,6]],[[NULL,8,9],[10,11,12]]]);
+INFO: ([[[1, 2, None], [None, 5, 6]], [[None, 8, 9], [10, 11, 12]]], <class 'list'>)
+ test_type_conversion_array_int4
+---------------------------------------------------
+ {{{1,2,NULL},{NULL,5,6}},{{NULL,8,9},{10,11,12}}}
+(1 row)
+
+CREATE FUNCTION test_type_conversion_array_int8(x int8[]) RETURNS int8[] AS $$
+plpy.info(x, type(x))
+return x
+$$ LANGUAGE plpython3u;
+SELECT * FROM test_type_conversion_array_int8(ARRAY[[[1,2,NULL],[NULL,5,6]],[[NULL,8,9],[10,11,12]]]::int8[]);
+INFO: ([[[1, 2, None], [None, 5, 6]], [[None, 8, 9], [10, 11, 12]]], <class 'list'>)
+ test_type_conversion_array_int8
+---------------------------------------------------
+ {{{1,2,NULL},{NULL,5,6}},{{NULL,8,9},{10,11,12}}}
+(1 row)
+
+CREATE FUNCTION test_type_conversion_array_float4(x float4[]) RETURNS float4[] AS $$
+plpy.info(x, type(x))
+return x
+$$ LANGUAGE plpython3u;
+SELECT * FROM test_type_conversion_array_float4(ARRAY[[[1.2,2.3,NULL],[NULL,5.7,6.8]],[[NULL,8.9,9.345],[10.123,11.456,12.6768]]]::float4[]);
+INFO: ([[[1.2000000476837158, 2.299999952316284, None], [None, 5.699999809265137, 6.800000190734863]], [[None, 8.899999618530273, 9.345000267028809], [10.123000144958496, 11.456000328063965, 12.676799774169922]]], <class 'list'>)
+ test_type_conversion_array_float4
+------------------------------------------------------------------------------
+ {{{1.2,2.3,NULL},{NULL,5.7,6.8}},{{NULL,8.9,9.345},{10.123,11.456,12.6768}}}
+(1 row)
+
+CREATE FUNCTION test_type_conversion_array_float8(x float8[]) RETURNS float8[] AS $$
+plpy.info(x, type(x))
+return x
+$$ LANGUAGE plpython3u;
+SELECT * FROM test_type_conversion_array_float8(ARRAY[[[1.2,2.3,NULL],[NULL,5.7,6.8]],[[NULL,8.9,9.345],[10.123,11.456,12.6768]]]::float8[]);
+INFO: ([[[1.2, 2.3, None], [None, 5.7, 6.8]], [[None, 8.9, 9.345], [10.123, 11.456, 12.6768]]], <class 'list'>)
+ test_type_conversion_array_float8
+------------------------------------------------------------------------------
+ {{{1.2,2.3,NULL},{NULL,5.7,6.8}},{{NULL,8.9,9.345},{10.123,11.456,12.6768}}}
+(1 row)
+
+CREATE FUNCTION test_type_conversion_array_date(x date[]) RETURNS date[] AS $$
+plpy.info(x, type(x))
+return x
+$$ LANGUAGE plpython3u;
+SELECT * FROM test_type_conversion_array_date(ARRAY[[['2016-09-21','2016-09-22',NULL],[NULL,'2016-10-21','2016-10-22']],
+ [[NULL,'2016-11-21','2016-10-21'],['2015-09-21','2015-09-22','2014-09-21']]]::date[]);
+INFO: ([[['09-21-2016', '09-22-2016', None], [None, '10-21-2016', '10-22-2016']], [[None, '11-21-2016', '10-21-2016'], ['09-21-2015', '09-22-2015', '09-21-2014']]], <class 'list'>)
+ test_type_conversion_array_date
+---------------------------------------------------------------------------------------------------------------------------------
+ {{{09-21-2016,09-22-2016,NULL},{NULL,10-21-2016,10-22-2016}},{{NULL,11-21-2016,10-21-2016},{09-21-2015,09-22-2015,09-21-2014}}}
+(1 row)
+
+CREATE FUNCTION test_type_conversion_array_timestamp(x timestamp[]) RETURNS timestamp[] AS $$
+plpy.info(x, type(x))
+return x
+$$ LANGUAGE plpython3u;
+SELECT * FROM test_type_conversion_array_timestamp(ARRAY[[['2016-09-21 15:34:24.078792-04','2016-10-22 11:34:24.078795-04',NULL],
+ [NULL,'2016-10-21 11:34:25.078792-04','2016-10-21 11:34:24.098792-04']],
+ [[NULL,'2016-01-21 11:34:24.078792-04','2016-11-21 11:34:24.108792-04'],
+ ['2015-09-21 11:34:24.079792-04','2014-09-21 11:34:24.078792-04','2013-09-21 11:34:24.078792-04']]]::timestamp[]);
+INFO: ([[['Wed Sep 21 15:34:24.078792 2016', 'Sat Oct 22 11:34:24.078795 2016', None], [None, 'Fri Oct 21 11:34:25.078792 2016', 'Fri Oct 21 11:34:24.098792 2016']], [[None, 'Thu Jan 21 11:34:24.078792 2016', 'Mon Nov 21 11:34:24.108792 2016'], ['Mon Sep 21 11:34:24.079792 2015', 'Sun Sep 21 11:34:24.078792 2014', 'Sat Sep 21 11:34:24.078792 2013']]], <class 'list'>)
+ test_type_conversion_array_timestamp
+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
+ {{{"Wed Sep 21 15:34:24.078792 2016","Sat Oct 22 11:34:24.078795 2016",NULL},{NULL,"Fri Oct 21 11:34:25.078792 2016","Fri Oct 21 11:34:24.098792 2016"}},{{NULL,"Thu Jan 21 11:34:24.078792 2016","Mon Nov 21 11:34:24.108792 2016"},{"Mon Sep 21 11:34:24.079792 2015","Sun Sep 21 11:34:24.078792 2014","Sat Sep 21 11:34:24.078792 2013"}}}
+(1 row)
+
+CREATE OR REPLACE FUNCTION pyreturnmultidemint4(h int4, i int4, j int4, k int4 ) RETURNS int4[] AS $BODY$
+m = [[[[x for x in range(h)] for y in range(i)] for z in range(j)] for w in range(k)]
+plpy.info(m, type(m))
+return m
+$BODY$ LANGUAGE plpython3u;
+select pyreturnmultidemint4(8,5,3,2);
+INFO: ([[[[0, 1, 2, 3, 4, 5, 6, 7], [0, 1, 2, 3, 4, 5, 6, 7], [0, 1, 2, 3, 4, 5, 6, 7], [0, 1, 2, 3, 4, 5, 6, 7], [0, 1, 2, 3, 4, 5, 6, 7]], [[0, 1, 2, 3, 4, 5, 6, 7], [0, 1, 2, 3, 4, 5, 6, 7], [0, 1, 2, 3, 4, 5, 6, 7], [0, 1, 2, 3, 4, 5, 6, 7], [0, 1, 2, 3, 4, 5, 6, 7]], [[0, 1, 2, 3, 4, 5, 6, 7], [0, 1, 2, 3, 4, 5, 6, 7], [0, 1, 2, 3, 4, 5, 6, 7], [0, 1, 2, 3, 4, 5, 6, 7], [0, 1, 2, 3, 4, 5, 6, 7]]], [[[0, 1, 2, 3, 4, 5, 6, 7], [0, 1, 2, 3, 4, 5, 6, 7], [0, 1, 2, 3, 4, 5, 6, 7], [0, 1, 2, 3, 4, 5, 6, 7], [0, 1, 2, 3, 4, 5, 6, 7]], [[0, 1, 2, 3, 4, 5, 6, 7], [0, 1, 2, 3, 4, 5, 6, 7], [0, 1, 2, 3, 4, 5, 6, 7], [0, 1, 2, 3, 4, 5, 6, 7], [0, 1, 2, 3, 4, 5, 6, 7]], [[0, 1, 2, 3, 4, 5, 6, 7], [0, 1, 2, 3, 4, 5, 6, 7], [0, 1, 2, 3, 4, 5, 6, 7], [0, 1, 2, 3, 4, 5, 6, 7], [0, 1, 2, 3, 4, 5, 6, 7]]]], <class 'list'>)
+ pyreturnmultidemint4
+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
+ {{{{0,1,2,3,4,5,6,7},{0,1,2,3,4,5,6,7},{0,1,2,3,4,5,6,7},{0,1,2,3,4,5,6,7},{0,1,2,3,4,5,6,7}},{{0,1,2,3,4,5,6,7},{0,1,2,3,4,5,6,7},{0,1,2,3,4,5,6,7},{0,1,2,3,4,5,6,7},{0,1,2,3,4,5,6,7}},{{0,1,2,3,4,5,6,7},{0,1,2,3,4,5,6,7},{0,1,2,3,4,5,6,7},{0,1,2,3,4,5,6,7},{0,1,2,3,4,5,6,7}}},{{{0,1,2,3,4,5,6,7},{0,1,2,3,4,5,6,7},{0,1,2,3,4,5,6,7},{0,1,2,3,4,5,6,7},{0,1,2,3,4,5,6,7}},{{0,1,2,3,4,5,6,7},{0,1,2,3,4,5,6,7},{0,1,2,3,4,5,6,7},{0,1,2,3,4,5,6,7},{0,1,2,3,4,5,6,7}},{{0,1,2,3,4,5,6,7},{0,1,2,3,4,5,6,7},{0,1,2,3,4,5,6,7},{0,1,2,3,4,5,6,7},{0,1,2,3,4,5,6,7}}}}
+(1 row)
+
+CREATE OR REPLACE FUNCTION pyreturnmultidemint8(h int4, i int4, j int4, k int4 ) RETURNS int8[] AS $BODY$
+m = [[[[x for x in range(h)] for y in range(i)] for z in range(j)] for w in range(k)]
+plpy.info(m, type(m))
+return m
+$BODY$ LANGUAGE plpython3u;
+select pyreturnmultidemint8(5,5,3,2);
+INFO: ([[[[0, 1, 2, 3, 4], [0, 1, 2, 3, 4], [0, 1, 2, 3, 4], [0, 1, 2, 3, 4], [0, 1, 2, 3, 4]], [[0, 1, 2, 3, 4], [0, 1, 2, 3, 4], [0, 1, 2, 3, 4], [0, 1, 2, 3, 4], [0, 1, 2, 3, 4]], [[0, 1, 2, 3, 4], [0, 1, 2, 3, 4], [0, 1, 2, 3, 4], [0, 1, 2, 3, 4], [0, 1, 2, 3, 4]]], [[[0, 1, 2, 3, 4], [0, 1, 2, 3, 4], [0, 1, 2, 3, 4], [0, 1, 2, 3, 4], [0, 1, 2, 3, 4]], [[0, 1, 2, 3, 4], [0, 1, 2, 3, 4], [0, 1, 2, 3, 4], [0, 1, 2, 3, 4], [0, 1, 2, 3, 4]], [[0, 1, 2, 3, 4], [0, 1, 2, 3, 4], [0, 1, 2, 3, 4], [0, 1, 2, 3, 4], [0, 1, 2, 3, 4]]]], <class 'list'>)
+ pyreturnmultidemint8
+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
+ {{{{0,1,2,3,4},{0,1,2,3,4},{0,1,2,3,4},{0,1,2,3,4},{0,1,2,3,4}},{{0,1,2,3,4},{0,1,2,3,4},{0,1,2,3,4},{0,1,2,3,4},{0,1,2,3,4}},{{0,1,2,3,4},{0,1,2,3,4},{0,1,2,3,4},{0,1,2,3,4},{0,1,2,3,4}}},{{{0,1,2,3,4},{0,1,2,3,4},{0,1,2,3,4},{0,1,2,3,4},{0,1,2,3,4}},{{0,1,2,3,4},{0,1,2,3,4},{0,1,2,3,4},{0,1,2,3,4},{0,1,2,3,4}},{{0,1,2,3,4},{0,1,2,3,4},{0,1,2,3,4},{0,1,2,3,4},{0,1,2,3,4}}}}
+(1 row)
+
+CREATE OR REPLACE FUNCTION pyreturnmultidemfloat4(h int4, i int4, j int4, k int4 ) RETURNS float4[] AS $BODY$
+m = [[[[x for x in range(h)] for y in range(i)] for z in range(j)] for w in range(k)]
+plpy.info(m, type(m))
+return m
+$BODY$ LANGUAGE plpython3u;
+select pyreturnmultidemfloat4(6,5,3,2);
+INFO: ([[[[0, 1, 2, 3, 4, 5], [0, 1, 2, 3, 4, 5], [0, 1, 2, 3, 4, 5], [0, 1, 2, 3, 4, 5], [0, 1, 2, 3, 4, 5]], [[0, 1, 2, 3, 4, 5], [0, 1, 2, 3, 4, 5], [0, 1, 2, 3, 4, 5], [0, 1, 2, 3, 4, 5], [0, 1, 2, 3, 4, 5]], [[0, 1, 2, 3, 4, 5], [0, 1, 2, 3, 4, 5], [0, 1, 2, 3, 4, 5], [0, 1, 2, 3, 4, 5], [0, 1, 2, 3, 4, 5]]], [[[0, 1, 2, 3, 4, 5], [0, 1, 2, 3, 4, 5], [0, 1, 2, 3, 4, 5], [0, 1, 2, 3, 4, 5], [0, 1, 2, 3, 4, 5]], [[0, 1, 2, 3, 4, 5], [0, 1, 2, 3, 4, 5], [0, 1, 2, 3, 4, 5], [0, 1, 2, 3, 4, 5], [0, 1, 2, 3, 4, 5]], [[0, 1, 2, 3, 4, 5], [0, 1, 2, 3, 4, 5], [0, 1, 2, 3, 4, 5], [0, 1, 2, 3, 4, 5], [0, 1, 2, 3, 4, 5]]]], <class 'list'>)
+ pyreturnmultidemfloat4
+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
+ {{{{0,1,2,3,4,5},{0,1,2,3,4,5},{0,1,2,3,4,5},{0,1,2,3,4,5},{0,1,2,3,4,5}},{{0,1,2,3,4,5},{0,1,2,3,4,5},{0,1,2,3,4,5},{0,1,2,3,4,5},{0,1,2,3,4,5}},{{0,1,2,3,4,5},{0,1,2,3,4,5},{0,1,2,3,4,5},{0,1,2,3,4,5},{0,1,2,3,4,5}}},{{{0,1,2,3,4,5},{0,1,2,3,4,5},{0,1,2,3,4,5},{0,1,2,3,4,5},{0,1,2,3,4,5}},{{0,1,2,3,4,5},{0,1,2,3,4,5},{0,1,2,3,4,5},{0,1,2,3,4,5},{0,1,2,3,4,5}},{{0,1,2,3,4,5},{0,1,2,3,4,5},{0,1,2,3,4,5},{0,1,2,3,4,5},{0,1,2,3,4,5}}}}
+(1 row)
+
+CREATE OR REPLACE FUNCTION pyreturnmultidemfloat8(h int4, i int4, j int4, k int4 ) RETURNS float8[] AS $BODY$
+m = [[[[x for x in range(h)] for y in range(i)] for z in range(j)] for w in range(k)]
+plpy.info(m, type(m))
+return m
+$BODY$ LANGUAGE plpython3u;
+select pyreturnmultidemfloat8(7,5,3,2);
+INFO: ([[[[0, 1, 2, 3, 4, 5, 6], [0, 1, 2, 3, 4, 5, 6], [0, 1, 2, 3, 4, 5, 6], [0, 1, 2, 3, 4, 5, 6], [0, 1, 2, 3, 4, 5, 6]], [[0, 1, 2, 3, 4, 5, 6], [0, 1, 2, 3, 4, 5, 6], [0, 1, 2, 3, 4, 5, 6], [0, 1, 2, 3, 4, 5, 6], [0, 1, 2, 3, 4, 5, 6]], [[0, 1, 2, 3, 4, 5, 6], [0, 1, 2, 3, 4, 5, 6], [0, 1, 2, 3, 4, 5, 6], [0, 1, 2, 3, 4, 5, 6], [0, 1, 2, 3, 4, 5, 6]]], [[[0, 1, 2, 3, 4, 5, 6], [0, 1, 2, 3, 4, 5, 6], [0, 1, 2, 3, 4, 5, 6], [0, 1, 2, 3, 4, 5, 6], [0, 1, 2, 3, 4, 5, 6]], [[0, 1, 2, 3, 4, 5, 6], [0, 1, 2, 3, 4, 5, 6], [0, 1, 2, 3, 4, 5, 6], [0, 1, 2, 3, 4, 5, 6], [0, 1, 2, 3, 4, 5, 6]], [[0, 1, 2, 3, 4, 5, 6], [0, 1, 2, 3, 4, 5, 6], [0, 1, 2, 3, 4, 5, 6], [0, 1, 2, 3, 4, 5, 6], [0, 1, 2, 3, 4, 5, 6]]]], <class 'list'>)
+ pyreturnmultidemfloat8
+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
+ {{{{0,1,2,3,4,5,6},{0,1,2,3,4,5,6},{0,1,2,3,4,5,6},{0,1,2,3,4,5,6},{0,1,2,3,4,5,6}},{{0,1,2,3,4,5,6},{0,1,2,3,4,5,6},{0,1,2,3,4,5,6},{0,1,2,3,4,5,6},{0,1,2,3,4,5,6}},{{0,1,2,3,4,5,6},{0,1,2,3,4,5,6},{0,1,2,3,4,5,6},{0,1,2,3,4,5,6},{0,1,2,3,4,5,6}}},{{{0,1,2,3,4,5,6},{0,1,2,3,4,5,6},{0,1,2,3,4,5,6},{0,1,2,3,4,5,6},{0,1,2,3,4,5,6}},{{0,1,2,3,4,5,6},{0,1,2,3,4,5,6},{0,1,2,3,4,5,6},{0,1,2,3,4,5,6},{0,1,2,3,4,5,6}},{{0,1,2,3,4,5,6},{0,1,2,3,4,5,6},{0,1,2,3,4,5,6},{0,1,2,3,4,5,6},{0,1,2,3,4,5,6}}}}
+(1 row)
+
CREATE FUNCTION test_type_conversion_array_text(x text[]) RETURNS text[] AS $$
plpy.info(x, type(x))
return x
@@ -551,6 +668,13 @@ INFO: (['foo', 'bar'], <class 'list'>)
{foo,bar}
(1 row)
+SELECT * FROM test_type_conversion_array_text(ARRAY[['foo', 'bar'],['foo2', 'bar2']]);
+INFO: ([['foo', 'bar'], ['foo2', 'bar2']], <class 'list'>)
+ test_type_conversion_array_text
+---------------------------------
+ {{foo,bar},{foo2,bar2}}
+(1 row)
+
CREATE FUNCTION test_type_conversion_array_bytea(x bytea[]) RETURNS bytea[] AS $$
plpy.info(x, type(x))
return x
@@ -578,6 +702,13 @@ SELECT * FROM test_type_conversion_array_mixed2();
ERROR: invalid input syntax for integer: "abc"
CONTEXT: while creating return value
PL/Python function "test_type_conversion_array_mixed2"
+CREATE FUNCTION test_type_conversion_mdarray_malformed() RETURNS int[] AS $$
+return [[1,2,3],[4,5]]
+$$ LANGUAGE plpython3u;
+SELECT * FROM test_type_conversion_mdarray_malformed();
+ERROR: Multidimensional arrays must have array expressions with matching dimensions. PL/Python function return value has sequence length 2 while expected 3
+CONTEXT: while creating return value
+PL/Python function "test_type_conversion_mdarray_malformed"
CREATE FUNCTION test_type_conversion_array_record() RETURNS type_record[] AS $$
return [{'first': 'one', 'second': 42}, {'first': 'two', 'second': 11}]
$$ LANGUAGE plpython3u;
diff --git a/src/pl/plpython/plpy_typeio.c b/src/pl/plpython/plpy_typeio.c
index 70f2e6d..9436412 100644
--- a/src/pl/plpython/plpy_typeio.c
+++ b/src/pl/plpython/plpy_typeio.c
@@ -631,46 +631,111 @@ PLyList_FromArray(PLyDatumToOb *arg, Datum d)
{
ArrayType *array = DatumGetArrayTypeP(d);
PLyDatumToOb *elm = arg->elm;
- PyObject *list;
- int length;
- int lbound;
+ int ndim;
+ int *dims;
+ int *lb;
+ char *dataptr;
+ bits8 *bitmap;
+ int bitmask;
int i;
+ int dim;
+ int indx[MAXDIM];
+ PyObject *lists[MAXDIM];
if (ARR_NDIM(array) == 0)
return PyList_New(0);
- if (ARR_NDIM(array) != 1)
- ereport(ERROR,
- (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
- errmsg("cannot convert multidimensional array to Python list"),
- errdetail("PL/Python only supports one-dimensional arrays.")));
+ /* Array dimensions and left bounds */
+ ndim = ARR_NDIM(array);
+ dims = ARR_DIMS(array);
+ lb = ARR_LBOUND(array);
- length = ARR_DIMS(array)[0];
- lbound = ARR_LBOUND(array)[0];
- list = PyList_New(length);
- if (list == NULL)
- PLy_elog(ERROR, "could not create new Python list");
+ /* Internal array representation pointers */
+ dataptr = ARR_DATA_PTR(array);
+ bitmap = ARR_NULLBITMAP(array);
+ bitmask = 1;
- for (i = 0; i < length; i++)
+ /* Iterators initialization */
+ for (i = 0; i < ndim; i++) {
+ indx[i] = lb[i];
+ lists[i] = NULL;
+ }
+ lists[0] = PyList_New(dims[0]);
+
+ /* We need this incref to keep the pointer valid after the array traversal
+ * terminates, as this traversal does DECREF for all the lists in array */
+ Py_INCREF(lists[0]);
+
+ /* In this cycle we are going over array dimensions. Postgres offers you an
+ * option to iterate over all the multi-dimensional array elemens in order.
+ * For 3-dimesnional array the order of iteration would be following - first
+ * you start with [0,0,0] elements through [0,0,k], then [0,1,0] till [0,1,k]
+ * till [0,m,k], then [1,0,0] till [1,0,k] till [1,m,k], and so on.
+ * In Python, each 1-d array is a separate list object, so 3-d array of
+ * [n,m,k] element is a list of n m-element arrays, each element of which is
+ * k-element array. In this cycle we traverse from outter dimensions to
+ * inner ones, creating nested Python lists during traversal */
+ dim = 0;
+ while (dim >= 0)
{
- Datum elem;
- bool isnull;
- int offset;
-
- offset = lbound + i;
- elem = array_ref(array, 1, &offset, arg->typlen,
- elm->typlen, elm->typbyval, elm->typalign,
- &isnull);
- if (isnull)
+ /* If we finished up iterating over current dimension - go one level up */
+ if (indx[dim] > dims[dim])
{
- Py_INCREF(Py_None);
- PyList_SET_ITEM(list, i, Py_None);
+ Py_DECREF(lists[dim]);
+ indx[dim] = 0;
+ dim -= 1;
+ }
+ /* If we are processing inner dimension - create one more list */
+ else if (dim < ndim - 1)
+ {
+ lists[dim+1] = PyList_New(dims[dim+1]);
+
+ /* We need this INCREF as we keep array pointer on our side,
+ * while PyList_SET_ITEM steals the reference */
+ Py_INCREF(lists[dim+1]);
+
+ PyList_SET_ITEM(lists[dim], indx[dim] - lb[dim], lists[dim+1]);
+ indx[dim] += 1;
+ dim += 1;
+ indx[dim] = lb[dim];
+ }
+ /* If we are iterating over the outter dimension, fill the list with
+ * values from the original Postgres array */
+ else if (dim == ndim - 1)
+ {
+ for (indx[dim] = lb[dim]; indx[dim] <= dims[dim]; indx[dim]++)
+ {
+ /* checking for NULL */
+ if (bitmap && (*bitmap & bitmask) == 0)
+ {
+ Py_INCREF(Py_None);
+ PyList_SET_ITEM(lists[dim], indx[dim] - lb[dim], Py_None);
+ }
+ else
+ {
+ Datum itemvalue;
+
+ itemvalue = fetch_att(dataptr, elm->typbyval, elm->typlen);
+ PyList_SET_ITEM(lists[dim], indx[dim] - lb[dim], elm->func(elm, itemvalue));
+ dataptr = att_addlength_pointer(dataptr, elm->typlen, dataptr);
+ dataptr = (char *) att_align_nominal(dataptr, elm->typalign);
+ }
+
+ /* advance bitmap pointer if any */
+ if (bitmap)
+ {
+ bitmask <<= 1;
+ if (bitmask == 0x100 /* (1<<8) */)
+ {
+ bitmap++;
+ bitmask = 1;
+ }
+ }
+ }
}
- else
- PyList_SET_ITEM(list, i, elm->func(elm, elem));
}
- return list;
+ return lists[0];
}
/*
@@ -864,39 +929,131 @@ static Datum
PLySequence_ToArray(PLyObToDatum *arg, int32 typmod, PyObject *plrv)
{
ArrayType *array;
- Datum rv;
int i;
Datum *elems;
bool *nulls;
int len;
- int lbs;
+ int ndim;
+ int dims[MAXDIM];
+ int lbs[MAXDIM];
+ int indx[MAXDIM];
+ PyObject *stack[MAXDIM];
+ int dim;
+ int idxelem;
+ Datum rv;
Assert(plrv != Py_None);
if (!PySequence_Check(plrv))
PLy_elog(ERROR, "return value of function with array return type is not a Python sequence");
- len = PySequence_Length(plrv);
+ ndim = 1;
+ dims[0] = PySequence_Length(plrv);
+ len = dims[0];
+ stack[0] = plrv;
+
+ /* We don't want to create multi-dimensional arrays when we have an empty sequence,
+ * when we need to parse sequence of composite objects, and when we have strings */
+ if (len > 0 && !type_is_rowtype(get_base_element_type(arg->typoid)) &&
+ !PyString_Check(plrv) && !PyBytes_Check(plrv) && !PyUnicode_Check(plrv))
+ {
+ PyObject *pyptr = PySequence_GetItem(plrv, 0);
+
+ /* We want to iterate through all iterable objects except by strings on nested levels */
+ while (pyptr != NULL && PySequence_Check(pyptr) &&
+ !(PyString_Check(pyptr) || PyBytes_Check(pyptr) || PyUnicode_Check(pyptr)))
+ {
+ dims[ndim] = PySequence_Length(pyptr);
+ if (dims[ndim] < 0)
+ PLy_elog(ERROR, "Cannot determine sequence length for function return value");
+ len *= dims[ndim];
+ stack[ndim] = pyptr;
+ ndim += 1;
+ if (dims[ndim - 1] == 0) {
+ pyptr = NULL;
+ break;
+ }
+ pyptr = PySequence_GetItem(pyptr, 0);
+ }
+
+ /* Pyptr points to element of n-dimensional array, we don't need its reference */
+ Py_XDECREF(pyptr);
+ }
+
+ /* We need this incref to keep the pointer valid after the array traversal
+ * terminates, as this traversal does DECREF for all the lists in array, and
+ * stack[0] corresponds to function return value */
+ Py_INCREF(stack[0]);
+
elems = palloc(sizeof(*elems) * len);
nulls = palloc(sizeof(*nulls) * len);
- for (i = 0; i < len; i++)
- {
- PyObject *obj = PySequence_GetItem(plrv, i);
+ for (i = 0; i < ndim; i++) {
+ indx[i] = 0;
+ lbs[i] = 1;
+ }
- if (obj == Py_None)
- nulls[i] = true;
- else
+ /* In this cycle we are going over nested Python lists, fetching elements
+ * from the deepest level and putting them into a linear array for Postgres
+ * to interpret them as n-dimensional array. This is a cycle implementation
+ * of DFS (recursive traversal of nested arrays here), keeping the stack in
+ * "stack" variable */
+ dim = 0;
+ idxelem = 0;
+ while (dim >= 0)
+ {
+ /* If we finished up iterating over current list - go one level up */
+ if (indx[dim] == dims[dim])
+ {
+ Py_DECREF(stack[dim]);
+ indx[dim] = 0;
+ dim -= 1;
+ }
+ /* If we are processing inner list - create one more list */
+ else if (dim < ndim - 1)
{
- nulls[i] = false;
- elems[i] = arg->elm->func(arg->elm, -1, obj);
+ stack[dim+1] = PySequence_GetItem(stack[dim], indx[dim]);
+ if (PySequence_Length(stack[dim+1]) != dims[dim+1])
+ PLy_elog(ERROR, "Multidimensional arrays must have array expressions with matching dimensions. "
+ "PL/Python function return value has sequence length %d while expected %d",
+ (int)PySequence_Length(stack[dim+1]), dims[dim+1]);
+ indx[dim] += 1;
+ dim += 1;
+ }
+ /* If we are iterating over the outter list, fill the output array */
+ else if (dim == ndim - 1)
+ {
+ for (indx[dim] = 0; indx[dim] < dims[dim]; indx[dim]++)
+ {
+ PyObject *obj = PySequence_GetItem(stack[dim], indx[dim]);
+
+ if (obj == Py_None)
+ nulls[idxelem] = true;
+ else
+ {
+ nulls[idxelem] = false;
+
+ /*
+ * We don't support arrays of row types yet, so the first argument
+ * can be NULL.
+ */
+ elems[idxelem] = arg->elm->func(arg->elm, -1, obj);
+ }
+ Py_XDECREF(obj);
+ idxelem += 1;
+ }
}
- Py_XDECREF(obj);
}
- lbs = 1;
- array = construct_md_array(elems, nulls, 1, &len, &lbs,
- get_base_element_type(arg->typoid), arg->elm->typlen, arg->elm->typbyval, arg->elm->typalign);
+ array = construct_md_array(elems,
+ nulls,
+ ndim,
+ dims,
+ lbs,
+ get_base_element_type(arg->typoid),
+ arg->elm->typlen,
+ arg->elm->typbyval,
+ arg->elm->typalign);
/*
* If the result type is a domain of array, the resulting array must be
diff --git a/src/pl/plpython/sql/plpython_types.sql b/src/pl/plpython/sql/plpython_types.sql
index 19d920d..b3599bc 100644
--- a/src/pl/plpython/sql/plpython_types.sql
+++ b/src/pl/plpython/sql/plpython_types.sql
@@ -237,7 +237,79 @@ SELECT * FROM test_type_conversion_array_int4(ARRAY[NULL,1]);
SELECT * FROM test_type_conversion_array_int4(ARRAY[]::integer[]);
SELECT * FROM test_type_conversion_array_int4(NULL);
SELECT * FROM test_type_conversion_array_int4(ARRAY[[1,2,3],[4,5,6]]);
+SELECT * FROM test_type_conversion_array_int4(ARRAY[[[1,2,NULL],[NULL,5,6]],[[NULL,8,9],[10,11,12]]]);
+CREATE FUNCTION test_type_conversion_array_int8(x int8[]) RETURNS int8[] AS $$
+plpy.info(x, type(x))
+return x
+$$ LANGUAGE plpythonu;
+
+SELECT * FROM test_type_conversion_array_int8(ARRAY[[[1,2,NULL],[NULL,5,6]],[[NULL,8,9],[10,11,12]]]::int8[]);
+
+CREATE FUNCTION test_type_conversion_array_float4(x float4[]) RETURNS float4[] AS $$
+plpy.info(x, type(x))
+return x
+$$ LANGUAGE plpythonu;
+
+SELECT * FROM test_type_conversion_array_float4(ARRAY[[[1.2,2.3,NULL],[NULL,5.7,6.8]],[[NULL,8.9,9.345],[10.123,11.456,12.6768]]]::float4[]);
+
+CREATE FUNCTION test_type_conversion_array_float8(x float8[]) RETURNS float8[] AS $$
+plpy.info(x, type(x))
+return x
+$$ LANGUAGE plpythonu;
+
+SELECT * FROM test_type_conversion_array_float8(ARRAY[[[1.2,2.3,NULL],[NULL,5.7,6.8]],[[NULL,8.9,9.345],[10.123,11.456,12.6768]]]::float8[]);
+
+CREATE FUNCTION test_type_conversion_array_date(x date[]) RETURNS date[] AS $$
+plpy.info(x, type(x))
+return x
+$$ LANGUAGE plpythonu;
+
+SELECT * FROM test_type_conversion_array_date(ARRAY[[['2016-09-21','2016-09-22',NULL],[NULL,'2016-10-21','2016-10-22']],
+ [[NULL,'2016-11-21','2016-10-21'],['2015-09-21','2015-09-22','2014-09-21']]]::date[]);
+
+CREATE FUNCTION test_type_conversion_array_timestamp(x timestamp[]) RETURNS timestamp[] AS $$
+plpy.info(x, type(x))
+return x
+$$ LANGUAGE plpythonu;
+
+SELECT * FROM test_type_conversion_array_timestamp(ARRAY[[['2016-09-21 15:34:24.078792-04','2016-10-22 11:34:24.078795-04',NULL],
+ [NULL,'2016-10-21 11:34:25.078792-04','2016-10-21 11:34:24.098792-04']],
+ [[NULL,'2016-01-21 11:34:24.078792-04','2016-11-21 11:34:24.108792-04'],
+ ['2015-09-21 11:34:24.079792-04','2014-09-21 11:34:24.078792-04','2013-09-21 11:34:24.078792-04']]]::timestamp[]);
+
+
+CREATE OR REPLACE FUNCTION pyreturnmultidemint4(h int4, i int4, j int4, k int4 ) RETURNS int4[] AS $BODY$
+m = [[[[x for x in range(h)] for y in range(i)] for z in range(j)] for w in range(k)]
+plpy.info(m, type(m))
+return m
+$BODY$ LANGUAGE plpythonu;
+
+select pyreturnmultidemint4(8,5,3,2);
+
+CREATE OR REPLACE FUNCTION pyreturnmultidemint8(h int4, i int4, j int4, k int4 ) RETURNS int8[] AS $BODY$
+m = [[[[x for x in range(h)] for y in range(i)] for z in range(j)] for w in range(k)]
+plpy.info(m, type(m))
+return m
+$BODY$ LANGUAGE plpythonu;
+
+select pyreturnmultidemint8(5,5,3,2);
+
+CREATE OR REPLACE FUNCTION pyreturnmultidemfloat4(h int4, i int4, j int4, k int4 ) RETURNS float4[] AS $BODY$
+m = [[[[x for x in range(h)] for y in range(i)] for z in range(j)] for w in range(k)]
+plpy.info(m, type(m))
+return m
+$BODY$ LANGUAGE plpythonu;
+
+select pyreturnmultidemfloat4(6,5,3,2);
+
+CREATE OR REPLACE FUNCTION pyreturnmultidemfloat8(h int4, i int4, j int4, k int4 ) RETURNS float8[] AS $BODY$
+m = [[[[x for x in range(h)] for y in range(i)] for z in range(j)] for w in range(k)]
+plpy.info(m, type(m))
+return m
+$BODY$ LANGUAGE plpythonu;
+
+select pyreturnmultidemfloat8(7,5,3,2);
CREATE FUNCTION test_type_conversion_array_text(x text[]) RETURNS text[] AS $$
plpy.info(x, type(x))
@@ -245,6 +317,7 @@ return x
$$ LANGUAGE plpythonu;
SELECT * FROM test_type_conversion_array_text(ARRAY['foo', 'bar']);
+SELECT * FROM test_type_conversion_array_text(ARRAY[['foo', 'bar'],['foo2', 'bar2']]);
CREATE FUNCTION test_type_conversion_array_bytea(x bytea[]) RETURNS bytea[] AS $$
@@ -268,6 +341,11 @@ $$ LANGUAGE plpythonu;
SELECT * FROM test_type_conversion_array_mixed2();
+CREATE FUNCTION test_type_conversion_mdarray_malformed() RETURNS int[] AS $$
+return [[1,2,3],[4,5]]
+$$ LANGUAGE plpythonu;
+
+SELECT * FROM test_type_conversion_mdarray_malformed();
CREATE FUNCTION test_type_conversion_array_record() RETURNS type_record[] AS $$
return [{'first': 'one', 'second': 42}, {'first': 'two', 'second': 11}]
Re: PL/Python adding support for multi-dimensional arrays
On 09/22/2016 10:28 AM, Pavel Stehule wrote:
Now, the tests are enough - so I'll mark this patch as ready for commiters.
I had to fix tests - there was lot of white spaces, and the result for
python3 was missing
Thanks Pavel!
This crashes with arrays with non-default lower bounds:
postgres=# SELECT * FROM test_type_conversion_array_int4('[2:4]={1,2,3}');
INFO: ([1, 2, <NULL>], <type 'list'>)
server closed the connection unexpectedly
This probably means the server terminated abnormally
before or while processing the request.
I'd like to see some updates to the docs for this. The manual doesn't
currently say anything about multi-dimensional arrays in pl/python, but
it should've mentioned that they're not supported. Now that it is
supported, should mention that, and explain briefly that a
multi-dimensional array is mapped to a python list of lists.
It seems we don't have any mention in the docs about arrays with
non-default lower-bounds ATM. That's not this patch's fault, but it
would be good to point out that the lower bounds are discarded when an
array is passed to python.
I find the loop in PLyList_FromArray() quite difficult to understand.
Are the comments there mixing up the "inner" and "outer" dimensions? I
wonder if that would be easier to read, if it was written in a
recursive-style, rather than iterative with stacks for the dimensions.
On 08/03/2016 02:49 PM, Alexey Grishchenko wrote:
This patch does not support multi-dimensional arrays of composite types, as
composite types in Python might be represented as iterators and there is no
obvious way to find out when the nested array stops and composite type
structure starts. For example, if we have a composite type of (int, text),
we can try to return "[ [ [1,'a'], [2,'b'] ], [ [3,'c'], [4,'d'] ] ]", and
it is hard to find out that the first two lists are lists, and the third
one represents structure. Things are getting even more complex when you
have arrays as members of composite type. This is why I think this
limitation is reasonable.
How do we handle single-dimensional arrays of composite types at the
moment? At a quick glance, it seems that the composite types are just
treated like strings, when they're in an array. That's probably OK, but
it means that there's nothing special about composite types in
multi-dimensional arrays. In any case, we should mention that in the docs.
- Heikki
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
Re: PL/Python adding support for multi-dimensional arrays
On 9/23/16 2:42 AM, Heikki Linnakangas wrote:
How do we handle single-dimensional arrays of composite types at the
moment? At a quick glance, it seems that the composite types are just
treated like strings, when they're in an array. That's probably OK, but
it means that there's nothing special about composite types in
multi-dimensional arrays. In any case, we should mention that in the docs.
That is how they're handled, but I'd really like to change that. I've
held off because I don't know how to handle the backwards
incompatibility that would introduce. (I've been wondering if we might
add a facility to allow specifying default TRANSFORMs that should be
used for specific data types in specific languages.)
The converse case (a composite with arrays) suffers the same problem
(array is just treated as a string).
--
Jim Nasby, Data Architect, Blue Treble Consulting, Austin TX
Experts in Analytics, Data Architecture and PostgreSQL
Data in Trouble? Get it in Treble! http://BlueTreble.com
855-TREBLE2 (855-873-2532) mobile: 512-569-9461
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
Re: PL/Python adding support for multi-dimensional arrays
This crashes with arrays with non-default lower bounds:
postgres=# SELECT * FROM test_type_conversion_array_int4('[2:4]={1,2,3}');
INFO: ([1, 2, <NULL>], <type 'list'>)
server closed the connection unexpectedly
This probably means the server terminated abnormally
before or while processing the request.Attached patch fixes this bug, and adds a test for it.
I'd like to see some updates to the docs for this. The manual doesn't
currently say anything about multi-dimensional arrays in pl/python, but it
should've mentioned that they're not supported. Now that it is supported,
should mention that, and explain briefly that a multi-dimensional array is
mapped to a python list of lists.If the code passes I'll fix the docs
It seems we don't have any mention in the docs about arrays with
non-default lower-bounds ATM. That's not this patch's fault, but it would
be good to point out that the lower bounds are discarded when an array is
passed to python.I find the loop in PLyList_FromArray() quite difficult to understand. Are
the comments there mixing up the "inner" and "outer" dimensions? I wonder
if that would be easier to read, if it was written in a recursive-style,
rather than iterative with stacks for the dimensions.Yes, it is fairly convoluted.
Dave Cramer
davec@postgresintl.com
www.postgresintl.com
Re: PL/Python adding support for multi-dimensional arrays
On 26 September 2016 at 14:52, Dave Cramer <pg@fastcrypt.com> wrote:
Show quoted text
This crashes with arrays with non-default lower bounds:
postgres=# SELECT * FROM test_type_conversion_array_int
4('[2:4]={1,2,3}');
INFO: ([1, 2, <NULL>], <type 'list'>)
server closed the connection unexpectedly
This probably means the server terminated abnormally
before or while processing the request.Attached patch fixes this bug, and adds a test for it.
I'd like to see some updates to the docs for this. The manual doesn't
currently say anything about multi-dimensional arrays in pl/python, but it
should've mentioned that they're not supported. Now that it is supported,
should mention that, and explain briefly that a multi-dimensional array is
mapped to a python list of lists.If the code passes I'll fix the docs
It seems we don't have any mention in the docs about arrays with
non-default lower-bounds ATM. That's not this patch's fault, but it would
be good to point out that the lower bounds are discarded when an array is
passed to python.I find the loop in PLyList_FromArray() quite difficult to understand. Are
the comments there mixing up the "inner" and "outer" dimensions? I wonder
if that would be easier to read, if it was written in a recursive-style,
rather than iterative with stacks for the dimensions.Yes, it is fairly convoluted.
Dave Cramer
davec@postgresintl.com
www.postgresintl.com
Attachments:
PL-Python-adding-support-for-multi-dimensional-arrays-20160926.patchapplication/octet-stream; name=PL-Python-adding-support-for-multi-dimensional-arrays-20160926.patchDownload
diff --git a/src/pl/plpython/expected/plpython_types.out b/src/pl/plpython/expected/plpython_types.out
index f0b6abd..dc6e36c 100644
--- a/src/pl/plpython/expected/plpython_types.out
+++ b/src/pl/plpython/expected/plpython_types.out
@@ -537,9 +537,133 @@ INFO: (None, <type 'NoneType'>)
(1 row)
SELECT * FROM test_type_conversion_array_int4(ARRAY[[1,2,3],[4,5,6]]);
-ERROR: cannot convert multidimensional array to Python list
-DETAIL: PL/Python only supports one-dimensional arrays.
-CONTEXT: PL/Python function "test_type_conversion_array_int4"
+INFO: ([[1, 2, 3], [4, 5, 6]], <type 'list'>)
+ test_type_conversion_array_int4
+---------------------------------
+ {{1,2,3},{4,5,6}}
+(1 row)
+
+SELECT * FROM test_type_conversion_array_int4(ARRAY[[[1,2,NULL],[NULL,5,6]],[[NULL,8,9],[10,11,12]]]);
+INFO: ([[[1, 2, None], [None, 5, 6]], [[None, 8, 9], [10, 11, 12]]], <type 'list'>)
+ test_type_conversion_array_int4
+---------------------------------------------------
+ {{{1,2,NULL},{NULL,5,6}},{{NULL,8,9},{10,11,12}}}
+(1 row)
+
+SELECT * FROM test_type_conversion_array_int4('[2:4]={1,2,3}');
+INFO: ([1, 2, 3], <type 'list'>)
+ test_type_conversion_array_int4
+---------------------------------
+ {1,2,3}
+(1 row)
+
+CREATE FUNCTION test_type_conversion_array_int8(x int8[]) RETURNS int8[] AS $$
+plpy.info(x, type(x))
+return x
+$$ LANGUAGE plpythonu;
+SELECT * FROM test_type_conversion_array_int8(ARRAY[[[1,2,NULL],[NULL,5,6]],[[NULL,8,9],[10,11,12]]]::int8[]);
+INFO: ([[[1L, 2L, None], [None, 5L, 6L]], [[None, 8L, 9L], [10L, 11L, 12L]]], <type 'list'>)
+ test_type_conversion_array_int8
+---------------------------------------------------
+ {{{1,2,NULL},{NULL,5,6}},{{NULL,8,9},{10,11,12}}}
+(1 row)
+
+CREATE FUNCTION test_type_conversion_array_float4(x float4[]) RETURNS float4[] AS $$
+plpy.info(x, type(x))
+return x
+$$ LANGUAGE plpythonu;
+SELECT * FROM test_type_conversion_array_float4(ARRAY[[[1.2,2.3,NULL],[NULL,5.7,6.8]],[[NULL,8.9,9.345],[10.123,11.456,12.6768]]]::float4[]);
+INFO: ([[[1.2000000476837158, 2.299999952316284, None], [None, 5.699999809265137, 6.800000190734863]], [[None, 8.899999618530273, 9.345000267028809], [10.123000144958496, 11.456000328063965, 12.676799774169922]]], <type 'list'>)
+ test_type_conversion_array_float4
+------------------------------------------------------------------------------
+ {{{1.2,2.3,NULL},{NULL,5.7,6.8}},{{NULL,8.9,9.345},{10.123,11.456,12.6768}}}
+(1 row)
+
+CREATE FUNCTION test_type_conversion_array_float8(x float8[]) RETURNS float8[] AS $$
+plpy.info(x, type(x))
+return x
+$$ LANGUAGE plpythonu;
+SELECT * FROM test_type_conversion_array_float8(ARRAY[[[1.2,2.3,NULL],[NULL,5.7,6.8]],[[NULL,8.9,9.345],[10.123,11.456,12.6768]]]::float8[]);
+INFO: ([[[1.2, 2.3, None], [None, 5.7, 6.8]], [[None, 8.9, 9.345], [10.123, 11.456, 12.6768]]], <type 'list'>)
+ test_type_conversion_array_float8
+------------------------------------------------------------------------------
+ {{{1.2,2.3,NULL},{NULL,5.7,6.8}},{{NULL,8.9,9.345},{10.123,11.456,12.6768}}}
+(1 row)
+
+CREATE FUNCTION test_type_conversion_array_date(x date[]) RETURNS date[] AS $$
+plpy.info(x, type(x))
+return x
+$$ LANGUAGE plpythonu;
+SELECT * FROM test_type_conversion_array_date(ARRAY[[['2016-09-21','2016-09-22',NULL],[NULL,'2016-10-21','2016-10-22']],
+ [[NULL,'2016-11-21','2016-10-21'],['2015-09-21','2015-09-22','2014-09-21']]]::date[]);
+INFO: ([[['09-21-2016', '09-22-2016', None], [None, '10-21-2016', '10-22-2016']], [[None, '11-21-2016', '10-21-2016'], ['09-21-2015', '09-22-2015', '09-21-2014']]], <type 'list'>)
+ test_type_conversion_array_date
+---------------------------------------------------------------------------------------------------------------------------------
+ {{{09-21-2016,09-22-2016,NULL},{NULL,10-21-2016,10-22-2016}},{{NULL,11-21-2016,10-21-2016},{09-21-2015,09-22-2015,09-21-2014}}}
+(1 row)
+
+CREATE FUNCTION test_type_conversion_array_timestamp(x timestamp[]) RETURNS timestamp[] AS $$
+plpy.info(x, type(x))
+return x
+$$ LANGUAGE plpythonu;
+SELECT * FROM test_type_conversion_array_timestamp(ARRAY[[['2016-09-21 15:34:24.078792-04','2016-10-22 11:34:24.078795-04',NULL],
+ [NULL,'2016-10-21 11:34:25.078792-04','2016-10-21 11:34:24.098792-04']],
+ [[NULL,'2016-01-21 11:34:24.078792-04','2016-11-21 11:34:24.108792-04'],
+ ['2015-09-21 11:34:24.079792-04','2014-09-21 11:34:24.078792-04','2013-09-21 11:34:24.078792-04']]]::timestamp[]);
+INFO: ([[['Wed Sep 21 15:34:24.078792 2016', 'Sat Oct 22 11:34:24.078795 2016', None], [None, 'Fri Oct 21 11:34:25.078792 2016', 'Fri Oct 21 11:34:24.098792 2016']], [[None, 'Thu Jan 21 11:34:24.078792 2016', 'Mon Nov 21 11:34:24.108792 2016'], ['Mon Sep 21 11:34:24.079792 2015', 'Sun Sep 21 11:34:24.078792 2014', 'Sat Sep 21 11:34:24.078792 2013']]], <type 'list'>)
+ test_type_conversion_array_timestamp
+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
+ {{{"Wed Sep 21 15:34:24.078792 2016","Sat Oct 22 11:34:24.078795 2016",NULL},{NULL,"Fri Oct 21 11:34:25.078792 2016","Fri Oct 21 11:34:24.098792 2016"}},{{NULL,"Thu Jan 21 11:34:24.078792 2016","Mon Nov 21 11:34:24.108792 2016"},{"Mon Sep 21 11:34:24.079792 2015","Sun Sep 21 11:34:24.078792 2014","Sat Sep 21 11:34:24.078792 2013"}}}
+(1 row)
+
+CREATE OR REPLACE FUNCTION pyreturnmultidemint4(h int4, i int4, j int4, k int4 ) RETURNS int4[] AS $BODY$
+m = [[[[x for x in range(h)] for y in range(i)] for z in range(j)] for w in range(k)]
+plpy.info(m, type(m))
+return m
+$BODY$ LANGUAGE plpythonu;
+select pyreturnmultidemint4(8,5,3,2);
+INFO: ([[[[0, 1, 2, 3, 4, 5, 6, 7], [0, 1, 2, 3, 4, 5, 6, 7], [0, 1, 2, 3, 4, 5, 6, 7], [0, 1, 2, 3, 4, 5, 6, 7], [0, 1, 2, 3, 4, 5, 6, 7]], [[0, 1, 2, 3, 4, 5, 6, 7], [0, 1, 2, 3, 4, 5, 6, 7], [0, 1, 2, 3, 4, 5, 6, 7], [0, 1, 2, 3, 4, 5, 6, 7], [0, 1, 2, 3, 4, 5, 6, 7]], [[0, 1, 2, 3, 4, 5, 6, 7], [0, 1, 2, 3, 4, 5, 6, 7], [0, 1, 2, 3, 4, 5, 6, 7], [0, 1, 2, 3, 4, 5, 6, 7], [0, 1, 2, 3, 4, 5, 6, 7]]], [[[0, 1, 2, 3, 4, 5, 6, 7], [0, 1, 2, 3, 4, 5, 6, 7], [0, 1, 2, 3, 4, 5, 6, 7], [0, 1, 2, 3, 4, 5, 6, 7], [0, 1, 2, 3, 4, 5, 6, 7]], [[0, 1, 2, 3, 4, 5, 6, 7], [0, 1, 2, 3, 4, 5, 6, 7], [0, 1, 2, 3, 4, 5, 6, 7], [0, 1, 2, 3, 4, 5, 6, 7], [0, 1, 2, 3, 4, 5, 6, 7]], [[0, 1, 2, 3, 4, 5, 6, 7], [0, 1, 2, 3, 4, 5, 6, 7], [0, 1, 2, 3, 4, 5, 6, 7], [0, 1, 2, 3, 4, 5, 6, 7], [0, 1, 2, 3, 4, 5, 6, 7]]]], <type 'list'>)
+ pyreturnmultidemint4
+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
+ {{{{0,1,2,3,4,5,6,7},{0,1,2,3,4,5,6,7},{0,1,2,3,4,5,6,7},{0,1,2,3,4,5,6,7},{0,1,2,3,4,5,6,7}},{{0,1,2,3,4,5,6,7},{0,1,2,3,4,5,6,7},{0,1,2,3,4,5,6,7},{0,1,2,3,4,5,6,7},{0,1,2,3,4,5,6,7}},{{0,1,2,3,4,5,6,7},{0,1,2,3,4,5,6,7},{0,1,2,3,4,5,6,7},{0,1,2,3,4,5,6,7},{0,1,2,3,4,5,6,7}}},{{{0,1,2,3,4,5,6,7},{0,1,2,3,4,5,6,7},{0,1,2,3,4,5,6,7},{0,1,2,3,4,5,6,7},{0,1,2,3,4,5,6,7}},{{0,1,2,3,4,5,6,7},{0,1,2,3,4,5,6,7},{0,1,2,3,4,5,6,7},{0,1,2,3,4,5,6,7},{0,1,2,3,4,5,6,7}},{{0,1,2,3,4,5,6,7},{0,1,2,3,4,5,6,7},{0,1,2,3,4,5,6,7},{0,1,2,3,4,5,6,7},{0,1,2,3,4,5,6,7}}}}
+(1 row)
+
+CREATE OR REPLACE FUNCTION pyreturnmultidemint8(h int4, i int4, j int4, k int4 ) RETURNS int8[] AS $BODY$
+m = [[[[x for x in range(h)] for y in range(i)] for z in range(j)] for w in range(k)]
+plpy.info(m, type(m))
+return m
+$BODY$ LANGUAGE plpythonu;
+select pyreturnmultidemint8(5,5,3,2);
+INFO: ([[[[0, 1, 2, 3, 4], [0, 1, 2, 3, 4], [0, 1, 2, 3, 4], [0, 1, 2, 3, 4], [0, 1, 2, 3, 4]], [[0, 1, 2, 3, 4], [0, 1, 2, 3, 4], [0, 1, 2, 3, 4], [0, 1, 2, 3, 4], [0, 1, 2, 3, 4]], [[0, 1, 2, 3, 4], [0, 1, 2, 3, 4], [0, 1, 2, 3, 4], [0, 1, 2, 3, 4], [0, 1, 2, 3, 4]]], [[[0, 1, 2, 3, 4], [0, 1, 2, 3, 4], [0, 1, 2, 3, 4], [0, 1, 2, 3, 4], [0, 1, 2, 3, 4]], [[0, 1, 2, 3, 4], [0, 1, 2, 3, 4], [0, 1, 2, 3, 4], [0, 1, 2, 3, 4], [0, 1, 2, 3, 4]], [[0, 1, 2, 3, 4], [0, 1, 2, 3, 4], [0, 1, 2, 3, 4], [0, 1, 2, 3, 4], [0, 1, 2, 3, 4]]]], <type 'list'>)
+ pyreturnmultidemint8
+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
+ {{{{0,1,2,3,4},{0,1,2,3,4},{0,1,2,3,4},{0,1,2,3,4},{0,1,2,3,4}},{{0,1,2,3,4},{0,1,2,3,4},{0,1,2,3,4},{0,1,2,3,4},{0,1,2,3,4}},{{0,1,2,3,4},{0,1,2,3,4},{0,1,2,3,4},{0,1,2,3,4},{0,1,2,3,4}}},{{{0,1,2,3,4},{0,1,2,3,4},{0,1,2,3,4},{0,1,2,3,4},{0,1,2,3,4}},{{0,1,2,3,4},{0,1,2,3,4},{0,1,2,3,4},{0,1,2,3,4},{0,1,2,3,4}},{{0,1,2,3,4},{0,1,2,3,4},{0,1,2,3,4},{0,1,2,3,4},{0,1,2,3,4}}}}
+(1 row)
+
+CREATE OR REPLACE FUNCTION pyreturnmultidemfloat4(h int4, i int4, j int4, k int4 ) RETURNS float4[] AS $BODY$
+m = [[[[x for x in range(h)] for y in range(i)] for z in range(j)] for w in range(k)]
+plpy.info(m, type(m))
+return m
+$BODY$ LANGUAGE plpythonu;
+select pyreturnmultidemfloat4(6,5,3,2);
+INFO: ([[[[0, 1, 2, 3, 4, 5], [0, 1, 2, 3, 4, 5], [0, 1, 2, 3, 4, 5], [0, 1, 2, 3, 4, 5], [0, 1, 2, 3, 4, 5]], [[0, 1, 2, 3, 4, 5], [0, 1, 2, 3, 4, 5], [0, 1, 2, 3, 4, 5], [0, 1, 2, 3, 4, 5], [0, 1, 2, 3, 4, 5]], [[0, 1, 2, 3, 4, 5], [0, 1, 2, 3, 4, 5], [0, 1, 2, 3, 4, 5], [0, 1, 2, 3, 4, 5], [0, 1, 2, 3, 4, 5]]], [[[0, 1, 2, 3, 4, 5], [0, 1, 2, 3, 4, 5], [0, 1, 2, 3, 4, 5], [0, 1, 2, 3, 4, 5], [0, 1, 2, 3, 4, 5]], [[0, 1, 2, 3, 4, 5], [0, 1, 2, 3, 4, 5], [0, 1, 2, 3, 4, 5], [0, 1, 2, 3, 4, 5], [0, 1, 2, 3, 4, 5]], [[0, 1, 2, 3, 4, 5], [0, 1, 2, 3, 4, 5], [0, 1, 2, 3, 4, 5], [0, 1, 2, 3, 4, 5], [0, 1, 2, 3, 4, 5]]]], <type 'list'>)
+ pyreturnmultidemfloat4
+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
+ {{{{0,1,2,3,4,5},{0,1,2,3,4,5},{0,1,2,3,4,5},{0,1,2,3,4,5},{0,1,2,3,4,5}},{{0,1,2,3,4,5},{0,1,2,3,4,5},{0,1,2,3,4,5},{0,1,2,3,4,5},{0,1,2,3,4,5}},{{0,1,2,3,4,5},{0,1,2,3,4,5},{0,1,2,3,4,5},{0,1,2,3,4,5},{0,1,2,3,4,5}}},{{{0,1,2,3,4,5},{0,1,2,3,4,5},{0,1,2,3,4,5},{0,1,2,3,4,5},{0,1,2,3,4,5}},{{0,1,2,3,4,5},{0,1,2,3,4,5},{0,1,2,3,4,5},{0,1,2,3,4,5},{0,1,2,3,4,5}},{{0,1,2,3,4,5},{0,1,2,3,4,5},{0,1,2,3,4,5},{0,1,2,3,4,5},{0,1,2,3,4,5}}}}
+(1 row)
+
+CREATE OR REPLACE FUNCTION pyreturnmultidemfloat8(h int4, i int4, j int4, k int4 ) RETURNS float8[] AS $BODY$
+m = [[[[x for x in range(h)] for y in range(i)] for z in range(j)] for w in range(k)]
+plpy.info(m, type(m))
+return m
+$BODY$ LANGUAGE plpythonu;
+select pyreturnmultidemfloat8(7,5,3,2);
+INFO: ([[[[0, 1, 2, 3, 4, 5, 6], [0, 1, 2, 3, 4, 5, 6], [0, 1, 2, 3, 4, 5, 6], [0, 1, 2, 3, 4, 5, 6], [0, 1, 2, 3, 4, 5, 6]], [[0, 1, 2, 3, 4, 5, 6], [0, 1, 2, 3, 4, 5, 6], [0, 1, 2, 3, 4, 5, 6], [0, 1, 2, 3, 4, 5, 6], [0, 1, 2, 3, 4, 5, 6]], [[0, 1, 2, 3, 4, 5, 6], [0, 1, 2, 3, 4, 5, 6], [0, 1, 2, 3, 4, 5, 6], [0, 1, 2, 3, 4, 5, 6], [0, 1, 2, 3, 4, 5, 6]]], [[[0, 1, 2, 3, 4, 5, 6], [0, 1, 2, 3, 4, 5, 6], [0, 1, 2, 3, 4, 5, 6], [0, 1, 2, 3, 4, 5, 6], [0, 1, 2, 3, 4, 5, 6]], [[0, 1, 2, 3, 4, 5, 6], [0, 1, 2, 3, 4, 5, 6], [0, 1, 2, 3, 4, 5, 6], [0, 1, 2, 3, 4, 5, 6], [0, 1, 2, 3, 4, 5, 6]], [[0, 1, 2, 3, 4, 5, 6], [0, 1, 2, 3, 4, 5, 6], [0, 1, 2, 3, 4, 5, 6], [0, 1, 2, 3, 4, 5, 6], [0, 1, 2, 3, 4, 5, 6]]]], <type 'list'>)
+ pyreturnmultidemfloat8
+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
+ {{{{0,1,2,3,4,5,6},{0,1,2,3,4,5,6},{0,1,2,3,4,5,6},{0,1,2,3,4,5,6},{0,1,2,3,4,5,6}},{{0,1,2,3,4,5,6},{0,1,2,3,4,5,6},{0,1,2,3,4,5,6},{0,1,2,3,4,5,6},{0,1,2,3,4,5,6}},{{0,1,2,3,4,5,6},{0,1,2,3,4,5,6},{0,1,2,3,4,5,6},{0,1,2,3,4,5,6},{0,1,2,3,4,5,6}}},{{{0,1,2,3,4,5,6},{0,1,2,3,4,5,6},{0,1,2,3,4,5,6},{0,1,2,3,4,5,6},{0,1,2,3,4,5,6}},{{0,1,2,3,4,5,6},{0,1,2,3,4,5,6},{0,1,2,3,4,5,6},{0,1,2,3,4,5,6},{0,1,2,3,4,5,6}},{{0,1,2,3,4,5,6},{0,1,2,3,4,5,6},{0,1,2,3,4,5,6},{0,1,2,3,4,5,6},{0,1,2,3,4,5,6}}}}
+(1 row)
+
CREATE FUNCTION test_type_conversion_array_text(x text[]) RETURNS text[] AS $$
plpy.info(x, type(x))
return x
@@ -551,6 +675,13 @@ INFO: (['foo', 'bar'], <type 'list'>)
{foo,bar}
(1 row)
+SELECT * FROM test_type_conversion_array_text(ARRAY[['foo', 'bar'],['foo2', 'bar2']]);
+INFO: ([['foo', 'bar'], ['foo2', 'bar2']], <type 'list'>)
+ test_type_conversion_array_text
+---------------------------------
+ {{foo,bar},{foo2,bar2}}
+(1 row)
+
CREATE FUNCTION test_type_conversion_array_bytea(x bytea[]) RETURNS bytea[] AS $$
plpy.info(x, type(x))
return x
@@ -578,6 +709,13 @@ SELECT * FROM test_type_conversion_array_mixed2();
ERROR: invalid input syntax for integer: "abc"
CONTEXT: while creating return value
PL/Python function "test_type_conversion_array_mixed2"
+CREATE FUNCTION test_type_conversion_mdarray_malformed() RETURNS int[] AS $$
+return [[1,2,3],[4,5]]
+$$ LANGUAGE plpythonu;
+SELECT * FROM test_type_conversion_mdarray_malformed();
+ERROR: Multidimensional arrays must have array expressions with matching dimensions. PL/Python function return value has sequence length 2 while expected 3
+CONTEXT: while creating return value
+PL/Python function "test_type_conversion_mdarray_malformed"
CREATE FUNCTION test_type_conversion_array_record() RETURNS type_record[] AS $$
return [{'first': 'one', 'second': 42}, {'first': 'two', 'second': 11}]
$$ LANGUAGE plpythonu;
diff --git a/src/pl/plpython/expected/plpython_types_3.out b/src/pl/plpython/expected/plpython_types_3.out
index 56b78e1..b244d2a 100644
--- a/src/pl/plpython/expected/plpython_types_3.out
+++ b/src/pl/plpython/expected/plpython_types_3.out
@@ -537,9 +537,126 @@ INFO: (None, <class 'NoneType'>)
(1 row)
SELECT * FROM test_type_conversion_array_int4(ARRAY[[1,2,3],[4,5,6]]);
-ERROR: cannot convert multidimensional array to Python list
-DETAIL: PL/Python only supports one-dimensional arrays.
-CONTEXT: PL/Python function "test_type_conversion_array_int4"
+INFO: ([[1, 2, 3], [4, 5, 6]], <class 'list'>)
+ test_type_conversion_array_int4
+---------------------------------
+ {{1,2,3},{4,5,6}}
+(1 row)
+
+SELECT * FROM test_type_conversion_array_int4(ARRAY[[[1,2,NULL],[NULL,5,6]],[[NULL,8,9],[10,11,12]]]);
+INFO: ([[[1, 2, None], [None, 5, 6]], [[None, 8, 9], [10, 11, 12]]], <class 'list'>)
+ test_type_conversion_array_int4
+---------------------------------------------------
+ {{{1,2,NULL},{NULL,5,6}},{{NULL,8,9},{10,11,12}}}
+(1 row)
+
+CREATE FUNCTION test_type_conversion_array_int8(x int8[]) RETURNS int8[] AS $$
+plpy.info(x, type(x))
+return x
+$$ LANGUAGE plpython3u;
+SELECT * FROM test_type_conversion_array_int8(ARRAY[[[1,2,NULL],[NULL,5,6]],[[NULL,8,9],[10,11,12]]]::int8[]);
+INFO: ([[[1, 2, None], [None, 5, 6]], [[None, 8, 9], [10, 11, 12]]], <class 'list'>)
+ test_type_conversion_array_int8
+---------------------------------------------------
+ {{{1,2,NULL},{NULL,5,6}},{{NULL,8,9},{10,11,12}}}
+(1 row)
+
+CREATE FUNCTION test_type_conversion_array_float4(x float4[]) RETURNS float4[] AS $$
+plpy.info(x, type(x))
+return x
+$$ LANGUAGE plpython3u;
+SELECT * FROM test_type_conversion_array_float4(ARRAY[[[1.2,2.3,NULL],[NULL,5.7,6.8]],[[NULL,8.9,9.345],[10.123,11.456,12.6768]]]::float4[]);
+INFO: ([[[1.2000000476837158, 2.299999952316284, None], [None, 5.699999809265137, 6.800000190734863]], [[None, 8.899999618530273, 9.345000267028809], [10.123000144958496, 11.456000328063965, 12.676799774169922]]], <class 'list'>)
+ test_type_conversion_array_float4
+------------------------------------------------------------------------------
+ {{{1.2,2.3,NULL},{NULL,5.7,6.8}},{{NULL,8.9,9.345},{10.123,11.456,12.6768}}}
+(1 row)
+
+CREATE FUNCTION test_type_conversion_array_float8(x float8[]) RETURNS float8[] AS $$
+plpy.info(x, type(x))
+return x
+$$ LANGUAGE plpython3u;
+SELECT * FROM test_type_conversion_array_float8(ARRAY[[[1.2,2.3,NULL],[NULL,5.7,6.8]],[[NULL,8.9,9.345],[10.123,11.456,12.6768]]]::float8[]);
+INFO: ([[[1.2, 2.3, None], [None, 5.7, 6.8]], [[None, 8.9, 9.345], [10.123, 11.456, 12.6768]]], <class 'list'>)
+ test_type_conversion_array_float8
+------------------------------------------------------------------------------
+ {{{1.2,2.3,NULL},{NULL,5.7,6.8}},{{NULL,8.9,9.345},{10.123,11.456,12.6768}}}
+(1 row)
+
+CREATE FUNCTION test_type_conversion_array_date(x date[]) RETURNS date[] AS $$
+plpy.info(x, type(x))
+return x
+$$ LANGUAGE plpython3u;
+SELECT * FROM test_type_conversion_array_date(ARRAY[[['2016-09-21','2016-09-22',NULL],[NULL,'2016-10-21','2016-10-22']],
+ [[NULL,'2016-11-21','2016-10-21'],['2015-09-21','2015-09-22','2014-09-21']]]::date[]);
+INFO: ([[['09-21-2016', '09-22-2016', None], [None, '10-21-2016', '10-22-2016']], [[None, '11-21-2016', '10-21-2016'], ['09-21-2015', '09-22-2015', '09-21-2014']]], <class 'list'>)
+ test_type_conversion_array_date
+---------------------------------------------------------------------------------------------------------------------------------
+ {{{09-21-2016,09-22-2016,NULL},{NULL,10-21-2016,10-22-2016}},{{NULL,11-21-2016,10-21-2016},{09-21-2015,09-22-2015,09-21-2014}}}
+(1 row)
+
+CREATE FUNCTION test_type_conversion_array_timestamp(x timestamp[]) RETURNS timestamp[] AS $$
+plpy.info(x, type(x))
+return x
+$$ LANGUAGE plpython3u;
+SELECT * FROM test_type_conversion_array_timestamp(ARRAY[[['2016-09-21 15:34:24.078792-04','2016-10-22 11:34:24.078795-04',NULL],
+ [NULL,'2016-10-21 11:34:25.078792-04','2016-10-21 11:34:24.098792-04']],
+ [[NULL,'2016-01-21 11:34:24.078792-04','2016-11-21 11:34:24.108792-04'],
+ ['2015-09-21 11:34:24.079792-04','2014-09-21 11:34:24.078792-04','2013-09-21 11:34:24.078792-04']]]::timestamp[]);
+INFO: ([[['Wed Sep 21 15:34:24.078792 2016', 'Sat Oct 22 11:34:24.078795 2016', None], [None, 'Fri Oct 21 11:34:25.078792 2016', 'Fri Oct 21 11:34:24.098792 2016']], [[None, 'Thu Jan 21 11:34:24.078792 2016', 'Mon Nov 21 11:34:24.108792 2016'], ['Mon Sep 21 11:34:24.079792 2015', 'Sun Sep 21 11:34:24.078792 2014', 'Sat Sep 21 11:34:24.078792 2013']]], <class 'list'>)
+ test_type_conversion_array_timestamp
+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
+ {{{"Wed Sep 21 15:34:24.078792 2016","Sat Oct 22 11:34:24.078795 2016",NULL},{NULL,"Fri Oct 21 11:34:25.078792 2016","Fri Oct 21 11:34:24.098792 2016"}},{{NULL,"Thu Jan 21 11:34:24.078792 2016","Mon Nov 21 11:34:24.108792 2016"},{"Mon Sep 21 11:34:24.079792 2015","Sun Sep 21 11:34:24.078792 2014","Sat Sep 21 11:34:24.078792 2013"}}}
+(1 row)
+
+CREATE OR REPLACE FUNCTION pyreturnmultidemint4(h int4, i int4, j int4, k int4 ) RETURNS int4[] AS $BODY$
+m = [[[[x for x in range(h)] for y in range(i)] for z in range(j)] for w in range(k)]
+plpy.info(m, type(m))
+return m
+$BODY$ LANGUAGE plpython3u;
+select pyreturnmultidemint4(8,5,3,2);
+INFO: ([[[[0, 1, 2, 3, 4, 5, 6, 7], [0, 1, 2, 3, 4, 5, 6, 7], [0, 1, 2, 3, 4, 5, 6, 7], [0, 1, 2, 3, 4, 5, 6, 7], [0, 1, 2, 3, 4, 5, 6, 7]], [[0, 1, 2, 3, 4, 5, 6, 7], [0, 1, 2, 3, 4, 5, 6, 7], [0, 1, 2, 3, 4, 5, 6, 7], [0, 1, 2, 3, 4, 5, 6, 7], [0, 1, 2, 3, 4, 5, 6, 7]], [[0, 1, 2, 3, 4, 5, 6, 7], [0, 1, 2, 3, 4, 5, 6, 7], [0, 1, 2, 3, 4, 5, 6, 7], [0, 1, 2, 3, 4, 5, 6, 7], [0, 1, 2, 3, 4, 5, 6, 7]]], [[[0, 1, 2, 3, 4, 5, 6, 7], [0, 1, 2, 3, 4, 5, 6, 7], [0, 1, 2, 3, 4, 5, 6, 7], [0, 1, 2, 3, 4, 5, 6, 7], [0, 1, 2, 3, 4, 5, 6, 7]], [[0, 1, 2, 3, 4, 5, 6, 7], [0, 1, 2, 3, 4, 5, 6, 7], [0, 1, 2, 3, 4, 5, 6, 7], [0, 1, 2, 3, 4, 5, 6, 7], [0, 1, 2, 3, 4, 5, 6, 7]], [[0, 1, 2, 3, 4, 5, 6, 7], [0, 1, 2, 3, 4, 5, 6, 7], [0, 1, 2, 3, 4, 5, 6, 7], [0, 1, 2, 3, 4, 5, 6, 7], [0, 1, 2, 3, 4, 5, 6, 7]]]], <class 'list'>)
+ pyreturnmultidemint4
+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
+ {{{{0,1,2,3,4,5,6,7},{0,1,2,3,4,5,6,7},{0,1,2,3,4,5,6,7},{0,1,2,3,4,5,6,7},{0,1,2,3,4,5,6,7}},{{0,1,2,3,4,5,6,7},{0,1,2,3,4,5,6,7},{0,1,2,3,4,5,6,7},{0,1,2,3,4,5,6,7},{0,1,2,3,4,5,6,7}},{{0,1,2,3,4,5,6,7},{0,1,2,3,4,5,6,7},{0,1,2,3,4,5,6,7},{0,1,2,3,4,5,6,7},{0,1,2,3,4,5,6,7}}},{{{0,1,2,3,4,5,6,7},{0,1,2,3,4,5,6,7},{0,1,2,3,4,5,6,7},{0,1,2,3,4,5,6,7},{0,1,2,3,4,5,6,7}},{{0,1,2,3,4,5,6,7},{0,1,2,3,4,5,6,7},{0,1,2,3,4,5,6,7},{0,1,2,3,4,5,6,7},{0,1,2,3,4,5,6,7}},{{0,1,2,3,4,5,6,7},{0,1,2,3,4,5,6,7},{0,1,2,3,4,5,6,7},{0,1,2,3,4,5,6,7},{0,1,2,3,4,5,6,7}}}}
+(1 row)
+
+CREATE OR REPLACE FUNCTION pyreturnmultidemint8(h int4, i int4, j int4, k int4 ) RETURNS int8[] AS $BODY$
+m = [[[[x for x in range(h)] for y in range(i)] for z in range(j)] for w in range(k)]
+plpy.info(m, type(m))
+return m
+$BODY$ LANGUAGE plpython3u;
+select pyreturnmultidemint8(5,5,3,2);
+INFO: ([[[[0, 1, 2, 3, 4], [0, 1, 2, 3, 4], [0, 1, 2, 3, 4], [0, 1, 2, 3, 4], [0, 1, 2, 3, 4]], [[0, 1, 2, 3, 4], [0, 1, 2, 3, 4], [0, 1, 2, 3, 4], [0, 1, 2, 3, 4], [0, 1, 2, 3, 4]], [[0, 1, 2, 3, 4], [0, 1, 2, 3, 4], [0, 1, 2, 3, 4], [0, 1, 2, 3, 4], [0, 1, 2, 3, 4]]], [[[0, 1, 2, 3, 4], [0, 1, 2, 3, 4], [0, 1, 2, 3, 4], [0, 1, 2, 3, 4], [0, 1, 2, 3, 4]], [[0, 1, 2, 3, 4], [0, 1, 2, 3, 4], [0, 1, 2, 3, 4], [0, 1, 2, 3, 4], [0, 1, 2, 3, 4]], [[0, 1, 2, 3, 4], [0, 1, 2, 3, 4], [0, 1, 2, 3, 4], [0, 1, 2, 3, 4], [0, 1, 2, 3, 4]]]], <class 'list'>)
+ pyreturnmultidemint8
+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
+ {{{{0,1,2,3,4},{0,1,2,3,4},{0,1,2,3,4},{0,1,2,3,4},{0,1,2,3,4}},{{0,1,2,3,4},{0,1,2,3,4},{0,1,2,3,4},{0,1,2,3,4},{0,1,2,3,4}},{{0,1,2,3,4},{0,1,2,3,4},{0,1,2,3,4},{0,1,2,3,4},{0,1,2,3,4}}},{{{0,1,2,3,4},{0,1,2,3,4},{0,1,2,3,4},{0,1,2,3,4},{0,1,2,3,4}},{{0,1,2,3,4},{0,1,2,3,4},{0,1,2,3,4},{0,1,2,3,4},{0,1,2,3,4}},{{0,1,2,3,4},{0,1,2,3,4},{0,1,2,3,4},{0,1,2,3,4},{0,1,2,3,4}}}}
+(1 row)
+
+CREATE OR REPLACE FUNCTION pyreturnmultidemfloat4(h int4, i int4, j int4, k int4 ) RETURNS float4[] AS $BODY$
+m = [[[[x for x in range(h)] for y in range(i)] for z in range(j)] for w in range(k)]
+plpy.info(m, type(m))
+return m
+$BODY$ LANGUAGE plpython3u;
+select pyreturnmultidemfloat4(6,5,3,2);
+INFO: ([[[[0, 1, 2, 3, 4, 5], [0, 1, 2, 3, 4, 5], [0, 1, 2, 3, 4, 5], [0, 1, 2, 3, 4, 5], [0, 1, 2, 3, 4, 5]], [[0, 1, 2, 3, 4, 5], [0, 1, 2, 3, 4, 5], [0, 1, 2, 3, 4, 5], [0, 1, 2, 3, 4, 5], [0, 1, 2, 3, 4, 5]], [[0, 1, 2, 3, 4, 5], [0, 1, 2, 3, 4, 5], [0, 1, 2, 3, 4, 5], [0, 1, 2, 3, 4, 5], [0, 1, 2, 3, 4, 5]]], [[[0, 1, 2, 3, 4, 5], [0, 1, 2, 3, 4, 5], [0, 1, 2, 3, 4, 5], [0, 1, 2, 3, 4, 5], [0, 1, 2, 3, 4, 5]], [[0, 1, 2, 3, 4, 5], [0, 1, 2, 3, 4, 5], [0, 1, 2, 3, 4, 5], [0, 1, 2, 3, 4, 5], [0, 1, 2, 3, 4, 5]], [[0, 1, 2, 3, 4, 5], [0, 1, 2, 3, 4, 5], [0, 1, 2, 3, 4, 5], [0, 1, 2, 3, 4, 5], [0, 1, 2, 3, 4, 5]]]], <class 'list'>)
+ pyreturnmultidemfloat4
+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
+ {{{{0,1,2,3,4,5},{0,1,2,3,4,5},{0,1,2,3,4,5},{0,1,2,3,4,5},{0,1,2,3,4,5}},{{0,1,2,3,4,5},{0,1,2,3,4,5},{0,1,2,3,4,5},{0,1,2,3,4,5},{0,1,2,3,4,5}},{{0,1,2,3,4,5},{0,1,2,3,4,5},{0,1,2,3,4,5},{0,1,2,3,4,5},{0,1,2,3,4,5}}},{{{0,1,2,3,4,5},{0,1,2,3,4,5},{0,1,2,3,4,5},{0,1,2,3,4,5},{0,1,2,3,4,5}},{{0,1,2,3,4,5},{0,1,2,3,4,5},{0,1,2,3,4,5},{0,1,2,3,4,5},{0,1,2,3,4,5}},{{0,1,2,3,4,5},{0,1,2,3,4,5},{0,1,2,3,4,5},{0,1,2,3,4,5},{0,1,2,3,4,5}}}}
+(1 row)
+
+CREATE OR REPLACE FUNCTION pyreturnmultidemfloat8(h int4, i int4, j int4, k int4 ) RETURNS float8[] AS $BODY$
+m = [[[[x for x in range(h)] for y in range(i)] for z in range(j)] for w in range(k)]
+plpy.info(m, type(m))
+return m
+$BODY$ LANGUAGE plpython3u;
+select pyreturnmultidemfloat8(7,5,3,2);
+INFO: ([[[[0, 1, 2, 3, 4, 5, 6], [0, 1, 2, 3, 4, 5, 6], [0, 1, 2, 3, 4, 5, 6], [0, 1, 2, 3, 4, 5, 6], [0, 1, 2, 3, 4, 5, 6]], [[0, 1, 2, 3, 4, 5, 6], [0, 1, 2, 3, 4, 5, 6], [0, 1, 2, 3, 4, 5, 6], [0, 1, 2, 3, 4, 5, 6], [0, 1, 2, 3, 4, 5, 6]], [[0, 1, 2, 3, 4, 5, 6], [0, 1, 2, 3, 4, 5, 6], [0, 1, 2, 3, 4, 5, 6], [0, 1, 2, 3, 4, 5, 6], [0, 1, 2, 3, 4, 5, 6]]], [[[0, 1, 2, 3, 4, 5, 6], [0, 1, 2, 3, 4, 5, 6], [0, 1, 2, 3, 4, 5, 6], [0, 1, 2, 3, 4, 5, 6], [0, 1, 2, 3, 4, 5, 6]], [[0, 1, 2, 3, 4, 5, 6], [0, 1, 2, 3, 4, 5, 6], [0, 1, 2, 3, 4, 5, 6], [0, 1, 2, 3, 4, 5, 6], [0, 1, 2, 3, 4, 5, 6]], [[0, 1, 2, 3, 4, 5, 6], [0, 1, 2, 3, 4, 5, 6], [0, 1, 2, 3, 4, 5, 6], [0, 1, 2, 3, 4, 5, 6], [0, 1, 2, 3, 4, 5, 6]]]], <class 'list'>)
+ pyreturnmultidemfloat8
+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
+ {{{{0,1,2,3,4,5,6},{0,1,2,3,4,5,6},{0,1,2,3,4,5,6},{0,1,2,3,4,5,6},{0,1,2,3,4,5,6}},{{0,1,2,3,4,5,6},{0,1,2,3,4,5,6},{0,1,2,3,4,5,6},{0,1,2,3,4,5,6},{0,1,2,3,4,5,6}},{{0,1,2,3,4,5,6},{0,1,2,3,4,5,6},{0,1,2,3,4,5,6},{0,1,2,3,4,5,6},{0,1,2,3,4,5,6}}},{{{0,1,2,3,4,5,6},{0,1,2,3,4,5,6},{0,1,2,3,4,5,6},{0,1,2,3,4,5,6},{0,1,2,3,4,5,6}},{{0,1,2,3,4,5,6},{0,1,2,3,4,5,6},{0,1,2,3,4,5,6},{0,1,2,3,4,5,6},{0,1,2,3,4,5,6}},{{0,1,2,3,4,5,6},{0,1,2,3,4,5,6},{0,1,2,3,4,5,6},{0,1,2,3,4,5,6},{0,1,2,3,4,5,6}}}}
+(1 row)
+
CREATE FUNCTION test_type_conversion_array_text(x text[]) RETURNS text[] AS $$
plpy.info(x, type(x))
return x
@@ -551,6 +668,13 @@ INFO: (['foo', 'bar'], <class 'list'>)
{foo,bar}
(1 row)
+SELECT * FROM test_type_conversion_array_text(ARRAY[['foo', 'bar'],['foo2', 'bar2']]);
+INFO: ([['foo', 'bar'], ['foo2', 'bar2']], <class 'list'>)
+ test_type_conversion_array_text
+---------------------------------
+ {{foo,bar},{foo2,bar2}}
+(1 row)
+
CREATE FUNCTION test_type_conversion_array_bytea(x bytea[]) RETURNS bytea[] AS $$
plpy.info(x, type(x))
return x
@@ -578,6 +702,13 @@ SELECT * FROM test_type_conversion_array_mixed2();
ERROR: invalid input syntax for integer: "abc"
CONTEXT: while creating return value
PL/Python function "test_type_conversion_array_mixed2"
+CREATE FUNCTION test_type_conversion_mdarray_malformed() RETURNS int[] AS $$
+return [[1,2,3],[4,5]]
+$$ LANGUAGE plpython3u;
+SELECT * FROM test_type_conversion_mdarray_malformed();
+ERROR: Multidimensional arrays must have array expressions with matching dimensions. PL/Python function return value has sequence length 2 while expected 3
+CONTEXT: while creating return value
+PL/Python function "test_type_conversion_mdarray_malformed"
CREATE FUNCTION test_type_conversion_array_record() RETURNS type_record[] AS $$
return [{'first': 'one', 'second': 42}, {'first': 'two', 'second': 11}]
$$ LANGUAGE plpython3u;
diff --git a/src/pl/plpython/plpy_typeio.c b/src/pl/plpython/plpy_typeio.c
index 70f2e6d..6414d47 100644
--- a/src/pl/plpython/plpy_typeio.c
+++ b/src/pl/plpython/plpy_typeio.c
@@ -631,46 +631,111 @@ PLyList_FromArray(PLyDatumToOb *arg, Datum d)
{
ArrayType *array = DatumGetArrayTypeP(d);
PLyDatumToOb *elm = arg->elm;
- PyObject *list;
- int length;
- int lbound;
+ int ndim;
+ int *dims;
+ int *lb;
+ char *dataptr;
+ bits8 *bitmap;
+ int bitmask;
int i;
+ int dim;
+ int indx[MAXDIM];
+ PyObject *lists[MAXDIM];
if (ARR_NDIM(array) == 0)
return PyList_New(0);
- if (ARR_NDIM(array) != 1)
- ereport(ERROR,
- (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
- errmsg("cannot convert multidimensional array to Python list"),
- errdetail("PL/Python only supports one-dimensional arrays.")));
+ /* Array dimensions and left bounds */
+ ndim = ARR_NDIM(array);
+ dims = ARR_DIMS(array);
+ lb = ARR_LBOUND(array);
- length = ARR_DIMS(array)[0];
- lbound = ARR_LBOUND(array)[0];
- list = PyList_New(length);
- if (list == NULL)
- PLy_elog(ERROR, "could not create new Python list");
+ /* Internal array representation pointers */
+ dataptr = ARR_DATA_PTR(array);
+ bitmap = ARR_NULLBITMAP(array);
+ bitmask = 1;
- for (i = 0; i < length; i++)
+ /* Iterators initialization */
+ for (i = 0; i < ndim; i++) {
+ indx[i] = lb[i];
+ lists[i] = NULL;
+ }
+ lists[0] = PyList_New(dims[0]);
+
+ /* We need this incref to keep the pointer valid after the array traversal
+ * terminates, as this traversal does DECREF for all the lists in array */
+ Py_INCREF(lists[0]);
+
+ /* In this cycle we are going over array dimensions. Postgres offers you an
+ * option to iterate over all the multi-dimensional array elements in order.
+ * For 3-dimensional array the order of iteration would be following - first
+ * you start with [0,0,0] elements through [0,0,k], then [0,1,0] till [0,1,k]
+ * till [0,m,k], then [1,0,0] till [1,0,k] till [1,m,k], and so on.
+ * In Python, each 1-d array is a separate list object, so 3-d array of
+ * [n,m,k] element is a list of n m-element arrays, each element of which is
+ * k-element array. In this cycle we traverse from outter dimensions to
+ * inner ones, creating nested Python lists during traversal */
+ dim = 0;
+ while (dim >= 0)
{
- Datum elem;
- bool isnull;
- int offset;
-
- offset = lbound + i;
- elem = array_ref(array, 1, &offset, arg->typlen,
- elm->typlen, elm->typbyval, elm->typalign,
- &isnull);
- if (isnull)
+ /* If we finished up iterating over current dimension - go one level up */
+ if (indx[dim] > dims[dim])
{
- Py_INCREF(Py_None);
- PyList_SET_ITEM(list, i, Py_None);
+ Py_DECREF(lists[dim]);
+ indx[dim] = 0;
+ dim -= 1;
+ }
+ /* If we are processing inner dimension - create one more list */
+ else if (dim < ndim - 1)
+ {
+ lists[dim+1] = PyList_New(dims[dim+1]);
+
+ /* We need this INCREF as we keep array pointer on our side,
+ * while PyList_SET_ITEM steals the reference */
+ Py_INCREF(lists[dim+1]);
+
+ PyList_SET_ITEM(lists[dim], indx[dim] - lb[dim], lists[dim+1]);
+ indx[dim] += 1;
+ dim += 1;
+ indx[dim] = lb[dim];
+ }
+ /* If we are iterating over the outer dimension, fill the list with
+ * values from the original Postgres array */
+ else if (dim == ndim - 1)
+ {
+ for (indx[dim] = lb[dim]; (indx[dim] - lb[dim]) < dims[dim]; indx[dim]++)
+ {
+ /* checking for NULL */
+ if (bitmap && (*bitmap & bitmask) == 0)
+ {
+ Py_INCREF(Py_None);
+ PyList_SET_ITEM(lists[dim], indx[dim] - lb[dim], Py_None);
+ }
+ else
+ {
+ Datum itemvalue;
+
+ itemvalue = fetch_att(dataptr, elm->typbyval, elm->typlen);
+ PyList_SET_ITEM(lists[dim], indx[dim] - lb[dim], elm->func(elm, itemvalue));
+ dataptr = att_addlength_pointer(dataptr, elm->typlen, dataptr);
+ dataptr = (char *) att_align_nominal(dataptr, elm->typalign);
+ }
+
+ /* advance bitmap pointer if any */
+ if (bitmap)
+ {
+ bitmask <<= 1;
+ if (bitmask == 0x100 /* (1<<8) */)
+ {
+ bitmap++;
+ bitmask = 1;
+ }
+ }
+ }
}
- else
- PyList_SET_ITEM(list, i, elm->func(elm, elem));
}
- return list;
+ return lists[0];
}
/*
@@ -864,39 +929,131 @@ static Datum
PLySequence_ToArray(PLyObToDatum *arg, int32 typmod, PyObject *plrv)
{
ArrayType *array;
- Datum rv;
int i;
Datum *elems;
bool *nulls;
int len;
- int lbs;
+ int ndim;
+ int dims[MAXDIM];
+ int lbs[MAXDIM];
+ int indx[MAXDIM];
+ PyObject *stack[MAXDIM];
+ int dim;
+ int idxelem;
+ Datum rv;
Assert(plrv != Py_None);
if (!PySequence_Check(plrv))
PLy_elog(ERROR, "return value of function with array return type is not a Python sequence");
- len = PySequence_Length(plrv);
+ ndim = 1;
+ dims[0] = PySequence_Length(plrv);
+ len = dims[0];
+ stack[0] = plrv;
+
+ /* We don't want to create multi-dimensional arrays when we have an empty sequence,
+ * when we need to parse sequence of composite objects, and when we have strings */
+ if (len > 0 && !type_is_rowtype(get_base_element_type(arg->typoid)) &&
+ !PyString_Check(plrv) && !PyBytes_Check(plrv) && !PyUnicode_Check(plrv))
+ {
+ PyObject *pyptr = PySequence_GetItem(plrv, 0);
+
+ /* We want to iterate through all iterable objects except by strings on nested levels */
+ while (pyptr != NULL && PySequence_Check(pyptr) &&
+ !(PyString_Check(pyptr) || PyBytes_Check(pyptr) || PyUnicode_Check(pyptr)))
+ {
+ dims[ndim] = PySequence_Length(pyptr);
+ if (dims[ndim] < 0)
+ PLy_elog(ERROR, "Cannot determine sequence length for function return value");
+ len *= dims[ndim];
+ stack[ndim] = pyptr;
+ ndim += 1;
+ if (dims[ndim - 1] == 0) {
+ pyptr = NULL;
+ break;
+ }
+ pyptr = PySequence_GetItem(pyptr, 0);
+ }
+
+ /* Pyptr points to element of n-dimensional array, we don't need its reference */
+ Py_XDECREF(pyptr);
+ }
+
+ /* We need this incref to keep the pointer valid after the array traversal
+ * terminates, as this traversal does DECREF for all the lists in array, and
+ * stack[0] corresponds to function return value */
+ Py_INCREF(stack[0]);
+
elems = palloc(sizeof(*elems) * len);
nulls = palloc(sizeof(*nulls) * len);
- for (i = 0; i < len; i++)
- {
- PyObject *obj = PySequence_GetItem(plrv, i);
+ for (i = 0; i < ndim; i++) {
+ indx[i] = 0;
+ lbs[i] = 1;
+ }
- if (obj == Py_None)
- nulls[i] = true;
- else
+ /* In this cycle we are going over nested Python lists, fetching elements
+ * from the deepest level and putting them into a linear array for Postgres
+ * to interpret them as n-dimensional array. This is a cycle implementation
+ * of DFS (recursive traversal of nested arrays here), keeping the stack in
+ * "stack" variable */
+ dim = 0;
+ idxelem = 0;
+ while (dim >= 0)
+ {
+ /* If we finished up iterating over current list - go one level up */
+ if (indx[dim] == dims[dim])
+ {
+ Py_DECREF(stack[dim]);
+ indx[dim] = 0;
+ dim -= 1;
+ }
+ /* If we are processing inner list - create one more list */
+ else if (dim < ndim - 1)
{
- nulls[i] = false;
- elems[i] = arg->elm->func(arg->elm, -1, obj);
+ stack[dim+1] = PySequence_GetItem(stack[dim], indx[dim]);
+ if (PySequence_Length(stack[dim+1]) != dims[dim+1])
+ PLy_elog(ERROR, "Multidimensional arrays must have array expressions with matching dimensions. "
+ "PL/Python function return value has sequence length %d while expected %d",
+ (int)PySequence_Length(stack[dim+1]), dims[dim+1]);
+ indx[dim] += 1;
+ dim += 1;
+ }
+ /* If we are iterating over the outter list, fill the output array */
+ else if (dim == ndim - 1)
+ {
+ for (indx[dim] = 0; indx[dim] < dims[dim]; indx[dim]++)
+ {
+ PyObject *obj = PySequence_GetItem(stack[dim], indx[dim]);
+
+ if (obj == Py_None)
+ nulls[idxelem] = true;
+ else
+ {
+ nulls[idxelem] = false;
+
+ /*
+ * We don't support arrays of row types yet, so the first argument
+ * can be NULL.
+ */
+ elems[idxelem] = arg->elm->func(arg->elm, -1, obj);
+ }
+ Py_XDECREF(obj);
+ idxelem += 1;
+ }
}
- Py_XDECREF(obj);
}
- lbs = 1;
- array = construct_md_array(elems, nulls, 1, &len, &lbs,
- get_base_element_type(arg->typoid), arg->elm->typlen, arg->elm->typbyval, arg->elm->typalign);
+ array = construct_md_array(elems,
+ nulls,
+ ndim,
+ dims,
+ lbs,
+ get_base_element_type(arg->typoid),
+ arg->elm->typlen,
+ arg->elm->typbyval,
+ arg->elm->typalign);
/*
* If the result type is a domain of array, the resulting array must be
diff --git a/src/pl/plpython/sql/plpython_types.sql b/src/pl/plpython/sql/plpython_types.sql
index 19d920d..0ada845 100644
--- a/src/pl/plpython/sql/plpython_types.sql
+++ b/src/pl/plpython/sql/plpython_types.sql
@@ -237,7 +237,80 @@ SELECT * FROM test_type_conversion_array_int4(ARRAY[NULL,1]);
SELECT * FROM test_type_conversion_array_int4(ARRAY[]::integer[]);
SELECT * FROM test_type_conversion_array_int4(NULL);
SELECT * FROM test_type_conversion_array_int4(ARRAY[[1,2,3],[4,5,6]]);
+SELECT * FROM test_type_conversion_array_int4(ARRAY[[[1,2,NULL],[NULL,5,6]],[[NULL,8,9],[10,11,12]]]);
+SELECT * FROM test_type_conversion_array_int4('[2:4]={1,2,3}');
+CREATE FUNCTION test_type_conversion_array_int8(x int8[]) RETURNS int8[] AS $$
+plpy.info(x, type(x))
+return x
+$$ LANGUAGE plpythonu;
+
+SELECT * FROM test_type_conversion_array_int8(ARRAY[[[1,2,NULL],[NULL,5,6]],[[NULL,8,9],[10,11,12]]]::int8[]);
+
+CREATE FUNCTION test_type_conversion_array_float4(x float4[]) RETURNS float4[] AS $$
+plpy.info(x, type(x))
+return x
+$$ LANGUAGE plpythonu;
+
+SELECT * FROM test_type_conversion_array_float4(ARRAY[[[1.2,2.3,NULL],[NULL,5.7,6.8]],[[NULL,8.9,9.345],[10.123,11.456,12.6768]]]::float4[]);
+
+CREATE FUNCTION test_type_conversion_array_float8(x float8[]) RETURNS float8[] AS $$
+plpy.info(x, type(x))
+return x
+$$ LANGUAGE plpythonu;
+
+SELECT * FROM test_type_conversion_array_float8(ARRAY[[[1.2,2.3,NULL],[NULL,5.7,6.8]],[[NULL,8.9,9.345],[10.123,11.456,12.6768]]]::float8[]);
+
+CREATE FUNCTION test_type_conversion_array_date(x date[]) RETURNS date[] AS $$
+plpy.info(x, type(x))
+return x
+$$ LANGUAGE plpythonu;
+
+SELECT * FROM test_type_conversion_array_date(ARRAY[[['2016-09-21','2016-09-22',NULL],[NULL,'2016-10-21','2016-10-22']],
+ [[NULL,'2016-11-21','2016-10-21'],['2015-09-21','2015-09-22','2014-09-21']]]::date[]);
+
+CREATE FUNCTION test_type_conversion_array_timestamp(x timestamp[]) RETURNS timestamp[] AS $$
+plpy.info(x, type(x))
+return x
+$$ LANGUAGE plpythonu;
+
+SELECT * FROM test_type_conversion_array_timestamp(ARRAY[[['2016-09-21 15:34:24.078792-04','2016-10-22 11:34:24.078795-04',NULL],
+ [NULL,'2016-10-21 11:34:25.078792-04','2016-10-21 11:34:24.098792-04']],
+ [[NULL,'2016-01-21 11:34:24.078792-04','2016-11-21 11:34:24.108792-04'],
+ ['2015-09-21 11:34:24.079792-04','2014-09-21 11:34:24.078792-04','2013-09-21 11:34:24.078792-04']]]::timestamp[]);
+
+
+CREATE OR REPLACE FUNCTION pyreturnmultidemint4(h int4, i int4, j int4, k int4 ) RETURNS int4[] AS $BODY$
+m = [[[[x for x in range(h)] for y in range(i)] for z in range(j)] for w in range(k)]
+plpy.info(m, type(m))
+return m
+$BODY$ LANGUAGE plpythonu;
+
+select pyreturnmultidemint4(8,5,3,2);
+
+CREATE OR REPLACE FUNCTION pyreturnmultidemint8(h int4, i int4, j int4, k int4 ) RETURNS int8[] AS $BODY$
+m = [[[[x for x in range(h)] for y in range(i)] for z in range(j)] for w in range(k)]
+plpy.info(m, type(m))
+return m
+$BODY$ LANGUAGE plpythonu;
+
+select pyreturnmultidemint8(5,5,3,2);
+
+CREATE OR REPLACE FUNCTION pyreturnmultidemfloat4(h int4, i int4, j int4, k int4 ) RETURNS float4[] AS $BODY$
+m = [[[[x for x in range(h)] for y in range(i)] for z in range(j)] for w in range(k)]
+plpy.info(m, type(m))
+return m
+$BODY$ LANGUAGE plpythonu;
+
+select pyreturnmultidemfloat4(6,5,3,2);
+
+CREATE OR REPLACE FUNCTION pyreturnmultidemfloat8(h int4, i int4, j int4, k int4 ) RETURNS float8[] AS $BODY$
+m = [[[[x for x in range(h)] for y in range(i)] for z in range(j)] for w in range(k)]
+plpy.info(m, type(m))
+return m
+$BODY$ LANGUAGE plpythonu;
+
+select pyreturnmultidemfloat8(7,5,3,2);
CREATE FUNCTION test_type_conversion_array_text(x text[]) RETURNS text[] AS $$
plpy.info(x, type(x))
@@ -245,6 +318,7 @@ return x
$$ LANGUAGE plpythonu;
SELECT * FROM test_type_conversion_array_text(ARRAY['foo', 'bar']);
+SELECT * FROM test_type_conversion_array_text(ARRAY[['foo', 'bar'],['foo2', 'bar2']]);
CREATE FUNCTION test_type_conversion_array_bytea(x bytea[]) RETURNS bytea[] AS $$
@@ -268,6 +342,11 @@ $$ LANGUAGE plpythonu;
SELECT * FROM test_type_conversion_array_mixed2();
+CREATE FUNCTION test_type_conversion_mdarray_malformed() RETURNS int[] AS $$
+return [[1,2,3],[4,5]]
+$$ LANGUAGE plpythonu;
+
+SELECT * FROM test_type_conversion_mdarray_malformed();
CREATE FUNCTION test_type_conversion_array_record() RETURNS type_record[] AS $$
return [{'first': 'one', 'second': 42}, {'first': 'two', 'second': 11}]
Re: PL/Python adding support for multi-dimensional arrays
On 09/27/2016 02:04 PM, Dave Cramer wrote:
On 26 September 2016 at 14:52, Dave Cramer <pg@fastcrypt.com> wrote:
This crashes with arrays with non-default lower bounds:
postgres=# SELECT * FROM test_type_conversion_array_int
4('[2:4]={1,2,3}');
INFO: ([1, 2, <NULL>], <type 'list'>)
server closed the connection unexpectedly
This probably means the server terminated abnormally
before or while processing the request.Attached patch fixes this bug, and adds a test for it.
I spent some more time massaging this:
* Changed the loops from iterative to recursive style. I think this
indeed is slightly easier to understand.
* Fixed another segfault, with too deeply nested lists:
CREATE or replace FUNCTION test_type_conversion_mdarray_toodeep()
RETURNS int[] AS $$
return [[[[[[[[[[[[[[[[[[1]]]]]]]]]]]]]]]]]]
$$ LANGUAGE plpythonu;
* Also, in PLySequence_ToArray(), we must check that the 'len' of the
array doesn't overflow.
* Fixed reference leak in the loop in PLySequence_ToArray() to count the
number of dimensions.
I'd like to see some updates to the docs for this. The manual doesn't
currently say anything about multi-dimensional arrays in pl/python, but it
should've mentioned that they're not supported. Now that it is supported,
should mention that, and explain briefly that a multi-dimensional array is
mapped to a python list of lists.If the code passes I'll fix the docs
Please do, thanks!
- Heikki
Attachments:
0001-WIP-Multi-dimensional-arrays-in-PL-python.patchtext/x-patch; name=0001-WIP-Multi-dimensional-arrays-in-PL-python.patchDownload
From 6bcff9d1787fc645118a3ecbb71d1ef7561a5bfd Mon Sep 17 00:00:00 2001
From: Heikki Linnakangas <heikki.linnakangas@iki.fi>
Date: Tue, 27 Sep 2016 21:53:17 +0300
Subject: [PATCH 1/1] WIP: Multi-dimensional arrays in PL/python
---
src/pl/plpython/expected/plpython_types.out | 151 +++++++++++++-
src/pl/plpython/expected/plpython_types_3.out | 151 +++++++++++++-
src/pl/plpython/plpy_typeio.c | 272 +++++++++++++++++++++-----
src/pl/plpython/sql/plpython_types.sql | 86 ++++++++
4 files changed, 608 insertions(+), 52 deletions(-)
diff --git a/src/pl/plpython/expected/plpython_types.out b/src/pl/plpython/expected/plpython_types.out
index f0b6abd..947244e 100644
--- a/src/pl/plpython/expected/plpython_types.out
+++ b/src/pl/plpython/expected/plpython_types.out
@@ -537,9 +537,133 @@ INFO: (None, <type 'NoneType'>)
(1 row)
SELECT * FROM test_type_conversion_array_int4(ARRAY[[1,2,3],[4,5,6]]);
-ERROR: cannot convert multidimensional array to Python list
-DETAIL: PL/Python only supports one-dimensional arrays.
-CONTEXT: PL/Python function "test_type_conversion_array_int4"
+INFO: ([[1, 2, 3], [4, 5, 6]], <type 'list'>)
+ test_type_conversion_array_int4
+---------------------------------
+ {{1,2,3},{4,5,6}}
+(1 row)
+
+SELECT * FROM test_type_conversion_array_int4(ARRAY[[[1,2,NULL],[NULL,5,6]],[[NULL,8,9],[10,11,12]]]);
+INFO: ([[[1, 2, None], [None, 5, 6]], [[None, 8, 9], [10, 11, 12]]], <type 'list'>)
+ test_type_conversion_array_int4
+---------------------------------------------------
+ {{{1,2,NULL},{NULL,5,6}},{{NULL,8,9},{10,11,12}}}
+(1 row)
+
+SELECT * FROM test_type_conversion_array_int4('[2:4]={1,2,3}');
+INFO: ([1, 2, 3], <type 'list'>)
+ test_type_conversion_array_int4
+---------------------------------
+ {1,2,3}
+(1 row)
+
+CREATE FUNCTION test_type_conversion_array_int8(x int8[]) RETURNS int8[] AS $$
+plpy.info(x, type(x))
+return x
+$$ LANGUAGE plpythonu;
+SELECT * FROM test_type_conversion_array_int8(ARRAY[[[1,2,NULL],[NULL,5,6]],[[NULL,8,9],[10,11,12]]]::int8[]);
+INFO: ([[[1L, 2L, None], [None, 5L, 6L]], [[None, 8L, 9L], [10L, 11L, 12L]]], <type 'list'>)
+ test_type_conversion_array_int8
+---------------------------------------------------
+ {{{1,2,NULL},{NULL,5,6}},{{NULL,8,9},{10,11,12}}}
+(1 row)
+
+CREATE FUNCTION test_type_conversion_array_float4(x float4[]) RETURNS float4[] AS $$
+plpy.info(x, type(x))
+return x
+$$ LANGUAGE plpythonu;
+SELECT * FROM test_type_conversion_array_float4(ARRAY[[[1.2,2.3,NULL],[NULL,5.7,6.8]],[[NULL,8.9,9.345],[10.123,11.456,12.6768]]]::float4[]);
+INFO: ([[[1.2000000476837158, 2.299999952316284, None], [None, 5.699999809265137, 6.800000190734863]], [[None, 8.899999618530273, 9.345000267028809], [10.123000144958496, 11.456000328063965, 12.676799774169922]]], <type 'list'>)
+ test_type_conversion_array_float4
+------------------------------------------------------------------------------
+ {{{1.2,2.3,NULL},{NULL,5.7,6.8}},{{NULL,8.9,9.345},{10.123,11.456,12.6768}}}
+(1 row)
+
+CREATE FUNCTION test_type_conversion_array_float8(x float8[]) RETURNS float8[] AS $$
+plpy.info(x, type(x))
+return x
+$$ LANGUAGE plpythonu;
+SELECT * FROM test_type_conversion_array_float8(ARRAY[[[1.2,2.3,NULL],[NULL,5.7,6.8]],[[NULL,8.9,9.345],[10.123,11.456,12.6768]]]::float8[]);
+INFO: ([[[1.2, 2.3, None], [None, 5.7, 6.8]], [[None, 8.9, 9.345], [10.123, 11.456, 12.6768]]], <type 'list'>)
+ test_type_conversion_array_float8
+------------------------------------------------------------------------------
+ {{{1.2,2.3,NULL},{NULL,5.7,6.8}},{{NULL,8.9,9.345},{10.123,11.456,12.6768}}}
+(1 row)
+
+CREATE FUNCTION test_type_conversion_array_date(x date[]) RETURNS date[] AS $$
+plpy.info(x, type(x))
+return x
+$$ LANGUAGE plpythonu;
+SELECT * FROM test_type_conversion_array_date(ARRAY[[['2016-09-21','2016-09-22',NULL],[NULL,'2016-10-21','2016-10-22']],
+ [[NULL,'2016-11-21','2016-10-21'],['2015-09-21','2015-09-22','2014-09-21']]]::date[]);
+INFO: ([[['09-21-2016', '09-22-2016', None], [None, '10-21-2016', '10-22-2016']], [[None, '11-21-2016', '10-21-2016'], ['09-21-2015', '09-22-2015', '09-21-2014']]], <type 'list'>)
+ test_type_conversion_array_date
+---------------------------------------------------------------------------------------------------------------------------------
+ {{{09-21-2016,09-22-2016,NULL},{NULL,10-21-2016,10-22-2016}},{{NULL,11-21-2016,10-21-2016},{09-21-2015,09-22-2015,09-21-2014}}}
+(1 row)
+
+CREATE FUNCTION test_type_conversion_array_timestamp(x timestamp[]) RETURNS timestamp[] AS $$
+plpy.info(x, type(x))
+return x
+$$ LANGUAGE plpythonu;
+SELECT * FROM test_type_conversion_array_timestamp(ARRAY[[['2016-09-21 15:34:24.078792-04','2016-10-22 11:34:24.078795-04',NULL],
+ [NULL,'2016-10-21 11:34:25.078792-04','2016-10-21 11:34:24.098792-04']],
+ [[NULL,'2016-01-21 11:34:24.078792-04','2016-11-21 11:34:24.108792-04'],
+ ['2015-09-21 11:34:24.079792-04','2014-09-21 11:34:24.078792-04','2013-09-21 11:34:24.078792-04']]]::timestamp[]);
+INFO: ([[['Wed Sep 21 15:34:24.078792 2016', 'Sat Oct 22 11:34:24.078795 2016', None], [None, 'Fri Oct 21 11:34:25.078792 2016', 'Fri Oct 21 11:34:24.098792 2016']], [[None, 'Thu Jan 21 11:34:24.078792 2016', 'Mon Nov 21 11:34:24.108792 2016'], ['Mon Sep 21 11:34:24.079792 2015', 'Sun Sep 21 11:34:24.078792 2014', 'Sat Sep 21 11:34:24.078792 2013']]], <type 'list'>)
+ test_type_conversion_array_timestamp
+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
+ {{{"Wed Sep 21 15:34:24.078792 2016","Sat Oct 22 11:34:24.078795 2016",NULL},{NULL,"Fri Oct 21 11:34:25.078792 2016","Fri Oct 21 11:34:24.098792 2016"}},{{NULL,"Thu Jan 21 11:34:24.078792 2016","Mon Nov 21 11:34:24.108792 2016"},{"Mon Sep 21 11:34:24.079792 2015","Sun Sep 21 11:34:24.078792 2014","Sat Sep 21 11:34:24.078792 2013"}}}
+(1 row)
+
+CREATE OR REPLACE FUNCTION pyreturnmultidemint4(h int4, i int4, j int4, k int4 ) RETURNS int4[] AS $BODY$
+m = [[[[x for x in range(h)] for y in range(i)] for z in range(j)] for w in range(k)]
+plpy.info(m, type(m))
+return m
+$BODY$ LANGUAGE plpythonu;
+select pyreturnmultidemint4(8,5,3,2);
+INFO: ([[[[0, 1, 2, 3, 4, 5, 6, 7], [0, 1, 2, 3, 4, 5, 6, 7], [0, 1, 2, 3, 4, 5, 6, 7], [0, 1, 2, 3, 4, 5, 6, 7], [0, 1, 2, 3, 4, 5, 6, 7]], [[0, 1, 2, 3, 4, 5, 6, 7], [0, 1, 2, 3, 4, 5, 6, 7], [0, 1, 2, 3, 4, 5, 6, 7], [0, 1, 2, 3, 4, 5, 6, 7], [0, 1, 2, 3, 4, 5, 6, 7]], [[0, 1, 2, 3, 4, 5, 6, 7], [0, 1, 2, 3, 4, 5, 6, 7], [0, 1, 2, 3, 4, 5, 6, 7], [0, 1, 2, 3, 4, 5, 6, 7], [0, 1, 2, 3, 4, 5, 6, 7]]], [[[0, 1, 2, 3, 4, 5, 6, 7], [0, 1, 2, 3, 4, 5, 6, 7], [0, 1, 2, 3, 4, 5, 6, 7], [0, 1, 2, 3, 4, 5, 6, 7], [0, 1, 2, 3, 4, 5, 6, 7]], [[0, 1, 2, 3, 4, 5, 6, 7], [0, 1, 2, 3, 4, 5, 6, 7], [0, 1, 2, 3, 4, 5, 6, 7], [0, 1, 2, 3, 4, 5, 6, 7], [0, 1, 2, 3, 4, 5, 6, 7]], [[0, 1, 2, 3, 4, 5, 6, 7], [0, 1, 2, 3, 4, 5, 6, 7], [0, 1, 2, 3, 4, 5, 6, 7], [0, 1, 2, 3, 4, 5, 6, 7], [0, 1, 2, 3, 4, 5, 6, 7]]]], <type 'list'>)
+ pyreturnmultidemint4
+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
+ {{{{0,1,2,3,4,5,6,7},{0,1,2,3,4,5,6,7},{0,1,2,3,4,5,6,7},{0,1,2,3,4,5,6,7},{0,1,2,3,4,5,6,7}},{{0,1,2,3,4,5,6,7},{0,1,2,3,4,5,6,7},{0,1,2,3,4,5,6,7},{0,1,2,3,4,5,6,7},{0,1,2,3,4,5,6,7}},{{0,1,2,3,4,5,6,7},{0,1,2,3,4,5,6,7},{0,1,2,3,4,5,6,7},{0,1,2,3,4,5,6,7},{0,1,2,3,4,5,6,7}}},{{{0,1,2,3,4,5,6,7},{0,1,2,3,4,5,6,7},{0,1,2,3,4,5,6,7},{0,1,2,3,4,5,6,7},{0,1,2,3,4,5,6,7}},{{0,1,2,3,4,5,6,7},{0,1,2,3,4,5,6,7},{0,1,2,3,4,5,6,7},{0,1,2,3,4,5,6,7},{0,1,2,3,4,5,6,7}},{{0,1,2,3,4,5,6,7},{0,1,2,3,4,5,6,7},{0,1,2,3,4,5,6,7},{0,1,2,3,4,5,6,7},{0,1,2,3,4,5,6,7}}}}
+(1 row)
+
+CREATE OR REPLACE FUNCTION pyreturnmultidemint8(h int4, i int4, j int4, k int4 ) RETURNS int8[] AS $BODY$
+m = [[[[x for x in range(h)] for y in range(i)] for z in range(j)] for w in range(k)]
+plpy.info(m, type(m))
+return m
+$BODY$ LANGUAGE plpythonu;
+select pyreturnmultidemint8(5,5,3,2);
+INFO: ([[[[0, 1, 2, 3, 4], [0, 1, 2, 3, 4], [0, 1, 2, 3, 4], [0, 1, 2, 3, 4], [0, 1, 2, 3, 4]], [[0, 1, 2, 3, 4], [0, 1, 2, 3, 4], [0, 1, 2, 3, 4], [0, 1, 2, 3, 4], [0, 1, 2, 3, 4]], [[0, 1, 2, 3, 4], [0, 1, 2, 3, 4], [0, 1, 2, 3, 4], [0, 1, 2, 3, 4], [0, 1, 2, 3, 4]]], [[[0, 1, 2, 3, 4], [0, 1, 2, 3, 4], [0, 1, 2, 3, 4], [0, 1, 2, 3, 4], [0, 1, 2, 3, 4]], [[0, 1, 2, 3, 4], [0, 1, 2, 3, 4], [0, 1, 2, 3, 4], [0, 1, 2, 3, 4], [0, 1, 2, 3, 4]], [[0, 1, 2, 3, 4], [0, 1, 2, 3, 4], [0, 1, 2, 3, 4], [0, 1, 2, 3, 4], [0, 1, 2, 3, 4]]]], <type 'list'>)
+ pyreturnmultidemint8
+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
+ {{{{0,1,2,3,4},{0,1,2,3,4},{0,1,2,3,4},{0,1,2,3,4},{0,1,2,3,4}},{{0,1,2,3,4},{0,1,2,3,4},{0,1,2,3,4},{0,1,2,3,4},{0,1,2,3,4}},{{0,1,2,3,4},{0,1,2,3,4},{0,1,2,3,4},{0,1,2,3,4},{0,1,2,3,4}}},{{{0,1,2,3,4},{0,1,2,3,4},{0,1,2,3,4},{0,1,2,3,4},{0,1,2,3,4}},{{0,1,2,3,4},{0,1,2,3,4},{0,1,2,3,4},{0,1,2,3,4},{0,1,2,3,4}},{{0,1,2,3,4},{0,1,2,3,4},{0,1,2,3,4},{0,1,2,3,4},{0,1,2,3,4}}}}
+(1 row)
+
+CREATE OR REPLACE FUNCTION pyreturnmultidemfloat4(h int4, i int4, j int4, k int4 ) RETURNS float4[] AS $BODY$
+m = [[[[x for x in range(h)] for y in range(i)] for z in range(j)] for w in range(k)]
+plpy.info(m, type(m))
+return m
+$BODY$ LANGUAGE plpythonu;
+select pyreturnmultidemfloat4(6,5,3,2);
+INFO: ([[[[0, 1, 2, 3, 4, 5], [0, 1, 2, 3, 4, 5], [0, 1, 2, 3, 4, 5], [0, 1, 2, 3, 4, 5], [0, 1, 2, 3, 4, 5]], [[0, 1, 2, 3, 4, 5], [0, 1, 2, 3, 4, 5], [0, 1, 2, 3, 4, 5], [0, 1, 2, 3, 4, 5], [0, 1, 2, 3, 4, 5]], [[0, 1, 2, 3, 4, 5], [0, 1, 2, 3, 4, 5], [0, 1, 2, 3, 4, 5], [0, 1, 2, 3, 4, 5], [0, 1, 2, 3, 4, 5]]], [[[0, 1, 2, 3, 4, 5], [0, 1, 2, 3, 4, 5], [0, 1, 2, 3, 4, 5], [0, 1, 2, 3, 4, 5], [0, 1, 2, 3, 4, 5]], [[0, 1, 2, 3, 4, 5], [0, 1, 2, 3, 4, 5], [0, 1, 2, 3, 4, 5], [0, 1, 2, 3, 4, 5], [0, 1, 2, 3, 4, 5]], [[0, 1, 2, 3, 4, 5], [0, 1, 2, 3, 4, 5], [0, 1, 2, 3, 4, 5], [0, 1, 2, 3, 4, 5], [0, 1, 2, 3, 4, 5]]]], <type 'list'>)
+ pyreturnmultidemfloat4
+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
+ {{{{0,1,2,3,4,5},{0,1,2,3,4,5},{0,1,2,3,4,5},{0,1,2,3,4,5},{0,1,2,3,4,5}},{{0,1,2,3,4,5},{0,1,2,3,4,5},{0,1,2,3,4,5},{0,1,2,3,4,5},{0,1,2,3,4,5}},{{0,1,2,3,4,5},{0,1,2,3,4,5},{0,1,2,3,4,5},{0,1,2,3,4,5},{0,1,2,3,4,5}}},{{{0,1,2,3,4,5},{0,1,2,3,4,5},{0,1,2,3,4,5},{0,1,2,3,4,5},{0,1,2,3,4,5}},{{0,1,2,3,4,5},{0,1,2,3,4,5},{0,1,2,3,4,5},{0,1,2,3,4,5},{0,1,2,3,4,5}},{{0,1,2,3,4,5},{0,1,2,3,4,5},{0,1,2,3,4,5},{0,1,2,3,4,5},{0,1,2,3,4,5}}}}
+(1 row)
+
+CREATE OR REPLACE FUNCTION pyreturnmultidemfloat8(h int4, i int4, j int4, k int4 ) RETURNS float8[] AS $BODY$
+m = [[[[x for x in range(h)] for y in range(i)] for z in range(j)] for w in range(k)]
+plpy.info(m, type(m))
+return m
+$BODY$ LANGUAGE plpythonu;
+select pyreturnmultidemfloat8(7,5,3,2);
+INFO: ([[[[0, 1, 2, 3, 4, 5, 6], [0, 1, 2, 3, 4, 5, 6], [0, 1, 2, 3, 4, 5, 6], [0, 1, 2, 3, 4, 5, 6], [0, 1, 2, 3, 4, 5, 6]], [[0, 1, 2, 3, 4, 5, 6], [0, 1, 2, 3, 4, 5, 6], [0, 1, 2, 3, 4, 5, 6], [0, 1, 2, 3, 4, 5, 6], [0, 1, 2, 3, 4, 5, 6]], [[0, 1, 2, 3, 4, 5, 6], [0, 1, 2, 3, 4, 5, 6], [0, 1, 2, 3, 4, 5, 6], [0, 1, 2, 3, 4, 5, 6], [0, 1, 2, 3, 4, 5, 6]]], [[[0, 1, 2, 3, 4, 5, 6], [0, 1, 2, 3, 4, 5, 6], [0, 1, 2, 3, 4, 5, 6], [0, 1, 2, 3, 4, 5, 6], [0, 1, 2, 3, 4, 5, 6]], [[0, 1, 2, 3, 4, 5, 6], [0, 1, 2, 3, 4, 5, 6], [0, 1, 2, 3, 4, 5, 6], [0, 1, 2, 3, 4, 5, 6], [0, 1, 2, 3, 4, 5, 6]], [[0, 1, 2, 3, 4, 5, 6], [0, 1, 2, 3, 4, 5, 6], [0, 1, 2, 3, 4, 5, 6], [0, 1, 2, 3, 4, 5, 6], [0, 1, 2, 3, 4, 5, 6]]]], <type 'list'>)
+ pyreturnmultidemfloat8
+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
+ {{{{0,1,2,3,4,5,6},{0,1,2,3,4,5,6},{0,1,2,3,4,5,6},{0,1,2,3,4,5,6},{0,1,2,3,4,5,6}},{{0,1,2,3,4,5,6},{0,1,2,3,4,5,6},{0,1,2,3,4,5,6},{0,1,2,3,4,5,6},{0,1,2,3,4,5,6}},{{0,1,2,3,4,5,6},{0,1,2,3,4,5,6},{0,1,2,3,4,5,6},{0,1,2,3,4,5,6},{0,1,2,3,4,5,6}}},{{{0,1,2,3,4,5,6},{0,1,2,3,4,5,6},{0,1,2,3,4,5,6},{0,1,2,3,4,5,6},{0,1,2,3,4,5,6}},{{0,1,2,3,4,5,6},{0,1,2,3,4,5,6},{0,1,2,3,4,5,6},{0,1,2,3,4,5,6},{0,1,2,3,4,5,6}},{{0,1,2,3,4,5,6},{0,1,2,3,4,5,6},{0,1,2,3,4,5,6},{0,1,2,3,4,5,6},{0,1,2,3,4,5,6}}}}
+(1 row)
+
CREATE FUNCTION test_type_conversion_array_text(x text[]) RETURNS text[] AS $$
plpy.info(x, type(x))
return x
@@ -551,6 +675,13 @@ INFO: (['foo', 'bar'], <type 'list'>)
{foo,bar}
(1 row)
+SELECT * FROM test_type_conversion_array_text(ARRAY[['foo', 'bar'],['foo2', 'bar2']]);
+INFO: ([['foo', 'bar'], ['foo2', 'bar2']], <type 'list'>)
+ test_type_conversion_array_text
+---------------------------------
+ {{foo,bar},{foo2,bar2}}
+(1 row)
+
CREATE FUNCTION test_type_conversion_array_bytea(x bytea[]) RETURNS bytea[] AS $$
plpy.info(x, type(x))
return x
@@ -578,6 +709,20 @@ SELECT * FROM test_type_conversion_array_mixed2();
ERROR: invalid input syntax for integer: "abc"
CONTEXT: while creating return value
PL/Python function "test_type_conversion_array_mixed2"
+CREATE FUNCTION test_type_conversion_mdarray_malformed() RETURNS int[] AS $$
+return [[1,2,3],[4,5]]
+$$ LANGUAGE plpythonu;
+SELECT * FROM test_type_conversion_mdarray_malformed();
+ERROR: multidimensional arrays must have array expressions with matching dimensions. PL/Python function return value has sequence length 2 while expected 3
+CONTEXT: while creating return value
+PL/Python function "test_type_conversion_mdarray_malformed"
+CREATE FUNCTION test_type_conversion_mdarray_toodeep() RETURNS int[] AS $$
+return [[[[[[[1]]]]]]]
+$$ LANGUAGE plpythonu;
+SELECT * FROM test_type_conversion_mdarray_toodeep();
+ERROR: number of array dimensions (7) exceeds the maximum allowed (6)
+CONTEXT: while creating return value
+PL/Python function "test_type_conversion_mdarray_toodeep"
CREATE FUNCTION test_type_conversion_array_record() RETURNS type_record[] AS $$
return [{'first': 'one', 'second': 42}, {'first': 'two', 'second': 11}]
$$ LANGUAGE plpythonu;
diff --git a/src/pl/plpython/expected/plpython_types_3.out b/src/pl/plpython/expected/plpython_types_3.out
index 56b78e1..f7d357d 100644
--- a/src/pl/plpython/expected/plpython_types_3.out
+++ b/src/pl/plpython/expected/plpython_types_3.out
@@ -537,9 +537,133 @@ INFO: (None, <class 'NoneType'>)
(1 row)
SELECT * FROM test_type_conversion_array_int4(ARRAY[[1,2,3],[4,5,6]]);
-ERROR: cannot convert multidimensional array to Python list
-DETAIL: PL/Python only supports one-dimensional arrays.
-CONTEXT: PL/Python function "test_type_conversion_array_int4"
+INFO: ([[1, 2, 3], [4, 5, 6]], <class 'list'>)
+ test_type_conversion_array_int4
+---------------------------------
+ {{1,2,3},{4,5,6}}
+(1 row)
+
+SELECT * FROM test_type_conversion_array_int4(ARRAY[[[1,2,NULL],[NULL,5,6]],[[NULL,8,9],[10,11,12]]]);
+INFO: ([[[1, 2, None], [None, 5, 6]], [[None, 8, 9], [10, 11, 12]]], <class 'list'>)
+ test_type_conversion_array_int4
+---------------------------------------------------
+ {{{1,2,NULL},{NULL,5,6}},{{NULL,8,9},{10,11,12}}}
+(1 row)
+
+SELECT * FROM test_type_conversion_array_int4('[2:4]={1,2,3}');
+INFO: ([1, 2, 3], <class 'list'>)
+ test_type_conversion_array_int4
+---------------------------------
+ {1,2,3}
+(1 row)
+
+CREATE FUNCTION test_type_conversion_array_int8(x int8[]) RETURNS int8[] AS $$
+plpy.info(x, type(x))
+return x
+$$ LANGUAGE plpython3u;
+SELECT * FROM test_type_conversion_array_int8(ARRAY[[[1,2,NULL],[NULL,5,6]],[[NULL,8,9],[10,11,12]]]::int8[]);
+INFO: ([[[1, 2, None], [None, 5, 6]], [[None, 8, 9], [10, 11, 12]]], <class 'list'>)
+ test_type_conversion_array_int8
+---------------------------------------------------
+ {{{1,2,NULL},{NULL,5,6}},{{NULL,8,9},{10,11,12}}}
+(1 row)
+
+CREATE FUNCTION test_type_conversion_array_float4(x float4[]) RETURNS float4[] AS $$
+plpy.info(x, type(x))
+return x
+$$ LANGUAGE plpython3u;
+SELECT * FROM test_type_conversion_array_float4(ARRAY[[[1.2,2.3,NULL],[NULL,5.7,6.8]],[[NULL,8.9,9.345],[10.123,11.456,12.6768]]]::float4[]);
+INFO: ([[[1.2000000476837158, 2.299999952316284, None], [None, 5.699999809265137, 6.800000190734863]], [[None, 8.899999618530273, 9.345000267028809], [10.123000144958496, 11.456000328063965, 12.676799774169922]]], <class 'list'>)
+ test_type_conversion_array_float4
+------------------------------------------------------------------------------
+ {{{1.2,2.3,NULL},{NULL,5.7,6.8}},{{NULL,8.9,9.345},{10.123,11.456,12.6768}}}
+(1 row)
+
+CREATE FUNCTION test_type_conversion_array_float8(x float8[]) RETURNS float8[] AS $$
+plpy.info(x, type(x))
+return x
+$$ LANGUAGE plpython3u;
+SELECT * FROM test_type_conversion_array_float8(ARRAY[[[1.2,2.3,NULL],[NULL,5.7,6.8]],[[NULL,8.9,9.345],[10.123,11.456,12.6768]]]::float8[]);
+INFO: ([[[1.2, 2.3, None], [None, 5.7, 6.8]], [[None, 8.9, 9.345], [10.123, 11.456, 12.6768]]], <class 'list'>)
+ test_type_conversion_array_float8
+------------------------------------------------------------------------------
+ {{{1.2,2.3,NULL},{NULL,5.7,6.8}},{{NULL,8.9,9.345},{10.123,11.456,12.6768}}}
+(1 row)
+
+CREATE FUNCTION test_type_conversion_array_date(x date[]) RETURNS date[] AS $$
+plpy.info(x, type(x))
+return x
+$$ LANGUAGE plpython3u;
+SELECT * FROM test_type_conversion_array_date(ARRAY[[['2016-09-21','2016-09-22',NULL],[NULL,'2016-10-21','2016-10-22']],
+ [[NULL,'2016-11-21','2016-10-21'],['2015-09-21','2015-09-22','2014-09-21']]]::date[]);
+INFO: ([[['09-21-2016', '09-22-2016', None], [None, '10-21-2016', '10-22-2016']], [[None, '11-21-2016', '10-21-2016'], ['09-21-2015', '09-22-2015', '09-21-2014']]], <class 'list'>)
+ test_type_conversion_array_date
+---------------------------------------------------------------------------------------------------------------------------------
+ {{{09-21-2016,09-22-2016,NULL},{NULL,10-21-2016,10-22-2016}},{{NULL,11-21-2016,10-21-2016},{09-21-2015,09-22-2015,09-21-2014}}}
+(1 row)
+
+CREATE FUNCTION test_type_conversion_array_timestamp(x timestamp[]) RETURNS timestamp[] AS $$
+plpy.info(x, type(x))
+return x
+$$ LANGUAGE plpython3u;
+SELECT * FROM test_type_conversion_array_timestamp(ARRAY[[['2016-09-21 15:34:24.078792-04','2016-10-22 11:34:24.078795-04',NULL],
+ [NULL,'2016-10-21 11:34:25.078792-04','2016-10-21 11:34:24.098792-04']],
+ [[NULL,'2016-01-21 11:34:24.078792-04','2016-11-21 11:34:24.108792-04'],
+ ['2015-09-21 11:34:24.079792-04','2014-09-21 11:34:24.078792-04','2013-09-21 11:34:24.078792-04']]]::timestamp[]);
+INFO: ([[['Wed Sep 21 15:34:24.078792 2016', 'Sat Oct 22 11:34:24.078795 2016', None], [None, 'Fri Oct 21 11:34:25.078792 2016', 'Fri Oct 21 11:34:24.098792 2016']], [[None, 'Thu Jan 21 11:34:24.078792 2016', 'Mon Nov 21 11:34:24.108792 2016'], ['Mon Sep 21 11:34:24.079792 2015', 'Sun Sep 21 11:34:24.078792 2014', 'Sat Sep 21 11:34:24.078792 2013']]], <class 'list'>)
+ test_type_conversion_array_timestamp
+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
+ {{{"Wed Sep 21 15:34:24.078792 2016","Sat Oct 22 11:34:24.078795 2016",NULL},{NULL,"Fri Oct 21 11:34:25.078792 2016","Fri Oct 21 11:34:24.098792 2016"}},{{NULL,"Thu Jan 21 11:34:24.078792 2016","Mon Nov 21 11:34:24.108792 2016"},{"Mon Sep 21 11:34:24.079792 2015","Sun Sep 21 11:34:24.078792 2014","Sat Sep 21 11:34:24.078792 2013"}}}
+(1 row)
+
+CREATE OR REPLACE FUNCTION pyreturnmultidemint4(h int4, i int4, j int4, k int4 ) RETURNS int4[] AS $BODY$
+m = [[[[x for x in range(h)] for y in range(i)] for z in range(j)] for w in range(k)]
+plpy.info(m, type(m))
+return m
+$BODY$ LANGUAGE plpython3u;
+select pyreturnmultidemint4(8,5,3,2);
+INFO: ([[[[0, 1, 2, 3, 4, 5, 6, 7], [0, 1, 2, 3, 4, 5, 6, 7], [0, 1, 2, 3, 4, 5, 6, 7], [0, 1, 2, 3, 4, 5, 6, 7], [0, 1, 2, 3, 4, 5, 6, 7]], [[0, 1, 2, 3, 4, 5, 6, 7], [0, 1, 2, 3, 4, 5, 6, 7], [0, 1, 2, 3, 4, 5, 6, 7], [0, 1, 2, 3, 4, 5, 6, 7], [0, 1, 2, 3, 4, 5, 6, 7]], [[0, 1, 2, 3, 4, 5, 6, 7], [0, 1, 2, 3, 4, 5, 6, 7], [0, 1, 2, 3, 4, 5, 6, 7], [0, 1, 2, 3, 4, 5, 6, 7], [0, 1, 2, 3, 4, 5, 6, 7]]], [[[0, 1, 2, 3, 4, 5, 6, 7], [0, 1, 2, 3, 4, 5, 6, 7], [0, 1, 2, 3, 4, 5, 6, 7], [0, 1, 2, 3, 4, 5, 6, 7], [0, 1, 2, 3, 4, 5, 6, 7]], [[0, 1, 2, 3, 4, 5, 6, 7], [0, 1, 2, 3, 4, 5, 6, 7], [0, 1, 2, 3, 4, 5, 6, 7], [0, 1, 2, 3, 4, 5, 6, 7], [0, 1, 2, 3, 4, 5, 6, 7]], [[0, 1, 2, 3, 4, 5, 6, 7], [0, 1, 2, 3, 4, 5, 6, 7], [0, 1, 2, 3, 4, 5, 6, 7], [0, 1, 2, 3, 4, 5, 6, 7], [0, 1, 2, 3, 4, 5, 6, 7]]]], <class 'list'>)
+ pyreturnmultidemint4
+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
+ {{{{0,1,2,3,4,5,6,7},{0,1,2,3,4,5,6,7},{0,1,2,3,4,5,6,7},{0,1,2,3,4,5,6,7},{0,1,2,3,4,5,6,7}},{{0,1,2,3,4,5,6,7},{0,1,2,3,4,5,6,7},{0,1,2,3,4,5,6,7},{0,1,2,3,4,5,6,7},{0,1,2,3,4,5,6,7}},{{0,1,2,3,4,5,6,7},{0,1,2,3,4,5,6,7},{0,1,2,3,4,5,6,7},{0,1,2,3,4,5,6,7},{0,1,2,3,4,5,6,7}}},{{{0,1,2,3,4,5,6,7},{0,1,2,3,4,5,6,7},{0,1,2,3,4,5,6,7},{0,1,2,3,4,5,6,7},{0,1,2,3,4,5,6,7}},{{0,1,2,3,4,5,6,7},{0,1,2,3,4,5,6,7},{0,1,2,3,4,5,6,7},{0,1,2,3,4,5,6,7},{0,1,2,3,4,5,6,7}},{{0,1,2,3,4,5,6,7},{0,1,2,3,4,5,6,7},{0,1,2,3,4,5,6,7},{0,1,2,3,4,5,6,7},{0,1,2,3,4,5,6,7}}}}
+(1 row)
+
+CREATE OR REPLACE FUNCTION pyreturnmultidemint8(h int4, i int4, j int4, k int4 ) RETURNS int8[] AS $BODY$
+m = [[[[x for x in range(h)] for y in range(i)] for z in range(j)] for w in range(k)]
+plpy.info(m, type(m))
+return m
+$BODY$ LANGUAGE plpython3u;
+select pyreturnmultidemint8(5,5,3,2);
+INFO: ([[[[0, 1, 2, 3, 4], [0, 1, 2, 3, 4], [0, 1, 2, 3, 4], [0, 1, 2, 3, 4], [0, 1, 2, 3, 4]], [[0, 1, 2, 3, 4], [0, 1, 2, 3, 4], [0, 1, 2, 3, 4], [0, 1, 2, 3, 4], [0, 1, 2, 3, 4]], [[0, 1, 2, 3, 4], [0, 1, 2, 3, 4], [0, 1, 2, 3, 4], [0, 1, 2, 3, 4], [0, 1, 2, 3, 4]]], [[[0, 1, 2, 3, 4], [0, 1, 2, 3, 4], [0, 1, 2, 3, 4], [0, 1, 2, 3, 4], [0, 1, 2, 3, 4]], [[0, 1, 2, 3, 4], [0, 1, 2, 3, 4], [0, 1, 2, 3, 4], [0, 1, 2, 3, 4], [0, 1, 2, 3, 4]], [[0, 1, 2, 3, 4], [0, 1, 2, 3, 4], [0, 1, 2, 3, 4], [0, 1, 2, 3, 4], [0, 1, 2, 3, 4]]]], <class 'list'>)
+ pyreturnmultidemint8
+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
+ {{{{0,1,2,3,4},{0,1,2,3,4},{0,1,2,3,4},{0,1,2,3,4},{0,1,2,3,4}},{{0,1,2,3,4},{0,1,2,3,4},{0,1,2,3,4},{0,1,2,3,4},{0,1,2,3,4}},{{0,1,2,3,4},{0,1,2,3,4},{0,1,2,3,4},{0,1,2,3,4},{0,1,2,3,4}}},{{{0,1,2,3,4},{0,1,2,3,4},{0,1,2,3,4},{0,1,2,3,4},{0,1,2,3,4}},{{0,1,2,3,4},{0,1,2,3,4},{0,1,2,3,4},{0,1,2,3,4},{0,1,2,3,4}},{{0,1,2,3,4},{0,1,2,3,4},{0,1,2,3,4},{0,1,2,3,4},{0,1,2,3,4}}}}
+(1 row)
+
+CREATE OR REPLACE FUNCTION pyreturnmultidemfloat4(h int4, i int4, j int4, k int4 ) RETURNS float4[] AS $BODY$
+m = [[[[x for x in range(h)] for y in range(i)] for z in range(j)] for w in range(k)]
+plpy.info(m, type(m))
+return m
+$BODY$ LANGUAGE plpython3u;
+select pyreturnmultidemfloat4(6,5,3,2);
+INFO: ([[[[0, 1, 2, 3, 4, 5], [0, 1, 2, 3, 4, 5], [0, 1, 2, 3, 4, 5], [0, 1, 2, 3, 4, 5], [0, 1, 2, 3, 4, 5]], [[0, 1, 2, 3, 4, 5], [0, 1, 2, 3, 4, 5], [0, 1, 2, 3, 4, 5], [0, 1, 2, 3, 4, 5], [0, 1, 2, 3, 4, 5]], [[0, 1, 2, 3, 4, 5], [0, 1, 2, 3, 4, 5], [0, 1, 2, 3, 4, 5], [0, 1, 2, 3, 4, 5], [0, 1, 2, 3, 4, 5]]], [[[0, 1, 2, 3, 4, 5], [0, 1, 2, 3, 4, 5], [0, 1, 2, 3, 4, 5], [0, 1, 2, 3, 4, 5], [0, 1, 2, 3, 4, 5]], [[0, 1, 2, 3, 4, 5], [0, 1, 2, 3, 4, 5], [0, 1, 2, 3, 4, 5], [0, 1, 2, 3, 4, 5], [0, 1, 2, 3, 4, 5]], [[0, 1, 2, 3, 4, 5], [0, 1, 2, 3, 4, 5], [0, 1, 2, 3, 4, 5], [0, 1, 2, 3, 4, 5], [0, 1, 2, 3, 4, 5]]]], <class 'list'>)
+ pyreturnmultidemfloat4
+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
+ {{{{0,1,2,3,4,5},{0,1,2,3,4,5},{0,1,2,3,4,5},{0,1,2,3,4,5},{0,1,2,3,4,5}},{{0,1,2,3,4,5},{0,1,2,3,4,5},{0,1,2,3,4,5},{0,1,2,3,4,5},{0,1,2,3,4,5}},{{0,1,2,3,4,5},{0,1,2,3,4,5},{0,1,2,3,4,5},{0,1,2,3,4,5},{0,1,2,3,4,5}}},{{{0,1,2,3,4,5},{0,1,2,3,4,5},{0,1,2,3,4,5},{0,1,2,3,4,5},{0,1,2,3,4,5}},{{0,1,2,3,4,5},{0,1,2,3,4,5},{0,1,2,3,4,5},{0,1,2,3,4,5},{0,1,2,3,4,5}},{{0,1,2,3,4,5},{0,1,2,3,4,5},{0,1,2,3,4,5},{0,1,2,3,4,5},{0,1,2,3,4,5}}}}
+(1 row)
+
+CREATE OR REPLACE FUNCTION pyreturnmultidemfloat8(h int4, i int4, j int4, k int4 ) RETURNS float8[] AS $BODY$
+m = [[[[x for x in range(h)] for y in range(i)] for z in range(j)] for w in range(k)]
+plpy.info(m, type(m))
+return m
+$BODY$ LANGUAGE plpython3u;
+select pyreturnmultidemfloat8(7,5,3,2);
+INFO: ([[[[0, 1, 2, 3, 4, 5, 6], [0, 1, 2, 3, 4, 5, 6], [0, 1, 2, 3, 4, 5, 6], [0, 1, 2, 3, 4, 5, 6], [0, 1, 2, 3, 4, 5, 6]], [[0, 1, 2, 3, 4, 5, 6], [0, 1, 2, 3, 4, 5, 6], [0, 1, 2, 3, 4, 5, 6], [0, 1, 2, 3, 4, 5, 6], [0, 1, 2, 3, 4, 5, 6]], [[0, 1, 2, 3, 4, 5, 6], [0, 1, 2, 3, 4, 5, 6], [0, 1, 2, 3, 4, 5, 6], [0, 1, 2, 3, 4, 5, 6], [0, 1, 2, 3, 4, 5, 6]]], [[[0, 1, 2, 3, 4, 5, 6], [0, 1, 2, 3, 4, 5, 6], [0, 1, 2, 3, 4, 5, 6], [0, 1, 2, 3, 4, 5, 6], [0, 1, 2, 3, 4, 5, 6]], [[0, 1, 2, 3, 4, 5, 6], [0, 1, 2, 3, 4, 5, 6], [0, 1, 2, 3, 4, 5, 6], [0, 1, 2, 3, 4, 5, 6], [0, 1, 2, 3, 4, 5, 6]], [[0, 1, 2, 3, 4, 5, 6], [0, 1, 2, 3, 4, 5, 6], [0, 1, 2, 3, 4, 5, 6], [0, 1, 2, 3, 4, 5, 6], [0, 1, 2, 3, 4, 5, 6]]]], <class 'list'>)
+ pyreturnmultidemfloat8
+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
+ {{{{0,1,2,3,4,5,6},{0,1,2,3,4,5,6},{0,1,2,3,4,5,6},{0,1,2,3,4,5,6},{0,1,2,3,4,5,6}},{{0,1,2,3,4,5,6},{0,1,2,3,4,5,6},{0,1,2,3,4,5,6},{0,1,2,3,4,5,6},{0,1,2,3,4,5,6}},{{0,1,2,3,4,5,6},{0,1,2,3,4,5,6},{0,1,2,3,4,5,6},{0,1,2,3,4,5,6},{0,1,2,3,4,5,6}}},{{{0,1,2,3,4,5,6},{0,1,2,3,4,5,6},{0,1,2,3,4,5,6},{0,1,2,3,4,5,6},{0,1,2,3,4,5,6}},{{0,1,2,3,4,5,6},{0,1,2,3,4,5,6},{0,1,2,3,4,5,6},{0,1,2,3,4,5,6},{0,1,2,3,4,5,6}},{{0,1,2,3,4,5,6},{0,1,2,3,4,5,6},{0,1,2,3,4,5,6},{0,1,2,3,4,5,6},{0,1,2,3,4,5,6}}}}
+(1 row)
+
CREATE FUNCTION test_type_conversion_array_text(x text[]) RETURNS text[] AS $$
plpy.info(x, type(x))
return x
@@ -551,6 +675,13 @@ INFO: (['foo', 'bar'], <class 'list'>)
{foo,bar}
(1 row)
+SELECT * FROM test_type_conversion_array_text(ARRAY[['foo', 'bar'],['foo2', 'bar2']]);
+INFO: ([['foo', 'bar'], ['foo2', 'bar2']], <class 'list'>)
+ test_type_conversion_array_text
+---------------------------------
+ {{foo,bar},{foo2,bar2}}
+(1 row)
+
CREATE FUNCTION test_type_conversion_array_bytea(x bytea[]) RETURNS bytea[] AS $$
plpy.info(x, type(x))
return x
@@ -578,6 +709,20 @@ SELECT * FROM test_type_conversion_array_mixed2();
ERROR: invalid input syntax for integer: "abc"
CONTEXT: while creating return value
PL/Python function "test_type_conversion_array_mixed2"
+CREATE FUNCTION test_type_conversion_mdarray_malformed() RETURNS int[] AS $$
+return [[1,2,3],[4,5]]
+$$ LANGUAGE plpython3u;
+SELECT * FROM test_type_conversion_mdarray_malformed();
+ERROR: multidimensional arrays must have array expressions with matching dimensions. PL/Python function return value has sequence length 2 while expected 3
+CONTEXT: while creating return value
+PL/Python function "test_type_conversion_mdarray_malformed"
+CREATE FUNCTION test_type_conversion_mdarray_toodeep() RETURNS int[] AS $$
+return [[[[[[[1]]]]]]]
+$$ LANGUAGE plpython3u;
+SELECT * FROM test_type_conversion_mdarray_toodeep();
+ERROR: number of array dimensions (7) exceeds the maximum allowed (6)
+CONTEXT: while creating return value
+PL/Python function "test_type_conversion_mdarray_toodeep"
CREATE FUNCTION test_type_conversion_array_record() RETURNS type_record[] AS $$
return [{'first': 'one', 'second': 42}, {'first': 'two', 'second': 11}]
$$ LANGUAGE plpython3u;
diff --git a/src/pl/plpython/plpy_typeio.c b/src/pl/plpython/plpy_typeio.c
index 70f2e6d..019fcb1 100644
--- a/src/pl/plpython/plpy_typeio.c
+++ b/src/pl/plpython/plpy_typeio.c
@@ -45,6 +45,8 @@ static PyObject *PLyBytes_FromBytea(PLyDatumToOb *arg, Datum d);
static PyObject *PLyString_FromDatum(PLyDatumToOb *arg, Datum d);
static PyObject *PLyObject_FromTransform(PLyDatumToOb *arg, Datum d);
static PyObject *PLyList_FromArray(PLyDatumToOb *arg, Datum d);
+static PyObject *PLyList_FromArray_recurse(PLyDatumToOb *elm, int *dims, int ndim, int dim,
+ char **dataptr_p, bits8 **bitmap_p, int *bitmask_p);
/* conversion from Python objects to Datums */
static Datum PLyObject_ToBool(PLyObToDatum *arg, int32 typmod, PyObject *plrv);
@@ -53,6 +55,9 @@ static Datum PLyObject_ToComposite(PLyObToDatum *arg, int32 typmod, PyObject *pl
static Datum PLyObject_ToDatum(PLyObToDatum *arg, int32 typmod, PyObject *plrv);
static Datum PLyObject_ToTransform(PLyObToDatum *arg, int32 typmod, PyObject *plrv);
static Datum PLySequence_ToArray(PLyObToDatum *arg, int32 typmod, PyObject *plrv);
+static void PLySequence_ToArray_recurse(PLyObToDatum *elm, PyObject *list,
+ int *dims, int ndim, int dim,
+ Datum *elems, bool *nulls, int *currelem);
/* conversion from Python objects to composite Datums (used by triggers and SRFs) */
static Datum PLyString_ToComposite(PLyTypeInfo *info, TupleDesc desc, PyObject *string);
@@ -631,43 +636,105 @@ PLyList_FromArray(PLyDatumToOb *arg, Datum d)
{
ArrayType *array = DatumGetArrayTypeP(d);
PLyDatumToOb *elm = arg->elm;
- PyObject *list;
- int length;
- int lbound;
- int i;
+ int ndim;
+ int *dims;
+ char *dataptr;
+ bits8 *bitmap;
+ int bitmask;
if (ARR_NDIM(array) == 0)
return PyList_New(0);
- if (ARR_NDIM(array) != 1)
- ereport(ERROR,
- (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
- errmsg("cannot convert multidimensional array to Python list"),
- errdetail("PL/Python only supports one-dimensional arrays.")));
+ /* Array dimensions and left bounds */
+ ndim = ARR_NDIM(array);
+ dims = ARR_DIMS(array);
+ Assert(ndim < MAXDIM);
- length = ARR_DIMS(array)[0];
- lbound = ARR_LBOUND(array)[0];
- list = PyList_New(length);
- if (list == NULL)
- PLy_elog(ERROR, "could not create new Python list");
+ /*
+ * We will iterate the Postgres array in the physical order its stored in
+ * the datum. For 3-dimensional array the order of iteration would be the
+ * following: first you start with [0,0,0] elements through [0,0,k], then
+ * [0,1,0] till [0,1,k] till [0,m,k], then [1,0,0] till [1,0,k] till
+ * [1,m,k], and so on.
+ *
+ * In Python, there is no multi-dimensional lists as such, but they are
+ * represented as a list of lists. So a 3-d array of [n,m,k] element is a
+ * list of n m-element arrays, each element of which is k-element array.
+ * PLyList_FromArray_recurse() builds the Python list for a single dimension,
+ * and recurses for the next inner dimension.
+ */
+ dataptr = ARR_DATA_PTR(array);
+ bitmap = ARR_NULLBITMAP(array);
+ bitmask = 1;
- for (i = 0; i < length; i++)
+ return PLyList_FromArray_recurse(elm, dims, ndim, 0,
+ &dataptr, &bitmap, &bitmask);
+}
+
+static PyObject *
+PLyList_FromArray_recurse(PLyDatumToOb *elm, int *dims, int ndim, int dim,
+ char **dataptr_p, bits8 **bitmap_p, int *bitmask_p)
+{
+ int i;
+ PyObject *list;
+
+ list = PyList_New(dims[dim]);
+
+ if (dim < ndim - 1)
{
- Datum elem;
- bool isnull;
- int offset;
-
- offset = lbound + i;
- elem = array_ref(array, 1, &offset, arg->typlen,
- elm->typlen, elm->typbyval, elm->typalign,
- &isnull);
- if (isnull)
+ /* Outer dimension. Recurse for each inner slice. */
+ for (i = 0; i < dims[dim]; i++)
{
- Py_INCREF(Py_None);
- PyList_SET_ITEM(list, i, Py_None);
+ PyObject *sublist;
+
+ sublist = PLyList_FromArray_recurse(elm, dims, ndim, dim + 1,
+ dataptr_p, bitmap_p, bitmask_p);
+ PyList_SET_ITEM(list, i, sublist);
}
- else
- PyList_SET_ITEM(list, i, elm->func(elm, elem));
+ }
+ else
+ {
+ /*
+ * Innermost dimension. Fill the list with the values from the array
+ * for this slice.
+ */
+ char *dataptr = *dataptr_p;
+ bits8 *bitmap = *bitmap_p;
+ int bitmask = *bitmask_p;
+
+ for (i = 0; i < dims[dim]; i++)
+ {
+ /* checking for NULL */
+ if (bitmap && (*bitmap & bitmask) == 0)
+ {
+ Py_INCREF(Py_None);
+ PyList_SET_ITEM(list, i, Py_None);
+ }
+ else
+ {
+ Datum itemvalue;
+
+ itemvalue = fetch_att(dataptr, elm->typbyval, elm->typlen);
+ PyList_SET_ITEM(list, i, elm->func(elm, itemvalue));
+ dataptr = att_addlength_pointer(dataptr, elm->typlen, dataptr);
+ dataptr = (char *) att_align_nominal(dataptr, elm->typalign);
+ }
+
+ /* advance bitmap pointer if any */
+ if (bitmap)
+ {
+ bitmask <<= 1;
+ if (bitmask == 0x100 /* (1<<8) */)
+ {
+ bitmap++;
+ bitmask = 1;
+ }
+ }
+ }
+
+ *dataptr_p = dataptr;
+ *bitmap_p = bitmap;
+ *bitmask_p = bitmask;
}
return list;
@@ -864,39 +931,100 @@ static Datum
PLySequence_ToArray(PLyObToDatum *arg, int32 typmod, PyObject *plrv)
{
ArrayType *array;
- Datum rv;
int i;
Datum *elems;
bool *nulls;
- int len;
- int lbs;
+ int64 len;
+ int ndim;
+ int dims[MAXDIM];
+ int lbs[MAXDIM];
+ int currelem;
+ Datum rv;
Assert(plrv != Py_None);
if (!PySequence_Check(plrv))
PLy_elog(ERROR, "return value of function with array return type is not a Python sequence");
- len = PySequence_Length(plrv);
- elems = palloc(sizeof(*elems) * len);
- nulls = palloc(sizeof(*nulls) * len);
-
- for (i = 0; i < len; i++)
+ /*
+ * Determine the number of dimensions, and their sizes.
+ *
+ * We don't want to create multi-dimensional arrays when we have an empty sequence,
+ * when we need to parse sequence of composite objects, or when we have strings.
+ */
+ if (type_is_rowtype(get_base_element_type(arg->typoid)) ||
+ PyString_Check(plrv) || PyBytes_Check(plrv) || PyUnicode_Check(plrv))
{
- PyObject *obj = PySequence_GetItem(plrv, i);
+ ndim = 1;
+ len = dims[0] = PySequence_Length(plrv);
+ }
+ else
+ {
+ PyObject *pyptr = plrv;
+ PyObject *next;
- if (obj == Py_None)
- nulls[i] = true;
- else
+ ndim = 0;
+ len = 1;
+
+ Py_INCREF(plrv);
+
+ for (;;)
{
- nulls[i] = false;
- elems[i] = arg->elm->func(arg->elm, -1, obj);
+ if (!PySequence_Check(pyptr))
+ break;
+
+ if (PyString_Check(pyptr) || PyBytes_Check(pyptr) || PyUnicode_Check(pyptr))
+ break;
+
+ dims[ndim] = PySequence_Length(pyptr);
+ if (dims[ndim] < 0)
+ PLy_elog(ERROR, "cannot determine sequence length for function return value");
+
+ if (dims[ndim] > MaxAllocSize)
+ PLy_elog(ERROR, "array size exceeds the maximum allowed (%d)", (int) MaxAllocSize);
+
+ len *= dims[ndim];
+ if (len > MaxAllocSize)
+ PLy_elog(ERROR, "array size exceeds the maximum allowed (%d)", (int) MaxAllocSize);
+
+ if (dims[ndim] == 0)
+ {
+ /* empty sequence */
+ break;
+ }
+
+ ndim++;
+
+ next = PySequence_GetItem(pyptr, 0);
+ Py_XDECREF(pyptr);
+ pyptr = next;
}
- Py_XDECREF(obj);
+ Py_XDECREF(pyptr);
}
- lbs = 1;
- array = construct_md_array(elems, nulls, 1, &len, &lbs,
- get_base_element_type(arg->typoid), arg->elm->typlen, arg->elm->typbyval, arg->elm->typalign);
+ /*
+ * Traverse the Python lists, in depth-first order, and collect all the
+ * elements at the bottom level into 'elems'/'nulls' arrays.
+ */
+ elems = palloc(sizeof(Datum) * len);
+ nulls = palloc(sizeof(bool) * len);
+ currelem = 0;
+ PLySequence_ToArray_recurse(arg->elm, plrv,
+ dims, ndim, 0,
+ elems, nulls, &currelem);
+
+ for (i = 0; i < ndim; i++)
+ lbs[i] = 1;
+
+ array = construct_md_array(elems,
+ nulls,
+ ndim,
+ dims,
+ lbs,
+ get_base_element_type(arg->typoid),
+ arg->elm->typlen,
+ arg->elm->typbyval,
+ arg->elm->typalign);
/*
* If the result type is a domain of array, the resulting array must be
@@ -908,6 +1036,58 @@ PLySequence_ToArray(PLyObToDatum *arg, int32 typmod, PyObject *plrv)
return rv;
}
+/*
+ * Helper function for PLySequence_ToArray. Traverse a Python list of lists in
+ * depth-first order, storing the elements in 'elems'.
+ */
+static void
+PLySequence_ToArray_recurse(PLyObToDatum *elm, PyObject *list,
+ int *dims, int ndim, int dim,
+ Datum *elems, bool *nulls, int *currelem)
+{
+ int i;
+
+ if (PySequence_Length(list) != dims[dim])
+ PLy_elog(ERROR,
+ "multidimensional arrays must have array expressions with matching dimensions. "
+ "PL/Python function return value has sequence length %d while expected %d",
+ (int) PySequence_Length(list), dims[dim]);
+
+ if (dim < ndim - 1)
+ {
+ for (i = 0; i < dims[dim]; i++)
+ {
+ PyObject *sublist = PySequence_GetItem(list, i);
+
+ PLySequence_ToArray_recurse(elm, sublist, dims, ndim, dim + 1,
+ elems, nulls, currelem);
+ Py_XDECREF(sublist);
+ }
+ }
+ else
+ {
+ for (i = 0; i < dims[dim]; i++)
+ {
+ PyObject *obj = PySequence_GetItem(list, i);
+
+ if (obj == Py_None)
+ nulls[*currelem] = true;
+ else
+ {
+ nulls[*currelem] = false;
+
+ /*
+ * We don't support arrays of row types yet, so the first argument
+ * can be NULL.
+ */
+ elems[*currelem] = elm->func(elm, -1, obj);
+ }
+ Py_XDECREF(obj);
+ (*currelem)++;
+ }
+ }
+}
+
static Datum
PLyString_ToComposite(PLyTypeInfo *info, TupleDesc desc, PyObject *string)
diff --git a/src/pl/plpython/sql/plpython_types.sql b/src/pl/plpython/sql/plpython_types.sql
index 19d920d..802dd41 100644
--- a/src/pl/plpython/sql/plpython_types.sql
+++ b/src/pl/plpython/sql/plpython_types.sql
@@ -237,7 +237,80 @@ SELECT * FROM test_type_conversion_array_int4(ARRAY[NULL,1]);
SELECT * FROM test_type_conversion_array_int4(ARRAY[]::integer[]);
SELECT * FROM test_type_conversion_array_int4(NULL);
SELECT * FROM test_type_conversion_array_int4(ARRAY[[1,2,3],[4,5,6]]);
+SELECT * FROM test_type_conversion_array_int4(ARRAY[[[1,2,NULL],[NULL,5,6]],[[NULL,8,9],[10,11,12]]]);
+SELECT * FROM test_type_conversion_array_int4('[2:4]={1,2,3}');
+CREATE FUNCTION test_type_conversion_array_int8(x int8[]) RETURNS int8[] AS $$
+plpy.info(x, type(x))
+return x
+$$ LANGUAGE plpythonu;
+
+SELECT * FROM test_type_conversion_array_int8(ARRAY[[[1,2,NULL],[NULL,5,6]],[[NULL,8,9],[10,11,12]]]::int8[]);
+
+CREATE FUNCTION test_type_conversion_array_float4(x float4[]) RETURNS float4[] AS $$
+plpy.info(x, type(x))
+return x
+$$ LANGUAGE plpythonu;
+
+SELECT * FROM test_type_conversion_array_float4(ARRAY[[[1.2,2.3,NULL],[NULL,5.7,6.8]],[[NULL,8.9,9.345],[10.123,11.456,12.6768]]]::float4[]);
+
+CREATE FUNCTION test_type_conversion_array_float8(x float8[]) RETURNS float8[] AS $$
+plpy.info(x, type(x))
+return x
+$$ LANGUAGE plpythonu;
+
+SELECT * FROM test_type_conversion_array_float8(ARRAY[[[1.2,2.3,NULL],[NULL,5.7,6.8]],[[NULL,8.9,9.345],[10.123,11.456,12.6768]]]::float8[]);
+
+CREATE FUNCTION test_type_conversion_array_date(x date[]) RETURNS date[] AS $$
+plpy.info(x, type(x))
+return x
+$$ LANGUAGE plpythonu;
+
+SELECT * FROM test_type_conversion_array_date(ARRAY[[['2016-09-21','2016-09-22',NULL],[NULL,'2016-10-21','2016-10-22']],
+ [[NULL,'2016-11-21','2016-10-21'],['2015-09-21','2015-09-22','2014-09-21']]]::date[]);
+
+CREATE FUNCTION test_type_conversion_array_timestamp(x timestamp[]) RETURNS timestamp[] AS $$
+plpy.info(x, type(x))
+return x
+$$ LANGUAGE plpythonu;
+
+SELECT * FROM test_type_conversion_array_timestamp(ARRAY[[['2016-09-21 15:34:24.078792-04','2016-10-22 11:34:24.078795-04',NULL],
+ [NULL,'2016-10-21 11:34:25.078792-04','2016-10-21 11:34:24.098792-04']],
+ [[NULL,'2016-01-21 11:34:24.078792-04','2016-11-21 11:34:24.108792-04'],
+ ['2015-09-21 11:34:24.079792-04','2014-09-21 11:34:24.078792-04','2013-09-21 11:34:24.078792-04']]]::timestamp[]);
+
+
+CREATE OR REPLACE FUNCTION pyreturnmultidemint4(h int4, i int4, j int4, k int4 ) RETURNS int4[] AS $BODY$
+m = [[[[x for x in range(h)] for y in range(i)] for z in range(j)] for w in range(k)]
+plpy.info(m, type(m))
+return m
+$BODY$ LANGUAGE plpythonu;
+
+select pyreturnmultidemint4(8,5,3,2);
+
+CREATE OR REPLACE FUNCTION pyreturnmultidemint8(h int4, i int4, j int4, k int4 ) RETURNS int8[] AS $BODY$
+m = [[[[x for x in range(h)] for y in range(i)] for z in range(j)] for w in range(k)]
+plpy.info(m, type(m))
+return m
+$BODY$ LANGUAGE plpythonu;
+
+select pyreturnmultidemint8(5,5,3,2);
+
+CREATE OR REPLACE FUNCTION pyreturnmultidemfloat4(h int4, i int4, j int4, k int4 ) RETURNS float4[] AS $BODY$
+m = [[[[x for x in range(h)] for y in range(i)] for z in range(j)] for w in range(k)]
+plpy.info(m, type(m))
+return m
+$BODY$ LANGUAGE plpythonu;
+
+select pyreturnmultidemfloat4(6,5,3,2);
+
+CREATE OR REPLACE FUNCTION pyreturnmultidemfloat8(h int4, i int4, j int4, k int4 ) RETURNS float8[] AS $BODY$
+m = [[[[x for x in range(h)] for y in range(i)] for z in range(j)] for w in range(k)]
+plpy.info(m, type(m))
+return m
+$BODY$ LANGUAGE plpythonu;
+
+select pyreturnmultidemfloat8(7,5,3,2);
CREATE FUNCTION test_type_conversion_array_text(x text[]) RETURNS text[] AS $$
plpy.info(x, type(x))
@@ -245,6 +318,7 @@ return x
$$ LANGUAGE plpythonu;
SELECT * FROM test_type_conversion_array_text(ARRAY['foo', 'bar']);
+SELECT * FROM test_type_conversion_array_text(ARRAY[['foo', 'bar'],['foo2', 'bar2']]);
CREATE FUNCTION test_type_conversion_array_bytea(x bytea[]) RETURNS bytea[] AS $$
@@ -268,6 +342,18 @@ $$ LANGUAGE plpythonu;
SELECT * FROM test_type_conversion_array_mixed2();
+CREATE FUNCTION test_type_conversion_mdarray_malformed() RETURNS int[] AS $$
+return [[1,2,3],[4,5]]
+$$ LANGUAGE plpythonu;
+
+SELECT * FROM test_type_conversion_mdarray_malformed();
+
+CREATE FUNCTION test_type_conversion_mdarray_toodeep() RETURNS int[] AS $$
+return [[[[[[[1]]]]]]]
+$$ LANGUAGE plpythonu;
+
+SELECT * FROM test_type_conversion_mdarray_toodeep();
+
CREATE FUNCTION test_type_conversion_array_record() RETURNS type_record[] AS $$
return [{'first': 'one', 'second': 42}, {'first': 'two', 'second': 11}]
--
2.9.3
Re: PL/Python adding support for multi-dimensional arrays
On 27 September 2016 at 14:58, Heikki Linnakangas <hlinnaka@iki.fi> wrote:
On 09/27/2016 02:04 PM, Dave Cramer wrote:
On 26 September 2016 at 14:52, Dave Cramer <pg@fastcrypt.com> wrote:
This crashes with arrays with non-default lower bounds:
postgres=# SELECT * FROM test_type_conversion_array_int
4('[2:4]={1,2,3}');
INFO: ([1, 2, <NULL>], <type 'list'>)
server closed the connection unexpectedly
This probably means the server terminated abnormally
before or while processing the request.Attached patch fixes this bug, and adds a test for it.
I spent some more time massaging this:
* Changed the loops from iterative to recursive style. I think this indeed
is slightly easier to understand.* Fixed another segfault, with too deeply nested lists:
CREATE or replace FUNCTION test_type_conversion_mdarray_toodeep() RETURNS
int[] AS $$
return [[[[[[[[[[[[[[[[[[1]]]]]]]]]]]]]]]]]]
$$ LANGUAGE plpythonu;* Also, in PLySequence_ToArray(), we must check that the 'len' of the
array doesn't overflow.* Fixed reference leak in the loop in PLySequence_ToArray() to count the
number of dimensions.I'd like to see some updates to the docs for this. The manual doesn't
currently say anything about multi-dimensional arrays in pl/python, but
it
should've mentioned that they're not supported. Now that it is
supported,
should mention that, and explain briefly that a multi-dimensional array
is
mapped to a python list of lists.If the code passes I'll fix the docs
Please do, thanks!
see attached
Dave Cramer
davec@postgresintl.com
www.postgresintl.com
Attachments:
0002-WIP-Multi-dimensional-arrays-in-PL-python.patchapplication/octet-stream; name=0002-WIP-Multi-dimensional-arrays-in-PL-python.patchDownload
diff --git a/doc/src/sgml/plpython.sgml b/doc/src/sgml/plpython.sgml
index 905e757..81e4052 100644
--- a/doc/src/sgml/plpython.sgml
+++ b/doc/src/sgml/plpython.sgml
@@ -467,6 +467,23 @@ SELECT return_arr();
(1 row)
</programlisting>
+ Multidimensional arrays are passed into PL/Python as a Python list
+ of lists, for example
+
+<programlisting>
+CREATE FUNCTION test_type_conversion_array_int4(x int4[]) RETURNS int4[] AS $$
+plpy.info(x, type(x))
+return x
+$$ LANGUAGE plpythonu;
+
+SELECT * FROM test_type_conversion_array_int4(ARRAY[[1,2,3],[4,5,6]]);
+INFO: ([[1, 2, 3], [4, 5, 6]], <type 'list'>)
+ test_type_conversion_array_int4
+---------------------------------
+ {{1,2,3},{4,5,6}}
+(1 row)
+</programlisting>
+
Note that in Python, strings are sequences, which can have
undesirable effects that might be familiar to Python programmers:
diff --git a/src/pl/plpython/expected/plpython_types.out b/src/pl/plpython/expected/plpython_types.out
index f0b6abd..947244e 100644
--- a/src/pl/plpython/expected/plpython_types.out
+++ b/src/pl/plpython/expected/plpython_types.out
@@ -537,9 +537,133 @@ INFO: (None, <type 'NoneType'>)
(1 row)
SELECT * FROM test_type_conversion_array_int4(ARRAY[[1,2,3],[4,5,6]]);
-ERROR: cannot convert multidimensional array to Python list
-DETAIL: PL/Python only supports one-dimensional arrays.
-CONTEXT: PL/Python function "test_type_conversion_array_int4"
+INFO: ([[1, 2, 3], [4, 5, 6]], <type 'list'>)
+ test_type_conversion_array_int4
+---------------------------------
+ {{1,2,3},{4,5,6}}
+(1 row)
+
+SELECT * FROM test_type_conversion_array_int4(ARRAY[[[1,2,NULL],[NULL,5,6]],[[NULL,8,9],[10,11,12]]]);
+INFO: ([[[1, 2, None], [None, 5, 6]], [[None, 8, 9], [10, 11, 12]]], <type 'list'>)
+ test_type_conversion_array_int4
+---------------------------------------------------
+ {{{1,2,NULL},{NULL,5,6}},{{NULL,8,9},{10,11,12}}}
+(1 row)
+
+SELECT * FROM test_type_conversion_array_int4('[2:4]={1,2,3}');
+INFO: ([1, 2, 3], <type 'list'>)
+ test_type_conversion_array_int4
+---------------------------------
+ {1,2,3}
+(1 row)
+
+CREATE FUNCTION test_type_conversion_array_int8(x int8[]) RETURNS int8[] AS $$
+plpy.info(x, type(x))
+return x
+$$ LANGUAGE plpythonu;
+SELECT * FROM test_type_conversion_array_int8(ARRAY[[[1,2,NULL],[NULL,5,6]],[[NULL,8,9],[10,11,12]]]::int8[]);
+INFO: ([[[1L, 2L, None], [None, 5L, 6L]], [[None, 8L, 9L], [10L, 11L, 12L]]], <type 'list'>)
+ test_type_conversion_array_int8
+---------------------------------------------------
+ {{{1,2,NULL},{NULL,5,6}},{{NULL,8,9},{10,11,12}}}
+(1 row)
+
+CREATE FUNCTION test_type_conversion_array_float4(x float4[]) RETURNS float4[] AS $$
+plpy.info(x, type(x))
+return x
+$$ LANGUAGE plpythonu;
+SELECT * FROM test_type_conversion_array_float4(ARRAY[[[1.2,2.3,NULL],[NULL,5.7,6.8]],[[NULL,8.9,9.345],[10.123,11.456,12.6768]]]::float4[]);
+INFO: ([[[1.2000000476837158, 2.299999952316284, None], [None, 5.699999809265137, 6.800000190734863]], [[None, 8.899999618530273, 9.345000267028809], [10.123000144958496, 11.456000328063965, 12.676799774169922]]], <type 'list'>)
+ test_type_conversion_array_float4
+------------------------------------------------------------------------------
+ {{{1.2,2.3,NULL},{NULL,5.7,6.8}},{{NULL,8.9,9.345},{10.123,11.456,12.6768}}}
+(1 row)
+
+CREATE FUNCTION test_type_conversion_array_float8(x float8[]) RETURNS float8[] AS $$
+plpy.info(x, type(x))
+return x
+$$ LANGUAGE plpythonu;
+SELECT * FROM test_type_conversion_array_float8(ARRAY[[[1.2,2.3,NULL],[NULL,5.7,6.8]],[[NULL,8.9,9.345],[10.123,11.456,12.6768]]]::float8[]);
+INFO: ([[[1.2, 2.3, None], [None, 5.7, 6.8]], [[None, 8.9, 9.345], [10.123, 11.456, 12.6768]]], <type 'list'>)
+ test_type_conversion_array_float8
+------------------------------------------------------------------------------
+ {{{1.2,2.3,NULL},{NULL,5.7,6.8}},{{NULL,8.9,9.345},{10.123,11.456,12.6768}}}
+(1 row)
+
+CREATE FUNCTION test_type_conversion_array_date(x date[]) RETURNS date[] AS $$
+plpy.info(x, type(x))
+return x
+$$ LANGUAGE plpythonu;
+SELECT * FROM test_type_conversion_array_date(ARRAY[[['2016-09-21','2016-09-22',NULL],[NULL,'2016-10-21','2016-10-22']],
+ [[NULL,'2016-11-21','2016-10-21'],['2015-09-21','2015-09-22','2014-09-21']]]::date[]);
+INFO: ([[['09-21-2016', '09-22-2016', None], [None, '10-21-2016', '10-22-2016']], [[None, '11-21-2016', '10-21-2016'], ['09-21-2015', '09-22-2015', '09-21-2014']]], <type 'list'>)
+ test_type_conversion_array_date
+---------------------------------------------------------------------------------------------------------------------------------
+ {{{09-21-2016,09-22-2016,NULL},{NULL,10-21-2016,10-22-2016}},{{NULL,11-21-2016,10-21-2016},{09-21-2015,09-22-2015,09-21-2014}}}
+(1 row)
+
+CREATE FUNCTION test_type_conversion_array_timestamp(x timestamp[]) RETURNS timestamp[] AS $$
+plpy.info(x, type(x))
+return x
+$$ LANGUAGE plpythonu;
+SELECT * FROM test_type_conversion_array_timestamp(ARRAY[[['2016-09-21 15:34:24.078792-04','2016-10-22 11:34:24.078795-04',NULL],
+ [NULL,'2016-10-21 11:34:25.078792-04','2016-10-21 11:34:24.098792-04']],
+ [[NULL,'2016-01-21 11:34:24.078792-04','2016-11-21 11:34:24.108792-04'],
+ ['2015-09-21 11:34:24.079792-04','2014-09-21 11:34:24.078792-04','2013-09-21 11:34:24.078792-04']]]::timestamp[]);
+INFO: ([[['Wed Sep 21 15:34:24.078792 2016', 'Sat Oct 22 11:34:24.078795 2016', None], [None, 'Fri Oct 21 11:34:25.078792 2016', 'Fri Oct 21 11:34:24.098792 2016']], [[None, 'Thu Jan 21 11:34:24.078792 2016', 'Mon Nov 21 11:34:24.108792 2016'], ['Mon Sep 21 11:34:24.079792 2015', 'Sun Sep 21 11:34:24.078792 2014', 'Sat Sep 21 11:34:24.078792 2013']]], <type 'list'>)
+ test_type_conversion_array_timestamp
+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
+ {{{"Wed Sep 21 15:34:24.078792 2016","Sat Oct 22 11:34:24.078795 2016",NULL},{NULL,"Fri Oct 21 11:34:25.078792 2016","Fri Oct 21 11:34:24.098792 2016"}},{{NULL,"Thu Jan 21 11:34:24.078792 2016","Mon Nov 21 11:34:24.108792 2016"},{"Mon Sep 21 11:34:24.079792 2015","Sun Sep 21 11:34:24.078792 2014","Sat Sep 21 11:34:24.078792 2013"}}}
+(1 row)
+
+CREATE OR REPLACE FUNCTION pyreturnmultidemint4(h int4, i int4, j int4, k int4 ) RETURNS int4[] AS $BODY$
+m = [[[[x for x in range(h)] for y in range(i)] for z in range(j)] for w in range(k)]
+plpy.info(m, type(m))
+return m
+$BODY$ LANGUAGE plpythonu;
+select pyreturnmultidemint4(8,5,3,2);
+INFO: ([[[[0, 1, 2, 3, 4, 5, 6, 7], [0, 1, 2, 3, 4, 5, 6, 7], [0, 1, 2, 3, 4, 5, 6, 7], [0, 1, 2, 3, 4, 5, 6, 7], [0, 1, 2, 3, 4, 5, 6, 7]], [[0, 1, 2, 3, 4, 5, 6, 7], [0, 1, 2, 3, 4, 5, 6, 7], [0, 1, 2, 3, 4, 5, 6, 7], [0, 1, 2, 3, 4, 5, 6, 7], [0, 1, 2, 3, 4, 5, 6, 7]], [[0, 1, 2, 3, 4, 5, 6, 7], [0, 1, 2, 3, 4, 5, 6, 7], [0, 1, 2, 3, 4, 5, 6, 7], [0, 1, 2, 3, 4, 5, 6, 7], [0, 1, 2, 3, 4, 5, 6, 7]]], [[[0, 1, 2, 3, 4, 5, 6, 7], [0, 1, 2, 3, 4, 5, 6, 7], [0, 1, 2, 3, 4, 5, 6, 7], [0, 1, 2, 3, 4, 5, 6, 7], [0, 1, 2, 3, 4, 5, 6, 7]], [[0, 1, 2, 3, 4, 5, 6, 7], [0, 1, 2, 3, 4, 5, 6, 7], [0, 1, 2, 3, 4, 5, 6, 7], [0, 1, 2, 3, 4, 5, 6, 7], [0, 1, 2, 3, 4, 5, 6, 7]], [[0, 1, 2, 3, 4, 5, 6, 7], [0, 1, 2, 3, 4, 5, 6, 7], [0, 1, 2, 3, 4, 5, 6, 7], [0, 1, 2, 3, 4, 5, 6, 7], [0, 1, 2, 3, 4, 5, 6, 7]]]], <type 'list'>)
+ pyreturnmultidemint4
+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
+ {{{{0,1,2,3,4,5,6,7},{0,1,2,3,4,5,6,7},{0,1,2,3,4,5,6,7},{0,1,2,3,4,5,6,7},{0,1,2,3,4,5,6,7}},{{0,1,2,3,4,5,6,7},{0,1,2,3,4,5,6,7},{0,1,2,3,4,5,6,7},{0,1,2,3,4,5,6,7},{0,1,2,3,4,5,6,7}},{{0,1,2,3,4,5,6,7},{0,1,2,3,4,5,6,7},{0,1,2,3,4,5,6,7},{0,1,2,3,4,5,6,7},{0,1,2,3,4,5,6,7}}},{{{0,1,2,3,4,5,6,7},{0,1,2,3,4,5,6,7},{0,1,2,3,4,5,6,7},{0,1,2,3,4,5,6,7},{0,1,2,3,4,5,6,7}},{{0,1,2,3,4,5,6,7},{0,1,2,3,4,5,6,7},{0,1,2,3,4,5,6,7},{0,1,2,3,4,5,6,7},{0,1,2,3,4,5,6,7}},{{0,1,2,3,4,5,6,7},{0,1,2,3,4,5,6,7},{0,1,2,3,4,5,6,7},{0,1,2,3,4,5,6,7},{0,1,2,3,4,5,6,7}}}}
+(1 row)
+
+CREATE OR REPLACE FUNCTION pyreturnmultidemint8(h int4, i int4, j int4, k int4 ) RETURNS int8[] AS $BODY$
+m = [[[[x for x in range(h)] for y in range(i)] for z in range(j)] for w in range(k)]
+plpy.info(m, type(m))
+return m
+$BODY$ LANGUAGE plpythonu;
+select pyreturnmultidemint8(5,5,3,2);
+INFO: ([[[[0, 1, 2, 3, 4], [0, 1, 2, 3, 4], [0, 1, 2, 3, 4], [0, 1, 2, 3, 4], [0, 1, 2, 3, 4]], [[0, 1, 2, 3, 4], [0, 1, 2, 3, 4], [0, 1, 2, 3, 4], [0, 1, 2, 3, 4], [0, 1, 2, 3, 4]], [[0, 1, 2, 3, 4], [0, 1, 2, 3, 4], [0, 1, 2, 3, 4], [0, 1, 2, 3, 4], [0, 1, 2, 3, 4]]], [[[0, 1, 2, 3, 4], [0, 1, 2, 3, 4], [0, 1, 2, 3, 4], [0, 1, 2, 3, 4], [0, 1, 2, 3, 4]], [[0, 1, 2, 3, 4], [0, 1, 2, 3, 4], [0, 1, 2, 3, 4], [0, 1, 2, 3, 4], [0, 1, 2, 3, 4]], [[0, 1, 2, 3, 4], [0, 1, 2, 3, 4], [0, 1, 2, 3, 4], [0, 1, 2, 3, 4], [0, 1, 2, 3, 4]]]], <type 'list'>)
+ pyreturnmultidemint8
+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
+ {{{{0,1,2,3,4},{0,1,2,3,4},{0,1,2,3,4},{0,1,2,3,4},{0,1,2,3,4}},{{0,1,2,3,4},{0,1,2,3,4},{0,1,2,3,4},{0,1,2,3,4},{0,1,2,3,4}},{{0,1,2,3,4},{0,1,2,3,4},{0,1,2,3,4},{0,1,2,3,4},{0,1,2,3,4}}},{{{0,1,2,3,4},{0,1,2,3,4},{0,1,2,3,4},{0,1,2,3,4},{0,1,2,3,4}},{{0,1,2,3,4},{0,1,2,3,4},{0,1,2,3,4},{0,1,2,3,4},{0,1,2,3,4}},{{0,1,2,3,4},{0,1,2,3,4},{0,1,2,3,4},{0,1,2,3,4},{0,1,2,3,4}}}}
+(1 row)
+
+CREATE OR REPLACE FUNCTION pyreturnmultidemfloat4(h int4, i int4, j int4, k int4 ) RETURNS float4[] AS $BODY$
+m = [[[[x for x in range(h)] for y in range(i)] for z in range(j)] for w in range(k)]
+plpy.info(m, type(m))
+return m
+$BODY$ LANGUAGE plpythonu;
+select pyreturnmultidemfloat4(6,5,3,2);
+INFO: ([[[[0, 1, 2, 3, 4, 5], [0, 1, 2, 3, 4, 5], [0, 1, 2, 3, 4, 5], [0, 1, 2, 3, 4, 5], [0, 1, 2, 3, 4, 5]], [[0, 1, 2, 3, 4, 5], [0, 1, 2, 3, 4, 5], [0, 1, 2, 3, 4, 5], [0, 1, 2, 3, 4, 5], [0, 1, 2, 3, 4, 5]], [[0, 1, 2, 3, 4, 5], [0, 1, 2, 3, 4, 5], [0, 1, 2, 3, 4, 5], [0, 1, 2, 3, 4, 5], [0, 1, 2, 3, 4, 5]]], [[[0, 1, 2, 3, 4, 5], [0, 1, 2, 3, 4, 5], [0, 1, 2, 3, 4, 5], [0, 1, 2, 3, 4, 5], [0, 1, 2, 3, 4, 5]], [[0, 1, 2, 3, 4, 5], [0, 1, 2, 3, 4, 5], [0, 1, 2, 3, 4, 5], [0, 1, 2, 3, 4, 5], [0, 1, 2, 3, 4, 5]], [[0, 1, 2, 3, 4, 5], [0, 1, 2, 3, 4, 5], [0, 1, 2, 3, 4, 5], [0, 1, 2, 3, 4, 5], [0, 1, 2, 3, 4, 5]]]], <type 'list'>)
+ pyreturnmultidemfloat4
+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
+ {{{{0,1,2,3,4,5},{0,1,2,3,4,5},{0,1,2,3,4,5},{0,1,2,3,4,5},{0,1,2,3,4,5}},{{0,1,2,3,4,5},{0,1,2,3,4,5},{0,1,2,3,4,5},{0,1,2,3,4,5},{0,1,2,3,4,5}},{{0,1,2,3,4,5},{0,1,2,3,4,5},{0,1,2,3,4,5},{0,1,2,3,4,5},{0,1,2,3,4,5}}},{{{0,1,2,3,4,5},{0,1,2,3,4,5},{0,1,2,3,4,5},{0,1,2,3,4,5},{0,1,2,3,4,5}},{{0,1,2,3,4,5},{0,1,2,3,4,5},{0,1,2,3,4,5},{0,1,2,3,4,5},{0,1,2,3,4,5}},{{0,1,2,3,4,5},{0,1,2,3,4,5},{0,1,2,3,4,5},{0,1,2,3,4,5},{0,1,2,3,4,5}}}}
+(1 row)
+
+CREATE OR REPLACE FUNCTION pyreturnmultidemfloat8(h int4, i int4, j int4, k int4 ) RETURNS float8[] AS $BODY$
+m = [[[[x for x in range(h)] for y in range(i)] for z in range(j)] for w in range(k)]
+plpy.info(m, type(m))
+return m
+$BODY$ LANGUAGE plpythonu;
+select pyreturnmultidemfloat8(7,5,3,2);
+INFO: ([[[[0, 1, 2, 3, 4, 5, 6], [0, 1, 2, 3, 4, 5, 6], [0, 1, 2, 3, 4, 5, 6], [0, 1, 2, 3, 4, 5, 6], [0, 1, 2, 3, 4, 5, 6]], [[0, 1, 2, 3, 4, 5, 6], [0, 1, 2, 3, 4, 5, 6], [0, 1, 2, 3, 4, 5, 6], [0, 1, 2, 3, 4, 5, 6], [0, 1, 2, 3, 4, 5, 6]], [[0, 1, 2, 3, 4, 5, 6], [0, 1, 2, 3, 4, 5, 6], [0, 1, 2, 3, 4, 5, 6], [0, 1, 2, 3, 4, 5, 6], [0, 1, 2, 3, 4, 5, 6]]], [[[0, 1, 2, 3, 4, 5, 6], [0, 1, 2, 3, 4, 5, 6], [0, 1, 2, 3, 4, 5, 6], [0, 1, 2, 3, 4, 5, 6], [0, 1, 2, 3, 4, 5, 6]], [[0, 1, 2, 3, 4, 5, 6], [0, 1, 2, 3, 4, 5, 6], [0, 1, 2, 3, 4, 5, 6], [0, 1, 2, 3, 4, 5, 6], [0, 1, 2, 3, 4, 5, 6]], [[0, 1, 2, 3, 4, 5, 6], [0, 1, 2, 3, 4, 5, 6], [0, 1, 2, 3, 4, 5, 6], [0, 1, 2, 3, 4, 5, 6], [0, 1, 2, 3, 4, 5, 6]]]], <type 'list'>)
+ pyreturnmultidemfloat8
+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
+ {{{{0,1,2,3,4,5,6},{0,1,2,3,4,5,6},{0,1,2,3,4,5,6},{0,1,2,3,4,5,6},{0,1,2,3,4,5,6}},{{0,1,2,3,4,5,6},{0,1,2,3,4,5,6},{0,1,2,3,4,5,6},{0,1,2,3,4,5,6},{0,1,2,3,4,5,6}},{{0,1,2,3,4,5,6},{0,1,2,3,4,5,6},{0,1,2,3,4,5,6},{0,1,2,3,4,5,6},{0,1,2,3,4,5,6}}},{{{0,1,2,3,4,5,6},{0,1,2,3,4,5,6},{0,1,2,3,4,5,6},{0,1,2,3,4,5,6},{0,1,2,3,4,5,6}},{{0,1,2,3,4,5,6},{0,1,2,3,4,5,6},{0,1,2,3,4,5,6},{0,1,2,3,4,5,6},{0,1,2,3,4,5,6}},{{0,1,2,3,4,5,6},{0,1,2,3,4,5,6},{0,1,2,3,4,5,6},{0,1,2,3,4,5,6},{0,1,2,3,4,5,6}}}}
+(1 row)
+
CREATE FUNCTION test_type_conversion_array_text(x text[]) RETURNS text[] AS $$
plpy.info(x, type(x))
return x
@@ -551,6 +675,13 @@ INFO: (['foo', 'bar'], <type 'list'>)
{foo,bar}
(1 row)
+SELECT * FROM test_type_conversion_array_text(ARRAY[['foo', 'bar'],['foo2', 'bar2']]);
+INFO: ([['foo', 'bar'], ['foo2', 'bar2']], <type 'list'>)
+ test_type_conversion_array_text
+---------------------------------
+ {{foo,bar},{foo2,bar2}}
+(1 row)
+
CREATE FUNCTION test_type_conversion_array_bytea(x bytea[]) RETURNS bytea[] AS $$
plpy.info(x, type(x))
return x
@@ -578,6 +709,20 @@ SELECT * FROM test_type_conversion_array_mixed2();
ERROR: invalid input syntax for integer: "abc"
CONTEXT: while creating return value
PL/Python function "test_type_conversion_array_mixed2"
+CREATE FUNCTION test_type_conversion_mdarray_malformed() RETURNS int[] AS $$
+return [[1,2,3],[4,5]]
+$$ LANGUAGE plpythonu;
+SELECT * FROM test_type_conversion_mdarray_malformed();
+ERROR: multidimensional arrays must have array expressions with matching dimensions. PL/Python function return value has sequence length 2 while expected 3
+CONTEXT: while creating return value
+PL/Python function "test_type_conversion_mdarray_malformed"
+CREATE FUNCTION test_type_conversion_mdarray_toodeep() RETURNS int[] AS $$
+return [[[[[[[1]]]]]]]
+$$ LANGUAGE plpythonu;
+SELECT * FROM test_type_conversion_mdarray_toodeep();
+ERROR: number of array dimensions (7) exceeds the maximum allowed (6)
+CONTEXT: while creating return value
+PL/Python function "test_type_conversion_mdarray_toodeep"
CREATE FUNCTION test_type_conversion_array_record() RETURNS type_record[] AS $$
return [{'first': 'one', 'second': 42}, {'first': 'two', 'second': 11}]
$$ LANGUAGE plpythonu;
diff --git a/src/pl/plpython/expected/plpython_types_3.out b/src/pl/plpython/expected/plpython_types_3.out
index 56b78e1..f7d357d 100644
--- a/src/pl/plpython/expected/plpython_types_3.out
+++ b/src/pl/plpython/expected/plpython_types_3.out
@@ -537,9 +537,133 @@ INFO: (None, <class 'NoneType'>)
(1 row)
SELECT * FROM test_type_conversion_array_int4(ARRAY[[1,2,3],[4,5,6]]);
-ERROR: cannot convert multidimensional array to Python list
-DETAIL: PL/Python only supports one-dimensional arrays.
-CONTEXT: PL/Python function "test_type_conversion_array_int4"
+INFO: ([[1, 2, 3], [4, 5, 6]], <class 'list'>)
+ test_type_conversion_array_int4
+---------------------------------
+ {{1,2,3},{4,5,6}}
+(1 row)
+
+SELECT * FROM test_type_conversion_array_int4(ARRAY[[[1,2,NULL],[NULL,5,6]],[[NULL,8,9],[10,11,12]]]);
+INFO: ([[[1, 2, None], [None, 5, 6]], [[None, 8, 9], [10, 11, 12]]], <class 'list'>)
+ test_type_conversion_array_int4
+---------------------------------------------------
+ {{{1,2,NULL},{NULL,5,6}},{{NULL,8,9},{10,11,12}}}
+(1 row)
+
+SELECT * FROM test_type_conversion_array_int4('[2:4]={1,2,3}');
+INFO: ([1, 2, 3], <class 'list'>)
+ test_type_conversion_array_int4
+---------------------------------
+ {1,2,3}
+(1 row)
+
+CREATE FUNCTION test_type_conversion_array_int8(x int8[]) RETURNS int8[] AS $$
+plpy.info(x, type(x))
+return x
+$$ LANGUAGE plpython3u;
+SELECT * FROM test_type_conversion_array_int8(ARRAY[[[1,2,NULL],[NULL,5,6]],[[NULL,8,9],[10,11,12]]]::int8[]);
+INFO: ([[[1, 2, None], [None, 5, 6]], [[None, 8, 9], [10, 11, 12]]], <class 'list'>)
+ test_type_conversion_array_int8
+---------------------------------------------------
+ {{{1,2,NULL},{NULL,5,6}},{{NULL,8,9},{10,11,12}}}
+(1 row)
+
+CREATE FUNCTION test_type_conversion_array_float4(x float4[]) RETURNS float4[] AS $$
+plpy.info(x, type(x))
+return x
+$$ LANGUAGE plpython3u;
+SELECT * FROM test_type_conversion_array_float4(ARRAY[[[1.2,2.3,NULL],[NULL,5.7,6.8]],[[NULL,8.9,9.345],[10.123,11.456,12.6768]]]::float4[]);
+INFO: ([[[1.2000000476837158, 2.299999952316284, None], [None, 5.699999809265137, 6.800000190734863]], [[None, 8.899999618530273, 9.345000267028809], [10.123000144958496, 11.456000328063965, 12.676799774169922]]], <class 'list'>)
+ test_type_conversion_array_float4
+------------------------------------------------------------------------------
+ {{{1.2,2.3,NULL},{NULL,5.7,6.8}},{{NULL,8.9,9.345},{10.123,11.456,12.6768}}}
+(1 row)
+
+CREATE FUNCTION test_type_conversion_array_float8(x float8[]) RETURNS float8[] AS $$
+plpy.info(x, type(x))
+return x
+$$ LANGUAGE plpython3u;
+SELECT * FROM test_type_conversion_array_float8(ARRAY[[[1.2,2.3,NULL],[NULL,5.7,6.8]],[[NULL,8.9,9.345],[10.123,11.456,12.6768]]]::float8[]);
+INFO: ([[[1.2, 2.3, None], [None, 5.7, 6.8]], [[None, 8.9, 9.345], [10.123, 11.456, 12.6768]]], <class 'list'>)
+ test_type_conversion_array_float8
+------------------------------------------------------------------------------
+ {{{1.2,2.3,NULL},{NULL,5.7,6.8}},{{NULL,8.9,9.345},{10.123,11.456,12.6768}}}
+(1 row)
+
+CREATE FUNCTION test_type_conversion_array_date(x date[]) RETURNS date[] AS $$
+plpy.info(x, type(x))
+return x
+$$ LANGUAGE plpython3u;
+SELECT * FROM test_type_conversion_array_date(ARRAY[[['2016-09-21','2016-09-22',NULL],[NULL,'2016-10-21','2016-10-22']],
+ [[NULL,'2016-11-21','2016-10-21'],['2015-09-21','2015-09-22','2014-09-21']]]::date[]);
+INFO: ([[['09-21-2016', '09-22-2016', None], [None, '10-21-2016', '10-22-2016']], [[None, '11-21-2016', '10-21-2016'], ['09-21-2015', '09-22-2015', '09-21-2014']]], <class 'list'>)
+ test_type_conversion_array_date
+---------------------------------------------------------------------------------------------------------------------------------
+ {{{09-21-2016,09-22-2016,NULL},{NULL,10-21-2016,10-22-2016}},{{NULL,11-21-2016,10-21-2016},{09-21-2015,09-22-2015,09-21-2014}}}
+(1 row)
+
+CREATE FUNCTION test_type_conversion_array_timestamp(x timestamp[]) RETURNS timestamp[] AS $$
+plpy.info(x, type(x))
+return x
+$$ LANGUAGE plpython3u;
+SELECT * FROM test_type_conversion_array_timestamp(ARRAY[[['2016-09-21 15:34:24.078792-04','2016-10-22 11:34:24.078795-04',NULL],
+ [NULL,'2016-10-21 11:34:25.078792-04','2016-10-21 11:34:24.098792-04']],
+ [[NULL,'2016-01-21 11:34:24.078792-04','2016-11-21 11:34:24.108792-04'],
+ ['2015-09-21 11:34:24.079792-04','2014-09-21 11:34:24.078792-04','2013-09-21 11:34:24.078792-04']]]::timestamp[]);
+INFO: ([[['Wed Sep 21 15:34:24.078792 2016', 'Sat Oct 22 11:34:24.078795 2016', None], [None, 'Fri Oct 21 11:34:25.078792 2016', 'Fri Oct 21 11:34:24.098792 2016']], [[None, 'Thu Jan 21 11:34:24.078792 2016', 'Mon Nov 21 11:34:24.108792 2016'], ['Mon Sep 21 11:34:24.079792 2015', 'Sun Sep 21 11:34:24.078792 2014', 'Sat Sep 21 11:34:24.078792 2013']]], <class 'list'>)
+ test_type_conversion_array_timestamp
+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
+ {{{"Wed Sep 21 15:34:24.078792 2016","Sat Oct 22 11:34:24.078795 2016",NULL},{NULL,"Fri Oct 21 11:34:25.078792 2016","Fri Oct 21 11:34:24.098792 2016"}},{{NULL,"Thu Jan 21 11:34:24.078792 2016","Mon Nov 21 11:34:24.108792 2016"},{"Mon Sep 21 11:34:24.079792 2015","Sun Sep 21 11:34:24.078792 2014","Sat Sep 21 11:34:24.078792 2013"}}}
+(1 row)
+
+CREATE OR REPLACE FUNCTION pyreturnmultidemint4(h int4, i int4, j int4, k int4 ) RETURNS int4[] AS $BODY$
+m = [[[[x for x in range(h)] for y in range(i)] for z in range(j)] for w in range(k)]
+plpy.info(m, type(m))
+return m
+$BODY$ LANGUAGE plpython3u;
+select pyreturnmultidemint4(8,5,3,2);
+INFO: ([[[[0, 1, 2, 3, 4, 5, 6, 7], [0, 1, 2, 3, 4, 5, 6, 7], [0, 1, 2, 3, 4, 5, 6, 7], [0, 1, 2, 3, 4, 5, 6, 7], [0, 1, 2, 3, 4, 5, 6, 7]], [[0, 1, 2, 3, 4, 5, 6, 7], [0, 1, 2, 3, 4, 5, 6, 7], [0, 1, 2, 3, 4, 5, 6, 7], [0, 1, 2, 3, 4, 5, 6, 7], [0, 1, 2, 3, 4, 5, 6, 7]], [[0, 1, 2, 3, 4, 5, 6, 7], [0, 1, 2, 3, 4, 5, 6, 7], [0, 1, 2, 3, 4, 5, 6, 7], [0, 1, 2, 3, 4, 5, 6, 7], [0, 1, 2, 3, 4, 5, 6, 7]]], [[[0, 1, 2, 3, 4, 5, 6, 7], [0, 1, 2, 3, 4, 5, 6, 7], [0, 1, 2, 3, 4, 5, 6, 7], [0, 1, 2, 3, 4, 5, 6, 7], [0, 1, 2, 3, 4, 5, 6, 7]], [[0, 1, 2, 3, 4, 5, 6, 7], [0, 1, 2, 3, 4, 5, 6, 7], [0, 1, 2, 3, 4, 5, 6, 7], [0, 1, 2, 3, 4, 5, 6, 7], [0, 1, 2, 3, 4, 5, 6, 7]], [[0, 1, 2, 3, 4, 5, 6, 7], [0, 1, 2, 3, 4, 5, 6, 7], [0, 1, 2, 3, 4, 5, 6, 7], [0, 1, 2, 3, 4, 5, 6, 7], [0, 1, 2, 3, 4, 5, 6, 7]]]], <class 'list'>)
+ pyreturnmultidemint4
+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
+ {{{{0,1,2,3,4,5,6,7},{0,1,2,3,4,5,6,7},{0,1,2,3,4,5,6,7},{0,1,2,3,4,5,6,7},{0,1,2,3,4,5,6,7}},{{0,1,2,3,4,5,6,7},{0,1,2,3,4,5,6,7},{0,1,2,3,4,5,6,7},{0,1,2,3,4,5,6,7},{0,1,2,3,4,5,6,7}},{{0,1,2,3,4,5,6,7},{0,1,2,3,4,5,6,7},{0,1,2,3,4,5,6,7},{0,1,2,3,4,5,6,7},{0,1,2,3,4,5,6,7}}},{{{0,1,2,3,4,5,6,7},{0,1,2,3,4,5,6,7},{0,1,2,3,4,5,6,7},{0,1,2,3,4,5,6,7},{0,1,2,3,4,5,6,7}},{{0,1,2,3,4,5,6,7},{0,1,2,3,4,5,6,7},{0,1,2,3,4,5,6,7},{0,1,2,3,4,5,6,7},{0,1,2,3,4,5,6,7}},{{0,1,2,3,4,5,6,7},{0,1,2,3,4,5,6,7},{0,1,2,3,4,5,6,7},{0,1,2,3,4,5,6,7},{0,1,2,3,4,5,6,7}}}}
+(1 row)
+
+CREATE OR REPLACE FUNCTION pyreturnmultidemint8(h int4, i int4, j int4, k int4 ) RETURNS int8[] AS $BODY$
+m = [[[[x for x in range(h)] for y in range(i)] for z in range(j)] for w in range(k)]
+plpy.info(m, type(m))
+return m
+$BODY$ LANGUAGE plpython3u;
+select pyreturnmultidemint8(5,5,3,2);
+INFO: ([[[[0, 1, 2, 3, 4], [0, 1, 2, 3, 4], [0, 1, 2, 3, 4], [0, 1, 2, 3, 4], [0, 1, 2, 3, 4]], [[0, 1, 2, 3, 4], [0, 1, 2, 3, 4], [0, 1, 2, 3, 4], [0, 1, 2, 3, 4], [0, 1, 2, 3, 4]], [[0, 1, 2, 3, 4], [0, 1, 2, 3, 4], [0, 1, 2, 3, 4], [0, 1, 2, 3, 4], [0, 1, 2, 3, 4]]], [[[0, 1, 2, 3, 4], [0, 1, 2, 3, 4], [0, 1, 2, 3, 4], [0, 1, 2, 3, 4], [0, 1, 2, 3, 4]], [[0, 1, 2, 3, 4], [0, 1, 2, 3, 4], [0, 1, 2, 3, 4], [0, 1, 2, 3, 4], [0, 1, 2, 3, 4]], [[0, 1, 2, 3, 4], [0, 1, 2, 3, 4], [0, 1, 2, 3, 4], [0, 1, 2, 3, 4], [0, 1, 2, 3, 4]]]], <class 'list'>)
+ pyreturnmultidemint8
+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
+ {{{{0,1,2,3,4},{0,1,2,3,4},{0,1,2,3,4},{0,1,2,3,4},{0,1,2,3,4}},{{0,1,2,3,4},{0,1,2,3,4},{0,1,2,3,4},{0,1,2,3,4},{0,1,2,3,4}},{{0,1,2,3,4},{0,1,2,3,4},{0,1,2,3,4},{0,1,2,3,4},{0,1,2,3,4}}},{{{0,1,2,3,4},{0,1,2,3,4},{0,1,2,3,4},{0,1,2,3,4},{0,1,2,3,4}},{{0,1,2,3,4},{0,1,2,3,4},{0,1,2,3,4},{0,1,2,3,4},{0,1,2,3,4}},{{0,1,2,3,4},{0,1,2,3,4},{0,1,2,3,4},{0,1,2,3,4},{0,1,2,3,4}}}}
+(1 row)
+
+CREATE OR REPLACE FUNCTION pyreturnmultidemfloat4(h int4, i int4, j int4, k int4 ) RETURNS float4[] AS $BODY$
+m = [[[[x for x in range(h)] for y in range(i)] for z in range(j)] for w in range(k)]
+plpy.info(m, type(m))
+return m
+$BODY$ LANGUAGE plpython3u;
+select pyreturnmultidemfloat4(6,5,3,2);
+INFO: ([[[[0, 1, 2, 3, 4, 5], [0, 1, 2, 3, 4, 5], [0, 1, 2, 3, 4, 5], [0, 1, 2, 3, 4, 5], [0, 1, 2, 3, 4, 5]], [[0, 1, 2, 3, 4, 5], [0, 1, 2, 3, 4, 5], [0, 1, 2, 3, 4, 5], [0, 1, 2, 3, 4, 5], [0, 1, 2, 3, 4, 5]], [[0, 1, 2, 3, 4, 5], [0, 1, 2, 3, 4, 5], [0, 1, 2, 3, 4, 5], [0, 1, 2, 3, 4, 5], [0, 1, 2, 3, 4, 5]]], [[[0, 1, 2, 3, 4, 5], [0, 1, 2, 3, 4, 5], [0, 1, 2, 3, 4, 5], [0, 1, 2, 3, 4, 5], [0, 1, 2, 3, 4, 5]], [[0, 1, 2, 3, 4, 5], [0, 1, 2, 3, 4, 5], [0, 1, 2, 3, 4, 5], [0, 1, 2, 3, 4, 5], [0, 1, 2, 3, 4, 5]], [[0, 1, 2, 3, 4, 5], [0, 1, 2, 3, 4, 5], [0, 1, 2, 3, 4, 5], [0, 1, 2, 3, 4, 5], [0, 1, 2, 3, 4, 5]]]], <class 'list'>)
+ pyreturnmultidemfloat4
+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
+ {{{{0,1,2,3,4,5},{0,1,2,3,4,5},{0,1,2,3,4,5},{0,1,2,3,4,5},{0,1,2,3,4,5}},{{0,1,2,3,4,5},{0,1,2,3,4,5},{0,1,2,3,4,5},{0,1,2,3,4,5},{0,1,2,3,4,5}},{{0,1,2,3,4,5},{0,1,2,3,4,5},{0,1,2,3,4,5},{0,1,2,3,4,5},{0,1,2,3,4,5}}},{{{0,1,2,3,4,5},{0,1,2,3,4,5},{0,1,2,3,4,5},{0,1,2,3,4,5},{0,1,2,3,4,5}},{{0,1,2,3,4,5},{0,1,2,3,4,5},{0,1,2,3,4,5},{0,1,2,3,4,5},{0,1,2,3,4,5}},{{0,1,2,3,4,5},{0,1,2,3,4,5},{0,1,2,3,4,5},{0,1,2,3,4,5},{0,1,2,3,4,5}}}}
+(1 row)
+
+CREATE OR REPLACE FUNCTION pyreturnmultidemfloat8(h int4, i int4, j int4, k int4 ) RETURNS float8[] AS $BODY$
+m = [[[[x for x in range(h)] for y in range(i)] for z in range(j)] for w in range(k)]
+plpy.info(m, type(m))
+return m
+$BODY$ LANGUAGE plpython3u;
+select pyreturnmultidemfloat8(7,5,3,2);
+INFO: ([[[[0, 1, 2, 3, 4, 5, 6], [0, 1, 2, 3, 4, 5, 6], [0, 1, 2, 3, 4, 5, 6], [0, 1, 2, 3, 4, 5, 6], [0, 1, 2, 3, 4, 5, 6]], [[0, 1, 2, 3, 4, 5, 6], [0, 1, 2, 3, 4, 5, 6], [0, 1, 2, 3, 4, 5, 6], [0, 1, 2, 3, 4, 5, 6], [0, 1, 2, 3, 4, 5, 6]], [[0, 1, 2, 3, 4, 5, 6], [0, 1, 2, 3, 4, 5, 6], [0, 1, 2, 3, 4, 5, 6], [0, 1, 2, 3, 4, 5, 6], [0, 1, 2, 3, 4, 5, 6]]], [[[0, 1, 2, 3, 4, 5, 6], [0, 1, 2, 3, 4, 5, 6], [0, 1, 2, 3, 4, 5, 6], [0, 1, 2, 3, 4, 5, 6], [0, 1, 2, 3, 4, 5, 6]], [[0, 1, 2, 3, 4, 5, 6], [0, 1, 2, 3, 4, 5, 6], [0, 1, 2, 3, 4, 5, 6], [0, 1, 2, 3, 4, 5, 6], [0, 1, 2, 3, 4, 5, 6]], [[0, 1, 2, 3, 4, 5, 6], [0, 1, 2, 3, 4, 5, 6], [0, 1, 2, 3, 4, 5, 6], [0, 1, 2, 3, 4, 5, 6], [0, 1, 2, 3, 4, 5, 6]]]], <class 'list'>)
+ pyreturnmultidemfloat8
+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
+ {{{{0,1,2,3,4,5,6},{0,1,2,3,4,5,6},{0,1,2,3,4,5,6},{0,1,2,3,4,5,6},{0,1,2,3,4,5,6}},{{0,1,2,3,4,5,6},{0,1,2,3,4,5,6},{0,1,2,3,4,5,6},{0,1,2,3,4,5,6},{0,1,2,3,4,5,6}},{{0,1,2,3,4,5,6},{0,1,2,3,4,5,6},{0,1,2,3,4,5,6},{0,1,2,3,4,5,6},{0,1,2,3,4,5,6}}},{{{0,1,2,3,4,5,6},{0,1,2,3,4,5,6},{0,1,2,3,4,5,6},{0,1,2,3,4,5,6},{0,1,2,3,4,5,6}},{{0,1,2,3,4,5,6},{0,1,2,3,4,5,6},{0,1,2,3,4,5,6},{0,1,2,3,4,5,6},{0,1,2,3,4,5,6}},{{0,1,2,3,4,5,6},{0,1,2,3,4,5,6},{0,1,2,3,4,5,6},{0,1,2,3,4,5,6},{0,1,2,3,4,5,6}}}}
+(1 row)
+
CREATE FUNCTION test_type_conversion_array_text(x text[]) RETURNS text[] AS $$
plpy.info(x, type(x))
return x
@@ -551,6 +675,13 @@ INFO: (['foo', 'bar'], <class 'list'>)
{foo,bar}
(1 row)
+SELECT * FROM test_type_conversion_array_text(ARRAY[['foo', 'bar'],['foo2', 'bar2']]);
+INFO: ([['foo', 'bar'], ['foo2', 'bar2']], <class 'list'>)
+ test_type_conversion_array_text
+---------------------------------
+ {{foo,bar},{foo2,bar2}}
+(1 row)
+
CREATE FUNCTION test_type_conversion_array_bytea(x bytea[]) RETURNS bytea[] AS $$
plpy.info(x, type(x))
return x
@@ -578,6 +709,20 @@ SELECT * FROM test_type_conversion_array_mixed2();
ERROR: invalid input syntax for integer: "abc"
CONTEXT: while creating return value
PL/Python function "test_type_conversion_array_mixed2"
+CREATE FUNCTION test_type_conversion_mdarray_malformed() RETURNS int[] AS $$
+return [[1,2,3],[4,5]]
+$$ LANGUAGE plpython3u;
+SELECT * FROM test_type_conversion_mdarray_malformed();
+ERROR: multidimensional arrays must have array expressions with matching dimensions. PL/Python function return value has sequence length 2 while expected 3
+CONTEXT: while creating return value
+PL/Python function "test_type_conversion_mdarray_malformed"
+CREATE FUNCTION test_type_conversion_mdarray_toodeep() RETURNS int[] AS $$
+return [[[[[[[1]]]]]]]
+$$ LANGUAGE plpython3u;
+SELECT * FROM test_type_conversion_mdarray_toodeep();
+ERROR: number of array dimensions (7) exceeds the maximum allowed (6)
+CONTEXT: while creating return value
+PL/Python function "test_type_conversion_mdarray_toodeep"
CREATE FUNCTION test_type_conversion_array_record() RETURNS type_record[] AS $$
return [{'first': 'one', 'second': 42}, {'first': 'two', 'second': 11}]
$$ LANGUAGE plpython3u;
diff --git a/src/pl/plpython/plpy_typeio.c b/src/pl/plpython/plpy_typeio.c
index 70f2e6d..019fcb1 100644
--- a/src/pl/plpython/plpy_typeio.c
+++ b/src/pl/plpython/plpy_typeio.c
@@ -45,6 +45,8 @@ static PyObject *PLyBytes_FromBytea(PLyDatumToOb *arg, Datum d);
static PyObject *PLyString_FromDatum(PLyDatumToOb *arg, Datum d);
static PyObject *PLyObject_FromTransform(PLyDatumToOb *arg, Datum d);
static PyObject *PLyList_FromArray(PLyDatumToOb *arg, Datum d);
+static PyObject *PLyList_FromArray_recurse(PLyDatumToOb *elm, int *dims, int ndim, int dim,
+ char **dataptr_p, bits8 **bitmap_p, int *bitmask_p);
/* conversion from Python objects to Datums */
static Datum PLyObject_ToBool(PLyObToDatum *arg, int32 typmod, PyObject *plrv);
@@ -53,6 +55,9 @@ static Datum PLyObject_ToComposite(PLyObToDatum *arg, int32 typmod, PyObject *pl
static Datum PLyObject_ToDatum(PLyObToDatum *arg, int32 typmod, PyObject *plrv);
static Datum PLyObject_ToTransform(PLyObToDatum *arg, int32 typmod, PyObject *plrv);
static Datum PLySequence_ToArray(PLyObToDatum *arg, int32 typmod, PyObject *plrv);
+static void PLySequence_ToArray_recurse(PLyObToDatum *elm, PyObject *list,
+ int *dims, int ndim, int dim,
+ Datum *elems, bool *nulls, int *currelem);
/* conversion from Python objects to composite Datums (used by triggers and SRFs) */
static Datum PLyString_ToComposite(PLyTypeInfo *info, TupleDesc desc, PyObject *string);
@@ -631,43 +636,105 @@ PLyList_FromArray(PLyDatumToOb *arg, Datum d)
{
ArrayType *array = DatumGetArrayTypeP(d);
PLyDatumToOb *elm = arg->elm;
- PyObject *list;
- int length;
- int lbound;
- int i;
+ int ndim;
+ int *dims;
+ char *dataptr;
+ bits8 *bitmap;
+ int bitmask;
if (ARR_NDIM(array) == 0)
return PyList_New(0);
- if (ARR_NDIM(array) != 1)
- ereport(ERROR,
- (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
- errmsg("cannot convert multidimensional array to Python list"),
- errdetail("PL/Python only supports one-dimensional arrays.")));
+ /* Array dimensions and left bounds */
+ ndim = ARR_NDIM(array);
+ dims = ARR_DIMS(array);
+ Assert(ndim < MAXDIM);
- length = ARR_DIMS(array)[0];
- lbound = ARR_LBOUND(array)[0];
- list = PyList_New(length);
- if (list == NULL)
- PLy_elog(ERROR, "could not create new Python list");
+ /*
+ * We will iterate the Postgres array in the physical order its stored in
+ * the datum. For 3-dimensional array the order of iteration would be the
+ * following: first you start with [0,0,0] elements through [0,0,k], then
+ * [0,1,0] till [0,1,k] till [0,m,k], then [1,0,0] till [1,0,k] till
+ * [1,m,k], and so on.
+ *
+ * In Python, there is no multi-dimensional lists as such, but they are
+ * represented as a list of lists. So a 3-d array of [n,m,k] element is a
+ * list of n m-element arrays, each element of which is k-element array.
+ * PLyList_FromArray_recurse() builds the Python list for a single dimension,
+ * and recurses for the next inner dimension.
+ */
+ dataptr = ARR_DATA_PTR(array);
+ bitmap = ARR_NULLBITMAP(array);
+ bitmask = 1;
- for (i = 0; i < length; i++)
+ return PLyList_FromArray_recurse(elm, dims, ndim, 0,
+ &dataptr, &bitmap, &bitmask);
+}
+
+static PyObject *
+PLyList_FromArray_recurse(PLyDatumToOb *elm, int *dims, int ndim, int dim,
+ char **dataptr_p, bits8 **bitmap_p, int *bitmask_p)
+{
+ int i;
+ PyObject *list;
+
+ list = PyList_New(dims[dim]);
+
+ if (dim < ndim - 1)
{
- Datum elem;
- bool isnull;
- int offset;
-
- offset = lbound + i;
- elem = array_ref(array, 1, &offset, arg->typlen,
- elm->typlen, elm->typbyval, elm->typalign,
- &isnull);
- if (isnull)
+ /* Outer dimension. Recurse for each inner slice. */
+ for (i = 0; i < dims[dim]; i++)
{
- Py_INCREF(Py_None);
- PyList_SET_ITEM(list, i, Py_None);
+ PyObject *sublist;
+
+ sublist = PLyList_FromArray_recurse(elm, dims, ndim, dim + 1,
+ dataptr_p, bitmap_p, bitmask_p);
+ PyList_SET_ITEM(list, i, sublist);
}
- else
- PyList_SET_ITEM(list, i, elm->func(elm, elem));
+ }
+ else
+ {
+ /*
+ * Innermost dimension. Fill the list with the values from the array
+ * for this slice.
+ */
+ char *dataptr = *dataptr_p;
+ bits8 *bitmap = *bitmap_p;
+ int bitmask = *bitmask_p;
+
+ for (i = 0; i < dims[dim]; i++)
+ {
+ /* checking for NULL */
+ if (bitmap && (*bitmap & bitmask) == 0)
+ {
+ Py_INCREF(Py_None);
+ PyList_SET_ITEM(list, i, Py_None);
+ }
+ else
+ {
+ Datum itemvalue;
+
+ itemvalue = fetch_att(dataptr, elm->typbyval, elm->typlen);
+ PyList_SET_ITEM(list, i, elm->func(elm, itemvalue));
+ dataptr = att_addlength_pointer(dataptr, elm->typlen, dataptr);
+ dataptr = (char *) att_align_nominal(dataptr, elm->typalign);
+ }
+
+ /* advance bitmap pointer if any */
+ if (bitmap)
+ {
+ bitmask <<= 1;
+ if (bitmask == 0x100 /* (1<<8) */)
+ {
+ bitmap++;
+ bitmask = 1;
+ }
+ }
+ }
+
+ *dataptr_p = dataptr;
+ *bitmap_p = bitmap;
+ *bitmask_p = bitmask;
}
return list;
@@ -864,39 +931,100 @@ static Datum
PLySequence_ToArray(PLyObToDatum *arg, int32 typmod, PyObject *plrv)
{
ArrayType *array;
- Datum rv;
int i;
Datum *elems;
bool *nulls;
- int len;
- int lbs;
+ int64 len;
+ int ndim;
+ int dims[MAXDIM];
+ int lbs[MAXDIM];
+ int currelem;
+ Datum rv;
Assert(plrv != Py_None);
if (!PySequence_Check(plrv))
PLy_elog(ERROR, "return value of function with array return type is not a Python sequence");
- len = PySequence_Length(plrv);
- elems = palloc(sizeof(*elems) * len);
- nulls = palloc(sizeof(*nulls) * len);
-
- for (i = 0; i < len; i++)
+ /*
+ * Determine the number of dimensions, and their sizes.
+ *
+ * We don't want to create multi-dimensional arrays when we have an empty sequence,
+ * when we need to parse sequence of composite objects, or when we have strings.
+ */
+ if (type_is_rowtype(get_base_element_type(arg->typoid)) ||
+ PyString_Check(plrv) || PyBytes_Check(plrv) || PyUnicode_Check(plrv))
{
- PyObject *obj = PySequence_GetItem(plrv, i);
+ ndim = 1;
+ len = dims[0] = PySequence_Length(plrv);
+ }
+ else
+ {
+ PyObject *pyptr = plrv;
+ PyObject *next;
- if (obj == Py_None)
- nulls[i] = true;
- else
+ ndim = 0;
+ len = 1;
+
+ Py_INCREF(plrv);
+
+ for (;;)
{
- nulls[i] = false;
- elems[i] = arg->elm->func(arg->elm, -1, obj);
+ if (!PySequence_Check(pyptr))
+ break;
+
+ if (PyString_Check(pyptr) || PyBytes_Check(pyptr) || PyUnicode_Check(pyptr))
+ break;
+
+ dims[ndim] = PySequence_Length(pyptr);
+ if (dims[ndim] < 0)
+ PLy_elog(ERROR, "cannot determine sequence length for function return value");
+
+ if (dims[ndim] > MaxAllocSize)
+ PLy_elog(ERROR, "array size exceeds the maximum allowed (%d)", (int) MaxAllocSize);
+
+ len *= dims[ndim];
+ if (len > MaxAllocSize)
+ PLy_elog(ERROR, "array size exceeds the maximum allowed (%d)", (int) MaxAllocSize);
+
+ if (dims[ndim] == 0)
+ {
+ /* empty sequence */
+ break;
+ }
+
+ ndim++;
+
+ next = PySequence_GetItem(pyptr, 0);
+ Py_XDECREF(pyptr);
+ pyptr = next;
}
- Py_XDECREF(obj);
+ Py_XDECREF(pyptr);
}
- lbs = 1;
- array = construct_md_array(elems, nulls, 1, &len, &lbs,
- get_base_element_type(arg->typoid), arg->elm->typlen, arg->elm->typbyval, arg->elm->typalign);
+ /*
+ * Traverse the Python lists, in depth-first order, and collect all the
+ * elements at the bottom level into 'elems'/'nulls' arrays.
+ */
+ elems = palloc(sizeof(Datum) * len);
+ nulls = palloc(sizeof(bool) * len);
+ currelem = 0;
+ PLySequence_ToArray_recurse(arg->elm, plrv,
+ dims, ndim, 0,
+ elems, nulls, &currelem);
+
+ for (i = 0; i < ndim; i++)
+ lbs[i] = 1;
+
+ array = construct_md_array(elems,
+ nulls,
+ ndim,
+ dims,
+ lbs,
+ get_base_element_type(arg->typoid),
+ arg->elm->typlen,
+ arg->elm->typbyval,
+ arg->elm->typalign);
/*
* If the result type is a domain of array, the resulting array must be
@@ -908,6 +1036,58 @@ PLySequence_ToArray(PLyObToDatum *arg, int32 typmod, PyObject *plrv)
return rv;
}
+/*
+ * Helper function for PLySequence_ToArray. Traverse a Python list of lists in
+ * depth-first order, storing the elements in 'elems'.
+ */
+static void
+PLySequence_ToArray_recurse(PLyObToDatum *elm, PyObject *list,
+ int *dims, int ndim, int dim,
+ Datum *elems, bool *nulls, int *currelem)
+{
+ int i;
+
+ if (PySequence_Length(list) != dims[dim])
+ PLy_elog(ERROR,
+ "multidimensional arrays must have array expressions with matching dimensions. "
+ "PL/Python function return value has sequence length %d while expected %d",
+ (int) PySequence_Length(list), dims[dim]);
+
+ if (dim < ndim - 1)
+ {
+ for (i = 0; i < dims[dim]; i++)
+ {
+ PyObject *sublist = PySequence_GetItem(list, i);
+
+ PLySequence_ToArray_recurse(elm, sublist, dims, ndim, dim + 1,
+ elems, nulls, currelem);
+ Py_XDECREF(sublist);
+ }
+ }
+ else
+ {
+ for (i = 0; i < dims[dim]; i++)
+ {
+ PyObject *obj = PySequence_GetItem(list, i);
+
+ if (obj == Py_None)
+ nulls[*currelem] = true;
+ else
+ {
+ nulls[*currelem] = false;
+
+ /*
+ * We don't support arrays of row types yet, so the first argument
+ * can be NULL.
+ */
+ elems[*currelem] = elm->func(elm, -1, obj);
+ }
+ Py_XDECREF(obj);
+ (*currelem)++;
+ }
+ }
+}
+
static Datum
PLyString_ToComposite(PLyTypeInfo *info, TupleDesc desc, PyObject *string)
diff --git a/src/pl/plpython/sql/plpython_types.sql b/src/pl/plpython/sql/plpython_types.sql
index 19d920d..802dd41 100644
--- a/src/pl/plpython/sql/plpython_types.sql
+++ b/src/pl/plpython/sql/plpython_types.sql
@@ -237,7 +237,80 @@ SELECT * FROM test_type_conversion_array_int4(ARRAY[NULL,1]);
SELECT * FROM test_type_conversion_array_int4(ARRAY[]::integer[]);
SELECT * FROM test_type_conversion_array_int4(NULL);
SELECT * FROM test_type_conversion_array_int4(ARRAY[[1,2,3],[4,5,6]]);
+SELECT * FROM test_type_conversion_array_int4(ARRAY[[[1,2,NULL],[NULL,5,6]],[[NULL,8,9],[10,11,12]]]);
+SELECT * FROM test_type_conversion_array_int4('[2:4]={1,2,3}');
+CREATE FUNCTION test_type_conversion_array_int8(x int8[]) RETURNS int8[] AS $$
+plpy.info(x, type(x))
+return x
+$$ LANGUAGE plpythonu;
+
+SELECT * FROM test_type_conversion_array_int8(ARRAY[[[1,2,NULL],[NULL,5,6]],[[NULL,8,9],[10,11,12]]]::int8[]);
+
+CREATE FUNCTION test_type_conversion_array_float4(x float4[]) RETURNS float4[] AS $$
+plpy.info(x, type(x))
+return x
+$$ LANGUAGE plpythonu;
+
+SELECT * FROM test_type_conversion_array_float4(ARRAY[[[1.2,2.3,NULL],[NULL,5.7,6.8]],[[NULL,8.9,9.345],[10.123,11.456,12.6768]]]::float4[]);
+
+CREATE FUNCTION test_type_conversion_array_float8(x float8[]) RETURNS float8[] AS $$
+plpy.info(x, type(x))
+return x
+$$ LANGUAGE plpythonu;
+
+SELECT * FROM test_type_conversion_array_float8(ARRAY[[[1.2,2.3,NULL],[NULL,5.7,6.8]],[[NULL,8.9,9.345],[10.123,11.456,12.6768]]]::float8[]);
+
+CREATE FUNCTION test_type_conversion_array_date(x date[]) RETURNS date[] AS $$
+plpy.info(x, type(x))
+return x
+$$ LANGUAGE plpythonu;
+
+SELECT * FROM test_type_conversion_array_date(ARRAY[[['2016-09-21','2016-09-22',NULL],[NULL,'2016-10-21','2016-10-22']],
+ [[NULL,'2016-11-21','2016-10-21'],['2015-09-21','2015-09-22','2014-09-21']]]::date[]);
+
+CREATE FUNCTION test_type_conversion_array_timestamp(x timestamp[]) RETURNS timestamp[] AS $$
+plpy.info(x, type(x))
+return x
+$$ LANGUAGE plpythonu;
+
+SELECT * FROM test_type_conversion_array_timestamp(ARRAY[[['2016-09-21 15:34:24.078792-04','2016-10-22 11:34:24.078795-04',NULL],
+ [NULL,'2016-10-21 11:34:25.078792-04','2016-10-21 11:34:24.098792-04']],
+ [[NULL,'2016-01-21 11:34:24.078792-04','2016-11-21 11:34:24.108792-04'],
+ ['2015-09-21 11:34:24.079792-04','2014-09-21 11:34:24.078792-04','2013-09-21 11:34:24.078792-04']]]::timestamp[]);
+
+
+CREATE OR REPLACE FUNCTION pyreturnmultidemint4(h int4, i int4, j int4, k int4 ) RETURNS int4[] AS $BODY$
+m = [[[[x for x in range(h)] for y in range(i)] for z in range(j)] for w in range(k)]
+plpy.info(m, type(m))
+return m
+$BODY$ LANGUAGE plpythonu;
+
+select pyreturnmultidemint4(8,5,3,2);
+
+CREATE OR REPLACE FUNCTION pyreturnmultidemint8(h int4, i int4, j int4, k int4 ) RETURNS int8[] AS $BODY$
+m = [[[[x for x in range(h)] for y in range(i)] for z in range(j)] for w in range(k)]
+plpy.info(m, type(m))
+return m
+$BODY$ LANGUAGE plpythonu;
+
+select pyreturnmultidemint8(5,5,3,2);
+
+CREATE OR REPLACE FUNCTION pyreturnmultidemfloat4(h int4, i int4, j int4, k int4 ) RETURNS float4[] AS $BODY$
+m = [[[[x for x in range(h)] for y in range(i)] for z in range(j)] for w in range(k)]
+plpy.info(m, type(m))
+return m
+$BODY$ LANGUAGE plpythonu;
+
+select pyreturnmultidemfloat4(6,5,3,2);
+
+CREATE OR REPLACE FUNCTION pyreturnmultidemfloat8(h int4, i int4, j int4, k int4 ) RETURNS float8[] AS $BODY$
+m = [[[[x for x in range(h)] for y in range(i)] for z in range(j)] for w in range(k)]
+plpy.info(m, type(m))
+return m
+$BODY$ LANGUAGE plpythonu;
+
+select pyreturnmultidemfloat8(7,5,3,2);
CREATE FUNCTION test_type_conversion_array_text(x text[]) RETURNS text[] AS $$
plpy.info(x, type(x))
@@ -245,6 +318,7 @@ return x
$$ LANGUAGE plpythonu;
SELECT * FROM test_type_conversion_array_text(ARRAY['foo', 'bar']);
+SELECT * FROM test_type_conversion_array_text(ARRAY[['foo', 'bar'],['foo2', 'bar2']]);
CREATE FUNCTION test_type_conversion_array_bytea(x bytea[]) RETURNS bytea[] AS $$
@@ -268,6 +342,18 @@ $$ LANGUAGE plpythonu;
SELECT * FROM test_type_conversion_array_mixed2();
+CREATE FUNCTION test_type_conversion_mdarray_malformed() RETURNS int[] AS $$
+return [[1,2,3],[4,5]]
+$$ LANGUAGE plpythonu;
+
+SELECT * FROM test_type_conversion_mdarray_malformed();
+
+CREATE FUNCTION test_type_conversion_mdarray_toodeep() RETURNS int[] AS $$
+return [[[[[[[1]]]]]]]
+$$ LANGUAGE plpythonu;
+
+SELECT * FROM test_type_conversion_mdarray_toodeep();
+
CREATE FUNCTION test_type_conversion_array_record() RETURNS type_record[] AS $$
return [{'first': 'one', 'second': 42}, {'first': 'two', 'second': 11}]
Re: PL/Python adding support for multi-dimensional arrays
On 09/23/2016 10:27 PM, Jim Nasby wrote:
On 9/23/16 2:42 AM, Heikki Linnakangas wrote:
How do we handle single-dimensional arrays of composite types at the
moment? At a quick glance, it seems that the composite types are just
treated like strings, when they're in an array. That's probably OK, but
it means that there's nothing special about composite types in
multi-dimensional arrays. In any case, we should mention that in the docs.That is how they're handled, but I'd really like to change that. I've
held off because I don't know how to handle the backwards
incompatibility that would introduce. (I've been wondering if we might
add a facility to allow specifying default TRANSFORMs that should be
used for specific data types in specific languages.)The converse case (a composite with arrays) suffers the same problem
(array is just treated as a string).
I take that back, I don't know what I was talking about. Without this
patch, an array of composite types can be returned, using any of the
three representations for the composite type explained in the docs: a
string, a sequence, or a dictionary. So, all these work, and return the
same value:
create table foo (a int4, b int4);
CREATE FUNCTION comp_array_string() RETURNS foo[] AS $$
return ["(1, 2)"]
$$ LANGUAGE plpythonu;
CREATE FUNCTION comp_array_sequence() RETURNS foo[] AS $$
return [[1, 2]]
$$ LANGUAGE plpythonu;
CREATE FUNCTION comp_array_dict() RETURNS foo[] AS $$
return [{"a": 1, "b": 2}]
$$ LANGUAGE plpythonu;
Jim, I was confused, but you agreed with me. Were you also confused, or
am I missing something?
Now, back to multi-dimensional arrays. I can see that the Sequence
representation is problematic, with arrays, because if you have a python
list of lists, like [[1, 2]], it's not immediately clear if that's a
one-dimensional array of tuples, or two-dimensional array of integers.
Then again, we do have the type definitions available. So is it really
ambiguous?
The string and dict representations don't have that ambiguity at all, so
I don't see why we wouldn't support those, at least.
- Heikki
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
Re: PL/Python adding support for multi-dimensional arrays
On 9/29/16 1:51 PM, Heikki Linnakangas wrote:
Jim, I was confused, but you agreed with me. Were you also confused, or
am I missing something?
I was confused by inputs:
CREATE FUNCTION repr(i foo[]) RETURNS text LANGUAGE plpythonu AS
$$return repr(i)$$;
select repr(array[row(1,2)::foo, row(3,4)::foo]);
repr
--------------------
['(1,2)', '(3,4)']
(1 row)
(in ipython...)
In [1]: i=['(1,2)', '(3,4)']
In [2]: type(i)
Out[2]: list
In [3]: type(i[0])
Out[3]: str
I wonder if your examples work only
Now, back to multi-dimensional arrays. I can see that the Sequence
representation is problematic, with arrays, because if you have a python
list of lists, like [[1, 2]], it's not immediately clear if that's a
one-dimensional array of tuples, or two-dimensional array of integers.
Then again, we do have the type definitions available. So is it really
ambiguous?
[[1,2]] is a list of lists...
In [4]: b=[[1,2]]
In [5]: type(b)
Out[5]: list
In [6]: type(b[0])
Out[6]: list
If you want a list of tuples...
In [7]: c=[(1,2)]
In [8]: type(c)
Out[8]: list
In [9]: type(c[0])
Out[9]: tuple
--
Jim Nasby, Data Architect, Blue Treble Consulting, Austin TX
Experts in Analytics, Data Architecture and PostgreSQL
Data in Trouble? Get it in Treble! http://BlueTreble.com
855-TREBLE2 (855-873-2532) mobile: 512-569-9461
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
Re: PL/Python adding support for multi-dimensional arrays
On Sat, Oct 1, 2016 at 8:45 AM, Jim Nasby <Jim.Nasby@bluetreble.com> wrote:
On 9/29/16 1:51 PM, Heikki Linnakangas wrote:
Jim, I was confused, but you agreed with me. Were you also confused, or
am I missing something?I was confused by inputs:
I have marked the patch as returned with feedback. Or Heikki, do you
plan on looking at it more and commit soon?
--
Michael
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
Re: PL/Python adding support for multi-dimensional arrays
On 10/01/2016 02:45 AM, Jim Nasby wrote:
On 9/29/16 1:51 PM, Heikki Linnakangas wrote:
Now, back to multi-dimensional arrays. I can see that the Sequence
representation is problematic, with arrays, because if you have a python
list of lists, like [[1, 2]], it's not immediately clear if that's a
one-dimensional array of tuples, or two-dimensional array of integers.
Then again, we do have the type definitions available. So is it really
ambiguous?[[1,2]] is a list of lists...
In [4]: b=[[1,2]]In [5]: type(b)
Out[5]: listIn [6]: type(b[0])
Out[6]: listIf you want a list of tuples...
In [7]: c=[(1,2)]In [8]: type(c)
Out[8]: listIn [9]: type(c[0])
Out[9]: tuple
Hmm, so we would start to treat lists and tuples differently? A Python
list would be converted into an array, and a Python tuple would be
converted into a composite type. That does make a lot of sense. The only
problem is that it's not backwards-compatible. A PL/python function that
returns an SQL array of rows, and does that by returning Python list of
lists, it would start failing.
I think we should bite the bullet and do that anyway. As long as it's
clearly documented, and the error message you get contains a clear hint
on how to fix it, I don't think it would be too painful to adjust
existing application.
We could continue to accept a Python list for a plain composite type,
this would only affect arrays of composite types.
I don't use PL/python much myself, so I don't feel qualified to make the
call, though. Any 3rd opinions?
- Heikki
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
Re: PL/Python adding support for multi-dimensional arrays
2016-10-10 12:31 GMT+02:00 Heikki Linnakangas <hlinnaka@iki.fi>:
On 10/01/2016 02:45 AM, Jim Nasby wrote:
On 9/29/16 1:51 PM, Heikki Linnakangas wrote:
Now, back to multi-dimensional arrays. I can see that the Sequence
representation is problematic, with arrays, because if you have a python
list of lists, like [[1, 2]], it's not immediately clear if that's a
one-dimensional array of tuples, or two-dimensional array of integers.
Then again, we do have the type definitions available. So is it really
ambiguous?[[1,2]] is a list of lists...
In [4]: b=[[1,2]]In [5]: type(b)
Out[5]: listIn [6]: type(b[0])
Out[6]: listIf you want a list of tuples...
In [7]: c=[(1,2)]In [8]: type(c)
Out[8]: listIn [9]: type(c[0])
Out[9]: tupleHmm, so we would start to treat lists and tuples differently? A Python
list would be converted into an array, and a Python tuple would be
converted into a composite type. That does make a lot of sense. The only
problem is that it's not backwards-compatible. A PL/python function that
returns an SQL array of rows, and does that by returning Python list of
lists, it would start failing.
is not possible do decision in last moment - on PL/Postgres interface?
There the expected type should be known.
Regards
Pavel
Show quoted text
I think we should bite the bullet and do that anyway. As long as it's
clearly documented, and the error message you get contains a clear hint on
how to fix it, I don't think it would be too painful to adjust existing
application.We could continue to accept a Python list for a plain composite type, this
would only affect arrays of composite types.I don't use PL/python much myself, so I don't feel qualified to make the
call, though. Any 3rd opinions?- Heikki
Re: PL/Python adding support for multi-dimensional arrays
On 10 October 2016 at 13:42, Pavel Stehule <pavel.stehule@gmail.com> wrote:
2016-10-10 12:31 GMT+02:00 Heikki Linnakangas <hlinnaka@iki.fi>:
On 10/01/2016 02:45 AM, Jim Nasby wrote:
On 9/29/16 1:51 PM, Heikki Linnakangas wrote:
Now, back to multi-dimensional arrays. I can see that the Sequence
representation is problematic, with arrays, because if you have a python
list of lists, like [[1, 2]], it's not immediately clear if that's a
one-dimensional array of tuples, or two-dimensional array of integers.
Then again, we do have the type definitions available. So is it really
ambiguous?[[1,2]] is a list of lists...
In [4]: b=[[1,2]]In [5]: type(b)
Out[5]: listIn [6]: type(b[0])
Out[6]: listIf you want a list of tuples...
In [7]: c=[(1,2)]In [8]: type(c)
Out[8]: listIn [9]: type(c[0])
Out[9]: tupleHmm, so we would start to treat lists and tuples differently? A Python
list would be converted into an array, and a Python tuple would be
converted into a composite type. That does make a lot of sense. The only
problem is that it's not backwards-compatible. A PL/python function that
returns an SQL array of rows, and does that by returning Python list of
lists, it would start failing.is not possible do decision in last moment - on PL/Postgres interface?
There the expected type should be known.Regards
Pavel
I think we should bite the bullet and do that anyway. As long as it's
clearly documented, and the error message you get contains a clear hint on
how to fix it, I don't think it would be too painful to adjust existing
application.We could continue to accept a Python list for a plain composite type,
this would only affect arrays of composite types.I don't use PL/python much myself, so I don't feel qualified to make the
call, though. Any 3rd opinions?
Can't you determine the correct output based on the function output
definition ?
For instance if the function output was an array type then we would return
the list as an array
if the function output was a set of then we return tuples ?
Dave Cramer
davec@postgresintl.com
www.postgresintl.com
Re: PL/Python adding support for multi-dimensional arrays
On 10/10/2016 08:42 PM, Pavel Stehule wrote:
2016-10-10 12:31 GMT+02:00 Heikki Linnakangas <hlinnaka@iki.fi>:
On 10/01/2016 02:45 AM, Jim Nasby wrote:
On 9/29/16 1:51 PM, Heikki Linnakangas wrote:
Now, back to multi-dimensional arrays. I can see that the Sequence
representation is problematic, with arrays, because if you have a python
list of lists, like [[1, 2]], it's not immediately clear if that's a
one-dimensional array of tuples, or two-dimensional array of integers.
Then again, we do have the type definitions available. So is it really
ambiguous?[[1,2]] is a list of lists...
In [4]: b=[[1,2]]In [5]: type(b)
Out[5]: listIn [6]: type(b[0])
Out[6]: listIf you want a list of tuples...
In [7]: c=[(1,2)]In [8]: type(c)
Out[8]: listIn [9]: type(c[0])
Out[9]: tupleHmm, so we would start to treat lists and tuples differently? A Python
list would be converted into an array, and a Python tuple would be
converted into a composite type. That does make a lot of sense. The only
problem is that it's not backwards-compatible. A PL/python function that
returns an SQL array of rows, and does that by returning Python list of
lists, it would start failing.is not possible do decision in last moment - on PL/Postgres interface?
There the expected type should be known.
Unfortunately there are cases that are fundamentally ambiguous.
create type comptype as (intarray int[]);
create function array_return() returns comptype[] as $$
return [[[[1]]]];
$$ language plpython;
What does the function return? It could be two-dimension array of
comptype, with a single-dimension intarray, or a single-dimension
comptype, with a two-dimension intarray.
We could resolve it for simpler cases, but not the general case. The
simple cases would probably cover most things people do in practice. But
if the distinction between a tuple and a list feels natural to Python
programmers, I think it would be more clear in the long run to have
people adjust their applications.
- Heikki
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
Re: PL/Python adding support for multi-dimensional arrays
2016-10-11 7:49 GMT+02:00 Heikki Linnakangas <hlinnaka@iki.fi>:
On 10/10/2016 08:42 PM, Pavel Stehule wrote:
2016-10-10 12:31 GMT+02:00 Heikki Linnakangas <hlinnaka@iki.fi>:
On 10/01/2016 02:45 AM, Jim Nasby wrote:
On 9/29/16 1:51 PM, Heikki Linnakangas wrote:
Now, back to multi-dimensional arrays. I can see that the Sequence
representation is problematic, with arrays, because if you have a
python
list of lists, like [[1, 2]], it's not immediately clear if that's a
one-dimensional array of tuples, or two-dimensional array of integers.
Then again, we do have the type definitions available. So is it really
ambiguous?[[1,2]] is a list of lists...
In [4]: b=[[1,2]]In [5]: type(b)
Out[5]: listIn [6]: type(b[0])
Out[6]: listIf you want a list of tuples...
In [7]: c=[(1,2)]In [8]: type(c)
Out[8]: listIn [9]: type(c[0])
Out[9]: tupleHmm, so we would start to treat lists and tuples differently? A Python
list would be converted into an array, and a Python tuple would be
converted into a composite type. That does make a lot of sense. The only
problem is that it's not backwards-compatible. A PL/python function that
returns an SQL array of rows, and does that by returning Python list of
lists, it would start failing.is not possible do decision in last moment - on PL/Postgres interface?
There the expected type should be known.Unfortunately there are cases that are fundamentally ambiguous.
create type comptype as (intarray int[]);
create function array_return() returns comptype[] as $$
return [[[[1]]]];
$$ language plpython;What does the function return? It could be two-dimension array of
comptype, with a single-dimension intarray, or a single-dimension comptype,
with a two-dimension intarray.We could resolve it for simpler cases, but not the general case. The
simple cases would probably cover most things people do in practice. But if
the distinction between a tuple and a list feels natural to Python
programmers, I think it would be more clear in the long run to have people
adjust their applications.
I agree. The distinction is natural - and it is our issue, so we don't
distinguish strongly.
Regards
Pavel
Show quoted text
- Heikki
Re: PL/Python adding support for multi-dimensional arrays
On 10/11/2016 08:56 AM, Pavel Stehule wrote:
2016-10-11 7:49 GMT+02:00 Heikki Linnakangas <hlinnaka@iki.fi>:
Unfortunately there are cases that are fundamentally ambiguous.
create type comptype as (intarray int[]);
create function array_return() returns comptype[] as $$
return [[[[1]]]];
$$ language plpython;What does the function return? It could be two-dimension array of
comptype, with a single-dimension intarray, or a single-dimension comptype,
with a two-dimension intarray.We could resolve it for simpler cases, but not the general case. The
simple cases would probably cover most things people do in practice. But if
the distinction between a tuple and a list feels natural to Python
programmers, I think it would be more clear in the long run to have people
adjust their applications.I agree. The distinction is natural - and it is our issue, so we don't
distinguish strongly.
Ok, let's do that then. Here is a patch set that does that. The first is
the main patch. The second patch adds some code to give a hint, if you
do that thing that whose behavior changed. That code isn't very pretty,
but I think a good error message is absolutely required, if we are to
make this change. Does anyone have better suggestions on how to catch
the common cases of that?
Please review. Are the docs and the error messages now clear enough on
this? We'll need a mention in the release notes too, when it's time for
that.
- Heikki
Attachments:
0001-Support-multi-dimensional-arrays-in-PL-python.patchtext/x-diff; name=0001-Support-multi-dimensional-arrays-in-PL-python.patchDownload
From 945b33bd77fae0605ae9ff9d9282ab543cb16ec8 Mon Sep 17 00:00:00 2001
From: Heikki Linnakangas <heikki.linnakangas@iki.fi>
Date: Fri, 14 Oct 2016 11:51:45 +0300
Subject: [PATCH 1/2] Support multi-dimensional arrays in PL/python.
Multi-dimensional arrays can now be used as arguments to a PL/python function
(used to throw an error), and they can be returned as nested Python lists.
This makes a backwards-incompatible change to the handling of composite
types in arrays. Previously, you could return an array of composite types
as "[[col1, col2], [col1, col2]]", but now that is interpreted as a two-
dimensional array. Composite types in arrays must now be returned as
Python tuples, not lists, to resolve the ambiguity. I.e. "[(col1, col2),
(col1, col2)]".
To avoid breaking backwards-compatibility, when not necessary, () is still
accepted for arrays at the top-level, but it is always treated as a
single-dimensional array. Likewise, [] is still accepted for composite types,
when they are not in an array. Update the documentation to recommend using []
for arrays, and () for composite types, with a mention that those other things
are also accepted.
This needs to be mentioned in the release notes.
Alexey Grishchenko, Dave Cramer and me.
Discussion: <CAH38_tmbqwaUyKs9yagyRra=SMaT45FPBxk1pmTYcM0TyXGG7Q@mail.gmail.com>
---
doc/src/sgml/plpython.sgml | 41 +++-
src/pl/plpython/expected/plpython_composite.out | 6 +-
src/pl/plpython/expected/plpython_types.out | 151 ++++++++++++-
src/pl/plpython/expected/plpython_types_3.out | 151 ++++++++++++-
src/pl/plpython/plpy_typeio.c | 280 ++++++++++++++++++++----
src/pl/plpython/sql/plpython_composite.sql | 6 +-
src/pl/plpython/sql/plpython_types.sql | 86 ++++++++
7 files changed, 658 insertions(+), 63 deletions(-)
diff --git a/doc/src/sgml/plpython.sgml b/doc/src/sgml/plpython.sgml
index bb69c75..4639778 100644
--- a/doc/src/sgml/plpython.sgml
+++ b/doc/src/sgml/plpython.sgml
@@ -451,13 +451,13 @@ $$ LANGUAGE plpythonu;
<para>
SQL array values are passed into PL/Python as a Python list. To
return an SQL array value out of a PL/Python function, return a
- Python sequence, for example a list or tuple:
+ Python list:
<programlisting>
CREATE FUNCTION return_arr()
RETURNS int[]
AS $$
-return (1, 2, 3, 4, 5)
+return [1, 2, 3, 4, 5]
$$ LANGUAGE plpythonu;
SELECT return_arr();
@@ -467,6 +467,34 @@ SELECT return_arr();
(1 row)
</programlisting>
+ Multidimensional arrays are passed into PL/Python as nested Python lists.
+ A 2-dimensional array is a list of lists, for example. When returning
+ a multi-dimensional SQL array out of a PL/Python function, the inner
+ lists at each level must all be of the same size. For example:
+
+<programlisting>
+CREATE FUNCTION test_type_conversion_array_int4(x int4[]) RETURNS int4[] AS $$
+plpy.info(x, type(x))
+return x
+$$ LANGUAGE plpythonu;
+
+SELECT * FROM test_type_conversion_array_int4(ARRAY[[1,2,3],[4,5,6]]);
+INFO: ([[1, 2, 3], [4, 5, 6]], <type 'list'>)
+ test_type_conversion_array_int4
+---------------------------------
+ {{1,2,3},{4,5,6}}
+(1 row)
+</programlisting>
+
+ Other Python sequences, like tuples, are also accepted for
+ backwards-compatibility with PostgreSQL versions 9.6 and below, when
+ multi-dimensional arrays were not supported. However, they are always
+ treated as one-dimensional arrays, because they are ambiguous with
+ composite types. For the same reason, when a composite type is used in a
+ multi-dimensional array, it must be represented by a tuple, rather than a
+ list.
+ </para>
+ <para>
Note that in Python, strings are sequences, which can have
undesirable effects that might be familiar to Python programmers:
@@ -541,14 +569,19 @@ CREATE TYPE named_value AS (
CREATE FUNCTION make_pair (name text, value integer)
RETURNS named_value
AS $$
- return [ name, value ]
- # or alternatively, as tuple: return ( name, value )
+ return ( name, value )
+ # or alternatively, as tuple: return [ name, value ]
$$ LANGUAGE plpythonu;
</programlisting>
To return a SQL null for any column, insert <symbol>None</symbol> at
the corresponding position.
</para>
+ <para>
+ When an array of composite types is returned, it cannot be returned as a list,
+ because it is ambiguous whether the Python list represents a composite type,
+ or another array dimension.
+ </para>
</listitem>
</varlistentry>
diff --git a/src/pl/plpython/expected/plpython_composite.out b/src/pl/plpython/expected/plpython_composite.out
index 0ef0c21..f23dccc 100644
--- a/src/pl/plpython/expected/plpython_composite.out
+++ b/src/pl/plpython/expected/plpython_composite.out
@@ -465,13 +465,13 @@ SELECT * FROM changing_test();
-- tables of composite types
CREATE FUNCTION composite_types_table(OUT tab table_record[], OUT typ type_record[] ) RETURNS SETOF record AS $$
-yield {'tab': [['first', 1], ['second', 2]],
+yield {'tab': [('first', 1), ('second', 2)],
'typ': [{'first': 'third', 'second': 3},
{'first': 'fourth', 'second': 4}]}
-yield {'tab': [['first', 1], ['second', 2]],
+yield {'tab': [('first', 1), ('second', 2)],
'typ': [{'first': 'third', 'second': 3},
{'first': 'fourth', 'second': 4}]}
-yield {'tab': [['first', 1], ['second', 2]],
+yield {'tab': [('first', 1), ('second', 2)],
'typ': [{'first': 'third', 'second': 3},
{'first': 'fourth', 'second': 4}]}
$$ LANGUAGE plpythonu;
diff --git a/src/pl/plpython/expected/plpython_types.out b/src/pl/plpython/expected/plpython_types.out
index f0b6abd..5243aab 100644
--- a/src/pl/plpython/expected/plpython_types.out
+++ b/src/pl/plpython/expected/plpython_types.out
@@ -537,9 +537,133 @@ INFO: (None, <type 'NoneType'>)
(1 row)
SELECT * FROM test_type_conversion_array_int4(ARRAY[[1,2,3],[4,5,6]]);
-ERROR: cannot convert multidimensional array to Python list
-DETAIL: PL/Python only supports one-dimensional arrays.
-CONTEXT: PL/Python function "test_type_conversion_array_int4"
+INFO: ([[1, 2, 3], [4, 5, 6]], <type 'list'>)
+ test_type_conversion_array_int4
+---------------------------------
+ {{1,2,3},{4,5,6}}
+(1 row)
+
+SELECT * FROM test_type_conversion_array_int4(ARRAY[[[1,2,NULL],[NULL,5,6]],[[NULL,8,9],[10,11,12]]]);
+INFO: ([[[1, 2, None], [None, 5, 6]], [[None, 8, 9], [10, 11, 12]]], <type 'list'>)
+ test_type_conversion_array_int4
+---------------------------------------------------
+ {{{1,2,NULL},{NULL,5,6}},{{NULL,8,9},{10,11,12}}}
+(1 row)
+
+SELECT * FROM test_type_conversion_array_int4('[2:4]={1,2,3}');
+INFO: ([1, 2, 3], <type 'list'>)
+ test_type_conversion_array_int4
+---------------------------------
+ {1,2,3}
+(1 row)
+
+CREATE FUNCTION test_type_conversion_array_int8(x int8[]) RETURNS int8[] AS $$
+plpy.info(x, type(x))
+return x
+$$ LANGUAGE plpythonu;
+SELECT * FROM test_type_conversion_array_int8(ARRAY[[[1,2,NULL],[NULL,5,6]],[[NULL,8,9],[10,11,12]]]::int8[]);
+INFO: ([[[1L, 2L, None], [None, 5L, 6L]], [[None, 8L, 9L], [10L, 11L, 12L]]], <type 'list'>)
+ test_type_conversion_array_int8
+---------------------------------------------------
+ {{{1,2,NULL},{NULL,5,6}},{{NULL,8,9},{10,11,12}}}
+(1 row)
+
+CREATE FUNCTION test_type_conversion_array_float4(x float4[]) RETURNS float4[] AS $$
+plpy.info(x, type(x))
+return x
+$$ LANGUAGE plpythonu;
+SELECT * FROM test_type_conversion_array_float4(ARRAY[[[1.2,2.3,NULL],[NULL,5.7,6.8]],[[NULL,8.9,9.345],[10.123,11.456,12.6768]]]::float4[]);
+INFO: ([[[1.2000000476837158, 2.299999952316284, None], [None, 5.699999809265137, 6.800000190734863]], [[None, 8.899999618530273, 9.345000267028809], [10.123000144958496, 11.456000328063965, 12.676799774169922]]], <type 'list'>)
+ test_type_conversion_array_float4
+------------------------------------------------------------------------------
+ {{{1.2,2.3,NULL},{NULL,5.7,6.8}},{{NULL,8.9,9.345},{10.123,11.456,12.6768}}}
+(1 row)
+
+CREATE FUNCTION test_type_conversion_array_float8(x float8[]) RETURNS float8[] AS $$
+plpy.info(x, type(x))
+return x
+$$ LANGUAGE plpythonu;
+SELECT * FROM test_type_conversion_array_float8(ARRAY[[[1.2,2.3,NULL],[NULL,5.7,6.8]],[[NULL,8.9,9.345],[10.123,11.456,12.6768]]]::float8[]);
+INFO: ([[[1.2, 2.3, None], [None, 5.7, 6.8]], [[None, 8.9, 9.345], [10.123, 11.456, 12.6768]]], <type 'list'>)
+ test_type_conversion_array_float8
+------------------------------------------------------------------------------
+ {{{1.2,2.3,NULL},{NULL,5.7,6.8}},{{NULL,8.9,9.345},{10.123,11.456,12.6768}}}
+(1 row)
+
+CREATE FUNCTION test_type_conversion_array_date(x date[]) RETURNS date[] AS $$
+plpy.info(x, type(x))
+return x
+$$ LANGUAGE plpythonu;
+SELECT * FROM test_type_conversion_array_date(ARRAY[[['2016-09-21','2016-09-22',NULL],[NULL,'2016-10-21','2016-10-22']],
+ [[NULL,'2016-11-21','2016-10-21'],['2015-09-21','2015-09-22','2014-09-21']]]::date[]);
+INFO: ([[['09-21-2016', '09-22-2016', None], [None, '10-21-2016', '10-22-2016']], [[None, '11-21-2016', '10-21-2016'], ['09-21-2015', '09-22-2015', '09-21-2014']]], <type 'list'>)
+ test_type_conversion_array_date
+---------------------------------------------------------------------------------------------------------------------------------
+ {{{09-21-2016,09-22-2016,NULL},{NULL,10-21-2016,10-22-2016}},{{NULL,11-21-2016,10-21-2016},{09-21-2015,09-22-2015,09-21-2014}}}
+(1 row)
+
+CREATE FUNCTION test_type_conversion_array_timestamp(x timestamp[]) RETURNS timestamp[] AS $$
+plpy.info(x, type(x))
+return x
+$$ LANGUAGE plpythonu;
+SELECT * FROM test_type_conversion_array_timestamp(ARRAY[[['2016-09-21 15:34:24.078792-04','2016-10-22 11:34:24.078795-04',NULL],
+ [NULL,'2016-10-21 11:34:25.078792-04','2016-10-21 11:34:24.098792-04']],
+ [[NULL,'2016-01-21 11:34:24.078792-04','2016-11-21 11:34:24.108792-04'],
+ ['2015-09-21 11:34:24.079792-04','2014-09-21 11:34:24.078792-04','2013-09-21 11:34:24.078792-04']]]::timestamp[]);
+INFO: ([[['Wed Sep 21 15:34:24.078792 2016', 'Sat Oct 22 11:34:24.078795 2016', None], [None, 'Fri Oct 21 11:34:25.078792 2016', 'Fri Oct 21 11:34:24.098792 2016']], [[None, 'Thu Jan 21 11:34:24.078792 2016', 'Mon Nov 21 11:34:24.108792 2016'], ['Mon Sep 21 11:34:24.079792 2015', 'Sun Sep 21 11:34:24.078792 2014', 'Sat Sep 21 11:34:24.078792 2013']]], <type 'list'>)
+ test_type_conversion_array_timestamp
+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
+ {{{"Wed Sep 21 15:34:24.078792 2016","Sat Oct 22 11:34:24.078795 2016",NULL},{NULL,"Fri Oct 21 11:34:25.078792 2016","Fri Oct 21 11:34:24.098792 2016"}},{{NULL,"Thu Jan 21 11:34:24.078792 2016","Mon Nov 21 11:34:24.108792 2016"},{"Mon Sep 21 11:34:24.079792 2015","Sun Sep 21 11:34:24.078792 2014","Sat Sep 21 11:34:24.078792 2013"}}}
+(1 row)
+
+CREATE OR REPLACE FUNCTION pyreturnmultidemint4(h int4, i int4, j int4, k int4 ) RETURNS int4[] AS $BODY$
+m = [[[[x for x in range(h)] for y in range(i)] for z in range(j)] for w in range(k)]
+plpy.info(m, type(m))
+return m
+$BODY$ LANGUAGE plpythonu;
+select pyreturnmultidemint4(8,5,3,2);
+INFO: ([[[[0, 1, 2, 3, 4, 5, 6, 7], [0, 1, 2, 3, 4, 5, 6, 7], [0, 1, 2, 3, 4, 5, 6, 7], [0, 1, 2, 3, 4, 5, 6, 7], [0, 1, 2, 3, 4, 5, 6, 7]], [[0, 1, 2, 3, 4, 5, 6, 7], [0, 1, 2, 3, 4, 5, 6, 7], [0, 1, 2, 3, 4, 5, 6, 7], [0, 1, 2, 3, 4, 5, 6, 7], [0, 1, 2, 3, 4, 5, 6, 7]], [[0, 1, 2, 3, 4, 5, 6, 7], [0, 1, 2, 3, 4, 5, 6, 7], [0, 1, 2, 3, 4, 5, 6, 7], [0, 1, 2, 3, 4, 5, 6, 7], [0, 1, 2, 3, 4, 5, 6, 7]]], [[[0, 1, 2, 3, 4, 5, 6, 7], [0, 1, 2, 3, 4, 5, 6, 7], [0, 1, 2, 3, 4, 5, 6, 7], [0, 1, 2, 3, 4, 5, 6, 7], [0, 1, 2, 3, 4, 5, 6, 7]], [[0, 1, 2, 3, 4, 5, 6, 7], [0, 1, 2, 3, 4, 5, 6, 7], [0, 1, 2, 3, 4, 5, 6, 7], [0, 1, 2, 3, 4, 5, 6, 7], [0, 1, 2, 3, 4, 5, 6, 7]], [[0, 1, 2, 3, 4, 5, 6, 7], [0, 1, 2, 3, 4, 5, 6, 7], [0, 1, 2, 3, 4, 5, 6, 7], [0, 1, 2, 3, 4, 5, 6, 7], [0, 1, 2, 3, 4, 5, 6, 7]]]], <type 'list'>)
+ pyreturnmultidemint4
+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
+ {{{{0,1,2,3,4,5,6,7},{0,1,2,3,4,5,6,7},{0,1,2,3,4,5,6,7},{0,1,2,3,4,5,6,7},{0,1,2,3,4,5,6,7}},{{0,1,2,3,4,5,6,7},{0,1,2,3,4,5,6,7},{0,1,2,3,4,5,6,7},{0,1,2,3,4,5,6,7},{0,1,2,3,4,5,6,7}},{{0,1,2,3,4,5,6,7},{0,1,2,3,4,5,6,7},{0,1,2,3,4,5,6,7},{0,1,2,3,4,5,6,7},{0,1,2,3,4,5,6,7}}},{{{0,1,2,3,4,5,6,7},{0,1,2,3,4,5,6,7},{0,1,2,3,4,5,6,7},{0,1,2,3,4,5,6,7},{0,1,2,3,4,5,6,7}},{{0,1,2,3,4,5,6,7},{0,1,2,3,4,5,6,7},{0,1,2,3,4,5,6,7},{0,1,2,3,4,5,6,7},{0,1,2,3,4,5,6,7}},{{0,1,2,3,4,5,6,7},{0,1,2,3,4,5,6,7},{0,1,2,3,4,5,6,7},{0,1,2,3,4,5,6,7},{0,1,2,3,4,5,6,7}}}}
+(1 row)
+
+CREATE OR REPLACE FUNCTION pyreturnmultidemint8(h int4, i int4, j int4, k int4 ) RETURNS int8[] AS $BODY$
+m = [[[[x for x in range(h)] for y in range(i)] for z in range(j)] for w in range(k)]
+plpy.info(m, type(m))
+return m
+$BODY$ LANGUAGE plpythonu;
+select pyreturnmultidemint8(5,5,3,2);
+INFO: ([[[[0, 1, 2, 3, 4], [0, 1, 2, 3, 4], [0, 1, 2, 3, 4], [0, 1, 2, 3, 4], [0, 1, 2, 3, 4]], [[0, 1, 2, 3, 4], [0, 1, 2, 3, 4], [0, 1, 2, 3, 4], [0, 1, 2, 3, 4], [0, 1, 2, 3, 4]], [[0, 1, 2, 3, 4], [0, 1, 2, 3, 4], [0, 1, 2, 3, 4], [0, 1, 2, 3, 4], [0, 1, 2, 3, 4]]], [[[0, 1, 2, 3, 4], [0, 1, 2, 3, 4], [0, 1, 2, 3, 4], [0, 1, 2, 3, 4], [0, 1, 2, 3, 4]], [[0, 1, 2, 3, 4], [0, 1, 2, 3, 4], [0, 1, 2, 3, 4], [0, 1, 2, 3, 4], [0, 1, 2, 3, 4]], [[0, 1, 2, 3, 4], [0, 1, 2, 3, 4], [0, 1, 2, 3, 4], [0, 1, 2, 3, 4], [0, 1, 2, 3, 4]]]], <type 'list'>)
+ pyreturnmultidemint8
+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
+ {{{{0,1,2,3,4},{0,1,2,3,4},{0,1,2,3,4},{0,1,2,3,4},{0,1,2,3,4}},{{0,1,2,3,4},{0,1,2,3,4},{0,1,2,3,4},{0,1,2,3,4},{0,1,2,3,4}},{{0,1,2,3,4},{0,1,2,3,4},{0,1,2,3,4},{0,1,2,3,4},{0,1,2,3,4}}},{{{0,1,2,3,4},{0,1,2,3,4},{0,1,2,3,4},{0,1,2,3,4},{0,1,2,3,4}},{{0,1,2,3,4},{0,1,2,3,4},{0,1,2,3,4},{0,1,2,3,4},{0,1,2,3,4}},{{0,1,2,3,4},{0,1,2,3,4},{0,1,2,3,4},{0,1,2,3,4},{0,1,2,3,4}}}}
+(1 row)
+
+CREATE OR REPLACE FUNCTION pyreturnmultidemfloat4(h int4, i int4, j int4, k int4 ) RETURNS float4[] AS $BODY$
+m = [[[[x for x in range(h)] for y in range(i)] for z in range(j)] for w in range(k)]
+plpy.info(m, type(m))
+return m
+$BODY$ LANGUAGE plpythonu;
+select pyreturnmultidemfloat4(6,5,3,2);
+INFO: ([[[[0, 1, 2, 3, 4, 5], [0, 1, 2, 3, 4, 5], [0, 1, 2, 3, 4, 5], [0, 1, 2, 3, 4, 5], [0, 1, 2, 3, 4, 5]], [[0, 1, 2, 3, 4, 5], [0, 1, 2, 3, 4, 5], [0, 1, 2, 3, 4, 5], [0, 1, 2, 3, 4, 5], [0, 1, 2, 3, 4, 5]], [[0, 1, 2, 3, 4, 5], [0, 1, 2, 3, 4, 5], [0, 1, 2, 3, 4, 5], [0, 1, 2, 3, 4, 5], [0, 1, 2, 3, 4, 5]]], [[[0, 1, 2, 3, 4, 5], [0, 1, 2, 3, 4, 5], [0, 1, 2, 3, 4, 5], [0, 1, 2, 3, 4, 5], [0, 1, 2, 3, 4, 5]], [[0, 1, 2, 3, 4, 5], [0, 1, 2, 3, 4, 5], [0, 1, 2, 3, 4, 5], [0, 1, 2, 3, 4, 5], [0, 1, 2, 3, 4, 5]], [[0, 1, 2, 3, 4, 5], [0, 1, 2, 3, 4, 5], [0, 1, 2, 3, 4, 5], [0, 1, 2, 3, 4, 5], [0, 1, 2, 3, 4, 5]]]], <type 'list'>)
+ pyreturnmultidemfloat4
+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
+ {{{{0,1,2,3,4,5},{0,1,2,3,4,5},{0,1,2,3,4,5},{0,1,2,3,4,5},{0,1,2,3,4,5}},{{0,1,2,3,4,5},{0,1,2,3,4,5},{0,1,2,3,4,5},{0,1,2,3,4,5},{0,1,2,3,4,5}},{{0,1,2,3,4,5},{0,1,2,3,4,5},{0,1,2,3,4,5},{0,1,2,3,4,5},{0,1,2,3,4,5}}},{{{0,1,2,3,4,5},{0,1,2,3,4,5},{0,1,2,3,4,5},{0,1,2,3,4,5},{0,1,2,3,4,5}},{{0,1,2,3,4,5},{0,1,2,3,4,5},{0,1,2,3,4,5},{0,1,2,3,4,5},{0,1,2,3,4,5}},{{0,1,2,3,4,5},{0,1,2,3,4,5},{0,1,2,3,4,5},{0,1,2,3,4,5},{0,1,2,3,4,5}}}}
+(1 row)
+
+CREATE OR REPLACE FUNCTION pyreturnmultidemfloat8(h int4, i int4, j int4, k int4 ) RETURNS float8[] AS $BODY$
+m = [[[[x for x in range(h)] for y in range(i)] for z in range(j)] for w in range(k)]
+plpy.info(m, type(m))
+return m
+$BODY$ LANGUAGE plpythonu;
+select pyreturnmultidemfloat8(7,5,3,2);
+INFO: ([[[[0, 1, 2, 3, 4, 5, 6], [0, 1, 2, 3, 4, 5, 6], [0, 1, 2, 3, 4, 5, 6], [0, 1, 2, 3, 4, 5, 6], [0, 1, 2, 3, 4, 5, 6]], [[0, 1, 2, 3, 4, 5, 6], [0, 1, 2, 3, 4, 5, 6], [0, 1, 2, 3, 4, 5, 6], [0, 1, 2, 3, 4, 5, 6], [0, 1, 2, 3, 4, 5, 6]], [[0, 1, 2, 3, 4, 5, 6], [0, 1, 2, 3, 4, 5, 6], [0, 1, 2, 3, 4, 5, 6], [0, 1, 2, 3, 4, 5, 6], [0, 1, 2, 3, 4, 5, 6]]], [[[0, 1, 2, 3, 4, 5, 6], [0, 1, 2, 3, 4, 5, 6], [0, 1, 2, 3, 4, 5, 6], [0, 1, 2, 3, 4, 5, 6], [0, 1, 2, 3, 4, 5, 6]], [[0, 1, 2, 3, 4, 5, 6], [0, 1, 2, 3, 4, 5, 6], [0, 1, 2, 3, 4, 5, 6], [0, 1, 2, 3, 4, 5, 6], [0, 1, 2, 3, 4, 5, 6]], [[0, 1, 2, 3, 4, 5, 6], [0, 1, 2, 3, 4, 5, 6], [0, 1, 2, 3, 4, 5, 6], [0, 1, 2, 3, 4, 5, 6], [0, 1, 2, 3, 4, 5, 6]]]], <type 'list'>)
+ pyreturnmultidemfloat8
+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
+ {{{{0,1,2,3,4,5,6},{0,1,2,3,4,5,6},{0,1,2,3,4,5,6},{0,1,2,3,4,5,6},{0,1,2,3,4,5,6}},{{0,1,2,3,4,5,6},{0,1,2,3,4,5,6},{0,1,2,3,4,5,6},{0,1,2,3,4,5,6},{0,1,2,3,4,5,6}},{{0,1,2,3,4,5,6},{0,1,2,3,4,5,6},{0,1,2,3,4,5,6},{0,1,2,3,4,5,6},{0,1,2,3,4,5,6}}},{{{0,1,2,3,4,5,6},{0,1,2,3,4,5,6},{0,1,2,3,4,5,6},{0,1,2,3,4,5,6},{0,1,2,3,4,5,6}},{{0,1,2,3,4,5,6},{0,1,2,3,4,5,6},{0,1,2,3,4,5,6},{0,1,2,3,4,5,6},{0,1,2,3,4,5,6}},{{0,1,2,3,4,5,6},{0,1,2,3,4,5,6},{0,1,2,3,4,5,6},{0,1,2,3,4,5,6},{0,1,2,3,4,5,6}}}}
+(1 row)
+
CREATE FUNCTION test_type_conversion_array_text(x text[]) RETURNS text[] AS $$
plpy.info(x, type(x))
return x
@@ -551,6 +675,13 @@ INFO: (['foo', 'bar'], <type 'list'>)
{foo,bar}
(1 row)
+SELECT * FROM test_type_conversion_array_text(ARRAY[['foo', 'bar'],['foo2', 'bar2']]);
+INFO: ([['foo', 'bar'], ['foo2', 'bar2']], <type 'list'>)
+ test_type_conversion_array_text
+---------------------------------
+ {{foo,bar},{foo2,bar2}}
+(1 row)
+
CREATE FUNCTION test_type_conversion_array_bytea(x bytea[]) RETURNS bytea[] AS $$
plpy.info(x, type(x))
return x
@@ -578,6 +709,20 @@ SELECT * FROM test_type_conversion_array_mixed2();
ERROR: invalid input syntax for integer: "abc"
CONTEXT: while creating return value
PL/Python function "test_type_conversion_array_mixed2"
+CREATE FUNCTION test_type_conversion_mdarray_malformed() RETURNS int[] AS $$
+return [[1,2,3],[4,5]]
+$$ LANGUAGE plpythonu;
+SELECT * FROM test_type_conversion_mdarray_malformed();
+ERROR: multidimensional arrays must have array expressions with matching dimensions. PL/Python function return value has sequence length 2 while expected 3
+CONTEXT: while creating return value
+PL/Python function "test_type_conversion_mdarray_malformed"
+CREATE FUNCTION test_type_conversion_mdarray_toodeep() RETURNS int[] AS $$
+return [[[[[[[1]]]]]]]
+$$ LANGUAGE plpythonu;
+SELECT * FROM test_type_conversion_mdarray_toodeep();
+ERROR: number of array dimensions exceeds the maximum allowed (6)
+CONTEXT: while creating return value
+PL/Python function "test_type_conversion_mdarray_toodeep"
CREATE FUNCTION test_type_conversion_array_record() RETURNS type_record[] AS $$
return [{'first': 'one', 'second': 42}, {'first': 'two', 'second': 11}]
$$ LANGUAGE plpythonu;
diff --git a/src/pl/plpython/expected/plpython_types_3.out b/src/pl/plpython/expected/plpython_types_3.out
index 56b78e1..1592219 100644
--- a/src/pl/plpython/expected/plpython_types_3.out
+++ b/src/pl/plpython/expected/plpython_types_3.out
@@ -537,9 +537,133 @@ INFO: (None, <class 'NoneType'>)
(1 row)
SELECT * FROM test_type_conversion_array_int4(ARRAY[[1,2,3],[4,5,6]]);
-ERROR: cannot convert multidimensional array to Python list
-DETAIL: PL/Python only supports one-dimensional arrays.
-CONTEXT: PL/Python function "test_type_conversion_array_int4"
+INFO: ([[1, 2, 3], [4, 5, 6]], <class 'list'>)
+ test_type_conversion_array_int4
+---------------------------------
+ {{1,2,3},{4,5,6}}
+(1 row)
+
+SELECT * FROM test_type_conversion_array_int4(ARRAY[[[1,2,NULL],[NULL,5,6]],[[NULL,8,9],[10,11,12]]]);
+INFO: ([[[1, 2, None], [None, 5, 6]], [[None, 8, 9], [10, 11, 12]]], <class 'list'>)
+ test_type_conversion_array_int4
+---------------------------------------------------
+ {{{1,2,NULL},{NULL,5,6}},{{NULL,8,9},{10,11,12}}}
+(1 row)
+
+SELECT * FROM test_type_conversion_array_int4('[2:4]={1,2,3}');
+INFO: ([1, 2, 3], <class 'list'>)
+ test_type_conversion_array_int4
+---------------------------------
+ {1,2,3}
+(1 row)
+
+CREATE FUNCTION test_type_conversion_array_int8(x int8[]) RETURNS int8[] AS $$
+plpy.info(x, type(x))
+return x
+$$ LANGUAGE plpython3u;
+SELECT * FROM test_type_conversion_array_int8(ARRAY[[[1,2,NULL],[NULL,5,6]],[[NULL,8,9],[10,11,12]]]::int8[]);
+INFO: ([[[1, 2, None], [None, 5, 6]], [[None, 8, 9], [10, 11, 12]]], <class 'list'>)
+ test_type_conversion_array_int8
+---------------------------------------------------
+ {{{1,2,NULL},{NULL,5,6}},{{NULL,8,9},{10,11,12}}}
+(1 row)
+
+CREATE FUNCTION test_type_conversion_array_float4(x float4[]) RETURNS float4[] AS $$
+plpy.info(x, type(x))
+return x
+$$ LANGUAGE plpython3u;
+SELECT * FROM test_type_conversion_array_float4(ARRAY[[[1.2,2.3,NULL],[NULL,5.7,6.8]],[[NULL,8.9,9.345],[10.123,11.456,12.6768]]]::float4[]);
+INFO: ([[[1.2000000476837158, 2.299999952316284, None], [None, 5.699999809265137, 6.800000190734863]], [[None, 8.899999618530273, 9.345000267028809], [10.123000144958496, 11.456000328063965, 12.676799774169922]]], <class 'list'>)
+ test_type_conversion_array_float4
+------------------------------------------------------------------------------
+ {{{1.2,2.3,NULL},{NULL,5.7,6.8}},{{NULL,8.9,9.345},{10.123,11.456,12.6768}}}
+(1 row)
+
+CREATE FUNCTION test_type_conversion_array_float8(x float8[]) RETURNS float8[] AS $$
+plpy.info(x, type(x))
+return x
+$$ LANGUAGE plpython3u;
+SELECT * FROM test_type_conversion_array_float8(ARRAY[[[1.2,2.3,NULL],[NULL,5.7,6.8]],[[NULL,8.9,9.345],[10.123,11.456,12.6768]]]::float8[]);
+INFO: ([[[1.2, 2.3, None], [None, 5.7, 6.8]], [[None, 8.9, 9.345], [10.123, 11.456, 12.6768]]], <class 'list'>)
+ test_type_conversion_array_float8
+------------------------------------------------------------------------------
+ {{{1.2,2.3,NULL},{NULL,5.7,6.8}},{{NULL,8.9,9.345},{10.123,11.456,12.6768}}}
+(1 row)
+
+CREATE FUNCTION test_type_conversion_array_date(x date[]) RETURNS date[] AS $$
+plpy.info(x, type(x))
+return x
+$$ LANGUAGE plpython3u;
+SELECT * FROM test_type_conversion_array_date(ARRAY[[['2016-09-21','2016-09-22',NULL],[NULL,'2016-10-21','2016-10-22']],
+ [[NULL,'2016-11-21','2016-10-21'],['2015-09-21','2015-09-22','2014-09-21']]]::date[]);
+INFO: ([[['09-21-2016', '09-22-2016', None], [None, '10-21-2016', '10-22-2016']], [[None, '11-21-2016', '10-21-2016'], ['09-21-2015', '09-22-2015', '09-21-2014']]], <class 'list'>)
+ test_type_conversion_array_date
+---------------------------------------------------------------------------------------------------------------------------------
+ {{{09-21-2016,09-22-2016,NULL},{NULL,10-21-2016,10-22-2016}},{{NULL,11-21-2016,10-21-2016},{09-21-2015,09-22-2015,09-21-2014}}}
+(1 row)
+
+CREATE FUNCTION test_type_conversion_array_timestamp(x timestamp[]) RETURNS timestamp[] AS $$
+plpy.info(x, type(x))
+return x
+$$ LANGUAGE plpython3u;
+SELECT * FROM test_type_conversion_array_timestamp(ARRAY[[['2016-09-21 15:34:24.078792-04','2016-10-22 11:34:24.078795-04',NULL],
+ [NULL,'2016-10-21 11:34:25.078792-04','2016-10-21 11:34:24.098792-04']],
+ [[NULL,'2016-01-21 11:34:24.078792-04','2016-11-21 11:34:24.108792-04'],
+ ['2015-09-21 11:34:24.079792-04','2014-09-21 11:34:24.078792-04','2013-09-21 11:34:24.078792-04']]]::timestamp[]);
+INFO: ([[['Wed Sep 21 15:34:24.078792 2016', 'Sat Oct 22 11:34:24.078795 2016', None], [None, 'Fri Oct 21 11:34:25.078792 2016', 'Fri Oct 21 11:34:24.098792 2016']], [[None, 'Thu Jan 21 11:34:24.078792 2016', 'Mon Nov 21 11:34:24.108792 2016'], ['Mon Sep 21 11:34:24.079792 2015', 'Sun Sep 21 11:34:24.078792 2014', 'Sat Sep 21 11:34:24.078792 2013']]], <class 'list'>)
+ test_type_conversion_array_timestamp
+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
+ {{{"Wed Sep 21 15:34:24.078792 2016","Sat Oct 22 11:34:24.078795 2016",NULL},{NULL,"Fri Oct 21 11:34:25.078792 2016","Fri Oct 21 11:34:24.098792 2016"}},{{NULL,"Thu Jan 21 11:34:24.078792 2016","Mon Nov 21 11:34:24.108792 2016"},{"Mon Sep 21 11:34:24.079792 2015","Sun Sep 21 11:34:24.078792 2014","Sat Sep 21 11:34:24.078792 2013"}}}
+(1 row)
+
+CREATE OR REPLACE FUNCTION pyreturnmultidemint4(h int4, i int4, j int4, k int4 ) RETURNS int4[] AS $BODY$
+m = [[[[x for x in range(h)] for y in range(i)] for z in range(j)] for w in range(k)]
+plpy.info(m, type(m))
+return m
+$BODY$ LANGUAGE plpython3u;
+select pyreturnmultidemint4(8,5,3,2);
+INFO: ([[[[0, 1, 2, 3, 4, 5, 6, 7], [0, 1, 2, 3, 4, 5, 6, 7], [0, 1, 2, 3, 4, 5, 6, 7], [0, 1, 2, 3, 4, 5, 6, 7], [0, 1, 2, 3, 4, 5, 6, 7]], [[0, 1, 2, 3, 4, 5, 6, 7], [0, 1, 2, 3, 4, 5, 6, 7], [0, 1, 2, 3, 4, 5, 6, 7], [0, 1, 2, 3, 4, 5, 6, 7], [0, 1, 2, 3, 4, 5, 6, 7]], [[0, 1, 2, 3, 4, 5, 6, 7], [0, 1, 2, 3, 4, 5, 6, 7], [0, 1, 2, 3, 4, 5, 6, 7], [0, 1, 2, 3, 4, 5, 6, 7], [0, 1, 2, 3, 4, 5, 6, 7]]], [[[0, 1, 2, 3, 4, 5, 6, 7], [0, 1, 2, 3, 4, 5, 6, 7], [0, 1, 2, 3, 4, 5, 6, 7], [0, 1, 2, 3, 4, 5, 6, 7], [0, 1, 2, 3, 4, 5, 6, 7]], [[0, 1, 2, 3, 4, 5, 6, 7], [0, 1, 2, 3, 4, 5, 6, 7], [0, 1, 2, 3, 4, 5, 6, 7], [0, 1, 2, 3, 4, 5, 6, 7], [0, 1, 2, 3, 4, 5, 6, 7]], [[0, 1, 2, 3, 4, 5, 6, 7], [0, 1, 2, 3, 4, 5, 6, 7], [0, 1, 2, 3, 4, 5, 6, 7], [0, 1, 2, 3, 4, 5, 6, 7], [0, 1, 2, 3, 4, 5, 6, 7]]]], <class 'list'>)
+ pyreturnmultidemint4
+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
+ {{{{0,1,2,3,4,5,6,7},{0,1,2,3,4,5,6,7},{0,1,2,3,4,5,6,7},{0,1,2,3,4,5,6,7},{0,1,2,3,4,5,6,7}},{{0,1,2,3,4,5,6,7},{0,1,2,3,4,5,6,7},{0,1,2,3,4,5,6,7},{0,1,2,3,4,5,6,7},{0,1,2,3,4,5,6,7}},{{0,1,2,3,4,5,6,7},{0,1,2,3,4,5,6,7},{0,1,2,3,4,5,6,7},{0,1,2,3,4,5,6,7},{0,1,2,3,4,5,6,7}}},{{{0,1,2,3,4,5,6,7},{0,1,2,3,4,5,6,7},{0,1,2,3,4,5,6,7},{0,1,2,3,4,5,6,7},{0,1,2,3,4,5,6,7}},{{0,1,2,3,4,5,6,7},{0,1,2,3,4,5,6,7},{0,1,2,3,4,5,6,7},{0,1,2,3,4,5,6,7},{0,1,2,3,4,5,6,7}},{{0,1,2,3,4,5,6,7},{0,1,2,3,4,5,6,7},{0,1,2,3,4,5,6,7},{0,1,2,3,4,5,6,7},{0,1,2,3,4,5,6,7}}}}
+(1 row)
+
+CREATE OR REPLACE FUNCTION pyreturnmultidemint8(h int4, i int4, j int4, k int4 ) RETURNS int8[] AS $BODY$
+m = [[[[x for x in range(h)] for y in range(i)] for z in range(j)] for w in range(k)]
+plpy.info(m, type(m))
+return m
+$BODY$ LANGUAGE plpython3u;
+select pyreturnmultidemint8(5,5,3,2);
+INFO: ([[[[0, 1, 2, 3, 4], [0, 1, 2, 3, 4], [0, 1, 2, 3, 4], [0, 1, 2, 3, 4], [0, 1, 2, 3, 4]], [[0, 1, 2, 3, 4], [0, 1, 2, 3, 4], [0, 1, 2, 3, 4], [0, 1, 2, 3, 4], [0, 1, 2, 3, 4]], [[0, 1, 2, 3, 4], [0, 1, 2, 3, 4], [0, 1, 2, 3, 4], [0, 1, 2, 3, 4], [0, 1, 2, 3, 4]]], [[[0, 1, 2, 3, 4], [0, 1, 2, 3, 4], [0, 1, 2, 3, 4], [0, 1, 2, 3, 4], [0, 1, 2, 3, 4]], [[0, 1, 2, 3, 4], [0, 1, 2, 3, 4], [0, 1, 2, 3, 4], [0, 1, 2, 3, 4], [0, 1, 2, 3, 4]], [[0, 1, 2, 3, 4], [0, 1, 2, 3, 4], [0, 1, 2, 3, 4], [0, 1, 2, 3, 4], [0, 1, 2, 3, 4]]]], <class 'list'>)
+ pyreturnmultidemint8
+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
+ {{{{0,1,2,3,4},{0,1,2,3,4},{0,1,2,3,4},{0,1,2,3,4},{0,1,2,3,4}},{{0,1,2,3,4},{0,1,2,3,4},{0,1,2,3,4},{0,1,2,3,4},{0,1,2,3,4}},{{0,1,2,3,4},{0,1,2,3,4},{0,1,2,3,4},{0,1,2,3,4},{0,1,2,3,4}}},{{{0,1,2,3,4},{0,1,2,3,4},{0,1,2,3,4},{0,1,2,3,4},{0,1,2,3,4}},{{0,1,2,3,4},{0,1,2,3,4},{0,1,2,3,4},{0,1,2,3,4},{0,1,2,3,4}},{{0,1,2,3,4},{0,1,2,3,4},{0,1,2,3,4},{0,1,2,3,4},{0,1,2,3,4}}}}
+(1 row)
+
+CREATE OR REPLACE FUNCTION pyreturnmultidemfloat4(h int4, i int4, j int4, k int4 ) RETURNS float4[] AS $BODY$
+m = [[[[x for x in range(h)] for y in range(i)] for z in range(j)] for w in range(k)]
+plpy.info(m, type(m))
+return m
+$BODY$ LANGUAGE plpython3u;
+select pyreturnmultidemfloat4(6,5,3,2);
+INFO: ([[[[0, 1, 2, 3, 4, 5], [0, 1, 2, 3, 4, 5], [0, 1, 2, 3, 4, 5], [0, 1, 2, 3, 4, 5], [0, 1, 2, 3, 4, 5]], [[0, 1, 2, 3, 4, 5], [0, 1, 2, 3, 4, 5], [0, 1, 2, 3, 4, 5], [0, 1, 2, 3, 4, 5], [0, 1, 2, 3, 4, 5]], [[0, 1, 2, 3, 4, 5], [0, 1, 2, 3, 4, 5], [0, 1, 2, 3, 4, 5], [0, 1, 2, 3, 4, 5], [0, 1, 2, 3, 4, 5]]], [[[0, 1, 2, 3, 4, 5], [0, 1, 2, 3, 4, 5], [0, 1, 2, 3, 4, 5], [0, 1, 2, 3, 4, 5], [0, 1, 2, 3, 4, 5]], [[0, 1, 2, 3, 4, 5], [0, 1, 2, 3, 4, 5], [0, 1, 2, 3, 4, 5], [0, 1, 2, 3, 4, 5], [0, 1, 2, 3, 4, 5]], [[0, 1, 2, 3, 4, 5], [0, 1, 2, 3, 4, 5], [0, 1, 2, 3, 4, 5], [0, 1, 2, 3, 4, 5], [0, 1, 2, 3, 4, 5]]]], <class 'list'>)
+ pyreturnmultidemfloat4
+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
+ {{{{0,1,2,3,4,5},{0,1,2,3,4,5},{0,1,2,3,4,5},{0,1,2,3,4,5},{0,1,2,3,4,5}},{{0,1,2,3,4,5},{0,1,2,3,4,5},{0,1,2,3,4,5},{0,1,2,3,4,5},{0,1,2,3,4,5}},{{0,1,2,3,4,5},{0,1,2,3,4,5},{0,1,2,3,4,5},{0,1,2,3,4,5},{0,1,2,3,4,5}}},{{{0,1,2,3,4,5},{0,1,2,3,4,5},{0,1,2,3,4,5},{0,1,2,3,4,5},{0,1,2,3,4,5}},{{0,1,2,3,4,5},{0,1,2,3,4,5},{0,1,2,3,4,5},{0,1,2,3,4,5},{0,1,2,3,4,5}},{{0,1,2,3,4,5},{0,1,2,3,4,5},{0,1,2,3,4,5},{0,1,2,3,4,5},{0,1,2,3,4,5}}}}
+(1 row)
+
+CREATE OR REPLACE FUNCTION pyreturnmultidemfloat8(h int4, i int4, j int4, k int4 ) RETURNS float8[] AS $BODY$
+m = [[[[x for x in range(h)] for y in range(i)] for z in range(j)] for w in range(k)]
+plpy.info(m, type(m))
+return m
+$BODY$ LANGUAGE plpython3u;
+select pyreturnmultidemfloat8(7,5,3,2);
+INFO: ([[[[0, 1, 2, 3, 4, 5, 6], [0, 1, 2, 3, 4, 5, 6], [0, 1, 2, 3, 4, 5, 6], [0, 1, 2, 3, 4, 5, 6], [0, 1, 2, 3, 4, 5, 6]], [[0, 1, 2, 3, 4, 5, 6], [0, 1, 2, 3, 4, 5, 6], [0, 1, 2, 3, 4, 5, 6], [0, 1, 2, 3, 4, 5, 6], [0, 1, 2, 3, 4, 5, 6]], [[0, 1, 2, 3, 4, 5, 6], [0, 1, 2, 3, 4, 5, 6], [0, 1, 2, 3, 4, 5, 6], [0, 1, 2, 3, 4, 5, 6], [0, 1, 2, 3, 4, 5, 6]]], [[[0, 1, 2, 3, 4, 5, 6], [0, 1, 2, 3, 4, 5, 6], [0, 1, 2, 3, 4, 5, 6], [0, 1, 2, 3, 4, 5, 6], [0, 1, 2, 3, 4, 5, 6]], [[0, 1, 2, 3, 4, 5, 6], [0, 1, 2, 3, 4, 5, 6], [0, 1, 2, 3, 4, 5, 6], [0, 1, 2, 3, 4, 5, 6], [0, 1, 2, 3, 4, 5, 6]], [[0, 1, 2, 3, 4, 5, 6], [0, 1, 2, 3, 4, 5, 6], [0, 1, 2, 3, 4, 5, 6], [0, 1, 2, 3, 4, 5, 6], [0, 1, 2, 3, 4, 5, 6]]]], <class 'list'>)
+ pyreturnmultidemfloat8
+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
+ {{{{0,1,2,3,4,5,6},{0,1,2,3,4,5,6},{0,1,2,3,4,5,6},{0,1,2,3,4,5,6},{0,1,2,3,4,5,6}},{{0,1,2,3,4,5,6},{0,1,2,3,4,5,6},{0,1,2,3,4,5,6},{0,1,2,3,4,5,6},{0,1,2,3,4,5,6}},{{0,1,2,3,4,5,6},{0,1,2,3,4,5,6},{0,1,2,3,4,5,6},{0,1,2,3,4,5,6},{0,1,2,3,4,5,6}}},{{{0,1,2,3,4,5,6},{0,1,2,3,4,5,6},{0,1,2,3,4,5,6},{0,1,2,3,4,5,6},{0,1,2,3,4,5,6}},{{0,1,2,3,4,5,6},{0,1,2,3,4,5,6},{0,1,2,3,4,5,6},{0,1,2,3,4,5,6},{0,1,2,3,4,5,6}},{{0,1,2,3,4,5,6},{0,1,2,3,4,5,6},{0,1,2,3,4,5,6},{0,1,2,3,4,5,6},{0,1,2,3,4,5,6}}}}
+(1 row)
+
CREATE FUNCTION test_type_conversion_array_text(x text[]) RETURNS text[] AS $$
plpy.info(x, type(x))
return x
@@ -551,6 +675,13 @@ INFO: (['foo', 'bar'], <class 'list'>)
{foo,bar}
(1 row)
+SELECT * FROM test_type_conversion_array_text(ARRAY[['foo', 'bar'],['foo2', 'bar2']]);
+INFO: ([['foo', 'bar'], ['foo2', 'bar2']], <class 'list'>)
+ test_type_conversion_array_text
+---------------------------------
+ {{foo,bar},{foo2,bar2}}
+(1 row)
+
CREATE FUNCTION test_type_conversion_array_bytea(x bytea[]) RETURNS bytea[] AS $$
plpy.info(x, type(x))
return x
@@ -578,6 +709,20 @@ SELECT * FROM test_type_conversion_array_mixed2();
ERROR: invalid input syntax for integer: "abc"
CONTEXT: while creating return value
PL/Python function "test_type_conversion_array_mixed2"
+CREATE FUNCTION test_type_conversion_mdarray_malformed() RETURNS int[] AS $$
+return [[1,2,3],[4,5]]
+$$ LANGUAGE plpython3u;
+SELECT * FROM test_type_conversion_mdarray_malformed();
+ERROR: multidimensional arrays must have array expressions with matching dimensions. PL/Python function return value has sequence length 2 while expected 3
+CONTEXT: while creating return value
+PL/Python function "test_type_conversion_mdarray_malformed"
+CREATE FUNCTION test_type_conversion_mdarray_toodeep() RETURNS int[] AS $$
+return [[[[[[[1]]]]]]]
+$$ LANGUAGE plpython3u;
+SELECT * FROM test_type_conversion_mdarray_toodeep();
+ERROR: number of array dimensions exceeds the maximum allowed (6)
+CONTEXT: while creating return value
+PL/Python function "test_type_conversion_mdarray_toodeep"
CREATE FUNCTION test_type_conversion_array_record() RETURNS type_record[] AS $$
return [{'first': 'one', 'second': 42}, {'first': 'two', 'second': 11}]
$$ LANGUAGE plpython3u;
diff --git a/src/pl/plpython/plpy_typeio.c b/src/pl/plpython/plpy_typeio.c
index 70f2e6d..56cfdd1 100644
--- a/src/pl/plpython/plpy_typeio.c
+++ b/src/pl/plpython/plpy_typeio.c
@@ -45,6 +45,8 @@ static PyObject *PLyBytes_FromBytea(PLyDatumToOb *arg, Datum d);
static PyObject *PLyString_FromDatum(PLyDatumToOb *arg, Datum d);
static PyObject *PLyObject_FromTransform(PLyDatumToOb *arg, Datum d);
static PyObject *PLyList_FromArray(PLyDatumToOb *arg, Datum d);
+static PyObject *PLyList_FromArray_recurse(PLyDatumToOb *elm, int *dims, int ndim, int dim,
+ char **dataptr_p, bits8 **bitmap_p, int *bitmask_p);
/* conversion from Python objects to Datums */
static Datum PLyObject_ToBool(PLyObToDatum *arg, int32 typmod, PyObject *plrv);
@@ -53,6 +55,9 @@ static Datum PLyObject_ToComposite(PLyObToDatum *arg, int32 typmod, PyObject *pl
static Datum PLyObject_ToDatum(PLyObToDatum *arg, int32 typmod, PyObject *plrv);
static Datum PLyObject_ToTransform(PLyObToDatum *arg, int32 typmod, PyObject *plrv);
static Datum PLySequence_ToArray(PLyObToDatum *arg, int32 typmod, PyObject *plrv);
+static void PLySequence_ToArray_recurse(PLyObToDatum *elm, PyObject *list,
+ int *dims, int ndim, int dim,
+ Datum *elems, bool *nulls, int *currelem);
/* conversion from Python objects to composite Datums (used by triggers and SRFs) */
static Datum PLyString_ToComposite(PLyTypeInfo *info, TupleDesc desc, PyObject *string);
@@ -631,43 +636,105 @@ PLyList_FromArray(PLyDatumToOb *arg, Datum d)
{
ArrayType *array = DatumGetArrayTypeP(d);
PLyDatumToOb *elm = arg->elm;
- PyObject *list;
- int length;
- int lbound;
- int i;
+ int ndim;
+ int *dims;
+ char *dataptr;
+ bits8 *bitmap;
+ int bitmask;
if (ARR_NDIM(array) == 0)
return PyList_New(0);
- if (ARR_NDIM(array) != 1)
- ereport(ERROR,
- (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
- errmsg("cannot convert multidimensional array to Python list"),
- errdetail("PL/Python only supports one-dimensional arrays.")));
+ /* Array dimensions and left bounds */
+ ndim = ARR_NDIM(array);
+ dims = ARR_DIMS(array);
+ Assert(ndim < MAXDIM);
+
+ /*
+ * We will iterate the Postgres array in the physical order its stored in
+ * the datum. For 3-dimensional array the order of iteration would be the
+ * following: first you start with [0,0,0] elements through [0,0,k], then
+ * [0,1,0] till [0,1,k] till [0,m,k], then [1,0,0] till [1,0,k] till
+ * [1,m,k], and so on.
+ *
+ * In Python, there is no multi-dimensional lists as such, but they are
+ * represented as a list of lists. So a 3-d array of [n,m,k] element is a
+ * list of n m-element arrays, each element of which is k-element array.
+ * PLyList_FromArray_recurse() builds the Python list for a single
+ * dimension, and recurses for the next inner dimension.
+ */
+ dataptr = ARR_DATA_PTR(array);
+ bitmap = ARR_NULLBITMAP(array);
+ bitmask = 1;
+
+ return PLyList_FromArray_recurse(elm, dims, ndim, 0,
+ &dataptr, &bitmap, &bitmask);
+}
+
+static PyObject *
+PLyList_FromArray_recurse(PLyDatumToOb *elm, int *dims, int ndim, int dim,
+ char **dataptr_p, bits8 **bitmap_p, int *bitmask_p)
+{
+ int i;
+ PyObject *list;
- length = ARR_DIMS(array)[0];
- lbound = ARR_LBOUND(array)[0];
- list = PyList_New(length);
- if (list == NULL)
- PLy_elog(ERROR, "could not create new Python list");
+ list = PyList_New(dims[dim]);
- for (i = 0; i < length; i++)
+ if (dim < ndim - 1)
{
- Datum elem;
- bool isnull;
- int offset;
-
- offset = lbound + i;
- elem = array_ref(array, 1, &offset, arg->typlen,
- elm->typlen, elm->typbyval, elm->typalign,
- &isnull);
- if (isnull)
+ /* Outer dimension. Recurse for each inner slice. */
+ for (i = 0; i < dims[dim]; i++)
{
- Py_INCREF(Py_None);
- PyList_SET_ITEM(list, i, Py_None);
+ PyObject *sublist;
+
+ sublist = PLyList_FromArray_recurse(elm, dims, ndim, dim + 1,
+ dataptr_p, bitmap_p, bitmask_p);
+ PyList_SET_ITEM(list, i, sublist);
}
- else
- PyList_SET_ITEM(list, i, elm->func(elm, elem));
+ }
+ else
+ {
+ /*
+ * Innermost dimension. Fill the list with the values from the array
+ * for this slice.
+ */
+ char *dataptr = *dataptr_p;
+ bits8 *bitmap = *bitmap_p;
+ int bitmask = *bitmask_p;
+
+ for (i = 0; i < dims[dim]; i++)
+ {
+ /* checking for NULL */
+ if (bitmap && (*bitmap & bitmask) == 0)
+ {
+ Py_INCREF(Py_None);
+ PyList_SET_ITEM(list, i, Py_None);
+ }
+ else
+ {
+ Datum itemvalue;
+
+ itemvalue = fetch_att(dataptr, elm->typbyval, elm->typlen);
+ PyList_SET_ITEM(list, i, elm->func(elm, itemvalue));
+ dataptr = att_addlength_pointer(dataptr, elm->typlen, dataptr);
+ dataptr = (char *) att_align_nominal(dataptr, elm->typalign);
+ }
+
+ /* advance bitmap pointer if any */
+ if (bitmap)
+ {
+ bitmask <<= 1;
+ if (bitmask == 0x100 /* (1<<8) */ )
+ {
+ bitmap++;
+ bitmask = 1;
+ }
+ }
+ }
+
+ *dataptr_p = dataptr;
+ *bitmap_p = bitmap;
+ *bitmask_p = bitmask;
}
return list;
@@ -864,39 +931,108 @@ static Datum
PLySequence_ToArray(PLyObToDatum *arg, int32 typmod, PyObject *plrv)
{
ArrayType *array;
- Datum rv;
int i;
Datum *elems;
bool *nulls;
- int len;
- int lbs;
+ int64 len;
+ int ndim;
+ int dims[MAXDIM];
+ int lbs[MAXDIM];
+ int currelem;
+ Datum rv;
+ PyObject *pyptr = plrv;
+ PyObject *next;
Assert(plrv != Py_None);
- if (!PySequence_Check(plrv))
- PLy_elog(ERROR, "return value of function with array return type is not a Python sequence");
+ /*
+ * Determine the number of dimensions, and their sizes
+ */
+ ndim = 0;
+ len = 1;
- len = PySequence_Length(plrv);
- elems = palloc(sizeof(*elems) * len);
- nulls = palloc(sizeof(*nulls) * len);
+ Py_INCREF(plrv);
- for (i = 0; i < len; i++)
+ for (;;)
{
- PyObject *obj = PySequence_GetItem(plrv, i);
+ if (!PySequence_Check(pyptr))
+ break;
- if (obj == Py_None)
- nulls[i] = true;
- else
+ /* composite type */
+ if (PyTuple_Check(pyptr))
+ break;
+
+ /* string */
+ if (PyString_Check(pyptr) || PyBytes_Check(pyptr) || PyUnicode_Check(pyptr))
+ break;
+
+ if (ndim == MAXDIM)
+ PLy_elog(ERROR, "number of array dimensions exceeds the maximum allowed (%d)", MAXDIM);
+
+ dims[ndim] = PySequence_Length(pyptr);
+ if (dims[ndim] < 0)
+ PLy_elog(ERROR, "cannot determine sequence length for function return value");
+
+ if (dims[ndim] > MaxAllocSize)
+ PLy_elog(ERROR, "array size exceeds the maximum allowed");
+
+ len *= dims[ndim];
+ if (len > MaxAllocSize)
+ PLy_elog(ERROR, "array size exceeds the maximum allowed");
+
+ if (dims[ndim] == 0)
{
- nulls[i] = false;
- elems[i] = arg->elm->func(arg->elm, -1, obj);
+ /* empty sequence */
+ break;
}
- Py_XDECREF(obj);
+
+ ndim++;
+
+ next = PySequence_GetItem(pyptr, 0);
+ Py_XDECREF(pyptr);
+ pyptr = next;
+ }
+ Py_XDECREF(pyptr);
+
+ /*
+ * Check for zero dimensions. This happens if the object is a tuple or a
+ * string, rather than a list, or is not a sequence at all. We don't map
+ * tuples or strings to arrays in general, but in the first level, be
+ * lenient, for historical reasons. So if the object is a sequence of any
+ * kind, treat it as a one-dimensional array.
+ */
+ if (ndim == 0)
+ {
+ if (!PySequence_Check(plrv))
+ PLy_elog(ERROR, "return value of function with array return type is not a Python sequence");
+
+ ndim = 1;
+ len = dims[0] = PySequence_Length(plrv);
}
- lbs = 1;
- array = construct_md_array(elems, nulls, 1, &len, &lbs,
- get_base_element_type(arg->typoid), arg->elm->typlen, arg->elm->typbyval, arg->elm->typalign);
+ /*
+ * Traverse the Python lists, in depth-first order, and collect all the
+ * elements at the bottom level into 'elems'/'nulls' arrays.
+ */
+ elems = palloc(sizeof(Datum) * len);
+ nulls = palloc(sizeof(bool) * len);
+ currelem = 0;
+ PLySequence_ToArray_recurse(arg->elm, plrv,
+ dims, ndim, 0,
+ elems, nulls, &currelem);
+
+ for (i = 0; i < ndim; i++)
+ lbs[i] = 1;
+
+ array = construct_md_array(elems,
+ nulls,
+ ndim,
+ dims,
+ lbs,
+ get_base_element_type(arg->typoid),
+ arg->elm->typlen,
+ arg->elm->typbyval,
+ arg->elm->typalign);
/*
* If the result type is a domain of array, the resulting array must be
@@ -908,6 +1044,56 @@ PLySequence_ToArray(PLyObToDatum *arg, int32 typmod, PyObject *plrv)
return rv;
}
+/*
+ * Helper function for PLySequence_ToArray. Traverse a Python list of lists in
+ * depth-first order, storing the elements in 'elems'.
+ */
+static void
+PLySequence_ToArray_recurse(PLyObToDatum *elm, PyObject *list,
+ int *dims, int ndim, int dim,
+ Datum *elems, bool *nulls, int *currelem)
+{
+ int i;
+
+ if (PySequence_Length(list) != dims[dim])
+ PLy_elog(ERROR,
+ "multidimensional arrays must have array expressions with matching dimensions. "
+ "PL/Python function return value has sequence length %d while expected %d",
+ (int) PySequence_Length(list), dims[dim]);
+
+ if (dim < ndim - 1)
+ {
+ for (i = 0; i < dims[dim]; i++)
+ {
+ PyObject *sublist = PySequence_GetItem(list, i);
+
+ PLySequence_ToArray_recurse(elm, sublist, dims, ndim, dim + 1,
+ elems, nulls, currelem);
+ Py_XDECREF(sublist);
+ }
+ }
+ else
+ {
+ for (i = 0; i < dims[dim]; i++)
+ {
+ PyObject *obj = PySequence_GetItem(list, i);
+
+ if (obj == Py_None)
+ {
+ nulls[*currelem] = true;
+ elems[*currelem] = (Datum) 0;
+ }
+ else
+ {
+ nulls[*currelem] = false;
+ elems[*currelem] = elm->func(elm, -1, obj);
+ }
+ Py_XDECREF(obj);
+ (*currelem)++;
+ }
+ }
+}
+
static Datum
PLyString_ToComposite(PLyTypeInfo *info, TupleDesc desc, PyObject *string)
diff --git a/src/pl/plpython/sql/plpython_composite.sql b/src/pl/plpython/sql/plpython_composite.sql
index 473342c..72a35c3 100644
--- a/src/pl/plpython/sql/plpython_composite.sql
+++ b/src/pl/plpython/sql/plpython_composite.sql
@@ -169,13 +169,13 @@ SELECT * FROM changing_test();
-- tables of composite types
CREATE FUNCTION composite_types_table(OUT tab table_record[], OUT typ type_record[] ) RETURNS SETOF record AS $$
-yield {'tab': [['first', 1], ['second', 2]],
+yield {'tab': [('first', 1), ('second', 2)],
'typ': [{'first': 'third', 'second': 3},
{'first': 'fourth', 'second': 4}]}
-yield {'tab': [['first', 1], ['second', 2]],
+yield {'tab': [('first', 1), ('second', 2)],
'typ': [{'first': 'third', 'second': 3},
{'first': 'fourth', 'second': 4}]}
-yield {'tab': [['first', 1], ['second', 2]],
+yield {'tab': [('first', 1), ('second', 2)],
'typ': [{'first': 'third', 'second': 3},
{'first': 'fourth', 'second': 4}]}
$$ LANGUAGE plpythonu;
diff --git a/src/pl/plpython/sql/plpython_types.sql b/src/pl/plpython/sql/plpython_types.sql
index 19d920d..802dd41 100644
--- a/src/pl/plpython/sql/plpython_types.sql
+++ b/src/pl/plpython/sql/plpython_types.sql
@@ -237,7 +237,80 @@ SELECT * FROM test_type_conversion_array_int4(ARRAY[NULL,1]);
SELECT * FROM test_type_conversion_array_int4(ARRAY[]::integer[]);
SELECT * FROM test_type_conversion_array_int4(NULL);
SELECT * FROM test_type_conversion_array_int4(ARRAY[[1,2,3],[4,5,6]]);
+SELECT * FROM test_type_conversion_array_int4(ARRAY[[[1,2,NULL],[NULL,5,6]],[[NULL,8,9],[10,11,12]]]);
+SELECT * FROM test_type_conversion_array_int4('[2:4]={1,2,3}');
+CREATE FUNCTION test_type_conversion_array_int8(x int8[]) RETURNS int8[] AS $$
+plpy.info(x, type(x))
+return x
+$$ LANGUAGE plpythonu;
+
+SELECT * FROM test_type_conversion_array_int8(ARRAY[[[1,2,NULL],[NULL,5,6]],[[NULL,8,9],[10,11,12]]]::int8[]);
+
+CREATE FUNCTION test_type_conversion_array_float4(x float4[]) RETURNS float4[] AS $$
+plpy.info(x, type(x))
+return x
+$$ LANGUAGE plpythonu;
+
+SELECT * FROM test_type_conversion_array_float4(ARRAY[[[1.2,2.3,NULL],[NULL,5.7,6.8]],[[NULL,8.9,9.345],[10.123,11.456,12.6768]]]::float4[]);
+
+CREATE FUNCTION test_type_conversion_array_float8(x float8[]) RETURNS float8[] AS $$
+plpy.info(x, type(x))
+return x
+$$ LANGUAGE plpythonu;
+
+SELECT * FROM test_type_conversion_array_float8(ARRAY[[[1.2,2.3,NULL],[NULL,5.7,6.8]],[[NULL,8.9,9.345],[10.123,11.456,12.6768]]]::float8[]);
+
+CREATE FUNCTION test_type_conversion_array_date(x date[]) RETURNS date[] AS $$
+plpy.info(x, type(x))
+return x
+$$ LANGUAGE plpythonu;
+
+SELECT * FROM test_type_conversion_array_date(ARRAY[[['2016-09-21','2016-09-22',NULL],[NULL,'2016-10-21','2016-10-22']],
+ [[NULL,'2016-11-21','2016-10-21'],['2015-09-21','2015-09-22','2014-09-21']]]::date[]);
+
+CREATE FUNCTION test_type_conversion_array_timestamp(x timestamp[]) RETURNS timestamp[] AS $$
+plpy.info(x, type(x))
+return x
+$$ LANGUAGE plpythonu;
+
+SELECT * FROM test_type_conversion_array_timestamp(ARRAY[[['2016-09-21 15:34:24.078792-04','2016-10-22 11:34:24.078795-04',NULL],
+ [NULL,'2016-10-21 11:34:25.078792-04','2016-10-21 11:34:24.098792-04']],
+ [[NULL,'2016-01-21 11:34:24.078792-04','2016-11-21 11:34:24.108792-04'],
+ ['2015-09-21 11:34:24.079792-04','2014-09-21 11:34:24.078792-04','2013-09-21 11:34:24.078792-04']]]::timestamp[]);
+
+
+CREATE OR REPLACE FUNCTION pyreturnmultidemint4(h int4, i int4, j int4, k int4 ) RETURNS int4[] AS $BODY$
+m = [[[[x for x in range(h)] for y in range(i)] for z in range(j)] for w in range(k)]
+plpy.info(m, type(m))
+return m
+$BODY$ LANGUAGE plpythonu;
+
+select pyreturnmultidemint4(8,5,3,2);
+
+CREATE OR REPLACE FUNCTION pyreturnmultidemint8(h int4, i int4, j int4, k int4 ) RETURNS int8[] AS $BODY$
+m = [[[[x for x in range(h)] for y in range(i)] for z in range(j)] for w in range(k)]
+plpy.info(m, type(m))
+return m
+$BODY$ LANGUAGE plpythonu;
+
+select pyreturnmultidemint8(5,5,3,2);
+
+CREATE OR REPLACE FUNCTION pyreturnmultidemfloat4(h int4, i int4, j int4, k int4 ) RETURNS float4[] AS $BODY$
+m = [[[[x for x in range(h)] for y in range(i)] for z in range(j)] for w in range(k)]
+plpy.info(m, type(m))
+return m
+$BODY$ LANGUAGE plpythonu;
+
+select pyreturnmultidemfloat4(6,5,3,2);
+
+CREATE OR REPLACE FUNCTION pyreturnmultidemfloat8(h int4, i int4, j int4, k int4 ) RETURNS float8[] AS $BODY$
+m = [[[[x for x in range(h)] for y in range(i)] for z in range(j)] for w in range(k)]
+plpy.info(m, type(m))
+return m
+$BODY$ LANGUAGE plpythonu;
+
+select pyreturnmultidemfloat8(7,5,3,2);
CREATE FUNCTION test_type_conversion_array_text(x text[]) RETURNS text[] AS $$
plpy.info(x, type(x))
@@ -245,6 +318,7 @@ return x
$$ LANGUAGE plpythonu;
SELECT * FROM test_type_conversion_array_text(ARRAY['foo', 'bar']);
+SELECT * FROM test_type_conversion_array_text(ARRAY[['foo', 'bar'],['foo2', 'bar2']]);
CREATE FUNCTION test_type_conversion_array_bytea(x bytea[]) RETURNS bytea[] AS $$
@@ -268,6 +342,18 @@ $$ LANGUAGE plpythonu;
SELECT * FROM test_type_conversion_array_mixed2();
+CREATE FUNCTION test_type_conversion_mdarray_malformed() RETURNS int[] AS $$
+return [[1,2,3],[4,5]]
+$$ LANGUAGE plpythonu;
+
+SELECT * FROM test_type_conversion_mdarray_malformed();
+
+CREATE FUNCTION test_type_conversion_mdarray_toodeep() RETURNS int[] AS $$
+return [[[[[[[1]]]]]]]
+$$ LANGUAGE plpythonu;
+
+SELECT * FROM test_type_conversion_mdarray_toodeep();
+
CREATE FUNCTION test_type_conversion_array_record() RETURNS type_record[] AS $$
return [{'first': 'one', 'second': 42}, {'first': 'two', 'second': 11}]
--
2.9.3
0002-Give-a-hint-when-is-incorrectly-used-for-a-composite.patchtext/x-diff; name=0002-Give-a-hint-when-is-incorrectly-used-for-a-composite.patchDownload
From d952cfc08e2f161fa99f6ab1ce568cce411de017 Mon Sep 17 00:00:00 2001
From: Heikki Linnakangas <heikki.linnakangas@iki.fi>
Date: Fri, 14 Oct 2016 11:52:05 +0300
Subject: [PATCH 2/2] Give a hint, when [] is incorrectly used for a composite
type in array.
That used to be accepted, so let's try to give a hint to users on why
their PL/python functions no longer work.
---
src/pl/plpython/plpy_cursorobject.c | 3 +-
src/pl/plpython/plpy_exec.c | 7 +-
src/pl/plpython/plpy_spi.c | 3 +-
src/pl/plpython/plpy_typeio.c | 117 +++++++++++++++++++++--------
src/pl/plpython/plpy_typeio.h | 17 +++--
src/pl/plpython/sql/plpython_composite.sql | 9 +++
6 files changed, 116 insertions(+), 40 deletions(-)
diff --git a/src/pl/plpython/plpy_cursorobject.c b/src/pl/plpython/plpy_cursorobject.c
index 0e17a03..e682bfe 100644
--- a/src/pl/plpython/plpy_cursorobject.c
+++ b/src/pl/plpython/plpy_cursorobject.c
@@ -240,7 +240,8 @@ PLy_cursor_plan(PyObject *ob, PyObject *args)
plan->values[j] =
plan->args[j].out.d.func(&(plan->args[j].out.d),
-1,
- elem);
+ elem,
+ false);
}
PG_CATCH();
{
diff --git a/src/pl/plpython/plpy_exec.c b/src/pl/plpython/plpy_exec.c
index 25e4744..fa583fa 100644
--- a/src/pl/plpython/plpy_exec.c
+++ b/src/pl/plpython/plpy_exec.c
@@ -245,7 +245,7 @@ PLy_exec_function(FunctionCallInfo fcinfo, PLyProcedure *proc)
desc = lookup_rowtype_tupdesc(proc->result.out.d.typoid,
proc->result.out.d.typmod);
- rv = PLyObject_ToCompositeDatum(&proc->result, desc, plrv);
+ rv = PLyObject_ToCompositeDatum(&proc->result, desc, plrv, false);
fcinfo->isnull = (rv == (Datum) NULL);
ReleaseTupleDesc(desc);
@@ -253,7 +253,7 @@ PLy_exec_function(FunctionCallInfo fcinfo, PLyProcedure *proc)
else
{
fcinfo->isnull = false;
- rv = (proc->result.out.d.func) (&proc->result.out.d, -1, plrv);
+ rv = (proc->result.out.d.func) (&proc->result.out.d, -1, plrv, false);
}
}
PG_CATCH();
@@ -984,7 +984,8 @@ PLy_modify_tuple(PLyProcedure *proc, PyObject *pltd, TriggerData *tdata,
modvalues[i] = (att->func) (att,
tupdesc->attrs[atti]->atttypmod,
- plval);
+ plval,
+ false);
modnulls[i] = ' ';
}
else
diff --git a/src/pl/plpython/plpy_spi.c b/src/pl/plpython/plpy_spi.c
index 09ee06d..0d556a2 100644
--- a/src/pl/plpython/plpy_spi.c
+++ b/src/pl/plpython/plpy_spi.c
@@ -264,7 +264,8 @@ PLy_spi_execute_plan(PyObject *ob, PyObject *list, long limit)
plan->values[j] =
plan->args[j].out.d.func(&(plan->args[j].out.d),
-1,
- elem);
+ elem,
+ false);
}
PG_CATCH();
{
diff --git a/src/pl/plpython/plpy_typeio.c b/src/pl/plpython/plpy_typeio.c
index 56cfdd1..9c62be7 100644
--- a/src/pl/plpython/plpy_typeio.c
+++ b/src/pl/plpython/plpy_typeio.c
@@ -14,6 +14,7 @@
#include "parser/parse_type.h"
#include "utils/array.h"
#include "utils/builtins.h"
+#include "utils/fmgroids.h"
#include "utils/lsyscache.h"
#include "utils/memutils.h"
#include "utils/numeric.h"
@@ -49,21 +50,21 @@ static PyObject *PLyList_FromArray_recurse(PLyDatumToOb *elm, int *dims, int ndi
char **dataptr_p, bits8 **bitmap_p, int *bitmask_p);
/* conversion from Python objects to Datums */
-static Datum PLyObject_ToBool(PLyObToDatum *arg, int32 typmod, PyObject *plrv);
-static Datum PLyObject_ToBytea(PLyObToDatum *arg, int32 typmod, PyObject *plrv);
-static Datum PLyObject_ToComposite(PLyObToDatum *arg, int32 typmod, PyObject *plrv);
-static Datum PLyObject_ToDatum(PLyObToDatum *arg, int32 typmod, PyObject *plrv);
-static Datum PLyObject_ToTransform(PLyObToDatum *arg, int32 typmod, PyObject *plrv);
-static Datum PLySequence_ToArray(PLyObToDatum *arg, int32 typmod, PyObject *plrv);
+static Datum PLyObject_ToBool(PLyObToDatum *arg, int32 typmod, PyObject *plrv, bool inarray);
+static Datum PLyObject_ToBytea(PLyObToDatum *arg, int32 typmod, PyObject *plrv, bool inarray);
+static Datum PLyObject_ToComposite(PLyObToDatum *arg, int32 typmod, PyObject *plrv, bool inarray);
+static Datum PLyObject_ToDatum(PLyObToDatum *arg, int32 typmod, PyObject *plrv, bool inarray);
+static Datum PLyObject_ToTransform(PLyObToDatum *arg, int32 typmod, PyObject *plrv, bool inarray);
+static Datum PLySequence_ToArray(PLyObToDatum *arg, int32 typmod, PyObject *plrv, bool inarray);
static void PLySequence_ToArray_recurse(PLyObToDatum *elm, PyObject *list,
int *dims, int ndim, int dim,
Datum *elems, bool *nulls, int *currelem);
/* conversion from Python objects to composite Datums (used by triggers and SRFs) */
-static Datum PLyString_ToComposite(PLyTypeInfo *info, TupleDesc desc, PyObject *string);
+static Datum PLyString_ToComposite(PLyTypeInfo *info, TupleDesc desc, PyObject *string, bool inarray);
static Datum PLyMapping_ToComposite(PLyTypeInfo *info, TupleDesc desc, PyObject *mapping);
static Datum PLySequence_ToComposite(PLyTypeInfo *info, TupleDesc desc, PyObject *sequence);
-static Datum PLyGenericObject_ToComposite(PLyTypeInfo *info, TupleDesc desc, PyObject *object);
+static Datum PLyGenericObject_ToComposite(PLyTypeInfo *info, TupleDesc desc, PyObject *object, bool inarray);
void
PLy_typeinfo_init(PLyTypeInfo *arg, MemoryContext mcxt)
@@ -341,12 +342,12 @@ PLyDict_FromTuple(PLyTypeInfo *info, HeapTuple tuple, TupleDesc desc)
* as an object that has __getattr__ support.
*/
Datum
-PLyObject_ToCompositeDatum(PLyTypeInfo *info, TupleDesc desc, PyObject *plrv)
+PLyObject_ToCompositeDatum(PLyTypeInfo *info, TupleDesc desc, PyObject *plrv, bool inarray)
{
Datum datum;
if (PyString_Check(plrv) || PyUnicode_Check(plrv))
- datum = PLyString_ToComposite(info, desc, plrv);
+ datum = PLyString_ToComposite(info, desc, plrv, inarray);
else if (PySequence_Check(plrv))
/* composite type as sequence (tuple, list etc) */
datum = PLySequence_ToComposite(info, desc, plrv);
@@ -355,7 +356,7 @@ PLyObject_ToCompositeDatum(PLyTypeInfo *info, TupleDesc desc, PyObject *plrv)
datum = PLyMapping_ToComposite(info, desc, plrv);
else
/* returned as smth, must provide method __getattr__(name) */
- datum = PLyGenericObject_ToComposite(info, desc, plrv);
+ datum = PLyGenericObject_ToComposite(info, desc, plrv, inarray);
return datum;
}
@@ -747,7 +748,7 @@ PLyList_FromArray_recurse(PLyDatumToOb *elm, int *dims, int ndim, int dim,
* type can parse.
*/
static Datum
-PLyObject_ToBool(PLyObToDatum *arg, int32 typmod, PyObject *plrv)
+PLyObject_ToBool(PLyObToDatum *arg, int32 typmod, PyObject *plrv, bool inarray)
{
Datum rv;
@@ -766,7 +767,7 @@ PLyObject_ToBool(PLyObToDatum *arg, int32 typmod, PyObject *plrv)
* with embedded nulls. And it's faster this way.
*/
static Datum
-PLyObject_ToBytea(PLyObToDatum *arg, int32 typmod, PyObject *plrv)
+PLyObject_ToBytea(PLyObToDatum *arg, int32 typmod, PyObject *plrv, bool inarray)
{
PyObject *volatile plrv_so = NULL;
Datum rv;
@@ -810,7 +811,7 @@ PLyObject_ToBytea(PLyObToDatum *arg, int32 typmod, PyObject *plrv)
* for obtaining PostgreSQL tuples.
*/
static Datum
-PLyObject_ToComposite(PLyObToDatum *arg, int32 typmod, PyObject *plrv)
+PLyObject_ToComposite(PLyObToDatum *arg, int32 typmod, PyObject *plrv, bool inarray)
{
Datum rv;
PLyTypeInfo info;
@@ -837,7 +838,7 @@ PLyObject_ToComposite(PLyObToDatum *arg, int32 typmod, PyObject *plrv)
* that info instead of looking it up every time a tuple is returned from
* the function.
*/
- rv = PLyObject_ToCompositeDatum(&info, desc, plrv);
+ rv = PLyObject_ToCompositeDatum(&info, desc, plrv, inarray);
ReleaseTupleDesc(desc);
@@ -909,26 +910,70 @@ PLyObject_AsString(PyObject *plrv)
* cstring into PostgreSQL type.
*/
static Datum
-PLyObject_ToDatum(PLyObToDatum *arg, int32 typmod, PyObject *plrv)
+PLyObject_ToDatum(PLyObToDatum *arg, int32 typmod, PyObject *plrv, bool inarray)
{
+ char *str;
+
Assert(plrv != Py_None);
+ str = PLyObject_AsString(plrv);
+
+ /*
+ * If we are parsing a composite type within an array, and the string
+ * isn't a valid record literal, there's a high chance that the function
+ * did something like:
+ *
+ * CREATE FUNCTION .. RETURNS comptype[] AS $$ return [['foo', 'bar']] $$
+ * LANGUAGE plpython;
+ *
+ * Before PostgreSQL 10, that was interpreted as a single-dimensional
+ * array, containing record ('foo', 'bar'). PostgreSQL 10 added support
+ * for multi-dimensional arrays, and it is now interpreted as a
+ * two-dimensional array, containing two records, 'foo', and 'bar'.
+ * record_in() will throw an error, because "foo" is not a valid record
+ * literal.
+ *
+ * To make that less confusing to users who are upgrading from older
+ * versions, try to give a hint in the typical cases of that. If we are
+ * parsing an array of composite types, and we see a string literal that
+ * is not a valid record literal, give a hint. We only want to give the
+ * hint in the narrow case of a malformed string literal, not any error
+ * from record_in(), so check for that case here specifically.
+ *
+ * This check better match the one in record_in(), so that we don't forbid
+ * literals that are actually valid!
+ */
+ if (inarray && arg->typfunc.fn_oid == F_RECORD_IN)
+ {
+ char *ptr = str;
+
+ /* Allow leading whitespace */
+ while (*ptr && isspace((unsigned char) *ptr))
+ ptr++;
+ if (*ptr++ != '(')
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_TEXT_REPRESENTATION),
+ errmsg("malformed record literal: \"%s\"", str),
+ errdetail("Missing left parenthesis."),
+ errhint("To return a composite type in an array, return the composite type as a Python tuple, e.g. \"[('foo')]\"")));
+ }
+
return InputFunctionCall(&arg->typfunc,
- PLyObject_AsString(plrv),
+ str,
arg->typioparam,
typmod);
}
static Datum
-PLyObject_ToTransform(PLyObToDatum *arg, int32 typmod, PyObject *plrv)
+PLyObject_ToTransform(PLyObToDatum *arg, int32 typmod, PyObject *plrv, bool inarray)
{
return FunctionCall1(&arg->typtransform, PointerGetDatum(plrv));
}
static Datum
-PLySequence_ToArray(PLyObToDatum *arg, int32 typmod, PyObject *plrv)
+PLySequence_ToArray(PLyObToDatum *arg, int32 typmod, PyObject *plrv, bool inarray)
{
ArrayType *array;
int i;
@@ -1086,7 +1131,7 @@ PLySequence_ToArray_recurse(PLyObToDatum *elm, PyObject *list,
else
{
nulls[*currelem] = false;
- elems[*currelem] = elm->func(elm, -1, obj);
+ elems[*currelem] = elm->func(elm, -1, obj, true);
}
Py_XDECREF(obj);
(*currelem)++;
@@ -1094,9 +1139,8 @@ PLySequence_ToArray_recurse(PLyObToDatum *elm, PyObject *list,
}
}
-
static Datum
-PLyString_ToComposite(PLyTypeInfo *info, TupleDesc desc, PyObject *string)
+PLyString_ToComposite(PLyTypeInfo *info, TupleDesc desc, PyObject *string, bool inarray)
{
Datum result;
HeapTuple typeTup;
@@ -1121,7 +1165,7 @@ PLyString_ToComposite(PLyTypeInfo *info, TupleDesc desc, PyObject *string)
ReleaseSysCache(typeTup);
- result = PLyObject_ToDatum(&locinfo.out.d, desc->tdtypmod, string);
+ result = PLyObject_ToDatum(&locinfo.out.d, desc->tdtypmod, string, inarray);
MemoryContextDelete(cxt);
@@ -1173,7 +1217,7 @@ PLyMapping_ToComposite(PLyTypeInfo *info, TupleDesc desc, PyObject *mapping)
}
else if (value)
{
- values[i] = (att->func) (att, -1, value);
+ values[i] = (att->func) (att, -1, value, false);
nulls[i] = false;
}
else
@@ -1266,7 +1310,7 @@ PLySequence_ToComposite(PLyTypeInfo *info, TupleDesc desc, PyObject *sequence)
}
else if (value)
{
- values[i] = (att->func) (att, -1, value);
+ values[i] = (att->func) (att, -1, value, false);
nulls[i] = false;
}
@@ -1295,7 +1339,7 @@ PLySequence_ToComposite(PLyTypeInfo *info, TupleDesc desc, PyObject *sequence)
static Datum
-PLyGenericObject_ToComposite(PLyTypeInfo *info, TupleDesc desc, PyObject *object)
+PLyGenericObject_ToComposite(PLyTypeInfo *info, TupleDesc desc, PyObject *object, bool inarray)
{
Datum result;
HeapTuple tuple;
@@ -1336,16 +1380,29 @@ PLyGenericObject_ToComposite(PLyTypeInfo *info, TupleDesc desc, PyObject *object
}
else if (value)
{
- values[i] = (att->func) (att, -1, value);
+ values[i] = (att->func) (att, -1, value, false);
nulls[i] = false;
}
else
+ {
+ /*
+ * No attribute for this column in the object.
+ *
+ * If we are parsing a composite type in an array, a likely
+ * cause is that the function contained something like "[[123,
+ * 'foo']]". Before PostgreSQL 10, that was interpreted as an
+ * array, with a composite type (123, 'foo') in it.. But now
+ * it's interpreted as a two-dimensional array, and we try to
+ * interpret "123" as the composite type. See also similar
+ * heuristic in PLyObject_ToDatum().
+ */
ereport(ERROR,
(errcode(ERRCODE_UNDEFINED_COLUMN),
errmsg("attribute \"%s\" does not exist in Python object", key),
- errhint("To return null in a column, "
- "let the returned object have an attribute named "
- "after column with value None.")));
+ inarray ?
+ errhint("To return a composite type in an array, return the composite type as a Python tuple, e.g. \"[('foo')]\"") :
+ errhint("To return null in a column, let the returned object have an attribute named after column with value None.")));
+ }
Py_XDECREF(value);
value = NULL;
diff --git a/src/pl/plpython/plpy_typeio.h b/src/pl/plpython/plpy_typeio.h
index 29fff61..5f5c1ad 100644
--- a/src/pl/plpython/plpy_typeio.h
+++ b/src/pl/plpython/plpy_typeio.h
@@ -10,8 +10,11 @@
#include "fmgr.h"
#include "storage/itemptr.h"
+/*
+ * Conversion from PostgreSQL Datum to a Python object.
+ */
struct PLyDatumToOb;
-typedef PyObject *(*PLyDatumToObFunc) (struct PLyDatumToOb *, Datum);
+typedef PyObject *(*PLyDatumToObFunc) (struct PLyDatumToOb *arg, Datum val);
typedef struct PLyDatumToOb
{
@@ -39,11 +42,15 @@ typedef union PLyTypeInput
PLyTupleToOb r;
} PLyTypeInput;
-/* convert PyObject to a Postgresql Datum or tuple.
- * output from Python
+/*
+ * Conversion from Python object to a Postgresql Datum.
+ *
+ * The 'inarray' argument to the conversion function is true, if the
+ * converted value was in an array (Python list). It is used to give a
+ * better error message in some cases.
*/
struct PLyObToDatum;
-typedef Datum (*PLyObToDatumFunc) (struct PLyObToDatum *, int32, PyObject *);
+typedef Datum (*PLyObToDatumFunc) (struct PLyObToDatum *arg, int32 typmod, PyObject *val, bool inarray);
typedef struct PLyObToDatum
{
@@ -104,7 +111,7 @@ extern void PLy_output_tuple_funcs(PLyTypeInfo *arg, TupleDesc desc);
extern void PLy_output_record_funcs(PLyTypeInfo *arg, TupleDesc desc);
/* conversion from Python objects to composite Datums */
-extern Datum PLyObject_ToCompositeDatum(PLyTypeInfo *info, TupleDesc desc, PyObject *plrv);
+extern Datum PLyObject_ToCompositeDatum(PLyTypeInfo *info, TupleDesc desc, PyObject *plrv, bool isarray);
/* conversion from heap tuples to Python dictionaries */
extern PyObject *PLyDict_FromTuple(PLyTypeInfo *info, HeapTuple tuple, TupleDesc desc);
diff --git a/src/pl/plpython/sql/plpython_composite.sql b/src/pl/plpython/sql/plpython_composite.sql
index 72a35c3..1a343af 100644
--- a/src/pl/plpython/sql/plpython_composite.sql
+++ b/src/pl/plpython/sql/plpython_composite.sql
@@ -207,3 +207,12 @@ SELECT * FROM return_record_2('v4') AS (v1 int, v3 int, v2 int);
-- works
SELECT * FROM return_record_2('v3') AS (v1 int, v3 int, v2 int);
SELECT * FROM return_record_2('v3') AS (v1 int, v2 int, v3 int);
+
+-- Starting with PostgreSQL 10, a composite type in an array cannot be represented as
+-- a Python list, because it's ambiguous with multi-dimensional arrays. So this
+-- throws an error now. The error should contain a useful hint on the issue.
+CREATE FUNCTION composite_type_as_list() RETURNS type_record[] AS $$
+ return [['first', 1]];
+$$ LANGUAGE plpythonu;
+
+SELECT * FROM composite_type_as_list();
--
2.9.3
Re: PL/Python adding support for multi-dimensional arrays
On 10/14/16 3:53 AM, Heikki Linnakangas wrote:
Composite types in arrays must now be returned as
Python tuples, not lists, to resolve the ambiguity. I.e. "[(col1, col2),
(col1, col2)]".
Shouldn't dicts be allowed as well? I'm not sure they would
automatically be considered as tuples (unlike something that extends
tuples, such as namedtuples).
--
Jim Nasby, Data Architect, Blue Treble Consulting, Austin TX
Experts in Analytics, Data Architecture and PostgreSQL
Data in Trouble? Get it in Treble! http://BlueTreble.com
855-TREBLE2 (855-873-2532) mobile: 512-569-9461
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
Re: PL/Python adding support for multi-dimensional arrays
On 14 October 2016 19:18:01 EEST, Jim Nasby <Jim.Nasby@BlueTreble.com> wrote:
On 10/14/16 3:53 AM, Heikki Linnakangas wrote:
Composite types in arrays must now be returned as
Python tuples, not lists, to resolve the ambiguity. I.e. "[(col1,col2),
(col1, col2)]".
Shouldn't dicts be allowed as well?
Ah yes, dicts are also allowed, as before. And strings. The only change is that a list is interpreted as an array dimension, instead of a composite type.
- Heikki
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
Re: PL/Python adding support for multi-dimensional arrays
Hi
2016-10-14 10:53 GMT+02:00 Heikki Linnakangas <hlinnaka@iki.fi>:
On 10/11/2016 08:56 AM, Pavel Stehule wrote:
2016-10-11 7:49 GMT+02:00 Heikki Linnakangas <hlinnaka@iki.fi>:
Unfortunately there are cases that are fundamentally ambiguous.
create type comptype as (intarray int[]);
create function array_return() returns comptype[] as $$
return [[[[1]]]];
$$ language plpython;What does the function return? It could be two-dimension array of
comptype, with a single-dimension intarray, or a single-dimension
comptype,
with a two-dimension intarray.We could resolve it for simpler cases, but not the general case. The
simple cases would probably cover most things people do in practice. But
if
the distinction between a tuple and a list feels natural to Python
programmers, I think it would be more clear in the long run to have
people
adjust their applications.I agree. The distinction is natural - and it is our issue, so we don't
distinguish strongly.Ok, let's do that then. Here is a patch set that does that. The first is
the main patch. The second patch adds some code to give a hint, if you do
that thing that whose behavior changed. That code isn't very pretty, but I
think a good error message is absolutely required, if we are to make this
change. Does anyone have better suggestions on how to catch the common
cases of that?Please review. Are the docs and the error messages now clear enough on
this? We'll need a mention in the release notes too, when it's time for
that.
The error message is clear.
I tested patches - and the regression test is broken (is not actualized)
+ -- Starting with PostgreSQL 10, a composite type in an array cannot be
represented as
+ -- a Python list, because it's ambiguous with multi-dimensional arrays.
So this
+ -- throws an error now. The error should contain a useful hint on the
issue.
+ CREATE FUNCTION composite_type_as_list() RETURNS type_record[] AS $$
+ return [['first', 1]];
+ $$ LANGUAGE plpythonu;
+ SELECT * FROM composite_type_as_list();
+ ERROR: malformed record literal: "first"
+ DETAIL: Missing left parenthesis.
+ HINT: To return a composite type in an array, return the composite type
as a Python tuple, e.g. "[('foo')]"
+ CONTEXT: while creating return value
+ PL/Python function "composite_type_as_list"I tested Pyhon 3.5 and 2.7 and there are not any other issues
There are no new tests for multidimensional array of composites - there is
only new negative test.
Regards
Pavel
Show quoted text
- Heikki
Re: PL/Python adding support for multi-dimensional arrays
On 10/24/2016 10:33 PM, Pavel Stehule wrote:
Hi
2016-10-14 10:53 GMT+02:00 Heikki Linnakangas <hlinnaka@iki.fi>:
Please review. Are the docs and the error messages now clear enough on
this? We'll need a mention in the release notes too, when it's time for
that.The error message is clear.
Ok, great!
I tested patches - and the regression test is broken (is not actualized)
Ah, fixed.
There are no new tests for multidimensional array of composites - there is
only new negative test.
Added one.
Thanks for the review! Committed, with those little fixes, and some
little last-minute comment tweaks.
- Heikki
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
Re: PL/Python adding support for multi-dimensional arrays
2016-10-26 10:03 GMT+02:00 Heikki Linnakangas <hlinnaka@iki.fi>:
On 10/24/2016 10:33 PM, Pavel Stehule wrote:
Hi
2016-10-14 10:53 GMT+02:00 Heikki Linnakangas <hlinnaka@iki.fi>:
Please review. Are the docs and the error messages now clear enough on
this? We'll need a mention in the release notes too, when it's time for
that.The error message is clear.
Ok, great!
I tested patches - and the regression test is broken (is not actualized)
Ah, fixed.
There are no new tests for multidimensional array of composites - there is
only new negative test.
Added one.
Thanks for the review! Committed, with those little fixes, and some little
last-minute comment tweaks.
Thank you very much
nice feature
Regards
Pavel
Show quoted text
- Heikki