[PATCH] Generalized JSON output functions

Started by Shulgin, Oleksandrover 10 years ago51 messages
#1Shulgin, Oleksandr
oleksandr.shulgin@zalando.de
1 attachment(s)

Hi, Hackers!

Attached is a patch against master to generalize the JSON-producing
functions in utils/adt/json.c and to provide a set of callbacks which can
be overridden the same way that is already provided for *parsing* JSON.

The motivation behind this to be able to produce specially-crafted JSON in
a logical replication output plugin, such that numeric (and bigint) values
are quoted. This requirement, in turn, arises from the fact that
JavaScript specification, which is quite natural to expect as a consumer
for this JSON data, allows to silently drop significant digits when
converting from string to number object.

I believe this is a well-known problem and I'm aware of a number of tricks
that might be used to avoid it, but none of them seems to be optimal from
my standpoint.

I can also imagine this can be used to convert date/time to string
differently, or adding indentation depending on the depth in object
hierarchy, etc.

What this patch does apart from providing callbacks, is abstracting most of
code for producing the correct JSON structure, which was previously
scattered and repeated in a number of functions with slight differences.
In the current code there are 5 styles for producing JSON object string,
differing in whitespace only:

a) no spaces

select to_json(row(1,2));
to_json
-----------------
{"f1":1,"f2":2}

b) some spaces (hstore_to_json)

select hstore(row(1,2))::json;
hstore
------------------------
{"f1": "1", "f2": "2"}

c) spaces around colon

select json_build_object('f1',1,'f2',2);
json_build_object
----------------------
{"f1" : 1, "f2" : 2}

d) spaces around colon *and* curly braces

select json_object_agg(x,x) from unnest('{1,2}'::int[]) x;
json_object_agg
----------------------
{ "1" : 1, "2" : 2 }

e) line feeds (row_to_json_pretty)

select row_to_json(row(1,2), true) as row;
row
----------
{"f1":1,+
"f2":2}

Personally, I think we should stick to (b), however that would break a lot
of test cases that already depend on (a). I've tried hard to minimize the
amount of changes in expected/json.out, but it is quickly becomes
cumbersome trying to support all of the above formats. So I've altered (c)
and (d) to look like (b), naturally only whitespace was affected.

There's one corner case I don't see a sensible way to support:

select json_agg(x) from generate_series(1,5) x;
json_agg
-----------------
[1, 2, 3, 4, 5]

With the patch applied it puts line feeds between the array elements
instead of spaces.

What also bothers me is that I've hard-coded output function oids for
cstring_out, and textout on assumption that they never change, but would
like to know that for sure.

Feedback is very much welcome!
--
Alex

PS: using a different email address this time, same Alex Shulgin. ;-)

PPS: sample code for mentioned use case with quoting numeric and bigint:

static void out_value_quote_numerics(JsonOutContext *out, Datum val,
JsonTypeCategory tcategory, Oid typoid, Oid
outfuncoid,
bool key_scalar) {
char *outputstr;

if (typoid == INT8OID || typoid == NUMERICOID) {
out->before_value(out);

outputstr = OidOutputFunctionCall(outfuncoid, val);
escape_json(&out->result, outputstr);
pfree(outputstr);

out->after_value(out, key_scalar);
} else {
json_out_value(out, val, tcategory, typoid, outfuncoid, key_scalar);
}
}
...
json_out_init_context(out, JSON_OUT_USE_SPACES);
out->value = out_value_quote_numerics;

Attachments:

json-output-generalized-v1.patchtext/x-patch; charset=US-ASCII; name=json-output-generalized-v1.patchDownload
diff --git a/contrib/hstore/hstore_io.c b/contrib/hstore/hstore_io.c
new file mode 100644
index 7d89867..1b1e857
*** a/contrib/hstore/hstore_io.c
--- b/contrib/hstore/hstore_io.c
*************** hstore_to_json_loose(PG_FUNCTION_ARGS)
*** 1241,1286 ****
  	int			count = HS_COUNT(in);
  	char	   *base = STRPTR(in);
  	HEntry	   *entries = ARRPTR(in);
! 	StringInfoData tmp,
! 				dst;
  
  	if (count == 0)
  		PG_RETURN_TEXT_P(cstring_to_text_with_len("{}", 2));
  
  	initStringInfo(&tmp);
! 	initStringInfo(&dst);
! 
! 	appendStringInfoChar(&dst, '{');
  
  	for (i = 0; i < count; i++)
  	{
  		resetStringInfo(&tmp);
  		appendBinaryStringInfo(&tmp, HS_KEY(entries, base, i), HS_KEYLEN(entries, i));
! 		escape_json(&dst, tmp.data);
! 		appendStringInfoString(&dst, ": ");
  		if (HS_VALISNULL(entries, i))
! 			appendStringInfoString(&dst, "null");
  		/* guess that values of 't' or 'f' are booleans */
  		else if (HS_VALLEN(entries, i) == 1 && *(HS_VAL(entries, base, i)) == 't')
! 			appendStringInfoString(&dst, "true");
  		else if (HS_VALLEN(entries, i) == 1 && *(HS_VAL(entries, base, i)) == 'f')
! 			appendStringInfoString(&dst, "false");
  		else
  		{
  			resetStringInfo(&tmp);
  			appendBinaryStringInfo(&tmp, HS_VAL(entries, base, i), HS_VALLEN(entries, i));
  			if (IsValidJsonNumber(tmp.data, tmp.len))
! 				appendBinaryStringInfo(&dst, tmp.data, tmp.len);
  			else
! 				escape_json(&dst, tmp.data);
  		}
- 
- 		if (i + 1 != count)
- 			appendStringInfoString(&dst, ", ");
  	}
- 	appendStringInfoChar(&dst, '}');
  
! 	PG_RETURN_TEXT_P(cstring_to_text(dst.data));
  }
  
  PG_FUNCTION_INFO_V1(hstore_to_json);
--- 1241,1289 ----
  	int			count = HS_COUNT(in);
  	char	   *base = STRPTR(in);
  	HEntry	   *entries = ARRPTR(in);
! 	StringInfoData	tmp;
! 	JsonOutContext	dst;
  
  	if (count == 0)
  		PG_RETURN_TEXT_P(cstring_to_text_with_len("{}", 2));
  
  	initStringInfo(&tmp);
! 	json_out_init_context(&dst, JSON_OUT_USE_SPACES);
! 	dst.object_start(&dst);
  
  	for (i = 0; i < count; i++)
  	{
  		resetStringInfo(&tmp);
  		appendBinaryStringInfo(&tmp, HS_KEY(entries, base, i), HS_KEYLEN(entries, i));
! 		json_out_cstring(&dst, tmp.data, true);
! 
  		if (HS_VALISNULL(entries, i))
! 			dst.value(&dst, (Datum) 0, JSONTYPE_NULL, InvalodOid, InvalidOid, false);
! 
  		/* guess that values of 't' or 'f' are booleans */
  		else if (HS_VALLEN(entries, i) == 1 && *(HS_VAL(entries, base, i)) == 't')
! 			dst.value(&dst, BoolGetDatum(true), JSONTYPE_BOOL,
! 					  InvalodOid, InvalidOid, false);
! 
  		else if (HS_VALLEN(entries, i) == 1 && *(HS_VAL(entries, base, i)) == 'f')
! 			dst.value(&dst, BoolGetDatum(false), JSONTYPE_BOOL,
! 					  InvalodOid, InvalidOid, false);
  		else
  		{
  			resetStringInfo(&tmp);
  			appendBinaryStringInfo(&tmp, HS_VAL(entries, base, i), HS_VALLEN(entries, i));
+ 
+ 			/* this is a bit of a hack, but strictly it is not incorrect */
  			if (IsValidJsonNumber(tmp.data, tmp.len))
! 				dst.value(&dst, CStringGetDatum(tmp.data), JSONTYPE_JSON,
! 						  CSTRINGOID, 2293 /* cstring_out */, false);
  			else
! 				json_out_cstring(&dst, tmp.data, false);
  		}
  	}
  
! 	dst.object_end(&dst);
! 	PG_RETURN_TEXT_P(cstring_to_text_with_len(dst.result.data, dst.result.len));
  }
  
  PG_FUNCTION_INFO_V1(hstore_to_json);
*************** hstore_to_json(PG_FUNCTION_ARGS)
*** 1292,1329 ****
  	int			count = HS_COUNT(in);
  	char	   *base = STRPTR(in);
  	HEntry	   *entries = ARRPTR(in);
! 	StringInfoData tmp,
! 				dst;
  
  	if (count == 0)
  		PG_RETURN_TEXT_P(cstring_to_text_with_len("{}", 2));
  
  	initStringInfo(&tmp);
! 	initStringInfo(&dst);
! 
! 	appendStringInfoChar(&dst, '{');
  
  	for (i = 0; i < count; i++)
  	{
  		resetStringInfo(&tmp);
  		appendBinaryStringInfo(&tmp, HS_KEY(entries, base, i), HS_KEYLEN(entries, i));
! 		escape_json(&dst, tmp.data);
! 		appendStringInfoString(&dst, ": ");
  		if (HS_VALISNULL(entries, i))
! 			appendStringInfoString(&dst, "null");
  		else
  		{
  			resetStringInfo(&tmp);
  			appendBinaryStringInfo(&tmp, HS_VAL(entries, base, i), HS_VALLEN(entries, i));
! 			escape_json(&dst, tmp.data);
  		}
- 
- 		if (i + 1 != count)
- 			appendStringInfoString(&dst, ", ");
  	}
- 	appendStringInfoChar(&dst, '}');
  
! 	PG_RETURN_TEXT_P(cstring_to_text(dst.data));
  }
  
  PG_FUNCTION_INFO_V1(hstore_to_jsonb);
--- 1295,1328 ----
  	int			count = HS_COUNT(in);
  	char	   *base = STRPTR(in);
  	HEntry	   *entries = ARRPTR(in);
! 	StringInfoData	tmp;
! 	JsonOutContext	dst;
  
  	if (count == 0)
  		PG_RETURN_TEXT_P(cstring_to_text_with_len("{}", 2));
  
  	initStringInfo(&tmp);
! 	json_out_init_context(&dst, JSON_OUT_USE_SPACES);
! 	dst.object_start(&dst);
  
  	for (i = 0; i < count; i++)
  	{
  		resetStringInfo(&tmp);
  		appendBinaryStringInfo(&tmp, HS_KEY(entries, base, i), HS_KEYLEN(entries, i));
! 		json_out_cstring(&dst, tmp.data, true);
! 
  		if (HS_VALISNULL(entries, i))
! 			dst.value(&dst, (Datum) 0, JSONTYPE_NULL, InvalodOid, InvalidOid, false);
  		else
  		{
  			resetStringInfo(&tmp);
  			appendBinaryStringInfo(&tmp, HS_VAL(entries, base, i), HS_VALLEN(entries, i));
! 			json_out_cstring(&dst, tmp.data, false);
  		}
  	}
  
! 	dst.object_end(&dst);
! 	PG_RETURN_TEXT_P(cstring_to_text_with_len(dst.result.data, dst.result.len));
  }
  
  PG_FUNCTION_INFO_V1(hstore_to_jsonb);
diff --git a/src/backend/utils/adt/json.c b/src/backend/utils/adt/json.c
new file mode 100644
index f08e288..58de4ad
*** a/src/backend/utils/adt/json.c
--- b/src/backend/utils/adt/json.c
*************** typedef enum					/* contexts of JSON par
*** 53,73 ****
  	JSON_PARSE_END				/* saw the end of a document, expect nothing */
  } JsonParseContext;
  
- typedef enum					/* type categories for datum_to_json */
- {
- 	JSONTYPE_NULL,				/* null, so we didn't bother to identify */
- 	JSONTYPE_BOOL,				/* boolean (built-in types only) */
- 	JSONTYPE_NUMERIC,			/* numeric (ditto) */
- 	JSONTYPE_DATE,				/* we use special formatting for datetimes */
- 	JSONTYPE_TIMESTAMP,
- 	JSONTYPE_TIMESTAMPTZ,
- 	JSONTYPE_JSON,				/* JSON itself (and JSONB) */
- 	JSONTYPE_ARRAY,				/* array */
- 	JSONTYPE_COMPOSITE,			/* composite */
- 	JSONTYPE_CAST,				/* something with an explicit cast to JSON */
- 	JSONTYPE_OTHER				/* all else */
- } JsonTypeCategory;
- 
  static inline void json_lex(JsonLexContext *lex);
  static inline void json_lex_string(JsonLexContext *lex);
  static inline void json_lex_number(JsonLexContext *lex, char *s, bool *num_err);
--- 53,58 ----
*************** static void report_parse_error(JsonParse
*** 80,102 ****
  static void report_invalid_token(JsonLexContext *lex);
  static int	report_json_context(JsonLexContext *lex);
  static char *extract_mb_char(char *s);
! static void composite_to_json(Datum composite, StringInfo result,
! 				  bool use_line_feeds);
! static void array_dim_to_json(StringInfo result, int dim, int ndims, int *dims,
  				  Datum *vals, bool *nulls, int *valcount,
! 				  JsonTypeCategory tcategory, Oid outfuncoid,
! 				  bool use_line_feeds);
! static void array_to_json_internal(Datum array, StringInfo result,
! 					   bool use_line_feeds);
  static void json_categorize_type(Oid typoid,
  					 JsonTypeCategory *tcategory,
  					 Oid *outfuncoid);
! static void datum_to_json(Datum val, bool is_null, StringInfo result,
! 			  JsonTypeCategory tcategory, Oid outfuncoid,
! 			  bool key_scalar);
! static void add_json(Datum val, bool is_null, StringInfo result,
  		 Oid val_type, bool key_scalar);
- static text *catenate_stringinfo_string(StringInfo buffer, const char *addon);
  
  /* the null action object used for pure validation */
  static JsonSemAction nullSemAction =
--- 65,80 ----
  static void report_invalid_token(JsonLexContext *lex);
  static int	report_json_context(JsonLexContext *lex);
  static char *extract_mb_char(char *s);
! 
! static void array_dim_to_json(JsonOutContext *out, int dim, int ndims, int *dims,
  				  Datum *vals, bool *nulls, int *valcount,
! 				  JsonTypeCategory tcategory, Oid elemtypoid, Oid outfuncoid);
! static void array_to_json_internal(Datum array, JsonOutContext *out);
  static void json_categorize_type(Oid typoid,
  					 JsonTypeCategory *tcategory,
  					 Oid *outfuncoid);
! static void add_json(Datum val, bool is_null, JsonOutContext *out,
  		 Oid val_type, bool key_scalar);
  
  /* the null action object used for pure validation */
  static JsonSemAction nullSemAction =
*************** json_categorize_type(Oid typoid,
*** 1377,1399 ****
   * If key_scalar is true, the value is being printed as a key, so insist
   * it's of an acceptable type, and force it to be quoted.
   */
! static void
! datum_to_json(Datum val, bool is_null, StringInfo result,
! 			  JsonTypeCategory tcategory, Oid outfuncoid,
  			  bool key_scalar)
  {
- 	char	   *outputstr;
- 	text	   *jsontext;
- 
  	/* callers are expected to ensure that null keys are not passed in */
  	Assert(!(key_scalar && is_null));
  
- 	if (is_null)
- 	{
- 		appendStringInfoString(result, "null");
- 		return;
- 	}
- 
  	if (key_scalar &&
  		(tcategory == JSONTYPE_ARRAY ||
  		 tcategory == JSONTYPE_COMPOSITE ||
--- 1355,1368 ----
   * If key_scalar is true, the value is being printed as a key, so insist
   * it's of an acceptable type, and force it to be quoted.
   */
! void
! datum_to_json(Datum val, bool is_null, JsonOutContext *out,
! 			  JsonTypeCategory tcategory, Oid typoid, Oid outfuncoid,
  			  bool key_scalar)
  {
  	/* callers are expected to ensure that null keys are not passed in */
  	Assert(!(key_scalar && is_null));
  
  	if (key_scalar &&
  		(tcategory == JSONTYPE_ARRAY ||
  		 tcategory == JSONTYPE_COMPOSITE ||
*************** datum_to_json(Datum val, bool is_null, S
*** 1406,1415 ****
  	switch (tcategory)
  	{
  		case JSONTYPE_ARRAY:
! 			array_to_json_internal(val, result, false);
  			break;
  		case JSONTYPE_COMPOSITE:
! 			composite_to_json(val, result, false);
  			break;
  		case JSONTYPE_BOOL:
  			outputstr = DatumGetBool(val) ? "true" : "false";
--- 1375,1500 ----
  	switch (tcategory)
  	{
  		case JSONTYPE_ARRAY:
! 			array_to_json_internal(val, out);
  			break;
  		case JSONTYPE_COMPOSITE:
! 			composite_to_json(val, out);
! 			break;
! 		default:
! 			out->value(out, val, tcategory, typoid, outfuncoid, key_scalar);
! 			break;
! 	}
! }
! 
! void
! json_out_init_context(JsonOutContext *out, int flags)
! {
! 	out->object_start = json_out_object_start;
! 	out->object_end = json_out_object_end;
! 	out->array_start = json_out_array_start;
! 	out->array_end = json_out_array_end;
! 	out->before_value = json_out_before_value;
! 	out->value = json_out_value;
! 	out->after_value = json_out_after_value;
! 
! 	initStringInfo(&out->result);
! 	out->flags = flags;
! 	out->need_comma = false;
! 	out->depth = 0;
! 
! 	out->agg_tcategory = JSONTYPE_OTHER;
! 	out->agg_outfuncoid = InvalidOid;
! }
! 
! void
! json_out_before_value(JsonOutContext *out)
! {
! 	if (out->need_comma)
! 	{
! 		out->need_comma = false;
! 		appendStringInfoChar(&out->result, ',');
! 
! 		/* don't get into all the prettiness deep in the object structure */
! 		if (out->depth == 1)
! 		{
! 			if ((out->flags & JSON_OUT_USE_SPACES) != 0)
! 				appendStringInfoChar(&out->result, ' ');
! 
! 			if ((out->flags & JSON_OUT_USE_LINE_FEEDS) != 0)
! 				appendStringInfoString(&out->result, "\n ");
! 		}
! 	}
! }
! 
! void
! json_out_after_value(JsonOutContext *out, bool key_scalar)
! {
! 	if (key_scalar)
! 	{
! 		appendStringInfoChar(&out->result, ':');
! 
! 		if (out->depth == 1 && (out->flags & JSON_OUT_USE_SPACES) != 0)
! 			appendStringInfoChar(&out->result, ' ');
! 
! 		out->need_comma = false;
! 	}
! 	else
! 		out->need_comma = true;
! }
! 
! void
! json_out_object_start(JsonOutContext *out)
! {
! 	out->before_value(out);
! 
! 	appendStringInfoChar(&out->result, '{');
! 	out->depth++;
! }
! 
! void
! json_out_object_end(JsonOutContext *out)
! {
! 	appendStringInfoChar(&out->result, '}');
! 
! 	out->after_value(out, false);
! 	out->depth--;
! }
! 
! void
! json_out_array_start(JsonOutContext *out)
! {
! 	out->before_value(out);
! 
! 	appendStringInfoChar(&out->result, '[');
! 	out->depth++;
! }
! 
! void
! json_out_array_end(JsonOutContext *out)
! {
! 	appendStringInfoChar(&out->result, ']');
! 
! 	out->after_value(out, false);
! 	out->depth--;
! }
! 
! void
! json_out_value(JsonOutContext *out, Datum val, JsonTypeCategory tcategory,
! 			Oid typoid, Oid outfuncoid, bool key_scalar)
! {
! 	char	   *outputstr;
! 	text	   *jsontext;
! 	StringInfo		result = &out->result;
! 
! 	Assert(!(tcategory == JSONTYPE_ARRAY || tcategory == JSONTYPE_COMPOSITE));
! 
! 	/* check if there was an element before this one and add a separator */
! 	out->before_value(out);
! 
! 	switch (tcategory)
! 	{
! 		case JSONTYPE_NULL:
! 			appendStringInfoString(result, "null");
  			break;
  		case JSONTYPE_BOOL:
  			outputstr = DatumGetBool(val) ? "true" : "false";
*************** datum_to_json(Datum val, bool is_null, S
*** 1520,1530 ****
  			pfree(jsontext);
  			break;
  		default:
! 			outputstr = OidOutputFunctionCall(outfuncoid, val);
! 			escape_json(result, outputstr);
! 			pfree(outputstr);
  			break;
  	}
  }
  
  /*
--- 1605,1630 ----
  			pfree(jsontext);
  			break;
  		default:
! 			if (typoid == CSTRINGOID)
! 				escape_json(result, DatumGetCString(val));
! 			else
! 			{
! 				outputstr = OidOutputFunctionCall(outfuncoid, val);
! 				escape_json(result, outputstr);
! 				pfree(outputstr);
! 			}
  			break;
  	}
+ 
+ 	/* output key-value separator if needed and set need_comma accordingly */
+ 	out->after_value(out, key_scalar);
+ }
+ 
+ void
+ json_out_cstring(JsonOutContext *out, const char *str, bool key_scalar)
+ {
+ 	out->value(out, CStringGetDatum(str), JSONTYPE_OTHER,
+ 			   CSTRINGOID, 2293 /* cstring_out */, key_scalar);
  }
  
  /*
*************** datum_to_json(Datum val, bool is_null, S
*** 1533,1581 ****
   * ourselves recursively to process the next dimension.
   */
  static void
! array_dim_to_json(StringInfo result, int dim, int ndims, int *dims, Datum *vals,
  				  bool *nulls, int *valcount, JsonTypeCategory tcategory,
! 				  Oid outfuncoid, bool use_line_feeds)
  {
  	int			i;
- 	const char *sep;
  
  	Assert(dim < ndims);
  
! 	sep = use_line_feeds ? ",\n " : ",";
! 
! 	appendStringInfoChar(result, '[');
  
  	for (i = 1; i <= dims[dim]; i++)
  	{
- 		if (i > 1)
- 			appendStringInfoString(result, sep);
- 
  		if (dim + 1 == ndims)
  		{
! 			datum_to_json(vals[*valcount], nulls[*valcount], result, tcategory,
! 						  outfuncoid, false);
  			(*valcount)++;
  		}
  		else
! 		{
! 			/*
! 			 * Do we want line feeds on inner dimensions of arrays? For now
! 			 * we'll say no.
! 			 */
! 			array_dim_to_json(result, dim + 1, ndims, dims, vals, nulls,
! 							  valcount, tcategory, outfuncoid, false);
! 		}
  	}
  
! 	appendStringInfoChar(result, ']');
  }
  
  /*
   * Turn an array into JSON.
   */
  static void
! array_to_json_internal(Datum array, StringInfo result, bool use_line_feeds)
  {
  	ArrayType  *v = DatumGetArrayTypeP(array);
  	Oid			element_type = ARR_ELEMTYPE(v);
--- 1633,1669 ----
   * ourselves recursively to process the next dimension.
   */
  static void
! array_dim_to_json(JsonOutContext *out, int dim, int ndims, int *dims, Datum *vals,
  				  bool *nulls, int *valcount, JsonTypeCategory tcategory,
! 				  Oid elemtypoid, Oid outfuncoid)
  {
  	int			i;
  
  	Assert(dim < ndims);
  
! 	out->array_start(out);
  
  	for (i = 1; i <= dims[dim]; i++)
  	{
  		if (dim + 1 == ndims)
  		{
! 			datum_to_json(vals[*valcount], nulls[*valcount], out, tcategory,
! 						  elemtypoid, outfuncoid, false);
  			(*valcount)++;
  		}
  		else
! 			array_dim_to_json(out, dim + 1, ndims, dims, vals, nulls,
! 							  valcount, tcategory, elemtypoid, outfuncoid);
  	}
  
! 	out->array_end(out);
  }
  
  /*
   * Turn an array into JSON.
   */
  static void
! array_to_json_internal(Datum array, JsonOutContext *out)
  {
  	ArrayType  *v = DatumGetArrayTypeP(array);
  	Oid			element_type = ARR_ELEMTYPE(v);
*************** array_to_json_internal(Datum array, Stri
*** 1597,1603 ****
  
  	if (nitems <= 0)
  	{
! 		appendStringInfoString(result, "[]");
  		return;
  	}
  
--- 1685,1692 ----
  
  	if (nitems <= 0)
  	{
! 		out->array_start(out);
! 		out->array_end(out);
  		return;
  	}
  
*************** array_to_json_internal(Datum array, Stri
*** 1611,1618 ****
  					  typalign, &elements, &nulls,
  					  &nitems);
  
! 	array_dim_to_json(result, 0, ndim, dim, elements, nulls, &count, tcategory,
! 					  outfuncoid, use_line_feeds);
  
  	pfree(elements);
  	pfree(nulls);
--- 1700,1707 ----
  					  typalign, &elements, &nulls,
  					  &nitems);
  
! 	array_dim_to_json(out, 0, ndim, dim, elements, nulls, &count, tcategory,
! 					  element_type, outfuncoid);
  
  	pfree(elements);
  	pfree(nulls);
*************** array_to_json_internal(Datum array, Stri
*** 1621,1628 ****
  /*
   * Turn a composite / record into JSON.
   */
! static void
! composite_to_json(Datum composite, StringInfo result, bool use_line_feeds)
  {
  	HeapTupleHeader td;
  	Oid			tupType;
--- 1710,1717 ----
  /*
   * Turn a composite / record into JSON.
   */
! void
! composite_to_json(Datum composite, JsonOutContext *out)
  {
  	HeapTupleHeader td;
  	Oid			tupType;
*************** composite_to_json(Datum composite, Strin
*** 1631,1640 ****
  	HeapTupleData tmptup,
  			   *tuple;
  	int			i;
- 	bool		needsep = false;
- 	const char *sep;
- 
- 	sep = use_line_feeds ? ",\n " : ",";
  
  	td = DatumGetHeapTupleHeader(composite);
  
--- 1720,1725 ----
*************** composite_to_json(Datum composite, Strin
*** 1648,1654 ****
  	tmptup.t_data = td;
  	tuple = &tmptup;
  
! 	appendStringInfoChar(result, '{');
  
  	for (i = 0; i < tupdesc->natts; i++)
  	{
--- 1733,1739 ----
  	tmptup.t_data = td;
  	tuple = &tmptup;
  
! 	out->object_start(out);
  
  	for (i = 0; i < tupdesc->natts; i++)
  	{
*************** composite_to_json(Datum composite, Strin
*** 1661,1676 ****
  		if (tupdesc->attrs[i]->attisdropped)
  			continue;
  
- 		if (needsep)
- 			appendStringInfoString(result, sep);
- 		needsep = true;
- 
  		attname = NameStr(tupdesc->attrs[i]->attname);
! 		escape_json(result, attname);
! 		appendStringInfoChar(result, ':');
  
  		val = heap_getattr(tuple, i + 1, tupdesc, &isnull);
- 
  		if (isnull)
  		{
  			tcategory = JSONTYPE_NULL;
--- 1746,1755 ----
  		if (tupdesc->attrs[i]->attisdropped)
  			continue;
  
  		attname = NameStr(tupdesc->attrs[i]->attname);
! 		json_out_cstring(out, attname, true);
  
  		val = heap_getattr(tuple, i + 1, tupdesc, &isnull);
  		if (isnull)
  		{
  			tcategory = JSONTYPE_NULL;
*************** composite_to_json(Datum composite, Strin
*** 1680,1689 ****
  			json_categorize_type(tupdesc->attrs[i]->atttypid,
  								 &tcategory, &outfuncoid);
  
! 		datum_to_json(val, isnull, result, tcategory, outfuncoid, false);
  	}
  
! 	appendStringInfoChar(result, '}');
  	ReleaseTupleDesc(tupdesc);
  }
  
--- 1759,1770 ----
  			json_categorize_type(tupdesc->attrs[i]->atttypid,
  								 &tcategory, &outfuncoid);
  
! 		datum_to_json(val, isnull, out, tcategory,
! 					  tupdesc->attrs[i]->atttypid, outfuncoid, false);
  	}
  
! 	out->object_end(out);
! 
  	ReleaseTupleDesc(tupdesc);
  }
  
*************** composite_to_json(Datum composite, Strin
*** 1695,1701 ****
   * lookups only once.
   */
  static void
! add_json(Datum val, bool is_null, StringInfo result,
  		 Oid val_type, bool key_scalar)
  {
  	JsonTypeCategory tcategory;
--- 1776,1782 ----
   * lookups only once.
   */
  static void
! add_json(Datum val, bool is_null, JsonOutContext *out,
  		 Oid val_type, bool key_scalar)
  {
  	JsonTypeCategory tcategory;
*************** add_json(Datum val, bool is_null, String
*** 1715,1721 ****
  		json_categorize_type(val_type,
  							 &tcategory, &outfuncoid);
  
! 	datum_to_json(val, is_null, result, tcategory, outfuncoid, key_scalar);
  }
  
  /*
--- 1796,1802 ----
  		json_categorize_type(val_type,
  							 &tcategory, &outfuncoid);
  
! 	datum_to_json(val, is_null, out, tcategory, val_type, outfuncoid, key_scalar);
  }
  
  /*
*************** extern Datum
*** 1725,1737 ****
  array_to_json(PG_FUNCTION_ARGS)
  {
  	Datum		array = PG_GETARG_DATUM(0);
! 	StringInfo	result;
! 
! 	result = makeStringInfo();
  
! 	array_to_json_internal(array, result, false);
  
! 	PG_RETURN_TEXT_P(cstring_to_text_with_len(result->data, result->len));
  }
  
  /*
--- 1806,1817 ----
  array_to_json(PG_FUNCTION_ARGS)
  {
  	Datum		array = PG_GETARG_DATUM(0);
! 	JsonOutContext	out;
  
! 	json_out_init_context(&out, 0);
! 	array_to_json_internal(array, &out);
  
! 	PG_RETURN_TEXT_P(cstring_to_text_with_len(out.result.data, out.result.len));
  }
  
  /*
*************** array_to_json_pretty(PG_FUNCTION_ARGS)
*** 1742,1754 ****
  {
  	Datum		array = PG_GETARG_DATUM(0);
  	bool		use_line_feeds = PG_GETARG_BOOL(1);
! 	StringInfo	result;
! 
! 	result = makeStringInfo();
  
! 	array_to_json_internal(array, result, use_line_feeds);
  
! 	PG_RETURN_TEXT_P(cstring_to_text_with_len(result->data, result->len));
  }
  
  /*
--- 1822,1833 ----
  {
  	Datum		array = PG_GETARG_DATUM(0);
  	bool		use_line_feeds = PG_GETARG_BOOL(1);
! 	JsonOutContext	out;
  
! 	json_out_init_context(&out, use_line_feeds ? JSON_OUT_USE_LINE_FEEDS : 0);
! 	array_to_json_internal(array, &out);
  
! 	PG_RETURN_TEXT_P(cstring_to_text_with_len(out.result.data, out.result.len));
  }
  
  /*
*************** extern Datum
*** 1758,1770 ****
  row_to_json(PG_FUNCTION_ARGS)
  {
  	Datum		array = PG_GETARG_DATUM(0);
! 	StringInfo	result;
! 
! 	result = makeStringInfo();
  
! 	composite_to_json(array, result, false);
  
! 	PG_RETURN_TEXT_P(cstring_to_text_with_len(result->data, result->len));
  }
  
  /*
--- 1837,1848 ----
  row_to_json(PG_FUNCTION_ARGS)
  {
  	Datum		array = PG_GETARG_DATUM(0);
! 	JsonOutContext	out;
  
! 	json_out_init_context(&out, 0);
! 	composite_to_json(array, &out);
  
! 	PG_RETURN_TEXT_P(cstring_to_text_with_len(out.result.data, out.result.len));
  }
  
  /*
*************** row_to_json_pretty(PG_FUNCTION_ARGS)
*** 1775,1787 ****
  {
  	Datum		array = PG_GETARG_DATUM(0);
  	bool		use_line_feeds = PG_GETARG_BOOL(1);
! 	StringInfo	result;
! 
! 	result = makeStringInfo();
  
! 	composite_to_json(array, result, use_line_feeds);
  
! 	PG_RETURN_TEXT_P(cstring_to_text_with_len(result->data, result->len));
  }
  
  /*
--- 1853,1864 ----
  {
  	Datum		array = PG_GETARG_DATUM(0);
  	bool		use_line_feeds = PG_GETARG_BOOL(1);
! 	JsonOutContext	out;
  
! 	json_out_init_context(&out, use_line_feeds ? JSON_OUT_USE_LINE_FEEDS : 0);
! 	composite_to_json(array, &out);
  
! 	PG_RETURN_TEXT_P(cstring_to_text_with_len(out.result.data, out.result.len));
  }
  
  /*
*************** to_json(PG_FUNCTION_ARGS)
*** 1792,1800 ****
  {
  	Datum		val = PG_GETARG_DATUM(0);
  	Oid			val_type = get_fn_expr_argtype(fcinfo->flinfo, 0);
- 	StringInfo	result;
  	JsonTypeCategory tcategory;
  	Oid			outfuncoid;
  
  	if (val_type == InvalidOid)
  		ereport(ERROR,
--- 1869,1879 ----
  {
  	Datum		val = PG_GETARG_DATUM(0);
  	Oid			val_type = get_fn_expr_argtype(fcinfo->flinfo, 0);
  	JsonTypeCategory tcategory;
  	Oid			outfuncoid;
+ 	JsonOutContext	out;
+ 
+ 	json_out_init_context(&out, 0);
  
  	if (val_type == InvalidOid)
  		ereport(ERROR,
*************** to_json(PG_FUNCTION_ARGS)
*** 1804,1814 ****
  	json_categorize_type(val_type,
  						 &tcategory, &outfuncoid);
  
! 	result = makeStringInfo();
! 
! 	datum_to_json(val, false, result, tcategory, outfuncoid, false);
  
! 	PG_RETURN_TEXT_P(cstring_to_text_with_len(result->data, result->len));
  }
  
  /*
--- 1883,1891 ----
  	json_categorize_type(val_type,
  						 &tcategory, &outfuncoid);
  
! 	datum_to_json(val, false, &out, tcategory, val_type, outfuncoid, false);
  
! 	PG_RETURN_TEXT_P(cstring_to_text_with_len(out.result.data, out.result.len));
  }
  
  /*
*************** json_agg_transfn(PG_FUNCTION_ARGS)
*** 1822,1831 ****
  	Oid			val_type = get_fn_expr_argtype(fcinfo->flinfo, 1);
  	MemoryContext aggcontext,
  				oldcontext;
! 	StringInfo	state;
  	Datum		val;
- 	JsonTypeCategory tcategory;
- 	Oid			outfuncoid;
  
  	if (val_type == InvalidOid)
  		ereport(ERROR,
--- 1899,1906 ----
  	Oid			val_type = get_fn_expr_argtype(fcinfo->flinfo, 1);
  	MemoryContext aggcontext,
  				oldcontext;
! 	JsonOutContext	*out = NULL;
  	Datum		val;
  
  	if (val_type == InvalidOid)
  		ereport(ERROR,
*************** json_agg_transfn(PG_FUNCTION_ARGS)
*** 1847,1891 ****
  		 * use the right context to enlarge the object if necessary.
  		 */
  		oldcontext = MemoryContextSwitchTo(aggcontext);
! 		state = makeStringInfo();
  		MemoryContextSwitchTo(oldcontext);
  
! 		appendStringInfoChar(state, '[');
  	}
  	else
! 	{
! 		state = (StringInfo) PG_GETARG_POINTER(0);
! 		appendStringInfoString(state, ", ");
! 	}
  
  	/* fast path for NULLs */
  	if (PG_ARGISNULL(1))
  	{
! 		datum_to_json((Datum) 0, true, state, JSONTYPE_NULL, InvalidOid, false);
! 		PG_RETURN_POINTER(state);
  	}
  
  	val = PG_GETARG_DATUM(1);
  
! 	/* XXX we do this every time?? */
! 	json_categorize_type(val_type,
! 						 &tcategory, &outfuncoid);
! 
! 	/* add some whitespace if structured type and not first item */
! 	if (!PG_ARGISNULL(0) &&
! 		(tcategory == JSONTYPE_ARRAY || tcategory == JSONTYPE_COMPOSITE))
! 	{
! 		appendStringInfoString(state, "\n ");
! 	}
  
! 	datum_to_json(val, false, state, tcategory, outfuncoid, false);
  
  	/*
  	 * The transition type for array_agg() is declared to be "internal", which
  	 * is a pass-by-value type the same size as a pointer.  So we can safely
! 	 * pass the ArrayBuildState pointer through nodeAgg.c's machinations.
  	 */
! 	PG_RETURN_POINTER(state);
  }
  
  /*
--- 1922,1959 ----
  		 * use the right context to enlarge the object if necessary.
  		 */
  		oldcontext = MemoryContextSwitchTo(aggcontext);
! 		out = palloc(sizeof(JsonOutContext));
! 		json_out_init_context(out, JSON_OUT_USE_LINE_FEEDS);
  		MemoryContextSwitchTo(oldcontext);
  
! 		out->array_start(out);
  	}
  	else
! 		out = (JsonOutContext *) PG_GETARG_POINTER(0);
  
  	/* fast path for NULLs */
  	if (PG_ARGISNULL(1))
  	{
! 		datum_to_json((Datum) 0, true, out,
! 					  JSONTYPE_NULL, InvalidOid, InvalidOid, false);
! 		PG_RETURN_POINTER(out);
  	}
  
  	val = PG_GETARG_DATUM(1);
  
! 	if (out->agg_outfuncoid == InvalidOid)
! 		json_categorize_type(val_type,
! 							 &out->agg_tcategory, &out->agg_outfuncoid);
  
! 	datum_to_json(val, false, out, out->agg_tcategory,
! 				  val_type, out->agg_outfuncoid, false);
  
  	/*
  	 * The transition type for array_agg() is declared to be "internal", which
  	 * is a pass-by-value type the same size as a pointer.  So we can safely
! 	 * pass the ArrayBuildOut pointer through nodeAgg.c's machinations.
  	 */
! 	PG_RETURN_POINTER(out);
  }
  
  /*
*************** json_agg_transfn(PG_FUNCTION_ARGS)
*** 1894,1912 ****
  Datum
  json_agg_finalfn(PG_FUNCTION_ARGS)
  {
! 	StringInfo	state;
  
  	/* cannot be called directly because of internal-type argument */
  	Assert(AggCheckCallContext(fcinfo, NULL));
  
! 	state = PG_ARGISNULL(0) ? NULL : (StringInfo) PG_GETARG_POINTER(0);
  
  	/* NULL result for no rows in, as is standard with aggregates */
! 	if (state == NULL)
  		PG_RETURN_NULL();
  
! 	/* Else return state with appropriate array terminator added */
! 	PG_RETURN_TEXT_P(catenate_stringinfo_string(state, "]"));
  }
  
  /*
--- 1962,1981 ----
  Datum
  json_agg_finalfn(PG_FUNCTION_ARGS)
  {
! 	JsonOutContext	*out;
  
  	/* cannot be called directly because of internal-type argument */
  	Assert(AggCheckCallContext(fcinfo, NULL));
  
! 	out = PG_ARGISNULL(0) ? NULL : (JsonOutContext *) PG_GETARG_POINTER(0);
  
  	/* NULL result for no rows in, as is standard with aggregates */
! 	if (out == NULL)
  		PG_RETURN_NULL();
  
! 	out->array_end(out);
! 
! 	PG_RETURN_TEXT_P(cstring_to_text_with_len(out->result.data, out->result.len));
  }
  
  /*
*************** json_object_agg_transfn(PG_FUNCTION_ARGS
*** 1920,1926 ****
  	Oid			val_type;
  	MemoryContext aggcontext,
  				oldcontext;
! 	StringInfo	state;
  	Datum		arg;
  
  	if (!AggCheckCallContext(fcinfo, &aggcontext))
--- 1989,1995 ----
  	Oid			val_type;
  	MemoryContext aggcontext,
  				oldcontext;
! 	JsonOutContext	*out = NULL;
  	Datum		arg;
  
  	if (!AggCheckCallContext(fcinfo, &aggcontext))
*************** json_object_agg_transfn(PG_FUNCTION_ARGS
*** 1938,1953 ****
  		 * use the right context to enlarge the object if necessary.
  		 */
  		oldcontext = MemoryContextSwitchTo(aggcontext);
! 		state = makeStringInfo();
  		MemoryContextSwitchTo(oldcontext);
  
! 		appendStringInfoString(state, "{ ");
  	}
  	else
! 	{
! 		state = (StringInfo) PG_GETARG_POINTER(0);
! 		appendStringInfoString(state, ", ");
! 	}
  
  	/*
  	 * Note: since json_object_agg() is declared as taking type "any", the
--- 2007,2020 ----
  		 * use the right context to enlarge the object if necessary.
  		 */
  		oldcontext = MemoryContextSwitchTo(aggcontext);
! 		out = palloc(sizeof(JsonOutContext));
! 		json_out_init_context(out, JSON_OUT_USE_SPACES);
  		MemoryContextSwitchTo(oldcontext);
  
! 		out->object_start(out);
  	}
  	else
! 		out = (JsonOutContext *) PG_GETARG_POINTER(0);
  
  	/*
  	 * Note: since json_object_agg() is declared as taking type "any", the
*************** json_object_agg_transfn(PG_FUNCTION_ARGS
*** 1970,1978 ****
  
  	arg = PG_GETARG_DATUM(1);
  
! 	add_json(arg, false, state, val_type, true);
! 
! 	appendStringInfoString(state, " : ");
  
  	val_type = get_fn_expr_argtype(fcinfo->flinfo, 2);
  
--- 2037,2043 ----
  
  	arg = PG_GETARG_DATUM(1);
  
! 	add_json(arg, false, out, val_type, true);
  
  	val_type = get_fn_expr_argtype(fcinfo->flinfo, 2);
  
*************** json_object_agg_transfn(PG_FUNCTION_ARGS
*** 1986,1994 ****
  	else
  		arg = PG_GETARG_DATUM(2);
  
! 	add_json(arg, PG_ARGISNULL(2), state, val_type, false);
  
! 	PG_RETURN_POINTER(state);
  }
  
  /*
--- 2051,2059 ----
  	else
  		arg = PG_GETARG_DATUM(2);
  
! 	add_json(arg, PG_ARGISNULL(2), out, val_type, false);
  
! 	PG_RETURN_POINTER(out);
  }
  
  /*
*************** json_object_agg_transfn(PG_FUNCTION_ARGS
*** 1997,2035 ****
  Datum
  json_object_agg_finalfn(PG_FUNCTION_ARGS)
  {
! 	StringInfo	state;
  
  	/* cannot be called directly because of internal-type argument */
  	Assert(AggCheckCallContext(fcinfo, NULL));
  
! 	state = PG_ARGISNULL(0) ? NULL : (StringInfo) PG_GETARG_POINTER(0);
  
  	/* NULL result for no rows in, as is standard with aggregates */
! 	if (state == NULL)
  		PG_RETURN_NULL();
  
! 	/* Else return state with appropriate object terminator added */
! 	PG_RETURN_TEXT_P(catenate_stringinfo_string(state, " }"));
! }
! 
! /*
!  * Helper function for aggregates: return given StringInfo's contents plus
!  * specified trailing string, as a text datum.  We need this because aggregate
!  * final functions are not allowed to modify the aggregate state.
!  */
! static text *
! catenate_stringinfo_string(StringInfo buffer, const char *addon)
! {
! 	/* custom version of cstring_to_text_with_len */
! 	int			buflen = buffer->len;
! 	int			addlen = strlen(addon);
! 	text	   *result = (text *) palloc(buflen + addlen + VARHDRSZ);
! 
! 	SET_VARSIZE(result, buflen + addlen + VARHDRSZ);
! 	memcpy(VARDATA(result), buffer->data, buflen);
! 	memcpy(VARDATA(result) + buflen, addon, addlen);
  
! 	return result;
  }
  
  /*
--- 2062,2081 ----
  Datum
  json_object_agg_finalfn(PG_FUNCTION_ARGS)
  {
! 	JsonOutContext	*out;
  
  	/* cannot be called directly because of internal-type argument */
  	Assert(AggCheckCallContext(fcinfo, NULL));
  
! 	out = PG_ARGISNULL(0) ? NULL : (JsonOutContext *) PG_GETARG_POINTER(0);
  
  	/* NULL result for no rows in, as is standard with aggregates */
! 	if (out == NULL)
  		PG_RETURN_NULL();
  
! 	out->object_end(out);
  
! 	PG_RETURN_TEXT_P(cstring_to_text_with_len(out->result.data, out->result.len));
  }
  
  /*
*************** json_build_object(PG_FUNCTION_ARGS)
*** 2041,2048 ****
  	int			nargs = PG_NARGS();
  	int			i;
  	Datum		arg;
! 	const char *sep = "";
! 	StringInfo	result;
  	Oid			val_type;
  
  	if (nargs % 2 != 0)
--- 2087,2093 ----
  	int			nargs = PG_NARGS();
  	int			i;
  	Datum		arg;
! 	JsonOutContext	out;
  	Oid			val_type;
  
  	if (nargs % 2 != 0)
*************** json_build_object(PG_FUNCTION_ARGS)
*** 2051,2059 ****
  				 errmsg("argument list must have even number of elements"),
  				 errhint("The arguments of json_build_object() must consist of alternating keys and values.")));
  
! 	result = makeStringInfo();
! 
! 	appendStringInfoChar(result, '{');
  
  	for (i = 0; i < nargs; i += 2)
  	{
--- 2096,2103 ----
  				 errmsg("argument list must have even number of elements"),
  				 errhint("The arguments of json_build_object() must consist of alternating keys and values.")));
  
! 	json_out_init_context(&out, JSON_OUT_USE_SPACES);
! 	out.object_start(&out);
  
  	for (i = 0; i < nargs; i += 2)
  	{
*************** json_build_object(PG_FUNCTION_ARGS)
*** 2064,2071 ****
  		 * here as type UNKNOWN, which fortunately does not matter to us,
  		 * since unknownout() works fine.
  		 */
- 		appendStringInfoString(result, sep);
- 		sep = ", ";
  
  		/* process key */
  		val_type = get_fn_expr_argtype(fcinfo->flinfo, i);
--- 2108,2113 ----
*************** json_build_object(PG_FUNCTION_ARGS)
*** 2084,2092 ****
  
  		arg = PG_GETARG_DATUM(i);
  
! 		add_json(arg, false, result, val_type, true);
! 
! 		appendStringInfoString(result, " : ");
  
  		/* process value */
  		val_type = get_fn_expr_argtype(fcinfo->flinfo, i + 1);
--- 2126,2132 ----
  
  		arg = PG_GETARG_DATUM(i);
  
! 		add_json(arg, false, &out, val_type, true);
  
  		/* process value */
  		val_type = get_fn_expr_argtype(fcinfo->flinfo, i + 1);
*************** json_build_object(PG_FUNCTION_ARGS)
*** 2102,2113 ****
  		else
  			arg = PG_GETARG_DATUM(i + 1);
  
! 		add_json(arg, PG_ARGISNULL(i + 1), result, val_type, false);
  	}
  
! 	appendStringInfoChar(result, '}');
  
! 	PG_RETURN_TEXT_P(cstring_to_text_with_len(result->data, result->len));
  }
  
  /*
--- 2142,2153 ----
  		else
  			arg = PG_GETARG_DATUM(i + 1);
  
! 		add_json(arg, PG_ARGISNULL(i + 1), &out, val_type, false);
  	}
  
! 	out.object_end(&out);
  
! 	PG_RETURN_TEXT_P(cstring_to_text_with_len(out.result.data, out.result.len));
  }
  
  /*
*************** json_build_array(PG_FUNCTION_ARGS)
*** 2128,2140 ****
  	int			nargs = PG_NARGS();
  	int			i;
  	Datum		arg;
! 	const char *sep = "";
! 	StringInfo	result;
  	Oid			val_type;
  
! 	result = makeStringInfo();
! 
! 	appendStringInfoChar(result, '[');
  
  	for (i = 0; i < nargs; i++)
  	{
--- 2168,2178 ----
  	int			nargs = PG_NARGS();
  	int			i;
  	Datum		arg;
! 	JsonOutContext	out;
  	Oid			val_type;
  
! 	json_out_init_context(&out, JSON_OUT_USE_SPACES);
! 	out.array_start(&out);
  
  	for (i = 0; i < nargs; i++)
  	{
*************** json_build_array(PG_FUNCTION_ARGS)
*** 2145,2153 ****
  		 * here as type UNKNOWN, which fortunately does not matter to us,
  		 * since unknownout() works fine.
  		 */
- 		appendStringInfoString(result, sep);
- 		sep = ", ";
- 
  		val_type = get_fn_expr_argtype(fcinfo->flinfo, i);
  
  		if (val_type == InvalidOid)
--- 2183,2188 ----
*************** json_build_array(PG_FUNCTION_ARGS)
*** 2161,2172 ****
  		else
  			arg = PG_GETARG_DATUM(i);
  
! 		add_json(arg, PG_ARGISNULL(i), result, val_type, false);
  	}
  
! 	appendStringInfoChar(result, ']');
  
! 	PG_RETURN_TEXT_P(cstring_to_text_with_len(result->data, result->len));
  }
  
  /*
--- 2196,2207 ----
  		else
  			arg = PG_GETARG_DATUM(i);
  
! 		add_json(arg, PG_ARGISNULL(i), &out, val_type, false);
  	}
  
! 	out.array_end(&out);
  
! 	PG_RETURN_TEXT_P(cstring_to_text_with_len(out.result.data, out.result.len));
  }
  
  /*
*************** json_object(PG_FUNCTION_ARGS)
*** 2189,2202 ****
  {
  	ArrayType  *in_array = PG_GETARG_ARRAYTYPE_P(0);
  	int			ndims = ARR_NDIM(in_array);
! 	StringInfoData result;
  	Datum	   *in_datums;
  	bool	   *in_nulls;
  	int			in_count,
  				count,
  				i;
- 	text	   *rval;
- 	char	   *v;
  
  	switch (ndims)
  	{
--- 2224,2235 ----
  {
  	ArrayType  *in_array = PG_GETARG_ARRAYTYPE_P(0);
  	int			ndims = ARR_NDIM(in_array);
! 	JsonOutContext	out;
  	Datum	   *in_datums;
  	bool	   *in_nulls;
  	int			in_count,
  				count,
  				i;
  
  	switch (ndims)
  	{
*************** json_object(PG_FUNCTION_ARGS)
*** 2230,2238 ****
  
  	count = in_count / 2;
  
! 	initStringInfo(&result);
! 
! 	appendStringInfoChar(&result, '{');
  
  	for (i = 0; i < count; ++i)
  	{
--- 2263,2270 ----
  
  	count = in_count / 2;
  
! 	json_out_init_context(&out, JSON_OUT_USE_SPACES);
! 	out.object_start(&out);
  
  	for (i = 0; i < count; ++i)
  	{
*************** json_object(PG_FUNCTION_ARGS)
*** 2241,2272 ****
  					(errcode(ERRCODE_NULL_VALUE_NOT_ALLOWED),
  					 errmsg("null value not allowed for object key")));
  
! 		v = TextDatumGetCString(in_datums[i * 2]);
! 		if (i > 0)
! 			appendStringInfoString(&result, ", ");
! 		escape_json(&result, v);
! 		appendStringInfoString(&result, " : ");
! 		pfree(v);
  		if (in_nulls[i * 2 + 1])
! 			appendStringInfoString(&result, "null");
  		else
! 		{
! 			v = TextDatumGetCString(in_datums[i * 2 + 1]);
! 			escape_json(&result, v);
! 			pfree(v);
! 		}
  	}
  
! 	appendStringInfoChar(&result, '}');
  
  	pfree(in_datums);
  	pfree(in_nulls);
  
! 	rval = cstring_to_text_with_len(result.data, result.len);
! 	pfree(result.data);
! 
! 	PG_RETURN_TEXT_P(rval);
! 
  }
  
  /*
--- 2273,2295 ----
  					(errcode(ERRCODE_NULL_VALUE_NOT_ALLOWED),
  					 errmsg("null value not allowed for object key")));
  
! 		out.value(&out, in_datums[i * 2], JSONTYPE_OTHER,
! 				  TEXTOID, 47 /* textout */, true);
! 
  		if (in_nulls[i * 2 + 1])
! 			out.value(&out, (Datum) 0, JSONTYPE_NULL,
! 					  InvalidOid, InvalidOid, false);
  		else
! 			out.value(&out, in_datums[i * 2 + 1], JSONTYPE_OTHER,
! 					  TEXTOID, 47 /* textout */, false);
  	}
  
! 	out.object_end(&out);
  
  	pfree(in_datums);
  	pfree(in_nulls);
  
! 	PG_RETURN_TEXT_P(cstring_to_text_with_len(out.result.data, out.result.len));
  }
  
  /*
*************** json_object_two_arg(PG_FUNCTION_ARGS)
*** 2282,2288 ****
  	ArrayType  *val_array = PG_GETARG_ARRAYTYPE_P(1);
  	int			nkdims = ARR_NDIM(key_array);
  	int			nvdims = ARR_NDIM(val_array);
! 	StringInfoData result;
  	Datum	   *key_datums,
  			   *val_datums;
  	bool	   *key_nulls,
--- 2305,2311 ----
  	ArrayType  *val_array = PG_GETARG_ARRAYTYPE_P(1);
  	int			nkdims = ARR_NDIM(key_array);
  	int			nvdims = ARR_NDIM(val_array);
! 	JsonOutContext	out;
  	Datum	   *key_datums,
  			   *val_datums;
  	bool	   *key_nulls,
*************** json_object_two_arg(PG_FUNCTION_ARGS)
*** 2290,2297 ****
  	int			key_count,
  				val_count,
  				i;
- 	text	   *rval;
- 	char	   *v;
  
  	if (nkdims > 1 || nkdims != nvdims)
  		ereport(ERROR,
--- 2313,2318 ----
*************** json_object_two_arg(PG_FUNCTION_ARGS)
*** 2314,2322 ****
  				(errcode(ERRCODE_ARRAY_SUBSCRIPT_ERROR),
  				 errmsg("mismatched array dimensions")));
  
! 	initStringInfo(&result);
! 
! 	appendStringInfoChar(&result, '{');
  
  	for (i = 0; i < key_count; ++i)
  	{
--- 2335,2342 ----
  				(errcode(ERRCODE_ARRAY_SUBSCRIPT_ERROR),
  				 errmsg("mismatched array dimensions")));
  
! 	json_out_init_context(&out, JSON_OUT_USE_SPACES);
! 	out.object_start(&out);
  
  	for (i = 0; i < key_count; ++i)
  	{
*************** json_object_two_arg(PG_FUNCTION_ARGS)
*** 2325,2357 ****
  					(errcode(ERRCODE_NULL_VALUE_NOT_ALLOWED),
  					 errmsg("null value not allowed for object key")));
  
! 		v = TextDatumGetCString(key_datums[i]);
! 		if (i > 0)
! 			appendStringInfoString(&result, ", ");
! 		escape_json(&result, v);
! 		appendStringInfoString(&result, " : ");
! 		pfree(v);
  		if (val_nulls[i])
! 			appendStringInfoString(&result, "null");
  		else
! 		{
! 			v = TextDatumGetCString(val_datums[i]);
! 			escape_json(&result, v);
! 			pfree(v);
! 		}
  	}
  
! 	appendStringInfoChar(&result, '}');
  
  	pfree(key_datums);
  	pfree(key_nulls);
  	pfree(val_datums);
  	pfree(val_nulls);
  
! 	rval = cstring_to_text_with_len(result.data, result.len);
! 	pfree(result.data);
! 
! 	PG_RETURN_TEXT_P(rval);
  }
  
  
--- 2345,2369 ----
  					(errcode(ERRCODE_NULL_VALUE_NOT_ALLOWED),
  					 errmsg("null value not allowed for object key")));
  
! 		out.value(&out, key_datums[i], JSONTYPE_OTHER,
! 				  TEXTOID, 47 /* textout */, true);
! 
  		if (val_nulls[i])
! 			out.value(&out, (Datum) 0, JSONTYPE_NULL,
! 					  InvalidOid, InvalidOid, false);
  		else
! 			out.value(&out, val_datums[i], JSONTYPE_OTHER,
! 					  TEXTOID, 47 /* textout */, false);
  	}
  
! 	out.object_end(&out);
  
  	pfree(key_datums);
  	pfree(key_nulls);
  	pfree(val_datums);
  	pfree(val_nulls);
  
! 	PG_RETURN_TEXT_P(cstring_to_text_with_len(out.result.data, out.result.len));
  }
  
  
diff --git a/src/include/utils/jsonapi.h b/src/include/utils/jsonapi.h
new file mode 100644
index 1d8293b..69464de
*** a/src/include/utils/jsonapi.h
--- b/src/include/utils/jsonapi.h
*************** extern JsonLexContext *makeJsonLexContex
*** 124,127 ****
--- 124,194 ----
   */
  extern bool IsValidJsonNumber(const char * str, int len);
  
+ 
+ /*
+  * Generalized structures for producing JSON output.
+  */
+ typedef enum					/* type categories for datum_to_json */
+ {
+ 	JSONTYPE_NULL,				/* null, so we didn't bother to identify */
+ 	JSONTYPE_BOOL,				/* boolean (built-in types only) */
+ 	JSONTYPE_NUMERIC,			/* numeric (ditto) */
+ 	JSONTYPE_DATE,				/* we use special formatting for datetimes */
+ 	JSONTYPE_TIMESTAMP,
+ 	JSONTYPE_TIMESTAMPTZ,
+ 	JSONTYPE_JSON,				/* JSON itself (and JSONB) */
+ 	JSONTYPE_ARRAY,				/* array */
+ 	JSONTYPE_COMPOSITE,			/* composite */
+ 	JSONTYPE_CAST,				/* something with an explicit cast to JSON */
+ 	JSONTYPE_OTHER				/* all else */
+ } JsonTypeCategory;
+ 
+ struct JsonOutContext;
+ 
+ typedef void (*json_out_struct_action)(struct JsonOutContext *out);
+ typedef void (*json_out_value_action)(struct JsonOutContext *out,
+ 			  Datum val, JsonTypeCategory tcategory,
+ 			  Oid typoid, Oid outfuncoid, bool key_scalar);
+ typedef void (*json_out_post_action)(struct JsonOutContext *out,
+ 			  bool key_scalar);
+ 
+ #define JSON_OUT_USE_LINE_FEEDS		1
+ #define JSON_OUT_USE_SPACES			2
+ 
+ typedef struct JsonOutContext {
+ 	json_out_struct_action	object_start;
+ 	json_out_struct_action	object_end;
+ 	json_out_struct_action	array_start;
+ 	json_out_struct_action	array_end;
+ 	json_out_struct_action	before_value;
+ 	json_out_value_action	value;
+ 	json_out_post_action	after_value;
+ 
+ 	StringInfoData	result;
+ 	int		flags;
+ 	bool	need_comma;
+ 	int		depth;
+ 	/* these are used in json_agg to cache the type information */
+ 	JsonTypeCategory	agg_tcategory;
+ 	Oid					agg_outfuncoid;
+ } JsonOutContext;
+ 
+ extern void json_out_init_context(JsonOutContext *out, int flags);
+ extern void json_out_before_value(JsonOutContext *out);
+ extern void json_out_after_value(JsonOutContext *out, bool key_scalar);
+ extern void json_out_object_start(JsonOutContext *out);
+ extern void json_out_object_end(JsonOutContext *out);
+ extern void json_out_array_start(JsonOutContext *out);
+ extern void json_out_array_end(JsonOutContext *out);
+ extern void json_out_value(JsonOutContext *out, Datum val,
+ 				JsonTypeCategory tcategory, Oid typoid, Oid outfuncoid,
+ 				bool key_scalar);
+ extern void json_out_cstring(JsonOutContext *out, const char *str,
+ 				bool key_scalar);
+ 
+ extern void composite_to_json(Datum composite, JsonOutContext *out);
+ extern void datum_to_json(Datum val, bool is_null, JsonOutContext *out,
+ 			  JsonTypeCategory tcategory, Oid typoid, Oid outfuncoid,
+ 			  bool key_scalar);
+ 
  #endif   /* JSONAPI_H */
diff --git a/src/test/regress/expected/json.out b/src/test/regress/expected/json.out
new file mode 100644
index 3942c3b..fbc9b5b
*** a/src/test/regress/expected/json.out
--- b/src/test/regress/expected/json.out
*************** SELECT json_agg(q)
*** 457,476 ****
                 ROW(y.*,ARRAY[4,5,6])] AS z
           FROM generate_series(1,2) x,
                generate_series(4,5) y) q;
!                                json_agg                                
! -----------------------------------------------------------------------
!  [{"b":"a1","c":4,"z":[{"f1":1,"f2":[1,2,3]},{"f1":4,"f2":[4,5,6]}]}, +
!   {"b":"a1","c":5,"z":[{"f1":1,"f2":[1,2,3]},{"f1":5,"f2":[4,5,6]}]}, +
!   {"b":"a2","c":4,"z":[{"f1":2,"f2":[1,2,3]},{"f1":4,"f2":[4,5,6]}]}, +
    {"b":"a2","c":5,"z":[{"f1":2,"f2":[1,2,3]},{"f1":5,"f2":[4,5,6]}]}]
  (1 row)
  
  SELECT json_agg(q)
    FROM rows q;
!        json_agg        
! -----------------------
!  [{"x":1,"y":"txt1"}, +
!   {"x":2,"y":"txt2"}, +
    {"x":3,"y":"txt3"}]
  (1 row)
  
--- 457,476 ----
                 ROW(y.*,ARRAY[4,5,6])] AS z
           FROM generate_series(1,2) x,
                generate_series(4,5) y) q;
!                                json_agg                               
! ----------------------------------------------------------------------
!  [{"b":"a1","c":4,"z":[{"f1":1,"f2":[1,2,3]},{"f1":4,"f2":[4,5,6]}]},+
!   {"b":"a1","c":5,"z":[{"f1":1,"f2":[1,2,3]},{"f1":5,"f2":[4,5,6]}]},+
!   {"b":"a2","c":4,"z":[{"f1":2,"f2":[1,2,3]},{"f1":4,"f2":[4,5,6]}]},+
    {"b":"a2","c":5,"z":[{"f1":2,"f2":[1,2,3]},{"f1":5,"f2":[4,5,6]}]}]
  (1 row)
  
  SELECT json_agg(q)
    FROM rows q;
!        json_agg       
! ----------------------
!  [{"x":1,"y":"txt1"},+
!   {"x":2,"y":"txt2"},+
    {"x":3,"y":"txt3"}]
  (1 row)
  
*************** SELECT json_build_array('a',1,'b',1.2,'c
*** 1505,1522 ****
  (1 row)
  
  SELECT json_build_object('a',1,'b',1.2,'c',true,'d',null,'e',json '{"x": 3, "y": [1,2,3]}');
!                              json_build_object                              
! ----------------------------------------------------------------------------
!  {"a" : 1, "b" : 1.2, "c" : true, "d" : null, "e" : {"x": 3, "y": [1,2,3]}}
  (1 row)
  
  SELECT json_build_object(
         'a', json_build_object('b',false,'c',99),
         'd', json_build_object('e',array[9,8,7]::int[],
             'f', (select row_to_json(r) from ( select relkind, oid::regclass as name from pg_class where relname = 'pg_class') r)));
!                                         json_build_object                                        
! -------------------------------------------------------------------------------------------------
!  {"a" : {"b" : false, "c" : 99}, "d" : {"e" : [9,8,7], "f" : {"relkind":"r","name":"pg_class"}}}
  (1 row)
  
  -- empty objects/arrays
--- 1505,1522 ----
  (1 row)
  
  SELECT json_build_object('a',1,'b',1.2,'c',true,'d',null,'e',json '{"x": 3, "y": [1,2,3]}');
!                            json_build_object                           
! -----------------------------------------------------------------------
!  {"a": 1, "b": 1.2, "c": true, "d": null, "e": {"x": 3, "y": [1,2,3]}}
  (1 row)
  
  SELECT json_build_object(
         'a', json_build_object('b',false,'c',99),
         'd', json_build_object('e',array[9,8,7]::int[],
             'f', (select row_to_json(r) from ( select relkind, oid::regclass as name from pg_class where relname = 'pg_class') r)));
!                                      json_build_object                                     
! -------------------------------------------------------------------------------------------
!  {"a": {"b": false, "c": 99}, "d": {"e": [9,8,7], "f": {"relkind":"r","name":"pg_class"}}}
  (1 row)
  
  -- empty objects/arrays
*************** SELECT json_build_object();
*** 1536,1542 ****
  SELECT json_build_object(1,2);
   json_build_object 
  -------------------
!  {"1" : 2}
  (1 row)
  
  -- keys must be scalar and not null
--- 1536,1542 ----
  SELECT json_build_object(1,2);
   json_build_object 
  -------------------
!  {"1": 2}
  (1 row)
  
  -- keys must be scalar and not null
*************** INSERT INTO foo VALUES (847002,'t16','GE
*** 1555,1578 ****
  INSERT INTO foo VALUES (847003,'sub-alpha','GESS90');
  SELECT json_build_object('turbines',json_object_agg(serial_num,json_build_object('name',name,'type',type)))
  FROM foo;
!                                                                             json_build_object                                                                            
! -------------------------------------------------------------------------------------------------------------------------------------------------------------------------
!  {"turbines" : { "847001" : {"name" : "t15", "type" : "GE1043"}, "847002" : {"name" : "t16", "type" : "GE1043"}, "847003" : {"name" : "sub-alpha", "type" : "GESS90"} }}
  (1 row)
  
  -- json_object
  -- one dimension
  SELECT json_object('{a,1,b,2,3,NULL,"d e f","a b c"}');
!                       json_object                      
! -------------------------------------------------------
!  {"a" : "1", "b" : "2", "3" : null, "d e f" : "a b c"}
  (1 row)
  
  -- same but with two dimensions
  SELECT json_object('{{a,1},{b,2},{3,NULL},{"d e f","a b c"}}');
!                       json_object                      
! -------------------------------------------------------
!  {"a" : "1", "b" : "2", "3" : null, "d e f" : "a b c"}
  (1 row)
  
  -- odd number error
--- 1555,1578 ----
  INSERT INTO foo VALUES (847003,'sub-alpha','GESS90');
  SELECT json_build_object('turbines',json_object_agg(serial_num,json_build_object('name',name,'type',type)))
  FROM foo;
!                                                                       json_build_object                                                                      
! -------------------------------------------------------------------------------------------------------------------------------------------------------------
!  {"turbines": {"847001": {"name": "t15", "type": "GE1043"}, "847002": {"name": "t16", "type": "GE1043"}, "847003": {"name": "sub-alpha", "type": "GESS90"}}}
  (1 row)
  
  -- json_object
  -- one dimension
  SELECT json_object('{a,1,b,2,3,NULL,"d e f","a b c"}');
!                     json_object                    
! ---------------------------------------------------
!  {"a": "1", "b": "2", "3": null, "d e f": "a b c"}
  (1 row)
  
  -- same but with two dimensions
  SELECT json_object('{{a,1},{b,2},{3,NULL},{"d e f","a b c"}}');
!                     json_object                    
! ---------------------------------------------------
!  {"a": "1", "b": "2", "3": null, "d e f": "a b c"}
  (1 row)
  
  -- odd number error
*************** SELECT json_object('{{{a,b},{c,d}},{{b,c
*** 1589,1597 ****
  ERROR:  wrong number of array subscripts
  --two argument form of json_object
  select json_object('{a,b,c,"d e f"}','{1,2,3,"a b c"}');
!                      json_object                      
! ------------------------------------------------------
!  {"a" : "1", "b" : "2", "c" : "3", "d e f" : "a b c"}
  (1 row)
  
  -- too many dimensions
--- 1589,1597 ----
  ERROR:  wrong number of array subscripts
  --two argument form of json_object
  select json_object('{a,b,c,"d e f"}','{1,2,3,"a b c"}');
!                    json_object                    
! --------------------------------------------------
!  {"a": "1", "b": "2", "c": "3", "d e f": "a b c"}
  (1 row)
  
  -- too many dimensions
*************** select json_object('{a,b,NULL,"d e f"}',
*** 1607,1615 ****
  ERROR:  null value not allowed for object key
  -- empty key is allowed
  select json_object('{a,b,"","d e f"}','{1,2,3,"a b c"}');
!                      json_object                     
! -----------------------------------------------------
!  {"a" : "1", "b" : "2", "" : "3", "d e f" : "a b c"}
  (1 row)
  
  -- json_to_record and json_to_recordset
--- 1607,1615 ----
  ERROR:  null value not allowed for object key
  -- empty key is allowed
  select json_object('{a,b,"","d e f"}','{1,2,3,"a b c"}');
!                    json_object                   
! -------------------------------------------------
!  {"a": "1", "b": "2", "": "3", "d e f": "a b c"}
  (1 row)
  
  -- json_to_record and json_to_recordset
#2Merlin Moncure
mmoncure@gmail.com
In reply to: Shulgin, Oleksandr (#1)
Re: [PATCH] Generalized JSON output functions

On Wed, May 20, 2015 at 8:16 AM, Shulgin, Oleksandr
<oleksandr.shulgin@zalando.de> wrote:

Hi, Hackers!

Attached is a patch against master to generalize the JSON-producing
functions in utils/adt/json.c and to provide a set of callbacks which can be
overridden the same way that is already provided for *parsing* JSON.

The motivation behind this to be able to produce specially-crafted JSON in a
logical replication output plugin, such that numeric (and bigint) values are
quoted. This requirement, in turn, arises from the fact that JavaScript
specification, which is quite natural to expect as a consumer for this JSON
data, allows to silently drop significant digits when converting from string
to number object.

I believe this is a well-known problem and I'm aware of a number of tricks
that might be used to avoid it, but none of them seems to be optimal from my
standpoint.

I can also imagine this can be used to convert date/time to string
differently, or adding indentation depending on the depth in object
hierarchy, etc.

What this patch does apart from providing callbacks, is abstracting most of
code for producing the correct JSON structure, which was previously
scattered and repeated in a number of functions with slight differences. In
the current code there are 5 styles for producing JSON object string,
differing in whitespace only:

a) no spaces

select to_json(row(1,2));
to_json
-----------------
{"f1":1,"f2":2}

b) some spaces (hstore_to_json)

select hstore(row(1,2))::json;
hstore
------------------------
{"f1": "1", "f2": "2"}

c) spaces around colon

select json_build_object('f1',1,'f2',2);
json_build_object
----------------------
{"f1" : 1, "f2" : 2}

d) spaces around colon *and* curly braces

select json_object_agg(x,x) from unnest('{1,2}'::int[]) x;
json_object_agg
----------------------
{ "1" : 1, "2" : 2 }

e) line feeds (row_to_json_pretty)

select row_to_json(row(1,2), true) as row;
row
----------
{"f1":1,+
"f2":2}

Personally, I think we should stick to (b), however that would break a lot
of test cases that already depend on (a). I've tried hard to minimize the
amount of changes in expected/json.out, but it is quickly becomes cumbersome
trying to support all of the above formats. So I've altered (c) and (d) to
look like (b), naturally only whitespace was affected.

Disagree. IMNSHO, the default should be (a), as it's the most compact
format and therefore the fastest. Whitespace injection should be
reserved for prettification functions.

merlin

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

#3Andrew Dunstan
andrew@dunslane.net
In reply to: Shulgin, Oleksandr (#1)
Re: [PATCH] Generalized JSON output functions

On 05/20/2015 09:16 AM, Shulgin, Oleksandr wrote:

Hi, Hackers!

Attached is a patch against master to generalize the JSON-producing
functions in utils/adt/json.c and to provide a set of callbacks which
can be overridden the same way that is already provided for *parsing*
JSON.

The motivation behind this to be able to produce specially-crafted
JSON in a logical replication output plugin, such that numeric (and
bigint) values are quoted. This requirement, in turn, arises from the
fact that JavaScript specification, which is quite natural to expect
as a consumer for this JSON data, allows to silently drop significant
digits when converting from string to number object.

I believe this is a well-known problem and I'm aware of a number of
tricks that might be used to avoid it, but none of them seems to be
optimal from my standpoint.

I can also imagine this can be used to convert date/time to string
differently, or adding indentation depending on the depth in object
hierarchy, etc.

I'm not necessarily opposed to this, but it sure seems like a lot of
changes, and moderately invasive ones, to support something that could
be done, at the cost of reparsing, with a simple loadable extension that
I could create in a few hours of programming. The parser API was created
precisely to make this sort of transformation close to trivial. Other
fairly obvious transformations include translating to XML or YAML, and a
less obvious one could be something very specialized, like translating
certain fields. Anyway, for this purpose I could imagine a function like:

json_format (
j json (or text),
indent_spaces smallint default 0,
space_after_colon boolean default false,
space_after_comma boolean default false,
quote_numerics boolean default false)
returns json

cheers

andrew

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

#4Shulgin, Oleksandr
oleksandr.shulgin@zalando.de
In reply to: Shulgin, Oleksandr (#1)
Re: [PATCH] Generalized JSON output functions

On Thu, May 21, 2015 at 6:43 PM, Merlin Moncure <mmoncure@gmail.com> wrote:

On Thu, May 21, 2015 at 2:23 AM, Shulgin, Oleksandr
<oleksandr.shulgin@zalando.de> wrote:

On Wed, May 20, 2015 at 4:06 PM, Merlin Moncure <mmoncure@gmail.com>

wrote:

On Wed, May 20, 2015 at 8:16 AM, Shulgin, Oleksandr
<oleksandr.shulgin@zalando.de> wrote:

a) no spaces

select to_json(row(1,2));
to_json
-----------------
{"f1":1,"f2":2}

b) some spaces (hstore_to_json)

select hstore(row(1,2))::json;
hstore
------------------------
{"f1": "1", "f2": "2"}

c) spaces around colon

select json_build_object('f1',1,'f2',2);
json_build_object
----------------------
{"f1" : 1, "f2" : 2}

d) spaces around colon *and* curly braces

select json_object_agg(x,x) from unnest('{1,2}'::int[]) x;
json_object_agg
----------------------
{ "1" : 1, "2" : 2 }

e) line feeds (row_to_json_pretty)

select row_to_json(row(1,2), true) as row;
row
----------
{"f1":1,+
"f2":2}

Personally, I think we should stick to (b), however that would break a
lot
of test cases that already depend on (a). I've tried hard to minimize
the
amount of changes in expected/json.out, but it is quickly becomes
cumbersome
trying to support all of the above formats. So I've altered (c) and

(d)

to
look like (b), naturally only whitespace was affected.

Disagree. IMNSHO, the default should be (a), as it's the most compact
format and therefore the fastest. Whitespace injection should be
reserved for prettification functions.

I have no strong opinion on choosing (a) over (b), just wanted to make

the

change minimally sensible. If at all, I think we should modify existing
code to make JSON output consistent: that is choose one format an stick

to

it.

sure -- did you mean to respond off-list?

No, just using an unusual mail agent :-p

anyways, inserting spacing
into the serialization function formatting (xx_to_json) for me will
raise memory profile of output json by 10%+ in nearly all cases. I
just don't see the benefit of doing that given that the json is still
not very 'pretty'.

I can agree that spaces are only useful for a human being trying to make
sense of the data. My vote is for reasonable default + an option to put
spaces/prettify on demand.

--
Alex

#5Alvaro Herrera
alvherre@2ndquadrant.com
In reply to: Andrew Dunstan (#3)
Re: [PATCH] Generalized JSON output functions

Andrew Dunstan wrote:

On 05/20/2015 09:16 AM, Shulgin, Oleksandr wrote:

Attached is a patch against master to generalize the JSON-producing
functions in utils/adt/json.c and to provide a set of callbacks which can
be overridden the same way that is already provided for *parsing* JSON.

I'm not necessarily opposed to this, but it sure seems like a lot of
changes, and moderately invasive ones, to support something that could be
done, at the cost of reparsing, with a simple loadable extension that I
could create in a few hours of programming.

But this seems like a pretty reasonable change to make, no? Doesn't the
total amount of code decrease after this patch? JSON stuff is pretty
new so some refactoring and generalization of what we have is to be
expected.

--
�lvaro Herrera http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

#6Merlin Moncure
mmoncure@gmail.com
In reply to: Alvaro Herrera (#5)
Re: [PATCH] Generalized JSON output functions

On Fri, May 22, 2015 at 9:43 AM, Alvaro Herrera
<alvherre@2ndquadrant.com> wrote:

Andrew Dunstan wrote:

On 05/20/2015 09:16 AM, Shulgin, Oleksandr wrote:

Attached is a patch against master to generalize the JSON-producing
functions in utils/adt/json.c and to provide a set of callbacks which can
be overridden the same way that is already provided for *parsing* JSON.

I'm not necessarily opposed to this, but it sure seems like a lot of
changes, and moderately invasive ones, to support something that could be
done, at the cost of reparsing, with a simple loadable extension that I
could create in a few hours of programming.

But this seems like a pretty reasonable change to make, no? Doesn't the
total amount of code decrease after this patch? JSON stuff is pretty
new so some refactoring and generalization of what we have is to be
expected.

Yeah. Also, there have been a few previous gripes about this, for
example, /messages/by-id/CAHbVmPzS+sVR+y-UgxjRq+XW4dqteVL-cOzc69zFFwmxjcKCxg@mail.gmail.com.
As noted, I definitely prefer 'space free' by default for efficiency
reasons, but standardizing the output has definitely got to be a
reasonable goal.

merlin

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

#7Ryan Pedela
rpedela@datalanche.com
In reply to: Merlin Moncure (#6)
Re: [PATCH] Generalized JSON output functions

On Fri, May 22, 2015 at 10:51 AM, Merlin Moncure <mmoncure@gmail.com> wrote:

On Fri, May 22, 2015 at 9:43 AM, Alvaro Herrera
<alvherre@2ndquadrant.com> wrote:

Andrew Dunstan wrote:

On 05/20/2015 09:16 AM, Shulgin, Oleksandr wrote:

Attached is a patch against master to generalize the JSON-producing
functions in utils/adt/json.c and to provide a set of callbacks which

can

be overridden the same way that is already provided for *parsing* JSON.

I'm not necessarily opposed to this, but it sure seems like a lot of
changes, and moderately invasive ones, to support something that could

be

done, at the cost of reparsing, with a simple loadable extension that I
could create in a few hours of programming.

But this seems like a pretty reasonable change to make, no? Doesn't the
total amount of code decrease after this patch? JSON stuff is pretty
new so some refactoring and generalization of what we have is to be
expected.

Yeah. Also, there have been a few previous gripes about this, for
example,
/messages/by-id/CAHbVmPzS+sVR+y-UgxjRq+XW4dqteVL-cOzc69zFFwmxjcKCxg@mail.gmail.com
.
As noted, I definitely prefer 'space free' by default for efficiency
reasons, but standardizing the output has definitely got to be a
reasonable goal.

Every JSON implementation I have ever used defaults to the minified version
of JSON (no whitespace) when printed.

#8Shulgin, Oleksandr
oleksandr.shulgin@zalando.de
In reply to: Ryan Pedela (#7)
Re: [PATCH] Generalized JSON output functions

On Sat, May 23, 2015 at 3:03 AM, Ryan Pedela <rpedela@datalanche.com> wrote:

On Fri, May 22, 2015 at 10:51 AM, Merlin Moncure <mmoncure@gmail.com>
wrote:

On Fri, May 22, 2015 at 9:43 AM, Alvaro Herrera
<alvherre@2ndquadrant.com> wrote:

Andrew Dunstan wrote:

On 05/20/2015 09:16 AM, Shulgin, Oleksandr wrote:

Attached is a patch against master to generalize the JSON-producing
functions in utils/adt/json.c and to provide a set of callbacks which

can

be overridden the same way that is already provided for *parsing*

JSON.

I'm not necessarily opposed to this, but it sure seems like a lot of
changes, and moderately invasive ones, to support something that could

be

done, at the cost of reparsing, with a simple loadable extension that I
could create in a few hours of programming.

But this seems like a pretty reasonable change to make, no? Doesn't the
total amount of code decrease after this patch? JSON stuff is pretty
new so some refactoring and generalization of what we have is to be
expected.

Yeah. Also, there have been a few previous gripes about this, for
example,
/messages/by-id/CAHbVmPzS+sVR+y-UgxjRq+XW4dqteVL-cOzc69zFFwmxjcKCxg@mail.gmail.com
.
As noted, I definitely prefer 'space free' by default for efficiency
reasons, but standardizing the output has definitely got to be a
reasonable goal.

Every JSON implementation I have ever used defaults to the minified
version of JSON (no whitespace) when printed.

Hashing of arrays seems to be an important issue: we'd rather make sure to
produce the same output in every code path. That would also mean: no
special logic to add the line feeds in json_agg either.

Is it reasonable to add this patch to CommitFest now?

--
Alex

#9Robert Haas
robertmhaas@gmail.com
In reply to: Shulgin, Oleksandr (#8)
Re: [PATCH] Generalized JSON output functions

On Tue, May 26, 2015 at 2:50 AM, Shulgin, Oleksandr
<oleksandr.shulgin@zalando.de> wrote:

Is it reasonable to add this patch to CommitFest now?

It's always reasonable to add a patch to the CommitFest if you would
like for it to be reviewed and avoid having it get forgotten about.
There seems to be some disagreement about whether we want this, but
don't let that stop you from adding it to the next CommitFest.

--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

#10Andrew Dunstan
andrew@dunslane.net
In reply to: Robert Haas (#9)
Re: [PATCH] Generalized JSON output functions

On 05/27/2015 02:37 PM, Robert Haas wrote:

On Tue, May 26, 2015 at 2:50 AM, Shulgin, Oleksandr
<oleksandr.shulgin@zalando.de> wrote:

Is it reasonable to add this patch to CommitFest now?

It's always reasonable to add a patch to the CommitFest if you would
like for it to be reviewed and avoid having it get forgotten about.
There seems to be some disagreement about whether we want this, but
don't let that stop you from adding it to the next CommitFest.

I'm not dead set against it either. When I have time I will take a
closer look.

cheers

andrew

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

#11Heikki Linnakangas
hlinnaka@iki.fi
In reply to: Andrew Dunstan (#10)
Re: [PATCH] Generalized JSON output functions

On 05/27/2015 09:51 PM, Andrew Dunstan wrote:

On 05/27/2015 02:37 PM, Robert Haas wrote:

On Tue, May 26, 2015 at 2:50 AM, Shulgin, Oleksandr
<oleksandr.shulgin@zalando.de> wrote:

Is it reasonable to add this patch to CommitFest now?

It's always reasonable to add a patch to the CommitFest if you would
like for it to be reviewed and avoid having it get forgotten about.
There seems to be some disagreement about whether we want this, but
don't let that stop you from adding it to the next CommitFest.

I'm not dead set against it either. When I have time I will take a
closer look.

Andrew, will you have the time to review this? Please add yourself as
reviewer in the commitfest app if you do.

My 2 cents is that I agree with your initial reaction: This is a lot of
infrastructure and generalizing things, for little benefit. Let's change
the current code where we generate JSON to be consistent with
whitespace, and call it a day.

- Heikki

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

#12Andrew Dunstan
andrew@dunslane.net
In reply to: Heikki Linnakangas (#11)
Re: [PATCH] Generalized JSON output functions

On 07/03/2015 06:27 AM, Heikki Linnakangas wrote:

On 05/27/2015 09:51 PM, Andrew Dunstan wrote:

On 05/27/2015 02:37 PM, Robert Haas wrote:

On Tue, May 26, 2015 at 2:50 AM, Shulgin, Oleksandr
<oleksandr.shulgin@zalando.de> wrote:

Is it reasonable to add this patch to CommitFest now?

It's always reasonable to add a patch to the CommitFest if you would
like for it to be reviewed and avoid having it get forgotten about.
There seems to be some disagreement about whether we want this, but
don't let that stop you from adding it to the next CommitFest.

I'm not dead set against it either. When I have time I will take a
closer look.

Andrew, will you have the time to review this? Please add yourself as
reviewer in the commitfest app if you do.

My 2 cents is that I agree with your initial reaction: This is a lot
of infrastructure and generalizing things, for little benefit. Let's
change the current code where we generate JSON to be consistent with
whitespace, and call it a day.

- Heikki

I'm somewhat on vacation for the next week or so, so I won't claim it,
but I'll try to make time to look at it. Other people (Merlin?) could
also provide reviews.

cheers

andrew

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

#13Pavel Stehule
pavel.stehule@gmail.com
In reply to: Heikki Linnakangas (#11)
Re: [PATCH] Generalized JSON output functions

2015-07-03 12:27 GMT+02:00 Heikki Linnakangas <hlinnaka@iki.fi>:

On 05/27/2015 09:51 PM, Andrew Dunstan wrote:

On 05/27/2015 02:37 PM, Robert Haas wrote:

On Tue, May 26, 2015 at 2:50 AM, Shulgin, Oleksandr
<oleksandr.shulgin@zalando.de> wrote:

Is it reasonable to add this patch to CommitFest now?

It's always reasonable to add a patch to the CommitFest if you would
like for it to be reviewed and avoid having it get forgotten about.
There seems to be some disagreement about whether we want this, but
don't let that stop you from adding it to the next CommitFest.

I'm not dead set against it either. When I have time I will take a
closer look.

Andrew, will you have the time to review this? Please add yourself as
reviewer in the commitfest app if you do.

My 2 cents is that I agree with your initial reaction: This is a lot of
infrastructure and generalizing things, for little benefit. Let's change
the current code where we generate JSON to be consistent with whitespace,
and call it a day.

I am thinking so it is not bad idea. This code can enforce uniform format,
and it can check if produced value is correct. It can be used in our code,
it can be used by extension's developers.

This patch is not small, but really new lines are not too much.

I'll do review today.

Regards

Pavel

Show quoted text

- Heikki

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

#14Pavel Stehule
pavel.stehule@gmail.com
In reply to: Pavel Stehule (#13)
Re: [PATCH] Generalized JSON output functions

Hi

I am sending review of this patch:

1. I reread a previous discussion and almost all are for this patch (me too)

2. I have to fix a typo in hstore_io.c function (update attached), other
(patching, regress tests) without problems

My objections:

1. comments - missing comment for some basic API, basic fields like
"key_scalar" and similar
2. why you did indirect call via JsonOutContext?

What is benefit

dst.value(&dst, (Datum) 0, JSONTYPE_NULL, InvalidOid, InvalidOid, false);

instead

json_out_value(&dst, ....)

? Is it necessary?

3. if it should be used everywhere, then in EXPLAIN statement too.

Regards

Pavel

2015-07-10 6:31 GMT+02:00 Pavel Stehule <pavel.stehule@gmail.com>:

Show quoted text

2015-07-03 12:27 GMT+02:00 Heikki Linnakangas <hlinnaka@iki.fi>:

On 05/27/2015 09:51 PM, Andrew Dunstan wrote:

On 05/27/2015 02:37 PM, Robert Haas wrote:

On Tue, May 26, 2015 at 2:50 AM, Shulgin, Oleksandr
<oleksandr.shulgin@zalando.de> wrote:

Is it reasonable to add this patch to CommitFest now?

It's always reasonable to add a patch to the CommitFest if you would
like for it to be reviewed and avoid having it get forgotten about.
There seems to be some disagreement about whether we want this, but
don't let that stop you from adding it to the next CommitFest.

I'm not dead set against it either. When I have time I will take a
closer look.

Andrew, will you have the time to review this? Please add yourself as
reviewer in the commitfest app if you do.

My 2 cents is that I agree with your initial reaction: This is a lot of
infrastructure and generalizing things, for little benefit. Let's change
the current code where we generate JSON to be consistent with whitespace,
and call it a day.

I am thinking so it is not bad idea. This code can enforce uniform
format, and it can check if produced value is correct. It can be used in
our code, it can be used by extension's developers.

This patch is not small, but really new lines are not too much.

I'll do review today.

Regards

Pavel

- Heikki

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

#15Pavel Stehule
pavel.stehule@gmail.com
In reply to: Pavel Stehule (#14)
1 attachment(s)
Re: [PATCH] Generalized JSON output functions

forgotten attachment

Regards

Pavel

2015-07-10 14:34 GMT+02:00 Pavel Stehule <pavel.stehule@gmail.com>:

Show quoted text

Hi

I am sending review of this patch:

1. I reread a previous discussion and almost all are for this patch (me
too)

2. I have to fix a typo in hstore_io.c function (update attached), other
(patching, regress tests) without problems

My objections:

1. comments - missing comment for some basic API, basic fields like
"key_scalar" and similar
2. why you did indirect call via JsonOutContext?

What is benefit

dst.value(&dst, (Datum) 0, JSONTYPE_NULL, InvalidOid, InvalidOid, false);

instead

json_out_value(&dst, ....)

? Is it necessary?

3. if it should be used everywhere, then in EXPLAIN statement too.

Regards

Pavel

2015-07-10 6:31 GMT+02:00 Pavel Stehule <pavel.stehule@gmail.com>:

2015-07-03 12:27 GMT+02:00 Heikki Linnakangas <hlinnaka@iki.fi>:

On 05/27/2015 09:51 PM, Andrew Dunstan wrote:

On 05/27/2015 02:37 PM, Robert Haas wrote:

On Tue, May 26, 2015 at 2:50 AM, Shulgin, Oleksandr
<oleksandr.shulgin@zalando.de> wrote:

Is it reasonable to add this patch to CommitFest now?

It's always reasonable to add a patch to the CommitFest if you would
like for it to be reviewed and avoid having it get forgotten about.
There seems to be some disagreement about whether we want this, but
don't let that stop you from adding it to the next CommitFest.

I'm not dead set against it either. When I have time I will take a
closer look.

Andrew, will you have the time to review this? Please add yourself as
reviewer in the commitfest app if you do.

My 2 cents is that I agree with your initial reaction: This is a lot of
infrastructure and generalizing things, for little benefit. Let's change
the current code where we generate JSON to be consistent with whitespace,
and call it a day.

I am thinking so it is not bad idea. This code can enforce uniform
format, and it can check if produced value is correct. It can be used in
our code, it can be used by extension's developers.

This patch is not small, but really new lines are not too much.

I'll do review today.

Regards

Pavel

- Heikki

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Attachments:

json-output-generalized-v1-2.patchtext/x-patch; charset=US-ASCII; name=json-output-generalized-v1-2.patchDownload
diff --git a/contrib/hstore/hstore_io.c b/contrib/hstore/hstore_io.c
new file mode 100644
index 7d89867..0ca223f
*** a/contrib/hstore/hstore_io.c
--- b/contrib/hstore/hstore_io.c
*************** hstore_to_json_loose(PG_FUNCTION_ARGS)
*** 1241,1286 ****
  	int			count = HS_COUNT(in);
  	char	   *base = STRPTR(in);
  	HEntry	   *entries = ARRPTR(in);
! 	StringInfoData tmp,
! 				dst;
  
  	if (count == 0)
  		PG_RETURN_TEXT_P(cstring_to_text_with_len("{}", 2));
  
  	initStringInfo(&tmp);
! 	initStringInfo(&dst);
! 
! 	appendStringInfoChar(&dst, '{');
  
  	for (i = 0; i < count; i++)
  	{
  		resetStringInfo(&tmp);
  		appendBinaryStringInfo(&tmp, HS_KEY(entries, base, i), HS_KEYLEN(entries, i));
! 		escape_json(&dst, tmp.data);
! 		appendStringInfoString(&dst, ": ");
  		if (HS_VALISNULL(entries, i))
! 			appendStringInfoString(&dst, "null");
  		/* guess that values of 't' or 'f' are booleans */
  		else if (HS_VALLEN(entries, i) == 1 && *(HS_VAL(entries, base, i)) == 't')
! 			appendStringInfoString(&dst, "true");
  		else if (HS_VALLEN(entries, i) == 1 && *(HS_VAL(entries, base, i)) == 'f')
! 			appendStringInfoString(&dst, "false");
  		else
  		{
  			resetStringInfo(&tmp);
  			appendBinaryStringInfo(&tmp, HS_VAL(entries, base, i), HS_VALLEN(entries, i));
  			if (IsValidJsonNumber(tmp.data, tmp.len))
! 				appendBinaryStringInfo(&dst, tmp.data, tmp.len);
  			else
! 				escape_json(&dst, tmp.data);
  		}
- 
- 		if (i + 1 != count)
- 			appendStringInfoString(&dst, ", ");
  	}
- 	appendStringInfoChar(&dst, '}');
  
! 	PG_RETURN_TEXT_P(cstring_to_text(dst.data));
  }
  
  PG_FUNCTION_INFO_V1(hstore_to_json);
--- 1241,1289 ----
  	int			count = HS_COUNT(in);
  	char	   *base = STRPTR(in);
  	HEntry	   *entries = ARRPTR(in);
! 	StringInfoData	tmp;
! 	JsonOutContext	dst;
  
  	if (count == 0)
  		PG_RETURN_TEXT_P(cstring_to_text_with_len("{}", 2));
  
  	initStringInfo(&tmp);
! 	json_out_init_context(&dst, JSON_OUT_USE_SPACES);
! 	dst.object_start(&dst);
  
  	for (i = 0; i < count; i++)
  	{
  		resetStringInfo(&tmp);
  		appendBinaryStringInfo(&tmp, HS_KEY(entries, base, i), HS_KEYLEN(entries, i));
! 		json_out_cstring(&dst, tmp.data, true);
! 
  		if (HS_VALISNULL(entries, i))
! 			dst.value(&dst, (Datum) 0, JSONTYPE_NULL, InvalidOid, InvalidOid, false);
! 
  		/* guess that values of 't' or 'f' are booleans */
  		else if (HS_VALLEN(entries, i) == 1 && *(HS_VAL(entries, base, i)) == 't')
! 			dst.value(&dst, BoolGetDatum(true), JSONTYPE_BOOL,
! 					  InvalidOid, InvalidOid, false);
! 
  		else if (HS_VALLEN(entries, i) == 1 && *(HS_VAL(entries, base, i)) == 'f')
! 			dst.value(&dst, BoolGetDatum(false), JSONTYPE_BOOL,
! 					  InvalidOid, InvalidOid, false);
  		else
  		{
  			resetStringInfo(&tmp);
  			appendBinaryStringInfo(&tmp, HS_VAL(entries, base, i), HS_VALLEN(entries, i));
+ 
+ 			/* this is a bit of a hack, but strictly it is not incorrect */
  			if (IsValidJsonNumber(tmp.data, tmp.len))
! 				dst.value(&dst, CStringGetDatum(tmp.data), JSONTYPE_JSON,
! 						  CSTRINGOID, 2293 /* cstring_out */, false);
  			else
! 				json_out_cstring(&dst, tmp.data, false);
  		}
  	}
  
! 	dst.object_end(&dst);
! 	PG_RETURN_TEXT_P(cstring_to_text_with_len(dst.result.data, dst.result.len));
  }
  
  PG_FUNCTION_INFO_V1(hstore_to_json);
*************** hstore_to_json(PG_FUNCTION_ARGS)
*** 1292,1329 ****
  	int			count = HS_COUNT(in);
  	char	   *base = STRPTR(in);
  	HEntry	   *entries = ARRPTR(in);
! 	StringInfoData tmp,
! 				dst;
  
  	if (count == 0)
  		PG_RETURN_TEXT_P(cstring_to_text_with_len("{}", 2));
  
  	initStringInfo(&tmp);
! 	initStringInfo(&dst);
! 
! 	appendStringInfoChar(&dst, '{');
  
  	for (i = 0; i < count; i++)
  	{
  		resetStringInfo(&tmp);
  		appendBinaryStringInfo(&tmp, HS_KEY(entries, base, i), HS_KEYLEN(entries, i));
! 		escape_json(&dst, tmp.data);
! 		appendStringInfoString(&dst, ": ");
  		if (HS_VALISNULL(entries, i))
! 			appendStringInfoString(&dst, "null");
  		else
  		{
  			resetStringInfo(&tmp);
  			appendBinaryStringInfo(&tmp, HS_VAL(entries, base, i), HS_VALLEN(entries, i));
! 			escape_json(&dst, tmp.data);
  		}
- 
- 		if (i + 1 != count)
- 			appendStringInfoString(&dst, ", ");
  	}
- 	appendStringInfoChar(&dst, '}');
  
! 	PG_RETURN_TEXT_P(cstring_to_text(dst.data));
  }
  
  PG_FUNCTION_INFO_V1(hstore_to_jsonb);
--- 1295,1328 ----
  	int			count = HS_COUNT(in);
  	char	   *base = STRPTR(in);
  	HEntry	   *entries = ARRPTR(in);
! 	StringInfoData	tmp;
! 	JsonOutContext	dst;
  
  	if (count == 0)
  		PG_RETURN_TEXT_P(cstring_to_text_with_len("{}", 2));
  
  	initStringInfo(&tmp);
! 	json_out_init_context(&dst, JSON_OUT_USE_SPACES);
! 	dst.object_start(&dst);
  
  	for (i = 0; i < count; i++)
  	{
  		resetStringInfo(&tmp);
  		appendBinaryStringInfo(&tmp, HS_KEY(entries, base, i), HS_KEYLEN(entries, i));
! 		json_out_cstring(&dst, tmp.data, true);
! 
  		if (HS_VALISNULL(entries, i))
! 			dst.value(&dst, (Datum) 0, JSONTYPE_NULL, InvalidOid, InvalidOid, false);
  		else
  		{
  			resetStringInfo(&tmp);
  			appendBinaryStringInfo(&tmp, HS_VAL(entries, base, i), HS_VALLEN(entries, i));
! 			json_out_cstring(&dst, tmp.data, false);
  		}
  	}
  
! 	dst.object_end(&dst);
! 	PG_RETURN_TEXT_P(cstring_to_text_with_len(dst.result.data, dst.result.len));
  }
  
  PG_FUNCTION_INFO_V1(hstore_to_jsonb);
diff --git a/src/backend/utils/adt/json.c b/src/backend/utils/adt/json.c
new file mode 100644
index 26d3843..1ed9deb
*** a/src/backend/utils/adt/json.c
--- b/src/backend/utils/adt/json.c
*************** typedef enum					/* contexts of JSON par
*** 53,73 ****
  	JSON_PARSE_END				/* saw the end of a document, expect nothing */
  } JsonParseContext;
  
- typedef enum					/* type categories for datum_to_json */
- {
- 	JSONTYPE_NULL,				/* null, so we didn't bother to identify */
- 	JSONTYPE_BOOL,				/* boolean (built-in types only) */
- 	JSONTYPE_NUMERIC,			/* numeric (ditto) */
- 	JSONTYPE_DATE,				/* we use special formatting for datetimes */
- 	JSONTYPE_TIMESTAMP,
- 	JSONTYPE_TIMESTAMPTZ,
- 	JSONTYPE_JSON,				/* JSON itself (and JSONB) */
- 	JSONTYPE_ARRAY,				/* array */
- 	JSONTYPE_COMPOSITE,			/* composite */
- 	JSONTYPE_CAST,				/* something with an explicit cast to JSON */
- 	JSONTYPE_OTHER				/* all else */
- } JsonTypeCategory;
- 
  static inline void json_lex(JsonLexContext *lex);
  static inline void json_lex_string(JsonLexContext *lex);
  static inline void json_lex_number(JsonLexContext *lex, char *s, bool *num_err);
--- 53,58 ----
*************** static void report_parse_error(JsonParse
*** 80,102 ****
  static void report_invalid_token(JsonLexContext *lex);
  static int	report_json_context(JsonLexContext *lex);
  static char *extract_mb_char(char *s);
! static void composite_to_json(Datum composite, StringInfo result,
! 				  bool use_line_feeds);
! static void array_dim_to_json(StringInfo result, int dim, int ndims, int *dims,
  				  Datum *vals, bool *nulls, int *valcount,
! 				  JsonTypeCategory tcategory, Oid outfuncoid,
! 				  bool use_line_feeds);
! static void array_to_json_internal(Datum array, StringInfo result,
! 					   bool use_line_feeds);
  static void json_categorize_type(Oid typoid,
  					 JsonTypeCategory *tcategory,
  					 Oid *outfuncoid);
! static void datum_to_json(Datum val, bool is_null, StringInfo result,
! 			  JsonTypeCategory tcategory, Oid outfuncoid,
! 			  bool key_scalar);
! static void add_json(Datum val, bool is_null, StringInfo result,
  		 Oid val_type, bool key_scalar);
- static text *catenate_stringinfo_string(StringInfo buffer, const char *addon);
  
  /* the null action object used for pure validation */
  static JsonSemAction nullSemAction =
--- 65,80 ----
  static void report_invalid_token(JsonLexContext *lex);
  static int	report_json_context(JsonLexContext *lex);
  static char *extract_mb_char(char *s);
! 
! static void array_dim_to_json(JsonOutContext *out, int dim, int ndims, int *dims,
  				  Datum *vals, bool *nulls, int *valcount,
! 				  JsonTypeCategory tcategory, Oid elemtypoid, Oid outfuncoid);
! static void array_to_json_internal(Datum array, JsonOutContext *out);
  static void json_categorize_type(Oid typoid,
  					 JsonTypeCategory *tcategory,
  					 Oid *outfuncoid);
! static void add_json(Datum val, bool is_null, JsonOutContext *out,
  		 Oid val_type, bool key_scalar);
  
  /* the null action object used for pure validation */
  static JsonSemAction nullSemAction =
*************** json_categorize_type(Oid typoid,
*** 1377,1399 ****
   * If key_scalar is true, the value is being printed as a key, so insist
   * it's of an acceptable type, and force it to be quoted.
   */
! static void
! datum_to_json(Datum val, bool is_null, StringInfo result,
! 			  JsonTypeCategory tcategory, Oid outfuncoid,
  			  bool key_scalar)
  {
- 	char	   *outputstr;
- 	text	   *jsontext;
- 
  	/* callers are expected to ensure that null keys are not passed in */
  	Assert(!(key_scalar && is_null));
  
- 	if (is_null)
- 	{
- 		appendStringInfoString(result, "null");
- 		return;
- 	}
- 
  	if (key_scalar &&
  		(tcategory == JSONTYPE_ARRAY ||
  		 tcategory == JSONTYPE_COMPOSITE ||
--- 1355,1368 ----
   * If key_scalar is true, the value is being printed as a key, so insist
   * it's of an acceptable type, and force it to be quoted.
   */
! void
! datum_to_json(Datum val, bool is_null, JsonOutContext *out,
! 			  JsonTypeCategory tcategory, Oid typoid, Oid outfuncoid,
  			  bool key_scalar)
  {
  	/* callers are expected to ensure that null keys are not passed in */
  	Assert(!(key_scalar && is_null));
  
  	if (key_scalar &&
  		(tcategory == JSONTYPE_ARRAY ||
  		 tcategory == JSONTYPE_COMPOSITE ||
*************** datum_to_json(Datum val, bool is_null, S
*** 1406,1415 ****
  	switch (tcategory)
  	{
  		case JSONTYPE_ARRAY:
! 			array_to_json_internal(val, result, false);
  			break;
  		case JSONTYPE_COMPOSITE:
! 			composite_to_json(val, result, false);
  			break;
  		case JSONTYPE_BOOL:
  			outputstr = DatumGetBool(val) ? "true" : "false";
--- 1375,1500 ----
  	switch (tcategory)
  	{
  		case JSONTYPE_ARRAY:
! 			array_to_json_internal(val, out);
  			break;
  		case JSONTYPE_COMPOSITE:
! 			composite_to_json(val, out);
! 			break;
! 		default:
! 			out->value(out, val, tcategory, typoid, outfuncoid, key_scalar);
! 			break;
! 	}
! }
! 
! void
! json_out_init_context(JsonOutContext *out, int flags)
! {
! 	out->object_start = json_out_object_start;
! 	out->object_end = json_out_object_end;
! 	out->array_start = json_out_array_start;
! 	out->array_end = json_out_array_end;
! 	out->before_value = json_out_before_value;
! 	out->value = json_out_value;
! 	out->after_value = json_out_after_value;
! 
! 	initStringInfo(&out->result);
! 	out->flags = flags;
! 	out->need_comma = false;
! 	out->depth = 0;
! 
! 	out->agg_tcategory = JSONTYPE_OTHER;
! 	out->agg_outfuncoid = InvalidOid;
! }
! 
! void
! json_out_before_value(JsonOutContext *out)
! {
! 	if (out->need_comma)
! 	{
! 		out->need_comma = false;
! 		appendStringInfoChar(&out->result, ',');
! 
! 		/* don't get into all the prettiness deep in the object structure */
! 		if (out->depth == 1)
! 		{
! 			if ((out->flags & JSON_OUT_USE_SPACES) != 0)
! 				appendStringInfoChar(&out->result, ' ');
! 
! 			if ((out->flags & JSON_OUT_USE_LINE_FEEDS) != 0)
! 				appendStringInfoString(&out->result, "\n ");
! 		}
! 	}
! }
! 
! void
! json_out_after_value(JsonOutContext *out, bool key_scalar)
! {
! 	if (key_scalar)
! 	{
! 		appendStringInfoChar(&out->result, ':');
! 
! 		if (out->depth == 1 && (out->flags & JSON_OUT_USE_SPACES) != 0)
! 			appendStringInfoChar(&out->result, ' ');
! 
! 		out->need_comma = false;
! 	}
! 	else
! 		out->need_comma = true;
! }
! 
! void
! json_out_object_start(JsonOutContext *out)
! {
! 	out->before_value(out);
! 
! 	appendStringInfoChar(&out->result, '{');
! 	out->depth++;
! }
! 
! void
! json_out_object_end(JsonOutContext *out)
! {
! 	appendStringInfoChar(&out->result, '}');
! 
! 	out->after_value(out, false);
! 	out->depth--;
! }
! 
! void
! json_out_array_start(JsonOutContext *out)
! {
! 	out->before_value(out);
! 
! 	appendStringInfoChar(&out->result, '[');
! 	out->depth++;
! }
! 
! void
! json_out_array_end(JsonOutContext *out)
! {
! 	appendStringInfoChar(&out->result, ']');
! 
! 	out->after_value(out, false);
! 	out->depth--;
! }
! 
! void
! json_out_value(JsonOutContext *out, Datum val, JsonTypeCategory tcategory,
! 			Oid typoid, Oid outfuncoid, bool key_scalar)
! {
! 	char	   *outputstr;
! 	text	   *jsontext;
! 	StringInfo		result = &out->result;
! 
! 	Assert(!(tcategory == JSONTYPE_ARRAY || tcategory == JSONTYPE_COMPOSITE));
! 
! 	/* check if there was an element before this one and add a separator */
! 	out->before_value(out);
! 
! 	switch (tcategory)
! 	{
! 		case JSONTYPE_NULL:
! 			appendStringInfoString(result, "null");
  			break;
  		case JSONTYPE_BOOL:
  			outputstr = DatumGetBool(val) ? "true" : "false";
*************** datum_to_json(Datum val, bool is_null, S
*** 1520,1530 ****
  			pfree(jsontext);
  			break;
  		default:
! 			outputstr = OidOutputFunctionCall(outfuncoid, val);
! 			escape_json(result, outputstr);
! 			pfree(outputstr);
  			break;
  	}
  }
  
  /*
--- 1605,1630 ----
  			pfree(jsontext);
  			break;
  		default:
! 			if (typoid == CSTRINGOID)
! 				escape_json(result, DatumGetCString(val));
! 			else
! 			{
! 				outputstr = OidOutputFunctionCall(outfuncoid, val);
! 				escape_json(result, outputstr);
! 				pfree(outputstr);
! 			}
  			break;
  	}
+ 
+ 	/* output key-value separator if needed and set need_comma accordingly */
+ 	out->after_value(out, key_scalar);
+ }
+ 
+ void
+ json_out_cstring(JsonOutContext *out, const char *str, bool key_scalar)
+ {
+ 	out->value(out, CStringGetDatum(str), JSONTYPE_OTHER,
+ 			   CSTRINGOID, 2293 /* cstring_out */, key_scalar);
  }
  
  /*
*************** datum_to_json(Datum val, bool is_null, S
*** 1533,1581 ****
   * ourselves recursively to process the next dimension.
   */
  static void
! array_dim_to_json(StringInfo result, int dim, int ndims, int *dims, Datum *vals,
  				  bool *nulls, int *valcount, JsonTypeCategory tcategory,
! 				  Oid outfuncoid, bool use_line_feeds)
  {
  	int			i;
- 	const char *sep;
  
  	Assert(dim < ndims);
  
! 	sep = use_line_feeds ? ",\n " : ",";
! 
! 	appendStringInfoChar(result, '[');
  
  	for (i = 1; i <= dims[dim]; i++)
  	{
- 		if (i > 1)
- 			appendStringInfoString(result, sep);
- 
  		if (dim + 1 == ndims)
  		{
! 			datum_to_json(vals[*valcount], nulls[*valcount], result, tcategory,
! 						  outfuncoid, false);
  			(*valcount)++;
  		}
  		else
! 		{
! 			/*
! 			 * Do we want line feeds on inner dimensions of arrays? For now
! 			 * we'll say no.
! 			 */
! 			array_dim_to_json(result, dim + 1, ndims, dims, vals, nulls,
! 							  valcount, tcategory, outfuncoid, false);
! 		}
  	}
  
! 	appendStringInfoChar(result, ']');
  }
  
  /*
   * Turn an array into JSON.
   */
  static void
! array_to_json_internal(Datum array, StringInfo result, bool use_line_feeds)
  {
  	ArrayType  *v = DatumGetArrayTypeP(array);
  	Oid			element_type = ARR_ELEMTYPE(v);
--- 1633,1669 ----
   * ourselves recursively to process the next dimension.
   */
  static void
! array_dim_to_json(JsonOutContext *out, int dim, int ndims, int *dims, Datum *vals,
  				  bool *nulls, int *valcount, JsonTypeCategory tcategory,
! 				  Oid elemtypoid, Oid outfuncoid)
  {
  	int			i;
  
  	Assert(dim < ndims);
  
! 	out->array_start(out);
  
  	for (i = 1; i <= dims[dim]; i++)
  	{
  		if (dim + 1 == ndims)
  		{
! 			datum_to_json(vals[*valcount], nulls[*valcount], out, tcategory,
! 						  elemtypoid, outfuncoid, false);
  			(*valcount)++;
  		}
  		else
! 			array_dim_to_json(out, dim + 1, ndims, dims, vals, nulls,
! 							  valcount, tcategory, elemtypoid, outfuncoid);
  	}
  
! 	out->array_end(out);
  }
  
  /*
   * Turn an array into JSON.
   */
  static void
! array_to_json_internal(Datum array, JsonOutContext *out)
  {
  	ArrayType  *v = DatumGetArrayTypeP(array);
  	Oid			element_type = ARR_ELEMTYPE(v);
*************** array_to_json_internal(Datum array, Stri
*** 1597,1603 ****
  
  	if (nitems <= 0)
  	{
! 		appendStringInfoString(result, "[]");
  		return;
  	}
  
--- 1685,1692 ----
  
  	if (nitems <= 0)
  	{
! 		out->array_start(out);
! 		out->array_end(out);
  		return;
  	}
  
*************** array_to_json_internal(Datum array, Stri
*** 1611,1618 ****
  					  typalign, &elements, &nulls,
  					  &nitems);
  
! 	array_dim_to_json(result, 0, ndim, dim, elements, nulls, &count, tcategory,
! 					  outfuncoid, use_line_feeds);
  
  	pfree(elements);
  	pfree(nulls);
--- 1700,1707 ----
  					  typalign, &elements, &nulls,
  					  &nitems);
  
! 	array_dim_to_json(out, 0, ndim, dim, elements, nulls, &count, tcategory,
! 					  element_type, outfuncoid);
  
  	pfree(elements);
  	pfree(nulls);
*************** array_to_json_internal(Datum array, Stri
*** 1621,1628 ****
  /*
   * Turn a composite / record into JSON.
   */
! static void
! composite_to_json(Datum composite, StringInfo result, bool use_line_feeds)
  {
  	HeapTupleHeader td;
  	Oid			tupType;
--- 1710,1717 ----
  /*
   * Turn a composite / record into JSON.
   */
! void
! composite_to_json(Datum composite, JsonOutContext *out)
  {
  	HeapTupleHeader td;
  	Oid			tupType;
*************** composite_to_json(Datum composite, Strin
*** 1631,1640 ****
  	HeapTupleData tmptup,
  			   *tuple;
  	int			i;
- 	bool		needsep = false;
- 	const char *sep;
- 
- 	sep = use_line_feeds ? ",\n " : ",";
  
  	td = DatumGetHeapTupleHeader(composite);
  
--- 1720,1725 ----
*************** composite_to_json(Datum composite, Strin
*** 1648,1654 ****
  	tmptup.t_data = td;
  	tuple = &tmptup;
  
! 	appendStringInfoChar(result, '{');
  
  	for (i = 0; i < tupdesc->natts; i++)
  	{
--- 1733,1739 ----
  	tmptup.t_data = td;
  	tuple = &tmptup;
  
! 	out->object_start(out);
  
  	for (i = 0; i < tupdesc->natts; i++)
  	{
*************** composite_to_json(Datum composite, Strin
*** 1661,1676 ****
  		if (tupdesc->attrs[i]->attisdropped)
  			continue;
  
- 		if (needsep)
- 			appendStringInfoString(result, sep);
- 		needsep = true;
- 
  		attname = NameStr(tupdesc->attrs[i]->attname);
! 		escape_json(result, attname);
! 		appendStringInfoChar(result, ':');
  
  		val = heap_getattr(tuple, i + 1, tupdesc, &isnull);
- 
  		if (isnull)
  		{
  			tcategory = JSONTYPE_NULL;
--- 1746,1755 ----
  		if (tupdesc->attrs[i]->attisdropped)
  			continue;
  
  		attname = NameStr(tupdesc->attrs[i]->attname);
! 		json_out_cstring(out, attname, true);
  
  		val = heap_getattr(tuple, i + 1, tupdesc, &isnull);
  		if (isnull)
  		{
  			tcategory = JSONTYPE_NULL;
*************** composite_to_json(Datum composite, Strin
*** 1680,1689 ****
  			json_categorize_type(tupdesc->attrs[i]->atttypid,
  								 &tcategory, &outfuncoid);
  
! 		datum_to_json(val, isnull, result, tcategory, outfuncoid, false);
  	}
  
! 	appendStringInfoChar(result, '}');
  	ReleaseTupleDesc(tupdesc);
  }
  
--- 1759,1770 ----
  			json_categorize_type(tupdesc->attrs[i]->atttypid,
  								 &tcategory, &outfuncoid);
  
! 		datum_to_json(val, isnull, out, tcategory,
! 					  tupdesc->attrs[i]->atttypid, outfuncoid, false);
  	}
  
! 	out->object_end(out);
! 
  	ReleaseTupleDesc(tupdesc);
  }
  
*************** composite_to_json(Datum composite, Strin
*** 1695,1701 ****
   * lookups only once.
   */
  static void
! add_json(Datum val, bool is_null, StringInfo result,
  		 Oid val_type, bool key_scalar)
  {
  	JsonTypeCategory tcategory;
--- 1776,1782 ----
   * lookups only once.
   */
  static void
! add_json(Datum val, bool is_null, JsonOutContext *out,
  		 Oid val_type, bool key_scalar)
  {
  	JsonTypeCategory tcategory;
*************** add_json(Datum val, bool is_null, String
*** 1715,1721 ****
  		json_categorize_type(val_type,
  							 &tcategory, &outfuncoid);
  
! 	datum_to_json(val, is_null, result, tcategory, outfuncoid, key_scalar);
  }
  
  /*
--- 1796,1802 ----
  		json_categorize_type(val_type,
  							 &tcategory, &outfuncoid);
  
! 	datum_to_json(val, is_null, out, tcategory, val_type, outfuncoid, key_scalar);
  }
  
  /*
*************** extern Datum
*** 1725,1737 ****
  array_to_json(PG_FUNCTION_ARGS)
  {
  	Datum		array = PG_GETARG_DATUM(0);
! 	StringInfo	result;
! 
! 	result = makeStringInfo();
  
! 	array_to_json_internal(array, result, false);
  
! 	PG_RETURN_TEXT_P(cstring_to_text_with_len(result->data, result->len));
  }
  
  /*
--- 1806,1817 ----
  array_to_json(PG_FUNCTION_ARGS)
  {
  	Datum		array = PG_GETARG_DATUM(0);
! 	JsonOutContext	out;
  
! 	json_out_init_context(&out, 0);
! 	array_to_json_internal(array, &out);
  
! 	PG_RETURN_TEXT_P(cstring_to_text_with_len(out.result.data, out.result.len));
  }
  
  /*
*************** array_to_json_pretty(PG_FUNCTION_ARGS)
*** 1742,1754 ****
  {
  	Datum		array = PG_GETARG_DATUM(0);
  	bool		use_line_feeds = PG_GETARG_BOOL(1);
! 	StringInfo	result;
! 
! 	result = makeStringInfo();
  
! 	array_to_json_internal(array, result, use_line_feeds);
  
! 	PG_RETURN_TEXT_P(cstring_to_text_with_len(result->data, result->len));
  }
  
  /*
--- 1822,1833 ----
  {
  	Datum		array = PG_GETARG_DATUM(0);
  	bool		use_line_feeds = PG_GETARG_BOOL(1);
! 	JsonOutContext	out;
  
! 	json_out_init_context(&out, use_line_feeds ? JSON_OUT_USE_LINE_FEEDS : 0);
! 	array_to_json_internal(array, &out);
  
! 	PG_RETURN_TEXT_P(cstring_to_text_with_len(out.result.data, out.result.len));
  }
  
  /*
*************** extern Datum
*** 1758,1770 ****
  row_to_json(PG_FUNCTION_ARGS)
  {
  	Datum		array = PG_GETARG_DATUM(0);
! 	StringInfo	result;
! 
! 	result = makeStringInfo();
  
! 	composite_to_json(array, result, false);
  
! 	PG_RETURN_TEXT_P(cstring_to_text_with_len(result->data, result->len));
  }
  
  /*
--- 1837,1848 ----
  row_to_json(PG_FUNCTION_ARGS)
  {
  	Datum		array = PG_GETARG_DATUM(0);
! 	JsonOutContext	out;
  
! 	json_out_init_context(&out, 0);
! 	composite_to_json(array, &out);
  
! 	PG_RETURN_TEXT_P(cstring_to_text_with_len(out.result.data, out.result.len));
  }
  
  /*
*************** row_to_json_pretty(PG_FUNCTION_ARGS)
*** 1775,1787 ****
  {
  	Datum		array = PG_GETARG_DATUM(0);
  	bool		use_line_feeds = PG_GETARG_BOOL(1);
! 	StringInfo	result;
! 
! 	result = makeStringInfo();
  
! 	composite_to_json(array, result, use_line_feeds);
  
! 	PG_RETURN_TEXT_P(cstring_to_text_with_len(result->data, result->len));
  }
  
  /*
--- 1853,1864 ----
  {
  	Datum		array = PG_GETARG_DATUM(0);
  	bool		use_line_feeds = PG_GETARG_BOOL(1);
! 	JsonOutContext	out;
  
! 	json_out_init_context(&out, use_line_feeds ? JSON_OUT_USE_LINE_FEEDS : 0);
! 	composite_to_json(array, &out);
  
! 	PG_RETURN_TEXT_P(cstring_to_text_with_len(out.result.data, out.result.len));
  }
  
  /*
*************** to_json(PG_FUNCTION_ARGS)
*** 1792,1800 ****
  {
  	Datum		val = PG_GETARG_DATUM(0);
  	Oid			val_type = get_fn_expr_argtype(fcinfo->flinfo, 0);
- 	StringInfo	result;
  	JsonTypeCategory tcategory;
  	Oid			outfuncoid;
  
  	if (val_type == InvalidOid)
  		ereport(ERROR,
--- 1869,1879 ----
  {
  	Datum		val = PG_GETARG_DATUM(0);
  	Oid			val_type = get_fn_expr_argtype(fcinfo->flinfo, 0);
  	JsonTypeCategory tcategory;
  	Oid			outfuncoid;
+ 	JsonOutContext	out;
+ 
+ 	json_out_init_context(&out, 0);
  
  	if (val_type == InvalidOid)
  		ereport(ERROR,
*************** to_json(PG_FUNCTION_ARGS)
*** 1804,1814 ****
  	json_categorize_type(val_type,
  						 &tcategory, &outfuncoid);
  
! 	result = makeStringInfo();
! 
! 	datum_to_json(val, false, result, tcategory, outfuncoid, false);
  
! 	PG_RETURN_TEXT_P(cstring_to_text_with_len(result->data, result->len));
  }
  
  /*
--- 1883,1891 ----
  	json_categorize_type(val_type,
  						 &tcategory, &outfuncoid);
  
! 	datum_to_json(val, false, &out, tcategory, val_type, outfuncoid, false);
  
! 	PG_RETURN_TEXT_P(cstring_to_text_with_len(out.result.data, out.result.len));
  }
  
  /*
*************** json_agg_transfn(PG_FUNCTION_ARGS)
*** 1822,1831 ****
  	Oid			val_type = get_fn_expr_argtype(fcinfo->flinfo, 1);
  	MemoryContext aggcontext,
  				oldcontext;
! 	StringInfo	state;
  	Datum		val;
- 	JsonTypeCategory tcategory;
- 	Oid			outfuncoid;
  
  	if (val_type == InvalidOid)
  		ereport(ERROR,
--- 1899,1906 ----
  	Oid			val_type = get_fn_expr_argtype(fcinfo->flinfo, 1);
  	MemoryContext aggcontext,
  				oldcontext;
! 	JsonOutContext	*out = NULL;
  	Datum		val;
  
  	if (val_type == InvalidOid)
  		ereport(ERROR,
*************** json_agg_transfn(PG_FUNCTION_ARGS)
*** 1847,1891 ****
  		 * use the right context to enlarge the object if necessary.
  		 */
  		oldcontext = MemoryContextSwitchTo(aggcontext);
! 		state = makeStringInfo();
  		MemoryContextSwitchTo(oldcontext);
  
! 		appendStringInfoChar(state, '[');
  	}
  	else
! 	{
! 		state = (StringInfo) PG_GETARG_POINTER(0);
! 		appendStringInfoString(state, ", ");
! 	}
  
  	/* fast path for NULLs */
  	if (PG_ARGISNULL(1))
  	{
! 		datum_to_json((Datum) 0, true, state, JSONTYPE_NULL, InvalidOid, false);
! 		PG_RETURN_POINTER(state);
  	}
  
  	val = PG_GETARG_DATUM(1);
  
! 	/* XXX we do this every time?? */
! 	json_categorize_type(val_type,
! 						 &tcategory, &outfuncoid);
! 
! 	/* add some whitespace if structured type and not first item */
! 	if (!PG_ARGISNULL(0) &&
! 		(tcategory == JSONTYPE_ARRAY || tcategory == JSONTYPE_COMPOSITE))
! 	{
! 		appendStringInfoString(state, "\n ");
! 	}
  
! 	datum_to_json(val, false, state, tcategory, outfuncoid, false);
  
  	/*
  	 * The transition type for array_agg() is declared to be "internal", which
  	 * is a pass-by-value type the same size as a pointer.  So we can safely
! 	 * pass the ArrayBuildState pointer through nodeAgg.c's machinations.
  	 */
! 	PG_RETURN_POINTER(state);
  }
  
  /*
--- 1922,1959 ----
  		 * use the right context to enlarge the object if necessary.
  		 */
  		oldcontext = MemoryContextSwitchTo(aggcontext);
! 		out = palloc(sizeof(JsonOutContext));
! 		json_out_init_context(out, JSON_OUT_USE_LINE_FEEDS);
  		MemoryContextSwitchTo(oldcontext);
  
! 		out->array_start(out);
  	}
  	else
! 		out = (JsonOutContext *) PG_GETARG_POINTER(0);
  
  	/* fast path for NULLs */
  	if (PG_ARGISNULL(1))
  	{
! 		datum_to_json((Datum) 0, true, out,
! 					  JSONTYPE_NULL, InvalidOid, InvalidOid, false);
! 		PG_RETURN_POINTER(out);
  	}
  
  	val = PG_GETARG_DATUM(1);
  
! 	if (out->agg_outfuncoid == InvalidOid)
! 		json_categorize_type(val_type,
! 							 &out->agg_tcategory, &out->agg_outfuncoid);
  
! 	datum_to_json(val, false, out, out->agg_tcategory,
! 				  val_type, out->agg_outfuncoid, false);
  
  	/*
  	 * The transition type for array_agg() is declared to be "internal", which
  	 * is a pass-by-value type the same size as a pointer.  So we can safely
! 	 * pass the ArrayBuildOut pointer through nodeAgg.c's machinations.
  	 */
! 	PG_RETURN_POINTER(out);
  }
  
  /*
*************** json_agg_transfn(PG_FUNCTION_ARGS)
*** 1894,1912 ****
  Datum
  json_agg_finalfn(PG_FUNCTION_ARGS)
  {
! 	StringInfo	state;
  
  	/* cannot be called directly because of internal-type argument */
  	Assert(AggCheckCallContext(fcinfo, NULL));
  
! 	state = PG_ARGISNULL(0) ? NULL : (StringInfo) PG_GETARG_POINTER(0);
  
  	/* NULL result for no rows in, as is standard with aggregates */
! 	if (state == NULL)
  		PG_RETURN_NULL();
  
! 	/* Else return state with appropriate array terminator added */
! 	PG_RETURN_TEXT_P(catenate_stringinfo_string(state, "]"));
  }
  
  /*
--- 1962,1981 ----
  Datum
  json_agg_finalfn(PG_FUNCTION_ARGS)
  {
! 	JsonOutContext	*out;
  
  	/* cannot be called directly because of internal-type argument */
  	Assert(AggCheckCallContext(fcinfo, NULL));
  
! 	out = PG_ARGISNULL(0) ? NULL : (JsonOutContext *) PG_GETARG_POINTER(0);
  
  	/* NULL result for no rows in, as is standard with aggregates */
! 	if (out == NULL)
  		PG_RETURN_NULL();
  
! 	out->array_end(out);
! 
! 	PG_RETURN_TEXT_P(cstring_to_text_with_len(out->result.data, out->result.len));
  }
  
  /*
*************** json_object_agg_transfn(PG_FUNCTION_ARGS
*** 1920,1926 ****
  	Oid			val_type;
  	MemoryContext aggcontext,
  				oldcontext;
! 	StringInfo	state;
  	Datum		arg;
  
  	if (!AggCheckCallContext(fcinfo, &aggcontext))
--- 1989,1995 ----
  	Oid			val_type;
  	MemoryContext aggcontext,
  				oldcontext;
! 	JsonOutContext	*out = NULL;
  	Datum		arg;
  
  	if (!AggCheckCallContext(fcinfo, &aggcontext))
*************** json_object_agg_transfn(PG_FUNCTION_ARGS
*** 1938,1953 ****
  		 * use the right context to enlarge the object if necessary.
  		 */
  		oldcontext = MemoryContextSwitchTo(aggcontext);
! 		state = makeStringInfo();
  		MemoryContextSwitchTo(oldcontext);
  
! 		appendStringInfoString(state, "{ ");
  	}
  	else
! 	{
! 		state = (StringInfo) PG_GETARG_POINTER(0);
! 		appendStringInfoString(state, ", ");
! 	}
  
  	/*
  	 * Note: since json_object_agg() is declared as taking type "any", the
--- 2007,2020 ----
  		 * use the right context to enlarge the object if necessary.
  		 */
  		oldcontext = MemoryContextSwitchTo(aggcontext);
! 		out = palloc(sizeof(JsonOutContext));
! 		json_out_init_context(out, JSON_OUT_USE_SPACES);
  		MemoryContextSwitchTo(oldcontext);
  
! 		out->object_start(out);
  	}
  	else
! 		out = (JsonOutContext *) PG_GETARG_POINTER(0);
  
  	/*
  	 * Note: since json_object_agg() is declared as taking type "any", the
*************** json_object_agg_transfn(PG_FUNCTION_ARGS
*** 1970,1978 ****
  
  	arg = PG_GETARG_DATUM(1);
  
! 	add_json(arg, false, state, val_type, true);
! 
! 	appendStringInfoString(state, " : ");
  
  	val_type = get_fn_expr_argtype(fcinfo->flinfo, 2);
  
--- 2037,2043 ----
  
  	arg = PG_GETARG_DATUM(1);
  
! 	add_json(arg, false, out, val_type, true);
  
  	val_type = get_fn_expr_argtype(fcinfo->flinfo, 2);
  
*************** json_object_agg_transfn(PG_FUNCTION_ARGS
*** 1986,1994 ****
  	else
  		arg = PG_GETARG_DATUM(2);
  
! 	add_json(arg, PG_ARGISNULL(2), state, val_type, false);
  
! 	PG_RETURN_POINTER(state);
  }
  
  /*
--- 2051,2059 ----
  	else
  		arg = PG_GETARG_DATUM(2);
  
! 	add_json(arg, PG_ARGISNULL(2), out, val_type, false);
  
! 	PG_RETURN_POINTER(out);
  }
  
  /*
*************** json_object_agg_transfn(PG_FUNCTION_ARGS
*** 1997,2035 ****
  Datum
  json_object_agg_finalfn(PG_FUNCTION_ARGS)
  {
! 	StringInfo	state;
  
  	/* cannot be called directly because of internal-type argument */
  	Assert(AggCheckCallContext(fcinfo, NULL));
  
! 	state = PG_ARGISNULL(0) ? NULL : (StringInfo) PG_GETARG_POINTER(0);
  
  	/* NULL result for no rows in, as is standard with aggregates */
! 	if (state == NULL)
  		PG_RETURN_NULL();
  
! 	/* Else return state with appropriate object terminator added */
! 	PG_RETURN_TEXT_P(catenate_stringinfo_string(state, " }"));
! }
! 
! /*
!  * Helper function for aggregates: return given StringInfo's contents plus
!  * specified trailing string, as a text datum.  We need this because aggregate
!  * final functions are not allowed to modify the aggregate state.
!  */
! static text *
! catenate_stringinfo_string(StringInfo buffer, const char *addon)
! {
! 	/* custom version of cstring_to_text_with_len */
! 	int			buflen = buffer->len;
! 	int			addlen = strlen(addon);
! 	text	   *result = (text *) palloc(buflen + addlen + VARHDRSZ);
! 
! 	SET_VARSIZE(result, buflen + addlen + VARHDRSZ);
! 	memcpy(VARDATA(result), buffer->data, buflen);
! 	memcpy(VARDATA(result) + buflen, addon, addlen);
  
! 	return result;
  }
  
  /*
--- 2062,2081 ----
  Datum
  json_object_agg_finalfn(PG_FUNCTION_ARGS)
  {
! 	JsonOutContext	*out;
  
  	/* cannot be called directly because of internal-type argument */
  	Assert(AggCheckCallContext(fcinfo, NULL));
  
! 	out = PG_ARGISNULL(0) ? NULL : (JsonOutContext *) PG_GETARG_POINTER(0);
  
  	/* NULL result for no rows in, as is standard with aggregates */
! 	if (out == NULL)
  		PG_RETURN_NULL();
  
! 	out->object_end(out);
  
! 	PG_RETURN_TEXT_P(cstring_to_text_with_len(out->result.data, out->result.len));
  }
  
  /*
*************** json_build_object(PG_FUNCTION_ARGS)
*** 2041,2048 ****
  	int			nargs = PG_NARGS();
  	int			i;
  	Datum		arg;
! 	const char *sep = "";
! 	StringInfo	result;
  	Oid			val_type;
  
  	if (nargs % 2 != 0)
--- 2087,2093 ----
  	int			nargs = PG_NARGS();
  	int			i;
  	Datum		arg;
! 	JsonOutContext	out;
  	Oid			val_type;
  
  	if (nargs % 2 != 0)
*************** json_build_object(PG_FUNCTION_ARGS)
*** 2051,2059 ****
  				 errmsg("argument list must have even number of elements"),
  				 errhint("The arguments of json_build_object() must consist of alternating keys and values.")));
  
! 	result = makeStringInfo();
! 
! 	appendStringInfoChar(result, '{');
  
  	for (i = 0; i < nargs; i += 2)
  	{
--- 2096,2103 ----
  				 errmsg("argument list must have even number of elements"),
  				 errhint("The arguments of json_build_object() must consist of alternating keys and values.")));
  
! 	json_out_init_context(&out, JSON_OUT_USE_SPACES);
! 	out.object_start(&out);
  
  	for (i = 0; i < nargs; i += 2)
  	{
*************** json_build_object(PG_FUNCTION_ARGS)
*** 2064,2071 ****
  		 * here as type UNKNOWN, which fortunately does not matter to us,
  		 * since unknownout() works fine.
  		 */
- 		appendStringInfoString(result, sep);
- 		sep = ", ";
  
  		/* process key */
  		val_type = get_fn_expr_argtype(fcinfo->flinfo, i);
--- 2108,2113 ----
*************** json_build_object(PG_FUNCTION_ARGS)
*** 2084,2092 ****
  
  		arg = PG_GETARG_DATUM(i);
  
! 		add_json(arg, false, result, val_type, true);
! 
! 		appendStringInfoString(result, " : ");
  
  		/* process value */
  		val_type = get_fn_expr_argtype(fcinfo->flinfo, i + 1);
--- 2126,2132 ----
  
  		arg = PG_GETARG_DATUM(i);
  
! 		add_json(arg, false, &out, val_type, true);
  
  		/* process value */
  		val_type = get_fn_expr_argtype(fcinfo->flinfo, i + 1);
*************** json_build_object(PG_FUNCTION_ARGS)
*** 2102,2113 ****
  		else
  			arg = PG_GETARG_DATUM(i + 1);
  
! 		add_json(arg, PG_ARGISNULL(i + 1), result, val_type, false);
  	}
  
! 	appendStringInfoChar(result, '}');
  
! 	PG_RETURN_TEXT_P(cstring_to_text_with_len(result->data, result->len));
  }
  
  /*
--- 2142,2153 ----
  		else
  			arg = PG_GETARG_DATUM(i + 1);
  
! 		add_json(arg, PG_ARGISNULL(i + 1), &out, val_type, false);
  	}
  
! 	out.object_end(&out);
  
! 	PG_RETURN_TEXT_P(cstring_to_text_with_len(out.result.data, out.result.len));
  }
  
  /*
*************** json_build_array(PG_FUNCTION_ARGS)
*** 2128,2140 ****
  	int			nargs = PG_NARGS();
  	int			i;
  	Datum		arg;
! 	const char *sep = "";
! 	StringInfo	result;
  	Oid			val_type;
  
! 	result = makeStringInfo();
! 
! 	appendStringInfoChar(result, '[');
  
  	for (i = 0; i < nargs; i++)
  	{
--- 2168,2178 ----
  	int			nargs = PG_NARGS();
  	int			i;
  	Datum		arg;
! 	JsonOutContext	out;
  	Oid			val_type;
  
! 	json_out_init_context(&out, JSON_OUT_USE_SPACES);
! 	out.array_start(&out);
  
  	for (i = 0; i < nargs; i++)
  	{
*************** json_build_array(PG_FUNCTION_ARGS)
*** 2145,2153 ****
  		 * here as type UNKNOWN, which fortunately does not matter to us,
  		 * since unknownout() works fine.
  		 */
- 		appendStringInfoString(result, sep);
- 		sep = ", ";
- 
  		val_type = get_fn_expr_argtype(fcinfo->flinfo, i);
  
  		if (val_type == InvalidOid)
--- 2183,2188 ----
*************** json_build_array(PG_FUNCTION_ARGS)
*** 2161,2172 ****
  		else
  			arg = PG_GETARG_DATUM(i);
  
! 		add_json(arg, PG_ARGISNULL(i), result, val_type, false);
  	}
  
! 	appendStringInfoChar(result, ']');
  
! 	PG_RETURN_TEXT_P(cstring_to_text_with_len(result->data, result->len));
  }
  
  /*
--- 2196,2207 ----
  		else
  			arg = PG_GETARG_DATUM(i);
  
! 		add_json(arg, PG_ARGISNULL(i), &out, val_type, false);
  	}
  
! 	out.array_end(&out);
  
! 	PG_RETURN_TEXT_P(cstring_to_text_with_len(out.result.data, out.result.len));
  }
  
  /*
*************** json_object(PG_FUNCTION_ARGS)
*** 2189,2202 ****
  {
  	ArrayType  *in_array = PG_GETARG_ARRAYTYPE_P(0);
  	int			ndims = ARR_NDIM(in_array);
! 	StringInfoData result;
  	Datum	   *in_datums;
  	bool	   *in_nulls;
  	int			in_count,
  				count,
  				i;
- 	text	   *rval;
- 	char	   *v;
  
  	switch (ndims)
  	{
--- 2224,2235 ----
  {
  	ArrayType  *in_array = PG_GETARG_ARRAYTYPE_P(0);
  	int			ndims = ARR_NDIM(in_array);
! 	JsonOutContext	out;
  	Datum	   *in_datums;
  	bool	   *in_nulls;
  	int			in_count,
  				count,
  				i;
  
  	switch (ndims)
  	{
*************** json_object(PG_FUNCTION_ARGS)
*** 2230,2238 ****
  
  	count = in_count / 2;
  
! 	initStringInfo(&result);
! 
! 	appendStringInfoChar(&result, '{');
  
  	for (i = 0; i < count; ++i)
  	{
--- 2263,2270 ----
  
  	count = in_count / 2;
  
! 	json_out_init_context(&out, JSON_OUT_USE_SPACES);
! 	out.object_start(&out);
  
  	for (i = 0; i < count; ++i)
  	{
*************** json_object(PG_FUNCTION_ARGS)
*** 2241,2272 ****
  					(errcode(ERRCODE_NULL_VALUE_NOT_ALLOWED),
  					 errmsg("null value not allowed for object key")));
  
! 		v = TextDatumGetCString(in_datums[i * 2]);
! 		if (i > 0)
! 			appendStringInfoString(&result, ", ");
! 		escape_json(&result, v);
! 		appendStringInfoString(&result, " : ");
! 		pfree(v);
  		if (in_nulls[i * 2 + 1])
! 			appendStringInfoString(&result, "null");
  		else
! 		{
! 			v = TextDatumGetCString(in_datums[i * 2 + 1]);
! 			escape_json(&result, v);
! 			pfree(v);
! 		}
  	}
  
! 	appendStringInfoChar(&result, '}');
  
  	pfree(in_datums);
  	pfree(in_nulls);
  
! 	rval = cstring_to_text_with_len(result.data, result.len);
! 	pfree(result.data);
! 
! 	PG_RETURN_TEXT_P(rval);
! 
  }
  
  /*
--- 2273,2295 ----
  					(errcode(ERRCODE_NULL_VALUE_NOT_ALLOWED),
  					 errmsg("null value not allowed for object key")));
  
! 		out.value(&out, in_datums[i * 2], JSONTYPE_OTHER,
! 				  TEXTOID, 47 /* textout */, true);
! 
  		if (in_nulls[i * 2 + 1])
! 			out.value(&out, (Datum) 0, JSONTYPE_NULL,
! 					  InvalidOid, InvalidOid, false);
  		else
! 			out.value(&out, in_datums[i * 2 + 1], JSONTYPE_OTHER,
! 					  TEXTOID, 47 /* textout */, false);
  	}
  
! 	out.object_end(&out);
  
  	pfree(in_datums);
  	pfree(in_nulls);
  
! 	PG_RETURN_TEXT_P(cstring_to_text_with_len(out.result.data, out.result.len));
  }
  
  /*
*************** json_object_two_arg(PG_FUNCTION_ARGS)
*** 2282,2288 ****
  	ArrayType  *val_array = PG_GETARG_ARRAYTYPE_P(1);
  	int			nkdims = ARR_NDIM(key_array);
  	int			nvdims = ARR_NDIM(val_array);
! 	StringInfoData result;
  	Datum	   *key_datums,
  			   *val_datums;
  	bool	   *key_nulls,
--- 2305,2311 ----
  	ArrayType  *val_array = PG_GETARG_ARRAYTYPE_P(1);
  	int			nkdims = ARR_NDIM(key_array);
  	int			nvdims = ARR_NDIM(val_array);
! 	JsonOutContext	out;
  	Datum	   *key_datums,
  			   *val_datums;
  	bool	   *key_nulls,
*************** json_object_two_arg(PG_FUNCTION_ARGS)
*** 2290,2297 ****
  	int			key_count,
  				val_count,
  				i;
- 	text	   *rval;
- 	char	   *v;
  
  	if (nkdims > 1 || nkdims != nvdims)
  		ereport(ERROR,
--- 2313,2318 ----
*************** json_object_two_arg(PG_FUNCTION_ARGS)
*** 2314,2322 ****
  				(errcode(ERRCODE_ARRAY_SUBSCRIPT_ERROR),
  				 errmsg("mismatched array dimensions")));
  
! 	initStringInfo(&result);
! 
! 	appendStringInfoChar(&result, '{');
  
  	for (i = 0; i < key_count; ++i)
  	{
--- 2335,2342 ----
  				(errcode(ERRCODE_ARRAY_SUBSCRIPT_ERROR),
  				 errmsg("mismatched array dimensions")));
  
! 	json_out_init_context(&out, JSON_OUT_USE_SPACES);
! 	out.object_start(&out);
  
  	for (i = 0; i < key_count; ++i)
  	{
*************** json_object_two_arg(PG_FUNCTION_ARGS)
*** 2325,2357 ****
  					(errcode(ERRCODE_NULL_VALUE_NOT_ALLOWED),
  					 errmsg("null value not allowed for object key")));
  
! 		v = TextDatumGetCString(key_datums[i]);
! 		if (i > 0)
! 			appendStringInfoString(&result, ", ");
! 		escape_json(&result, v);
! 		appendStringInfoString(&result, " : ");
! 		pfree(v);
  		if (val_nulls[i])
! 			appendStringInfoString(&result, "null");
  		else
! 		{
! 			v = TextDatumGetCString(val_datums[i]);
! 			escape_json(&result, v);
! 			pfree(v);
! 		}
  	}
  
! 	appendStringInfoChar(&result, '}');
  
  	pfree(key_datums);
  	pfree(key_nulls);
  	pfree(val_datums);
  	pfree(val_nulls);
  
! 	rval = cstring_to_text_with_len(result.data, result.len);
! 	pfree(result.data);
! 
! 	PG_RETURN_TEXT_P(rval);
  }
  
  
--- 2345,2369 ----
  					(errcode(ERRCODE_NULL_VALUE_NOT_ALLOWED),
  					 errmsg("null value not allowed for object key")));
  
! 		out.value(&out, key_datums[i], JSONTYPE_OTHER,
! 				  TEXTOID, 47 /* textout */, true);
! 
  		if (val_nulls[i])
! 			out.value(&out, (Datum) 0, JSONTYPE_NULL,
! 					  InvalidOid, InvalidOid, false);
  		else
! 			out.value(&out, val_datums[i], JSONTYPE_OTHER,
! 					  TEXTOID, 47 /* textout */, false);
  	}
  
! 	out.object_end(&out);
  
  	pfree(key_datums);
  	pfree(key_nulls);
  	pfree(val_datums);
  	pfree(val_nulls);
  
! 	PG_RETURN_TEXT_P(cstring_to_text_with_len(out.result.data, out.result.len));
  }
  
  
diff --git a/src/include/utils/jsonapi.h b/src/include/utils/jsonapi.h
new file mode 100644
index 296d20a..8b9af6c
*** a/src/include/utils/jsonapi.h
--- b/src/include/utils/jsonapi.h
*************** extern JsonLexContext *makeJsonLexContex
*** 124,127 ****
--- 124,194 ----
   */
  extern bool IsValidJsonNumber(const char *str, int len);
  
+ 
+ /*
+  * Generalized structures for producing JSON output.
+  */
+ typedef enum					/* type categories for datum_to_json */
+ {
+ 	JSONTYPE_NULL,				/* null, so we didn't bother to identify */
+ 	JSONTYPE_BOOL,				/* boolean (built-in types only) */
+ 	JSONTYPE_NUMERIC,			/* numeric (ditto) */
+ 	JSONTYPE_DATE,				/* we use special formatting for datetimes */
+ 	JSONTYPE_TIMESTAMP,
+ 	JSONTYPE_TIMESTAMPTZ,
+ 	JSONTYPE_JSON,				/* JSON itself (and JSONB) */
+ 	JSONTYPE_ARRAY,				/* array */
+ 	JSONTYPE_COMPOSITE,			/* composite */
+ 	JSONTYPE_CAST,				/* something with an explicit cast to JSON */
+ 	JSONTYPE_OTHER				/* all else */
+ } JsonTypeCategory;
+ 
+ struct JsonOutContext;
+ 
+ typedef void (*json_out_struct_action)(struct JsonOutContext *out);
+ typedef void (*json_out_value_action)(struct JsonOutContext *out,
+ 			  Datum val, JsonTypeCategory tcategory,
+ 			  Oid typoid, Oid outfuncoid, bool key_scalar);
+ typedef void (*json_out_post_action)(struct JsonOutContext *out,
+ 			  bool key_scalar);
+ 
+ #define JSON_OUT_USE_LINE_FEEDS		1
+ #define JSON_OUT_USE_SPACES			2
+ 
+ typedef struct JsonOutContext {
+ 	json_out_struct_action	object_start;
+ 	json_out_struct_action	object_end;
+ 	json_out_struct_action	array_start;
+ 	json_out_struct_action	array_end;
+ 	json_out_struct_action	before_value;
+ 	json_out_value_action	value;
+ 	json_out_post_action	after_value;
+ 
+ 	StringInfoData	result;
+ 	int		flags;
+ 	bool	need_comma;
+ 	int		depth;
+ 	/* these are used in json_agg to cache the type information */
+ 	JsonTypeCategory	agg_tcategory;
+ 	Oid					agg_outfuncoid;
+ } JsonOutContext;
+ 
+ extern void json_out_init_context(JsonOutContext *out, int flags);
+ extern void json_out_before_value(JsonOutContext *out);
+ extern void json_out_after_value(JsonOutContext *out, bool key_scalar);
+ extern void json_out_object_start(JsonOutContext *out);
+ extern void json_out_object_end(JsonOutContext *out);
+ extern void json_out_array_start(JsonOutContext *out);
+ extern void json_out_array_end(JsonOutContext *out);
+ extern void json_out_value(JsonOutContext *out, Datum val,
+ 				JsonTypeCategory tcategory, Oid typoid, Oid outfuncoid,
+ 				bool key_scalar);
+ extern void json_out_cstring(JsonOutContext *out, const char *str,
+ 				bool key_scalar);
+ 
+ extern void composite_to_json(Datum composite, JsonOutContext *out);
+ extern void datum_to_json(Datum val, bool is_null, JsonOutContext *out,
+ 			  JsonTypeCategory tcategory, Oid typoid, Oid outfuncoid,
+ 			  bool key_scalar);
+ 
  #endif   /* JSONAPI_H */
diff --git a/src/test/regress/expected/json.out b/src/test/regress/expected/json.out
new file mode 100644
index 3942c3b..fbc9b5b
*** a/src/test/regress/expected/json.out
--- b/src/test/regress/expected/json.out
*************** SELECT json_agg(q)
*** 457,476 ****
                 ROW(y.*,ARRAY[4,5,6])] AS z
           FROM generate_series(1,2) x,
                generate_series(4,5) y) q;
!                                json_agg                                
! -----------------------------------------------------------------------
!  [{"b":"a1","c":4,"z":[{"f1":1,"f2":[1,2,3]},{"f1":4,"f2":[4,5,6]}]}, +
!   {"b":"a1","c":5,"z":[{"f1":1,"f2":[1,2,3]},{"f1":5,"f2":[4,5,6]}]}, +
!   {"b":"a2","c":4,"z":[{"f1":2,"f2":[1,2,3]},{"f1":4,"f2":[4,5,6]}]}, +
    {"b":"a2","c":5,"z":[{"f1":2,"f2":[1,2,3]},{"f1":5,"f2":[4,5,6]}]}]
  (1 row)
  
  SELECT json_agg(q)
    FROM rows q;
!        json_agg        
! -----------------------
!  [{"x":1,"y":"txt1"}, +
!   {"x":2,"y":"txt2"}, +
    {"x":3,"y":"txt3"}]
  (1 row)
  
--- 457,476 ----
                 ROW(y.*,ARRAY[4,5,6])] AS z
           FROM generate_series(1,2) x,
                generate_series(4,5) y) q;
!                                json_agg                               
! ----------------------------------------------------------------------
!  [{"b":"a1","c":4,"z":[{"f1":1,"f2":[1,2,3]},{"f1":4,"f2":[4,5,6]}]},+
!   {"b":"a1","c":5,"z":[{"f1":1,"f2":[1,2,3]},{"f1":5,"f2":[4,5,6]}]},+
!   {"b":"a2","c":4,"z":[{"f1":2,"f2":[1,2,3]},{"f1":4,"f2":[4,5,6]}]},+
    {"b":"a2","c":5,"z":[{"f1":2,"f2":[1,2,3]},{"f1":5,"f2":[4,5,6]}]}]
  (1 row)
  
  SELECT json_agg(q)
    FROM rows q;
!        json_agg       
! ----------------------
!  [{"x":1,"y":"txt1"},+
!   {"x":2,"y":"txt2"},+
    {"x":3,"y":"txt3"}]
  (1 row)
  
*************** SELECT json_build_array('a',1,'b',1.2,'c
*** 1505,1522 ****
  (1 row)
  
  SELECT json_build_object('a',1,'b',1.2,'c',true,'d',null,'e',json '{"x": 3, "y": [1,2,3]}');
!                              json_build_object                              
! ----------------------------------------------------------------------------
!  {"a" : 1, "b" : 1.2, "c" : true, "d" : null, "e" : {"x": 3, "y": [1,2,3]}}
  (1 row)
  
  SELECT json_build_object(
         'a', json_build_object('b',false,'c',99),
         'd', json_build_object('e',array[9,8,7]::int[],
             'f', (select row_to_json(r) from ( select relkind, oid::regclass as name from pg_class where relname = 'pg_class') r)));
!                                         json_build_object                                        
! -------------------------------------------------------------------------------------------------
!  {"a" : {"b" : false, "c" : 99}, "d" : {"e" : [9,8,7], "f" : {"relkind":"r","name":"pg_class"}}}
  (1 row)
  
  -- empty objects/arrays
--- 1505,1522 ----
  (1 row)
  
  SELECT json_build_object('a',1,'b',1.2,'c',true,'d',null,'e',json '{"x": 3, "y": [1,2,3]}');
!                            json_build_object                           
! -----------------------------------------------------------------------
!  {"a": 1, "b": 1.2, "c": true, "d": null, "e": {"x": 3, "y": [1,2,3]}}
  (1 row)
  
  SELECT json_build_object(
         'a', json_build_object('b',false,'c',99),
         'd', json_build_object('e',array[9,8,7]::int[],
             'f', (select row_to_json(r) from ( select relkind, oid::regclass as name from pg_class where relname = 'pg_class') r)));
!                                      json_build_object                                     
! -------------------------------------------------------------------------------------------
!  {"a": {"b": false, "c": 99}, "d": {"e": [9,8,7], "f": {"relkind":"r","name":"pg_class"}}}
  (1 row)
  
  -- empty objects/arrays
*************** SELECT json_build_object();
*** 1536,1542 ****
  SELECT json_build_object(1,2);
   json_build_object 
  -------------------
!  {"1" : 2}
  (1 row)
  
  -- keys must be scalar and not null
--- 1536,1542 ----
  SELECT json_build_object(1,2);
   json_build_object 
  -------------------
!  {"1": 2}
  (1 row)
  
  -- keys must be scalar and not null
*************** INSERT INTO foo VALUES (847002,'t16','GE
*** 1555,1578 ****
  INSERT INTO foo VALUES (847003,'sub-alpha','GESS90');
  SELECT json_build_object('turbines',json_object_agg(serial_num,json_build_object('name',name,'type',type)))
  FROM foo;
!                                                                             json_build_object                                                                            
! -------------------------------------------------------------------------------------------------------------------------------------------------------------------------
!  {"turbines" : { "847001" : {"name" : "t15", "type" : "GE1043"}, "847002" : {"name" : "t16", "type" : "GE1043"}, "847003" : {"name" : "sub-alpha", "type" : "GESS90"} }}
  (1 row)
  
  -- json_object
  -- one dimension
  SELECT json_object('{a,1,b,2,3,NULL,"d e f","a b c"}');
!                       json_object                      
! -------------------------------------------------------
!  {"a" : "1", "b" : "2", "3" : null, "d e f" : "a b c"}
  (1 row)
  
  -- same but with two dimensions
  SELECT json_object('{{a,1},{b,2},{3,NULL},{"d e f","a b c"}}');
!                       json_object                      
! -------------------------------------------------------
!  {"a" : "1", "b" : "2", "3" : null, "d e f" : "a b c"}
  (1 row)
  
  -- odd number error
--- 1555,1578 ----
  INSERT INTO foo VALUES (847003,'sub-alpha','GESS90');
  SELECT json_build_object('turbines',json_object_agg(serial_num,json_build_object('name',name,'type',type)))
  FROM foo;
!                                                                       json_build_object                                                                      
! -------------------------------------------------------------------------------------------------------------------------------------------------------------
!  {"turbines": {"847001": {"name": "t15", "type": "GE1043"}, "847002": {"name": "t16", "type": "GE1043"}, "847003": {"name": "sub-alpha", "type": "GESS90"}}}
  (1 row)
  
  -- json_object
  -- one dimension
  SELECT json_object('{a,1,b,2,3,NULL,"d e f","a b c"}');
!                     json_object                    
! ---------------------------------------------------
!  {"a": "1", "b": "2", "3": null, "d e f": "a b c"}
  (1 row)
  
  -- same but with two dimensions
  SELECT json_object('{{a,1},{b,2},{3,NULL},{"d e f","a b c"}}');
!                     json_object                    
! ---------------------------------------------------
!  {"a": "1", "b": "2", "3": null, "d e f": "a b c"}
  (1 row)
  
  -- odd number error
*************** SELECT json_object('{{{a,b},{c,d}},{{b,c
*** 1589,1597 ****
  ERROR:  wrong number of array subscripts
  --two argument form of json_object
  select json_object('{a,b,c,"d e f"}','{1,2,3,"a b c"}');
!                      json_object                      
! ------------------------------------------------------
!  {"a" : "1", "b" : "2", "c" : "3", "d e f" : "a b c"}
  (1 row)
  
  -- too many dimensions
--- 1589,1597 ----
  ERROR:  wrong number of array subscripts
  --two argument form of json_object
  select json_object('{a,b,c,"d e f"}','{1,2,3,"a b c"}');
!                    json_object                    
! --------------------------------------------------
!  {"a": "1", "b": "2", "c": "3", "d e f": "a b c"}
  (1 row)
  
  -- too many dimensions
*************** select json_object('{a,b,NULL,"d e f"}',
*** 1607,1615 ****
  ERROR:  null value not allowed for object key
  -- empty key is allowed
  select json_object('{a,b,"","d e f"}','{1,2,3,"a b c"}');
!                      json_object                     
! -----------------------------------------------------
!  {"a" : "1", "b" : "2", "" : "3", "d e f" : "a b c"}
  (1 row)
  
  -- json_to_record and json_to_recordset
--- 1607,1615 ----
  ERROR:  null value not allowed for object key
  -- empty key is allowed
  select json_object('{a,b,"","d e f"}','{1,2,3,"a b c"}');
!                    json_object                   
! -------------------------------------------------
!  {"a": "1", "b": "2", "": "3", "d e f": "a b c"}
  (1 row)
  
  -- json_to_record and json_to_recordset
#16Shulgin, Oleksandr
oleksandr.shulgin@zalando.de
In reply to: Pavel Stehule (#15)
Re: [PATCH] Generalized JSON output functions

2015-07-10 14:34 GMT+02:00 Pavel Stehule <pavel.stehule@gmail.com>:

Hi

I am sending review of this patch:

1. I reread a previous discussion and almost all are for this patch (me
too)

2. I have to fix a typo in hstore_io.c function (update attached), other
(patching, regress tests) without problems

My objections:

1. comments - missing comment for some basic API, basic fields like
"key_scalar" and similar

I thought it was pretty obvious from the code, because it's sort of the
only source for docs on the subject right now. Should we add proper
documentation section, this would have been documented for sure.

2. why you did indirect call via JsonOutContext?

What is benefit

dst.value(&dst, (Datum) 0, JSONTYPE_NULL, InvalidOid, InvalidOid, false);

instead

json_out_value(&dst, ....)

For consistency. Even though we initialize the output context ourselves,
there might be some code introduced between json_out_init_context() and
dst.value() calls that replaces some of the callbacks, and then there would
be a difference.

3. if it should be used everywhere, then in EXPLAIN statement too.

Ahh.. good catch. I'll have a look on this now.

Thanks for the review!

--
Alex

#17Pavel Stehule
pavel.stehule@gmail.com
In reply to: Shulgin, Oleksandr (#16)
Re: [PATCH] Generalized JSON output functions

2015-07-10 15:57 GMT+02:00 Shulgin, Oleksandr <oleksandr.shulgin@zalando.de>
:

2015-07-10 14:34 GMT+02:00 Pavel Stehule <pavel.stehule@gmail.com>:

Hi

I am sending review of this patch:

1. I reread a previous discussion and almost all are for this patch (me
too)

2. I have to fix a typo in hstore_io.c function (update attached), other
(patching, regress tests) without problems

My objections:

1. comments - missing comment for some basic API, basic fields like
"key_scalar" and similar

I thought it was pretty obvious from the code, because it's sort of the
only source for docs on the subject right now. Should we add proper
documentation section, this would have been documented for sure.

2. why you did indirect call via JsonOutContext?

What is benefit

dst.value(&dst, (Datum) 0, JSONTYPE_NULL, InvalidOid, InvalidOid, false);

instead

json_out_value(&dst, ....)

For consistency. Even though we initialize the output context ourselves,
there might be some code introduced between json_out_init_context() and
dst.value() calls that replaces some of the callbacks, and then there would
be a difference.

with this consistency? I didn't see this style everywhere in Postgres?
Isn't it premature optimization?

Show quoted text

3. if it should be used everywhere, then in EXPLAIN statement too.

Ahh.. good catch. I'll have a look on this now.

Thanks for the review!

--
Alex

#18Shulgin, Oleksandr
oleksandr.shulgin@zalando.de
In reply to: Pavel Stehule (#17)
Re: [PATCH] Generalized JSON output functions

On Fri, Jul 10, 2015 at 4:04 PM, Pavel Stehule <pavel.stehule@gmail.com>
wrote:

2. why you did indirect call via JsonOutContext?

What is benefit

dst.value(&dst, (Datum) 0, JSONTYPE_NULL, InvalidOid, InvalidOid,
false);

instead

json_out_value(&dst, ....)

For consistency. Even though we initialize the output context ourselves,
there might be some code introduced between json_out_init_context() and
dst.value() calls that replaces some of the callbacks, and then there would
be a difference.

with this consistency? I didn't see this style everywhere in Postgres?
Isn't it premature optimization?

Well, one could call it premature pessimization due to dynamic call
overhead.

IMO, the fact that json_out_init_context() sets the value callback to
json_out_value is an implementation detail, the other parts of code should
not rely on. And for the Explain output, there definitely going to be
*some* code between context initialization and output callbacks: these are
done in a number of different functions.

--
Alex

#19Pavel Stehule
pavel.stehule@gmail.com
In reply to: Shulgin, Oleksandr (#18)
Re: [PATCH] Generalized JSON output functions

2015-07-10 16:16 GMT+02:00 Shulgin, Oleksandr <oleksandr.shulgin@zalando.de>
:

On Fri, Jul 10, 2015 at 4:04 PM, Pavel Stehule <pavel.stehule@gmail.com>
wrote:

2. why you did indirect call via JsonOutContext?

What is benefit

dst.value(&dst, (Datum) 0, JSONTYPE_NULL, InvalidOid, InvalidOid,
false);

instead

json_out_value(&dst, ....)

For consistency. Even though we initialize the output context
ourselves, there might be some code introduced between
json_out_init_context() and dst.value() calls that replaces some of the
callbacks, and then there would be a difference.

with this consistency? I didn't see this style everywhere in Postgres?
Isn't it premature optimization?

Well, one could call it premature pessimization due to dynamic call
overhead.

IMO, the fact that json_out_init_context() sets the value callback to
json_out_value is an implementation detail, the other parts of code should
not rely on. And for the Explain output, there definitely going to be
*some* code between context initialization and output callbacks: these are
done in a number of different functions.

Again - it is necessary? Postgres still use modular code, not OOP code. I
can understand the using of this technique, when I need a possibility to
change behave. But these function are used for printing JSON, not printing
any others.

Pavel

Show quoted text

--
Alex

#20Shulgin, Oleksandr
oleksandr.shulgin@zalando.de
In reply to: Pavel Stehule (#19)
1 attachment(s)
Re: [PATCH] Generalized JSON output functions

On Fri, Jul 10, 2015 at 5:16 PM, Pavel Stehule <pavel.stehule@gmail.com>
wrote:

Well, one could call it premature pessimization due to dynamic call
overhead.

IMO, the fact that json_out_init_context() sets the value callback to
json_out_value is an implementation detail, the other parts of code should
not rely on. And for the Explain output, there definitely going to be
*some* code between context initialization and output callbacks: these are
done in a number of different functions.

Again - it is necessary? Postgres still use modular code, not OOP code. I
can understand the using of this technique, when I need a possibility to
change behave. But these function are used for printing JSON, not printing
any others.

No, it's not strictly necessary.

For me it's not about procedural- vs. object- style, but rather about being
able to override/extend the behavior consistently. And for that I would
prefer that if I override the value callback in a JSON output context, that
it would be called for every value being printed, not only for some of them.

Thank you for pointing out the case of Explain format, I've totally
overlooked it in my first version. Trying to apply the proposed approach
in the explain printing code led me to reorganize things slightly. I've
added explicit functions for printing keys vs. values, thus no need to
expose that key_scalar param anymore. There are now separate before/after
key and before/after value functions as well, but I believe it makes for a
cleaner code.

The most of the complexity is still in the code that decides whether or not
to put spaces (between the values or for indentation) and newlines at
certain points. Should we decide to unify the style we emit ourselves,
this could be simplified, while still leaving room for great flexibility if
overridden by an extension, for example.

Have a nice weekend.
--
Alex

Attachments:

json-output-generalized-v1-3.patchtext/x-patch; charset=US-ASCII; name=json-output-generalized-v1-3.patchDownload
diff --git a/contrib/hstore/hstore_io.c b/contrib/hstore/hstore_io.c
new file mode 100644
index 7d89867..1f365f5
*** a/contrib/hstore/hstore_io.c
--- b/contrib/hstore/hstore_io.c
*************** hstore_to_json_loose(PG_FUNCTION_ARGS)
*** 1241,1286 ****
  	int			count = HS_COUNT(in);
  	char	   *base = STRPTR(in);
  	HEntry	   *entries = ARRPTR(in);
! 	StringInfoData tmp,
! 				dst;
  
  	if (count == 0)
  		PG_RETURN_TEXT_P(cstring_to_text_with_len("{}", 2));
  
  	initStringInfo(&tmp);
! 	initStringInfo(&dst);
! 
! 	appendStringInfoChar(&dst, '{');
  
  	for (i = 0; i < count; i++)
  	{
  		resetStringInfo(&tmp);
  		appendBinaryStringInfo(&tmp, HS_KEY(entries, base, i), HS_KEYLEN(entries, i));
! 		escape_json(&dst, tmp.data);
! 		appendStringInfoString(&dst, ": ");
  		if (HS_VALISNULL(entries, i))
! 			appendStringInfoString(&dst, "null");
  		/* guess that values of 't' or 'f' are booleans */
  		else if (HS_VALLEN(entries, i) == 1 && *(HS_VAL(entries, base, i)) == 't')
! 			appendStringInfoString(&dst, "true");
  		else if (HS_VALLEN(entries, i) == 1 && *(HS_VAL(entries, base, i)) == 'f')
! 			appendStringInfoString(&dst, "false");
  		else
  		{
  			resetStringInfo(&tmp);
  			appendBinaryStringInfo(&tmp, HS_VAL(entries, base, i), HS_VALLEN(entries, i));
  			if (IsValidJsonNumber(tmp.data, tmp.len))
! 				appendBinaryStringInfo(&dst, tmp.data, tmp.len);
  			else
! 				escape_json(&dst, tmp.data);
  		}
- 
- 		if (i + 1 != count)
- 			appendStringInfoString(&dst, ", ");
  	}
- 	appendStringInfoChar(&dst, '}');
  
! 	PG_RETURN_TEXT_P(cstring_to_text(dst.data));
  }
  
  PG_FUNCTION_INFO_V1(hstore_to_json);
--- 1241,1293 ----
  	int			count = HS_COUNT(in);
  	char	   *base = STRPTR(in);
  	HEntry	   *entries = ARRPTR(in);
! 	StringInfoData	tmp;
! 	Datum			num;
! 	JsonOutContext	dst;
  
  	if (count == 0)
  		PG_RETURN_TEXT_P(cstring_to_text_with_len("{}", 2));
  
  	initStringInfo(&tmp);
! 	json_out_init_context(&dst, JSON_OUT_USE_SPACES);
! 	dst.object_start(&dst);
  
  	for (i = 0; i < count; i++)
  	{
  		resetStringInfo(&tmp);
  		appendBinaryStringInfo(&tmp, HS_KEY(entries, base, i), HS_KEYLEN(entries, i));
! 		json_out_cstring_key(&dst, tmp.data);
! 
  		if (HS_VALISNULL(entries, i))
! 			dst.value(&dst, (Datum) 0, JSONTYPE_NULL, InvalidOid, InvalidOid);
! 
  		/* guess that values of 't' or 'f' are booleans */
  		else if (HS_VALLEN(entries, i) == 1 && *(HS_VAL(entries, base, i)) == 't')
! 			dst.value(&dst, BoolGetDatum(true), JSONTYPE_BOOL,
! 					  InvalidOid, InvalidOid);
! 
  		else if (HS_VALLEN(entries, i) == 1 && *(HS_VAL(entries, base, i)) == 'f')
! 			dst.value(&dst, BoolGetDatum(false), JSONTYPE_BOOL,
! 					  InvalidOid, InvalidOid);
  		else
  		{
  			resetStringInfo(&tmp);
  			appendBinaryStringInfo(&tmp, HS_VAL(entries, base, i), HS_VALLEN(entries, i));
+ 
  			if (IsValidJsonNumber(tmp.data, tmp.len))
! 			{
! 				num = DirectFunctionCall3(numeric_in, CStringGetDatum(tmp.data), 0, -1);
! 				dst.value(&dst, num, JSONTYPE_NUMERIC,
! 						  NUMERICOID, 1702 /* numeric_out */);
! 				pfree(DatumGetPointer(num));
! 			}
  			else
! 				json_out_cstring_value(&dst, tmp.data);
  		}
  	}
  
! 	dst.object_end(&dst);
! 	PG_RETURN_TEXT_P(cstring_to_text_with_len(dst.result.data, dst.result.len));
  }
  
  PG_FUNCTION_INFO_V1(hstore_to_json);
*************** hstore_to_json(PG_FUNCTION_ARGS)
*** 1292,1329 ****
  	int			count = HS_COUNT(in);
  	char	   *base = STRPTR(in);
  	HEntry	   *entries = ARRPTR(in);
! 	StringInfoData tmp,
! 				dst;
  
  	if (count == 0)
  		PG_RETURN_TEXT_P(cstring_to_text_with_len("{}", 2));
  
  	initStringInfo(&tmp);
! 	initStringInfo(&dst);
! 
! 	appendStringInfoChar(&dst, '{');
  
  	for (i = 0; i < count; i++)
  	{
  		resetStringInfo(&tmp);
  		appendBinaryStringInfo(&tmp, HS_KEY(entries, base, i), HS_KEYLEN(entries, i));
! 		escape_json(&dst, tmp.data);
! 		appendStringInfoString(&dst, ": ");
  		if (HS_VALISNULL(entries, i))
! 			appendStringInfoString(&dst, "null");
  		else
  		{
  			resetStringInfo(&tmp);
  			appendBinaryStringInfo(&tmp, HS_VAL(entries, base, i), HS_VALLEN(entries, i));
! 			escape_json(&dst, tmp.data);
  		}
- 
- 		if (i + 1 != count)
- 			appendStringInfoString(&dst, ", ");
  	}
- 	appendStringInfoChar(&dst, '}');
  
! 	PG_RETURN_TEXT_P(cstring_to_text(dst.data));
  }
  
  PG_FUNCTION_INFO_V1(hstore_to_jsonb);
--- 1299,1332 ----
  	int			count = HS_COUNT(in);
  	char	   *base = STRPTR(in);
  	HEntry	   *entries = ARRPTR(in);
! 	StringInfoData	tmp;
! 	JsonOutContext	dst;
  
  	if (count == 0)
  		PG_RETURN_TEXT_P(cstring_to_text_with_len("{}", 2));
  
  	initStringInfo(&tmp);
! 	json_out_init_context(&dst, JSON_OUT_USE_SPACES);
! 	dst.object_start(&dst);
  
  	for (i = 0; i < count; i++)
  	{
  		resetStringInfo(&tmp);
  		appendBinaryStringInfo(&tmp, HS_KEY(entries, base, i), HS_KEYLEN(entries, i));
! 		json_out_cstring_key(&dst, tmp.data);
! 
  		if (HS_VALISNULL(entries, i))
! 			dst.value(&dst, (Datum) 0, JSONTYPE_NULL, InvalidOid, InvalidOid);
  		else
  		{
  			resetStringInfo(&tmp);
  			appendBinaryStringInfo(&tmp, HS_VAL(entries, base, i), HS_VALLEN(entries, i));
! 			json_out_cstring_value(&dst, tmp.data);
  		}
  	}
  
! 	dst.object_end(&dst);
! 	PG_RETURN_TEXT_P(cstring_to_text_with_len(dst.result.data, dst.result.len));
  }
  
  PG_FUNCTION_INFO_V1(hstore_to_jsonb);
diff --git a/src/backend/commands/explain.c b/src/backend/commands/explain.c
new file mode 100644
index 0d1ecc2..19c7db2
*** a/src/backend/commands/explain.c
--- b/src/backend/commands/explain.c
*************** static void ExplainCloseGroup(const char
*** 126,132 ****
  static void ExplainDummyGroup(const char *objtype, const char *labelname,
  				  ExplainState *es);
  static void ExplainXMLTag(const char *tagname, int flags, ExplainState *es);
- static void ExplainJSONLineEnding(ExplainState *es);
  static void ExplainYAMLLineStarting(ExplainState *es);
  static void escape_yaml(StringInfo buf, const char *str);
  
--- 126,131 ----
*************** ExplainQuery(ExplainStmt *stmt, const ch
*** 173,179 ****
--- 172,189 ----
  			else if (strcmp(p, "xml") == 0)
  				es->format = EXPLAIN_FORMAT_XML;
  			else if (strcmp(p, "json") == 0)
+ 			{
  				es->format = EXPLAIN_FORMAT_JSON;
+ 
+ 				json_out_init_context(&es->json_cxt,
+ 									  JSON_OUT_USE_SPACES | JSON_OUT_USE_LINE_FEEDS);
+ 				es->json_cxt.indent_spaces = 2;
+ 
+ 				/* point our result buffer straight to json output context */
+ 				pfree(es->str->data);
+ 				pfree(es->str);
+ 				es->str = &es->json_cxt.result;
+ 			}
  			else if (strcmp(p, "yaml") == 0)
  				es->format = EXPLAIN_FORMAT_YAML;
  			else
*************** ExplainPropertyList(const char *qlabel,
*** 2711,2728 ****
  			break;
  
  		case EXPLAIN_FORMAT_JSON:
! 			ExplainJSONLineEnding(es);
! 			appendStringInfoSpaces(es->str, es->indent * 2);
! 			escape_json(es->str, qlabel);
! 			appendStringInfoString(es->str, ": [");
  			foreach(lc, data)
  			{
! 				if (!first)
! 					appendStringInfoString(es->str, ", ");
! 				escape_json(es->str, (const char *) lfirst(lc));
! 				first = false;
  			}
! 			appendStringInfoChar(es->str, ']');
  			break;
  
  		case EXPLAIN_FORMAT_YAML:
--- 2721,2733 ----
  			break;
  
  		case EXPLAIN_FORMAT_JSON:
! 			json_out_cstring_key(&es->json_cxt, qlabel);
! 			es->json_cxt.array_start(&es->json_cxt);
  			foreach(lc, data)
  			{
! 				json_out_cstring_value(&es->json_cxt, (const char *) lfirst(lc));
  			}
! 			es->json_cxt.array_end(&es->json_cxt);
  			break;
  
  		case EXPLAIN_FORMAT_YAML:
*************** ExplainPropertyListNested(const char *ql
*** 2757,2773 ****
  			return;
  
  		case EXPLAIN_FORMAT_JSON:
! 			ExplainJSONLineEnding(es);
! 			appendStringInfoSpaces(es->str, es->indent * 2);
! 			appendStringInfoChar(es->str, '[');
  			foreach(lc, data)
  			{
! 				if (!first)
! 					appendStringInfoString(es->str, ", ");
! 				escape_json(es->str, (const char *) lfirst(lc));
! 				first = false;
  			}
! 			appendStringInfoChar(es->str, ']');
  			break;
  
  		case EXPLAIN_FORMAT_YAML:
--- 2762,2773 ----
  			return;
  
  		case EXPLAIN_FORMAT_JSON:
! 			es->json_cxt.array_start(&es->json_cxt);
  			foreach(lc, data)
  			{
! 				json_out_cstring_value(&es->json_cxt, (const char *) lfirst(lc));
  			}
! 			es->json_cxt.array_end(&es->json_cxt);
  			break;
  
  		case EXPLAIN_FORMAT_YAML:
*************** ExplainProperty(const char *qlabel, cons
*** 2820,2833 ****
  			break;
  
  		case EXPLAIN_FORMAT_JSON:
! 			ExplainJSONLineEnding(es);
! 			appendStringInfoSpaces(es->str, es->indent * 2);
! 			escape_json(es->str, qlabel);
! 			appendStringInfoString(es->str, ": ");
! 			if (numeric)
! 				appendStringInfoString(es->str, value);
! 			else
! 				escape_json(es->str, value);
  			break;
  
  		case EXPLAIN_FORMAT_YAML:
--- 2820,2839 ----
  			break;
  
  		case EXPLAIN_FORMAT_JSON:
! 			{
! 				Datum		num;
! 
! 				json_out_cstring_key(&es->json_cxt, qlabel);
! 				if (numeric)
! 				{
! 					num = DirectFunctionCall3(numeric_in, CStringGetDatum(value), 0, -1);
! 					es->json_cxt.value(&es->json_cxt, num, JSONTYPE_NUMERIC,
! 									   NUMERICOID, 1702 /* numeric_out */);
! 					pfree(DatumGetPointer(num));
! 				}
! 				else
! 					json_out_cstring_value(&es->json_cxt, value);
! 			}
  			break;
  
  		case EXPLAIN_FORMAT_YAML:
*************** ExplainOpenGroup(const char *objtype, co
*** 2913,2935 ****
  			break;
  
  		case EXPLAIN_FORMAT_JSON:
- 			ExplainJSONLineEnding(es);
- 			appendStringInfoSpaces(es->str, 2 * es->indent);
  			if (labelname)
! 			{
! 				escape_json(es->str, labelname);
! 				appendStringInfoString(es->str, ": ");
! 			}
! 			appendStringInfoChar(es->str, labeled ? '{' : '[');
  
! 			/*
! 			 * In JSON format, the grouping_stack is an integer list.  0 means
! 			 * we've emitted nothing at this grouping level, 1 means we've
! 			 * emitted something (and so the next item needs a comma). See
! 			 * ExplainJSONLineEnding().
! 			 */
! 			es->grouping_stack = lcons_int(0, es->grouping_stack);
! 			es->indent++;
  			break;
  
  		case EXPLAIN_FORMAT_YAML:
--- 2919,2931 ----
  			break;
  
  		case EXPLAIN_FORMAT_JSON:
  			if (labelname)
! 				json_out_cstring_key(&es->json_cxt, labelname);
  
! 			if (labeled)
! 				es->json_cxt.object_start(&es->json_cxt);
! 			else
! 				es->json_cxt.array_start(&es->json_cxt);
  			break;
  
  		case EXPLAIN_FORMAT_YAML:
*************** ExplainCloseGroup(const char *objtype, c
*** 2976,2986 ****
  			break;
  
  		case EXPLAIN_FORMAT_JSON:
! 			es->indent--;
! 			appendStringInfoChar(es->str, '\n');
! 			appendStringInfoSpaces(es->str, 2 * es->indent);
! 			appendStringInfoChar(es->str, labeled ? '}' : ']');
! 			es->grouping_stack = list_delete_first(es->grouping_stack);
  			break;
  
  		case EXPLAIN_FORMAT_YAML:
--- 2972,2981 ----
  			break;
  
  		case EXPLAIN_FORMAT_JSON:
! 			if (labeled)
! 				es->json_cxt.object_end(&es->json_cxt);
! 			else
! 				es->json_cxt.array_end(&es->json_cxt);
  			break;
  
  		case EXPLAIN_FORMAT_YAML:
*************** ExplainDummyGroup(const char *objtype, c
*** 3010,3023 ****
  			break;
  
  		case EXPLAIN_FORMAT_JSON:
- 			ExplainJSONLineEnding(es);
- 			appendStringInfoSpaces(es->str, 2 * es->indent);
  			if (labelname)
! 			{
! 				escape_json(es->str, labelname);
! 				appendStringInfoString(es->str, ": ");
! 			}
! 			escape_json(es->str, objtype);
  			break;
  
  		case EXPLAIN_FORMAT_YAML:
--- 3005,3014 ----
  			break;
  
  		case EXPLAIN_FORMAT_JSON:
  			if (labelname)
! 				json_out_cstring_key(&es->json_cxt, labelname);
! 
! 			json_out_cstring_value(&es->json_cxt, objtype);
  			break;
  
  		case EXPLAIN_FORMAT_YAML:
*************** ExplainBeginOutput(ExplainState *es)
*** 3059,3067 ****
  
  		case EXPLAIN_FORMAT_JSON:
  			/* top-level structure is an array of plans */
! 			appendStringInfoChar(es->str, '[');
! 			es->grouping_stack = lcons_int(0, es->grouping_stack);
! 			es->indent++;
  			break;
  
  		case EXPLAIN_FORMAT_YAML:
--- 3050,3056 ----
  
  		case EXPLAIN_FORMAT_JSON:
  			/* top-level structure is an array of plans */
! 			es->json_cxt.array_start(&es->json_cxt);
  			break;
  
  		case EXPLAIN_FORMAT_YAML:
*************** ExplainEndOutput(ExplainState *es)
*** 3088,3096 ****
  			break;
  
  		case EXPLAIN_FORMAT_JSON:
! 			es->indent--;
! 			appendStringInfoString(es->str, "\n]");
! 			es->grouping_stack = list_delete_first(es->grouping_stack);
  			break;
  
  		case EXPLAIN_FORMAT_YAML:
--- 3077,3083 ----
  			break;
  
  		case EXPLAIN_FORMAT_JSON:
! 			es->json_cxt.array_end(&es->json_cxt);
  			break;
  
  		case EXPLAIN_FORMAT_YAML:
*************** ExplainXMLTag(const char *tagname, int f
*** 3150,3173 ****
  }
  
  /*
-  * Emit a JSON line ending.
-  *
-  * JSON requires a comma after each property but the last.  To facilitate this,
-  * in JSON format, the text emitted for each property begins just prior to the
-  * preceding line-break (and comma, if applicable).
-  */
- static void
- ExplainJSONLineEnding(ExplainState *es)
- {
- 	Assert(es->format == EXPLAIN_FORMAT_JSON);
- 	if (linitial_int(es->grouping_stack) != 0)
- 		appendStringInfoChar(es->str, ',');
- 	else
- 		linitial_int(es->grouping_stack) = 1;
- 	appendStringInfoChar(es->str, '\n');
- }
- 
- /*
   * Indent a YAML line.
   *
   * YAML lines are ordinarily indented by two spaces per indentation level.
--- 3137,3142 ----
diff --git a/src/backend/utils/adt/json.c b/src/backend/utils/adt/json.c
new file mode 100644
index 26d3843..48dcc6b
*** a/src/backend/utils/adt/json.c
--- b/src/backend/utils/adt/json.c
*************** typedef enum					/* contexts of JSON par
*** 53,73 ****
  	JSON_PARSE_END				/* saw the end of a document, expect nothing */
  } JsonParseContext;
  
- typedef enum					/* type categories for datum_to_json */
- {
- 	JSONTYPE_NULL,				/* null, so we didn't bother to identify */
- 	JSONTYPE_BOOL,				/* boolean (built-in types only) */
- 	JSONTYPE_NUMERIC,			/* numeric (ditto) */
- 	JSONTYPE_DATE,				/* we use special formatting for datetimes */
- 	JSONTYPE_TIMESTAMP,
- 	JSONTYPE_TIMESTAMPTZ,
- 	JSONTYPE_JSON,				/* JSON itself (and JSONB) */
- 	JSONTYPE_ARRAY,				/* array */
- 	JSONTYPE_COMPOSITE,			/* composite */
- 	JSONTYPE_CAST,				/* something with an explicit cast to JSON */
- 	JSONTYPE_OTHER				/* all else */
- } JsonTypeCategory;
- 
  static inline void json_lex(JsonLexContext *lex);
  static inline void json_lex_string(JsonLexContext *lex);
  static inline void json_lex_number(JsonLexContext *lex, char *s, bool *num_err);
--- 53,58 ----
*************** static void report_parse_error(JsonParse
*** 80,102 ****
  static void report_invalid_token(JsonLexContext *lex);
  static int	report_json_context(JsonLexContext *lex);
  static char *extract_mb_char(char *s);
! static void composite_to_json(Datum composite, StringInfo result,
! 				  bool use_line_feeds);
! static void array_dim_to_json(StringInfo result, int dim, int ndims, int *dims,
  				  Datum *vals, bool *nulls, int *valcount,
! 				  JsonTypeCategory tcategory, Oid outfuncoid,
! 				  bool use_line_feeds);
! static void array_to_json_internal(Datum array, StringInfo result,
! 					   bool use_line_feeds);
  static void json_categorize_type(Oid typoid,
  					 JsonTypeCategory *tcategory,
  					 Oid *outfuncoid);
! static void datum_to_json(Datum val, bool is_null, StringInfo result,
! 			  JsonTypeCategory tcategory, Oid outfuncoid,
! 			  bool key_scalar);
! static void add_json(Datum val, bool is_null, StringInfo result,
  		 Oid val_type, bool key_scalar);
- static text *catenate_stringinfo_string(StringInfo buffer, const char *addon);
  
  /* the null action object used for pure validation */
  static JsonSemAction nullSemAction =
--- 65,85 ----
  static void report_invalid_token(JsonLexContext *lex);
  static int	report_json_context(JsonLexContext *lex);
  static char *extract_mb_char(char *s);
! 
! static void json_out_comma(JsonOutContext *out);
! static void json_out_indent(JsonOutContext *out);
! static void json_out_internal(JsonOutContext *out, Datum val,
! 				  JsonTypeCategory tcategory, Oid typoid, Oid outfuncoid, bool escape);
! 
! static void array_dim_to_json(JsonOutContext *out, int dim, int ndims, int *dims,
  				  Datum *vals, bool *nulls, int *valcount,
! 				  JsonTypeCategory tcategory, Oid elemtypoid, Oid outfuncoid);
! static void array_to_json_internal(Datum array, JsonOutContext *out);
  static void json_categorize_type(Oid typoid,
  					 JsonTypeCategory *tcategory,
  					 Oid *outfuncoid);
! static void add_json(Datum val, bool is_null, JsonOutContext *out,
  		 Oid val_type, bool key_scalar);
  
  /* the null action object used for pure validation */
  static JsonSemAction nullSemAction =
*************** json_categorize_type(Oid typoid,
*** 1377,1399 ****
   * If key_scalar is true, the value is being printed as a key, so insist
   * it's of an acceptable type, and force it to be quoted.
   */
! static void
! datum_to_json(Datum val, bool is_null, StringInfo result,
! 			  JsonTypeCategory tcategory, Oid outfuncoid,
  			  bool key_scalar)
  {
- 	char	   *outputstr;
- 	text	   *jsontext;
- 
  	/* callers are expected to ensure that null keys are not passed in */
  	Assert(!(key_scalar && is_null));
  
- 	if (is_null)
- 	{
- 		appendStringInfoString(result, "null");
- 		return;
- 	}
- 
  	if (key_scalar &&
  		(tcategory == JSONTYPE_ARRAY ||
  		 tcategory == JSONTYPE_COMPOSITE ||
--- 1360,1373 ----
   * If key_scalar is true, the value is being printed as a key, so insist
   * it's of an acceptable type, and force it to be quoted.
   */
! void
! datum_to_json(Datum val, bool is_null, JsonOutContext *out,
! 			  JsonTypeCategory tcategory, Oid typoid, Oid outfuncoid,
  			  bool key_scalar)
  {
  	/* callers are expected to ensure that null keys are not passed in */
  	Assert(!(key_scalar && is_null));
  
  	if (key_scalar &&
  		(tcategory == JSONTYPE_ARRAY ||
  		 tcategory == JSONTYPE_COMPOSITE ||
*************** datum_to_json(Datum val, bool is_null, S
*** 1406,1419 ****
  	switch (tcategory)
  	{
  		case JSONTYPE_ARRAY:
! 			array_to_json_internal(val, result, false);
  			break;
  		case JSONTYPE_COMPOSITE:
! 			composite_to_json(val, result, false);
  			break;
  		case JSONTYPE_BOOL:
  			outputstr = DatumGetBool(val) ? "true" : "false";
! 			if (key_scalar)
  				escape_json(result, outputstr);
  			else
  				appendStringInfoString(result, outputstr);
--- 1380,1591 ----
  	switch (tcategory)
  	{
  		case JSONTYPE_ARRAY:
! 			array_to_json_internal(val, out);
  			break;
  		case JSONTYPE_COMPOSITE:
! 			composite_to_json(val, out);
! 			break;
! 		default:
! 			if (key_scalar)
! 				out->key(out, val, tcategory, typoid, outfuncoid);
! 			else
! 				out->value(out, val, tcategory, typoid, outfuncoid);
! 			break;
! 	}
! }
! 
! void
! json_out_init_context(JsonOutContext *out, int flags)
! {
! 	out->object_start = json_out_object_start;
! 	out->object_end = json_out_object_end;
! 	out->array_start = json_out_array_start;
! 	out->array_end = json_out_array_end;
! 	out->before_key = json_out_before_key;
! 	out->key = json_out_key;
! 	out->after_key = json_out_after_key;
! 	out->before_value = json_out_before_value;
! 	out->value = json_out_value;
! 	out->after_value = json_out_after_value;
! 
! 	initStringInfo(&out->result);
! 	out->flags = flags;
! 	out->indent_spaces = 0;
! 	out->need_comma = false;
! 	out->need_colon = false;
! 	out->depth = 0;
! 
! 	out->agg_tcategory = JSONTYPE_OTHER;
! 	out->agg_outfuncoid = InvalidOid;
! }
! 
! void
! json_out_object_start(JsonOutContext *out)
! {
! 	out->before_value(out);
! 
! 	appendStringInfoChar(&out->result, '{');
! 	if (out->indent_spaces > 0)
! 		appendStringInfoChar(&out->result, '\n');
! 
! 	out->depth++;
! }
! 
! void
! json_out_object_end(JsonOutContext *out)
! {
! 	if (out->indent_spaces > 0)
! 		appendStringInfoChar(&out->result, '\n');
! 
! 	out->depth--;
! 
! 	json_out_indent(out);
! 	appendStringInfoChar(&out->result, '}');
! 
! 	out->after_value(out);
! }
! 
! void
! json_out_array_start(JsonOutContext *out)
! {
! 	out->before_value(out);
! 
! 	appendStringInfoChar(&out->result, '[');
! 	if (out->indent_spaces > 0)
! 		appendStringInfoChar(&out->result, '\n');
! 
! 	out->depth++;
! }
! 
! void
! json_out_array_end(JsonOutContext *out)
! {
! 	if (out->indent_spaces > 0)
! 		appendStringInfoChar(&out->result, '\n');
! 
! 	out->depth--;
! 
! 	json_out_indent(out);
! 	appendStringInfoChar(&out->result, ']');
! 
! 	out->after_value(out);
! }
! 
! void
! json_out_before_key(JsonOutContext *out)
! {
! 	Assert(!out->need_colon);
! 
! 	json_out_comma(out);
! 	json_out_indent(out);
! }
! 
! void
! json_out_after_key(JsonOutContext *out)
! {
! 	out->need_colon = true;
! 	out->need_comma = false;
! }
! 
! void
! json_out_before_value(JsonOutContext *out)
! {
! 	json_out_comma(out);
! 
! 	if (out->need_colon)
! 	{
! 		out->need_colon = false;
! 		appendStringInfoChar(&out->result, ':');
! 
! 		if ((out->flags & JSON_OUT_USE_SPACES) != 0)
! 			appendStringInfoChar(&out->result, ' ');
! 	}
! 	else
! 		json_out_indent(out);
! }
! 
! void
! json_out_after_value(JsonOutContext *out)
! {
! 	out->need_comma = true;
! }
! 
! static void
! json_out_comma(JsonOutContext *out)
! {
! 	if (out->need_comma)
! 	{
! 		out->need_comma = false;
! 		appendStringInfoChar(&out->result, ',');
! 
! 		/* don't get into all the prettiness deep in the object structure,
! 		 * unless indentation was required */
! 		if (out->depth == 1 || out->indent_spaces > 0)
! 		{
! 			if ((out->flags & JSON_OUT_USE_SPACES) != 0 &&
! 				(out->flags & JSON_OUT_USE_LINE_FEEDS) == 0)
! 				appendStringInfoChar(&out->result, ' ');
! 
! 			if ((out->flags & JSON_OUT_USE_LINE_FEEDS) != 0)
! 			{
! 				appendStringInfoChar(&out->result, '\n');
! 
! 				/* this is only to support json_agg's current output style;
! 				 * but if indent is set we don't need the extra space */
! 				if (out->indent_spaces <= 0)
! 					appendStringInfoChar(&out->result, ' ');
! 			}
! 		}
! 	}
! }
! 
! static void
! json_out_indent(JsonOutContext *out)
! {
! 	if (out->indent_spaces > 0)
! 		appendStringInfoSpaces(&out->result, out->depth * out->indent_spaces);
! }
! 
! void
! json_out_key(JsonOutContext *out, Datum val, JsonTypeCategory tcategory,
! 			 Oid typoid, Oid outfuncoid)
! {
! 	out->before_key(out);
! 
! 	json_out_internal(out, val, tcategory, typoid, outfuncoid, true);
! 
! 	out->after_key(out);
! }
! 
! void
! json_out_value(JsonOutContext *out, Datum val, JsonTypeCategory tcategory,
! 			   Oid typoid, Oid outfuncoid)
! {
! 	out->before_value(out);
! 
! 	json_out_internal(out, val, tcategory, typoid, outfuncoid, false);
! 
! 	out->after_value(out);
! }
! 
! static void
! json_out_internal(JsonOutContext *out, Datum val, JsonTypeCategory tcategory,
! 				  Oid typoid, Oid outfuncoid, bool escape)
! {
! 	char	   *outputstr;
! 	text	   *jsontext;
! 	StringInfo	result = &out->result;
! 
! 	Assert(!(tcategory == JSONTYPE_ARRAY || tcategory == JSONTYPE_COMPOSITE));
! 
! 	switch (tcategory)
! 	{
! 		case JSONTYPE_NULL:
! 			appendStringInfoString(result, "null");
  			break;
  		case JSONTYPE_BOOL:
  			outputstr = DatumGetBool(val) ? "true" : "false";
! 			if (escape)
  				escape_json(result, outputstr);
  			else
  				appendStringInfoString(result, outputstr);
*************** datum_to_json(Datum val, bool is_null, S
*** 1425,1431 ****
  			 * Don't call escape_json for a non-key if it's a valid JSON
  			 * number.
  			 */
! 			if (!key_scalar && IsValidJsonNumber(outputstr, strlen(outputstr)))
  				appendStringInfoString(result, outputstr);
  			else
  				escape_json(result, outputstr);
--- 1597,1603 ----
  			 * Don't call escape_json for a non-key if it's a valid JSON
  			 * number.
  			 */
! 			if (!escape && IsValidJsonNumber(outputstr, strlen(outputstr)))
  				appendStringInfoString(result, outputstr);
  			else
  				escape_json(result, outputstr);
*************** datum_to_json(Datum val, bool is_null, S
*** 1520,1581 ****
  			pfree(jsontext);
  			break;
  		default:
! 			outputstr = OidOutputFunctionCall(outfuncoid, val);
! 			escape_json(result, outputstr);
! 			pfree(outputstr);
  			break;
  	}
  }
  
  /*
   * Process a single dimension of an array.
   * If it's the innermost dimension, output the values, otherwise call
   * ourselves recursively to process the next dimension.
   */
  static void
! array_dim_to_json(StringInfo result, int dim, int ndims, int *dims, Datum *vals,
  				  bool *nulls, int *valcount, JsonTypeCategory tcategory,
! 				  Oid outfuncoid, bool use_line_feeds)
  {
  	int			i;
- 	const char *sep;
  
  	Assert(dim < ndims);
  
! 	sep = use_line_feeds ? ",\n " : ",";
! 
! 	appendStringInfoChar(result, '[');
  
  	for (i = 1; i <= dims[dim]; i++)
  	{
- 		if (i > 1)
- 			appendStringInfoString(result, sep);
- 
  		if (dim + 1 == ndims)
  		{
! 			datum_to_json(vals[*valcount], nulls[*valcount], result, tcategory,
! 						  outfuncoid, false);
  			(*valcount)++;
  		}
  		else
! 		{
! 			/*
! 			 * Do we want line feeds on inner dimensions of arrays? For now
! 			 * we'll say no.
! 			 */
! 			array_dim_to_json(result, dim + 1, ndims, dims, vals, nulls,
! 							  valcount, tcategory, outfuncoid, false);
! 		}
  	}
  
! 	appendStringInfoChar(result, ']');
  }
  
  /*
   * Turn an array into JSON.
   */
  static void
! array_to_json_internal(Datum array, StringInfo result, bool use_line_feeds)
  {
  	ArrayType  *v = DatumGetArrayTypeP(array);
  	Oid			element_type = ARR_ELEMTYPE(v);
--- 1692,1760 ----
  			pfree(jsontext);
  			break;
  		default:
! 			if (typoid == CSTRINGOID)
! 				escape_json(result, DatumGetCString(val));
! 			else
! 			{
! 				outputstr = OidOutputFunctionCall(outfuncoid, val);
! 				escape_json(result, outputstr);
! 				pfree(outputstr);
! 			}
  			break;
  	}
  }
  
+ void
+ json_out_cstring_key(JsonOutContext *out, const char *str)
+ {
+ 	out->key(out, CStringGetDatum(str), JSONTYPE_OTHER,
+ 			 CSTRINGOID, 2293 /* cstring_out */);
+ }
+ 
+ void
+ json_out_cstring_value(JsonOutContext *out, const char *str)
+ {
+ 	out->value(out, CStringGetDatum(str), JSONTYPE_OTHER,
+ 			   CSTRINGOID, 2293 /* cstring_out */);
+ }
+ 
  /*
   * Process a single dimension of an array.
   * If it's the innermost dimension, output the values, otherwise call
   * ourselves recursively to process the next dimension.
   */
  static void
! array_dim_to_json(JsonOutContext *out, int dim, int ndims, int *dims, Datum *vals,
  				  bool *nulls, int *valcount, JsonTypeCategory tcategory,
! 				  Oid elemtypoid, Oid outfuncoid)
  {
  	int			i;
  
  	Assert(dim < ndims);
  
! 	out->array_start(out);
  
  	for (i = 1; i <= dims[dim]; i++)
  	{
  		if (dim + 1 == ndims)
  		{
! 			datum_to_json(vals[*valcount], nulls[*valcount], out, tcategory,
! 						  elemtypoid, outfuncoid, false);
  			(*valcount)++;
  		}
  		else
! 			array_dim_to_json(out, dim + 1, ndims, dims, vals, nulls,
! 							  valcount, tcategory, elemtypoid, outfuncoid);
  	}
  
! 	out->array_end(out);
  }
  
  /*
   * Turn an array into JSON.
   */
  static void
! array_to_json_internal(Datum array, JsonOutContext *out)
  {
  	ArrayType  *v = DatumGetArrayTypeP(array);
  	Oid			element_type = ARR_ELEMTYPE(v);
*************** array_to_json_internal(Datum array, Stri
*** 1597,1603 ****
  
  	if (nitems <= 0)
  	{
! 		appendStringInfoString(result, "[]");
  		return;
  	}
  
--- 1776,1783 ----
  
  	if (nitems <= 0)
  	{
! 		out->array_start(out);
! 		out->array_end(out);
  		return;
  	}
  
*************** array_to_json_internal(Datum array, Stri
*** 1611,1618 ****
  					  typalign, &elements, &nulls,
  					  &nitems);
  
! 	array_dim_to_json(result, 0, ndim, dim, elements, nulls, &count, tcategory,
! 					  outfuncoid, use_line_feeds);
  
  	pfree(elements);
  	pfree(nulls);
--- 1791,1798 ----
  					  typalign, &elements, &nulls,
  					  &nitems);
  
! 	array_dim_to_json(out, 0, ndim, dim, elements, nulls, &count, tcategory,
! 					  element_type, outfuncoid);
  
  	pfree(elements);
  	pfree(nulls);
*************** array_to_json_internal(Datum array, Stri
*** 1621,1628 ****
  /*
   * Turn a composite / record into JSON.
   */
! static void
! composite_to_json(Datum composite, StringInfo result, bool use_line_feeds)
  {
  	HeapTupleHeader td;
  	Oid			tupType;
--- 1801,1808 ----
  /*
   * Turn a composite / record into JSON.
   */
! void
! composite_to_json(Datum composite, JsonOutContext *out)
  {
  	HeapTupleHeader td;
  	Oid			tupType;
*************** composite_to_json(Datum composite, Strin
*** 1631,1640 ****
  	HeapTupleData tmptup,
  			   *tuple;
  	int			i;
- 	bool		needsep = false;
- 	const char *sep;
- 
- 	sep = use_line_feeds ? ",\n " : ",";
  
  	td = DatumGetHeapTupleHeader(composite);
  
--- 1811,1816 ----
*************** composite_to_json(Datum composite, Strin
*** 1648,1654 ****
  	tmptup.t_data = td;
  	tuple = &tmptup;
  
! 	appendStringInfoChar(result, '{');
  
  	for (i = 0; i < tupdesc->natts; i++)
  	{
--- 1824,1830 ----
  	tmptup.t_data = td;
  	tuple = &tmptup;
  
! 	out->object_start(out);
  
  	for (i = 0; i < tupdesc->natts; i++)
  	{
*************** composite_to_json(Datum composite, Strin
*** 1661,1676 ****
  		if (tupdesc->attrs[i]->attisdropped)
  			continue;
  
- 		if (needsep)
- 			appendStringInfoString(result, sep);
- 		needsep = true;
- 
  		attname = NameStr(tupdesc->attrs[i]->attname);
! 		escape_json(result, attname);
! 		appendStringInfoChar(result, ':');
  
  		val = heap_getattr(tuple, i + 1, tupdesc, &isnull);
- 
  		if (isnull)
  		{
  			tcategory = JSONTYPE_NULL;
--- 1837,1846 ----
  		if (tupdesc->attrs[i]->attisdropped)
  			continue;
  
  		attname = NameStr(tupdesc->attrs[i]->attname);
! 		json_out_cstring_key(out, attname);
  
  		val = heap_getattr(tuple, i + 1, tupdesc, &isnull);
  		if (isnull)
  		{
  			tcategory = JSONTYPE_NULL;
*************** composite_to_json(Datum composite, Strin
*** 1680,1689 ****
  			json_categorize_type(tupdesc->attrs[i]->atttypid,
  								 &tcategory, &outfuncoid);
  
! 		datum_to_json(val, isnull, result, tcategory, outfuncoid, false);
  	}
  
! 	appendStringInfoChar(result, '}');
  	ReleaseTupleDesc(tupdesc);
  }
  
--- 1850,1861 ----
  			json_categorize_type(tupdesc->attrs[i]->atttypid,
  								 &tcategory, &outfuncoid);
  
! 		datum_to_json(val, isnull, out, tcategory,
! 					  tupdesc->attrs[i]->atttypid, outfuncoid, false);
  	}
  
! 	out->object_end(out);
! 
  	ReleaseTupleDesc(tupdesc);
  }
  
*************** composite_to_json(Datum composite, Strin
*** 1695,1701 ****
   * lookups only once.
   */
  static void
! add_json(Datum val, bool is_null, StringInfo result,
  		 Oid val_type, bool key_scalar)
  {
  	JsonTypeCategory tcategory;
--- 1867,1873 ----
   * lookups only once.
   */
  static void
! add_json(Datum val, bool is_null, JsonOutContext *out,
  		 Oid val_type, bool key_scalar)
  {
  	JsonTypeCategory tcategory;
*************** add_json(Datum val, bool is_null, String
*** 1712,1721 ****
  		outfuncoid = InvalidOid;
  	}
  	else
! 		json_categorize_type(val_type,
! 							 &tcategory, &outfuncoid);
  
! 	datum_to_json(val, is_null, result, tcategory, outfuncoid, key_scalar);
  }
  
  /*
--- 1884,1892 ----
  		outfuncoid = InvalidOid;
  	}
  	else
! 		json_categorize_type(val_type, &tcategory, &outfuncoid);
  
! 	datum_to_json(val, is_null, out, tcategory, val_type, outfuncoid, key_scalar);
  }
  
  /*
*************** extern Datum
*** 1725,1737 ****
  array_to_json(PG_FUNCTION_ARGS)
  {
  	Datum		array = PG_GETARG_DATUM(0);
! 	StringInfo	result;
! 
! 	result = makeStringInfo();
  
! 	array_to_json_internal(array, result, false);
  
! 	PG_RETURN_TEXT_P(cstring_to_text_with_len(result->data, result->len));
  }
  
  /*
--- 1896,1907 ----
  array_to_json(PG_FUNCTION_ARGS)
  {
  	Datum		array = PG_GETARG_DATUM(0);
! 	JsonOutContext	out;
  
! 	json_out_init_context(&out, 0);
! 	array_to_json_internal(array, &out);
  
! 	PG_RETURN_TEXT_P(cstring_to_text_with_len(out.result.data, out.result.len));
  }
  
  /*
*************** array_to_json_pretty(PG_FUNCTION_ARGS)
*** 1742,1754 ****
  {
  	Datum		array = PG_GETARG_DATUM(0);
  	bool		use_line_feeds = PG_GETARG_BOOL(1);
! 	StringInfo	result;
! 
! 	result = makeStringInfo();
  
! 	array_to_json_internal(array, result, use_line_feeds);
  
! 	PG_RETURN_TEXT_P(cstring_to_text_with_len(result->data, result->len));
  }
  
  /*
--- 1912,1923 ----
  {
  	Datum		array = PG_GETARG_DATUM(0);
  	bool		use_line_feeds = PG_GETARG_BOOL(1);
! 	JsonOutContext	out;
  
! 	json_out_init_context(&out, use_line_feeds ? JSON_OUT_USE_LINE_FEEDS : 0);
! 	array_to_json_internal(array, &out);
  
! 	PG_RETURN_TEXT_P(cstring_to_text_with_len(out.result.data, out.result.len));
  }
  
  /*
*************** extern Datum
*** 1758,1770 ****
  row_to_json(PG_FUNCTION_ARGS)
  {
  	Datum		array = PG_GETARG_DATUM(0);
! 	StringInfo	result;
! 
! 	result = makeStringInfo();
  
! 	composite_to_json(array, result, false);
  
! 	PG_RETURN_TEXT_P(cstring_to_text_with_len(result->data, result->len));
  }
  
  /*
--- 1927,1938 ----
  row_to_json(PG_FUNCTION_ARGS)
  {
  	Datum		array = PG_GETARG_DATUM(0);
! 	JsonOutContext	out;
  
! 	json_out_init_context(&out, 0);
! 	composite_to_json(array, &out);
  
! 	PG_RETURN_TEXT_P(cstring_to_text_with_len(out.result.data, out.result.len));
  }
  
  /*
*************** row_to_json_pretty(PG_FUNCTION_ARGS)
*** 1775,1787 ****
  {
  	Datum		array = PG_GETARG_DATUM(0);
  	bool		use_line_feeds = PG_GETARG_BOOL(1);
! 	StringInfo	result;
! 
! 	result = makeStringInfo();
  
! 	composite_to_json(array, result, use_line_feeds);
  
! 	PG_RETURN_TEXT_P(cstring_to_text_with_len(result->data, result->len));
  }
  
  /*
--- 1943,1954 ----
  {
  	Datum		array = PG_GETARG_DATUM(0);
  	bool		use_line_feeds = PG_GETARG_BOOL(1);
! 	JsonOutContext	out;
  
! 	json_out_init_context(&out, use_line_feeds ? JSON_OUT_USE_LINE_FEEDS : 0);
! 	composite_to_json(array, &out);
  
! 	PG_RETURN_TEXT_P(cstring_to_text_with_len(out.result.data, out.result.len));
  }
  
  /*
*************** to_json(PG_FUNCTION_ARGS)
*** 1792,1800 ****
  {
  	Datum		val = PG_GETARG_DATUM(0);
  	Oid			val_type = get_fn_expr_argtype(fcinfo->flinfo, 0);
- 	StringInfo	result;
  	JsonTypeCategory tcategory;
  	Oid			outfuncoid;
  
  	if (val_type == InvalidOid)
  		ereport(ERROR,
--- 1959,1969 ----
  {
  	Datum		val = PG_GETARG_DATUM(0);
  	Oid			val_type = get_fn_expr_argtype(fcinfo->flinfo, 0);
  	JsonTypeCategory tcategory;
  	Oid			outfuncoid;
+ 	JsonOutContext	out;
+ 
+ 	json_out_init_context(&out, 0);
  
  	if (val_type == InvalidOid)
  		ereport(ERROR,
*************** to_json(PG_FUNCTION_ARGS)
*** 1804,1814 ****
  	json_categorize_type(val_type,
  						 &tcategory, &outfuncoid);
  
! 	result = makeStringInfo();
! 
! 	datum_to_json(val, false, result, tcategory, outfuncoid, false);
  
! 	PG_RETURN_TEXT_P(cstring_to_text_with_len(result->data, result->len));
  }
  
  /*
--- 1973,1981 ----
  	json_categorize_type(val_type,
  						 &tcategory, &outfuncoid);
  
! 	datum_to_json(val, false, &out, tcategory, val_type, outfuncoid, false);
  
! 	PG_RETURN_TEXT_P(cstring_to_text_with_len(out.result.data, out.result.len));
  }
  
  /*
*************** json_agg_transfn(PG_FUNCTION_ARGS)
*** 1822,1831 ****
  	Oid			val_type = get_fn_expr_argtype(fcinfo->flinfo, 1);
  	MemoryContext aggcontext,
  				oldcontext;
! 	StringInfo	state;
  	Datum		val;
- 	JsonTypeCategory tcategory;
- 	Oid			outfuncoid;
  
  	if (val_type == InvalidOid)
  		ereport(ERROR,
--- 1989,1996 ----
  	Oid			val_type = get_fn_expr_argtype(fcinfo->flinfo, 1);
  	MemoryContext aggcontext,
  				oldcontext;
! 	JsonOutContext	*out = NULL;
  	Datum		val;
  
  	if (val_type == InvalidOid)
  		ereport(ERROR,
*************** json_agg_transfn(PG_FUNCTION_ARGS)
*** 1847,1891 ****
  		 * use the right context to enlarge the object if necessary.
  		 */
  		oldcontext = MemoryContextSwitchTo(aggcontext);
! 		state = makeStringInfo();
  		MemoryContextSwitchTo(oldcontext);
  
! 		appendStringInfoChar(state, '[');
  	}
  	else
! 	{
! 		state = (StringInfo) PG_GETARG_POINTER(0);
! 		appendStringInfoString(state, ", ");
! 	}
  
  	/* fast path for NULLs */
  	if (PG_ARGISNULL(1))
  	{
! 		datum_to_json((Datum) 0, true, state, JSONTYPE_NULL, InvalidOid, false);
! 		PG_RETURN_POINTER(state);
  	}
  
  	val = PG_GETARG_DATUM(1);
  
! 	/* XXX we do this every time?? */
! 	json_categorize_type(val_type,
! 						 &tcategory, &outfuncoid);
! 
! 	/* add some whitespace if structured type and not first item */
! 	if (!PG_ARGISNULL(0) &&
! 		(tcategory == JSONTYPE_ARRAY || tcategory == JSONTYPE_COMPOSITE))
! 	{
! 		appendStringInfoString(state, "\n ");
! 	}
  
! 	datum_to_json(val, false, state, tcategory, outfuncoid, false);
  
  	/*
  	 * The transition type for array_agg() is declared to be "internal", which
  	 * is a pass-by-value type the same size as a pointer.  So we can safely
! 	 * pass the ArrayBuildState pointer through nodeAgg.c's machinations.
  	 */
! 	PG_RETURN_POINTER(state);
  }
  
  /*
--- 2012,2049 ----
  		 * use the right context to enlarge the object if necessary.
  		 */
  		oldcontext = MemoryContextSwitchTo(aggcontext);
! 		out = palloc(sizeof(JsonOutContext));
! 		json_out_init_context(out, JSON_OUT_USE_LINE_FEEDS);
  		MemoryContextSwitchTo(oldcontext);
  
! 		out->array_start(out);
  	}
  	else
! 		out = (JsonOutContext *) PG_GETARG_POINTER(0);
  
  	/* fast path for NULLs */
  	if (PG_ARGISNULL(1))
  	{
! 		datum_to_json((Datum) 0, true, out,
! 					  JSONTYPE_NULL, InvalidOid, InvalidOid, false);
! 		PG_RETURN_POINTER(out);
  	}
  
  	val = PG_GETARG_DATUM(1);
  
! 	if (out->agg_outfuncoid == InvalidOid)
! 		json_categorize_type(val_type,
! 							 &out->agg_tcategory, &out->agg_outfuncoid);
  
! 	datum_to_json(val, false, out, out->agg_tcategory,
! 				  val_type, out->agg_outfuncoid, false);
  
  	/*
  	 * The transition type for array_agg() is declared to be "internal", which
  	 * is a pass-by-value type the same size as a pointer.  So we can safely
! 	 * pass the ArrayBuildOut pointer through nodeAgg.c's machinations.
  	 */
! 	PG_RETURN_POINTER(out);
  }
  
  /*
*************** json_agg_transfn(PG_FUNCTION_ARGS)
*** 1894,1912 ****
  Datum
  json_agg_finalfn(PG_FUNCTION_ARGS)
  {
! 	StringInfo	state;
  
  	/* cannot be called directly because of internal-type argument */
  	Assert(AggCheckCallContext(fcinfo, NULL));
  
! 	state = PG_ARGISNULL(0) ? NULL : (StringInfo) PG_GETARG_POINTER(0);
  
  	/* NULL result for no rows in, as is standard with aggregates */
! 	if (state == NULL)
  		PG_RETURN_NULL();
  
! 	/* Else return state with appropriate array terminator added */
! 	PG_RETURN_TEXT_P(catenate_stringinfo_string(state, "]"));
  }
  
  /*
--- 2052,2071 ----
  Datum
  json_agg_finalfn(PG_FUNCTION_ARGS)
  {
! 	JsonOutContext	*out;
  
  	/* cannot be called directly because of internal-type argument */
  	Assert(AggCheckCallContext(fcinfo, NULL));
  
! 	out = PG_ARGISNULL(0) ? NULL : (JsonOutContext *) PG_GETARG_POINTER(0);
  
  	/* NULL result for no rows in, as is standard with aggregates */
! 	if (out == NULL)
  		PG_RETURN_NULL();
  
! 	out->array_end(out);
! 
! 	PG_RETURN_TEXT_P(cstring_to_text_with_len(out->result.data, out->result.len));
  }
  
  /*
*************** json_object_agg_transfn(PG_FUNCTION_ARGS
*** 1920,1926 ****
  	Oid			val_type;
  	MemoryContext aggcontext,
  				oldcontext;
! 	StringInfo	state;
  	Datum		arg;
  
  	if (!AggCheckCallContext(fcinfo, &aggcontext))
--- 2079,2085 ----
  	Oid			val_type;
  	MemoryContext aggcontext,
  				oldcontext;
! 	JsonOutContext	*out = NULL;
  	Datum		arg;
  
  	if (!AggCheckCallContext(fcinfo, &aggcontext))
*************** json_object_agg_transfn(PG_FUNCTION_ARGS
*** 1938,1953 ****
  		 * use the right context to enlarge the object if necessary.
  		 */
  		oldcontext = MemoryContextSwitchTo(aggcontext);
! 		state = makeStringInfo();
  		MemoryContextSwitchTo(oldcontext);
  
! 		appendStringInfoString(state, "{ ");
  	}
  	else
! 	{
! 		state = (StringInfo) PG_GETARG_POINTER(0);
! 		appendStringInfoString(state, ", ");
! 	}
  
  	/*
  	 * Note: since json_object_agg() is declared as taking type "any", the
--- 2097,2110 ----
  		 * use the right context to enlarge the object if necessary.
  		 */
  		oldcontext = MemoryContextSwitchTo(aggcontext);
! 		out = palloc(sizeof(JsonOutContext));
! 		json_out_init_context(out, JSON_OUT_USE_SPACES);
  		MemoryContextSwitchTo(oldcontext);
  
! 		out->object_start(out);
  	}
  	else
! 		out = (JsonOutContext *) PG_GETARG_POINTER(0);
  
  	/*
  	 * Note: since json_object_agg() is declared as taking type "any", the
*************** json_object_agg_transfn(PG_FUNCTION_ARGS
*** 1970,1978 ****
  
  	arg = PG_GETARG_DATUM(1);
  
! 	add_json(arg, false, state, val_type, true);
! 
! 	appendStringInfoString(state, " : ");
  
  	val_type = get_fn_expr_argtype(fcinfo->flinfo, 2);
  
--- 2127,2133 ----
  
  	arg = PG_GETARG_DATUM(1);
  
! 	add_json(arg, false, out, val_type, true);
  
  	val_type = get_fn_expr_argtype(fcinfo->flinfo, 2);
  
*************** json_object_agg_transfn(PG_FUNCTION_ARGS
*** 1986,1994 ****
  	else
  		arg = PG_GETARG_DATUM(2);
  
! 	add_json(arg, PG_ARGISNULL(2), state, val_type, false);
  
! 	PG_RETURN_POINTER(state);
  }
  
  /*
--- 2141,2149 ----
  	else
  		arg = PG_GETARG_DATUM(2);
  
! 	add_json(arg, PG_ARGISNULL(2), out, val_type, false);
  
! 	PG_RETURN_POINTER(out);
  }
  
  /*
*************** json_object_agg_transfn(PG_FUNCTION_ARGS
*** 1997,2035 ****
  Datum
  json_object_agg_finalfn(PG_FUNCTION_ARGS)
  {
! 	StringInfo	state;
  
  	/* cannot be called directly because of internal-type argument */
  	Assert(AggCheckCallContext(fcinfo, NULL));
  
! 	state = PG_ARGISNULL(0) ? NULL : (StringInfo) PG_GETARG_POINTER(0);
  
  	/* NULL result for no rows in, as is standard with aggregates */
! 	if (state == NULL)
  		PG_RETURN_NULL();
  
! 	/* Else return state with appropriate object terminator added */
! 	PG_RETURN_TEXT_P(catenate_stringinfo_string(state, " }"));
! }
! 
! /*
!  * Helper function for aggregates: return given StringInfo's contents plus
!  * specified trailing string, as a text datum.  We need this because aggregate
!  * final functions are not allowed to modify the aggregate state.
!  */
! static text *
! catenate_stringinfo_string(StringInfo buffer, const char *addon)
! {
! 	/* custom version of cstring_to_text_with_len */
! 	int			buflen = buffer->len;
! 	int			addlen = strlen(addon);
! 	text	   *result = (text *) palloc(buflen + addlen + VARHDRSZ);
! 
! 	SET_VARSIZE(result, buflen + addlen + VARHDRSZ);
! 	memcpy(VARDATA(result), buffer->data, buflen);
! 	memcpy(VARDATA(result) + buflen, addon, addlen);
  
! 	return result;
  }
  
  /*
--- 2152,2171 ----
  Datum
  json_object_agg_finalfn(PG_FUNCTION_ARGS)
  {
! 	JsonOutContext	*out;
  
  	/* cannot be called directly because of internal-type argument */
  	Assert(AggCheckCallContext(fcinfo, NULL));
  
! 	out = PG_ARGISNULL(0) ? NULL : (JsonOutContext *) PG_GETARG_POINTER(0);
  
  	/* NULL result for no rows in, as is standard with aggregates */
! 	if (out == NULL)
  		PG_RETURN_NULL();
  
! 	out->object_end(out);
  
! 	PG_RETURN_TEXT_P(cstring_to_text_with_len(out->result.data, out->result.len));
  }
  
  /*
*************** json_build_object(PG_FUNCTION_ARGS)
*** 2041,2048 ****
  	int			nargs = PG_NARGS();
  	int			i;
  	Datum		arg;
! 	const char *sep = "";
! 	StringInfo	result;
  	Oid			val_type;
  
  	if (nargs % 2 != 0)
--- 2177,2183 ----
  	int			nargs = PG_NARGS();
  	int			i;
  	Datum		arg;
! 	JsonOutContext	out;
  	Oid			val_type;
  
  	if (nargs % 2 != 0)
*************** json_build_object(PG_FUNCTION_ARGS)
*** 2051,2059 ****
  				 errmsg("argument list must have even number of elements"),
  				 errhint("The arguments of json_build_object() must consist of alternating keys and values.")));
  
! 	result = makeStringInfo();
! 
! 	appendStringInfoChar(result, '{');
  
  	for (i = 0; i < nargs; i += 2)
  	{
--- 2186,2193 ----
  				 errmsg("argument list must have even number of elements"),
  				 errhint("The arguments of json_build_object() must consist of alternating keys and values.")));
  
! 	json_out_init_context(&out, JSON_OUT_USE_SPACES);
! 	out.object_start(&out);
  
  	for (i = 0; i < nargs; i += 2)
  	{
*************** json_build_object(PG_FUNCTION_ARGS)
*** 2064,2071 ****
  		 * here as type UNKNOWN, which fortunately does not matter to us,
  		 * since unknownout() works fine.
  		 */
- 		appendStringInfoString(result, sep);
- 		sep = ", ";
  
  		/* process key */
  		val_type = get_fn_expr_argtype(fcinfo->flinfo, i);
--- 2198,2203 ----
*************** json_build_object(PG_FUNCTION_ARGS)
*** 2084,2092 ****
  
  		arg = PG_GETARG_DATUM(i);
  
! 		add_json(arg, false, result, val_type, true);
! 
! 		appendStringInfoString(result, " : ");
  
  		/* process value */
  		val_type = get_fn_expr_argtype(fcinfo->flinfo, i + 1);
--- 2216,2222 ----
  
  		arg = PG_GETARG_DATUM(i);
  
! 		add_json(arg, false, &out, val_type, true);
  
  		/* process value */
  		val_type = get_fn_expr_argtype(fcinfo->flinfo, i + 1);
*************** json_build_object(PG_FUNCTION_ARGS)
*** 2102,2113 ****
  		else
  			arg = PG_GETARG_DATUM(i + 1);
  
! 		add_json(arg, PG_ARGISNULL(i + 1), result, val_type, false);
  	}
  
! 	appendStringInfoChar(result, '}');
  
! 	PG_RETURN_TEXT_P(cstring_to_text_with_len(result->data, result->len));
  }
  
  /*
--- 2232,2243 ----
  		else
  			arg = PG_GETARG_DATUM(i + 1);
  
! 		add_json(arg, PG_ARGISNULL(i + 1), &out, val_type, false);
  	}
  
! 	out.object_end(&out);
  
! 	PG_RETURN_TEXT_P(cstring_to_text_with_len(out.result.data, out.result.len));
  }
  
  /*
*************** json_build_array(PG_FUNCTION_ARGS)
*** 2128,2140 ****
  	int			nargs = PG_NARGS();
  	int			i;
  	Datum		arg;
! 	const char *sep = "";
! 	StringInfo	result;
  	Oid			val_type;
  
! 	result = makeStringInfo();
! 
! 	appendStringInfoChar(result, '[');
  
  	for (i = 0; i < nargs; i++)
  	{
--- 2258,2268 ----
  	int			nargs = PG_NARGS();
  	int			i;
  	Datum		arg;
! 	JsonOutContext	out;
  	Oid			val_type;
  
! 	json_out_init_context(&out, JSON_OUT_USE_SPACES);
! 	out.array_start(&out);
  
  	for (i = 0; i < nargs; i++)
  	{
*************** json_build_array(PG_FUNCTION_ARGS)
*** 2145,2153 ****
  		 * here as type UNKNOWN, which fortunately does not matter to us,
  		 * since unknownout() works fine.
  		 */
- 		appendStringInfoString(result, sep);
- 		sep = ", ";
- 
  		val_type = get_fn_expr_argtype(fcinfo->flinfo, i);
  
  		if (val_type == InvalidOid)
--- 2273,2278 ----
*************** json_build_array(PG_FUNCTION_ARGS)
*** 2161,2172 ****
  		else
  			arg = PG_GETARG_DATUM(i);
  
! 		add_json(arg, PG_ARGISNULL(i), result, val_type, false);
  	}
  
! 	appendStringInfoChar(result, ']');
  
! 	PG_RETURN_TEXT_P(cstring_to_text_with_len(result->data, result->len));
  }
  
  /*
--- 2286,2297 ----
  		else
  			arg = PG_GETARG_DATUM(i);
  
! 		add_json(arg, PG_ARGISNULL(i), &out, val_type, false);
  	}
  
! 	out.array_end(&out);
  
! 	PG_RETURN_TEXT_P(cstring_to_text_with_len(out.result.data, out.result.len));
  }
  
  /*
*************** json_object(PG_FUNCTION_ARGS)
*** 2189,2202 ****
  {
  	ArrayType  *in_array = PG_GETARG_ARRAYTYPE_P(0);
  	int			ndims = ARR_NDIM(in_array);
! 	StringInfoData result;
  	Datum	   *in_datums;
  	bool	   *in_nulls;
  	int			in_count,
  				count,
  				i;
- 	text	   *rval;
- 	char	   *v;
  
  	switch (ndims)
  	{
--- 2314,2325 ----
  {
  	ArrayType  *in_array = PG_GETARG_ARRAYTYPE_P(0);
  	int			ndims = ARR_NDIM(in_array);
! 	JsonOutContext	out;
  	Datum	   *in_datums;
  	bool	   *in_nulls;
  	int			in_count,
  				count,
  				i;
  
  	switch (ndims)
  	{
*************** json_object(PG_FUNCTION_ARGS)
*** 2230,2238 ****
  
  	count = in_count / 2;
  
! 	initStringInfo(&result);
! 
! 	appendStringInfoChar(&result, '{');
  
  	for (i = 0; i < count; ++i)
  	{
--- 2353,2360 ----
  
  	count = in_count / 2;
  
! 	json_out_init_context(&out, JSON_OUT_USE_SPACES);
! 	out.object_start(&out);
  
  	for (i = 0; i < count; ++i)
  	{
*************** json_object(PG_FUNCTION_ARGS)
*** 2241,2272 ****
  					(errcode(ERRCODE_NULL_VALUE_NOT_ALLOWED),
  					 errmsg("null value not allowed for object key")));
  
! 		v = TextDatumGetCString(in_datums[i * 2]);
! 		if (i > 0)
! 			appendStringInfoString(&result, ", ");
! 		escape_json(&result, v);
! 		appendStringInfoString(&result, " : ");
! 		pfree(v);
  		if (in_nulls[i * 2 + 1])
! 			appendStringInfoString(&result, "null");
  		else
! 		{
! 			v = TextDatumGetCString(in_datums[i * 2 + 1]);
! 			escape_json(&result, v);
! 			pfree(v);
! 		}
  	}
  
! 	appendStringInfoChar(&result, '}');
  
  	pfree(in_datums);
  	pfree(in_nulls);
  
! 	rval = cstring_to_text_with_len(result.data, result.len);
! 	pfree(result.data);
! 
! 	PG_RETURN_TEXT_P(rval);
! 
  }
  
  /*
--- 2363,2385 ----
  					(errcode(ERRCODE_NULL_VALUE_NOT_ALLOWED),
  					 errmsg("null value not allowed for object key")));
  
! 		out.key(&out, in_datums[i * 2], JSONTYPE_OTHER,
! 				TEXTOID, 47 /* textout */);
! 
  		if (in_nulls[i * 2 + 1])
! 			out.value(&out, (Datum) 0, JSONTYPE_NULL,
! 					  InvalidOid, InvalidOid);
  		else
! 			out.value(&out, in_datums[i * 2 + 1], JSONTYPE_OTHER,
! 					  TEXTOID, 47 /* textout */);
  	}
  
! 	out.object_end(&out);
  
  	pfree(in_datums);
  	pfree(in_nulls);
  
! 	PG_RETURN_TEXT_P(cstring_to_text_with_len(out.result.data, out.result.len));
  }
  
  /*
*************** json_object_two_arg(PG_FUNCTION_ARGS)
*** 2282,2288 ****
  	ArrayType  *val_array = PG_GETARG_ARRAYTYPE_P(1);
  	int			nkdims = ARR_NDIM(key_array);
  	int			nvdims = ARR_NDIM(val_array);
! 	StringInfoData result;
  	Datum	   *key_datums,
  			   *val_datums;
  	bool	   *key_nulls,
--- 2395,2401 ----
  	ArrayType  *val_array = PG_GETARG_ARRAYTYPE_P(1);
  	int			nkdims = ARR_NDIM(key_array);
  	int			nvdims = ARR_NDIM(val_array);
! 	JsonOutContext	out;
  	Datum	   *key_datums,
  			   *val_datums;
  	bool	   *key_nulls,
*************** json_object_two_arg(PG_FUNCTION_ARGS)
*** 2290,2297 ****
  	int			key_count,
  				val_count,
  				i;
- 	text	   *rval;
- 	char	   *v;
  
  	if (nkdims > 1 || nkdims != nvdims)
  		ereport(ERROR,
--- 2403,2408 ----
*************** json_object_two_arg(PG_FUNCTION_ARGS)
*** 2314,2322 ****
  				(errcode(ERRCODE_ARRAY_SUBSCRIPT_ERROR),
  				 errmsg("mismatched array dimensions")));
  
! 	initStringInfo(&result);
! 
! 	appendStringInfoChar(&result, '{');
  
  	for (i = 0; i < key_count; ++i)
  	{
--- 2425,2432 ----
  				(errcode(ERRCODE_ARRAY_SUBSCRIPT_ERROR),
  				 errmsg("mismatched array dimensions")));
  
! 	json_out_init_context(&out, JSON_OUT_USE_SPACES);
! 	out.object_start(&out);
  
  	for (i = 0; i < key_count; ++i)
  	{
*************** json_object_two_arg(PG_FUNCTION_ARGS)
*** 2325,2357 ****
  					(errcode(ERRCODE_NULL_VALUE_NOT_ALLOWED),
  					 errmsg("null value not allowed for object key")));
  
! 		v = TextDatumGetCString(key_datums[i]);
! 		if (i > 0)
! 			appendStringInfoString(&result, ", ");
! 		escape_json(&result, v);
! 		appendStringInfoString(&result, " : ");
! 		pfree(v);
  		if (val_nulls[i])
! 			appendStringInfoString(&result, "null");
  		else
! 		{
! 			v = TextDatumGetCString(val_datums[i]);
! 			escape_json(&result, v);
! 			pfree(v);
! 		}
  	}
  
! 	appendStringInfoChar(&result, '}');
  
  	pfree(key_datums);
  	pfree(key_nulls);
  	pfree(val_datums);
  	pfree(val_nulls);
  
! 	rval = cstring_to_text_with_len(result.data, result.len);
! 	pfree(result.data);
! 
! 	PG_RETURN_TEXT_P(rval);
  }
  
  
--- 2435,2459 ----
  					(errcode(ERRCODE_NULL_VALUE_NOT_ALLOWED),
  					 errmsg("null value not allowed for object key")));
  
! 		out.key(&out, key_datums[i], JSONTYPE_OTHER,
! 				TEXTOID, 47 /* textout */);
! 
  		if (val_nulls[i])
! 			out.value(&out, (Datum) 0, JSONTYPE_NULL,
! 					  InvalidOid, InvalidOid);
  		else
! 			out.value(&out, val_datums[i], JSONTYPE_OTHER,
! 					  TEXTOID, 47 /* textout */);
  	}
  
! 	out.object_end(&out);
  
  	pfree(key_datums);
  	pfree(key_nulls);
  	pfree(val_datums);
  	pfree(val_nulls);
  
! 	PG_RETURN_TEXT_P(cstring_to_text_with_len(out.result.data, out.result.len));
  }
  
  
diff --git a/src/include/commands/explain.h b/src/include/commands/explain.h
new file mode 100644
index 26fcc5b..f485d37
*** a/src/include/commands/explain.h
--- b/src/include/commands/explain.h
***************
*** 15,20 ****
--- 15,21 ----
  
  #include "executor/executor.h"
  #include "lib/stringinfo.h"
+ #include "utils/jsonapi.h"
  
  typedef enum ExplainFormat
  {
*************** typedef struct ExplainState
*** 42,47 ****
--- 43,49 ----
  	int			indent;			/* current indentation level */
  	List	   *grouping_stack; /* format-specific grouping state */
  	List	   *deparse_cxt;	/* context list for deparsing expressions */
+ 	JsonOutContext	json_cxt;	/* JSON output context */
  } ExplainState;
  
  /* Hook for plugins to get control in ExplainOneQuery() */
diff --git a/src/include/utils/jsonapi.h b/src/include/utils/jsonapi.h
new file mode 100644
index 296d20a..f31332c
*** a/src/include/utils/jsonapi.h
--- b/src/include/utils/jsonapi.h
*************** extern JsonLexContext *makeJsonLexContex
*** 124,127 ****
--- 124,202 ----
   */
  extern bool IsValidJsonNumber(const char *str, int len);
  
+ 
+ /*
+  * Generalized structures for producing JSON output.
+  */
+ typedef enum					/* type categories for datum_to_json */
+ {
+ 	JSONTYPE_NULL,				/* null, so we didn't bother to identify */
+ 	JSONTYPE_BOOL,				/* boolean (built-in types only) */
+ 	JSONTYPE_NUMERIC,			/* numeric (ditto) */
+ 	JSONTYPE_DATE,				/* we use special formatting for datetimes */
+ 	JSONTYPE_TIMESTAMP,
+ 	JSONTYPE_TIMESTAMPTZ,
+ 	JSONTYPE_JSON,				/* JSON itself (and JSONB) */
+ 	JSONTYPE_ARRAY,				/* array */
+ 	JSONTYPE_COMPOSITE,			/* composite */
+ 	JSONTYPE_CAST,				/* something with an explicit cast to JSON */
+ 	JSONTYPE_OTHER				/* all else */
+ } JsonTypeCategory;
+ 
+ struct JsonOutContext;
+ 
+ typedef void (*json_out_struct_action)(struct JsonOutContext *out);
+ typedef void (*json_out_key_value_action)(struct JsonOutContext *out,
+ 			  Datum key, JsonTypeCategory tcategory,
+ 			  Oid typoid, Oid outfuncoid);
+ 
+ #define JSON_OUT_USE_LINE_FEEDS		1
+ #define JSON_OUT_USE_SPACES			2
+ 
+ typedef struct JsonOutContext {
+ 	json_out_struct_action		object_start;
+ 	json_out_struct_action		object_end;
+ 	json_out_struct_action		array_start;
+ 	json_out_struct_action		array_end;
+ 	json_out_struct_action		before_key;
+ 	json_out_key_value_action	key;
+ 	json_out_struct_action		after_key;
+ 	json_out_struct_action		before_value;
+ 	json_out_key_value_action	value;
+ 	json_out_struct_action		after_value;
+ 
+ 	StringInfoData	result;
+ 	int		flags;
+ 	int		indent_spaces;
+ 	bool	need_comma;
+ 	bool	need_colon;
+ 	int		depth;
+ 	/* these are used in json_agg to cache the type information */
+ 	JsonTypeCategory	agg_tcategory;
+ 	Oid					agg_outfuncoid;
+ } JsonOutContext;
+ 
+ extern void json_out_init_context(JsonOutContext *out, int flags);
+ extern void json_out_object_start(JsonOutContext *out);
+ extern void json_out_object_end(JsonOutContext *out);
+ extern void json_out_array_start(JsonOutContext *out);
+ extern void json_out_array_end(JsonOutContext *out);
+ extern void json_out_before_key(JsonOutContext *out);
+ extern void json_out_key(JsonOutContext *out, Datum val,
+ 				JsonTypeCategory tcategory, Oid typoid, Oid outfuncoid);
+ extern void json_out_after_key(JsonOutContext *out);
+ extern void json_out_before_value(JsonOutContext *out);
+ extern void json_out_value(JsonOutContext *out, Datum val,
+ 				JsonTypeCategory tcategory, Oid typoid, Oid outfuncoid);
+ extern void json_out_after_value(JsonOutContext *out);
+ 
+ /* cstring helpers */
+ extern void json_out_cstring_key(JsonOutContext *out, const char *str);
+ extern void json_out_cstring_value(JsonOutContext *out, const char *str);
+ 
+ extern void composite_to_json(Datum composite, JsonOutContext *out);
+ extern void datum_to_json(Datum val, bool is_null, JsonOutContext *out,
+ 			  JsonTypeCategory tcategory, Oid typoid, Oid outfuncoid,
+ 			  bool key_scalar);
+ 
  #endif   /* JSONAPI_H */
diff --git a/src/test/regress/expected/insert_conflict.out b/src/test/regress/expected/insert_conflict.out
new file mode 100644
index eca9690..203f334
*** a/src/test/regress/expected/insert_conflict.out
--- b/src/test/regress/expected/insert_conflict.out
*************** explain (costs off, format json) insert
*** 190,196 ****
         "Relation Name": "insertconflicttest",                          +
         "Alias": "insertconflicttest",                                  +
         "Conflict Resolution": "UPDATE",                                +
!        "Conflict Arbiter Indexes": ["key_index"],                      +
         "Conflict Filter": "(insertconflicttest.fruit <> 'Lime'::text)",+
         "Plans": [                                                      +
           {                                                             +
--- 190,198 ----
         "Relation Name": "insertconflicttest",                          +
         "Alias": "insertconflicttest",                                  +
         "Conflict Resolution": "UPDATE",                                +
!        "Conflict Arbiter Indexes": [                                   +
!          "key_index"                                                   +
!        ],                                                              +
         "Conflict Filter": "(insertconflicttest.fruit <> 'Lime'::text)",+
         "Plans": [                                                      +
           {                                                             +
diff --git a/src/test/regress/expected/json.out b/src/test/regress/expected/json.out
new file mode 100644
index 3942c3b..fbc9b5b
*** a/src/test/regress/expected/json.out
--- b/src/test/regress/expected/json.out
*************** SELECT json_agg(q)
*** 457,476 ****
                 ROW(y.*,ARRAY[4,5,6])] AS z
           FROM generate_series(1,2) x,
                generate_series(4,5) y) q;
!                                json_agg                                
! -----------------------------------------------------------------------
!  [{"b":"a1","c":4,"z":[{"f1":1,"f2":[1,2,3]},{"f1":4,"f2":[4,5,6]}]}, +
!   {"b":"a1","c":5,"z":[{"f1":1,"f2":[1,2,3]},{"f1":5,"f2":[4,5,6]}]}, +
!   {"b":"a2","c":4,"z":[{"f1":2,"f2":[1,2,3]},{"f1":4,"f2":[4,5,6]}]}, +
    {"b":"a2","c":5,"z":[{"f1":2,"f2":[1,2,3]},{"f1":5,"f2":[4,5,6]}]}]
  (1 row)
  
  SELECT json_agg(q)
    FROM rows q;
!        json_agg        
! -----------------------
!  [{"x":1,"y":"txt1"}, +
!   {"x":2,"y":"txt2"}, +
    {"x":3,"y":"txt3"}]
  (1 row)
  
--- 457,476 ----
                 ROW(y.*,ARRAY[4,5,6])] AS z
           FROM generate_series(1,2) x,
                generate_series(4,5) y) q;
!                                json_agg                               
! ----------------------------------------------------------------------
!  [{"b":"a1","c":4,"z":[{"f1":1,"f2":[1,2,3]},{"f1":4,"f2":[4,5,6]}]},+
!   {"b":"a1","c":5,"z":[{"f1":1,"f2":[1,2,3]},{"f1":5,"f2":[4,5,6]}]},+
!   {"b":"a2","c":4,"z":[{"f1":2,"f2":[1,2,3]},{"f1":4,"f2":[4,5,6]}]},+
    {"b":"a2","c":5,"z":[{"f1":2,"f2":[1,2,3]},{"f1":5,"f2":[4,5,6]}]}]
  (1 row)
  
  SELECT json_agg(q)
    FROM rows q;
!        json_agg       
! ----------------------
!  [{"x":1,"y":"txt1"},+
!   {"x":2,"y":"txt2"},+
    {"x":3,"y":"txt3"}]
  (1 row)
  
*************** SELECT json_build_array('a',1,'b',1.2,'c
*** 1505,1522 ****
  (1 row)
  
  SELECT json_build_object('a',1,'b',1.2,'c',true,'d',null,'e',json '{"x": 3, "y": [1,2,3]}');
!                              json_build_object                              
! ----------------------------------------------------------------------------
!  {"a" : 1, "b" : 1.2, "c" : true, "d" : null, "e" : {"x": 3, "y": [1,2,3]}}
  (1 row)
  
  SELECT json_build_object(
         'a', json_build_object('b',false,'c',99),
         'd', json_build_object('e',array[9,8,7]::int[],
             'f', (select row_to_json(r) from ( select relkind, oid::regclass as name from pg_class where relname = 'pg_class') r)));
!                                         json_build_object                                        
! -------------------------------------------------------------------------------------------------
!  {"a" : {"b" : false, "c" : 99}, "d" : {"e" : [9,8,7], "f" : {"relkind":"r","name":"pg_class"}}}
  (1 row)
  
  -- empty objects/arrays
--- 1505,1522 ----
  (1 row)
  
  SELECT json_build_object('a',1,'b',1.2,'c',true,'d',null,'e',json '{"x": 3, "y": [1,2,3]}');
!                            json_build_object                           
! -----------------------------------------------------------------------
!  {"a": 1, "b": 1.2, "c": true, "d": null, "e": {"x": 3, "y": [1,2,3]}}
  (1 row)
  
  SELECT json_build_object(
         'a', json_build_object('b',false,'c',99),
         'd', json_build_object('e',array[9,8,7]::int[],
             'f', (select row_to_json(r) from ( select relkind, oid::regclass as name from pg_class where relname = 'pg_class') r)));
!                                      json_build_object                                     
! -------------------------------------------------------------------------------------------
!  {"a": {"b": false, "c": 99}, "d": {"e": [9,8,7], "f": {"relkind":"r","name":"pg_class"}}}
  (1 row)
  
  -- empty objects/arrays
*************** SELECT json_build_object();
*** 1536,1542 ****
  SELECT json_build_object(1,2);
   json_build_object 
  -------------------
!  {"1" : 2}
  (1 row)
  
  -- keys must be scalar and not null
--- 1536,1542 ----
  SELECT json_build_object(1,2);
   json_build_object 
  -------------------
!  {"1": 2}
  (1 row)
  
  -- keys must be scalar and not null
*************** INSERT INTO foo VALUES (847002,'t16','GE
*** 1555,1578 ****
  INSERT INTO foo VALUES (847003,'sub-alpha','GESS90');
  SELECT json_build_object('turbines',json_object_agg(serial_num,json_build_object('name',name,'type',type)))
  FROM foo;
!                                                                             json_build_object                                                                            
! -------------------------------------------------------------------------------------------------------------------------------------------------------------------------
!  {"turbines" : { "847001" : {"name" : "t15", "type" : "GE1043"}, "847002" : {"name" : "t16", "type" : "GE1043"}, "847003" : {"name" : "sub-alpha", "type" : "GESS90"} }}
  (1 row)
  
  -- json_object
  -- one dimension
  SELECT json_object('{a,1,b,2,3,NULL,"d e f","a b c"}');
!                       json_object                      
! -------------------------------------------------------
!  {"a" : "1", "b" : "2", "3" : null, "d e f" : "a b c"}
  (1 row)
  
  -- same but with two dimensions
  SELECT json_object('{{a,1},{b,2},{3,NULL},{"d e f","a b c"}}');
!                       json_object                      
! -------------------------------------------------------
!  {"a" : "1", "b" : "2", "3" : null, "d e f" : "a b c"}
  (1 row)
  
  -- odd number error
--- 1555,1578 ----
  INSERT INTO foo VALUES (847003,'sub-alpha','GESS90');
  SELECT json_build_object('turbines',json_object_agg(serial_num,json_build_object('name',name,'type',type)))
  FROM foo;
!                                                                       json_build_object                                                                      
! -------------------------------------------------------------------------------------------------------------------------------------------------------------
!  {"turbines": {"847001": {"name": "t15", "type": "GE1043"}, "847002": {"name": "t16", "type": "GE1043"}, "847003": {"name": "sub-alpha", "type": "GESS90"}}}
  (1 row)
  
  -- json_object
  -- one dimension
  SELECT json_object('{a,1,b,2,3,NULL,"d e f","a b c"}');
!                     json_object                    
! ---------------------------------------------------
!  {"a": "1", "b": "2", "3": null, "d e f": "a b c"}
  (1 row)
  
  -- same but with two dimensions
  SELECT json_object('{{a,1},{b,2},{3,NULL},{"d e f","a b c"}}');
!                     json_object                    
! ---------------------------------------------------
!  {"a": "1", "b": "2", "3": null, "d e f": "a b c"}
  (1 row)
  
  -- odd number error
*************** SELECT json_object('{{{a,b},{c,d}},{{b,c
*** 1589,1597 ****
  ERROR:  wrong number of array subscripts
  --two argument form of json_object
  select json_object('{a,b,c,"d e f"}','{1,2,3,"a b c"}');
!                      json_object                      
! ------------------------------------------------------
!  {"a" : "1", "b" : "2", "c" : "3", "d e f" : "a b c"}
  (1 row)
  
  -- too many dimensions
--- 1589,1597 ----
  ERROR:  wrong number of array subscripts
  --two argument form of json_object
  select json_object('{a,b,c,"d e f"}','{1,2,3,"a b c"}');
!                    json_object                    
! --------------------------------------------------
!  {"a": "1", "b": "2", "c": "3", "d e f": "a b c"}
  (1 row)
  
  -- too many dimensions
*************** select json_object('{a,b,NULL,"d e f"}',
*** 1607,1615 ****
  ERROR:  null value not allowed for object key
  -- empty key is allowed
  select json_object('{a,b,"","d e f"}','{1,2,3,"a b c"}');
!                      json_object                     
! -----------------------------------------------------
!  {"a" : "1", "b" : "2", "" : "3", "d e f" : "a b c"}
  (1 row)
  
  -- json_to_record and json_to_recordset
--- 1607,1615 ----
  ERROR:  null value not allowed for object key
  -- empty key is allowed
  select json_object('{a,b,"","d e f"}','{1,2,3,"a b c"}');
!                    json_object                   
! -------------------------------------------------
!  {"a": "1", "b": "2", "": "3", "d e f": "a b c"}
  (1 row)
  
  -- json_to_record and json_to_recordset
#21Pavel Stehule
pavel.stehule@gmail.com
In reply to: Shulgin, Oleksandr (#20)
Re: [PATCH] Generalized JSON output functions

2015-07-11 18:02 GMT+02:00 Shulgin, Oleksandr <oleksandr.shulgin@zalando.de>
:

On Fri, Jul 10, 2015 at 5:16 PM, Pavel Stehule <pavel.stehule@gmail.com>
wrote:

Well, one could call it premature pessimization due to dynamic call
overhead.

IMO, the fact that json_out_init_context() sets the value callback to
json_out_value is an implementation detail, the other parts of code should
not rely on. And for the Explain output, there definitely going to be
*some* code between context initialization and output callbacks: these are
done in a number of different functions.

Again - it is necessary? Postgres still use modular code, not OOP code. I
can understand the using of this technique, when I need a possibility to
change behave. But these function are used for printing JSON, not printing
any others.

No, it's not strictly necessary.

For me it's not about procedural- vs. object- style, but rather about
being able to override/extend the behavior consistently. And for that I
would prefer that if I override the value callback in a JSON output
context, that it would be called for every value being printed, not only
for some of them.

please, can me show any real use case? JSON is JSON, not art work. Still I
don't see any value of this.

Thank you for pointing out the case of Explain format, I've totally
overlooked it in my first version. Trying to apply the proposed approach
in the explain printing code led me to reorganize things slightly. I've
added explicit functions for printing keys vs. values, thus no need to
expose that key_scalar param anymore. There are now separate before/after
key and before/after value functions as well, but I believe it makes for a
cleaner code.

The most of the complexity is still in the code that decides whether or
not to put spaces (between the values or for indentation) and newlines at
certain points. Should we decide to unify the style we emit ourselves,
this could be simplified, while still leaving room for great flexibility if
overridden by an extension, for example.

Have a nice weekend.

you too

Regards

Pavel

Show quoted text

--
Alex

#22Shulgin, Oleksandr
oleksandr.shulgin@zalando.de
In reply to: Pavel Stehule (#21)
Re: [PATCH] Generalized JSON output functions

On Jul 11, 2015 6:19 PM, "Pavel Stehule" <pavel.stehule@gmail.com> wrote:

2015-07-11 18:02 GMT+02:00 Shulgin, Oleksandr <

oleksandr.shulgin@zalando.de>:

On Fri, Jul 10, 2015 at 5:16 PM, Pavel Stehule <pavel.stehule@gmail.com>

wrote:

Well, one could call it premature pessimization due to dynamic call

overhead.

IMO, the fact that json_out_init_context() sets the value callback to

json_out_value is an implementation detail, the other parts of code should
not rely on. And for the Explain output, there definitely going to be
*some* code between context initialization and output callbacks: these are
done in a number of different functions.

Again - it is necessary? Postgres still use modular code, not OOP code.

I can understand the using of this technique, when I need a possibility to
change behave. But these function are used for printing JSON, not printing
any others.

No, it's not strictly necessary.

For me it's not about procedural- vs. object- style, but rather about

being able to override/extend the behavior consistently. And for that I
would prefer that if I override the value callback in a JSON output
context, that it would be called for every value being printed, not only
for some of them.

please, can me show any real use case? JSON is JSON, not art work.

To quote my first mail:

The motivation behind this to be able to produce specially-crafted JSON in
a logical replication output plugin, such that numeric (and bigint) values
are quoted. This requirement, in turn, arises from the fact that
JavaScript specification, which is quite natural to expect as a consumer
for this JSON data, allows to silently drop significant digits when
converting from string to number object.

I believe this is a well-known problem and I'm aware of a number of tricks
that might be used to avoid it, but none of them seems to be optimal from
my standpoint.

I can also imagine this can be used to convert date/time to string
differently, or adding indentation depending on the depth in object
hierarchy, etc.

Still I don't see any value of this.

Huh? Why then do you spend time on review?

#23Pavel Stehule
pavel.stehule@gmail.com
In reply to: Shulgin, Oleksandr (#22)
Re: [PATCH] Generalized JSON output functions

2015-07-11 19:57 GMT+02:00 Shulgin, Oleksandr <oleksandr.shulgin@zalando.de>
:

On Jul 11, 2015 6:19 PM, "Pavel Stehule" <pavel.stehule@gmail.com> wrote:

2015-07-11 18:02 GMT+02:00 Shulgin, Oleksandr <

oleksandr.shulgin@zalando.de>:

On Fri, Jul 10, 2015 at 5:16 PM, Pavel Stehule <pavel.stehule@gmail.com>

wrote:

Well, one could call it premature pessimization due to dynamic call

overhead.

IMO, the fact that json_out_init_context() sets the value callback to

json_out_value is an implementation detail, the other parts of code should
not rely on. And for the Explain output, there definitely going to be
*some* code between context initialization and output callbacks: these are
done in a number of different functions.

Again - it is necessary? Postgres still use modular code, not OOP

code. I can understand the using of this technique, when I need a
possibility to change behave. But these function are used for printing
JSON, not printing any others.

No, it's not strictly necessary.

For me it's not about procedural- vs. object- style, but rather about

being able to override/extend the behavior consistently. And for that I
would prefer that if I override the value callback in a JSON output
context, that it would be called for every value being printed, not only
for some of them.

please, can me show any real use case? JSON is JSON, not art work.

To quote my first mail:

The motivation behind this to be able to produce specially-crafted JSON in
a logical replication output plugin, such that numeric (and bigint) values
are quoted. This requirement, in turn, arises from the fact that
JavaScript specification, which is quite natural to expect as a consumer
for this JSON data, allows to silently drop significant digits when
converting from string to number object.

I believe this is a well-known problem and I'm aware of a number of tricks
that might be used to avoid it, but none of them seems to be optimal from
my standpoint.

I can also imagine this can be used to convert date/time to string
differently, or adding indentation depending on the depth in object
hierarchy, etc.

There is simple rule - be strict on output and tolerant on input. If I
understand to sense of this patch - the target is one same format of JSON
documents - so there are no space for any variability.

Still I don't see any value of this.

Huh? Why then do you spend time on review?

I am thinking so general json functions has sense, but I partially disagree
with your implementation.

Regards

Pavel

#24Shulgin, Oleksandr
oleksandr.shulgin@zalando.de
In reply to: Pavel Stehule (#23)
Re: [PATCH] Generalized JSON output functions

On Jul 11, 2015 8:41 PM, "Pavel Stehule" <pavel.stehule@gmail.com> wrote:

There is simple rule - be strict on output and tolerant on input. If I

understand to sense of this patch - the target is one same format of JSON
documents - so there are no space for any variability.

So, would you prefer explain json format on a single line - no indentation
or whitespace whatsoever?

This far it was only about whitespace, but it can be useful for tweaking
other aspects of output, as I've mentioned before.

I can imagine the ability for 3rd-party code to override certain aspects of
the output would be really useful for extensions or background workers,
decoding plugins, etc.

I am thinking so general json functions has sense, but I partially

disagree with your implementation.

Then what would you differently exactly?

--
Alex

#25Pavel Stehule
pavel.stehule@gmail.com
In reply to: Shulgin, Oleksandr (#24)
Re: [PATCH] Generalized JSON output functions

2015-07-12 10:29 GMT+02:00 Shulgin, Oleksandr <oleksandr.shulgin@zalando.de>
:

On Jul 11, 2015 8:41 PM, "Pavel Stehule" <pavel.stehule@gmail.com> wrote:

There is simple rule - be strict on output and tolerant on input. If I

understand to sense of this patch - the target is one same format of JSON
documents - so there are no space for any variability.

So, would you prefer explain json format on a single line - no indentation
or whitespace whatsoever?

yes, - if you need pretty format - there is function json_pretty - any more
styles is wrong on Postgres side.

This far it was only about whitespace, but it can be useful for tweaking
other aspects of output, as I've mentioned before.

Postgres is database, not presentation server - it have to to any database
operations, quickly as possible - and formatting is part of client side.

I can imagine the ability for 3rd-party code to override certain aspects
of the output would be really useful for extensions or background workers,
decoding plugins, etc.

we talking about output - I can imagine, so there is only two possibilities
- plain join, and pretty formatted join (but with only one style).

I am thinking so general json functions has sense, but I partially

disagree with your implementation.

Then what would you differently exactly?

simple code.

Show quoted text

--
Alex

#26Merlin Moncure
mmoncure@gmail.com
In reply to: Pavel Stehule (#25)
Re: [PATCH] Generalized JSON output functions

On Sun, Jul 12, 2015 at 4:35 AM, Pavel Stehule <pavel.stehule@gmail.com> wrote:

2015-07-12 10:29 GMT+02:00 Shulgin, Oleksandr
<oleksandr.shulgin@zalando.de>:

On Jul 11, 2015 8:41 PM, "Pavel Stehule" <pavel.stehule@gmail.com> wrote:

There is simple rule - be strict on output and tolerant on input. If I
understand to sense of this patch - the target is one same format of JSON
documents - so there are no space for any variability.

So, would you prefer explain json format on a single line - no indentation
or whitespace whatsoever?

yes, - if you need pretty format - there is function json_pretty - any more
styles is wrong on Postgres side.

This far it was only about whitespace, but it can be useful for tweaking
other aspects of output, as I've mentioned before.

Postgres is database, not presentation server - it have to to any database
operations, quickly as possible - and formatting is part of client side.

I can imagine the ability for 3rd-party code to override certain aspects
of the output would be really useful for extensions or background workers,
decoding plugins, etc.

we talking about output - I can imagine, so there is only two possibilities
- plain join, and pretty formatted join (but with only one style).

This makes sense. Postgres core really only needs to support the
minimum styles necessary for core requirements. This means raw
unformatted json for data productions to client and an appropriate
formatting for explain. Fancier stuff like a generic formatted is
fine but those features *belong in an extension*.

merlin

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

#27Shulgin, Oleksandr
oleksandr.shulgin@zalando.de
In reply to: Merlin Moncure (#26)
Re: [PATCH] Generalized JSON output functions

we talking about output - I can imagine, so there is only two

possibilities

- plain join, and pretty formatted join (but with only one style).

This makes sense. Postgres core really only needs to support the
minimum styles necessary for core requirements. This means raw
unformatted json for data productions to client and an appropriate
formatting for explain. Fancier stuff like a generic formatted is
fine but those features *belong in an extension*.

The thing is - it's not only about whitespace, otherwise I would probably
not bother with the generic interface. For my original problem, there is
simply no way to do this correctly in an extension w/o copying over all of
the logic from json.c, which I have to do right now, would rather not.

Alex

#28Pavel Stehule
pavel.stehule@gmail.com
In reply to: Shulgin, Oleksandr (#27)
Re: [PATCH] Generalized JSON output functions

2015-07-12 20:11 GMT+02:00 Shulgin, Oleksandr <oleksandr.shulgin@zalando.de>
:

we talking about output - I can imagine, so there is only two

possibilities

- plain join, and pretty formatted join (but with only one style).

This makes sense. Postgres core really only needs to support the
minimum styles necessary for core requirements. This means raw
unformatted json for data productions to client and an appropriate
formatting for explain. Fancier stuff like a generic formatted is
fine but those features *belong in an extension*.

The thing is - it's not only about whitespace, otherwise I would probably
not bother with the generic interface. For my original problem, there is
simply no way to do this correctly in an extension w/o copying over all of
the logic from json.c, which I have to do right now, would rather not.

I am sorry - we are talking about JSON, not about any styled document. I
disagree, so it has not be implemented as extension - the backport of JSON
support is a extension.

Regards

Pavel

Show quoted text

Alex

#29Shulgin, Oleksandr
oleksandr.shulgin@zalando.de
In reply to: Pavel Stehule (#28)
Re: [PATCH] Generalized JSON output functions

On Sun, Jul 12, 2015 at 8:39 PM, Pavel Stehule <pavel.stehule@gmail.com>
wrote:

The thing is - it's not only about whitespace, otherwise I would probably

not bother with the generic interface. For my original problem, there is
simply no way to do this correctly in an extension w/o copying over all of
the logic from json.c, which I have to do right now, would rather not.

I am sorry - we are talking about JSON, not about any styled document. I
disagree, so it has not be implemented as extension - the backport of JSON
support is a extension.

Hm... I'm having a hard time making sense of that statement, sorry.

To reiterate: for my problem, that is escaping numerics that can
potentially overflow[1]http://stackoverflow.com/questions/307179/what-is-javascripts-highest-integer-value-that-a-number-can-go-to-without-losin under ECMAScript standard, I want to be able to
override the code that outputs the numeric converted to string. There is
no way in current implementation to do that *at all*, short of copying all
the code involved in producing JSON output and changing it at certain
points. One could try re-parsing JSON instead, but that doesn't actually
solve the issue, because type information is lost forever at that point.

The whitespace unification was a mere side-effect of the original effort on
this patch.

--
Best regards,
Alex

[1]: http://stackoverflow.com/questions/307179/what-is-javascripts-highest-integer-value-that-a-number-can-go-to-without-losin
http://stackoverflow.com/questions/307179/what-is-javascripts-highest-integer-value-that-a-number-can-go-to-without-losin

#30Pavel Stehule
pavel.stehule@gmail.com
In reply to: Shulgin, Oleksandr (#29)
Re: [PATCH] Generalized JSON output functions

2015-07-13 9:30 GMT+02:00 Shulgin, Oleksandr <oleksandr.shulgin@zalando.de>:

On Sun, Jul 12, 2015 at 8:39 PM, Pavel Stehule <pavel.stehule@gmail.com>
wrote:

The thing is - it's not only about whitespace, otherwise I would probably

not bother with the generic interface. For my original problem, there is
simply no way to do this correctly in an extension w/o copying over all of
the logic from json.c, which I have to do right now, would rather not.

I am sorry - we are talking about JSON, not about any styled document. I
disagree, so it has not be implemented as extension - the backport of JSON
support is a extension.

Hm... I'm having a hard time making sense of that statement, sorry.

To reiterate: for my problem, that is escaping numerics that can
potentially overflow[1] under ECMAScript standard, I want to be able to
override the code that outputs the numeric converted to string. There is
no way in current implementation to do that *at all*, short of copying all
the code involved in producing JSON output and changing it at certain
points. One could try re-parsing JSON instead, but that doesn't actually
solve the issue, because type information is lost forever at that point.

The whitespace unification was a mere side-effect of the original effort
on this patch.

The dynamic type change is some what I would not to do in database, really
:)

If you afraid about overflow, then convert numeric to string immediately -
in this case, the client have to support both variant - so immediate cast
should not be a problem.

Anyway this check on max number should be implemented in our JSON(b) out
functions (as warning?).

Regards

Pavel

Show quoted text

--
Best regards,
Alex

[1]
http://stackoverflow.com/questions/307179/what-is-javascripts-highest-integer-value-that-a-number-can-go-to-without-losin

#31Shulgin, Oleksandr
oleksandr.shulgin@zalando.de
In reply to: Pavel Stehule (#30)
Re: [PATCH] Generalized JSON output functions

On Mon, Jul 13, 2015 at 9:44 AM, Pavel Stehule <pavel.stehule@gmail.com>
wrote:

To reiterate: for my problem, that is escaping numerics that can

potentially overflow[1] under ECMAScript standard, I want to be able to
override the code that outputs the numeric converted to string. There is
no way in current implementation to do that *at all*, short of copying all
the code involved in producing JSON output and changing it at certain
points. One could try re-parsing JSON instead, but that doesn't actually
solve the issue, because type information is lost forever at that point.

The whitespace unification was a mere side-effect of the original effort
on this patch.

The dynamic type change is some what I would not to do in database, really
:)

If you afraid about overflow, then convert numeric to string immediately -
in this case, the client have to support both variant - so immediate cast
should not be a problem.

Yeah, but how would you do that in context of a logical replication
decoding plugin? I've tried a number of tricks for that, including, but
not limited to registering phony types to wrap numeric type and replacing
the OID of numeric with this custom type OID in TupleDesc, but then again
one has to register that as known record type, etc.

Anyway this check on max number should be implemented in our JSON(b) out

functions (as warning?).

Not really, since this is a problem of ECMAScript standard, not JSON spec.
For example, Python module for handling JSON doesn't suffer from this
overflow problem,

The thing is, we cannot know which clients are going to consume the stream
of decoded events, and if it's some implementation of JavaScript, it can
suffer silent data corruption if we don't guard against that in the logical
decoding plugin.

Hope that makes it clear. :-)

--
Alex

#32Ryan Pedela
rpedela@datalanche.com
In reply to: Shulgin, Oleksandr (#29)
Re: [PATCH] Generalized JSON output functions

On Mon, Jul 13, 2015 at 1:30 AM, Shulgin, Oleksandr <
oleksandr.shulgin@zalando.de> wrote:

To reiterate: for my problem, that is escaping numerics that can
potentially overflow[1] under ECMAScript standard, I want to be able to
override the code that outputs the numeric converted to string. There is
no way in current implementation to do that *at all*, short of copying all
the code involved in producing JSON output and changing it at certain
points. One could try re-parsing JSON instead, but that doesn't actually
solve the issue, because type information is lost forever at that point.

I had the exact same problem with Node.js and client-side Javascript. That
is why I wrote json-bignum [1] for Node.js. There is a bower version [2] as
well. The only caveat is that it is slower than the native JSON functions,
but I am happy to receive PRs to improve performance.

1. https://github.com/datalanche/json-bignum
2. https://libraries.io/bower/json-bignum

As far as large numbers in JSON, I think Postgres is doing the right thing
and should not be changed. It is Javascript that is stupid here, and I
don't think it is wise to add something to core just because one client
does stupid things with large numbers. In addition, ES7 is introducing
"value types" which will hopefully solve the large number problem in
Javascript.

The random whitespace issue is valid in my opinion and should be fixed.

Thanks,
Ryan Pedela

#33Andrew Dunstan
andrew@dunslane.net
In reply to: Shulgin, Oleksandr (#31)
Re: [PATCH] Generalized JSON output functions

On 07/13/2015 05:41 AM, Shulgin, Oleksandr wrote:

On Mon, Jul 13, 2015 at 9:44 AM, Pavel Stehule
<pavel.stehule@gmail.com <mailto:pavel.stehule@gmail.com>> wrote:

To reiterate: for my problem, that is escaping numerics that
can potentially overflow[1] under ECMAScript standard, I want
to be able to override the code that outputs the numeric
converted to string. There is no way in current
implementation to do that *at all*, short of copying all the
code involved in producing JSON output and changing it at
certain points. One could try re-parsing JSON instead, but
that doesn't actually solve the issue, because type
information is lost forever at that point.

The whitespace unification was a mere side-effect of the
original effort on this patch.

The dynamic type change is some what I would not to do in
database, really :)

If you afraid about overflow, then convert numeric to string
immediately - in this case, the client have to support both
variant - so immediate cast should not be a problem.

Yeah, but how would you do that in context of a logical replication
decoding plugin? I've tried a number of tricks for that, including,
but not limited to registering phony types to wrap numeric type and
replacing the OID of numeric with this custom type OID in TupleDesc,
but then again one has to register that as known record type, etc.

Anyway this check on max number should be implemented in our
JSON(b) out functions (as warning?).

Not really, since this is a problem of ECMAScript standard, not JSON
spec. For example, Python module for handling JSON doesn't suffer
from this overflow problem,

The thing is, we cannot know which clients are going to consume the
stream of decoded events, and if it's some implementation of
JavaScript, it can suffer silent data corruption if we don't guard
against that in the logical decoding plugin.

Hope that makes it clear. :-)

Yes, but I think the plugin is the right place to do it. What is more,
this won't actually prevent you completely from producing non-ECMAScript
compliant JSON, since json or jsonb values containing offending numerics
won't be caught, AIUI. But a fairly simple to write function that
reparsed and fixed the JSON inside the decoder would work.

cheers

andrew

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

#34Andrew Dunstan
andrew@dunslane.net
In reply to: Ryan Pedela (#32)
Re: [PATCH] Generalized JSON output functions

On 07/13/2015 10:46 AM, Ryan Pedela wrote:

On Mon, Jul 13, 2015 at 1:30 AM, Shulgin, Oleksandr
<oleksandr.shulgin@zalando.de <mailto:oleksandr.shulgin@zalando.de>>
wrote:

To reiterate: for my problem, that is escaping numerics that can
potentially overflow[1] under ECMAScript standard, I want to be
able to override the code that outputs the numeric converted to
string. There is no way in current implementation to do that *at
all*, short of copying all the code involved in producing JSON
output and changing it at certain points. One could try
re-parsing JSON instead, but that doesn't actually solve the
issue, because type information is lost forever at that point.

I had the exact same problem with Node.js and client-side Javascript.
That is why I wrote json-bignum [1] for Node.js. There is a bower
version [2] as well. The only caveat is that it is slower than the
native JSON functions, but I am happy to receive PRs to improve
performance.

1. https://github.com/datalanche/json-bignum
2. https://libraries.io/bower/json-bignum

As far as large numbers in JSON, I think Postgres is doing the right
thing and should not be changed. It is Javascript that is stupid here,
and I don't think it is wise to add something to core just because one
client does stupid things with large numbers. In addition, ES7 is
introducing "value types" which will hopefully solve the large number
problem in Javascript.

The random whitespace issue is valid in my opinion and should be fixed.

OK, I think we're getting a consensus here. It's good to know the JS
world is acquiring some sanity in this area.

Let's just fix the whitespace and be done, without all the callback
stuff. That should be a much smaller patch.

cheers

andrew

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

#35Shulgin, Oleksandr
oleksandr.shulgin@zalando.de
In reply to: Andrew Dunstan (#33)
Re: [PATCH] Generalized JSON output functions

Yes, but I think the plugin is the right place to do it. What is more,

this won't actually prevent you completely from producing non-ECMAScript
compliant JSON, since json or jsonb values containing offending numerics
won't be caught, AIUI.

Ah, that's a good catch indeed.

But a fairly simple to write function that reparsed and fixed the JSON

inside the decoder would work.

Need to rethink this, but reparsing was never my favorite option here. :-)

--
Alex

#36Robert Haas
robertmhaas@gmail.com
In reply to: Ryan Pedela (#32)
Re: [PATCH] Generalized JSON output functions

On Mon, Jul 13, 2015 at 10:46 AM, Ryan Pedela <rpedela@datalanche.com> wrote:

As far as large numbers in JSON, I think Postgres is doing the right thing
and should not be changed. It is Javascript that is stupid here, and I don't
think it is wise to add something to core just because one client does
stupid things with large numbers. In addition, ES7 is introducing "value
types" which will hopefully solve the large number problem in Javascript.

FWIW, I don't agree. If it's not easy to read the JSON that
PostgreSQL generates using JavaScript, then a lot of people are just
going to give up on doing it, and IMO that would be sad. Telling
people that they have to parse the JSON using some parser other than
the one built into their JavaScript engine, whack it around, and then
render it as text and parse it again is not really an acceptable
answer. The reason why the logical decoding stuff allows multiple
output formats is because Andres, quite correctly, foresaw that
different people would need different output formats. He could have
designed that system to output only one output format and just said,
everybody's got to read and parse this, but that would have been slow.
Instead, he tried to set things up so that you could get the output in
the format that was most convenient for your client, whatever that is.
On this thread, we're back-pedaling from that idea: sorry, you can get
JSON output, but if you want JSON output that will be properly
interpreted by your JSON parser, you can't have that. Regardless of
the details of this particular patch, I can't endorse that approach.
If we want people to use our software, we need to meet them where they
are at, especially when we are only (IIUC) talking about inserting a
few extra quotation marks.

--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

#37Andrew Dunstan
andrew@dunslane.net
In reply to: Robert Haas (#36)
Re: [PATCH] Generalized JSON output functions

On 07/15/2015 10:52 AM, Robert Haas wrote:

On Mon, Jul 13, 2015 at 10:46 AM, Ryan Pedela <rpedela@datalanche.com> wrote:

As far as large numbers in JSON, I think Postgres is doing the right thing
and should not be changed. It is Javascript that is stupid here, and I don't
think it is wise to add something to core just because one client does
stupid things with large numbers. In addition, ES7 is introducing "value
types" which will hopefully solve the large number problem in Javascript.

FWIW, I don't agree. If it's not easy to read the JSON that
PostgreSQL generates using JavaScript, then a lot of people are just
going to give up on doing it, and IMO that would be sad. Telling
people that they have to parse the JSON using some parser other than
the one built into their JavaScript engine, whack it around, and then
render it as text and parse it again is not really an acceptable
answer. The reason why the logical decoding stuff allows multiple
output formats is because Andres, quite correctly, foresaw that
different people would need different output formats. He could have
designed that system to output only one output format and just said,
everybody's got to read and parse this, but that would have been slow.
Instead, he tried to set things up so that you could get the output in
the format that was most convenient for your client, whatever that is.
On this thread, we're back-pedaling from that idea: sorry, you can get
JSON output, but if you want JSON output that will be properly
interpreted by your JSON parser, you can't have that. Regardless of
the details of this particular patch, I can't endorse that approach.
If we want people to use our software, we need to meet them where they
are at, especially when we are only (IIUC) talking about inserting a
few extra quotation marks.

The question for me is where is the best place to transform the data.
The approach take was both invasive and broken. The approach I
suggested, reparsing and transforming it in the logical decoder, would
be both fairly simple and completely noninvasive. If someone gives me a
test for what is an acceptable number for JS processors, I bet I could
write a transformation function in an hour or two, and in a hundred
lines or so. I admit that I probably have more experience doing this
than anyone else, but the parser API was designed to be fairly simple,
and I believe it is.

cheers

andrew

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

#38Ryan Pedela
rpedela@datalanche.com
In reply to: Robert Haas (#36)
Re: [PATCH] Generalized JSON output functions

On Wed, Jul 15, 2015 at 8:52 AM, Robert Haas <robertmhaas@gmail.com> wrote:

FWIW, I don't agree. If it's not easy to read the JSON that
PostgreSQL generates using JavaScript, then a lot of people are just
going to give up on doing it, and IMO that would be sad. Telling
people that they have to parse the JSON using some parser other than
the one built into their JavaScript engine, whack it around, and then
render it as text and parse it again is not really an acceptable
answer.

The vast majority of Javascript users are going to be using Node.js when
they connect to Postgres if only for security reasons. If they use Node,
they will be using node-postgres [1] or something that builds on top of it.
For int64 and numerics in a row, the default is to return a string, and
there is a flag you can set to round returned numbers if you prefer. There
is also a way to override the default parsing of each Postgres type [2]. So
in the case of JSON using my json-bignum module [3], the code looks like
this:

var pgTypes = require('pg').types;
var bignumJSON = require('json-bignum');

types.setTypeParser(JSON_TYPE_OID, function (value) {
return bignumJSON.parse(value);
});

types.setTypeParser(JSONB_TYPE_OID, function (value) {
return bignumJSON.parse(value);
});

To me that code is super simple, and no a pain in the ass. In other words,
it is not "Telling people that they have to parse the JSON using some
parser other than the one built into their JavaScript engine, whack it
around, and then render it as text and parse it again". Like I said
previously, the situation with Javascript will hopefully be remedied in a
few years with ES7 anyway.

1. https://github.com/brianc/node-postgres
2. https://github.com/brianc/node-pg-types
3. https://github.com/datalanche/json-bignum

On Wed, Jul 15, 2015 at 8:52 AM, Robert Haas <robertmhaas@gmail.com> wrote:

The reason why the logical decoding stuff allows multiple
output formats is because Andres, quite correctly, foresaw that
different people would need different output formats. He could have
designed that system to output only one output format and just said,
everybody's got to read and parse this, but that would have been slow.
Instead, he tried to set things up so that you could get the output in
the format that was most convenient for your client, whatever that is.
On this thread, we're back-pedaling from that idea: sorry, you can get
JSON output, but if you want JSON output that will be properly
interpreted by your JSON parser, you can't have that. Regardless of
the details of this particular patch, I can't endorse that approach.
If we want people to use our software, we need to meet them where they
are at, especially when we are only (IIUC) talking about inserting a
few extra quotation marks.

I would be okay with a generic way to specify output formats if there are
many use cases beyond Javascript and JSON. I vaguely remember someone
suggesting a FORMAT clause on CREATE TABLE which would specify how a
particular column would output from a SELECT. For example, returning a date
with a non-ISO format. I liked that idea. However if the only reason for
different output formats is Javascript, that is silly. I have a very long
list of feature requests that would probably only be beneficial to me or a
handful of users. Should we implement them? No, of course not! If we did
that Postgres would cease to be the best open-source database. You can't
have the best product and say yes to everything. Feature creep is the enemy
of quality. If Javascript is the sole reason for supporting multiple output
formats, then that is the definition of feature creep in my opinion. If
there are many use cases beyond Javascript and JSON, then that is different
and a conversation worth having.

#39Ryan Pedela
rpedela@datalanche.com
In reply to: Ryan Pedela (#38)
Re: [PATCH] Generalized JSON output functions

On Wed, Jul 15, 2015 at 11:10 AM, Ryan Pedela <rpedela@datalanche.com>
wrote:

On Wed, Jul 15, 2015 at 8:52 AM, Robert Haas <robertmhaas@gmail.com>
wrote:

FWIW, I don't agree. If it's not easy to read the JSON that
PostgreSQL generates using JavaScript, then a lot of people are just
going to give up on doing it, and IMO that would be sad. Telling
people that they have to parse the JSON using some parser other than
the one built into their JavaScript engine, whack it around, and then
render it as text and parse it again is not really an acceptable
answer.

The vast majority of Javascript users are going to be using Node.js when
they connect to Postgres if only for security reasons. If they use Node,
they will be using node-postgres [1] or something that builds on top of it.
For int64 and numerics in a row, the default is to return a string, and
there is a flag you can set to round returned numbers if you prefer. There
is also a way to override the default parsing of each Postgres type [2]. So
in the case of JSON using my json-bignum module [3], the code looks like
this:

var pgTypes = require('pg').types;
var bignumJSON = require('json-bignum');

types.setTypeParser(JSON_TYPE_OID, function (value) {
return bignumJSON.parse(value);
});

types.setTypeParser(JSONB_TYPE_OID, function (value) {
return bignumJSON.parse(value);
});

To me that code is super simple, and no a pain in the ass. In other words,
it is not "Telling people that they have to parse the JSON using some
parser other than the one built into their JavaScript engine, whack it
around, and then render it as text and parse it again". Like I said
previously, the situation with Javascript will hopefully be remedied in a
few years with ES7 anyway.

1. https://github.com/brianc/node-postgres
2. https://github.com/brianc/node-pg-types
3. https://github.com/datalanche/json-bignum

On Wed, Jul 15, 2015 at 8:52 AM, Robert Haas <robertmhaas@gmail.com>
wrote:

The reason why the logical decoding stuff allows multiple
output formats is because Andres, quite correctly, foresaw that
different people would need different output formats. He could have
designed that system to output only one output format and just said,
everybody's got to read and parse this, but that would have been slow.
Instead, he tried to set things up so that you could get the output in
the format that was most convenient for your client, whatever that is.
On this thread, we're back-pedaling from that idea: sorry, you can get
JSON output, but if you want JSON output that will be properly
interpreted by your JSON parser, you can't have that. Regardless of
the details of this particular patch, I can't endorse that approach.
If we want people to use our software, we need to meet them where they
are at, especially when we are only (IIUC) talking about inserting a
few extra quotation marks.

I would be okay with a generic way to specify output formats if there are
many use cases beyond Javascript and JSON. I vaguely remember someone
suggesting a FORMAT clause on CREATE TABLE which would specify how a
particular column would output from a SELECT. For example, returning a date
with a non-ISO format. I liked that idea. However if the only reason for
different output formats is Javascript, that is silly. I have a very long
list of feature requests that would probably only be beneficial to me or a
handful of users. Should we implement them? No, of course not! If we did
that Postgres would cease to be the best open-source database. You can't
have the best product and say yes to everything. Feature creep is the enemy
of quality. If Javascript is the sole reason for supporting multiple output
formats, then that is the definition of feature creep in my opinion. If
there are many use cases beyond Javascript and JSON, then that is different
and a conversation worth having.

Bottom line: Large numbers are a pain to deal with in Javascript regardless
of where they come from or what format they are in. Adding code to Postgres
core will never change that.

#40Robert Haas
robertmhaas@gmail.com
In reply to: Andrew Dunstan (#37)
Re: [PATCH] Generalized JSON output functions

On Wed, Jul 15, 2015 at 12:58 PM, Andrew Dunstan <andrew@dunslane.net> wrote:

The approach take was both invasive and broken.

Well, then let's not do it that way.

--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

#41Robert Haas
robertmhaas@gmail.com
In reply to: Ryan Pedela (#38)
Re: [PATCH] Generalized JSON output functions

On Wed, Jul 15, 2015 at 1:10 PM, Ryan Pedela <rpedela@datalanche.com> wrote:

Like I said previously, the
situation with Javascript will hopefully be remedied in a few years with ES7
anyway.

I don't understand these issues in great technical depth, but if
somebody is arguing that it's OK for PostgreSQL to be difficult to use
for a certain category of user for several years until the next
language rev becomes mainstream, then I disagree. The fact that
somebody wrote a patch to try to solve a problem means that the thing
in question is a problem for at least that one user. If he's the only
one, maybe we don't need to care all that much. If his needs are
representative of a significant user community, we should not turn our
backs on that community, regardless of whether we like the patch he
wrote, and regardless of how well we are meeting the needs of other
communities (like node.js users).

--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

#42Pavel Stehule
pavel.stehule@gmail.com
In reply to: Robert Haas (#41)
Re: [PATCH] Generalized JSON output functions

2015-07-16 19:51 GMT+02:00 Robert Haas <robertmhaas@gmail.com>:

On Wed, Jul 15, 2015 at 1:10 PM, Ryan Pedela <rpedela@datalanche.com>
wrote:

Like I said previously, the
situation with Javascript will hopefully be remedied in a few years with

ES7

anyway.

I don't understand these issues in great technical depth, but if
somebody is arguing that it's OK for PostgreSQL to be difficult to use
for a certain category of user for several years until the next
language rev becomes mainstream, then I disagree. The fact that
somebody wrote a patch to try to solve a problem means that the thing
in question is a problem for at least that one user. If he's the only
one, maybe we don't need to care all that much. If his needs are
representative of a significant user community, we should not turn our
backs on that community, regardless of whether we like the patch he
wrote, and regardless of how well we are meeting the needs of other
communities (like node.js users).

I don't think so this issue is too hot. How long we support XML? The output
format is static - the date format is fixed. How much issues was there?
Was there any issue, that was not solvable by casting?

If somebody needs different quoting, then it can be solved by explicit cast
in SQL query, and not in hacking our output routines.

Regards

Pavel

Show quoted text

--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

#43Ryan Pedela
rpedela@datalanche.com
In reply to: Robert Haas (#41)
Re: [PATCH] Generalized JSON output functions

On Thu, Jul 16, 2015 at 11:51 AM, Robert Haas <robertmhaas@gmail.com> wrote:

I don't understand these issues in great technical depth, but if
somebody is arguing that it's OK for PostgreSQL to be difficult to use
for a certain category of user for several years until the next
language rev becomes mainstream, then I disagree. The fact that
somebody wrote a patch to try to solve a problem means that the thing
in question is a problem for at least that one user. If he's the only
one, maybe we don't need to care all that much. If his needs are
representative of a significant user community, we should not turn our
backs on that community, regardless of whether we like the patch he
wrote, and regardless of how well we are meeting the needs of other
communities (like node.js users).

I completely agree. However we aren't talking about a usability problem
with Postgres. We are actually talking about a usability problem with
Javascript, and trying to implement a band-aid for it with Postgres.
Javascript doesn't support large numbers, it just doesn't. There is nothing
the Postgres community can do about that. Only the ECMAscript standards
committee and implementers can fix Javascript.

Here is the current user flow of reading numerics from Postgres and then
doing some math with them in Javascript.

1. SELECT json
2. Use json-bignum [1] module or custom JSON parser to correctly parse
numerics.
3. Perform addition, subtraction, etc of numerics using either custom
numeric math library or an existing library such as bigdecimal.js [2].

Here is the user flow if this patch is accepted.

1. SELECT json with quoting flags set
2. Custom parser to find numeric strings within JSON and convert them into
numerics. This is easy if JSON is simple, but may be difficult with a very
complex JSON.
3. Perform addition, subtraction, etc of numerics using either custom
numeric math library or an existing library such as bigdecimal.js [2].

It is almost the exact same user flow so what is the point? This patch
makes Postgres core more complex while not really solving the problem in
Javascript. If this would help other languages, domains, etc then maybe it
is worth implementing, but Javascript can't be fixed by the Postgres
community. It just can't.

To me the question is: does this help anyone besides Javascript users?

References
1. https://github.com/datalanche/json-bignum
2. https://github.com/iriscouch/bigdecimal.js

Thanks,
Ryan

#44Shulgin, Oleksandr
oleksandr.shulgin@zalando.de
In reply to: Ryan Pedela (#43)
Re: [PATCH] Generalized JSON output functions

On Jul 17, 2015 12:23 AM, "Ryan Pedela" <rpedela@datalanche.com> wrote:

On Thu, Jul 16, 2015 at 11:51 AM, Robert Haas <robertmhaas@gmail.com>

wrote:

I don't understand these issues in great technical depth, but if
somebody is arguing that it's OK for PostgreSQL to be difficult to use
for a certain category of user for several years until the next
language rev becomes mainstream, then I disagree. The fact that
somebody wrote a patch to try to solve a problem means that the thing
in question is a problem for at least that one user. If he's the only
one, maybe we don't need to care all that much. If his needs are
representative of a significant user community, we should not turn our
backs on that community, regardless of whether we like the patch he
wrote, and regardless of how well we are meeting the needs of other
communities (like node.js users).

I completely agree. However we aren't talking about a usability problem

with Postgres. We are actually talking about a usability problem with
Javascript, and trying to implement a band-aid for it with Postgres.
Javascript doesn't support large numbers, it just doesn't. There is nothing
the Postgres community can do about that. Only the ECMAscript standards
committee and implementers can fix Javascript.

Here is the current user flow of reading numerics from Postgres and then

doing some math with them in Javascript.

1. SELECT json
2. Use json-bignum [1] module or custom JSON parser to correctly parse

numerics.

3. Perform addition, subtraction, etc of numerics using either custom

numeric math library or an existing library such as bigdecimal.js [2].

Here is the user flow if this patch is accepted.

1. SELECT json with quoting flags set
2. Custom parser to find numeric strings within JSON and convert them

into numerics. This is easy if JSON is simple, but may be difficult with a
very complex JSON.

3. Perform addition, subtraction, etc of numerics using either custom

numeric math library or an existing library such as bigdecimal.js [2].

It is almost the exact same user flow so what is the point?

In my case there's no select: we're running this in the context of a
logical decoding plugin.

The all safeguarding idea that is enabled by this patch is that if the
client *expects* big numbers *and* it needs to perform arithmetic on them,
it'll have the special handling anyway. And IMO, it would actually make
more sense to use big numbers module only at the point where you have the
need for special handling, not to parse the whole input in a nonstandard
way.

But the clients that are unaware of big numbers or don't care about them
shouldn't be *forced* to use external modules for parsing json.

This patch makes Postgres core more complex

Yes, it does. But, that was not the purpose, obviously. :-)

while not really solving the problem in Javascript.

It still allows for less risk of silent data corruption on the js side.

--
Alex

#45Andrew Dunstan
andrew@dunslane.net
In reply to: Shulgin, Oleksandr (#44)
Re: [PATCH] Generalized JSON output functions

On 07/17/2015 08:20 AM, Shulgin, Oleksandr wrote:

This patch makes Postgres core more complex

Yes, it does. But, that was not the purpose, obviously. :-)

while not really solving the problem in Javascript.

It still allows for less risk of silent data corruption on the js side.

I have already pointed out how this patch is fundamentally broken. You
can achieve your aims by a fairly small amount of code inside your
logical decoder, and with no core code changes whatsoever. So I'm
puzzled why we are even still debating this broken design.

cheers

andrew

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

#46Alvaro Herrera
alvherre@2ndquadrant.com
In reply to: Andrew Dunstan (#45)
Re: [PATCH] Generalized JSON output functions

Andrew Dunstan wrote:

I have already pointed out how this patch is fundamentally broken. You can
achieve your aims by a fairly small amount of code inside your logical
decoder, and with no core code changes whatsoever. So I'm puzzled why we are
even still debating this broken design.

I went through all your responses over the entire thread and I couldn't
find your argument about how this is fundamentally broken. Can you
restate, or maybe give an archive link if I just missed it?

(Saying "but it changes so much of the existing code" is not really a
fundamental problem to me. I mean, it's not like the existing code is
perfect and needs no changes.)

--
�lvaro Herrera http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

#47Andrew Dunstan
andrew@dunslane.net
In reply to: Andrew Dunstan (#45)
Re: [PATCH] Generalized JSON output functions

On 07/17/2015 10:11 AM, Andrew Dunstan wrote:

On 07/17/2015 08:20 AM, Shulgin, Oleksandr wrote:

This patch makes Postgres core more complex

Yes, it does. But, that was not the purpose, obviously. :-)

while not really solving the problem in Javascript.

It still allows for less risk of silent data corruption on the js side.

I have already pointed out how this patch is fundamentally broken. You
can achieve your aims by a fairly small amount of code inside your
logical decoder, and with no core code changes whatsoever. So I'm
puzzled why we are even still debating this broken design.

Incidentally, this doesn't look acceptable anyway:

! es->json_cxt.value(&es->json_cxt, num, JSONTYPE_NUMERIC,
! NUMERICOID, 1702 /* numeric_out */);

We don't hardcode function oids elsewhere. So this is also something
that makes the patch unacceptable.

cheers

andrew

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

#48Shulgin, Oleksandr
oleksandr.shulgin@zalando.de
In reply to: Andrew Dunstan (#47)
Re: [PATCH] Generalized JSON output functions

On Jul 17, 2015 4:31 PM, "Andrew Dunstan" <andrew@dunslane.net> wrote:

On 07/17/2015 10:11 AM, Andrew Dunstan wrote:

On 07/17/2015 08:20 AM, Shulgin, Oleksandr wrote:

This patch makes Postgres core more complex

Yes, it does. But, that was not the purpose, obviously. :-)

while not really solving the problem in Javascript.

It still allows for less risk of silent data corruption on the js side.

I have already pointed out how this patch is fundamentally broken. You

can achieve your aims by a fairly small amount of code inside your logical
decoder, and with no core code changes whatsoever. So I'm puzzled why we
are even still debating this broken design.

Incidentally, this doesn't look acceptable anyway:

!

es->json_cxt.value(&es->json_cxt, num, JSONTYPE_NUMERIC,

!

NUMERICOID, 1702 /* numeric_out */);

We don't hardcode function oids elsewhere. So this is also something that

makes the patch unacceptable.

Well, good to know (I believe I've asked about this in the first mail
specifically).

Is there any way a built-in function oid would change/differ on different
server versions? What would be the recommended way to do this?

--
Alex

#49Alvaro Herrera
alvherre@2ndquadrant.com
In reply to: Shulgin, Oleksandr (#48)
Re: [PATCH] Generalized JSON output functions

Shulgin, Oleksandr wrote:

On Jul 17, 2015 4:31 PM, "Andrew Dunstan" <andrew@dunslane.net> wrote:

Incidentally, this doesn't look acceptable anyway:

! es->json_cxt.value(&es->json_cxt, num, JSONTYPE_NUMERIC,
! NUMERICOID, 1702 /* numeric_out */);

We don't hardcode function oids elsewhere. So this is also something that
makes the patch unacceptable.

Well, good to know (I believe I've asked about this in the first mail
specifically).

Is there any way a built-in function oid would change/differ on different
server versions? What would be the recommended way to do this?

C'mon, that's a trivial problem. Just use getTypeOutputInfo();
numeric's OID is hardcoded as NUMERICOID.

--
�lvaro Herrera http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

#50Andrew Dunstan
andrew@dunslane.net
In reply to: Alvaro Herrera (#46)
Re: [PATCH] Generalized JSON output functions

On 07/17/2015 10:30 AM, Alvaro Herrera wrote:

Andrew Dunstan wrote:

I have already pointed out how this patch is fundamentally broken. You can
achieve your aims by a fairly small amount of code inside your logical
decoder, and with no core code changes whatsoever. So I'm puzzled why we are
even still debating this broken design.

I went through all your responses over the entire thread and I couldn't
find your argument about how this is fundamentally broken. Can you
restate, or maybe give an archive link if I just missed it?

(Saying "but it changes so much of the existing code" is not really a
fundamental problem to me. I mean, it's not like the existing code is
perfect and needs no changes.)

On July 13 I wrote:

Yes, but I think the plugin is the right place to do it. What is more,
this won't actually prevent you completely from producing
non-ECMAScript compliant JSON, since json or jsonb values containing
offending numerics won't be caught, AIUI. But a fairly simple to write
function that reparsed and fixed the JSON inside the decoder would work.

The OP admitted that this was a serious flaw in his approach. In fact,
given that a json value can contain an offending numeric value, any
approach which doesn't involve reparsing is pretty much bound to fail.

cheers

andrew

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

#51Shulgin, Oleksandr
oleksandr.shulgin@zalando.de
In reply to: Andrew Dunstan (#50)
Re: [PATCH] Generalized JSON output functions

On July 13 I wrote:

Yes, but I think the plugin is the right place to do it. What is more,

this won't actually prevent you completely from producing non-ECMAScript
compliant JSON, since json or jsonb values containing offending numerics
won't be caught, AIUI. But a fairly simple to write function that reparsed
and fixed the JSON inside the decoder would work.

The OP admitted that this was a serious flaw in his approach. In fact,
given that a json value can contain an offending numeric value, any
approach which doesn't involve reparsing is pretty much bound to fail.

I agree that was a critical omission in my thinking.

Now, back to the whitespace issue: I could submit a patch to unify the
whitespace w/o all the hairy callbacks. Did we have the consensus here: no
spaces whatsoever unless some *_pretty function is used?

--
Alex