Speed up JSON escape processing with SIMD plus other optimisations
Currently the escape_json() function takes a cstring and char-by-char
checks each character in the string up to the NUL and adds the escape
sequence if the character requires it.
Because this function requires a NUL terminated string, we're having
to do a little more work in some places. For example, in
jsonb_put_escaped_value() we call pnstrdup() on the non-NUL-terminated
string to make a NUL-terminated string to pass to escape_json().
To make this faster, we can just have a version of escape_json which
takes a 'len' and stops after doing that many chars rather than
stopping when the NUL char is reached. Now there's no need to
pnstrdup() which saves some palloc()/memcpy() work.
There are also a few places where we do escape_json() with a "text"
typed Datum where we go and convert the text to a NUL-terminated
cstring so we can pass that along to ecape_json(). That's wasteful as
we could just pass the payload of the text Datum directly, and only
allocate memory if the text Datum needs to be de-toasted. That saves
a useless palloc/memcpy/pfree cycle.
Now, to make this more interesting, since we have a version of
escape_json which takes a 'len', we could start looking at more than 1
character at a time. If you look closely add escape_json() all the
special chars apart from " and \ are below the space character.
pg_lfind8() and pg_lfind8_le() allow processing of 16 bytes at a time,
so we only need to search the 16 bytes 3 times to ensure that no
special chars exist within. When that test fails, just go into
byte-at-a-time processing first copying over the portion of the string
that passed the vector test up until that point.
I've attached 2 patches:
0001 does everything I've described aside from SIMD.
0002 does SIMD
I've not personally done too much work in the area of JSON, so I don't
have any canned workloads to throw at this. I did try the following:
create table j1 (very_long_column_name_to_test_json_escape text);
insert into j1 select repeat('x', x) from generate_series(0,1024)x;
vacuum freeze j1;
bench.sql:
select row_to_json(j1)::jsonb from j1;
Master:
$ pgbench -n -f bench.sql -T 10 -M prepared postgres | grep tps
tps = 362.494309 (without initial connection time)
tps = 363.182458 (without initial connection time)
tps = 362.679654 (without initial connection time)
Master + 0001 + 0002
$ pgbench -n -f bench.sql -T 10 -M prepared postgres | grep tps
tps = 426.456885 (without initial connection time)
tps = 430.573046 (without initial connection time)
tps = 431.142917 (without initial connection time)
About 18% faster.
It would be much faster if we could also get rid of the
escape_json_cstring() call in the switch default case of
datum_to_json_internal(). row_to_json() would be heaps faster with
that done. I considered adding a special case for the "text" type
there, but in the end felt that we should just fix that with some
hypothetical other patch that changes how output functions work.
Others may feel it's worthwhile. I certainly could be convinced of it.
I did add a new regression test. I'm not sure I'd want to keep that,
but felt it's worth leaving in there for now.
Other things I considered were if doing 16 bytes at a time is too much
as it puts quite a bit of work into byte-at-a-time processing if just
1 special char exists in a 16-byte chunk. I considered doing SWAR [1]https://en.wikipedia.org/wiki/SWAR
processing to do the job of vector8_has_le() and vector8_has() byte
maybe with just uint32s. It might be worth doing that. However, I've
not done it yet as it raises the bar for this patch quite a bit. SWAR
vector processing is pretty much write-only code. Imagine trying to
write comments for the code in [2]https://dotat.at/@/2022-06-27-tolower-swar.html so that the average person could
understand what's going on!?
I'd be happy to hear from anyone that can throw these patches at a
real-world JSON workload to see if it runs more quickly.
Parking for July CF.
David
[1]: https://en.wikipedia.org/wiki/SWAR
[2]: https://dotat.at/@/2022-06-27-tolower-swar.html
Attachments:
v1-0001-Add-len-parameter-to-escape_json.patchapplication/octet-stream; name=v1-0001-Add-len-parameter-to-escape_json.patchDownload
From 695ab50057adb87caa95d8ca3ff77849f33ba399 Mon Sep 17 00:00:00 2001
From: David Rowley <dgrowley@gmail.com>
Date: Tue, 21 May 2024 17:14:06 +1200
Subject: [PATCH v1 1/2] Add 'len' parameter to escape_json()
---
contrib/hstore/hstore_io.c | 41 +++-----
src/backend/backup/backup_manifest.c | 2 +-
src/backend/commands/explain.c | 30 ++++--
src/backend/parser/parse_jsontable.c | 2 +-
src/backend/utils/adt/json.c | 150 +++++++++++++++++----------
src/backend/utils/adt/jsonb.c | 2 +-
src/backend/utils/adt/jsonfuncs.c | 15 +--
src/backend/utils/adt/jsonpath.c | 15 ++-
src/backend/utils/error/jsonlog.c | 8 +-
src/include/utils/json.h | 4 +-
10 files changed, 162 insertions(+), 107 deletions(-)
diff --git a/contrib/hstore/hstore_io.c b/contrib/hstore/hstore_io.c
index 999ddad76d..374b8d05ef 100644
--- a/contrib/hstore/hstore_io.c
+++ b/contrib/hstore/hstore_io.c
@@ -1343,23 +1343,20 @@ hstore_to_json_loose(PG_FUNCTION_ARGS)
int count = HS_COUNT(in);
char *base = STRPTR(in);
HEntry *entries = ARRPTR(in);
- StringInfoData tmp,
- dst;
+ StringInfoData dst;
if (count == 0)
PG_RETURN_TEXT_P(cstring_to_text_with_len("{}", 2));
- initStringInfo(&tmp);
initStringInfo(&dst);
appendStringInfoChar(&dst, '{');
for (i = 0; i < count; i++)
{
- resetStringInfo(&tmp);
- appendBinaryStringInfo(&tmp, HSTORE_KEY(entries, base, i),
- HSTORE_KEYLEN(entries, i));
- escape_json(&dst, tmp.data);
+ escape_json(&dst,
+ HSTORE_KEY(entries, base, i),
+ HSTORE_KEYLEN(entries, i));
appendStringInfoString(&dst, ": ");
if (HSTORE_VALISNULL(entries, i))
appendStringInfoString(&dst, "null");
@@ -1372,13 +1369,13 @@ hstore_to_json_loose(PG_FUNCTION_ARGS)
appendStringInfoString(&dst, "false");
else
{
- resetStringInfo(&tmp);
- appendBinaryStringInfo(&tmp, HSTORE_VAL(entries, base, i),
- HSTORE_VALLEN(entries, i));
- if (IsValidJsonNumber(tmp.data, tmp.len))
- appendBinaryStringInfo(&dst, tmp.data, tmp.len);
+ char *str = HSTORE_VAL(entries, base, i);
+ int len = HSTORE_VALLEN(entries, i);
+
+ if (IsValidJsonNumber(str, len))
+ appendBinaryStringInfo(&dst, str, len);
else
- escape_json(&dst, tmp.data);
+ escape_json(&dst, str, len);
}
if (i + 1 != count)
@@ -1398,32 +1395,28 @@ hstore_to_json(PG_FUNCTION_ARGS)
int count = HS_COUNT(in);
char *base = STRPTR(in);
HEntry *entries = ARRPTR(in);
- StringInfoData tmp,
- dst;
+ StringInfoData dst;
if (count == 0)
PG_RETURN_TEXT_P(cstring_to_text_with_len("{}", 2));
- initStringInfo(&tmp);
initStringInfo(&dst);
appendStringInfoChar(&dst, '{');
for (i = 0; i < count; i++)
{
- resetStringInfo(&tmp);
- appendBinaryStringInfo(&tmp, HSTORE_KEY(entries, base, i),
- HSTORE_KEYLEN(entries, i));
- escape_json(&dst, tmp.data);
+ escape_json(&dst,
+ HSTORE_KEY(entries, base, i),
+ HSTORE_KEYLEN(entries, i));
appendStringInfoString(&dst, ": ");
if (HSTORE_VALISNULL(entries, i))
appendStringInfoString(&dst, "null");
else
{
- resetStringInfo(&tmp);
- appendBinaryStringInfo(&tmp, HSTORE_VAL(entries, base, i),
- HSTORE_VALLEN(entries, i));
- escape_json(&dst, tmp.data);
+ escape_json(&dst,
+ HSTORE_VAL(entries, base, i),
+ HSTORE_VALLEN(entries, i));
}
if (i + 1 != count)
diff --git a/src/backend/backup/backup_manifest.c b/src/backend/backup/backup_manifest.c
index b360a13547..3da8da00d6 100644
--- a/src/backend/backup/backup_manifest.c
+++ b/src/backend/backup/backup_manifest.c
@@ -148,7 +148,7 @@ AddFileToBackupManifest(backup_manifest_info *manifest, Oid spcoid,
pg_verify_mbstr(PG_UTF8, pathname, pathlen, true))
{
appendStringInfoString(&buf, "{ \"Path\": ");
- escape_json(&buf, pathname);
+ escape_json(&buf, pathname, pathlen);
appendStringInfoString(&buf, ", ");
}
else
diff --git a/src/backend/commands/explain.c b/src/backend/commands/explain.c
index 94511a5a02..399025aed3 100644
--- a/src/backend/commands/explain.c
+++ b/src/backend/commands/explain.c
@@ -4661,13 +4661,15 @@ ExplainPropertyList(const char *qlabel, List *data, ExplainState *es)
case EXPLAIN_FORMAT_JSON:
ExplainJSONLineEnding(es);
appendStringInfoSpaces(es->str, es->indent * 2);
- escape_json(es->str, qlabel);
+ escape_json_cstring(es->str, qlabel);
appendStringInfoString(es->str, ": [");
foreach(lc, data)
{
+ const char *str = (const char *) lfirst(lc);
+
if (!first)
appendStringInfoString(es->str, ", ");
- escape_json(es->str, (const char *) lfirst(lc));
+ escape_json_cstring(es->str, str);
first = false;
}
appendStringInfoChar(es->str, ']');
@@ -4678,10 +4680,12 @@ ExplainPropertyList(const char *qlabel, List *data, ExplainState *es)
appendStringInfo(es->str, "%s: ", qlabel);
foreach(lc, data)
{
+ const char *str = (const char *) lfirst(lc);
+
appendStringInfoChar(es->str, '\n');
appendStringInfoSpaces(es->str, es->indent * 2 + 2);
appendStringInfoString(es->str, "- ");
- escape_yaml(es->str, (const char *) lfirst(lc));
+ escape_yaml(es->str, str);
}
break;
}
@@ -4710,9 +4714,11 @@ ExplainPropertyListNested(const char *qlabel, List *data, ExplainState *es)
appendStringInfoChar(es->str, '[');
foreach(lc, data)
{
+ const char *str = (const char *) lfirst(lc);
+
if (!first)
appendStringInfoString(es->str, ", ");
- escape_json(es->str, (const char *) lfirst(lc));
+ escape_json_cstring(es->str, str);
first = false;
}
appendStringInfoChar(es->str, ']');
@@ -4723,9 +4729,11 @@ ExplainPropertyListNested(const char *qlabel, List *data, ExplainState *es)
appendStringInfoString(es->str, "- [");
foreach(lc, data)
{
+ const char *str = (const char *) lfirst(lc);
+
if (!first)
appendStringInfoString(es->str, ", ");
- escape_yaml(es->str, (const char *) lfirst(lc));
+ escape_yaml(es->str, str);
first = false;
}
appendStringInfoChar(es->str, ']');
@@ -4775,12 +4783,12 @@ ExplainProperty(const char *qlabel, const char *unit, const char *value,
case EXPLAIN_FORMAT_JSON:
ExplainJSONLineEnding(es);
appendStringInfoSpaces(es->str, es->indent * 2);
- escape_json(es->str, qlabel);
+ escape_json_cstring(es->str, qlabel);
appendStringInfoString(es->str, ": ");
if (numeric)
appendStringInfoString(es->str, value);
else
- escape_json(es->str, value);
+ escape_json_cstring(es->str, value);
break;
case EXPLAIN_FORMAT_YAML:
@@ -4882,7 +4890,7 @@ ExplainOpenGroup(const char *objtype, const char *labelname,
appendStringInfoSpaces(es->str, 2 * es->indent);
if (labelname)
{
- escape_json(es->str, labelname);
+ escape_json_cstring(es->str, labelname);
appendStringInfoString(es->str, ": ");
}
appendStringInfoChar(es->str, labeled ? '{' : '[');
@@ -5090,10 +5098,10 @@ ExplainDummyGroup(const char *objtype, const char *labelname, ExplainState *es)
appendStringInfoSpaces(es->str, 2 * es->indent);
if (labelname)
{
- escape_json(es->str, labelname);
+ escape_json_cstring(es->str, labelname);
appendStringInfoString(es->str, ": ");
}
- escape_json(es->str, objtype);
+ escape_json_cstring(es->str, objtype);
break;
case EXPLAIN_FORMAT_YAML:
@@ -5297,7 +5305,7 @@ ExplainYAMLLineStarting(ExplainState *es)
static void
escape_yaml(StringInfo buf, const char *str)
{
- escape_json(buf, str);
+ escape_json_cstring(buf, str);
}
diff --git a/src/backend/parser/parse_jsontable.c b/src/backend/parser/parse_jsontable.c
index b2519c2f32..03aac2f0cd 100644
--- a/src/backend/parser/parse_jsontable.c
+++ b/src/backend/parser/parse_jsontable.c
@@ -427,7 +427,7 @@ transformJsonTableColumn(JsonTableColumn *jtc, Node *contextItemExpr,
initStringInfo(&path);
appendStringInfoString(&path, "$.");
- escape_json(&path, jtc->name);
+ escape_json_cstring(&path, jtc->name);
pathspec = makeStringConst(path.data, -1);
}
diff --git a/src/backend/utils/adt/json.c b/src/backend/utils/adt/json.c
index d719a61f16..7934cf62fb 100644
--- a/src/backend/utils/adt/json.c
+++ b/src/backend/utils/adt/json.c
@@ -286,7 +286,7 @@ datum_to_json_internal(Datum val, bool is_null, StringInfo result,
break;
default:
outputstr = OidOutputFunctionCall(outfuncoid, val);
- escape_json(result, outputstr);
+ escape_json_cstring(result, outputstr);
pfree(outputstr);
break;
}
@@ -560,7 +560,7 @@ composite_to_json(Datum composite, StringInfo result, bool use_line_feeds)
needsep = true;
attname = NameStr(att->attname);
- escape_json(result, attname);
+ escape_json_cstring(result, attname);
appendStringInfoChar(result, ':');
val = heap_getattr(tuple, i + 1, tupdesc, &isnull);
@@ -1391,7 +1391,6 @@ json_object(PG_FUNCTION_ARGS)
count,
i;
text *rval;
- char *v;
switch (ndims)
{
@@ -1434,19 +1433,17 @@ json_object(PG_FUNCTION_ARGS)
(errcode(ERRCODE_NULL_VALUE_NOT_ALLOWED),
errmsg("null value not allowed for object key")));
- v = TextDatumGetCString(in_datums[i * 2]);
if (i > 0)
appendStringInfoString(&result, ", ");
- escape_json(&result, v);
+ escape_json_from_text(&result,
+ (text *) DatumGetPointer(in_datums[i * 2]));
appendStringInfoString(&result, " : ");
- pfree(v);
if (in_nulls[i * 2 + 1])
appendStringInfoString(&result, "null");
else
{
- v = TextDatumGetCString(in_datums[i * 2 + 1]);
- escape_json(&result, v);
- pfree(v);
+ escape_json_from_text(&result,
+ (text *) DatumGetPointer(in_datums[i * 2 + 1]));
}
}
@@ -1483,7 +1480,6 @@ json_object_two_arg(PG_FUNCTION_ARGS)
val_count,
i;
text *rval;
- char *v;
if (nkdims > 1 || nkdims != nvdims)
ereport(ERROR,
@@ -1512,20 +1508,17 @@ json_object_two_arg(PG_FUNCTION_ARGS)
(errcode(ERRCODE_NULL_VALUE_NOT_ALLOWED),
errmsg("null value not allowed for object key")));
- v = TextDatumGetCString(key_datums[i]);
if (i > 0)
appendStringInfoString(&result, ", ");
- escape_json(&result, v);
+ escape_json_from_text(&result,
+ (text *) DatumGetPointer(key_datums[i]));
+
appendStringInfoString(&result, " : ");
- pfree(v);
if (val_nulls[i])
appendStringInfoString(&result, "null");
else
- {
- v = TextDatumGetCString(val_datums[i]);
- escape_json(&result, v);
- pfree(v);
- }
+ escape_json_from_text(&result,
+ (text *) DatumGetPointer(val_datums[i]));
}
appendStringInfoChar(&result, '}');
@@ -1541,52 +1534,101 @@ json_object_two_arg(PG_FUNCTION_ARGS)
PG_RETURN_TEXT_P(rval);
}
+/*
+ * escape_json_char
+ * Inline helper function for escape_json* functions
+ */
+static pg_attribute_always_inline void
+escape_json_char(StringInfo buf, char c)
+{
+ switch (c)
+ {
+ case '\b':
+ appendStringInfoString(buf, "\\b");
+ break;
+ case '\f':
+ appendStringInfoString(buf, "\\f");
+ break;
+ case '\n':
+ appendStringInfoString(buf, "\\n");
+ break;
+ case '\r':
+ appendStringInfoString(buf, "\\r");
+ break;
+ case '\t':
+ appendStringInfoString(buf, "\\t");
+ break;
+ case '"':
+ appendStringInfoString(buf, "\\\"");
+ break;
+ case '\\':
+ appendStringInfoString(buf, "\\\\");
+ break;
+ default:
+ if ((unsigned char) c < ' ')
+ appendStringInfo(buf, "\\u%04x", (int) c);
+ else
+ appendStringInfoCharMacro(buf, c);
+ break;
+ }
+}
/*
- * Produce a JSON string literal, properly escaping characters in the text.
+ * escape_json_cstring
+ * Produce a JSON string literal. Same as escape_json() except takes a
+ * NUL-terminated string as input.
*/
void
-escape_json(StringInfo buf, const char *str)
+escape_json_cstring(StringInfo buf, const char *str)
{
- const char *p;
+ appendStringInfoCharMacro(buf, '"');
+
+ for (; *str != '\0'; str++)
+ escape_json_char(buf, *str);
appendStringInfoCharMacro(buf, '"');
- for (p = str; *p; p++)
- {
- switch (*p)
- {
- case '\b':
- appendStringInfoString(buf, "\\b");
- break;
- case '\f':
- appendStringInfoString(buf, "\\f");
- break;
- case '\n':
- appendStringInfoString(buf, "\\n");
- break;
- case '\r':
- appendStringInfoString(buf, "\\r");
- break;
- case '\t':
- appendStringInfoString(buf, "\\t");
- break;
- case '"':
- appendStringInfoString(buf, "\\\"");
- break;
- case '\\':
- appendStringInfoString(buf, "\\\\");
- break;
- default:
- if ((unsigned char) *p < ' ')
- appendStringInfo(buf, "\\u%04x", (int) *p);
- else
- appendStringInfoCharMacro(buf, *p);
- break;
- }
- }
+}
+
+/*
+ * Produce a JSON string literal, properly escaping the possibly not
+ * NUL-terminated characters in 'str'. 'len' defines the number of bytes from
+ * 'str' to process.
+ */
+void
+escape_json(StringInfo buf, const char *str, int len)
+{
+ appendStringInfoCharMacro(buf, '"');
+
+ for (int i = 0; i < len; i++)
+ escape_json_char(buf, str[i]);
+
appendStringInfoCharMacro(buf, '"');
}
+/*
+ * escape_json_from_text
+ * Append 't' onto 'buf' and escape using escape_json.
+ *
+ * This is more efficient than calling text_to_cstring and appending the
+ * result as that could require an additional palloc and memcpy.
+ */
+void
+escape_json_from_text(StringInfo buf, const text *t)
+{
+ /* must cast away the const, unfortunately */
+ text *tunpacked = pg_detoast_datum_packed(unconstify(text *, t));
+ int len = VARSIZE_ANY_EXHDR(tunpacked);
+ char *str;
+
+ str = VARDATA_ANY(tunpacked);
+
+ escape_json(buf, str, len);
+
+ /* pfree any detoasted values */
+ if (tunpacked != t)
+ pfree(tunpacked);
+}
+
/* Semantic actions for key uniqueness check */
static JsonParseErrorType
json_unique_object_start(void *_state)
diff --git a/src/backend/utils/adt/jsonb.c b/src/backend/utils/adt/jsonb.c
index e4562b3c6c..f24d8003be 100644
--- a/src/backend/utils/adt/jsonb.c
+++ b/src/backend/utils/adt/jsonb.c
@@ -354,7 +354,7 @@ jsonb_put_escaped_value(StringInfo out, JsonbValue *scalarVal)
appendBinaryStringInfo(out, "null", 4);
break;
case jbvString:
- escape_json(out, pnstrdup(scalarVal->val.string.val, scalarVal->val.string.len));
+ escape_json(out, scalarVal->val.string.val, scalarVal->val.string.len);
break;
case jbvNumeric:
appendStringInfoString(out,
diff --git a/src/backend/utils/adt/jsonfuncs.c b/src/backend/utils/adt/jsonfuncs.c
index 83125b06a4..743d47dae6 100644
--- a/src/backend/utils/adt/jsonfuncs.c
+++ b/src/backend/utils/adt/jsonfuncs.c
@@ -3139,7 +3139,10 @@ populate_scalar(ScalarIOData *io, Oid typid, int32 typmod, JsValue *jsv,
str[len] = '\0';
}
else
- str = json; /* string is already null-terminated */
+ {
+ str = json; /* string is already null-terminated */
+ len = strlen(str);
+ }
/* If converting to json/jsonb, make string into valid JSON literal */
if ((typid == JSONOID || typid == JSONBOID) &&
@@ -3148,7 +3151,7 @@ populate_scalar(ScalarIOData *io, Oid typid, int32 typmod, JsValue *jsv,
StringInfoData buf;
initStringInfo(&buf);
- escape_json(&buf, str);
+ escape_json(&buf, str, len);
/* free temporary buffer */
if (str != json)
pfree(str);
@@ -4425,7 +4428,7 @@ sn_object_field_start(void *state, char *fname, bool isnull)
* Unfortunately we don't have the quoted and escaped string any more, so
* we have to re-escape it.
*/
- escape_json(_state->strval, fname);
+ escape_json_cstring(_state->strval, fname);
appendStringInfoCharMacro(_state->strval, ':');
@@ -4456,7 +4459,7 @@ sn_scalar(void *state, char *token, JsonTokenType tokentype)
}
if (tokentype == JSON_TOKEN_STRING)
- escape_json(_state->strval, token);
+ escape_json_cstring(_state->strval, token);
else
appendStringInfoString(_state->strval, token);
@@ -5888,7 +5891,7 @@ transform_string_values_object_field_start(void *state, char *fname, bool isnull
* Unfortunately we don't have the quoted and escaped string any more, so
* we have to re-escape it.
*/
- escape_json(_state->strval, fname);
+ escape_json_cstring(_state->strval, fname);
appendStringInfoCharMacro(_state->strval, ':');
return JSON_SUCCESS;
@@ -5914,7 +5917,7 @@ transform_string_values_scalar(void *state, char *token, JsonTokenType tokentype
{
text *out = _state->action(_state->action_state, token, strlen(token));
- escape_json(_state->strval, text_to_cstring(out));
+ escape_json_from_text(_state->strval, out);
}
else
appendStringInfoString(_state->strval, token);
diff --git a/src/backend/utils/adt/jsonpath.c b/src/backend/utils/adt/jsonpath.c
index 11e6193e96..82d65e4d4f 100644
--- a/src/backend/utils/adt/jsonpath.c
+++ b/src/backend/utils/adt/jsonpath.c
@@ -523,6 +523,8 @@ printJsonPathItem(StringInfo buf, JsonPathItem *v, bool inKey,
{
JsonPathItem elem;
int i;
+ int32 len;
+ char *str;
check_stack_depth();
CHECK_FOR_INTERRUPTS();
@@ -533,7 +535,8 @@ printJsonPathItem(StringInfo buf, JsonPathItem *v, bool inKey,
appendStringInfoString(buf, "null");
break;
case jpiString:
- escape_json(buf, jspGetString(v, NULL));
+ str = jspGetString(v, &len);
+ escape_json(buf, str, len);
break;
case jpiNumeric:
if (jspHasNext(v))
@@ -662,7 +665,8 @@ printJsonPathItem(StringInfo buf, JsonPathItem *v, bool inKey,
case jpiKey:
if (inKey)
appendStringInfoChar(buf, '.');
- escape_json(buf, jspGetString(v, NULL));
+ str = jspGetString(v, &len);
+ escape_json(buf, str, len);
break;
case jpiCurrent:
Assert(!inKey);
@@ -674,7 +678,8 @@ printJsonPathItem(StringInfo buf, JsonPathItem *v, bool inKey,
break;
case jpiVariable:
appendStringInfoChar(buf, '$');
- escape_json(buf, jspGetString(v, NULL));
+ str = jspGetString(v, &len);
+ escape_json(buf, str, len);
break;
case jpiFilter:
appendStringInfoString(buf, "?(");
@@ -732,7 +737,9 @@ printJsonPathItem(StringInfo buf, JsonPathItem *v, bool inKey,
appendStringInfoString(buf, " like_regex ");
- escape_json(buf, v->content.like_regex.pattern);
+ escape_json(buf,
+ v->content.like_regex.pattern,
+ v->content.like_regex.patternlen);
if (v->content.like_regex.flags)
{
diff --git a/src/backend/utils/error/jsonlog.c b/src/backend/utils/error/jsonlog.c
index bd0124869d..bd21f6ef58 100644
--- a/src/backend/utils/error/jsonlog.c
+++ b/src/backend/utils/error/jsonlog.c
@@ -49,11 +49,11 @@ appendJSONKeyValue(StringInfo buf, const char *key, const char *value,
return;
appendStringInfoChar(buf, ',');
- escape_json(buf, key);
+ escape_json_cstring(buf, key);
appendStringInfoChar(buf, ':');
if (escape_value)
- escape_json(buf, value);
+ escape_json_cstring(buf, value);
else
appendStringInfoString(buf, value);
}
@@ -143,9 +143,9 @@ write_jsonlog(ErrorData *edata)
* First property does not use appendJSONKeyValue as it does not have
* comma prefix.
*/
- escape_json(&buf, "timestamp");
+ escape_json(&buf, "timestamp", strlen("timestamp"));
appendStringInfoChar(&buf, ':');
- escape_json(&buf, log_time);
+ escape_json_cstring(&buf, log_time);
/* username */
if (MyProcPort)
diff --git a/src/include/utils/json.h b/src/include/utils/json.h
index 6d7f1b387d..8ede8d0bd6 100644
--- a/src/include/utils/json.h
+++ b/src/include/utils/json.h
@@ -17,7 +17,9 @@
#include "lib/stringinfo.h"
/* functions in json.c */
-extern void escape_json(StringInfo buf, const char *str);
+extern void escape_json_cstring(StringInfo buf, const char *str);
+extern void escape_json(StringInfo buf, const char *str, int len);
+extern void escape_json_from_text(StringInfo buf, const text *t);
extern char *JsonEncodeDateTime(char *buf, Datum value, Oid typid,
const int *tzp);
extern bool to_json_is_immutable(Oid typoid);
--
2.34.1
v1-0002-Use-SIMD-processing-for-escape_json.patchapplication/octet-stream; name=v1-0002-Use-SIMD-processing-for-escape_json.patchDownload
From d9dfd04b40c451cf28fc146d6242aa7f96e02db2 Mon Sep 17 00:00:00 2001
From: David Rowley <dgrowley@gmail.com>
Date: Thu, 23 May 2024 10:53:23 +1200
Subject: [PATCH v1 2/2] Use SIMD processing for escape_json()
---
src/backend/utils/adt/json.c | 72 +++++++++++++++++++++++++++++-
src/test/regress/expected/json.out | 44 ++++++++++++++++++
src/test/regress/sql/json.sql | 8 ++++
3 files changed, 122 insertions(+), 2 deletions(-)
diff --git a/src/backend/utils/adt/json.c b/src/backend/utils/adt/json.c
index 7934cf62fb..a266f60ff3 100644
--- a/src/backend/utils/adt/json.c
+++ b/src/backend/utils/adt/json.c
@@ -19,6 +19,7 @@
#include "funcapi.h"
#include "libpq/pqformat.h"
#include "miscadmin.h"
+#include "port/simd.h"
#include "utils/array.h"
#include "utils/builtins.h"
#include "utils/date.h"
@@ -1597,11 +1598,78 @@ escape_json_cstring(StringInfo buf, const char *str)
void
escape_json(StringInfo buf, const char *str, int len)
{
+ int i = 0;
+ int copypos = 0;
+
+ Assert(len >= 0);
+
appendStringInfoCharMacro(buf, '"');
- for (int i = 0; i < len; i++)
- escape_json_char(buf, str[i]);
+ for (;;)
+ {
+ Vector8 chunk;
+ int vlen;
+
+ /*
+ * Figure out how many bytes to process using SIMD. Round 'len' down
+ * to the previous multiple of sizeof(Vector8), assuming that's a
+ * power-of-2.
+ */
+ vlen = len & (int) (~(sizeof(Vector8) - 1));
+
+ /*
+ * To speed this up try searching sizeof(Vector8) bytes at once for
+ * special characters that we need to escape. When we find one, we
+ * fall out of this first loop and copy the parts we've vector
+ * searched before processing the special-char vector byte-by-byte.
+ * Once we're done with that, come back and try doing vector searching
+ * again. We'll also process the tail end of the string byte-by-byte.
+ */
+ for (; i < vlen; i += sizeof(Vector8))
+ {
+ vector8_load(&chunk, (const uint8 *) &str[i]);
+
+ /*
+ * Break on anything less than ' ' or if we find a '"' or '\\'.
+ * Those need special handling. That's done in the per-byte loop.
+ */
+ if (vector8_has_le(chunk, (unsigned char) 0x1F) ||
+ vector8_has(chunk, (unsigned char) '"') ||
+ vector8_has(chunk, (unsigned char) '\\'))
+ break;
+ }
+
+ /*
+ * Write to the destination up to the point of that we've vector
+ * searched so far. Do this only when switching into per-byte mode
+ * rather than once every sizeof(Vector8) bytes.
+ */
+ if (copypos < i)
+ {
+ appendBinaryStringInfo(buf, &str[copypos], i - copypos);
+ copypos = i;
+ }
+
+ /*
+ * Per-byte loop for Vector8s containing special chars and for
+ * processing the tail of the string.
+ */
+ for (int b = 0; b < sizeof(Vector8); b++)
+ {
+ /* check if we've finished */
+ if (i == len)
+ goto done;
+
+ Assert(i < len);
+
+ escape_json_char(buf, str[i++]);
+ }
+
+ copypos = i;
+ /* We're not done yet. Try the SIMD search again */
+ }
+done:
appendStringInfoCharMacro(buf, '"');
}
diff --git a/src/test/regress/expected/json.out b/src/test/regress/expected/json.out
index aa29bc597b..bfcc26c531 100644
--- a/src/test/regress/expected/json.out
+++ b/src/test/regress/expected/json.out
@@ -55,6 +55,50 @@ SELECT ('"'||repeat('.', 12)||'abc\n"')::json; -- OK, legal escapes
"............abc\n"
(1 row)
+-- Stress testing of JSON escape code
+CREATE TABLE json_escape (very_long_column_name_to_test_json_escape text);
+INSERT INTO json_escape SELECT repeat('a', a) FROM generate_series(0,33) a;
+-- Test various lengths of strings to validate SIMD processing to escape
+-- special chars in the JSON.
+SELECT row_to_json(j)::jsonb FROM json_escape j;
+ row_to_json
+------------------------------------------------------------------------------------
+ {"very_long_column_name_to_test_json_escape": ""}
+ {"very_long_column_name_to_test_json_escape": "a"}
+ {"very_long_column_name_to_test_json_escape": "aa"}
+ {"very_long_column_name_to_test_json_escape": "aaa"}
+ {"very_long_column_name_to_test_json_escape": "aaaa"}
+ {"very_long_column_name_to_test_json_escape": "aaaaa"}
+ {"very_long_column_name_to_test_json_escape": "aaaaaa"}
+ {"very_long_column_name_to_test_json_escape": "aaaaaaa"}
+ {"very_long_column_name_to_test_json_escape": "aaaaaaaa"}
+ {"very_long_column_name_to_test_json_escape": "aaaaaaaaa"}
+ {"very_long_column_name_to_test_json_escape": "aaaaaaaaaa"}
+ {"very_long_column_name_to_test_json_escape": "aaaaaaaaaaa"}
+ {"very_long_column_name_to_test_json_escape": "aaaaaaaaaaaa"}
+ {"very_long_column_name_to_test_json_escape": "aaaaaaaaaaaaa"}
+ {"very_long_column_name_to_test_json_escape": "aaaaaaaaaaaaaa"}
+ {"very_long_column_name_to_test_json_escape": "aaaaaaaaaaaaaaa"}
+ {"very_long_column_name_to_test_json_escape": "aaaaaaaaaaaaaaaa"}
+ {"very_long_column_name_to_test_json_escape": "aaaaaaaaaaaaaaaaa"}
+ {"very_long_column_name_to_test_json_escape": "aaaaaaaaaaaaaaaaaa"}
+ {"very_long_column_name_to_test_json_escape": "aaaaaaaaaaaaaaaaaaa"}
+ {"very_long_column_name_to_test_json_escape": "aaaaaaaaaaaaaaaaaaaa"}
+ {"very_long_column_name_to_test_json_escape": "aaaaaaaaaaaaaaaaaaaaa"}
+ {"very_long_column_name_to_test_json_escape": "aaaaaaaaaaaaaaaaaaaaaa"}
+ {"very_long_column_name_to_test_json_escape": "aaaaaaaaaaaaaaaaaaaaaaa"}
+ {"very_long_column_name_to_test_json_escape": "aaaaaaaaaaaaaaaaaaaaaaaa"}
+ {"very_long_column_name_to_test_json_escape": "aaaaaaaaaaaaaaaaaaaaaaaaa"}
+ {"very_long_column_name_to_test_json_escape": "aaaaaaaaaaaaaaaaaaaaaaaaaa"}
+ {"very_long_column_name_to_test_json_escape": "aaaaaaaaaaaaaaaaaaaaaaaaaaa"}
+ {"very_long_column_name_to_test_json_escape": "aaaaaaaaaaaaaaaaaaaaaaaaaaaa"}
+ {"very_long_column_name_to_test_json_escape": "aaaaaaaaaaaaaaaaaaaaaaaaaaaaa"}
+ {"very_long_column_name_to_test_json_escape": "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaa"}
+ {"very_long_column_name_to_test_json_escape": "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa"}
+ {"very_long_column_name_to_test_json_escape": "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa"}
+ {"very_long_column_name_to_test_json_escape": "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa"}
+(34 rows)
+
-- see json_encoding test for input with unicode escapes
-- Numbers.
SELECT '1'::json; -- OK
diff --git a/src/test/regress/sql/json.sql b/src/test/regress/sql/json.sql
index ec57dfe707..0e7ca2f5af 100644
--- a/src/test/regress/sql/json.sql
+++ b/src/test/regress/sql/json.sql
@@ -12,6 +12,14 @@ SELECT '"\v"'::json; -- ERROR, not a valid JSON escape
SELECT ('"'||repeat('.', 12)||'abc"')::json; -- OK
SELECT ('"'||repeat('.', 12)||'abc\n"')::json; -- OK, legal escapes
+-- Stress testing of JSON escape code
+CREATE TABLE json_escape (very_long_column_name_to_test_json_escape text);
+INSERT INTO json_escape SELECT repeat('a', a) FROM generate_series(0,33) a;
+
+-- Test various lengths of strings to validate SIMD processing to escape
+-- special chars in the JSON.
+SELECT row_to_json(j)::jsonb FROM json_escape j;
+
-- see json_encoding test for input with unicode escapes
-- Numbers.
--
2.34.1
On Thu, 23 May 2024 at 13:23, David Rowley <dgrowleyml@gmail.com> wrote:
Master:
$ pgbench -n -f bench.sql -T 10 -M prepared postgres | grep tps
tps = 362.494309 (without initial connection time)
tps = 363.182458 (without initial connection time)
tps = 362.679654 (without initial connection time)Master + 0001 + 0002
$ pgbench -n -f bench.sql -T 10 -M prepared postgres | grep tps
tps = 426.456885 (without initial connection time)
tps = 430.573046 (without initial connection time)
tps = 431.142917 (without initial connection time)About 18% faster.
It would be much faster if we could also get rid of the
escape_json_cstring() call in the switch default case of
datum_to_json_internal(). row_to_json() would be heaps faster with
that done. I considered adding a special case for the "text" type
there, but in the end felt that we should just fix that with some
hypothetical other patch that changes how output functions work.
Others may feel it's worthwhile. I certainly could be convinced of it.
Just to turn that into performance numbers, I tried the attached
patch. The numbers came out better than I thought.
Same test as before:
master + 0001 + 0002 + attached hacks:
$ pgbench -n -f bench.sql -T 10 -M prepared postgres | grep tps
tps = 616.094394 (without initial connection time)
tps = 615.928236 (without initial connection time)
tps = 614.175494 (without initial connection time)
About 70% faster than master.
David
Attachments:
datum_to_json_internal.patch.txttext/plain; charset=US-ASCII; name=datum_to_json_internal.patch.txtDownload
diff --git a/src/backend/utils/adt/json.c b/src/backend/utils/adt/json.c
index a266f60ff3..b15f6c5e64 100644
--- a/src/backend/utils/adt/json.c
+++ b/src/backend/utils/adt/json.c
@@ -24,6 +24,7 @@
#include "utils/builtins.h"
#include "utils/date.h"
#include "utils/datetime.h"
+#include "utils/fmgroids.h"
#include "utils/json.h"
#include "utils/jsonfuncs.h"
#include "utils/lsyscache.h"
@@ -286,9 +287,15 @@ datum_to_json_internal(Datum val, bool is_null, StringInfo result,
pfree(jsontext);
break;
default:
- outputstr = OidOutputFunctionCall(outfuncoid, val);
- escape_json_cstring(result, outputstr);
- pfree(outputstr);
+ /* special case common case for text types */
+ if (outfuncoid == F_TEXTOUT)
+ escape_json_from_text(result, (text *) DatumGetPointer(val));
+ else
+ {
+ outputstr = OidOutputFunctionCall(outfuncoid, val);
+ escape_json_cstring(result, outputstr);
+ pfree(outputstr);
+ }
break;
}
}
On 2024-05-22 We 22:15, David Rowley wrote:
On Thu, 23 May 2024 at 13:23, David Rowley <dgrowleyml@gmail.com> wrote:
Master:
$ pgbench -n -f bench.sql -T 10 -M prepared postgres | grep tps
tps = 362.494309 (without initial connection time)
tps = 363.182458 (without initial connection time)
tps = 362.679654 (without initial connection time)Master + 0001 + 0002
$ pgbench -n -f bench.sql -T 10 -M prepared postgres | grep tps
tps = 426.456885 (without initial connection time)
tps = 430.573046 (without initial connection time)
tps = 431.142917 (without initial connection time)About 18% faster.
It would be much faster if we could also get rid of the
escape_json_cstring() call in the switch default case of
datum_to_json_internal(). row_to_json() would be heaps faster with
that done. I considered adding a special case for the "text" type
there, but in the end felt that we should just fix that with some
hypothetical other patch that changes how output functions work.
Others may feel it's worthwhile. I certainly could be convinced of it.Just to turn that into performance numbers, I tried the attached
patch. The numbers came out better than I thought.Same test as before:
master + 0001 + 0002 + attached hacks:
$ pgbench -n -f bench.sql -T 10 -M prepared postgres | grep tps
tps = 616.094394 (without initial connection time)
tps = 615.928236 (without initial connection time)
tps = 614.175494 (without initial connection time)About 70% faster than master.
That's all pretty nice! I'd take the win on this rather than wait for
some hypothetical patch that changes how output functions work.
cheers
andrew
--
Andrew Dunstan
EDB: https://www.enterprisedb.com
On Fri, 24 May 2024 at 08:34, Andrew Dunstan <andrew@dunslane.net> wrote:
That's all pretty nice! I'd take the win on this rather than wait for
some hypothetical patch that changes how output functions work.
On re-think of that, even if we changed the output functions to write
directly to a StringInfo, we wouldn't get the same speedup. All it
would get us is a better ability to know the length of the string the
output function generated by looking at the StringInfoData.len before
and after calling the output function. That *would* allow us to use
the SIMD escaping, but not save the palloc/memcpy cycle for
non-toasted Datums. In other words, if we want this speedup then I
don't see another way other than this special case.
I've attached a rebased patch series which includes the 3rd patch in a
more complete form. This one also adds handling for varchar and
char(n) output functions. Ideally, these would also use textout() to
save from having the ORs in the if condition. The output function code
is the same in each.
Updated benchmarks from the test in [1]/messages/by-id/CAApHDvpLXwMZvbCKcdGfU9XQjGCDm7tFpRdTXuB9PVgpNUYfEQ@mail.gmail.com.
master @ 7c655a04a
$ for i in {1..3}; do pgbench -n -f bench.sql -T 10 -M prepared
postgres | grep tps; done
tps = 366.211426
tps = 359.707014
tps = 362.204383
master + 0001
$ for i in {1..3}; do pgbench -n -f bench.sql -T 10 -M prepared
postgres | grep tps; done
tps = 362.641668
tps = 367.986495
tps = 368.698193 (+1% vs master)
master + 0001 + 0002
$ for i in {1..3}; do pgbench -n -f bench.sql -T 10 -M prepared
postgres | grep tps; done
tps = 430.477314
tps = 425.173469
tps = 431.013275 (+18% vs master)
master + 0001 + 0002 + 0003
$ for i in {1..3}; do pgbench -n -f bench.sql -T 10 -M prepared
postgres | grep tps; done
tps = 606.702305
tps = 625.727031
tps = 617.164822 (+70% vs master)
David
[1]: /messages/by-id/CAApHDvpLXwMZvbCKcdGfU9XQjGCDm7tFpRdTXuB9PVgpNUYfEQ@mail.gmail.com
Attachments:
v2-0001-Add-len-parameter-to-escape_json.patchapplication/octet-stream; name=v2-0001-Add-len-parameter-to-escape_json.patchDownload
From e2b7fec5c3bdf12f8339309dd6b59f061e0d12dc Mon Sep 17 00:00:00 2001
From: David Rowley <dgrowley@gmail.com>
Date: Tue, 21 May 2024 17:14:06 +1200
Subject: [PATCH v2 1/3] Add 'len' parameter to escape_json()
---
contrib/hstore/hstore_io.c | 41 +++-----
src/backend/backup/backup_manifest.c | 2 +-
src/backend/commands/explain.c | 30 ++++--
src/backend/parser/parse_jsontable.c | 2 +-
src/backend/utils/adt/json.c | 150 +++++++++++++++++----------
src/backend/utils/adt/jsonb.c | 2 +-
src/backend/utils/adt/jsonfuncs.c | 15 +--
src/backend/utils/adt/jsonpath.c | 15 ++-
src/backend/utils/error/jsonlog.c | 8 +-
src/include/utils/json.h | 4 +-
10 files changed, 162 insertions(+), 107 deletions(-)
diff --git a/contrib/hstore/hstore_io.c b/contrib/hstore/hstore_io.c
index 999ddad76d..374b8d05ef 100644
--- a/contrib/hstore/hstore_io.c
+++ b/contrib/hstore/hstore_io.c
@@ -1343,23 +1343,20 @@ hstore_to_json_loose(PG_FUNCTION_ARGS)
int count = HS_COUNT(in);
char *base = STRPTR(in);
HEntry *entries = ARRPTR(in);
- StringInfoData tmp,
- dst;
+ StringInfoData dst;
if (count == 0)
PG_RETURN_TEXT_P(cstring_to_text_with_len("{}", 2));
- initStringInfo(&tmp);
initStringInfo(&dst);
appendStringInfoChar(&dst, '{');
for (i = 0; i < count; i++)
{
- resetStringInfo(&tmp);
- appendBinaryStringInfo(&tmp, HSTORE_KEY(entries, base, i),
- HSTORE_KEYLEN(entries, i));
- escape_json(&dst, tmp.data);
+ escape_json(&dst,
+ HSTORE_KEY(entries, base, i),
+ HSTORE_KEYLEN(entries, i));
appendStringInfoString(&dst, ": ");
if (HSTORE_VALISNULL(entries, i))
appendStringInfoString(&dst, "null");
@@ -1372,13 +1369,13 @@ hstore_to_json_loose(PG_FUNCTION_ARGS)
appendStringInfoString(&dst, "false");
else
{
- resetStringInfo(&tmp);
- appendBinaryStringInfo(&tmp, HSTORE_VAL(entries, base, i),
- HSTORE_VALLEN(entries, i));
- if (IsValidJsonNumber(tmp.data, tmp.len))
- appendBinaryStringInfo(&dst, tmp.data, tmp.len);
+ char *str = HSTORE_VAL(entries, base, i);
+ int len = HSTORE_VALLEN(entries, i);
+
+ if (IsValidJsonNumber(str, len))
+ appendBinaryStringInfo(&dst, str, len);
else
- escape_json(&dst, tmp.data);
+ escape_json(&dst, str, len);
}
if (i + 1 != count)
@@ -1398,32 +1395,28 @@ hstore_to_json(PG_FUNCTION_ARGS)
int count = HS_COUNT(in);
char *base = STRPTR(in);
HEntry *entries = ARRPTR(in);
- StringInfoData tmp,
- dst;
+ StringInfoData dst;
if (count == 0)
PG_RETURN_TEXT_P(cstring_to_text_with_len("{}", 2));
- initStringInfo(&tmp);
initStringInfo(&dst);
appendStringInfoChar(&dst, '{');
for (i = 0; i < count; i++)
{
- resetStringInfo(&tmp);
- appendBinaryStringInfo(&tmp, HSTORE_KEY(entries, base, i),
- HSTORE_KEYLEN(entries, i));
- escape_json(&dst, tmp.data);
+ escape_json(&dst,
+ HSTORE_KEY(entries, base, i),
+ HSTORE_KEYLEN(entries, i));
appendStringInfoString(&dst, ": ");
if (HSTORE_VALISNULL(entries, i))
appendStringInfoString(&dst, "null");
else
{
- resetStringInfo(&tmp);
- appendBinaryStringInfo(&tmp, HSTORE_VAL(entries, base, i),
- HSTORE_VALLEN(entries, i));
- escape_json(&dst, tmp.data);
+ escape_json(&dst,
+ HSTORE_VAL(entries, base, i),
+ HSTORE_VALLEN(entries, i));
}
if (i + 1 != count)
diff --git a/src/backend/backup/backup_manifest.c b/src/backend/backup/backup_manifest.c
index b360a13547..3da8da00d6 100644
--- a/src/backend/backup/backup_manifest.c
+++ b/src/backend/backup/backup_manifest.c
@@ -148,7 +148,7 @@ AddFileToBackupManifest(backup_manifest_info *manifest, Oid spcoid,
pg_verify_mbstr(PG_UTF8, pathname, pathlen, true))
{
appendStringInfoString(&buf, "{ \"Path\": ");
- escape_json(&buf, pathname);
+ escape_json(&buf, pathname, pathlen);
appendStringInfoString(&buf, ", ");
}
else
diff --git a/src/backend/commands/explain.c b/src/backend/commands/explain.c
index 94511a5a02..399025aed3 100644
--- a/src/backend/commands/explain.c
+++ b/src/backend/commands/explain.c
@@ -4661,13 +4661,15 @@ ExplainPropertyList(const char *qlabel, List *data, ExplainState *es)
case EXPLAIN_FORMAT_JSON:
ExplainJSONLineEnding(es);
appendStringInfoSpaces(es->str, es->indent * 2);
- escape_json(es->str, qlabel);
+ escape_json_cstring(es->str, qlabel);
appendStringInfoString(es->str, ": [");
foreach(lc, data)
{
+ const char *str = (const char *) lfirst(lc);
+
if (!first)
appendStringInfoString(es->str, ", ");
- escape_json(es->str, (const char *) lfirst(lc));
+ escape_json_cstring(es->str, str);
first = false;
}
appendStringInfoChar(es->str, ']');
@@ -4678,10 +4680,12 @@ ExplainPropertyList(const char *qlabel, List *data, ExplainState *es)
appendStringInfo(es->str, "%s: ", qlabel);
foreach(lc, data)
{
+ const char *str = (const char *) lfirst(lc);
+
appendStringInfoChar(es->str, '\n');
appendStringInfoSpaces(es->str, es->indent * 2 + 2);
appendStringInfoString(es->str, "- ");
- escape_yaml(es->str, (const char *) lfirst(lc));
+ escape_yaml(es->str, str);
}
break;
}
@@ -4710,9 +4714,11 @@ ExplainPropertyListNested(const char *qlabel, List *data, ExplainState *es)
appendStringInfoChar(es->str, '[');
foreach(lc, data)
{
+ const char *str = (const char *) lfirst(lc);
+
if (!first)
appendStringInfoString(es->str, ", ");
- escape_json(es->str, (const char *) lfirst(lc));
+ escape_json_cstring(es->str, str);
first = false;
}
appendStringInfoChar(es->str, ']');
@@ -4723,9 +4729,11 @@ ExplainPropertyListNested(const char *qlabel, List *data, ExplainState *es)
appendStringInfoString(es->str, "- [");
foreach(lc, data)
{
+ const char *str = (const char *) lfirst(lc);
+
if (!first)
appendStringInfoString(es->str, ", ");
- escape_yaml(es->str, (const char *) lfirst(lc));
+ escape_yaml(es->str, str);
first = false;
}
appendStringInfoChar(es->str, ']');
@@ -4775,12 +4783,12 @@ ExplainProperty(const char *qlabel, const char *unit, const char *value,
case EXPLAIN_FORMAT_JSON:
ExplainJSONLineEnding(es);
appendStringInfoSpaces(es->str, es->indent * 2);
- escape_json(es->str, qlabel);
+ escape_json_cstring(es->str, qlabel);
appendStringInfoString(es->str, ": ");
if (numeric)
appendStringInfoString(es->str, value);
else
- escape_json(es->str, value);
+ escape_json_cstring(es->str, value);
break;
case EXPLAIN_FORMAT_YAML:
@@ -4882,7 +4890,7 @@ ExplainOpenGroup(const char *objtype, const char *labelname,
appendStringInfoSpaces(es->str, 2 * es->indent);
if (labelname)
{
- escape_json(es->str, labelname);
+ escape_json_cstring(es->str, labelname);
appendStringInfoString(es->str, ": ");
}
appendStringInfoChar(es->str, labeled ? '{' : '[');
@@ -5090,10 +5098,10 @@ ExplainDummyGroup(const char *objtype, const char *labelname, ExplainState *es)
appendStringInfoSpaces(es->str, 2 * es->indent);
if (labelname)
{
- escape_json(es->str, labelname);
+ escape_json_cstring(es->str, labelname);
appendStringInfoString(es->str, ": ");
}
- escape_json(es->str, objtype);
+ escape_json_cstring(es->str, objtype);
break;
case EXPLAIN_FORMAT_YAML:
@@ -5297,7 +5305,7 @@ ExplainYAMLLineStarting(ExplainState *es)
static void
escape_yaml(StringInfo buf, const char *str)
{
- escape_json(buf, str);
+ escape_json_cstring(buf, str);
}
diff --git a/src/backend/parser/parse_jsontable.c b/src/backend/parser/parse_jsontable.c
index b2519c2f32..03aac2f0cd 100644
--- a/src/backend/parser/parse_jsontable.c
+++ b/src/backend/parser/parse_jsontable.c
@@ -427,7 +427,7 @@ transformJsonTableColumn(JsonTableColumn *jtc, Node *contextItemExpr,
initStringInfo(&path);
appendStringInfoString(&path, "$.");
- escape_json(&path, jtc->name);
+ escape_json_cstring(&path, jtc->name);
pathspec = makeStringConst(path.data, -1);
}
diff --git a/src/backend/utils/adt/json.c b/src/backend/utils/adt/json.c
index d719a61f16..7934cf62fb 100644
--- a/src/backend/utils/adt/json.c
+++ b/src/backend/utils/adt/json.c
@@ -286,7 +286,7 @@ datum_to_json_internal(Datum val, bool is_null, StringInfo result,
break;
default:
outputstr = OidOutputFunctionCall(outfuncoid, val);
- escape_json(result, outputstr);
+ escape_json_cstring(result, outputstr);
pfree(outputstr);
break;
}
@@ -560,7 +560,7 @@ composite_to_json(Datum composite, StringInfo result, bool use_line_feeds)
needsep = true;
attname = NameStr(att->attname);
- escape_json(result, attname);
+ escape_json_cstring(result, attname);
appendStringInfoChar(result, ':');
val = heap_getattr(tuple, i + 1, tupdesc, &isnull);
@@ -1391,7 +1391,6 @@ json_object(PG_FUNCTION_ARGS)
count,
i;
text *rval;
- char *v;
switch (ndims)
{
@@ -1434,19 +1433,17 @@ json_object(PG_FUNCTION_ARGS)
(errcode(ERRCODE_NULL_VALUE_NOT_ALLOWED),
errmsg("null value not allowed for object key")));
- v = TextDatumGetCString(in_datums[i * 2]);
if (i > 0)
appendStringInfoString(&result, ", ");
- escape_json(&result, v);
+ escape_json_from_text(&result,
+ (text *) DatumGetPointer(in_datums[i * 2]));
appendStringInfoString(&result, " : ");
- pfree(v);
if (in_nulls[i * 2 + 1])
appendStringInfoString(&result, "null");
else
{
- v = TextDatumGetCString(in_datums[i * 2 + 1]);
- escape_json(&result, v);
- pfree(v);
+ escape_json_from_text(&result,
+ (text *) DatumGetPointer(in_datums[i * 2 + 1]));
}
}
@@ -1483,7 +1480,6 @@ json_object_two_arg(PG_FUNCTION_ARGS)
val_count,
i;
text *rval;
- char *v;
if (nkdims > 1 || nkdims != nvdims)
ereport(ERROR,
@@ -1512,20 +1508,17 @@ json_object_two_arg(PG_FUNCTION_ARGS)
(errcode(ERRCODE_NULL_VALUE_NOT_ALLOWED),
errmsg("null value not allowed for object key")));
- v = TextDatumGetCString(key_datums[i]);
if (i > 0)
appendStringInfoString(&result, ", ");
- escape_json(&result, v);
+ escape_json_from_text(&result,
+ (text *) DatumGetPointer(key_datums[i]));
+
appendStringInfoString(&result, " : ");
- pfree(v);
if (val_nulls[i])
appendStringInfoString(&result, "null");
else
- {
- v = TextDatumGetCString(val_datums[i]);
- escape_json(&result, v);
- pfree(v);
- }
+ escape_json_from_text(&result,
+ (text *) DatumGetPointer(val_datums[i]));
}
appendStringInfoChar(&result, '}');
@@ -1541,52 +1534,101 @@ json_object_two_arg(PG_FUNCTION_ARGS)
PG_RETURN_TEXT_P(rval);
}
+/*
+ * escape_json_char
+ * Inline helper function for escape_json* functions
+ */
+static pg_attribute_always_inline void
+escape_json_char(StringInfo buf, char c)
+{
+ switch (c)
+ {
+ case '\b':
+ appendStringInfoString(buf, "\\b");
+ break;
+ case '\f':
+ appendStringInfoString(buf, "\\f");
+ break;
+ case '\n':
+ appendStringInfoString(buf, "\\n");
+ break;
+ case '\r':
+ appendStringInfoString(buf, "\\r");
+ break;
+ case '\t':
+ appendStringInfoString(buf, "\\t");
+ break;
+ case '"':
+ appendStringInfoString(buf, "\\\"");
+ break;
+ case '\\':
+ appendStringInfoString(buf, "\\\\");
+ break;
+ default:
+ if ((unsigned char) c < ' ')
+ appendStringInfo(buf, "\\u%04x", (int) c);
+ else
+ appendStringInfoCharMacro(buf, c);
+ break;
+ }
+}
/*
- * Produce a JSON string literal, properly escaping characters in the text.
+ * escape_json_cstring
+ * Produce a JSON string literal. Same as escape_json() except takes a
+ * NUL-terminated string as input.
*/
void
-escape_json(StringInfo buf, const char *str)
+escape_json_cstring(StringInfo buf, const char *str)
{
- const char *p;
+ appendStringInfoCharMacro(buf, '"');
+
+ for (; *str != '\0'; str++)
+ escape_json_char(buf, *str);
appendStringInfoCharMacro(buf, '"');
- for (p = str; *p; p++)
- {
- switch (*p)
- {
- case '\b':
- appendStringInfoString(buf, "\\b");
- break;
- case '\f':
- appendStringInfoString(buf, "\\f");
- break;
- case '\n':
- appendStringInfoString(buf, "\\n");
- break;
- case '\r':
- appendStringInfoString(buf, "\\r");
- break;
- case '\t':
- appendStringInfoString(buf, "\\t");
- break;
- case '"':
- appendStringInfoString(buf, "\\\"");
- break;
- case '\\':
- appendStringInfoString(buf, "\\\\");
- break;
- default:
- if ((unsigned char) *p < ' ')
- appendStringInfo(buf, "\\u%04x", (int) *p);
- else
- appendStringInfoCharMacro(buf, *p);
- break;
- }
- }
+}
+
+/*
+ * Produce a JSON string literal, properly escaping the possibly not
+ * NUL-terminated characters in 'str'. 'len' defines the number of bytes from
+ * 'str' to process.
+ */
+void
+escape_json(StringInfo buf, const char *str, int len)
+{
+ appendStringInfoCharMacro(buf, '"');
+
+ for (int i = 0; i < len; i++)
+ escape_json_char(buf, str[i]);
+
appendStringInfoCharMacro(buf, '"');
}
+/*
+ * escape_json_from_text
+ * Append 't' onto 'buf' and escape using escape_json.
+ *
+ * This is more efficient than calling text_to_cstring and appending the
+ * result as that could require an additional palloc and memcpy.
+ */
+void
+escape_json_from_text(StringInfo buf, const text *t)
+{
+ /* must cast away the const, unfortunately */
+ text *tunpacked = pg_detoast_datum_packed(unconstify(text *, t));
+ int len = VARSIZE_ANY_EXHDR(tunpacked);
+ char *str;
+
+ str = VARDATA_ANY(tunpacked);
+
+ escape_json(buf, str, len);
+
+ /* pfree any detoasted values */
+ if (tunpacked != t)
+ pfree(tunpacked);
+}
+
/* Semantic actions for key uniqueness check */
static JsonParseErrorType
json_unique_object_start(void *_state)
diff --git a/src/backend/utils/adt/jsonb.c b/src/backend/utils/adt/jsonb.c
index e4562b3c6c..f24d8003be 100644
--- a/src/backend/utils/adt/jsonb.c
+++ b/src/backend/utils/adt/jsonb.c
@@ -354,7 +354,7 @@ jsonb_put_escaped_value(StringInfo out, JsonbValue *scalarVal)
appendBinaryStringInfo(out, "null", 4);
break;
case jbvString:
- escape_json(out, pnstrdup(scalarVal->val.string.val, scalarVal->val.string.len));
+ escape_json(out, scalarVal->val.string.val, scalarVal->val.string.len);
break;
case jbvNumeric:
appendStringInfoString(out,
diff --git a/src/backend/utils/adt/jsonfuncs.c b/src/backend/utils/adt/jsonfuncs.c
index 83125b06a4..743d47dae6 100644
--- a/src/backend/utils/adt/jsonfuncs.c
+++ b/src/backend/utils/adt/jsonfuncs.c
@@ -3139,7 +3139,10 @@ populate_scalar(ScalarIOData *io, Oid typid, int32 typmod, JsValue *jsv,
str[len] = '\0';
}
else
- str = json; /* string is already null-terminated */
+ {
+ str = json; /* string is already null-terminated */
+ len = strlen(str);
+ }
/* If converting to json/jsonb, make string into valid JSON literal */
if ((typid == JSONOID || typid == JSONBOID) &&
@@ -3148,7 +3151,7 @@ populate_scalar(ScalarIOData *io, Oid typid, int32 typmod, JsValue *jsv,
StringInfoData buf;
initStringInfo(&buf);
- escape_json(&buf, str);
+ escape_json(&buf, str, len);
/* free temporary buffer */
if (str != json)
pfree(str);
@@ -4425,7 +4428,7 @@ sn_object_field_start(void *state, char *fname, bool isnull)
* Unfortunately we don't have the quoted and escaped string any more, so
* we have to re-escape it.
*/
- escape_json(_state->strval, fname);
+ escape_json_cstring(_state->strval, fname);
appendStringInfoCharMacro(_state->strval, ':');
@@ -4456,7 +4459,7 @@ sn_scalar(void *state, char *token, JsonTokenType tokentype)
}
if (tokentype == JSON_TOKEN_STRING)
- escape_json(_state->strval, token);
+ escape_json_cstring(_state->strval, token);
else
appendStringInfoString(_state->strval, token);
@@ -5888,7 +5891,7 @@ transform_string_values_object_field_start(void *state, char *fname, bool isnull
* Unfortunately we don't have the quoted and escaped string any more, so
* we have to re-escape it.
*/
- escape_json(_state->strval, fname);
+ escape_json_cstring(_state->strval, fname);
appendStringInfoCharMacro(_state->strval, ':');
return JSON_SUCCESS;
@@ -5914,7 +5917,7 @@ transform_string_values_scalar(void *state, char *token, JsonTokenType tokentype
{
text *out = _state->action(_state->action_state, token, strlen(token));
- escape_json(_state->strval, text_to_cstring(out));
+ escape_json_from_text(_state->strval, out);
}
else
appendStringInfoString(_state->strval, token);
diff --git a/src/backend/utils/adt/jsonpath.c b/src/backend/utils/adt/jsonpath.c
index 11e6193e96..82d65e4d4f 100644
--- a/src/backend/utils/adt/jsonpath.c
+++ b/src/backend/utils/adt/jsonpath.c
@@ -523,6 +523,8 @@ printJsonPathItem(StringInfo buf, JsonPathItem *v, bool inKey,
{
JsonPathItem elem;
int i;
+ int32 len;
+ char *str;
check_stack_depth();
CHECK_FOR_INTERRUPTS();
@@ -533,7 +535,8 @@ printJsonPathItem(StringInfo buf, JsonPathItem *v, bool inKey,
appendStringInfoString(buf, "null");
break;
case jpiString:
- escape_json(buf, jspGetString(v, NULL));
+ str = jspGetString(v, &len);
+ escape_json(buf, str, len);
break;
case jpiNumeric:
if (jspHasNext(v))
@@ -662,7 +665,8 @@ printJsonPathItem(StringInfo buf, JsonPathItem *v, bool inKey,
case jpiKey:
if (inKey)
appendStringInfoChar(buf, '.');
- escape_json(buf, jspGetString(v, NULL));
+ str = jspGetString(v, &len);
+ escape_json(buf, str, len);
break;
case jpiCurrent:
Assert(!inKey);
@@ -674,7 +678,8 @@ printJsonPathItem(StringInfo buf, JsonPathItem *v, bool inKey,
break;
case jpiVariable:
appendStringInfoChar(buf, '$');
- escape_json(buf, jspGetString(v, NULL));
+ str = jspGetString(v, &len);
+ escape_json(buf, str, len);
break;
case jpiFilter:
appendStringInfoString(buf, "?(");
@@ -732,7 +737,9 @@ printJsonPathItem(StringInfo buf, JsonPathItem *v, bool inKey,
appendStringInfoString(buf, " like_regex ");
- escape_json(buf, v->content.like_regex.pattern);
+ escape_json(buf,
+ v->content.like_regex.pattern,
+ v->content.like_regex.patternlen);
if (v->content.like_regex.flags)
{
diff --git a/src/backend/utils/error/jsonlog.c b/src/backend/utils/error/jsonlog.c
index bd0124869d..bd21f6ef58 100644
--- a/src/backend/utils/error/jsonlog.c
+++ b/src/backend/utils/error/jsonlog.c
@@ -49,11 +49,11 @@ appendJSONKeyValue(StringInfo buf, const char *key, const char *value,
return;
appendStringInfoChar(buf, ',');
- escape_json(buf, key);
+ escape_json_cstring(buf, key);
appendStringInfoChar(buf, ':');
if (escape_value)
- escape_json(buf, value);
+ escape_json_cstring(buf, value);
else
appendStringInfoString(buf, value);
}
@@ -143,9 +143,9 @@ write_jsonlog(ErrorData *edata)
* First property does not use appendJSONKeyValue as it does not have
* comma prefix.
*/
- escape_json(&buf, "timestamp");
+ escape_json(&buf, "timestamp", strlen("timestamp"));
appendStringInfoChar(&buf, ':');
- escape_json(&buf, log_time);
+ escape_json_cstring(&buf, log_time);
/* username */
if (MyProcPort)
diff --git a/src/include/utils/json.h b/src/include/utils/json.h
index 6d7f1b387d..8ede8d0bd6 100644
--- a/src/include/utils/json.h
+++ b/src/include/utils/json.h
@@ -17,7 +17,9 @@
#include "lib/stringinfo.h"
/* functions in json.c */
-extern void escape_json(StringInfo buf, const char *str);
+extern void escape_json_cstring(StringInfo buf, const char *str);
+extern void escape_json(StringInfo buf, const char *str, int len);
+extern void escape_json_from_text(StringInfo buf, const text *t);
extern char *JsonEncodeDateTime(char *buf, Datum value, Oid typid,
const int *tzp);
extern bool to_json_is_immutable(Oid typoid);
--
2.34.1
v2-0002-Use-SIMD-processing-for-escape_json.patchapplication/octet-stream; name=v2-0002-Use-SIMD-processing-for-escape_json.patchDownload
From 36e226c368d2eb37c41124f52ef819bc626fd5a8 Mon Sep 17 00:00:00 2001
From: David Rowley <dgrowley@gmail.com>
Date: Thu, 23 May 2024 10:53:23 +1200
Subject: [PATCH v2 2/3] Use SIMD processing for escape_json()
---
src/backend/utils/adt/json.c | 72 +++++++++++++++++++++++++++++-
src/test/regress/expected/json.out | 44 ++++++++++++++++++
src/test/regress/sql/json.sql | 8 ++++
3 files changed, 122 insertions(+), 2 deletions(-)
diff --git a/src/backend/utils/adt/json.c b/src/backend/utils/adt/json.c
index 7934cf62fb..a266f60ff3 100644
--- a/src/backend/utils/adt/json.c
+++ b/src/backend/utils/adt/json.c
@@ -19,6 +19,7 @@
#include "funcapi.h"
#include "libpq/pqformat.h"
#include "miscadmin.h"
+#include "port/simd.h"
#include "utils/array.h"
#include "utils/builtins.h"
#include "utils/date.h"
@@ -1597,11 +1598,78 @@ escape_json_cstring(StringInfo buf, const char *str)
void
escape_json(StringInfo buf, const char *str, int len)
{
+ int i = 0;
+ int copypos = 0;
+
+ Assert(len >= 0);
+
appendStringInfoCharMacro(buf, '"');
- for (int i = 0; i < len; i++)
- escape_json_char(buf, str[i]);
+ for (;;)
+ {
+ Vector8 chunk;
+ int vlen;
+
+ /*
+ * Figure out how many bytes to process using SIMD. Round 'len' down
+ * to the previous multiple of sizeof(Vector8), assuming that's a
+ * power-of-2.
+ */
+ vlen = len & (int) (~(sizeof(Vector8) - 1));
+
+ /*
+ * To speed this up try searching sizeof(Vector8) bytes at once for
+ * special characters that we need to escape. When we find one, we
+ * fall out of this first loop and copy the parts we've vector
+ * searched before processing the special-char vector byte-by-byte.
+ * Once we're done with that, come back and try doing vector searching
+ * again. We'll also process the tail end of the string byte-by-byte.
+ */
+ for (; i < vlen; i += sizeof(Vector8))
+ {
+ vector8_load(&chunk, (const uint8 *) &str[i]);
+
+ /*
+ * Break on anything less than ' ' or if we find a '"' or '\\'.
+ * Those need special handling. That's done in the per-byte loop.
+ */
+ if (vector8_has_le(chunk, (unsigned char) 0x1F) ||
+ vector8_has(chunk, (unsigned char) '"') ||
+ vector8_has(chunk, (unsigned char) '\\'))
+ break;
+ }
+
+ /*
+ * Write to the destination up to the point of that we've vector
+ * searched so far. Do this only when switching into per-byte mode
+ * rather than once every sizeof(Vector8) bytes.
+ */
+ if (copypos < i)
+ {
+ appendBinaryStringInfo(buf, &str[copypos], i - copypos);
+ copypos = i;
+ }
+
+ /*
+ * Per-byte loop for Vector8s containing special chars and for
+ * processing the tail of the string.
+ */
+ for (int b = 0; b < sizeof(Vector8); b++)
+ {
+ /* check if we've finished */
+ if (i == len)
+ goto done;
+
+ Assert(i < len);
+
+ escape_json_char(buf, str[i++]);
+ }
+
+ copypos = i;
+ /* We're not done yet. Try the SIMD search again */
+ }
+done:
appendStringInfoCharMacro(buf, '"');
}
diff --git a/src/test/regress/expected/json.out b/src/test/regress/expected/json.out
index aa29bc597b..bfcc26c531 100644
--- a/src/test/regress/expected/json.out
+++ b/src/test/regress/expected/json.out
@@ -55,6 +55,50 @@ SELECT ('"'||repeat('.', 12)||'abc\n"')::json; -- OK, legal escapes
"............abc\n"
(1 row)
+-- Stress testing of JSON escape code
+CREATE TABLE json_escape (very_long_column_name_to_test_json_escape text);
+INSERT INTO json_escape SELECT repeat('a', a) FROM generate_series(0,33) a;
+-- Test various lengths of strings to validate SIMD processing to escape
+-- special chars in the JSON.
+SELECT row_to_json(j)::jsonb FROM json_escape j;
+ row_to_json
+------------------------------------------------------------------------------------
+ {"very_long_column_name_to_test_json_escape": ""}
+ {"very_long_column_name_to_test_json_escape": "a"}
+ {"very_long_column_name_to_test_json_escape": "aa"}
+ {"very_long_column_name_to_test_json_escape": "aaa"}
+ {"very_long_column_name_to_test_json_escape": "aaaa"}
+ {"very_long_column_name_to_test_json_escape": "aaaaa"}
+ {"very_long_column_name_to_test_json_escape": "aaaaaa"}
+ {"very_long_column_name_to_test_json_escape": "aaaaaaa"}
+ {"very_long_column_name_to_test_json_escape": "aaaaaaaa"}
+ {"very_long_column_name_to_test_json_escape": "aaaaaaaaa"}
+ {"very_long_column_name_to_test_json_escape": "aaaaaaaaaa"}
+ {"very_long_column_name_to_test_json_escape": "aaaaaaaaaaa"}
+ {"very_long_column_name_to_test_json_escape": "aaaaaaaaaaaa"}
+ {"very_long_column_name_to_test_json_escape": "aaaaaaaaaaaaa"}
+ {"very_long_column_name_to_test_json_escape": "aaaaaaaaaaaaaa"}
+ {"very_long_column_name_to_test_json_escape": "aaaaaaaaaaaaaaa"}
+ {"very_long_column_name_to_test_json_escape": "aaaaaaaaaaaaaaaa"}
+ {"very_long_column_name_to_test_json_escape": "aaaaaaaaaaaaaaaaa"}
+ {"very_long_column_name_to_test_json_escape": "aaaaaaaaaaaaaaaaaa"}
+ {"very_long_column_name_to_test_json_escape": "aaaaaaaaaaaaaaaaaaa"}
+ {"very_long_column_name_to_test_json_escape": "aaaaaaaaaaaaaaaaaaaa"}
+ {"very_long_column_name_to_test_json_escape": "aaaaaaaaaaaaaaaaaaaaa"}
+ {"very_long_column_name_to_test_json_escape": "aaaaaaaaaaaaaaaaaaaaaa"}
+ {"very_long_column_name_to_test_json_escape": "aaaaaaaaaaaaaaaaaaaaaaa"}
+ {"very_long_column_name_to_test_json_escape": "aaaaaaaaaaaaaaaaaaaaaaaa"}
+ {"very_long_column_name_to_test_json_escape": "aaaaaaaaaaaaaaaaaaaaaaaaa"}
+ {"very_long_column_name_to_test_json_escape": "aaaaaaaaaaaaaaaaaaaaaaaaaa"}
+ {"very_long_column_name_to_test_json_escape": "aaaaaaaaaaaaaaaaaaaaaaaaaaa"}
+ {"very_long_column_name_to_test_json_escape": "aaaaaaaaaaaaaaaaaaaaaaaaaaaa"}
+ {"very_long_column_name_to_test_json_escape": "aaaaaaaaaaaaaaaaaaaaaaaaaaaaa"}
+ {"very_long_column_name_to_test_json_escape": "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaa"}
+ {"very_long_column_name_to_test_json_escape": "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa"}
+ {"very_long_column_name_to_test_json_escape": "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa"}
+ {"very_long_column_name_to_test_json_escape": "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa"}
+(34 rows)
+
-- see json_encoding test for input with unicode escapes
-- Numbers.
SELECT '1'::json; -- OK
diff --git a/src/test/regress/sql/json.sql b/src/test/regress/sql/json.sql
index ec57dfe707..0e7ca2f5af 100644
--- a/src/test/regress/sql/json.sql
+++ b/src/test/regress/sql/json.sql
@@ -12,6 +12,14 @@ SELECT '"\v"'::json; -- ERROR, not a valid JSON escape
SELECT ('"'||repeat('.', 12)||'abc"')::json; -- OK
SELECT ('"'||repeat('.', 12)||'abc\n"')::json; -- OK, legal escapes
+-- Stress testing of JSON escape code
+CREATE TABLE json_escape (very_long_column_name_to_test_json_escape text);
+INSERT INTO json_escape SELECT repeat('a', a) FROM generate_series(0,33) a;
+
+-- Test various lengths of strings to validate SIMD processing to escape
+-- special chars in the JSON.
+SELECT row_to_json(j)::jsonb FROM json_escape j;
+
-- see json_encoding test for input with unicode escapes
-- Numbers.
--
2.34.1
v2-0003-Special-case-text-type-conversion-in-datum_to_jso.patchapplication/octet-stream; name=v2-0003-Special-case-text-type-conversion-in-datum_to_jso.patchDownload
From bb25974cd77c5afb273c0acfbf150d35da0ea0cd Mon Sep 17 00:00:00 2001
From: David Rowley <dgrowley@gmail.com>
Date: Thu, 23 May 2024 15:12:59 +1200
Subject: [PATCH v2 3/3] Special-case text type conversion in
datum_to_json_internal
---
src/backend/utils/adt/json.c | 13 ++++++++++---
1 file changed, 10 insertions(+), 3 deletions(-)
diff --git a/src/backend/utils/adt/json.c b/src/backend/utils/adt/json.c
index a266f60ff3..371881dfd0 100644
--- a/src/backend/utils/adt/json.c
+++ b/src/backend/utils/adt/json.c
@@ -24,6 +24,7 @@
#include "utils/builtins.h"
#include "utils/date.h"
#include "utils/datetime.h"
+#include "utils/fmgroids.h"
#include "utils/json.h"
#include "utils/jsonfuncs.h"
#include "utils/lsyscache.h"
@@ -286,9 +287,15 @@ datum_to_json_internal(Datum val, bool is_null, StringInfo result,
pfree(jsontext);
break;
default:
- outputstr = OidOutputFunctionCall(outfuncoid, val);
- escape_json_cstring(result, outputstr);
- pfree(outputstr);
+ /* special-case text types to save useless palloc/memcpy cycles */
+ if (outfuncoid == F_TEXTOUT || outfuncoid == F_VARCHAROUT || outfuncoid == F_BPCHAROUT)
+ escape_json_from_text(result, (text *) DatumGetPointer(val));
+ else
+ {
+ outputstr = OidOutputFunctionCall(outfuncoid, val);
+ escape_json_cstring(result, outputstr);
+ pfree(outputstr);
+ }
break;
}
}
--
2.34.1
Hi David,
Thanks for the patch.
In 0001 patch, I see that there are some escape_json() calls with
NUL-terminated strings and gets the length by calling strlen(), like below:
- escape_json(&buf, "timestamp");
+ escape_json(&buf, "timestamp", strlen("timestamp"));
Wouldn't using escape_json_cstring() be better instead? IIUC there isn't
much difference between escape_json() and escape_json_cstring(), right? We
would avoid strlen() with escape_json_cstring().
Regards,
--
Melih Mutlu
Microsoft
On 2024-06-11 Tu 08:08, Melih Mutlu wrote:
Hi David,
Thanks for the patch.
In 0001 patch, I see that there are some escape_json() calls with
NUL-terminated strings and gets the length by calling strlen(), like
below:- escape_json(&buf, "timestamp"); + escape_json(&buf, "timestamp", strlen("timestamp"));Wouldn't using escape_json_cstring() be better instead? IIUC there
isn't much difference between escape_json() and escape_json_cstring(),
right? We would avoid strlen() with escape_json_cstring().
or maybe use sizeof("timestamp") - 1
cheers
andrew
--
Andrew Dunstan
EDB:https://www.enterprisedb.com
Thanks for having a look.
On Wed, 12 Jun 2024 at 00:08, Melih Mutlu <m.melihmutlu@gmail.com> wrote:
In 0001 patch, I see that there are some escape_json() calls with NUL-terminated strings and gets the length by calling strlen(), like below:
- escape_json(&buf, "timestamp"); + escape_json(&buf, "timestamp", strlen("timestamp"));Wouldn't using escape_json_cstring() be better instead? IIUC there isn't much difference between escape_json() and escape_json_cstring(), right? We would avoid strlen() with escape_json_cstring().
It maybe would be better, but not for this reason. Most compilers will
be able to perform constant folding to transform the
strlen("timestamp") into 9. You can see that's being done by both gcc
and clang in [1]https://godbolt.org/z/xqj4rKara.
It might be better to use escape_json_cstring() regardless of that as
the SIMD only kicks in when there are >= 16 chars, so there might be a
few more instructions calling the SIMD version for such a short
string. Probably, if we're worried about performance here we could
just not bother passing the string through the escape function to
search for something we know isn't there and just
appendBinaryStringInfo \""timestamp\":" directly.
I don't really have a preference as to which of these we use. I doubt
the JSON escaping rules would ever change sufficiently that the latter
of these methods would be a bad idea. I just doubt it's worth the
debate as I imagine the performance won't matter that much.
David
I've attached a rebased set of patches. The previous set no longer applied.
David
Attachments:
v3-0001-Add-len-parameter-to-escape_json.patchapplication/octet-stream; name=v3-0001-Add-len-parameter-to-escape_json.patchDownload
From f67eafe621ba8612c1d69a0a957707554bd88670 Mon Sep 17 00:00:00 2001
From: David Rowley <dgrowley@gmail.com>
Date: Tue, 21 May 2024 17:14:06 +1200
Subject: [PATCH v3 1/3] Add 'len' parameter to escape_json()
---
contrib/hstore/hstore_io.c | 41 +++-----
src/backend/backup/backup_manifest.c | 2 +-
src/backend/commands/explain.c | 30 ++++--
src/backend/parser/parse_jsontable.c | 2 +-
src/backend/utils/adt/json.c | 150 +++++++++++++++++----------
src/backend/utils/adt/jsonb.c | 2 +-
src/backend/utils/adt/jsonfuncs.c | 12 +--
src/backend/utils/adt/jsonpath.c | 15 ++-
src/backend/utils/error/jsonlog.c | 8 +-
src/include/utils/json.h | 4 +-
10 files changed, 159 insertions(+), 107 deletions(-)
diff --git a/contrib/hstore/hstore_io.c b/contrib/hstore/hstore_io.c
index 999ddad76d..374b8d05ef 100644
--- a/contrib/hstore/hstore_io.c
+++ b/contrib/hstore/hstore_io.c
@@ -1343,23 +1343,20 @@ hstore_to_json_loose(PG_FUNCTION_ARGS)
int count = HS_COUNT(in);
char *base = STRPTR(in);
HEntry *entries = ARRPTR(in);
- StringInfoData tmp,
- dst;
+ StringInfoData dst;
if (count == 0)
PG_RETURN_TEXT_P(cstring_to_text_with_len("{}", 2));
- initStringInfo(&tmp);
initStringInfo(&dst);
appendStringInfoChar(&dst, '{');
for (i = 0; i < count; i++)
{
- resetStringInfo(&tmp);
- appendBinaryStringInfo(&tmp, HSTORE_KEY(entries, base, i),
- HSTORE_KEYLEN(entries, i));
- escape_json(&dst, tmp.data);
+ escape_json(&dst,
+ HSTORE_KEY(entries, base, i),
+ HSTORE_KEYLEN(entries, i));
appendStringInfoString(&dst, ": ");
if (HSTORE_VALISNULL(entries, i))
appendStringInfoString(&dst, "null");
@@ -1372,13 +1369,13 @@ hstore_to_json_loose(PG_FUNCTION_ARGS)
appendStringInfoString(&dst, "false");
else
{
- resetStringInfo(&tmp);
- appendBinaryStringInfo(&tmp, HSTORE_VAL(entries, base, i),
- HSTORE_VALLEN(entries, i));
- if (IsValidJsonNumber(tmp.data, tmp.len))
- appendBinaryStringInfo(&dst, tmp.data, tmp.len);
+ char *str = HSTORE_VAL(entries, base, i);
+ int len = HSTORE_VALLEN(entries, i);
+
+ if (IsValidJsonNumber(str, len))
+ appendBinaryStringInfo(&dst, str, len);
else
- escape_json(&dst, tmp.data);
+ escape_json(&dst, str, len);
}
if (i + 1 != count)
@@ -1398,32 +1395,28 @@ hstore_to_json(PG_FUNCTION_ARGS)
int count = HS_COUNT(in);
char *base = STRPTR(in);
HEntry *entries = ARRPTR(in);
- StringInfoData tmp,
- dst;
+ StringInfoData dst;
if (count == 0)
PG_RETURN_TEXT_P(cstring_to_text_with_len("{}", 2));
- initStringInfo(&tmp);
initStringInfo(&dst);
appendStringInfoChar(&dst, '{');
for (i = 0; i < count; i++)
{
- resetStringInfo(&tmp);
- appendBinaryStringInfo(&tmp, HSTORE_KEY(entries, base, i),
- HSTORE_KEYLEN(entries, i));
- escape_json(&dst, tmp.data);
+ escape_json(&dst,
+ HSTORE_KEY(entries, base, i),
+ HSTORE_KEYLEN(entries, i));
appendStringInfoString(&dst, ": ");
if (HSTORE_VALISNULL(entries, i))
appendStringInfoString(&dst, "null");
else
{
- resetStringInfo(&tmp);
- appendBinaryStringInfo(&tmp, HSTORE_VAL(entries, base, i),
- HSTORE_VALLEN(entries, i));
- escape_json(&dst, tmp.data);
+ escape_json(&dst,
+ HSTORE_VAL(entries, base, i),
+ HSTORE_VALLEN(entries, i));
}
if (i + 1 != count)
diff --git a/src/backend/backup/backup_manifest.c b/src/backend/backup/backup_manifest.c
index b360a13547..3da8da00d6 100644
--- a/src/backend/backup/backup_manifest.c
+++ b/src/backend/backup/backup_manifest.c
@@ -148,7 +148,7 @@ AddFileToBackupManifest(backup_manifest_info *manifest, Oid spcoid,
pg_verify_mbstr(PG_UTF8, pathname, pathlen, true))
{
appendStringInfoString(&buf, "{ \"Path\": ");
- escape_json(&buf, pathname);
+ escape_json(&buf, pathname, pathlen);
appendStringInfoString(&buf, ", ");
}
else
diff --git a/src/backend/commands/explain.c b/src/backend/commands/explain.c
index 94511a5a02..399025aed3 100644
--- a/src/backend/commands/explain.c
+++ b/src/backend/commands/explain.c
@@ -4661,13 +4661,15 @@ ExplainPropertyList(const char *qlabel, List *data, ExplainState *es)
case EXPLAIN_FORMAT_JSON:
ExplainJSONLineEnding(es);
appendStringInfoSpaces(es->str, es->indent * 2);
- escape_json(es->str, qlabel);
+ escape_json_cstring(es->str, qlabel);
appendStringInfoString(es->str, ": [");
foreach(lc, data)
{
+ const char *str = (const char *) lfirst(lc);
+
if (!first)
appendStringInfoString(es->str, ", ");
- escape_json(es->str, (const char *) lfirst(lc));
+ escape_json_cstring(es->str, str);
first = false;
}
appendStringInfoChar(es->str, ']');
@@ -4678,10 +4680,12 @@ ExplainPropertyList(const char *qlabel, List *data, ExplainState *es)
appendStringInfo(es->str, "%s: ", qlabel);
foreach(lc, data)
{
+ const char *str = (const char *) lfirst(lc);
+
appendStringInfoChar(es->str, '\n');
appendStringInfoSpaces(es->str, es->indent * 2 + 2);
appendStringInfoString(es->str, "- ");
- escape_yaml(es->str, (const char *) lfirst(lc));
+ escape_yaml(es->str, str);
}
break;
}
@@ -4710,9 +4714,11 @@ ExplainPropertyListNested(const char *qlabel, List *data, ExplainState *es)
appendStringInfoChar(es->str, '[');
foreach(lc, data)
{
+ const char *str = (const char *) lfirst(lc);
+
if (!first)
appendStringInfoString(es->str, ", ");
- escape_json(es->str, (const char *) lfirst(lc));
+ escape_json_cstring(es->str, str);
first = false;
}
appendStringInfoChar(es->str, ']');
@@ -4723,9 +4729,11 @@ ExplainPropertyListNested(const char *qlabel, List *data, ExplainState *es)
appendStringInfoString(es->str, "- [");
foreach(lc, data)
{
+ const char *str = (const char *) lfirst(lc);
+
if (!first)
appendStringInfoString(es->str, ", ");
- escape_yaml(es->str, (const char *) lfirst(lc));
+ escape_yaml(es->str, str);
first = false;
}
appendStringInfoChar(es->str, ']');
@@ -4775,12 +4783,12 @@ ExplainProperty(const char *qlabel, const char *unit, const char *value,
case EXPLAIN_FORMAT_JSON:
ExplainJSONLineEnding(es);
appendStringInfoSpaces(es->str, es->indent * 2);
- escape_json(es->str, qlabel);
+ escape_json_cstring(es->str, qlabel);
appendStringInfoString(es->str, ": ");
if (numeric)
appendStringInfoString(es->str, value);
else
- escape_json(es->str, value);
+ escape_json_cstring(es->str, value);
break;
case EXPLAIN_FORMAT_YAML:
@@ -4882,7 +4890,7 @@ ExplainOpenGroup(const char *objtype, const char *labelname,
appendStringInfoSpaces(es->str, 2 * es->indent);
if (labelname)
{
- escape_json(es->str, labelname);
+ escape_json_cstring(es->str, labelname);
appendStringInfoString(es->str, ": ");
}
appendStringInfoChar(es->str, labeled ? '{' : '[');
@@ -5090,10 +5098,10 @@ ExplainDummyGroup(const char *objtype, const char *labelname, ExplainState *es)
appendStringInfoSpaces(es->str, 2 * es->indent);
if (labelname)
{
- escape_json(es->str, labelname);
+ escape_json_cstring(es->str, labelname);
appendStringInfoString(es->str, ": ");
}
- escape_json(es->str, objtype);
+ escape_json_cstring(es->str, objtype);
break;
case EXPLAIN_FORMAT_YAML:
@@ -5297,7 +5305,7 @@ ExplainYAMLLineStarting(ExplainState *es)
static void
escape_yaml(StringInfo buf, const char *str)
{
- escape_json(buf, str);
+ escape_json_cstring(buf, str);
}
diff --git a/src/backend/parser/parse_jsontable.c b/src/backend/parser/parse_jsontable.c
index 8a72e498e8..7a8aa4a2d3 100644
--- a/src/backend/parser/parse_jsontable.c
+++ b/src/backend/parser/parse_jsontable.c
@@ -427,7 +427,7 @@ transformJsonTableColumn(JsonTableColumn *jtc, Node *contextItemExpr,
initStringInfo(&path);
appendStringInfoString(&path, "$.");
- escape_json(&path, jtc->name);
+ escape_json_cstring(&path, jtc->name);
pathspec = makeStringConst(path.data, -1);
}
diff --git a/src/backend/utils/adt/json.c b/src/backend/utils/adt/json.c
index d719a61f16..7934cf62fb 100644
--- a/src/backend/utils/adt/json.c
+++ b/src/backend/utils/adt/json.c
@@ -286,7 +286,7 @@ datum_to_json_internal(Datum val, bool is_null, StringInfo result,
break;
default:
outputstr = OidOutputFunctionCall(outfuncoid, val);
- escape_json(result, outputstr);
+ escape_json_cstring(result, outputstr);
pfree(outputstr);
break;
}
@@ -560,7 +560,7 @@ composite_to_json(Datum composite, StringInfo result, bool use_line_feeds)
needsep = true;
attname = NameStr(att->attname);
- escape_json(result, attname);
+ escape_json_cstring(result, attname);
appendStringInfoChar(result, ':');
val = heap_getattr(tuple, i + 1, tupdesc, &isnull);
@@ -1391,7 +1391,6 @@ json_object(PG_FUNCTION_ARGS)
count,
i;
text *rval;
- char *v;
switch (ndims)
{
@@ -1434,19 +1433,17 @@ json_object(PG_FUNCTION_ARGS)
(errcode(ERRCODE_NULL_VALUE_NOT_ALLOWED),
errmsg("null value not allowed for object key")));
- v = TextDatumGetCString(in_datums[i * 2]);
if (i > 0)
appendStringInfoString(&result, ", ");
- escape_json(&result, v);
+ escape_json_from_text(&result,
+ (text *) DatumGetPointer(in_datums[i * 2]));
appendStringInfoString(&result, " : ");
- pfree(v);
if (in_nulls[i * 2 + 1])
appendStringInfoString(&result, "null");
else
{
- v = TextDatumGetCString(in_datums[i * 2 + 1]);
- escape_json(&result, v);
- pfree(v);
+ escape_json_from_text(&result,
+ (text *) DatumGetPointer(in_datums[i * 2 + 1]));
}
}
@@ -1483,7 +1480,6 @@ json_object_two_arg(PG_FUNCTION_ARGS)
val_count,
i;
text *rval;
- char *v;
if (nkdims > 1 || nkdims != nvdims)
ereport(ERROR,
@@ -1512,20 +1508,17 @@ json_object_two_arg(PG_FUNCTION_ARGS)
(errcode(ERRCODE_NULL_VALUE_NOT_ALLOWED),
errmsg("null value not allowed for object key")));
- v = TextDatumGetCString(key_datums[i]);
if (i > 0)
appendStringInfoString(&result, ", ");
- escape_json(&result, v);
+ escape_json_from_text(&result,
+ (text *) DatumGetPointer(key_datums[i]));
+
appendStringInfoString(&result, " : ");
- pfree(v);
if (val_nulls[i])
appendStringInfoString(&result, "null");
else
- {
- v = TextDatumGetCString(val_datums[i]);
- escape_json(&result, v);
- pfree(v);
- }
+ escape_json_from_text(&result,
+ (text *) DatumGetPointer(val_datums[i]));
}
appendStringInfoChar(&result, '}');
@@ -1541,52 +1534,101 @@ json_object_two_arg(PG_FUNCTION_ARGS)
PG_RETURN_TEXT_P(rval);
}
+/*
+ * escape_json_char
+ * Inline helper function for escape_json* functions
+ */
+static pg_attribute_always_inline void
+escape_json_char(StringInfo buf, char c)
+{
+ switch (c)
+ {
+ case '\b':
+ appendStringInfoString(buf, "\\b");
+ break;
+ case '\f':
+ appendStringInfoString(buf, "\\f");
+ break;
+ case '\n':
+ appendStringInfoString(buf, "\\n");
+ break;
+ case '\r':
+ appendStringInfoString(buf, "\\r");
+ break;
+ case '\t':
+ appendStringInfoString(buf, "\\t");
+ break;
+ case '"':
+ appendStringInfoString(buf, "\\\"");
+ break;
+ case '\\':
+ appendStringInfoString(buf, "\\\\");
+ break;
+ default:
+ if ((unsigned char) c < ' ')
+ appendStringInfo(buf, "\\u%04x", (int) c);
+ else
+ appendStringInfoCharMacro(buf, c);
+ break;
+ }
+}
/*
- * Produce a JSON string literal, properly escaping characters in the text.
+ * escape_json_cstring
+ * Produce a JSON string literal. Same as escape_json() except takes a
+ * NUL-terminated string as input.
*/
void
-escape_json(StringInfo buf, const char *str)
+escape_json_cstring(StringInfo buf, const char *str)
{
- const char *p;
+ appendStringInfoCharMacro(buf, '"');
+
+ for (; *str != '\0'; str++)
+ escape_json_char(buf, *str);
appendStringInfoCharMacro(buf, '"');
- for (p = str; *p; p++)
- {
- switch (*p)
- {
- case '\b':
- appendStringInfoString(buf, "\\b");
- break;
- case '\f':
- appendStringInfoString(buf, "\\f");
- break;
- case '\n':
- appendStringInfoString(buf, "\\n");
- break;
- case '\r':
- appendStringInfoString(buf, "\\r");
- break;
- case '\t':
- appendStringInfoString(buf, "\\t");
- break;
- case '"':
- appendStringInfoString(buf, "\\\"");
- break;
- case '\\':
- appendStringInfoString(buf, "\\\\");
- break;
- default:
- if ((unsigned char) *p < ' ')
- appendStringInfo(buf, "\\u%04x", (int) *p);
- else
- appendStringInfoCharMacro(buf, *p);
- break;
- }
- }
+}
+
+/*
+ * Produce a JSON string literal, properly escaping the possibly not
+ * NUL-terminated characters in 'str'. 'len' defines the number of bytes from
+ * 'str' to process.
+ */
+void
+escape_json(StringInfo buf, const char *str, int len)
+{
+ appendStringInfoCharMacro(buf, '"');
+
+ for (int i = 0; i < len; i++)
+ escape_json_char(buf, str[i]);
+
appendStringInfoCharMacro(buf, '"');
}
+/*
+ * escape_json_from_text
+ * Append 't' onto 'buf' and escape using escape_json.
+ *
+ * This is more efficient than calling text_to_cstring and appending the
+ * result as that could require an additional palloc and memcpy.
+ */
+void
+escape_json_from_text(StringInfo buf, const text *t)
+{
+ /* must cast away the const, unfortunately */
+ text *tunpacked = pg_detoast_datum_packed(unconstify(text *, t));
+ int len = VARSIZE_ANY_EXHDR(tunpacked);
+ char *str;
+
+ str = VARDATA_ANY(tunpacked);
+
+ escape_json(buf, str, len);
+
+ /* pfree any detoasted values */
+ if (tunpacked != t)
+ pfree(tunpacked);
+}
+
/* Semantic actions for key uniqueness check */
static JsonParseErrorType
json_unique_object_start(void *_state)
diff --git a/src/backend/utils/adt/jsonb.c b/src/backend/utils/adt/jsonb.c
index e4562b3c6c..f24d8003be 100644
--- a/src/backend/utils/adt/jsonb.c
+++ b/src/backend/utils/adt/jsonb.c
@@ -354,7 +354,7 @@ jsonb_put_escaped_value(StringInfo out, JsonbValue *scalarVal)
appendBinaryStringInfo(out, "null", 4);
break;
case jbvString:
- escape_json(out, pnstrdup(scalarVal->val.string.val, scalarVal->val.string.len));
+ escape_json(out, scalarVal->val.string.val, scalarVal->val.string.len);
break;
case jbvNumeric:
appendStringInfoString(out,
diff --git a/src/backend/utils/adt/jsonfuncs.c b/src/backend/utils/adt/jsonfuncs.c
index 48c3f88140..fd3c6b8f43 100644
--- a/src/backend/utils/adt/jsonfuncs.c
+++ b/src/backend/utils/adt/jsonfuncs.c
@@ -3142,8 +3142,8 @@ populate_scalar(ScalarIOData *io, Oid typid, int32 typmod, JsValue *jsv,
}
else
{
- /* string is already null-terminated */
str = unconstify(char *, json);
+ len = strlen(str);
}
/* If converting to json/jsonb, make string into valid JSON literal */
@@ -3153,7 +3153,7 @@ populate_scalar(ScalarIOData *io, Oid typid, int32 typmod, JsValue *jsv,
StringInfoData buf;
initStringInfo(&buf);
- escape_json(&buf, str);
+ escape_json(&buf, str, len);
/* free temporary buffer */
if (str != json)
pfree(str);
@@ -4446,7 +4446,7 @@ sn_object_field_start(void *state, char *fname, bool isnull)
* Unfortunately we don't have the quoted and escaped string any more, so
* we have to re-escape it.
*/
- escape_json(_state->strval, fname);
+ escape_json_cstring(_state->strval, fname);
appendStringInfoCharMacro(_state->strval, ':');
@@ -4477,7 +4477,7 @@ sn_scalar(void *state, char *token, JsonTokenType tokentype)
}
if (tokentype == JSON_TOKEN_STRING)
- escape_json(_state->strval, token);
+ escape_json_cstring(_state->strval, token);
else
appendStringInfoString(_state->strval, token);
@@ -5909,7 +5909,7 @@ transform_string_values_object_field_start(void *state, char *fname, bool isnull
* Unfortunately we don't have the quoted and escaped string any more, so
* we have to re-escape it.
*/
- escape_json(_state->strval, fname);
+ escape_json_cstring(_state->strval, fname);
appendStringInfoCharMacro(_state->strval, ':');
return JSON_SUCCESS;
@@ -5935,7 +5935,7 @@ transform_string_values_scalar(void *state, char *token, JsonTokenType tokentype
{
text *out = _state->action(_state->action_state, token, strlen(token));
- escape_json(_state->strval, text_to_cstring(out));
+ escape_json_from_text(_state->strval, out);
}
else
appendStringInfoString(_state->strval, token);
diff --git a/src/backend/utils/adt/jsonpath.c b/src/backend/utils/adt/jsonpath.c
index 11e6193e96..82d65e4d4f 100644
--- a/src/backend/utils/adt/jsonpath.c
+++ b/src/backend/utils/adt/jsonpath.c
@@ -523,6 +523,8 @@ printJsonPathItem(StringInfo buf, JsonPathItem *v, bool inKey,
{
JsonPathItem elem;
int i;
+ int32 len;
+ char *str;
check_stack_depth();
CHECK_FOR_INTERRUPTS();
@@ -533,7 +535,8 @@ printJsonPathItem(StringInfo buf, JsonPathItem *v, bool inKey,
appendStringInfoString(buf, "null");
break;
case jpiString:
- escape_json(buf, jspGetString(v, NULL));
+ str = jspGetString(v, &len);
+ escape_json(buf, str, len);
break;
case jpiNumeric:
if (jspHasNext(v))
@@ -662,7 +665,8 @@ printJsonPathItem(StringInfo buf, JsonPathItem *v, bool inKey,
case jpiKey:
if (inKey)
appendStringInfoChar(buf, '.');
- escape_json(buf, jspGetString(v, NULL));
+ str = jspGetString(v, &len);
+ escape_json(buf, str, len);
break;
case jpiCurrent:
Assert(!inKey);
@@ -674,7 +678,8 @@ printJsonPathItem(StringInfo buf, JsonPathItem *v, bool inKey,
break;
case jpiVariable:
appendStringInfoChar(buf, '$');
- escape_json(buf, jspGetString(v, NULL));
+ str = jspGetString(v, &len);
+ escape_json(buf, str, len);
break;
case jpiFilter:
appendStringInfoString(buf, "?(");
@@ -732,7 +737,9 @@ printJsonPathItem(StringInfo buf, JsonPathItem *v, bool inKey,
appendStringInfoString(buf, " like_regex ");
- escape_json(buf, v->content.like_regex.pattern);
+ escape_json(buf,
+ v->content.like_regex.pattern,
+ v->content.like_regex.patternlen);
if (v->content.like_regex.flags)
{
diff --git a/src/backend/utils/error/jsonlog.c b/src/backend/utils/error/jsonlog.c
index bd0124869d..bd21f6ef58 100644
--- a/src/backend/utils/error/jsonlog.c
+++ b/src/backend/utils/error/jsonlog.c
@@ -49,11 +49,11 @@ appendJSONKeyValue(StringInfo buf, const char *key, const char *value,
return;
appendStringInfoChar(buf, ',');
- escape_json(buf, key);
+ escape_json_cstring(buf, key);
appendStringInfoChar(buf, ':');
if (escape_value)
- escape_json(buf, value);
+ escape_json_cstring(buf, value);
else
appendStringInfoString(buf, value);
}
@@ -143,9 +143,9 @@ write_jsonlog(ErrorData *edata)
* First property does not use appendJSONKeyValue as it does not have
* comma prefix.
*/
- escape_json(&buf, "timestamp");
+ escape_json(&buf, "timestamp", strlen("timestamp"));
appendStringInfoChar(&buf, ':');
- escape_json(&buf, log_time);
+ escape_json_cstring(&buf, log_time);
/* username */
if (MyProcPort)
diff --git a/src/include/utils/json.h b/src/include/utils/json.h
index 6d7f1b387d..8ede8d0bd6 100644
--- a/src/include/utils/json.h
+++ b/src/include/utils/json.h
@@ -17,7 +17,9 @@
#include "lib/stringinfo.h"
/* functions in json.c */
-extern void escape_json(StringInfo buf, const char *str);
+extern void escape_json_cstring(StringInfo buf, const char *str);
+extern void escape_json(StringInfo buf, const char *str, int len);
+extern void escape_json_from_text(StringInfo buf, const text *t);
extern char *JsonEncodeDateTime(char *buf, Datum value, Oid typid,
const int *tzp);
extern bool to_json_is_immutable(Oid typoid);
--
2.34.1
v3-0002-Use-SIMD-processing-for-escape_json.patchapplication/octet-stream; name=v3-0002-Use-SIMD-processing-for-escape_json.patchDownload
From 490f83765f7765659d5efc4218da6ef807853d79 Mon Sep 17 00:00:00 2001
From: David Rowley <dgrowley@gmail.com>
Date: Thu, 23 May 2024 10:53:23 +1200
Subject: [PATCH v3 2/3] Use SIMD processing for escape_json()
---
src/backend/utils/adt/json.c | 72 +++++++++++++++++++++++++++++-
src/test/regress/expected/json.out | 44 ++++++++++++++++++
src/test/regress/sql/json.sql | 8 ++++
3 files changed, 122 insertions(+), 2 deletions(-)
diff --git a/src/backend/utils/adt/json.c b/src/backend/utils/adt/json.c
index 7934cf62fb..a266f60ff3 100644
--- a/src/backend/utils/adt/json.c
+++ b/src/backend/utils/adt/json.c
@@ -19,6 +19,7 @@
#include "funcapi.h"
#include "libpq/pqformat.h"
#include "miscadmin.h"
+#include "port/simd.h"
#include "utils/array.h"
#include "utils/builtins.h"
#include "utils/date.h"
@@ -1597,11 +1598,78 @@ escape_json_cstring(StringInfo buf, const char *str)
void
escape_json(StringInfo buf, const char *str, int len)
{
+ int i = 0;
+ int copypos = 0;
+
+ Assert(len >= 0);
+
appendStringInfoCharMacro(buf, '"');
- for (int i = 0; i < len; i++)
- escape_json_char(buf, str[i]);
+ for (;;)
+ {
+ Vector8 chunk;
+ int vlen;
+
+ /*
+ * Figure out how many bytes to process using SIMD. Round 'len' down
+ * to the previous multiple of sizeof(Vector8), assuming that's a
+ * power-of-2.
+ */
+ vlen = len & (int) (~(sizeof(Vector8) - 1));
+
+ /*
+ * To speed this up try searching sizeof(Vector8) bytes at once for
+ * special characters that we need to escape. When we find one, we
+ * fall out of this first loop and copy the parts we've vector
+ * searched before processing the special-char vector byte-by-byte.
+ * Once we're done with that, come back and try doing vector searching
+ * again. We'll also process the tail end of the string byte-by-byte.
+ */
+ for (; i < vlen; i += sizeof(Vector8))
+ {
+ vector8_load(&chunk, (const uint8 *) &str[i]);
+
+ /*
+ * Break on anything less than ' ' or if we find a '"' or '\\'.
+ * Those need special handling. That's done in the per-byte loop.
+ */
+ if (vector8_has_le(chunk, (unsigned char) 0x1F) ||
+ vector8_has(chunk, (unsigned char) '"') ||
+ vector8_has(chunk, (unsigned char) '\\'))
+ break;
+ }
+
+ /*
+ * Write to the destination up to the point of that we've vector
+ * searched so far. Do this only when switching into per-byte mode
+ * rather than once every sizeof(Vector8) bytes.
+ */
+ if (copypos < i)
+ {
+ appendBinaryStringInfo(buf, &str[copypos], i - copypos);
+ copypos = i;
+ }
+
+ /*
+ * Per-byte loop for Vector8s containing special chars and for
+ * processing the tail of the string.
+ */
+ for (int b = 0; b < sizeof(Vector8); b++)
+ {
+ /* check if we've finished */
+ if (i == len)
+ goto done;
+
+ Assert(i < len);
+
+ escape_json_char(buf, str[i++]);
+ }
+
+ copypos = i;
+ /* We're not done yet. Try the SIMD search again */
+ }
+done:
appendStringInfoCharMacro(buf, '"');
}
diff --git a/src/test/regress/expected/json.out b/src/test/regress/expected/json.out
index aa29bc597b..bfcc26c531 100644
--- a/src/test/regress/expected/json.out
+++ b/src/test/regress/expected/json.out
@@ -55,6 +55,50 @@ SELECT ('"'||repeat('.', 12)||'abc\n"')::json; -- OK, legal escapes
"............abc\n"
(1 row)
+-- Stress testing of JSON escape code
+CREATE TABLE json_escape (very_long_column_name_to_test_json_escape text);
+INSERT INTO json_escape SELECT repeat('a', a) FROM generate_series(0,33) a;
+-- Test various lengths of strings to validate SIMD processing to escape
+-- special chars in the JSON.
+SELECT row_to_json(j)::jsonb FROM json_escape j;
+ row_to_json
+------------------------------------------------------------------------------------
+ {"very_long_column_name_to_test_json_escape": ""}
+ {"very_long_column_name_to_test_json_escape": "a"}
+ {"very_long_column_name_to_test_json_escape": "aa"}
+ {"very_long_column_name_to_test_json_escape": "aaa"}
+ {"very_long_column_name_to_test_json_escape": "aaaa"}
+ {"very_long_column_name_to_test_json_escape": "aaaaa"}
+ {"very_long_column_name_to_test_json_escape": "aaaaaa"}
+ {"very_long_column_name_to_test_json_escape": "aaaaaaa"}
+ {"very_long_column_name_to_test_json_escape": "aaaaaaaa"}
+ {"very_long_column_name_to_test_json_escape": "aaaaaaaaa"}
+ {"very_long_column_name_to_test_json_escape": "aaaaaaaaaa"}
+ {"very_long_column_name_to_test_json_escape": "aaaaaaaaaaa"}
+ {"very_long_column_name_to_test_json_escape": "aaaaaaaaaaaa"}
+ {"very_long_column_name_to_test_json_escape": "aaaaaaaaaaaaa"}
+ {"very_long_column_name_to_test_json_escape": "aaaaaaaaaaaaaa"}
+ {"very_long_column_name_to_test_json_escape": "aaaaaaaaaaaaaaa"}
+ {"very_long_column_name_to_test_json_escape": "aaaaaaaaaaaaaaaa"}
+ {"very_long_column_name_to_test_json_escape": "aaaaaaaaaaaaaaaaa"}
+ {"very_long_column_name_to_test_json_escape": "aaaaaaaaaaaaaaaaaa"}
+ {"very_long_column_name_to_test_json_escape": "aaaaaaaaaaaaaaaaaaa"}
+ {"very_long_column_name_to_test_json_escape": "aaaaaaaaaaaaaaaaaaaa"}
+ {"very_long_column_name_to_test_json_escape": "aaaaaaaaaaaaaaaaaaaaa"}
+ {"very_long_column_name_to_test_json_escape": "aaaaaaaaaaaaaaaaaaaaaa"}
+ {"very_long_column_name_to_test_json_escape": "aaaaaaaaaaaaaaaaaaaaaaa"}
+ {"very_long_column_name_to_test_json_escape": "aaaaaaaaaaaaaaaaaaaaaaaa"}
+ {"very_long_column_name_to_test_json_escape": "aaaaaaaaaaaaaaaaaaaaaaaaa"}
+ {"very_long_column_name_to_test_json_escape": "aaaaaaaaaaaaaaaaaaaaaaaaaa"}
+ {"very_long_column_name_to_test_json_escape": "aaaaaaaaaaaaaaaaaaaaaaaaaaa"}
+ {"very_long_column_name_to_test_json_escape": "aaaaaaaaaaaaaaaaaaaaaaaaaaaa"}
+ {"very_long_column_name_to_test_json_escape": "aaaaaaaaaaaaaaaaaaaaaaaaaaaaa"}
+ {"very_long_column_name_to_test_json_escape": "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaa"}
+ {"very_long_column_name_to_test_json_escape": "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa"}
+ {"very_long_column_name_to_test_json_escape": "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa"}
+ {"very_long_column_name_to_test_json_escape": "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa"}
+(34 rows)
+
-- see json_encoding test for input with unicode escapes
-- Numbers.
SELECT '1'::json; -- OK
diff --git a/src/test/regress/sql/json.sql b/src/test/regress/sql/json.sql
index ec57dfe707..0e7ca2f5af 100644
--- a/src/test/regress/sql/json.sql
+++ b/src/test/regress/sql/json.sql
@@ -12,6 +12,14 @@ SELECT '"\v"'::json; -- ERROR, not a valid JSON escape
SELECT ('"'||repeat('.', 12)||'abc"')::json; -- OK
SELECT ('"'||repeat('.', 12)||'abc\n"')::json; -- OK, legal escapes
+-- Stress testing of JSON escape code
+CREATE TABLE json_escape (very_long_column_name_to_test_json_escape text);
+INSERT INTO json_escape SELECT repeat('a', a) FROM generate_series(0,33) a;
+
+-- Test various lengths of strings to validate SIMD processing to escape
+-- special chars in the JSON.
+SELECT row_to_json(j)::jsonb FROM json_escape j;
+
-- see json_encoding test for input with unicode escapes
-- Numbers.
--
2.34.1
v3-0003-Special-case-text-type-conversion-in-datum_to_jso.patchapplication/octet-stream; name=v3-0003-Special-case-text-type-conversion-in-datum_to_jso.patchDownload
From 9c5caba16f1ebe046e61dc08b92917f4cf603b2f Mon Sep 17 00:00:00 2001
From: David Rowley <dgrowley@gmail.com>
Date: Thu, 23 May 2024 15:12:59 +1200
Subject: [PATCH v3 3/3] Special-case text type conversion in
datum_to_json_internal
---
src/backend/utils/adt/json.c | 13 ++++++++++---
1 file changed, 10 insertions(+), 3 deletions(-)
diff --git a/src/backend/utils/adt/json.c b/src/backend/utils/adt/json.c
index a266f60ff3..371881dfd0 100644
--- a/src/backend/utils/adt/json.c
+++ b/src/backend/utils/adt/json.c
@@ -24,6 +24,7 @@
#include "utils/builtins.h"
#include "utils/date.h"
#include "utils/datetime.h"
+#include "utils/fmgroids.h"
#include "utils/json.h"
#include "utils/jsonfuncs.h"
#include "utils/lsyscache.h"
@@ -286,9 +287,15 @@ datum_to_json_internal(Datum val, bool is_null, StringInfo result,
pfree(jsontext);
break;
default:
- outputstr = OidOutputFunctionCall(outfuncoid, val);
- escape_json_cstring(result, outputstr);
- pfree(outputstr);
+ /* special-case text types to save useless palloc/memcpy cycles */
+ if (outfuncoid == F_TEXTOUT || outfuncoid == F_VARCHAROUT || outfuncoid == F_BPCHAROUT)
+ escape_json_from_text(result, (text *) DatumGetPointer(val));
+ else
+ {
+ outputstr = OidOutputFunctionCall(outfuncoid, val);
+ escape_json_cstring(result, outputstr);
+ pfree(outputstr);
+ }
break;
}
}
--
2.34.1
On 02/07/2024 07:49, David Rowley wrote:
I've attached a rebased set of patches. The previous set no longer applied.
I looked briefly at the first patch. Seems reasonable.
One little thing that caught my eye is that in populate_scalar(), you
sometimes make a temporary copy of the string to add the
null-terminator, but then call escape_json() which doesn't need the
null-terminator anymore. See attached patch to avoid that. However, it's
not clear to me how to reach that codepath, or if it reachable at all. I
tried to add a NOTICE there and ran the regression tests, but got no
failures.
--
Heikki Linnakangas
Neon (https://neon.tech)
Attachments:
avoid-possibly-unreachable-palloc.patch.txttext/plain; charset=UTF-8; name=avoid-possibly-unreachable-palloc.patch.txtDownload
diff --git a/src/backend/utils/adt/jsonfuncs.c b/src/backend/utils/adt/jsonfuncs.c
index fd3c6b8f432..55c0fc4d73d 100644
--- a/src/backend/utils/adt/jsonfuncs.c
+++ b/src/backend/utils/adt/jsonfuncs.c
@@ -3133,18 +3133,6 @@ populate_scalar(ScalarIOData *io, Oid typid, int32 typmod, JsValue *jsv,
json = jsv->val.json.str;
Assert(json);
- if (len >= 0)
- {
- /* Need to copy non-null-terminated string */
- str = palloc(len + 1 * sizeof(char));
- memcpy(str, json, len);
- str[len] = '\0';
- }
- else
- {
- str = unconstify(char *, json);
- len = strlen(str);
- }
/* If converting to json/jsonb, make string into valid JSON literal */
if ((typid == JSONOID || typid == JSONBOID) &&
@@ -3153,12 +3141,24 @@ populate_scalar(ScalarIOData *io, Oid typid, int32 typmod, JsValue *jsv,
StringInfoData buf;
initStringInfo(&buf);
- escape_json(&buf, str, len);
- /* free temporary buffer */
- if (str != json)
- pfree(str);
+ if (len >= 0)
+ escape_json(&buf, json, len);
+ else
+ escape_json_cstring(&buf, json);
str = buf.data;
}
+ else if (len >= 0)
+ {
+ /* Need to copy non-null-terminated string */
+ str = palloc(len + 1 * sizeof(char));
+ memcpy(str, json, len);
+ str[len] = '\0';
+ }
+ else
+ {
+ /* string is already null-terminated */
+ str = unconstify(char *, json);
+ }
}
else
{
On Wed, 24 Jul 2024 at 22:55, Heikki Linnakangas <hlinnaka@iki.fi> wrote:
On 02/07/2024 07:49, David Rowley wrote:
I've attached a rebased set of patches. The previous set no longer applied.
I looked briefly at the first patch. Seems reasonable.
One little thing that caught my eye is that in populate_scalar(), you
sometimes make a temporary copy of the string to add the
null-terminator, but then call escape_json() which doesn't need the
null-terminator anymore. See attached patch to avoid that. However, it's
not clear to me how to reach that codepath, or if it reachable at all. I
tried to add a NOTICE there and ran the regression tests, but got no
failures.
Thanks for noticing that. It seems like a good simplification
regardless. I've incorporated it.
I made another pass over the 0001 and 0003 patches and after a bit of
renaming, I pushed the result. I ended up keeping escape_json() as-is
and giving the new function the name escape_json_with_len(). The text
version is named ecape_json_text(). I think originally I did it the
other way as thought I'd have been able to adjust more locations than
I did. Having it this way around is slightly less churn.
I did another round of testing on the SIMD patch (attached as v5-0001)
as I wondered if the SIMD loop maybe shouldn't wait too long before
copying the bytes to the destination string. I had wondered if the
JSON string was very large that if we looked ahead too far that by the
time we flush those bytes out to the destination buffer, we'd have
started eviction of L1 cachelines for parts of the buffer that are
still to be flushed. I put this to the test (test 3) and found that
with a 1MB JSON string it is faster to flush every 512 bytes than it
is to only flush after checking the entire 1MB. With a 10kB JSON
string (test 2), the extra code to flush every 512 bytes seems to slow
things down. I'm a bit undecided about whether the flushing is
worthwhile or not. It really depend on the length of JSON strings we'd
like to optimise for. It might be possible to get the best of both but
I think it might require manually implementing portions of
appendBinaryStringInfo(). I'd rather not go there. Does anyone have
any thoughts about that?
Test 2 (10KB) does show a ~261% performance increase but dropped to
~227% flushing every 512 bytes. Test 3 (1MB) increased performance by
~99% without early flushing and increased to ~156% flushing every 512
bytes.
bench.sql: select row_to_json(j1)::jsonb from j1;
## Test 1 (variable JSON strings up to 1KB)
create table j1 (very_long_column_name_to_test_json_escape text);
insert into j1 select repeat('x', x) from generate_series(0,1024)x;
vacuum freeze j1;
master @ 17a5871d:
$ for i in {1..3}; do pgbench -n -f bench.sql -T 10 -M prepared
postgres | grep tps; done
tps = 364.410386 (without initial connection time)
tps = 367.914165 (without initial connection time)
tps = 365.794513 (without initial connection time)
master + v5-0001
$ for i in {1..3}; do pgbench -n -f bench.sql -T 10 -M prepared
postgres | grep tps; done
tps = 683.570613 (without initial connection time)
tps = 685.206578 (without initial connection time)
tps = 679.014056 (without initial connection time)
## Test 2 (10KB JSON strings)
create table j1 (very_long_column_name_to_test_json_escape text);
insert into j1 select repeat('x', 1024*10) from generate_series(0,1024)x;
vacuum freeze j1;
master @ 17a5871d:
$ for i in {1..3}; do pgbench -n -f bench.sql -T 10 -M prepared
postgres | grep tps; done
tps = 23.872630 (without initial connection time)
tps = 26.232014 (without initial connection time)
tps = 26.495739 (without initial connection time)
master + v5-0001
$ for i in {1..3}; do pgbench -n -f bench.sql -T 10 -M prepared
postgres | grep tps; done
tps = 96.813515 (without initial connection time)
tps = 96.023632 (without initial connection time)
tps = 99.630428 (without initial connection time)
master + v5-0001 ESCAPE_JSON_MAX_LOOKHEAD 512
$ for i in {1..3}; do pgbench -n -f bench.sql -T 10 -M prepared
postgres | grep tps; done
tps = 83.597442 (without initial connection time)
tps = 85.045554 (without initial connection time)
tps = 82.105907 (without initial connection time)
## Test 3 (1MB JSON strings)
create table j1 (very_long_column_name_to_test_json_escape text);
insert into j1 select repeat('x', 1024*1024) from generate_series(0,10)x;
vacuum freeze j1;
master @ 17a5871d:
$ for i in {1..3}; do pgbench -n -f bench.sql -T 10 -M prepared
postgres | grep tps; done
tps = 18.885922 (without initial connection time)
tps = 18.829701 (without initial connection time)
tps = 18.889369 (without initial connection time)
master v5-0001
$ for i in {1..3}; do pgbench -n -f bench.sql -T 10 -M prepared
postgres | grep tps; done
tps = 37.464967 (without initial connection time)
tps = 37.536676 (without initial connection time)
tps = 37.561387 (without initial connection time)
master + v5-0001 ESCAPE_JSON_MAX_LOOKHEAD 512
$ for i in {1..3}; do pgbench -n -f bench.sql -T 10 -M prepared
postgres | grep tps; done
tps = 48.296320 (without initial connection time)
tps = 48.118151 (without initial connection time)
tps = 48.507530 (without initial connection time)
David
Attachments:
v5-0001-Optimize-escaping-of-JSON-strings-using-SIMD.patchapplication/octet-stream; name=v5-0001-Optimize-escaping-of-JSON-strings-using-SIMD.patchDownload
From 58a913589b0b89a8c5ece50b5f8de6c9321a8366 Mon Sep 17 00:00:00 2001
From: David Rowley <dgrowley@gmail.com>
Date: Thu, 23 May 2024 10:53:23 +1200
Subject: [PATCH v5] Optimize escaping of JSON strings using SIMD
Here we adjust escape_json_with_len() to make use of SIMD to allow
processing of up to 16-bytes at a time rather than processing a single
byte at a time. This has been shown to speed up escaping of JSON
strings significantly, especially when no escaping is required.
Reviewed-by: Melih Mutlu
Discussion: https://postgr.es/m/CAApHDvpLXwMZvbCKcdGfU9XQjGCDm7tFpRdTXuB9PVgpNUYfEQ@mail.gmail.com
---
src/backend/utils/adt/json.c | 82 +++++++++++++++++++++++++++++-
src/test/regress/expected/json.out | 48 +++++++++++++++++
src/test/regress/sql/json.sql | 7 +++
3 files changed, 135 insertions(+), 2 deletions(-)
diff --git a/src/backend/utils/adt/json.c b/src/backend/utils/adt/json.c
index be7bc46038..4e86d734e4 100644
--- a/src/backend/utils/adt/json.c
+++ b/src/backend/utils/adt/json.c
@@ -19,6 +19,7 @@
#include "funcapi.h"
#include "libpq/pqformat.h"
#include "miscadmin.h"
+#include "port/simd.h"
#include "utils/array.h"
#include "utils/builtins.h"
#include "utils/date.h"
@@ -1603,11 +1604,88 @@ escape_json(StringInfo buf, const char *str)
void
escape_json_with_len(StringInfo buf, const char *str, int len)
{
+ int i = 0;
+ int copypos = 0;
+ int vlen;
+
+ Assert(len >= 0);
+
+ /*
+ * Figure out how many bytes to process using SIMD. Round 'len' down to
+ * the previous multiple of sizeof(Vector8), assuming that's a power-of-2.
+ */
+ vlen = len & (int) (~(sizeof(Vector8) - 1));
+
appendStringInfoCharMacro(buf, '"');
- for (int i = 0; i < len; i++)
- escape_json_char(buf, str[i]);
+ for (;;)
+ {
+ /*
+ * To speed this up try searching sizeof(Vector8) bytes at once for
+ * special characters that we need to escape. When we find one, we
+ * fall out of the Vector8 loop and copy the portion we've vector
+ * searched and then we process sizeof(Vector8) bytes one byte at a
+ * time. Once done, come back and try doing vector searching again.
+ * We'll also process any remaining bytes at the tail end of the
+ * string byte-by-byte. This optimization assumes special characters
+ * are not that common.
+ */
+ for (; i < vlen; i += sizeof(Vector8))
+ {
+ Vector8 chunk;
+
+ vector8_load(&chunk, (const uint8 *) &str[i]);
+
+ /*
+ * Break on anything less than ' ' or if we find a '"' or '\\'.
+ * Those need special handling. That's done in the per-byte loop.
+ */
+ if (vector8_has_le(chunk, (unsigned char) 0x1F) ||
+ vector8_has(chunk, (unsigned char) '"') ||
+ vector8_has(chunk, (unsigned char) '\\'))
+ break;
+
+/* #define ESCAPE_JSON_MAX_LOOKHEAD 512 */
+#ifdef ESCAPE_JSON_MAX_LOOKHEAD
+ if (i - copypos >= ESCAPE_JSON_MAX_LOOKHEAD)
+ {
+ appendBinaryStringInfo(buf, &str[copypos], i - copypos);
+ copypos = i;
+ }
+#endif
+ }
+
+ /*
+ * Write to the destination up to the point of that we've vector
+ * searched so far. Do this only when switching into per-byte mode
+ * rather than once every sizeof(Vector8) bytes.
+ */
+ if (copypos < i)
+ {
+ appendBinaryStringInfo(buf, &str[copypos], i - copypos);
+ copypos = i;
+ }
+
+ /*
+ * Per-byte loop for Vector8s containing special chars and for
+ * processing the tail of the string.
+ */
+ for (int b = 0; b < sizeof(Vector8); b++)
+ {
+ /* check if we've finished */
+ if (i == len)
+ goto done;
+
+ Assert(i < len);
+
+ escape_json_char(buf, str[i++]);
+ }
+
+ copypos = i;
+ /* We're not done yet. Try the vector search again */
+ }
+done:
appendStringInfoCharMacro(buf, '"');
}
diff --git a/src/test/regress/expected/json.out b/src/test/regress/expected/json.out
index aa29bc597b..c8e9b97f0a 100644
--- a/src/test/regress/expected/json.out
+++ b/src/test/regress/expected/json.out
@@ -55,6 +55,54 @@ SELECT ('"'||repeat('.', 12)||'abc\n"')::json; -- OK, legal escapes
"............abc\n"
(1 row)
+-- Test various lengths of strings to validate SIMD processing to escape
+-- special chars in the JSON.
+SELECT row_to_json(j)::jsonb FROM (
+ SELECT left(E'abcdefghijklmnopqrstuvwxyz0123456"\t78', a) AS very_long_column_name_to_test_json_escape
+ FROM generate_series(0,37) a
+) j;
+ row_to_json
+------------------------------------------------------------------------------------------
+ {"very_long_column_name_to_test_json_escape": ""}
+ {"very_long_column_name_to_test_json_escape": "a"}
+ {"very_long_column_name_to_test_json_escape": "ab"}
+ {"very_long_column_name_to_test_json_escape": "abc"}
+ {"very_long_column_name_to_test_json_escape": "abcd"}
+ {"very_long_column_name_to_test_json_escape": "abcde"}
+ {"very_long_column_name_to_test_json_escape": "abcdef"}
+ {"very_long_column_name_to_test_json_escape": "abcdefg"}
+ {"very_long_column_name_to_test_json_escape": "abcdefgh"}
+ {"very_long_column_name_to_test_json_escape": "abcdefghi"}
+ {"very_long_column_name_to_test_json_escape": "abcdefghij"}
+ {"very_long_column_name_to_test_json_escape": "abcdefghijk"}
+ {"very_long_column_name_to_test_json_escape": "abcdefghijkl"}
+ {"very_long_column_name_to_test_json_escape": "abcdefghijklm"}
+ {"very_long_column_name_to_test_json_escape": "abcdefghijklmn"}
+ {"very_long_column_name_to_test_json_escape": "abcdefghijklmno"}
+ {"very_long_column_name_to_test_json_escape": "abcdefghijklmnop"}
+ {"very_long_column_name_to_test_json_escape": "abcdefghijklmnopq"}
+ {"very_long_column_name_to_test_json_escape": "abcdefghijklmnopqr"}
+ {"very_long_column_name_to_test_json_escape": "abcdefghijklmnopqrs"}
+ {"very_long_column_name_to_test_json_escape": "abcdefghijklmnopqrst"}
+ {"very_long_column_name_to_test_json_escape": "abcdefghijklmnopqrstu"}
+ {"very_long_column_name_to_test_json_escape": "abcdefghijklmnopqrstuv"}
+ {"very_long_column_name_to_test_json_escape": "abcdefghijklmnopqrstuvw"}
+ {"very_long_column_name_to_test_json_escape": "abcdefghijklmnopqrstuvwx"}
+ {"very_long_column_name_to_test_json_escape": "abcdefghijklmnopqrstuvwxy"}
+ {"very_long_column_name_to_test_json_escape": "abcdefghijklmnopqrstuvwxyz"}
+ {"very_long_column_name_to_test_json_escape": "abcdefghijklmnopqrstuvwxyz0"}
+ {"very_long_column_name_to_test_json_escape": "abcdefghijklmnopqrstuvwxyz01"}
+ {"very_long_column_name_to_test_json_escape": "abcdefghijklmnopqrstuvwxyz012"}
+ {"very_long_column_name_to_test_json_escape": "abcdefghijklmnopqrstuvwxyz0123"}
+ {"very_long_column_name_to_test_json_escape": "abcdefghijklmnopqrstuvwxyz01234"}
+ {"very_long_column_name_to_test_json_escape": "abcdefghijklmnopqrstuvwxyz012345"}
+ {"very_long_column_name_to_test_json_escape": "abcdefghijklmnopqrstuvwxyz0123456"}
+ {"very_long_column_name_to_test_json_escape": "abcdefghijklmnopqrstuvwxyz0123456\""}
+ {"very_long_column_name_to_test_json_escape": "abcdefghijklmnopqrstuvwxyz0123456\"\t"}
+ {"very_long_column_name_to_test_json_escape": "abcdefghijklmnopqrstuvwxyz0123456\"\t7"}
+ {"very_long_column_name_to_test_json_escape": "abcdefghijklmnopqrstuvwxyz0123456\"\t78"}
+(38 rows)
+
-- see json_encoding test for input with unicode escapes
-- Numbers.
SELECT '1'::json; -- OK
diff --git a/src/test/regress/sql/json.sql b/src/test/regress/sql/json.sql
index ec57dfe707..9bf33115d4 100644
--- a/src/test/regress/sql/json.sql
+++ b/src/test/regress/sql/json.sql
@@ -12,6 +12,13 @@ SELECT '"\v"'::json; -- ERROR, not a valid JSON escape
SELECT ('"'||repeat('.', 12)||'abc"')::json; -- OK
SELECT ('"'||repeat('.', 12)||'abc\n"')::json; -- OK, legal escapes
+-- Test various lengths of strings to validate SIMD processing to escape
+-- special chars in the JSON.
+SELECT row_to_json(j)::jsonb FROM (
+ SELECT left(E'abcdefghijklmnopqrstuvwxyz0123456"\t78', a) AS very_long_column_name_to_test_json_escape
+ FROM generate_series(0,37) a
+) j;
+
-- see json_encoding test for input with unicode escapes
-- Numbers.
--
2.34.1
On Sun, 28 Jul 2024 at 00:51, David Rowley <dgrowleyml@gmail.com> wrote:
I did another round of testing on the SIMD patch (attached as v5-0001)
as I wondered if the SIMD loop maybe shouldn't wait too long before
copying the bytes to the destination string. I had wondered if the
JSON string was very large that if we looked ahead too far that by the
time we flush those bytes out to the destination buffer, we'd have
started eviction of L1 cachelines for parts of the buffer that are
still to be flushed. I put this to the test (test 3) and found that
with a 1MB JSON string it is faster to flush every 512 bytes than it
is to only flush after checking the entire 1MB. With a 10kB JSON
string (test 2), the extra code to flush every 512 bytes seems to slow
things down.
I'd been wondering why test 2 (10KB) with v5-0001
ESCAPE_JSON_MAX_LOOKHEAD 512 was not better than v5-0001. It occurred
to me that when using 10KB vs 1MB and flushing the buffer every 512
bytes that enlargeStringInfo() is called more often proportionally to
the length of the string. Doing that causes more repalloc/memcpy work
in stringinfo.c.
We can reduce the repalloc/memcpy work by calling enlargeStringInfo()
once at the beginning of escape_json_with_len(). We already know the
minimum length we're going to append so we might as well do that.
After making that change, doing the 512-byte flushing no longer slows
down test 2.
Here are the results of testing v6-0001. I've added test 4, which
tests a very short string to ensure there are no performance
regressions when we can't do SIMD. Test 2 patched came out 3.74x
faster than master.
## Test 1:
echo "select row_to_json(j1)::jsonb from j1;" > test1.sql
for i in {1..3}; do pgbench -n -f test1.sql -T 10 -M prepared postgres
| grep tps; done
master @ e6a963748:
tps = 339.560611
tps = 344.649009
tps = 343.246659
v6-0001:
tps = 610.734018
tps = 628.297298
tps = 630.028225
v6-0001 ESCAPE_JSON_MAX_LOOKHEAD 512:
tps = 557.562866
tps = 626.476618
tps = 618.665045
## Test 2:
echo "select row_to_json(j2)::jsonb from j2;" > test2.sql
for i in {1..3}; do pgbench -n -f test2.sql -T 10 -M prepared postgres
| grep tps; done
master @ e6a963748:
tps = 25.633934
tps = 18.580632
tps = 25.395866
v6-0001:
tps = 89.325752
tps = 91.277016
tps = 86.289533
v6-0001 ESCAPE_JSON_MAX_LOOKHEAD 512:
tps = 85.194479
tps = 90.054279
tps = 85.483279
## Test 3:
echo "select row_to_json(j3)::jsonb from j3;" > test3.sql
for i in {1..3}; do pgbench -n -f test3.sql -T 10 -M prepared postgres
| grep tps; done
master @ e6a963748:
tps = 18.863420
tps = 18.866374
tps = 18.791395
v6-0001:
tps = 38.990681
tps = 37.893820
tps = 38.057235
v6-0001 ESCAPE_JSON_MAX_LOOKHEAD 512:
tps = 46.076842
tps = 46.400413
tps = 46.165491
## Test 4:
echo "select row_to_json(j4)::jsonb from j4;" > test4.sql
for i in {1..3}; do pgbench -n -f test4.sql -T 10 -M prepared postgres
| grep tps; done
master @ e6a963748:
tps = 1700.888458
tps = 1684.753818
tps = 1690.262772
v6-0001:
tps = 1721.821561
tps = 1699.189207
tps = 1663.618117
v6-0001 ESCAPE_JSON_MAX_LOOKHEAD 512:
tps = 1701.565562
tps = 1706.310398
tps = 1687.585128
I'm pretty happy with this now so I'd like to commit this and move on
to other work. Doing "#define ESCAPE_JSON_MAX_LOOKHEAD 512", seems
like the right thing. If anyone else wants to verify my results or
take a look at the patch, please do so.
David
Attachments:
v6-0001-Optimize-escaping-of-JSON-strings-using-SIMD.patchapplication/octet-stream; name=v6-0001-Optimize-escaping-of-JSON-strings-using-SIMD.patchDownload
From 02ae2dc0f53dbefab972e5efb47821e66a6cd678 Mon Sep 17 00:00:00 2001
From: David Rowley <dgrowley@gmail.com>
Date: Thu, 23 May 2024 10:53:23 +1200
Subject: [PATCH v6] Optimize escaping of JSON strings using SIMD
Here we adjust escape_json_with_len() to make use of SIMD to allow
processing of up to 16-bytes at a time rather than processing a single
byte at a time. This has been shown to speed up escaping of JSON
strings significantly, especially when no escaping is required.
Reviewed-by: Melih Mutlu
Discussion: https://postgr.es/m/CAApHDvpLXwMZvbCKcdGfU9XQjGCDm7tFpRdTXuB9PVgpNUYfEQ@mail.gmail.com
---
src/backend/utils/adt/json.c | 89 +++++++++++++++++++++++++++++-
src/test/regress/expected/json.out | 48 ++++++++++++++++
src/test/regress/sql/json.sql | 7 +++
3 files changed, 142 insertions(+), 2 deletions(-)
diff --git a/src/backend/utils/adt/json.c b/src/backend/utils/adt/json.c
index be7bc46038..eb5548384e 100644
--- a/src/backend/utils/adt/json.c
+++ b/src/backend/utils/adt/json.c
@@ -19,6 +19,7 @@
#include "funcapi.h"
#include "libpq/pqformat.h"
#include "miscadmin.h"
+#include "port/simd.h"
#include "utils/array.h"
#include "utils/builtins.h"
#include "utils/date.h"
@@ -1603,11 +1604,95 @@ escape_json(StringInfo buf, const char *str)
void
escape_json_with_len(StringInfo buf, const char *str, int len)
{
+ int i = 0;
+ int copypos = 0;
+ int vlen;
+
+ Assert(len >= 0);
+
+ /*
+ * Since we know the minimum length we'll need to append, let's just
+ * enlarge the buffer now rather than than incrementally making more space
+ * when we run out. Add two extra bytes for the enclosing quotes.
+ */
+ enlargeStringInfo(buf, len + 2);
+
+ /*
+ * Figure out how many bytes to process using SIMD. Round 'len' down to
+ * the previous multiple of sizeof(Vector8), assuming that's a power-of-2.
+ */
+ vlen = len & (int) (~(sizeof(Vector8) - 1));
+
appendStringInfoCharMacro(buf, '"');
- for (int i = 0; i < len; i++)
- escape_json_char(buf, str[i]);
+ for (;;)
+ {
+ /*
+ * To speed this up try searching sizeof(Vector8) bytes at once for
+ * special characters that we need to escape. When we find one, we
+ * fall out of the Vector8 loop and copy the portion we've vector
+ * searched and then we process sizeof(Vector8) bytes one byte at a
+ * time. Once done, come back and try doing vector searching again.
+ * We'll also process any remaining bytes at the tail end of the
+ * string byte-by-byte. This optimization assumes special characters
+ * are not that common.
+ */
+ for (; i < vlen; i += sizeof(Vector8))
+ {
+ Vector8 chunk;
+
+ vector8_load(&chunk, (const uint8 *) &str[i]);
+
+ /*
+ * Break on anything less than ' ' or if we find a '"' or '\\'.
+ * Those need special handling. That's done in the per-byte loop.
+ */
+ if (vector8_has_le(chunk, (unsigned char) 0x1F) ||
+ vector8_has(chunk, (unsigned char) '"') ||
+ vector8_has(chunk, (unsigned char) '\\'))
+ break;
+
+/* #define ESCAPE_JSON_MAX_LOOKHEAD 512 */
+#ifdef ESCAPE_JSON_MAX_LOOKHEAD
+ if (i - copypos >= ESCAPE_JSON_MAX_LOOKHEAD)
+ {
+ appendBinaryStringInfo(buf, &str[copypos], i - copypos);
+ copypos = i;
+ }
+#endif
+ }
+
+ /*
+ * Write to the destination up to the point of that we've vector
+ * searched so far. Do this only when switching into per-byte mode
+ * rather than once every sizeof(Vector8) bytes.
+ */
+ if (copypos < i)
+ {
+ appendBinaryStringInfo(buf, &str[copypos], i - copypos);
+ copypos = i;
+ }
+
+ /*
+ * Per-byte loop for Vector8s containing special chars and for
+ * processing the tail of the string.
+ */
+ for (int b = 0; b < sizeof(Vector8); b++)
+ {
+ /* check if we've finished */
+ if (i == len)
+ goto done;
+
+ Assert(i < len);
+
+ escape_json_char(buf, str[i++]);
+ }
+
+ copypos = i;
+ /* We're not done yet. Try the vector search again */
+ }
+done:
appendStringInfoCharMacro(buf, '"');
}
diff --git a/src/test/regress/expected/json.out b/src/test/regress/expected/json.out
index aa29bc597b..c8e9b97f0a 100644
--- a/src/test/regress/expected/json.out
+++ b/src/test/regress/expected/json.out
@@ -55,6 +55,54 @@ SELECT ('"'||repeat('.', 12)||'abc\n"')::json; -- OK, legal escapes
"............abc\n"
(1 row)
+-- Test various lengths of strings to validate SIMD processing to escape
+-- special chars in the JSON.
+SELECT row_to_json(j)::jsonb FROM (
+ SELECT left(E'abcdefghijklmnopqrstuvwxyz0123456"\t78', a) AS very_long_column_name_to_test_json_escape
+ FROM generate_series(0,37) a
+) j;
+ row_to_json
+------------------------------------------------------------------------------------------
+ {"very_long_column_name_to_test_json_escape": ""}
+ {"very_long_column_name_to_test_json_escape": "a"}
+ {"very_long_column_name_to_test_json_escape": "ab"}
+ {"very_long_column_name_to_test_json_escape": "abc"}
+ {"very_long_column_name_to_test_json_escape": "abcd"}
+ {"very_long_column_name_to_test_json_escape": "abcde"}
+ {"very_long_column_name_to_test_json_escape": "abcdef"}
+ {"very_long_column_name_to_test_json_escape": "abcdefg"}
+ {"very_long_column_name_to_test_json_escape": "abcdefgh"}
+ {"very_long_column_name_to_test_json_escape": "abcdefghi"}
+ {"very_long_column_name_to_test_json_escape": "abcdefghij"}
+ {"very_long_column_name_to_test_json_escape": "abcdefghijk"}
+ {"very_long_column_name_to_test_json_escape": "abcdefghijkl"}
+ {"very_long_column_name_to_test_json_escape": "abcdefghijklm"}
+ {"very_long_column_name_to_test_json_escape": "abcdefghijklmn"}
+ {"very_long_column_name_to_test_json_escape": "abcdefghijklmno"}
+ {"very_long_column_name_to_test_json_escape": "abcdefghijklmnop"}
+ {"very_long_column_name_to_test_json_escape": "abcdefghijklmnopq"}
+ {"very_long_column_name_to_test_json_escape": "abcdefghijklmnopqr"}
+ {"very_long_column_name_to_test_json_escape": "abcdefghijklmnopqrs"}
+ {"very_long_column_name_to_test_json_escape": "abcdefghijklmnopqrst"}
+ {"very_long_column_name_to_test_json_escape": "abcdefghijklmnopqrstu"}
+ {"very_long_column_name_to_test_json_escape": "abcdefghijklmnopqrstuv"}
+ {"very_long_column_name_to_test_json_escape": "abcdefghijklmnopqrstuvw"}
+ {"very_long_column_name_to_test_json_escape": "abcdefghijklmnopqrstuvwx"}
+ {"very_long_column_name_to_test_json_escape": "abcdefghijklmnopqrstuvwxy"}
+ {"very_long_column_name_to_test_json_escape": "abcdefghijklmnopqrstuvwxyz"}
+ {"very_long_column_name_to_test_json_escape": "abcdefghijklmnopqrstuvwxyz0"}
+ {"very_long_column_name_to_test_json_escape": "abcdefghijklmnopqrstuvwxyz01"}
+ {"very_long_column_name_to_test_json_escape": "abcdefghijklmnopqrstuvwxyz012"}
+ {"very_long_column_name_to_test_json_escape": "abcdefghijklmnopqrstuvwxyz0123"}
+ {"very_long_column_name_to_test_json_escape": "abcdefghijklmnopqrstuvwxyz01234"}
+ {"very_long_column_name_to_test_json_escape": "abcdefghijklmnopqrstuvwxyz012345"}
+ {"very_long_column_name_to_test_json_escape": "abcdefghijklmnopqrstuvwxyz0123456"}
+ {"very_long_column_name_to_test_json_escape": "abcdefghijklmnopqrstuvwxyz0123456\""}
+ {"very_long_column_name_to_test_json_escape": "abcdefghijklmnopqrstuvwxyz0123456\"\t"}
+ {"very_long_column_name_to_test_json_escape": "abcdefghijklmnopqrstuvwxyz0123456\"\t7"}
+ {"very_long_column_name_to_test_json_escape": "abcdefghijklmnopqrstuvwxyz0123456\"\t78"}
+(38 rows)
+
-- see json_encoding test for input with unicode escapes
-- Numbers.
SELECT '1'::json; -- OK
diff --git a/src/test/regress/sql/json.sql b/src/test/regress/sql/json.sql
index ec57dfe707..9bf33115d4 100644
--- a/src/test/regress/sql/json.sql
+++ b/src/test/regress/sql/json.sql
@@ -12,6 +12,13 @@ SELECT '"\v"'::json; -- ERROR, not a valid JSON escape
SELECT ('"'||repeat('.', 12)||'abc"')::json; -- OK
SELECT ('"'||repeat('.', 12)||'abc\n"')::json; -- OK, legal escapes
+-- Test various lengths of strings to validate SIMD processing to escape
+-- special chars in the JSON.
+SELECT row_to_json(j)::jsonb FROM (
+ SELECT left(E'abcdefghijklmnopqrstuvwxyz0123456"\t78', a) AS very_long_column_name_to_test_json_escape
+ FROM generate_series(0,37) a
+) j;
+
-- see json_encoding test for input with unicode escapes
-- Numbers.
--
2.34.1
On Thu, 1 Aug 2024 at 16:15, David Rowley <dgrowleyml@gmail.com> wrote:
I'm pretty happy with this now so I'd like to commit this and move on
to other work. Doing "#define ESCAPE_JSON_MAX_LOOKHEAD 512", seems
like the right thing. If anyone else wants to verify my results or
take a look at the patch, please do so.
I did some more testing on this on a few different machines; apple M2
Ultra, AMD 7945HX and with a Raspberry Pi 4.
I've attached the results as graphs with the master time normalised to
1. I tried out quite a few different values for flushing the buffer,
256 bytes in powers of 2 up to 8192 bytes. It seems like each machine
has its own preference to what this should be set to, but no machine
seems to be too picky about the exact value. They're all small enough
values to fit in L1d cache on each of the CPUs. Test 4 shouldn't
change much as there's no SIMD going on in that test. You might notice
a bit of noise from all machines for test 4, apart from the M2. You
can assume a similar level of noise for tests 1 to 3 on each of the
machines. The Raspberry Pi does seem to prefer not flushing the
buffer until the end (listed as "patched" in the graphs). I suspect
that's because that CPU does better with less code. I've not taken
these results quite as seriously since it's likely a platform that we
wouldn't want to prefer when it comes to tuning optimisations. I was
mostly interested in not seeing regressions.
I think, if nobody else thinks differently, I'll rename
ESCAPE_JSON_MAX_LOOKHEAD to ESCAPE_JSON_FLUSH_AFTER and set it to 512.
The exact value does not seem to matter too much and 512 seems fine.
It's better for the M2 than the 7945HX, but not by much.
I've also attached the script I ran to get these results and also the
full results.
David
Attachments:
raspberrypi4.pngimage/png; name=raspberrypi4.pngDownload
�PNG
IHDR � � ��� sRGB ��� gAMA ���a pHYs � ����e ��IDATx^�� �Us��_�mJ��bI�D=Q"��D�,!�"k!I��.IJ�li!���V����5����s����3���L�y�^�5����{g�����~�_�� �L��� � ��@ �Dr @&"� 2� ��r���� �d���6p�@���w&z�
��Z�r��J�*V�B;��c�[���K����y��V�@w��!-_y��u[E����+[��U�D��+W.���������c��ow���y���l���83m����{�=��s�;������+�t� ���B �lj��=�y�f�5k�}�����sg�4i����y� �������������k��f={��u��y���:d�G�N� @�C �QB�@�|���1�bcc�� �j�����wo�;w�w&k�1c�-]��; Y� Y@�|�\�i���#�*V�h�>��N+Q�bn��i� ��}�E����w���Tu���_��U��3Y��5kl��q�? @��r �k\w�q���[�b��q��Z�����g'���-ZX����3�k����K��������~K�������^Cn��}��G�����3�a
9 �*� ��0�z��V�~�$�r[�l�?���;-�iM�49,�RZ���HSh8e�[�b�w der ds
�.��2;��3�3 �-[�Zr@�o���.�R�Jyg��M-\��;�:T8y�d��}kR, ��� 8
s�1v�9�xG �>w���@j-Z��>�l�(��
�a����3������O?�t� ��� 8J���g��=��9��S��
�bbb��#Kq����M{��v�5���! YC 8�k�C��3g�����#�%JX���]�OJ�m�fs�������+����N���3��Z�j�I'�t�t����4�u��Enm;?$L��2������?3=�[(��k(,��;���*���m��$7�AUQ?���-^��v���n/X���.]�=�^'O�<�|4�?���_�>��K���F?�Q���'�h��Us�?������$�:���n7et��u��J���5����|�%��k�g������:u��|��t��%K������?w-���P�FS�������*T�`��{��I����Z�`�}����Q���#1�A���~ht�}�
���/w�V������ d5T� pP��W��4iU�Kr������u��������I�8Q��y�f�>}����;��gON�d���6h� ��?���]+�B=��O�t�o���}���}�aV\\�m����Ho����>���EC�7z�h�5q��D��
D�t�������z��n=�>?L���� �#F��/�l���O���s�����{���
����C���*�Z����8�>��7��u
��0U�� +���d����;=�B���qa��M�^q�������.~���.���^z�� Y� GU)X ����B
?����hB��k����p���������#}n=����W�V�\�������=h�l��}S�o ��
�4�R!Trv��m�|��M�81�k���HT������?�`��������I�M` �b��.�
���~�[0�G���V�\�$�b
I�C�p�����-x@���`Q_��#�V�(����]0*��k��6K� �</��� �D�Fj��+���v���j�'U�)RE����y�"E�3I)0�����}P������������S�U>��S-x��s
�t�r���}1��X
�|z?����V�s�w�y��
��K?u=��{n��B�W~u�>;�����^�~��v�
7X��u]�����>(:������
.��U ������s���][�>�����A�B]��9���ZB�^���(��D�����}F�k��2e���J���|
���H�C��{��7��=�����w�����^�z�����X�S���p� ������N������������Z
���D���I�+���\pAb�]j����*'L������4���SNq������q�UV��������BL���oF��a���w�����_yG�BN
{ Y� G@Z�\�XZLS_}�U�VL��\kH)�Ea���#]��Sh����:�,��}hS��P�f��n_�R~x��A����@j{Uk�m��v�[oK�*�[�ZwM�30X��/^<��[�a����z�-Z��A���_a�B3�Nj����5�n��/8� �^|Z3��t!�����h�;}ozOj��/?��2��K-����=h
:}��<��H�=����5��S���t���g��7X��P8�����3����Q�F��(�3�)���k�A�w�M���9��~p��A�KJ��i�<?8�}FF��_r�+�����0Z�M�1T�\fr���n��s@�w� >��&� k�e �,@���dO?�t����>k�;w�>��UZ�����*x�Q�X�����Z���JUg>�~H�X~ �������.�U]�
�"]�L�U��MC$z}
3����\8�S[��k�����_�U(�]g�a��[AS0�UX���TU}�}�zmb
E�����;E��CDK�w�������A�>����&��?���;2��H��2�>W�^�=�-��w�MRa��}E2(%���BA�_������ �z� 8J(8���[\x�\U��f�E�E�C�Z[�������Pk��a\4T
�`P�DZ�L���Ac8
�492�*,�rz^�*�4�5%
`��)�D
�t������L>u����/��bx�D-���O��u��Q5��'Z
#�X�g��A�~��5��60U�]J�_jDx?��3��S'����T��@�HR�?W�gzT ��G @6� G�I�&M�}����-�pG��j�T������-%�G��y�8��`jM��{��'��������������*~2�FL��Z3���Sk�^#j�\WL����6c�h�J�*E���]�]Fr�|�[�.9
##������jH�7��6�~7z��]�S ���^����L)��H
/5��o����
Q ��� ���JU\��u���,8�Q@����;�c�=���"���
�{�����v��!�Zm>�%Z�����{����S��00��:\o���}��G.�IM](
F+�R��L��Ok��
�Z(#
��~��n�����&��-�M�{�����p��O!i����T�)]O`k�I����G���"���Ljs�����#���E���S��kH�� @�����= 2����&�S��z����.����Vx����kZ�qjS���������/Ux�X�}��.H��:j�(�>}�w&)���}��T-�jO�4T2dHb����-[�����5j�=�<U
>��CI*������%0RZ�_�S}��z��7�}tj���C��!�:��q���F(~��^���/R�f�r����
��~'4pBtz�~��}l)G���G�=FR�S���O���Q����\��_������y�{����~�7n�������~� @�@� ��MRT�X������)���*n�t���U�)HP��PI�\Ja\8z�W_}u��C
������^x�W}7g�7�2R
J�R���\K/�h� �B�!���Zs/�t
�SvCQ�bj>+=&��1������i��C,D�E�DV�_��(���x�m���)�R��i��~�E�����_jt ���@ �lD���bIUQ>�>��Skh��U���m��a�-p�(�PX�n]W� -���Y37}���������C������\$�D���������M�~)�
��)��U�W��)�=ztb����
��
@6G�* G@�-�����������E��Z0SZ�L��\(H�mj��Z^Z;M����{�����e5����^Sp�g�v��������_�����@����^U��}��-��� ���h��T���E������N�������|) ��S�5�Uk�N4��i�����B�1c�������$�_��-����^��� ��
9 �!�`
9i��w�}�������qjKl������K�uUSN�X�^C�8��8�T�S�u�l=�����+��p��K.qm�����:
�X�x�w&4�j&w����'m;fJ�C��!���HU>��j�]i2�B]�����s��[�u?� '!� RH��������Knh�Z�.M����u��qm�)���J�����}{��W/�g�����?�����")R�w�N2C��)rt
U3R��S{m��j
�[9��]
\CP��~���cM��iRn$�{ G9 �)�;
6t��OA�?�v ����4I2�VAUsm���;
M��8q������N?Sd�6Y
�����T���&7�A�\4��&�����;J�^K)��FZ��o�>[�v�w����i���y������(��OU=*`U�d���JB�\�������Hh��N9���8���O?�� r dc
=*W��%P(4u��d�i��J,����J) SK�&��
OUQ��������N�zH����$�R�����D��Zr��e����k��Dm�
�|������$�*O����Z�l��(T����o�����>�U6+PY�|y�(��Rk *��i���*O$�v��n�)�D�����\�q���j��n G� ��ZW��<�`��)�C*p
�GA�&=���VU��0
��E��V�)�I)�S����?���R�A>UER!� ��o�M�p�>c��v
��;�<W%H!U���������Dx����;��,+N���f`����*9?���j��%iU �)� ���(�$V{��R����c
�����9y��� )��.��������<�A�A�u��S��������6����p�=�\���_ �=�n�m#F�H��:(�sH��<���p��>���h�����j`������v�`�>?���$�e
a���];�w��T�P�J�,����O�6-��h��s 9V��4R
d*U�����8S%��^xXEU����&�=TUW*��@��L[�f��|j�T���"����z�D|����/�$�D*(�F;��3�4+V��{�?������:m���n_����s����t�a�jSu�MN%��6�?�&w��u
z�����n3g�L�)(��������E���W���W��m����}�?�F��dX�nWK�D��#�v�����}��Q���7U���Q���P5�O�_�n]+S���������������X������-����������U���"#��������3}��/i���B�y��o 9r ����D�.L �vS��J3x�B(�8
�����?���U�)�SU�*�t^��yn��F�X��N�Sp ���?����,��Ihm9�o����VR�������N��o�=dfp �u����z�
����~p;����7��M2�{Q��B]��)0S����OM��]k��5K���*�t}
M��+�S��K����`�?���~��v�����*�`�Hr���u�a�O�c�)���@�@ ����U �����H��j
�jSp���`.����iS4i
��VO7�&�* �������.
���B��[`Kgr.�����;R��n���[�}�YoM���V�\%�n����M��D�W��X��}w��7�+���}��~�G����6�6k�U @NF �QB!�e�]�B�@�
��Mn|���]��*���M!�B8���eK{��g���?xh�*�T���/U����v�5�����z��m:V�N�&M�����������{T������<zM=w������_�,�y
�Z�jeg�uVb��>U>�y�qk���k����G}��x� �Y���^L���s��u�����O�����.�y��'��g���9 �D���/s �,l��!���w��w�I���&M�dc��q�
8^�r ��B Fkj@�O-��v @vA �����T_��j��h�� �6r ���;c�;x��;.Y��U�P�� �dr Hwqqq��/������3 Sq�-� �\r H��Z��=m��an�F��]��o����w���V�^�� �tr H�B�
����m��Yn��;�[����k
4����{g r69 ����k�������;wn���+�j��� � �T!W�lY+X��;�$���:�Z�nm�_~���
������ �`T� ��@ �Dr @&"� 2� ��� �LD d"9 � ��@ �Dr @&"� 2� ��� �LD d"9 � ��@ �Dr @&"� 2� ��� �LD d"9 � ��@ �D����� $����6������ _^�[/:��]��;�r �a�Y� �����Q~kx^^�8�����i��7�{�1�T��������/o��v�M�8�����{Fg��a����Z�li���� @����Xo/c�Z�9�drico���]z����gO[�x�;�j�*�������/�'�|������Gc��%� �iT������g�}�����@.�F�a�;w�c�9��~�m��u�����{�}���V�\9��������bc#O����k�W�v��F����7��^{�5+P��� 8z�3�7nl������hA ���msUp�v��n�����?n��w�*T����N{��W�H�"���_����n��*�V�\iU�Tqm��|��X�b�u ]�����\,_���O�n_|�]u�U�c:�[��U�^��N�U �n�:��z�)�X�R��� ����@r�^+S��w�q��������hm����.]jg�q��� )�����V�~}����l��5��K/�E]d�K�v�����)�L�g��e����:u���hk������{�e���I<x�������{��s�u�i���u�����_�_�������i���x�
w]��/���k�RYz.�G�����C]C���h�"<x��v=��g��M�6y�DF"�K�+���&L��WT�h(Z�M��'�|rTk����������ns���������E�����Lo '���_\P6`�������'�#�<b�G��i���p*���!�h�wh�_�}���.8���^~�e�)`�����oo-Z��q�������y�������Q#��U��%G����o���n��_=1G�s6������������`�������]�S��/_>��HrHCF�i�����5k�p-J�����8�r4�
����H�2� �����i�?��[JJ�M)tRE���>�1*<k����S5�n�={��t�B��7z���)�8p��s�9��'����U���:�y������5�A�<W\q�M�4���_��{T�������Y���T���W/w����C�I\����~s�����>�x�������r���>}?��������b�
.����a����2�Ru��j�C��-�hR��m��%n_�[��?��PN�\K�(�JR{���"� ������{�-5�u���sU�)�R���S
h<�����'�x�=��C�v�[�7�O�Wu[����C�V�^=�ov�O?���X���}����l��n��pa[�� q��/����k�`��n-�H�RO����*��~���k����SO=���~�L��7�|���SW���#W|��i��[���/vX�z�)��7�wk��f�o{���.
�����[�+:t����5|�p���C���e��# @F���Z�(�xj�~����J��r��y���_=������~���{�����nIx�
]�
�]aV�5�yU�������S�g
����9�n��FW������]�tqm��=�����tI+u�i9U�)8�)/PU��y�>�l�H���xS�����JJ���A ���q����.,S�$^[zf�����o�F��~���?Z�����{G ��v�)����Q���YT(\��K��@�p��s��f����tS�g` 'j
�[eU!��8����+���� ��;������^'��:
���#O�<Q]2�\:RJ��_�_r����^e�*;Mo���+�M��6��FC����$ s�*U�����{ ���-����~oZ_NC�2JpP����b��k����K.��n��v78R-���r��H�u�].?Pe�\�@ �N��R�W^y���UUr����F����c��n"���t �^���]�������E������w�yv�Yg�m��v�-���\���_��Mp?~|�pN��VXU����^uH*Q�b�Z�M�'��T�Z��i��������]D��0 ���)�={�tk����6o�<7�TY@���������p��wMZ����'Nt�\)���s���fK��7�q�nmxM`MnSe^���!s����8���*V��ZV�6m�81%5��?~W��e��lR:�OiI�@ ��~���w��W�
�PE8ZK.��
��G
O��r>=^Uu?��=�������K�k��|��v� '���6m��";!�KU��\a\�r�\���(�]�-�J^5!e��)n�2����q��1�� ���u�Bu��[��~��G��?ZW^��ZsNUv������:�4156�����N;�j��is�������������������l����}#� �K�?����y����m[7E�k��}����������;��M�v*w��F"kD�*�d���n�8�LV������� �#U��:>�^��Z��p?�i�j��
��ZU����X���m��.���$��*<U�i��O�i���k��s�����~�z�)x[�b�[o^x�j����N>�?��T�/��Q�\y�h1��%K&���.��S9����������
.���O<��W��U�� d�K/��--��RU�U�P�J�.�~*
{���l���n]x�9v������&>V�����������;�-������wE@j�]�r��MrZNB�>}�z����e��q�[G�����/Z�J��c�u����E��7N%J����~����;���[]K��z��'��Y�f��I� �x�JS�Z��]�V�Z���g����&��j��k�qa����]��;��c]0����F��������/�����s�q�Q�����J9w7�p��;��
�|z;vt���������r�'�� r��������[�,�\~n^{����p���
�Dr @&�e p���������?X.o{��N����<�Ns� �|��>`�����2N��Z�+iYE�A 8����v��Y3��3I��f+��3� 2���q�n�~��+�����r�[w��
1x 9� ��������v�w^�������!=�<`�-�����q�|�-�igX����1 �o�������+�]]5���C���@ p����}���{�Q�J����CZ����4�;J���g���r�� �~� 8b��<d�]���;��<cKw�v�_��xg#�n{������R���������l���xG 2r �#f���Z��M�Q���ly���e�����W�c�m����O����=���|�C��f1�f������o�pRn�S1��v���U��������W���-o���# ��
9 �iw�Q���?C�q2b�?��������;2����������M �ji��H�,[�� Hor d2U��d��������������;S.n���������M�= ��@ �L��PL���`���[��3U��=��B��-� ��F ��fl�o��|���}:q���g����{{)[��[�D��������F�@ �8r �������O/����m������u^�I������YL�" D�@ ���o�lq�hq�Z%lK�Y��1���}��g�n�$�(4���%����F=e6�e���f�'{7�ML���[l��� dr ������s����gq+_������-v^c��~��o���9������~
���m������M�������{��
��d���|5jy{ �� ��U%�������Ybq�ZxGI��{������������A����j� �� ����-~���Ax
�B�����z{�y?��&��~��l��� $ � dK��~��R�-�u����'o'��_� @�C ���c�x{)�����K'��x��� H@ 8�����I'���� �(r ��^����^:!� �r �#&�Y���w��I'1��| r9 � �z�K?�����-� �� ��R������m�l���6~�x������?w�\��{�wO �����sJq��4 ���d-B���U����?��#G��3�[B�������n�[o����+g�r��n �l[�f�d�w���}'�v��-j�%O-��+��%�1����������<�Z�M�����~IZ\���m��\�%O�\�U��}����yG�kue>+��9�����0���eo��3 G��%m�w�-W�<��t�'���&�S��w�"
�t�E�Y�n�l��������Y��-��w��i��Ow��"E�X�&M�]�vV�R%�9 �C �<9 p��;�����s�E��C,���#���b���M��}����Ma�~���k��o���]�\X7m�4��������8��u�M�<�^~�e+_��
4�=V��� Y��C����o�g��3�
��i=�������y����_����������[����m#��+^����]�^x�������{�q��]z�������� YS����^;x���O���y��V�ti�~V�lHY����#R'l �
�7�|��U�f�f�����������v���s[����W�^6�|������� @v����K�����d"�
�{����O{���]�iz�s����=�^ ��3g�5j�����$l W�`�TW��D����k dw����{�n�H^�C��Rm��;w�
6xg D*�@n���6i�$���/l����Y �������5�@-�O>���{��V�bE�����������w��5�
���4XA?z�!�����<��o����'v�g$�~ =n����y��m���
<��fr��Yc�:u�:u���5h��z��m[�l��q8�:t��v=���ns��{���8�:uj�������?������*�����]�v�I'�d%K��� d=�'O�-Z��a�\a�ZJ�5k���t�2�R�v�w�#�<b���K,H��1c��<`/��R�Z�M�6Y�6m\8����w��i��}{�z�O(?���������������kW{������q��X���W=���o��g�s-[�����������w��[n�%�}j�gx�}��/����iU W�H�Z������ d��z�����SO�o���U�)���I�8p�}��Gn�7��;���_��3g��7�Wz���k�����V�pa�98{�l�������!C��/�t�
��:?������w������?�|���Om�������.��2���\�r�{��V�JW����Z��U���'L���W���T5�����v�=r�H�=V��k�siI2]s�o���}F�w����[nr��;U���Y���y*��AT�\�|�\��%�\�f}����t ����.pm�*.��;�(P�n��&{��]���Ia�o������/v��i���'��������OU��-R
���s�9.\SHv�1����{�kU�P������z�W��\�R%���y����_�*�D�r{��q�2v�X�Z�_�E]�����T���WyS�L��H~�����^~�ew��{��Y�\9{��W\�-�.�@N}�:t�����������ov���B����6l�lO3 @zS����@�)j��m�^z���3'I ��O�������J�*����
�J�(��#�9T�4k�,r)X;����[��J��u���g�i[�n���G���p� �Q]��g����^:���:j�j��/��F��}U��������O-��j�r�Xz�@��T���*���~*o����� �"X
�B)Z�hb��~y����n�g�y�����rU��mS�k�>��
0��wdv�k��BN�LA�������`�B�p����SEu�(��g��o[U�=���l��v�y�yGH��9�s�D�F��1z, @fPG_�B����T��c
����zoC�uk�)�S���L��v�c�����p��'���
��s�����Z+N�UCmW_}�m����q
��o���xj�
EA�_%���*���?�[�O���lz� ���k����*xk���+0:����np�4Da��nm} ��
� ����B��~U��EU1��?��M;�Zo�F������k�[�v�4�5�j5Q�w�����u����I�O����������i����Ij��hx�_Q��KS �/A�G^}�Uk���=���n���15�W�2 @f���%K�xGI����.\��5�Ttn���n_���F<�@A��\F�`��v�)�����'i�
������:{���\��Zj'-S�L������>�p��B�P�`�Tr111���z��V�^=W�� N�K�M��5�d��I�� 2��#�T7a�7�R�E���[���#����z\r�������X��R��?v������r����=��~mjh`���J��]����S��k��'��+W�.��Bw�_��a�����p*��.�oX_�[o�ew�q�Kv/�����{�����������sS: 2��O5hRSGB)P��n����o��6W& ��n�Mu��Y.�������Z�l��CQ���M�[pV�Z5�����D�v�����S�s��i����S'?~������j���5h��]�7U�������Z����U�:� ���G���gOw?�_�����]��G��"m���K��G� �3�~��7{��7��[������Z�jeS�Lq�T ����s���_���P��=���nR���?�ZD��T�%��iS7UUb�_����Z���+��aC�������9$�4 �J�*����Sn��79��c��'�pA���3��[nq���R�W]�5j�����'�7��J�r��z�q�����^�^Ok���R*�{���]P��[�vmw�Z��s�����*����k�9�MT�������{�/��/�/4\e�"E\��F���M%� �E�l
4�U�w�qn���.U���)���>��k��Eb���(u_}������T�����m��
�>��K]�
�Dm��7ov����W�^����T�&z��]����~����A������w�����������i��
(�����4i�����[�/�.�w�u~U!R/W��#��
w�}��0d���E 5}��;�t�:���*�T���6� �Z7��%�-[��`�{V%��,n�@�[��;J^��X�J�zG jk���l�m#�=���fS�{)���Y�����]�������U(R�z���%�;����-�E~[]��J���8;8s��t-a��Q�
��g �N;����s��2N��X��:{Gi����GqC(��3 ���BN��*i��?��R���h� G��9�S�,Q��]�����z� ���C��������Qr�7>���l��y������nSi������� {(p���^�������Qr���)#Z����_�:��4�A�&j���xh
I�� ��hp�y�Y�{F���*�cE��l�k]��"�3D5�At��C��i$�v�r����E�;�<[�b��Z��MZ�H\M� � c�C�� ����k
�'�,\���n��6M1QW�bE7*���o'� <Qr�����/��S����sm�����[7�����~��7�?�5i��MZ � MiY��y�j��v����3�<����[[�����
�U �m�6{�������>
�n��F��e�w ���
����l��5n�~Jbbbl�����~� ��%�:t�f���5h��q��ow�����������3\k+k� )r�s��P�^=�]��.`��}�|��C��k�.�]��-Z�{V �J�lM�[�-����r![�%��}�Kn���k�o������Z�<y�g r��R�J�{��g?���
>�j���6��\r��1c�U�VV�P!�� ��-�������&�j�>