BUG #19600: pgcrypto crypt() can return text with invalid encoding
The following bug has been logged on the website:
Bug reference: 19600
Logged by: Michael Malis
Email address: malis@pgrust.com
PostgreSQL version: 19beta1
Operating system: Debian
Description:
pgcrypto's crypt() copies bytes from the caller-supplied setting/salt
verbatim into its text result without regard to character boundaries. When a
multibyte character straddles the fixed byte count that gets copied, the
result is a text value containing a truncated UTF-8 sequence.
That value is invalid in the database encoding. The server rejects the exact
same bytes when they are offered as input, yet it accepts them from crypt()
and lets them into a table, after which upper() fails on the stored value
and
the row cannot be restored from a dump.
This is the same class that was already accepted and fixed for pgcrypto's
other text-returning functions. The thread "Encoding protection for
pgcrypto"
added pg_verifymbstr() to pgp_sym_decrypt_text and pgp_pub_decrypt_text on
the
stated principle that "by default the PostgreSQL TEXT type should have the
same encoding as the database encoding". crypt() is the same class and was
NOT covered. Unlike those functions it has no bytea counterpart, so the
escape
hatch offered in that thread ("use pgp_*_decrypt_bytea") does not exist
here.
Reproducer
----------
Needs pgcrypto and a UTF8 database. Nothing here requires superuser; crypt()
is EXECUTE-able by PUBLIC.
CREATE EXTENSION IF NOT EXISTS pgcrypto;
-- the input salt is VALID UTF-8: U+20AC then 'A' -> e2 82 ac 41
SELECT encode(E'€A'::bytea, 'hex');
-- crypt() returns text whose bytes are NOT valid UTF-8
SELECT encode(crypt('password', E'€A')::bytea, 'hex');
-- e282555a6f49796a2f48792f63 <-- begins e2 82 55
-- the server rejects those same bytes on input
SELECT convert_from(decode('e282', 'hex'), 'UTF8');
-- ERROR: invalid byte sequence for encoding "UTF8": 0xe2 0x82
-- yet the value stores fine ...
CREATE TEMP TABLE t (h text);
INSERT INTO t SELECT crypt('password', E'€A');
SELECT octet_length(h) FROM t; -- 13
-- ... and then breaks a core text function
SELECT upper(h) FROM t;
-- ERROR: invalid byte sequence for encoding "UTF8": 0xe2 0x82 0x55
-- and the row does not survive dump/restore
\copy t TO '/tmp/crypt_invalid.txt'
CREATE TEMP TABLE t2 (h text);
\copy t2 FROM '/tmp/crypt_invalid.txt'
-- ERROR: invalid byte sequence for encoding "UTF8": 0xe2 0x82 0x55
-- CONTEXT: COPY t2, line 1
Two controls, both of which pass, so this is the geometry and not an
artifact of the test:
-- the documented idiom cannot trigger it: gen_salt() emits ASCII only
SELECT convert_from(crypt('password', gen_salt('des'))::bytea, 'UTF8')
IS NOT NULL; -- t
-- a multibyte character PAST the 2-byte cut is fine
SELECT convert_from(crypt('password', E'AB€')::bytea, 'UTF8')
IS NOT NULL; -- t
Expected vs. actual
-------------------
- Expected: crypt() either rejects a setting it cannot represent in the
database encoding, or returns a value that is valid in that encoding —
the same contract pgp_sym_decrypt_text and pgp_pub_decrypt_text now
honour.
- Actual: crypt() returns text containing a truncated multibyte sequence.
The
value is storable, breaks upper(), and blocks restore.
Mechanism, with file:line
-------------------------
contrib/pgcrypto/pgcrypto.c, pg_crypt(): the result is built with
cres = px_crypt(buf0, buf1, resbuf, PX_MAX_CRYPT);
...
res = cstring_to_text(cres);
PG_RETURN_TEXT_P(res);
and there is no encoding validation on that path. The string "verifymbstr"
does not appear anywhere in pgcrypto.c, on 18.3 or on master.
contrib/pgcrypto/crypt-des.c, the traditional-DES branch, copies a FIXED
two bytes of the setting into the output:
output[0] = setting[0];
output[1] = setting[1] ? setting[1] : output[0];
with no character-boundary awareness. Identical on master.
So the rule is deterministic rather than intermittent: for traditional DES
the output is invalid exactly when a multibyte character straddles byte
offset 2, i.e. begins at byte 0 or byte 1 of the setting. The same mechanism
exists at different cut points elsewhere — xdes copies 9 bytes
(strlcpy(output, setting, 10)), and crypt-md5.c re-emits the raw salt run
capped at 8 bytes.
For contrast, contrib/pgcrypto/pgp-pgsql.c DOES validate, at two sites:
pg_verifymbstr(VARDATA_ANY(res), VARSIZE_ANY_EXHDR(res), false);
Observed behaviour
------------------
Verified live on both versions, in a UTF8 database:
crypt('password', E'€A') -> e282555a6f49796a2f48792f63
convert_from(decode('e282','hex'),'UTF8') -> ERROR, 0xe2 0x82
INSERT into a text column -> succeeds, octet_length 13
upper(stored) -> ERROR, 0xe2 0x82 0x55
COPY out then COPY in -> ERROR, 0xe2 0x82 0x55, COPY t2 line 1
Hi, Michael!
Big thanks for the steady flow of bug reports. Thanks for caring about
every detail. Really impressive productivity, keep it up!
The attached patch adds pg_verifymbstr() in pg_crypt(). A regress case
based on the report is included. Non-UTF8 buildfarm animals skip that
file.
пн, 3 авг. 2026 г. в 16:51, PG Bug reporting form <noreply@postgresql.org>:
The following bug has been logged on the website:
Bug reference: 19600
Logged by: Michael Malis
Email address: malis@pgrust.com
PostgreSQL version: 19beta1
Operating system: Debian
Description:pgcrypto's crypt() copies bytes from the caller-supplied setting/salt
verbatim into its text result without regard to character boundaries. When
a
multibyte character straddles the fixed byte count that gets copied, the
result is a text value containing a truncated UTF-8 sequence.That value is invalid in the database encoding. The server rejects the
exact
same bytes when they are offered as input, yet it accepts them from crypt()
and lets them into a table, after which upper() fails on the stored value
and
the row cannot be restored from a dump.This is the same class that was already accepted and fixed for pgcrypto's
other text-returning functions. The thread "Encoding protection for
pgcrypto"
added pg_verifymbstr() to pgp_sym_decrypt_text and pgp_pub_decrypt_text on
the
stated principle that "by default the PostgreSQL TEXT type should have the
same encoding as the database encoding". crypt() is the same class and was
NOT covered. Unlike those functions it has no bytea counterpart, so the
escape
hatch offered in that thread ("use pgp_*_decrypt_bytea") does not exist
here.Reproducer
----------
Needs pgcrypto and a UTF8 database. Nothing here requires superuser;
crypt()
is EXECUTE-able by PUBLIC.CREATE EXTENSION IF NOT EXISTS pgcrypto;
-- the input salt is VALID UTF-8: U+20AC then 'A' -> e2 82 ac 41
SELECT encode(E'€A'::bytea, 'hex');-- crypt() returns text whose bytes are NOT valid UTF-8
SELECT encode(crypt('password', E'€A')::bytea, 'hex');
-- e282555a6f49796a2f48792f63 <-- begins e2 82 55-- the server rejects those same bytes on input
SELECT convert_from(decode('e282', 'hex'), 'UTF8');
-- ERROR: invalid byte sequence for encoding "UTF8": 0xe2 0x82-- yet the value stores fine ...
CREATE TEMP TABLE t (h text);
INSERT INTO t SELECT crypt('password', E'€A');
SELECT octet_length(h) FROM t; -- 13-- ... and then breaks a core text function
SELECT upper(h) FROM t;
-- ERROR: invalid byte sequence for encoding "UTF8": 0xe2 0x82 0x55-- and the row does not survive dump/restore
\copy t TO '/tmp/crypt_invalid.txt'
CREATE TEMP TABLE t2 (h text);
\copy t2 FROM '/tmp/crypt_invalid.txt'
-- ERROR: invalid byte sequence for encoding "UTF8": 0xe2 0x82 0x55
-- CONTEXT: COPY t2, line 1Two controls, both of which pass, so this is the geometry and not an
artifact of the test:-- the documented idiom cannot trigger it: gen_salt() emits ASCII only
SELECT convert_from(crypt('password', gen_salt('des'))::bytea, 'UTF8')
IS NOT NULL; -- t-- a multibyte character PAST the 2-byte cut is fine
SELECT convert_from(crypt('password', E'AB€')::bytea, 'UTF8')
IS NOT NULL; -- tExpected vs. actual
-------------------
- Expected: crypt() either rejects a setting it cannot represent in the
database encoding, or returns a value that is valid in that encoding —
the same contract pgp_sym_decrypt_text and pgp_pub_decrypt_text now
honour.
- Actual: crypt() returns text containing a truncated multibyte sequence.
The
value is storable, breaks upper(), and blocks restore.Mechanism, with file:line
-------------------------
contrib/pgcrypto/pgcrypto.c, pg_crypt(): the result is built withcres = px_crypt(buf0, buf1, resbuf, PX_MAX_CRYPT);
...
res = cstring_to_text(cres);
PG_RETURN_TEXT_P(res);and there is no encoding validation on that path. The string "verifymbstr"
does not appear anywhere in pgcrypto.c, on 18.3 or on master.contrib/pgcrypto/crypt-des.c, the traditional-DES branch, copies a FIXED
two bytes of the setting into the output:output[0] = setting[0];
output[1] = setting[1] ? setting[1] : output[0];with no character-boundary awareness. Identical on master.
So the rule is deterministic rather than intermittent: for traditional DES
the output is invalid exactly when a multibyte character straddles byte
offset 2, i.e. begins at byte 0 or byte 1 of the setting. The same
mechanism
exists at different cut points elsewhere — xdes copies 9 bytes
(strlcpy(output, setting, 10)), and crypt-md5.c re-emits the raw salt run
capped at 8 bytes.For contrast, contrib/pgcrypto/pgp-pgsql.c DOES validate, at two sites:
pg_verifymbstr(VARDATA_ANY(res), VARSIZE_ANY_EXHDR(res), false);
Observed behaviour
------------------
Verified live on both versions, in a UTF8 database:
crypt('password', E'€A') -> e282555a6f49796a2f48792f63
convert_from(decode('e282','hex'),'UTF8') -> ERROR, 0xe2 0x82
INSERT into a text column -> succeeds, octet_length 13
upper(stored) -> ERROR, 0xe2 0x82 0x55
COPY out then COPY in -> ERROR, 0xe2 0x82 0x55, COPY t2 line 1
--
Regards,
Rachitskiy Andrey