Weird behaviour of C extension function

On Fri, 2020-04-24 at 14:53 +0200, Amaury Bouchard wrote:

I have a really strange behaviour with a C function, wich gets a text as parameter.
Everything works fine when I call the function directly, giving a text string as parameter. But a problem occurs when I try to read data from a table.

To illustrate the problem, I stripped the function down to the minimum. The source code is below, but first, here is the behaviour :

Direct call
-----------

select passthru('hello world!'), passthru('utf8 çhàràtérs'), passthru(' h3110 123 456 ');

INFO: INPUT STRING: 'hello world!' (12)
INFO: INPUT STRING: 'utf8 çhàràtérs' (18)
INFO: INPUT STRING: ' h3110 123 456 ' (15)

(as you can see, the log messages show the correct input, with the number of bytes between parentheses)

Reading a table data
--------------------

create table mytable ( str text);
insert into mytable (str) values ('hello world!'), ('utf8 çhàràtérs'), (' h3110 123 456 ');
select passthru(str) from mytable;

INFO: INPUT STRING: 'lo world!' (12)
INFO: INPUT STRING: '8 çhàràtérs' (18)
INFO: INPUT STRING: '110 123 456 �
' (15)
INFO: INPUT STRING: '��' (5)
INFO: INPUT STRING: '' (3)

There, you can see that the pointer seems to be shifted 3 bytes farther.

Do you have any clue for this strange behaviour?

The source code
---------------

#include "postgres.h"
#include "fmgr.h"
#include "funcapi.h"

// PG module init
#ifdef PG_MODULE_MAGIC
PG_MODULE_MAGIC;
#endif
void _PG_init(void);
Datum passthru(PG_FUNCTION_ARGS);
PG_FUNCTION_INFO_V1(passthru);

void _PG_init() {
}

Datum passthru(PG_FUNCTION_ARGS) {
// get the input string
text *input = PG_GETARG_TEXT_PP(0);
char *input_pt = (char*)VARDATA(input);
int32 input_len = VARSIZE_ANY_EXHDR(input);
// create a null terminated copy of the input string
char *str_copy = calloc(1, input_len + 1);
memcpy(str_copy, input_pt, input_len);
// log message
elog(INFO, "INPUT STRING: '%s' (%d)", str_copy, input_len);
free(str_copy);
PG_RETURN_NULL();
}

You find this in "postgres.h":

* In consumers oblivious to data alignment, call PG_DETOAST_DATUM_PACKED(),
* VARDATA_ANY(), VARSIZE_ANY() and VARSIZE_ANY_EXHDR(). Elsewhere, call
* PG_DETOAST_DATUM(), VARDATA() and VARSIZE(). Directly fetching an int16,
* int32 or wider field in the struct representing the datum layout requires
* aligned data. memcpy() is alignment-oblivious, as are most operations on
* datatypes, such as text, whose layout struct contains only char fields.

So you should use VARDATA_ANY.

What happens is that these short text columns have a 1-byte TOAST header,
but you ship the first 4 bytes unconditionally, assuming they were detoasted.

Yours,
Laurenz Albe
--
Cybertec | https://www.cybertec-postgresql.com