Error creating gin index on jsonb columns
We are getting an error on the following statement:
CREATE INDEX CONCURRENTLY customer_jsonb_fields_idx ON customer USING gin
(jsonb_fields jsonb_path_ops);
ERROR: invalid memory alloc request size 2013265920
Anyone know what is causing it? It does not seem to be data corruption as
when we deploy to multiple databases we got exactly the same error.
I did try to set maintenance_work_mam to 4GB and still the same error
occurred.
Thank you.
-glenn
--
Glenn Zhu
SaaS Operations | ❖ Medallia, Inc.
Glenn Zhu <gzhu@medallia.com> writes:
We are getting an error on the following statement:
CREATE INDEX CONCURRENTLY customer_jsonb_fields_idx ON customer USING gin
(jsonb_fields jsonb_path_ops);
ERROR: invalid memory alloc request size 2013265920
Anyone know what is causing it?
Sounds like a bug from here. What PG version is this exactly? If it's
not the latest minor releases, try updating. If it's still there, see if
you can extract a test case that you can share.
regards, tom lane
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
Hi Tom,
Thanks for the reply.
We are currently running 9.4.4. 9.4.5 notes have two references to gin
index but does not seem to address the issue.
We are however able to create same index on some other databases. So it
maybe data related or size of table related? So far, the failed cases
reported are from large tables, with number of rows between 150 to 350
millions.
What would you need in terms of test cases?
Thank you very much.
-glenn
On Fri, Oct 16, 2015 at 6:28 AM, Tom Lane <tgl@sss.pgh.pa.us> wrote:
Glenn Zhu <gzhu@medallia.com> writes:
We are getting an error on the following statement:
CREATE INDEX CONCURRENTLY customer_jsonb_fields_idx ON customer USING gin
(jsonb_fields jsonb_path_ops);ERROR: invalid memory alloc request size 2013265920
Anyone know what is causing it?
Sounds like a bug from here. What PG version is this exactly? If it's
not the latest minor releases, try updating. If it's still there, see if
you can extract a test case that you can share.regards, tom lane
--
Glenn Zhu
SaaS Operations | ❖ Medallia, Inc.
Glenn Zhu <gzhu@medallia.com> writes:
We are currently running 9.4.4. 9.4.5 notes have two references to gin
index but does not seem to address the issue.
We are however able to create same index on some other databases. So it
maybe data related or size of table related?
I'd guess that it's triggered by a specific data item or set of data
items. Doubt it has anything to do with table size per se. The quoted
value is 0x78000000, which makes me think that something is
misinterpreting a plain C string as a varlena value (with a length word),
or something along that line.
regards, tom lane
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
Currently, after hitting the error, the indexes were still created but
marked with status of "invalid"
Looks like we shall see inserts to fail with the index on the column,
regardless of the index status ("valid" or "invalid"), if we start to
receive the "bad" values? Maybe I shall drop all these indexes.
If it's a bug, what information would be needed to trigger a bug fix? Is
there a formal channel?
Thanks.
-glenn
On Fri, Oct 16, 2015 at 6:43 AM, Tom Lane <tgl@sss.pgh.pa.us> wrote:
Glenn Zhu <gzhu@medallia.com> writes:
We are currently running 9.4.4. 9.4.5 notes have two references to gin
index but does not seem to address the issue.We are however able to create same index on some other databases. So it
maybe data related or size of table related?I'd guess that it's triggered by a specific data item or set of data
items. Doubt it has anything to do with table size per se. The quoted
value is 0x78000000, which makes me think that something is
misinterpreting a plain C string as a varlena value (with a length word),
or something along that line.regards, tom lane
--
Glenn Zhu
SaaS Operations | ❖ Medallia, Inc.
Glenn Zhu <gzhu@medallia.com> writes:
Currently, after hitting the error, the indexes were still created but
marked with status of "invalid"
That's just what CREATE INDEX CONCURRENTLY would do with any error.
(It might be worth checking whether a non-CONCURRENTLY build hits the
same error, though I'm betting it will.)
If it's a bug, what information would be needed to trigger a bug fix?
A reproducible test case would move things along quite a bit; without that
we're just guessing.
Is there a formal channel?
Well, you could move the thread to pgsql-bugs but you might as well
keep it where it is.
regards, tom lane
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
We can't test a non-concurrent index build in production - but your word is
just as good.
We only see this on some of production databases. We did not see it in QA
testing. But we will try to get a test case in QA.
Is this categorized as a bug specific to GIN indexes or a PostgreSQL bug in
general?
-glenn
On Fri, Oct 16, 2015 at 7:02 AM, Tom Lane <tgl@sss.pgh.pa.us> wrote:
Glenn Zhu <gzhu@medallia.com> writes:
Currently, after hitting the error, the indexes were still created but
marked with status of "invalid"That's just what CREATE INDEX CONCURRENTLY would do with any error.
(It might be worth checking whether a non-CONCURRENTLY build hits the
same error, though I'm betting it will.)If it's a bug, what information would be needed to trigger a bug fix?
A reproducible test case would move things along quite a bit; without that
we're just guessing.Is there a formal channel?
Well, you could move the thread to pgsql-bugs but you might as well
keep it where it is.regards, tom lane
--
Glenn Zhu
SaaS Operations | ❖ Medallia, Inc.
Glenn Zhu <gzhu@medallia.com> writes:
Is this categorized as a bug specific to GIN indexes or a PostgreSQL bug in
general?
My gut says it's GIN-specific, but that's really only an educated guess;
we have too little info.
What I would recommend is that you get the data onto a non-production
machine where you can play around a bit more. One thing you could do
then is run a build with debug symbols, attach to the backend process
with gdb, and set a breakpoint at "errfinish". Then provoke the error,
and capture a backtrace from the call to errfinish. That would greatly
narrow things down, though it might not be enough to isolate the bug
immediately. (If I had a test case in hand, that's exactly the first
step I would take with it, but maybe you can do it for me.)
regards, tom lane
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
We will follow your instructions and get back to you.
Thank you Tom. Much appreciated!
-glenn
On Fri, Oct 16, 2015 at 7:16 AM, Tom Lane <tgl@sss.pgh.pa.us> wrote:
Glenn Zhu <gzhu@medallia.com> writes:
Is this categorized as a bug specific to GIN indexes or a PostgreSQL bug
in
general?
My gut says it's GIN-specific, but that's really only an educated guess;
we have too little info.What I would recommend is that you get the data onto a non-production
machine where you can play around a bit more. One thing you could do
then is run a build with debug symbols, attach to the backend process
with gdb, and set a breakpoint at "errfinish". Then provoke the error,
and capture a backtrace from the call to errfinish. That would greatly
narrow things down, though it might not be enough to isolate the bug
immediately. (If I had a test case in hand, that's exactly the first
step I would take with it, but maybe you can do it for me.)regards, tom lane
--
Glenn Zhu
SaaS Operations | ❖ Medallia, Inc.