Memory leak in RelationBuildRowSecurity
Hi,
One of our customers faced with backend memory bloat (up to 70Gb).
This customer is actively usnig row based security and temporary tables.
Looks like this combination cause memory bloat.
There is memory leak in RelationBuildRowSecurity function - it converts
node to expression tree in current memory context:
with_check_qual = (Expr *) stringToNode(with_check_value);
qual_expr = (Expr *) stringToNode(qual_value);
then this trees are copied to "row security descriptor" memory context:
policy->with_check_qual = copyObject(with_check_qual);
and finally we do cleanup:
/* clean up some (not all) of the junk ... */
if (qual_expr != NULL)
pfree(qual_expr);
if (with_check_qual != NULL)
pfree(with_check_qual);
The problem is that pfree deletes only top element, but not the whole tree.
As far as we do not have deleteObject function, the only solution I see
is to create this tree in separate memory context.
But RelationBuildRowSecurity has created its own context.
I wonder if it is enough just to switch to this context earlier?
(proposed patch is attached).
Now a little bit more about the role temporary tables in this problem.
This the stack trace I observed in debugger:
(gdb) bt
#0 AllocSetAlloc (context=0x5634b7389fb8, size=<optimized out>) at
/home/builder/postgrespro-9.5.7.1/build/../src/backend/utils/mmgr/aset.c:891
#1 0x00005634b6c348c1 in MemoryContextAllocZero (context=<optimized
out>, size=size@entry=28) at
/home/builder/postgrespro-9.5.7.1/build/../src/backend/utils/mmgr/mcxt.c:737
#2 0x00005634b6a66ddd in _readParam () at
/home/builder/postgrespro-9.5.7.1/build/../src/backend/nodes/readfuncs.c:492
#3 0x00005634b6a68f75 in parseNodeString () at
/home/builder/postgrespro-9.5.7.1/build/../src/backend/nodes/readfuncs.c:1417
#4 0x00005634b6a6bc65 in nodeRead (
token=0x5634d39034bb "{PARAM :paramkind 2 :paramid 1 :paramtype
1700 :paramtypmod 1048580 :paramcollid 0 :location -1}) :location 112}
:operName (\"=\") :subselect {QUERY :commandType 1 :querySource 0
:canSetTag true :utilit"..., tok_len=1) at
/home/builder/postgrespro-9.5.7.1/build/../src/backend/nodes/read.c:296
#5 0x00005634b6a6bc17 in nodeRead (token=<optimized out>,
token@entry=0x0, tok_len=1, tok_len@entry=0) at
/home/builder/postgrespro-9.5.7.1/build/../src/backend/nodes/read.c:362
#6 0x00005634b6a67119 in _readOpExpr () at
/home/builder/postgrespro-9.5.7.1/build/../src/backend/nodes/readfuncs.c:651
#7 0x00005634b6a6a4e5 in parseNodeString () at
/home/builder/postgrespro-9.5.7.1/build/../src/backend/nodes/readfuncs.c:1431
#8 0x00005634b6a6bc65 in nodeRead (
token=0x5634d39033d5 "{OPEXPR :opno 1752 :opfuncid 1718
:opresulttype 16 :opretset false :opcollid 0 :inputcollid 0 :args ({VAR
:varno 1 :varattno 14 :vartype 1700 :vartypmod 1048580 :varcollid 0
:varlevelsup 0 :varnoold 1"..., token@entry=0x0, tok_len=1,
tok_len@entry=0) at
/home/builder/postgrespro-9.5.7.1/build/../src/backend/nodes/read.c:296
#9 0x00005634b6a6aa17 in _readSubLink () at
/home/builder/postgrespro-9.5.7.1/build/../src/backend/nodes/readfuncs.c:784
#10 parseNodeString () at
/home/builder/postgrespro-9.5.7.1/build/../src/backend/nodes/readfuncs.c:1441
#11 0x00005634b6a6bc65 in nodeRead (
token=0x5634d39033a6 "{SUBLINK :subLinkType 2 :subLinkId 0
:testexpr {OPEXPR :opno 1752 :opfuncid 1718 :opresulttype 16 :opretset
false :opcollid 0 :inputcollid 0 :args ({VAR :varno 1 :varattno 14
:vartype 1700 :vartypmod "..., tok_len=1) at
/home/builder/postgrespro-9.5.7.1/build/../src/backend/nodes/read.c:296
#12 0x00005634b6a6bc17 in nodeRead (token=<optimized out>,
token@entry=0x0, tok_len=1, tok_len@entry=0) at
/home/builder/postgrespro-9.5.7.1/build/../src/backend/nodes/read.c:362
#13 0x00005634b6a69b46 in _readBoolExpr () at
/home/builder/postgrespro-9.5.7.1/build/../src/backend/nodes/readfuncs.c:768
#14 parseNodeString () at
/home/builder/postgrespro-9.5.7.1/build/../src/backend/nodes/readfuncs.c:1439
#15 0x00005634b6a6bc65 in nodeRead (
token=0x5634d3903389 "{BOOLEXPR :boolop and :args ({SUBLINK
:subLinkType 2 :subLinkId 0 :testexpr {OPEXPR :opno 1752 :opfuncid 1718
:opresulttype 16 :opretset false :opcollid 0 :inputcollid 0 :args ({VAR
:varno 1 :varattno"..., tok_len=1) at
/home/builder/postgrespro-9.5.7.1/build/../src/backend/nodes/read.c:296
#16 0x00005634b6a6bc17 in nodeRead (token=<optimized out>,
token@entry=0x0, tok_len=1, tok_len@entry=0) at
/home/builder/postgrespro-9.5.7.1/build/../src/backend/nodes/read.c:362
#17 0x00005634b6a69b46 in _readBoolExpr () at
/home/builder/postgrespro-9.5.7.1/build/../src/backend/nodes/readfuncs.c:768
#18 parseNodeString () at
/home/builder/postgrespro-9.5.7.1/build/../src/backend/nodes/readfuncs.c:1439
#19 0x00005634b6a6bc65 in nodeRead (
token=0x5634d39032d8 "{BOOLEXPR :boolop and :args ({CONST
:consttype 16 :consttypmod -1 :constcollid 0 :constlen 1 :constbyval
true :constisnull false :location 89 :constvalue 1 [ 1 0 0 0 0 0 0 0 ]}
{BOOLEXPR :boolop and :"..., token@entry=0x0, tok_len=1,
tok_len@entry=0) at
/home/builder/postgrespro-9.5.7.1/build/../src/backend/nodes/read.c:296
#20 0x00005634b6a6bf00 in stringToNode (str=<optimized out>) at
/home/builder/postgrespro-9.5.7.1/build/../src/backend/nodes/read.c:53
#21 0x00005634b69d18dd in RelationBuildRowSecurity
(relation=relation@entry=0x7f2b5dcee8c8) at
/home/builder/postgrespro-9.5.7.1/build/../src/backend/commands/policy.c:281
#22 0x00005634b6c03e56 in RelationBuildDesc
(targetRelId=targetRelId@entry=158077, insertIt=insertIt@entry=0 '\000')
at
/home/builder/postgrespro-9.5.7.1/build/../src/backend/utils/cache/relcache.c:1052
#23 0x00005634b6c042cf in RelationClearRelation
(relation=0x7f2b5dce3f28, rebuild=rebuild@entry=1 '\001') at
/home/builder/postgrespro-9.5.7.1/build/../src/backend/utils/cache/relcache.c:2200
#24 0x00005634b6c04f95 in RelationCacheInvalidate () at
/home/builder/postgrespro-9.5.7.1/build/../src/backend/utils/cache/relcache.c:2500
#25 0x00005634b6bfdecf in InvalidateSystemCaches () at
/home/builder/postgrespro-9.5.7.1/build/../src/backend/utils/cache/inval.c:628
#26 0x00005634b6afca8f in ReceiveSharedInvalidMessages
(invalFunction=invalFunction@entry=0x5634b6bfe6d0
<LocalExecuteInvalidationMessage>,
resetFunction=resetFunction@entry=0x5634b6bfdeb0
<InvalidateSystemCaches>) at
/home/builder/postgrespro-9.5.7.1/build/../src/backend/storage/ipc/sinval.c:108
#27 0x00005634b6bfe018 in AcceptInvalidationMessages () at
/home/builder/postgrespro-9.5.7.1/build/../src/backend/utils/cache/inval.c:662
#28 0x00005634b6affb79 in LockRelationOid (relid=2601, lockmode=1) at
/home/builder/postgrespro-9.5.7.1/build/../src/backend/storage/lmgr/lmgr.c:125
#29 0x00005634b68d23b5 in relation_open (relationId=2601,
lockmode=<optimized out>) at
/home/builder/postgrespro-9.5.7.1/build/../src/backend/access/heap/heapam.c:1068
#30 0x00005634b68d264e in heap_open (relationId=<optimized out>,
lockmode=lockmode@entry=1) at
/home/builder/postgrespro-9.5.7.1/build/../src/backend/access/heap/heapam.c:1246
#31 0x00005634b6bfc957 in SearchCatCache (cache=0x5634b72fb138,
v1=<optimized out>, v2=v2@entry=0, v3=v3@entry=0, v4=v4@entry=0)
at
/home/builder/postgrespro-9.5.7.1/build/../src/backend/utils/cache/catcache.c:1239
#32 0x00005634b6c0964e in SearchSysCache (cacheId=cacheId@entry=2,
key1=<optimized out>, key2=key2@entry=0, key3=key3@entry=0,
key4=key4@entry=0)
at
/home/builder/postgrespro-9.5.7.1/build/../src/backend/utils/cache/syscache.c:981
#33 0x00005634b6c02940 in RelationInitIndexAccessInfo
(relation=relation@entry=0x7f2946a9dfa8) at
/home/builder/postgrespro-9.5.7.1/build/../src/backend/utils/cache/relcache.c:1211
---Type <return> to continue, or q <return> to quit---
#34 0x00005634b6c03e6d in RelationBuildDesc (targetRelId=<optimized
out>, insertIt=insertIt@entry=1 '\001') at
/home/builder/postgrespro-9.5.7.1/build/../src/backend/utils/cache/relcache.c:1060
#35 0x00005634b6c04d05 in RelationIdGetRelation
(relationId=relationId@entry=2656) at
/home/builder/postgrespro-9.5.7.1/build/../src/backend/utils/cache/relcache.c:1782
#36 0x00005634b68d236f in relation_open
(relationId=relationId@entry=2656, lockmode=lockmode@entry=1) at
/home/builder/postgrespro-9.5.7.1/build/../src/backend/access/heap/heapam.c:1071
#37 0x00005634b68e3c6e in index_open (relationId=relationId@entry=2656,
lockmode=lockmode@entry=1) at
/home/builder/postgrespro-9.5.7.1/build/../src/backend/access/index/indexam.c:167
#38 0x00005634b68e36e3 in systable_beginscan
(heapRelation=heapRelation@entry=0x7f2946a32ce8,
indexId=indexId@entry=2656, indexOK=indexOK@entry=1 '\001',
snapshot=snapshot@entry=0x0,
nkeys=nkeys@entry=1, key=key@entry=0x7ffed5111320) at
/home/builder/postgrespro-9.5.7.1/build/../src/backend/access/index/genam.c:335
#39 0x00005634b6c03736 in AttrDefaultFetch (relation=0x7f2b5dc97258) at
/home/builder/postgrespro-9.5.7.1/build/../src/backend/utils/cache/relcache.c:3655
#40 RelationBuildTupleDesc (relation=0x7f2b5dc97258) at
/home/builder/postgrespro-9.5.7.1/build/../src/backend/utils/cache/relcache.c:607
#41 RelationBuildDesc (targetRelId=targetRelId@entry=158077,
insertIt=insertIt@entry=0 '\000') at
/home/builder/postgrespro-9.5.7.1/build/../src/backend/utils/cache/relcache.c:1033
#42 0x00005634b6c042cf in RelationClearRelation
(relation=0x7f2b5dce3f28, rebuild=rebuild@entry=1 '\001') at
/home/builder/postgrespro-9.5.7.1/build/../src/backend/utils/cache/relcache.c:2200
#43 0x00005634b6c04f95 in RelationCacheInvalidate () at
/home/builder/postgrespro-9.5.7.1/build/../src/backend/utils/cache/relcache.c:2500
#44 0x00005634b6bfdecf in InvalidateSystemCaches () at
/home/builder/postgrespro-9.5.7.1/build/../src/backend/utils/cache/inval.c:628
#45 0x00005634b6afca8f in ReceiveSharedInvalidMessages
(invalFunction=invalFunction@entry=0x5634b6bfe6d0
<LocalExecuteInvalidationMessage>,
resetFunction=resetFunction@entry=0x5634b6bfdeb0
<InvalidateSystemCaches>) at
/home/builder/postgrespro-9.5.7.1/build/../src/backend/storage/ipc/sinval.c:108
#46 0x00005634b6bfe018 in AcceptInvalidationMessages () at
/home/builder/postgrespro-9.5.7.1/build/../src/backend/utils/cache/inval.c:662
#47 0x00005634b6affb79 in LockRelationOid (relid=2679, lockmode=1) at
/home/builder/postgrespro-9.5.7.1/build/../src/backend/storage/lmgr/lmgr.c:125
#48 0x00005634b68d23b5 in relation_open
(relationId=relationId@entry=2679, lockmode=lockmode@entry=1) at
/home/builder/postgrespro-9.5.7.1/build/../src/backend/access/heap/heapam.c:1068
#49 0x00005634b68e3c6e in index_open (relationId=relationId@entry=2679,
lockmode=lockmode@entry=1) at
/home/builder/postgrespro-9.5.7.1/build/../src/backend/access/index/indexam.c:167
#50 0x00005634b68e36e3 in systable_beginscan
(heapRelation=heapRelation@entry=0x7f2946aa0748, indexId=2679,
indexOK=<optimized out>, snapshot=snapshot@entry=0x0, nkeys=1,
key=key@entry=0x7ffed5111790) at
/home/builder/postgrespro-9.5.7.1/build/../src/backend/access/index/genam.c:335
#51 0x00005634b6bfc981 in SearchCatCache (cache=0x5634b7301ec8,
v1=<optimized out>, v2=v2@entry=0, v3=v3@entry=0, v4=v4@entry=0)
at
/home/builder/postgrespro-9.5.7.1/build/../src/backend/utils/cache/catcache.c:1241
#52 0x00005634b6c0964e in SearchSysCache (cacheId=cacheId@entry=32,
key1=<optimized out>, key2=key2@entry=0, key3=key3@entry=0,
key4=key4@entry=0)
at
/home/builder/postgrespro-9.5.7.1/build/../src/backend/utils/cache/syscache.c:981
#53 0x00005634b6c028d8 in RelationInitIndexAccessInfo
(relation=relation@entry=0x7f2946969f38) at
/home/builder/postgrespro-9.5.7.1/build/../src/backend/utils/cache/relcache.c:1197
#54 0x00005634b6c03e6d in RelationBuildDesc (targetRelId=<optimized
out>, insertIt=insertIt@entry=1 '\001') at
/home/builder/postgrespro-9.5.7.1/build/../src/backend/utils/cache/relcache.c:1060
#55 0x00005634b6c04d05 in RelationIdGetRelation
(relationId=relationId@entry=2656) at
/home/builder/postgrespro-9.5.7.1/build/../src/backend/utils/cache/relcache.c:1782
#56 0x00005634b68d236f in relation_open
(relationId=relationId@entry=2656, lockmode=lockmode@entry=1) at
/home/builder/postgrespro-9.5.7.1/build/../src/backend/access/heap/heapam.c:1071
#57 0x00005634b68e3c6e in index_open (relationId=relationId@entry=2656,
lockmode=lockmode@entry=1) at
/home/builder/postgrespro-9.5.7.1/build/../src/backend/access/index/indexam.c:167
#58 0x00005634b68e36e3 in systable_beginscan
(heapRelation=heapRelation@entry=0x7f2946a32ce8,
indexId=indexId@entry=2656, indexOK=indexOK@entry=1 '\001',
snapshot=snapshot@entry=0x0,
nkeys=nkeys@entry=1, key=key@entry=0x7ffed5111bf0) at
/home/builder/postgrespro-9.5.7.1/build/../src/backend/access/index/genam.c:335
#59 0x00005634b6c03736 in AttrDefaultFetch (relation=0x7f2b5dcedf78) at
/home/builder/postgrespro-9.5.7.1/build/../src/backend/utils/cache/relcache.c:3655
#60 RelationBuildTupleDesc (relation=0x7f2b5dcedf78) at
/home/builder/postgrespro-9.5.7.1/build/../src/backend/utils/cache/relcache.c:607
#61 RelationBuildDesc (targetRelId=targetRelId@entry=158077,
insertIt=insertIt@entry=0 '\000') at
/home/builder/postgrespro-9.5.7.1/build/../src/backend/utils/cache/relcache.c:1033
#62 0x00005634b6c042cf in RelationClearRelation
(relation=0x7f2b5dce3f28, rebuild=rebuild@entry=1 '\001') at
/home/builder/postgrespro-9.5.7.1/build/../src/backend/utils/cache/relcache.c:2200
#63 0x00005634b6c04f95 in RelationCacheInvalidate () at
/home/builder/postgrespro-9.5.7.1/build/../src/backend/utils/cache/relcache.c:2500
#64 0x00005634b6bfdecf in InvalidateSystemCaches () at
/home/builder/postgrespro-9.5.7.1/build/../src/backend/utils/cache/inval.c:628
#65 0x00005634b6afca8f in ReceiveSharedInvalidMessages
(invalFunction=invalFunction@entry=0x5634b6bfe6d0
<LocalExecuteInvalidationMessage>,
resetFunction=resetFunction@entry=0x5634b6bfdeb0
<InvalidateSystemCaches>) at
/home/builder/postgrespro-9.5.7.1/build/../src/backend/storage/ipc/sinval.c:108
#66 0x00005634b6bfe018 in AcceptInvalidationMessages () at
/home/builder/postgrespro-9.5.7.1/build/../src/backend/utils/cache/inval.c:662
#67 0x00005634b6affb79 in LockRelationOid (relid=2610, lockmode=1) at
/home/builder/postgrespro-9.5.7.1/build/../src/backend/storage/lmgr/lmgr.c:125
#68 0x00005634b68d23b5 in relation_open (relationId=2610,
lockmode=<optimized out>) at
/home/builder/postgrespro-9.5.7.1/build/../src/backend/access/heap/heapam.c:1068
#69 0x00005634b68d264e in heap_open (relationId=<optimized out>,
lockmode=lockmode@entry=1) at
/home/builder/postgrespro-9.5.7.1/build/../src/backend/access/heap/heapam.c:1246
#70 0x00005634b6bfc957 in SearchCatCache (cache=0x5634b7301ec8,
v1=<optimized out>, v2=v2@entry=0, v3=v3@entry=0, v4=v4@entry=0)
at
/home/builder/postgrespro-9.5.7.1/build/../src/backend/utils/cache/catcache.c:1239
#71 0x00005634b6c0964e in SearchSysCache (cacheId=cacheId@entry=32,
key1=<optimized out>, key2=key2@entry=0, key3=key3@entry=0,
key4=key4@entry=0)
at
/home/builder/postgrespro-9.5.7.1/build/../src/backend/utils/cache/syscache.c:981
#72 0x00005634b6c028d8 in RelationInitIndexAccessInfo
(relation=relation@entry=0x7f2946dcc698) at
/home/builder/postgrespro-9.5.7.1/build/../src/backend/utils/cache/relcache.c:1197
---Type <return> to continue, or q <return> to quit---
#73 0x00005634b6c03e6d in RelationBuildDesc (targetRelId=<optimized
out>, insertIt=insertIt@entry=1 '\001') at
/home/builder/postgrespro-9.5.7.1/build/../src/backend/utils/cache/relcache.c:1060
#74 0x00005634b6c04d05 in RelationIdGetRelation
(relationId=relationId@entry=2656) at
/home/builder/postgrespro-9.5.7.1/build/../src/backend/utils/cache/relcache.c:1782
#75 0x00005634b68d236f in relation_open
(relationId=relationId@entry=2656, lockmode=lockmode@entry=1) at
/home/builder/postgrespro-9.5.7.1/build/../src/backend/access/heap/heapam.c:1071
#76 0x00005634b68e3c6e in index_open (relationId=relationId@entry=2656,
lockmode=lockmode@entry=1) at
/home/builder/postgrespro-9.5.7.1/build/../src/backend/access/index/indexam.c:167
#77 0x00005634b68e36e3 in systable_beginscan
(heapRelation=heapRelation@entry=0x7f2946a32ce8,
indexId=indexId@entry=2656, indexOK=indexOK@entry=1 '\001',
snapshot=snapshot@entry=0x0,
nkeys=nkeys@entry=1, key=key@entry=0x7ffed5112470) at
/home/builder/postgrespro-9.5.7.1/build/../src/backend/access/index/genam.c:335
#78 0x00005634b6c03736 in AttrDefaultFetch (relation=0x7f2946f4d1f8) at
/home/builder/postgrespro-9.5.7.1/build/../src/backend/utils/cache/relcache.c:3655
#79 RelationBuildTupleDesc (relation=0x7f2946f4d1f8) at
/home/builder/postgrespro-9.5.7.1/build/../src/backend/utils/cache/relcache.c:607
#80 RelationBuildDesc (targetRelId=targetRelId@entry=158077,
insertIt=insertIt@entry=0 '\000') at
/home/builder/postgrespro-9.5.7.1/build/../src/backend/utils/cache/relcache.c:1033
#81 0x00005634b6c042cf in RelationClearRelation
(relation=0x7f2b5dce3f28, rebuild=rebuild@entry=1 '\001') at
/home/builder/postgrespro-9.5.7.1/build/../src/backend/utils/cache/relcache.c:2200
#82 0x00005634b6c04f95 in RelationCacheInvalidate () at
/home/builder/postgrespro-9.5.7.1/build/../src/backend/utils/cache/relcache.c:2500
#83 0x00005634b6bfdecf in InvalidateSystemCaches () at
/home/builder/postgrespro-9.5.7.1/build/../src/backend/utils/cache/inval.c:628
#84 0x00005634b6afca8f in ReceiveSharedInvalidMessages
(invalFunction=invalFunction@entry=0x5634b6bfe6d0
<LocalExecuteInvalidationMessage>,
resetFunction=resetFunction@entry=0x5634b6bfdeb0
<InvalidateSystemCaches>) at
/home/builder/postgrespro-9.5.7.1/build/../src/backend/storage/ipc/sinval.c:108
#85 0x00005634b6bfe018 in AcceptInvalidationMessages () at
/home/builder/postgrespro-9.5.7.1/build/../src/backend/utils/cache/inval.c:662
#86 0x00005634b6affb79 in LockRelationOid (relid=2656, lockmode=1) at
/home/builder/postgrespro-9.5.7.1/build/../src/backend/storage/lmgr/lmgr.c:125
#87 0x00005634b68d23b5 in relation_open
(relationId=relationId@entry=2656, lockmode=lockmode@entry=1) at
/home/builder/postgrespro-9.5.7.1/build/../src/backend/access/heap/heapam.c:1068
#88 0x00005634b68e3c6e in index_open (relationId=relationId@entry=2656,
lockmode=lockmode@entry=1) at
/home/builder/postgrespro-9.5.7.1/build/../src/backend/access/index/indexam.c:167
#89 0x00005634b68e36e3 in systable_beginscan
(heapRelation=heapRelation@entry=0x7f2946a32ce8,
indexId=indexId@entry=2656, indexOK=indexOK@entry=1 '\001',
snapshot=snapshot@entry=0x0,
nkeys=nkeys@entry=1, key=key@entry=0x7ffed51128f0) at
/home/builder/postgrespro-9.5.7.1/build/../src/backend/access/index/genam.c:335
#90 0x00005634b6c03736 in AttrDefaultFetch (relation=0x7f2b5dce79d8) at
/home/builder/postgrespro-9.5.7.1/build/../src/backend/utils/cache/relcache.c:3655
#91 RelationBuildTupleDesc (relation=0x7f2b5dce79d8) at
/home/builder/postgrespro-9.5.7.1/build/../src/backend/utils/cache/relcache.c:607
#92 RelationBuildDesc (targetRelId=targetRelId@entry=158077,
insertIt=insertIt@entry=0 '\000') at
/home/builder/postgrespro-9.5.7.1/build/../src/backend/utils/cache/relcache.c:1033
#93 0x00005634b6c042cf in RelationClearRelation
(relation=0x7f2b5dce3f28, rebuild=rebuild@entry=1 '\001') at
/home/builder/postgrespro-9.5.7.1/build/../src/backend/utils/cache/relcache.c:2200
#94 0x00005634b6c04f95 in RelationCacheInvalidate () at
/home/builder/postgrespro-9.5.7.1/build/../src/backend/utils/cache/relcache.c:2500
#95 0x00005634b6bfdecf in InvalidateSystemCaches () at
/home/builder/postgrespro-9.5.7.1/build/../src/backend/utils/cache/inval.c:628
#96 0x00005634b6afca8f in ReceiveSharedInvalidMessages
(invalFunction=invalFunction@entry=0x5634b6bfe6d0
<LocalExecuteInvalidationMessage>,
resetFunction=resetFunction@entry=0x5634b6bfdeb0
<InvalidateSystemCaches>) at
/home/builder/postgrespro-9.5.7.1/build/../src/backend/storage/ipc/sinval.c:108
#97 0x00005634b6bfe018 in AcceptInvalidationMessages () at
/home/builder/postgrespro-9.5.7.1/build/../src/backend/utils/cache/inval.c:662
#98 0x00005634b6affb79 in LockRelationOid (relid=2604, lockmode=1) at
/home/builder/postgrespro-9.5.7.1/build/../src/backend/storage/lmgr/lmgr.c:125
#99 0x00005634b68d23b5 in relation_open (relationId=2604,
lockmode=<optimized out>) at
/home/builder/postgrespro-9.5.7.1/build/../src/backend/access/heap/heapam.c:1068
#100 0x00005634b68d264e in heap_open (relationId=relationId@entry=2604,
lockmode=lockmode@entry=1) at
/home/builder/postgrespro-9.5.7.1/build/../src/backend/access/heap/heapam.c:1246
#101 0x00005634b6c0370e in AttrDefaultFetch (relation=0x5634b7310cd8) at
/home/builder/postgrespro-9.5.7.1/build/../src/backend/utils/cache/relcache.c:3654
#102 RelationBuildTupleDesc (relation=0x5634b7310cd8) at
/home/builder/postgrespro-9.5.7.1/build/../src/backend/utils/cache/relcache.c:607
#103 RelationBuildDesc (targetRelId=targetRelId@entry=158077,
insertIt=insertIt@entry=0 '\000') at
/home/builder/postgrespro-9.5.7.1/build/../src/backend/utils/cache/relcache.c:1033
#104 0x00005634b6c042cf in RelationClearRelation
(relation=0x7f2b5dce3f28, rebuild=rebuild@entry=1 '\001') at
/home/builder/postgrespro-9.5.7.1/build/../src/backend/utils/cache/relcache.c:2200
#105 0x00005634b6c04f95 in RelationCacheInvalidate () at
/home/builder/postgrespro-9.5.7.1/build/../src/backend/utils/cache/relcache.c:2500
#106 0x00005634b6bfdecf in InvalidateSystemCaches () at
/home/builder/postgrespro-9.5.7.1/build/../src/backend/utils/cache/inval.c:628
#107 0x00005634b6afca8f in ReceiveSharedInvalidMessages
(invalFunction=invalFunction@entry=0x5634b6bfe6d0
<LocalExecuteInvalidationMessage>,
resetFunction=resetFunction@entry=0x5634b6bfdeb0
<InvalidateSystemCaches>) at
/home/builder/postgrespro-9.5.7.1/build/../src/backend/storage/ipc/sinval.c:108
#108 0x00005634b6bfe018 in AcceptInvalidationMessages () at
/home/builder/postgrespro-9.5.7.1/build/../src/backend/utils/cache/inval.c:662
#109 0x00005634b6affb79 in LockRelationOid (relid=2659, lockmode=1) at
/home/builder/postgrespro-9.5.7.1/build/../src/backend/storage/lmgr/lmgr.c:125
#110 0x00005634b68d23b5 in relation_open
(relationId=relationId@entry=2659, lockmode=lockmode@entry=1) at
/home/builder/postgrespro-9.5.7.1/build/../src/backend/access/heap/heapam.c:1068
#111 0x00005634b68e3c6e in index_open (relationId=relationId@entry=2659,
lockmode=lockmode@entry=1) at
/home/builder/postgrespro-9.5.7.1/build/../src/backend/access/index/indexam.c:167
#112 0x00005634b68e36e3 in systable_beginscan
(heapRelation=heapRelation@entry=0x7f2b5dc94f38,
indexId=indexId@entry=2659, indexOK=<optimized out>,
snapshot=snapshot@entry=0x0,
nkeys=nkeys@entry=2, key=key@entry=0x7ffed51131f0) at
/home/builder/postgrespro-9.5.7.1/build/../src/backend/access/index/genam.c:335
#113 0x00005634b6c03536 in RelationBuildTupleDesc
(relation=0x7f2b5dc991c8) at
/home/builder/postgrespro-9.5.7.1/build/../src/backend/utils/cache/relcache.c:514
#114 RelationBuildDesc (targetRelId=targetRelId@entry=158077,
insertIt=insertIt@entry=0 '\000') at
/home/builder/postgrespro-9.5.7.1/build/../src/backend/utils/cache/relcache.c:1033
---Type <return> to continue, or q <return> to quit---
#115 0x00005634b6c042cf in RelationClearRelation
(relation=0x7f2b5dce3f28, rebuild=rebuild@entry=1 '\001') at
/home/builder/postgrespro-9.5.7.1/build/../src/backend/utils/cache/relcache.c:2200
#116 0x00005634b6c04f95 in RelationCacheInvalidate () at
/home/builder/postgrespro-9.5.7.1/build/../src/backend/utils/cache/relcache.c:2500
#117 0x00005634b6bfdecf in InvalidateSystemCaches () at
/home/builder/postgrespro-9.5.7.1/build/../src/backend/utils/cache/inval.c:628
#118 0x00005634b6afca8f in ReceiveSharedInvalidMessages
(invalFunction=invalFunction@entry=0x5634b6bfe6d0
<LocalExecuteInvalidationMessage>,
resetFunction=resetFunction@entry=0x5634b6bfdeb0
<InvalidateSystemCaches>) at
/home/builder/postgrespro-9.5.7.1/build/../src/backend/storage/ipc/sinval.c:108
#119 0x00005634b6bfe018 in AcceptInvalidationMessages () at
/home/builder/postgrespro-9.5.7.1/build/../src/backend/utils/cache/inval.c:662
#120 0x00005634b6affb79 in LockRelationOid (relid=1249, lockmode=1) at
/home/builder/postgrespro-9.5.7.1/build/../src/backend/storage/lmgr/lmgr.c:125
#121 0x00005634b68d23b5 in relation_open (relationId=1249,
lockmode=<optimized out>) at
/home/builder/postgrespro-9.5.7.1/build/../src/backend/access/heap/heapam.c:1068
#122 0x00005634b68d264e in heap_open (relationId=relationId@entry=1249,
lockmode=lockmode@entry=1) at
/home/builder/postgrespro-9.5.7.1/build/../src/backend/access/heap/heapam.c:1246
#123 0x00005634b6c03505 in RelationBuildTupleDesc
(relation=0x7f2946aa0a68) at
/home/builder/postgrespro-9.5.7.1/build/../src/backend/utils/cache/relcache.c:513
#124 RelationBuildDesc (targetRelId=targetRelId@entry=158077,
insertIt=insertIt@entry=0 '\000') at
/home/builder/postgrespro-9.5.7.1/build/../src/backend/utils/cache/relcache.c:1033
#125 0x00005634b6c042cf in RelationClearRelation
(relation=0x7f2b5dce3f28, rebuild=rebuild@entry=1 '\001') at
/home/builder/postgrespro-9.5.7.1/build/../src/backend/utils/cache/relcache.c:2200
#126 0x00005634b6c04f95 in RelationCacheInvalidate () at
/home/builder/postgrespro-9.5.7.1/build/../src/backend/utils/cache/relcache.c:2500
#127 0x00005634b6bfdecf in InvalidateSystemCaches () at
/home/builder/postgrespro-9.5.7.1/build/../src/backend/utils/cache/inval.c:628
#128 0x00005634b6afca8f in ReceiveSharedInvalidMessages
(invalFunction=invalFunction@entry=0x5634b6bfe6d0
<LocalExecuteInvalidationMessage>,
resetFunction=resetFunction@entry=0x5634b6bfdeb0
<InvalidateSystemCaches>) at
/home/builder/postgrespro-9.5.7.1/build/../src/backend/storage/ipc/sinval.c:108
#129 0x00005634b6bfe018 in AcceptInvalidationMessages () at
/home/builder/postgrespro-9.5.7.1/build/../src/backend/utils/cache/inval.c:662
#130 0x00005634b6affb79 in LockRelationOid (relid=2662, lockmode=1) at
/home/builder/postgrespro-9.5.7.1/build/../src/backend/storage/lmgr/lmgr.c:125
#131 0x00005634b68d23b5 in relation_open
(relationId=relationId@entry=2662, lockmode=lockmode@entry=1) at
/home/builder/postgrespro-9.5.7.1/build/../src/backend/access/heap/heapam.c:1068
#132 0x00005634b68e3c6e in index_open (relationId=relationId@entry=2662,
lockmode=lockmode@entry=1) at
/home/builder/postgrespro-9.5.7.1/build/../src/backend/access/index/indexam.c:167
#133 0x00005634b68e36e3 in systable_beginscan
(heapRelation=heapRelation@entry=0x7f2b5dcf0318,
indexId=indexId@entry=2662, indexOK=<optimized out>,
snapshot=0x5634b7048f60,
nkeys=nkeys@entry=1, key=key@entry=0x7ffed5113a10) at
/home/builder/postgrespro-9.5.7.1/build/../src/backend/access/index/genam.c:335
#134 0x00005634b6c0076e in ScanPgRelation
(targetRelId=targetRelId@entry=158077, indexOK=indexOK@entry=1 '\001',
force_non_historic=force_non_historic@entry=0 '\000')
at
/home/builder/postgrespro-9.5.7.1/build/../src/backend/utils/cache/relcache.c:339
#135 0x00005634b6c032a7 in RelationBuildDesc
(targetRelId=targetRelId@entry=158077, insertIt=insertIt@entry=0 '\000')
at
/home/builder/postgrespro-9.5.7.1/build/../src/backend/utils/cache/relcache.c:950
#136 0x00005634b6c042cf in RelationClearRelation
(relation=0x7f2b5dce3f28, rebuild=rebuild@entry=1 '\001') at
/home/builder/postgrespro-9.5.7.1/build/../src/backend/utils/cache/relcache.c:2200
#137 0x00005634b6c04f95 in RelationCacheInvalidate () at
/home/builder/postgrespro-9.5.7.1/build/../src/backend/utils/cache/relcache.c:2500
#138 0x00005634b6bfdecf in InvalidateSystemCaches () at
/home/builder/postgrespro-9.5.7.1/build/../src/backend/utils/cache/inval.c:628
#139 0x00005634b6afca8f in ReceiveSharedInvalidMessages
(invalFunction=invalFunction@entry=0x5634b6bfe6d0
<LocalExecuteInvalidationMessage>,
resetFunction=resetFunction@entry=0x5634b6bfdeb0
<InvalidateSystemCaches>) at
/home/builder/postgrespro-9.5.7.1/build/../src/backend/storage/ipc/sinval.c:108
#140 0x00005634b6bfe018 in AcceptInvalidationMessages () at
/home/builder/postgrespro-9.5.7.1/build/../src/backend/utils/cache/inval.c:662
#141 0x00005634b6affb79 in LockRelationOid (relid=3256, lockmode=1) at
/home/builder/postgrespro-9.5.7.1/build/../src/backend/storage/lmgr/lmgr.c:125
#142 0x00005634b68d23b5 in relation_open (relationId=3256,
lockmode=<optimized out>) at
/home/builder/postgrespro-9.5.7.1/build/../src/backend/access/heap/heapam.c:1068
#143 0x00005634b68d264e in heap_open (relationId=relationId@entry=3256,
lockmode=lockmode@entry=1) at
/home/builder/postgrespro-9.5.7.1/build/../src/backend/access/heap/heapam.c:1246
#144 0x00005634b69d1562 in RelationBuildRowSecurity
(relation=relation@entry=0x7f2946e51c98) at
/home/builder/postgrespro-9.5.7.1/build/../src/backend/commands/policy.c:222
#145 0x00005634b6c03e56 in RelationBuildDesc
(targetRelId=targetRelId@entry=158077, insertIt=insertIt@entry=0 '\000')
at
/home/builder/postgrespro-9.5.7.1/build/../src/backend/utils/cache/relcache.c:1052
#146 0x00005634b6c042cf in RelationClearRelation
(relation=0x7f2b5dce3f28, rebuild=rebuild@entry=1 '\001') at
/home/builder/postgrespro-9.5.7.1/build/../src/backend/utils/cache/relcache.c:2200
#147 0x00005634b6c04f95 in RelationCacheInvalidate () at
/home/builder/postgrespro-9.5.7.1/build/../src/backend/utils/cache/relcache.c:2500
#148 0x00005634b6bfdecf in InvalidateSystemCaches () at
/home/builder/postgrespro-9.5.7.1/build/../src/backend/utils/cache/inval.c:628
#149 0x00005634b6afca8f in ReceiveSharedInvalidMessages
(invalFunction=invalFunction@entry=0x5634b6bfe6d0
<LocalExecuteInvalidationMessage>,
resetFunction=resetFunction@entry=0x5634b6bfdeb0
<InvalidateSystemCaches>) at
/home/builder/postgrespro-9.5.7.1/build/../src/backend/storage/ipc/sinval.c:108
#150 0x00005634b6bfe018 in AcceptInvalidationMessages () at
/home/builder/postgrespro-9.5.7.1/build/../src/backend/utils/cache/inval.c:662
#151 0x00005634b6affb79 in LockRelationOid (relid=1259, lockmode=1) at
/home/builder/postgrespro-9.5.7.1/build/../src/backend/storage/lmgr/lmgr.c:125
#152 0x00005634b68d23b5 in relation_open (relationId=1259,
lockmode=<optimized out>) at
/home/builder/postgrespro-9.5.7.1/build/../src/backend/access/heap/heapam.c:1068
#153 0x00005634b68d264e in heap_open (relationId=<optimized out>,
lockmode=lockmode@entry=1) at
/home/builder/postgrespro-9.5.7.1/build/../src/backend/access/heap/heapam.c:1246
#154 0x00005634b6bfc957 in SearchCatCache (cache=0x5634b7306b68,
v1=v1@entry=158077, v2=v2@entry=0, v3=v3@entry=0, v4=v4@entry=0)
---Type <return> to continue, or q <return> to quit---
at
/home/builder/postgrespro-9.5.7.1/build/../src/backend/utils/cache/catcache.c:1239
#155 0x00005634b6c0964e in SearchSysCache (cacheId=cacheId@entry=45,
key1=key1@entry=158077, key2=key2@entry=0, key3=key3@entry=0,
key4=key4@entry=0)
at
/home/builder/postgrespro-9.5.7.1/build/../src/backend/utils/cache/syscache.c:981
#156 0x00005634b6c31c94 in check_enable_rls (relid=158077,
checkAsUser=<optimized out>, noError=noError@entry=0 '\000')
at
/home/builder/postgrespro-9.5.7.1/build/../src/backend/utils/misc/rls.c:66
#157 0x00005634b6ae9d2d in get_row_security_policies
(root=root@entry=0x5634b7390838, rte=rte@entry=0x5634b7390948,
rt_index=rt_index@entry=1,
securityQuals=securityQuals@entry=0x7ffed5114780,
withCheckOptions=withCheckOptions@entry=0x7ffed5114788,
hasRowSecurity=hasRowSecurity@entry=0x7ffed511477d "",
hasSubLinks=hasSubLinks@entry=0x7ffed511477e "")
at
/home/builder/postgrespro-9.5.7.1/build/../src/backend/rewrite/rowsecurity.c:132
#158 0x00005634b6ae4e4d in fireRIRrules (parsetree=0x5634b7390838,
activeRIRs=<optimized out>, activeRIRs@entry=0x0,
forUpdatePushedDown=forUpdatePushedDown@entry=0 '\000')
at
/home/builder/postgrespro-9.5.7.1/build/../src/backend/rewrite/rewriteHandler.c:1782
#159 0x00005634b6ae7bb4 in QueryRewrite
(parsetree=parsetree@entry=0x5634b7390838) at
/home/builder/postgrespro-9.5.7.1/build/../src/backend/rewrite/rewriteHandler.c:3504
#160 0x00005634b6b152ca in pg_rewrite_query (query=0x5634b7390838) at
/home/builder/postgrespro-9.5.7.1/build/../src/backend/tcop/postgres.c:759
#161 0x00005634b6b1556d in pg_analyze_and_rewrite_params
(parsetree=<optimized out>, query_string=<optimized out>,
parserSetup=parserSetup@entry=0x5634b6a17cd0 <sql_fn_parser_setup>,
parserSetupArg=<optimized out>) at
/home/builder/postgrespro-9.5.7.1/build/../src/backend/tcop/postgres.c:726
#162 0x00005634b6a19032 in init_sql_fcache (lazyEvalOK=1 '\001',
collation=<optimized out>, finfo=<optimized out>)
at
/home/builder/postgrespro-9.5.7.1/build/../src/backend/executor/functions.c:710
#163 fmgr_sql (fcinfo=0x5634b72d2fb8) at
/home/builder/postgrespro-9.5.7.1/build/../src/backend/executor/functions.c:1060
#164 0x00005634b6a11617 in ExecMakeFunctionResultNoSets
(fcache=0x5634b72d2f48, econtext=0x5634b72d2d58, isNull=0x5634b72d38d0
"Xu5\267\064V", isDone=<optimized out>)
at
/home/builder/postgrespro-9.5.7.1/build/../src/backend/executor/execQual.c:2020
#165 0x00005634b6a15a02 in ExecTargetList (isDone=0x7ffed5114a4c,
itemIsDone=0x5634b72d39e8, isnull=0x5634b72d38d0 "Xu5\267\064V",
values=0x5634b72d38b8, econtext=0x5634b72d2d58,
targetlist=0x5634b72d39b8, tupdesc=<optimized out>) at
/home/builder/postgrespro-9.5.7.1/build/../src/backend/executor/execQual.c:5417
#166 ExecProject (projInfo=<optimized out>,
isDone=isDone@entry=0x7ffed5114a4c) at
/home/builder/postgrespro-9.5.7.1/build/../src/backend/executor/execQual.c:5641
#167 0x00005634b6a2a012 in ExecResult (node=node@entry=0x5634b72d2c48)
at
/home/builder/postgrespro-9.5.7.1/build/../src/backend/executor/nodeResult.c:155
#168 0x00005634b6a0ea38 in ExecProcNode (node=node@entry=0x5634b72d2c48)
at
/home/builder/postgrespro-9.5.7.1/build/../src/backend/executor/execProcnode.c:392
#169 0x00005634b6a0b6b6 in ExecutePlan (dest=0x5634b73371c0,
direction=<optimized out>, numberTuples=0, sendTuples=1 '\001',
operation=CMD_SELECT, planstate=0x5634b72d2c48,
estate=0x5634b72d2b38) at
/home/builder/postgrespro-9.5.7.1/build/../src/backend/executor/execMain.c:1601
#170 standard_ExecutorRun (queryDesc=0x5634bfe215e8,
direction=<optimized out>, count=0) at
/home/builder/postgrespro-9.5.7.1/build/../src/backend/executor/execMain.c:341
#171 0x00007f2b56a63395 in pgss_ExecutorRun (queryDesc=0x5634bfe215e8,
direction=ForwardScanDirection, count=0)
at
/home/builder/postgrespro-9.5.7.1/build/../contrib/pg_stat_statements/pg_stat_statements.c:878
#172 0x00005634b6b187ff in PortalRunSelect
(portal=portal@entry=0x5634b7396c18, forward=forward@entry=1 '\001',
count=0, count@entry=9223372036854775807, dest=dest@entry=0x5634b73371c0)
at
/home/builder/postgrespro-9.5.7.1/build/../src/backend/tcop/pquery.c:947
#173 0x00005634b6b19c97 in PortalRun
(portal=portal@entry=0x5634b7396c18,
count=count@entry=9223372036854775807, isTopLevel=isTopLevel@entry=1
'\001', dest=dest@entry=0x5634b73371c0,
altdest=altdest@entry=0x5634b73371c0,
completionTag=completionTag@entry=0x7ffed5114fe0 "") at
/home/builder/postgrespro-9.5.7.1/build/../src/backend/tcop/pquery.c:791
#174 0x00005634b6b16d12 in exec_simple_query
(query_string=0x5634b73352a8 "select norders();") at
/home/builder/postgrespro-9.5.7.1/build/../src/backend/tcop/postgres.c:1104
#175 PostgresMain (argc=<optimized out>, argv=argv@entry=0x5634b72b23c0,
dbname=0x5634b72b2270 "tsup", username=<optimized out>)
at
/home/builder/postgrespro-9.5.7.1/build/../src/backend/tcop/postgres.c:4102
#176 0x00005634b68a2ab8 in BackendRun (port=0x5634b72e2310) at
/home/builder/postgrespro-9.5.7.1/build/../src/backend/postmaster/postmaster.c:4277
#177 BackendStartup (port=0x5634b72e2310) at
/home/builder/postgrespro-9.5.7.1/build/../src/backend/postmaster/postmaster.c:3951
#178 ServerLoop () at
/home/builder/postgrespro-9.5.7.1/build/../src/backend/postmaster/postmaster.c:1716
#179 0x00005634b6abad5b in PostmasterMain (argc=argc@entry=3,
argv=argv@entry=0x5634b72b1380) at
/home/builder/postgrespro-9.5.7.1/build/../src/backend/postmaster/postmaster.c:1324
#180 0x00005634b68a3a54 in main (argc=3, argv=0x5634b72b1380) at
/home/builder/postgrespro-9.5.7.1/build/../src/backend/main/main.c:253
As you can see system catalog updates caused by frequent create/drop
temp tables lead to nested invocation of InvalidateSystemCaches()
This is itself seems to be bad smell and suspicious behavior. And in
conjunction with memory leak in RelationBuildRowSecurity it leads to
backend memory bloat.
I do not know how to fix cache invalidation problem. Global temp tables
can help here.
But right now I just want to fix the bug with memory leak in
RelationBuildRowSecurity.
--
Konstantin Knizhnik
Postgres Professional: http://www.postgrespro.com
The Russian Postgres Company
Attachments:
build_row_security_memory_leak.patchtext/x-patch; name=build_row_security_memory_leak.patchDownload+4-11
Konstantin Knizhnik <k.knizhnik@postgrespro.ru> writes:
There is memory leak in RelationBuildRowSecurity function - it converts
node to expression tree in current memory context:
Hm ... this smells a whole lot like the issue we fixed recently for
partition expressions, namely: what the devil are we doing reloading
these expressions during RelationClearRelation? We should delay
populating that cache until the value is requested.
regards, tom lane
On 25.09.2020 16:48, Tom Lane wrote:
Konstantin Knizhnik <k.knizhnik@postgrespro.ru> writes:
There is memory leak in RelationBuildRowSecurity function - it converts
node to expression tree in current memory context:Hm ... this smells a whole lot like the issue we fixed recently for
partition expressions, namely: what the devil are we doing reloading
these expressions during RelationClearRelation? We should delay
populating that cache until the value is requested.
Sorry, this stack trace was obtained at 9.6 version of Postgres which
the customer is using.
So may be the problem with cache invalidation is already fixed.
But the memory leak in RelationBuildRowSecurity is still present.
May be it is not so critical if RelationBuildRowSecurity will not be
called many times in the same memory context.
But at least calling pfree for a tree is very strange idea: we should
not do it all or allocate tree in separate memory context.
By the way - it is illustration how "SQL function data" is growing after
each invocation of RelationBuildRowSecurity:
SQL function data: 57344 total in 3 blocks; 53112 free (278 chunks);
4232 used
SQL function data: 289213248 total in 3137 blocks; 3281544 free (217
chunks); 285931704 used
SQL function data: 578434688 total in 6265 blocks; 6581320 free (234
chunks); 571853368 used
SQL function data: 1148488960 total in 12520 blocks; 4793680 free (17
chunks); 1143695280 used
SQL function data: 2288597504 total in 25030 blocks; 1222064 free (221
chunks); 2287375440 used
Konstantin Knizhnik <k.knizhnik@postgrespro.ru> writes:
On 25.09.2020 16:48, Tom Lane wrote:
Hm ... this smells a whole lot like the issue we fixed recently for
partition expressions, namely: what the devil are we doing reloading
these expressions during RelationClearRelation? We should delay
populating that cache until the value is requested.
Sorry, this stack trace was obtained at 9.6 version of Postgres which
the customer is using.
So may be the problem with cache invalidation is already fixed.
Not sure if we've changed anything much in that area. I've occasionally
wondered if we should not increase the size of the sinval message queue,
but it hasn't happened yet.
Anyway, looking more closely at RelationBuildRowSecurity, it doesn't
seem to be doing anything more dangerous than RelationBuildRuleLock
is; it's definitely not risking recursion, as the partition stuff did.
So I take back the idea that we need to postpone it till the data is
referenced. However, it is leaking memory.
Attached is a proposed rewrite that borrows some ideas from
RelationBuildRuleLock, and also makes use of MemoryContextSetParent
so we don't need a PG_TRY block. In a very simple test
(using regress_rls_schema.rls_tbl from the regression database),
this doesn't seem to leak any memory in the caller context,
and it also doesn't make the rscxt any bigger than it was before.
But at least calling pfree for a tree is very strange idea: we should
not do it all or allocate tree in separate memory context.
Yeah, that's pretty useless. But also I think the string forms of
the expressions are leaking as much memory as the tree forms.
This is against HEAD but I think it wouldn't be hard to back-patch.
Are you in a position to see if it fixes the problem in your customer's
environment?
regards, tom lane
Attachments:
rewrite-RelationBuildRowSecurity-1.patchtext/x-diff; charset=us-ascii; name=rewrite-RelationBuildRowSecurity-1.patchDownload+98-118
On 25.09.2020 21:07, Tom Lane wrote:
Konstantin Knizhnik <k.knizhnik@postgrespro.ru> writes:
On 25.09.2020 16:48, Tom Lane wrote:
Hm ... this smells a whole lot like the issue we fixed recently for
partition expressions, namely: what the devil are we doing reloading
these expressions during RelationClearRelation? We should delay
populating that cache until the value is requested.Sorry, this stack trace was obtained at 9.6 version of Postgres which
the customer is using.
So may be the problem with cache invalidation is already fixed.Not sure if we've changed anything much in that area. I've occasionally
wondered if we should not increase the size of the sinval message queue,
but it hasn't happened yet.Anyway, looking more closely at RelationBuildRowSecurity, it doesn't
seem to be doing anything more dangerous than RelationBuildRuleLock
is; it's definitely not risking recursion, as the partition stuff did.
So I take back the idea that we need to postpone it till the data is
referenced. However, it is leaking memory.Attached is a proposed rewrite that borrows some ideas from
RelationBuildRuleLock, and also makes use of MemoryContextSetParent
so we don't need a PG_TRY block. In a very simple test
(using regress_rls_schema.rls_tbl from the regression database),
this doesn't seem to leak any memory in the caller context,
and it also doesn't make the rscxt any bigger than it was before.But at least calling pfree for a tree is very strange idea: we should
not do it all or allocate tree in separate memory context.Yeah, that's pretty useless. But also I think the string forms of
the expressions are leaking as much memory as the tree forms.This is against HEAD but I think it wouldn't be hard to back-patch.
Are you in a position to see if it fixes the problem in your customer's
environment?
Thank you.
Your patch fixes the problem.
Konstantin Knizhnik <k.knizhnik@postgrespro.ru> writes:
On 25.09.2020 21:07, Tom Lane wrote:
This is against HEAD but I think it wouldn't be hard to back-patch.
Are you in a position to see if it fixes the problem in your customer's
environment?
Thank you.
Your patch fixes the problem.
Thanks for testing! I'll try to get this committed later today.
regards, tom lane
On 26.09.2020 20:05, Tom Lane wrote:
Konstantin Knizhnik <k.knizhnik@postgrespro.ru> writes:
On 25.09.2020 21:07, Tom Lane wrote:
This is against HEAD but I think it wouldn't be hard to back-patch.
Are you in a position to see if it fixes the problem in your customer's
environment?Thank you.
Your patch fixes the problem.Thanks for testing! I'll try to get this committed later today.
regards, tom lane
Just want to convey gratitude from the customer: after apply your patch
memory consumption is decreased 3 times which significantly improves
performance of their system.
--
Konstantin Knizhnik
Postgres Professional: http://www.postgrespro.com
The Russian Postgres Company