Vectorization of some functions and improving pg_list interface

Started by Maxim Orlovover 2 years ago4 messages
#1Maxim Orlov
orlovmg@gmail.com
4 attachment(s)

Hi!

Recently, I've been playing around with pg_lists and realize how annoying
(maybe, I was a bit tired) some stuff related to the lists.
For an example, see this code
List *l1 = list_make4(1, 2, 3, 4),
*l2 = list_make4(5, 6, 7, 8),
*l3 = list_make4(9, 0, 1, 2);
ListCell *lc1, *lc2, *lc3;

forthree(lc1, l1, lc2, l2, lc3, l3) {
...
}

list_free(l1);
list_free(l2);
list_free(l3);

There are several questions:
1) Why do I need to specify the number of elements in the list in the
function name?
Compiler already knew how much arguments do I use.
2) Why I have to call free for every list? I don't know how to call it
right, for now I call it vectorization.
Why not to use simple wrapper to "vectorize" function args?

So, my proposal is:
1) Add a simple macro to "vectorize" functions.
2) Use this macro to "vectorize" list_free and list_free_deep functions.
3) Use this macro to "vectorize" bms_free function.
4) "Vectorize" list_makeN functions.

For this V1 version, I do not remove all list_makeN calls in order to
reduce diff, but I'll address
this in future, if it will be needed.

In my view, one thing still waiting to be improved if foreach loop. It is
not very handy to have a bunch of
similar calls foreach, forboth, forthree and etc. It will be ideal to have
single foreach interface, but I don't know how
to do it without overall interface of the loop.

Any opinions are very welcome!

--
Best regards,
Maxim Orlov.

Attachments:

v1-0002-Make-use-of-vectorized-lists_free-and-lists_free_.patchapplication/octet-stream; name=v1-0002-Make-use-of-vectorized-lists_free-and-lists_free_.patchDownload
From fe5cd43c2f8ed5b08aa8bcf8da6e8a5a23c89307 Mon Sep 17 00:00:00 2001
From: Maxim Orlov <orlovmg@gmail.com>
Date: Thu, 24 Aug 2023 16:13:20 +0300
Subject: [PATCH v1 2/4] Make use of vectorized lists_free and lists_free_deep
 functions

---
 src/backend/catalog/namespace.c             |  3 +--
 src/backend/catalog/pg_inherits.c           |  3 +--
 src/backend/catalog/pg_publication.c        |  3 +--
 src/backend/commands/publicationcmds.c      |  3 +--
 src/backend/executor/nodeModifyTable.c      |  4 ++--
 src/backend/optimizer/path/costsize.c       |  3 +--
 src/backend/optimizer/path/joinpath.c       | 12 ++++--------
 src/backend/optimizer/plan/planner.c        | 12 ++++--------
 src/backend/optimizer/util/predtest.c       |  3 +--
 src/backend/partitioning/partbounds.c       |  7 ++-----
 src/backend/partitioning/partprune.c        |  3 +--
 src/backend/replication/pgoutput/pgoutput.c |  4 +---
 src/backend/utils/cache/relcache.c          |  9 +++------
 src/backend/utils/misc/guc.c                |  3 +--
 14 files changed, 24 insertions(+), 48 deletions(-)

diff --git a/src/backend/catalog/namespace.c b/src/backend/catalog/namespace.c
index 4ceae038ec..4c2dfb1cd1 100644
--- a/src/backend/catalog/namespace.c
+++ b/src/backend/catalog/namespace.c
@@ -3794,8 +3794,7 @@ recomputeNamespacePath(void)
 
 	/* Clean up. */
 	pfree(rawname);
-	list_free(namelist);
-	list_free(oidlist);
+	lists_free(namelist, oidlist);
 }
 
 /*
diff --git a/src/backend/catalog/pg_inherits.c b/src/backend/catalog/pg_inherits.c
index da969bd2f9..c0929e69d8 100644
--- a/src/backend/catalog/pg_inherits.c
+++ b/src/backend/catalog/pg_inherits.c
@@ -496,8 +496,7 @@ typeInheritsFrom(Oid subclassTypeId, Oid superclassTypeId)
 	/* clean up ... */
 	table_close(inhrel, AccessShareLock);
 
-	list_free(visited);
-	list_free(queue);
+	lists_free(visited, queue);
 
 	return result;
 }
diff --git a/src/backend/catalog/pg_publication.c b/src/backend/catalog/pg_publication.c
index c488b6370b..81075a86b3 100644
--- a/src/backend/catalog/pg_publication.c
+++ b/src/backend/catalog/pg_publication.c
@@ -349,8 +349,7 @@ GetTopMostAncestorInPublication(Oid puboid, List *ancestors, int *ancestor_level
 			}
 		}
 
-		list_free(apubids);
-		list_free(aschemaPubids);
+		lists_free(apubids, aschemaPubids);
 	}
 
 	return topmost_relid;
diff --git a/src/backend/commands/publicationcmds.c b/src/backend/commands/publicationcmds.c
index f4ba572697..e4fe6cc72a 100644
--- a/src/backend/commands/publicationcmds.c
+++ b/src/backend/commands/publicationcmds.c
@@ -1693,8 +1693,7 @@ OpenTableList(List *tables)
 		}
 	}
 
-	list_free(relids);
-	list_free(relids_with_rf);
+	lists_free(relids, relids_with_rf);
 
 	return rels;
 }
diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c
index 5005d8c0d1..ebfcfc9ce2 100644
--- a/src/backend/executor/nodeModifyTable.c
+++ b/src/backend/executor/nodeModifyTable.c
@@ -1293,8 +1293,8 @@ ExecPendingInserts(EState *estate)
 						estate, mtstate->canSetTag);
 	}
 
-	list_free(estate->es_insert_pending_result_relations);
-	list_free(estate->es_insert_pending_modifytables);
+	lists_free(estate->es_insert_pending_result_relations,
+			   estate->es_insert_pending_modifytables);
 	estate->es_insert_pending_result_relations = NIL;
 	estate->es_insert_pending_modifytables = NIL;
 }
diff --git a/src/backend/optimizer/path/costsize.c b/src/backend/optimizer/path/costsize.c
index d6ceafd51c..c5de63bc4c 100644
--- a/src/backend/optimizer/path/costsize.c
+++ b/src/backend/optimizer/path/costsize.c
@@ -5461,8 +5461,7 @@ calc_joinrel_size_estimate(PlannerInfo *root,
 										sjinfo);
 
 		/* Avoid leaking a lot of ListCells */
-		list_free(joinquals);
-		list_free(pushedquals);
+		lists_free(joinquals, pushedquals);
 	}
 	else
 	{
diff --git a/src/backend/optimizer/path/joinpath.c b/src/backend/optimizer/path/joinpath.c
index 821d282497..f9fd89e734 100644
--- a/src/backend/optimizer/path/joinpath.c
+++ b/src/backend/optimizer/path/joinpath.c
@@ -465,8 +465,7 @@ paraminfo_get_equal_hashops(PlannerInfo *root, ParamPathInfo *param_info,
 			if (!IsA(opexpr, OpExpr) || list_length(opexpr->args) != 2 ||
 				!clause_sides_match_join(rinfo, outerrel, innerrel))
 			{
-				list_free(*operators);
-				list_free(*param_exprs);
+				lists_free(*operators, *param_exprs);
 				return false;
 			}
 
@@ -484,8 +483,7 @@ paraminfo_get_equal_hashops(PlannerInfo *root, ParamPathInfo *param_info,
 			/* can't do memoize if we can't hash the outer type */
 			if (!OidIsValid(hasheqoperator))
 			{
-				list_free(*operators);
-				list_free(*param_exprs);
+				lists_free(*operators, *param_exprs);
 				return false;
 			}
 
@@ -517,8 +515,7 @@ paraminfo_get_equal_hashops(PlannerInfo *root, ParamPathInfo *param_info,
 		/* Reject if there are any volatile functions in lateral vars */
 		if (contain_volatile_functions(expr))
 		{
-			list_free(*operators);
-			list_free(*param_exprs);
+			lists_free(*operators, *param_exprs);
 			return false;
 		}
 
@@ -528,8 +525,7 @@ paraminfo_get_equal_hashops(PlannerInfo *root, ParamPathInfo *param_info,
 		/* can't use memoize without a valid hash proc and equals operator */
 		if (!OidIsValid(typentry->hash_proc) || !OidIsValid(typentry->eq_opr))
 		{
-			list_free(*operators);
-			list_free(*param_exprs);
+			lists_free(*operators, *param_exprs);
 			return false;
 		}
 
diff --git a/src/backend/optimizer/plan/planner.c b/src/backend/optimizer/plan/planner.c
index 44efb1f4eb..165fd9241b 100644
--- a/src/backend/optimizer/plan/planner.c
+++ b/src/backend/optimizer/plan/planner.c
@@ -5413,8 +5413,7 @@ make_group_input_target(PlannerInfo *root, PathTarget *final_target)
 	add_new_columns_to_pathtarget(input_target, non_group_vars);
 
 	/* clean up cruft */
-	list_free(non_group_vars);
-	list_free(non_group_cols);
+	lists_free(non_group_vars, non_group_cols);
 
 	/* XXX this causes some redundant cost calculation ... */
 	return set_pathtarget_cost_width(root, input_target);
@@ -5526,8 +5525,7 @@ make_partial_grouping_target(PlannerInfo *root,
 	}
 
 	/* clean up cruft */
-	list_free(non_group_exprs);
-	list_free(non_group_cols);
+	lists_free(non_group_exprs, non_group_cols);
 
 	/* XXX this causes some redundant cost calculation ... */
 	return set_pathtarget_cost_width(root, partial_target);
@@ -6009,8 +6007,7 @@ make_window_input_target(PlannerInfo *root,
 	add_new_columns_to_pathtarget(input_target, flattenable_vars);
 
 	/* clean up cruft */
-	list_free(flattenable_vars);
-	list_free(flattenable_cols);
+	lists_free(flattenable_vars, flattenable_cols);
 
 	/* XXX this causes some redundant cost calculation ... */
 	return set_pathtarget_cost_width(root, input_target);
@@ -6309,8 +6306,7 @@ make_sort_input_target(PlannerInfo *root,
 	add_new_columns_to_pathtarget(input_target, postponable_vars);
 
 	/* clean up cruft */
-	list_free(postponable_vars);
-	list_free(postponable_cols);
+	lists_free(postponable_vars, postponable_cols);
 
 	/* XXX this represents even more redundant cost calculation ... */
 	return set_pathtarget_cost_width(root, input_target);
diff --git a/src/backend/optimizer/util/predtest.c b/src/backend/optimizer/util/predtest.c
index 237c883874..1ca1f45e2e 100644
--- a/src/backend/optimizer/util/predtest.c
+++ b/src/backend/optimizer/util/predtest.c
@@ -2166,8 +2166,7 @@ lookup_proof_cache(Oid pred_op, Oid clause_op, bool refute_it)
 			break;
 	}
 
-	list_free_deep(pred_op_infos);
-	list_free_deep(clause_op_infos);
+	lists_free_deep(pred_op_infos, clause_op_infos);
 
 	if (!found)
 	{
diff --git a/src/backend/partitioning/partbounds.c b/src/backend/partitioning/partbounds.c
index 5436cc302d..66b8479075 100644
--- a/src/backend/partitioning/partbounds.c
+++ b/src/backend/partitioning/partbounds.c
@@ -1477,8 +1477,7 @@ merge_list_bounds(FmgrInfo *partsupfunc, Oid *partcollation,
 
 cleanup:
 	/* Free local memory before returning. */
-	list_free(merged_datums);
-	list_free(merged_indexes);
+	lists_free(merged_datums, merged_indexes);
 	free_partition_map(&outer_map);
 	free_partition_map(&inner_map);
 
@@ -1795,9 +1794,7 @@ merge_range_bounds(int partnatts, FmgrInfo *partsupfuncs,
 
 cleanup:
 	/* Free local memory before returning. */
-	list_free(merged_datums);
-	list_free(merged_kinds);
-	list_free(merged_indexes);
+	lists_free(merged_datums, merged_kinds, merged_indexes);
 	free_partition_map(&outer_map);
 	free_partition_map(&inner_map);
 
diff --git a/src/backend/partitioning/partprune.c b/src/backend/partitioning/partprune.c
index 7179b22a05..9a95986ad6 100644
--- a/src/backend/partitioning/partprune.c
+++ b/src/backend/partitioning/partprune.c
@@ -2501,8 +2501,7 @@ get_steps_using_prefix_recurse(GeneratePruningStepsContext *context,
 													   step_cmpfns1);
 			result = list_concat(result, moresteps);
 
-			list_free(step_exprs1);
-			list_free(step_cmpfns1);
+			lists_free(step_exprs1, step_cmpfns1);
 		}
 	}
 	else
diff --git a/src/backend/replication/pgoutput/pgoutput.c b/src/backend/replication/pgoutput/pgoutput.c
index b08ca55041..1596cd163b 100644
--- a/src/backend/replication/pgoutput/pgoutput.c
+++ b/src/backend/replication/pgoutput/pgoutput.c
@@ -2198,9 +2198,7 @@ get_rel_sync_entry(PGOutputData *data, Relation relation)
 			pgoutput_column_list_init(data, rel_publications, entry);
 		}
 
-		list_free(pubids);
-		list_free(schemaPubids);
-		list_free(rel_publications);
+		lists_free(pubids, schemaPubids, rel_publications);
 
 		entry->replicate_valid = true;
 	}
diff --git a/src/backend/utils/cache/relcache.c b/src/backend/utils/cache/relcache.c
index 8e08ca1c68..1f813e5946 100644
--- a/src/backend/utils/cache/relcache.c
+++ b/src/backend/utils/cache/relcache.c
@@ -2439,8 +2439,7 @@ RelationDestroyRelation(Relation relation, bool remember_tupdesc)
 	}
 	FreeTriggerDesc(relation->trigdesc);
 	list_free_deep(relation->rd_fkeylist);
-	list_free(relation->rd_indexlist);
-	list_free(relation->rd_statlist);
+	lists_free(relation->rd_indexlist, relation->rd_statlist);
 	bms_free(relation->rd_keyattr);
 	bms_free(relation->rd_pkattr);
 	bms_free(relation->rd_idattr);
@@ -5365,14 +5364,12 @@ restart:
 		relreplindex == relation->rd_replidindex)
 	{
 		/* Still the same index set, so proceed */
-		list_free(newindexoidlist);
-		list_free(indexoidlist);
+		lists_free(newindexoidlist, indexoidlist);
 	}
 	else
 	{
 		/* Gotta do it over ... might as well not leak memory */
-		list_free(newindexoidlist);
-		list_free(indexoidlist);
+		lists_free(newindexoidlist, indexoidlist);
 		bms_free(uindexattrs);
 		bms_free(pkindexattrs);
 		bms_free(idindexattrs);
diff --git a/src/backend/utils/misc/guc.c b/src/backend/utils/misc/guc.c
index 84e7ad4d90..c474fe74c0 100644
--- a/src/backend/utils/misc/guc.c
+++ b/src/backend/utils/misc/guc.c
@@ -6299,8 +6299,7 @@ ProcessGUCArray(ArrayType *array,
 		pfree(value);
 	}
 
-	list_free(gucNames);
-	list_free(gucValues);
+	lists_free(gucNames, gucValues);
 }
 
 
-- 
2.41.0

v1-0004-vectorize-list_make-functions.patchapplication/octet-stream; name=v1-0004-vectorize-list_make-functions.patchDownload
From a0c4a73410a988fa770fbd265f030c2007d63aa1 Mon Sep 17 00:00:00 2001
From: Maxim Orlov <orlovmg@gmail.com>
Date: Wed, 23 Aug 2023 18:52:05 +0300
Subject: [PATCH v1 4/4] vectorize list_make* functions

---
 src/backend/nodes/list.c    |  70 +++++++++++-----------
 src/include/nodes/pg_list.h | 112 +++++++++++++-----------------------
 2 files changed, 75 insertions(+), 107 deletions(-)

diff --git a/src/backend/nodes/list.c b/src/backend/nodes/list.c
index 750ee5a7e5..9e616d7e4c 100644
--- a/src/backend/nodes/list.c
+++ b/src/backend/nodes/list.c
@@ -229,67 +229,69 @@ enlarge_list(List *list, int min_size)
 
 /*
  * Convenience functions to construct short Lists from given values.
- * (These are normally invoked via the list_makeN macros.)
+ * (These are normally invoked via the list_make* macros.)
  */
+#define list_make_ptr_cell(v)	((ListCell) {.ptr_value = (v)})
+#define list_make_int_cell(v)	((ListCell) {.int_value = (v)})
+#define list_make_oid_cell(v)	((ListCell) {.oid_value = (v)})
+#define list_make_xid_cell(v)	((ListCell) {.xid_value = (v)})
+
 List *
-list_make1_impl(NodeTag t, ListCell datum1)
+list_make_impl(NodeTag t, int n, ...)
 {
-	List	   *list = new_list(t, 1);
+	List	   *list = new_list(t, n);
 
-	list->elements[0] = datum1;
-	check_list_invariants(list);
-	return list;
-}
+	va_list datum;
+	va_start(datum, n);
 
-List *
-list_make2_impl(NodeTag t, ListCell datum1, ListCell datum2)
-{
-	List	   *list = new_list(t, 2);
+	for (int i = 0; i < n; i++)
+		list->elements[i] = list_make_ptr_cell(va_arg(datum, void *));
 
-	list->elements[0] = datum1;
-	list->elements[1] = datum2;
 	check_list_invariants(list);
 	return list;
 }
 
 List *
-list_make3_impl(NodeTag t, ListCell datum1, ListCell datum2,
-				ListCell datum3)
+list_make_int_impl(NodeTag t, int n, ...)
 {
-	List	   *list = new_list(t, 3);
+	List	   *list = new_list(t, n);
+
+	va_list datum;
+	va_start(datum, n);
+
+	for (int i = 0; i < n; i++)
+		list->elements[i] = list_make_int_cell(va_arg(datum, int));
 
-	list->elements[0] = datum1;
-	list->elements[1] = datum2;
-	list->elements[2] = datum3;
 	check_list_invariants(list);
 	return list;
 }
 
 List *
-list_make4_impl(NodeTag t, ListCell datum1, ListCell datum2,
-				ListCell datum3, ListCell datum4)
+list_make_oid_impl(NodeTag t, int n, ...)
 {
-	List	   *list = new_list(t, 4);
+	List	   *list = new_list(t, n);
+
+	va_list datum;
+	va_start(datum, n);
+
+	for (int i = 0; i < n; i++)
+		list->elements[i] = list_make_oid_cell(va_arg(datum, Oid));
 
-	list->elements[0] = datum1;
-	list->elements[1] = datum2;
-	list->elements[2] = datum3;
-	list->elements[3] = datum4;
 	check_list_invariants(list);
 	return list;
 }
 
 List *
-list_make5_impl(NodeTag t, ListCell datum1, ListCell datum2,
-				ListCell datum3, ListCell datum4, ListCell datum5)
+list_make_xid_impl(NodeTag t, int n, ...)
 {
-	List	   *list = new_list(t, 5);
+	List	   *list = new_list(t, n);
+
+	va_list datum;
+	va_start(datum, n);
+
+	for (int i = 0; i < n; i++)
+		list->elements[i] = list_make_xid_cell(va_arg(datum, TransactionId));
 
-	list->elements[0] = datum1;
-	list->elements[1] = datum2;
-	list->elements[2] = datum3;
-	list->elements[3] = datum4;
-	list->elements[4] = datum5;
 	check_list_invariants(list);
 	return list;
 }
diff --git a/src/include/nodes/pg_list.h b/src/include/nodes/pg_list.h
index daae56ec32..5592dc3de1 100644
--- a/src/include/nodes/pg_list.h
+++ b/src/include/nodes/pg_list.h
@@ -204,70 +204,41 @@ list_length(const List *l)
 /*
  * Convenience macros for building fixed-length lists
  */
-#define list_make_ptr_cell(v)	((ListCell) {.ptr_value = (v)})
-#define list_make_int_cell(v)	((ListCell) {.int_value = (v)})
-#define list_make_oid_cell(v)	((ListCell) {.oid_value = (v)})
-#define list_make_xid_cell(v)	((ListCell) {.xid_value = (v)})
-
-#define list_make1(x1) \
-	list_make1_impl(T_List, list_make_ptr_cell(x1))
-#define list_make2(x1,x2) \
-	list_make2_impl(T_List, list_make_ptr_cell(x1), list_make_ptr_cell(x2))
-#define list_make3(x1,x2,x3) \
-	list_make3_impl(T_List, list_make_ptr_cell(x1), list_make_ptr_cell(x2), \
-					list_make_ptr_cell(x3))
-#define list_make4(x1,x2,x3,x4) \
-	list_make4_impl(T_List, list_make_ptr_cell(x1), list_make_ptr_cell(x2), \
-					list_make_ptr_cell(x3), list_make_ptr_cell(x4))
-#define list_make5(x1,x2,x3,x4,x5) \
-	list_make5_impl(T_List, list_make_ptr_cell(x1), list_make_ptr_cell(x2), \
-					list_make_ptr_cell(x3), list_make_ptr_cell(x4), \
-					list_make_ptr_cell(x5))
-
-#define list_make1_int(x1) \
-	list_make1_impl(T_IntList, list_make_int_cell(x1))
-#define list_make2_int(x1,x2) \
-	list_make2_impl(T_IntList, list_make_int_cell(x1), list_make_int_cell(x2))
-#define list_make3_int(x1,x2,x3) \
-	list_make3_impl(T_IntList, list_make_int_cell(x1), list_make_int_cell(x2), \
-					list_make_int_cell(x3))
-#define list_make4_int(x1,x2,x3,x4) \
-	list_make4_impl(T_IntList, list_make_int_cell(x1), list_make_int_cell(x2), \
-					list_make_int_cell(x3), list_make_int_cell(x4))
-#define list_make5_int(x1,x2,x3,x4,x5) \
-	list_make5_impl(T_IntList, list_make_int_cell(x1), list_make_int_cell(x2), \
-					list_make_int_cell(x3), list_make_int_cell(x4), \
-					list_make_int_cell(x5))
-
-#define list_make1_oid(x1) \
-	list_make1_impl(T_OidList, list_make_oid_cell(x1))
-#define list_make2_oid(x1,x2) \
-	list_make2_impl(T_OidList, list_make_oid_cell(x1), list_make_oid_cell(x2))
-#define list_make3_oid(x1,x2,x3) \
-	list_make3_impl(T_OidList, list_make_oid_cell(x1), list_make_oid_cell(x2), \
-					list_make_oid_cell(x3))
-#define list_make4_oid(x1,x2,x3,x4) \
-	list_make4_impl(T_OidList, list_make_oid_cell(x1), list_make_oid_cell(x2), \
-					list_make_oid_cell(x3), list_make_oid_cell(x4))
-#define list_make5_oid(x1,x2,x3,x4,x5) \
-	list_make5_impl(T_OidList, list_make_oid_cell(x1), list_make_oid_cell(x2), \
-					list_make_oid_cell(x3), list_make_oid_cell(x4), \
-					list_make_oid_cell(x5))
-
-#define list_make1_xid(x1) \
-	list_make1_impl(T_XidList, list_make_xid_cell(x1))
-#define list_make2_xid(x1,x2) \
-	list_make2_impl(T_XidList, list_make_xid_cell(x1), list_make_xid_cell(x2))
-#define list_make3_xid(x1,x2,x3) \
-	list_make3_impl(T_XidList, list_make_xid_cell(x1), list_make_xid_cell(x2), \
-					list_make_xid_cell(x3))
-#define list_make4_xid(x1,x2,x3,x4) \
-	list_make4_impl(T_XidList, list_make_xid_cell(x1), list_make_xid_cell(x2), \
-					list_make_xid_cell(x3), list_make_xid_cell(x4))
-#define list_make5_xid(x1,x2,x3,x4,x5) \
-	list_make5_impl(T_XidList, list_make_xid_cell(x1), list_make_xid_cell(x2), \
-					list_make_xid_cell(x3), list_make_xid_cell(x4), \
-					list_make_xid_cell(x5))
+#define list_make1(x1) list_make(x1)
+#define list_make2(x1,x2) list_make(x1,x2)
+#define list_make3(x1,x2,x3) list_make(x1,x2,x3)
+#define list_make4(x1,x2,x3,x4) list_make(x1,x2,x3,x4)
+#define list_make5(x1,x2,x3,x4,x5) list_make(x1,x2,x3,x4,x5)
+
+#define list_make(...) \
+	list_make_impl(T_List, VA_ARGS_NARGS(__VA_ARGS__), __VA_ARGS__)
+
+#define list_make1_int(x1) list_make_int(x1)
+#define list_make2_int(x1,x2) list_make_int(_intx1,x2)
+#define list_make3_int(x1,x2,x3) list_make_int(x1,x2,x3)
+#define list_make4_int(x1,x2,x3,x4) list_make_int(x1,x2,x3,x4)
+#define list_make5_int(x1,x2,x3,x4,x5) list_make_int(x1,x2,x3,x4,x5)
+
+#define list_make_int(...) \
+	list_make_int_impl(T_IntList, VA_ARGS_NARGS(__VA_ARGS__), __VA_ARGS__)
+
+#define list_make1_oid(x1) list_make_oid((x1))
+#define list_make2_oid(x1,x2) list_make_oid((x1),(x2))
+#define list_make3_oid(x1,x2,x3) list_make_oid((x1),(x2),(x3))
+#define list_make4_oid(x1,x2,x3,x4) list_make_oid((x1),(x2),(x3),(x4))
+#define list_make5_oid(x1,x2,x3,x4,x5) list_make_oid((x1),(x2),(x3),(x4),(x5))
+
+#define list_make_oid(...) \
+	list_make_oid_impl(T_OidList, VA_ARGS_NARGS(__VA_ARGS__), __VA_ARGS__)
+
+#define list_make1_xid(x1) list_make_xid(x1)
+#define list_make2_xid(x1,x2) list_make_xid(_intx1,x2)
+#define list_make3_xid(x1,x2,x3) list_make_xid(x1,x2,x3)
+#define list_make4_xid(x1,x2,x3,x4) list_make_xid(x1,x2,x3,x4)
+#define list_make5_xid(x1,x2,x3,x4,x5) list_make_xid(x1,x2,x3,x4,x5)
+
+#define list_make_xid(...) \
+	list_make_xid_impl(T_XidList, VA_ARGS_NARGS(__VA_ARGS__), __VA_ARGS__)
 
 /*
  * Locate the n'th cell (counting from 0) of the list.
@@ -547,15 +518,10 @@ for_both_cell_setup(const List *list1, const ListCell *initcell1,
 
 /* Functions in src/backend/nodes/list.c */
 
-extern List *list_make1_impl(NodeTag t, ListCell datum1);
-extern List *list_make2_impl(NodeTag t, ListCell datum1, ListCell datum2);
-extern List *list_make3_impl(NodeTag t, ListCell datum1, ListCell datum2,
-							 ListCell datum3);
-extern List *list_make4_impl(NodeTag t, ListCell datum1, ListCell datum2,
-							 ListCell datum3, ListCell datum4);
-extern List *list_make5_impl(NodeTag t, ListCell datum1, ListCell datum2,
-							 ListCell datum3, ListCell datum4,
-							 ListCell datum5);
+extern List *list_make_impl(NodeTag t, int n, ...);
+extern List *list_make_int_impl(NodeTag t, int n, ...);
+extern List *list_make_oid_impl(NodeTag t, int n, ...);
+extern List *list_make_xid_impl(NodeTag t, int n, ...);
 
 extern pg_nodiscard List *lappend(List *list, void *datum);
 extern pg_nodiscard List *lappend_int(List *list, int datum);
-- 
2.41.0

v1-0001-Add-lists_free-and-lists_free_deep.patchapplication/octet-stream; name=v1-0001-Add-lists_free-and-lists_free_deep.patchDownload
From 31b7ea9a4251057dfd2dc9e02e72440d957011e5 Mon Sep 17 00:00:00 2001
From: Maxim Orlov <orlovmg@gmail.com>
Date: Thu, 24 Aug 2023 15:54:19 +0300
Subject: [PATCH v1 1/4] Add lists_free and lists_free_deep

This macro functions can handle multiple args at a time.
---
 src/include/c.h             | 10 ++++++++++
 src/include/nodes/pg_list.h |  5 +++++
 2 files changed, 15 insertions(+)

diff --git a/src/include/c.h b/src/include/c.h
index 82f8e9d4c7..3f8cee0361 100644
--- a/src/include/c.h
+++ b/src/include/c.h
@@ -1108,6 +1108,16 @@ extern void ExceptionalCondition(const char *conditionName,
 #define FLOAT8_FITS_IN_INT64(num) \
 	((num) >= (float8) PG_INT64_MIN && (num) < -((float8) PG_INT64_MIN))
 
+/*
+ * TODO: add comment
+ */
+#define func_vectorizor(func, ...) \
+do { \
+	void  *end = (int[]) {0}; \
+	void **args = (void *[]) {__VA_ARGS__, end}; \
+	for (int i = 0; args[i] != end; i++) \
+		func(args[i]); \
+} while (0)
 
 /* ----------------------------------------------------------------
  *				Section 8:	random stuff
diff --git a/src/include/nodes/pg_list.h b/src/include/nodes/pg_list.h
index 529a382d28..daae56ec32 100644
--- a/src/include/nodes/pg_list.h
+++ b/src/include/nodes/pg_list.h
@@ -621,6 +621,11 @@ extern void list_deduplicate_oid(List *list);
 extern void list_free(List *list);
 extern void list_free_deep(List *list);
 
+#define lists_free(...) \
+	func_vectorizor(list_free, __VA_ARGS__)
+#define lists_free_deep(...) \
+	func_vectorizor(list_free_deep, __VA_ARGS__)
+
 extern pg_nodiscard List *list_copy(const List *oldlist);
 extern pg_nodiscard List *list_copy_head(const List *oldlist, int len);
 extern pg_nodiscard List *list_copy_tail(const List *oldlist, int nskip);
-- 
2.41.0

v1-0003-Make-use-of-vectorized-bmss_free-function.patchapplication/octet-stream; name=v1-0003-Make-use-of-vectorized-bmss_free-function.patchDownload
From 922bef2bb73f22b215e07baf2550998cd8d6694b Mon Sep 17 00:00:00 2001
From: Maxim Orlov <orlovmg@gmail.com>
Date: Thu, 24 Aug 2023 16:28:50 +0300
Subject: [PATCH v1 3/4] Make use of vectorized bmss_free function

---
 src/backend/access/heap/heapam.c        | 16 ++++------------
 src/backend/commands/publicationcmds.c  |  3 +--
 src/backend/executor/nodeAppend.c       |  3 +--
 src/backend/optimizer/path/indxpath.c   |  3 +--
 src/backend/optimizer/path/joinpath.c   |  3 +--
 src/backend/statistics/extended_stats.c |  3 +--
 src/backend/utils/cache/relcache.c      | 21 ++++++---------------
 src/include/nodes/bitmapset.h           |  3 +++
 8 files changed, 18 insertions(+), 37 deletions(-)

diff --git a/src/backend/access/heap/heapam.c b/src/backend/access/heap/heapam.c
index 6a66214a58..dba20614f8 100644
--- a/src/backend/access/heap/heapam.c
+++ b/src/backend/access/heap/heapam.c
@@ -3372,12 +3372,8 @@ l2:
 			ReleaseBuffer(vmbuffer);
 		*update_indexes = TU_None;
 
-		bms_free(hot_attrs);
-		bms_free(sum_attrs);
-		bms_free(key_attrs);
-		bms_free(id_attrs);
-		bms_free(modified_attrs);
-		bms_free(interesting_attrs);
+		bmss_free(hot_attrs, sum_attrs, key_attrs, id_attrs, modified_attrs,
+				  interesting_attrs);
 		return result;
 	}
 
@@ -3887,12 +3883,8 @@ l2:
 	if (old_key_tuple != NULL && old_key_copied)
 		heap_freetuple(old_key_tuple);
 
-	bms_free(hot_attrs);
-	bms_free(sum_attrs);
-	bms_free(key_attrs);
-	bms_free(id_attrs);
-	bms_free(modified_attrs);
-	bms_free(interesting_attrs);
+	bmss_free(hot_attrs, sum_attrs, key_attrs, id_attrs, modified_attrs,
+			  interesting_attrs);
 
 	return TM_Ok;
 }
diff --git a/src/backend/commands/publicationcmds.c b/src/backend/commands/publicationcmds.c
index e4fe6cc72a..3dbbf878b1 100644
--- a/src/backend/commands/publicationcmds.c
+++ b/src/backend/commands/publicationcmds.c
@@ -431,8 +431,7 @@ pub_collist_contains_invalid_column(Oid pubid, Relation relation, List *ancestor
 			}
 		}
 
-		bms_free(idattrs);
-		bms_free(columns);
+		bmss_free(idattrs, columns);
 	}
 
 	ReleaseSysCache(tuple);
diff --git a/src/backend/executor/nodeAppend.c b/src/backend/executor/nodeAppend.c
index 609df6b9e6..4eef4cb53a 100644
--- a/src/backend/executor/nodeAppend.c
+++ b/src/backend/executor/nodeAppend.c
@@ -418,9 +418,8 @@ ExecReScanAppend(AppendState *node)
 					node->as_prune_state->execparamids))
 	{
 		node->as_valid_subplans_identified = false;
-		bms_free(node->as_valid_subplans);
+		bmss_free(node->as_valid_subplans, node->as_valid_asyncplans);
 		node->as_valid_subplans = NULL;
-		bms_free(node->as_valid_asyncplans);
 		node->as_valid_asyncplans = NULL;
 	}
 
diff --git a/src/backend/optimizer/path/indxpath.c b/src/backend/optimizer/path/indxpath.c
index 6a93d767a5..d840fb5e10 100644
--- a/src/backend/optimizer/path/indxpath.c
+++ b/src/backend/optimizer/path/indxpath.c
@@ -1847,8 +1847,7 @@ check_index_only(RelOptInfo *rel, IndexOptInfo *index)
 	/* Do we have all the necessary attributes? */
 	result = bms_is_subset(attrs_used, index_canreturn_attrs);
 
-	bms_free(attrs_used);
-	bms_free(index_canreturn_attrs);
+	bmss_free(attrs_used, index_canreturn_attrs);
 
 	return result;
 }
diff --git a/src/backend/optimizer/path/joinpath.c b/src/backend/optimizer/path/joinpath.c
index f9fd89e734..6b597fc812 100644
--- a/src/backend/optimizer/path/joinpath.c
+++ b/src/backend/optimizer/path/joinpath.c
@@ -412,8 +412,7 @@ have_unsafe_outer_join_ref(PlannerInfo *root,
 	}
 
 	/* Waste no memory when we reject a path here */
-	bms_free(unsatisfied);
-	bms_free(satisfied);
+	bmss_free(unsatisfied, satisfied);
 
 	return result;
 }
diff --git a/src/backend/statistics/extended_stats.c b/src/backend/statistics/extended_stats.c
index 9f67a57724..2af13b9e3e 100644
--- a/src/backend/statistics/extended_stats.c
+++ b/src/backend/statistics/extended_stats.c
@@ -1261,8 +1261,7 @@ choose_best_statistics(List *stats, char requiredkind, bool inh,
 
 		num_matched = bms_num_members(matched_attnums) + bms_num_members(matched_exprs);
 
-		bms_free(matched_attnums);
-		bms_free(matched_exprs);
+		bmss_free(matched_attnums, matched_exprs);
 
 		/*
 		 * save the actual number of keys in the stats so that we can choose
diff --git a/src/backend/utils/cache/relcache.c b/src/backend/utils/cache/relcache.c
index 1f813e5946..adef3eee55 100644
--- a/src/backend/utils/cache/relcache.c
+++ b/src/backend/utils/cache/relcache.c
@@ -2440,11 +2440,8 @@ RelationDestroyRelation(Relation relation, bool remember_tupdesc)
 	FreeTriggerDesc(relation->trigdesc);
 	list_free_deep(relation->rd_fkeylist);
 	lists_free(relation->rd_indexlist, relation->rd_statlist);
-	bms_free(relation->rd_keyattr);
-	bms_free(relation->rd_pkattr);
-	bms_free(relation->rd_idattr);
-	bms_free(relation->rd_hotblockingattr);
-	bms_free(relation->rd_summarizedattr);
+	bmss_free(relation->rd_keyattr, relation->rd_pkattr, relation->rd_idattr,
+			  relation->rd_hotblockingattr, relation->rd_summarizedattr);
 	if (relation->rd_pubdesc)
 		pfree(relation->rd_pubdesc);
 	if (relation->rd_options)
@@ -5370,26 +5367,20 @@ restart:
 	{
 		/* Gotta do it over ... might as well not leak memory */
 		lists_free(newindexoidlist, indexoidlist);
-		bms_free(uindexattrs);
-		bms_free(pkindexattrs);
-		bms_free(idindexattrs);
-		bms_free(hotblockingattrs);
-		bms_free(summarizedattrs);
+		bmss_free(uindexattrs, pkindexattrs, idindexattrs, hotblockingattrs,
+				  summarizedattrs);
 
 		goto restart;
 	}
 
 	/* Don't leak the old values of these bitmaps, if any */
 	relation->rd_attrsvalid = false;
-	bms_free(relation->rd_keyattr);
+	bmss_free(relation->rd_keyattr, relation->rd_pkattr, relation->rd_idattr,
+			  relation->rd_hotblockingattr, relation->rd_summarizedattr);
 	relation->rd_keyattr = NULL;
-	bms_free(relation->rd_pkattr);
 	relation->rd_pkattr = NULL;
-	bms_free(relation->rd_idattr);
 	relation->rd_idattr = NULL;
-	bms_free(relation->rd_hotblockingattr);
 	relation->rd_hotblockingattr = NULL;
-	bms_free(relation->rd_summarizedattr);
 	relation->rd_summarizedattr = NULL;
 
 	/*
diff --git a/src/include/nodes/bitmapset.h b/src/include/nodes/bitmapset.h
index 14de6a9ff1..0c1bdbb4c1 100644
--- a/src/include/nodes/bitmapset.h
+++ b/src/include/nodes/bitmapset.h
@@ -84,6 +84,9 @@ extern int	bms_compare(const Bitmapset *a, const Bitmapset *b);
 extern Bitmapset *bms_make_singleton(int x);
 extern void bms_free(Bitmapset *a);
 
+#define bmss_free(...) \
+	func_vectorizor(bms_free, __VA_ARGS__)
+
 extern Bitmapset *bms_union(const Bitmapset *a, const Bitmapset *b);
 extern Bitmapset *bms_intersect(const Bitmapset *a, const Bitmapset *b);
 extern Bitmapset *bms_difference(const Bitmapset *a, const Bitmapset *b);
-- 
2.41.0

#2Chapman Flack
chap@anastigmatix.net
In reply to: Maxim Orlov (#1)
Re: Vectorization of some functions and improving pg_list interface

On 2023-08-24 10:07, Maxim Orlov wrote:

1) Why do I need to specify the number of elements in the list in the
function name?

This is reminding me of something someone (Tom?) worked on sort of
recently.

Ah, yes:
https://git.postgresql.org/gitweb/?p=postgresql.git;a=commit;h=1cff1b9

I wasn't following closely, but that and the discussion link
may answer some questions.

Regards,
-Chap

#3Yura Sokolov
y.sokolov@postgrespro.ru
In reply to: Maxim Orlov (#1)
Re: Vectorization of some functions and improving pg_list interface

24.08.2023 17:07, Maxim Orlov wrote:

Hi!

Recently, I've been playing around with pg_lists and realize how
annoying (maybe, I was a bit tired) some stuff related to the lists.
For an example, see this code
List     *l1 = list_make4(1, 2, 3, 4),
         *l2 = list_make4(5, 6, 7, 8),
         *l3 = list_make4(9, 0, 1, 2);
ListCell *lc1, *lc2, *lc3;

forthree(lc1, l1, lc2, l2, lc3, l3) {
...
}

list_free(l1);
list_free(l2);
list_free(l3);

There are several questions:
1) Why do I need to specify the number of elements in the list in the
function name?
Compiler already knew how much arguments do I use.
2) Why I have to call free for every list? I don't know how to call it
right, for now I call it vectorization.
    Why not to use simple wrapper to "vectorize" function args?

So, my proposal is:
1) Add a simple macro to "vectorize" functions.
2) Use this macro to "vectorize" list_free and list_free_deep functions.
3) Use this macro to "vectorize" bms_free function.
4) "Vectorize" list_makeN functions.

For this V1 version, I do not remove all list_makeN calls in order to
reduce diff, but I'll address
this in future, if it will be needed.

In my view, one thing still waiting to be improved if foreach loop. It
is not very handy to have a bunch of
similar calls foreach, forboth, forthree and etc. It will be ideal to
have single foreach interface, but I don't know how
to do it without overall interface of the loop.

Any opinions are very welcome!

Given use case doesn't assume "zero" arguments, it is possible to
implement "lists_free" with just macro expansion (following code is not
checked, but close to valid):

#define VA_FOR_EACH(invoke, join, ...) \
CppConcat(VA_FOR_EACH_, VA_ARGS_NARGS(__VA_ARGS__))( \
invoke, join, __VA_ARGS__)
#define VA_FOR_EACH_1(invoke, join, a1) \
invoke(a1)
#define VA_FOR_EACH_2(invoke, join, a1, a2) \
invoke(a1) join() invoke(a2)
#define VA_FOR_EACH_3(invoke, join, a1, a2, a3) \
invoke(a1) join() invoke(a2) join() invoke(a3)
... up to 63 args

#define VA_SEMICOLON() ;

#define lists_free(...) \
VA_FOR_EACH(list_free, VA_SEMICOLON, __VA_ARGS__)

#define lists_free_deep(...) \
VA_FOR_EACH(list_free_deep, VA_SEMICOLON, __VA_ARGS__)

There could be couple of issues with msvc, but they are solvable.

------

Regards,
Yura

#4Yura Sokolov
y.sokolov@postgrespro.ru
In reply to: Yura Sokolov (#3)
Re: Vectorization of some functions and improving pg_list interface

06.09.2023 13:24, Yura Sokolov wrote:

24.08.2023 17:07, Maxim Orlov wrote:

Hi!

Recently, I've been playing around with pg_lists and realize how
annoying (maybe, I was a bit tired) some stuff related to the lists.
For an example, see this code
List     *l1 = list_make4(1, 2, 3, 4),
          *l2 = list_make4(5, 6, 7, 8),
          *l3 = list_make4(9, 0, 1, 2);
ListCell *lc1, *lc2, *lc3;

forthree(lc1, l1, lc2, l2, lc3, l3) {
...
}

list_free(l1);
list_free(l2);
list_free(l3);

There are several questions:
1) Why do I need to specify the number of elements in the list in the
function name?
Compiler already knew how much arguments do I use.
2) Why I have to call free for every list? I don't know how to call it
right, for now I call it vectorization.
     Why not to use simple wrapper to "vectorize" function args?

So, my proposal is:
1) Add a simple macro to "vectorize" functions.
2) Use this macro to "vectorize" list_free and list_free_deep functions.
3) Use this macro to "vectorize" bms_free function.
4) "Vectorize" list_makeN functions.

For this V1 version, I do not remove all list_makeN calls in order to
reduce diff, but I'll address
this in future, if it will be needed.

In my view, one thing still waiting to be improved if foreach loop. It
is not very handy to have a bunch of
similar calls foreach, forboth, forthree and etc. It will be ideal to
have single foreach interface, but I don't know how
to do it without overall interface of the loop.

Any opinions are very welcome!

Given use case doesn't assume "zero" arguments, it is possible to
implement "lists_free" with just macro expansion (following code is not
checked, but close to valid):

#define VA_FOR_EACH(invoke, join, ...) \
    CppConcat(VA_FOR_EACH_, VA_ARGS_NARGS(__VA_ARGS__))( \
        invoke, join, __VA_ARGS__)
#define VA_FOR_EACH_1(invoke, join, a1) \
    invoke(a1)
#define VA_FOR_EACH_2(invoke, join, a1, a2) \
    invoke(a1) join() invoke(a2)
#define VA_FOR_EACH_3(invoke, join, a1, a2, a3) \
    invoke(a1) join() invoke(a2) join() invoke(a3)
... up to 63 args

#define VA_SEMICOLON() ;

#define lists_free(...) \
    VA_FOR_EACH(list_free, VA_SEMICOLON, __VA_ARGS__)

#define lists_free_deep(...) \
    VA_FOR_EACH(list_free_deep, VA_SEMICOLON, __VA_ARGS__)

There could be couple of issues with msvc, but they are solvable.

Given we could use C99 compound literals, list contruction could be
implemented without C vaarg functions as well

List *
list_make_impl(NodeTag t, int n, ListCell *datums)
{
List *list = new_list(t, n);
memcpy(list->elements, datums, sizeof(ListCell)*n);
return list;
}

#define VA_COMMA() ,

#define list_make__m(Tag, type, ...) \
list_make_impl(Tag, VA_ARGS_NARGS(__VA_ARGS__), \
((ListCell[]){ \
VA_FOR_EACH(list_make_##type##_cell, VA_COMMA, __VA_ARGS__) \
}))

#define list_make(...) list_make__m(T_List, ptr, __VA_ARGS__)
#define list_make_int(...) list_make__m(T_IntList, int, __VA_ARGS__)
#define list_make_oid(...) list_make__m(T_OidList, oid, __VA_ARGS__)
#define list_make_xid(...) list_make__m(T_XidList, xid, __VA_ARGS__)

(code is not checked)

If zero arguments (no arguments) should be supported, it is tricky
because of mvsc, but solvable.