Add minimal C example and SQL registration example for custom table access methods.

Started by Phil Eatonabout 2 years ago15 messages
#1Phil Eaton
phil@eatonphil.com
1 attachment(s)

Hello!

I was looking into table access methods recently and found the
existing page a bit sparse. Here's a small patch adding a little more
example code to the table access methods page.

Let me know if there's anything I can do to fix my patch up!

Cheers,
Phil

Attachments:

v1-0001-Add-minimal-C-example-and-SQL-registration-exampl.patchapplication/octet-stream; name=v1-0001-Add-minimal-C-example-and-SQL-registration-exampl.patchDownload
From 383d7aff0100023c89cf66cf78b3b5eceba5da83 Mon Sep 17 00:00:00 2001
From: Phil Eaton <phil@eatonphil.com>
Date: Thu, 2 Nov 2023 16:15:42 +0000
Subject: [PATCH v1] Add minimal C example and SQL registration example for
 custom table access methods.

---
 doc/src/sgml/tableam.sgml | 53 +++++++++++++++++++++++++++++++++++----
 1 file changed, 48 insertions(+), 5 deletions(-)

diff --git a/doc/src/sgml/tableam.sgml b/doc/src/sgml/tableam.sgml
index 6a6eb2b766..8f30eaaf2b 100644
--- a/doc/src/sgml/tableam.sgml
+++ b/doc/src/sgml/tableam.sgml
@@ -35,13 +35,56 @@
   argument of type <type>internal</type> and to return the pseudo-type
   <type>table_am_handler</type>.  The argument is a dummy value that simply
   serves to prevent handler functions from being called directly from SQL commands.
+ </para>
+
+ <para>
+  Here is an example of how to register an extension that provides a
+  table access method handler:
+ </para>
+
+<screen>
+CREATE OR REPLACE FUNCTION my_tableam_handler(internal)
+RETURNS table_am_handler AS 'myextension', 'my_tableam_handler'
+LANGUAGE C STRICT;
 
+CREATE ACCESS METHOD myam TYPE TABLE HANDLER my_tableam_handler;
+</screen>
+
+ <para>
   The result of the function must be a pointer to a struct of type
-  <structname>TableAmRoutine</structname>, which contains everything that the
-  core code needs to know to make use of the table access method. The return
-  value needs to be of server lifetime, which is typically achieved by
-  defining it as a <literal>static const</literal> variable in global
-  scope. The <structname>TableAmRoutine</structname> struct, also called the
+  <structname>TableAmRoutine</structname>, which contains everything
+  that the core code needs to know to make use of the table access
+  method. The return value needs to be of server lifetime, which is
+  typically achieved by defining it as a <literal>static
+  const</literal> variable in global scope.
+ </para>
+
+ <para>
+  Here is what <filename>myextension.c</filename> with the table
+  access method handler might look like:
+ </para>
+
+<screen>
+#include "postgres.h"
+#include "fmgr.h"
+#include "access/tableam.h"
+
+PG_MODULE_MAGIC;
+
+static const TableAmRoutine my_am_methods = {
+  .type = T_TableAmRoutine,
+  /* Methods from TableAmRoutine omitted from example, but all
+     non-optional ones must be provided here. */
+};
+
+PG_FUNCTION_INFO_V1(my_tableam_handler);
+Datum my_tableam_handler(PG_FUNCTION_ARGS) {
+  PG_RETURN_POINTER(&#038;my_am_methods);
+}
+</screen>
+
+ <para>
+  The <structname>TableAmRoutine</structname> struct, also called the
   access method's <firstterm>API struct</firstterm>, defines the behavior of
   the access method using callbacks. These callbacks are pointers to plain C
   functions and are not visible or callable at the SQL level. All the
-- 
2.41.0

#2Paul A Jungwirth
pj@illuminatedcomputing.com
In reply to: Phil Eaton (#1)
Re: Add minimal C example and SQL registration example for custom table access methods.

On Sat, Nov 11, 2023 at 1:00 PM Phil Eaton <phil@eatonphil.com> wrote:

I was looking into table access methods recently and found the
existing page a bit sparse. Here's a small patch adding a little more
example code to the table access methods page.

I agree this is a helpful addition for people exploring table AMs. The
patch applies and builds. No spelling/grammar errors. Integrates
smoothly with the surrounding text.

I didn't write a full table AM (maybe someday! :-) but I put the
example code into a new extension and made sure it builds. I don't
think this snippet is likely to go out of date, but is there anything
we do in doc examples to test that? (I'm not aware of anything.)

There is no commitfest entry yet. Phil, would you mind adding one? (It
will need to be against the Jan 2024 commitfest.) I started one myself
but I don't think you're registered yet so I couldn't enter you as the
patch author. Let me know if you have any trouble.

+1 from me!

Regards,
Paul

#3Roberto Mello
roberto.mello@gmail.com
In reply to: Paul A Jungwirth (#2)
Re: Add minimal C example and SQL registration example for custom table access methods.

Suggestion:

In the C example you added you mention in the comment:

+  /* Methods from TableAmRoutine omitted from example, but all
+     non-optional ones must be provided here. */

Perhaps you could provide a "see <xyz>" to point the reader finding your example where he could find these non-optional methods he must provide?

Nitpicking a little: your patch appears to change more lines than it does, because it added line breaks earlier in the lines. I would generally avoid that unless there's good reason to do so.

#4Tristen Raab
tristen.raab@highgo.ca
In reply to: Roberto Mello (#3)
Re: Add minimal C example and SQL registration example for custom table access methods.

The following review has been posted through the commitfest application:
make installcheck-world: tested, passed
Implements feature: not tested
Spec compliant: not tested
Documentation: tested, passed

Hello,

I've reviewed your patch and it applies correctly and the documentation builds without any error. The built documentation also looks good with no formatting errors. It's always great to see more examples when reading through documentation so I think this patch is a good addition.

thanks,

-----------------------
Tristen Raab
Highgo Software Canada
www.highgo.ca

#5Fabrízio de Royes Mello
fabriziomello@gmail.com
In reply to: Roberto Mello (#3)
Re: Add minimal C example and SQL registration example for custom table access methods.

On Wed, Nov 15, 2023 at 8:29 PM Roberto Mello <roberto.mello@gmail.com>
wrote:

Suggestion:

In the C example you added you mention in the comment:

+  /* Methods from TableAmRoutine omitted from example, but all
+     non-optional ones must be provided here. */

Perhaps you could provide a "see <xyz>" to point the reader finding your

example where he could find these non-optional methods he must provide?

Nitpicking a little: your patch appears to change more lines than it

does, because it added line breaks earlier in the lines. I would generally
avoid that unless there's good reason to do so.

Hey folks,

There is a previous patch [1]https://commitfest.postgresql.org/46/4588/ around the same topic. What about joining
efforts on pointing these documentation changes to the proposed test module?

[1]: https://commitfest.postgresql.org/46/4588/

--
Fabrízio de Royes Mello

#6Robert Haas
robertmhaas@gmail.com
In reply to: Fabrízio de Royes Mello (#5)
Re: Add minimal C example and SQL registration example for custom table access methods.

On Fri, Jan 26, 2024 at 3:03 PM Fabrízio de Royes Mello
<fabriziomello@gmail.com> wrote:

On Wed, Nov 15, 2023 at 8:29 PM Roberto Mello <roberto.mello@gmail.com> wrote:

Suggestion:

In the C example you added you mention in the comment:

+  /* Methods from TableAmRoutine omitted from example, but all
+     non-optional ones must be provided here. */

Perhaps you could provide a "see <xyz>" to point the reader finding your example where he could find these non-optional methods he must provide?

Nitpicking a little: your patch appears to change more lines than it does, because it added line breaks earlier in the lines. I would generally avoid that unless there's good reason to do so.

Hey folks,

There is a previous patch [1] around the same topic. What about joining efforts on pointing these documentation changes to the proposed test module?

[1] https://commitfest.postgresql.org/46/4588/

Looking over this thread, I see that it was moved from pgsql-docs to
pgsql-hackers while at the same time dropping the original poster from
the Cc list. That seems rather unfortunate. I suspect there's a pretty
good chance that Phil Eaton hasn't seen any of the replies other than
the first one from Paul Jungwirth, which is also the only one that
didn't ask for anything to be changed.

Re-adding Phil. Phil, you should have a look over
/messages/by-id/CAByiw+r+CS-ojBDP7Dm=9YeOLkZTXVnBmOe_ajK=en8C_zB3_g@mail.gmail.com
and respond to the various emails and probably update the patch
somehow. Note that feature freeze is in 2 weeks, so if we can't reach
agreement on what is to be done here soon, this will have to wait for
the next cycle, or later.

--
Robert Haas
EDB: http://www.enterprisedb.com

#7Phil Eaton
phil@eatonphil.com
In reply to: Robert Haas (#6)
1 attachment(s)
Re: Add minimal C example and SQL registration example for custom table access methods.

Thanks Robert for mentioning this! I indeed did not notice the switch.

Nitpicking a little: your patch appears to change more lines than it does, because it added line breaks earlier in the lines. I would generally avoid that unless there's good reason to do so.

Thanks! I'm not sure why that happened since I normally run
fill-region in emacs and when I re-ran it now, it looked as it used
to. I've fixed it up in this patch.

Perhaps you could provide a "see <xyz>" to point the reader finding your example where he could find these non-optional methods he must provide?

Since the responses were positive, I've taken the liberty to extend
the sample code by simply including all the stub methods and the full
struct. Marking which methods are optional and not.

If that looks like too much, I can revert back. Perhaps only
mentioning the struct like we do for the index AM here:
https://www.postgresql.org/docs/current/index-api.html. However, as a
reader, I feel like the full stubs are a bit more useful.

Happy for feedback. Updated patch is attached.

Cheers,
Phil

Show quoted text

On Fri, Mar 22, 2024 at 1:40 PM Robert Haas <robertmhaas@gmail.com> wrote:

On Fri, Jan 26, 2024 at 3:03 PM Fabrízio de Royes Mello
<fabriziomello@gmail.com> wrote:

On Wed, Nov 15, 2023 at 8:29 PM Roberto Mello <roberto.mello@gmail.com> wrote:

Suggestion:

In the C example you added you mention in the comment:

+  /* Methods from TableAmRoutine omitted from example, but all
+     non-optional ones must be provided here. */

Perhaps you could provide a "see <xyz>" to point the reader finding your example where he could find these non-optional methods he must provide?

Nitpicking a little: your patch appears to change more lines than it does, because it added line breaks earlier in the lines. I would generally avoid that unless there's good reason to do so.

Hey folks,

There is a previous patch [1] around the same topic. What about joining efforts on pointing these documentation changes to the proposed test module?

[1] https://commitfest.postgresql.org/46/4588/

Looking over this thread, I see that it was moved from pgsql-docs to
pgsql-hackers while at the same time dropping the original poster from
the Cc list. That seems rather unfortunate. I suspect there's a pretty
good chance that Phil Eaton hasn't seen any of the replies other than
the first one from Paul Jungwirth, which is also the only one that
didn't ask for anything to be changed.

Re-adding Phil. Phil, you should have a look over
/messages/by-id/CAByiw+r+CS-ojBDP7Dm=9YeOLkZTXVnBmOe_ajK=en8C_zB3_g@mail.gmail.com
and respond to the various emails and probably update the patch
somehow. Note that feature freeze is in 2 weeks, so if we can't reach
agreement on what is to be done here soon, this will have to wait for
the next cycle, or later.

--
Robert Haas
EDB: http://www.enterprisedb.com

Attachments:

v2-0001-Add-boilerplate-C-code-and-SQL-registration-examp.patchapplication/octet-stream; name=v2-0001-Add-boilerplate-C-code-and-SQL-registration-examp.patchDownload
From 83a4138f6e3bb7fa80b32b1a7f7b579e327e9489 Mon Sep 17 00:00:00 2001
From: Phil Eaton <phil@eatonphil.com>
Date: Fri, 3 May 2024 12:08:55 -0400
Subject: [PATCH v2] Add boilerplate C code and SQL registration example for
 custom table access methods.

---
 doc/src/sgml/tableam.sgml | 353 +++++++++++++++++++++++++++++++++++++-
 1 file changed, 351 insertions(+), 2 deletions(-)

diff --git a/doc/src/sgml/tableam.sgml b/doc/src/sgml/tableam.sgml
index 4b37f2e5a6..3d91407ae4 100644
--- a/doc/src/sgml/tableam.sgml
+++ b/doc/src/sgml/tableam.sgml
@@ -35,13 +35,362 @@
   argument of type <type>internal</type> and to return the pseudo-type
   <type>table_am_handler</type>.  The argument is a dummy value that simply
   serves to prevent handler functions from being called directly from SQL commands.
+ </para>
+
+ <para>
+  Here is an example of how to register an extension that provides a
+  table access method handler:
+ </para>
+
+<screen>
+CREATE OR REPLACE FUNCTION my_tableam_handler(internal)
+RETURNS table_am_handler AS 'myextension', 'my_tableam_handler'
+LANGUAGE C STRICT;
 
+CREATE ACCESS METHOD myam TYPE TABLE HANDLER my_tableam_handler;
+</screen>
+
+ <para>
   The result of the function must be a pointer to a struct of type
   <structname>TableAmRoutine</structname>, which contains everything that the
   core code needs to know to make use of the table access method. The return
   value needs to be of server lifetime, which is typically achieved by
-  defining it as a <literal>static const</literal> variable in global
-  scope. The <structname>TableAmRoutine</structname> struct, also called the
+  defining it as a <literal>static const</literal> variable in global scope.
+ </para>
+
+ <para>
+  Here is what <filename>myextension.c</filename> with the table
+  access method handler might look like:
+ </para>
+
+<screen>
+#include "postgres.h"
+#include "fmgr.h"
+#include "access/tableam.h"
+#include "access/heapam.h"
+#include "nodes/execnodes.h"
+#include "catalog/index.h"
+#include "commands/vacuum.h"
+#include "utils/builtins.h"
+#include "executor/tuptable.h"
+
+PG_MODULE_MAGIC;
+
+const TableAmRoutine my_am_methods;
+
+static const TupleTableSlotOps* my_am_slot_callbacks(Relation relation)
+{
+        return NULL;
+}
+
+static TableScanDesc my_am_beginscan(Relation relation,
+                                     Snapshot snapshot,
+                                     int nkeys,
+                                     struct ScanKeyData *key,
+                                     ParallelTableScanDesc parallel_scan,
+                                     uint32 flags)
+{
+        return NULL;
+}
+
+static void my_am_rescan(TableScanDesc sscan,
+                         struct ScanKeyData *key,
+                         bool set_params,
+                         bool allow_strat,
+                         bool allow_sync,
+                         bool allow_pagemode) {}
+
+static void my_am_endscan(TableScanDesc sscan) {}
+
+static bool my_am_getnextslot(TableScanDesc sscan,
+                              ScanDirection direction,
+                              TupleTableSlot *slot)
+{
+        return false;
+}
+
+static IndexFetchTableData* my_am_index_fetch_begin(Relation rel)
+{
+        return NULL;
+}
+
+static void my_am_index_fetch_reset(IndexFetchTableData *scan) {}
+
+static void my_am_index_fetch_end(IndexFetchTableData *scan) {}
+
+static bool my_am_index_fetch_tuple(struct IndexFetchTableData *scan,
+                                    ItemPointer tid,
+                                    Snapshot snapshot,
+                                    TupleTableSlot *slot,
+                                    bool *call_again,
+                                    bool *all_dead)
+{
+        return false;
+}
+
+static void my_am_tuple_insert(Relation relation,
+                               TupleTableSlot *slot,
+                               CommandId cid,
+                               int options,
+                               BulkInsertState bistate) {}
+
+static void my_am_tuple_insert_speculative(Relation relation,
+                                           TupleTableSlot *slot,
+                                           CommandId cid,
+                                           int options,
+                                           BulkInsertState bistate,
+                                           uint32 specToken) {}
+
+static void my_am_tuple_complete_speculative(Relation relation,
+                                             TupleTableSlot *slot,
+                                             uint32 specToken,
+                                             bool succeeded) {}
+
+static void my_am_multi_insert(Relation relation,
+                               TupleTableSlot **slots,
+                               int ntuples,
+                               CommandId cid,
+                               int options,
+                               BulkInsertState bistate) {}
+
+static TM_Result my_am_tuple_delete(Relation relation,
+                                    ItemPointer tid,
+                                    CommandId cid,
+                                    Snapshot snapshot,
+                                    Snapshot crosscheck,
+                                    bool wait,
+                                    TM_FailureData *tmfd,
+                                    bool changingPart)
+{
+        return TM_Ok;
+}
+
+static TM_Result my_am_tuple_update(Relation relation,
+                                    ItemPointer otid,
+                                    TupleTableSlot *slot,
+                                    CommandId cid,
+                                    Snapshot snapshot,
+                                    Snapshot crosscheck,
+                                    bool wait,
+                                    TM_FailureData *tmfd,
+                                    LockTupleMode *lockmode,
+                                    TU_UpdateIndexes *update_indexes)
+{
+        return TM_Ok;
+}
+
+static TM_Result my_am_tuple_lock(Relation relation,
+                                  ItemPointer tid,
+                                  Snapshot snapshot,
+                                  TupleTableSlot *slot,
+                                  CommandId cid,
+                                  LockTupleMode mode,
+                                  LockWaitPolicy wait_policy,
+                                  uint8 flags,
+                                  TM_FailureData *tmfd)
+{
+        return TM_Ok;
+}
+
+static bool my_am_fetch_row_version(Relation relation,
+                                    ItemPointer tid,
+                                    Snapshot snapshot,
+                                    TupleTableSlot *slot)
+{
+        return false;
+}
+
+static void my_am_get_latest_tid(TableScanDesc sscan, ItemPointer tid) {}
+
+static bool my_am_tuple_tid_valid(TableScanDesc scan, ItemPointer tid)
+{
+        return false;
+}
+
+static bool my_am_tuple_satisfies_snapshot(Relation rel,
+                                           TupleTableSlot *slot,
+                                           Snapshot snapshot)
+{
+        return false;
+}
+
+static TransactionId my_am_index_delete_tuples(Relation rel,
+                                               TM_IndexDeleteOp *delstate)
+{
+        return 0;
+}
+
+static void my_am_relation_set_new_filelocator(Relation rel,
+                                               const RelFileLocator *newrlocator,
+                                               char persistence,
+                                               TransactionId *freezeXid,
+                                               MultiXactId *minmulti) {}
+
+static void my_am_relation_nontransactional_truncate(Relation rel) {}
+
+static void my_am_relation_copy_data(Relation rel,
+                                     const RelFileLocator *newrlocator) {}
+
+static void my_am_relation_copy_for_cluster(Relation OldHeap,
+                                            Relation NewHeap,
+                                            Relation OldIndex,
+                                            bool use_sort,
+                                            TransactionId OldestXmin,
+                                            TransactionId *xid_cutoff,
+                                            MultiXactId *multi_cutoff,
+                                            double *num_tuples,
+                                            double *tups_vacuumed,
+                                            double *tups_recently_dead) {}
+
+static void my_am_vacuum_rel(Relation rel,
+                             VacuumParams *params,
+                             BufferAccessStrategy bstrategy) {}
+
+static bool my_am_scan_analyze_next_block(TableScanDesc scan,
+                                          ReadStream *stream)
+{
+        return false;
+}
+
+static bool my_am_scan_analyze_next_tuple(TableScanDesc scan,
+                                          TransactionId OldestXmin,
+                                          double *liverows,
+                                          double *deadrows,
+                                          TupleTableSlot *slot)
+{
+        return false;
+}
+
+static double my_am_index_build_range_scan(Relation heapRelation,
+                                           Relation indexRelation,
+                                           IndexInfo *indexInfo,
+                                           bool allow_sync,
+                                           bool anyvisible,
+                                           bool progress,
+                                           BlockNumber start_blockno,
+                                           BlockNumber numblocks,
+                                           IndexBuildCallback callback,
+                                           void *callback_state,
+                                           TableScanDesc scan)
+{
+        return 0;
+}
+
+static void my_am_index_validate_scan(Relation heapRelation,
+                                      Relation indexRelation,
+                                      IndexInfo *indexInfo,
+                                      Snapshot snapshot,
+                                      ValidateIndexState *state) {}
+
+static bool my_am_relation_needs_toast_table(Relation rel)
+{
+        return false;
+}
+
+static Oid my_am_relation_toast_am(Relation rel)
+{
+        return InvalidOid;
+}
+
+static void my_am_fetch_toast_slice(Relation toastrel,
+                                    Oid valueid,
+                                    int32 attrsize,
+                                    int32 sliceoffset,
+                                    int32 slicelength,
+                                    struct varlena *result) {}
+
+static void my_am_estimate_rel_size(Relation rel,
+                                    int32 *attr_widths,
+                                    BlockNumber *pages,
+                                    double *tuples,
+                                    double *allvisfrac) {}
+
+static bool my_am_scan_sample_next_block(TableScanDesc scan,
+                                         SampleScanState *scanstate)
+{
+        return false;
+}
+
+static bool my_am_scan_sample_next_tuple(TableScanDesc scan,
+                                         SampleScanState *scanstate,
+                                         TupleTableSlot *slot)
+{
+        return false;
+}
+
+const TableAmRoutine my_am_methods = {
+        .type = T_TableAmRoutine,
+
+        /* Required TableAmRoutine methods. */
+
+        .slot_callbacks = my_am_slot_callbacks,
+
+        .scan_begin = my_am_beginscan,
+        .scan_end = my_am_endscan,
+        .scan_rescan = my_am_rescan,
+        .scan_getnextslot = my_am_getnextslot,
+
+        .parallelscan_estimate = table_block_parallelscan_estimate,
+        .parallelscan_initialize = table_block_parallelscan_initialize,
+        .parallelscan_reinitialize = table_block_parallelscan_reinitialize,
+
+        .index_fetch_begin = my_am_index_fetch_begin,
+        .index_fetch_reset = my_am_index_fetch_reset,
+        .index_fetch_end = my_am_index_fetch_end,
+        .index_fetch_tuple = my_am_index_fetch_tuple,
+
+        .tuple_insert = my_am_tuple_insert,
+        .tuple_insert_speculative = my_am_tuple_insert_speculative,
+        .tuple_complete_speculative = my_am_tuple_complete_speculative,
+        .multi_insert = my_am_multi_insert,
+        .tuple_delete = my_am_tuple_delete,
+        .tuple_update = my_am_tuple_update,
+        .tuple_lock = my_am_tuple_lock,
+
+        .tuple_fetch_row_version = my_am_fetch_row_version,
+        .tuple_get_latest_tid = my_am_get_latest_tid,
+        .tuple_tid_valid = my_am_tuple_tid_valid,
+        .tuple_satisfies_snapshot = my_am_tuple_satisfies_snapshot,
+        .index_delete_tuples = my_am_index_delete_tuples,
+
+        .relation_set_new_filelocator = my_am_relation_set_new_filelocator,
+        .relation_nontransactional_truncate = my_am_relation_nontransactional_truncate,
+        .relation_copy_data = my_am_relation_copy_data,
+        .relation_copy_for_cluster = my_am_relation_copy_for_cluster,
+        .relation_vacuum = my_am_vacuum_rel,
+        .scan_analyze_next_block = my_am_scan_analyze_next_block,
+        .scan_analyze_next_tuple = my_am_scan_analyze_next_tuple,
+        .index_build_range_scan = my_am_index_build_range_scan,
+        .index_validate_scan = my_am_index_validate_scan,
+
+        .relation_size = table_block_relation_size,
+        .relation_needs_toast_table = my_am_relation_needs_toast_table,
+        .relation_toast_am = my_am_relation_toast_am,
+        .relation_fetch_toast_slice = my_am_fetch_toast_slice,
+
+        .relation_estimate_size = my_am_estimate_rel_size,
+
+        .scan_sample_next_block = my_am_scan_sample_next_block,
+        .scan_sample_next_tuple = my_am_scan_sample_next_tuple
+
+        /*
+         * Optional TableAmRoutine methods:
+         * .finish_bulk_insert = my_am_finish_bulk_insert,
+         * .scan_set_tidrange = my_am_scan_set_tidrange,
+         * .scan_getnextslot_tidrange = my_am_scan_getnextslot_tidrange,
+         */
+};
+
+PG_FUNCTION_INFO_V1(mem_tableam_handler);
+
+Datum mem_tableam_handler(PG_FUNCTION_ARGS)
+{
+        PG_RETURN_POINTER(&#038;my_am_methods);
+}
+</screen>
+
+ <para>
+  The <structname>TableAmRoutine</structname> struct, also called the
   access method's <firstterm>API struct</firstterm>, defines the behavior of
   the access method using callbacks. These callbacks are pointers to plain C
   functions and are not visible or callable at the SQL level. All the
-- 
2.39.3 (Apple Git-146)

#8Robert Haas
robertmhaas@gmail.com
In reply to: Phil Eaton (#7)
Re: Add minimal C example and SQL registration example for custom table access methods.

On Fri, May 3, 2024 at 1:35 PM Phil Eaton <phil@eatonphil.com> wrote:

Happy for feedback. Updated patch is attached.

I took a look at this patch and I don't think this is a very good
idea, for two reasons:

1. We change the table access method interface definitions not all
that infrequently, so I think this will become out of date, and fail
to get updated.

2. Writing a table access method is really hard, and if you need this
in order to be able to attempt it, you're probably shouldn't be
attmempting it.

I wouldn't mind patching the documentation to add the SQL part of
this; that seems short enough, non-obvious enough, and sufficiently
unlikely to change that I can believe it would be a worthwhile
addition. But there have been 21 commits to tableam.h in the last 6
months and most of those would have needed to update this example, and
I think it's very likely that some of them would have forgotten it.

--
Robert Haas
EDB: http://www.enterprisedb.com

#9Phil Eaton
phil@eatonphil.com
In reply to: Robert Haas (#8)
1 attachment(s)
Re: Add minimal C example and SQL registration example for custom table access methods.

I took a look at this patch and I don't think this is a very good
idea,

No problem! I've dropped the v2 code additions and stuck with the v1
attempt plus feedback.

Thank you!

Phil

Attachments:

v3-0001-Add-minimal-C-example-and-SQL-registration-exampl.patchapplication/octet-stream; name=v3-0001-Add-minimal-C-example-and-SQL-registration-exampl.patchDownload
From 318cfaded13c1a4c674c189865c2b07aaf09972d Mon Sep 17 00:00:00 2001
From: Phil Eaton <phil.eaton@enterprisedb.com>
Date: Tue, 14 May 2024 14:50:57 -0400
Subject: [PATCH v3] Add minimal C example and SQL registration example for
 custom table access methods.

---
 doc/src/sgml/tableam.sgml | 55 +++++++++++++++++++++++++++++++++++++--
 1 file changed, 53 insertions(+), 2 deletions(-)

diff --git a/doc/src/sgml/tableam.sgml b/doc/src/sgml/tableam.sgml
index 4b37f2e5a6..df12afe73f 100644
--- a/doc/src/sgml/tableam.sgml
+++ b/doc/src/sgml/tableam.sgml
@@ -35,13 +35,64 @@
   argument of type <type>internal</type> and to return the pseudo-type
   <type>table_am_handler</type>.  The argument is a dummy value that simply
   serves to prevent handler functions from being called directly from SQL commands.
+ </para>
+
+ <para>
+  Here is an example of how to register an extension that provides a
+  table access method handler:
+ </para>
+
+<screen>
+CREATE OR REPLACE FUNCTION my_tableam_handler(internal)
+RETURNS table_am_handler AS 'myextension', 'my_tableam_handler'
+LANGUAGE C STRICT;
 
+CREATE ACCESS METHOD myam TYPE TABLE HANDLER my_tableam_handler;
+</screen>
+
+ <para>
   The result of the function must be a pointer to a struct of type
   <structname>TableAmRoutine</structname>, which contains everything that the
   core code needs to know to make use of the table access method. The return
   value needs to be of server lifetime, which is typically achieved by
-  defining it as a <literal>static const</literal> variable in global
-  scope. The <structname>TableAmRoutine</structname> struct, also called the
+  defining it as a <literal>static const</literal> variable in global scope.
+ </para>
+
+ <para>
+  Here is what <filename>myextension.c</filename> with the table
+  access method handler might look like:
+ </para>
+
+<screen>
+#include "postgres.h"
+#include "fmgr.h"
+#include "access/tableam.h"
+#include "access/heapam.h"
+#include "nodes/execnodes.h"
+#include "catalog/index.h"
+#include "commands/vacuum.h"
+#include "utils/builtins.h"
+#include "executor/tuptable.h"
+
+PG_MODULE_MAGIC;
+
+const TableAmRoutine my_am_methods = {
+  .type = T_TableAmRoutine,
+
+  /* Methods from TableAmRoutine omitted from example, but all
+     non-optional ones must be provided here. */
+};
+
+PG_FUNCTION_INFO_V1(mem_tableam_handler);
+
+Datum mem_tableam_handler(PG_FUNCTION_ARGS)
+{
+        PG_RETURN_POINTER(&#038;my_am_methods);
+}
+</screen>
+
+ <para>
+  The <structname>TableAmRoutine</structname> struct, also called the
   access method's <firstterm>API struct</firstterm>, defines the behavior of
   the access method using callbacks. These callbacks are pointers to plain C
   functions and are not visible or callable at the SQL level. All the
-- 
2.39.3 (Apple Git-146)

#10Robert Haas
robertmhaas@gmail.com
In reply to: Phil Eaton (#9)
Re: Add minimal C example and SQL registration example for custom table access methods.

On Tue, May 14, 2024 at 3:02 PM Phil Eaton <phil@eatonphil.com> wrote:

I took a look at this patch and I don't think this is a very good
idea,

No problem! I've dropped the v2 code additions and stuck with the v1
attempt plus feedback.

That looks more reasonable. I'd like to quibble with this text:

+. Here is an example of how to register an extension that provides a
+  table access method handler:

I think this should say something more like "Here is how an extension
SQL script might create a table access method handler". I'm not sure
if we have a standard term in our documentation that should be used
instead of "extension SQL script"; perhaps look for similar examples,
or the documentation of extensions themselves, and copy the wording.

Shouldn't "mem_tableam_handler" be "my_tableam_handler"?

--
Robert Haas
EDB: http://www.enterprisedb.com

#11Phil Eaton
phil@eatonphil.com
In reply to: Robert Haas (#10)
1 attachment(s)
Re: Add minimal C example and SQL registration example for custom table access methods.

I think this should say something more like "Here is how an extension
SQL script might create a table access method handler".

Fair point. It is referred to elsewhere [0]https://www.postgresql.org/docs/current/extend-extensions.html in docs as a "script
file", so I've done that.

Shouldn't "mem_tableam_handler" be "my_tableam_handler"?

Sorry about that, fixed.

[0]: https://www.postgresql.org/docs/current/extend-extensions.html

Phil

Attachments:

v4-0001-Add-minimal-C-example-and-SQL-registration-exampl.patchapplication/octet-stream; name=v4-0001-Add-minimal-C-example-and-SQL-registration-exampl.patchDownload
From 79a8058f971c55fca4696e841976815ae5fabb80 Mon Sep 17 00:00:00 2001
From: Phil Eaton <phil.eaton@enterprisedb.com>
Date: Fri, 24 May 2024 14:02:06 -0400
Subject: [PATCH v4] Add minimal C example and SQL registration example for
 custom table access methods

---
 doc/src/sgml/tableam.sgml | 55 +++++++++++++++++++++++++++++++++++++--
 1 file changed, 53 insertions(+), 2 deletions(-)

diff --git a/doc/src/sgml/tableam.sgml b/doc/src/sgml/tableam.sgml
index 4b37f2e5a6..09f0b74631 100644
--- a/doc/src/sgml/tableam.sgml
+++ b/doc/src/sgml/tableam.sgml
@@ -35,13 +35,64 @@
   argument of type <type>internal</type> and to return the pseudo-type
   <type>table_am_handler</type>.  The argument is a dummy value that simply
   serves to prevent handler functions from being called directly from SQL commands.
+ </para>
+
+ <para>
+  Here is how an extension script file might create a table access method
+  handler:
+ </para>
+
+<screen>
+CREATE OR REPLACE FUNCTION my_tableam_handler(internal)
+RETURNS table_am_handler AS 'myextension', 'my_tableam_handler'
+LANGUAGE C STRICT;
 
+CREATE ACCESS METHOD myam TYPE TABLE HANDLER my_tableam_handler;
+</screen>
+
+ <para>
   The result of the function must be a pointer to a struct of type
   <structname>TableAmRoutine</structname>, which contains everything that the
   core code needs to know to make use of the table access method. The return
   value needs to be of server lifetime, which is typically achieved by
-  defining it as a <literal>static const</literal> variable in global
-  scope. The <structname>TableAmRoutine</structname> struct, also called the
+  defining it as a <literal>static const</literal> variable in global scope.
+ </para>
+
+ <para>
+  Here is what <filename>myextension.c</filename> with the table
+  access method handler might look like:
+ </para>
+
+<screen>
+#include "postgres.h"
+#include "fmgr.h"
+#include "access/tableam.h"
+#include "access/heapam.h"
+#include "nodes/execnodes.h"
+#include "catalog/index.h"
+#include "commands/vacuum.h"
+#include "utils/builtins.h"
+#include "executor/tuptable.h"
+
+PG_MODULE_MAGIC;
+
+const TableAmRoutine my_am_methods = {
+  .type = T_TableAmRoutine,
+
+  /* Methods from TableAmRoutine omitted from example, but all
+     non-optional ones must be provided here. */
+};
+
+PG_FUNCTION_INFO_V1(my_tableam_handler);
+
+Datum my_tableam_handler(PG_FUNCTION_ARGS)
+{
+        PG_RETURN_POINTER(&#038;my_am_methods);
+}
+</screen>
+
+ <para>
+  The <structname>TableAmRoutine</structname> struct, also called the
   access method's <firstterm>API struct</firstterm>, defines the behavior of
   the access method using callbacks. These callbacks are pointers to plain C
   functions and are not visible or callable at the SQL level. All the
-- 
2.39.3 (Apple Git-146)

#12Fabrízio de Royes Mello
fabriziomello@gmail.com
In reply to: Phil Eaton (#11)
Re: Add minimal C example and SQL registration example for custom table access methods.

On Fri, May 24, 2024 at 3:11 PM Phil Eaton <phil@eatonphil.com> wrote:

I think this should say something more like "Here is how an extension
SQL script might create a table access method handler".

Fair point. It is referred to elsewhere [0] in docs as a "script
file", so I've done that.

Shouldn't "mem_tableam_handler" be "my_tableam_handler"?

Sorry about that, fixed.

[0] https://www.postgresql.org/docs/current/extend-extensions.html

Phil

Nice... LGTM!

--
Fabrízio de Royes Mello

#13Tom Lane
tgl@sss.pgh.pa.us
In reply to: Fabrízio de Royes Mello (#12)
Re: Add minimal C example and SQL registration example for custom table access methods.

I noticed that there were two CF entries pointing at this thread:

https://commitfest.postgresql.org/48/4655/
https://commitfest.postgresql.org/48/4973/

That doesn't seem helpful, so I've marked the second one "Withdrawn".

regards, tom lane

#14Michael Paquier
michael@paquier.xyz
In reply to: Fabrízio de Royes Mello (#12)
Re: Add minimal C example and SQL registration example for custom table access methods.

On Fri, May 24, 2024 at 03:59:08PM -0300, Fabrízio de Royes Mello wrote:

Nice... LGTM!

I have noticed that this was still in the CF. After fixing a couple
of inconsistencies in the markups and the names, trimming down the
list of headers to avoid rot and adding a static in from of the const,
the result looked pretty much OK, so applied.
--
Michael

#15Phil Eaton
phil@eatonphil.com
In reply to: Michael Paquier (#14)
Re: Add minimal C example and SQL registration example for custom table access methods.

Glad to hear it. Thank you!

Show quoted text

On Mon, Oct 7, 2024 at 2:50 AM Michael Paquier <michael@paquier.xyz> wrote:

On Fri, May 24, 2024 at 03:59:08PM -0300, Fabrízio de Royes Mello wrote:

Nice... LGTM!

I have noticed that this was still in the CF. After fixing a couple
of inconsistencies in the markups and the names, trimming down the
list of headers to avoid rot and adding a static in from of the const,
the result looked pretty much OK, so applied.
--
Michael