BackgroundWorkerInitializeConnection(NULL, ...) doesn't work
The documentation for 9.4 says, in the "Background Worker Processes"
section:
Once running, the process can connect to a database by calling
BackgroundWorkerInitializeConnection(char *dbname, char *username). This
allows the process to run transactions and queries using the SPI interface.
If dbname is NULL, the session is not connected to any particular database,
but shared catalogs can be accessed. If username is NULL, the process will
run as the superuser created during initdb.
However, should one attempt to pass NULL for the dbname parameter, the
process will die with:
FATAL: database 0 does not exist
BackgroundWorkerInitializeConnection() is essentially just a wrapper around
InitPostgres(), passing it the supplied dbname and username. (And passing
InvalidOid for the dboid parameter.) When InitPostgres() finds that dbname
is null, it falls back on dboid. GetDatabaseTupleByOid() returns NULL when
given InvalidOid, resulting in the aforementioned elog(FATAL).
Based on my quick read through InitPostgres() and postinit.c, it's not even
clear that there's a way to access the shared catalogs without connecting
to a specific database. Certainly the rest of the code in InitPostgres
expects MyDatabaseId to be set to something useful.
Regards,
Andrew Tipton
Andrew Tipton wrote:
However, should one attempt to pass NULL for the dbname parameter, the
process will die with:FATAL: database 0 does not exist
BackgroundWorkerInitializeConnection() is essentially just a wrapper around
InitPostgres(), passing it the supplied dbname and username. (And passing
InvalidOid for the dboid parameter.) When InitPostgres() finds that dbname
is null, it falls back on dboid. GetDatabaseTupleByOid() returns NULL when
given InvalidOid, resulting in the aforementioned elog(FATAL).
Hmm, the intention is that this code path mimics what the autovacuum
launcher does to establish its connection. It did work at some point;
maybe I broke this before commit. I will give it a look next week.
--
�lvaro Herrera http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Training & Services
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
Andrew Tipton <andrew@kiwidrew.com> writes:
The documentation for 9.4 says, in the "Background Worker Processes"
section:
Once running, the process can connect to a database by calling
BackgroundWorkerInitializeConnection(char *dbname, char *username). This
allows the process to run transactions and queries using the SPI interface.
If dbname is NULL, the session is not connected to any particular database,
but shared catalogs can be accessed.
Based on my quick read through InitPostgres() and postinit.c, it's not even
clear that there's a way to access the shared catalogs without connecting
to a specific database.
There is not, because you need e.g. pg_class and pg_attribute to be
available in order to open any relation whatsoever. So this documentation
is just wrong. A bg worker that chooses not to connect to a database
is shutting itself out of access to all relations.
There's been occasional talk of trying to refactor system catalogs into
separate shared and local components. The main motivation for that was
to reduce the bloat from having many copies of pg_proc etc, but it could
conceivably also make it practical for processes to run with only a
view of the shared catalogs. The work required seems far out of
proportion to these benefits, unfortunately ...
regards, tom lane
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
Alvaro Herrera <alvherre@2ndquadrant.com> writes:
Andrew Tipton wrote:
However, should one attempt to pass NULL for the dbname parameter, the
process will die with:
FATAL: database 0 does not exist
Hmm, the intention is that this code path mimics what the autovacuum
launcher does to establish its connection. It did work at some point;
maybe I broke this before commit. I will give it a look next week.
I'm pretty sure InitPostgres skips a bunch of stuff if
IsAutoVacuumLauncherProcess(). If you just want to make an environment
equivalent to that process's, maybe those tests should be replaced with
"if (dbname == NULL)". But the claim that you have access to shared
catalogs in this state is still wrong.
regards, tom lane
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
On Sat, Aug 10, 2013 at 10:40 PM, Tom Lane <tgl@sss.pgh.pa.us> wrote:
Alvaro Herrera <alvherre@2ndquadrant.com> writes:
Hmm, the intention is that this code path mimics what the autovacuum
launcher does to establish its connection. It did work at some point;
maybe I broke this before commit. I will give it a look next week.I'm pretty sure InitPostgres skips a bunch of stuff if
IsAutoVacuumLauncherProcess(). If you just want to make an environment
equivalent to that process's, maybe those tests should be replaced with
"if (dbname == NULL)". But the claim that you have access to shared
catalogs in this state is still wrong.
I've written up a quick proof-of-concept that splits InitPostgres(dbname,
username) into two phases; InitPostgresPhase1() is equivalent to what was
set up prior to the autovacuum launcher bailing out, while
InitPostgresPhase2(dbname, username) contains all of the authentication and
database selection logic.
After calling InitPostgresPhase1(), certain accesses to shared catalogs
*do* work. For example, ClientAuthentication() searches the AUTHNAME
syscache. Later, a call to InitPostgresPhase2() with the dbname and
username brings things to a state where normal SPI can be used. This
behaviour would be quite useful for certain classes of bgworker.
Documentation-wise, it would be helpful to specify precisely what is/isn't
allowed in this "shared catalogs only" mode.
See the attached patch -- which is just a proof-of-concept -- for the
details.
Regards,
Andrew Tipton
Attachments:
bgworker-shared-catalogs-v1.patchapplication/octet-stream; name=bgworker-shared-catalogs-v1.patchDownload
diff --git a/src/include/miscadmin.h b/src/include/miscadmin.h
index edced29..8e13cde 100644
--- a/src/include/miscadmin.h
+++ b/src/include/miscadmin.h
@@ -398,6 +398,9 @@ extern void pg_split_opts(char **argv, int *argcp, char *optstr);
extern void InitializeMaxBackends(void);
extern void InitPostgres(const char *in_dbname, Oid dboid, const char *username,
char *out_dbname);
+extern void InitPostgresPhase1(void);
+extern void InitPostgresPhase2(const char *in_dbname, Oid dboid,
+ const char *username, char *out_dbname);
extern void BaseInit(void);
/* in utils/init/miscinit.c */
diff --git a/src/backend/utils/init/postinit.c b/src/backend/utils/init/postinit.c
index 2c7f0f1..92ff971 100644
--- a/src/backend/utils/init/postinit.c
+++ b/src/backend/utils/init/postinit.c
@@ -493,139 +493,166 @@ void
InitPostgres(const char *in_dbname, Oid dboid, const char *username,
char *out_dbname)
{
+ /* Gain access to shared catalogs. */
+ InitPostgresPhase1();
+
+ /* The autovacuum launcher is done here */
+ if (IsAutoVacuumLauncherProcess())
+ return;
+
+ /* Perform client authentication, set MyDatabaseId, etc. */
+ InitPostgresPhase2(in_dbname, dboid, username, out_dbname);
+}
+
+
+/*
+ * InitPostgresPhase1
+ *
+ * Perform enough initialization to get us to a stage where access to the
+ * shared catalogs (at least, those that the relcache's formrdesc() helper
+ * knows how to bootstrap) can be done through the syscache.
+ */
+void InitPostgresPhase1()
+{
bool bootstrap = IsBootstrapProcessingMode();
- bool am_superuser;
- char *fullpath;
- char dbname[NAMEDATALEN];
- elog(DEBUG3, "InitPostgres");
+ elog(DEBUG3, "InitPostgresPhase1");
/*
* Add my PGPROC struct to the ProcArray.
*
* Once I have done this, I am visible to other backends!
*/
InitProcessPhase2();
/*
* Initialize my entry in the shared-invalidation manager's array of
* per-backend data.
*
* Sets up MyBackendId, a unique backend identifier.
*/
MyBackendId = InvalidBackendId;
SharedInvalBackendInit(false);
if (MyBackendId > MaxBackends || MyBackendId <= 0)
elog(FATAL, "bad backend ID: %d", MyBackendId);
/* Now that we have a BackendId, we can participate in ProcSignal */
ProcSignalInit(MyBackendId);
/*
* Also set up timeout handlers needed for backend operation. We need
* these in every case except bootstrap.
*/
if (!bootstrap)
{
RegisterTimeout(DEADLOCK_TIMEOUT, CheckDeadLock);
RegisterTimeout(STATEMENT_TIMEOUT, StatementTimeoutHandler);
RegisterTimeout(LOCK_TIMEOUT, LockTimeoutHandler);
}
/*
* bufmgr needs another initialization call too
*/
InitBufferPoolBackend();
/*
* Initialize local process's access to XLOG.
*/
if (IsUnderPostmaster)
{
/*
* The postmaster already started the XLOG machinery, but we need to
* call InitXLOGAccess(), if the system isn't in hot-standby mode.
* This is handled by calling RecoveryInProgress and ignoring the
* result.
*/
(void) RecoveryInProgress();
}
else
{
/*
* We are either a bootstrap process or a standalone backend. Either
* way, start up the XLOG machinery, and register to have it closed
* down at exit.
*/
StartupXLOG();
on_shmem_exit(ShutdownXLOG, 0);
}
/*
* Initialize the relation cache and the system catalog caches. Note that
* no catalog access happens here; we only set up the hashtable structure.
* We must do this before starting a transaction because transaction abort
* would try to touch these hashtables.
*/
RelationCacheInitialize();
InitCatalogCache();
InitPlanCache();
/* Initialize portal manager */
EnablePortalManager();
/* Initialize stats collection --- must happen before first xact */
if (!bootstrap)
pgstat_initialize();
/*
* Load relcache entries for the shared system catalogs. This must create
* at least entries for pg_database and catalogs used for authentication.
*/
RelationCacheInitializePhase2();
/*
* Set up process-exit callback to do pre-shutdown cleanup. This has to
* be after we've initialized all the low-level modules like the buffer
* manager, because during shutdown this has to run before the low-level
* modules start to close down. On the other hand, we want it in place
* before we begin our first transaction --- if we fail during the
* initialization transaction, as is entirely possible, we need the
* AbortTransaction call to clean up.
*/
on_shmem_exit(ShutdownPostgres, 0);
+}
- /* The autovacuum launcher is done here */
- if (IsAutoVacuumLauncherProcess())
- return;
+
+void
+InitPostgresPhase2(const char *in_dbname, Oid dboid, const char *username,
+ char *out_dbname)
+{
+ bool bootstrap = IsBootstrapProcessingMode();
+ bool am_superuser;
+ char *fullpath;
+ char dbname[NAMEDATALEN];
+
+ elog(DEBUG3, "InitPostgresPhase2");
/*
* Start a new transaction here before first access to db, and get a
* snapshot. We don't have a use for the snapshot itself, but we're
* interested in the secondary effect that it sets RecentGlobalXmin. (This
* is critical for anything that reads heap pages, because HOT may decide
* to prune them even if the process doesn't attempt to modify any
* tuples.)
*/
if (!bootstrap)
{
/* statement_timestamp must be set for timeouts to work correctly */
SetCurrentStatementStartTimestamp();
StartTransactionCommand();
/*
* transaction_isolation will have been set to the default by the
* above. If the default is "serializable", and we are in hot
* standby, we will fail if we don't change it to something lower.
* Fortunately, "read committed" is plenty good enough.
*/
XactIsoLevel = XACT_READ_COMMITTED;
(void) GetTransactionSnapshot();
}
/*
* Perform client authentication if necessary, then figure out our
* postgres user ID, and see if we are a superuser.
diff --git a/src/include/postmaster/bgworker.h b/src/include/postmaster/bgworker.h
index 0bb897b..802230b 100644
--- a/src/include/postmaster/bgworker.h
+++ b/src/include/postmaster/bgworker.h
@@ -92,15 +92,20 @@ extern bool RegisterDynamicBackgroundWorker(BackgroundWorker *worker);
extern BackgroundWorker *MyBgworkerEntry;
/*
- * Connect to the specified database, as the specified user. Only a worker
- * that passed BGWORKER_BACKEND_DATABASE_CONNECTION during registration may
- * call this.
- *
- * If username is NULL, bootstrapping superuser is used.
- * If dbname is NULL, connection is made to no specific database;
- * only shared catalogs can be accessed.
+ * Initialize and open the specified database, as the specified user. Only a
+ * worker that passed BGWORKER_BACKEND_DATABASE_CONNECTION during registration
+ * may call these functions.
+ *
+ * Once InitializeConnection() has been called, shared catalogs may be
+ * accessed. [XXX: but only through the syscache interfaces?]
+ *
+ * After calling OpenDatabase(), the SPI functions may be used to access all
+ * relations. Once a database has been opened, it is not possible to
+ * disconnect and change to another database. If username is NULL, the
+ * bootstrapping superuser is used.
*/
-extern void BackgroundWorkerInitializeConnection(char *dbname, char *username);
+extern void BackgroundWorkerInitializeConnection();
+extern void BackgroundWorkerOpenDatabase(char *dbname, char *username);
/* Block/unblock signals in a background worker process */
extern void BackgroundWorkerBlockSignals(void);
diff --git a/src/backend/postmaster/postmaster.c b/src/backend/postmaster/postmaster.c
index afdb53f..05b463f 100644
--- a/src/backend/postmaster/postmaster.c
+++ b/src/backend/postmaster/postmaster.c
@@ -5222,32 +5222,50 @@ int
MaxLivePostmasterChildren(void)
{
return 2 * (MaxConnections + autovacuum_max_workers + 1 +
max_worker_processes);
}
/*
- * Connect background worker to a database.
+ * Connect background worker to the shared catalogs.
*/
void
-BackgroundWorkerInitializeConnection(char *dbname, char *username)
+BackgroundWorkerInitializeConnection()
{
BackgroundWorker *worker = MyBgworkerEntry;
/* XXX is this the right errcode? */
if (!(worker->bgw_flags & BGWORKER_BACKEND_DATABASE_CONNECTION))
ereport(FATAL,
(errcode(ERRCODE_PROGRAM_LIMIT_EXCEEDED),
errmsg("database connection requirement not indicated during registration")));
- InitPostgres(dbname, InvalidOid, username, NULL);
+ InitPostgresPhase1();
+}
+
+/*
+ * Connect background worker to a database, and enter normal processing
+ * mode.
+ */
+void
+BackgroundWorkerOpenDatabase(char *dbname, char *username)
+{
+ BackgroundWorker *worker = MyBgworkerEntry;
+
+ /* XXX is this the right errcode? */
+ if (!(worker->bgw_flags & BGWORKER_BACKEND_DATABASE_CONNECTION))
+ ereport(FATAL,
+ (errcode(ERRCODE_PROGRAM_LIMIT_EXCEEDED),
+ errmsg("database connection requirement not indicated during registration")));
+
+ InitPostgresPhase2(dbname, InvalidOid, username, NULL);
/* it had better not gotten out of "init" mode yet */
if (!IsInitProcessingMode())
ereport(ERROR,
(errmsg("invalid processing mode in background worker")));
SetProcessingMode(NormalProcessing);
}
/*
* Block/unblock signals in a background worker
*/