Insert records in the tavke only if they are not exist

On Friday, December 6, 2024, Igor Korot <ikorot01@gmail.com> wrote:

When my application starts up, ot creates some tables and insert records
in them.

When the app starts for the second time it should check if the tables and
the records in them are exist and skip the process.

Everything is good, except what if I have a connection from 2 different
users?

I can run this inside transaction, but will this be enough? Will stating
transaction lock the DB and the second user will wait for transaction to
complete?

Seems like letting the create table fail would be a reliable way to
determine what is happening. But this procedure overall just seems better
avoided. Can’t you just run an installer/updater separate from running the
application?

David J.

Hi, Dvid,

On Fri, Dec 6, 2024, 9:55 PM David G. Johnston <david.g.johnston@gmail.com>
wrote:

On Friday, December 6, 2024, Igor Korot <ikorot01@gmail.com> wrote:

When my application starts up, ot creates some tables and insert records
in them.

When the app starts for the second time it should check if the tables and
the records in them are exist and skip the process.

Everything is good, except what if I have a connection from 2 different
users?

I can run this inside transaction, but will this be enough? Will stating
transaction lock the DB and the second user will wait for transaction to
complete?

Seems like letting the create table fail would be a reliable way to
determine what is happening. But this procedure overall just seems better
avoided. Can’t you just run an installer/updater separate from running the
application?

Interesting idea bout the installer.
Except those tables will contain some additional info about the schema and
the application.

So if I create/remove new table, either in my app or from psql the record
will need to be inserted/deleted.

Thank you.

On 12/6/24 19:36, Igor Korot wrote:

Hi, All,

When my application starts up, ot creates some tables and insert records
in them.

When the app starts for the second time it should check if the tables
and the records in them are exist and skip the process.

Does this also apply to starts after the second time?

Is the data expected to change over time?

Everything is good, except what if I have a connection from 2 different
users?

Is it possible to have the app start up without external users connecting?

I can run this inside transaction, but will this be enough? Will stating
transaction lock the DB and the second user will wait for transaction to
complete?

Thank you.

--
Adrian Klaver
adrian.klaver@aklaver.com

Hi, Adrian,

On Sat, Dec 7, 2024 at 11:18 AM Adrian Klaver <adrian.klaver@aklaver.com> wrote:

On 12/6/24 19:36, Igor Korot wrote:

Hi, All,

When my application starts up, ot creates some tables and insert records
in them.

When the app starts for the second time it should check if the tables
and the records in them are exist and skip the process.

Does this also apply to starts after the second time?

Yes, it does.

However, if the new table is created in the meantime - the new record
should be created, because it is not there yet.

Is the data expected to change over time?

Data change is possible.

This is one of the tables:

"CREATE TABLE IF NOT EXISTS \"sys.abcatfmt\"(\"abf_name\" char(30)
NOT NULL, \"abf_frmt\" char(254), \"abf_type\" smallint, \"abf_cntr\"
integer" ));";
"CREATE UNIQUE INDEX IF NOT EXISTS pbcatf_x ON
\"sys.abcatfmt\"(\"abf_name\" ASC);";

My understanding is that "INSERT OR IGNORE" will check the unique
index and will not do anything if the recrd
with such data on the index already exists.

Everything is good, except what if I have a connection from 2 different
users?

Is it possible to have the app start up without external users connecting?

Not sure what you mean here.

Are you asking if the computer can run the app?
Then the answer is NO. App is always started by the user.
Please clarify.

Thank you.

On 12/7/24 09:59, Igor Korot wrote:

Hi, Adrian,

On Sat, Dec 7, 2024 at 11:18 AM Adrian Klaver <adrian.klaver@aklaver.com> wrote:

On 12/6/24 19:36, Igor Korot wrote:

Hi, All,

When my application starts up, ot creates some tables and insert records
in them.

When the app starts for the second time it should check if the tables
and the records in them are exist and skip the process.

Does this also apply to starts after the second time?

Yes, it does.

However, if the new table is created in the meantime - the new record
should be created, because it is not there yet.

What determines that a new table needs to be created and populated?

Is the data expected to change over time?

Data change is possible.

This is one of the tables:

"CREATE TABLE IF NOT EXISTS \"sys.abcatfmt\"(\"abf_name\" char(30)
NOT NULL, \"abf_frmt\" char(254), \"abf_type\" smallint, \"abf_cntr\"
integer" ));";
"CREATE UNIQUE INDEX IF NOT EXISTS pbcatf_x ON
\"sys.abcatfmt\"(\"abf_name\" ASC);";

My understanding is that "INSERT OR IGNORE" will check the unique

It is INSERT ... ON CONFLICT DO NOTHING | UPDATE.

index and will not do anything if the recrd
with such data on the index already exists.

So you have the option of either skipping the insert or updating
selected fields in the row.

Everything is good, except what if I have a connection from 2 different
users?

Is it possible to have the app start up without external users connecting?

Not sure what you mean here.

Are you asking if the computer can run the app?
Then the answer is NO. App is always started by the user.
Please clarify.

This is going to need a more detailed explanation of what 'app' means.

Are you talking about the front end that the user launches or the
backend that runs the database or something else?

Thank you.

I can run this inside transaction, but will this be enough? Will stating
transaction lock the DB and the second user will wait for transaction to
complete?

Thank you.

--
Adrian Klaver
adrian.klaver@aklaver.com

--
Adrian Klaver
adrian.klaver@aklaver.com

Hi, Adrian,

On Sat, Dec 7, 2024 at 12:32 PM Adrian Klaver <adrian.klaver@aklaver.com> wrote:

On 12/7/24 09:59, Igor Korot wrote:

Hi, Adrian,

On Sat, Dec 7, 2024 at 11:18 AM Adrian Klaver <adrian.klaver@aklaver.com> wrote:

On 12/6/24 19:36, Igor Korot wrote:

Hi, All,

When my application starts up, ot creates some tables and insert records
in them.

When the app starts for the second time it should check if the tables
and the records in them are exist and skip the process.

Does this also apply to starts after the second time?

Yes, it does.

However, if the new table is created in the meantime - the new record
should be created, because it is not there yet.

What determines that a new table needs to be created and populated?

Consider the following scenario:

1. Program is installed.
2. Program is started for the first time.
3. My tables are created and populated
4. Program is executed.
5. User closes the program.
6.. Later on the user decides that there is a need
for another table (inside psql or any other client)
7. Then the program starts for the second time.

At this time all my tables that were created will stay (courtesy
of CREATE TABLE IF NOT EXIST).
All records that were there are staying unchanged.
However, for the table that was made in between the runs
will be added

Now if the program is installed on 2 different machines
and started simultaneously on both - I want to ensure that
only 1 set of tables is made and only 1 set of records in them
is available

Now, the creation/population is done inside a transaction.

Is the data expected to change over time?

Data change is possible.

This is one of the tables:

"CREATE TABLE IF NOT EXISTS \"sys.abcatfmt\"(\"abf_name\" char(30)
NOT NULL, \"abf_frmt\" char(254), \"abf_type\" smallint, \"abf_cntr\"
integer" ));";
"CREATE UNIQUE INDEX IF NOT EXISTS pbcatf_x ON
\"sys.abcatfmt\"(\"abf_name\" ASC);";

My understanding is that "INSERT OR IGNORE" will check the unique

It is INSERT ... ON CONFLICT DO NOTHING | UPDATE.

index and will not do anything if the recrd
with such data on the index already exists.

Good.
I'm using INSERT ON CONFLICT DO NOTHING.

So you have the option of either skipping the insert or updating
selected fields in the row.

Everything is good, except what if I have a connection from 2 different
users?

Is it possible to have the app start up without external users connecting?

Not sure what you mean here.

Are you asking if the computer can run the app?
Then the answer is NO. App is always started by the user.
Please clarify.

This is going to need a more detailed explanation of what 'app' means.

Are you talking about the front end that the user launches or the
backend that runs the database or something else?

Front-end.
This is a C++ app.

On 12/7/24 12:17, Igor Korot wrote:

Hi, Adrian,

On Sat, Dec 7, 2024 at 12:32 PM Adrian Klaver <adrian.klaver@aklaver.com> wrote:

On 12/7/24 09:59, Igor Korot wrote:

Hi, Adrian,

On Sat, Dec 7, 2024 at 11:18 AM Adrian Klaver <adrian.klaver@aklaver.com> wrote:

On 12/6/24 19:36, Igor Korot wrote:

Hi, All,

When my application starts up, ot creates some tables and insert records
in them.

When the app starts for the second time it should check if the tables
and the records in them are exist and skip the process.

Does this also apply to starts after the second time?

Yes, it does.

However, if the new table is created in the meantime - the new record
should be created, because it is not there yet.

What determines that a new table needs to be created and populated?

Consider the following scenario:

I did and it made me start twitching.

1. Program is installed.
2. Program is started for the first time.
3. My tables are created and populated

What distinguishes your tables from other users tables?

4. Program is executed.
5. User closes the program.
6.. Later on the user decides that there is a need
for another table (inside psql or any other client)

The above is where I started twitching.

How do you keep them out of your tables?

How do you get these changes to play nice with the existing structure?

7. Then the program starts for the second time.

At this time all my tables that were created will stay (courtesy
of CREATE TABLE IF NOT EXIST).

You are depending on folks not knowing about DROP TABLE and/or you
having thought out the permissions for access thoroughly.

All records that were there are staying unchanged.

INSERT/UPDATE against your tables is not a possibility?

However, for the table that was made in between the runs
will be added

Now if the program is installed on 2 different machines
and started simultaneously on both - I want to ensure that
only 1 set of tables is made and only 1 set of records in them
is available

I assume this means they are both pointing at the same instance of a
database?

This is the part that confuses me.

If you are going to allow ad hoc and at will changes how do you know
what is actually the correct change?

Now, the creation/population is done inside a transaction.

I'm not sure that a transaction is going to solve the issue I raised
above, it will just make one thing happen with no guarantee that it is
the correct outcome.

Are you talking about the front end that the user launches or the
backend that runs the database or something else?

Front-end.
This is a C++ app.

To me this is the tail wagging the dog. The thought of allowing users to
change the database structure and you dealing with it after the fact is
just disturbing to me.

Thank you.

--
Adrian Klaver
adrian.klaver@aklaver.com

Hi, Adrian,

On Sat, Dec 7, 2024 at 5:07 PM Adrian Klaver <adrian.klaver@aklaver.com> wrote:

On 12/7/24 12:17, Igor Korot wrote:

Hi, Adrian,

On Sat, Dec 7, 2024 at 12:32 PM Adrian Klaver <adrian.klaver@aklaver.com> wrote:

On 12/7/24 09:59, Igor Korot wrote:

Hi, Adrian,

On Sat, Dec 7, 2024 at 11:18 AM Adrian Klaver <adrian.klaver@aklaver.com> wrote:

On 12/6/24 19:36, Igor Korot wrote:

Hi, All,

When my application starts up, ot creates some tables and insert records
in them.

When the app starts for the second time it should check if the tables
and the records in them are exist and skip the process.

Does this also apply to starts after the second time?

Yes, it does.

However, if the new table is created in the meantime - the new record
should be created, because it is not there yet.

What determines that a new table needs to be created and populated?

Consider the following scenario:

I did and it made me start twitching.

Good..

1. Program is installed.
2. Program is started for the first time.
3. My tables are created and populated

What distinguishes your tables from other users' tables?

I am making the tables and naming them with a specific pattern.

Now I'm curious - if I start psql and will want to create a table
named pg_am, what will happen?
I presume psql will produce error saying that the system table with
that name exists and the user can't create ome.
Am I right?

4. Program is executed.
5. User closes the program.
6.. Later on the user decides that there is a need
for another table (inside psql or any other client)

The above is where I started twitching.

How do you keep them out of your tables?

My tables are named with the specific pattern.

Also - see above.

How do you get these changes to play nice with the existing structure?

Again - not sure what you mean here....

7. Then the program starts for the second time.

At this time all my tables that were created will stay (courtesy
of CREATE TABLE IF NOT EXIST).

You are depending on folks not knowing about DROP TABLE and/or you
having thought out the permissions for access thoroughly.

If one of my tables will be dropped - it will be re-created.
I'm using CREATE TABLE IF NOT EXIST.

All records that were there are staying unchanged.

INSERT/UPDATE against your tables is not a possibility?

It is..
All I;m saying that when the app starts-up, this is done automatically

And if the user decides to insert some data - that's on him

However, for the table that was made in between the runs
will be added

Now if the program is installed on 2 different machines
and started simultaneously on both - I want to ensure that
only 1 set of tables is made and only 1 set of records in them
is available

I assume this means they are both pointing at the same instance of a
database?

Correct.

This is the part that confuses me.

If you are going to allow ad hoc and at will changes how do you know
what is actually the correct change?

I don't.
Think about MS ACCESS-like applications.
ACCESS creates the internal tables to keep track of some
internals.
Is there a possibility of a clash with the user table? Ofc there is.
But MS is still doing it nevertheless, because chances of this occurring
are really slim.
And if a user acquire access to those internal tables and start modify
them and in the process screw something up, well he uses MS product
as a developer and so should know better as a developer
And if MS can do it - why can't I?

Now, the creation/population is done inside a transaction.

I'm not sure that a transaction is going to solve the issue I raised
above, it will just make one thing happen with no guarantee that it is
the correct outcome.

What is considered correct outcome is strongly on user if we are talking
about my app.

Are you talking about the front end that the user launches or the
backend that runs the database or something else?

Front-end.
This is a C++ app.

To me this is the tail wagging the dog. The thought of allowing users to
change the database structure and you dealing with it after the fact is
just disturbing to me.

But not to me.
My application targets developers, not end users.

Thank you.

But not to me.
My application targets developers, not end users.

Thank you.

What does your app enable developers to do?

On Sat, Dec 7, 2024 at 4:59 PM Igor Korot <ikorot01@gmail.com> wrote:

I am making the tables and naming them with a specific pattern.

Now I'm curious - if I start psql and will want to create a table
named pg_am, what will happen?
I presume psql will produce error saying that the system table with
that name exists and the user can't create ome.
Am I right?

You are wasting people's time asking a question like this when it takes but
seconds to experiment.

Most of what you are doing should be considered something to test out and
observe the behavior yourself. Formalize it as a test if possible if you
rely heavily on it so you can be aware if a bug were to be introduced that
broke such a behavior, or even just to get lots of exposure to concurrency
if you are designing a system especially reliant on specific concurrent
behavior.

As it stands the methodology you describe is quite non-traditional for a
relational client-server database system.

David J.

On 12/7/24 15:58, Igor Korot wrote:

Hi, Adrian,

What distinguishes your tables from other users' tables?

I am making the tables and naming them with a specific pattern.

Now I'm curious - if I start psql and will want to create a table
named pg_am, what will happen?
I presume psql will produce error saying that the system table with
that name exists and the user can't create ome.
Am I right?

When you refer to psql are you talking about the CLI program or the
Postgres server in general?

As to table names it depends. A schema is a namespace so if you put the
table in it's own schema then the server will not complain:

create table test_sch.pg_am(id integer);
CREATE TABLE

\d *.pg_am
Table "pg_catalog.pg_am"
Column | Type | Collation | Nullable | Default
-----------+---------+-----------+----------+---------
oid | oid | | not null |
amname | name | | not null |
amhandler | regproc | | not null |
amtype | "char" | | not null |
Indexes:
"pg_am_oid_index" PRIMARY KEY, btree (oid)
"pg_am_name_index" UNIQUE CONSTRAINT, btree (amname)

Table "test_sch.pg_am"
Column | Type | Collation | Nullable | Default
--------+---------+-----------+----------+---------
id | integer | | |

I would strongly advise against this naming schema for the above reason.

4. Program is executed.
5. User closes the program.
6.. Later on the user decides that there is a need
for another table (inside psql or any other client)

The above is where I started twitching.

How do you keep them out of your tables?

My tables are named with the specific pattern.

Also - see above.

Yes, but if you allow users into the database:

a) They can figure out what that pattern is.

b) Mess with the tables.

How do you get these changes to play nice with the existing structure?

Again - not sure what you mean here....

Examples:

1) A user creates table that creates a FK relationship to one of 'your'
tables and you then change 'your' table and invalidate that relationship.

2) A user changes the structure of 'your' tables.

If one of my tables will be dropped - it will be re-created.
I'm using CREATE TABLE IF NOT EXIST.

This means every time the app is run it could potentially run one or
more schema/data migrations.

How are you going to track the state of the database in order to get it
back to what it was when the change or changes occurred?

All records that were there are staying unchanged.

INSERT/UPDATE against your tables is not a possibility?

It is..
All I;m saying that when the app starts-up, this is done automatically

Again, using what point in time state?

And if the user decides to insert some data - that's on him

No the point of this thread is it's on you or you would not be asking
how to restore to a known state.

This is the part that confuses me.

If you are going to allow ad hoc and at will changes how do you know
what is actually the correct change?

I don't.
Think about MS ACCESS-like applications.
ACCESS creates the internal tables to keep track of some
internals.
Is there a possibility of a clash with the user table? Ofc there is.
But MS is still doing it nevertheless, because chances of this occurring
are really slim.
And if a user acquire access to those internal tables and start modify
them and in the process screw something up, well he uses MS product
as a developer and so should know better as a developer
And if MS can do it - why can't I?

Because MS has the lawyers to keep at bay anyone complaining that Access
blew up their application.

Now, the creation/population is done inside a transaction.

I'm not sure that a transaction is going to solve the issue I raised
above, it will just make one thing happen with no guarantee that it is
the correct outcome.

What is considered correct outcome is strongly on user if we are talking
about my app.

Except you are asking how to deal with changes and so you are taking
ownership of them. And in the case you describe, two different instances
of the app making changes, how do you decide which one wins?

To me this is the tail wagging the dog. The thought of allowing users to
change the database structure and you dealing with it after the fact is
just disturbing to me.

But not to me.
My application targets developers, not end users.

Who are users in your application.

Thank you.

Thank you.

--
Adrian Klaver
adrian.klaver@aklaver.com

--
Adrian Klaver
adrian.klaver@aklaver.com