pgsql: Introduce hash_search_with_hash_value() function

Hi

st 7. 8. 2024 v 6:08 odesílatel Alexander Korotkov <akorotkov@postgresql.org>
napsal:

Introduce hash_search_with_hash_value() function

This new function iterates hash entries with given hash values. This
function
is designed to avoid full sequential hash search in the syscache
invalidation
callbacks.

Discussion:
/messages/by-id/5812a6e5-68ae-4d84-9d85-b443176966a1@sigaev.ru
Author: Teodor Sigaev
Reviewed-by: Aleksander Alekseev, Tom Lane, Michael Paquier, Roman Zharkov
Reviewed-by: Andrei Lepikhov

I tried to use hash_seq_init_with_hash_value in session variables patch,
but it doesn't work there.

<-->if (!sessionvars)
<--><-->return;

elog(NOTICE, "%u", hashvalue);

<-->hash_seq_init(&status, sessionvars);

<-->while ((svar = (SVariable) hash_seq_search(&status)) != NULL)
<-->{
<--><-->if (hashvalue == 0 || svar->hashvalue == hashvalue)
<--><-->{
<--><--><-->elog(NOTICE, "FOUND OLD");
<--><--><-->svar->is_valid = false;
<--><-->}
<-->}

<-->/*
<--> * If the hashvalue is not specified, we have to recheck all currently
<--> * used session variables. Since we can't tell the exact session
variable
<--> * from its hashvalue, we have to iterate over all items in the hash
bucket.
<--> */
<-->if (hashvalue == 0)
<--><-->hash_seq_init(&status, sessionvars);
<-->else
<--><-->hash_seq_init_with_hash_value(&status, sessionvars, hashvalue);

<-->while ((svar = (SVariable) hash_seq_search(&status)) != NULL)
<-->{
<--><-->Assert(hashvalue == 0 || svar->hashvalue == hashvalue);

elog(NOTICE, "found");

<--><-->svar->is_valid = false;
<--><-->needs_validation = true;
<-->}
}

Old methods found an entry, but new not.

What am I doing wrong?

Regards

Pavel

Hi, Pavel!

On Wed, Aug 7, 2024 at 9:35 AM Pavel Stehule <pavel.stehule@gmail.com> wrote:

st 7. 8. 2024 v 6:08 odesílatel Alexander Korotkov <akorotkov@postgresql.org> napsal:

Introduce hash_search_with_hash_value() function

This new function iterates hash entries with given hash values. This function
is designed to avoid full sequential hash search in the syscache invalidation
callbacks.

Discussion: /messages/by-id/5812a6e5-68ae-4d84-9d85-b443176966a1@sigaev.ru
Author: Teodor Sigaev
Reviewed-by: Aleksander Alekseev, Tom Lane, Michael Paquier, Roman Zharkov
Reviewed-by: Andrei Lepikhov

I tried to use hash_seq_init_with_hash_value in session variables patch, but it doesn't work there.

<-->if (!sessionvars)
<--><-->return;

elog(NOTICE, "%u", hashvalue);

<-->hash_seq_init(&status, sessionvars);

<-->while ((svar = (SVariable) hash_seq_search(&status)) != NULL)
<-->{
<--><-->if (hashvalue == 0 || svar->hashvalue == hashvalue)
<--><-->{
<--><--><-->elog(NOTICE, "FOUND OLD");
<--><--><-->svar->is_valid = false;
<--><-->}
<-->}

<-->/*
<--> * If the hashvalue is not specified, we have to recheck all currently
<--> * used session variables. Since we can't tell the exact session variable
<--> * from its hashvalue, we have to iterate over all items in the hash bucket.
<--> */
<-->if (hashvalue == 0)
<--><-->hash_seq_init(&status, sessionvars);
<-->else
<--><-->hash_seq_init_with_hash_value(&status, sessionvars, hashvalue);

<-->while ((svar = (SVariable) hash_seq_search(&status)) != NULL)
<-->{
<--><-->Assert(hashvalue == 0 || svar->hashvalue == hashvalue);

elog(NOTICE, "found");

<--><-->svar->is_valid = false;
<--><-->needs_validation = true;
<-->}
}

Old methods found an entry, but new not.

What am I doing wrong?

I'm trying to check this. Applying this patch [1], but got conflicts.
Could you please, rebase the patch, so I can recheck the issue?

Links.
1. /messages/by-id/CAFj8pRD053CY_N4=6SvPe7ke6xPbh=K50LUAOwjC3jm8Me9Obg@mail.gmail.com

------
Regards,
Alexander Korotkov
Supabase

st 7. 8. 2024 v 10:52 odesílatel Alexander Korotkov <aekorotkov@gmail.com>
napsal:

Hi, Pavel!

On Wed, Aug 7, 2024 at 9:35 AM Pavel Stehule <pavel.stehule@gmail.com>
wrote:

st 7. 8. 2024 v 6:08 odesílatel Alexander Korotkov <

akorotkov@postgresql.org> napsal:

Introduce hash_search_with_hash_value() function

This new function iterates hash entries with given hash values. This

function

is designed to avoid full sequential hash search in the syscache

invalidation

callbacks.

Discussion:

/messages/by-id/5812a6e5-68ae-4d84-9d85-b443176966a1@sigaev.ru

Author: Teodor Sigaev
Reviewed-by: Aleksander Alekseev, Tom Lane, Michael Paquier, Roman

Zharkov

Reviewed-by: Andrei Lepikhov

I tried to use hash_seq_init_with_hash_value in session variables patch,

but it doesn't work there.

<-->if (!sessionvars)
<--><-->return;

elog(NOTICE, "%u", hashvalue);

<-->hash_seq_init(&status, sessionvars);

<-->while ((svar = (SVariable) hash_seq_search(&status)) != NULL)
<-->{
<--><-->if (hashvalue == 0 || svar->hashvalue == hashvalue)
<--><-->{
<--><--><-->elog(NOTICE, "FOUND OLD");
<--><--><-->svar->is_valid = false;
<--><-->}
<-->}

<-->/*
<--> * If the hashvalue is not specified, we have to recheck all

currently

<--> * used session variables. Since we can't tell the exact session

variable

<--> * from its hashvalue, we have to iterate over all items in the hash

bucket.

<--> */
<-->if (hashvalue == 0)
<--><-->hash_seq_init(&status, sessionvars);
<-->else
<--><-->hash_seq_init_with_hash_value(&status, sessionvars, hashvalue);

<-->while ((svar = (SVariable) hash_seq_search(&status)) != NULL)
<-->{
<--><-->Assert(hashvalue == 0 || svar->hashvalue == hashvalue);

elog(NOTICE, "found");

<--><-->svar->is_valid = false;
<--><-->needs_validation = true;
<-->}
}

Old methods found an entry, but new not.

What am I doing wrong?

I'm trying to check this. Applying this patch [1], but got conflicts.
Could you please, rebase the patch, so I can recheck the issue?

I sent rebased patchset

Message-ID:
CAFj8pRAskimJmB9Q8pHDa8YoLphVoZMH1xPeGBK8Eze=u+_hBQ@mail.gmail.com
</messages/by-id/CAFj8pRAskimJmB9Q8pHDa8YoLphVoZMH1xPeGBK8Eze=u+_hBQ@mail.gmail.com&gt;

Regards

Pavel

On Wed, Aug 7, 2024 at 1:03 PM Pavel Stehule <pavel.stehule@gmail.com> wrote:

st 7. 8. 2024 v 10:52 odesílatel Alexander Korotkov <aekorotkov@gmail.com> napsal:

Hi, Pavel!

On Wed, Aug 7, 2024 at 9:35 AM Pavel Stehule <pavel.stehule@gmail.com> wrote:

st 7. 8. 2024 v 6:08 odesílatel Alexander Korotkov <akorotkov@postgresql.org> napsal:

Introduce hash_search_with_hash_value() function

This new function iterates hash entries with given hash values. This function
is designed to avoid full sequential hash search in the syscache invalidation
callbacks.

Discussion: /messages/by-id/5812a6e5-68ae-4d84-9d85-b443176966a1@sigaev.ru
Author: Teodor Sigaev
Reviewed-by: Aleksander Alekseev, Tom Lane, Michael Paquier, Roman Zharkov
Reviewed-by: Andrei Lepikhov

I tried to use hash_seq_init_with_hash_value in session variables patch, but it doesn't work there.

<-->if (!sessionvars)
<--><-->return;

elog(NOTICE, "%u", hashvalue);

<-->hash_seq_init(&status, sessionvars);

<-->while ((svar = (SVariable) hash_seq_search(&status)) != NULL)
<-->{
<--><-->if (hashvalue == 0 || svar->hashvalue == hashvalue)
<--><-->{
<--><--><-->elog(NOTICE, "FOUND OLD");
<--><--><-->svar->is_valid = false;
<--><-->}
<-->}

<-->/*
<--> * If the hashvalue is not specified, we have to recheck all currently
<--> * used session variables. Since we can't tell the exact session variable
<--> * from its hashvalue, we have to iterate over all items in the hash bucket.
<--> */
<-->if (hashvalue == 0)
<--><-->hash_seq_init(&status, sessionvars);
<-->else
<--><-->hash_seq_init_with_hash_value(&status, sessionvars, hashvalue);

<-->while ((svar = (SVariable) hash_seq_search(&status)) != NULL)
<-->{
<--><-->Assert(hashvalue == 0 || svar->hashvalue == hashvalue);

elog(NOTICE, "found");

<--><-->svar->is_valid = false;
<--><-->needs_validation = true;
<-->}
}

Old methods found an entry, but new not.

What am I doing wrong?

I'm trying to check this. Applying this patch [1], but got conflicts.
Could you please, rebase the patch, so I can recheck the issue?

I sent rebased patchset

Thank you.
Please see 40064a8ee1 takes special efforts to match HTAB hash
function to syscache hash function. By default, their hash values
don't match and you can't simply use syscache hash value to search
HTAB. This probably should be mentioned in the header comment of
hash_seq_init_with_hash_value().

------
Regards,
Alexander Korotkov
Supabase

st 7. 8. 2024 v 12:22 odesílatel Alexander Korotkov <aekorotkov@gmail.com>
napsal:

On Wed, Aug 7, 2024 at 1:03 PM Pavel Stehule <pavel.stehule@gmail.com>
wrote:

st 7. 8. 2024 v 10:52 odesílatel Alexander Korotkov <

aekorotkov@gmail.com> napsal:

Hi, Pavel!

On Wed, Aug 7, 2024 at 9:35 AM Pavel Stehule <pavel.stehule@gmail.com>

wrote:

st 7. 8. 2024 v 6:08 odesílatel Alexander Korotkov <

akorotkov@postgresql.org> napsal:

Introduce hash_search_with_hash_value() function

This new function iterates hash entries with given hash values.

This function

is designed to avoid full sequential hash search in the syscache

invalidation

callbacks.

Discussion:

/messages/by-id/5812a6e5-68ae-4d84-9d85-b443176966a1@sigaev.ru

Author: Teodor Sigaev
Reviewed-by: Aleksander Alekseev, Tom Lane, Michael Paquier, Roman

Zharkov

Reviewed-by: Andrei Lepikhov

I tried to use hash_seq_init_with_hash_value in session variables

patch, but it doesn't work there.

<-->if (!sessionvars)
<--><-->return;

elog(NOTICE, "%u", hashvalue);

<-->hash_seq_init(&status, sessionvars);

<-->while ((svar = (SVariable) hash_seq_search(&status)) != NULL)
<-->{
<--><-->if (hashvalue == 0 || svar->hashvalue == hashvalue)
<--><-->{
<--><--><-->elog(NOTICE, "FOUND OLD");
<--><--><-->svar->is_valid = false;
<--><-->}
<-->}

<-->/*
<--> * If the hashvalue is not specified, we have to recheck all

currently

<--> * used session variables. Since we can't tell the exact session

variable

<--> * from its hashvalue, we have to iterate over all items in the

hash bucket.

<--> */
<-->if (hashvalue == 0)
<--><-->hash_seq_init(&status, sessionvars);
<-->else
<--><-->hash_seq_init_with_hash_value(&status, sessionvars,

hashvalue);

<-->while ((svar = (SVariable) hash_seq_search(&status)) != NULL)
<-->{
<--><-->Assert(hashvalue == 0 || svar->hashvalue == hashvalue);

elog(NOTICE, "found");

<--><-->svar->is_valid = false;
<--><-->needs_validation = true;
<-->}
}

Old methods found an entry, but new not.

What am I doing wrong?

I'm trying to check this. Applying this patch [1], but got conflicts.
Could you please, rebase the patch, so I can recheck the issue?

I sent rebased patchset

Thank you.
Please see 40064a8ee1 takes special efforts to match HTAB hash
function to syscache hash function. By default, their hash values
don't match and you can't simply use syscache hash value to search
HTAB. This probably should be mentioned in the header comment of
hash_seq_init_with_hash_value().

yes, enhancing doc should be great + maybe assert

Regards

Pavel

You may have already realized this, but the name of the function the
patch adds is not the same as the name that appears in the commit
message.

...Robert

On Wed, Aug 7, 2024 at 3:24 PM Robert Haas <robertmhaas@gmail.com> wrote:

You may have already realized this, but the name of the function the
patch adds is not the same as the name that appears in the commit
message.

:sigh:
I didn't realize that before your message. That would be another item
for my checklist: ensure entities referenced from commit message and
comments didn't change their names.

------
Regards,
Alexander Korotkov
Supabase

On Wed, Aug 7, 2024 at 11:55 AM Alexander Korotkov <aekorotkov@gmail.com> wrote:

On Wed, Aug 7, 2024 at 3:24 PM Robert Haas <robertmhaas@gmail.com> wrote:

You may have already realized this, but the name of the function the
patch adds is not the same as the name that appears in the commit
message.

:sigh:
I didn't realize that before your message. That would be another item
for my checklist: ensure entities referenced from commit message and
comments didn't change their names.

I really wish there was some way to fix commit messages. I had a typo
in mine today, too.

--
Robert Haas
EDB: http://www.enterprisedb.com

On 2024-Aug-07, Robert Haas wrote:

I really wish there was some way to fix commit messages. I had a typo
in mine today, too.

We could use git notes. The UI is a bit inconvenient (they have to be
pushed and pulled separately from commits), but they seem useful enough.

https://initialcommit.com/blog/git-notes

--
Álvaro Herrera 48°01'N 7°57'E — https://www.EnterpriseDB.com/
"This is what I like so much about PostgreSQL. Most of the surprises
are of the "oh wow! That's cool" Not the "oh shit!" kind. :)"
Scott Marlowe, http://archives.postgresql.org/pgsql-admin/2008-10/msg00152.php

On Wed, Aug 07, 2024 at 01:08:35PM -0400, Alvaro Herrera wrote:

On 2024-Aug-07, Robert Haas wrote:

I really wish there was some way to fix commit messages. I had a typo
in mine today, too.

We could use git notes. The UI is a bit inconvenient (they have to be
pushed and pulled separately from commits), but they seem useful enough.

Yeah, I spend a lot of time on commit messages because they're pretty much
written in stone once pushed. I'd definitely use git notes to add errata,
follow-up commits that fixed/reverted things, etc.

--
nathan

On Wed, Aug 7, 2024 at 1:15 PM Nathan Bossart <nathandbossart@gmail.com> wrote:

We could use git notes. The UI is a bit inconvenient (they have to be
pushed and pulled separately from commits), but they seem useful enough.

Yeah, I spend a lot of time on commit messages because they're pretty much
written in stone once pushed. I'd definitely use git notes to add errata,
follow-up commits that fixed/reverted things, etc.

I think this could be a good idea, although it wouldn't really fix the
problem, because in the case of both this commit and the one I
mentioned earlier today, all you could do with the note is point out
the earlier mistake. You couldn't actually fix it.

Also, for the notes to be useful, we'd probably need some conventions
about how we, as a project, want to use them. If everyone does
something different, the result isn't likely to be all that great.

--
Robert Haas
EDB: http://www.enterprisedb.com

On Wed, Aug 7, 2024 at 7:30 PM Robert Haas <robertmhaas@gmail.com> wrote:

On Wed, Aug 7, 2024 at 11:55 AM Alexander Korotkov <aekorotkov@gmail.com> wrote:

On Wed, Aug 7, 2024 at 3:24 PM Robert Haas <robertmhaas@gmail.com> wrote:

You may have already realized this, but the name of the function the
patch adds is not the same as the name that appears in the commit
message.

:sigh:
I didn't realize that before your message. That would be another item
for my checklist: ensure entities referenced from commit message and
comments didn't change their names.

I really wish there was some way to fix commit messages. I had a typo
in mine today, too.

+1,
One of the scariest things that happened to me was forgetting to
mention reviewers or even authors. People don't get credit for their
work, and you can't fix that.

------
Regards,
Alexander Korotkov
Supabase

On Wed, Aug 7, 2024 at 9:01 PM Robert Haas <robertmhaas@gmail.com> wrote:

On Wed, Aug 7, 2024 at 1:15 PM Nathan Bossart <nathandbossart@gmail.com> wrote:

We could use git notes. The UI is a bit inconvenient (they have to be
pushed and pulled separately from commits), but they seem useful enough.

Yeah, I spend a lot of time on commit messages because they're pretty much
written in stone once pushed. I'd definitely use git notes to add errata,
follow-up commits that fixed/reverted things, etc.

I think this could be a good idea, although it wouldn't really fix the
problem, because in the case of both this commit and the one I
mentioned earlier today, all you could do with the note is point out
the earlier mistake. You couldn't actually fix it.

Correct, but something looks better than nothing.

Also, for the notes to be useful, we'd probably need some conventions
about how we, as a project, want to use them. If everyone does
something different, the result isn't likely to be all that great.

+1

------
Regards,
Alexander Korotkov
Supabase

On Wed, Aug 7, 2024 at 1:34 PM Pavel Stehule <pavel.stehule@gmail.com> wrote:

st 7. 8. 2024 v 12:22 odesílatel Alexander Korotkov <aekorotkov@gmail.com> napsal:

Thank you.
Please see 40064a8ee1 takes special efforts to match HTAB hash
function to syscache hash function. By default, their hash values
don't match and you can't simply use syscache hash value to search
HTAB. This probably should be mentioned in the header comment of
hash_seq_init_with_hash_value().

yes, enhancing doc should be great + maybe assert

Please check the patch, which adds a caveat to the function header
comment. I don't particularly like an assert here, because there
could be use-cases besides syscache callbacks, which could legally use
default hash function.

------
Regards,
Alexander Korotkov
Supabase

st 7. 8. 2024 v 22:25 odesílatel Alexander Korotkov <aekorotkov@gmail.com>
napsal:

On Wed, Aug 7, 2024 at 1:34 PM Pavel Stehule <pavel.stehule@gmail.com>
wrote:

st 7. 8. 2024 v 12:22 odesílatel Alexander Korotkov <

aekorotkov@gmail.com> napsal:

Thank you.
Please see 40064a8ee1 takes special efforts to match HTAB hash
function to syscache hash function. By default, their hash values
don't match and you can't simply use syscache hash value to search
HTAB. This probably should be mentioned in the header comment of
hash_seq_init_with_hash_value().

yes, enhancing doc should be great + maybe assert

Please check the patch, which adds a caveat to the function header
comment. I don't particularly like an assert here, because there
could be use-cases besides syscache callbacks, which could legally use
default hash function.

looks well

Regards

Pavel

On Thu, Aug 8, 2024 at 7:45 AM Pavel Stehule <pavel.stehule@gmail.com> wrote:

st 7. 8. 2024 v 22:25 odesílatel Alexander Korotkov <aekorotkov@gmail.com> napsal:

On Wed, Aug 7, 2024 at 1:34 PM Pavel Stehule <pavel.stehule@gmail.com> wrote:

st 7. 8. 2024 v 12:22 odesílatel Alexander Korotkov <aekorotkov@gmail.com> napsal:

Thank you.
Please see 40064a8ee1 takes special efforts to match HTAB hash
function to syscache hash function. By default, their hash values
don't match and you can't simply use syscache hash value to search
HTAB. This probably should be mentioned in the header comment of
hash_seq_init_with_hash_value().

yes, enhancing doc should be great + maybe assert

Please check the patch, which adds a caveat to the function header
comment. I don't particularly like an assert here, because there
could be use-cases besides syscache callbacks, which could legally use
default hash function.

looks well

Thank you, pushed.

------
Regards,
Alexander Korotkov
Supabase

On Wed, Aug 07, 2024 at 02:01:30PM -0400, Robert Haas wrote:

Also, for the notes to be useful, we'd probably need some conventions
about how we, as a project, want to use them. If everyone does
something different, the result isn't likely to be all that great.

What did you have in mind? Would it be sufficient to propose a template
that, once ratified, would be available in the committing checklist [0]https://wiki.postgresql.org/wiki/Committing_checklist?

[0]: https://wiki.postgresql.org/wiki/Committing_checklist

--
nathan

On Thu, Aug 15, 2024 at 5:02 PM Nathan Bossart <nathandbossart@gmail.com> wrote:

On Wed, Aug 07, 2024 at 02:01:30PM -0400, Robert Haas wrote:

Also, for the notes to be useful, we'd probably need some conventions
about how we, as a project, want to use them. If everyone does
something different, the result isn't likely to be all that great.

What did you have in mind? Would it be sufficient to propose a template
that, once ratified, would be available in the committing checklist [0]?

[0] https://wiki.postgresql.org/wiki/Committing_checklist

Yes, that would suffice.

--
Robert Haas
EDB: http://www.enterprisedb.com