Hstore::TooBig ?

On Oct 31, 2013, at 3:46 PM, john gale <john@smadness.com> wrote:

I don't quite know where this error is coming from. The ActiveRecord source doesn't seem to have it, and I'm not familiar enough with Rails or ActiveRecord to track definitively whether the failing function is actually performing a DB operation. Moreso I cannot seem to find any mentions (post 9.0 at least) of Hstore size limitations, either key/val pairs (which 8.x docs said was 65k) or the hstore field in its entirety.

I should note there is no error thrown into my postgres log at all.

~ john

On 10/31/2013 03:46 PM, john gale wrote:

I am running a somewhat unfamiliar Ruby automation results app using ActiveRecord to manage the postgres 9.0 backend. During our automation runs we sometimes get bursts of HTTP 500 errors coming back at us, and the Ruby app log shows an Hstore::TooBig error:

Hstore::TooBig (Hstore::TooBig):
vendor/bundle/ruby/1.8/gems/activerecord-2.3.12/lib/active_record/base.rb:3045:in `attributes_with_quotes'
vendor/bundle/ruby/1.8/gems/activerecord-2.3.12/lib/active_record/base.rb:3036:in `each'
vendor/bundle/ruby/1.8/gems/activerecord-2.3.12/lib/active_record/base.rb:3036:in `attributes_with_quotes'
vendor/bundle/ruby/1.8/gems/activerecord-2.3.12/lib/active_record/base.rb:2934:in `update_without_lock'
vendor/bundle/ruby/1.8/gems/activerecord-2.3.12/lib/active_record/locking/optimistic.rb:81:in `update_without_dirty'
vendor/bundle/ruby/1.8/gems/activerecord-2.3.12/lib/active_record/dirty.rb:146:in `update_without_timestamps'
vendor/bundle/ruby/1.8/gems/activerecord-2.3.12/lib/active_record/timestamp.rb:64:in `update_without_callbacks'
vendor/bundle/ruby/1.8/gems/activerecord-2.3.12/lib/active_record/callbacks.rb:282:in `update'
vendor/bundle/ruby/1.8/gems/activerecord-2.3.12/lib/active_record/base.rb:2927:in `create_or_update_without_callbacks'
vendor/bundle/ruby/1.8/gems/activerecord-2.3.12/lib/active_record/callbacks.rb:250:in `create_or_update'
vendor/bundle/ruby/1.8/gems/activerecord-2.3.12/lib/active_record/base.rb:2594:in `save_without_validation!'
vendor/bundle/ruby/1.8/gems/activerecord-2.3.12/lib/active_record/validations.rb:1100:in `save_without_dirty!'
vendor/bundle/ruby/1.8/gems/activerecord-2.3.12/lib/active_record/dirty.rb:87:in `save_without_transactions!'
vendor/bundle/ruby/1.8/gems/activerecord-2.3.12/lib/active_record/transactions.rb:200:in `save!'
vendor/bundle/ruby/1.8/gems/activerecord-2.3.12/lib/active_record/connection_adapters/abstract/database_statements.rb:136:in `transaction'
vendor/bundle/ruby/1.8/gems/activerecord-2.3.12/lib/active_record/transactions.rb:182:in `transaction'
vendor/bundle/ruby/1.8/gems/activerecord-2.3.12/lib/active_record/transactions.rb:200:in `save!'
vendor/bundle/ruby/1.8/gems/activerecord-2.3.12/lib/active_record/transactions.rb:208:in `rollback_active_record_state!'
vendor/bundle/ruby/1.8/gems/activerecord-2.3.12/lib/active_record/transactions.rb:200:in `save!'
app/helpers/logger_helper.rb:72:in `update_testrun'
app/models/testrun.rb:267:in `transaction_with_isolation_level'
...

I don't quite know where this error is coming from. The ActiveRecord source doesn't seem to have it, and I'm not familiar enough with Rails or ActiveRecord to track definitively whether the failing function is actually performing a DB operation. Moreso I cannot seem to find any mentions (post 9.0 at least) of Hstore size limitations, either key/val pairs (which 8.x docs said was 65k) or the hstore field in its entirety.

So to piece out the questions,
- is there a total size limitation of the hstore field? or is it theoretically large enough (1GB) that it really shouldn't matter?
- is there a string size limitation of each key/val in 9.x, or is it still 65k from 8.x?
- is Hstore::TooBig an error that is likely coming from postgres? or should I continue poking at ActiveRecord or other places?

Two things I can think of.

1) Grep the above gems for the Hstore::TooBig message

2) Ask on a Rails list, in light of your second post where you say the
error does not touch the Postgres log. Would seem to indicate a
Ruby/Rails issue.

thanks,

~ john

--
Adrian Klaver
adrian.klaver@gmail.com

--
Sent via pgsql-general mailing list (pgsql-general@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-general

On Oct 31, 2013, at 4:44 PM, Adrian Klaver <adrian.klaver@gmail.com> wrote:

So to piece out the questions,
- is there a total size limitation of the hstore field? or is it theoretically large enough (1GB) that it really shouldn't matter?
- is there a string size limitation of each key/val in 9.x, or is it still 65k from 8.x?
- is Hstore::TooBig an error that is likely coming from postgres? or should I continue poking at ActiveRecord or other places?

Two things I can think of.

1) Grep the above gems for the Hstore::TooBig message

2) Ask on a Rails list, in light of your second post where you say the error does not touch the Postgres log. Would seem to indicate a Ruby/Rails issue.

These were done in parallel, which is why I was asking the question on a postgres list.

The further questions still apply though; are there documented size limitations for hstore?

~ john

On Oct 31, 2013, at 4:44 PM, Adrian Klaver <adrian.klaver@gmail.com> wrote:

So to piece out the questions,
- is there a total size limitation of the hstore field? or is it theoretically large enough (1GB) that it really shouldn't matter?
- is there a string size limitation of each key/val in 9.x, or is it still 65k from 8.x?
- is Hstore::TooBig an error that is likely coming from postgres? or should I continue poking at ActiveRecord or other places?

Two things I can think of.

1) Grep the above gems for the Hstore::TooBig message

2) Ask on a Rails list, in light of your second post where you say the error does not touch the Postgres log. Would seem to indicate a Ruby/Rails issue.

These were done in parallel, which is why I was asking the question on a postgres list.

The further questions still apply though; are there documented size limitations for hstore?

~ john

On 10/31/2013 5:11 PM, john gale wrote:

The further questions still apply though; are there documented size
limitations for hstore?

afaik only the practical limits of how big you want your tables to get,
and the 1GB limit of the underlying text representation. having many
1000s of keys in a single hstore would likely make for some painful
performance, and nasty memory requirements (returning all keynames, for
instance, so you could enumerate them, ouch)

--
john r pierce 37N 122W
somewhere on the middle of the left coast

--
Sent via pgsql-general mailing list (pgsql-general@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-general