TSearch2 memory usage

On Mon, 21 May 2007, Hannes Dorbath wrote:

We have a fixed amount of 16 PG backend processes. Once created they stay
forever. OLTP load is distributed over them in a SQF fashion.

As loading a TSearch dictionary takes a few moments, we have a script that
connects to each backend on start up and pushes the dictionary into ram by
simply calling ts_debug('foo');

The dictionary has a file size of 9.8MB.

If we don't call the script `free' prints:

total used free shared buffers cached
Mem: 4048056 953192 3094864 0 4 359300
-/+ buffers/cache: 593888 3454168

After the script has called ts_debug('foo') on each backend:

total used free shared buffers cached
Mem: 4048056 2374508 1673548 0 4 370340
-/+ buffers/cache: 2004164 2043892

Is it supposed to blast so much memory?

dictionary is just a program ! Are you usin custom dictionaries ?

what does 'select * from ts_debug('foo');' prints ?

This is PG 8.2.4 on x86_64.

Regards,
Oleg
_____________________________________________________________
Oleg Bartunov, Research Scientist, Head of AstroNet (www.astronet.ru),
Sternberg Astronomical Institute, Moscow University, Russia
Internet: oleg@sai.msu.su, http://www.sai.msu.su/~megera/
phone: +007(495)939-16-83, +007(495)939-23-83

On 21.05.2007 11:42, Oleg Bartunov wrote:

dictionary is just a program ! Are you usin custom dictionaries ?

Depends on what you mean, it's a standard ispell dict, defined as:

DictFile="/foo/bar/german_utf8.dict",AffFile="/foo/bar/german_utf8.aff"

You can grab both files here: http://falcon.imos.net/dict.bz2

IIRC I have created them from an older, german OO dict, iconv'd them to
UTF-8 and run compound.pl on them.

The compound stuff stopped to function on 8.2 and as I couldn't find a
solution I stripped the compound z line from the affix file.

what does 'select * from ts_debug('foo');' prints ?

ts_name | tok_type | description | token | dict_name | tsvector
-------------+----------+-------------+-------+----------------+----------
utf8_german | lword | Latin word | foo | {de_ispell,de} | 'foo'

Thanks for your time.

--
Regards,
Hannes Dorbath