What is the simpliest text search configuration?

Dear Jerome,

from personal experience full-text searching in PostgreSQL can be quite
powerful
but it's not simple, it requires thought, planning and coding. PostgreSQL
mainly
provides an efficient token matching mechanism supporting positional
information
and weights, but natural language processing and normalization is pretty
basic.

If you don't mind writing a couple of user-defined functions to take control
of lexeme
normalization, then tsvector/tsquery support can be a very powerful tool for
custom
search engines.

regards,

Michael

2009/11/12 Jérôme Etévé <jerome.eteve@gmail.com>

Hi Michael,

I actually found that the 'simple' dictionary doesn't enforce a
stopword list by default. so i defined my search conf like this and it
works:

create text search configuration sbsimple ( parser = 'default' ) ;
alter text search configuration sbsimple ALTER MAPPING FOR
word,hword,asciiword,asciihword WITH simple

Cheers!

J.

2009/11/12 Michael Nacos <m.nacos@gmail.com>:

Dear Jerome,

from personal experience full-text searching in PostgreSQL can be quite
powerful
but it's not simple, it requires thought, planning and coding. PostgreSQL
mainly
provides an efficient token matching mechanism supporting positional
information
and weights, but natural language processing and normalization is pretty
basic.

If you don't mind writing a couple of user-defined functions to take control
of lexeme
normalization, then tsvector/tsquery support can be a very powerful tool for
custom
search engines.

regards,

Michael

2009/11/12 Jérôme Etévé <jerome.eteve@gmail.com>

Hi all,

I'd like to implement a full text search with postgresql, and I can't
find
a text search configuration that would just:

map unicode accentuated letters to an un-accentuated equivalent
tokenize the words (and skip any non word characters)
no stopwords
lower case the tokens

How can I achieve this? I'm particularly interested in deactivating
the stopwords filtering.

I tried pg_catalog.simple, but despite its name, it still considers stop
words.

Thanks for your help!

Jerome.

--
Jerome Eteve.
http://www.eteve.net
jerome@eteve.net

=?UTF-8?B?SsOpcsO0bWUgRXTDqXbDqQ==?= <jerome.eteve@gmail.com> writes:

I'd like to implement a full text search with postgresql, and I can't find
a text search configuration that would just:

map unicode accentuated letters to an un-accentuated equivalent
tokenize the words (and skip any non word characters)
no stopwords
lower case the tokens

How can I achieve this? I'm particularly interested in deactivating
the stopwords filtering.

I tried pg_catalog.simple, but despite its name, it still considers stop words.

What's wrong with specifying an empty stopword list?

(To me, removing accents is already past what I'd expect of a "simple"
configuration, so I doubt you're going to find a dictionary that
provides exactly that set of features and no other ones.)

regards, tom lane

We submitted unaccent dictionary for 8.5
See http://www.sai.msu.su/~megera/wiki/unaccent for some information

Oleg
On Thu, 12 Nov 2009, Jrme Etv wrote:

Hi Michael,

I actually found that the 'simple' dictionary doesn't enforce a
stopword list by default. so i defined my search conf like this and it
works:

create text search configuration sbsimple ( parser = 'default' ) ;
alter text search configuration sbsimple ALTER MAPPING FOR
word,hword,asciiword,asciihword WITH simple

Cheers!

J.

2009/11/12 Michael Nacos <m.nacos@gmail.com>:

Dear Jerome,

from personal experience full-text searching in PostgreSQL can be quite
powerful
but it's not simple, it requires thought, planning and coding. PostgreSQL
mainly
provides an efficient token matching mechanism supporting positional
information
and weights, but natural language processing and normalization is pretty
basic.

If you don't mind writing a couple of user-defined functions to take control
of lexeme
normalization, then tsvector/tsquery support can be a very powerful tool for
custom
search engines.

regards,

Michael

2009/11/12 JЪЪrЪЪme EtЪЪvЪЪ <jerome.eteve@gmail.com>

Hi all,

I'd like to implement a full text search with postgresql, and I can't
find
a text search configuration that would just:

map unicode accentuated letters to an un-accentuated equivalent
tokenize the words (and skip any non word characters)
no stopwords
lower case the tokens

How can I achieve this? I'm particularly interested in deactivating
the stopwords filtering.

I tried pg_catalog.simple, but despite its name, it still considers stop
words.

Thanks for your help!

Jerome.

Regards,
Oleg
_____________________________________________________________
Oleg Bartunov, Research Scientist, Head of AstroNet (www.astronet.ru),
Sternberg Astronomical Institute, Moscow University, Russia
Internet: oleg@sai.msu.su, http://www.sai.msu.su/~megera/
phone: +007(495)939-16-83, +007(495)939-23-83