[Fwd: Re: fulltext search stemming/ spelling problems]

Hi Corin

It looks like you have it working correctly, however you are expecting the
FTS to do a task it does not. The FTS will not automattically correct a
spelling error.
If the FTS auto corrected search tokens, this would likely lead to
undesirable results. I believe you are approaching the problem in the
wrong manner, a system should
not assume the input is incorrect, a user may wish to search for 'gitar'.
Additionally an incorrect term will expand into a range of possible terms,
user interaction is
needed to pick the desired search term.

Therefore your application will need to provide the spell checking
support. There are many libraries available like GNU Aspell
http://aspell.net/ or http://jazzy.sourceforge.net/.
I use Jazzy to provide spelling correction within the search engine I have
developed, for example:

http://search.shropshire.gov.uk/RICE/index.jsc?p=0&q=gitar&simple=Search&simplebutton=Search

The system checks tokenised input queries by using Jazzy. So a search for
'gitar' will offer the user a choice to search for 'guitar'.

Obviously, I do not fully know what you are trying to implement and
therefore can only tell you how I approached a similar problem.

I hope that is of some use.

Chris Ellis

pgsql-general-owner@postgresql.org wrote on 04/09/2010 02:53:19 PM:

Hi,

nobody here who knows how to get the postgre fulltext working with
ispell and stemming? :-(

So that when I search for 'gitar' also records containing 'guitar',
'guitars', ... will be found.

Any help would be really appreciated! :)

Thanks,
Corin

----- Message from Corin <wakathane@gmail.com> on Thu, 08 Apr 2010
22:54:38 +0200 -----

To:

Oleg Bartunov <oleg@sai.msu.su>, pgsql-general@postgresql.org

Subject:

Re: [GENERAL] fulltext search stemming/ spelling problems

On 08.04.2010 21:27, Oleg Bartunov wrote:

it means, that (from
http://www.postgresql.org/docs/current/static/textsearch-

dictionaries.html#TEXTSEARCH-ISPELL-DICTIONARY)

12.6.5. Ispell Dictionary

The Ispell dictionary template supports morphological dictionaries,
which can normalize many different linguistic forms of a word into the

same lexeme. For example, an English Ispell dictionary can match all
declensions and conjugations of the search term bank, e.g., banking,
banked, banks, banks', and bank's.

I already read this but I don't know how to solve my problems with this
information.

SELECT ts_lexize('english_ispell','guitar');
{guitar}
(1 line)

SELECT ts_lexize('english_ispell','bank');
{bank}
(1 line)

SELECT ts_debug('english_ispell','bank');
(asciiword,"Word, all
ASCII",bank,"{english_ispell,english_stem}",english_ispell,{bank})
(1 line)

SELECT plainto_tsquery('english_ispell','bank');
'bank'
(1 line)

Regards,
Oleg

It would be very nice if you (or anyone else) could provide me with
concrete instructions or any howto. What can I do to find the error in
my setup? What output should I expect from the above comments if
everything worked correctly?

Thanks,
Corin

--
Sent via pgsql-general mailing list (pgsql-general@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-general

******************************************************************************
If you are not the intended recipient of this email please do not send it on
to others, open any attachments or file the email locally.
Please inform the sender of the error and then delete the original email.
For more information, please refer to http://www.shropshire.gov.uk/privacy.nsf
******************************************************************************
Help prevent the spread of swine flu. CATCH IT. BIN IT. KILL IT.
******************************************************************************