Re: PostgreSQL search engine (Perl)
On Fri, 31 May 2002, Kaare Rasmussen wrote:
Search the documentation does not work right now:
What I was trying to find was information about full text indexing.
It's supposed to be used in a search engine. But maybe someone knows a
search engine in Perl optimized for PostgreSQL?One that works ? ;-)
It works but it's not in perl and it doesn't do scoring but it is available
from the contrib tree as tsearch. I seem to remember there is another search
facility in there that is much closer to the original [open]FTS but I can't
remember what it is called. It's fairly obvious from the directory name though.
--
Nigel J. Andrews
Director
---
Logictree Systems Limited
Computer Consultants
Import Notes
Reply to msg id not found: 20020531105659.13013.qmail@post.webline.dk
I can recommend using the Lucene search-engine, it's in Java and it's got
nothing to do with PostgreSQL, but it performs excellently.
-----Original Message-----
From: Kaare Rasmussen [mailto:kar@kakidata.dk]
Sent: 31. maj 2002 12:57
To: pgsql-general@postgresql.org
Subject: [GENERAL] PostgreSQL search engine (Perl)
Search the documentation does not work right now:
What I was trying to find was information about full text indexing.
It's supposed to be used in a search engine. But maybe someone knows a
search engine in Perl optimized for PostgreSQL?
One that works ? ;-)
--
Kaare Rasmussen --Linux, spil,-- Tlf: 3816 2582
Kaki Data tshirts, merchandize Fax: 3816 2501
Howitzvej 75 Åben 14.00-18.00 Web: www.suse.dk
2000 Frederiksberg Lørdag 11.00-17.00 Email: kar@kakidata.dk
---------------------------(end of broadcast)---------------------------
TIP 1: subscribe and unsubscribe commands go to majordomo@postgresql.org
Import Notes
Resolved by subject fallback
You certainly need openfts ( openfts.sourceforge.net ) which is based
on contrib/tsearch module. tsearch is our step to integrate openfts search
engine into postgresql. I'd recommend download openfts from CVS -
it's stable and better than last official release. The only thing
we need is to add some documentation. Also I've finished web crawler
for personal site and cgi search script. But didn't commited yet to CVS.
The problem is lack of time. Sorry
Oleg
On Fri, 31 May 2002, Nigel J. Andrews wrote:
On Fri, 31 May 2002, Kaare Rasmussen wrote:
Search the documentation does not work right now:
What I was trying to find was information about full text indexing.
It's supposed to be used in a search engine. But maybe someone knows a
search engine in Perl optimized for PostgreSQL?One that works ? ;-)
It works but it's not in perl and it doesn't do scoring but it is available
from the contrib tree as tsearch. I seem to remember there is another search
facility in there that is much closer to the original [open]FTS but I can't
remember what it is called. It's fairly obvious from the directory name though.
Regards,
Oleg
_____________________________________________________________
Oleg Bartunov, sci.researcher, hostmaster of AstroNet,
Sternberg Astronomical Institute, Moscow University (Russia)
Internet: oleg@sai.msu.su, http://www.sai.msu.su/~megera/
phone: +007(095)939-16-83, +007(095)939-23-83
On Fri, 31 May 2002, Anders Nielsen wrote:
I can recommend using the Lucene search-engine, it's in Java and it's got
nothing to do with PostgreSQL, but it performs excellently.
main drawback of not-db-affiliated search engines is problem to access
metadata stored in db. Modern portals require a lot of such metadata,
think about authorization layer, versioning etc.
Besides that Lucene and many others like namazu (www.namazu.org)
are good enough. OpenFTS os our attempt to overcome metadata barrier.
-----Original Message-----
From: Kaare Rasmussen [mailto:kar@kakidata.dk]
Sent: 31. maj 2002 12:57
To: pgsql-general@postgresql.org
Subject: [GENERAL] PostgreSQL search engine (Perl)Search the documentation does not work right now:
What I was trying to find was information about full text indexing.
It's supposed to be used in a search engine. But maybe someone knows a
search engine in Perl optimized for PostgreSQL?One that works ? ;-)
--
Kaare Rasmussen --Linux, spil,-- Tlf: 3816 2582
Kaki Data tshirts, merchandize Fax: 3816 2501
Howitzvej 75 О©╫ben 14.00-18.00 Web: www.suse.dk
2000 Frederiksberg LО©╫rdag 11.00-17.00 Email: kar@kakidata.dk---------------------------(end of broadcast)---------------------------
TIP 1: subscribe and unsubscribe commands go to majordomo@postgresql.org---------------------------(end of broadcast)---------------------------
TIP 5: Have you checked our extensive FAQ?
Regards,
Oleg
_____________________________________________________________
Oleg Bartunov, sci.researcher, hostmaster of AstroNet,
Sternberg Astronomical Institute, Moscow University (Russia)
Internet: oleg@sai.msu.su, http://www.sai.msu.su/~megera/
phone: +007(095)939-16-83, +007(095)939-23-83
The problem with in-database search-engines are that they're usually tagged
on as a add-on feature and doesn't recieve the concentrated programming
effort that is required to create a high-performance full-text indexing
solution.
I'd be interested in comparing performance stats with people who use
OpenFTS, as I have a 8.5 gig fulltext Lucene index (including the indexed
text) and I am more than happy with the response times I'm seeing.
regards,
Anders Nielsen
-----Original Message-----
From: Oleg Bartunov [mailto:oleg@sai.msu.su]
Sent: 31. maj 2002 13:05
To: Anders Nielsen
Cc: 'Kaare Rasmussen'; pgsql-general@postgresql.org
Subject: Re: [GENERAL] PostgreSQL search engine (Perl)
On Fri, 31 May 2002, Anders Nielsen wrote:
I can recommend using the Lucene search-engine, it's in Java and it's got
nothing to do with PostgreSQL, but it performs excellently.
main drawback of not-db-affiliated search engines is problem to access
metadata stored in db. Modern portals require a lot of such metadata,
think about authorization layer, versioning etc.
Besides that Lucene and many others like namazu (www.namazu.org)
are good enough. OpenFTS os our attempt to overcome metadata barrier.
-----Original Message-----
From: Kaare Rasmussen [mailto:kar@kakidata.dk]
Sent: 31. maj 2002 12:57
To: pgsql-general@postgresql.org
Subject: [GENERAL] PostgreSQL search engine (Perl)Search the documentation does not work right now:
What I was trying to find was information about full text indexing.
It's supposed to be used in a search engine. But maybe someone knows a
search engine in Perl optimized for PostgreSQL?One that works ? ;-)
--
Kaare Rasmussen --Linux, spil,-- Tlf: 3816 2582
Kaki Data tshirts, merchandize Fax: 3816 2501
Howitzvej 75 еben 14.00-18.00 Web: www.suse.dk
2000 Frederiksberg LЬrdag 11.00-17.00 Email: kar@kakidata.dk---------------------------(end of broadcast)---------------------------
TIP 1: subscribe and unsubscribe commands go to majordomo@postgresql.org---------------------------(end of broadcast)---------------------------
TIP 5: Have you checked our extensive FAQ?
Regards,
Oleg
_____________________________________________________________
Oleg Bartunov, sci.researcher, hostmaster of AstroNet,
Sternberg Astronomical Institute, Moscow University (Russia)
Internet: oleg@sai.msu.su, http://www.sai.msu.su/~megera/
phone: +007(095)939-16-83, +007(095)939-23-83
Import Notes
Resolved by subject fallback
What kind of hardware resources do you have it sitting on? CPUs, memory,
disks?
On Fri, 31 May 2002, Anders Nielsen wrote:
Show quoted text
The problem with in-database search-engines are that they're usually tagged
on as a add-on feature and doesn't recieve the concentrated programming
effort that is required to create a high-performance full-text indexing
solution.I'd be interested in comparing performance stats with people who use
OpenFTS, as I have a 8.5 gig fulltext Lucene index (including the indexed
text) and I am more than happy with the response times I'm seeing.regards,
Anders Nielsen-----Original Message-----
From: Oleg Bartunov [mailto:oleg@sai.msu.su]
Sent: 31. maj 2002 13:05
To: Anders Nielsen
Cc: 'Kaare Rasmussen'; pgsql-general@postgresql.org
Subject: Re: [GENERAL] PostgreSQL search engine (Perl)On Fri, 31 May 2002, Anders Nielsen wrote:
I can recommend using the Lucene search-engine, it's in Java and it's got
nothing to do with PostgreSQL, but it performs excellently.main drawback of not-db-affiliated search engines is problem to access
metadata stored in db. Modern portals require a lot of such metadata,
think about authorization layer, versioning etc.
Besides that Lucene and many others like namazu (www.namazu.org)
are good enough. OpenFTS os our attempt to overcome metadata barrier.-----Original Message-----
From: Kaare Rasmussen [mailto:kar@kakidata.dk]
Sent: 31. maj 2002 12:57
To: pgsql-general@postgresql.org
Subject: [GENERAL] PostgreSQL search engine (Perl)Search the documentation does not work right now:
What I was trying to find was information about full text indexing.
It's supposed to be used in a search engine. But maybe someone knows a
search engine in Perl optimized for PostgreSQL?One that works ? ;-)
--
Kaare Rasmussen --Linux, spil,-- Tlf: 3816 2582
Kaki Data tshirts, merchandize Fax: 3816 2501
Howitzvej 75 ���ben 14.00-18.00 Web: www.suse.dk
2000 Frederiksberg L���rdag 11.00-17.00 Email: kar@kakidata.dk---------------------------(end of broadcast)---------------------------
TIP 1: subscribe and unsubscribe commands go to majordomo@postgresql.org---------------------------(end of broadcast)---------------------------
TIP 5: Have you checked our extensive FAQ?Regards,
Oleg
_____________________________________________________________
Oleg Bartunov, sci.researcher, hostmaster of AstroNet,
Sternberg Astronomical Institute, Moscow University (Russia)
Internet: oleg@sai.msu.su, http://www.sai.msu.su/~megera/
phone: +007(095)939-16-83, +007(095)939-23-83---------------------------(end of broadcast)---------------------------
TIP 1: subscribe and unsubscribe commands go to majordomo@postgresql.org
On Fri, 31 May 2002, Anders Nielsen wrote:
The problem with in-database search-engines are that they're usually tagged
on as a add-on feature and doesn't recieve the concentrated programming
effort that is required to create a high-performance full-text indexing
solution.
You're right.
RDBMS isn't good for high-performance full-text indexing, im my
opinion. I dont' know commercial databases which have built-in
fts support. I mean 'fair' built-in search engine with
transactions support and simultaneous indexing.
We did all the best we could at the moment and for small and
medium sites openfts is very good. There is room for improving
and we hope we'll keep our hands on it.
I'd be interested in comparing performance stats with people who use
OpenFTS, as I have a 8.5 gig fulltext Lucene index (including the indexed
text) and I am more than happy with the response times I'm seeing.
I bet Lucene will be a winner.
There is no magick in Lucene performance. Inverted indices are fast
for searching but very-very-very slow for updating.
Probably, your application doesnt require so much from database.
It's another story. We've built many search engines like Lucene and we do
know all pro's and contra's, but there are applicatios where transactional
integrity and access to metadata are much more important than performance.
regards,
Anders Nielsen-----Original Message-----
From: Oleg Bartunov [mailto:oleg@sai.msu.su]
Sent: 31. maj 2002 13:05
To: Anders Nielsen
Cc: 'Kaare Rasmussen'; pgsql-general@postgresql.org
Subject: Re: [GENERAL] PostgreSQL search engine (Perl)On Fri, 31 May 2002, Anders Nielsen wrote:
I can recommend using the Lucene search-engine, it's in Java and it's got
nothing to do with PostgreSQL, but it performs excellently.main drawback of not-db-affiliated search engines is problem to access
metadata stored in db. Modern portals require a lot of such metadata,
think about authorization layer, versioning etc.
Besides that Lucene and many others like namazu (www.namazu.org)
are good enough. OpenFTS os our attempt to overcome metadata barrier.-----Original Message-----
From: Kaare Rasmussen [mailto:kar@kakidata.dk]
Sent: 31. maj 2002 12:57
To: pgsql-general@postgresql.org
Subject: [GENERAL] PostgreSQL search engine (Perl)Search the documentation does not work right now:
What I was trying to find was information about full text indexing.
It's supposed to be used in a search engine. But maybe someone knows a
search engine in Perl optimized for PostgreSQL?One that works ? ;-)
--
Kaare Rasmussen --Linux, spil,-- Tlf: 3816 2582
Kaki Data tshirts, merchandize Fax: 3816 2501
Howitzvej 75 О©╫ben 14.00-18.00 Web: www.suse.dk
2000 Frederiksberg LО©╫rdag 11.00-17.00 Email: kar@kakidata.dk---------------------------(end of broadcast)---------------------------
TIP 1: subscribe and unsubscribe commands go to majordomo@postgresql.org---------------------------(end of broadcast)---------------------------
TIP 5: Have you checked our extensive FAQ?Regards,
Oleg
_____________________________________________________________
Oleg Bartunov, sci.researcher, hostmaster of AstroNet,
Sternberg Astronomical Institute, Moscow University (Russia)
Internet: oleg@sai.msu.su, http://www.sai.msu.su/~megera/
phone: +007(095)939-16-83, +007(095)939-23-83
Regards,
Oleg
_____________________________________________________________
Oleg Bartunov, sci.researcher, hostmaster of AstroNet,
Sternberg Astronomical Institute, Moscow University (Russia)
Internet: oleg@sai.msu.su, http://www.sai.msu.su/~megera/
phone: +007(095)939-16-83, +007(095)939-23-83
Single CPU Athlon Palomino 1800
2 IDE HD RAID 1
1.5 gigabytes DDR RAM
-----Original Message-----
From: Marc G. Fournier
To: Anders Nielsen
Cc: 'Oleg Bartunov'; 'Kaare Rasmussen'; pgsql-general@postgresql.org
Sent: 31-05-2002 16:25
Subject: Re: [GENERAL] PostgreSQL search engine (Perl)
This message uses a character set that is not supported by the Internet
Service. To view the original message content, open the attached
message. If the text doesn't display correctly, save the attachment to
disk, and then open it using a viewer that can display the original
character set. <<message.txt>>
Import Notes
Resolved by subject fallback