Problem with pl/python procedure connecting to the internet

Started by Igor Sosa Mayorover 10 years ago42 messagesgeneral
Jump to latest
#1Igor Sosa Mayor
joseleopoldo1792@gmail.com

Hi,

I'm a beginner with pgsql and have the following problem. I have written
a pl/python procedure to query geolocations (with the library geopy). It
inserts them into a table. All this works perfectly.

The problem is the configuration with the network connections. I'm using
pgsql in a private laptop. In order to let the procedure get results
from the internet I had to put

listen_addresses = '*' (in postgresql.conf) and
host all all 0.0.0.0/0 trust (in pg_hba.conf)

I don't think this is a good solution. Moreover: it doesn't work in all
places (work, home, etc.), since I imagine it depends on the fact that
the port is open.

My question is therefore:
1. is there a way to permit the pl/python to connect to the internet all
the time and with a better configuration?
2. or should I forget the procedure and write a python script outside
the database?

Many thanks in advance.

Igor Sosa Mayor

--
:: Igor Sosa Mayor :: joseleopoldo1792@gmail.com ::
:: GnuPG: 0x1C1E2890 :: http://www.gnupg.org/ ::
:: jabberid: rogorido :: ::

--
Sent via pgsql-general mailing list (pgsql-general@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-general

#2John R Pierce
pierce@hogranch.com
In reply to: Igor Sosa Mayor (#1)
Re: Problem with pl/python procedure connecting to the internet

listen_addresses should only affect the interfaces that the postgres
server is listening to connections from. as long as your app is on the
same machine, and uses localhost:someport to connect to the postgres
server, then the default listen_addresses='localhost' should be sufficient.'

if you want clients to connect from other systems, then listen_addresses
= '*' is appropriate.

listen_addresses should have no impact on what your plpython app can
connect to outside of postgres, unless you're running the 'safe' version
of pl***

--
john r pierce, recycling bits in santa cruz

--
Sent via pgsql-general mailing list (pgsql-general@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-general

#3Adrian Klaver
adrian.klaver@aklaver.com
In reply to: Igor Sosa Mayor (#1)
Re: Problem with pl/python procedure connecting to the internet

On 08/20/2015 10:30 PM, Igor Sosa Mayor wrote:

Hi,

I'm a beginner with pgsql and have the following problem. I have written
a pl/python procedure to query geolocations (with the library geopy). It
inserts them into a table. All this works perfectly.

The problem is the configuration with the network connections. I'm using
pgsql in a private laptop. In order to let the procedure get results
from the internet I had to put

This part I am uncertain about.

Are you talking about getting data into the procedure, or out of it, or
both?

listen_addresses = '*' (in postgresql.conf) and
host all all 0.0.0.0/0 trust (in pg_hba.conf)

I would at least change trust to md5, which would require a password.
You can further restrict by user and to database. If you set up
SSL(http://www.postgresql.org/docs/9.4/interactive/ssl-tcp.html) you can
use hostssl.

I don't think this is a good solution. Moreover: it doesn't work in all
places (work, home, etc.), since I imagine it depends on the fact that
the port is open.

listen_addresses just applies to what address Postgres listens to, in
this case all IP interfaces. The port it listens on is set by port =. In
any case this applies to the server, not to what your laptop can see.
That is probably shaped by any firewalls you are behind and changing
Postgres settings is not going to really help that.

My question is therefore:
1. is there a way to permit the pl/python to connect to the internet all
the time and with a better configuration?
2. or should I forget the procedure and write a python script outside
the database?

Not seeing that is going to help, you still have to connect to whatever
machine is hosting the script. Even if the script is on your laptop it
will still need to connect to the remote database and you are back to
trying to connect to Postgres.

Many thanks in advance.

Igor Sosa Mayor

--
Adrian Klaver
adrian.klaver@aklaver.com

--
Sent via pgsql-general mailing list (pgsql-general@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-general

#4Adrian Klaver
adrian.klaver@aklaver.com
In reply to: John R Pierce (#2)
Re: Problem with pl/python procedure connecting to the internet

On 08/21/2015 12:34 AM, John R Pierce wrote:

listen_addresses should only affect the interfaces that the postgres
server is listening to connections from. as long as your app is on the
same machine, and uses localhost:someport to connect to the postgres
server, then the default listen_addresses='localhost' should be
sufficient.'

if you want clients to connect from other systems, then listen_addresses
= '*' is appropriate.

listen_addresses should have no impact on what your plpython app can
connect to outside of postgres, unless you're running the 'safe' version
of pl***

There is no safe(trusted) version of plpython, it is only available as
plpythonu(ntrusted). So you are allowed to run with scissors:)

--
Adrian Klaver
adrian.klaver@aklaver.com

--
Sent via pgsql-general mailing list (pgsql-general@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-general

#5Igor Sosa Mayor
joseleopoldo1792@gmail.com
In reply to: Igor Sosa Mayor (#1)
Re: Problem with pl/python procedure connecting to the internet

John R Pierce <pierce@hogranch.com> writes:

listen_addresses should only affect the interfaces that the postgres
server is listening to connections from. as long as your app is on
the same machine, and uses localhost:someport to connect to the
postgres server, then the default listen_addresses='localhost' should
be sufficient.'

if you want clients to connect from other systems, then
listen_addresses = '*' is appropriate.

listen_addresses should have no impact on what your plpython app can
connect to outside of postgres, unless you're running the 'safe'
version of pl***

thanks a lot for your answer. I also thought that this config should not
be important, but it seems to be necessary. In any case: server and app
are on the same machine (my laptop), but for some reason the plpython
procedure cannot connect to the internet. It works very unreliable.

Have you maybe other hints?

Many thanks in advance.

--
:: Igor Sosa Mayor :: joseleopoldo1792@gmail.com ::
:: GnuPG: 0x1C1E2890 :: http://www.gnupg.org/ ::
:: jabberid: rogorido :: ::

--
Sent via pgsql-general mailing list (pgsql-general@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-general

#6Igor Sosa Mayor
joseleopoldo1792@gmail.com
In reply to: Igor Sosa Mayor (#1)
Re: Problem with pl/python procedure connecting to the internet

Adrian Klaver <adrian.klaver@aklaver.com> writes:

Are you talking about getting data into the procedure, or out of it,
or both?

[...]
(I reduce your email only to this question in order to be clearer; sorry
for this).

thanks for your answer.

To your questions: my setup is very easy and primitive:
1. postgres is in my laptop; I'm the only user and I will be the only
user of the system;
2. the script is also in my laptop;
3. the only thing which connects to the internet is the plpython I
wrote: it gets coordinates from Openstreetmap. The code I'm using is
the following: http://pastie.org/10366558

Then a trigger uses this code to insert the coordinates in a table.
Maybe I'm doing something wrong...

--
:: Igor Sosa Mayor :: joseleopoldo1792@gmail.com ::
:: GnuPG: 0x1C1E2890 :: http://www.gnupg.org/ ::
:: jabberid: rogorido :: ::

--
Sent via pgsql-general mailing list (pgsql-general@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-general

#7Adrian Klaver
adrian.klaver@aklaver.com
In reply to: Igor Sosa Mayor (#6)
Re: Problem with pl/python procedure connecting to the internet

On 08/21/2015 08:38 AM, Igor Sosa Mayor wrote:

Adrian Klaver <adrian.klaver@aklaver.com> writes:

Are you talking about getting data into the procedure, or out of it,
or both?

[...]
(I reduce your email only to this question in order to be clearer; sorry
for this).

thanks for your answer.

To your questions: my setup is very easy and primitive:
1. postgres is in my laptop; I'm the only user and I will be the only
user of the system;
2. the script is also in my laptop;
3. the only thing which connects to the internet is the plpython I
wrote: it gets coordinates from Openstreetmap. The code I'm using is
the following: http://pastie.org/10366558

Then a trigger uses this code to insert the coordinates in a table.
Maybe I'm doing something wrong...

Does the code work reliably outside Postgres?
It seems you are reliant on the availability of the OpenStreetMap
service and its ability to provide a timely response.

So what exactly is the trigger event, INSERT, UPDATE, both?

Is it absolutely necessary that this happen in a trigger?

--
Adrian Klaver
adrian.klaver@aklaver.com

--
Sent via pgsql-general mailing list (pgsql-general@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-general

#8Igor Sosa Mayor
joseleopoldo1792@gmail.com
In reply to: Igor Sosa Mayor (#1)
Re: Problem with pl/python procedure connecting to the internet

Adrian Klaver <adrian.klaver@aklaver.com> writes:

Does the code work reliably outside Postgres?
It seems you are reliant on the availability of the OpenStreetMap
service and its ability to provide a timely response.

So what exactly is the trigger event, INSERT, UPDATE, both?

Is it absolutely necessary that this happen in a trigger?

yes, it works reliable. I can also add a timeout option, but it does not
make any difference.

As for the trigger: well, I don't know. I wanted to be triggered every
time I add a city in the table 'ciudad'.postgres deals

Somehow I have the impression postgres is trying to make a connection
without having to rights to get the answer (I'm on linux). I dont
understand exactly which process (and with which rights) is executing
the plpython procedure...

Thanks in any case for your answer.

--
:: Igor Sosa Mayor :: joseleopoldo1792@gmail.com ::
:: GnuPG: 0x1C1E2890 :: http://www.gnupg.org/ ::
:: jabberid: rogorido :: ::

--
Sent via pgsql-general mailing list (pgsql-general@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-general

#9John R Pierce
pierce@hogranch.com
In reply to: Igor Sosa Mayor (#8)
Re: Problem with pl/python procedure connecting to the internet

On 8/22/2015 12:23 AM, Igor Sosa Mayor wrote:

Somehow I have the impression postgres is trying to make a connection
without having to rights to get the answer (I'm on linux). I dont
understand exactly which process (and with which rights) is executing
the plpython procedure...

Thanks in any case for your answer.

each client connection creates a postgresql server process. pl-anything
runs in that context, along with all sql for that client.

say... is by any chance selinux running in enforcing mode ? you might
try temporarily setting it to 'permissive' with the setenforce command,
and see if your python works.

--
john r pierce, recycling bits in santa cruz

if this is a reply to a list, I'm subscribed to that list, please reply to the list not me personally.

--
Sent via pgsql-general mailing list (pgsql-general@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-general

#10Igor Sosa Mayor
joseleopoldo1792@gmail.com
In reply to: Igor Sosa Mayor (#1)
Re: Problem with pl/python procedure connecting to the internet

John R Pierce <pierce@hogranch.com> writes:

On 8/22/2015 12:23 AM, Igor Sosa Mayor wrote:

Somehow I have the impression postgres is trying to make a connection
without having to rights to get the answer (I'm on linux). I dont
understand exactly which process (and with which rights) is executing
the plpython procedure...

Thanks in any case for your answer.

each client connection creates a postgresql server process.
pl-anything runs in that context, along with all sql for that client.

say... is by any chance selinux running in enforcing mode ? you
might try temporarily setting it to 'permissive' with the setenforce
command, and see if your python works.

thanks. No selinux is running.

Is there a way to check whether a process has the rights to make a
connection to the internet (TCP or something like that)?

--
Sent via pgsql-general mailing list (pgsql-general@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-general

#11Adrian Klaver
adrian.klaver@aklaver.com
In reply to: Igor Sosa Mayor (#8)
Re: Problem with pl/python procedure connecting to the internet

On 08/22/2015 12:23 AM, Igor Sosa Mayor wrote:

Adrian Klaver <adrian.klaver@aklaver.com> writes:

Does the code work reliably outside Postgres?
It seems you are reliant on the availability of the OpenStreetMap
service and its ability to provide a timely response.

So what exactly is the trigger event, INSERT, UPDATE, both?

Is it absolutely necessary that this happen in a trigger?

yes, it works reliable. I can also add a timeout option, but it does not
make any difference.

As for the trigger: well, I don't know. I wanted to be triggered every
time I add a city in the table 'ciudad'.postgres deals

Aargh, I was not paying attention.

Is the function you show here:

http://pastie.org/10366558

the actual function you are using in the trigger?

If so, there is no mechanism for getting the location coordinates into
the table, if that is what you want.

If that is not what you want, then what is the function supposed to do?

If it is not the trigger function, what is the actual function?

Somehow I have the impression postgres is trying to make a connection
without having to rights to get the answer (I'm on linux). I dont
understand exactly which process (and with which rights) is executing
the plpython procedure...

Thanks in any case for your answer.

--
Adrian Klaver
adrian.klaver@aklaver.com

--
Sent via pgsql-general mailing list (pgsql-general@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-general

#12Igor Sosa Mayor
joseleopoldo1792@gmail.com
In reply to: Igor Sosa Mayor (#1)
Re: Problem with pl/python procedure connecting to the internet

Adrian Klaver <adrian.klaver@aklaver.com> writes:

Is the function you show here:

http://pastie.org/10366558

the actual function you are using in the trigger?

If so, there is no mechanism for getting the location coordinates into
the table, if that is what you want.

this is the function and there is a trigger which gets the coordinates
into the table. It works perfectly when postgres is able to make a
connection and get the data. But this is exactly what works only
rarely... And I dont know the reason...

PS: this is the trigger, but as I said, it works if there is a
connection...
http://pastie.org/10368578

--
:: Igor Sosa Mayor :: joseleopoldo1792@gmail.com ::
:: GnuPG: 0x1C1E2890 :: http://www.gnupg.org/ ::
:: jabberid: rogorido :: ::

--
Sent via pgsql-general mailing list (pgsql-general@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-general

#13Adrian Klaver
adrian.klaver@aklaver.com
In reply to: Igor Sosa Mayor (#12)
Re: Problem with pl/python procedure connecting to the internet

On 08/22/2015 10:16 AM, Igor Sosa Mayor wrote:

Adrian Klaver <adrian.klaver@aklaver.com> writes:

Is the function you show here:

http://pastie.org/10366558

the actual function you are using in the trigger?

If so, there is no mechanism for getting the location coordinates into
the table, if that is what you want.

this is the function and there is a trigger which gets the coordinates
into the table. It works perfectly when postgres is able to make a
connection and get the data. But this is exactly what works only
rarely... And I dont know the reason...

What is your trigger definition? Please post here not on Pastie.

So if you just run the Postgres function on its own, not via the
trigger, does it always return a value?

Are you taking lack of coordinates in the table as evidence of no
connection or are you determining that some other way?

Does anything show up in the Postgres logs when the trigger runs?

Have you tried using plpy.notice in plpythonu and RAISE NOTICE in
plpgsql to track what is happening?

Also from here:

http://geopy.readthedocs.org/en/latest/#module-geopy.geocoders

geocode()

exactly_one (bool) � Return one result or a list of results, if available.

Not sure what is going to happen to your code if it gets a list instead
of a single value.

PS: this is the trigger, but as I said, it works if there is a
connection...
http://pastie.org/10368578

FYI, just include code snippets in the post. This keeps the code in the
thread and makes the thread not dependent on an external source. To that
end:

CREATE TYPE coordenadas AS (lat numeric, lon numeric);

CREATE OR REPLACE FUNCTION geocodificar(direccion text)
returns coordenadas
AS $$
from geopy.geocoders import Nominatim
geoloc = Nominatim()
location = geoloc.geocode(direccion)
return(location.latitude, location.longitude)
$$
LANGUAGE 'plpython2u';

CREATE OR REPLACE FUNCTION anadirgeocoord() RETURNS TRIGGER

AS $$
DECLARE
coord coordenadas;
BEGIN
SELECT * INTO coord from geocodificar(NEW.ciudad || ' ' ||
NEW.pais) ;
NEW.latitute := (coord).lat;
NEW.longitude := (coord).lon;
RETURN NEW;
END;
$$
LANGUAGE plpgsql;

--
Adrian Klaver
adrian.klaver@aklaver.com

--
Sent via pgsql-general mailing list (pgsql-general@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-general

#14Igor Sosa Mayor
joseleopoldo1792@gmail.com
In reply to: Igor Sosa Mayor (#1)
Re: Problem with pl/python procedure connecting to the internet

Adrian Klaver <adrian.klaver@aklaver.com> writes:

Are you taking lack of coordinates in the table as evidence of no
connection or are you determining that some other way?

thanks again for your answer. The evidence is very easy. I get this[1]ERROR: geopy.exc.GeocoderServiceError: <urlopen error [Errno -2] Nombre o servicio desconocido> CONTEXTO: Traceback (most recent call last): PL/Python function "geocodificar", line 4, in <module> location = geoloc.geocode(direccion) PL/Python function "geocodificar", line 190, in geocode PL/Python function "geocodificar", line 159, in _call_geocoder funci�n PL/Python �geocodificar� sentencia SQL: �SELECT * from geocodificar(NEW.ciudad || ' ' || NEW.pais)� funci�n PL/pgSQL anadirgeocoord() en la l�nea 5 en sentencia SQL.
BUT: if I try exactly the same python code in a console there is no
error and the coordinates are there.

Have you tried using plpy.notice in plpythonu and RAISE NOTICE in
plpgsql to track what is happening?

Yes, if the connection works, I get the correct answer. As I say: for
some reason postges can not make the connection OR get the answer (but
not always, it has worked only 3-4 times). All other things are working:
internet connection, python code...

[1]: ERROR: geopy.exc.GeocoderServiceError: <urlopen error [Errno -2] Nombre o servicio desconocido> CONTEXTO: Traceback (most recent call last): PL/Python function "geocodificar", line 4, in <module> location = geoloc.geocode(direccion) PL/Python function "geocodificar", line 190, in geocode PL/Python function "geocodificar", line 159, in _call_geocoder funci�n PL/Python �geocodificar� sentencia SQL: �SELECT * from geocodificar(NEW.ciudad || ' ' || NEW.pais)� funci�n PL/pgSQL anadirgeocoord() en la l�nea 5 en sentencia SQL
ERROR: geopy.exc.GeocoderServiceError: <urlopen error [Errno -2] Nombre o servicio desconocido>
CONTEXTO: Traceback (most recent call last):
PL/Python function "geocodificar", line 4, in <module>
location = geoloc.geocode(direccion)
PL/Python function "geocodificar", line 190, in geocode
PL/Python function "geocodificar", line 159, in _call_geocoder
funci�n PL/Python �geocodificar�
sentencia SQL: �SELECT * from geocodificar(NEW.ciudad || ' ' || NEW.pais)�
funci�n PL/pgSQL anadirgeocoord() en la l�nea 5 en sentencia SQL

--
Sent via pgsql-general mailing list (pgsql-general@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-general

#15Charles Clavadetscher
clavadetscher@swisspug.org
In reply to: Igor Sosa Mayor (#14)
Re: Problem with pl/python procedure connecting to the internet

-----Original Message-----
From: pgsql-general-owner@postgresql.org [mailto:pgsql-general-
owner@postgresql.org] On Behalf Of Igor Sosa Mayor
Sent: Sonntag, 23. August 2015 07:22
To: pgsql-general@postgresql.org
Subject: Re: [GENERAL] Problem with pl/python procedure connecting to the
internet

Adrian Klaver <adrian.klaver@aklaver.com> writes:

Are you taking lack of coordinates in the table as evidence of no
connection or are you determining that some other way?

thanks again for your answer. The evidence is very easy. I get this[1].
BUT: if I try exactly the same python code in a console there is no error

and

the coordinates are there.

Have you tried using plpy.notice in plpythonu and RAISE NOTICE in
plpgsql to track what is happening?

Yes, if the connection works, I get the correct answer. As I say: for some
reason postges can not make the connection OR get the answer (but not
always, it has worked only 3-4 times). All other things are working:
internet connection, python code...

[1]
ERROR: geopy.exc.GeocoderServiceError: <urlopen error [Errno -2] Nombre
o servicio desconocido>
CONTEXTO: Traceback (most recent call last):
PL/Python function "geocodificar", line 4, in <module>
location = geoloc.geocode(direccion)
PL/Python function "geocodificar", line 190, in geocode
PL/Python function "geocodificar", line 159, in _call_geocoder función
PL/Python «geocodificar»
sentencia SQL: «SELECT * from geocodificar(NEW.ciudad || ' '

||

NEW.pais)»
función PL/pgSQL anadirgeocoord() en la línea 5 en sentencia SQL

Are you sure that you really use exactly the same code? The indication
"survice unknown" seems to point to an error in the URL and not to an
authorization problem. Maybe some encoding problem?
The irritating thing is that you mention that it works sometimes and
sometimes not. Getting back to the encoding problem idea, is there a
difference if you call the function with a name containing special
characters (e.g. "é", "à", "ñ") or not?

Regards
Charles

--
Sent via pgsql-general mailing list (pgsql-general@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-general

#16Igor Sosa Mayor
joseleopoldo1792@gmail.com
In reply to: Igor Sosa Mayor (#1)
Re: Problem with pl/python procedure connecting to the internet

"Charles Clavadetscher" <clavadetscher@swisspug.org> writes:

Are you sure that you really use exactly the same code? The indication
"survice unknown" seems to point to an error in the URL and not to an
authorization problem. Maybe some encoding problem?

THnaks for your hints.

Yes, exactly the same code. Exactly the same query: two consoles, one
with postgres, other with ipython. Postgres gives the error; ipython
works.

The irritating thing is that you mention that it works sometimes and
sometimes not. Getting back to the encoding problem idea, is there a
difference if you call the function with a name containing special
characters (e.g. "�", "�", "�") or not?

No special characters in the query. To be honest: it worked only 2-3
times at work and I thought 'finally I managed to do it work'. But since
then, it is not working anymore...

In any case: I'm pretty sure it is a problem of postgres which is not
able to make the connection. If I put a parameter timeout=15,
postgres gives immediately the same error.

--
Sent via pgsql-general mailing list (pgsql-general@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-general

#17Charles Clavadetscher
clavadetscher@swisspug.org
In reply to: Igor Sosa Mayor (#16)
Re: Problem with pl/python procedure connecting to the internet

Hi

Weird. According to
http://geopy.readthedocs.org/en/latest/#geopy.exc.GeocoderServiceError this
is the most generic exception, only used when there is not a more specific
one (which include by the way problems caused by connectivity and
authorization). The message of the exception is supposed to be the one
delivered by the original application. In this case: "Nombre o servicio
desconocido":

I am wondering if there is an error message matching the one you receive in
PostgreSQL.

Are you sure that you really use exactly the same code? The indication
"survice unknown" seems to point to an error in the URL and not to an
authorization problem. Maybe some encoding problem?

THnaks for your hints.

Yes, exactly the same code. Exactly the same query: two consoles, one with
postgres, other with ipython. Postgres gives the error; ipython works.

Somehow I expected that answer. I am not a python expert, but it happens
that different pieces of code at the end do not use the same libraries or
framework code by configuration. But if you are positive on that, no more
discussion.

The irritating thing is that you mention that it works sometimes and
sometimes not. Getting back to the encoding problem idea, is there a
difference if you call the function with a name containing special
characters (e.g. "é", "à", "ñ") or not?

No special characters in the query. To be honest: it worked only 2-3 times

at

work and I thought 'finally I managed to do it work'. But since then, it

is not

working anymore...

Well, if it worked a few times and then never again, then something must
have changed in your system. This could also be a possible reason for the
error.
Sorry for asking obvious things, but it happened to me to oversee some of
them in the past: Are you making the calls from the same computer?

In any case: I'm pretty sure it is a problem of postgres which is not able

to

make the connection. If I put a parameter timeout=15, postgres gives
immediately the same error.

So it looks like a timeout problem with a confusing error message.
A possible way to follow this up would be to call another webservice from a
python function in PG and see if it behaves the same.
Another way to analyze the problem could be to sniff the network traffic and
see what really happens when you make the function call. This could help at
least to discard network problems and may lead to some useful information.

I hope that you will find a solution.
Bye
Charles

--
Sent via pgsql-general mailing list (pgsql-general@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-general

#18Adrian Klaver
adrian.klaver@aklaver.com
In reply to: Igor Sosa Mayor (#16)
Re: Problem with pl/python procedure connecting to the internet

On 08/23/2015 12:09 AM, Igor Sosa Mayor wrote:

"Charles Clavadetscher" <clavadetscher@swisspug.org> writes:

Are you sure that you really use exactly the same code? The indication
"survice unknown" seems to point to an error in the URL and not to an
authorization problem. Maybe some encoding problem?

THnaks for your hints.

Yes, exactly the same code. Exactly the same query: two consoles, one
with postgres, other with ipython. Postgres gives the error; ipython
works.

The irritating thing is that you mention that it works sometimes and
sometimes not. Getting back to the encoding problem idea, is there a
difference if you call the function with a name containing special
characters (e.g. "�", "�", "�") or not?

No special characters in the query. To be honest: it worked only 2-3
times at work and I thought 'finally I managed to do it work'. But since
then, it is not working anymore...

In any case: I'm pretty sure it is a problem of postgres which is not
able to make the connection. If I put a parameter timeout=15,
postgres gives immediately the same error.

What happens if you run the function below directly in psql and not
through the trigger?

CREATE OR REPLACE FUNCTION geocodificar(direccion text)
returns coordenadas
AS $$
from geopy.geocoders import Nominatim
geoloc = Nominatim()
location = geoloc.geocode(direccion)
return(location.latitude, location.longitude)
$$
LANGUAGE 'plpython2u';

Hmm, just noticed plpython2u.

Do you have both Python 2 and Python 3 on this machine?

Did you create both the 2u and 3u variants of plpythonu?

--
Adrian Klaver
adrian.klaver@aklaver.com

--
Sent via pgsql-general mailing list (pgsql-general@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-general

#19Sándor Daku
daku.sandor@gmail.com
In reply to: Igor Sosa Mayor (#16)
Re: Problem with pl/python procedure connecting to the internet

It might be a dumb thought, but...
Did you run the script from ipython as postgres user?
External programs called by triggers run as postgres, and maybe your
postgres user doesn't have rights to reach the remote service.

Regards,
Sandor

On 23 August 2015 at 09:09, Igor Sosa Mayor <joseleopoldo1792@gmail.com>
wrote:

Show quoted text

"Charles Clavadetscher" <clavadetscher@swisspug.org> writes:

Are you sure that you really use exactly the same code? The indication
"survice unknown" seems to point to an error in the URL and not to an
authorization problem. Maybe some encoding problem?

THnaks for your hints.

Yes, exactly the same code. Exactly the same query: two consoles, one
with postgres, other with ipython. Postgres gives the error; ipython
works.

The irritating thing is that you mention that it works sometimes and
sometimes not. Getting back to the encoding problem idea, is there a
difference if you call the function with a name containing special
characters (e.g. "é", "à", "ñ") or not?

No special characters in the query. To be honest: it worked only 2-3
times at work and I thought 'finally I managed to do it work'. But since
then, it is not working anymore...

In any case: I'm pretty sure it is a problem of postgres which is not
able to make the connection. If I put a parameter timeout=15,
postgres gives immediately the same error.

--
Sent via pgsql-general mailing list (pgsql-general@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-general

#20Adrian Klaver
adrian.klaver@aklaver.com
In reply to: Igor Sosa Mayor (#16)
Re: Problem with pl/python procedure connecting to the internet

On 08/23/2015 12:09 AM, Igor Sosa Mayor wrote:

"Charles Clavadetscher" <clavadetscher@swisspug.org> writes:

Are you sure that you really use exactly the same code? The indication
"survice unknown" seems to point to an error in the URL and not to an
authorization problem. Maybe some encoding problem?

THnaks for your hints.

Yes, exactly the same code. Exactly the same query: two consoles, one
with postgres, other with ipython. Postgres gives the error; ipython
works.

The irritating thing is that you mention that it works sometimes and
sometimes not. Getting back to the encoding problem idea, is there a
difference if you call the function with a name containing special
characters (e.g. "�", "�", "�") or not?

No special characters in the query. To be honest: it worked only 2-3
times at work and I thought 'finally I managed to do it work'. But since
then, it is not working anymore...

In any case: I'm pretty sure it is a problem of postgres which is not
able to make the connection. If I put a parameter timeout=15,
postgres gives immediately the same error.

I tried it here, using your functions, and could not get it to fail:

Running as postgres user.

test=# select version();
version

-----------------------------------------------------------------------------------------------------------------------------
PostgreSQL 9.4.2 on i686-pc-linux-gnu, compiled by gcc (SUSE Linux)
4.8.1 20130909 [gcc-4_8-branch revision 202388], 32-bit
(1 row)

Python 2. I did change the plpython2u to plpythonu in geocodificar as I
have only the one version of plpythonu installed.

test=# create table lat_long(ciudad text, pais text, latitute numeric
not null, longitude numeric not null);

create trigger lat_lon before insert or update on lat_long for each row
execute procedure anadirgeocoord();

test=# insert into lat_long (ciudad, pais) values ('Bellingham', 'USA');
INSERT 0 1
test=# insert into lat_long (ciudad, pais) values ('Seattle', 'USA');
INSERT 0 1
test=# insert into lat_long (ciudad, pais) values ('London', 'UK');
INSERT 0 1
test=# insert into lat_long (ciudad, pais) values ('Perth', 'AU');
INSERT 0 1
test=# select * from lat_long ;
ciudad | pais | latitute | longitude
------------+------+-------------+--------------
Bellingham | USA | 48.754402 | -122.4788602
Seattle | USA | 47.6038321 | -122.3300624
London | UK | 51.5073219 | -0.1276474
Perth | AU | -31.9527121 | 115.8604796
(4 rows)

test=# update lat_long set pais= 'UK' where ciudad ='Bellingham';
UPDATE 1
test=# select * from lat_long ;
ciudad | pais | latitute | longitude
------------+------+-------------+--------------
Seattle | USA | 47.6038321 | -122.3300624
London | UK | 51.5073219 | -0.1276474
Perth | AU | -31.9527121 | 115.8604796
Bellingham | UK | 55.1443709 | -2.2549272
(4 rows)

So it is something to do with the setup on your laptop.

--
Adrian Klaver
adrian.klaver@aklaver.com

--
Sent via pgsql-general mailing list (pgsql-general@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-general

#21Igor Sosa Mayor
joseleopoldo1792@gmail.com
In reply to: Igor Sosa Mayor (#1)
#22Igor Sosa Mayor
joseleopoldo1792@gmail.com
In reply to: Igor Sosa Mayor (#1)
#23Igor Sosa Mayor
joseleopoldo1792@gmail.com
In reply to: Igor Sosa Mayor (#1)
#24Igor Sosa Mayor
joseleopoldo1792@gmail.com
In reply to: Igor Sosa Mayor (#1)
#25Adrian Klaver
adrian.klaver@aklaver.com
In reply to: Igor Sosa Mayor (#22)
#26Adrian Klaver
adrian.klaver@aklaver.com
In reply to: Igor Sosa Mayor (#22)
#27Adrian Klaver
adrian.klaver@aklaver.com
In reply to: Igor Sosa Mayor (#24)
#28Sándor Daku
daku.sandor@gmail.com
In reply to: Igor Sosa Mayor (#23)
#29Igor Sosa Mayor
joseleopoldo1792@gmail.com
In reply to: Igor Sosa Mayor (#1)
#30Igor Sosa Mayor
joseleopoldo1792@gmail.com
In reply to: Igor Sosa Mayor (#1)
#31Adrian Klaver
adrian.klaver@aklaver.com
In reply to: Igor Sosa Mayor (#29)
#32Tom Lane
tgl@sss.pgh.pa.us
In reply to: Igor Sosa Mayor (#29)
#33John R Pierce
pierce@hogranch.com
In reply to: Tom Lane (#32)
#34Dave Potts
dave.potts@pinan.co.uk
In reply to: Tom Lane (#32)
#35Igor Sosa Mayor
joseleopoldo1792@gmail.com
In reply to: Igor Sosa Mayor (#1)
#36Adrian Klaver
adrian.klaver@aklaver.com
In reply to: Igor Sosa Mayor (#35)
#37Igor Sosa Mayor
joseleopoldo1792@gmail.com
In reply to: Igor Sosa Mayor (#1)
#38Igor Sosa Mayor
joseleopoldo1792@gmail.com
In reply to: Igor Sosa Mayor (#1)
#39Igor Sosa Mayor
joseleopoldo1792@gmail.com
In reply to: Igor Sosa Mayor (#1)
#40Igor Sosa Mayor
joseleopoldo1792@gmail.com
In reply to: Igor Sosa Mayor (#1)
#41Adrian Klaver
adrian.klaver@aklaver.com
In reply to: Igor Sosa Mayor (#37)
#42Igor Sosa Mayor
joseleopoldo1792@gmail.com
In reply to: Igor Sosa Mayor (#1)