plperl & sort

Jeff <threshar@threshar.is-a-geek.com> writes:

I've ran into this interesting problem.
It seems that while you can call sort() in a trusted plperl func you
cannot access $a & $b which effectively makes it useless.

I've tested this on 8.2.11, 8.3.5, and the nov 4 snapshot on ftp.postgresql.org
In all cases its on a mac with perl 5.8.8.

I can confirm this behavior with perl 5.10 on Fedora 9. I suppose the
Safe module is somehow blocking the variable accesses, but if so why
doesn't it throw an outright error? Is this a Safe bug, or are we
failing to enable something we should, or perhaps it's actually
necessary to block this for security reasons?? Requires more perl-fu
than I have, unfortunately.

regards, tom lane

On Tue, Nov 4, 2008 at 09:02, Jeff <threshar@torgo.978.org> wrote:

I've ran into this interesting problem.
It seems that while you can call sort() in a trusted plperl func you cannot
access $a & $b which effectively makes it useless.

Hrm works for me if I take out the elog from sort()

create or replace function trustedsort()
returns int
as $$

my @arr = qw(5 4 3 2 1);

my @sorted = sort { $a <=> $b } @arr;

elog(NOTICE, join(' ', @sorted));

return 1;

$$
language 'plperl';

SELECT trustedsort();
NOTICE: 1 2 3 4 5
trustedsort
-------------
1
(1 row)

"Alex Hunsaker" <badalex@gmail.com> writes:

Hrm works for me if I take out the elog from sort()

Even more interesting, this variant *doesn't* work:

regression=# create or replace function trustedsort()
returns int
as $$
my @arr = qw(5 4 3 2 1);
my @sorted = sort { "$a" <=> "$b" } @arr;
elog(NOTICE, join(' ', @sorted));
return 1;
$$
language 'plperl';
CREATE FUNCTION
regression=# select trustedsort();
NOTICE: 5 4 3 2 1
trustedsort
-------------
1
(1 row)

Seems like it's the interpolation into a string that is failing.

regards, tom lane

On Tue, Nov 4, 2008 at 12:39, Tom Lane <tgl@sss.pgh.pa.us> wrote:

"Alex Hunsaker" <badalex@gmail.com> writes:

Hrm works for me if I take out the elog from sort()

Even more interesting, this variant *doesn't* work:

regression=# create or replace function trustedsort()
returns int
as $$
my @arr = qw(5 4 3 2 1);
my @sorted = sort { "$a" <=> "$b" } @arr;
elog(NOTICE, join(' ', @sorted));
return 1;
$$
language 'plperl';
CREATE FUNCTION
regression=# select trustedsort();
NOTICE: 5 4 3 2 1
trustedsort
-------------
1
(1 row)

Seems like it's the interpolation into a string that is failing.

It has something to do with anon subs not sure what...
see below test case

This works:

require Safe;

my $safe = Safe->new('PLPerl');
$safe->permit_only(':default');
$safe->permit(qw(sort));
$safe->share(qw(&j));

sub j
{
print "j called ". (shift) . "\n";
}

my $f = $safe->reval(<<'z');
sub blah {
my @c = sort { j("$a $b"); $a <=> $b } qw(5 4 3 2 1);
j(join(" ", @c));
return;
}

blah();
z

$ perl tsafe.pl
j called 5 4
j called 3 2
j called 4 2
j called 4 3
j called 2 1
j called 1 2 3 4 5

This fails: (which is what we do in plperl.c)
my $f = $safe->reval(<<'z');
sub {
my @c = sort { j("$a $b"); $a <=> $b } qw(5 4 3 2 1);
j(join(" ", @c));
return;
}
z

$f->();

$ perl tsafe.pl
j called
j called
j called
j called
j called
j called
j called
j called
j called 5 4 3 2 1

This works:
$safe->reval(<<'z');
my @c = sort { j("$a $b"); $a <=> $b } qw(5 4 3 2 1);
j(join(" ", @c));
return;
z

$ perl tsafe.pl
j called 5 4
j called 3 2
j called 4 2
j called 4 3
j called 2 1
j called 1 2 3 4 5

Dunno...

On Tue, Nov 4, 2008 at 12:43, Alex Hunsaker <badalex@gmail.com> wrote:

It has something to do with anon subs not sure what...

It has to do with us returning the anonymous sub inside of the safe
and then calling the function outside of the safe (or at least in a
different namespace)

we do something eqvilient to this:
my $func_ptr = $safe->reval('sub { ... }');
$func_ptr->();

because safe makes its own namespace from perldoc Safe
The "root" of the namespace (i.e. "main::") is changed to a
different package and code evaluated in the compartment cannot
refer to variables outside this namespace, even with run-time
glob lookups and other tricks.

I only see one way to "fix" this which is to do something groddy like
share a global variable between the safe and the real interpreter.
Something like:

my $_pl_sub;
sub call_pl_sub
{
retrun $_pl_sub;
}

$safe->share(qw(call_pl_sub);

my $sub = $safe->reval('sub { ...}');

$_pl_sub = $sub;
$safe->reval('call_pl_sub();');

Note I tried just sharing $_pl_sub and doing
$safe->reval('$_pl_sub->()'); but I just get 'Undefined subroutine
&main::'

Should I work up a patch? Assuming someone confirm this?

On Nov 4, 2008, at 2:27 PM, Alex Hunsaker wrote:

On Tue, Nov 4, 2008 at 09:02, Jeff <threshar@torgo.978.org> wrote:

I've ran into this interesting problem.
It seems that while you can call sort() in a trusted plperl func
you cannot
access $a & $b which effectively makes it useless.

Hrm works for me if I take out the elog from sort()

I came across this because I was attempting to sort some data (an
array of hashrefs) in to reverse order and got very odd results.. some
elogging showed $a and $b were not what they should have been and
after more and more digging I was able to widdle it down to the simple
case I posted. When I tried having it call a sub instead of an
anonymous block it would complain the sub didn't exist. (I have other
plperl functions that have subs declared and they all work fine, but I
never used them with sort before).

I'll have some more time to tinker with it tomorrow. I'm reasonably
sure its got something to do with the Safe module and some magic-fu we
may need. Looking at plperl we do allow sort so I'm not sure why $a &
$b disappear..

--
Jeff Trout <jeff@jefftrout.com>
http://www.stuarthamm.net/
http://www.dellsmartexitin.com/

Alex Hunsaker wrote:

On Tue, Nov 4, 2008 at 12:43, Alex Hunsaker <badalex@gmail.com> wrote:

It has something to do with anon subs not sure what...

It has to do with us returning the anonymous sub inside of the safe
and then calling the function outside of the safe (or at least in a
different namespace)

we do something eqvilient to this:
my $func_ptr = $safe->reval('sub { ... }');
$func_ptr->();

because safe makes its own namespace from perldoc Safe
The "root" of the namespace (i.e. "main::") is changed to a
different package and code evaluated in the compartment cannot
refer to variables outside this namespace, even with run-time
glob lookups and other tricks.

I only see one way to "fix" this which is to do something groddy like
share a global variable between the safe and the real interpreter.
Something like:

my $_pl_sub;
sub call_pl_sub
{
retrun $_pl_sub;
}

$safe->share(qw(call_pl_sub);

my $sub = $safe->reval('sub { ...}');

$_pl_sub = $sub;
$safe->reval('call_pl_sub();');

Note I tried just sharing $_pl_sub and doing
$safe->reval('$_pl_sub->()'); but I just get 'Undefined subroutine
&main::'

Should I work up a patch? Assuming someone confirm this?

OK, the first thing to note is that there is an easy workaround, which
is to use a sort routine that doesn't need $a/$b. Example:

create or replace function mysort() returns text language plperl as $f$

my $sfunc = sub ($$) { $_[0] <=> $_[1] };

my @vals = (5,3,4,2,7);

return join(' ',sort $sfunc @vals);

$f$;

We need to document that, and given that this exists I think we don't
need to backpatch old versions.

Beyond that, we need to be very careful with any "solution" that we
don't upset the moderately fragile security of trusted plperl, and I'm
going to look fairly skeptically at anything that changes the way we set
up and call functions. But by all means if you can come up with a robust
way of allowing the more traditional way of calling sort routines, send
it in. Sharing globals between the Safe and non-Safe worlds is not a
solution - we removed an instance of that not long ago for security reasons.

cheers

andrew

On Tue, Nov 4, 2008 at 14:43, Andrew Dunstan <andrew@dunslane.net> wrote:

We need to document that, and given that this exists I think we don't need
to backpatch old versions.

Agreed.

Beyond that, we need to be very careful with any "solution" that we don't
upset the moderately fragile security of trusted plperl, and I'm going to
look fairly skeptically at anything that changes the way we set up and call
functions. But by all means if you can come up with a robust way of allowing
the more traditional way of calling sort routines, send it in.

Well its not just sort its anything that uses main:: right?

Sharing
globals between the Safe and non-Safe worlds is not a solution - we removed
an instance of that not long ago for security reasons.

Oh defiantly :) just tossing out ideas. Instead of storing the sub we
could just call Safe::reval() everytime... that seems the safest way
to me.

The other idea Ive been toying this is instead of calling reval we can
just call Opcode::_safe_call_sv() something like the below:

I verified it on perl 5.10.0 only but I looked at 5.8.8 and those
routines in Safe.pm are the same so it should be relatively safe...
Note this is *exactly* what reval does except we already do our own
strict import. and it only works for CODE refs.

*** a/src/pl/plperl/plperl.c
--- b/src/pl/plperl/plperl.c
***************
*** 283,295 **** _PG_init(void)
  	"&_plperl_to_pg_array " \
  	"&DEBUG &LOG &INFO &NOTICE &WARNING &ERROR %_SHARED ]);" \
  	"sub ::mksafefunc {" \
! 	"      my $ret = $PLContainer->reval(qq[sub { $_[0] $_[1] }]); " \
! 	"      $@ =~ s/\\(eval \\d+\\) //g if $@; return $ret; }" \
  	"$PLContainer->permit(qw[require caller]); $PLContainer->reval('use
strict;');" \
  	"$PLContainer->deny(qw[require caller]); " \
  	"sub ::mk_strict_safefunc {" \
! 	"      my $ret = $PLContainer->reval(qq[sub { BEGIN {
strict->import(); } $_[0] $_[1] }]); " \
! 	"      $@ =~ s/\\(eval \\d+\\) //g if $@; return $ret; }"

  #define SAFE_BAD \
  	"use vars qw($PLContainer); $PLContainer = new Safe('PLPerl');" \
--- 283,299 ----
  	"&_plperl_to_pg_array " \
  	"&DEBUG &LOG &INFO &NOTICE &WARNING &ERROR %_SHARED ]);" \
  	"sub ::mksafefunc {" \
! 	"      my $__ExPr__ = $PLContainer->reval(qq[sub { $_[0] $_[1] }]); " \
! 	"      $@ =~ s/\\(eval \\d+\\) //g if $@; " \
! 	"      my $sub = eval 'package '. $PLContainer->{Root} .'; sub {
@_=(); $__ExPr__->(); }'; " \
! 	"      return sub { Opcode::_safe_call_sv($PLContainer->{Root},
$PLContainer->{Mask}, $sub); }; } "\
  	"$PLContainer->permit(qw[require caller]); $PLContainer->reval('use
strict;');" \
  	"$PLContainer->deny(qw[require caller]); " \
  	"sub ::mk_strict_safefunc {" \
! 	"      my $__ExPr__ = $PLContainer->reval(qq[sub { BEGIN {
strict->import(); } $_[0] $_[1] }]); " \
! 	"      $@ =~ s/\\(eval \\d+\\) //g if $@; "\
! 	"      my $sub = eval 'package '. $PLContainer->{Root} .'; sub {
@_=(); $__ExPr__->(); }'; " \
! 	"      return sub { Opcode::_safe_call_sv($PLContainer->{Root},
$PLContainer->{Mask}, $sub); }; }"

#define SAFE_BAD \
"use vars qw($PLContainer); $PLContainer = new Safe('PLPerl');" \

On Tue, Nov 4, 2008 at 15:02, Alex Hunsaker <badalex@gmail.com> wrote:

The other idea Ive been toying this is instead of calling reval we can
just call Opcode::_safe_call_sv() something like the below:

Argh gmail probably ate the whitespace in the patch... see attached

On Tue, Nov 4, 2008 at 15:02, Alex Hunsaker <badalex@gmail.com> wrote:

On Tue, Nov 4, 2008 at 14:43, Andrew Dunstan <andrew@dunslane.net> wrote:

But by all means if you can come up with a robust way of allowing

the more traditional way of calling sort routines, send it in.

Well its not just sort its anything that uses main:: right?

Err no you're right its only builtins that use main:: sort being the
only one I know of off the top of my head... its a shame
PLContainer->share('$main::a'); does not seem to work..

Alex Hunsaker wrote:

On Tue, Nov 4, 2008 at 15:02, Alex Hunsaker <badalex@gmail.com> wrote:

On Tue, Nov 4, 2008 at 14:43, Andrew Dunstan <andrew@dunslane.net> wrote:

But by all means if you can come up with a robust way of allowing

the more traditional way of calling sort routines, send it in.

Well its not just sort its anything that uses main:: right?

Err no you're right its only builtins that use main:: sort being the
only one I know of off the top of my head... its a shame
PLContainer->share('$main::a'); does not seem to work..

$a and $b are magical *package* variables. See "perldoc perlvar". This
has nothing whatever to do with main::

cheers

andrew

On Tue, Nov 4, 2008 at 15:17, Andrew Dunstan <andrew@dunslane.net> wrote:

Alex Hunsaker wrote:

Err no you're right its only builtins that use main:: sort being the
only one I know of off the top of my head... its a shame
PLContainer->share('$main::a'); does not seem to work..

$a and $b are magical *package* variables. See "perldoc perlvar". This has
nothing whatever to do with main::

Hah right! The perl is strong in this one! =)

I was just remember seeing warnings from typos like:
$ perl -We '$a = $b;'
Name "main::a" used only once: possible typo at -e line 1.
Name "main::b" used only once: possible typo at -e line 1.

... but that's neither here nor there

Tom Lane wrote:

Jeff <threshar@threshar.is-a-geek.com> writes:

I've ran into this interesting problem.
It seems that while you can call sort() in a trusted plperl func you
cannot access $a & $b which effectively makes it useless.

I've tested this on 8.2.11, 8.3.5, and the nov 4 snapshot on ftp.postgresql.org
In all cases its on a mac with perl 5.8.8.

I can confirm this behavior with perl 5.10 on Fedora 9. I suppose the
Safe module is somehow blocking the variable accesses, but if so why
doesn't it throw an outright error? Is this a Safe bug, or are we
failing to enable something we should, or perhaps it's actually
necessary to block this for security reasons?? Requires more perl-fu
than I have, unfortunately.

Completely untested speculation based on my knowledge of perl and
a bit of reading:

The reason you can't see $a and $b is that sort internally sets
these variables in the main package. That is, sort is setting
$main::a and $main::b, and when you run the plperl code in the
safe compartment, main:: isn't visible any more.

The reason you don't get an error is that unadorned $a and $b
which you reference in the sort routine is relative to the
namespace you give to Safe. That is, your sort sub is trying
to access $PLPerl::a and $PLPerl::b which isn't what is
set by sort.

It looks like there are two fixes that should work, one sort based
and one Safe based.

sort based: use a subroutine with a prototype. From perldoc -f sort:

If the subroutine�s prototype is "($$)", the elements to be
compared are passed by reference in @_, as for a normal
subroutine.

Safe based: share the $a and $b variables with the compartment.

$compartment->share_from('main', '$a', '$b');

I'm not sure how postgres embeds perl. Depending on how the
interpreters are set up, it is conceivable that the contents
of $a and $b could be leaked to other "threads" or similar that
are using the same interpreter. In any case, using the
share_from() method of Safe would have to be changed at
the postgres level rather than the untrusted language
function writer's level.

I can do some testing if anyone needs something more than
the above suggestions.

--
nw

On Nov 4, 2008, at 4:43 PM, Andrew Dunstan wrote:

OK, the first thing to note is that there is an easy workaround,
which is to use a sort routine that doesn't need $a/$b. Example:

create or replace function mysort() returns text language plperl
as $f$

my $sfunc = sub ($$) { $_[0] <=> $_[1] };

my @vals = (5,3,4,2,7);

return join(' ',sort $sfunc @vals);

$f$;

Andrew for the win!

Thanks a lot!

I agree, a documentation note would be fine for this rather doing all
sorts of complicated perl trickery.

--
Jeff Trout <jeff@jefftrout.com>
http://www.stuarthamm.net/
http://www.dellsmartexitin.com/

"nathan" == nathan wagner <nw@hydaspes.if.org> writes:

nathan> Completely untested speculation based on my knowledge of perl
nathan> and a bit of reading:

nathan> The reason you can't see $a and $b is that sort internally
nathan> sets these variables in the main package. That is, sort is
nathan> setting $main::a and $main::b, and when you run the plperl
nathan> code in the safe compartment, main:: isn't visible any more.

Nice theory, but completely wrong: sort creates $a and $b in the
current package, not in main::.

--
Andrew (irc:RhodiumToad)

Andrew Gierth <andrew@tao11.riddles.org.uk> writes:

Nice theory, but completely wrong: sort creates $a and $b in the
current package, not in main::.

Hmm ... so then why are we seeing a failure?

regards, tom lane

On Wed, Nov 5, 2008 at 10:54, Andrew Gierth <andrew@tao11.riddles.org.uk> wrote:

"nathan" == nathan wagner <nw@hydaspes.if.org> writes:

nathan> Completely untested speculation based on my knowledge of perl
nathan> and a bit of reading:

nathan> The reason you can't see $a and $b is that sort internally
nathan> sets these variables in the main package. That is, sort is
nathan> setting $main::a and $main::b, and when you run the plperl
nathan> code in the safe compartment, main:: isn't visible any more.

Nice theory, but completely wrong: sort creates $a and $b in the
current package, not in main::.

current package is main ;)

On Wed, Nov 5, 2008 at 11:14, Tom Lane <tgl@sss.pgh.pa.us> wrote:

Andrew Gierth <andrew@tao11.riddles.org.uk> writes:

Nice theory, but completely wrong: sort creates $a and $b in the
current package, not in main::.

Hmm ... so then why are we seeing a failure?

Because Safe runs in a different namespace altogether (part of why its
Safe). We build the sub under Safe but then execute it in the real
namespace. The patch I posted fixes this but Id like someone with
more knowledge of safe to look over it. From a quick cvs log it
*looked* like that was Andrew Dunstan which is why I cc'ed him. This
is not a Safe bug IMHO its our (ab)use of it that is causing the
problem. Of course if its only sort that gets affected maybe the cure
is worse than the disease...

"Tom" == Tom Lane <tgl@sss.pgh.pa.us> writes:

Nice theory, but completely wrong: sort creates $a and $b in the
current package, not in main::.

Tom> Hmm ... so then why are we seeing a failure?

FWIW, I _don't_ see the failure. It seems to occur ONLY if perl was
built with threading support (ithreads). Without ithreads, I can't
reproduce it (I've tried enabling and disabling multiplicity with no
effect, so it's not that).

Ithreads seem to be the default on many linux package builds of perl.
It is _not_ the default on FreeBSD.

--
Andrew.