DNS SRV support for LDAP authentication
Hello hackers,
Some people like to use DNS SRV records to advertise LDAP servers on
their network. Microsoft Active Directory is usually (always?) set up
that way. Here is a patch to allow our LDAP auth module to support
that kind of discovery. It copies the convention of the OpenLDAP
command line tools: if you give it a URL that has no hostname, it'll
try to extract a domain name from the bind DN, and then ask your DNS
server for a SRV record for LDAP-over-TCP at that domain. The
OpenLDAP version of libldap.so exports the magic to do that, so the
patch is very small (but the infrastructure set-up to test it is a bit
of a schlep, see below). I'll add this to the next Commitfest.
Testing instructions for (paths and commands given for FreeBSD, adjust
as appropriate):
1. Install BIND:
$ sudo pkg install bind99
2. Define a new zone for testing, by adding the following to the end
of /usr/local/etc/namedb/named.conf:
===== 8< =====
zone "my.test.domain" {
type master;
file "/usr/local/etc/namedb/master/my.test.domain";
};
===== 8< =====
3. Create that zone file in /usr/local/etc/namedb/master/my.test.domain:
===== 8< =====
$TTL 10
@ IN SOA ns.my.test.domain. admin.my.test.domain. (
2 ; Serial
604800 ; Refresh
86400 ; Retry
2419200 ; Expire
604800 ) ; Negative Cache TTL
IN NS ns.my.test.domain.
ns.my.test.domain. IN A 127.0.0.1
my.test.domain. IN A 127.0.0.1
ldap-server.my.test.domain. IN A 127.0.0.1
_ldap._tcp.my.test.domain. IN SRV 0 0 389
ldap-server
===== 8< =====
4. Start up bind:
# service named onestart
5. Confirm that SRV lookups find our record:
$ dig @localhost _ldap._tcp.my-domain.com SRV
...
;; ANSWER SECTION:
_ldap._tcp.my-domain.com. 10 IN SRV 0 0 389
ldap-server.my-domain.com.
6. Tell your system libraries to use this DNS server by temporarily
changing /etc/resolv.conf to say:
===== 8< =====
nameserver 127.0.0.1
===== 8< =====
7. Confirm that the OpenLDAP tools can look that SRV record up:
$ ldapsearch -H 'ldap:///ou%3Dblah%2Cdc%3Dmy-domain%2Cdc%3Dcom'
(That's "ou=blah,dc=my-domain,dc=com" URL-encoded, from which
"my-domain.com" will be extracted.) You should see that it's trying
to connect to ldap-server port 389, and you can stick 'x' on the end
of it to see what it looks like when it can't find a SRV record, as a
sanity check:
$ ldapsearch -H 'ldap:///ou%3Dblah%2Cdc%3Dmy-domain%2Cdc%3Dcomx'
DNS SRV: Could not turn domain=my-domain.comx into a hostlist
8. Set up an LDAP server listening on localhost port 389, and create
a user, such that you can actually authenticate from PostgreSQL with
it. Gory details omitted. First test that you can log in with LDAP
authentication when using a pg_hba.conf line like this:
host all fred 127.0.0.1/32 ldap
ldapurl="ldap://ldap-server.my-domain.com/dc=my-domain,dc=com?cn?sub"
9. Next apply the patch and verify that you can take out the hostname
and let it be discovered via DNS SRV:
host all fred 127.0.0.1/32 ldap ldapurl="ldap:///dc=my-domain,dc=com?cn?sub"
(You can stick some elog(LOG, ...) lines into
InitializeLDAPConnection() if you want to check that
ldap_domain2hostlist() is in fact finding the hostname and port.)
This is a first draft. Not tested much yet. I wonder if
HAVE_LDAP_INITIALIZE is a reasonable way to detact OpenLDAP. The
documentation was written in about 7 seconds so probably needs work.
There is probably a Windowsy way to do this too but I didn't look into
that.
--
Thomas Munro
http://www.enterprisedb.com
Attachments:
0001-Add-DNS-SRV-support-for-LDAP-server-discovery.patchapplication/octet-stream; name=0001-Add-DNS-SRV-support-for-LDAP-server-discovery.patchDownload
From 4166e0133649c3c66c37af36bc79efb836a238d1 Mon Sep 17 00:00:00 2001
From: Thomas Munro <thomas.munro@enterprisedb.com>
Date: Tue, 25 Sep 2018 13:47:34 +1200
Subject: [PATCH] Add DNS SRV support for LDAP server discovery.
LDAP servers can be advertised on a network by registering DNS SRV
records for _ldap._tcp.<domain>. The OpenLDAP command-line tools
know how to find servers via those records, if no server name is
provided by the user. Teach PostgreSQL to follow the same convention,
using non-standard extensions provided by OpenLDAP, where available.
Author: Thomas Munro
Reviewed-by:
Discussion:
---
doc/src/sgml/client-auth.sgml | 18 +++++++++++
src/backend/libpq/auth.c | 58 +++++++++++++++++++++++++++++++++--
src/backend/libpq/hba.c | 2 ++
3 files changed, 76 insertions(+), 2 deletions(-)
diff --git a/doc/src/sgml/client-auth.sgml b/doc/src/sgml/client-auth.sgml
index c2114021c3..2116e636be 100644
--- a/doc/src/sgml/client-auth.sgml
+++ b/doc/src/sgml/client-auth.sgml
@@ -1671,6 +1671,15 @@ ldap[s]://<replaceable>host</replaceable>[:<replaceable>port</replaceable>]/<rep
<literal>ldapsearchattribute=uid</literal>.
</para>
+ <para>
+ If <productname>PostgreSQL</productname> was compiled with OpenLDAP as
+ the LDAP client library, the <literal>ldapserver</literal> setting may be
+ omitted. In that case, the hostname and port are looked up via DNS
+ service records. The "SRV" records for the service
+ <literal>_ldap._tcp.domain</literal> are requested, where
+ <literal>domain</literal> is extracted from <literal>basedn</literal>.
+ </para>
+
<para>
Here is an example for a simple-bind LDAP configuration:
<programlisting>
@@ -1716,6 +1725,15 @@ host ... ldap ldapserver=ldap.example.net ldapbasedn="dc=example, dc=net" ldapse
</programlisting>
</para>
+ <para>
+ Here is an example for a search+bind configuration that uses DNS SRV
+ discovery to find the hostname and port for the LDAP service using the
+ domain name <literal>example.net</literal>":
+<programlisting>
+host ... ldap ldapurl="ldap:///ou=people,dc=example,dc=net?cn"
+</programlisting>
+ </para>
+
<tip>
<para>
Since LDAP often uses commas and spaces to separate the different
diff --git a/src/backend/libpq/auth.c b/src/backend/libpq/auth.c
index 8517565535..203f08adac 100644
--- a/src/backend/libpq/auth.c
+++ b/src/backend/libpq/auth.c
@@ -2354,8 +2354,51 @@ InitializeLDAPConnection(Port *port, LDAP **ldap)
{
char *uri;
- uri = psprintf("%s://%s:%d", scheme, port->hba->ldapserver,
- port->hba->ldapport);
+ /*
+ * If the user provided no hostname, we can ask OpenLDAP to try to
+ * find one by extracting a domain name from the base DN and then
+ * using a DSN SRV record for _ldap._tcp.<domain>. If one or more
+ * such SRV records have been defined, we can get a hostname and
+ * port.
+ */
+ if (!port->hba->ldapserver || port->hba->ldapserver[0] == '\0')
+ {
+ char *domain;
+ char *hostlist;
+ char *end;
+
+ /* ou=blah,dc=foo,dc=bar -> foo.bar */
+ if (ldap_dn2domain(port->hba->ldapbasedn, &domain))
+ {
+ ereport(LOG,
+ (errmsg("could not extract domain name from basedn")));
+ return STATUS_ERROR;
+ }
+ /* Look up host:port using DNS SRV for _ldap._tcp.foo.bar */
+ if (ldap_domain2hostlist(domain, &hostlist))
+ {
+ ereport(LOG,
+ (errmsg("could not look up a hostlist for %s",
+ domain)));
+ ldap_memfree(domain);
+ return STATUS_ERROR;
+ }
+ ldap_memfree(domain);
+ /*
+ * OpenLDAP already ordered by weight and shuffled equal weight
+ * servers, so we'll just take the first one. The string is
+ * of the format "host:port", separated by spaces.
+ */
+ if ((end = strchr(hostlist, ' ')))
+ *end = '\0';
+ uri = psprintf("%s://%s", scheme, hostlist);
+ ldap_memfree(hostlist);
+ }
+ else
+ {
+ uri = psprintf("%s://%s:%d", scheme, port->hba->ldapserver,
+ port->hba->ldapport);
+ }
r = ldap_initialize(ldap, uri);
pfree(uri);
if (r != LDAP_SUCCESS)
@@ -2504,12 +2547,23 @@ CheckLDAPAuth(Port *port)
int r;
char *fulluser;
+#ifdef HAVE_LDAP_INITIALIZE
+ /* OpenLDAP allows empty hostname, if we have a basedn. */
+ if ((!port->hba->ldapserver || port->hba->ldapserver[0] == '\0') &&
+ (!port->hba->ldapbasedn || port->hba->ldapbasedn[0] == '\0'))
+ {
+ ereport(LOG,
+ (errmsg("LDAP server not specified, and no ldapbasedn")));
+ return STATUS_ERROR;
+ }
+#else
if (!port->hba->ldapserver || port->hba->ldapserver[0] == '\0')
{
ereport(LOG,
(errmsg("LDAP server not specified")));
return STATUS_ERROR;
}
+#endif
if (port->hba->ldapport == 0)
{
diff --git a/src/backend/libpq/hba.c b/src/backend/libpq/hba.c
index 1a65ec87bd..8cd09a49b5 100644
--- a/src/backend/libpq/hba.c
+++ b/src/backend/libpq/hba.c
@@ -1500,7 +1500,9 @@ parse_hba_line(TokenizedLine *tok_line, int elevel)
*/
if (parsedline->auth_method == uaLDAP)
{
+#ifndef HAVE_LDAP_INITIALIZE
MANDATORY_AUTH_ARG(parsedline->ldapserver, "ldapserver", "ldap");
+#endif
/*
* LDAP can operate in two modes: either with a direct bind, using
--
2.17.0
On Tue, Sep 25, 2018 at 2:09 PM Thomas Munro
<thomas.munro@enterprisedb.com> wrote:
2. Define a new zone for testing, by adding the following to the end
3. Create that zone file in /usr/local/etc/namedb/master/my.test.domain:
Oops, I changed my testing domain name in the middle of my experiment,
but pasted the older version into the previous message. Here are the
corrected steps 2 and 3, consistent with the rest:
===== end of /usr/local/etc/namedb/named.conf =====
zone "my-domain.com" {
type master;
file "/usr/local/etc/namedb/master/my-domain.com";
};
=====
===== /usr/local/etc/namedb/master/my-domain.com =====
$TTL 10
@ IN SOA ns.my-domain.com. admin.my-domain.com. (
2 ; Serial
604800 ; Refresh
86400 ; Retry
2419200 ; Expire
604800 ) ; Negative Cache TTL
IN NS ns.my-domain.com.
ns.my-domain.com. IN A 127.0.0.1
my-domain.com. IN A 127.0.0.1
ldap-server.my-domain.com. IN A 127.0.0.1
_ldap._tcp.my-domain.com. IN SRV 0 0 389
ldap-server
=====
--
Thomas Munro
http://www.enterprisedb.com
On Tue, Sep 25, 2018 at 2:09 PM Thomas Munro
<thomas.munro@enterprisedb.com> wrote:
Some people like to use DNS SRV records to advertise LDAP servers on
their network. Microsoft Active Directory is usually (always?) set up
that way. Here is a patch to allow our LDAP auth module to support
that kind of discovery. It copies the convention of the OpenLDAP
command line tools: if you give it a URL that has no hostname, it'll
try to extract a domain name from the bind DN, and then ask your DNS
server for a SRV record for LDAP-over-TCP at that domain. The
OpenLDAP version of libldap.so exports the magic to do that, so the
patch is very small (but the infrastructure set-up to test it is a bit
of a schlep, see below). I'll add this to the next Commitfest.[long tedious explanation of how to set up a test with BIND and OpenLDAP on Unix]
Of course the point of this is not really for the Unix-based set-up I
described, but for Microsoft environments with one or more AD servers
and a PostgreSQL server running on (eg) Linux that wants to find AD.
In such environments, from what I can tell, the following should work:
Standard DNS lookup tools should be able to find SRV records
advertising the host, port and weight (priority) of any AD servers on
the network:
$ nslookup -type=any _ldap._tcp.YOUR.DOMAIN
$ dig srv _ldap._tcp.YOUR.DOMAIN
$ host -t srv _ldp._tcp.YOUR.DOMAIN
OpenLDAP command line tools should be able to find the AD server via
those SRV records, extracting YOUR.DOMAIN from the base DN:
$ ldapsearch -H 'ldap:///dc%3DYOUR%2Cdc%3DDOMAIN' ...
pg_hba.conf with an explicit LDAP server name should be able to talk
to Active Directory without using this patch with something like:
host all all 127.0.0.1/32 ldap
ldapurl="ldap://YOUR-AD-SERVER.YOUR.DOMAIN/dc=YOUR,dc=DOMAIN?cn?sub"
pg_hba.conf using this patch should be able to discover the LDAP
server via SRV if you take out the server name:
host all all 127.0.0.1/32 ldap ldapurl="ldap:///dc=YOUR,dc=DOMAIN?cn?sub"
I'm hoping someone can help test this in a real Active Directory environment.
--
Thomas Munro
http://www.enterprisedb.com
On Wed, Nov 7, 2018 at 4:39 PM Thomas Munro
<thomas.munro@enterprisedb.com> wrote:
On Tue, Sep 25, 2018 at 2:09 PM Thomas Munro
<thomas.munro@enterprisedb.com> wrote:Some people like to use DNS SRV records to advertise LDAP servers on
their network. Microsoft Active Directory is usually (always?) set up
that way. Here is a patch to allow our LDAP auth module to support
that kind of discovery.
Rebased.
I took the liberty of CCing Mark Cave-Ayland, who had some great
advice on the last round of LDAP feature tweaks[1]/messages/by-id/CAEepm=0XTkYvMci0WRubZcf_1am8=gP=7oJErpsUfRYcKF2gwg@mail.gmail.com. Mark, if you have
any comments on the sanity of this proposal, they'd be much
appreciated, otherwise of course please feel free to ignore. Thanks!
[1]: /messages/by-id/CAEepm=0XTkYvMci0WRubZcf_1am8=gP=7oJErpsUfRYcKF2gwg@mail.gmail.com
--
Thomas Munro
http://www.enterprisedb.com
Attachments:
0001-Add-DNS-SRV-support-for-LDAP-server-discovery-v2.patchapplication/x-patch; name=0001-Add-DNS-SRV-support-for-LDAP-server-discovery-v2.patchDownload
From 4be9142998e76614146f88c825179264573bcf7c Mon Sep 17 00:00:00 2001
From: Thomas Munro <thomas.munro@enterprisedb.com>
Date: Fri, 16 Nov 2018 14:32:00 +1300
Subject: [PATCH] Add DNS SRV support for LDAP server discovery.
LDAP servers can be advertised on a network by registering DNS SRV
records for _ldap._tcp.<domain>. The OpenLDAP command-line tools
know how to find servers via those records, if no server name is
provided by the user. Teach PostgreSQL to follow the same convention
using non-standard extensions provided by OpenLDAP, where available.
Author: Thomas Munro
Reviewed-by:
Discussion: https://postgr.es/m/CAEepm=2hAnSfhdsd6vXsM6VZVN0br-FbAZ-O+Swk18S5HkCP=A@mail.gmail.com
---
doc/src/sgml/client-auth.sgml | 19 ++++++
src/backend/libpq/auth.c | 113 +++++++++++++++++++++++++---------
src/backend/libpq/hba.c | 2 +
3 files changed, 105 insertions(+), 29 deletions(-)
diff --git a/doc/src/sgml/client-auth.sgml b/doc/src/sgml/client-auth.sgml
index c2114021c3..f9e7416c79 100644
--- a/doc/src/sgml/client-auth.sgml
+++ b/doc/src/sgml/client-auth.sgml
@@ -1671,6 +1671,16 @@ ldap[s]://<replaceable>host</replaceable>[:<replaceable>port</replaceable>]/<rep
<literal>ldapsearchattribute=uid</literal>.
</para>
+ <para>
+ If <productname>PostgreSQL</productname> was compiled with OpenLDAP as
+ the LDAP client library, the <literal>ldapserver</literal> setting may be
+ omitted. In that case, the hostname and port are looked up via DNS
+ service records. The "SRV" records for the service
+ <literal>_ldap._tcp.domain</literal> are requested, where
+ <literal>domain</literal> is extracted from <literal>basedn</literal>.
+ This follows a convention used by OpenLDAP command-line tools.
+ </para>
+
<para>
Here is an example for a simple-bind LDAP configuration:
<programlisting>
@@ -1716,6 +1726,15 @@ host ... ldap ldapserver=ldap.example.net ldapbasedn="dc=example, dc=net" ldapse
</programlisting>
</para>
+ <para>
+ Here is an example for a search+bind configuration that uses DNS SRV
+ discovery to find the hostname and port for the LDAP service using the
+ domain name <literal>example.net</literal>":
+<programlisting>
+host ... ldap ldapurl="ldap:///ou=people,dc=example,dc=net?cn"
+</programlisting>
+ </para>
+
<tip>
<para>
Since LDAP often uses commas and spaces to separate the different
diff --git a/src/backend/libpq/auth.c b/src/backend/libpq/auth.c
index 4f9d697d6d..cb62540c9f 100644
--- a/src/backend/libpq/auth.c
+++ b/src/backend/libpq/auth.c
@@ -2356,37 +2356,81 @@ InitializeLDAPConnection(Port *port, LDAP **ldap)
char *uris = NULL;
/*
- * We have a space-separated list of hostnames. Convert it
- * to a space-separated list of URIs.
+ * If the user provided no hostname, we can ask OpenLDAP to try to
+ * find one by extracting a domain name from the base DN and then
+ * using a DSN SRV record for _ldap._tcp.<domain>. If one or more
+ * such SRV records have been defined, we can get a hostname and
+ * port. The same convention is used by the OpenLDAP command line
+ * tools.
*/
- do
+ if (!hostnames || hostnames[0] == '\0')
{
- char *hostname;
- size_t hostname_size;
- char *new_uris;
-
- /* Find the leading hostname. */
- hostname_size = strcspn(hostnames, " ");
- hostname = pnstrdup(hostnames, hostname_size);
-
- /* Append a URI for this hostname. */
- new_uris = psprintf("%s%s%s://%s:%d",
- uris ? uris : "",
- uris ? " " : "",
- scheme,
- hostname,
- port->hba->ldapport);
-
- pfree(hostname);
- if (uris)
- pfree(uris);
- uris = new_uris;
-
- /* Step over this hostname and any spaces. */
- hostnames += hostname_size;
- while (*hostnames == ' ')
- ++hostnames;
- } while (*hostnames);
+ char *domain;
+ char *hostlist;
+ char *end;
+
+ /* ou=blah,dc=foo,dc=bar -> foo.bar */
+ if (ldap_dn2domain(port->hba->ldapbasedn, &domain))
+ {
+ ereport(LOG,
+ (errmsg("could not extract domain name from basedn")));
+ return STATUS_ERROR;
+ }
+ /* Look up host:port using DNS SRV for _ldap._tcp.foo.bar */
+ if (ldap_domain2hostlist(domain, &hostlist))
+ {
+ ereport(LOG,
+ (errmsg("could not look up a hostlist for %s",
+ domain)));
+ ldap_memfree(domain);
+ return STATUS_ERROR;
+ }
+ ldap_memfree(domain);
+ /*
+ * OpenLDAP already ordered by weight and shuffled equal weight
+ * servers, so we'll just take the first one. The string is
+ * of the format "host:port", separated by spaces.
+ */
+ if ((end = strchr(hostlist, ' ')))
+ *end = '\0';
+ uris = psprintf("%s://%s", scheme, hostlist);
+ ldap_memfree(hostlist);
+ }
+ else
+ {
+ /*
+ * We have a space-separated list of hostnames. Convert it
+ * to a space-separated list of URIs.
+ */
+ do
+ {
+ char *hostname;
+ size_t hostname_size;
+ char *new_uris;
+
+ /* Find the leading hostname. */
+ hostname_size = strcspn(hostnames, " ");
+ hostname = pnstrdup(hostnames, hostname_size);
+
+ /* Append a URI for this hostname. */
+ new_uris = psprintf("%s%s%s://%s:%d",
+ uris ? uris : "",
+ uris ? " " : "",
+ scheme,
+ hostname,
+ port->hba->ldapport);
+
+ pfree(hostname);
+ if (uris)
+ pfree(uris);
+ uris = new_uris;
+
+ /* Step over this hostname and any spaces. */
+ hostnames += hostname_size;
+ while (*hostnames == ' ')
+ ++hostnames;
+ } while (*hostnames);
+ }
r = ldap_initialize(ldap, uris);
pfree(uris);
@@ -2536,12 +2580,23 @@ CheckLDAPAuth(Port *port)
int r;
char *fulluser;
+#ifdef HAVE_LDAP_INITIALIZE
+ /* OpenLDAP allows empty hostname, if we have a basedn. */
+ if ((!port->hba->ldapserver || port->hba->ldapserver[0] == '\0') &&
+ (!port->hba->ldapbasedn || port->hba->ldapbasedn[0] == '\0'))
+ {
+ ereport(LOG,
+ (errmsg("LDAP server not specified, and no ldapbasedn")));
+ return STATUS_ERROR;
+ }
+#else
if (!port->hba->ldapserver || port->hba->ldapserver[0] == '\0')
{
ereport(LOG,
(errmsg("LDAP server not specified")));
return STATUS_ERROR;
}
+#endif
if (port->hba->ldapport == 0)
{
diff --git a/src/backend/libpq/hba.c b/src/backend/libpq/hba.c
index 0129dd24d0..ab4cc99001 100644
--- a/src/backend/libpq/hba.c
+++ b/src/backend/libpq/hba.c
@@ -1500,7 +1500,9 @@ parse_hba_line(TokenizedLine *tok_line, int elevel)
*/
if (parsedline->auth_method == uaLDAP)
{
+#ifndef HAVE_LDAP_INITIALIZE
MANDATORY_AUTH_ARG(parsedline->ldapserver, "ldapserver", "ldap");
+#endif
/*
* LDAP can operate in two modes: either with a direct bind, using
--
2.19.1
On 25 Sep 2018, at 04:09, Thomas Munro <thomas.munro@enterprisedb.com> wrote:
Some people like to use DNS SRV records to advertise LDAP servers on
their network. Microsoft Active Directory is usually (always?) set up
that way. Here is a patch to allow our LDAP auth module to support
that kind of discovery. It copies the convention of the OpenLDAP
command line tools: if you give it a URL that has no hostname, it'll
try to extract a domain name from the bind DN, and then ask your DNS
server for a SRV record for LDAP-over-TCP at that domain. The
OpenLDAP version of libldap.so exports the magic to do that, so the
patch is very small (but the infrastructure set-up to test it is a bit
of a schlep, see below). I'll add this to the next Commitfest.
Sounds like a reasonable feature.
Testing instructions for (paths and commands given for FreeBSD, adjust
as appropriate):
Trying this quickly on macOS while at a conference didn’t yield much success,
will do another attempt when I’m on a more reliable connection.
This is a first draft. Not tested much yet. I wonder if
HAVE_LDAP_INITIALIZE is a reasonable way to detact OpenLDAP. The
documentation was written in about 7 seconds so probably needs work.
There is probably a Windowsy way to do this too but I didn't look into
that.
Reading through the patch, and related OpenLDAP code, this seems like a good
approach. A few small comments:
+ If <productname>PostgreSQL</productname> was compiled with OpenLDAP as
Should OpenLDAP be wrapped in <productname> tags as well? If so, there is
another “bare” instance in client-auth.sgml which perhaps can be wrapped into
this patch while at it.
+ ereport(LOG,
+ (errmsg("could not look up a hostlist for %s",
+ domain)));Should this be \”%s\”?
+ new_uris = psprintf("%s%s%s://%s:%d",
While this construction isn't introduced in this patch, would it not make sense
to convert uris to StringInfo instead to improve readability?
+ /* Step over this hostname and any spaces. */
Nitpicking on a moved hunk, but single-line comments shouldn’t end in a period
I believe.
cheers ./daniel
On 25 Sep 2018, at 04:09, Thomas Munro <thomas.munro@enterprisedb.com> wrote:
Some people like to use DNS SRV records to advertise LDAP servers on
their network. Microsoft Active Directory is usually (always?) set up
that way. Here is a patch to allow our LDAP auth module to support
that kind of discovery.
Does this support SSL/TLS?
Regards,
Graham
—
On Sat, Feb 2, 2019 at 9:25 AM Graham Leggett <minfrin@sharp.fm> wrote:
On 25 Sep 2018, at 04:09, Thomas Munro <thomas.munro@enterprisedb.com> wrote:
Some people like to use DNS SRV records to advertise LDAP servers on
their network. Microsoft Active Directory is usually (always?) set up
that way. Here is a patch to allow our LDAP auth module to support
that kind of discovery.Does this support SSL/TLS?
I didn't try it myself but I found several claims that it works. I
see complaints that it always looks for _ldap._tcp and not _ldaps._tcp
as you might expect when using ldascheme=ldaps, but that doesn't seem
to be a big problem. As for ldaptls=1, that must work because it
doesn't even negotiate that until after the connection is made.
--
Thomas Munro
http://www.enterprisedb.com
On 02 Feb 2019, at 01:57, Thomas Munro <thomas.munro@enterprisedb.com> wrote:
On Sat, Feb 2, 2019 at 9:25 AM Graham Leggett <minfrin@sharp.fm> wrote:
On 25 Sep 2018, at 04:09, Thomas Munro <thomas.munro@enterprisedb.com> wrote:
Some people like to use DNS SRV records to advertise LDAP servers on
their network. Microsoft Active Directory is usually (always?) set up
that way. Here is a patch to allow our LDAP auth module to support
that kind of discovery.Does this support SSL/TLS?
I didn't try it myself but I found several claims that it works. I
see complaints that it always looks for _ldap._tcp and not _ldaps._tcp
as you might expect when using ldascheme=ldaps, but that doesn't seem
to be a big problem. As for ldaptls=1, that must work because it
doesn't even negotiate that until after the connection is made.
If the LDAP server was bound to port 636, how would the client know to use a direct SSL/TLS connection and not STARTTLS?
Regards,
Graham
—
On Sat, Feb 2, 2019 at 10:34 PM Graham Leggett <minfrin@sharp.fm> wrote:
On 02 Feb 2019, at 01:57, Thomas Munro <thomas.munro@enterprisedb.com> wrote:
On Sat, Feb 2, 2019 at 9:25 AM Graham Leggett <minfrin@sharp.fm> wrote:
Does this support SSL/TLS?
I didn't try it myself but I found several claims that it works. I
see complaints that it always looks for _ldap._tcp and not _ldaps._tcp
as you might expect when using ldascheme=ldaps, but that doesn't seem
to be a big problem. As for ldaptls=1, that must work because it
doesn't even negotiate that until after the connection is made.If the LDAP server was bound to port 636, how would the client know to use a direct SSL/TLS connection and not STARTTLS?
SRV records don't control that, so it looks like the person
configuring pg_hba.conf would simply have to know which of the
following formats to use:
ldapurl=ldap:///dc=example,dc=net
ldapurl=ldap:///dc=example,dc=net ldaptls=1
ldapurl=ldaps:///dc=example,dc=net
Only the port and host are obtained from the SRV record, not those
protocol details. Nothing in RFC 2782 prevents you from setting up
separate "_ldaps._tcp" SRV records (and I can find discussions of that
idea on the net) and then writing custom resolver code that knows to
look for that, but the OpenLDAP code we're using (for compatibility
with the command line tools) is hard coded to use "_ldap._tcp"
always[1]https://github.com/openldap/openldap/blob/b06f5b0493937fc28f2cc86df1d7f464aa4504d8/libraries/libldap/dnssrv.c#L276. Active Directory apparently automatically creates only
"_ldap._tcp" SRV records according to its documentation and that's the
user base I was aiming for with this patch, so I think it makes sense
to just use the routines they provide, despite this weakness.
--
Thomas Munro
http://www.enterprisedb.com
On Sat, Feb 2, 2019 at 12:48 AM Daniel Gustafsson <daniel@yesql.se> wrote:
+ new_uris = psprintf("%s%s%s://%s:%d",
While this construction isn't introduced in this patch, would it not make sense
to convert uris to StringInfo instead to improve readability?
Yeah. This coding is ugly and StringInfo would be much nicer.
Thinking about that made me realise that the proposed SRV case should
also handle multiple SRV records by building a multi-URL string too
(instead of just taking the first one). I will make it so.
--
Thomas Munro
http://www.enterprisedb.com
On Sat, Feb 16, 2019 at 10:57 PM Thomas Munro
<thomas.munro@enterprisedb.com> wrote:
Yeah. This coding is ugly and StringInfo would be much nicer.
Thinking about that made me realise that the proposed SRV case should
also handle multiple SRV records by building a multi-URL string too
(instead of just taking the first one). I will make it so.
Done, in the attached. Reviewing your comments again, from the top:
On Sat, Feb 2, 2019 at 12:48 AM Daniel Gustafsson <daniel@yesql.se> wrote:
+ If <productname>PostgreSQL</productname> was compiled with OpenLDAP as
Should OpenLDAP be wrapped in <productname> tags as well? If so, there is
another “bare” instance in client-auth.sgml which perhaps can be wrapped into
this patch while at it.
Fixed.
+ ereport(LOG, + (errmsg("could not look up a hostlist for %s", + domain)));Should this be \”%s\”?
Yep, fixed.
+ new_uris = psprintf("%s%s%s://%s:%d",
While this construction isn't introduced in this patch, would it not make sense
to convert uris to StringInfo instead to improve readability?
Agreed, fixed.
+ /* Step over this hostname and any spaces. */
Nitpicking on a moved hunk, but single-line comments shouldn’t end in a period
I believe.
Huh. And yet they are sentences.
tmunro@dogmatix $ git grep '/\* [A-Za-z].*\. \*/' | wc -l
5607
tmunro@dogmatix $ git grep '/\* [A-Za-z].*[a-z] \*/' | wc -l
59500
Yep, you win!
I also fixed a bug where some error messages could pass a NULL pointer
for %s when we don't have a server name.
I also added a hint to the error message you get if it can't find DNS
SRV records, so that if you accidentally activate this feature by
forgetting to set the server name, it'll remind you that you could do
that:
LOG: LDAP authentication could not find DNS SRV records for "example.net"
HINT: Set an LDAP server name explicitly.
Unfortunately, no feedback from MS Active Directory users has been
forthcoming, but I guess that might take a beta release. See below
for new more complete instructions for testing this with an open
source stack (now that I know there is a lazy way to stand up an LDAP
server using the TAP test stuff, I've adjusted the instructions to
work with that).
I'd like to commit this soon.
Some random things I noticed that I am not fixing in this patch but
wanted to mention: I don't like the asymmetry initStringInfo(si),
pfree(si->data). I don't like si->data as a way to get a C string
from a StringInfo. There are a couple of references to StringBuffer
that surely mean StringInfo in comments.
=== How to test ===
1. Start up an LDAP server that has a user test1/secret1 under
dc=example,dc=net (it runs in the background and you can stop it with
SIGINT):
$ make -C src/test/ldap check
$ /usr/local/libexec/slapd -f src/test/ldap/tmp_check/slapd.conf -h
ldap://127.0.0.1:5555
2. Start up a BIND daemon that has multiple SRV records for LDAP at
example.com:
$ tail -4 /usr/local/etc/namedb/named.conf
zone "example.net" {
type master;
file "/usr/local/etc/namedb/master/example.net";
};
$ cat /usr/local/etc/namedb/master/example.net
$TTL 10
@ IN SOA ns.example.net. admin.example.net. (
2 ; Serial
604800 ; Refresh
86400 ; Retry
2419200 ; Expire
604800 ) ; Negative Cache TTL
IN NS ns.example.net.
ns.example.net. IN A 127.0.0.1
example.net. IN A 127.0.0.1
ldap1.example.net. IN A 127.0.0.1
ldap2.example.net. IN A 127.0.0.1
_ldap._tcp.example.net. IN SRV 0 0 5555 ldap1
_ldap._tcp.example.net. IN SRV 1 0 5555 ldap2
3. Tell your OS to talk to that DNS server (and, erm, keep what you
had here so you can restore it later):
$ cat /etc/resolv.conf
nameserver 127.0.0.1
4. Check that standard DNS and LDAP tools can find their way to your
LDAP servers via these breadcrumbs:
$ host -t srv _ldap._tcp.example.net
_ldap._tcp.example.net has SRV record 0 0 5555 ldap1.example.net.
_ldap._tcp.example.net has SRV record 1 0 5555 ldap2.example.net.
$ ldapsearch -H 'ldap:///dc%3Dexample%2Cdc%3Dnet' -b 'dc=example,dc=net'
5. Tell PostgreSQL to use SRV records in pg_hba.conf using either of
these styles:
host all test1 127.0.0.1/32 ldap basedn="dc=example,dc=net"
host all test1 127.0.0.1/32 ldap ldapurl="ldap:///dc=example,dc=net?uid?sub"
6. Check that you now log in as test1/secret1:
$ psql -h 127.0.0.1 postgres test1
--
Thomas Munro
https://enterprisedb.com
Attachments:
0001-Add-DNS-SRV-support-for-LDAP-server-discovery-v3.patchapplication/octet-stream; name=0001-Add-DNS-SRV-support-for-LDAP-server-discovery-v3.patchDownload
From 5d0a9f43f66bc2e9830df802048cf527f6ee4f16 Mon Sep 17 00:00:00 2001
From: Thomas Munro <thomas.munro@gmail.com>
Date: Tue, 19 Mar 2019 18:53:29 +1300
Subject: [PATCH] Add DNS SRV support for LDAP server discovery.
LDAP servers can be advertised on a network by registering DNS SRV
records for _ldap._tcp.<domain>. The OpenLDAP command-line tools
know how to find servers via those records, if no server name is
provided by the user. Teach PostgreSQL to follow the same convention
using non-standard extensions provided by OpenLDAP, where available.
Author: Thomas Munro
Reviewed-by: Daniel Gustafsson
Discussion: https://postgr.es/m/CAEepm=2hAnSfhdsd6vXsM6VZVN0br-FbAZ-O+Swk18S5HkCP=A@mail.gmail.com
---
doc/src/sgml/client-auth.sgml | 21 ++++-
src/backend/libpq/auth.c | 145 ++++++++++++++++++++++++----------
src/backend/libpq/hba.c | 2 +
3 files changed, 125 insertions(+), 43 deletions(-)
diff --git a/doc/src/sgml/client-auth.sgml b/doc/src/sgml/client-auth.sgml
index 411f1e1679..b6d44f2d66 100644
--- a/doc/src/sgml/client-auth.sgml
+++ b/doc/src/sgml/client-auth.sgml
@@ -1655,7 +1655,8 @@ ldap[s]://<replaceable>host</replaceable>[:<replaceable>port</replaceable>]/<rep
</para>
<para>
- LDAP URLs are currently only supported with OpenLDAP, not on Windows.
+ LDAP URLs are currently only supported with
+ <productname>OpenLDAP</productname>, not on Windows.
</para>
</listitem>
</varlistentry>
@@ -1678,6 +1679,15 @@ ldap[s]://<replaceable>host</replaceable>[:<replaceable>port</replaceable>]/<rep
<literal>ldapsearchattribute=uid</literal>.
</para>
+ <para>
+ If <productname>PostgreSQL</productname> was compiled with
+ <productname>OpenLDAP</productname> as the LDAP client library, the
+ <literal>ldapserver</literal> setting may be omitted. In that case, a
+ list of hostnames and ports is looked up via RFC 2782 DNS service records.
+ The name <literal>_ldap._tcp.DOMAIN</literal> is looked up, where
+ <literal>DOMAIN</literal> is extracted from <literal>basedn</literal>.
+ </para>
+
<para>
Here is an example for a simple-bind LDAP configuration:
<programlisting>
@@ -1723,6 +1733,15 @@ host ... ldap ldapserver=ldap.example.net ldapbasedn="dc=example, dc=net" ldapse
</programlisting>
</para>
+ <para>
+ Here is an example for a search+bind configuration that uses DNS SRV
+ discovery to find the hostname(s) and port(s) for the LDAP service for the
+ domain name <literal>example.net</literal>:
+<programlisting>
+host ... ldap ldapbasedn="dc=example,dc=net"
+</programlisting>
+ </para>
+
<tip>
<para>
Since LDAP often uses commas and spaces to separate the different
diff --git a/src/backend/libpq/auth.c b/src/backend/libpq/auth.c
index fb86e9e9d4..d3bbbdac6d 100644
--- a/src/backend/libpq/auth.c
+++ b/src/backend/libpq/auth.c
@@ -2369,44 +2369,87 @@ InitializeLDAPConnection(Port *port, LDAP **ldap)
#else
#ifdef HAVE_LDAP_INITIALIZE
{
- const char *hostnames = port->hba->ldapserver;
- char *uris = NULL;
+ StringInfoData uris;
+ char *hostlist = NULL;
+ char *p;
+ bool append_port;
+
+ /* We'll build the list of scheme://hostname:port in a StringInfo */
+ initStringInfo(&uris);
/*
- * We have a space-separated list of hostnames. Convert it
- * to a space-separated list of URIs.
+ * If pg_hba.conf provided no hostnames, we can ask OpenLDAP to try to
+ * find some by extracting a domain name from the base DN and looking
+ * up DSN SRV records for _ldap._tcp.<domain>. The same convention
+ * is used by the OpenLDAP command line tools.
*/
+ if (!port->hba->ldapserver || port->hba->ldapserver[0] == '\0')
+ {
+ char *domain;
+
+ /* ou=blah,dc=foo,dc=bar -> foo.bar */
+ if (ldap_dn2domain(port->hba->ldapbasedn, &domain))
+ {
+ ereport(LOG,
+ (errmsg("could not extract domain name from basedn")));
+ return STATUS_ERROR;
+ }
+
+ /* Look up a list of LDAP server hosts and port numbers */
+ if (ldap_domain2hostlist(domain, &hostlist))
+ {
+ ereport(LOG,
+ (errmsg("LDAP authentication could not find DNS SRV records for \"%s\"",
+ domain),
+ (errhint("Set an LDAP server name explicitly."))));
+ ldap_memfree(domain);
+ return STATUS_ERROR;
+ }
+ ldap_memfree(domain);
+
+ /* We have a space-separated list of host:port entries */
+ p = hostlist;
+ append_port = false;
+ }
+ else
+ {
+ /* We have a space-separated list of hosts from pg_hba.conf */
+ p = port->hba->ldapserver;
+ append_port = true;
+ }
+
+ /* Build a space-separated list of full URIs */
do
{
- char *hostname;
- size_t hostname_size;
- char *new_uris;
-
- /* Find the leading hostname. */
- hostname_size = strcspn(hostnames, " ");
- hostname = pnstrdup(hostnames, hostname_size);
-
- /* Append a URI for this hostname. */
- new_uris = psprintf("%s%s%s://%s:%d",
- uris ? uris : "",
- uris ? " " : "",
- scheme,
- hostname,
- port->hba->ldapport);
-
- pfree(hostname);
- if (uris)
- pfree(uris);
- uris = new_uris;
-
- /* Step over this hostname and any spaces. */
- hostnames += hostname_size;
- while (*hostnames == ' ')
- ++hostnames;
- } while (*hostnames);
-
- r = ldap_initialize(ldap, uris);
- pfree(uris);
+ size_t size;
+
+ /* Find the span of the next entry */
+ size = strcspn(p, " ");
+
+ /* Append a space separator if this isn't the first URI */
+ if (uris.len > 0)
+ appendStringInfoChar(&uris, ' ');
+
+ /* Append scheme://host:port */
+ appendStringInfoString(&uris, scheme);
+ appendStringInfoString(&uris, "://");
+ appendBinaryStringInfo(&uris, p, size);
+ if (append_port)
+ appendStringInfo(&uris, ":%d", port->hba->ldapport);
+
+ /* Step over this entry and any number of trailing spaces */
+ p += size;
+ while (*p == ' ')
+ ++p;
+ } while (*p);
+
+ /* Free memory from OpenLDAP if we looked up SRV records */
+ if (hostlist)
+ ldap_memfree(hostlist);
+
+ /* Finally, try to connect using the URI list */
+ r = ldap_initialize(ldap, uris.data);
+ pfree(uris.data);
if (r != LDAP_SUCCESS)
{
ereport(LOG,
@@ -2552,13 +2595,31 @@ CheckLDAPAuth(Port *port)
LDAP *ldap;
int r;
char *fulluser;
+ const char *server_name;
+#ifdef HAVE_LDAP_INITIALIZE
+ /* OpenLDAP allows empty hostname, if we have a basedn. */
+ if ((!port->hba->ldapserver || port->hba->ldapserver[0] == '\0') &&
+ (!port->hba->ldapbasedn || port->hba->ldapbasedn[0] == '\0'))
+ {
+ ereport(LOG,
+ (errmsg("LDAP server not specified, and no ldapbasedn")));
+ return STATUS_ERROR;
+ }
+#else
if (!port->hba->ldapserver || port->hba->ldapserver[0] == '\0')
{
ereport(LOG,
(errmsg("LDAP server not specified")));
return STATUS_ERROR;
}
+#endif
+
+ /*
+ * If we're using SRV records, we don't have a server name so we'll
+ * just show an empty string in error messages.
+ */
+ server_name = port->hba->ldapserver ? port->hba->ldapserver : "";
if (port->hba->ldapport == 0)
{
@@ -2630,7 +2691,7 @@ CheckLDAPAuth(Port *port)
ereport(LOG,
(errmsg("could not perform initial LDAP bind for ldapbinddn \"%s\" on server \"%s\": %s",
port->hba->ldapbinddn ? port->hba->ldapbinddn : "",
- port->hba->ldapserver,
+ server_name,
ldap_err2string(r)),
errdetail_for_ldap(ldap)));
ldap_unbind(ldap);
@@ -2658,7 +2719,9 @@ CheckLDAPAuth(Port *port)
{
ereport(LOG,
(errmsg("could not search LDAP for filter \"%s\" on server \"%s\": %s",
- filter, port->hba->ldapserver, ldap_err2string(r)),
+ filter,
+ server_name,
+ ldap_err2string(r)),
errdetail_for_ldap(ldap)));
ldap_unbind(ldap);
pfree(passwd);
@@ -2673,14 +2736,13 @@ CheckLDAPAuth(Port *port)
ereport(LOG,
(errmsg("LDAP user \"%s\" does not exist", port->user_name),
errdetail("LDAP search for filter \"%s\" on server \"%s\" returned no entries.",
- filter, port->hba->ldapserver)));
+ filter, server_name)));
else
ereport(LOG,
(errmsg("LDAP user \"%s\" is not unique", port->user_name),
errdetail_plural("LDAP search for filter \"%s\" on server \"%s\" returned %d entry.",
"LDAP search for filter \"%s\" on server \"%s\" returned %d entries.",
- count,
- filter, port->hba->ldapserver, count)));
+ count, filter, server_name, count)));
ldap_unbind(ldap);
pfree(passwd);
@@ -2698,8 +2760,7 @@ CheckLDAPAuth(Port *port)
(void) ldap_get_option(ldap, LDAP_OPT_ERROR_NUMBER, &error);
ereport(LOG,
(errmsg("could not get dn for the first entry matching \"%s\" on server \"%s\": %s",
- filter, port->hba->ldapserver,
- ldap_err2string(error)),
+ filter, server_name, ldap_err2string(error)),
errdetail_for_ldap(ldap)));
ldap_unbind(ldap);
pfree(passwd);
@@ -2719,7 +2780,7 @@ CheckLDAPAuth(Port *port)
{
ereport(LOG,
(errmsg("could not unbind after searching for user \"%s\" on server \"%s\"",
- fulluser, port->hba->ldapserver)));
+ fulluser, server_name)));
pfree(passwd);
pfree(fulluser);
return STATUS_ERROR;
@@ -2750,7 +2811,7 @@ CheckLDAPAuth(Port *port)
{
ereport(LOG,
(errmsg("LDAP login failed for user \"%s\" on server \"%s\": %s",
- fulluser, port->hba->ldapserver, ldap_err2string(r)),
+ fulluser, server_name, ldap_err2string(r)),
errdetail_for_ldap(ldap)));
ldap_unbind(ldap);
pfree(passwd);
diff --git a/src/backend/libpq/hba.c b/src/backend/libpq/hba.c
index 59de1b7639..9c4e81a0e9 100644
--- a/src/backend/libpq/hba.c
+++ b/src/backend/libpq/hba.c
@@ -1500,7 +1500,9 @@ parse_hba_line(TokenizedLine *tok_line, int elevel)
*/
if (parsedline->auth_method == uaLDAP)
{
+#ifndef HAVE_LDAP_INITIALIZE
MANDATORY_AUTH_ARG(parsedline->ldapserver, "ldapserver", "ldap");
+#endif
/*
* LDAP can operate in two modes: either with a direct bind, using
--
2.21.0
On Tue, Mar 19, 2019 at 9:01 PM Thomas Munro <thomas.munro@gmail.com> wrote:
I'd like to commit this soon.
Done, after some more comment adjustments. Thanks Daniel and Graham
for your feedback!
--
Thomas Munro
https://enterprisedb.com