[Patch] Mingw: Fix import library extension, build actual static libraries

Started by Sandro Manialmost 7 years ago7 messages

manisandro@gmail.com

almost 7 years ago

2 attachment(s)

Currently building postgresql for Win32 with a mingw toolchain produces
import libraries with *.a extension, whereas the extension should be
*.dll.a. There are various downstream workarounds for this, see i.e. [1]https://src.fedoraproject.org/rpms/mingw-postgresql/blob/master/f/mingw-postgresql.spec#_144
and [2]https://aur.archlinux.org/cgit/aur.git/tree/0001-Use-.dll.a-as-extension-for-import-libraries.patch?h=mingw-w64-postgresql. The attached patch 0001-Fix-import-library-extension.patch
addresses this.

Related, no actual static libraries are produced alongside the
respective dlls. The attached patch 0002-Build-static-libraries.patch
addresses this, in a similar fashion as is already done for the AIX case
in Makefile.shlib.

Thanks Sandro

[1]: https://src.fedoraproject.org/rpms/mingw-postgresql/blob/master/f/mingw-postgresql.spec#_144
https://src.fedoraproject.org/rpms/mingw-postgresql/blob/master/f/mingw-postgresql.spec#_144
[2]: https://aur.archlinux.org/cgit/aur.git/tree/0001-Use-.dll.a-as-extension-for-import-libraries.patch?h=mingw-w64-postgresql
https://aur.archlinux.org/cgit/aur.git/tree/0001-Use-.dll.a-as-extension-for-import-libraries.patch?h=mingw-w64-postgresql

Attachments:

0001-Fix-import-library-extension.patchtext/x-patch; name=0001-Fix-import-library-extension.patchDownload

From 9cf2d3076750f0214c448aaabcbcf7ba5aed6594 Mon Sep 17 00:00:00 2001
From: Sandro Mani <manisandro@gmail.com>
Date: Thu, 7 Mar 2019 14:48:57 +0100
Subject: [PATCH 1/2] Fix import library extension

---
 src/Makefile.shlib   |  5 +++--
 src/backend/Makefile | 24 ++++++++++++------------
 2 files changed, 15 insertions(+), 14 deletions(-)

diff --git a/src/Makefile.shlib b/src/Makefile.shlib
index 373d73caef..aeaa46e89b 100644
--- a/src/Makefile.shlib
+++ b/src/Makefile.shlib
@@ -383,12 +383,12 @@ $(stlib): $(shlib)
 # Else we just use --export-all-symbols.
 ifeq (,$(SHLIB_EXPORTS))
 $(shlib): $(OBJS) | $(SHLIB_PREREQS)
-	$(CC) $(CFLAGS)  -shared -static-libgcc -o $@  $(OBJS) $(LDFLAGS) $(LDFLAGS_SL) $(SHLIB_LINK) $(LIBS) -Wl,--export-all-symbols -Wl,--out-implib=$(stlib)
+	$(CC) $(CFLAGS)  -shared -static-libgcc -o $@  $(OBJS) $(LDFLAGS) $(LDFLAGS_SL) $(SHLIB_LINK) $(LIBS) -Wl,--export-all-symbols -Wl,--out-implib=lib$(NAME).dll.a
 else
 DLL_DEFFILE = lib$(NAME)dll.def
 
 $(shlib): $(OBJS) $(DLL_DEFFILE) | $(SHLIB_PREREQS)
-	$(CC) $(CFLAGS)  -shared -static-libgcc -o $@  $(OBJS) $(DLL_DEFFILE) $(LDFLAGS) $(LDFLAGS_SL) $(SHLIB_LINK) $(LIBS) -Wl,--out-implib=$(stlib)
+	$(CC) $(CFLAGS)  -shared -static-libgcc -o $@  $(OBJS) $(DLL_DEFFILE) $(LDFLAGS) $(LDFLAGS_SL) $(SHLIB_LINK) $(LIBS) -Wl,--out-implib=lib$(NAME).dll.a
 endif
 
 endif # PORTNAME == cygwin
@@ -482,6 +482,7 @@ endif # not cygwin
 endif # not aix
 ifneq (,$(findstring $(PORTNAME),win32 cygwin))
 	$(INSTALL_SHLIB) $< '$(DESTDIR)$(bindir)/$(shlib)'
+	$(INSTALL_DATA) $< '$(DESTDIR)$(libdir)/lib$(NAME).dll.a'
 endif
 else # no soname
 	$(INSTALL_SHLIB) $< '$(DESTDIR)$(pkglibdir)/$(shlib)'
diff --git a/src/backend/Makefile b/src/backend/Makefile
index 478a96db9b..e4f52d0b53 100644
--- a/src/backend/Makefile
+++ b/src/backend/Makefile
@@ -69,11 +69,11 @@ endif
 ifeq ($(PORTNAME), cygwin)
 
 postgres: $(OBJS)
-	$(CC) $(CFLAGS) $(call expand_subsys,$^) $(LDFLAGS) $(LDFLAGS_EX) $(export_dynamic) -Wl,--stack,$(WIN32_STACK_RLIMIT) -Wl,--export-all-symbols -Wl,--out-implib=libpostgres.a $(LIBS) -o $@
+	$(CC) $(CFLAGS) $(call expand_subsys,$^) $(LDFLAGS) $(LDFLAGS_EX) $(export_dynamic) -Wl,--stack,$(WIN32_STACK_RLIMIT) -Wl,--export-all-symbols -Wl,--out-implib=libpostgres.dll.a $(LIBS) -o $@
 
-# libpostgres.a is actually built in the preceding rule, but we need this to
+# libpostgres.dll.a is actually built in the preceding rule, but we need this to
 # ensure it's newer than postgres; see notes in src/backend/parser/Makefile
-libpostgres.a: postgres
+libpostgres.dll.a: postgres
 	touch $@
 
 endif # cygwin
@@ -82,11 +82,11 @@ ifeq ($(PORTNAME), win32)
 LIBS += -lsecur32
 
 postgres: $(OBJS) $(WIN32RES)
-	$(CC) $(CFLAGS) $(call expand_subsys,$(OBJS)) $(WIN32RES) $(LDFLAGS) $(LDFLAGS_EX) -Wl,--stack=$(WIN32_STACK_RLIMIT) -Wl,--export-all-symbols -Wl,--out-implib=libpostgres.a $(LIBS) -o $@$(X)
+	$(CC) $(CFLAGS) $(call expand_subsys,$(OBJS)) $(WIN32RES) $(LDFLAGS) $(LDFLAGS_EX) -Wl,--stack=$(WIN32_STACK_RLIMIT) -Wl,--export-all-symbols -Wl,--out-implib=libpostgres.dll.a $(LIBS) -o $@$(X)
 
-# libpostgres.a is actually built in the preceding rule, but we need this to
+# libpostgres.dll.a is actually built in the preceding rule, but we need this to
 # ensure it's newer than postgres; see notes in src/backend/parser/Makefile
-libpostgres.a: postgres
+libpostgres.dll.a: postgres
 	touch $@
 
 endif # win32
@@ -195,12 +195,12 @@ distprep:
 install: all installdirs install-bin
 ifeq ($(PORTNAME), cygwin)
 ifeq ($(MAKE_DLL), true)
-	$(INSTALL_DATA) libpostgres.a '$(DESTDIR)$(libdir)/libpostgres.a'
+	$(INSTALL_DATA) libpostgres.dll.a '$(DESTDIR)$(libdir)/libpostgres.dll.a'
 endif
 endif
 ifeq ($(PORTNAME), win32)
 ifeq ($(MAKE_DLL), true)
-	$(INSTALL_DATA) libpostgres.a '$(DESTDIR)$(libdir)/libpostgres.a'
+	$(INSTALL_DATA) libpostgres.dll.a '$(DESTDIR)$(libdir)/libpostgres.dll.a'
 endif
 endif
 	$(MAKE) -C catalog install-data
@@ -260,12 +260,12 @@ ifeq ($(MAKE_EXPORTS), true)
 endif
 ifeq ($(PORTNAME), cygwin)
 ifeq ($(MAKE_DLL), true)
-	rm -f '$(DESTDIR)$(libdir)/libpostgres.a'
+	rm -f '$(DESTDIR)$(libdir)/libpostgres.dll.a'
 endif
 endif
 ifeq ($(PORTNAME), win32)
 ifeq ($(MAKE_DLL), true)
-	rm -f '$(DESTDIR)$(libdir)/libpostgres.a'
+	rm -f '$(DESTDIR)$(libdir)/libpostgres.dll.a'
 endif
 endif
 	$(MAKE) -C catalog uninstall-data
@@ -284,10 +284,10 @@ endif
 clean:
 	rm -f $(LOCALOBJS) postgres$(X) $(POSTGRES_IMP)
 ifeq ($(PORTNAME), cygwin)
-	rm -f postgres.dll libpostgres.a
+	rm -f postgres.dll libpostgres.dll.a
 endif
 ifeq ($(PORTNAME), win32)
-	rm -f postgres.dll libpostgres.a $(WIN32RES)
+	rm -f postgres.dll libpostgres.dll.a $(WIN32RES)
 endif
 
 distclean: clean
-- 
2.21.0

0002-Build-static-libraries.patchtext/x-patch; name=0002-Build-static-libraries.patchDownload

From 6e0b9e2a8ea761d712788e5f178d53cecf54382e Mon Sep 17 00:00:00 2001
From: Sandro Mani <manisandro@gmail.com>
Date: Thu, 7 Mar 2019 14:49:35 +0100
Subject: [PATCH 2/2] Build static libraries

---
 src/Makefile.shlib | 6 ++++++
 1 file changed, 6 insertions(+)

diff --git a/src/Makefile.shlib b/src/Makefile.shlib
index aeaa46e89b..2b8c7ddc2b 100644
--- a/src/Makefile.shlib
+++ b/src/Makefile.shlib
@@ -383,11 +383,17 @@ $(stlib): $(shlib)
 # Else we just use --export-all-symbols.
 ifeq (,$(SHLIB_EXPORTS))
 $(shlib): $(OBJS) | $(SHLIB_PREREQS)
+	rm -f $(stlib)
+	$(LINK.static) $(stlib) $(OBJS)
+	$(RANLIB) $(stlib)
 	$(CC) $(CFLAGS)  -shared -static-libgcc -o $@  $(OBJS) $(LDFLAGS) $(LDFLAGS_SL) $(SHLIB_LINK) $(LIBS) -Wl,--export-all-symbols -Wl,--out-implib=lib$(NAME).dll.a
 else
 DLL_DEFFILE = lib$(NAME)dll.def
 
 $(shlib): $(OBJS) $(DLL_DEFFILE) | $(SHLIB_PREREQS)
+	rm -f $(stlib)
+	$(LINK.static) $(stlib) $(OBJS)
+	$(RANLIB) $(stlib)
 	$(CC) $(CFLAGS)  -shared -static-libgcc -o $@  $(OBJS) $(DLL_DEFFILE) $(LDFLAGS) $(LDFLAGS_SL) $(SHLIB_LINK) $(LIBS) -Wl,--out-implib=lib$(NAME).dll.a
 endif
 
-- 
2.21.0

Sandro Mani

manisandro@gmail.com

almost 7 years ago

In reply to: Sandro Mani (#1)

Re: [Patch] Mingw: Fix import library extension, build actual static libraries

Any chance these patches could be considered?

Thanks
Sandro

On Thu, Mar 7, 2019 at 3:23 PM Sandro Mani <manisandro@gmail.com> wrote:

Show quoted text

Currently building postgresql for Win32 with a mingw toolchain produces
import libraries with *.a extension, whereas the extension should be
*.dll.a. There are various downstream workarounds for this, see i.e. [1]
and [2]. The attached patch 0001-Fix-import-library-extension.patch
addresses this.

Related, no actual static libraries are produced alongside the respective
dlls. The attached patch 0002-Build-static-libraries.patch addresses this,
in a similar fashion as is already done for the AIX case in Makefile.shlib.

Thanks Sandro

[1]
https://src.fedoraproject.org/rpms/mingw-postgresql/blob/master/f/mingw-postgresql.spec#_144
[2]
https://aur.archlinux.org/cgit/aur.git/tree/0001-Use-.dll.a-as-extension-for-import-libraries.patch?h=mingw-w64-postgresql

Robert Haas

robertmhaas@gmail.com

almost 7 years ago

In reply to: Sandro Mani (#2)

Re: [Patch] Mingw: Fix import library extension, build actual static libraries

On Tue, Apr 2, 2019 at 5:32 AM Sandro Mani <manisandro@gmail.com> wrote:

Any chance these patches could be considered?

You should add them to the Open CommitFest, currently
https://commitfest.postgresql.org/23/

That CommitFest is scheduled to start in July; we are about to go to
feature freeze for v12, and your patch was submitted after the last
CommitFest for v12 had already started. That doesn't *necessarily*
mean that someone won't make an argument for including your patch in
v12 rather than making it wait another year, if for example it is
viewed to be a bug fix, but the odds are against you, because a lot of
other people's patches have been in line for much longer.

--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

Noah Misch

noah@leadboat.com

over 6 years ago

In reply to: Sandro Mani (#1)

Re: [Patch] Mingw: Fix import library extension, build actual static libraries

On Thu, Mar 07, 2019 at 03:23:50PM +0100, Sandro Mani wrote:

Related, no actual static libraries are produced alongside the respective
dlls. The attached patch 0002-Build-static-libraries.patch addresses this,
in a similar fashion as is already done for the AIX case in Makefile.shlib.

We don't build static libraries on AIX, though Makefile.shlib uses the
$(stlib) variable to get a name for the *.a shared library it makes. Here's
an example of one AIX Makefile.shlib build sequence, from
https://buildfarm.postgresql.org/cgi-bin/show_stage_log.pl?nm=hornet&dt=2019-04-15%2022%3A35%3A52&stg=make

rm -f libpq.a
ar crs libpq.a fe-auth.o fe-auth-scram.o fe-connect.o fe-exec.o fe-misc.o fe-print.o fe-lobj.o fe-protocol2.o fe-protocol3.o pqexpbuffer.o fe-secure.o libpq-events.o encnames.o wchar.o fe-secure-openssl.o fe-secure-common.o
touch libpq.a
../../../src/backend/port/aix/mkldexport.sh libpq.a libpq.so.5 >libpq.exp
xlc_r -qmaxmem=33554432 -D_LARGE_FILES=1 -qnoansialias -g -O2 -qmaxmem=16384 -qsrcmsg -D_REENTRANT -D_THREAD_SAFE -D_POSIX_PTHREAD_SEMANTICS -o libpq.so.5 libpq.a -Wl,-bE:libpq.exp -L../../../src/port -L../../../src/common -lpgcommon_shlib -lpgport_shlib -L/home/nm/sw/nopath/libxml2-64/lib -L/home/nm/sw/nopath/icu58.2-64/lib -L/home/nm/sw/nopath/uuid-64/lib -L/home/nm/sw/nopath/openldap-64/lib -L/home/nm/sw/nopath/icu58.2-64/lib -L/home/nm/sw/nopath/libxml2-64/lib -Wl,-blibpath:'/home/nm/farm/xlc64/HEAD/inst/lib:/home/nm/sw/nopath/libxml2-64/lib:/home/nm/sw/nopath/icu58.2-64/lib:/home/nm/sw/nopath/uuid-64/lib:/home/nm/sw/nopath/openldap-64/lib:/home/nm/sw/nopath/icu58.2-64/lib:/home/nm/sw/nopath/libxml2-64/lib:/usr/lib:/lib' -Wl,-bnoentry -Wl,-H512 -Wl,-bM:SRE -lintl -lssl -lcrypto -lm -lldap_r -llber -lpthreads
rm -f libpq.a
ar crs libpq.a libpq.so.5

Andrew Dunstan

andrew.dunstan@2ndquadrant.com

over 6 years ago

In reply to: Noah Misch (#4)

Re: [Patch] Mingw: Fix import library extension, build actual static libraries

On 4/16/19 1:22 AM, Noah Misch wrote:

On Thu, Mar 07, 2019 at 03:23:50PM +0100, Sandro Mani wrote:

Related, no actual static libraries are produced alongside the respective
dlls. The attached patch 0002-Build-static-libraries.patch addresses this,
in a similar fashion as is already done for the AIX case in Makefile.shlib.

We don't build static libraries on AIX, though Makefile.shlib uses the
$(stlib) variable to get a name for the *.a shared library it makes. Here's
an example of one AIX Makefile.shlib build sequence, from
https://buildfarm.postgresql.org/cgi-bin/show_stage_log.pl?nm=hornet&dt=2019-04-15%2022%3A35%3A52&stg=make

rm -f libpq.a
ar crs libpq.a fe-auth.o fe-auth-scram.o fe-connect.o fe-exec.o fe-misc.o fe-print.o fe-lobj.o fe-protocol2.o fe-protocol3.o pqexpbuffer.o fe-secure.o libpq-events.o encnames.o wchar.o fe-secure-openssl.o fe-secure-common.o
touch libpq.a
../../../src/backend/port/aix/mkldexport.sh libpq.a libpq.so.5 >libpq.exp
xlc_r -qmaxmem=33554432 -D_LARGE_FILES=1 -qnoansialias -g -O2 -qmaxmem=16384 -qsrcmsg -D_REENTRANT -D_THREAD_SAFE -D_POSIX_PTHREAD_SEMANTICS -o libpq.so.5 libpq.a -Wl,-bE:libpq.exp -L../../../src/port -L../../../src/common -lpgcommon_shlib -lpgport_shlib -L/home/nm/sw/nopath/libxml2-64/lib -L/home/nm/sw/nopath/icu58.2-64/lib -L/home/nm/sw/nopath/uuid-64/lib -L/home/nm/sw/nopath/openldap-64/lib -L/home/nm/sw/nopath/icu58.2-64/lib -L/home/nm/sw/nopath/libxml2-64/lib -Wl,-blibpath:'/home/nm/farm/xlc64/HEAD/inst/lib:/home/nm/sw/nopath/libxml2-64/lib:/home/nm/sw/nopath/icu58.2-64/lib:/home/nm/sw/nopath/uuid-64/lib:/home/nm/sw/nopath/openldap-64/lib:/home/nm/sw/nopath/icu58.2-64/lib:/home/nm/sw/nopath/libxml2-64/lib:/usr/lib:/lib' -Wl,-bnoentry -Wl,-H512 -Wl,-bM:SRE -lintl -lssl -lcrypto -lm -lldap_r -llber -lpthreads
rm -f libpq.a
ar crs libpq.a libpq.so.5

I'm wondering if it wouldn't be better to set the value of stlib for
windows, instead of changes like this:

-    $(CC) $(CFLAGS)  -shared -static-libgcc -o $@  $(OBJS) $(LDFLAGS)
$(LDFLAGS_SL) $(SHLIB_LINK) $(LIBS) -Wl,--export-all-symbols
-Wl,--out-implib=$(stlib)
+    $(CC) $(CFLAGS)  -shared -static-libgcc -o $@  $(OBJS) $(LDFLAGS)
$(LDFLAGS_SL) $(SHLIB_LINK) $(LIBS) -Wl,--export-all-symbols
-Wl,--out-implib=lib$(NAME).dll.a

I'm also wondering if changing this will upset third party authors.

Thoughts?

cheers

andrew

--
Andrew Dunstan https://www.2ndQuadrant.com
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

Noah Misch

noah@leadboat.com

over 6 years ago

In reply to: Andrew Dunstan (#5)

Re: [Patch] Mingw: Fix import library extension, build actual static libraries

On Tue, Jul 09, 2019 at 08:26:52AM -0400, Andrew Dunstan wrote:

On 4/16/19 1:22 AM, Noah Misch wrote:

On Thu, Mar 07, 2019 at 03:23:50PM +0100, Sandro Mani wrote:

Related, no actual static libraries are produced alongside the respective
dlls. The attached patch 0002-Build-static-libraries.patch addresses this,
in a similar fashion as is already done for the AIX case in Makefile.shlib.

We don't build static libraries on AIX, though Makefile.shlib uses the
$(stlib) variable to get a name for the *.a shared library it makes. Here's
an example of one AIX Makefile.shlib build sequence, from
https://buildfarm.postgresql.org/cgi-bin/show_stage_log.pl?nm=hornet&dt=2019-04-15%2022%3A35%3A52&stg=make

rm -f libpq.a
ar crs libpq.a fe-auth.o fe-auth-scram.o fe-connect.o fe-exec.o fe-misc.o fe-print.o fe-lobj.o fe-protocol2.o fe-protocol3.o pqexpbuffer.o fe-secure.o libpq-events.o encnames.o wchar.o fe-secure-openssl.o fe-secure-common.o
touch libpq.a
../../../src/backend/port/aix/mkldexport.sh libpq.a libpq.so.5 >libpq.exp
xlc_r -qmaxmem=33554432 -D_LARGE_FILES=1 -qnoansialias -g -O2 -qmaxmem=16384 -qsrcmsg -D_REENTRANT -D_THREAD_SAFE -D_POSIX_PTHREAD_SEMANTICS -o libpq.so.5 libpq.a -Wl,-bE:libpq.exp -L../../../src/port -L../../../src/common -lpgcommon_shlib -lpgport_shlib -L/home/nm/sw/nopath/libxml2-64/lib -L/home/nm/sw/nopath/icu58.2-64/lib -L/home/nm/sw/nopath/uuid-64/lib -L/home/nm/sw/nopath/openldap-64/lib -L/home/nm/sw/nopath/icu58.2-64/lib -L/home/nm/sw/nopath/libxml2-64/lib -Wl,-blibpath:'/home/nm/farm/xlc64/HEAD/inst/lib:/home/nm/sw/nopath/libxml2-64/lib:/home/nm/sw/nopath/icu58.2-64/lib:/home/nm/sw/nopath/uuid-64/lib:/home/nm/sw/nopath/openldap-64/lib:/home/nm/sw/nopath/icu58.2-64/lib:/home/nm/sw/nopath/libxml2-64/lib:/usr/lib:/lib' -Wl,-bnoentry -Wl,-H512 -Wl,-bM:SRE -lintl -lssl -lcrypto -lm -lldap_r -llber -lpthreads
rm -f libpq.a
ar crs libpq.a libpq.so.5

I'm wondering if it wouldn't be better to set the value of stlib for
windows, instead of changes like this:
-ï¿½ï¿½ï¿½ $(CC) $(CFLAGS)ï¿½ -shared -static-libgcc -o $@ï¿½ $(OBJS) $(LDFLAGS)
$(LDFLAGS_SL) $(SHLIB_LINK) $(LIBS) -Wl,--export-all-symbols
-Wl,--out-implib=$(stlib)
+ï¿½ï¿½ï¿½ $(CC) $(CFLAGS)ï¿½ -shared -static-libgcc -o $@ï¿½ $(OBJS) $(LDFLAGS)
$(LDFLAGS_SL) $(SHLIB_LINK) $(LIBS) -Wl,--export-all-symbols
-Wl,--out-implib=lib$(NAME).dll.a

If we're going to have both static libs and import libs, they need different
names. But it may be an improvement to set implib = lib$(NAME).dll.a and use
$(implib) in Make recipes like this one.

I'm also wondering if changing this will upset third party authors.

I'd make it master-only, but I don't expect big trouble. GNU ld does pick
libFOO.dll.a when both that and libFOO.a are available. Folks probably have
scripts that copy libpq.a to libpq.lib for the benefit of MSVC, and those
would need to change.

Peter Eisentraut

peter.eisentraut@2ndquadrant.com

over 6 years ago

In reply to: Sandro Mani (#1)

Re: [Patch] Mingw: Fix import library extension, build actual static libraries

On 2019-03-07 15:23, Sandro Mani wrote:

Currently building postgresql for Win32 with a mingw toolchain produces
import libraries with *.a extension, whereas the extension should be
*.dll.a.

I have read the MinGW documentation starting from
<http://www.mingw.org/wiki/DLL> and have found no information supporting
this claim. All their examples match our existing naming. What is your
source for this information -- other than ...

There are various downstream workarounds for this, see i.e. [1]
and [2]. The attached patch 0001-Fix-import-library-extension.patch
addresses this.

I'm confused what Fedora and Arch Linux have to do with this. Are they
distributing Windows binaries of third-party software?

--
Peter Eisentraut http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services