libtool, -fstack-protector, -nostdlib, and while we're at it -Bdirect on SunOS

[Richard PALO]

In order to get over suppression of -fstack-protector namely in SunOS
builds for perl and cups, I've been testing for a real long time now

Re: Bug: linking shared libraries on Cygwin results in undefined references to __stack_chck_guard for code compiled with -fstack-protector

This patch prevents libtools link phase from suppressing
'-fstack-protector*' switches such that gcc can issue the proper link
statement.

Well, that was fine for gcc, but when building with g++ libtool adds for
some reason '-nostdlib' which foils the above patch, so that needed
attending to. It is useful to know that this patch fixes similar
problems with, for example, '-pthread' and others!

Then, Mr Orth found a problem on SunOS x86_64 with -Bdirect as mentioned
here: http://debbugs.gnu.org/cgi/bugreport.cgi?bug=16452 and
http://gcc.gnu.org/bugzilla/show_bug.cgi?id=59788

So I have some patches found here:
http://www.netbsd.org/~richard/libtool-SunOS/ that I've been using
without issue on both i386 and x86_64.

Summary:

0001-add-fstack-protector-link-support-to-libtool.patch
permits libtool to generate -fstack-protector* on all platforms.

0002-x86-64-sunos-fix-for-g-and-Wl-Bdirect-involving-libt.patch
affects only SunOS x86_64 libtool, gcc47 & gcc48

0003-remove-nostdlib-from-libtool-shared-libraries-for-fs.patch
suppresses -nostdlib with g++ on SunOS only (see note # below)

0004-use-libtool-fstack-protector-support-in-cups.patch
re-enable in perl

0005-put-back-stack-protector-on-solaris-in-perl.patch
re-enable in perl

Note # other platforms that wish to test suppression of '-nostdlib' will
need to apply a similar patch for their target. Apparently
the darwin platform may have (or have had) issues...

I encourage direct participation in the libtool@ discussion
'g++ and -nostdlib'

NB libtool-base is *not* revbump'd in the any of the above patches,
although gcc*, cups, and perl are.

[Joerg Sonnenberger]

Post by Richard PALO
0001-add-fstack-protector-link-support-to-libtool.patch
permits libtool to generate -fstack-protector* on all platforms.

*sigh* Stupid, stupid, crappy specs. I dislike this. A lot. It should
never have become a spec modifier. Is this for broken GCC instances
outside pkgsrc or for the versions we have some control over?

Post by Richard PALO
0002-x86-64-sunos-fix-for-g-and-Wl-Bdirect-involving-libt.patch
affects only SunOS x86_64 libtool, gcc47 & gcc48

I believe the correct fix is the same I am going to apply for NetBSD
soon -- to just not include GCC's unwinder in libgcc_s.

Joerg

[Richard PALO]

Post by Joerg Sonnenberger

Post by Richard PALO
0001-add-fstack-protector-link-support-to-libtool.patch
permits libtool to generate -fstack-protector* on all platforms.

*sigh* Stupid, stupid, crappy specs. I dislike this. A lot. It should
never have become a spec modifier. Is this for broken GCC instances
outside pkgsrc or for the versions we have some control over?

Well, this patch only prevents libtool invoked with '--mode=link' from
suppressing '-fstack-protector*' allowing the flag to be passed through
unchanged to gcc/link where it will 'do the right thing'.

In particular, for SunOS, this permits gcc to add the following to the
link statement: '-lssp_nonshared -lssp'.

I should reiterate that although this is sufficient for gcc, for g++
0003-remove-nostdlib-from-libtool-shared-libraries-for-fs.patch is
needed as well in order to prevent g++ from consequently stripping via
'-nostdlib' what it should be adding.

To answer your question, this particular patch (0001) is solely a
libtool patch, which works with both native and pkgsrc gcc/g++.

In theory, and in my testing thus far on SunOS, this should simplify
pkgsrc as opposed to complicate it given libtool won't swallow any
longer the '-fstack-protector*' flag (which affected at very least perl
and cups).

Post by Joerg Sonnenberger

Post by Richard PALO
0002-x86-64-sunos-fix-for-g-and-Wl-Bdirect-involving-libt.patch
affects only SunOS x86_64 libtool, gcc47 & gcc48

I believe the correct fix is the same I am going to apply for NetBSD
soon -- to just not include GCC's unwinder in libgcc_s.

I'm not so sure, as this is the equivalent of the upstream gcc *and*
libtool patch.

Post by Joerg Sonnenberger
commit 525cddd2bcea4c565d6dd1d2d55dd1de1a476b67
Date: Sat Jan 18 10:07:52 2014 +1300
libtool: opt_duplicate_compiler_generated_deps is harmful on Solaris
Fix for http://debbugs.gnu.org/cgi/bugreport.cgi?bug=16452.
* build-aux/ltmain.in (libtool_validate_options): disable the
opt_duplicate_compiler_generated_deps optimization for Solaris2 so
that gcc-4.9+ compiled C++ code with -Wl,-Bdirect on 64-bit Solaris
x86 can avoid unwinding failures caused by accidental mixing of the
libc and libgcc_s unwinders in a single executable.

A link to the gcc bugreports is indicated in the patchset as well.
Again, this patcheset only affects SunOS platforms (gcc* and libtool).

cordially,

richard

[Richard PALO]

Post by Richard PALO
In order to get over suppression of -fstack-protector namely in SunOS
builds for perl and cups, I've been testing for a real long time now

Re: Bug: linking shared libraries on Cygwin results in undefined
references to __stack_chck_guard for code compiled with -fstack-protector

This patch prevents libtools link phase from suppressing
'-fstack-protector*' switches such that gcc can issue the proper link
statement.
Well, that was fine for gcc, but when building with g++ libtool adds for
some reason '-nostdlib' which foils the above patch, so that needed
attending to. It is useful to know that this patch fixes similar
problems with, for example, '-pthread' and others!
Then, Mr Orth found a problem on SunOS x86_64 with -Bdirect as mentioned
here: http://debbugs.gnu.org/cgi/bugreport.cgi?bug=16452 and
http://gcc.gnu.org/bugzilla/show_bug.cgi?id=59788
http://www.netbsd.org/~richard/libtool-SunOS/ that I've been using
without issue on both i386 and x86_64.
0001-add-fstack-protector-link-support-to-libtool.patch
permits libtool to generate -fstack-protector* on all platforms.
0002-x86-64-sunos-fix-for-g-and-Wl-Bdirect-involving-libt.patch
affects only SunOS x86_64 libtool, gcc47 & gcc48
0003-remove-nostdlib-from-libtool-shared-libraries-for-fs.patch
suppresses -nostdlib with g++ on SunOS only (see note # below)
0004-use-libtool-fstack-protector-support-in-cups.patch
re-enable in perl
0005-put-back-stack-protector-on-solaris-in-perl.patch
re-enable in perl
Note # other platforms that wish to test suppression of '-nostdlib' will
need to apply a similar patch for their target. Apparently
the darwin platform may have (or have had) issues...
'g++ and -nostdlib'
NB libtool-base is *not* revbump'd in the any of the above patches,
although gcc*, cups, and perl are.

If there is no motivated objection, I plan on pushing this early may.

Again, for non SunOS platforms, the only effect is to see libtool pass
through '-fstack-protector*' for gcc with mode=link having been
suppressed up to now.

For SunOS, '-fstack-protector*' will now work for both gcc/g++ as well
as picking up an fix for '-Wl,-Bdirect' when using either gcc47 or gcc48
and should be ready now for upstream gcc49.

[Richard PALO]

I have updated http://www.netbsd.org/~richard/libtool-SunOS/
in order to fix existing problems with manual-libtool.m4 initially
skipped, allowing applying portions of this patch correctly there (in
manual-libtool.m4) allowing regeneration of patch-ab and patch-ad.

Restructured a bit and revbump'd gcc47* and gcc48.

[Richard PALO]

I have updated http://www.netbsd.org/~richard/libtool-SunOS/
in order to fix existing problems with manual-libtool.m4 initially
skipped, allowing applying portions of this patch correctly there (in
manual-libtool.m4) allowing regeneration of patch-ab and patch-ad.

Restructured a bit and revbump'd gcc47* and gcc48.

[Richard PALO]

As requested by jperkin@, I'm posting here the attached patch to
complete this libtool update.

Concerning manual-libtool.m4 and patch-ab, I had omitted removing
'$predep_objects' and '$postdep_objects' in my initial patch removing
'-nostdlib'...

ld: warning: file /usr/lib/crti.o: attempted multiple inclusion of file
ld: warning: file /opt/local/gcc47/lib/gcc/i386-sun-solaris2.11/4.7.3/crtbegin.o: attempted multiple inclusion of file
..
ld: warning: file /opt/local/gcc47/lib/gcc/i386-sun-solaris2.11/4.7.3/crtend.o: attempted multiple inclusion of file
ld: warning: file /usr/lib/crtn.o: attempted multiple inclusion of file

Thus updated, the g++ case becomes more or less the same as the gcc case

archive_cmds="\$CC -shared \$pic_flag \${wl}-z \${wl}text \${wl}-h \${wl}\$soname -o \$lib \$libobjs \$deplibs \$compiler_flags"
archive_expsym_cmds="echo \\\"{ global:\\\" > \$lib.exp~cat \$export_symbols | \$SED -e \\\"s/\\\\(.*\\\\)/\\\\1;/\\\" >> \$lib.exp~echo \\\"local: *; };\\\" >> \$lib.exp~
\$CC -shared \$pic_flag \${wl}-z \${wl}text \${wl}-M \${wl}\$lib.exp \${wl}-h \${wl}\$soname -o \$lib \$libobjs \$deplibs \$compiler_flags~\$RM \$lib.exp"
--
archive_cmds="\$CC -shared \$pic_flag \$libobjs \$deplibs \$compiler_flags \${wl}-h \$wl\$soname -o \$lib"
archive_expsym_cmds="echo \\\"{ global:\\\" > \$lib.exp~cat \$export_symbols | \$SED -e \\\"s/\\\\(.*\\\\)/\\\\1;/\\\" >> \$lib.exp~echo \\\"local: *; };\\\" >> \$lib.exp~
\$CC -shared \$pic_flag \${wl}-M \$wl\$lib.exp \$wl-h \$wl\$soname -o \$lib \$libobjs \$deplibs \$compiler_flags~\$RM \$lib.exp"

Notice the only real difference is that for gcc '-z text' is added.

For what it is worth, I also tried that in my test cases, and so far
there is no issue...but I'm not specifically recommending it at the
moment (perhaps someone sees the need).

patch-ac is the same as before, and for the case of passing
'-fstack-protector*' through to linking is the driver for this patchset
in the beginning. The second case of fixing the problem on SunOS with
'-Bdirect' is also thrown in as before (cf references in patch).

Observations as to why the attached patchset should not be committed?

For information, my test case is simply graphviz, with the following
options: gd gtk guile lua pangocairo perl rsvg tcl x11
has had these issues building libgv_perl.la.

[Jonathan Perkin]

Post by Richard PALO
this libtool update.

Thanks Richard. I will run this through a full bulk build. Results
as I get them.

Regards,

[Jonathan Perkin]

Post by Jonathan Perkin

Post by Richard PALO
this libtool update.

Thanks Richard. I will run this through a full bulk build. Results
as I get them.

These are now in:

http://us-east.manta.joyent.com/pkgsrc/public/reports/upstream-bulktest/20140710.1609/meta/report.html

Compared to a run of clean upstream-trunk32 at a similar time:

http://us-east.manta.joyent.com/pkgsrc/public/reports/upstream-trunk32/20140710.2233/meta/report.html

the patch clearly has positive benefits, with about 70 additional
pacakges, mainly due to fixes in graphviz and libsodium.

On that basis it's a +1 from me.

[Richard PALO]

There is one thing that has been bugging me for months is the fact that
(using my test case for graphviz) frequently the need for
'-fstack-protector' comes from a dependant library, such as perl or cups
or something.

That may be fine for programs, but for shared libraries, that can get
quite complicated.

In libtool terms, there is a variable in the .la file named
inherited_linker_flags that is added to the link command to the shared
library... I guess a reasonably good example of this is '-pthread'.

I believe that '-fstack-protector*' is quite similar in this respect,
and that if '-fstack-protector*' is required to build the shared library
with libtool, then it will be necessary to ultimately link to this new
shared library.

libgvc.so.6 => /var/tmp/pkgsrc/graphics/graphviz/work/graphviz-2.38.0/lib/gvc/.libs/libgvc.so.6
libcgraph.so.6 => /opt/local/lib/libcgraph.so.6
libcdt.so.5 => /opt/local/lib/libcdt.so.5
libdl.so.1 => /lib/libdl.so.1
libltdl.so.7 => /opt/local/lib/libltdl.so.7
libxdot.so.4 => /opt/local/lib/libxdot.so.4
libpathplan.so.4 => /opt/local/lib/libpathplan.so.4
libexpat.so.1 => /opt/local/lib/libexpat.so.1
libpthread.so.1 => /lib/libpthread.so.1
librt.so.1 => /lib/librt.so.1
libsocket.so.1 => /lib/libsocket.so.1
libnsl.so.1 => /lib/libnsl.so.1
libz.so.1 => /opt/local/lib/libz.so.1
libm.so.2 => /lib/libm.so.2
libperl.so => /opt/local/lib/perl5/5.20.0/i386-solaris/CORE/libperl.so
libstdc++.so.6 => /opt/local/gcc47/i386-sun-solaris2.11/lib/./libstdc++.so.6
libgcc_s.so.1 => /opt/local/gcc47/i386-sun-solaris2.11/lib/./libgcc_s.so.1
libc.so.1 => /lib/libc.so.1
libssp.so.0 => /opt/local/gcc47/i386-sun-solaris2.11/lib/./libssp.so.0 <<<<<<<<<<<<<<<<<<<<<<<<<
libmp.so.2 => /lib/libmp.so.2
libmd.so.1 => /lib/libmd.so.1
[680] | 0| 0|FUNC |GLOB |0 |UNDEF |__stack_chk_fail
[47] | 83792| 29|FUNC |LOCL |2 |14 |__stack_chk_fail_local
[684] | 0| 0|OBJT |GLOB |0 |UNDEF |__stack_chk_guard

Even though on Solaris libssp_unshared.o is no longer needed, libssp.so
still is, and that is provided with the spec '-fstack-protector*'.

So I revived my initial work with "inherited_linker_flags" and attach
the complementary patch for testing prior to pushing patch-ac which is

# Linker flags that can not go in dependency_libs.
inherited_linker_flags=' -fstack-protector -pthread'

Observations anyone?

[Richard PALO]

Post by Richard PALO
There is one thing that has been bugging me for months is the fact that
(using my test case for graphviz) frequently the need for
'-fstack-protector' comes from a dependant library, such as perl or cups
or something.
That may be fine for programs, but for shared libraries, that can get
quite complicated.
In libtool terms, there is a variable in the .la file named
inherited_linker_flags that is added to the link command to the shared
library... I guess a reasonably good example of this is '-pthread'.
I believe that '-fstack-protector*' is quite similar in this respect,
and that if '-fstack-protector*' is required to build the shared library
with libtool, then it will be necessary to ultimately link to this new
shared library.

ldd -D libgv_perl.so
libgvc.so.6 =>
/var/tmp/pkgsrc/graphics/graphviz/work/graphviz-2.38.0/lib/gvc/.libs/libgvc.so.6
libcgraph.so.6 => /opt/local/lib/libcgraph.so.6
libcdt.so.5 => /opt/local/lib/libcdt.so.5
libdl.so.1 => /lib/libdl.so.1
libltdl.so.7 => /opt/local/lib/libltdl.so.7
libxdot.so.4 => /opt/local/lib/libxdot.so.4
libpathplan.so.4 => /opt/local/lib/libpathplan.so.4
libexpat.so.1 => /opt/local/lib/libexpat.so.1
libpthread.so.1 => /lib/libpthread.so.1
librt.so.1 => /lib/librt.so.1
libsocket.so.1 => /lib/libsocket.so.1
libnsl.so.1 => /lib/libnsl.so.1
libz.so.1 => /opt/local/lib/libz.so.1
libm.so.2 => /lib/libm.so.2
libperl.so =>
/opt/local/lib/perl5/5.20.0/i386-solaris/CORE/libperl.so
libstdc++.so.6 =>
/opt/local/gcc47/i386-sun-solaris2.11/lib/./libstdc++.so.6
libgcc_s.so.1 =>
/opt/local/gcc47/i386-sun-solaris2.11/lib/./libgcc_s.so.1
libc.so.1 => /lib/libc.so.1
libssp.so.0 =>
/opt/local/gcc47/i386-sun-solaris2.11/lib/./libssp.so.0
<<<<<<<<<<<<<<<<<<<<<<<<<
libmp.so.2 => /lib/libmp.so.2
libmd.so.1 => /lib/libmd.so.1
nm libgv_perl.so |grep __stack
[680] | 0| 0|FUNC |GLOB |0 |UNDEF
|__stack_chk_fail
[47] | 83792| 29|FUNC |LOCL |2 |14
|__stack_chk_fail_local
[684] | 0| 0|OBJT |GLOB |0 |UNDEF
|__stack_chk_guard

Even though on Solaris libssp_unshared.o is no longer needed, libssp.so
still is, and that is provided with the spec '-fstack-protector*'.
So I revived my initial work with "inherited_linker_flags" and attach
the complementary patch for testing prior to pushing patch-ac which is

# Linker flags that can not go in dependency_libs.
inherited_linker_flags=' -fstack-protector -pthread'

Observations anyone?

I'm closeting [again] this inherited_linker_flags work...
It is moot as any client program needing the ssp libs will have already
added necessarily a flavour of '-fstack-protector*'.

+1 to push the lt-refix-update.patch as is.

[Joerg Sonnenberger]

Post by Richard PALO
I believe that '-fstack-protector*' is quite similar in this
respect, and that if '-fstack-protector*' is required to build the
shared library with libtool, then it will be necessary to ultimately
link to this new shared library.

No, it isn't. Once the stupid static linkage issue is fixed, it becomes
only necessary when *statically* linking. E.g. if libperl.so was built
with -fstack-protector, something linking against libperl.so doesn't
have to.

Joerg