Optimising check-shlibs-elf.awk

[Jonathan Perkin]

I noticed that x11/kde-workspace4 was taking an extraordinarily long
time to complete the _check-shlibs phase so took a look at it.

This package is somewhat unusual in the number of files it tests
(298), due to the considerable number of small .so's it builds, so is
a good candidate for this test.

A baseline run on a fast SmartOS machine takes:

$ ptime bmake _check-shlibs
=> Checking for missing run-time search paths in kde-workspace4-4.11.5nb5

real 7:55.251878017
user 2:08.013799404
sys 5:14.145580838

With the corresponding DTrace for number of exec calls:

$ dtrace -n 'syscall::exece:return { @num[execname] = count(); }'
dtrace: description 'syscall::exece:return ' matched 1 probe
[...]
greadelf 298
pkg_info 5809
ksh93 95612

The reason for the high number of shell calls is that the awk script
checks for the existence of every possible shared library until it
gets a match, but repeats this for every file that is checked. There
are also corresponding pkg_info calls to find which package the
library belongs to.

By adding a couple of caches (one for the "test -f" checks, one for
the pkg_info results), we can significantly reduce the runtime:

$ ptime bmake _check-shlibs
=> Checking for missing run-time search paths in kde-workspace4-4.11.5nb5

real 18.503489661
user 6.115494568
sys 11.551809938

$ dtrace -n 'syscall::exece:return { @num[execname] = count(); }'
dtrace: description 'syscall::exece:return ' matched 1 probe
[...]
pkg_info 114
greadelf 298
ksh93 3028

The diff is here:

http://us-east.manta.joyent.com/pkgsrc/public/patches/check-shlibs-elf.diff

Comments and testing on other systems welcome. As always I will push this
through full bulk builds for testing on SmartOS.

[Joerg Sonnenberger]

Post by Jonathan Perkin
http://us-east.manta.joyent.com/pkgsrc/public/patches/check-shlibs-elf.diff

LGTM.

Joerg