Jonathan Perkin
2014-09-10 14:33:08 UTC
This bulk build is of particular note, as it is the first performed
using the cwrappers - joerg's GSOC 2007 project to replace the
shell-based wrappers we currently use with C implementations.
This branch is still under development to bring it up to parity with
the current wrappers, but already it is showing significant
performance improvements, primarily on platforms which have expensive
fork() (SunOS and OSX at least).
The full from-scratch bulk build (11,042 successful package builds
plus 1,140 attempted builds which failed) took 5hrs 2 minutes from
first to last package. Whilst the package count is down a little on
pristine builds, this is still around 4 hours faster than an upstream
build.
Some individual package build time improvements:
+-------------+-----------------+--------------+---------+
| Package | Legacy wrappers | C wrappers | Speedup |
+-------------+-----------------+--------------+---------+
| wireshark | 3376 seconds | 1098 seconds | 3.07x |
| webkit1-gtk | 11684 seconds | 4622 seconds | 2.52x |
| qt4-libs | 11866 seconds | 5134 seconds | 2.31x |
| xulrunner24 | 10574 seconds | 5058 seconds | 2.09x |
| ghc6 | 2026 seconds | 1328 seconds | 1.52x |
+-------------+-----------------+--------------+---------+
In testing on OSX I've seen even larger performance increases, with
libsodium around 3.5x faster at various -j levels as one example.
I'd like to get this in prior to 2014Q3 so that people can test, as
while it is still under development it should be a nop unless
USE_CWRAPPERS=yes is set. I'll send mail about that separately if I
feel it's ready before the freeze.
using the cwrappers - joerg's GSOC 2007 project to replace the
shell-based wrappers we currently use with C implementations.
This branch is still under development to bring it up to parity with
the current wrappers, but already it is showing significant
performance improvements, primarily on platforms which have expensive
fork() (SunOS and OSX at least).
The full from-scratch bulk build (11,042 successful package builds
plus 1,140 attempted builds which failed) took 5hrs 2 minutes from
first to last package. Whilst the package count is down a little on
pristine builds, this is still around 4 hours faster than an upstream
build.
Some individual package build time improvements:
+-------------+-----------------+--------------+---------+
| Package | Legacy wrappers | C wrappers | Speedup |
+-------------+-----------------+--------------+---------+
| wireshark | 3376 seconds | 1098 seconds | 3.07x |
| webkit1-gtk | 11684 seconds | 4622 seconds | 2.52x |
| qt4-libs | 11866 seconds | 5134 seconds | 2.31x |
| xulrunner24 | 10574 seconds | 5058 seconds | 2.09x |
| ghc6 | 2026 seconds | 1328 seconds | 1.52x |
+-------------+-----------------+--------------+---------+
In testing on OSX I've seen even larger performance increases, with
libsodium around 3.5x faster at various -j levels as one example.
I'd like to get this in prior to 2014Q3 so that people can test, as
while it is still under development it should be a nop unless
USE_CWRAPPERS=yes is set. I'll send mail about that separately if I
feel it's ready before the freeze.
--
Jonathan Perkin - Joyent, Inc. - www.joyent.com
Jonathan Perkin - Joyent, Inc. - www.joyent.com