Discussion:
Optimising _check-wrkref
Jonathan Perkin
2014-09-02 13:32:41 UTC
Permalink
The current implementation of _check-wrkref iterates over a list of
directories and runs grep for each of them against every file listed
in a package, at least when PKG_DEVELOPER is set.

This can be pretty slow, and I'd like to change it so we use a single
egrep and a regexp instead. A diff to implement that is here:

http://us-east.manta.joyent.com/pkgsrc/public/patches/check-wrkref-opt.diff

Testing with lang/ruby21-base we go from this:

$ ptime bmake _check-wrkref
=> Checking for work-directory references in ruby212-base-2.1.2nb1

real 2:41.018259391
user 54.498007133
sys 3:46.198545074

to this:

$ ptime bmake _check-wrkref
=> Checking for work-directory references in ruby212-base-2.1.2nb1

real 1:43.642373148
user 48.184716342
sys 53.025063252

Users who set additional entries in CHECK_WRKREF should see scalable
performance improvements.

As usual I will push this through a full bulk build to ensure there
are no regressions, but wanted to post it here for comments first.

Cheers,
--
Jonathan Perkin - Joyent, Inc. - www.joyent.com
Jonathan Perkin
2014-09-02 14:00:16 UTC
Permalink
Post by Jonathan Perkin
The current implementation of _check-wrkref iterates over a list of
directories and runs grep for each of them against every file listed
in a package, at least when PKG_DEVELOPER is set.
This can be pretty slow, and I'd like to change it so we use a single
http://us-east.manta.joyent.com/pkgsrc/public/patches/check-wrkref-opt.diff
Joerg suggested using the :ts| operator instead, which results in this
much simpler diff:

http://us-east.manta.joyent.com/pkgsrc/public/patches/check-wrkref-opt-2.diff

Same performance improvement and functionality.
--
Jonathan Perkin - Joyent, Inc. - www.joyent.com
Loading...