Apache should never be a mandatory dependency

[Volkmar Seifert]

Hello everyone,

While building a few packages, I noticed that while some already have
learned that there are more web-servers out there than just apache,
some still pull in apache without leaving the user a choice.

As I am using nginx myself, instead of Apache, this is a really
annoying thing. Some packages have already adopted to this, and offer
option to leave apache out of the picture. Examples for this are

www/drupal
www/wordpress

An example of forcing the apache is

mail/roundcube

My suggestion to a general solution to this problem is to introduce
some kind of "global option", like this:

PKG_DEFAULT_OPTIONS= apache

for including apache (could be default), and

PKG_DEFAULT_OPTIONS= -apache

for excluding it. All packages that think they require a web-server
then should honor this.
It's kind of apache-specific, because apache seems to be the one server
that is generally the server of choice for all packages, opposite to
lighttpd, nginx and whatever other server is out there.

I do understand that a web-server should be included in these packages,
but I dislike being unable to tell the package "let this be my own
concern" or "Look, I am using this web-server, be satisfied with that".

What do you think?

Volkmar

[D'Arcy J.M. Cain]

On Fri, 16 May 2014 13:10:09 +0200

Post by Volkmar Seifert
While building a few packages, I noticed that while some already have
learned that there are more web-servers out there than just apache,
some still pull in apache without leaving the user a choice.

See my fix to www/dojo. Not only does it not need Apache, it doesn't
really need a web server at all. The files could be used on an NFS
mounted drive. The web server may not even run on that server. I
simply removed the dependency there.

Post by Volkmar Seifert
My suggestion to a general solution to this problem is to introduce
PKG_DEFAULT_OPTIONS= apache

Sounds like it could get more complicated with special cases. I think
that if someone is building roundcube or something like that then they
probably already have a web server where they need it. Unless it
requires a specific version why include the dependency at all? And if
it does require a specific version, it would probably requires a
specific web server anyway.

If a package generated CSV files, would we include a dependency on
LibreOffice?

[Volkmar Seifert]

Post by D'Arcy J.M. Cain

Post by Volkmar Seifert
My suggestion to a general solution to this problem is to introduce
PKG_DEFAULT_OPTIONS= apache

Sounds like it could get more complicated with special cases. I think
that if someone is building roundcube or something like that then they
probably already have a web server where they need it. Unless it
requires a specific version why include the dependency at all? And if
it does require a specific version, it would probably requires a
specific web server anyway.
If a package generated CSV files, would we include a dependency on
LibreOffice?

That's exactly what I am talking about.

Leaving out the dependencies to any kind of web-server completely would
be fine with me, because, yes, I do have webservers running on all
hosts on which I need them. I usually do this prior to installing
software like drupal, roundcube or whatever. As you already said, this
will probably be the general case anyway.

[Greg Troxel]

Post by Volkmar Seifert

Post by D'Arcy J.M. Cain

Post by Volkmar Seifert
My suggestion to a general solution to this problem is to introduce
PKG_DEFAULT_OPTIONS= apache

Sounds like it could get more complicated with special cases. I think
that if someone is building roundcube or something like that then they
probably already have a web server where they need it. Unless it
requires a specific version why include the dependency at all? And if
it does require a specific version, it would probably requires a
specific web server anyway.
If a package generated CSV files, would we include a dependency on
LibreOffice?

That's exactly what I am talking about.
Leaving out the dependencies to any kind of web-server completely would
be fine with me, because, yes, I do have webservers running on all
hosts on which I need them. I usually do this prior to installing
software like drupal, roundcube or whatever. As you already said, this
will probably be the general case anyway.

In general, options are problematic because while people compiling from
source can set them, they are not really options with respect to the
bulk build.

So if some kind of web server is a soft run-time dependency, then I
think it makes sense to just not include it as a dependency at all.

[OBATA Akio]

Post by Greg Troxel
In general, options are problematic because while people compiling from
source can set them, they are not really options with respect to the
bulk build.
So if some kind of web server is a soft run-time dependency, then I
think it makes sense to just not include it as a dependency at all.

Any reason APACHE_USER/APACHE_GROUP and NGINX_USER/NGINX_GROUP must differ?
Some packages depend on apache and set WWW_USER=APACHE_USER/WWW_GROUP=APACHE_GROUP,
then set permissions with WWW_USER/WWW_GROUP.

[Volkmar Seifert]

Post by OBATA Akio
Any reason APACHE_USER/APACHE_GROUP and NGINX_USER/NGINX_GROUP must
differ? Some packages depend on apache and set
WWW_USER=APACHE_USER/WWW_GROUP=APACHE_GROUP, then set permissions
with WWW_USER/WWW_GROUP.

I've asked myself the same thing, and actually set everything to
www:www manually where needed. That's a hack, ugly, and error-prone, so
not really an advice I'd give to anyone working in a larger scale
production environment.

Unifying that to one user (www:www is nicely application-independent
and task-related, I think) would be really great.

Volkmar

[Taylor R Campbell]

Date: Fri, 16 May 2014 22:43:31 +0900
From: "OBATA Akio" <***@lins.jp>

Any reason APACHE_USER/APACHE_GROUP and NGINX_USER/NGINX_GROUP must differ?
Some packages depend on apache and set WWW_USER=APACHE_USER/WWW_GROUP=APACHE_GROUP,
then set permissions with WWW_USER/WWW_GROUP.

I may want to have Apache and nginx both running simultaneously in
isolated privilege domains.

[Volkmar Seifert]

Post by OBATA Akio
Any reason APACHE_USER/APACHE_GROUP and NGINX_USER/NGINX_GROUP
must differ? Some packages depend on apache and set
WWW_USER=APACHE_USER/WWW_GROUP=APACHE_GROUP, then set permissions
with WWW_USER/WWW_GROUP.
I may want to have Apache and nginx both running simultaneously in
isolated privilege domains.

In that case, the user has a special requirement, and most likely the
knowledge to take care of setting the appropriate PKG_OPTIONS him- or
herself. It might be a good idea to keep the possibility of using
different users/groups, but for mainstream, having them use the same
user/group.
It might even be a possibility, to have a general "WWW_USER" and
"WWW_GROUP" parameter introduced, that will be used as a default, if
(APACHE|NGINX)_(USER|GROUP) is empty.

That way, everything is possible, while the packages in general will
use a unified user/group.

- Volkmar

[Volkmar Seifert]

Post by Volkmar Seifert

Post by Taylor R Campbell
I may want to have Apache and nginx both running simultaneously in
isolated privilege domains.

In that case, the user has a special requirement, and most likely the
knowledge to take care of setting the appropriate PKG_OPTIONS him- or
herself. It might be a good idea to keep the possibility of using
different users/groups, but for mainstream, having them use the same
user/group.
It might even be a possibility, to have a general "WWW_USER" and
"WWW_GROUP" parameter introduced, that will be used as a default, if
(APACHE|NGINX)_(USER|GROUP) is empty.

I just grepped a bit through the pkgsrc-tree, and found this:

WWW_USER is used by the following packages:

mail/roundcube
security/base
www/drupal6
www/drupal7
www/fengoffice
www/php-ja-wordpress
www/wordpress

All of them use APACHE_USER to initialize WWW_USER, though some seem to
use ROOT_USER under certain conditions, too. Which is IMHO even worse,
though admittedly it doesn't mean a new user is created.

These are the packages that use APACHE_USER, be that for
initialization or directly (the packages listed above are included in
this list):

comms/op_panel
databases/phpmyadmin
devel/bugzilla
devel/bugzilla3
devel/mantis
devel/rt3
devel/rt4
devel/startbug1
mail/dspam
mail/imp
mail/ingo
mail/mimp
mail/roundcube
mail/squirrelmail
mail/turba
misc/php-orangehrm
security/base
security/zoneminder
time/kronolith
www/ap2-suphp
www/apache
www/apache2
www/apache22
www/apache24
www/awstats
www/contao
www/contao211
www/contao32
www/davical
www/drupal6
www/drupal7
www/fengoffice
www/gallery2
www/horde
www/mediawiki
www/moodle
www/ocsigen
www/php-basercms
www/php-concrete5
www/php-fpm
www/php-ja-wordpress
www/php-owncloud
www/php-piwigo
www/php-soycms
www/php-sugarcrm
www/php-tiki6
www/php-tt-rss
www/phraseanet
www/sencha-sns
www/wordpress

When we are trying to get away from a forced apache, which actually is
what this thread is all about, this dependency on APACHE_(USER|GROUP) is
something that should be changed, too.
May be by making "WWW_(USER|GROUP)" a global variable, set through the
mk/* config-files and adjustable per user through mk.conf .

Those packages that already use this variable simply need to remove the
initialization, and the other packages have to be adjusted
appropriately - most basic by replacing APACHE_(USER|GROUP) with WWW_
(USER|GROUP).

- Volkmar

[Joerg Sonnenberger]

[snip]

I don't think any of those dependencies are really justified.
Part of the problem where is trying to support mod_php -- sane
deployments using FastCGI certainly want to run the application as a
separate user and don't need all this mess.

Joerg

[Volkmar Seifert]

Post by Joerg Sonnenberger
I don't think any of those dependencies are really justified.

That's one reason why I brought this to discussion :)
I am very glad to see we are in agreement to a large degree.

Now we need to find a proper solution and implement it.

Post by Joerg Sonnenberger
Part of the problem where is trying to support mod_php -- sane
deployments using FastCGI certainly want to run the application as a
separate user and don't need all this mess.

Well, if support for mod_php is required, how about implementing either
an option or a direct check if mod_php is installed?
Yes I know, this would have no effect on bulk-builds, so what do we do
with them? I mean, bulk-builds shouldn't worry about what the effective
users are on the system the packages will finally be installed. That's
an install-time issue, so we should have those checks run at
install-time, instead of putting them into the build.

Volkmar

[Joerg Sonnenberger]

Post by OBATA Akio
Any reason APACHE_USER/APACHE_GROUP and NGINX_USER/NGINX_GROUP must differ?
Some packages depend on apache and set WWW_USER=APACHE_USER/WWW_GROUP=APACHE_GROUP,
then set permissions with WWW_USER/WWW_GROUP.

I think we should consolidate a WWW_GROUP, but not a user. It often
makes sense to limit accessibility for the group, e.g. for FastCGI
sockets. Nothing should run as the unprivileged web server user though.

Joerg

[Volkmar Seifert]

Post by Joerg Sonnenberger

Post by OBATA Akio
Any reason APACHE_USER/APACHE_GROUP and NGINX_USER/NGINX_GROUP must
differ? Some packages depend on apache and set
WWW_USER=APACHE_USER/WWW_GROUP=APACHE_GROUP, then set permissions
with WWW_USER/WWW_GROUP.

I think we should consolidate a WWW_GROUP, but not a user. It often
makes sense to limit accessibility for the group, e.g. for FastCGI
sockets. Nothing should run as the unprivileged web server user though.

Fair enough, though in order to prevent uncontrolled growth of users,
consolidating a WWW_USER isn't a bad idea, either.
Especially when it comes to web-applications, I wouldn't want them to
be owned by "root". The WWW_USER would be the expected user and safer
then root.
See my other emails for elaboration on a suggestion to a solution while
maintaining the possibility of customization by the user.

- Volkmar

[David Holland]

Post by Volkmar Seifert

Post by Joerg Sonnenberger

Post by OBATA Akio
Any reason APACHE_USER/APACHE_GROUP and NGINX_USER/NGINX_GROUP must
differ? Some packages depend on apache and set
WWW_USER=APACHE_USER/WWW_GROUP=APACHE_GROUP, then set permissions
with WWW_USER/WWW_GROUP.

I think we should consolidate a WWW_GROUP, but not a user. It often
makes sense to limit accessibility for the group, e.g. for FastCGI
sockets. Nothing should run as the unprivileged web server user though.

Fair enough, though in order to prevent uncontrolled growth of users,
consolidating a WWW_USER isn't a bad idea, either.
Especially when it comes to web-applications, I wouldn't want them to
be owned by "root". The WWW_USER would be the expected user and safer
then root.
See my other emails for elaboration on a suggestion to a solution while
maintaining the possibility of customization by the user.

So we should have a WWWAPPS_USER, and make sure that the web servers
are set up by default to switch to it when needed? There are programs
that need read-write files and directories and expect to own them;
having these files owned by the apache/nginx user (and the apps
running as that user) isn't a great idea.

Miscellaneous readonly files should be owned by root though.

[Volkmar Seifert]

Post by David Holland
So we should have a WWWAPPS_USER, and make sure that the web servers
are set up by default to switch to it when needed? There are programs
that need read-write files and directories and expect to own them;
having these files owned by the apache/nginx user (and the apps
running as that user) isn't a great idea.
Miscellaneous readonly files should be owned by root though.

Well, drupal and wordpress are probably good examples for what you
describe.
They have parts that can be owned by root, but read-only accessible to
the web-server, and parts that need to be read-write accessible to the
web-server or whatever runs PHP (in case of drupal/wordpress).

As far as I understand pkgsrc, it should be no real problem to have the
correct mode and ownership set, it's just a matter the maintainer has
to implement properly.

As ROOT_USER is available already, something like your WWWAPPS_USER
could be introduced for those special parts.
However, as Joerg already pointed out in a differen part of the thread,
this only works properly as long as something else but apache is used.
mod_perl, mod_php, mod_whatever run as the same user as the apache -
naturally, as it's the same process. But, of course, in those cases,
WWWAPPS_USER can be set to APACHE_USER. You will still have the
separation of ROOT_USER owned files vs WWWAPPS_USER, but WWWAPPS_USER
will be the same as WWW_USER, because that's APACHE_USER.

Volkmar

[Volkmar Seifert]

Post by Greg Troxel
In general, options are problematic because while people compiling
from source can set them, they are not really options with respect to
the bulk build.

I agree. Which is why my intention was to make the suggestion I made in
order to stir up a discussion to find the optimal solution.

Post by Greg Troxel
So if some kind of web server is a soft run-time dependency, then I
think it makes sense to just not include it as a dependency at all.

Indeed. As long as not required for building, I'd strongly suggest to
remove all those dependencies, be that to a web-server, or to a
programming-language meant to somehow run in/behind a web-server (php,
ruby-on-rails, etc). Though with languages, build-dependencies -are-
more likely, so these should be included in a way that web-servers are
not pulled into the build unsolicited, and may even end up as a
dependency when installing the packages.

- Volkmar

[David Holland]

Post by Volkmar Seifert

Post by Greg Troxel
So if some kind of web server is a soft run-time dependency, then I
think it makes sense to just not include it as a dependency at all.

Indeed. As long as not required for building, I'd strongly suggest to
remove all those dependencies, be that to a web-server, or to a
programming-language meant to somehow run in/behind a web-server (php,
ruby-on-rails, etc). Though with languages, build-dependencies -are-
more likely, so these should be included in a way that web-servers are
not pulled into the build unsolicited, and may even end up as a
dependency when installing the packages.

I do think it's a good idea to indicate somehow that some packages
require a web server. It isn't always obvious from DESCR; also,
making such requirements implicit makes turnkey installs harder.

The standard thing we do in this kind of situation is to make
everything that requires a web server include mk/webserver.mk or
something like that (it wouldn't probably be a bl3 file) and this
figures out what webserver you're using based on config vars and
depends on that.

Not sure if that's really what we want here though.

[Jonathan Perkin]

Post by David Holland

Post by Volkmar Seifert

Post by Greg Troxel
So if some kind of web server is a soft run-time dependency, then I
think it makes sense to just not include it as a dependency at all.

Indeed. As long as not required for building, I'd strongly suggest to
remove all those dependencies, be that to a web-server, or to a
programming-language meant to somehow run in/behind a web-server (php,
ruby-on-rails, etc). Though with languages, build-dependencies -are-
more likely, so these should be included in a way that web-servers are
not pulled into the build unsolicited, and may even end up as a
dependency when installing the packages.

I do think it's a good idea to indicate somehow that some packages
require a web server. It isn't always obvious from DESCR; also,
making such requirements implicit makes turnkey installs harder.
The standard thing we do in this kind of situation is to make
everything that requires a web server include mk/webserver.mk or
something like that (it wouldn't probably be a bl3 file) and this
figures out what webserver you're using based on config vars and
depends on that.
Not sure if that's really what we want here though.

It definitely is not something we, as a provider of binary packages,
would ever want.

[David Holland]

Post by Jonathan Perkin

Post by David Holland
The standard thing we do in this kind of situation is to make
everything that requires a web server include mk/webserver.mk or
something like that (it wouldn't probably be a bl3 file) and this
figures out what webserver you're using based on config vars and
depends on that.
Not sure if that's really what we want here though.

It definitely is not something we, as a provider of binary packages,
would ever want.

Right, and this isn't the only time that's come up either, not that I
can remember any others offhand.

I think we ought to have package metadata for providing and depending
on abstractions.

[Volkmar Seifert]

Post by David Holland
Right, and this isn't the only time that's come up either, not that I
can remember any others offhand.
I think we ought to have package metadata for providing and depending
on abstractions.

I actually don't really now any pkgsrc-based "turnkey-installation".
That's usually something found with Linux, and there usually Apache is
indeed the web-server of choice.

I think we should pursue a practical approach that has high probability
of being used, instead of hunting special cases that -may- be -nice-.

While talking to someone else about this whole topic, the term
"bikeshedding" came up. When I take a look at this discussion, there
has indeed been some of that, because a lot of energy suddenly went
into the question of users and groups. It is not an important
discussion, and probably something that is to be thought about while
solving this issue, but the discussion got a little bit side-tracked.

What pkgsrc needs now, regarding the apache-dependency-issue, is a
decision that helps to improve it.

One good way is leaving out the dependencies to web-servers, because
honestly, any web-server can be used, no matter what web-application we
are talking about. Some will require more work then others, but that's
actually the user's problem. If I, as a user, want to use nginx, then
so be it, but then -I- will have to live with that decision and solve
all puzzles that would have been easier with apache.
As long as there are packages with dependencies to apache or may be
some apache module (hello meta-pkgs/php5[3-5]-extensions ...), apache
will always be pulled into the build, and that is definitely not in
every user's interest.

It might pose a problem with said turnkey-installations, but I believe
that these kinds of installation are not the usual way pkgsrc is used.
I think it's safe to defer the solution of this to those who use pkgsrc
for realizing it - which means outside of pkgsrc. Who ever provides
turn-key installations will also have the knowledge of how to realize
it.
Whoever needs turnkey-installations because he has no clue of what's
necessary to install a webserver will probably use Ubuntu anyway.

If we really want packages to be able to state "You know, I need a
web-server", there's always the post-install message and DESCR. Yes, it
has to be maintained, but meta-data has to be, too. And opposite to
meta-data, post-install-messages and DESCR-files do already exist.
There doesn't need to be a lengthy process of developing a proper
structure etc., with endless discussions, just to realize a simple "I
need a webserver"-message. And honestly - if someone installs something
like drupal, wordpress, squirrelmail, roundcube or whatever - if that
one doesn't know it requires a webserver, too, then for all that's
sacred, I can't help that person.
Plus - all these packages need PHP, not a webserver. So I agree that it
would be ok to let them pull in PHP. I don't agree with having them
pulling in -how- PHP should be executed, however, that's the user's
decision.
So, as was already suggested a couple of times, please, remove the
dependencies to the web-servers or their modules.
If we can tackle those USER/GROUP issues at the same time, all the
better.

If no one feels inclined to do that, I'd happily jump in and take the
task. I just don't have commit-rights.

- Volkmar

[Alistair Crooks]

Post by Volkmar Seifert

Post by David Holland
Right, and this isn't the only time that's come up either, not that I
can remember any others offhand.
I think we ought to have package metadata for providing and depending
on abstractions.

I actually don't really now any pkgsrc-based "turnkey-installation".
That's usually something found with Linux, and there usually Apache is
indeed the web-server of choice.
I think we should pursue a practical approach that has high probability
of being used, instead of hunting special cases that -may- be -nice-.
While talking to someone else about this whole topic, the term
"bikeshedding" came up. When I take a look at this discussion, there
has indeed been some of that, because a lot of energy suddenly went
into the question of users and groups. It is not an important
discussion, and probably something that is to be thought about while
solving this issue, but the discussion got a little bit side-tracked.
What pkgsrc needs now, regarding the apache-dependency-issue, is a
decision that helps to improve it.

Aren't we getting into ALTERNATIVES territory here?

What I would suggest is a "generic" style of package, let's call it

pkgsrc/metapackages/webserver

It's not just webservers this can be used for - I can think of editors
(and, within editors, sub classes of vi, emacs and other packages),
browsers, versions of scheme, common lisp, shells, etc.

Anyway, to get back to the worked example - this will make sure a
webserver is installed. It can take preferences from a setting in
/etc/mk.conf (on a per-platform basis), of the form

GENERIC.webserver= apache22

If no preference is given at package installation time, it will select
the default one for you.

So the line in the webserver pkgsrc entry Makefile would look like:

.include "../../mk/bsd.prefs.mk"

.if defined(GENERIC.webserver)
_WEBSERVER=${GENERIC.webserver}
.else
_WEBSERVER=${_DEFAULT_GENERIC_WEBSERVER}
.endif

DEPENDS+= {bozohttpd>=20140201,apache22>=2.2.27,apache24>=2.4.9,nginx>=1.6.0}:../../www/${DEFAULT_GENERIC_WEBSERVER}

Seems simple to me, and, in lieu of any other mechanism within pkgsrc,
should work. I'm less chuffed with the version numbers there, but I
wonder if there's some way to generalise them.

Thoughts?

Thanks,
Alistair

[Jonathan Perkin]

Post by Alistair Crooks
What I would suggest is a "generic" style of package, let's call it
pkgsrc/metapackages/webserver
It's not just webservers this can be used for - I can think of editors
(and, within editors, sub classes of vi, emacs and other packages),
browsers, versions of scheme, common lisp, shells, etc.
Anyway, to get back to the worked example - this will make sure a
webserver is installed. It can take preferences from a setting in
/etc/mk.conf (on a per-platform basis), of the form
GENERIC.webserver= apache22
If no preference is given at package installation time, it will select
the default one for you.
.include "../../mk/bsd.prefs.mk"
.if defined(GENERIC.webserver)
_WEBSERVER=${GENERIC.webserver}
.else
_WEBSERVER=${_DEFAULT_GENERIC_WEBSERVER}
.endif
DEPENDS+= {bozohttpd>=20140201,apache22>=2.2.27,apache24>=2.4.9,nginx>=1.6.0}:../../www/${DEFAULT_GENERIC_WEBSERVER}
Seems simple to me, and, in lieu of any other mechanism within pkgsrc,
should work. I'm less chuffed with the version numbers there, but I
wonder if there's some way to generalise them.

We use this type of mechanism at Joyent to support all the various
MySQL forks we've added. It's ... not great.

Users regularly get confused because some random package happens to
need to pull in a MySQL client library, and due to the DEPENDS
matching will pull in the first match, which is often not what they
want to use. When they come later to install their preferred MySQL
server, they can't due to conflicts, and have to unwind the
installation back so that they start with their preferred version.

This is somewhat unavoidable due to hard dependencies on
libmysqlclient, which changes version between forks, so currently is a
necessity we have to put up with (and we produce specific images to
make it easier for users to pick their preferred option).

When the dependency is soft, like with a web server, then I would
definitely not want any of this getting in the way. If people really
want it regardless, that's fine, but please can we have a global knob
to turn it off completely to save us having more diffs.

We're working to remove dependencies to make packages more flexible,
rather than the other way around.

Thanks,

[Volkmar Seifert]

On Sun, 18 May 2014 08:52:13 +0100

Post by Jonathan Perkin

Post by Alistair Crooks
What I would suggest is a "generic" style of package, let's call it
pkgsrc/metapackages/webserver
It's not just webservers this can be used for - I can think of
editors (and, within editors, sub classes of vi, emacs and other
packages), browsers, versions of scheme, common lisp, shells, etc.
Anyway, to get back to the worked example - this will make sure a
webserver is installed. It can take preferences from a setting in
/etc/mk.conf (on a per-platform basis), of the form
GENERIC.webserver= apache22
If no preference is given at package installation time, it will
select the default one for you.

This is certainly one possible way. However, it won't reduce
dependencies, and over the course of this discussion, it became clear
to me that a shifting of dependencies, though solving my personal
issue, would not really simplify pkgsrc - neither for maintainers nor
for users. This being said, I now do not think it's wise to have the
user define which web-browser should be used, but that it's much
better to remove such dependencies at all.

When someone is installing e.g. PHP, it's most likely he will install a
webserver, too - but not even necessarily on the same machine. With
tools like FastCGI, you can have a scaled environment, with web-servers
on several different hosts, and PHP on other hosts. Software like
Roundcube, Drupal, etc. will require PHP on the machine they are
installed, but not a webserver. If we have a setup like the above, one
webserver would always be installed, and that is simply not necessary.

We shouldn't treat our users to be of spoon-feed-age, and assume that
they do possess a certain knowledge. If they don't, they have to either
learn by reading manuals and asking questions or they better don't run
their own server.

Therefore, like it's being said by others several times before, I'd vote
for a removal of dependencies, instead of shifting them.

Post by Jonathan Perkin
Users regularly get confused because some random package happens to
need to pull in a MySQL client library, and due to the DEPENDS
matching will pull in the first match, which is often not what they
want to use. When they come later to install their preferred MySQL
server, they can't due to conflicts, and have to unwind the
installation back so that they start with their preferred version.

Yes, that is something we already have in pkgsrc right now, too.
And not just with mysql, but as this topic is about apache - with
apache, which is pulled in even though not required / wanted.

That's why I started this discussion, to find a solution for this
problem. If we can find a solution for similar problems with other
packages (Alistair mentioned editors already) all the better.

Post by Jonathan Perkin
When the dependency is soft, like with a web server, then I would
definitely not want any of this getting in the way. If people really
want it regardless, that's fine, but please can we have a global knob
to turn it off completely to save us having more diffs.
We're working to remove dependencies to make packages more flexible,
rather than the other way around.

Exactly.

- Volkmar

[Alistair Crooks]

Post by Jonathan Perkin

Post by Alistair Crooks
What I would suggest is a "generic" style of package, let's call it
pkgsrc/metapackages/webserver
It's not just webservers this can be used for - I can think of editors
(and, within editors, sub classes of vi, emacs and other packages),
browsers, versions of scheme, common lisp, shells, etc.
Anyway, to get back to the worked example - this will make sure a
webserver is installed. It can take preferences from a setting in
/etc/mk.conf (on a per-platform basis), of the form
GENERIC.webserver= apache22
If no preference is given at package installation time, it will select
the default one for you.
.include "../../mk/bsd.prefs.mk"
.if defined(GENERIC.webserver)
_WEBSERVER=${GENERIC.webserver}
.else
_WEBSERVER=${_DEFAULT_GENERIC_WEBSERVER}
.endif
DEPENDS+= {bozohttpd>=20140201,apache22>=2.2.27,apache24>=2.4.9,nginx>=1.6.0}:../../www/${DEFAULT_GENERIC_WEBSERVER}
Seems simple to me, and, in lieu of any other mechanism within pkgsrc,
should work. I'm less chuffed with the version numbers there, but I
wonder if there's some way to generalise them.

We use this type of mechanism at Joyent to support all the various
MySQL forks we've added. It's ... not great.
Users regularly get confused because some random package happens to
need to pull in a MySQL client library, and due to the DEPENDS
matching will pull in the first match, which is often not what they
want to use. When they come later to install their preferred MySQL
server, they can't due to conflicts, and have to unwind the
installation back so that they start with their preferred version.
This is somewhat unavoidable due to hard dependencies on
libmysqlclient, which changes version between forks, so currently is a
necessity we have to put up with (and we produce specific images to
make it easier for users to pick their preferred option).
When the dependency is soft, like with a web server, then I would
definitely not want any of this getting in the way. If people really
want it regardless, that's fine, but please can we have a global knob
to turn it off completely to save us having more diffs.
We're working to remove dependencies to make packages more flexible,
rather than the other way around.

[Previous message left for context, as I've left this far too long to
reply to - agc]

Thanks for the insight - I can see that this would be a problem with
a real package with a library in it, but for indirecting through a
meta-package, I'm not sure the case is quite so similar.

And so onto plan B.

Second suggestion, then, is to create some form of lightweight
package for the indirection, like, say, a site-specific symbolic
link which can be made (again through a mk.conf setting), and the
"make package" command. e.g.

in /etc/mk.conf

# strict version numbers would be a lose here
PKG_GENERIC.webserver= apache

and use the "webserver" keyword in nginx, apache, bozohttpd, lighthttpd,
et al pkgsrc Makefiles:

PKG_GENERIC.category= webserver

which would create a webserver-0.0.tgz binary package when the
desired binary package was built.

Yes, I know this is subclassing the existing categories, but it seems
useful to me to have such a mechanism. Depending on a generic and
abstract webserver-0.0 package would trigger the build mechanism
(by checking whther PKG_GENERIC.${PKG_GENERIC.category} was defined).

It's possible to change the desired specific package by using a
different value in /etc/mk.conf. Failure to flush out the old one
would have the same results.

Whatever way this is done, it needs another layer of indirection (with
a tip of the hat to Butler Lampson/David Wheeler). I'm sure people
can improve on my rather handwavy explanations above.

Regards,
Alistair

[Greg Troxel]

Post by Alistair Crooks

[packages dpeending on web servers]

Aren't we getting into ALTERNATIVES territory here?
What I would suggest is a "generic" style of package, let's call it
pkgsrc/metapackages/webserver
It's not just webservers this can be used for - I can think of editors
(and, within editors, sub classes of vi, emacs and other packages),
browsers, versions of scheme, common lisp, shells, etc.

I think the key point is how to enable as much as we can with unmodified
binary packages (no options). Within that, I suspect there are several
flavors of dependecies.

One situation seems pretty common, where a package does not actually
depend at build time on a web server, but needs some web server to run.
For those, it seems best to not have a dependency. If that's not going
to be obvious, it can be put in DESCR.

The other situation, where one really needs one of a class of things,
and can't actually build/install a package without one of them, needs
explicit support.


For the first situation, resolved by just omitting the dependency, one
can also think of inverting the dependency tree, so rather than have foo
depend on apache (when really it's just that sensible use of foo needs
some web server), foo could have no web server dependency, and then have
packages foo-apache, foo-nginx, foo-thttpd, etc. The problem is not
conceptual so much as it's annoying to have -- in the source tree --
many metapackages with no real content. A possible solution would be
to have a more compact representation where these packages could be
enumerated, essentially a metapackage generation language. Then,
full binary builds would have them. I'm not sure this is worth it, but
thought it was worth mentioning.