Discussion:
Purpose of mk/find-prefix.mk
J. Lewis Muir
2014-10-17 17:20:50 UTC
Permalink
Hello, pkgsrc experts!

I've noticed some packages (e.g. editors/OmegaT) use mk/find-prefix.mk
to find the installation prefix of a particular package (e.g.
devel/apache-ant) while others (e.g. devel/opengrok) just assume an
installation prefix of a particular package (e.g. devel/apache-ant) to
be "${PREFIX}". What is the reason for using mk/find-prefix.mk instead
of "${PREFIX}"?

I might guess the following reason: if a package dependency, let's say
devel/apache-ant, were satisfied as a "built-in" package, that package
would reside outside of the pkgsrc PREFIX directory and therefore
"${PREFIX}/bin/ant" wouldn't work. Is this the main reason? Are there
other reasons?

Thank you!

Lewis
OBATA Akio
2014-10-18 01:57:53 UTC
Permalink
Hi,
Post by J. Lewis Muir
Hello, pkgsrc experts!
I've noticed some packages (e.g. editors/OmegaT) use mk/find-prefix.mk
to find the installation prefix of a particular package (e.g.
devel/apache-ant) while others (e.g. devel/opengrok) just assume an
installation prefix of a particular package (e.g. devel/apache-ant) to
be "${PREFIX}". What is the reason for using mk/find-prefix.mk instead
of "${PREFIX}"?
Use mk/find-prefix.mk, it is the right solution.

PREFIX is the installation prefix of currently handling package,
it may not same as other packages.

LOCALBASE is better than PREFIX to refer prefix of other packages,
because it is the default variable.
Post by J. Lewis Muir
I might guess the following reason: if a package dependency, let's say
devel/apache-ant, were satisfied as a "built-in" package, that package
would reside outside of the pkgsrc PREFIX directory and therefore
"${PREFIX}/bin/ant" wouldn't work. Is this the main reason? Are there
other reasons?
mk/find-prefix.mk itself dose not handle built-in package,
but used by pkgsrc tools framework.
--
OBATA Akio / ***@lins.jp
Joerg Sonnenberger
2014-10-18 02:11:24 UTC
Permalink
Post by J. Lewis Muir
I've noticed some packages (e.g. editors/OmegaT) use mk/find-prefix.mk
to find the installation prefix of a particular package (e.g.
devel/apache-ant) while others (e.g. devel/opengrok) just assume an
installation prefix of a particular package (e.g. devel/apache-ant) to
be "${PREFIX}". What is the reason for using mk/find-prefix.mk instead
of "${PREFIX}"?
Mostly historical, as pkgviews would effectively give a different PREFIX
for each installed package.

Joerg
Alistair Crooks
2014-10-18 04:59:58 UTC
Permalink
Post by Joerg Sonnenberger
Post by J. Lewis Muir
I've noticed some packages (e.g. editors/OmegaT) use mk/find-prefix.mk
to find the installation prefix of a particular package (e.g.
devel/apache-ant) while others (e.g. devel/opengrok) just assume an
installation prefix of a particular package (e.g. devel/apache-ant) to
be "${PREFIX}". What is the reason for using mk/find-prefix.mk instead
of "${PREFIX}"?
Mostly historical, as pkgviews would effectively give a different PREFIX
for each installed package.
Even more historically...

At the start, X11 packages were installed into X11BASE, and others
into LOCALBASE (the names come from FreeBSD's ports). PREFIX was set
to whichever location (X11BASE or LOCALBASE) was pertinent. Over the
first few years we moved things so that everything installed into
LOCALBASE.

Regards,
Alistair
J. Lewis Muir
2014-10-20 16:09:28 UTC
Permalink
Post by Alistair Crooks
Even more historically...
At the start, X11 packages were installed into X11BASE, and others
into LOCALBASE (the names come from FreeBSD's ports). PREFIX was set
to whichever location (X11BASE or LOCALBASE) was pertinent. Over the
first few years we moved things so that everything installed into
LOCALBASE.
That's interesting because the pkgsrc developer's guide still talks
about X11BASE (and X11PREFIX) in the section titled "Program location":

https://www.netbsd.org/docs/pkgsrc/build.html#build.prefix

Does the developer's guide need to be updated?

Regards,

Lewis
J. Lewis Muir
2014-10-20 16:08:44 UTC
Permalink
Thank you very much to all who replied!

My goal in asking my original question was to ensure I write my package
in the correct way. I thought I could determine this based on the
answer to my initial question, but now I'm still wondering. It seems
like the answers conflict a little, or I'm misunderstanding (quite
possible).

So, PREFIX is the right variable to use when referring to the package's
own installation path. I think all the answers agree with this.

But now I start to become unclear on the best approach for the rest of
the use cases.

I'm hearing that LOCALBASE is the right variable to use at build-time if
constructing "-I" or "-L" arguments to the compiler, or, more generally,
when referring to the prefix of other packages.

On the other hand, some answers say mk/find-prefix.mk (or EVAL_PREFIX)
is the right thing to be using for determining the prefix of a
build-time dependency. Why is mk/find-prefix.mk better than LOCALBASE?

Is LOCALBASE the right variable to use at *run-time* if referring to
the prefix of another package? For example, say I want to embed in a
program provided by my package the absolute path of a program on which
my package has a run-time dependency. Is LOCALBASE the right choice?

I'm still wondering about the best way to invoke a build-dependency
program during the build phase of my package. In particular, I'm
concerned about the case where the build-dependency is satisfied by a
built-in. I think a concrete example is the easiest way to discuss
this.

Let's say I have a build dependency on apache-ant because I want to
use the ant program it provides to build some Java source files for my
package. I would specify my build dependency like this:

BUILD_DEPENDS+= apache-ant-[0-9]*:../../devel/apache-ant

Then, I would specify a do-build target like this:

FIND_PREFIX:= ANT_PREFIX=apache-ant
.include "../../mk/find-prefix.mk"

do-build:
cd ${WORKSRC} && "${ANT_PREFIX}/bin/ant"

Why is that better than "${PREFIX}/bin/ant" or "${LOCALBASE}/bin/ant"?

Now, let's go one step further and pretend that devel/apache-ant is
satisfied as a built-in (i.e. my base system comes with apache-ant by
default and devel/apache-ant has a builtin.mk). In this case, the
ant program might be at /usr/bin/ant, or some other path *not* under
the pkgsrc LOCALBASE. Will mk/find-prefix.mk work in this case? I'm
worried that it won't. This makes me wonder if a bare invocation of
"ant" is more appropriate, like this:

do-build:
cd ${WORKSRC} && ant

This assumes pkgsrc sets up the PATH environment variable (or wrappers?)
to make my do-build ant invocation find the correct ant program (from
pkgsrc or from the (built-in) base system). Is this valid? If so, is
this actually better than any absolute path invocation because it works
regardless of whether the program is from a pkgsrc package or the base
system (built-in)?

Thank you!

Lewis
Joerg Sonnenberger
2014-10-20 16:15:38 UTC
Permalink
Post by J. Lewis Muir
I'm hearing that LOCALBASE is the right variable to use at build-time if
constructing "-I" or "-L" arguments to the compiler, or, more generally,
when referring to the prefix of other packages.
No, always use PREFIX. LOCALBASE is for the user to set.

Joerg
Greg Troxel
2014-10-20 16:52:50 UTC
Permalink
Post by J. Lewis Muir
So, PREFIX is the right variable to use when referring to the package's
own installation path. I think all the answers agree with this.
I think so, but in general it's helpful to read the makefiles. Prefix
is set in mk/bsd.pkg.use.mk, and you can see that it can be LOCALBASE or
other values, depending. So it seems clear that PREFIX is determined,
and then should be used.
Post by J. Lewis Muir
I'm hearing that LOCALBASE is the right variable to use at build-time if
constructing "-I" or "-L" arguments to the compiler, or, more generally,
when referring to the prefix of other packages.
I don't think that's true. I would argue that if you need to refer to
prefixes of other packages, then you should be including a bl3 file and
that file should set the variables you need. That way, the information
is modularized behind an abstraction boundary, even if newly defined.
Post by J. Lewis Muir
On the other hand, some answers say mk/find-prefix.mk (or EVAL_PREFIX)
is the right thing to be using for determining the prefix of a
build-time dependency. Why is mk/find-prefix.mk better than LOCALBASE?
The basic issue is that LOCALBASE is the configured location where
packages normally should be built. Which is different from where they
were built, maybe.
Post by J. Lewis Muir
Is LOCALBASE the right variable to use at *run-time* if referring to
the prefix of another package? For example, say I want to embed in a
program provided by my package the absolute path of a program on which
my package has a run-time dependency. Is LOCALBASE the right choice?
I find this question surprising, because it seems to be that the proper
way is for the upstream program to use configure and find the path and
then substitute it, all without pkgsrc's help. Or are you trying to
patch a program that doesn't do this and should?
Post by J. Lewis Muir
I'm still wondering about the best way to invoke a build-dependency
program during the build phase of my package. In particular, I'm
concerned about the case where the build-dependency is satisfied by a
built-in. I think a concrete example is the easiest way to discuss
this.
We have talked about this off and on in the context of cross building.
Thinking about cross is harder, but makes things easier to understand.
If you are running a program at build time, then it isn't a build
dependency (something you link against) but a tool dependency (a
program you run). A key point is that build deps need to be in the
target architecture/os, and tool deps in the host architecture/os (using
target and host in the normal sense, not the Canadian cross autoonf
sense).

So a build-dependency satisfied by a builtin when you run a program is
really a tool dependency and should be added to the tools framework.
Maybe the tools framework should make this easier.
Post by J. Lewis Muir
Let's say I have a build dependency on apache-ant because I want to
use the ant program it provides to build some Java source files for my
BUILD_DEPENDS+= apache-ant-[0-9]*:../../devel/apache-ant
FIND_PREFIX:= ANT_PREFIX=apache-ant
.include "../../mk/find-prefix.mk"
cd ${WORKSRC} && "${ANT_PREFIX}/bin/ant"
Why is that better than "${PREFIX}/bin/ant" or "${LOCALBASE}/bin/ant"?
Because ant might be built with some other JAVABASE prefix in some
future change to pkgsrc that you don't know about.

But really ant should be a tool.
Post by J. Lewis Muir
Now, let's go one step further and pretend that devel/apache-ant is
satisfied as a built-in (i.e. my base system comes with apache-ant by
default and devel/apache-ant has a builtin.mk). In this case, the
ant program might be at /usr/bin/ant, or some other path *not* under
the pkgsrc LOCALBASE. Will mk/find-prefix.mk work in this case? I'm
worried that it won't. This makes me wonder if a bare invocation of
cd ${WORKSRC} && ant
The builtin.mk should be figuring out and setting the path to ant.
Post by J. Lewis Muir
This assumes pkgsrc sets up the PATH environment variable (or wrappers?)
to make my do-build ant invocation find the correct ant program (from
pkgsrc or from the (built-in) base system). Is this valid? If so, is
this actually better than any absolute path invocation because it works
regardless of whether the program is from a pkgsrc package or the base
system (built-in)?
Generally, using the user PATH is not so great, because it leads to
inconsistent results.
J. Lewis Muir
2014-10-20 18:12:13 UTC
Permalink
Post by Greg Troxel
Post by J. Lewis Muir
Is LOCALBASE the right variable to use at *run-time* if referring to
the prefix of another package? For example, say I want to embed in
a program provided by my package the absolute path of a program on
which my package has a run-time dependency. Is LOCALBASE the right
choice?
I find this question surprising, because it seems to be that the
proper way is for the upstream program to use configure and find the
path and then substitute it, all without pkgsrc's help. Or are you
trying to patch a program that doesn't do this and should?
This is for any software that does not use GNU Autotools. I actually
prefer not to use Autotools for software I write. I prefer just default
make variables in Makefile that can be overridden from the make command
line invocation (e.g. "make PREFIX=/opt"). This works well, I think,
and it works really nicely for packaging since the packager knows where
things are and can just set them rather than trying to trick Autotools
into figuring out the right things.

But maybe my no-Autotools approach doesn't scale well; I don't know.
Post by Greg Troxel
Post by J. Lewis Muir
I'm still wondering about the best way to invoke a build-dependency
program during the build phase of my package. In particular, I'm
concerned about the case where the build-dependency is satisfied by a
built-in. I think a concrete example is the easiest way to discuss
this.
We have talked about this off and on in the context of cross building.
Thinking about cross is harder, but makes things easier to understand.
If you are running a program at build time, then it isn't a build
dependency (something you link against) but a tool dependency (a
program you run). A key point is that build deps need to be in the
target architecture/os, and tool deps in the host architecture/os
(using target and host in the normal sense, not the Canadian cross
autoonf sense).
So a build-dependency satisfied by a builtin when you run a program is
really a tool dependency and should be added to the tools framework.
Maybe the tools framework should make this easier.
Interesting. Yes, that sounds really good.

It would be cool if the tools framework would let me specify a tool via
USE_TOOLS in my package Makefile without it actually being an official
tool. So, all the existing tool stuff would stay, but the syntax could
be extended to allow specifying the package of a tool, and for any tool
that is not an official tool, the tools framework could create a symlink
or wrapper script for the tool that just invokes the right tool just
like the other official tools but without any additional behavior.

For example, in the case of ant, I could have this in my Makefile:

USE_TOOLS += ant:../../devel/apache-ant

The tools framework would create a symlink or wrapper script for the
ant program which would in turn use the ant program installed by
devel/apache-ant or the built-in found by devel/apache-ant/builtin.mk.
I could then use a bare "ant" just like an officially provided tool:

do-build:
cd ${WORKSRC} && ant

Since ant has been declared as a tool, I am assured that the tools
framework has arranged for the ant symlink or wrapper script to be found
on the shell's executable search path first.
Post by Greg Troxel
Post by J. Lewis Muir
Let's say I have a build dependency on apache-ant because I want to
use the ant program it provides to build some Java source files for
BUILD_DEPENDS+= apache-ant-[0-9]*:../../devel/apache-ant
FIND_PREFIX:= ANT_PREFIX=apache-ant
.include "../../mk/find-prefix.mk"
cd ${WORKSRC} && "${ANT_PREFIX}/bin/ant"
Why is that better than "${PREFIX}/bin/ant" or
"${LOCALBASE}/bin/ant"?
Because ant might be built with some other JAVABASE prefix in some
future change to pkgsrc that you don't know about.
OK, so you're saying it's possible that the ant program would not be
installed at ${PREFIX}/bin/ant in the future, and for this reason, using
mk/find-prefix.mk is better because it would be able to figure out
where ant was actually installed (obviously unlike the case of using
${PREFIX}/bin/ant).
Post by Greg Troxel
But really ant should be a tool.
Got it. And a tool is normally invoked in a bare form (e.g. "ant") not
as an absolute path, right? The pkgsrc developer's guide says that the
tools framework defines the variable TOOLS_PATH.foo for tool "foo" which
contains the full path to the tool. But I'm assuming that's helpful for
debugging, but I don't need to use that to invoke the tool.
Post by Greg Troxel
Post by J. Lewis Muir
Now, let's go one step further and pretend that devel/apache-ant is
satisfied as a built-in (i.e. my base system comes with apache-ant by
default and devel/apache-ant has a builtin.mk). In this case, the
ant program might be at /usr/bin/ant, or some other path *not* under
the pkgsrc LOCALBASE. Will mk/find-prefix.mk work in this case? I'm
worried that it won't. This makes me wonder if a bare invocation of
cd ${WORKSRC} && ant
The builtin.mk should be figuring out and setting the path to ant.
Post by J. Lewis Muir
This assumes pkgsrc sets up the PATH environment variable (or
wrappers?) to make my do-build ant invocation find the correct
ant program (from pkgsrc or from the (built-in) base system). Is
this valid? If so, is this actually better than any absolute path
invocation because it works regardless of whether the program is from
a pkgsrc package or the base system (built-in)?
Generally, using the user PATH is not so great, because it leads to
inconsistent results.
Understood. But for a tool, it is good to depend upon the PATH variable
of the environment because it has been set by the tools framework to the
correct value, right?

Thank you!

Lewis
Greg Troxel
2014-10-20 18:21:41 UTC
Permalink
Post by J. Lewis Muir
This is for any software that does not use GNU Autotools. I actually
prefer not to use Autotools for software I write. I prefer just default
make variables in Makefile that can be overridden from the make command
line invocation (e.g. "make PREFIX=/opt"). This works well, I think,
and it works really nicely for packaging since the packager knows where
things are and can just set them rather than trying to trick Autotools
into figuring out the right things.
But maybe my no-Autotools approach doesn't scale well; I don't know.
It doesn't scale; packagers need to read and understand your docs,
rather than just saying "this uses autoconf". Telling autoconf-built
packages how to set prefix, and so on is already worked out as a general
case.
Greg Troxel
2014-10-21 12:26:20 UTC
Permalink
Post by J. Lewis Muir
It would be cool if the tools framework would let me specify a tool via
USE_TOOLS in my package Makefile without it actually being an official
tool.
That might be useful, but I wonder how many tools there are that truly
expect to be used by only one package. ant is certainly not in that
category. I would suggest that you look at what it takes to add it to
the existing framework before deciding to add a mechanism to declare
local tools. Or at least until we find a tool needed for some package
that we can confidently declare to be so local to that package that it
would be clutter under mk/.
Post by J. Lewis Muir
Got it. And a tool is normally invoked in a bare form (e.g. "ant") not
as an absolute path, right? The pkgsrc developer's guide says that the
tools framework defines the variable TOOLS_PATH.foo for tool "foo" which
contains the full path to the tool. But I'm assuming that's helpful for
debugging, but I don't need to use that to invoke the tool.
I think ${TOOLS_PATH.foo} should be used to invoke the tool. That's the
point of figuring out the path - to make sure that the same program that
was judged adequate is invoked.
Post by J. Lewis Muir
Post by Greg Troxel
Generally, using the user PATH is not so great, because it leads to
inconsistent results.
Understood. But for a tool, it is good to depend upon the PATH variable
of the environment because it has been set by the tools framework to the
correct value, right?
I think the tools framework sets the pathname of the tool itself, not a
search path. The tools framework and builtin.mk may have OS-specific
paths to check for tools that are adequate.

Loading...