Perl5 update

[Jens Rehsack]

Hi Adam,

with you perl5 update (I think you should revert unless you want to go the extra mile), did you mind to revbump all perl-modules and prove each p5-* module against requiring removed modules from core (Module::Build, CGI, ...)?

Best regards

[Thomas Klausner]

with you perl5 update, did you mind to revbump all perl-modules

I'm currently doing that. Might take a few more hours though.

and prove each p5-* module against requiring removed modules from core (Module::Build, CGI, ...)?

There's no automation for that yet, so we'll fix these as we find
them.
Thomas

[Thomas Klausner]

Post by Thomas Klausner

with you perl5 update, did you mind to revbump all perl-modules

I'm currently doing that. Might take a few more hours though.

I've just finished. Like last time for 5.18, I bumped all packages that
* mention "perl", or
* have a directory name starting with p5-*, or
* depend on a package starting with p5-

Let me know if you find other packages that should be bumped.

Thanks,
Thomas

[OBATA Akio]

Post by Thomas Klausner

Post by Jens Rehsack
and prove each p5-* module against requiring removed modules from core (Module::Build, CGI, ...)?

There's no automation for that yet, so we'll fix these as we find
them.

Those module are not removed yet, just marked as deprecated
http://search.cpan.org/~rjbs/perl-5.20.0/pod/perldelta.pod#Module_removals
Should we prevent warnings?

And following also should be done:
http://mail-index.netbsd.org/tech-pkg/2013/11/22/msg012262.html

Update CONFLICTS in lang/perl5/Makefile.
http://search.cpan.org/~rjbs/perl-5.20.0/pod/perldelta.pod#Modules_and_Pragmata
Check packages marked as "older for.." core module may be sufficient now.
...

[Jens Rehsack]

Post by OBATA Akio

Post by Thomas Klausner

Post by Jens Rehsack
and prove each p5-* module against requiring removed modules from core (Module::Build, CGI, ...)?

There's no automation for that yet, so we'll fix these as we find
them.

Those module are not removed yet, just marked as deprecated
http://search.cpan.org/~rjbs/perl-5.20.0/pod/perldelta.pod#Module_removals

True - I was mislead by behavior of last blead/release ...
Those modules were marked depreciated in blead and got removed with 1st release.

Post by OBATA Akio
Should we prevent warnings?

Nope, we should prefer dedicated packages for CPAN versions of the modules (as we do already for Module::Build).
Regarding to Saywer X statement "CGI.pm must die" (in favor of Catalyst, Dancer, Mojo, ...) I don't check CGI to often, so sorry for filtering it :)

Post by OBATA Akio
http://mail-index.netbsd.org/tech-pkg/2013/11/22/msg012262.html

Some say so, some say just add a comment.
In https://github.com/rehsack/Packager-Utils I meanwhile do comments of the recommended line.

Post by OBATA Akio
Update CONFLICTS in lang/perl5/Makefile.
http://search.cpan.org/~rjbs/perl-5.20.0/pod/perldelta.pod#Modules_and_Pragmata
Check packages marked as "older for.." core module may be sufficient now.

This would have been a nice approval whether P::U does that nicely out of the box.
I see how far I come using a not updated git-clone ;)

Cheers

[Thomas Klausner]

Post by OBATA Akio
http://mail-index.netbsd.org/tech-pkg/2013/11/22/msg012262.html

I've done step 5 (or 1? what's the difference?) of that now -- for all
existing CONFLICTS lines, I've fixed the pattern to match the corelist
output. I had to remove one line because the package was removed from
core, and added one line, because it was added to core
(p5-experimental).

I don't understand the "old for" lines, so I didn't touch them. What
do they mean, can you please explain?

There are many more corelist changes than those for existing CONFLICTS
lines. How did you find out if any of them affect existing packages
when you last did that?

Are there volunteers for steps 2 and 3?

2: update p5-* packages that are now older than perl core (bulk builds
will tell us about them)

3: fix DEPENDS for packages depending on packages now in perl core

Thomas

[OBATA Akio]

Post by Thomas Klausner

Post by OBATA Akio
http://mail-index.netbsd.org/tech-pkg/2013/11/22/msg012262.html

I've done step 5 (or 1? what's the difference?) of that now -- for all
existing CONFLICTS lines, I've fixed the pattern to match the corelist
output. I had to remove one line because the package was removed from
core, and added one line, because it was added to core
(p5-experimental).

Thanks!

Post by Thomas Klausner
I don't understand the "old for" lines, so I didn't touch them. What
do they mean, can you please explain?

For example,

CONFLICTS+= p5-CPAN-Meta<2.120921
# older for devel/p5-Module-CPANTS-Analyse

It means that CPAN::Meta in perl core is 2.120921, but
devel/p5-Module-CPANTS-Analyse require newer version (>=2.131490).

DEPENDS+= p5-CPAN-Meta>=2.131490:../../devel/p5-CPAN-Meta

Now in perl5/Makefile,
CONFLICTS+= p5-CPAN-Meta<2.140640

Then update dependency of devel/p5-Module-CPANTS-Analyse as following

DEPENDS+= {perl>=5.19.1,p5-CPAN-Meta>=2.131490}:../../devel/p5-CPAN-Meta
(and drop "old for")

Post by Thomas Klausner
There are many more corelist changes than those for existing CONFLICTS
lines. How did you find out if any of them affect existing packages
when you last did that?

Find pkgsrc by name, ex, find IO::Compress::Gzip with p5-IO-Compress-Gzip.
If not found, try with p5-IO-Compress.
Then go to http://search.cpan.org/dist/IO-Compress/, and it is confirmed
that p5-IO-Compress contains IO::Compress::Gzip.
Moreover, it is noticed that this package also contains Compress::Zlib,
Then,
CONFLICTS+= p5-IO-Compress<2.060 # IO::Compress::*, Compress::*

Post by Thomas Klausner
Are there volunteers for steps 2 and 3?
2: update p5-* packages that are now older than perl core (bulk builds
will tell us about them)

Yes, it should help us.

Post by Thomas Klausner
3: fix DEPENDS for packages depending on packages now in perl core

At least, for packages
* noted "older for" in lang/Makefile
* depending on modules removed from core
* depending on modules added to core

[Thomas Klausner]

Post by OBATA Akio

Post by Thomas Klausner
I don't understand the "old for" lines, so I didn't touch them. What
do they mean, can you please explain?

For example,
CONFLICTS+= p5-CPAN-Meta<2.120921
# older for devel/p5-Module-CPANTS-Analyse
It means that CPAN::Meta in perl core is 2.120921, but
devel/p5-Module-CPANTS-Analyse require newer version (>=2.131490).
DEPENDS+= p5-CPAN-Meta>=2.131490:../../devel/p5-CPAN-Meta
Now in perl5/Makefile,
CONFLICTS+= p5-CPAN-Meta<2.140640
Then update dependency of devel/p5-Module-CPANTS-Analyse as following
DEPENDS+= {perl>=5.19.1,p5-CPAN-Meta>=2.131490}:../../devel/p5-CPAN-Meta
(and drop "old for")

Thanks for the explanation and the fix.

Post by OBATA Akio

Post by Thomas Klausner
There are many more corelist changes than those for existing CONFLICTS
lines. How did you find out if any of them affect existing packages
when you last did that?

Find pkgsrc by name, ex, find IO::Compress::Gzip with p5-IO-Compress-Gzip.
If not found, try with p5-IO-Compress.
Then go to http://search.cpan.org/dist/IO-Compress/, and it is confirmed
that p5-IO-Compress contains IO::Compress::Gzip.
Moreover, it is noticed that this package also contains Compress::Zlib,
Then,
CONFLICTS+= p5-IO-Compress<2.060 # IO::Compress::*, Compress::*

Ok, that's still open.

Post by OBATA Akio

Post by Thomas Klausner
Are there volunteers for steps 2 and 3?
2: update p5-* packages that are now older than perl core (bulk builds
will tell us about them)

Yes, it should help us.

I've updated CPAN today and I saw you updated external. Let's see if
there are others.

Post by OBATA Akio

Post by Thomas Klausner
3: fix DEPENDS for packages depending on packages now in perl core

At least, for packages
* noted "older for" in lang/Makefile

I think you did that.

Post by OBATA Akio
* depending on modules removed from core

I think I've handled all of these now.

Post by OBATA Akio
* depending on modules added to core

I think only p5-external was relevant, and you handled that already.

Cheers,
Thomas

[Edgar Fuß]

Shouldn't there be an infrastructure mechanism that allows a Perl Module's
Makefile to express either
I need Foo::Bar 47.11, and I assume it's in Core
(which sets PKG_FAIL_REASON if that assertion is false)
or better
I need Foo::Bar 47.11, take it from Core or devel/p5-Foo-Bar
(adding a dependency on the latter if the former is false)?

I don't think that would be too hard to implement. You need a pre-digested
corelist output in lang/perl5/somewhere, an fgrep and a version comparison.

You could even automatically add a CONFLICTS+= p5-Foo-Bar if you take
Foo::Bar from Core. I guess Perl Module packages automatically depend on
"this Perl Version and not the next one", don't tey?

Do I overlook something?

[Greg Troxel]

Post by Edgar Fuß
Shouldn't there be an infrastructure mechanism that allows a Perl Module's
Makefile to express either
I need Foo::Bar 47.11, and I assume it's in Core
(which sets PKG_FAIL_REASON if that assertion is false)
or better
I need Foo::Bar 47.11, take it from Core or devel/p5-Foo-Bar
(adding a dependency on the latter if the former is false)?
I don't think that would be too hard to implement. You need a pre-digested
corelist output in lang/perl5/somewhere, an fgrep and a version comparison.
You could even automatically add a CONFLICTS+= p5-Foo-Bar if you take
Foo::Bar from Core. I guess Perl Module packages automatically depend on
"this Perl Version and not the next one", don't tey?

A big question is the cost/benefit of adding this flexibility. In
general, one has to build from a consistent checkout of pkgsrc. And
building packages with other packages being out of date gets one into
tricky territory. So saying that p5-* packages in pkgsrc should expect
the current perl5 version seems like a simplifying assumption that saves
a lot of work, but on the other hand expects people to replace their
perl interpreter with the one from the pkgsrc version they are using.

[Edgar Fuß]

In general, one has to build from a consistent checkout of pkgsrc.

Yes, exactly.

So saying that p5-* packages in pkgsrc should expect the current perl5
version seems like a simplifying assumption that saves a lot of work,

I'm afraid you mis-understood me. With "current", I meant "the version
included in this checkout of pkgsrc".

but on the other hand expects people to replace their perl interpreter
with the one from the pkgsrc version they are using.

I don't get that.
If I have 5.14 installed and build a package containing a Perl Module
from a pkgsrc checkout that includes 5.16, I get 5.16 built because I
get a dependency on perl>=5.16<5.18, don't I?

[Greg Troxel]

Post by Edgar Fuß

In general, one has to build from a consistent checkout of pkgsrc.

Yes, exactly.

So saying that p5-* packages in pkgsrc should expect the current perl5
version seems like a simplifying assumption that saves a lot of work,

I'm afraid you mis-understood me. With "current", I meant "the version
included in this checkout of pkgsrc".

That's what I was trying to say. When one does a cvs checkout/update
to some branch or date, then it's the p5-* package in that checkout can
expect that the perl5 package in that checkout is installed, or more
carefully can depend on it.

Post by Edgar Fuß

but on the other hand expects people to replace their perl interpreter
with the one from the pkgsrc version they are using.

I don't get that.
If I have 5.14 installed and build a package containing a Perl Module
from a pkgsrc checkout that includes 5.16, I get 5.16 built because I
get a dependency on perl>=5.16<5.18, don't I?

Probably. But if building any perl module requires that you have the
up-to-date (with respect to your checkout) perl version, then why is
this OR dependency method needed? Each p5-* depends on modules and the
current base version, so there are no choices.

[Edgar Fuß]

Post by Greg Troxel
But if building any perl module requires that you have the
up-to-date (with respect to your checkout) perl version, then why is
this OR dependency method needed? Each p5-* depends on modules and the
current base version, so there are no choices.

Currently, if Perl is updated, someone has to manually look into each
packaged module's source (Build.PL or even *.pm) to find dependencies
on Core modules that have been removed from Core. And you have to inspect
all the pkgsrc Makefile's for dependencies on pkgsrc packages containing
Perl modules that have been added to Core.

I proposed to automate that process.

In addition to that, currently, in a pkgsrc Makefile, I can't express
what I mean. I don't mean
I need Perl at least 5.4 but older than 5.20
or I depend on ../devel/p5-CGI
I mean
I need the CGI module, version 1.23
and can hint
You can get that from ../devel/p5-CGI if it's not in Core
.

[OBATA Akio]

Post by Edgar Fuß
I proposed to automate that process.
In addition to that, currently, in a pkgsrc Makefile, I can't express
what I mean. I don't mean
I need Perl at least 5.4 but older than 5.20
or I depend on ../devel/p5-CGI
I mean
I need the CGI module, version 1.23
and can hint
You can get that from ../devel/p5-CGI if it's not in Core

We require big database for perl module name and correspondent package,
CGI module is in ../www/p5-CGI.

-- =

OBATA Akio / ***@lins.jp

[Edgar Fuß]

EF> I mean
EF> I need the CGI module, version 1.23
EF> and can hint
EF> You can get that from ../devel/p5-CGI if it's not in Core
OA> We require big database for perl module name and correspondent package,
No, why? You don't need that database if the Makefile contains
DEPENDS+= p5-CGI>=1.23:../../www/p5-CGI
so why do you require it if you write something like
PERL_MODULE_DEPENDS+= CGI:1.23:www/p5-CGI
or the like?

OA> CGI module is in ../www/p5-CGI.
Sorry, I just mis-spelled that.

[OBATA Akio]

Post by Edgar Fuß
EF> I mean
EF> I need the CGI module, version 1.23
EF> and can hint
EF> You can get that from ../devel/p5-CGI if it's not in Core
OA> We require big database for perl module name and correspondent pac=

kage,

Post by Edgar Fuß
No, why? You don't need that database if the Makefile contains
DEPENDS+=3D p5-CGI>=3D1.23:../../www/p5-CGI
so why do you require it if you write something like
PERL_MODULE_DEPENDS+=3D CGI:1.23:www/p5-CGI
or the like?

I felt that you suggested following style
PERL_MODULE_DEPENDS+=3D CGI:1.23
and list whole dependency modules (either in core or not)...

Post by Edgar Fuß
OA> CGI module is in ../www/p5-CGI.
Sorry, I just mis-spelled that.

-- =

OBATA Akio / ***@lins.jp

[OBATA Akio]

Post by Edgar Fuß
I don't get that.
If I have 5.14 installed and build a package containing a Perl Module
from a pkgsrc checkout that includes 5.16, I get 5.16 built because I
get a dependency on perl>=3D5.16<5.18, don't I?

If I have buitin perl 5.14 and build a package requiring a Perl Module
but in perl 5.14 core, additional packages will not be installed.
Because required perl version will be came from only PERL5_REQD,
it's 5.0 by default.

-- =

OBATA Akio / ***@lins.jp

[Edgar Fuß]

Post by OBATA Akio
If I have buitin perl 5.14

Ah, OK, I forgot about built-in Perls.
You will need multiple versions of the large pre-compiled table then,
one for every major release of Perl. Then choose the one apropriate for
your built-in Perl.

[David Holland]

Post by Edgar Fuß

Post by OBATA Akio
If I have buitin perl 5.14

Ah, OK, I forgot about built-in Perls.
You will need multiple versions of the large pre-compiled table then,
one for every major release of Perl. Then choose the one apropriate for
your built-in Perl.

For best results, the table should be stuffed in the perl binary
package. Then it will all work regardless of what perl you happen to
have installed.

(Can this info be extracted automatically from a perl install? Because
then builtin perls can provide it on the fly.)

[Thomas Klausner]

Post by David Holland
For best results, the table should be stuffed in the perl binary
package. Then it will all work regardless of what perl you happen to
have installed.

The information needs to be available before perl is built -- e.g.,
pbulk needs it for dependency tracking.
Thomas

[Jens Rehsack]

Post by David Holland

Post by Edgar Fuß

Post by OBATA Akio
If I have buitin perl 5.14

Ah, OK, I forgot about built-in Perls.
You will need multiple versions of the large pre-compiled table then,
one for every major release of Perl. Then choose the one apropriate for
your built-in Perl.

For best results, the table should be stuffed in the perl binary
package. Then it will all work regardless of what perl you happen to
have installed.

I assume you're talking about Module::CoreList. Yes, there is a bundled
one since 5.8.9 and 5.9.2.

Post by David Holland
(Can this info be extracted automatically from a perl install? Because
then builtin perls can provide it on the fly.)

Not as easy as the idea would require. But I'm interested to see the
patch and see how good it performs.

Best regards

[OBATA Akio]

Shouldn't there be an infrastructure mechanism that allows a Perl Modu=

le's

Makefile to express either
I need Foo::Bar 47.11, and I assume it's in Core
(which sets PKG_FAIL_REASON if that assertion is false)
or better
I need Foo::Bar 47.11, take it from Core or devel/p5-Foo-Bar
(adding a dependency on the latter if the former is false)?
I don't think that would be too hard to implement. You need a pre-dige=

sted

corelist output in lang/perl5/somewhere, an fgrep and a version compar=

ison.

You could even automatically add a CONFLICTS+=3D p5-Foo-Bar if you tak=

e

Foo::Bar from Core. I guess Perl Module packages automatically depend =

on

"this Perl Version and not the next one", don't tey?
Do I overlook something?

Foo::Bar::Baz perl module is not always in p5-Foo-Bar-Baz package.

-- =

OBATA Akio / ***@lins.jp

[Edgar Fuß]

EF> I need Foo::Bar 47.11, take it from Core or devel/p5-Foo-Bar
[...]
EF> Do I overlook something?
OA> Foo::Bar::Baz perl module is not always in p5-Foo-Bar-Baz package.
That's what the "or devel/p5-Foo-Bar" part was about.

[OBATA Akio]

Post by Edgar Fuß
EF> I need Foo::Bar 47.11, take it from Core or devel/p5-Foo-Bar
[...]
EF> Do I overlook something?
OA> Foo::Bar::Baz perl module is not always in p5-Foo-Bar-Baz package.=
That's what the "or devel/p5-Foo-Bar" part was about.

As a result of my whole pkgsrc perl related scanning,
it is not so simple.

-- =

OBATA Akio / ***@lins.jp

[Jens Rehsack]

Post by Edgar Fuß
EF> I need Foo::Bar 47.11, take it from Core or devel/p5-Foo-Bar
[...]
EF> Do I overlook something?
OA> Foo::Bar::Baz perl module is not always in p5-Foo-Bar-Baz package.
That's what the "or devel/p5-Foo-Bar" part was about.

Check devel/p5-IO-Async/Makefile - what do you miss there?

Cheers

[Jens Rehsack]

Post by Edgar Fuß
Shouldn't there be an infrastructure mechanism that allows a Perl Module's
Makefile to express either
I need Foo::Bar 47.11, and I assume it's in Core
(which sets PKG_FAIL_REASON if that assertion is false)
or better
I need Foo::Bar 47.11, take it from Core or devel/p5-Foo-Bar
(adding a dependency on the latter if the former is false)?
I don't think that would be too hard to implement. You need a pre-digested
corelist output in lang/perl5/somewhere, an fgrep and a version comparison.

How about Test::Harness::Env? Separate module or bundled into p5-Test-Harness?

Post by Edgar Fuß
You could even automatically add a CONFLICTS+= p5-Foo-Bar if you take
Foo::Bar from Core.

Installing two applications in the same pkgsrc-prefix - one requires Foo::Bar
47.11 (in Core), the other Foo::Bar 47::17 - conflicts will prevent 2nd app
being installed.

Post by Edgar Fuß
I guess Perl Module packages automatically depend on
"this Perl Version and not the next one", don't tey?
Do I overlook something?

You think an 80% solution. In real world perl modules move between distributions
(resulting in different packages) - we would have to create a buillink3.mk for
each perl module then - and we have much more issues on updating.

As I said: come to next niederrhein.pm and I explain that in full detail.
I'm working on a cpan-package creator and it's not that trivial you see for
simple modules. Check DBIx::Class, Moose, Catalyst or IO::Async for pitfalls.

My question is: what do you want to reach with that proposal?

Cheers

[Edgar Fuß]

I am NOT proposing a mechanism that will find out that Foo::Bar::Bar
is provided by devel/p5-Foo-Bar.
I repeat: I am NOT proposing an automated mapping from Module names to
pkgsrc packages.
Did I mention that I DON'T want to automatically decide which pkgsrc
package a given (non-Core) Perl Module resides in?

JR> How about Test::Harness::Env?
JR> Separate module or bundled into p5-Test-Harness?
?

EF> You could even automatically add a CONFLICTS+= p5-Foo-Bar if you take
EF> Foo::Bar from Core.
JR> Installing two applications in the same pkgsrc-prefix - one requires
JR> Foo::Bar 47.11 (in Core), the other Foo::Bar 47::17 - conflicts will
JR> prevent 2nd app being installed.
You may omit that part, but recording no conflict may prevent one of them
to work properly, doesn't it?

JR> My question is: what do you want to reach with that proposal?
I guess I tried to explain that before.
1. Currently, when I pkgsrc a Perl Distribution, I have to manually
go through all the required modules and find out since which release
of Perl they have been in Core. Then, for every module in Core,
I have to include a line
DEPENDS+= {perl>=5.nn,p5-Foo-Bar>=1.23}:../../devel/p5-Foo-Bar
or
DEPENDS+= p5-Foo-Bar>=1.23:../../devel/p5-Foo-Bar
in the Makefile. I nail down the result of my research given what
I know about Perl releases up to now.
2. When a new release of Perl adds/removes modules to/from Core, someone
has to re-visit all the Makefiles and go through the same dance again.
(3. I prefer to express what I mean, not something that amounts to the same
thing under current circumstances.)

If I was allowed to replace the above with something like
PERL_MODULE_DEPENDS+= Foo::Bar>=1.23;devel/p5-Foo-Bar
(I would like to point out there's a pkrsrc package name "devel/p5-Foo-Bar"
in that line) then a computer can make these decisions.
Not only are computers better at making bulk decisions, it also allows
to automatically re-make that decision for every Perl release to come.

In real world perl modules move between distributions (resulting in
different packages)

Yes. Both the current DEPENDS+= way and the way I propose you have to
manually adjust all Makefiles depending on a module that moves from one
package to another. I did mention that I don't want to automate the way
you find out which pkgsrc package or Perl Distribution a given (non-Core)
module resides in?

[Thomas Klausner]

Post by Edgar Fuß
1. Currently, when I pkgsrc a Perl Distribution, I have to manually
go through all the required modules and find out since which release
of Perl they have been in Core. Then, for every module in Core,
I have to include a line
DEPENDS+= {perl>=5.nn,p5-Foo-Bar>=1.23}:../../devel/p5-Foo-Bar
or
DEPENDS+= p5-Foo-Bar>=1.23:../../devel/p5-Foo-Bar
in the Makefile. I nail down the result of my research given what
I know about Perl releases up to now.
2. When a new release of Perl adds/removes modules to/from Core, someone
has to re-visit all the Makefiles and go through the same dance again.
(3. I prefer to express what I mean, not something that amounts to the same
thing under current circumstances.)
If I was allowed to replace the above with something like
PERL_MODULE_DEPENDS+= Foo::Bar>=1.23;devel/p5-Foo-Bar
(I would like to point out there's a pkrsrc package name "devel/p5-Foo-Bar"
in that line) then a computer can make these decisions.
Not only are computers better at making bulk decisions, it also allows
to automatically re-make that decision for every Perl release to come.

I think this would work in principle. However, there's also the
implementation step. Since it's a major change for perl in pkgsrc, I'd
like to see a working change for a simple example, then we could think
about how to increase the coverage to all perl modules.

Post by Edgar Fuß
From what I see right now, we'd need at least

* a file in mk, e.g. perl-module-mappings.mk (name pulled out of hat,
bring your own hat) mapping either

* all modules ever included in perl to pkgsrc paths, or, probably
better,

* all perl modules to pkgsrc paths

* a file containing the current state of what perl provides itself

* a script that updates this file based on corelist output

* some dependency logic that reads these files to decide if pulling in
perl is enough or if a different pkgsrc package is needed

* some way to semiautomatically add lines to perl-module-mappings.mk
when a new perl package is added to pkgsrc


Reading this list, I'm not sure it's really worth the effort or even
an improvement to what we have now, but I'm willing to be surprised by
your design and patch :)
Thomas

[Edgar Fuß]

Post by Edgar Fuß
From what I see right now, we'd need at least

[...]

Post by Edgar Fuß
* all modules ever included in perl to pkgsrc paths, or, probably
better,
* all perl modules to pkgsrc paths

Hugh?
Eh, I did mention that I'm NOT proposing pkgsrc's infrastructure to map
Perl Modules to pkgsrc, paths, right?

Post by Edgar Fuß
* a file containing the current state of what perl provides itself
* a script that updates this file based on corelist output

Something like that. Basically, for every release of Perl, a list of
Module/Version pairs.

Post by Edgar Fuß
* some dependency logic that reads these files to decide if pulling in
perl is enough or if a different pkgsrc package is needed

Yes. grep the version number and compare to the required one.

Post by Edgar Fuß
* some way to semiautomatically add lines to perl-module-mappings.mk
when a new perl package is added to pkgsrc

I seem to have forgotten to mention that I'm NOT proposing to automatically
have pkgsrc map Perl Module names to pkgsrc package names.

[Jens Rehsack]

Post by Edgar Fuß

Post by Edgar Fuß
From what I see right now, we'd need at least

[...]

Post by Edgar Fuß
* all modules ever included in perl to pkgsrc paths, or, probably
better,
* all perl modules to pkgsrc paths

Hugh?
Eh, I did mention that I'm NOT proposing pkgsrc's infrastructure to map
Perl Modules to pkgsrc, paths, right?

Post by Edgar Fuß
* a file containing the current state of what perl provides itself
* a script that updates this file based on corelist output

Something like that. Basically, for every release of Perl, a list of
Module/Version pairs.

Post by Edgar Fuß
* some dependency logic that reads these files to decide if pulling in
perl is enough or if a different pkgsrc package is needed

Yes. grep the version number and compare to the required one.

Post by Edgar Fuß
* some way to semiautomatically add lines to perl-module-mappings.mk
when a new perl package is added to pkgsrc

I seem to have forgotten to mention that I'm NOT proposing to automatically
have pkgsrc map Perl Module names to pkgsrc package names.

As Thomas said: I'd like to be surprised by the patch. I'm interested in
getting an answer what you're missing in devel/p5-IO-Async/Makefile or
similar package?

Please note that the dependency format

DEPENDS += {perl>=5.10.1,p5-CPAN>=1.9301}:../../devel/p5-CPAN

has been discussed and is discouraged.

Best regards

[Edgar Fuß]

Post by Jens Rehsack
As Thomas said: I'd like to be surprised by the patch.

Let's see when I have time to sketch one.

Post by Jens Rehsack
I'm interested in getting an answer what you're missing in
devel/p5-IO-Async/Makefile or similar package?

I thought I imlicitly answered that multiple times:
1. Someone has to manually add all these PERL5_REQD+= lines to the Makefile,
which can only be done by inspecting corelist.
2. Someone has to re-do that when modules get added to/removed from Core.
(3. You only in comment fileds do write down what you really mean.)
4. What do you write if you need a module that has been removed from Core?
You can't express a range in PERL5_REQD, do you?

There's another issue:
5. Suppose you have a native Perl 5.40 satisfying all requirements but
lacking the required Foo::Bar module, which is included in pkgsrc's Perl
5.42 (and also available in pkgsrc as devel/p5-Foo-Bar).
Then, your approach, contrary to the, as I just learned, deprecated
DEPENDS+= {perl>=5.42,p5-Foo-Bar>=1.23}:../../devel/p5-Foo-Bar
practise, will pull in the pkgsrc Perl 5.42, not p5-Foo-Bar.
Is that what is desired?

[Jens Rehsack]

Post by Edgar Fuß

Post by Jens Rehsack
As Thomas said: I'd like to be surprised by the patch.

Let's see when I have time to sketch one.

Post by Jens Rehsack
I'm interested in getting an answer what you're missing in
devel/p5-IO-Async/Makefile or similar package?

1. Someone has to manually add all these PERL5_REQD+= lines to the Makefile,
which can only be done by inspecting corelist.

This is not true anymore. Packager::Utils will do that for (some)one.

Post by Edgar Fuß
2. Someone has to re-do that when modules get added to/removed from Core.

See RE: 1
Also: Packager::Utils doesn't add that line when it knows it's removed
from core (marked as depreciated even in next devel-version).

Post by Edgar Fuß
(3. You only in comment fileds do write down what you really mean.)

No, Packager::Utils inspects META uploaded to CPAN. Packager

Post by Edgar Fuß
4. What do you write if you need a module that has been removed from Core?
You can't express a range in PERL5_REQD, do you?

No, in that case always the separate package is referred.

Post by Edgar Fuß
5. Suppose you have a native Perl 5.40 satisfying all requirements but
lacking the required Foo::Bar module, which is included in pkgsrc's Perl
5.42 (and also available in pkgsrc as devel/p5-Foo-Bar).
Then, your approach, contrary to the, as I just learned, deprecated
DEPENDS+= {perl>=5.42,p5-Foo-Bar>=1.23}:../../devel/p5-Foo-Bar
practise, will pull in the pkgsrc Perl 5.42, not p5-Foo-Bar.
Is that what is desired?

Currently pkgsrc comes with one perl only. If we do multiple perls as
we do for python, we (I, patches welcome) have to modify Packager::Utils
to fall back to ›DEPENDS += {perl>=5.10.1,p5-CPAN>=1.9301}:../../devel/p5-CPAN‹

Currently P::U cares only for the version contained in pkgsrc (or better:
packaged version - theoretically any packaging system can be plugged in)
or later in case of deprecation.

I recently talked to the MetaCPAN folks to enhance metacpan to allow updating
packages by knowing the distribution only (refer DESCRIPTION etc. from
primary module of distribution - what is somewhat difficult for eg. IO-Compress).

Cheers

[Edgar Fuß]

JR> This is not true anymore. Packager::Utils will do that for (some)one.
That sounds great.

EF> 2. Someone has to re-do that when modules get added to/removed from Core.
JR> See RE: 1
Oh, it automatically inspects all Makefiles and replaces the outdated
information with uptdated one? That sounds even better.

EF> 4. What do you write if you need a module that has been removed from Core?
EF> You can't express a range in PERL5_REQD, do you?
JR> No, in that case always the separate package is referred.
Hm, and if my native Perl still has that module in Core?

EF> 5. Suppose you have a native Perl 5.40 satisfying all requirements but
EF> lacking the required Foo::Bar module, which is included in pkgsrc's Perl
EF> 5.42 (and also available in pkgsrc as devel/p5-Foo-Bar).
EF> Then, your approach, contrary to the, as I just learned, deprecated
EF> DEPENDS+= {perl>=5.42,p5-Foo-Bar>=1.23}:../../devel/p5-Foo-Bar
EF> practise, will pull in the pkgsrc Perl 5.42, not p5-Foo-Bar.
EF> Is that what is desired?
JR> Currently pkgsrc comes with one perl only.
Yes, but in another part of this thread I was told I had to care about
native (e.g. non-pkgsrc) Perl. I really don't know about that. Do I have
to care about native Perl or not?

[Thomas Klausner]

Post by Edgar Fuß
Yes, but in another part of this thread I was told I had to care about
native (e.g. non-pkgsrc) Perl. I really don't know about that. Do I have
to care about native Perl or not?

No.
Thomas

[OBATA Akio]

Yes, but in another part of this thread I was told I had to care abou=

t

native (e.g. non-pkgsrc) Perl. I really don't know about that. Do I h=

ave

to care about native Perl or not?

No.

I'm using native perl for some minor platforms (Cygwin, Haiku,...).
Because perl is used in various packages (even for openssl), so if perl
package is broken, many packages are not buildable.
I ever fixed perl package for such platforms, but now broken again.
After following changes
http://mail-index.netbsd.org/pkgsrc-changes/2014/04/28/msg105057.html=

native perl is usable if it satisfies whole dependencies related to perl=

[Jens Rehsack]

Post by OBATA Akio

Post by Edgar Fuß
Yes, but in another part of this thread I was told I had to care about
native (e.g. non-pkgsrc) Perl. I really don't know about that. Do I have
to care about native Perl or not?

No.

I'm using native perl for some minor platforms (Cygwin, Haiku,...).
Because perl is used in various packages (even for openssl), so if perl
package is broken, many packages are not buildable.
I ever fixed perl package for such platforms, but now broken again.
After following changes
http://mail-index.netbsd.org/pkgsrc-changes/2014/04/28/msg105057.html
native perl is usable if it satisfies whole dependencies related to perl

I'm very surprised how you deal with

devel/p5-ExtUtils-MakeMaker for system perl and a 2nd
devel/p5-ExtUtils-MakeMaker for pkgsrc perl

I rate the idea itself as dangerous.

Cheers

[OBATA Akio]

Post by Jens Rehsack

Post by OBATA Akio
After following changes
http://mail-index.netbsd.org/pkgsrc-changes/2014/04/28/msg105057.html
native perl is usable if it satisfies whole dependencies related to perl

I'm very surprised how you deal with
devel/p5-ExtUtils-MakeMaker for system perl and a 2nd
devel/p5-ExtUtils-MakeMaker for pkgsrc perl
I rate the idea itself as dangerous.

In my point of view, I want to use packages requiring perl as a tool.
Not for perl itself, including additional perl modules.
Please look at the diff, only for USE_TOOLS=perl.

The next step is split runtime and buildtime dependency handling,
to improve cross build support.

[Jens Rehsack]

Post by OBATA Akio

Post by Jens Rehsack

Post by OBATA Akio
After following changes
http://mail-index.netbsd.org/pkgsrc-changes/2014/04/28/msg105057.html
native perl is usable if it satisfies whole dependencies related to perl

I'm very surprised how you deal with
devel/p5-ExtUtils-MakeMaker for system perl and a 2nd
devel/p5-ExtUtils-MakeMaker for pkgsrc perl
I rate the idea itself as dangerous.

In my point of view, I want to use packages requiring perl as a tool.
Not for perl itself, including additional perl modules.
Please look at the diff, only for USE_TOOLS=perl.

I think you understand my concerns regarding reliable packages being
built. And I trust in your senior skills and see where it leads :)

The commit-msg was a bit misleading and you might know ENOTIME issues ;)

But you might agree that the style of p5-IO-Async's Makefile is
optimal for your use-case. Digging ~700 entries in an additional table
in mk/ would complicate that decision.

Post by OBATA Akio
The next step is split runtime and buildtime dependency handling,
to improve cross build support.

Please feel encouraged to go ahead and take my concerns as a
chance to improve ;)

Cheers

[Jens Rehsack]

Post by Edgar Fuß
JR> This is not true anymore. Packager::Utils will do that for (some)one.
That sounds great.

It does - and in many cases it works fine. I have to do some final polishing
and than I can do a first release to pkgsrc developers. wiz@ and joerg@ provided
very useful feedback and I expect that not to much bullshit happens at early
adopters :)

Post by Edgar Fuß
EF> 2. Someone has to re-do that when modules get added to/removed from Core.
JR> See RE: 1
Oh, it automatically inspects all Makefiles and replaces the outdated
information with uptdated one? That sounds even better.

Depends - some information are reused, some are replaced. It's also a 90%
tool - but a "${FAV_SCMCMD} diff" should be reasonable for pkgsrc developers :)

Post by Edgar Fuß
EF> 4. What do you write if you need a module that has been removed from Core?
EF> You can't express a range in PERL5_REQD, do you?
JR> No, in that case always the separate package is referred.
Hm, and if my native Perl still has that module in Core?

Than the pkgsrc one installed into vendor_lib is used anyway. It generates
bullet-prof dependencies - not optimized for size. I'm not sure if that is
the goal ;)

Post by Edgar Fuß
EF> 5. Suppose you have a native Perl 5.40 satisfying all requirements but
EF> lacking the required Foo::Bar module, which is included in pkgsrc's Perl
EF> 5.42 (and also available in pkgsrc as devel/p5-Foo-Bar).
EF> Then, your approach, contrary to the, as I just learned, deprecated
EF> DEPENDS+= {perl>=5.42,p5-Foo-Bar>=1.23}:../../devel/p5-Foo-Bar
EF> practise, will pull in the pkgsrc Perl 5.42, not p5-Foo-Bar.
EF> Is that what is desired?
JR> Currently pkgsrc comes with one perl only.
Yes, but in another part of this thread I was told I had to care about
native (e.g. non-pkgsrc) Perl. I really don't know about that. Do I have
to care about native Perl or not?

Nope, you don't (Compare output of different perl -V when searching for @INC).

perl -MData::Dumper -le 'print Dumper \@INC, $^X'

might enligthen those who fear.

Any pkgsrc package will replace the perl-interpreter and
$^X will always give the FQPN to the interpreter. If you have an application
what runs

system( "perl ..." )

open a ticket against that. This is horribly wrong.

Cheers

[Jens Rehsack]

Post by Thomas Klausner

with you perl5 update, did you mind to revbump all perl-modules

I'm currently doing that. Might take a few more hours though.

and prove each p5-* module against requiring removed modules from core (Module::Build, CGI, ...)?

There's no automation for that yet, so we'll fix these as we find

That's why I offered to do it in next weeks - I'm currently busy with some low-level modules (List::MoreUtils, Params::Util) which should be finished with highest priority to remove stuck in p5 toolchain.

Cheers