Mutex implementation on m68k/netbsd-7?

[John Klos]

Hi,

I have a Mac running mac68k netbsd-7, compiled from yesterday's sources,
with pkgsrc from today, which is failing in databases/db4 like so:

checking for getopt optreset variable... yes
checking for mutexes... UNIX/fcntl
configure: WARNING: NO SHARED LATCH IMPLEMENTATION FOUND FOR THIS
PLATFORM.
configure: error: Unable to find a mutex implementation
*** Error code 1

Stop.
make[2]: stopped in /usr/pkgsrc/databases/db4

Ideas? Suggestions?

Thanks,
John

[David Holland]

Post by John Klos
I have a Mac running mac68k netbsd-7, compiled from yesterday's sources,
checking for getopt optreset variable... yes
checking for mutexes... UNIX/fcntl
configure: WARNING: NO SHARED LATCH IMPLEMENTATION FOUND FOR THIS PLATFORM.
configure: error: Unable to find a mutex implementation
*** Error code 1
Stop.
make[2]: stopped in /usr/pkgsrc/databases/db4
Ideas? Suggestions?

Reading config.log is probably the best starting point... do you know
what it ought to be finding?

[Hauke Fath]

Post by John Klos
I have a Mac running mac68k netbsd-7, compiled from yesterday's sources,
checking for getopt optreset variable... yes
checking for mutexes... UNIX/fcntl
configure: WARNING: NO SHARED LATCH IMPLEMENTATION FOUND FOR THIS
PLATFORM.
configure: error: Unable to find a mutex implementation
*** Error code 1
Stop.
make[2]: stopped in /usr/pkgsrc/databases/db4
Ideas? Suggestions?

Hm. I have a patch to the db4 package for sparc where I ran into the same
problem, years back before my second pair of SM71s went belly-up, and I
moved the installation to a B/W G3 Macintosh. db4's test routines were fine
with these patches, but as nobody else appeared to run into the issue,
and/or care, they still reside on my harddrive.

Why you would run into the issue for m68k netbsd-7, and not earlier,
though, is funny - it's not like the db4 sources have changed much
recently, is it?

hauke


--
"It's never straight up and down" (DEVO)

[John Klos]

Hi,

Post by Hauke Fath
Hm. I have a patch to the db4 package for sparc where I ran into the same
problem, years back before my second pair of SM71s went belly-up, and I
moved the installation to a B/W G3 Macintosh. db4's test routines were fine
with these patches, but as nobody else appeared to run into the issue,
and/or care, they still reside on my harddrive.
Why you would run into the issue for m68k netbsd-7, and not earlier,
though, is funny - it's not like the db4 sources have changed much
recently, is it?

It seems it doesn't work on netbsd-6, either, right now, since I get the
same issue on NetBSD 6.1 (netbsd-6 built at the end of 2013). But
apparently it worked at some point on m68k:

http://ftp.netbsd.org/pub/pkgsrc/current/pkgsrc/databases/db4/README.html

I wonder what changed. It seems that SPARC, VAX, SH3, Alpha, and possibly
others currently don't build. So sure, let's try out those patches of
yours.

Thanks,
John

[John D. Baker]

On Sat, 23 Aug 2014 18:14:08 +0200 Hauke Fath

Post by Hauke Fath
Hm. I have a patch to the db4 package for sparc where I ran into the
same problem, years back before my second pair of SM71s went belly-up,
and I moved the installation to a B/W G3 Macintosh. db4's test routines
were fine with these patches, but as nobody else appeared to run into
the issue, and/or care, they still reside on my harddrive.
Why you would run into the issue for m68k netbsd-7, and not earlier,
though, is funny - it's not like the db4 sources have changed much
recently, is it?

My understanding was that starting with db4-4.8 (and subsequent "dbN")
Oracle changed things so db4 would only build on x86 and ultraSPARC
platforms (apparently powerpc works too). ISTR it was something about
the way file locking was done that eliminated calls to platform-independent
mechanisms in favor of local platform-specific code. (But I may be
wrong and/or injecting too much opinion in this--I have not looked at
the code myself.)

I figured there was nothing for it and have been keeping my "db4" rolled
back to the last version that was buildable on sparc (db4-4.7.25.3 from
pkgsrc-2010Q1, or a date tag just before the update to 4.8.0).

If you have patches to restore current db4 to building on sparc (and
apparently m68k platforms), I'd be happy to use them and even happier
if they could be committed to pkgsrc. (Perhaps even updated for db5
and following?)

Thanks.

[Hauke Fath]

Post by John D. Baker
If you have patches to restore current db4 to building on sparc (and
apparently m68k platforms), I'd be happy to use them and even happier
if they could be committed to pkgsrc. (Perhaps even updated for db5
and following?)

I have uploaded the relevant patches to database/db4 to
<http://la.causeuse.org/hauke/pkgsrc/database_db4_sparc.tar.gz>. They date
back to 2010 and most likely will not apply cleanly to the current db4
package. But they will point to the processor specific primitives that have
to be provided, both for the configure script and for the
"dbinc/mutex_int.h" header.

For lack of round tuits to bring up a sparcv8 machine for testing, I'd much
prefer someone with sparc fu to pick up the patches and run with them.
OTOH, if you can test them and confirm they are working, I'll be happy to
commit.

hauke

--
"It's never straight up and down" (DEVO)

[John D. Baker]

Post by Hauke Fath

Post by John D. Baker
If you have patches to restore current db4 to building on sparc (and

I have uploaded the relevant patches to database/db4 to
<http://la.causeuse.org/hauke/pkgsrc/database_db4_sparc.tar.gz>. They
date back to 2010 and most likely will not apply cleanly to the current
db4 package. But they will point to the processor specific primitives
that have to be provided, both for the configure script and for the
"dbinc/mutex_int.h" header.

I've finally got a tuit of the circular variety to investigate the
patches on db4{8}. The only reason they didn't apply cleanly is because
everything except the sparc-specific parts were already in pkgsrc patches.

I first adapted them as local patches to be applied after the pkgsrc
patches. Actually, I applied the patch to "dbinc/mutex_int.h" manually
as there was an unnecessary change in the original patch. I cleaned up
local-patch versions and saved comprehensive patches that can replace
the ones in "databases/db4/patches/"

As I write, it passed the configure test and is building. I'll need to
rearrange some things to perform a functional test.

I then took a look at "databases/db5" figuring it hadn't changed all
that much. Sure enough, the smaller local patch for "dbinc/mutex_int.h"
(now "src/dbinc/mutex_int.h" in "databases/db5") was the only change
needed and applied cleanly. The local patch for "dist/configure applied
with some offset and I've regenerated a clean local patch and full patch
to replace that in "databases/db5/patches/"

I'll run config/build test of db5 when db48 is finished.

Thanks for the help.

[Hauke Fath]

Post by John D. Baker
I've finally got a tuit of the circular variety to investigate the
patches on db4{8}. The only reason they didn't apply cleanly is because
everything except the sparc-specific parts were already in pkgsrc patches.

[...]

Post by John D. Baker
Thanks for the help.

Well, thanks to you for picking up the ball!

hauke

[John D. Baker]

Post by Hauke Fath

Post by John D. Baker
I've finally got a tuit of the circular variety to investigate the
patches on db4{8}. The only reason they didn't apply cleanly is because
Thanks for the help.

Well, thanks to you for picking up the ball!

Mind you, I don't actually have any sparc-fu, but am mainly cargo-culting
the patches and examining the db4-4.7.25.3 sources for comparison.

The only difference I noticed is that your patch inserts the following
in "dbinc/mutex_int.h":

#define MUTEX_UNSET(tsl) ({ \
__asm__ volatile ("stbar"); \
*(tsl) = 0; })

while the db4-4.7.25.3 version of that file simply has:

#define MUTEX_UNSET(tsl) (*(tsl) = 0)

Lacking sparc-fu of my own, I'd hazard that's a cache-coherency tactic?


Also, the variant of the "asm" reserved word used is different. In your
patch (like the sparc-v9 macros):

#define MUTEX_MEMBAR(x) \
({ __asm__ volatile ("stbar"); })

while in the db4-4.7.25.3 source:

#define MUTEX_MEMBAR(x) ({asm volatile("stbar");})

GCC likes either one, so I guess that's not really an issue.

[Hauke Fath]

Post by John D. Baker
Mind you, I don't actually have any sparc-fu, but am mainly cargo-culting
the patches and examining the db4-4.7.25.3 sources for comparison.
The only difference I noticed is that your patch inserts the following
#define MUTEX_UNSET(tsl) ({ \
__asm__ volatile ("stbar"); \
*(tsl) = 0; })
#define MUTEX_UNSET(tsl) (*(tsl) = 0)
Lacking sparc-fu of my own, I'd hazard that's a cache-coherency tactic?

The 'stbar' mandates strict ordering of memory accesses, see
<http://docs.oracle.com/cd/E19683-01/806-5222/hwovr-17/index.html>.

Post by John D. Baker
Also, the variant of the "asm" reserved word used is different. In your
#define MUTEX_MEMBAR(x) \
({ __asm__ volatile ("stbar"); })
#define MUTEX_MEMBAR(x) ({asm volatile("stbar");})
GCC likes either one, so I guess that's not really an issue.

Unless you invoke it with -ansi, which makes 'asm' illegal. Is there a
general preference within pkgsrc?

hauke