posted by Tony Bourke on Thu 18th Mar 2004 18:16 UTC

"NetBSD on SPARC, Page 2/3"
Compiling Under NetBSD

Compiling applications under NetBSD can be a bit tricky, and there are a few gotchas that you should keep in mind.

In a previous article on compiler optimizations, I discussed the differences between the V7, V8, and V9 SPARC architectures and their impact on performance. You don't need to worry too much about it with NetBSD SPARC64, since the included compiler (2.95.3) builds V9-based 64-bit binaries (-m64 -mcpu=v9) by default, although specifying -mcpu=ultrasparc can help tune the code (although it did cause me some signal 11 problems with GCC 2.95.3, but not 3.3.2). Also, specifying -mcpu for v7 or v8 won't make much difference, since it ends up building a V9 binary anyway (with the performance advantages associated with a V9 build).

Not only does the compiler build for 64-bit by default (-m32 is the default switch for stock GCC, while -m64 is the default for NetBSD SPARC64's included GCC), the build environment is incomplete to even attempt a 32-bit build.

In general, getting software to compile on NetBSD SPARC64 can be pretty tough going. Even if you get something to compile cleanly, it may not work right. Take OpenSSL 0.9.7c for example. When I ran into an SSH issue covered in the next section, I went to compile a newer version of the OpenSSL libraries. The system comes installed with 0.9.6g, but I wanted to go with the 0.9.7 series to be consistent with the other installs so I downloaded the source from the OpenSSL site.

I used -mcpu=ultrasparc as a compile flag, but the compiler would die with a signal 11:

gcc: Internal compiler error: program cc1 got fatal signal 11
*** Error code 1
I used -mcpu=v9, and for some reason it compiled. But while it compiled cleanly, the binary did not work correctly.
netbsd1# apps/openssl speed rsa dsa
To get the most accurate results, try to run this
program when this computer is idle.
Doing 512 bit private rsa's for 10s: openssl in free(): warning: modified (chunk-) pointer.
openssl in free(): warning: modified (chunk-) pointer.
Bus error (core dumped)
netbsd1# 
Which also shot out this kernel error to syslog:
Feb 19 05:27:00  /netbsd: Alignment error: pid=24954 comm=openssl 
dsfsr=00000000:00800005 dsfar=fffffa48:4959cdd isfsr=00000000:00000000 
pc=40508674
I tried using OpenSSL 0.9.6l (and g) downloaded from the OpenSSL site, and while the binary compiled, it gave another type of error:
netbsd1# apps/openssl speed rsa dsa
To get the most accurate results, try to run this
program when this computer is idle.
Doing 512 bit private rsa's for 10s: 242 512 bit private RSA's in 10.05s
RSA verify failure.  No RSA verify will be done.
24940:error:0407006A:rsa routines:RSA_padding_check_PKCS1_type_1:block type is not 01:rsa_pk1.c:100:
24940:error:04067072:rsa routines:RSA_EAY_PUBLIC_DECRYPT:padding check failed:rsa_eay.c:459:
I might be tempted to presume that this issue has something to do with the code not being 64-bit clean, but I know it is, since I compiled the very same source under Solaris 9 with both GCC and Sun's compiler as a 64-bit working binary.

I poked around and realized when I ran ./config, it configured the system for NetBSD-sparc, and not NetBSD-sparc64:

./config 
Operating system: sparc64-whatever-netbsd
Configuring for NetBSD-sparc

While there is a FreeBSD-sparc64 and an OpenBSD-sparc64 definition, there is no NetBSD-sparc64 definition in either the 0.9.7 or 0.9.6 branch's Configure script. So while the code is clean, it looks like perhaps the configure script isn't setting up the right environment. I even tried using both the FreeBSD-sparc64 and OpenBSD-sparc64 build environment to compile, and while they both produced binaries, they still exhibited the same behavior.

I went into the pkgsrc directory, and found the openssl directory. Operating much like the ports envrionment in FreeBSD, it automatically downloaded the OpenSSL 0.9.6l source and compiled cleanly from source, applied patches (which included a NetBSD-sparc64 definition) and produced a working OpenSSL binary.

I used the same NetBSD-sparc64 defination from the OpenSSL 0.9.6g patch and applied it to the OpenSSL 0.9.7c source, and it finally produced a working binary.

The pkgsrc tree

I found the pkgsrc environment, which is downloaded as a tar ball from NetBSD's site or mirrors, to be very important for getting various applications compiled. While I was able to get tcsh 6.12.00 and OpenSSH 3.7.1p2 to compile on my own, I was not able to get a working copy of OpenSSL compiled without it. The pkgsrc environment also built me a working GCC 3.3.2 specially modified to produce 64-bit binaries by default.

Packages that are compiled from pkgsrc are usually installed under /usr/pkg.

Unfortunately, even pkgsrc couldn't get mysql 4 to compile, so I don't have much in the way of benchmarks for this system (see Appendix A for details).

OpenSSL Mystery

Here's one thing I can't figure out: How the openssl package distributed with NetBSD ended up being so slow.

I first realized something was wrong when logging into the system via SSH took about 3 seconds to get a prompt back (an issue I wrote about in my GCC optimization article).

When I ran the openssl speed test on the included OpenSSL binary, I got results that were slower than I've ever been able to produce in any environment on my Ultra 5, including V7 in 32-bit Solaris.

OpenSSL 0.9.6g 9 Aug 2002
built on: NetBSD 1.6.1
options:bn(32,32) md2(int) rc4(ptr,int) des(ptr,risc1,16,int) blowfish(idx) 
compiler: gcc version 2.95.3 20010315 (release) (NetBSD nb3)
sign verify sign/s verify/s
rsa 512 bits 0.0248s 0.0022s 40.2 449.9
rsa 1024 bits 0.1279s 0.0076s 7.8 131.7
rsa 2048 bits 0.9217s 0.0276s 1.1 36.2
rsa 4096 bits 6.4647s 0.0928s 0.2 10.8
sign verify sign/s verify/s
dsa 512 bits 0.0224s 0.0281s 44.7 35.5
dsa 1024 bits 0.0750s 0.0927s 13.3 10.8
Take RSA verify/second for 512 bits for instance. In the 0.9.6g that came with the system, the results was 449.9. When I compiled 0.9.6l on my own with -mcpu=v9, the result for RSA verify/second 512 bits was around 1,100.

During my Solaris tests, I compiled 0.9.7c with -mcpu=v7, and it produced a 512-bit RSA verify result of 627, which was the slowest result I could produce and it was still faster than NetBSD's binary. So how NetBSD managed to build a VV9-based binary so slow is a mystery to me.

Bare-Bones

The default installation of NetBSD is pretty baren, and doesn't even include a Perl binary. Since there are no pre-compiled packages available for SPARC64 on the NetBSD site, you'll have to compile them on your own. Using pkgsrc is probably your best bet for getting applications to compile.

File System

NetBSD comes with the standard FFS. Enabling soft updates can help file system performance, particularly when untaring a file. To do so, edit /etc/fstab and add "softdep" after the rw option.

/dev/wd0f   /usr  ffs  rw,softdep  1  2  
After a reboot, the system will come up with those file systems set for soft updates.

Conclusion

The NetBSD SPARC64 feels fairly incomplete to me. The lack of a 32-bit userland is a significant hindrance to its usability. I would imagine that the SPARC 32-bit port for sun4/m/d/c systems is much more solid, but the SPARC64 port is difficult to recommend.

Even if you're an avid user of NetBSD and are very comfortable in that environment, you'll likely still find this particular port a challenge and would need to put in a lot of work to make it a useful system. Compiling requires a higher level of aptitude on the ins and outs of a build environment.

Still, for all its weaknesses, it's not a matter of shitty technology or bad craftsmanship, it seems to be more of a lack of attention than anything else. Someone could possibly make a name for themselves by taking on the task of making the SPARC64 port of NetBSD 1.6.2 more complete.

Table of contents
  1. "NetBSD on SPARC, Page 1/3"
  2. "NetBSD on SPARC, Page 2/3"
  3. "NetBSD on SPARC, Page 3/3"
e p (0)    29 Comment(s)

Technology White Papers

See More