OSNews:

OSNews: http://www.osnews.com/story/17977/The_Hoard_Scalable_Memory_Allocator_3_6_2_Released Exploring the Future of Computing en-us Copyright 2001-2012, David Adams adam+nospam@osnews.com Wed, 15 Feb 2012 09:16:07 GMT http://www.osnews.com/images/osnews.gif OSNews.com http://www.osnews.com how to know i'm using hoard? http://www.osnews.com/thread?243245 http://www.osnews.com/thread?243245 i have downloaded and build it on my box, and setup LD_PRELOAD envrion variable, but how did i know that i'm using libhoard instead of standard ones? Sat, 26 May 2007 05:42:00 GMT donotreply@osnews.com (gerryxiao) Comments RE: how to know i'm using hoard? http://www.osnews.com/thread?243253 http://www.osnews.com/thread?243253 1. Use man ld.so to get some information whether hoard is loaded at all.
2. Use a debugger and trace a malloc. You should at least see a hoard method in the traceback. Sat, 26 May 2007 07:30:00 GMT donotreply@osnews.com (Ponto) Comments RE: how to know i'm using hoard? http://www.osnews.com/thread?243283 http://www.osnews.com/thread?243283 On Solaris, just run: pldd

Where is the PID of the process you want to make sure is runing with the Horde library. pldd is just like ldd, but is for use against running processes rather than just binaries.

I don't know if Linux or other OSes provide a tool for doing this.Edited 2007-05-26 14:46 Sat, 26 May 2007 14:46:00 GMT donotreply@osnews.com (Elektronkind) Comments RE: how to know i'm using hoard? http://www.osnews.com/thread?243284 http://www.osnews.com/thread?243284

i have downloaded and build it on my box, and setup LD_PRELOAD envrion variable, but how did i know that i'm using libhoard instead of standard ones?

Compile your program. Then run "ldd" on it to see which (dynamic) library it is linked to:
> ldd myapp Sat, 26 May 2007 15:01:00 GMT donotreply@osnews.com (big_gie) Comments RE[2]: how to know i'm using hoard? http://www.osnews.com/thread?243285 http://www.osnews.com/thread?243285 Compile your program. Then run "ldd" on it to see which (dynamic) library it is linked to:
> ldd myapp

i'm not using it for developing, just want some programs in my box apply libhoard.so to improve performance

there aren't any programs in linux which have the same functions as pldd in solaris, but i'm not sure

pmap seems working in linux
pmap Edited 2007-05-26 15:19 Sat, 26 May 2007 15:14:00 GMT donotreply@osnews.com (gerryxiao) Comments RE[3]: how to know i'm using hoard? http://www.osnews.com/thread?243286 http://www.osnews.com/thread?243286

i'm not using it for developing, just want some programs in my box apply libhoard.so to improve performance

So you want to replace the existing library with that one? Interesting thing. Can't help for that though

Sat, 26 May 2007 15:27:00 GMT donotreply@osnews.com (big_gie) Comments RE[4]: how to know i'm using hoard? http://www.osnews.com/thread?243289 http://www.osnews.com/thread?243289 i'm not using it for developing, just want some programs in my box apply libhoard.so to improve performance
So you want to replace the existing library with that one? Interesting thing. Can't help for that though

libhoard.so is a dynamic link share lib file which includes much same functions suchas malloc(),free() etc with standard GNU c lib, if LD_PRELOAD variable has been setup, any program depending on share libs first look at libhoard.so, if found any functions which is needed , it will not call standard GNU c share lib functions Sat, 26 May 2007 15:49:00 GMT donotreply@osnews.com (gerryxiao) Comments RE[2]: how to know i'm using hoard? http://www.osnews.com/thread?243290 http://www.osnews.com/thread?243290 "I don't know if Linux or other OSes provide a tool for doing this. "

Besides Solaris, I've seen the pldd command only on HP-UX. Sat, 26 May 2007 15:59:00 GMT donotreply@osnews.com (Doc Pain) Comments Comment by bnolsen http://www.osnews.com/thread?243299 http://www.osnews.com/thread?243299 another allocator is called "nedmalloc" which claims to thrash horde in real world applications.

This article reminded me. When I get back to work on Monday or Tuesday I'll test horde and nedmalloc on our 8 core dell workstation with our threaded ortho rectification and see if there's any noticeable performance improvements. Sat, 26 May 2007 17:33:00 GMT donotreply@osnews.com (bnolsen) Comments Also see Google's tcmalloc and Sun's umem http://www.osnews.com/thread?243309 http://www.osnews.com/thread?243309 http://goog-perftools.sourceforge.net/doc/tcmalloc.html

https://labs.omniti.com/trac/portableumem/

(Emery Berger seems pretty good at marketing for a university professor.) Sat, 26 May 2007 20:19:00 GMT donotreply@osnews.com (Wes Felter) Comments RE: Comment by bnolsen http://www.osnews.com/thread?243596 http://www.osnews.com/thread?243596 Hi,

I just downloaded and tried nedmalloc against a particularly brutal benchmark called "larson" on a dual processor Linux box. With Hoard, its throughput was 790,614 memory operations per second, while with Nedmalloc it was 188,706 ops/sec. Compare to GNU libc (the default allocator), whose throughput was 192,485 ops/sec. In short, Hoard outperforms Nedmalloc by more than a factor of 4X.

I'd be interested to hear what happens with your application.

Best,
-- Emery Berger Mon, 28 May 2007 00:54:00 GMT donotreply@osnews.com (EmeryBerger) Comments RE: Also see Google's tcmalloc and Sun's umem http://www.osnews.com/thread?243598 http://www.osnews.com/thread?243598 Hi,

Thanks for the pointers.

I ran the "larson" benchmark with both of the allocators you pointed out.

libumem: shockingly poor performance. 76,110 ops per second -- Hoard is 10X faster. Which makes me unsure whether this port is true to the original libumem algorithm (from Bonwick).

tcmalloc: for this benchmark, it outperforms Hoard by about 20%, but also consumes 40% more memory (172M versus 122M). In addition, tcmalloc does some things that Hoard doesn't do because they can seriously harm performance. I tried another benchmark ("cache-thrash") which tests whether an allocator can contribute to false sharing of cache lines (really bad for performance). On this benchmark, Hoard performed around 6.5X faster.

Best,
-- Emery

P.S. I'm going to consider 'good at marketing' as a compliment...

. Mon, 28 May 2007 01:17:00 GMT donotreply@osnews.com (EmeryBerger) Comments RE[2]: Comment by bnolsen http://www.osnews.com/thread?243860 http://www.osnews.com/thread?243860 I got to work and ran some tests.

Machine: 8 core clovertown 1.6GHz, 8GB ram. gentoo ~amd64

Process:
threaded ortho rectification.
3 pipelines, 8 threads each
- Input IO
- Ray production & solid intersection
- Pixel Rasterization and output tiling.

Test dataset: 210MP input image (smallest I coul find)
Run each data set twice, take second timing.

Process is 64bit optimized. Memory usage numbers are somewhat misleading because of the kernel paging, but hover between 3.2GB & 3.8GB regardless of the allocator.

Interesting observation:
During libhoard runs occasionally the system levels would dramatically spike (seen in both gkrellm2 & htop)

Timing:

default gcc allocator
real 1m19.477s
user 9m10.210s
sys 0m8.993s

hoard:
real 1m9.135s
user 7m23.508s
sys 0m37.102s

tcmalloc:
real 1m2.222s
user 6m58.014s
sys 0m5.032sEdited 2007-05-29 15:28 UTC Tue, 29 May 2007 15:24:00 GMT donotreply@osnews.com (bnolsen) Comments RE[3]: Comment by bnolsen http://www.osnews.com/thread?243872 http://www.osnews.com/thread?243872 nedmalloc numbers:

real 0m58.323s
user 6m11.111s
sys 0m11.933s

Looks like the nedmalloc guy has the most realistic claims. Tue, 29 May 2007 16:48:00 GMT donotreply@osnews.com (bnolsen) Comments RE: Comment by bnolsen http://www.osnews.com/thread?243948 http://www.osnews.com/thread?243948 Yet another test case:

Normal system accumulator.
Equations are objects which are assembled into a normal system and then solved via SVD.

Tons of 3x3 and 4x4 matrices.
Single threaded application.

Base case:

real 0m3.596s
user 0m3.560s
sys 0m0.036s

hoard:

real 0m4.112s
user 0m4.076s
sys 0m0.036s

nedmalloc:

real 0m3.692s
user 0m3.664s
sys 0m0.028s

tcmalloc: CORE DUMP (hehe) Tue, 29 May 2007 23:09:00 GMT donotreply@osnews.com (bnolsen) Comments RE[2]: Comment by bnolsen http://www.osnews.com/thread?244530 http://www.osnews.com/thread?244530 Hi -

Could you please send me your benchmarks? There is some sort of performance issue with 64-bit that I am trying to resolve.

Thanks,
-- Emery Fri, 01 Jun 2007 00:58:00 GMT donotreply@osnews.com (EmeryBerger) Comments