Xserve Outside the Reality Distortion Field

The release of Apple’s latest hardware offerings this month has caused quite a stir among Apple loyalists. This is only Apple’s second foray into creating a truly dedicated server machine, the first of which being the “Network Server 500/700”, which Apple sold in 1995 running AIX. The question on everyone’s mind now is going to be what this means for Apple’s prospects as a server platform. Certainly everyone agrees that the new machine looks nice, but how will it on the duties it will be required to perform. Furthermore, what kind of price/performance ratio is Apple offering when compared to the current group of servers.

Editorial notice: All opinions are those of the author and not necessarily those of osnews.com


Even Steve Jobs, the master of spin, states that the Xserve
device is not meant to be a competition with high end servers. In fact no one PC manufacturer puts up hardware that competes directly in the world of big iron servers. Mr. Jobs also is making sure that this isn’t perceived as direct competition with middle-tier server platforms. These would be the multi-processor, multi-unit machines offered by other PC manufacturers such as Dell or Compaq. Instead, Jobs made it clear that this was Apple’s first, and humble, push into the entry level server market.

Neither Apple nor Apple’s faithful are naive enough to think that this one device will be able to please everyone. Just as in the design of every computer, Apple took trade-offs of performance and cost. Furthermore, Apple always has the stigma of not being an x86 platform in the minds of some people. What I am going to try to investigate is how Apple’s offerings stack up against
similar competitors in both individual components, software and in price.

To create a fair sample, I went to other major manufacturer sites–IBM, Compaq/HP, Dell and Sun– and looked at all of their 1U rack mounted server offerings. I then went to Apple’s site and pulled down their three recommended server configurations. Next, I attempted to recreate the offerings to match Apple’s for each of
the three configurations. This was the basis of the
price/performance chart at the end. Of course, in and of itself this chart is meaningless without first looking at each of the individual components.

Processor

Apple of course is using the venerable PowerPC G4 running at its top speed of 1 GHz as the basis of the server. Each configuration is capable of housing two of these processors, however the low end configuration has only one at purchase time. The PowerPC chip itself is surrounded by benchmark controversy. How much power is the user going to get out of the PowerPC G4 when compared to the same offerings on the other platforms? This question is going to depend on how memory starved the G4 was with
the old style memory bus. If the bottleneck was in the processor, and not in the memory system, then we can reasonably expect these G4’s to perform on par with a mid-1.5GHz Pentium chip on normal operations, and maybe up to a 2GHz Pentium on certain optimized floating point operations. If there truly was a memory bottleneck in the old memory system, then the performance will be even higher. How high will be a question for the end users and
benchmarks to determine.

In the case of IBM, Compaq and Dell, the processor of choice is the Intel Pentium III. None of the machines use the Pentium 4 or the Xeon version of each of these chips. The Xeon version of Intel’s Pentium line basically adds larger caches to the chips, which allow them to keep more data readily available, without having to go out to the main system memory for data. The choice of processors in this case probably has to do with the heat dissipation requirements of having one Xeon chip, much less two, in the confined space of a 1U rackmount. If historical benchmarks are still accurate, these chips should give about the same performance as the 1GHz G4, however in memory intensive operation the 2MB cache on the Xserve’s G4 may give it an extra performance boost. Also, as stated previously, the new memory system may make the G4 more effective than in previous benchmarks.

Sun’s offerings of course use the UltraSPARC processor as the basis for its system. The UltraSPARC IIe is the chip of choice for sun’s low end systems. This SPARC processor is hardly the same chip as the UltraSPARC III, which Sun puts in their high end machines. On top of this it runs at a relatively slow 500MHz. This puts Sun near the bottom of the heap on the processor power front. Although the benchmarks aren’t as well studied in the
Macintosh and PC circles, it has been stated that the UltraSPARC line is not as powerful as the PowerPC or Pentium lines in most operations.

Since the G4 in the dual processor Apple workstations, was recently defeated in multimedia benchmarks by a dual 1.5GHz Athlon, it will be interesting to see how the Xserve will benchmark against the single and dual processor Pentiums with their new memory subsystem. This will be especially telling with the very generous L3 cache in these systems as well. In the end, the new memory
system could put the PowerPC back on par with current generation Pentiums and Athlons. Once again, only time will tell how much benefit the new memory system will give the Xserve.

Memory

One of the bigger sources of controversy in the Xserve is the memory system being used. With only one exception, the low end IBM web appliance, every single 1U system uses PC133 ECC SDRAM. For the uninitiated, this is error correcting RAM running at half the bandwidth of the RAM in Apple’s system. Apple’s RAM is PC2100 DDR RAM. This actually has the same clock speed as the PC133 RAM, but data transactions can occur twice per cycle, not just once.
This is where the name “dual data rate” comes from. What this translates to is much higher performance than the SDRAM. However, this is not error correcting memory. This could be a relatively sore point for Apple to some potential customers. In some usages, large scale databases for example, it is not acceptable that the memory isn’t error correcting. Time will tell if this was a wise decision on the part of Apple.

In terms of raw performance there is a great deal of benefit to be gained from using the DDR RAM. In the article “Inside RAM” on ExtremeTech.com, Jimmy Rimmer points out that DDR RAM not only provides a good deal of improvement on latency compared to SDRAM, but it also provides nearly double the bandwidth. On top of this, in real world usage,
the DDR RAM can be almost as fast, or faster than the Rambus RAM in high end Xeon workstations. Performance-wise, this could provide a significant performance advantage to the Xserve on CPU intensive tasks. This trade-off will be more than acceptable for render farms and other users, who don’t truly require ECC RAM.

Disk I/O

Even bigger than the issue of RAM usage is Apple’s decision to offer the system with only Ultra ATA disk interfaces. Almost all the other selected systems, and most certainly the high end systems, all used Ultra160 SCSI drives. It is without question that the ATA disk interface is slower than the SCSI disk interface. On top of this, the drives themselves are only 7200 RPM drives, not the 10,000 RPM drives used in the competitor’s 1U servers. The question is how this will affect the Xserve’s
overall performance.

Apple has addressed this issue in a few different ways. The first thing to notice is that the SCSI systems themselves only have one SCSI bus that all their drives attach to. The Xserve on the other hand has each ATA drive attached to its own interface. According to simple arithmetic, the combination of four ATA interfaces should provide a theoretical maximum data throughput of 266MB/s, compared to the Ultra160 SCSI maximum bandwidth of 160MB/s. On paper that sounds nice, but how the actual maximum translates into actual performance remains to be seen. If it works, it could provide a truly impressive disk I/O performance advantage for the Xserve, especially since the four disks act as one large disk via software RAID software built into the OS.

The first possible problem will occurs with the raw disk access speed itself. After a request for data, the Ultra ATA drives take a performance penalty compared to SCSI drives in their data access. A brief look at the statistical data compiled for all the fastest hard disks at StorageReview.com, show that even the fastest ATA drives pull in dismal performance compared to the top end SCSI drives. This is also true for total disk bandwidth and other benchmarks. The data also shows one other interesting thing however, Ultra160 SCSI drives don’t always beat ATA drives, which
are almost always significantly less expensive per megabyte. It is therefore possible, that the Xserve will be able to reach the same transfer rates as its SCSI-based cousins. The determining factor in this case will be the drive manufacturer used for the servers. It is therefore theoretically possible that the SCSI-based servers could turn in lower disk I/O performances than Xserve’s ATA drives. Again, only time and actual use will tell us the rest of that story.

Network Bandwidth

With only one exception, the second generation Compaq servers, are the gigabit ethernet options standard outside the Xserve. In every other case, the gigabit ethernet is installed on the standard PCI buses of the systems, or it is not an option at all. This puts the Xserve at the top of the list in terms of network bandwidth. The Xserve has two gigabit ethernet ports, standard. One is directly integrated into the motherboard. The other is installed via the high speed 4xAGP/PCI bus slot. This gives the
Xserve more network bandwidth potential than any other machine being reviewed. In some cases, such as the IBM machines, the systems could not be configured with more than one gigabit ethernet ports, however these systems generally had two on-board 100 megabit ethernet ports. The Sun server has by far the worst network performance, with no option for gigabit ethernet, and only two 100 megabit ethernet ports.

Software

Apple is shipping the Xserve pre-configured with MacOS X Server and all the necessary software to begin file serving, web server et cetera. Later this year Oracle will be done porting their database software to OS X, which will add high quality industry standard database serving to the list of tasks the Xserve can perform. Because MacOS X is Unix based, the underlying system has
all the networking and file system performance which goes along with a Unix operating system. Apple has also packaged a software RAID solution to allow the operating system to see all the disks stored in the Xserve as one large hard disk. This offers all the standard RAID options that is typical of the standard hardware RAID systems. From a server software and operating system
standpoint, Xserve gives the user quite a bit of quality software to work with.

Along with the operating system, the Xserve will have remote
server management software. This will allow the operator to
determine the operational status of each of the nodes, and each of
the drives in those nodes, for the entire cluster of Xserves on
the same network. This remote monitoring system is assisted by
built in hardware monitors, which can help predict future problems
on each of the nodes. This makes managing the cluster easier,
however it doesn’t help with controlling the distribution of tasks
on a cluster of Xserves. Nothing in the documentation says
anything about automatic load balancing in the operating system,
or in any of the built in standard software packages.

The software situation on each of the other machines is
variable. By default, the Sun machines have Solaris 9 and
services associated with Solaris. This includes standard Unix
utilities, such as Apache or POP mail. However, this doesn’t
include different windows connectivity software, or media
streaming software, as on the XServe. In the case of the
Intel-based servers, the software is configurable. In many cases,
even if an operating system is purchased with the server, it is
not installed by the manufacturer. This heavily impacts the
instant usability of the other servers in the list.

The choice of software also is a problem when it comes to
pricing the systems. Because Apple ships OS X with unlimited
simultaneous users, the cost is fixed at the price of the server.
Since Apple makes both the operating system and the hardware, the
cost is the same as Sun’s price–zero. However, in the case of
the Intel servers, the operating systems come with user
constraints on them. This heavily impacts the system cost.
Operating system costs, even for many non-Windows options, can be
anywhere between $700 and several thousands of dollars. This can
translate into heavy cost penalties later on.

The price issue is what burns people a lot about Apple
Computer. Despite all the “industrial design” which piques many
people’s interests, the power of the buck rules in most purchasing
cases. Assuming that Apple’s performance benchmarks come up to
their claim, which puts their capabilities at least on par or
better than the competitors listed here, how much do the systems
cost?

As stated in the beginning of the review, the three recommended
configurations were used directly from Apple’s web site. In
essence there is the entry level model, which only has one
processor, one hard disk drive, and 256MB of RAM. Then there is
the mid-range version, which doubles the number of processors and
RAM. Lastly, there is the fully decked out Xserve, with 2GB of
RAM and four of the biggest ATA drives available. The competitor
systems were reviewed with similar configurations.

Of course there are issues with such a direct comparison. The
first point of contention occurs with the size of the hard disks.
The cost per gigabyte of SCSI drives is many times that of Ultra
ATA drives. This extra cost is one of the reasons why Apple chose
not to use them. However, the performance of that more expensive
drive can be, as stated early, twice that of an ATA drive. The
actual drives used are not compared against the tabulated data or
anything like that in this article however, since the hardware
isn’t available for direct testing. Therefore to keep things fair,
multiple configurations are presented for each competitor machine
that uses SCSI drives. The first price is for the standard size
drive, generally 18GB, the other price is for a comparable storage
capacity.

Another possible problem comes in with software and services
bundling. Especially in the case of IBM, the number of potential
service contract combinations is enormous. However, in some
cases, service contracts are built into the standard price.
Therefore, service contracts are not built into the price of the
systems. The software services available were not directly
tabulated for all systems, but most claim server management and
basic server software for e-mail, web hosting, et cetera. How
Apple’s bundle compare is not directly comparable from a quick
review of the descriptions.

As stated in the software section, the cost of operating
systems can double our triple the cost of the server. Therefore,
no operating system option was chosen for systems that did not
bundle the operating system for free. This applies to almost all
the Intel-based machines. In this case, it should be expected that
a Windows based implementation will require at least an additional
$700. This is the cheapest the Windows2000 operating system
options come. Those who would rather use Linux should know that
not all Linux operating system bundles are free either. Some may
feel this gives a distinct disadvantage to Apple, since all the
operating systems are included, however I didn’t want
potential operating system costs skewing the numbers. This
is especially true since the user could spend the time and put it
on for free later.








Entry Level Configuration
Name Company Processor RAM HD(GB) Ethernet
(100/1000)
OS Price
US$
Xserve Apple 1-1GHz PPC 256 60(ATA) 0/2 MacOSX
Server
2999
xSeries 300 IBM 1-1GHz PIII 256 18GB(SCSI) 2/1 None 1813
xSeries 300 IBM 1-1GHz PIII 256 72.4GB(SCSI) 2/1 None 2562
xSeries 330 IBM 1-1.26GHz PIII 256 18GB(SCSI) 2/2 None 2374
xSeries 330 IBM 1-1.26GHz PIII 256 72.4GB(SCSI) 2/2 None 3197
xSeries WebAppliance IBM 1-950MHz Celeron 256
100Mhz
80GB(ATA) 2/0 Linux 2728
ProLiant DL 360 Compaq 1-1.26GHz PIII 256 18GB(SCSI) 2/1 None 3106
ProLiant DL 360 Compaq 1-1.26GHz PIII 256 72.4GB(SCSI) 2/1 None 3881
ProLiant DL 360 G2 Compaq 1-1.4GHz PIII 256 18GB(SCSI) 0/2 None 2728
ProLiant DL 360 G2 Compaq 1-1.4GHz PIII 256 72.4GB(SCSI) 0/2 None 3503
ProLiant DL 320 Compaq 1-1.13GHz PIII 256 80GB(ATA) 2/1 None 2553
Fire V100 Sun 1-500MHz UltraSPARC-IIe 256 80GB(ATA) 2/0 Solaris 8 1648
Netra X1 Sun 1-500MHz UltraSPARC-IIe 128 80GB(ATA) 2/0 Solaris 8 1223
PowerEdge 1650 Dell 1-1.4GHz PIII 256 18GB(SCSI) 0/2 None 2474
PowerEdge 1650 Dell 1-1.4GHz PIII 256 72.4GB(SCSI) 0/2 None 3123

The entry level Xserve configuration is as bare bones as Apple
lets the customer get. This system comes in costing nearly $3000.
Dell’s offering only beat the price of the Apple system by
offering less disk space. The one Compaq system that manages to
beat the price only does so with one third the disk space. The
IBM systems all mostly beat the price as well, however none of the
lower priced options are have an equivalent processor capability,
nor network capability. Furthermore, the total hard disk capacity
is not always up to the same level as the Apple version. In the
end the none of the Intel systems came configured with software,
which in the case of the minimum Windows configuration,
makes each of them more expensive than the Apple system.

The surprise showing in this case comes in with the Sun
servers. Both of these servers are priced far below any of the
Intel or Apple systems. It should be noted that the Sun systems
use an ATA interface, as does the Xserve, however they don’t offer
any option for gigabit ethernet. This will put their network
throughput far below any of the other systems in the entry level
specification for network bandwidth intensive operations.
Considering they cost half as much as the next less expensive
system, they may still be decent deals.

In this case the SCSI versus ATA argument, assuming that the
manufacturers used the high end SCSI drives, will work against
Apple and Sun. The real advantage to fully independent ATA buses
for each drive will only be apparent once more than one drive is
used. Once again however, as the drive benchmarks show, many
times, the ATA drives can hold their own against the SCSI drives,
even if they never finish up on top.

The final conclusion, without any benchmarks, is that the Apple
systems will only cost less than their Intel counterparts with
similar capabilities only if Windows is used as the operating
system. In the case of Linux, it may be possible to keep the cost
of the final configuration as low as stated here, which can offer
a savings of up to $500 against the similarly equiped Xserve.
However, when factoring in $500 of savings, factor in the cost per
hour to get the system up and running in the end. That could be
on the same scale. In the end, without benchmarks, I’d say it is
a draw with the price advantage leaning towards Apple for those
people who are looking for totally pre-configured system. For
more do-it-yourself IT people, they can easily find a better
priced Intel-based system.


Mid-Range Configuration
Name Company Processor RAM HD(GB) Ethernet
(100/1000)
OS Price
$US
Xserve Apple 2-1GHz PPC 512 60GB (ATA) 0/2 MacOSX
Server
3999
xSeries 330 IBM 2-1.4GHz PIII 512 18GB(SCSI) 2/2 None 4293
xSeries 330 IBM 2-1.4GHz PIII 512 72.4GB(SCSI) 2/2 None 5116
ProLiant DL 360 Compaq 2-1.26GHz PIII 512 18GB(SCSI) 2/1 None 12764
ProLiant DL 360 Compaq 2-1.26GHz PIII 512 72.4GB(SCSI) 2/1 None 13433
ProLiant DL 360 G2 Compaq 2-1.4GHz PIII 512 18GB(SCSI) 2/0 None 12231
ProLiant DL 360 G2 Compaq 2-1.4GHz PIII 512 72.4GB(SCSI) 2/0 None 14556
PowerEdge 1650 Dell 2-1.4GHz PIII 512 18GB(SCSI) 2/0 None 3018
PowerEdge 1650 Dell 2-1.4GHz PIII 512 72.4GB(SCSI) 2/0 None 3667

Apple’s mid-range server takes the basic server configuration
and adds an additional processor, and doubles the memory to 512MB.
To be a competitor in this category, the manufacturer had to at
least be able to provide a dual processor version of their
systems. This caused IBM’s 300 series and web appliance servers to
drop out, since none of those systems could support additional
processors. Sun is also noticeably absent from this, and future
lists as well. Unfortunately their 1U blades apparently can only
be configured with single processors. The next step up for them
is a multi-unit sized device which can support many more than two
processors. There is nothing between Sun’s low end and high end
systems. For the remaining systems, the same rules applied in
terms of hard disk and memory matching as was stated above.

The first thing to note is the outrageous pricing of the Compaq servers in this and future categories, which is literally more than twice as much as the next most expensive machines. I can not see any reason for their machines to start at over $12000–without any software installed no less! Back in the realm of reasonable pricing lie the Apple, IBM and Dell machines. Xserve
easily beat the IBM machine pricing for both hard disk
configurations. Once again, this IBM system has no operating system installed, and the gigabit ethernet had to be added by filling up the PCI bus.

The PowerEdge machines by Dell show very strong pricing
compared against the Apple machine. Without the operating system price, both hard disk configurations come in less than the Xserve. Since the machine that matches the capacity is using two SCSI hard disks, it’s hard to imagine that the disk I/O of the Xserve would pose a challenge to the PowerEdge server. Even if the cost of the
operating systems is included, the lower capacity hard drive configuration of the PowerEdge server is cheaper than Xserve. This puts the PowerEdge server as the price/performance champion of these machines in the mid-range category, with Apple’s Xserve coming in a close second.


High-End Configuration
Name Company Processor RAM HD(GB) Ethernet
(100/1000)
OS Price
US$
Xserve Apple 2-1GHz PPC 2024 480(ATA) 0/2 MacOSXM
Server
6849
xSeries 330 IBM 2-1.4GHz PIII 1536 72.4GB(SCSI) 2/2 None 6114
xSeries 330 IBM 2-1.4GHz PIII 2560 72.4GB(SCSI) 2/2 None 7094
xSeries 330 IBM 2-1.4GHz PIII 1536 146.8GB(SCSI) 2/2 None 7214
xSeries 330 IBM 2-1.4GHz PIII 2560 146.8GB(SCSI) 2/2 None 8194
ProLiant DL 360 Compaq 2-1.26GHz PIII 2048 72.4GB(SCSI) 2/1 None 19259
ProLiant DL 360 Compaq 2-1.26GHz PIII 2048 146.8GB(SCSI) 2/1 None 21470
ProLiant DL 360 G2 Compaq 2-1.4GHz PIII 2048 72.4GB(SCSI) 2/0 None 22389
ProLiant DL 360 G2 Compaq 2-1.4GHz PIII 2048 146.8GB(SCSI) 2/0 None 24600
PowerEdge 1650 Dell 2-1.4GHz PIII 2048 146.8GB(SCSI) 2/0 None 5915
PowerEdge 1650 Dell 2-1.4GHz PIII 2048 217.2GB(SCSI) 2/0 None 6716

Apple’s high end Xserve configuration has everything maxed out. Like the mid-range configuration, this system has two 1GHz PowerPC G4 processors. The system has the full amount of RAM it can currently support, 2GB, and every drive bay full of 120GB Ultra ATA drives. This puts the total storage size of this unit at 480GB, or almost half a terabyte. Because there are four drives for the RAID software to stripe across, the tremendous bandwidth
Apple is claiming may be possible in this configuration. The same competitor machines that were in the mid-range line are used, but the memory and hard disk storage is now update to reflect the new storage capacity of the Xserve.

Once again Compaq’s offering are priced outrageously compared to any of the other machines listed here. Their prices start at almost $17000, and end at almost $25000. No other manufacturer price is close to this, and from the spec, there is nothing special about the Compaq boxes this time either. The Dell and IBM machines once again are coming in less than the corresponding Xserve in many cases. One problem that exists in this case is the
fact that none of these machines allow for a configuration that stores 480GB. The most that can be squeezed into the Intel boxes is 147GB-219GB. This is due to the fact that SCSI hard disks don’t come in a 120GB capacity, and none of these systems can take four drives. The closest one to this is the PowerEdge server, that can take three 73GB drives.

A second problem occurs with the IBM xServe machines, because memory has to be added in banks of two. Therefore, the memory can only be 1.5GB or 2.5GB, since there is already half a gigabyte of memory installed, and that is not changeable. I therefore, have machines with both configurations. Because the Xserve has half a
terabyte of disk space, I started the price comparison with the 73GB versions, and went from there. The IBM machine, with comparable memory and a fraction of the hard disk storage space and bandwidth, costs a few hundred dollars less. The similarly equipped Dell is nearly $1000 less. When these unit’s hard disk capacities were maxed out, the IBM prices were higher than the Xserve, in some cases significantly. The Dell PowerEdge machine, with more hard disk space than the IBM, but less than half the
storage space of the Xserve, is practically the same price as the Xserve. Again however, there is no software cost in the any of the IBM or Dell prices.

If the Apple RAID system meets up to their hype, the clear winner in this case is the Xserve, for both total price and performance. In this category, the Xserve, even without the external RAID system, could be a serious contender to the Intel-based 1U servers

Apple’s first serious foray into the server world definitely have some controversial design decisions. The impact of these will be determined once these units get into the field. From a price standpoint, the Xserve shows up reasonably close to its Intel brethren, and in many cases surpasses the cost effectiveness of the Intel machines. From a performance standpoint, the Xserve
should certainly be able to holds its own in many cases, and if Apple’s statements are verified, it even will surpass the performance of these Intel based servers on all the major tests. The Xserve can easily be a contender in the low end, low profile server market.

More important than this, is some of the technology which Apple is showing off in the Xserve product. Finally, Apple is up to speed with their internal bus speeds. This could cause a large increase in the performance of the upcoming workstations, even without an increase in the clock speed of the G4. Couple the new
memory bus with faster G4’s, or next generation PowerPC chips that may be on their way along with MacOS X 10.2, and Apple could pose a serious challenge to high end Intel workstations.

Outside the reality distortion field, these machines offer the potential for a decently valued, comparably powered server for the standard low-end server market. On top of that, they also look neat!


Update, 22 May 2002:
At the time of the article writing, the prices for the mid-range and high-end Compaq servers were pulled from the 3-pack option, not the standard options. While these new prices are still on the high side of the sampled platforms, they are no longer a factor of two higher. So, the new price range for the Compaq’s on the mid range server are $4077-4852, and on the high end $6420-8200. That doesn’t change the final conclusions however.


Xserve


About the Author
Hank Grabowski is a self admitted geek who finds a lot of thrills in working with and studying cutting edge computer and science concepts. He currently works as a
programmer and aerospace engineer for Analytical Graphics, makers of the satellite mission analysis software Satellite Tool Kit (STK), where he’s been employed for 2.5 years. Previous to that he did his graduate work in
aerospace engineering at Virginia Tech. The biggest highlight of his life so far was working at CNN for the live Mir reentry coverage–which included air time as a bonus. His current computer hobbies include working with
alternative operating systems, and studying parallel processing and vector processing for scientific algorithms. Hank can be reached at [email protected].

105 Comments

  1. 2002-05-19 8:51 pm
  2. 2002-05-19 9:12 pm
  3. 2002-05-19 9:24 pm
  4. 2002-05-19 10:08 pm
  5. 2002-05-19 10:53 pm
  6. 2002-05-19 11:17 pm
  7. 2002-05-20 1:25 am
  8. 2002-05-20 1:47 am
  9. 2002-05-20 1:55 am
  10. 2002-05-20 2:15 am
  11. 2002-05-20 2:24 am
  12. 2002-05-20 2:43 am
  13. 2002-05-20 2:43 am
  14. 2002-05-20 3:05 am
  15. 2002-05-20 4:02 am
  16. 2002-05-20 4:03 am
  17. 2002-05-20 4:32 am
  18. 2002-05-20 5:29 am
  19. 2002-05-20 7:32 am
  20. 2002-05-20 9:01 am
  21. 2002-05-20 10:51 am
  22. 2002-05-20 11:11 am
  23. 2002-05-20 1:45 pm
  24. 2002-05-20 1:46 pm
  25. 2002-05-20 1:55 pm
  26. 2002-05-20 2:32 pm
  27. 2002-05-20 2:53 pm
  28. 2002-05-20 2:57 pm
  29. 2002-05-20 3:01 pm
  30. 2002-05-20 3:02 pm
  31. 2002-05-20 3:17 pm
  32. 2002-05-20 3:46 pm
  33. 2002-05-20 3:48 pm
  34. 2002-05-20 4:13 pm
  35. 2002-05-20 5:09 pm
  36. 2002-05-20 5:37 pm
  37. 2002-05-20 5:41 pm
  38. 2002-05-20 5:48 pm
  39. 2002-05-20 6:44 pm
  40. 2002-05-20 7:34 pm
  41. 2002-05-20 8:16 pm
  42. 2002-05-20 8:42 pm
  43. 2002-05-21 1:12 am
  44. 2002-05-21 2:04 am
  45. 2002-05-21 4:00 am
  46. 2002-05-21 7:22 am
  47. 2002-05-21 10:50 am
  48. 2002-05-21 2:03 pm
  49. 2002-05-21 2:39 pm
  50. 2002-05-21 3:12 pm
  51. 2002-05-21 3:28 pm
  52. 2002-05-21 5:20 pm
  53. 2002-05-22 10:17 am
  54. 2002-05-22 12:41 pm
  55. 2002-05-22 4:48 pm
  56. 2002-05-22 5:50 pm
  57. 2002-05-22 5:53 pm
  58. 2002-05-22 7:58 pm
  59. 2002-05-22 8:18 pm
  60. 2002-05-23 4:38 am
  61. 2002-05-23 4:52 am
  62. 2002-05-23 4:58 pm
  63. 2002-05-23 7:57 pm
  64. 2002-05-23 9:21 pm
  65. 2002-05-23 9:48 pm
  66. 2002-05-23 11:24 pm
  67. 2002-05-24 12:13 am
  68. 2002-05-24 1:28 am
  69. 2002-05-24 5:50 am
  70. 2002-05-24 12:13 pm
  71. 2002-05-24 1:33 pm
  72. 2002-05-24 9:02 pm
  73. 2002-05-24 9:45 pm
  74. 2002-05-24 10:10 pm
  75. 2002-05-24 10:16 pm
  76. 2002-05-24 10:59 pm
  77. 2002-05-25 1:26 am
  78. 2002-05-25 1:59 am
  79. 2002-05-25 3:09 pm
  80. 2002-05-25 3:26 pm
  81. 2002-05-25 8:08 pm
  82. 2002-05-25 9:31 pm
  83. 2002-05-25 10:38 pm
  84. 2002-05-25 10:52 pm
  85. 2002-05-26 5:17 am
  86. 2002-05-26 3:30 pm
  87. 2002-05-26 5:20 pm
  88. 2002-05-27 12:46 am
  89. 2002-05-27 4:37 pm
  90. 2002-05-27 4:44 pm
  91. 2002-05-27 5:44 pm
  92. 2002-05-27 7:12 pm
  93. 2002-05-27 7:53 pm
  94. 2002-05-27 10:18 pm
  95. 2002-05-27 11:46 pm
  96. 2002-05-28 12:53 am
  97. 2002-05-28 1:06 am
  98. 2002-05-28 1:28 am
  99. 2002-05-28 3:35 am
  100. 2002-05-28 7:02 am