Linked by Thom Holwerda on Fri 18th Aug 2017 10:51 UTC
AMD

In this mini-test, we compared AMD's Game Mode as originally envisioned by AMD. Game Mode sits as an extra option in the AMD Ryzen Master software, compared to Creator Mode which is enabled by default. Game Mode does two things: firstly, it adjusts the memory configuration. Rather than seeing the DRAM as one uniform block of memory with an ‘average’ latency, the system splits the memory into near memory closest to the active CPU, and far memory for DRAM connected via the other silicon die. The second thing that Game Mode does is disable the cores on one of the silicon dies, but retains the PCIe lanes, IO, and DRAM support. This disables cross-die thread migration, offers faster memory for applications that need it, and aims to lower the latency of the cores used for gaming by simplifying the layout. The downside of Game Mode is raw performance when peak CPU is needed: by disabling half the cores, any throughput limited task is going to be cut by losing half of the throughput resources. The argument here is that Game mode is designed for games, which rarely use above 8 cores, while optimizing the memory latency and PCIe connectivity.

I like how AnandTech calls this a "mini" test.

In any event - even though Threadripper is probably way out of the league of us regular people, I'm really loving how AMD's recent products have lit a fire under the processor market specifically and the self-built desktop market in general. Ever since Ryzen hit the market, now joined by Vega and Threadripper, we're back to comparing numbers and arguing over which numbers are better. We're back to the early 2000s, and it feels comforting and innocent - because everyone is right and everyone is wrong, all at the same time, because everything 100% depends on your personal budget and your personal use cases and no amount of benchmarks or number crunching is going to change your budget or personal use case.

I'm loving every second of this.

Order by: Score:
Interesting
by ahferroin7 on Fri 18th Aug 2017 12:26 UTC
ahferroin7
Member since:
2015-10-30

I find it somewhat odd that they're essentially acknowledging that it's NUMA on a package, and yet they don't differentiate which MC memory is on normally. Are they part of the whole 'NUMA is bad' cult, do they just think people won't care about having proper topology information and also having all cores running, or do they think that the OS can't possibly know better than they do how to place threads of execution intelligently across multiple nodes?

Reply Score: 3

RE: Interesting
by dpJudas on Fri 18th Aug 2017 12:47 UTC in reply to "Interesting"
dpJudas Member since:
2009-12-10

What really annoys me is that they essentially give you two shitty options: a default mode where they hide the NUMA setup to the applications and OS, or "Game Mode" where they disable half the chip.

It has actually caused me to think twice about buying a threadripper as I don't like hardware where I have to toggle between two modes as an end user. This solution sucks.

Reply Score: 3

RE[2]: Interesting
by ahferroin7 on Fri 18th Aug 2017 13:00 UTC in reply to "RE: Interesting"
ahferroin7 Member since:
2015-10-30

Entirely agreed. They'll be shooting themselves in the foot if they try this crap with the server/workstation options.

Reply Score: 1

RE[2]: Interesting
by Alfman on Fri 18th Aug 2017 13:49 UTC in reply to "RE: Interesting"
Alfman Member since:
2011-01-28

dpJudas,

What really annoys me is that they essentially give you two shitty options: a default mode where they hide the NUMA setup to the applications and OS, or "Game Mode" where they disable half the chip.

It has actually caused me to think twice about buying a threadripper as I don't like hardware where I have to toggle between two modes as an end user. This solution sucks.


Agree too. Any solution that requires rebooting is an awful solution. There should be no interruptions. Not only that but it shouldn't have to be a system-wide toggle ether. If a process doesn't cope well with NUMA, then it should be possible to switch the mode for individual processes independently from the rest of the system.

AMD's solution here certainly seems very odd. Unless the issues they're trying to work around are in the windows kernel itself? I don't know if it's the case, but it is a real possibility.

Not that this makes the options any less disappointing, however it would explain AMD's motivation for disabling NUMA at the system level rather than at the process level.

Reply Score: 3

RE[3]: Interesting
by dpJudas on Fri 18th Aug 2017 14:03 UTC in reply to "RE[2]: Interesting"
dpJudas Member since:
2009-12-10

Not that this makes the options any less disappointing, however it would explain AMD's motivation for disabling NUMA at the system level rather than at the process level.

I think their motivation is mainly to win benchmarks at any cost. By offering two modes they can always show the highest numbers that they can get.

I can certainly understand their motivation, but, as one that actually wanted to buy this chip, this really sucks because not even the default mode does the Right Thing(tm). Future applications do not have the ability to detect the NUMA layout and optimize for it, because even though I might be able to configure that in its control panel, the average end user does not know what all this stuff is. He will either keep it on default or select "Game Mode" because it sounds cool.

Bottom line I get from all this is that it makes no sense to write applications that take full advantage of threadripper. Ugh. And that's where my interest in this chip died. ;)

Reply Score: 2

RE[4]: Interesting
by _txf_ on Fri 18th Aug 2017 14:39 UTC in reply to "RE[3]: Interesting"
_txf_ Member since:
2008-03-17


I can certainly understand their motivation, but, as one that actually wanted to buy this chip, this really sucks because not even the default mode does the Right Thing(tm). Future applications do not have the ability to detect the NUMA layout and optimize for it, because even though I might be able to configure that in its control panel, the average end user does not know what all this stuff is. He will either keep it on default or select "Game Mode" because it sounds cool.


The average user isn't buying threadripper.

Reply Score: 3

RE[5]: Interesting
by dpJudas on Fri 18th Aug 2017 17:20 UTC in reply to "RE[4]: Interesting"
dpJudas Member since:
2009-12-10

The average user isn't buying threadripper.

It is a high end consumer product. Such people are not expected to know what NUMA even is.

Just because poor people can't afford it doesn't mean it is a device for professionals only.

Reply Score: 2

RE[6]: Interesting
by _txf_ on Sat 19th Aug 2017 01:29 UTC in reply to "RE[5]: Interesting"
_txf_ Member since:
2008-03-17

A person buying a high end consumer product is going to want to fiddle with it. Let them figure it out, it isn't like this information is hard to find. They don't have to care about NUMA or not, all they have to do is see whether all cores are enabled or not, and that is simple to do.

And if they're interested in fire-and-forget, they should just leave it in normal mode. It doesn't make that much of a difference.

Edited 2017-08-19 01:29 UTC

Reply Score: 3

RE[5]: Interesting
by shotsman on Fri 18th Aug 2017 19:51 UTC in reply to "RE[4]: Interesting"
shotsman Member since:
2005-07-22

And not every power user plays games.
And not every power user runs Windows.
Just saying...

Reply Score: 2

RE[4]: Interesting
by tylerdurden on Fri 18th Aug 2017 22:50 UTC in reply to "RE[3]: Interesting"
tylerdurden Member since:
2009-03-17

Honestly, it seems you're looking for drama. This is just a toggle to get a couple extra FPS on game benchmarks, it basically adds single percentage performance boost. Unless you're some kind of hardcore FPS-obsessed gamer, which does not sound like you are, it's basically of no consequence.

It's mostly an issue with the Windows Scheduler, that will be fixed eventually. NUMA is mainly about the HW/OS, the apps that can take advantage of NUMA usually are not optimized manually... Most coarse-threaded apps will do just fine. The thing is that most games, as it stand, are programmed by overworked monkeys who do not quite understand multithreading (those monkeys cost extra, and the industry is about shipping ASAP, and let brute force HW take care of it all).

Reply Score: 3

RE[5]: Interesting
by _txf_ on Sat 19th Aug 2017 01:23 UTC in reply to "RE[4]: Interesting"
_txf_ Member since:
2008-03-17

Honestly, it seems you're looking for drama. This is just a toggle to get a couple extra FPS on game benchmarks, it basically adds single percentage performance boost. Unless you're some kind of hardcore FPS-obsessed gamer, which does not sound like you are, it's basically of no consequence.


It does help with the minimum framerates (which is what should matter the most in terms of smoothness and consistency). But even there it isn't that much of a changer.

A person is perfectly fine leaving it in Creator mode for most games.

Reply Score: 2

RE[3]: Interesting
by _txf_ on Fri 18th Aug 2017 14:34 UTC in reply to "RE[2]: Interesting"
_txf_ Member since:
2008-03-17

AMD's solution here certainly seems very odd. Unless the issues they're trying to work around are in the windows kernel itself? I don't know if it's the case, but it is a real possibility.


If you read the article, they say that is precisely what it is.

Edited 2017-08-18 14:36 UTC

Reply Score: 3

RE[4]: Interesting
by Alfman on Fri 18th Aug 2017 20:05 UTC in reply to "RE[3]: Interesting"
Alfman Member since:
2011-01-28

_txf_,

If you read the article, they say that is precisely what it is.


Can I assume you are referring to this?

http://www.anandtech.com/show/11726/retesting-amd-ryzen-threadrippe...

Update:

Robert Hallock on AMD's Threadripper webcast has stated that Windows Scheduler is not capable of specifically zeroing out a full die to have the same effect. The UMA/NUMA implementation can be managed with Windows Scheduler to assign threads to where the data is (or assign data to where the threads are), but as far as fully disabling a die in the OS requires a restart.


The reboot is needed to hide the second CPU from windows (aka "game mode"). However this still does not explain why we need a system-wide switch in the first place. Disabling the second CPU is an awful solution because it completely defeats the purpose of having a second NUMA CPU. Even if no reboot were required, the solution is still bad because a user might want to run a game on the first CPU and run a video stream on the second CPU at the same time.

This would be an excellent use of NUMA, and yet it's mindboggling why this processor can't operate optimally this way. All it would take is to give users a way to move processes to either one of the NUMA CPUs. A simple user-friendly widget could do that without disabling the second CPU system-wide. So why didn't AMD do this? I can't explain it unless they were working around NUMA performance bottlenecks with the windows operating system itself (rather than strictly with the game & encoding processes).

Edited 2017-08-18 20:11 UTC

Reply Score: 2

RE[5]: Interesting
by cb88 on Fri 18th Aug 2017 21:38 UTC in reply to "RE[4]: Interesting"
cb88 Member since:
2009-04-23

I wouldn't be surprised if different microcode is loaded for the 2 modes... necessitating a reboot.

There is a lot of stuff most likely the CPU doesn't have to do when there is only 1 die. Also there are probably timing assumptions in the microcode that can be changed on this basis... could they potentially do that on the fly yes probably but that is harder.

Reply Score: 2

RE[6]: Interesting
by Alfman on Fri 18th Aug 2017 22:22 UTC in reply to "RE[5]: Interesting"
Alfman Member since:
2011-01-28

cb88,

I wouldn't be surprised if different microcode is loaded for the 2 modes... necessitating a reboot.

There is a lot of stuff most likely the CPU doesn't have to do when there is only 1 die. Also there are probably timing assumptions in the microcode that can be changed on this basis... could they potentially do that on the fly yes probably but that is harder.



Maybe, but if that's the case, then it really does set the platform back as a general purpose desktop. Nobody wants to reboot to switch to/from "gaming mode".

To make matters worse, end users will be confused about which mode to use for given software. "Gaming mode", despite the name, isn't just for games and "creator mode" isn't just for creation apps apps. It's a misnomer and will depend on the software.


This rebooting mess actually reminds me of the conditional sections in config.sys and autoexec.bat back in the days of DOS when we had to switch EMS/TSR settings depending on the software. This was a process of painstaking trial and error with many reboots to see which configurations worked with which programs.

Reply Score: 2

RE[6]: Interesting
by tylerdurden on Sat 19th Aug 2017 01:32 UTC in reply to "RE[5]: Interesting"
tylerdurden Member since:
2009-03-17

It's a windows issue, more than anything. I pretty much doubt MS included any core hot swapping/load migration code in the Pro and Home versions of Windows 10.

Reply Score: 2

RE[7]: Interesting
by bert64 on Sat 19th Aug 2017 13:47 UTC in reply to "RE[6]: Interesting"
bert64 Member since:
2007-04-23

It probably doesn't understand NUMA very well, if at all either... Otherwise the OS should take care of all these things, and ensure that processes are located in the correct memory zones for the CPU they're running on.

Linux basically owns the HPC market, which has had NUMA systems for years, so the support there is tried and tested and known to scale to very large NUMA systems. Windows only exists in the lowend where NUMA is far less common, and until now didn't exist at all in single socket desktops.

Reply Score: 2

RE[2]: Interesting
by jing on Sat 19th Aug 2017 16:27 UTC in reply to "RE: Interesting"
jing Member since:
2017-08-19

dpJudas,
If you were paying more attention to the reviews instead of getting annoyed you'd know that you can mix and match the options as you please.(idem for others who comment on the same lines)

It's well explained in the first Anandtech review. (http://www.anandtech.com/show/11697/the-amd-ryzen-threadripper-1950...)

The newest review happened because Anandtech misunderstood which flags defined the AMD game-mode.
(original review game-mode test had disabled SMT but retained all cores active with NUMA enabled)

Oddly "Anandtech game-mode" had better performance than the "AMD game-mode"... AMD probably made a power/performance trade-off here.

Mind that Threadripper is still an enthusiast thing!
And for a workstation is definitely better as-is (configurable via reboot) than a hypothetical "fixed" dual 1800X build.

Also is nice that if you wanted a 1800X with extra lanes: Threadripper game-mode delivers...

Edited 2017-08-19 16:28 UTC

Reply Score: 1

RE[3]: Interesting
by dpJudas on Mon 21st Aug 2017 09:58 UTC in reply to "RE[2]: Interesting"
dpJudas Member since:
2009-12-10

dpJudas,
If you were paying more attention to the reviews instead of getting annoyed you'd know that you can mix and match the options as you please.(idem for others who comment on the same lines)

I know I can do that. My primary point is that the default mode has NUMA disabled (as seen by the OS/applications), which means the apps have no chance of arranging their threads and memory accesses for maximum performance.

As this is a rather advanced subject, no enthusiast user is able to tell when to turn that on or off. You'd have to constant benchmark and reboot based on which application you are going to run. The only reasonable solution is therefore to pick one setting and stick to it.

There are basically three choices there:

1) Keep it at creator mode. This provides suboptimal performance for time critical things, as it is effectively rolling a dice where memory and threads will run.

2) Gimp it with the game mode. Probably the worst choice to pick.

3) Create a custom setup. Problem with this is that you will be the only user then with your particular setup - developers cannot target the setup directly as a result. If my threadripper is likely going to be the only one around with that particular setup, then I can no longer justify the development costs.

I like to treat my CPUs as fire-and-forget. That means this CPU must be evaluated as if I had it in creator mode. I get that not everyone will reach this conclusion, but that's how I look at it.

Reply Score: 2

A solution in search of a problem....
by grat on Fri 18th Aug 2017 16:00 UTC
grat
Member since:
2006-02-02

Not sure what problem AMD is trying to solve here with Game Mode.

If you're a serious gamer who needs every single FPS, out to three decimal places, you won't buy ThreadRipper. In fact, you're probably going to stick with Intel, as it has a higher single-threaded performance.

If you're a gamer in general, but not so paranoid about numbers, get a Ryzen 7 1700, and overclock it a bit. If you still don't have enough spare horsepower to stream, take the money you saved by not buying threadripper/X399, and build a mini-ITX rig with a Ryzen 3 or 5.

If you're buying ThreadRipper, you're doing it because you need lots of CPU cores and L3 cache is just to keep the numbers flowing to the cores.

Even then, if 8C/16T is all you need and you need more PCIe lanes and/or quad-channel memory, then buy the 1900x when it comes out.

Reply Score: 1

Carewolf Member since:
2005-09-08

The components of the game mode makes sense though. I wouldn't disable cores, but the option to have memory near each core instead of interleaved over all banks on all core is meaningful, it is choice between latency and bandwidth. That can be useful outside of just games, and for games it is kind of pointless anyway, we are talking small percentages.

Reply Score: 2

Anyone tried Threadripper in Linux?
by rklrkl on Sat 19th Aug 2017 08:15 UTC
rklrkl
Member since:
2005-07-06

If we think it's Windows deficiencies as to why Creator Mode vs. Game Mode can't be switched on the fly in the OS without a reboot, has anyone tried to do something similar in Linux with Threadripper?

In other words, set Creator Mode in the Threadripper BIOS, boot into Linux and then see if it's possible to simulate Game Mode in Linux without a reboot (and to switch back to Creator Mode without a reboot too of course). I've no idea if the current Linux kernel has the options to do this, but if it did, then it would be very neat indeed.

Sadly, almost the only place that bothers to test new stuff on Linux - phoronix.com - hasn't got a Threadripper to play with (I believe AMD has only just realised the site exists and has started to send Michael some freebie test kit).

Edited 2017-08-19 08:17 UTC

Reply Score: 2

Delgarde Member since:
2008-08-19

I believe AMD has only just realised the site exists and has started to send Michael some freebie test kit.


That's a bit of an exaggeration, since a number of AMD engineers - pretty much all the Linux-focused people - are regular contributors to the Phoronix forums. But yeah, their marketing people seem a bit slow on the uptake...

Reply Score: 2

good
by yahyah on Wed 23rd Aug 2017 09:38 UTC
yahyah
Member since:
2017-08-23

Thanks for sharing this information. I really like your blog post very much. You have really shared a informative and interesting blog post with people.
http://transformiceonline.com

Edited 2017-08-23 09:39 UTC

Reply Score: 1