PCI Express technology has served as the de facto interconnect of choice for nearly two decades. The PCIe 6.0 specification doubles the bandwidth and power efficiency of the PCIe 5.0 specification (32 GT/s), while providing low latency and reduced bandwidth overhead.
We’re barely seeing the rollout of PCIe 5.0 begin, and we’re already moving ahead. Also, who knew the standards organisation for PCIe is headquartered in Beaverton, Oregon, of all places. Although, to be fair, any city that understands and caters to the beautiful, thrilling, and honest sport of curling is a great city. And I’m not joking here – curling is exquisite, and quite probably the noblest of sports.
Love the comments on curling. In Canada, curling is very popular. Perhaps it is because of the prominent role of Scottish immigrants in the history of Canadian settlement. Even more likely it is because it is one of the few sports where it is a totally accepted part of the game to be drinking while playing.
That’s part of why I love disc golf so much.
Drumhellar,
Does your disc golf course have a liquor license? I’m pretty sure here you’d risk getting a ticket for public drinking. You’re allowed to have fun, but not too much, haha.
A couple of the ones I frequent are on private property, with no rules against drinking (within reason). But those are a bit further away and are usually weekend camping trips.
But, the several that are near by are in public parks, which means breaking the rules. But, nobody actually cares as long as you don’t cause trouble. A six pack between a couple of friends is something nobody cares about.
Regular golf is certainly one of the other sports where you can drink while playing. What is it with sports dreamed up by the Scots? I was unaware that you could drink while playing disc golf. I will have to check it out!
A specification iteration is just another way of selling stuff. If you look around all the announcements of the big American tech companies a few months ago you will notice each and every one of them took a dominate the market attitude and forced obsolescence and similar was triggered. There’s also been issues with control of the market and somewhat dubious political actions to rig things in their favour. The US has also tried to con Europe into “sharing technology” while denying it to countries on their naughty list, they’ve poked the admittedly problematic Russia with a stick which is going to provoke increased military costs in Europe to cover their tilt towards the Pacific. America is very good at talking up a con such as during the Alan Greenspan bubble as well as passing their costs onto everyone else such as with the financial crisis. Once you see this you begin to understand how you’re being played.
But yes I think the economy and finance have a man problem. Now it’s possible people may have thought of it themselves independently but I know after I opened my gob for a few weeks this was a talking point in the media and lots of women had their say on economics including promoting the concept of watering the roots of the tree (there were articles on this as well as Keynsianism and Marxism funnily enough) which as we know with (mostly male) billionaires becoming even wealthier post financial crash and post-pandemic this is not happening. Feminist economics and gender economics are actually formal academic topics, just so you know.
Yesterday, the Guardian did an on the surface serious but more a spiteful comedy article on emotional labour. This is actually a thing. Now as much as I have a problem with men I don’t agree with articles like this and I’m fairly sure it’s the kind of article Deborah Orr would not have approved. So before everyone goes omg omg you hate men you need to park the mouth and think things through a bit.
As for PCIE 6 I have nothing against it. I’m sure some people will find it useful even if it’s just an excuse to sell more stuff. I just don’t need it myself not even remotely. The majority of people don’t. And men call women spending money on clothes and makeup vain and frivolous…
HollyB,
Yes but osnews is for technology first and foremost. Feminism is a topic we can cover every now and then, but not every damn article needs to be turned into a sexist anti-male crusade.
“Feminist economics and gender economics are actually formal academic topics, just so you know.”
Seems reason for Crazy to write about them on a tech blog all the time.
OK Karen this is an article on a formal tech spec, not an invitation for you to go spew your man hate. Go watch some cat videos, you’ll feel better.
BTW does anybody else miss being able to vote down and bury the nutballs and spammers? Maybe Thom should bring that back, I know I’d be grateful.
bassbeast,
Yeah but one of their goals in moving to a standard wordpress site was less customization & maintenance. Maybe they would install an existing plugin though, something like this maybe?
https://wordpress.org/plugins/bbp-voting/
Note that most of the voting plugins seem to be for rating the articles rather than the comments.
That would be nice as I’ve noticed if you don’t have a way to filter out the cranks and trolls? Eventually they will crap on everyone’s cornflakes and ruin the place.
Yes, but…
PCI is relatively good when it comes to backward and forward compatibility – e.g. if you plug an older PCIe card into a newer bus, or plug a newer PCIe card into an older bus; then it just falls back to the fastest speed supported by both and everything works (you’re not forced to upgrade all your PCIe cards when you upgrade your computer or..).
This compatibility also extends to software (if you upgrade your hardware you don’t need to upgrade your OS/software).
I’m sure that if you replaced every man with an equally skilled women nothing would change. More likely (given that some sexism does exist) is that the women would need to be more skilled to reach the same positions; where “more skilled” means “more skilled at maximizing a company’s profit at the expense of both competitors and consumers” which means “worse for all the things Holly complains about”.
In other words, capitalism has a capitalism problem (that has nothing to do with men vs. women).
This is something we’ve known for hundreds (thousands?) of years – that unregulated capitalism is extremely bad, and that well regulated capitalism can be good.
With this in mind we can say that the problem is that USA’s capitalism is not well regulated.
Further; we can say that the problem is democracy – to delude the maximum number of voters you need to sink a huge amount of $$ into spam (“campaigning”), and to obtain a huge amount of $$ you need to french kiss the butts of large companies (primarily, by promising to not regulate capitalism well).
Not sure if the increased bandwidth is worth the extra power consumption and latency.
Already with PCIe 4.0 (16Gb/s lanes) there were problems with chipset and M.2 SSD cooling and the overall system performance was initially only marginally better (SSDs are now catching up, GPUs still don’t really benefit from the extra bandwidth).
PCIe 5.0 (32Gb/s/lane) consumes significantly more power, so, at least for now, it can’t really be used in M.2 SSDs.
PCIe 6.0 (64Gb/s/lane) uses the same baud rate as PCIe 5.0 but it sends 2 bits per symbol (PAM4) to double the bandwidth. Power consumption should not increase significantly (maybe 1.5x) but, as PAM4 requires FEC, it comes with extra latency.
PCIe 7.0 (guessing 128Gb/s/lane and assuming it sticks to copper at all), will likely use PAM4@64Gbaud/s. This is close to the physical limits of an electrical channel and it requires shorter connections, different connectors, digital signal processing and fairly heavy FEC. At that point, it would be more efficient to use more slower lanes.
For PCs, PCIe 5&6 are a solution waiting for a problem, at least until U.3 SSDs become mainstream. In small factor computers bandwidth needs are generally better addressed by tighter integration (like what M1 CPU did with RAM) than by adding costly interconnects.
ndrw,
Where are you getting information for latency & power consumption?
I searched and found this about PCI6.
https://pcisig.com/sites/default/files/files/PCIe%206.0%20Webinar_Final_.pdf
It looks like the worse case scenario was a 10ns increase over PCIe5 for the smallest transfers and the latency for large transfers will improve (search PDF for “Meets or exceeds the latency expectations”).
About FEC, this paper says…
I’m curious about power consumption, although I didn’t find much information about that. The same PDF mentions “Addition of a new power state (L0p) to support scalable power consumption with bandwidth usage without interrupting traffic” where lanes can wake/sleep without interruption.
It’s hard to imagine the need for such high data rates at least in consumer applications. Personally I’d rather see more integrated high speed cache than integrated ram. Integrated ram, while being good for latency, also has negative trade-offs. Obviously there’s the capacity and upgrade issues, but also more subtly in terms of the implied thermal constraints of the CPU package. But small factor computers aren’t generally used for high end or server grade applications, so it’s probably a non-issue for that. Still though, can anyone think of an application where consumers would need this much bandwidth? As always, with servers you can never get enough.
Alfman,
One potential consumer benefit is multiplexing more PCIe lanes on smaller CPU packages. Intel was notoriously bad with 16+16 (?) lanes, which were not enough for modern applications. Now they can use one PCIe 6 lane to get 4 PCIe 4 lanes (i.e.: one nvme SSD).
Yes, otherwise PCIe 4 is more than enough for any modern desktop need for the foreseeable future.
(Checking https://en.wikipedia.org/wiki/PCI_Express)
One PCIe 4 x1 lane is ~2GB/s. That is sufficient even for transferring raw uncompressed 4K streams, or connecting 10GBe Ethernet devices.
sukru,
You are right about that, there aren’t enough PCIe3 lanes, especially when the 16 lanes get divided between the first two PCIe slots. PCIe4 helps restore the lost bandwidth. Arguably you can repeat this PCIe5 and 6, but I think you’d be looking at higher power, higher costs, and less range.
Today we can only fit two high end cards in the entire system. I learned this the hard way, unless it’s water cooled, the slots are too close together for cooling and on top of that with two cards there is no physical room for even one more peripheral such as a 10gbe card. This makes PCIe extenders an absolute necessity.
I tried using a PCIv4 riser for a second video card and it just wasn’t stable, contrast to PCIv3 risers that are stable even if you daisy chain them several meters. For PCIe 5 and 6 I’m guessing risers will be out all together. This is a bit of a problem because today’s video cards are getting too big to fit in standard PCI slot spacing. I’m admittedly introducing niche issues here, but even if we had more free lanes (which is good), I don’t think the standard PCIe slot placement gives us a way to use them effectively without a new physical layout.
This (multiplexing) is what AM5 boards are said to be: PCIe5 lanes to the CPU, 2x more PCIe4 lanes on board.
Applications will catch up at some point, as they always do. The main use for faster IO in datacenters is storage and networking. Home users may (say, 2+ years from now) be interested in faster SSDs but these would probably have to come in a different format than M.2, unless technology gets significantly more efficient.
For graphics cards – it depends if direct access to disk catches on. If yes, that may put some pressure on bandwidth. Otherwise PCIe4 will be good enough for quite some time.
I expect PCIe edge connectors to remain at PCIe3/4 level. Anything faster than 16Gb/s/lane should either use on-board connections or cables.
ndrw,
I don’t know about consumer applications always needing more. On paper you can double the bandwidth, number of pixels on a display, ray tracing cores, etc. But the reality is average consumers will have more and more trouble even noticing improvements due to diminishing returns. Disk I/O was very slow in the past but it’s gotten much better and I think any gains from now on will be rather marginal to average consumers. Most users won’t benefit from the memory capacity or CPU cores available at the high end. There are specialized niche applications of course, but I’m talking about mainstream. I’ve been waiting for faster *commodity* ethernet like 10gbe for decades and it’s been excruciatingly slow to become mainstream because they don’t need it.
Don’t get me wrong, I’m not against the hardware improvements and there are obvious advantages for hardware to become lighter, more efficient, and cheaper, but I think the high end performance is already more than most consumers need. At this point I think the biggest improvement for consumers will be increasing availability and having costs come down a lot more.
Good point about spec “sensitivity”. People generally don’t care about specs that are at any given moment too bad (e.g. current VR headsets) or too good (speed of ethernet at home). It is the middle ground that makes all the difference and receives customers’ attention. I agree storage speed is currently in the “too good” category, as software hasn’t caught up with the recent hardware performance jump yet but that’s coming the moment developers start depending on SSDs.
Another potential is connecting nvme directly to the GPU. Add in sampling feedback streaming for textures, and you get a truly next-gen feature, avoiding CPU or RAM overhead, while dramatically improving visual quality of games.
The pieces are already here, but I am not sure there is any existing software using it at the moment.
The first piece is similar to “zero copy” operations under Linux. One I/O device (nvme) is directly fed to another one (GPU): https://en.wikipedia.org/wiki/Zero-copy.
As far as I know, nvidia already supports this:
https://github.com/ZaidQureshi/gpudirect-nvme
The second piece is using the actual rendered frames to choose which textures to load. As a reminder, games have different resolutions of the same textures (mipmaps), to save on RAM, but game engines need to figure out the quality of each polygon. Here the GPU itself figures out what quality is actually needed, and gives feedback to the CPU:
https://www.reddit.com/r/hardware/comments/fx0bbj/sampler_feedback_streaming_how_games_will_use/
It is already implemented in AMD RNDA2 GPUs (and Xbox Series consoles). But I think nvidia does not have it yet (and PS5 has no hardware support).
The obvious next step is connecting these technologies. To be fair, I have not followed the recent developments, so it could already be done. But in theory the GPU can figure out which textures to load, and directly ask them from the NVMe, reducing load times to essentially zero (for graphics at least). And significantly free up memory on both the CPU and the GPU.
(It would also need collaboration from the OS, giving information from the filesystem, and locking those disk pages in place).
Anyway, future will be interesting. The CPU centric computer model is moving towards a “local distributed network” with each innovation.
RE: RAM and cache.
I wouldn’t mind having 8/16GB of fast in-package RAM and treating the on-board RAM as a disk cache and swap. To benefit from it, OS would have to support it, though (be able to distinguish between these two areas of RAM and implement two-level disk caching and swapping).
Thermals are generally not a problem, unless you mean 3d-integrated RAM.
ndrw,
To the extent that there’s extra headroom, I’d rather see them use it to bump up L1/L2 cache specs than onchip dram which is going to be slower anyways.
The problem is when you keep stacking RAM, CPU cores, GPU cores, and AI cores, etc into the same package, you end up shortening their lifespan at extreme temps and/or thermal throttling them. Even with water cooling the temperature delta between the coolant and CPU die is pretty high because there’s so much heat energy to dissipate.
The M1 benefited greatly from TSMC’s smaller cutting edge node size, but heat presents a major challenge for scaling it for more cores/ram/gpu/etc. It’s possible to actively cool the package below atmospheric temps, people even use frozen nitrogen for this. But the problem is this is not efficient and not practical, especially not for something that’s meant to be portable and run on a battery. IMHO it still makes sense to spread the heat across a larger area.
That wouldn’t compete with L1/L2, or even L3 cache. I am talking about adding a large DRAM die to the CPU package. Anything >4GB should definitely offer random access and be exposed to the OS. Operating temperature may indeed matter for DRAM but that mainly affects refresh rate, not longevity (assuming DRAM was designed for this temperature).
3d integration (die stacking) thermals are still a challenge. 3d stacking will for sure be adopted in mobile applications, likely in multi-level configurations. For high performance CPUs/GPUs it is not clear at the moment. In limited configurations (2 thinned dies, CPU+static RAM) bandwidth benefits seem to outweigh the cost but with improved packaging placing these dies next to each other could work almost just as well and offer better cooling.
ndrw,
Yes I know what you’re saying, but my point was that when they’re in the same package it eats into the thermal headroom available for the entire package, so there is an implicit tradeoff. I’d rather use the headroom for cache, which is much faster than DRAM.
sukru brought up loading data from storage into GPU. Nvidia has been working on this for a few years…
https://developer.nvidia.com/blog/gpudirect-storage/
https://docs.nvidia.com/gpudirect-storage/design-guide/index.html
I’ve heard that this works in Tesla and Quadro cards, I don’t know if nvidia is adding it to consumer GPUs though.
It’s interesting that at some point the roles between CPU and GPU could flip, the CPU becoming a “co-processor” to handle peripherals while the GPU does all the heavy lifting. If someone found a way to run the keyboard/mouse/networking on the GPU you almost wouldn’t need a CPU at all, haha.