On Thursday, May 26, 2022 at 12:37:56 PM UTC-5, Anton Ertl wrote:
> EricP <ThatWould...@thevillage.com> writes:
> >Anton Ertl wrote:

> >For example, if the DRAM internal row was a multiple of 72 then
> >an internal scrubber could read a row and chew away on it over
> >multiple cycles, ideally without interfering with normal operations.
> From the things I have read about DDR5's on-die ECC, it really does
> per-die ECC, and fixes errors on the die, but does not send any ECC
> information outside.
>
> My guess is that they correct a whole row (minimizing the additional
> bits necessary, e.g., 14 extra bits for SECDED of an 8Kb row); it's
> possible that it's too expensive to compute ECC for an 8K+14bit row,
> and in that case they may choose to use more extra bits and a cheaper
> ECC function.
>
> My other guess is that they do the SECDED on every refresh. That
> mechanism is already in place, the refreshes occur anyway, and given
> that the reason for on-die ECC is flaky RAM cells, it makes little
> sense to do a refresh without correcting errors. The stated benefit
> of on-die ECC is that this counteracts the tendency of RAM to become
> less reliable as processes shrink.
<
Given an ability to perform SECDED, and the fact that reading a DRAM is
a destructive operation, DRAMs with SECDED would be performing
correction on EVERY activate, not just refreshes. {A refresh is just an
activate without using the data pins.}
>
> But DDR5 also has ECC DIMMs that have 40 bits per sub-channel, which
> allows end-to-end ECC checking. And I guess you run a scrubber on
> that, too.
<
Fundamentally, there is no reason to check ECC at the DRAM controller,
then pass data over the interconnect without ECC, and reconstruct another
ECC as data reaches the cache hierarchy. It is at least as good to just send
the RAW DRAM ECC through out the memory hierarchy.
<
This allows the DRAM controller to forward data faster than it can correct
ECC and allow the cache hierarchy to make a correction when needed after
arrival, lowering overall latency.
<
> - anton
> --
> 'Anyone trying for "industrial quality" ISA should avoid undefined behavior.'
> Mitch Alsup, <c17fcd89-f024-40e7...@googlegroups.com>

On Thursday, May 26, 2022 at 8:37:56 PM UTC+3, Anton Ertl wrote:
> EricP <ThatWould...@thevillage.com> writes:
> >Anton Ertl wrote:
> >> DDR5-4800 has ~38GB/s bandwidth per 64-bit superchannel. If you have
> >> 4096GB of memory to scrub with 8 superchannels, and you invest 1% of
> >> the bandwidth (0.38GB/s per superchannel), it takes
> >> 4096GB/8/0.38(GB/s)=1347s, i.e. less than half an hour. If it takes
> >> longer, it's because somebody decided to spend less bandwidth on it.
> >
> >The number I had from 2010 came from Intel's recommendation
> >for one of its memory controller hubs and was 1.5 hours/GB.
> >DRAMs haven't gotten faster since then, just bigger.
> DRAM bandwidth has gotten much bigger. 2010 would be DDR3 memory.
> But even DDR3 memory supports mempory bandwidths that support much
> higher scrubbing rates even if you use only 1% of the bandwidth. If
> Intel scrubbed this slowly by default, it's because they felt that
> there is no need for faster scrubbing.
> >> A DRAM chip does not have the ECC information in DDR4, it's on a
> >> separate chip. DDR5 has something called on-die ECC, but apparently
> >> that is not real ECC (I have not seen any information that explains it
> >> and that looked trustworthy).
> >
> >I know. I'm exploring new, different configurations.
> >
> >For example, if the DRAM internal row was a multiple of 72 then
> >an internal scrubber could read a row and chew away on it over
> >multiple cycles, ideally without interfering with normal operations.
> From the things I have read about DDR5's on-die ECC, it really does
> per-die ECC, and fixes errors on the die, but does not send any ECC
> information outside.
>
> My guess is that they correct a whole row (minimizing the additional
> bits necessary, e.g., 14 extra bits for SECDED of an 8Kb row); it's
> possible that it's too expensive to compute ECC for an 8K+14bit row,
> and in that case they may choose to use more extra bits and a cheaper
> ECC function.
>

I suspect it's neither.
Nothing is checked except the content read either by external client or by internal scrubber.

> My other guess is that they do the SECDED on every refresh.

My guess is they don't. Refresh has to be relatively fast, order of 15-25ns.
Checking the whole row in such short time would be expensive, but in terms
of area and in terms of power consumption.
Now, it's likely that the do use refresh time for scrub but check only 64 or at best 128 words,
rather than full row.
As to SECDED, at least in case of Micron it is known that they are doing
SEC, but not DED. They didn't supply enough of redundancy for the later -
only 8 parity bits per 128 data bits. So some dual-error will be detected, but
many others would be confused for single errors and turned into triple errors.

>That
> mechanism is already in place, the refreshes occur anyway, and given
> that the reason for on-die ECC is flaky RAM cells, it makes little
> sense to do a refresh without correcting errors. The stated benefit
> of on-die ECC is that this counteracts the tendency of RAM to become
> less reliable as processes shrink.
>
> But DDR5 also has ECC DIMMs that have 40 bits per sub-channel, which
> allows end-to-end ECC checking. And I guess you run a scrubber on
> that, too.

And since the overhead is +25% instead of +12.5% of previous generations, it could
very likely mean the end to availability of "real" ECC in consumer/prosumer space.
Not that it was too common until now...

> - anton
> --
> 'Anyone trying for "industrial quality" ISA should avoid undefined behavior.'
> Mitch Alsup, <c17fcd89-f024-40e7...@googlegroups.com>

Quadibloc <jsavard@ecn.ab.ca> wrote:
>On Wednesday, May 25, 2022 at 6:33:53 AM UTC-6, Torbjorn Lindgren wrote:
>> And we now know it was pretty close to the low-bar, AMD's Direct of
>> Technical Marketing Robert Hallock later disclosed in a PCWorld stream
>> that the 5.5GHz peak was done without overclocking using a reference
>> AM5 motherboard and a "consumer-based" Asetek 280mm - which build MOST
>> PC AIO's thanks to various patents on placing the pump on top of the
>> CPU (hence some model instead placing the pump in the radiator).
>>
>> So more substantial cooling than I expect most people have on their
>> 5950X (I'm guessing 240mm or big-air) but not chilled water (Intel 28
>> core 5GHz demo anyone) or anything crazy and it's a setup that should
>> be easy to match using consumer parts.
>
>Given that people have achieved over 6 GHz with previous generations of
>Ryzen parts, if the 5.5 GHz had been achieved with liquid nitrogen, I think
>the Securities and Exchange Commission would be having a word with
>Lisa Su.

Except... it's not launched yet and won't be for many months ("fall"),
and it's possible not all SKUs is in the first batch.

There's plenty of cases where pre-production HW has needed, well,
extra encouragement lets say, to reach the specs the final product
will do.

Yes, it's close enough to the launch that this should hopefully not be
the case but I think you're vastly overstating the case especially
given that the official listed spec when I was writing was 5GHz+, NOT
the 5.5GHz they were showing! But we can always agree to disagree on
that.

The situation when you're writing is very different since they've
since made additional statement about how this was done (no
overclocking and on 280mm AIO) and that at least one SKU's should be
able to hit 5.5GHz boost without overclocking under the right
circumstances, but both that and the "system setup" was conspiciously
missing from the presentation.

>Chilled water, like in an IBM mainframe, indeed counts as "something
>crazy", but water cooling using a fan and radiator on the water outside
>the case is what AMD _recommended_ for their 16-core chips, both the
>5950 and its predecessor the 3950. (In fact, it was also recommended
>for the 5900 and other members of that series.)
>
>So one can't fault them for using the standard, recommended cooling
>solution for their chip!

The only actual statement I can find from AMD on the subject is that
for Ryzen they recommend "To get maximum performance... AMD recommends
a high performance all-in-one liquid or air cooler"[1].

The title does give additional prominence to the liquid coolers not
found in the text or page title but the list also includes a number of
air-cooler of varying capacity, down to at one single-tower cooler
with just a single 120mm fan! Not sure that I'd count the Dark Rock as
"high-performance air cooling" but it's in AMD's official list. They
also list several much bigger air tower coolers that would work fine!

Their recommendation for the up to 280W TDP ThreadRipper[2] OTOH does
only list 240mm or larger AIO, or custom water-block. Sure you didn't
get them mixed up?

So, how does that 280mm AIO rate in that list of officially
recommended Ryzen coolers? Pretty close to the top actually, the
"largest" coolers listed are 360mm but those are barely better than
the significantly wider 280mm AIOs - 360mm radiators have only ~10%
more surface area than 280mm and 140mm fans tends to have better
fanblade-to-hub ratio which will reduce this advantage so say 0-10%.

Or to look at it differently I'm guessing the 280mm AIO has about
35-40% more cooling capacity than the 240mm AIO and probably has at
least twice the cooling capacity of that Dark Rock Slim!

So I think I was fair when I said it was "pretty close to the lower
bar" that I think I earlier defined as 240mm AIO or equivivalent.
If anything the presence of the Dark Rock Slim suggest I might have
been kind.

Of course after that AMD's clarified things multiple times, including
AM5 socket power, but I see that they've already had to correct their
first statements about that. I don't think we've seen AMD handle a
reveal this badly before (at least not in the Ryzen era). Which is
very high marks - it wasn't BAD as such, just "scattered".

Originally AMD said the AM5 was designed for max 170W peak power, we
know that the corresponding numbers for AM4 was 142W peak power and
105W TDP. This has now been corrected to 230W peak power and 170W TDP
max for AM5.

Their statement also for the first time mention that they'll have
three TDP groups (170/105/65) for AM5 rather than AM4's two (105/65),
though it's still unclear if they say plan multiple 16-core with
different TDP or not.

Intel's KS and AMD's 5800X3D SKU's has shown that some people will pay
extra for higher (specific) performance even at higher TDP or other
drawback (non-gaming performance for X3D), perhaps the plan is for the
best binned chip to go into a 170W & 5.5GHz boost "premium" SKU.

1. https://www.amd.com/en/processors/ryzen-thermal-solutions
2. https://www.amd.com/en/thermal-solutions-threadripper
3. https://www.tomshardware.com/news/amd-corrects-socket-am5-for-ryzen-7000-power-specs-230w-peak-power-170w-tdp

Michael S <already5chosen@yahoo.com> writes:
>On Thursday, May 26, 2022 at 8:37:56 PM UTC+3, Anton Ertl wrote:
>> My guess is that they correct a whole row (minimizing the additional
>> bits necessary, e.g., 14 extra bits for SECDED of an 8Kb row); it's
>> possible that it's too expensive to compute ECC for an 8K+14bit row,
>> and in that case they may choose to use more extra bits and a cheaper
>> ECC function.
>>
>
>I suspect it's neither.
>Nothing is checked except the content read either by external client or by internal scrubber.
>
>> My other guess is that they do the SECDED on every refresh.
>
>My guess is they don't. Refresh has to be relatively fast, order of 15-25ns.
>Checking the whole row in such short time would be expensive, but in terms
>of area and in terms of power consumption.

I think that the area is small compared to the DRAM array, and the
power consumption is small compared to the driving power for
refreshing the row (which is done anyway). But speed might be an
issue. Caches often have ECC, so error correction can be speedy, but
they only do much shorter blocks.

One way would be to have shorter blocks, e.g., 9 blocks of 920-921
bits (10 of which are for ECC) for an 8Kb row. But if you have that
many extra bits (90 for an 8Kb row), there may be better ways to do
ECC than just to split to row into subblocks and check them
separately.

>Now, it's likely that the do use refresh time for scrub but check only 64 or at best 128 words,
>rather than full row.

I guess your idea is to use the transfer lines that are there anyway
for the read and write accesses for the accessed part of a row and do
ECC on that (e.g. for an 8-bit-wide device with 16-beat minimum burst
length, that would be 128 bits). Concerning area, that requires 8 ECC
bits per 128 data bits, which is probably more than when doing ECC for
bigger blocks (bigger ECC circuits, but less extra DRAM). Concerning
timing, the ECC computation may be shorter, but you now have the
transfer there and back.

>As to SECDED, at least in case of Micron it is known that they are doing
>SEC, but not DED.

I guess there is also no way for the device to report a detected error.

>They didn't supply enough of redundancy for the later -
>only 8 parity bits per 128 data bits.

Maybe I should have looked at the data sheets instead of speculating.
So they have documented that they do it this way. Interesting.

So for scrubbing an 8Kb line they use 64 refreshes.

>> But DDR5 also has ECC DIMMs that have 40 bits per sub-channel, which
>> allows end-to-end ECC checking. And I guess you run a scrubber on
>> that, too.
>
>And since the overhead is +25% instead of +12.5% of previous generations, it could
> very likely mean the end to availability of "real" ECC in consumer/prosumer space.
>Not that it was too common until now...

Currently I see a factor of 1.8 between 32GB DDR4-3200 UDIMMs with and
without ECC, which should be plenty for absorbing the 25% overhead in
DRAM chips; of course that partly reflects the fact that there are low
sales numbers for ECC UDIMMs. Interestingly, a 32GB DDR4-3200 RDIMM
can be had for less than a UDIMM (and slower RDIMMs are significantly
cheaper) despite needing additional chips.

So basically, those who buy ECC UDIMMs are already paying a hefty
premium now (and if they buy Intel, they also pay it on the board and
maybe on the CPU; and AMD is partially following Intel by disabling
ECC on their non-Pro APUs, but they idiotically don't sell Pro APUs in
retail). I expect that this will stay the same.

In the long run, the (sub)channels are going to become narrower, which
would mean increasing hardware overhead for ECC. But given on-die
ECC, there is another option: each chip could generate a parity bit
and transmit/receive that on a separate line in order to detect
transmission errors. If there is a transmission error, the receiver
would request a retransmit (or maybe use more lines and perform error
correction if retransmission is impractical).

As for commercial considerations, there is obviously little revenue to
be made with ECC UDIMMs, so market segmentation for UDIMMs with ECC
does not really work, so there is no commercial reason to block such a
development. Market segmentation between UDIMMs in the cheap segment
and RDIMMs and LRDIMMs in the expensive segment seems to work, but
there is no need to block ECC for UDIMMs for that.

- anton
--
'Anyone trying for "industrial quality" ISA should avoid undefined behavior.'
Mitch Alsup, <c17fcd89-f024-40e7-a594-88a85ac10d20o@googlegroups.com>

On Friday, May 27, 2022 at 10:53:05 AM UTC+3, Anton Ertl wrote:
> Michael S <already...@yahoo.com> writes:
> >On Thursday, May 26, 2022 at 8:37:56 PM UTC+3, Anton Ertl wrote:
> >> My guess is that they correct a whole row (minimizing the additional
> >> bits necessary, e.g., 14 extra bits for SECDED of an 8Kb row); it's
> >> possible that it's too expensive to compute ECC for an 8K+14bit row,
> >> and in that case they may choose to use more extra bits and a cheaper
> >> ECC function.
> >>
> >
> >I suspect it's neither.
> >Nothing is checked except the content read either by external client or by internal scrubber.
> >
> >> My other guess is that they do the SECDED on every refresh.
> >
> >My guess is they don't. Refresh has to be relatively fast, order of 15-25ns.
> >Checking the whole row in such short time would be expensive, but in terms
> >of area and in terms of power consumption.
> I think that the area is small compared to the DRAM array, and the
> power consumption is small compared to the driving power for
> refreshing the row (which is done anyway). But speed might be an
> issue. Caches often have ECC, so error correction can be speedy, but
> they only do much shorter blocks.
>
> One way would be to have shorter blocks, e.g., 9 blocks of 920-921
> bits (10 of which are for ECC) for an 8Kb row. But if you have that
> many extra bits (90 for an 8Kb row), there may be better ways to do
> ECC than just to split to row into subblocks and check them
> separately.
> >Now, it's likely that the do use refresh time for scrub but check only 64 or at best 128 words,
> >rather than full row.
> I guess your idea is to use the transfer lines that are there anyway
> for the read and write accesses for the accessed part of a row and do
> ECC on that (e.g. for an 8-bit-wide device with 16-beat minimum burst
> length, that would be 128 bits). Concerning area, that requires 8 ECC
> bits per 128 data bits, which is probably more than when doing ECC for
> bigger blocks (bigger ECC circuits, but less extra DRAM). Concerning
> timing, the ECC computation may be shorter, but you now have the
> transfer there and back.
> >As to SECDED, at least in case of Micron it is known that they are doing
> >SEC, but not DED.
> I guess there is also no way for the device to report a detected error.
> >They didn't supply enough of redundancy for the later -
> >only 8 parity bits per 128 data bits.
> Maybe I should have looked at the data sheets instead of speculating.
> So they have documented that they do it this way. Interesting.

I didn't read datasheet either, just their marketing materials named
"Introducing Micron ® DDR5 SDRAM: More Than a Generational Update."

Today I finally opened a datasheet and found out that my guesses were too optimistic.
It is not stated explicitly, but it could be easily deduced that in automatic ESC mode
each Refresh command (REFab) handles exactly 1 code block (128 bits) rather than
4 or 8 code blocks that I was hoping for.
That can be deduced from the following statement in the datasheet:
'The maximum spacing between REFab commands or self refresh entry for the
device to complete the automatic scrub within the recommended 24 hours is t ECSint."
>
> So for scrubbing an 8Kb line they use 64 refreshes.

Exactly :(

> >> But DDR5 also has ECC DIMMs that have 40 bits per sub-channel, which
> >> allows end-to-end ECC checking. And I guess you run a scrubber on
> >> that, too.
> >
> >And since the overhead is +25% instead of +12.5% of previous generations, it could
> > very likely mean the end to availability of "real" ECC in consumer/prosumer space.
> >Not that it was too common until now...
> Currently I see a factor of 1.8 between 32GB DDR4-3200 UDIMMs with and
> without ECC, which should be plenty for absorbing the 25% overhead in
> DRAM chips; of course that partly reflects the fact that there are low
> sales numbers for ECC UDIMMs. Interestingly, a 32GB DDR4-3200 RDIMM
> can be had for less than a UDIMM (and slower RDIMMs are significantly
> cheaper) despite needing additional chips.
>
> So basically, those who buy ECC UDIMMs are already paying a hefty
> premium now (and if they buy Intel, they also pay it on the board and
> maybe on the CPU; and AMD is partially following Intel by disabling
> ECC on their non-Pro APUs, but they idiotically don't sell Pro APUs in
> retail). I expect that this will stay the same.
>
> In the long run, the (sub)channels are going to become narrower,

You still assume that we are ahead of long run [of Moore law].
I am less optimistic. I seems plausible that provisions for 64gb devices
that is already in DDR5 will suffice for more than decade.
May be, close to 2 decades. And we, Anton and Michael, are less
young then we used to be.

> which
> would mean increasing hardware overhead for ECC. But given on-die
> ECC, there is another option: each chip could generate a parity bit
> and transmit/receive that on a separate line in order to detect
> transmission errors. If there is a transmission error, the receiver
> would request a retransmit (or maybe use more lines and perform error
> correction if retransmission is impractical).
>
> As for commercial considerations, there is obviously little revenue to
> be made with ECC UDIMMs, so market segmentation for UDIMMs with ECC
> does not really work, so there is no commercial reason to block such a
> development. Market segmentation between UDIMMs in the cheap segment
> and RDIMMs and LRDIMMs in the expensive segment seems to work, but
> there is no need to block ECC for UDIMMs for that.
> - anton
> --
> 'Anyone trying for "industrial quality" ISA should avoid undefined behavior.'
> Mitch Alsup, <c17fcd89-f024-40e7...@googlegroups.com>

Michael S <already5chosen@yahoo.com> writes:
>On Friday, May 27, 2022 at 10:53:05 AM UTC+3, Anton Ertl wrote:
>> In the long run, the (sub)channels are going to become narrower,=20
>
>You still assume that we are ahead of long run [of Moore law].
>I am less optimistic. I seems plausible that provisions for 64gb devices
>that is already in DDR5 will suffice for more than decade.
>May be, close to 2 decades.

Newer DDR standards are not primarily about chip capacity (currently
the largest DDR5 chips are not larger than supported by DDR4: 16Gb),
but about transfer rate. I.e., if the transfer rate becomes faster
than 16x the internal cycle time of DRAM, they have to increase the
minimum burst length (and decrease the (sub)channel width) to benefit
from that increase. And currently I see no sign that the transfer
rate increases are slowing down. DDR4 was released 7 years after
DDR3; DDR5 was released 6 years after DDR4. On the PCIe front, the
release cadence seems to be even higher.

- anton
--
'Anyone trying for "industrial quality" ISA should avoid undefined behavior.'
Mitch Alsup, <c17fcd89-f024-40e7-a594-88a85ac10d20o@googlegroups.com>

On Friday, May 27, 2022 at 3:15:03 PM UTC+3, Anton Ertl wrote:
> Michael S <already...@yahoo.com> writes:
> >On Friday, May 27, 2022 at 10:53:05 AM UTC+3, Anton Ertl wrote:
> >> In the long run, the (sub)channels are going to become narrower,=20
> >
> >You still assume that we are ahead of long run [of Moore law].
> >I am less optimistic. I seems plausible that provisions for 64gb devices
> >that is already in DDR5 will suffice for more than decade.
> >May be, close to 2 decades.
> Newer DDR standards are not primarily about chip capacity (currently
> the largest DDR5 chips are not larger than supported by DDR4: 16Gb),
> but about transfer rate. I.e., if the transfer rate becomes faster
> than 16x the internal cycle time of DRAM, they have to increase the
> minimum burst length (and decrease the (sub)channel width) to benefit
> from that increase. And currently I see no sign that the transfer
> rate increases are slowing down. DDR4 was released 7 years after
> DDR3; DDR5 was released 6 years after DDR4. On the PCIe front, the
> release cadence seems to be even higher.
> - anton
> --
> 'Anyone trying for "industrial quality" ISA should avoid undefined behavior.'
> Mitch Alsup, <c17fcd89-f024-40e7...@googlegroups.com>

I can not formulate it with sufficient rigor, but my opinion is that need
in new DDR standards is very much a function of non-stability (hopefully growth)
of the market in term of average capacity.
In terms of trade offs between bandwidth, capacity. latency, energy consumption
and configurability, DDR3/4/5 interfaces are jacks of all trades, masters of none.
That is very valuable, when there is a rapid change and no time for specialization.
In other, more stable market conditions it is something that pushes your out of
all commodities into area that economists call "long tail of Pareto curve".
That's not the worst case to be, but also not good enough to invent the next
generation of the JEDEC Standard.

As to commodities, we already see enough to guess with good certainty how
they will look if growth of capacity stops.

On low-capacity (consumer) side of things it would be either LPDDR4 or something
based on LPDDR4 competing against current and future generations of HBM.

On high-capacity (server) side it would be yet another variation of "fully-buffered"
idea, most likely based on the phy as PCIeN. Yes, it failed all previous times,
but it failed exactly because the market needs were dynamic. This time it would
be different. Also, it's already different from last 40 years in terms of much bigger
centralization of the server market and because of "this or that shite as a service".
As you can guess, I don't like these new trends, but can't say that they are not here
because of my dislike.

On 5/27/2022 9:11 AM, Michael S wrote:
> On Friday, May 27, 2022 at 3:15:03 PM UTC+3, Anton Ertl wrote:
>> Michael S <already...@yahoo.com> writes:
>>> On Friday, May 27, 2022 at 10:53:05 AM UTC+3, Anton Ertl wrote:
>>>> In the long run, the (sub)channels are going to become narrower,=20
>>>
>>> You still assume that we are ahead of long run [of Moore law].
>>> I am less optimistic. I seems plausible that provisions for 64gb devices
>>> that is already in DDR5 will suffice for more than decade.
>>> May be, close to 2 decades.
>> Newer DDR standards are not primarily about chip capacity (currently
>> the largest DDR5 chips are not larger than supported by DDR4: 16Gb),
>> but about transfer rate. I.e., if the transfer rate becomes faster
>> than 16x the internal cycle time of DRAM, they have to increase the
>> minimum burst length (and decrease the (sub)channel width) to benefit
>> from that increase. And currently I see no sign that the transfer
>> rate increases are slowing down. DDR4 was released 7 years after
>> DDR3; DDR5 was released 6 years after DDR4. On the PCIe front, the
>> release cadence seems to be even higher.
>> - anton
>> --
>> 'Anyone trying for "industrial quality" ISA should avoid undefined behavior.'
>> Mitch Alsup, <c17fcd89-f024-40e7...@googlegroups.com>
>
> I can not formulate it with sufficient rigor, but my opinion is that need
> in new DDR standards is very much a function of non-stability (hopefully growth)
> of the market in term of average capacity.

I disagree. If it were true, then newer generations of the standard
would be much simpler; just add an address bit or two. The fact that
each standard provides a substantial increase in data rate/bandwidth
over the previous one, which is harder than adding an address pin, says
that data rate/bandwidth is the driving factor, and capacity comes in
second. Unfortunately, latency has proven resistant to substantial
improvements :-(

--
- Stephen Fuld
(e-mail address disguised to prevent spam)

Michael S <already5chosen@yahoo.com> writes:
>As to commodities, we already see enough to guess with good certainty how
>they will look if growth of capacity stops.
>
>On low-capacity (consumer) side of things it would be either LPDDR4 or something
>based on LPDDR4 competing against current and future generations of HBM.

The main shortcoming of LPDDRx memory is that it is soldered on the
board (you gain transfer rate and lower power consumption from that).
That seems to be too unflexible for most desktops (except Apple
customers).

HBM is connected through an MCM, and generally seems too expensive for
consumers.

>On high-capacity (server) side it would be yet another variation of "fully-buffered"
>idea, most likely based on the phy as PCIeN.

There is something new called CXL; the next version of CXL is expected
soon and based on PCIe 6.0 PHY. But from what I read about it, it
does not seem to be a replacement for RDIMMs or LRDIMMs; it's probably
more that you have some boards that have memory in the form of LRDIMMs
if high-capacity is relevant that probably also have one or several
CPUs, and are connected to other boards through CXL. I expect that
certain specialized big iron things will make use of it somewhere, but
I don't expect that it's a replacement for RDIMMs or LRDIMMs.

- anton
--
'Anyone trying for "industrial quality" ISA should avoid undefined behavior.'
Mitch Alsup, <c17fcd89-f024-40e7-a594-88a85ac10d20o@googlegroups.com>

On Friday, May 27, 2022 at 11:58:14 AM UTC-5, Stephen Fuld wrote:
> On 5/27/2022 9:11 AM, Michael S wrote:
> > On Friday, May 27, 2022 at 3:15:03 PM UTC+3, Anton Ertl wrote:
> >> Michael S <already...@yahoo.com> writes:
> >>> On Friday, May 27, 2022 at 10:53:05 AM UTC+3, Anton Ertl wrote:
> >>>> In the long run, the (sub)channels are going to become narrower,=20
> >>>
> >>> You still assume that we are ahead of long run [of Moore law].
> >>> I am less optimistic. I seems plausible that provisions for 64gb devices
> >>> that is already in DDR5 will suffice for more than decade.
> >>> May be, close to 2 decades.
> >> Newer DDR standards are not primarily about chip capacity (currently
> >> the largest DDR5 chips are not larger than supported by DDR4: 16Gb),
> >> but about transfer rate. I.e., if the transfer rate becomes faster
> >> than 16x the internal cycle time of DRAM, they have to increase the
> >> minimum burst length (and decrease the (sub)channel width) to benefit
> >> from that increase. And currently I see no sign that the transfer
> >> rate increases are slowing down. DDR4 was released 7 years after
> >> DDR3; DDR5 was released 6 years after DDR4. On the PCIe front, the
> >> release cadence seems to be even higher.
> >> - anton
> >> --
> >> 'Anyone trying for "industrial quality" ISA should avoid undefined behavior.'
> >> Mitch Alsup, <c17fcd89-f024-40e7...@googlegroups.com>
> >
> > I can not formulate it with sufficient rigor, but my opinion is that need
> > in new DDR standards is very much a function of non-stability (hopefully growth)
> > of the market in term of average capacity.
<
> I disagree. If it were true, then newer generations of the standard
> would be much simpler; just add an address bit or two. The fact that
> each standard provides a substantial increase in data rate/bandwidth
> over the previous one, which is harder than adding an address pin, says
> that data rate/bandwidth is the driving factor, and capacity comes in
> second. Unfortunately, latency has proven resistant to substantial
> improvements :-(
<
DRAM chips have rather constant area. Passing over a linear amount of
real-estate is a wire delay problem. So that transistors are getting faster
and wire is getting slower has finally become manifest in DRAM. {actually
DRAM bank access to pin drivers has stayed rather constant since DDR2.}
<
But, does anyone know the ratio of consumer DRAMs (all those PCs) versus
the server DRAM (all those servers) in total chip sold/bought/delivered ?
>
>
>
> --
> - Stephen Fuld
> (e-mail address disguised to prevent spam)

On Friday, May 27, 2022 at 1:14:35 PM UTC-5, Anton Ertl wrote:
> Michael S <already...@yahoo.com> writes:
> >As to commodities, we already see enough to guess with good certainty how
> >they will look if growth of capacity stops.
> >
> >On low-capacity (consumer) side of things it would be either LPDDR4 or something
> >based on LPDDR4 competing against current and future generations of HBM.
> The main shortcoming of LPDDRx memory is that it is soldered on the
> board (you gain transfer rate and lower power consumption from that).
> That seems to be too unflexible for most desktops (except Apple
> customers).
>
> HBM is connected through an MCM, and generally seems too expensive for
> consumers.
<
LPDDR and HBM are fine for cell phones and GPUs and NON-Upgradable PCs.
<
Last time I looked, cell phones are 10× the market of PCs.
<
> >On high-capacity (server) side it would be yet another variation of "fully-buffered"
> >idea, most likely based on the phy as PCIeN.
<
PCIe 6.0 ? PAM4 and 16GHz clock rate.
<
> There is something new called CXL; the next version of CXL is expected
> soon and based on PCIe 6.0 PHY. But from what I read about it, it
> does not seem to be a replacement for RDIMMs or LRDIMMs; it's probably
> more that you have some boards that have memory in the form of LRDIMMs
> if high-capacity is relevant that probably also have one or several
> CPUs, and are connected to other boards through CXL. I expect that
> certain specialized big iron things will make use of it somewhere, but
> I don't expect that it's a replacement for RDIMMs or LRDIMMs.
<
The main problem is that pins are expensive (area, poser, and skew)
<
> - anton
> --
> 'Anyone trying for "industrial quality" ISA should avoid undefined behavior.'
> Mitch Alsup, <c17fcd89-f024-40e7...@googlegroups.com>

On Friday, May 27, 2022 at 9:14:35 PM UTC+3, Anton Ertl wrote:
> Michael S <already...@yahoo.com> writes:
> >As to commodities, we already see enough to guess with good certainty how
> >they will look if growth of capacity stops.
> >
> >On low-capacity (consumer) side of things it would be either LPDDR4 or something
> >based on LPDDR4 competing against current and future generations of HBM.
> The main shortcoming of LPDDRx memory is that it is soldered on the
> board (you gain transfer rate and lower power consumption from that).
> That seems to be too unflexible for most desktops (except Apple
> customers).

As mentioned by Mitch, DDR4/5 is already outsold by LPDDR4.
And certainly Apple is not the only factor in that. I didn't check
but would expect that any big laptop manufacturer sells more
LPDDR4 machines than of those base on DDR4/5.
Also I'd expect that all AIO desktops and majority of miniPC desktops
are LPDDR4.

>
> HBM is connected through an MCM, and generally seems too expensive for
> consumers.

Not every consumer is cheap. Think gamers, not the casual ones, but addicts.
Or everybody buying Apple.

> >On high-capacity (server) side it would be yet another variation of "fully-buffered"
> >idea, most likely based on the phy as PCIeN.
> There is something new called CXL; the next version of CXL is expected
> soon and based on PCIe 6.0 PHY. But from what I read about it, it
> does not seem to be a replacement for RDIMMs or LRDIMMs; it's probably
> more that you have some boards that have memory in the form of LRDIMMs
> if high-capacity is relevant that probably also have one or several
> CPUs, and are connected to other boards through CXL. I expect that
> certain specialized big iron things will make use of it somewhere, but
> I don't expect that it's a replacement for RDIMMs or LRDIMMs.

Yes, except that I expect that if they went to trouble of doing extension box
they probably are going to build it for cheaper and faster RDIMMs rather
than for LRDIMMS
But my point is that people that build this boxen don't need new memory standard
every 5-7 years. Esp. when capacity does not go up. Even with DDR4 they likely
have internal bandwidth that easily exceeds bandwidth of their external COMM
channels, because there is no other way to get required capacity.
So, if they need DDR5 then only because of promised extra capacity.

My another point is that in stable market middlemen are pushed out.
This boxen would be built either by system manufacturers or, more
likely, by memory manufacturers. And in the second case
memory-to-COMM-ASIC interface does not even have to follow any
JEDEC standard.

> - anton
> --
> 'Anyone trying for "industrial quality" ISA should avoid undefined behavior.'
> Mitch Alsup, <c17fcd89-f024-40e7...@googlegroups.com>

On Saturday, May 28, 2022 at 1:44:33 PM UTC-5, Michael S wrote:
> On Friday, May 27, 2022 at 9:14:35 PM UTC+3, Anton Ertl wrote:
> > Michael S <already...@yahoo.com> writes:
> > >As to commodities, we already see enough to guess with good certainty how
> > >they will look if growth of capacity stops.
> > >
> > >On low-capacity (consumer) side of things it would be either LPDDR4 or something
> > >based on LPDDR4 competing against current and future generations of HBM.
> > The main shortcoming of LPDDRx memory is that it is soldered on the
> > board (you gain transfer rate and lower power consumption from that).
> > That seems to be too unflexible for most desktops (except Apple
> > customers).
> As mentioned by Mitch, DDR4/5 is already outsold by LPDDR4.
> And certainly Apple is not the only factor in that. I didn't check
> but would expect that any big laptop manufacturer sells more
> LPDDR4 machines than of those base on DDR4/5.
> Also I'd expect that all AIO desktops and majority of miniPC desktops
> are LPDDR4.
> >
> > HBM is connected through an MCM, and generally seems too expensive for
> > consumers.
> Not every consumer is cheap. Think gamers, not the casual ones, but addicts.
> Or everybody buying Apple.
> > >On high-capacity (server) side it would be yet another variation of "fully-buffered"
> > >idea, most likely based on the phy as PCIeN.
> > There is something new called CXL; the next version of CXL is expected
> > soon and based on PCIe 6.0 PHY. But from what I read about it, it
> > does not seem to be a replacement for RDIMMs or LRDIMMs; it's probably
> > more that you have some boards that have memory in the form of LRDIMMs
> > if high-capacity is relevant that probably also have one or several
> > CPUs, and are connected to other boards through CXL. I expect that
> > certain specialized big iron things will make use of it somewhere, but
> > I don't expect that it's a replacement for RDIMMs or LRDIMMs.
> Yes, except that I expect that if they went to trouble of doing extension box
> they probably are going to build it for cheaper and faster RDIMMs rather
> than for LRDIMMS
<
> But my point is that people that build this boxen don't need new memory standard
> every 5-7 years. Esp. when capacity does not go up. Even with DDR4 they likely
> have internal bandwidth that easily exceeds bandwidth of their external COMM
> channels, because there is no other way to get required capacity.
> So, if they need DDR5 then only because of promised extra capacity.
<
I bought a new machine about 6 months ago. It has more CPUs (12) and more
memory (32 GB) than I think I will ever need. When I look while having 6 WORD
documents open (200 pages each), 3 spreadsheets, and 4 CorelDraw files open
30-70 pages each), I am sitting at 25%-30% memory capacity.
>
> My another point is that in stable market middlemen are pushed out.
> This boxen would be built either by system manufacturers or, more
> likely, by memory manufacturers. And in the second case
> memory-to-COMM-ASIC interface does not even have to follow any
> JEDEC standard.
<
Standards are for people who want "choice between" not simply "choice of".
The PC market may have already reached that "stable market" criterion
mentioned above. The only thing I want in a new PC is the ability to insert
my old SATA drives rather than moving many hours of SATA copying when I
get a new machine. {And I certainly don't want any of this in the cloud !}
<
> > - anton
> > --
> > 'Anyone trying for "industrial quality" ISA should avoid undefined behavior.'
> > Mitch Alsup, <c17fcd89-f024-40e7...@googlegroups.com>

Michael S <already5chosen@yahoo.com> writes:
>On Friday, May 27, 2022 at 9:14:35 PM UTC+3, Anton Ertl wrote:
>> Michael S <already...@yahoo.com> writes:
>> >As to commodities, we already see enough to guess with good certainty how
>> >they will look if growth of capacity stops.
>> >
>> >On low-capacity (consumer) side of things it would be either LPDDR4 or something
>> >based on LPDDR4 competing against current and future generations of HBM.
>> The main shortcoming of LPDDRx memory is that it is soldered on the
>> board (you gain transfer rate and lower power consumption from that).
>> That seems to be too unflexible for most desktops (except Apple
>> customers).
>
>As mentioned by Mitch, DDR4/5 is already outsold by LPDDR4.

[Citation needed]

Whether that's true or not, it does not change the shortcomings, or
the situation for desktops.

>And certainly Apple is not the only factor in that. I didn't check
>but would expect that any big laptop manufacturer sells more
>LPDDR4 machines than of those base on DDR4/5.

Laptops are not desktops, but anyway, let's get some data on this:

Looking at the "RAM type" category in <https://geizhals.eu/?cat=nb>:

models
5235 DDR4
1578 LPDDR4
154 DDR5
255 LPDDR5

So there are certainly many more models with DDR4/5 than with LPDDR4,
and this probably also means more sales.

The DDR5 vs. LPDDR5 numbers may indicate that the situation will be
different for DDR5 vs. LPDDR5, or it may just have to do with the kind
of models and CPUs available this early in the game. Comparing the
CPU manufacturers for DDR5 vs. LPDDR5 models, I see:

DDR5 LPDDR5
37 6 AMD
117 87 Intel
0 162 Apple

So AMD and Intel CPUs are more often built into laptops with DDR5
rather than LPDDR5 memory.

The numbers of RAM slots are also interesting:

Slots
3223 unknown (includes 0)
4349 1+
3022 2+
143 4+

And the amount of soldered RAM (note that many laptops have some RAM
soldered and a slot for expansion):

3762 none
795 4GB
1428 8GB
1316 16GB
228 32GB
43 64GB (all Apple)

>Also I'd expect that all AIO desktops and majority of miniPC desktops
>are LPDDR4.

Ok, let's check this:

All-in-one <https://geizhals.eu/?cat=sysdiv&xf=450_All-in-One>:

348 DDR4
95 LPDDR4 (all Apple)

Mini PC <https://geizhals.eu/?cat=sysdiv&xf=450_Mini-PC>:
336 DDR4
3 DDR5
25 LPDDR4
36 LPDDR5 (all Apple)

>> HBM is connected through an MCM, and generally seems too expensive for
>> consumers.
>
>Not every consumer is cheap. Think gamers, not the casual ones, but addicts.

I doubt that they are a big enough market to justify the expense of
spinning separate HBM CPUs. And I am not sure if they would be
interested. From what I read, they are into specialty DIMMs with high
clocks and low latency numbers.

>Or everybody buying Apple.

None of the Apple systems use HBM. They use LPDDR4/5. And what would
HBM buy them? Apple gets more than enough bandwidth for their larger
configurations from LPDDR5; they are still weak on capacity, but HBM
would not change that.

Concerning consumers who are willing to pay, the marketing expert
knows how to skim that willingness by giving them something that costs
only a little more to produce (such as the specialty DIMMs discussed
above). An HBM system costs a lot more to produce: changed memory
controllers, changed connectors on the CPU die, MCM, changed socket
(or maybe something like the Pentium II/III and early Athlon slots),
changed mainboards, ...

For Apple's ecosystem it's more in reach, but even they don't go
there.

[CXL]
>But my point is that people that build this boxen don't need new memory standard
>every 5-7 years. Esp. when capacity does not go up. Even with DDR4 they likely
>have internal bandwidth that easily exceeds bandwidth of their external COMM
>channels, because there is no other way to get required capacity.
>So, if they need DDR5 then only because of promised extra capacity.

A pure memory box with bandwidth that cannot even compete with DDR4 in
bandwidth (and certainly not latency) does not look to me like something
that many people would buy. I doubt that we will see such products.

- anton
--
'Anyone trying for "industrial quality" ISA should avoid undefined behavior.'
Mitch Alsup, <c17fcd89-f024-40e7-a594-88a85ac10d20o@googlegroups.com>

On Sunday, May 29, 2022 at 1:14:20 PM UTC+3, Anton Ertl wrote:
> Michael S <already...@yahoo.com> writes:
> >On Friday, May 27, 2022 at 9:14:35 PM UTC+3, Anton Ertl wrote:
> >> Michael S <already...@yahoo.com> writes:
> >> >As to commodities, we already see enough to guess with good certainty how
> >> >they will look if growth of capacity stops.
> >> >
> >> >On low-capacity (consumer) side of things it would be either LPDDR4 or something
> >> >based on LPDDR4 competing against current and future generations of HBM.
> >> The main shortcoming of LPDDRx memory is that it is soldered on the
> >> board (you gain transfer rate and lower power consumption from that).
> >> That seems to be too unflexible for most desktops (except Apple
> >> customers).
> >
> >As mentioned by Mitch, DDR4/5 is already outsold by LPDDR4.
> [Citation needed]
>
> Whether that's true or not, it does not change the shortcomings, or
> the situation for desktops.
> >And certainly Apple is not the only factor in that. I didn't check
> >but would expect that any big laptop manufacturer sells more
> >LPDDR4 machines than of those base on DDR4/5.
> Laptops are not desktops, but anyway, let's get some data on this:
>
> Looking at the "RAM type" category in <https://geizhals.eu/?cat=nb>:
>
> models
> 5235 DDR4
> 1578 LPDDR4
> 154 DDR5
> 255 LPDDR5
>
> So there are certainly many more models with DDR4/5 than with LPDDR4,
> and this probably also means more sales.
>

I'm not sure that it follows.
Simple models like Dell Latitude 3320 probably outsel more advanced models by significant factor.
But overall I agree that I overestimated popularity of LPDDR in non-apple PCs.

> The DDR5 vs. LPDDR5 numbers may indicate that the situation will be
> different for DDR5 vs. LPDDR5, or it may just have to do with the kind
> of models and CPUs available this early in the game. Comparing the
> CPU manufacturers for DDR5 vs. LPDDR5 models, I see:
>
> DDR5 LPDDR5
> 37 6 AMD
> 117 87 Intel
> 0 162 Apple
>
> So AMD and Intel CPUs are more often built into laptops with DDR5
> rather than LPDDR5 memory.
>
> The numbers of RAM slots are also interesting:
>
> Slots
> 3223 unknown (includes 0)
> 4349 1+
> 3022 2+
> 143 4+
>
> And the amount of soldered RAM (note that many laptops have some RAM
> soldered and a slot for expansion):
>
> 3762 none
> 795 4GB
> 1428 8GB
> 1316 16GB
> 228 32GB
> 43 64GB (all Apple)
> >Also I'd expect that all AIO desktops and majority of miniPC desktops
> >are LPDDR4.
> Ok, let's check this:
>
> All-in-one <https://geizhals.eu/?cat=sysdiv&xf=450_All-in-One>:
>
> 348 DDR4
> 95 LPDDR4 (all Apple)
>
> Mini PC <https://geizhals.eu/?cat=sysdiv&xf=450_Mini-PC>:
> 336 DDR4
> 3 DDR5
> 25 LPDDR4
> 36 LPDDR5 (all Apple)
> >> HBM is connected through an MCM, and generally seems too expensive for
> >> consumers.
> >
> >Not every consumer is cheap. Think gamers, not the casual ones, but addicts.
> I doubt that they are a big enough market to justify the expense of
> spinning separate HBM CPUs. And I am not sure if they would be
> interested. From what I read, they are into specialty DIMMs with high
> clocks and low latency numbers.
>
> >Or everybody buying Apple.
>
> None of the Apple systems use HBM. They use LPDDR4/5. And what would
> HBM buy them?

I suppose, they want to reach the level of 3D graphics performance that
is comparable with best discrete cards of NVidia and AMD. Not necessarily
to match the very top, but to be close. For that they will need HBM.

> Apple gets more than enough bandwidth for their larger
> configurations from LPDDR5; they are still weak on capacity, but HBM
> would not change that.
>
> Concerning consumers who are willing to pay, the marketing expert
> knows how to skim that willingness by giving them something that costs
> only a little more to produce (such as the specialty DIMMs discussed
> above). An HBM system costs a lot more to produce: changed memory
> controllers, changed connectors on the CPU die, MCM, changed socket
> (or maybe something like the Pentium II/III and early Athlon slots),
> changed mainboards, ...
>
> For Apple's ecosystem it's more in reach, but even they don't go
> there.
>
> [CXL]
> >But my point is that people that build this boxen don't need new memory standard
> >every 5-7 years. Esp. when capacity does not go up. Even with DDR4 they likely
> >have internal bandwidth that easily exceeds bandwidth of their external COMM
> >channels, because there is no other way to get required capacity.
> >So, if they need DDR5 then only because of promised extra capacity.
> A pure memory box with bandwidth that cannot even compete with DDR4 in
> bandwidth (and certainly not latency) does not look to me like something
> that many people would buy. I doubt that we will see such products.

Who says it does not compete with DDR4 in bandwidth?
I expect that such product, in order to be sell-able, has to beat DDR4 in bandwidth
per pin, but not necessarily by large factor. Factor of 1.5 to 2 is enough.
But most importantly it has to beat anything else on the market (i.e. mostly
RL-DIMMs) at least by factor of 4 in *capacity* per pin. Preferably, it should
promise factor of 6-8, at least as option for future upgrade.
In order to achieve that it would need pretty wide internal buses, at very least
8x wider that external COMM links, more likely 12x wider.
So, even with DDR4 internal bandwidth easily beats bandwidth of the link.
I just do not see how they can possible get required capacity otherwise.

As to latency, of course it would be bad. But alternatives of similar capacity
are "super-NUMA" like HP Superdome Flex, which is likely even worse if the app
isn't very NUMA-friendly. Another alternative is big IBM z box. Here latency is
better (than HP), due to relatively tightly connected processor racks, but the
price is high.

> - anton
> --
> 'Anyone trying for "industrial quality" ISA should avoid undefined behavior.'
> Mitch Alsup, <c17fcd89-f024-40e7...@googlegroups.com>

Michael S <already5chosen@yahoo.com> writes:
>On Sunday, May 29, 2022 at 1:14:20 PM UTC+3, Anton Ertl wrote:
>> Looking at the "RAM type" category in <https://geizhals.eu/?cat=nb>:
>>
>> models
>> 5235 DDR4
>> 1578 LPDDR4
>> 154 DDR5
>> 255 LPDDR5
>>
>> So there are certainly many more models with DDR4/5 than with LPDDR4,
>> and this probably also means more sales.
>>
>
>I'm not sure that it follows.
>Simple models like Dell Latitude 3320 probably outsel more advanced models by significant factor.

Maybe. And maybe even cheaper models like, say, the ASUS Business P1
P1511CEA-BQ749 (offered by 81 dealers vs. 57 for the most-offered Dell
Latitude 3320 variant) outsell the Dell Latitude 3320 by a large
margin. Given that we have no sales numbers, I'll use the model
numbers as approximation.

>> None of the Apple systems use HBM. They use LPDDR4/5. And what would
>> HBM buy them?
>
>I suppose, they want to reach the level of 3D graphics performance that
>is comparable with best discrete cards of NVidia and AMD. Not necessarily
>to match the very top, but to be close. For that they will need HBM.

bits bandwidth
1024 816GB/s M1 Ultra
256 576GB/s Radeon RX 6950XT
384 1008GB/s Geforce RTX 3090 Ti

Given that the M1 Ultra has 96MB of system level cache (compared to
128MB of Infinity Cache for the 6950XT, with cache being no feature
that Nvidia touts), Apple already has comparable bandwidth. No HBM
needed.

>> [CXL]
>> >But my point is that people that build this boxen don't need new memory standard
>> >every 5-7 years. Esp. when capacity does not go up. Even with DDR4 they likely
>> >have internal bandwidth that easily exceeds bandwidth of their external COMM
>> >channels, because there is no other way to get required capacity.
>> >So, if they need DDR5 then only because of promised extra capacity.
>> A pure memory box with bandwidth that cannot even compete with DDR4 in
>> bandwidth (and certainly not latency) does not look to me like something
>> that many people would buy. I doubt that we will see such products.
>
>Who says it does not compete with DDR4 in bandwidth?

You do: "Even with DDR4 they likely have internal bandwidth that
easily exceeds bandwidth of their external COMM channels".

As for the rest: A box with more memory than supported by LRDIMMs on,
e.g., EPYC boards will probably find a market, but I doubt that it
will replace LRDIMMs.

- anton
--
'Anyone trying for "industrial quality" ISA should avoid undefined behavior.'
Mitch Alsup, <c17fcd89-f024-40e7-a594-88a85ac10d20o@googlegroups.com>

Anton Ertl <anton@mips.complang.tuwien.ac.at> wrote:
> Michael S <already5chosen@yahoo.com> writes:
>> On Friday, May 27, 2022 at 9:14:35 PM UTC+3, Anton Ertl wrote:
>>> Michael S <already...@yahoo.com> writes:
>>>> As to commodities, we already see enough to guess with good certainty how
>>>> they will look if growth of capacity stops.
>>>>
>>>> On low-capacity (consumer) side of things it would be either LPDDR4 or something
>>>> based on LPDDR4 competing against current and future generations of HBM.
>>> The main shortcoming of LPDDRx memory is that it is soldered on the
>>> board (you gain transfer rate and lower power consumption from that).
>>> That seems to be too unflexible for most desktops (except Apple
>>> customers).
>>
>> As mentioned by Mitch, DDR4/5 is already outsold by LPDDR4.
>
> [Citation needed]
>
> Whether that's true or not, it does not change the shortcomings, or
> the situation for desktops.
>
>> And certainly Apple is not the only factor in that. I didn't check
>> but would expect that any big laptop manufacturer sells more
>> LPDDR4 machines than of those base on DDR4/5.
>
> Laptops are not desktops, but anyway, let's get some data on this:
>
> Looking at the "RAM type" category in <https://geizhals.eu/?cat=nb>:
>
> models
> 5235 DDR4
> 1578 LPDDR4
> 154 DDR5
> 255 LPDDR5

The soldered ram in a laptop should be LPDDR while the DIMM in the same
laptop is not low power, so you may have some double counts in this list.

In the fifth generation LPDDR is clearly winning.

> So there are certainly many more models with DDR4/5 than with LPDDR4,
> and this probably also means more sales.
>
> The DDR5 vs. LPDDR5 numbers may indicate that the situation will be
> different for DDR5 vs. LPDDR5, or it may just have to do with the kind
> of models and CPUs available this early in the game. Comparing the
> CPU manufacturers for DDR5 vs. LPDDR5 models, I see:
>
> DDR5 LPDDR5
> 37 6 AMD
> 117 87 Intel
> 0 162 Apple
>
> So AMD and Intel CPUs are more often built into laptops with DDR5
> rather than LPDDR5 memory.
>
> The numbers of RAM slots are also interesting:
>
> Slots
> 3223 unknown (includes 0)
> 4349 1+
> 3022 2+
> 143 4+
>
> And the amount of soldered RAM (note that many laptops have some RAM
> soldered and a slot for expansion):
>
> 3762 none
> 795 4GB
> 1428 8GB
> 1316 16GB
> 228 32GB
> 43 64GB (all Apple)
>
>> Also I'd expect that all AIO desktops and majority of miniPC desktops
>> are LPDDR4.
>
> Ok, let's check this:
>
> All-in-one <https://geizhals.eu/?cat=sysdiv&xf=450_All-in-One>:
>
> 348 DDR4
> 95 LPDDR4 (all Apple)
>
> Mini PC <https://geizhals.eu/?cat=sysdiv&xf=450_Mini-PC>:
> 336 DDR4
> 3 DDR5
> 25 LPDDR4
> 36 LPDDR5 (all Apple)
>
>>> HBM is connected through an MCM, and generally seems too expensive for
>>> consumers.
>>
>> Not every consumer is cheap. Think gamers, not the casual ones, but addicts.
>
> I doubt that they are a big enough market to justify the expense of
> spinning separate HBM CPUs. And I am not sure if they would be
> interested. From what I read, they are into specialty DIMMs with high
> clocks and low latency numbers.
>
>> Or everybody buying Apple.
>
> None of the Apple systems use HBM. They use LPDDR4/5. And what would
> HBM buy them? Apple gets more than enough bandwidth for their larger
> configurations from LPDDR5; they are still weak on capacity, but HBM
> would not change that.
>
> Concerning consumers who are willing to pay, the marketing expert
> knows how to skim that willingness by giving them something that costs
> only a little more to produce (such as the specialty DIMMs discussed
> above). An HBM system costs a lot more to produce: changed memory
> controllers, changed connectors on the CPU die, MCM, changed socket
> (or maybe something like the Pentium II/III and early Athlon slots),
> changed mainboards, ...
>
> For Apple's ecosystem it's more in reach, but even they don't go
> there.
>
> [CXL]
>> But my point is that people that build this boxen don't need new memory standard
>> every 5-7 years. Esp. when capacity does not go up. Even with DDR4 they likely
>> have internal bandwidth that easily exceeds bandwidth of their external COMM
>> channels, because there is no other way to get required capacity.
>> So, if they need DDR5 then only because of promised extra capacity.
>
> A pure memory box with bandwidth that cannot even compete with DDR4 in
> bandwidth (and certainly not latency) does not look to me like something
> that many people would buy. I doubt that we will see such products.
>
> - anton

On Sunday, May 29, 2022 at 7:06:16 PM UTC+3, Anton Ertl wrote:
> Michael S <already...@yahoo.com> writes:
> >On Sunday, May 29, 2022 at 1:14:20 PM UTC+3, Anton Ertl wrote:
> >> Looking at the "RAM type" category in <https://geizhals.eu/?cat=nb>:
> >>
> >> models
> >> 5235 DDR4
> >> 1578 LPDDR4
> >> 154 DDR5
> >> 255 LPDDR5
> >>
> >> So there are certainly many more models with DDR4/5 than with LPDDR4,
> >> and this probably also means more sales.
> >>
> >
> >I'm not sure that it follows.
> >Simple models like Dell Latitude 3320 probably outsel more advanced models by significant factor.
> Maybe. And maybe even cheaper models like, say, the ASUS Business P1
> P1511CEA-BQ749 (offered by 81 dealers vs. 57 for the most-offered Dell
> Latitude 3320 variant)

Or. may be, even lower end ASUS VivoBook E12, which resembles ACER gears.
Who knows?

> outsell the Dell Latitude 3320 by a large
> margin. Given that we have no sales numbers, I'll use the model
> numbers as approximation.

At very least, we know that according to Gartner estimates Dell PC division
as a whole outsells ASUS PC division as whole, 2.5 to 1. So, fewer dealers,
but more PCs sold by each dealer.
Also, I would think that [at retail] ASUS has many more models than Dell.
At least that's what I see in a couple of shops that I just checked. In one shop
the difference in favor of ASUS was 5 to 1.
So, it seems, Dell has to sell on average many more units of each model.
Of course, it does not mean that most sold Dell model is L3320.

By the way, I don't know how it works in your country, but here Latitude is not
seen in shops at all. Only Inspiron , Vostro and occasional Alienware.
But at work Latitude is everywhere.

> >> None of the Apple systems use HBM. They use LPDDR4/5. And what would
> >> HBM buy them?
> >
> >I suppose, they want to reach the level of 3D graphics performance that
> >is comparable with best discrete cards of NVidia and AMD. Not necessarily
> >to match the very top, but to be close. For that they will need HBM.
> bits bandwidth
> 1024 816GB/s M1 Ultra
> 256 576GB/s Radeon RX 6950XT
> 384 1008GB/s Geforce RTX 3090 Ti
>
> Given that the M1 Ultra has 96MB of system level cache (compared to
> 128MB of Infinity Cache for the 6950XT, with cache being no feature
> that Nvidia touts), Apple already has comparable bandwidth. No HBM
> needed.
> >> [CXL]
> >> >But my point is that people that build this boxen don't need new memory standard
> >> >every 5-7 years. Esp. when capacity does not go up. Even with DDR4 they likely
> >> >have internal bandwidth that easily exceeds bandwidth of their external COMM
> >> >channels, because there is no other way to get required capacity.
> >> >So, if they need DDR5 then only because of promised extra capacity.
> >> A pure memory box with bandwidth that cannot even compete with DDR4 in
> >> bandwidth (and certainly not latency) does not look to me like something
> >> that many people would buy. I doubt that we will see such products.
> >
> >Who says it does not compete with DDR4 in bandwidth?
> You do: "Even with DDR4 they likely have internal bandwidth that
> easily exceeds bandwidth of their external COMM channels".
>

Hopefully, now you paid attention to qualifiers "internal and "external".

> As for the rest: A box with more memory than supported by LRDIMMs on,
> e.g., EPYC boards will probably find a market, but I doubt that it
> will replace LRDIMMs.

It does not have to replace LR-DIMM in order to make DDR6 unnecessary or,
at least, to delay its introduction.
I rather see it as an opposite - LR-DIMM are in the same boat with those future
external boxes. Both serve to extend the usefulness of given generation of
DDRn, helping to reduce the necessity of the next generation.

> - anton
> --
> 'Anyone trying for "industrial quality" ISA should avoid undefined behavior.'
> Mitch Alsup, <c17fcd89-f024-40e7...@googlegroups.com>

Brett <ggtgp@yahoo.com> writes:
>Anton Ertl <anton@mips.complang.tuwien.ac.at> wrote:
>> Looking at the "RAM type" category in <https://geizhals.eu/?cat=nb>:
>>
>> models
>> 5235 DDR4
>> 1578 LPDDR4
>> 154 DDR5
>> 255 LPDDR5
>
>The soldered ram in a laptop should be LPDDR while the DIMM in the same
>laptop is not low power,

While the memory controller in laptop chips like the Tiger Lake can
handle DDR4 or LPDDR4, they cannot mix them, for whatever reason. So
if a laptop has soldered RAM and a DIMM slot, the soldered RAM is DDR4
(or DDR5), not LPDDRx.

E.g., the Fujitsu Lifebook U7311 has 8GB of soldered RAM and a slot
for a SO-DIMM. dmidecode -t memory tells me [I shortened the output]:

Handle 0x000D, DMI type 17, 92 bytes
Memory Device
Size: 8 GB
Locator: Onboard (Controller0-ChannelA)
Type: DDR4
Type Detail: Synchronous
Speed: 3200 MT/s
Manufacturer: Samsung

Handle 0x0011, DMI type 17, 92 bytes
Memory Device
Size: 32 GB
Locator: DIMM0 (Controller1-ChannelA)
Bank Locator: BANK 0
Type: DDR4
Type Detail: Synchronous
Speed: 3200 MT/s
Manufacturer: Crucial Technology

The first is the soldered-on memory (Locator: Onboard), the second is
an additional SO-DIMM. Both are DDR4.

- anton
--
'Anyone trying for "industrial quality" ISA should avoid undefined behavior.'
Mitch Alsup, <c17fcd89-f024-40e7-a594-88a85ac10d20o@googlegroups.com>

On Sunday, May 29, 2022 at 4:41:30 PM UTC-5, Anton Ertl wrote:
> Brett <gg...@yahoo.com> writes:
> >Anton Ertl <an...@mips.complang.tuwien.ac.at> wrote:
> >> Looking at the "RAM type" category in <https://geizhals.eu/?cat=nb>:
> >>
> >> models
> >> 5235 DDR4
> >> 1578 LPDDR4
> >> 154 DDR5
> >> 255 LPDDR5
> >
> >The soldered ram in a laptop should be LPDDR while the DIMM in the same
> >laptop is not low power,
> While the memory controller in laptop chips like the Tiger Lake can
> handle DDR4 or LPDDR4, they cannot mix them, for whatever reason. So
> if a laptop has soldered RAM and a DIMM slot, the soldered RAM is DDR4
> (or DDR5), not LPDDRx.
<
The inability to mix and match in laptop chips is likely a fuse programmed
at testing time.
>
> E.g., the Fujitsu Lifebook U7311 has 8GB of soldered RAM and a slot
> for a SO-DIMM. dmidecode -t memory tells me [I shortened the output]:
>
> Handle 0x000D, DMI type 17, 92 bytes
> Memory Device
> Size: 8 GB
> Locator: Onboard (Controller0-ChannelA)
> Type: DDR4
> Type Detail: Synchronous
> Speed: 3200 MT/s
> Manufacturer: Samsung
>
> Handle 0x0011, DMI type 17, 92 bytes
> Memory Device
> Size: 32 GB
> Locator: DIMM0 (Controller1-ChannelA)
> Bank Locator: BANK 0
> Type: DDR4
> Type Detail: Synchronous
> Speed: 3200 MT/s
> Manufacturer: Crucial Technology
>
> The first is the soldered-on memory (Locator: Onboard), the second is
> an additional SO-DIMM. Both are DDR4.
> - anton
> --
> 'Anyone trying for "industrial quality" ISA should avoid undefined behavior.'
> Mitch Alsup, <c17fcd89-f024-40e7...@googlegroups.com>

On 2022-05-28, MitchAlsup wrote:
> On Saturday, May 28, 2022 at 1:44:33 PM UTC-5, Michael S wrote:

[snip]

>> But my point is that people that build this boxen don't need new memory standard
>> every 5-7 years. Esp. when capacity does not go up. Even with DDR4 they likely
>> have internal bandwidth that easily exceeds bandwidth of their external COMM
>> channels, because there is no other way to get required capacity.
>> So, if they need DDR5 then only because of promised extra capacity.
> <
> I bought a new machine about 6 months ago. It has more CPUs (12) and more
> memory (32 GB) than I think I will ever need. When I look while having 6 WORD
> documents open (200 pages each), 3 spreadsheets, and 4 CorelDraw files open
> 30-70 pages each), I am sitting at 25%-30% memory capacity.

A few things that I have learned about CPU cores and memory consumption:

1. With every extra core you want more RAM. Twice the number of cores
roughly translates to twice the memory requirements. My typical use case
is parallel compilation (e.g. building LLVM), where each CPU core gets
its own compiler process, and each process consumes RAM independently of
the other. For embarrassingly parallel tasks multi-core CPUs are king.

2. I regularly enough run virtual machines to care about having enough
RAM to support both a fully working host environment and a working VM
environment. Usually the VM uses 25-50% of the host RAM, so out of my
32 GB RAM I sometimes have to do with 16 GB available RAM when the VM
is running.

/Marcus

Michael S <already5chosen@yahoo.com> writes:
>On Sunday, May 29, 2022 at 7:06:16 PM UTC+3, Anton Ertl wrote:
>> Michael S <already...@yahoo.com> writes:
>> >On Sunday, May 29, 2022 at 1:14:20 PM UTC+3, Anton Ertl wrote:
>> >> Looking at the "RAM type" category in <https://geizhals.eu/?cat=nb>:
>> >>
>> >> models
>> >> 5235 DDR4
>> >> 1578 LPDDR4
>> >> 154 DDR5
>> >> 255 LPDDR5
>> >>
>> >> So there are certainly many more models with DDR4/5 than with LPDDR4,
>> >> and this probably also means more sales.
>> >>
>> >
>> >I'm not sure that it follows.
>> >Simple models like Dell Latitude 3320 probably outsel more advanced models by significant factor.
>> Maybe. And maybe even cheaper models like, say, the ASUS Business P1
>> P1511CEA-BQ749 (offered by 81 dealers vs. 57 for the most-offered Dell
>> Latitude 3320 variant)
>
>Or. may be, even lower end ASUS VivoBook E12

Unlikely at the moment, because none are offered. Looking at one
model, I see that it was last offered in February 2019.

>At very least, we know that according to Gartner estimates Dell PC division
>as a whole outsells ASUS PC division as whole, 2.5 to 1. So, fewer dealers,
>but more PCs sold by each dealer.
>Also, I would think that [at retail] ASUS has many more models than Dell.

ASUS Dell
662 468 <https://geizhals.eu/?cat=nb> (Laptops).
61 227 <https://geizhals.eu/?cat=sysdiv> (Systems)

>By the way, I don't know how it works in your country, but here Latitude is not
>seen in shops at all. Only Inspiron , Vostro and occasional Alienware.
>But at work Latitude is everywhere.

I don't know how it works in my country, and I don't go to shops
looking what brands of laptops they have, but I don't see that many
Dells overall when I see other people's laptops. For those that I
see, I have no idea what models they are. There's a central
laptop/tablet ordering system two times per year from the University
of Vienna that my university participates in, and I guess that many
people buy laptops through this system; the most recent one was
<https://ubook.at/fileadmin/content/broschueren/ubook_sose2022.pdf>.
The only Dell Latitude there cost EUR 2239, and I doubt that many
people bought it.

Anyway, if we assume that Dell sells particularly many laptops, and
that we need to study them separately, there the RAM type statistics
is:

models
362 DDR4
57 LPDDR4
17 DDR5
4 LPDDR5

So for Dell laptops the balance is even less in favour of LPDDR4. Do
people all buy these 57 LPDDR4 models, and Dell offers the other
models just for fun?

>> >> [CXL]
>> >> >But my point is that people that build this boxen don't need new memory standard
>> >> >every 5-7 years. Esp. when capacity does not go up. Even with DDR4 they likely
>> >> >have internal bandwidth that easily exceeds bandwidth of their external COMM
>> >> >channels, because there is no other way to get required capacity.
>> >> >So, if they need DDR5 then only because of promised extra capacity.
>> >> A pure memory box with bandwidth that cannot even compete with DDR4 in
>> >> bandwidth (and certainly not latency) does not look to me like something
>> >> that many people would buy. I doubt that we will see such products.
>> >
>> >Who says it does not compete with DDR4 in bandwidth?
>> You do: "Even with DDR4 they likely have internal bandwidth that
>> easily exceeds bandwidth of their external COMM channels".
>>
>
>Hopefully, now you paid attention to qualifiers "internal and "external".
>
>> As for the rest: A box with more memory than supported by LRDIMMs on,
>> e.g., EPYC boards will probably find a market, but I doubt that it
>> will replace LRDIMMs.
>
>It does not have to replace LR-DIMM in order to make DDR6 unnecessary or,
>at least, to delay its introduction.

Like DDR5, DDR6 will probably be about bandwidth, not capacity. The
fact that the largest DDR5 UDIMMs and RDIMMs at the moment are not
bigger than the largest DDR4 UDIMMs and RDIMMs, respectively, shows
that DDR5 is not about capacity (no DDR5 LRDIMMs yet, probably for
lack of corresponding servers).

If they can improve signaling speed enough that DDR5 is insufficient,
there will be DDR6, offering improved bandwidth. It will be there for
those who use UDIMMs, those who use RDIMMs, and those who use LRDIMMs.
And the box you have in mind will have to compete with that.

>I rather see it as an opposite - LR-DIMM are in the same boat with those future
>external boxes. Both serve to extend the usefulness of given generation of
>DDRn, helping to reduce the necessity of the next generation.

LRDIMMs increase the capacity, but not the bandwidth. But bandwidth
is the main reason for the next DDR generation.

- anton
--
'Anyone trying for "industrial quality" ISA should avoid undefined behavior.'
Mitch Alsup, <c17fcd89-f024-40e7-a594-88a85ac10d20o@googlegroups.com>

On 5/27/2022 1:54 PM, MitchAlsup wrote:
> On Friday, May 27, 2022 at 11:58:14 AM UTC-5, Stephen Fuld wrote:
>> On 5/27/2022 9:11 AM, Michael S wrote:
>>> On Friday, May 27, 2022 at 3:15:03 PM UTC+3, Anton Ertl wrote:
>>>> Michael S <already...@yahoo.com> writes:
>>>>> On Friday, May 27, 2022 at 10:53:05 AM UTC+3, Anton Ertl wrote:
>>>>>> In the long run, the (sub)channels are going to become narrower,=20
>>>>>
>>>>> You still assume that we are ahead of long run [of Moore law].
>>>>> I am less optimistic. I seems plausible that provisions for 64gb devices
>>>>> that is already in DDR5 will suffice for more than decade.
>>>>> May be, close to 2 decades.
>>>> Newer DDR standards are not primarily about chip capacity (currently
>>>> the largest DDR5 chips are not larger than supported by DDR4: 16Gb),
>>>> but about transfer rate. I.e., if the transfer rate becomes faster
>>>> than 16x the internal cycle time of DRAM, they have to increase the
>>>> minimum burst length (and decrease the (sub)channel width) to benefit
>>>> from that increase. And currently I see no sign that the transfer
>>>> rate increases are slowing down. DDR4 was released 7 years after
>>>> DDR3; DDR5 was released 6 years after DDR4. On the PCIe front, the
>>>> release cadence seems to be even higher.
>>>> - anton
>>>> --
>>>> 'Anyone trying for "industrial quality" ISA should avoid undefined behavior.'
>>>> Mitch Alsup, <c17fcd89-f024-40e7...@googlegroups.com>
>>>
>>> I can not formulate it with sufficient rigor, but my opinion is that need
>>> in new DDR standards is very much a function of non-stability (hopefully growth)
>>> of the market in term of average capacity.
> <
>> I disagree. If it were true, then newer generations of the standard
>> would be much simpler; just add an address bit or two. The fact that
>> each standard provides a substantial increase in data rate/bandwidth
>> over the previous one, which is harder than adding an address pin, says
>> that data rate/bandwidth is the driving factor, and capacity comes in
>> second. Unfortunately, latency has proven resistant to substantial
>> improvements :-(
> <
> DRAM chips have rather constant area. Passing over a linear amount of
> real-estate is a wire delay problem. So that transistors are getting faster
> and wire is getting slower has finally become manifest in DRAM. {actually
> DRAM bank access to pin drivers has stayed rather constant since DDR2.}

While all of that is true, I think it is only a part of the reason for
the long latency. Consider a hypothetical chip of the same physical
size as a current DRAM, but instead uses SRAM. Of course, it will have
smaller capacity and perhaps require more pins than the DRAM, but I
expect it will have lower latency. Is that correct?

I think that the original DRAM designers made the (IMO correct) tradeoff
to gain lower cost (fewer pins), at the cost of higher latency. This
was reinforced by the introduction of CPU caches, which are essentially
average latency reducers, hiding much of the longer latency. These
allowed DRAMS to successfully dominate the market.

However, cache is a diminishing returns game. Adding more improves
performance, but not as much as the last time you increased it. So it
may be true that reducing DRAM latency may become more important.

So what could be done? A lower overhead (fewer transfers) protocol
coupled with a different internal design and more pins (i.e. higher
cost) might be achievable. It is way out of my area of expertise, but I
just want to stimulate thoughts on this.

Of course, the marketability of such a design, even if achievable, is a
different question. :-(

--
- Stephen Fuld
(e-mail address disguised to prevent spam)

On Monday, May 30, 2022 at 1:05:22 PM UTC-5, Stephen Fuld wrote:
> On 5/27/2022 1:54 PM, MitchAlsup wrote:
> > On Friday, May 27, 2022 at 11:58:14 AM UTC-5, Stephen Fuld wrote:
> >> On 5/27/2022 9:11 AM, Michael S wrote:
> >>> On Friday, May 27, 2022 at 3:15:03 PM UTC+3, Anton Ertl wrote:
> >>>> Michael S <already...@yahoo.com> writes:
> >>>>> On Friday, May 27, 2022 at 10:53:05 AM UTC+3, Anton Ertl wrote:
> >>>>>> In the long run, the (sub)channels are going to become narrower,=20
> >>>>>
> >>>>> You still assume that we are ahead of long run [of Moore law].
> >>>>> I am less optimistic. I seems plausible that provisions for 64gb devices
> >>>>> that is already in DDR5 will suffice for more than decade.
> >>>>> May be, close to 2 decades.
> >>>> Newer DDR standards are not primarily about chip capacity (currently
> >>>> the largest DDR5 chips are not larger than supported by DDR4: 16Gb),
> >>>> but about transfer rate. I.e., if the transfer rate becomes faster
> >>>> than 16x the internal cycle time of DRAM, they have to increase the
> >>>> minimum burst length (and decrease the (sub)channel width) to benefit
> >>>> from that increase. And currently I see no sign that the transfer
> >>>> rate increases are slowing down. DDR4 was released 7 years after
> >>>> DDR3; DDR5 was released 6 years after DDR4. On the PCIe front, the
> >>>> release cadence seems to be even higher.
> >>>> - anton
> >>>> --
> >>>> 'Anyone trying for "industrial quality" ISA should avoid undefined behavior.'
> >>>> Mitch Alsup, <c17fcd89-f024-40e7...@googlegroups.com>
> >>>
> >>> I can not formulate it with sufficient rigor, but my opinion is that need
> >>> in new DDR standards is very much a function of non-stability (hopefully growth)
> >>> of the market in term of average capacity.
> > <
> >> I disagree. If it were true, then newer generations of the standard
> >> would be much simpler; just add an address bit or two. The fact that
> >> each standard provides a substantial increase in data rate/bandwidth
> >> over the previous one, which is harder than adding an address pin, says
> >> that data rate/bandwidth is the driving factor, and capacity comes in
> >> second. Unfortunately, latency has proven resistant to substantial
> >> improvements :-(
> > <
> > DRAM chips have rather constant area. Passing over a linear amount of
> > real-estate is a wire delay problem. So that transistors are getting faster
> > and wire is getting slower has finally become manifest in DRAM. {actually
> > DRAM bank access to pin drivers has stayed rather constant since DDR2.}
<
> While all of that is true, I think it is only a part of the reason for
> the long latency. Consider a hypothetical chip of the same physical
> size as a current DRAM, but instead uses SRAM. Of course, it will have
> smaller capacity and perhaps require more pins than the DRAM, but I
> expect it will have lower latency. Is that correct?
<
Back in 1998, I designed a DRAM macro for a logic process. The DRAM
macro had 4× the density of SRAM and only a few picoseconds longer
access time (in the era of 250-500 MHz processors (4ns-2ns)). So, I take
issue that DRAM is inherently slower than SRAM. DRAM implemented in
a DRAM process (moderate P-channels slow N-channels) would not be
as fast a DRAM implemented in a logic process.
<
Where designed = circuit design, timing design, layout, parasitic extraction,
SPICE simulation, sequencer, and interface to the cache busses.
<
In order to deal with the leakage of the logic process, the DRAMs were
refreshed on every cycle the cache was not being accessed. This averages
out to be some 1000× more often than the DRAM process DRAMs.
>
> I think that the original DRAM designers made the (IMO correct) tradeoff
> to gain lower cost (fewer pins), at the cost of higher latency. This
> was reinforced by the introduction of CPU caches, which are essentially
> average latency reducers, hiding much of the longer latency. These
> allowed DRAMS to successfully dominate the market.
<
packaged DRAMs are successful in the moderate latency, lowest cost per bit
{likely supplanted by Flash} market. Here, package cost is significant!
<
Did anyone ever wonder why DRAM manufactures build $2B+ FABs in order
to produce $5 dies that go in $3 packages made in $50M factory..........
>
> However, cache is a diminishing returns game. Adding more improves
> performance, but not as much as the last time you increased it. So it
> may be true that reducing DRAM latency may become more important.
<
Cache gains go in 1/SQRT( size ). 4× the cache size buys ½ the miss rate.
>
> So what could be done? A lower overhead (fewer transfers) protocol
> coupled with a different internal design and more pins (i.e. higher
> cost) might be achievable. It is way out of my area of expertise, but I
> just want to stimulate thoughts on this.
<
Servers trade a couple of clocks of latency for lots more DRAM DIMMs
and the advantages that come from more DIMMs {banks, open rows,
refresh banks not in use,...} and for them this is a good tradeoff.
>
> Of course, the marketability of such a design, even if achievable, is a
> different question. :-(
> --
> - Stephen Fuld
> (e-mail address disguised to prevent spam)

On Monday, May 30, 2022 at 9:05:22 PM UTC+3, Stephen Fuld wrote:
> On 5/27/2022 1:54 PM, MitchAlsup wrote:
> > On Friday, May 27, 2022 at 11:58:14 AM UTC-5, Stephen Fuld wrote:
> >> On 5/27/2022 9:11 AM, Michael S wrote:
> >>> On Friday, May 27, 2022 at 3:15:03 PM UTC+3, Anton Ertl wrote:
> >>>> Michael S <already...@yahoo.com> writes:
> >>>>> On Friday, May 27, 2022 at 10:53:05 AM UTC+3, Anton Ertl wrote:
> >>>>>> In the long run, the (sub)channels are going to become narrower,=20
> >>>>>
> >>>>> You still assume that we are ahead of long run [of Moore law].
> >>>>> I am less optimistic. I seems plausible that provisions for 64gb devices
> >>>>> that is already in DDR5 will suffice for more than decade.
> >>>>> May be, close to 2 decades.
> >>>> Newer DDR standards are not primarily about chip capacity (currently
> >>>> the largest DDR5 chips are not larger than supported by DDR4: 16Gb),
> >>>> but about transfer rate. I.e., if the transfer rate becomes faster
> >>>> than 16x the internal cycle time of DRAM, they have to increase the
> >>>> minimum burst length (and decrease the (sub)channel width) to benefit
> >>>> from that increase. And currently I see no sign that the transfer
> >>>> rate increases are slowing down. DDR4 was released 7 years after
> >>>> DDR3; DDR5 was released 6 years after DDR4. On the PCIe front, the
> >>>> release cadence seems to be even higher.
> >>>> - anton
> >>>> --
> >>>> 'Anyone trying for "industrial quality" ISA should avoid undefined behavior.'
> >>>> Mitch Alsup, <c17fcd89-f024-40e7...@googlegroups.com>
> >>>
> >>> I can not formulate it with sufficient rigor, but my opinion is that need
> >>> in new DDR standards is very much a function of non-stability (hopefully growth)
> >>> of the market in term of average capacity.
> > <
> >> I disagree. If it were true, then newer generations of the standard
> >> would be much simpler; just add an address bit or two. The fact that
> >> each standard provides a substantial increase in data rate/bandwidth
> >> over the previous one, which is harder than adding an address pin, says
> >> that data rate/bandwidth is the driving factor, and capacity comes in
> >> second. Unfortunately, latency has proven resistant to substantial
> >> improvements :-(
> > <
> > DRAM chips have rather constant area. Passing over a linear amount of
> > real-estate is a wire delay problem. So that transistors are getting faster
> > and wire is getting slower has finally become manifest in DRAM. {actually
> > DRAM bank access to pin drivers has stayed rather constant since DDR2.}
> While all of that is true, I think it is only a part of the reason for
> the long latency. Consider a hypothetical chip of the same physical
> size as a current DRAM, but instead uses SRAM. Of course, it will have
> smaller capacity and perhaps require more pins than the DRAM, but I
> expect it will have lower latency. Is that correct?
>
> I think that the original DRAM designers made the (IMO correct) tradeoff
> to gain lower cost (fewer pins), at the cost of higher latency. This
> was reinforced by the introduction of CPU caches, which are essentially
> average latency reducers, hiding much of the longer latency. These
> allowed DRAMS to successfully dominate the market.
>
> However, cache is a diminishing returns game. Adding more improves
> performance, but not as much as the last time you increased it. So it
> may be true that reducing DRAM latency may become more important.
>
> So what could be done? A lower overhead (fewer transfers) protocol
> coupled with a different internal design and more pins (i.e. higher
> cost) might be achievable. It is way out of my area of expertise, but I
> just want to stimulate thoughts on this.
>
> Of course, the marketability of such a design, even if achievable, is a
> different question. :-(
> --
> - Stephen Fuld
> (e-mail address disguised to prevent spam)

What you described exists:
https://www.micron.com/products/dram/rldram-memory
It is very very very niche, but its existence proves that it is not absolutely non-marketable.