I just noticed a YouTube video where someone was talking about how we
were heading from the "megabyte era" to the "gigabyte era" in server
processors.
He cited the recent disclosure by AMD of new technology, similar to what
Intel was doing with two-layer chips, but with a higher bandwidth between
the layers.
But the prototype that AMD showed just had double the cache of a normal
chip, not many times more.
Even so, I don't think it's unreasonable to look in that direction.
Instead of doubling the amount of cache, after all, the second layer could
be DRAM, and then the higher bandwidth between memory and cache
would bring great benefits. And if it's DRAM, putting gigabytes in the chip
module wouldn't be beyond present technology.
The Vega 56 in my computer, after all, uses HBM for its video memory. That's a technology that puts gigabytes in the chip-sized module, so this
has all been around for a while.
Gigabytes of static RAM, on the other hand, will take a while.

John Savard

On 6/8/2021 3:22 PM, Quadibloc wrote:
> I just noticed a YouTube video where someone was talking about how we
> were heading from the "megabyte era" to the "gigabyte era" in server
> processors.
> He cited the recent disclosure by AMD of new technology, similar to what
> Intel was doing with two-layer chips, but with a higher bandwidth between
> the layers.
> But the prototype that AMD showed just had double the cache of a normal
> chip, not many times more.
> Even so, I don't think it's unreasonable to look in that direction.
> Instead of doubling the amount of cache, after all, the second layer could
> be DRAM, and then the higher bandwidth between memory and cache
> would bring great benefits. And if it's DRAM, putting gigabytes in the chip
> module wouldn't be beyond present technology.
> The Vega 56 in my computer, after all, uses HBM for its video memory. That's a technology that puts gigabytes in the chip-sized module, so this
> has all been around for a while.
> Gigabytes of static RAM, on the other hand, will take a while.

Fwiw, I always thought about putting gigs of DRAM memory in a cpu. So,
when you added more memory to your system, you automatically got
additional cpu's.

On Tuesday, June 8, 2021 at 5:47:06 PM UTC-5, Chris M. Thomasson wrote:
> On 6/8/2021 3:22 PM, Quadibloc wrote:
> > I just noticed a YouTube video where someone was talking about how we
> > were heading from the "megabyte era" to the "gigabyte era" in server
> > processors.
> > He cited the recent disclosure by AMD of new technology, similar to what
> > Intel was doing with two-layer chips, but with a higher bandwidth between
> > the layers.
> > But the prototype that AMD showed just had double the cache of a normal
> > chip, not many times more.
<
With SRAM taking up more than 1/2 of the chi now anyway, doubling the chip
count only doubles the caches.
<
> > Even so, I don't think it's unreasonable to look in that direction.
> > Instead of doubling the amount of cache, after all, the second layer could
> > be DRAM, and then the higher bandwidth between memory and cache
> > would bring great benefits. And if it's DRAM, putting gigabytes in the chip
> > module wouldn't be beyond present technology.
<
Point 1: yes this would add lots more GBs and you could access it faster
with hundreds of banks instead of 4-16.
<
Point 2: The DRAM industry would not want to make these for you--they like
the current model.
<
> > The Vega 56 in my computer, after all, uses HBM for its video memory. That's a technology that puts gigabytes in the chip-sized module, so this
> > has all been around for a while.
> > Gigabytes of static RAM, on the other hand, will take a while.
<
> Fwiw, I always thought about putting gigs of DRAM memory in a cpu. So,
> when you added more memory to your system, you automatically got
> additional cpu's.
<
CPU architects (like me) have advocated for DRAM caches since caches
got ECC. Test and marketing guys wave red flags at the people with money.

On Tuesday, June 8, 2021 at 6:20:57 PM UTC-6, MitchAlsup wrote:

> CPU architects (like me) have advocated for DRAM caches since caches
> got ECC. Test and marketing guys wave red flags at the people with money.

I remember that at one point IBM made a chip with an L3 cache made out of DRAM.
This, of course, made it slower, but it allowed it to be bigger - but IIRC, only 2x bigger.

I can't remember the name of the chip.

John Savard

MitchAlsup <MitchAlsup@aol.com> schrieb:

> CPU architects (like me) have advocated for DRAM caches since caches
> got ECC. Test and marketing guys wave red flags at the people with money.

IBM z uses DRAMs for their level 2, level 3 and level 4 caches.

Here's an interesting article about this:

https://fuse.wikichip.org/news/3383/ibm-doubles-its-14nm-edram-density-adds-hundreds-of-megabytes-of-cache/

On 6/8/2021 11:25 PM, Quadibloc wrote:
> On Tuesday, June 8, 2021 at 6:20:57 PM UTC-6, MitchAlsup wrote:
>
>> CPU architects (like me) have advocated for DRAM caches since caches
>> got ECC. Test and marketing guys wave red flags at the people with money.
>
> I remember that at one point IBM made a chip with an L3 cache made out of DRAM.
> This, of course, made it slower, but it allowed it to be bigger - but IIRC, only 2x bigger.
>
> I can't remember the name of the chip.

Power7. It used eDRAM. Other chips use or used eDRAM.

https://en.wikipedia.org/wiki/EDRAM

I am surprised it isn't more popular.

--
- Stephen Fuld
(e-mail address disguised to prevent spam)

On 6/8/2021 5:20 PM, MitchAlsup wrote:
> On Tuesday, June 8, 2021 at 5:47:06 PM UTC-5, Chris M. Thomasson wrote:
>> On 6/8/2021 3:22 PM, Quadibloc wrote:
>>> I just noticed a YouTube video where someone was talking about how we
>>> were heading from the "megabyte era" to the "gigabyte era" in server
>>> processors.
>>> He cited the recent disclosure by AMD of new technology, similar to what
>>> Intel was doing with two-layer chips, but with a higher bandwidth between
>>> the layers.
>>> But the prototype that AMD showed just had double the cache of a normal
>>> chip, not many times more.
> <
> With SRAM taking up more than 1/2 of the chi now anyway, doubling the chip
> count only doubles the caches.
> <
>>> Even so, I don't think it's unreasonable to look in that direction.
>>> Instead of doubling the amount of cache, after all, the second layer could
>>> be DRAM, and then the higher bandwidth between memory and cache
>>> would bring great benefits. And if it's DRAM, putting gigabytes in the chip
>>> module wouldn't be beyond present technology.
> <
> Point 1: yes this would add lots more GBs and you could access it faster
> with hundreds of banks instead of 4-16.
> <
> Point 2: The DRAM industry would not want to make these for you--they like
> the current model.
> <
>>> The Vega 56 in my computer, after all, uses HBM for its video memory. That's a technology that puts gigabytes in the chip-sized module, so this
>>> has all been around for a while.
>>> Gigabytes of static RAM, on the other hand, will take a while.
> <
>> Fwiw, I always thought about putting gigs of DRAM memory in a cpu. So,
>> when you added more memory to your system, you automatically got
>> additional cpu's.
> <
> CPU architects (like me) have advocated for DRAM caches since caches
> got ECC. Test and marketing guys wave red flags at the people with money.
>

Strange. It would be funny to transition into a place where adding more
ram would require one to buy the memory from the same company that
created the processor family in the system. So, say you have an Intel
system, you must buy memory from Intel that automatically comes with a
lot of extra processing power in the form of cpus that are integrated
with the DRAM, or at least really close to it... So, could it be a
"possible" threat to pure memory providers? Because now the memory has
to come with several multicore cpus?

I can think of a system without any extra memory, that has its onboard
memory and very close cpus on the motherboard. Then a user expands the
system by adding more memory, and extra cpus magically appear riding
along with the memory. It would be akin to a NUMA like thing.
Distributed programming model.

On Wednesday, June 9, 2021 at 3:47:02 PM UTC-6, Chris M. Thomasson wrote:

> Strange. It would be funny to transition into a place where adding more
> ram would require one to buy the memory from the same company that
> created the processor family in the system.

DRAM caches don't mean that.

If there is DRAM in the chip module, that had to come from the chipmaker,
then no more DRAM of that kind could be added later.

External DRAM could, as normal, come from any DRAM supplier. Whether
it's used as main memory... or the main memory is in the chip module, and
the external DRAM is used as a fast hard disk... is another matter.

John Savard

Chris M. Thomasson <chris.m.thomasson.1@gmail.com> schrieb:

> Strange. It would be funny to transition into a place where adding more
> ram would require one to buy the memory from the same company that
> created the processor family in the system. So, say you have an Intel
> system, you must buy memory from Intel that automatically comes with a
> lot of extra processing power in the form of cpus that are integrated
> with the DRAM, or at least really close to it... So, could it be a
> "possible" threat to pure memory providers? Because now the memory has
> to come with several multicore cpus?

Or another way - there are also people who are trying out
computational RAM.

It's certainly an idea. I could see a vector processor integrated
tightly with the RAM so it can bypass all of the caches etc and
only the end result of some calculation needs to reach the CPU.

However, it hasn't caught on, and there are several reasons why this
could be the case. Optimizing gates for arithmetic operations and
optimizing RAM are two very different things. Graphics cards are
now doing what computational RAM could be doing, and catching up
with that development could be extremely challenging, and also not
very promising; usually, graphics cards have their own, faster RAM.

There are probably other reasons, as well.

On Wednesday, June 9, 2021 at 3:20:57 AM UTC+3, MitchAlsup wrote:
> On Tuesday, June 8, 2021 at 5:47:06 PM UTC-5, Chris M. Thomasson wrote:
> > On 6/8/2021 3:22 PM, Quadibloc wrote:
> > > I just noticed a YouTube video where someone was talking about how we
> > > were heading from the "megabyte era" to the "gigabyte era" in server
> > > processors.
> > > He cited the recent disclosure by AMD of new technology, similar to what
> > > Intel was doing with two-layer chips, but with a higher bandwidth between
> > > the layers.
> > > But the prototype that AMD showed just had double the cache of a normal
> > > chip, not many times more.
> <
> With SRAM taking up more than 1/2 of the chi now anyway, doubling the chip
> count only doubles the caches.

I don't know how much of modern CPUs/APUs/SoCs is "SRAM total", but it does not appear relevant to current discussion.
What is relevant is how much of it is LLC.
And LLC on modern chips is significantly less than 1/2, either by transistors count (which, by itself, does not matter) or by area.
I think, it would be hard to find CPUs/APUs/SoCs developed after 2015, where LLC occupies 30% of area or more.
On the other hand, it's not hard at all to find chips with LLC occupying less than 20% of the area.
So, by going external, AMD style, with just one additional die layer, it's possible to do *much* better than mere doubling of LLC.
And that's before we even consider DRAM/eDRAM for companion chip.

> <
> > > Even so, I don't think it's unreasonable to look in that direction.
> > > Instead of doubling the amount of cache, after all, the second layer could
> > > be DRAM, and then the higher bandwidth between memory and cache
> > > would bring great benefits. And if it's DRAM, putting gigabytes in the chip
> > > module wouldn't be beyond present technology.
> <
> Point 1: yes this would add lots more GBs and you could access it faster
> with hundreds of banks instead of 4-16.
> <
> Point 2: The DRAM industry would not want to make these for you--they like
> the current model.
> <
> > > The Vega 56 in my computer, after all, uses HBM for its video memory. That's a technology that puts gigabytes in the chip-sized module, so this
> > > has all been around for a while.
> > > Gigabytes of static RAM, on the other hand, will take a while.
> <
> > Fwiw, I always thought about putting gigs of DRAM memory in a cpu. So,
> > when you added more memory to your system, you automatically got
> > additional cpu's.
> <
> CPU architects (like me) have advocated for DRAM caches since caches
> got ECC. Test and marketing guys wave red flags at the people with money.

On Wednesday, June 9, 2021 at 2:21:09 PM UTC+3, Stephen Fuld wrote:
> On 6/8/2021 11:25 PM, Quadibloc wrote:
> > On Tuesday, June 8, 2021 at 6:20:57 PM UTC-6, MitchAlsup wrote:
> >
> >> CPU architects (like me) have advocated for DRAM caches since caches
> >> got ECC. Test and marketing guys wave red flags at the people with money.
> >
> > I remember that at one point IBM made a chip with an L3 cache made out of DRAM.
> > This, of course, made it slower, but it allowed it to be bigger - but IIRC, only 2x bigger.
> >
> > I can't remember the name of the chip.
> Power7. It used eDRAM. Other chips use or used eDRAM.
>
> https://en.wikipedia.org/wiki/EDRAM
>
>
> I am surprised it isn't more popular.
>

There is a good reason for that.
Good DRAM takes good capacitors. IBM knew how to integrate good capacitors into logic process, other manufacturers didn't.
I wonder, what it means for future generations of IBM server chips now, when IBM became fabless.
That is, for POWER line the answer is obvious - no more eDRAM L3 cache after POWER9. But what about mainframe chips?

I'd think that there are only 2 feasible answer:
1. IBM continues to use their own variation of 14nm GlobalFoundries process for the next generations
2. eDRAM-based L3 cache on separate dies.
Or combination of the two.

Of course, the 3rd answer (make SRAM L3 cache, as the rest of the industry) is not totally impossible, but it would be serious contradiction to IBM mainframe design philosophy that always was "Don't trust SRAM for long-term storage".

>
>
> --
> - Stephen Fuld
> (e-mail address disguised to prevent spam)

"Chris M. Thomasson" <chris.m.thomasson.1@gmail.com> writes:
>Strange. It would be funny to transition into a place where adding more
>ram would require one to buy the memory from the same company that
>created the processor family in the system.

That's not new or anything. You used to have to buy RAM from the
computer manufacturer at boutique prices. With PC standardization, we
are in the lucky situation to be able to buy components at commodity
prices, but if you buy a complete PC from, say, HP, they use so many
non-standard components (in particular, no ATX-conforming power supply
and no ATX-conforming mainboard), so you cannot use commodity
components to repair it. Last I looked, they at least have standard
DIMM sockets, standard PCIe slots and standard mass storage
connectors, so you at least can expand these systems with free-market
components (if the power supply is strong enough). There are
manufacturers that have tried to hinder you from using third-party
mass storage for a long time; in particular, IIRC Apple did this
already a long time ago.

And these days, not just Apple, but many laptop manufacturers deliver
laptops with soldered-on RAM, soldered-on SSD, and glued-in battery,
not user-servicable. And if you compare the prices for, e.g., Apples
with more RAM or more SSD with RAM and SSD prices, you see that Apply
charges a steep premium.

> So, could it be a
>"possible" threat to pure memory providers?

You mean the eDRAM used as cache by CPUs? No.

As for the trend towards soldered-on stuff, the computer manufacturers
buy the RAM and SSDs from memory manufacturers (is Samsung a "pure
memory provider"?), so in a way it's not a threat to them. OTOH, if
Apple charges 4 times as much for RAM as memory manufacturers do,
Apple customers will buy configurations with less RAM than they would
if they could buy commodity RAM, which means that, even at the same
price, the memory manufacturer loses revenue.

- anton
--
'Anyone trying for "industrial quality" ISA should avoid undefined behavior.'
Mitch Alsup, <c17fcd89-f024-40e7-a594-88a85ac10d20o@googlegroups.com>

On 6/9/2021 11:26 PM, Thomas Koenig wrote:
> Chris M. Thomasson <chris.m.thomasson.1@gmail.com> schrieb:
>
>> Strange. It would be funny to transition into a place where adding more
>> ram would require one to buy the memory from the same company that
>> created the processor family in the system. So, say you have an Intel
>> system, you must buy memory from Intel that automatically comes with a
>> lot of extra processing power in the form of cpus that are integrated
>> with the DRAM, or at least really close to it... So, could it be a
>> "possible" threat to pure memory providers? Because now the memory has
>> to come with several multicore cpus?
>
> Or another way - there are also people who are trying out
> computational RAM.
>
> It's certainly an idea. I could see a vector processor integrated
> tightly with the RAM so it can bypass all of the caches etc and
> only the end result of some calculation needs to reach the CPU.
>
> However, it hasn't caught on, and there are several reasons why this
> could be the case. Optimizing gates for arithmetic operations and
> optimizing RAM are two very different things. Graphics cards are
> now doing what computational RAM could be doing, and catching up
> with that development could be extremely challenging, and also not
> very promising; usually, graphics cards have their own, faster RAM.
>
> There are probably other reasons, as well.

Yes, including communication. Suppose for example that you have a chip
with one core and 1 GB of RAM. What happens if the data you want to
process is larger than 1 GB? You need some mechanism for multiple of
these chips to communicate to each other and pass data, or at least some
kind of "message". These present both a programming issue and a
performance issue. This brings up memories of systems like N-Cube, the
Intel Paragon, etc. They didn't work out all that well, except for a
small subset of problems.

--
- Stephen Fuld
(e-mail address disguised to prevent spam)

On 6/10/2021 1:07 AM, Michael S wrote:
> On Wednesday, June 9, 2021 at 2:21:09 PM UTC+3, Stephen Fuld wrote:
>> On 6/8/2021 11:25 PM, Quadibloc wrote:
>>> On Tuesday, June 8, 2021 at 6:20:57 PM UTC-6, MitchAlsup wrote:
>>>
>>>> CPU architects (like me) have advocated for DRAM caches since caches
>>>> got ECC. Test and marketing guys wave red flags at the people with money.
>>>
>>> I remember that at one point IBM made a chip with an L3 cache made out of DRAM.
>>> This, of course, made it slower, but it allowed it to be bigger - but IIRC, only 2x bigger.
>>>
>>> I can't remember the name of the chip.
>> Power7. It used eDRAM. Other chips use or used eDRAM.
>>
>> https://en.wikipedia.org/wiki/EDRAM
>>
>>
>> I am surprised it isn't more popular.
>>
>
> There is a good reason for that.
> Good DRAM takes good capacitors. IBM knew how to integrate good capacitors into logic process, other manufacturers didn't.

The Wikipedia article says several Intel chips used it.

--
- Stephen Fuld
(e-mail address disguised to prevent spam)

On Thursday, June 10, 2021 at 5:55:47 PM UTC+3, Stephen Fuld wrote:
> On 6/10/2021 1:07 AM, Michael S wrote:
> > On Wednesday, June 9, 2021 at 2:21:09 PM UTC+3, Stephen Fuld wrote:
> >> On 6/8/2021 11:25 PM, Quadibloc wrote:
> >>> On Tuesday, June 8, 2021 at 6:20:57 PM UTC-6, MitchAlsup wrote:
> >>>
> >>>> CPU architects (like me) have advocated for DRAM caches since caches
> >>>> got ECC. Test and marketing guys wave red flags at the people with money.
> >>>
> >>> I remember that at one point IBM made a chip with an L3 cache made out of DRAM.
> >>> This, of course, made it slower, but it allowed it to be bigger - but IIRC, only 2x bigger.
> >>>
> >>> I can't remember the name of the chip.
> >> Power7. It used eDRAM. Other chips use or used eDRAM.
> >>
> >> https://en.wikipedia.org/wiki/EDRAM
> >>
> >>
> >> I am surprised it isn't more popular.
> >>
> >
> > There is a good reason for that.
> > Good DRAM takes good capacitors. IBM knew how to integrate good capacitors into logic process, other manufacturers didn't.
> The Wikipedia article says several Intel chips used it.

It's not the first time when Wikipedia is wrong.
In recent memories, Intel used eDRAM in product called Crystal Well (based on Haswell) and in several Broadwell SKUs.
In both cases eDRAM was on the separate die, so the problem of integration of good capacitors into high-performance logic die did not apply to them.

Intel did R&D related to high-perf capacitors integrated with logic, but in completely different context - filtering of high-frequency supply noise caused by quick changes in consumed power.
I would think that capacitors that are used here are quite different from those that hold data in DRAM/eDRAM.

> --
> - Stephen Fuld
> (e-mail address disguised to prevent spam)

On 6/10/2021 9:55 AM, Michael S wrote:
> On Thursday, June 10, 2021 at 5:55:47 PM UTC+3, Stephen Fuld wrote:
>> On 6/10/2021 1:07 AM, Michael S wrote:
>>> On Wednesday, June 9, 2021 at 2:21:09 PM UTC+3, Stephen Fuld wrote:
>>>> On 6/8/2021 11:25 PM, Quadibloc wrote:
>>>>> On Tuesday, June 8, 2021 at 6:20:57 PM UTC-6, MitchAlsup wrote:
>>>>>
>>>>>> CPU architects (like me) have advocated for DRAM caches since caches
>>>>>> got ECC. Test and marketing guys wave red flags at the people with money.
>>>>>
>>>>> I remember that at one point IBM made a chip with an L3 cache made out of DRAM.
>>>>> This, of course, made it slower, but it allowed it to be bigger - but IIRC, only 2x bigger.
>>>>>
>>>>> I can't remember the name of the chip.
>>>> Power7. It used eDRAM. Other chips use or used eDRAM.
>>>>
>>>> https://en.wikipedia.org/wiki/EDRAM
>>>>
>>>>
>>>> I am surprised it isn't more popular.
>>>>
>>>
>>> There is a good reason for that.
>>> Good DRAM takes good capacitors. IBM knew how to integrate good capacitors into logic process, other manufacturers didn't.
>> The Wikipedia article says several Intel chips used it.
>
> It's not the first time when Wikipedia is wrong.

True, although they are usually right. But it did seem a little odd, so
I did some more research. Note that the article says the e-DRAM was
used in the chips with IRIS graphics. So I looked at

https://en.wikipedia.org/wiki/Intel_Graphics_Technology#cite_note-anand-iris-6

and it said the graphics processor used e-DRAM, with a reference to an
Anandtech article that says the same thing

https://www.anandtech.com/show/6993/intel-iris-pro-5200-graphics-review-core-i74950hq-tested/2

I have absolutely no direct knowledge, one way or the other, but if
Wikipedia is wrong, they at least have a pretty good reference.

--
- Stephen Fuld
(e-mail address disguised to prevent spam)

On Thursday, June 10, 2021 at 8:28:10 PM UTC+3, Stephen Fuld wrote:
> On 6/10/2021 9:55 AM, Michael S wrote:
> > On Thursday, June 10, 2021 at 5:55:47 PM UTC+3, Stephen Fuld wrote:
> >> On 6/10/2021 1:07 AM, Michael S wrote:
> >>> On Wednesday, June 9, 2021 at 2:21:09 PM UTC+3, Stephen Fuld wrote:
> >>>> On 6/8/2021 11:25 PM, Quadibloc wrote:
> >>>>> On Tuesday, June 8, 2021 at 6:20:57 PM UTC-6, MitchAlsup wrote:
> >>>>>
> >>>>>> CPU architects (like me) have advocated for DRAM caches since caches
> >>>>>> got ECC. Test and marketing guys wave red flags at the people with money.
> >>>>>
> >>>>> I remember that at one point IBM made a chip with an L3 cache made out of DRAM.
> >>>>> This, of course, made it slower, but it allowed it to be bigger - but IIRC, only 2x bigger.
> >>>>>
> >>>>> I can't remember the name of the chip.
> >>>> Power7. It used eDRAM. Other chips use or used eDRAM.
> >>>>
> >>>> https://en.wikipedia.org/wiki/EDRAM
> >>>>
> >>>>
> >>>> I am surprised it isn't more popular.
> >>>>
> >>>
> >>> There is a good reason for that.
> >>> Good DRAM takes good capacitors. IBM knew how to integrate good capacitors into logic process, other manufacturers didn't.
> >> The Wikipedia article says several Intel chips used it.
> >
> > It's not the first time when Wikipedia is wrong.
> True, although they are usually right. But it did seem a little odd, so
> I did some more research. Note that the article says the e-DRAM was
> used in the chips with IRIS graphics. So I looked at
>
> https://en.wikipedia.org/wiki/Intel_Graphics_Technology#cite_note-anand-iris-6
>
> and it said the graphics processor used e-DRAM, with a reference to an
> Anandtech article that says the same thing
>
> https://www.anandtech.com/show/6993/intel-iris-pro-5200-graphics-review-core-i74950hq-tested/2
>
> I have absolutely no direct knowledge, one way or the other, but if
> Wikipedia is wrong, they at least have a pretty good reference.

Anand explains it well. Read the 3rd page.
"Despite its name, the eDRAM silicon is actually separate from the main microprocessor die - it’s simply housed on the same package."

> --
> - Stephen Fuld
> (e-mail address disguised to prevent spam)

On 6/10/2021 11:53 AM, Michael S wrote:
> On Thursday, June 10, 2021 at 8:28:10 PM UTC+3, Stephen Fuld wrote:
>> On 6/10/2021 9:55 AM, Michael S wrote:
>>> On Thursday, June 10, 2021 at 5:55:47 PM UTC+3, Stephen Fuld wrote:
>>>> On 6/10/2021 1:07 AM, Michael S wrote:
>>>>> On Wednesday, June 9, 2021 at 2:21:09 PM UTC+3, Stephen Fuld wrote:
>>>>>> On 6/8/2021 11:25 PM, Quadibloc wrote:
>>>>>>> On Tuesday, June 8, 2021 at 6:20:57 PM UTC-6, MitchAlsup wrote:
>>>>>>>
>>>>>>>> CPU architects (like me) have advocated for DRAM caches since caches
>>>>>>>> got ECC. Test and marketing guys wave red flags at the people with money.
>>>>>>>
>>>>>>> I remember that at one point IBM made a chip with an L3 cache made out of DRAM.
>>>>>>> This, of course, made it slower, but it allowed it to be bigger - but IIRC, only 2x bigger.
>>>>>>>
>>>>>>> I can't remember the name of the chip.
>>>>>> Power7. It used eDRAM. Other chips use or used eDRAM.
>>>>>>
>>>>>> https://en.wikipedia.org/wiki/EDRAM
>>>>>>
>>>>>>
>>>>>> I am surprised it isn't more popular.
>>>>>>
>>>>>
>>>>> There is a good reason for that.
>>>>> Good DRAM takes good capacitors. IBM knew how to integrate good capacitors into logic process, other manufacturers didn't.
>>>> The Wikipedia article says several Intel chips used it.
>>>
>>> It's not the first time when Wikipedia is wrong.
>> True, although they are usually right. But it did seem a little odd, so
>> I did some more research. Note that the article says the e-DRAM was
>> used in the chips with IRIS graphics. So I looked at
>>
>> https://en.wikipedia.org/wiki/Intel_Graphics_Technology#cite_note-anand-iris-6
>>
>> and it said the graphics processor used e-DRAM, with a reference to an
>> Anandtech article that says the same thing
>>
>> https://www.anandtech.com/show/6993/intel-iris-pro-5200-graphics-review-core-i74950hq-tested/2
>>
>> I have absolutely no direct knowledge, one way or the other, but if
>> Wikipedia is wrong, they at least have a pretty good reference.
>
> Anand explains it well. Read the 3rd page.
> "Despite its name, the eDRAM silicon is actually separate from the main microprocessor die - it’s simply housed on the same package."

Yes. I hadn't read past the first section. :-( But it is a, albeit
modified, Intel logic process, not a DRAM process. See the quote from
the article

> As we’ve been talking about for a while now, the highest end Haswell graphics configuration includes 128MB of eDRAM on-package. The eDRAM itself is a custom design by Intel and it’s built on a variant of Intel’s P1271 22nm SoC process (not P1270, the CPU process). Intel needed a set of low leakage 22nm transistors rather than the ability to drive very high frequencies which is why it’s using the mobile SoC 22nm process variant here.

The process change didn't seem to be about the capacitors, but I am way
beyond my depth here.

--
- Stephen Fuld
(e-mail address disguised to prevent spam)

On Wednesday, June 9, 2021 at 11:49:32 PM UTC-5, Quadibloc wrote:
> On Wednesday, June 9, 2021 at 3:47:02 PM UTC-6, Chris M. Thomasson wrote:
>
> > Strange. It would be funny to transition into a place where adding more
> > ram would require one to buy the memory from the same company that
> > created the processor family in the system.
<
Some of the unexposed architects around here are unaware of the process
differences between DRAM and CPU logic. I won't get into it now, But (BUT)
<
It suffices for you to understand that a DRAM chip DRAM is required to not
need refresh for a dozen milliseconds--this requires low leakage transistors
(almost always P-channel).
<
On the other hand DRAM caches can be refreshed at a dozen nanosecond rate
and this requires no process changes whatsoever. DRAM DRAMs need 1M×
lower leakage.
<
And when RAM starts with a D it requires ECC!
<
CPU DRAM might not be as small as DRAM DRAM but it will be in spitting
distance. It is not well understood but accessing CPU DRAM is only 20%
slower than accessing CPU SRAM. DRAM blocks of reasonable size can
be accessed in a single CPU cycle--that block cannot be accessed again
for 3-5 cycles but the address-to-RAM-out is only 20% slower than CPU
SRAM of the size (nanoacres not bits).
<
> DRAM caches don't mean that.
>
> If there is DRAM in the chip module, that had to come from the chipmaker,
> then no more DRAM of that kind could be added later.
>
> External DRAM could, as normal, come from any DRAM supplier. Whether
> it's used as main memory... or the main memory is in the chip module, and
> the external DRAM is used as a fast hard disk... is another matter.
>
> John Savard

On Thursday, June 10, 2021 at 1:26:57 AM UTC-5, Thomas Koenig wrote:
> Chris M. Thomasson <chris.m.t...@gmail.com> schrieb:
> > Strange. It would be funny to transition into a place where adding more
> > ram would require one to buy the memory from the same company that
> > created the processor family in the system. So, say you have an Intel
> > system, you must buy memory from Intel that automatically comes with a
> > lot of extra processing power in the form of cpus that are integrated
> > with the DRAM, or at least really close to it... So, could it be a
> > "possible" threat to pure memory providers? Because now the memory has
> > to come with several multicore cpus?
> Or another way - there are also people who are trying out
> computational RAM.
>
> It's certainly an idea. I could see a vector processor integrated
> tightly with the RAM so it can bypass all of the caches etc and
> only the end result of some calculation needs to reach the CPU.
>
> However, it hasn't caught on, and there are several reasons why this
> could be the case.
<
FFTs cannot be performed on such an architecture. FFTs have the
aspect that sooner (Decimation in time) or later (decimation in
frequency) you are touching memory that is very far apart. That is
it is not suitable for bringing the compute to memory architecture.
>
> Optimizing gates for arithmetic operations and
> optimizing RAM are two very different things. Graphics cards are
> now doing what computational RAM could be doing, and catching up
> with that development could be extremely challenging, and also not
> very promising; usually, graphics cards have their own, faster RAM.
>
> There are probably other reasons, as well.

On Thursday, June 10, 2021 at 1:56:27 PM UTC-6, MitchAlsup wrote:

> CPU DRAM might not be as small as DRAM DRAM but it will be in spitting
> distance. It is not well understood but accessing CPU DRAM is only 20%
> slower than accessing CPU SRAM.

Ah! I remember when reading whatever it was I originally saw, I had thought
that doubling the size of the cache was a poor benefit to obtain from
switching to DRAM, and getting a hugely slower cache. Glad to hear I was
mistaken.
In other news... AMD managed to get out of its contract to buy stuff from
GlobalFoundries when they wimped out on 10nm. IBM, on the other hand,
has felt it will have to go to the courts and sue them.. now that they've got
a chip shortage to make money from.

John Savard

On Thursday, June 10, 2021 at 1:59:19 PM UTC-6, MitchAlsup wrote:
> On Thursday, June 10, 2021 at 1:26:57 AM UTC-5, Thomas Koenig wrote:

> > It's certainly an idea. I could see a vector processor integrated
> > tightly with the RAM so it can bypass all of the caches etc and
> > only the end result of some calculation needs to reach the CPU.
> >
> > However, it hasn't caught on, and there are several reasons why this
> > could be the case.
> <
> FFTs cannot be performed on such an architecture. FFTs have the
> aspect that sooner (Decimation in time) or later (decimation in
> frequency) you are touching memory that is very far apart. That is
> it is not suitable for bringing the compute to memory architecture.

Yes.

I would have used *matrix multiplications* as my example, and once
again you have to touch memory far apart to do them.

That's not to say that processing in memory integrated with DRAM
isn't of _some_ use; *other* kinds of dumb calculations that don't
require accessing far-apart memory could at least be done massively.

However, my concept of doing processing in memory is based on not
just one, but *two*, historic failures in the field of computing.

The Illiac IV and the PDP-8/S.

The Illiac IV was SIMD instead of MIMD, and that was why it failed;
unlike modern "MIMD" computing, it was constructed in such a way
that the cost savings from making it SIMD instead of MIMD were
modest - even though the loss in usability and power was large.

The PDP-8/S had a serial ALU, and that made it slow.

The idea is:

You have to make the DRAM chips 64 bits wide to do calculations on
64 bit values. That's unfortunate. However, since today's DRAM only
works at maximum efficiency when you pump out 16 consecutive values
at a time... you could leave the pins at 4 bits wide, and just re-organize
how values are stored in memory.

The downside is now that you _always_ have to read the memory in the
maximum efficiency value, and can never read just one value if that's
all you need.

Each DRAM chip gets one, or a bunch of, seriously minimal CPUs.

These consist of:

A 64-bit shift register or three...

A serial ALU that is controlled from an external pin of the DRAM chip,

Storage for a handful of logic flags.

So this adds a _minimal_ amount of circuitry to the DRAM chip, on the
basis that usually most systems will seldom if ever use this capability,
so providing it can't threaten cost-competitiveness.

When it is used...

The central CPU steps a range of memory through a calculation. Say it wants
to replace the 64-bit floating-point numbers in an enormous swath of memory
with their sines.

It steps the serial ALUs through the calculation. One shift and add at a time.

Sometimes, there will be "test" instructions that set a logic flag, and sometimes
code sequences will be predicated on one of those logic flags. (The Illiac IV
did that kind of stuff.)

You can actually manage to do 64-bit floating arithmetic with a single-bit ALU.
It just takes time. But if you have thousands of them all working at once, it's
not too bad.

John Savard

On Thursday, June 10, 2021 at 10:10:50 PM UTC+3, Stephen Fuld wrote:
> On 6/10/2021 11:53 AM, Michael S wrote:
> > On Thursday, June 10, 2021 at 8:28:10 PM UTC+3, Stephen Fuld wrote:
> >> On 6/10/2021 9:55 AM, Michael S wrote:
> >>> On Thursday, June 10, 2021 at 5:55:47 PM UTC+3, Stephen Fuld wrote:
> >>>> On 6/10/2021 1:07 AM, Michael S wrote:
> >>>>> On Wednesday, June 9, 2021 at 2:21:09 PM UTC+3, Stephen Fuld wrote:
> >>>>>> On 6/8/2021 11:25 PM, Quadibloc wrote:
> >>>>>>> On Tuesday, June 8, 2021 at 6:20:57 PM UTC-6, MitchAlsup wrote:
> >>>>>>>
> >>>>>>>> CPU architects (like me) have advocated for DRAM caches since caches
> >>>>>>>> got ECC. Test and marketing guys wave red flags at the people with money.
> >>>>>>>
> >>>>>>> I remember that at one point IBM made a chip with an L3 cache made out of DRAM.
> >>>>>>> This, of course, made it slower, but it allowed it to be bigger - but IIRC, only 2x bigger.
> >>>>>>>
> >>>>>>> I can't remember the name of the chip.
> >>>>>> Power7. It used eDRAM. Other chips use or used eDRAM.
> >>>>>>
> >>>>>> https://en.wikipedia.org/wiki/EDRAM
> >>>>>>
> >>>>>>
> >>>>>> I am surprised it isn't more popular.
> >>>>>>
> >>>>>
> >>>>> There is a good reason for that.
> >>>>> Good DRAM takes good capacitors. IBM knew how to integrate good capacitors into logic process, other manufacturers didn't.
> >>>> The Wikipedia article says several Intel chips used it.
> >>>
> >>> It's not the first time when Wikipedia is wrong.
> >> True, although they are usually right. But it did seem a little odd, so
> >> I did some more research. Note that the article says the e-DRAM was
> >> used in the chips with IRIS graphics. So I looked at
> >>
> >> https://en.wikipedia.org/wiki/Intel_Graphics_Technology#cite_note-anand-iris-6
> >>
> >> and it said the graphics processor used e-DRAM, with a reference to an
> >> Anandtech article that says the same thing
> >>
> >> https://www.anandtech.com/show/6993/intel-iris-pro-5200-graphics-review-core-i74950hq-tested/2
> >>
> >> I have absolutely no direct knowledge, one way or the other, but if
> >> Wikipedia is wrong, they at least have a pretty good reference.
> >
> > Anand explains it well. Read the 3rd page.
> > "Despite its name, the eDRAM silicon is actually separate from the main microprocessor die - it’s simply housed on the same package."
> Yes. I hadn't read past the first section. :-( But it is a, albeit
> modified, Intel logic process, not a DRAM process. See the quote from
> the article
>
> > As we’ve been talking about for a while now, the highest end Haswell graphics configuration includes 128MB of eDRAM on-package. The eDRAM itself is a custom design by Intel and it’s built on a variant of Intel’s P1271 22nm SoC process (not P1270, the CPU process). Intel needed a set of low leakage 22nm transistors rather than the ability to drive very high frequencies which is why it’s using the mobile SoC 22nm process variant here.
>
> The process change didn't seem to be about the capacitors, but I am way
> beyond my depth here.

The process they used for Crystal Well eDRAM die is not fast enough for 3+GHz CPU.
It's probably different now, but back 5-6 years ago "mobile process" meant "optimized for reduced leakage and generally for low static power consumption at big price in increased dynamic power consumption".

To be clear, I don't say that other silicon manufacturers don't know, how to do eDRAM. They do know, foundries even have eDRAM in their libraries.
But all those non-IBM eDRAMs are not good enough to serve as L3 cache on the same die with 4-5 GHz CPU cores. Even less so, to serve as L2 cache, like in IBM z15.

> --
> - Stephen Fuld
> (e-mail address disguised to prevent spam)

Quadibloc wrote:
> On Thursday, June 10, 2021 at 1:59:19 PM UTC-6, MitchAlsup wrote:
>> On Thursday, June 10, 2021 at 1:26:57 AM UTC-5, Thomas Koenig wrote:
>
>>> It's certainly an idea. I could see a vector processor integrated
>>> tightly with the RAM so it can bypass all of the caches etc and
>>> only the end result of some calculation needs to reach the CPU.
>>>
>>> However, it hasn't caught on, and there are several reasons why this
>>> could be the case.
>> <
>> FFTs cannot be performed on such an architecture. FFTs have the
>> aspect that sooner (Decimation in time) or later (decimation in
>> frequency) you are touching memory that is very far apart. That is
>> it is not suitable for bringing the compute to memory architecture.
>
> Yes.
>
> I would have used *matrix multiplications* as my example, and once
> again you have to touch memory far apart to do them.
>
> That's not to say that processing in memory integrated with DRAM
> isn't of _some_ use; *other* kinds of dumb calculations that don't
> require accessing far-apart memory could at least be done massively.
>
> However, my concept of doing processing in memory is based on not
> just one, but *two*, historic failures in the field of computing.
>
> The Illiac IV and the PDP-8/S.
>
> The Illiac IV was SIMD instead of MIMD, and that was why it failed;
> unlike modern "MIMD" computing, it was constructed in such a way
> that the cost savings from making it SIMD instead of MIMD were
> modest - even though the loss in usability and power was large.
>
> The PDP-8/S had a serial ALU, and that made it slow.
>
> The idea is:
>
> You have to make the DRAM chips 64 bits wide to do calculations on
> 64 bit values. That's unfortunate. However, since today's DRAM only
> works at maximum efficiency when you pump out 16 consecutive values
> at a time... you could leave the pins at 4 bits wide, and just re-organize
> how values are stored in memory.
>
> The downside is now that you _always_ have to read the memory in the
> maximum efficiency value, and can never read just one value if that's
> all you need.
>
> Each DRAM chip gets one, or a bunch of, seriously minimal CPUs.
>
> These consist of:
>
> A 64-bit shift register or three...
>
> A serial ALU that is controlled from an external pin of the DRAM chip,
>
> Storage for a handful of logic flags.
>
> So this adds a _minimal_ amount of circuitry to the DRAM chip, on the
> basis that usually most systems will seldom if ever use this capability,
> so providing it can't threaten cost-competitiveness.
>
> When it is used...
>
> The central CPU steps a range of memory through a calculation. Say it wants
> to replace the 64-bit floating-point numbers in an enormous swath of memory
> with their sines.
>
> It steps the serial ALUs through the calculation. One shift and add at a time.
>
> Sometimes, there will be "test" instructions that set a logic flag, and sometimes
> code sequences will be predicated on one of those logic flags. (The Illiac IV
> did that kind of stuff.)
>
> You can actually manage to do 64-bit floating arithmetic with a single-bit ALU.
> It just takes time. But if you have thousands of them all working at once, it's
> not too bad.

What would the power efficiency be of such a system, i.e. compared to
streaming the same data past a GPU array?

Terje

--
- <Terje.Mathisen at tmsw.no>
"almost all programming can be viewed as an exercise in caching"

Quadibloc <jsavard@ecn.ab.ca> schrieb:
> On Thursday, June 10, 2021 at 1:59:19 PM UTC-6, MitchAlsup wrote:
>> On Thursday, June 10, 2021 at 1:26:57 AM UTC-5, Thomas Koenig wrote:
>
>> > It's certainly an idea. I could see a vector processor integrated
>> > tightly with the RAM so it can bypass all of the caches etc and
>> > only the end result of some calculation needs to reach the CPU.
>> >
>> > However, it hasn't caught on, and there are several reasons why this
>> > could be the case.
>> <
>> FFTs cannot be performed on such an architecture. FFTs have the
>> aspect that sooner (Decimation in time) or later (decimation in
>> frequency) you are touching memory that is very far apart. That is
>> it is not suitable for bringing the compute to memory architecture.
>
> Yes.
>
> I would have used *matrix multiplications* as my example, and once
> again you have to touch memory far apart to do them.

Depends how big your matrices are, how the data is actually arranged
in memory and where at the interface the auxiliary processor is.

AI stuff is a lot of matrix multiplications, but I don't really
know what sizes they are or how the data is usually arranged.

If there is anything worthwhile in there, I suspect people will
start looking at this again.