On 6/11/2021 4:05 AM, Michael S wrote:
> On Thursday, June 10, 2021 at 10:10:50 PM UTC+3, Stephen Fuld wrote:
>> On 6/10/2021 11:53 AM, Michael S wrote:
>>> On Thursday, June 10, 2021 at 8:28:10 PM UTC+3, Stephen Fuld wrote:
>>>> On 6/10/2021 9:55 AM, Michael S wrote:
>>>>> On Thursday, June 10, 2021 at 5:55:47 PM UTC+3, Stephen Fuld wrote:
>>>>>> On 6/10/2021 1:07 AM, Michael S wrote:
>>>>>>> On Wednesday, June 9, 2021 at 2:21:09 PM UTC+3, Stephen Fuld wrote:
>>>>>>>> On 6/8/2021 11:25 PM, Quadibloc wrote:
>>>>>>>>> On Tuesday, June 8, 2021 at 6:20:57 PM UTC-6, MitchAlsup wrote:
>>>>>>>>>
>>>>>>>>>> CPU architects (like me) have advocated for DRAM caches since caches
>>>>>>>>>> got ECC. Test and marketing guys wave red flags at the people with money.
>>>>>>>>>
>>>>>>>>> I remember that at one point IBM made a chip with an L3 cache made out of DRAM.
>>>>>>>>> This, of course, made it slower, but it allowed it to be bigger - but IIRC, only 2x bigger.
>>>>>>>>>
>>>>>>>>> I can't remember the name of the chip.
>>>>>>>> Power7. It used eDRAM. Other chips use or used eDRAM.
>>>>>>>>
>>>>>>>> https://en.wikipedia.org/wiki/EDRAM
>>>>>>>>
>>>>>>>>
>>>>>>>> I am surprised it isn't more popular.
>>>>>>>>
>>>>>>>
>>>>>>> There is a good reason for that.
>>>>>>> Good DRAM takes good capacitors. IBM knew how to integrate good capacitors into logic process, other manufacturers didn't.
>>>>>> The Wikipedia article says several Intel chips used it.
>>>>>
>>>>> It's not the first time when Wikipedia is wrong.
>>>> True, although they are usually right. But it did seem a little odd, so
>>>> I did some more research. Note that the article says the e-DRAM was
>>>> used in the chips with IRIS graphics. So I looked at
>>>>
>>>> https://en.wikipedia.org/wiki/Intel_Graphics_Technology#cite_note-anand-iris-6
>>>>
>>>> and it said the graphics processor used e-DRAM, with a reference to an
>>>> Anandtech article that says the same thing
>>>>
>>>> https://www.anandtech.com/show/6993/intel-iris-pro-5200-graphics-review-core-i74950hq-tested/2
>>>>
>>>> I have absolutely no direct knowledge, one way or the other, but if
>>>> Wikipedia is wrong, they at least have a pretty good reference.
>>>
>>> Anand explains it well. Read the 3rd page.
>>> "Despite its name, the eDRAM silicon is actually separate from the main microprocessor die - it’s simply housed on the same package."
>> Yes. I hadn't read past the first section. :-( But it is a, albeit
>> modified, Intel logic process, not a DRAM process. See the quote from
>> the article
>>
>>> As we’ve been talking about for a while now, the highest end Haswell graphics configuration includes 128MB of eDRAM on-package. The eDRAM itself is a custom design by Intel and it’s built on a variant of Intel’s P1271 22nm SoC process (not P1270, the CPU process). Intel needed a set of low leakage 22nm transistors rather than the ability to drive very high frequencies which is why it’s using the mobile SoC 22nm process variant here.
>>
>> The process change didn't seem to be about the capacitors, but I am way
>> beyond my depth here.
>
> The process they used for Crystal Well eDRAM die is not fast enough for 3+GHz CPU.

I certainly agree with that.

> It's probably different now, but back 5-6 years ago "mobile process" meant "optimized for reduced leakage and generally for low static power consumption at big price in increased dynamic power consumption".

Seems very reasonable.

>
> To be clear, I don't say that other silicon manufacturers don't know, how to do eDRAM. They do know, foundries even have eDRAM in their libraries.
> But all those non-IBM eDRAMs are not good enough to serve as L3 cache on the same die with 4-5 GHz CPU cores. Even less so, to serve as L2 cache, like in IBM z15.

OK, I see what you are saying. To paraphrase, anyone can do eDRAM, but
IBM does it enough better than anyone else to make it fast enough to be
used as an L3, much less an L2 cache. Is that a reasonable restatement?

Regarding L3 caches, certainly "fast enough" is a little too concise.
Obviously eDRAM will be slower than SRAM, but the idea is that the
larger sized caches allowed by the smaller cells increases the hit rate
enough to compensate for the slower access time. I take your comments
to mean that, except for IBM, that tradeoff doesn't work out. Is that fair?

Since you are far more knowledgeable than I am in this area, let me ask
a question. One of the advantages of eDRAM over "commodity DRAM" is
that one could, at least in theory, configure the bank sizes to meet
your needs. Given that the communication is all on chip, so number of
wires isn't a big issue, couldn't you configure the row size to be equal
to the cache line size such that once you get the row into the buffers,
you can transfer the whole row in one clock, essentially eliminating the
whole CAS part of the access time and transferring a whole cache line at
once instead of the eight clocks typically used for external DRAM? Does
this work?

--
- Stephen Fuld
(e-mail address disguised to prevent spam)

On 6/11/2021 8:31 AM, Stephen Fuld wrote:
> On 6/11/2021 4:05 AM, Michael S wrote:
>> On Thursday, June 10, 2021 at 10:10:50 PM UTC+3, Stephen Fuld wrote:
>>> On 6/10/2021 11:53 AM, Michael S wrote:
>>>> On Thursday, June 10, 2021 at 8:28:10 PM UTC+3, Stephen Fuld wrote:
>>>>> On 6/10/2021 9:55 AM, Michael S wrote:
>>>>>> On Thursday, June 10, 2021 at 5:55:47 PM UTC+3, Stephen Fuld wrote:
>>>>>>> On 6/10/2021 1:07 AM, Michael S wrote:
>>>>>>>> On Wednesday, June 9, 2021 at 2:21:09 PM UTC+3, Stephen Fuld wrote:
>>>>>>>>> On 6/8/2021 11:25 PM, Quadibloc wrote:
>>>>>>>>>> On Tuesday, June 8, 2021 at 6:20:57 PM UTC-6, MitchAlsup wrote:
>>>>>>>>>>
>>>>>>>>>>> CPU architects (like me) have advocated for DRAM caches since
>>>>>>>>>>> caches
>>>>>>>>>>> got ECC. Test and marketing guys wave red flags at the people
>>>>>>>>>>> with money.
>>>>>>>>>>
>>>>>>>>>> I remember that at one point IBM made a chip with an L3 cache
>>>>>>>>>> made out of DRAM.
>>>>>>>>>> This, of course, made it slower, but it allowed it to be
>>>>>>>>>> bigger - but IIRC, only 2x bigger.
>>>>>>>>>>
>>>>>>>>>> I can't remember the name of the chip.
>>>>>>>>> Power7. It used eDRAM. Other chips use or used eDRAM.
>>>>>>>>>
>>>>>>>>> https://en.wikipedia.org/wiki/EDRAM
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> I am surprised it isn't more popular.
>>>>>>>>>
>>>>>>>>
>>>>>>>> There is a good reason for that.
>>>>>>>> Good DRAM takes good capacitors. IBM knew how to integrate good
>>>>>>>> capacitors into logic process, other manufacturers didn't.
>>>>>>> The Wikipedia article says several Intel chips used it.
>>>>>>
>>>>>> It's not the first time when Wikipedia is wrong.
>>>>> True, although they are usually right. But it did seem a little
>>>>> odd, so
>>>>> I did some more research. Note that the article says the e-DRAM was
>>>>> used in the chips with IRIS graphics. So I looked at
>>>>>
>>>>> https://en.wikipedia.org/wiki/Intel_Graphics_Technology#cite_note-anand-iris-6
>>>>>
>>>>>
>>>>> and it said the graphics processor used e-DRAM, with a reference to an
>>>>> Anandtech article that says the same thing
>>>>>
>>>>> https://www.anandtech.com/show/6993/intel-iris-pro-5200-graphics-review-core-i74950hq-tested/2
>>>>>
>>>>>
>>>>> I have absolutely no direct knowledge, one way or the other, but if
>>>>> Wikipedia is wrong, they at least have a pretty good reference.
>>>>
>>>> Anand explains it well. Read the 3rd page.
>>>> "Despite its name, the eDRAM silicon is actually separate from the
>>>> main microprocessor die - it’s simply housed on the same package."
>>> Yes. I hadn't read past the first section. :-( But it is a, albeit
>>> modified, Intel logic process, not a DRAM process. See the quote from
>>> the article
>>>
>>>> As we’ve been talking about for a while now, the highest end Haswell
>>>> graphics configuration includes 128MB of eDRAM on-package. The eDRAM
>>>> itself is a custom design by Intel and it’s built on a variant of
>>>> Intel’s P1271 22nm SoC process (not P1270, the CPU process). Intel
>>>> needed a set of low leakage 22nm transistors rather than the ability
>>>> to drive very high frequencies which is why it’s using the mobile
>>>> SoC 22nm process variant here.
>>>
>>> The process change didn't seem to be about the capacitors, but I am way
>>> beyond my depth here.
>>
>> The process they used for Crystal Well eDRAM die is not fast enough
>> for 3+GHz CPU.
>
> I certainly agree with that.
>
>
>
>> It's probably different now, but back 5-6 years ago "mobile process"
>> meant "optimized for reduced leakage and generally for low static
>> power consumption at big price in increased dynamic power consumption".
>
> Seems very reasonable.
>
>
>
>>
>> To be clear, I don't say that other silicon manufacturers don't know,
>> how to do eDRAM. They do know, foundries even have eDRAM in their
>> libraries.
>> But all those non-IBM eDRAMs are not good enough to serve as L3 cache
>> on the same die with 4-5 GHz CPU cores. Even less so, to serve as L2
>> cache, like in IBM z15.
>
> OK, I see what you are saying.  To paraphrase, anyone can do eDRAM, but
> IBM does it enough better than anyone else to make it fast enough to be
> used as an L3, much less an L2 cache.

Sorry, I got the logic messed up. Should be "even an L2 cache."

  Is that a reasonable restatement?
>
> Regarding L3 caches, certainly "fast enough" is a little too concise.
> Obviously eDRAM will be slower than SRAM, but the idea is that the
> larger sized caches allowed by the smaller cells increases the hit rate
> enough to compensate for the slower access time.  I take your comments
> to mean that, except for IBM, that tradeoff doesn't work out.  Is that
> fair?
>
> Since you are far more knowledgeable than I am in this area, let me ask
> a question.  One of the advantages of eDRAM over "commodity DRAM" is
> that one could, at least in theory, configure the bank sizes to meet
> your needs.  Given that the communication is all on chip, so number of
> wires isn't a big issue, couldn't you configure the row size to be equal
> to the cache line size such that once you get the row into the  buffers,
> you can transfer the whole row in one clock, essentially eliminating the
> whole CAS part of the access time and transferring a whole cache line at
> once instead of the eight clocks typically used for external DRAM?  Does
> this work?
>
>

--
- Stephen Fuld
(e-mail address disguised to prevent spam)

On Friday, June 11, 2021 at 10:31:08 AM UTC-5, Stephen Fuld wrote:
> On 6/11/2021 4:05 AM, Michael S wrote:
> > On Thursday, June 10, 2021 at 10:10:50 PM UTC+3, Stephen Fuld wrote:
> >> On 6/10/2021 11:53 AM, Michael S wrote:

> >
> > To be clear, I don't say that other silicon manufacturers don't know, how to do eDRAM. They do know, foundries even have eDRAM in their libraries.
> > But all those non-IBM eDRAMs are not good enough to serve as L3 cache on the same die with 4-5 GHz CPU cores. Even less so, to serve as L2 cache, like in IBM z15.
<
> OK, I see what you are saying. To paraphrase, anyone can do eDRAM, but
> IBM does it enough better than anyone else to make it fast enough to be
> used as an L3, much less an L2 cache. Is that a reasonable restatement?
>
> Regarding L3 caches, certainly "fast enough" is a little too concise.
> Obviously eDRAM will be slower than SRAM, but the idea is that the
> larger sized caches allowed by the smaller cells increases the hit rate
> enough to compensate for the slower access time. I take your comments
> to mean that, except for IBM, that tradeoff doesn't work out. Is that fair?
<
The problem is that L2 and L3 architectures are designed for SRAM where
there is no penalty for hitting the same macro every clock. DRAM has an
acceptably fast access time address to data at sense amp, but after this
the stored data must be refreshed and the bank put back into a controlled
precharge condition. This takes 3-5 more cycles.
<
So, in order to utilize an L2-L3 made of DRAM, you need to have multiple
outstanding requests to different partitions simultaneously, rather than
strip mining a line from the L2-L3 and then another.
>
> Since you are far more knowledgeable than I am in this area, let me ask
> a question. One of the advantages of eDRAM over "commodity DRAM" is
> that one could, at least in theory, configure the bank sizes to meet
> your needs. Given that the communication is all on chip, so number of
> wires isn't a big issue, couldn't you configure the row size to be equal
> to the cache line size such that once you get the row into the buffers,
> you can transfer the whole row in one clock, essentially eliminating the
> whole CAS part of the access time and transferring a whole cache line at
> once instead of the eight clocks typically used for external DRAM? Does
> this work?
<
Commodity DRAMs use 1024-4096 bits per bank read, so if you are willing
to make you line size 128B-512B then yes you could do this. But, these DRAMs
allow you to access "all the bits" by sending multiple CAS requests and
strip mining the data through the pins.
<
See my patent 5,367,494 for a better organization--more appropriate
organization for a block of DRAM on a die.
> --
> - Stephen Fuld
> (e-mail address disguised to prevent spam)

On 6/11/2021 6:03 AM, Terje Mathisen wrote:
> Quadibloc wrote:
>> On Thursday, June 10, 2021 at 1:59:19 PM UTC-6, MitchAlsup wrote:
>>> On Thursday, June 10, 2021 at 1:26:57 AM UTC-5, Thomas Koenig wrote:
>>
>>>> It's certainly an idea. I could see a vector processor integrated
>>>> tightly with the RAM so it can bypass all of the caches etc and
>>>> only the end result of some calculation needs to reach the CPU.
>>>>
>>>> However, it hasn't caught on, and there are several reasons why this
>>>> could be the case.
>>> <
>>> FFTs cannot be performed on such an architecture. FFTs have the
>>> aspect that sooner (Decimation in time) or later (decimation in
>>> frequency) you are touching memory that is very far apart. That is
>>> it is not suitable for bringing the compute to memory architecture.
>>
>> Yes.
>>
>> I would have used *matrix multiplications* as my example, and once
>> again you have to touch memory far apart to do them.
>>
>> That's not to say that processing in memory integrated with DRAM
>> isn't of _some_ use; *other* kinds of dumb calculations that don't
>> require accessing far-apart memory could at least be done massively.
>>
>> However, my concept of doing processing in memory is based on not
>> just one, but *two*, historic failures in the field of computing.
>>
>> The Illiac IV and the PDP-8/S.
>>
>> The Illiac IV was SIMD instead of MIMD, and that was why it failed;
>> unlike modern "MIMD" computing, it was constructed in such a way
>> that the cost savings from making it SIMD instead of MIMD were
>> modest - even though the loss in usability and power was large.
>>
>> The PDP-8/S had a serial ALU, and that made it slow.
>>
>> The idea is:
>>
>> You have to make the DRAM chips 64 bits wide to do calculations on
>> 64 bit values. That's unfortunate. However, since today's DRAM only
>> works at maximum efficiency when you pump out 16 consecutive values
>> at a time... you could leave the pins at 4 bits wide, and just
>> re-organize
>> how values are stored in memory.
>>
>> The downside is now that you _always_ have to read the memory in the
>> maximum efficiency value, and can never read just one value if that's
>> all you need.
>>
>> Each DRAM chip gets one, or a bunch of, seriously minimal CPUs.
>>
>> These consist of:
>>
>> A 64-bit shift register or three...
>>
>> A serial ALU that is controlled from an external pin of the DRAM chip,
>>
>> Storage for a handful of logic flags.
>>
>> So this adds a _minimal_ amount of circuitry to the DRAM chip, on the
>> basis that usually most systems will seldom if ever use this capability,
>> so providing it can't threaten cost-competitiveness.
>>
>> When it is used...
>>
>> The central CPU steps a range of memory through a calculation. Say it
>> wants
>> to replace the 64-bit floating-point numbers in an enormous swath of
>> memory
>> with their sines.
>>
>> It steps the serial ALUs through the calculation. One shift and add at
>> a time.
>>
>> Sometimes, there will be "test" instructions that set a logic flag,
>> and sometimes
>> code sequences will be predicated on one of those logic flags. (The
>> Illiac IV
>> did that kind of stuff.)
>>
>> You can actually manage to do 64-bit floating arithmetic with a
>> single-bit ALU.
>> It just takes time. But if you have thousands of them all working at
>> once, it's
>> not too bad.
>
> What would the power efficiency be of such a system, i.e. compared to
> streaming the same data past a GPU array?

If it can get 60fps rendering a hardcore scene, it should be okay.
Shadertoy is a nice friend to have:

https://www.shadertoy.com/view/Ms2SD1

https://www.shadertoy.com/view/Nt2GRW

;^)

On Friday, June 11, 2021 at 2:43:23 PM UTC-5, Chris M. Thomasson wrote:
> On 6/11/2021 6:03 AM, Terje Mathisen wrote:
> > Quadibloc wrote:
> >> On Thursday, June 10, 2021 at 1:59:19 PM UTC-6, MitchAlsup wrote:
> >>> On Thursday, June 10, 2021 at 1:26:57 AM UTC-5, Thomas Koenig wrote:
> >>
> >>>> It's certainly an idea. I could see a vector processor integrated
> >>>> tightly with the RAM so it can bypass all of the caches etc and
> >>>> only the end result of some calculation needs to reach the CPU.
> >>>>
> >>>> However, it hasn't caught on, and there are several reasons why this
> >>>> could be the case.
> >>> <
> >>> FFTs cannot be performed on such an architecture. FFTs have the
> >>> aspect that sooner (Decimation in time) or later (decimation in
> >>> frequency) you are touching memory that is very far apart. That is
> >>> it is not suitable for bringing the compute to memory architecture.
> >>
> >> Yes.
> >>
> >> I would have used *matrix multiplications* as my example, and once
> >> again you have to touch memory far apart to do them.
> >>
> >> That's not to say that processing in memory integrated with DRAM
> >> isn't of _some_ use; *other* kinds of dumb calculations that don't
> >> require accessing far-apart memory could at least be done massively.
> >>
> >> However, my concept of doing processing in memory is based on not
> >> just one, but *two*, historic failures in the field of computing.
> >>
> >> The Illiac IV and the PDP-8/S.
> >>
> >> The Illiac IV was SIMD instead of MIMD, and that was why it failed;
> >> unlike modern "MIMD" computing, it was constructed in such a way
> >> that the cost savings from making it SIMD instead of MIMD were
> >> modest - even though the loss in usability and power was large.
> >>
> >> The PDP-8/S had a serial ALU, and that made it slow.
> >>
> >> The idea is:
> >>
> >> You have to make the DRAM chips 64 bits wide to do calculations on
> >> 64 bit values. That's unfortunate. However, since today's DRAM only
> >> works at maximum efficiency when you pump out 16 consecutive values
> >> at a time... you could leave the pins at 4 bits wide, and just
> >> re-organize
> >> how values are stored in memory.
> >>
> >> The downside is now that you _always_ have to read the memory in the
> >> maximum efficiency value, and can never read just one value if that's
> >> all you need.
> >>
> >> Each DRAM chip gets one, or a bunch of, seriously minimal CPUs.
> >>
> >> These consist of:
> >>
> >> A 64-bit shift register or three...
> >>
> >> A serial ALU that is controlled from an external pin of the DRAM chip,
> >>
> >> Storage for a handful of logic flags.
> >>
> >> So this adds a _minimal_ amount of circuitry to the DRAM chip, on the
> >> basis that usually most systems will seldom if ever use this capability,
> >> so providing it can't threaten cost-competitiveness.
> >>
> >> When it is used...
> >>
> >> The central CPU steps a range of memory through a calculation. Say it
> >> wants
> >> to replace the 64-bit floating-point numbers in an enormous swath of
> >> memory
> >> with their sines.
> >>
> >> It steps the serial ALUs through the calculation. One shift and add at
> >> a time.
> >>
> >> Sometimes, there will be "test" instructions that set a logic flag,
> >> and sometimes
> >> code sequences will be predicated on one of those logic flags. (The
> >> Illiac IV
> >> did that kind of stuff.)
> >>
> >> You can actually manage to do 64-bit floating arithmetic with a
> >> single-bit ALU.
> >> It just takes time. But if you have thousands of them all working at
> >> once, it's
> >> not too bad.
> >
> > What would the power efficiency be of such a system, i.e. compared to
> > streaming the same data past a GPU array?
<
> If it can get 60fps rendering a hardcore scene, it should be okay.
<
<
Well GPU tends to have embarisingly large amounts of parallelism, so maybe
1,000,000,000 1-bit CPUs at 1 GHz would be sufficient.
<
> Shadertoy is a nice friend to have:
>
> https://www.shadertoy.com/view/Ms2SD1
>
> https://www.shadertoy.com/view/Nt2GRW
>
> ;^)

On Friday, June 11, 2021 at 6:31:08 PM UTC+3, Stephen Fuld wrote:
> On 6/11/2021 4:05 AM, Michael S wrote:
> > On Thursday, June 10, 2021 at 10:10:50 PM UTC+3, Stephen Fuld wrote:
> >> On 6/10/2021 11:53 AM, Michael S wrote:
> >>> On Thursday, June 10, 2021 at 8:28:10 PM UTC+3, Stephen Fuld wrote:
> >>>> On 6/10/2021 9:55 AM, Michael S wrote:
> >>>>> On Thursday, June 10, 2021 at 5:55:47 PM UTC+3, Stephen Fuld wrote:
> >>>>>> On 6/10/2021 1:07 AM, Michael S wrote:
> >>>>>>> On Wednesday, June 9, 2021 at 2:21:09 PM UTC+3, Stephen Fuld wrote:
> >>>>>>>> On 6/8/2021 11:25 PM, Quadibloc wrote:
> >>>>>>>>> On Tuesday, June 8, 2021 at 6:20:57 PM UTC-6, MitchAlsup wrote:
> >>>>>>>>>
> >>>>>>>>>> CPU architects (like me) have advocated for DRAM caches since caches
> >>>>>>>>>> got ECC. Test and marketing guys wave red flags at the people with money.
> >>>>>>>>>
> >>>>>>>>> I remember that at one point IBM made a chip with an L3 cache made out of DRAM.
> >>>>>>>>> This, of course, made it slower, but it allowed it to be bigger - but IIRC, only 2x bigger.
> >>>>>>>>>
> >>>>>>>>> I can't remember the name of the chip.
> >>>>>>>> Power7. It used eDRAM. Other chips use or used eDRAM.
> >>>>>>>>
> >>>>>>>> https://en.wikipedia.org/wiki/EDRAM
> >>>>>>>>
> >>>>>>>>
> >>>>>>>> I am surprised it isn't more popular.
> >>>>>>>>
> >>>>>>>
> >>>>>>> There is a good reason for that.
> >>>>>>> Good DRAM takes good capacitors. IBM knew how to integrate good capacitors into logic process, other manufacturers didn't.
> >>>>>> The Wikipedia article says several Intel chips used it.
> >>>>>
> >>>>> It's not the first time when Wikipedia is wrong.
> >>>> True, although they are usually right. But it did seem a little odd, so
> >>>> I did some more research. Note that the article says the e-DRAM was
> >>>> used in the chips with IRIS graphics. So I looked at
> >>>>
> >>>> https://en.wikipedia.org/wiki/Intel_Graphics_Technology#cite_note-anand-iris-6
> >>>>
> >>>> and it said the graphics processor used e-DRAM, with a reference to an
> >>>> Anandtech article that says the same thing
> >>>>
> >>>> https://www.anandtech.com/show/6993/intel-iris-pro-5200-graphics-review-core-i74950hq-tested/2
> >>>>
> >>>> I have absolutely no direct knowledge, one way or the other, but if
> >>>> Wikipedia is wrong, they at least have a pretty good reference.
> >>>
> >>> Anand explains it well. Read the 3rd page.
> >>> "Despite its name, the eDRAM silicon is actually separate from the main microprocessor die - it’s simply housed on the same package."
> >> Yes. I hadn't read past the first section. :-( But it is a, albeit
> >> modified, Intel logic process, not a DRAM process. See the quote from
> >> the article
> >>
> >>> As we’ve been talking about for a while now, the highest end Haswell graphics configuration includes 128MB of eDRAM on-package. The eDRAM itself is a custom design by Intel and it’s built on a variant of Intel’s P1271 22nm SoC process (not P1270, the CPU process). Intel needed a set of low leakage 22nm transistors rather than the ability to drive very high frequencies which is why it’s using the mobile SoC 22nm process variant here.
> >>
> >> The process change didn't seem to be about the capacitors, but I am way
> >> beyond my depth here.
> >
> > The process they used for Crystal Well eDRAM die is not fast enough for 3+GHz CPU.
> I certainly agree with that.
> > It's probably different now, but back 5-6 years ago "mobile process" meant "optimized for reduced leakage and generally for low static power consumption at big price in increased dynamic power consumption".
> Seems very reasonable.
> >
> > To be clear, I don't say that other silicon manufacturers don't know, how to do eDRAM. They do know, foundries even have eDRAM in their libraries.
> > But all those non-IBM eDRAMs are not good enough to serve as L3 cache on the same die with 4-5 GHz CPU cores. Even less so, to serve as L2 cache, like in IBM z15.
> OK, I see what you are saying. To paraphrase, anyone can do eDRAM, but
> IBM does it enough better than anyone else to make it fast enough to be
> used as an L3, much less an L2 cache. Is that a reasonable restatement?
>
> Regarding L3 caches, certainly "fast enough" is a little too concise.
> Obviously eDRAM will be slower than SRAM, but the idea is that the
> larger sized caches allowed by the smaller cells increases the hit rate
> enough to compensate for the slower access time. I take your comments
> to mean that, except for IBM, that tradeoff doesn't work out. Is that fair?
>
> Since you are far more knowledgeable than I am in this area, let me ask
> a question.

My knowledge is at the "blackbox" level, interfaces and parameters visible to outside world, rather than internal operations.

> One of the advantages of eDRAM over "commodity DRAM" is
> that one could, at least in theory, configure the bank sizes to meet
> your needs. Given that the communication is all on chip, so number of
> wires isn't a big issue, couldn't you configure the row size to be equal
> to the cache line size such that once you get the row into the buffers,
> you can transfer the whole row in one clock, essentially eliminating the
> whole CAS part of the access time and transferring a whole cache line at
> once instead of the eight clocks typically used for external DRAM? Does
> this work?

I think that more that, say, 8K rows per bank wouldn't work.
But a lot of small banks, say 8K rows * 128B * 256 banks for total size of 256 MB, does not seem unreasonable if one is willing to sacrifice a bit of density (more banks==more sense amplifiers).
I wouldn't be surprised if that's how things were done in POWER9 except that the total was only 120 MB

> --
> - Stephen Fuld
> (e-mail address disguised to prevent spam)

On Saturday, June 12, 2021 at 2:12:40 PM UTC-5, Michael S wrote:
> On Friday, June 11, 2021 at 6:31:08 PM UTC+3, Stephen Fuld wrote:
> > On 6/11/2021 4:05 AM, Michael S wrote:
> > > On Thursday, June 10, 2021 at 10:10:50 PM UTC+3, Stephen Fuld wrote:
> > >> On 6/10/2021 11:53 AM, Michael S wrote:
> > >>> On Thursday, June 10, 2021 at 8:28:10 PM UTC+3, Stephen Fuld wrote:
> > >>>> On 6/10/2021 9:55 AM, Michael S wrote:
> > >>>>> On Thursday, June 10, 2021 at 5:55:47 PM UTC+3, Stephen Fuld wrote:
> > >>>>>> On 6/10/2021 1:07 AM, Michael S wrote:
> > >>>>>>> On Wednesday, June 9, 2021 at 2:21:09 PM UTC+3, Stephen Fuld wrote:
> > >>>>>>>> On 6/8/2021 11:25 PM, Quadibloc wrote:
> > >>>>>>>>> On Tuesday, June 8, 2021 at 6:20:57 PM UTC-6, MitchAlsup wrote:
> > >>>>>>>>>
> > >>>>>>>>>> CPU architects (like me) have advocated for DRAM caches since caches
> > >>>>>>>>>> got ECC. Test and marketing guys wave red flags at the people with money.
> > >>>>>>>>>
> > >>>>>>>>> I remember that at one point IBM made a chip with an L3 cache made out of DRAM.
> > >>>>>>>>> This, of course, made it slower, but it allowed it to be bigger - but IIRC, only 2x bigger.
> > >>>>>>>>>
> > >>>>>>>>> I can't remember the name of the chip.
> > >>>>>>>> Power7. It used eDRAM. Other chips use or used eDRAM.
> > >>>>>>>>
> > >>>>>>>> https://en.wikipedia.org/wiki/EDRAM
> > >>>>>>>>
> > >>>>>>>>
> > >>>>>>>> I am surprised it isn't more popular.
> > >>>>>>>>
> > >>>>>>>
> > >>>>>>> There is a good reason for that.
> > >>>>>>> Good DRAM takes good capacitors. IBM knew how to integrate good capacitors into logic process, other manufacturers didn't.
> > >>>>>> The Wikipedia article says several Intel chips used it.
> > >>>>>
> > >>>>> It's not the first time when Wikipedia is wrong.
> > >>>> True, although they are usually right. But it did seem a little odd, so
> > >>>> I did some more research. Note that the article says the e-DRAM was
> > >>>> used in the chips with IRIS graphics. So I looked at
> > >>>>
> > >>>> https://en.wikipedia.org/wiki/Intel_Graphics_Technology#cite_note-anand-iris-6
> > >>>>
> > >>>> and it said the graphics processor used e-DRAM, with a reference to an
> > >>>> Anandtech article that says the same thing
> > >>>>
> > >>>> https://www.anandtech.com/show/6993/intel-iris-pro-5200-graphics-review-core-i74950hq-tested/2
> > >>>>
> > >>>> I have absolutely no direct knowledge, one way or the other, but if
> > >>>> Wikipedia is wrong, they at least have a pretty good reference.
> > >>>
> > >>> Anand explains it well. Read the 3rd page.
> > >>> "Despite its name, the eDRAM silicon is actually separate from the main microprocessor die - it’s simply housed on the same package."
> > >> Yes. I hadn't read past the first section. :-( But it is a, albeit
> > >> modified, Intel logic process, not a DRAM process. See the quote from
> > >> the article
> > >>
> > >>> As we’ve been talking about for a while now, the highest end Haswell graphics configuration includes 128MB of eDRAM on-package. The eDRAM itself is a custom design by Intel and it’s built on a variant of Intel’s P1271 22nm SoC process (not P1270, the CPU process). Intel needed a set of low leakage 22nm transistors rather than the ability to drive very high frequencies which is why it’s using the mobile SoC 22nm process variant here.
> > >>
> > >> The process change didn't seem to be about the capacitors, but I am way
> > >> beyond my depth here.
> > >
> > > The process they used for Crystal Well eDRAM die is not fast enough for 3+GHz CPU.
> > I certainly agree with that.
> > > It's probably different now, but back 5-6 years ago "mobile process" meant "optimized for reduced leakage and generally for low static power consumption at big price in increased dynamic power consumption".
> > Seems very reasonable.
> > >
> > > To be clear, I don't say that other silicon manufacturers don't know, how to do eDRAM. They do know, foundries even have eDRAM in their libraries.
> > > But all those non-IBM eDRAMs are not good enough to serve as L3 cache on the same die with 4-5 GHz CPU cores. Even less so, to serve as L2 cache, like in IBM z15.
> > OK, I see what you are saying. To paraphrase, anyone can do eDRAM, but
> > IBM does it enough better than anyone else to make it fast enough to be
> > used as an L3, much less an L2 cache. Is that a reasonable restatement?
> >
> > Regarding L3 caches, certainly "fast enough" is a little too concise.
> > Obviously eDRAM will be slower than SRAM, but the idea is that the
> > larger sized caches allowed by the smaller cells increases the hit rate
> > enough to compensate for the slower access time. I take your comments
> > to mean that, except for IBM, that tradeoff doesn't work out. Is that fair?
> >
> > Since you are far more knowledgeable than I am in this area, let me ask
> > a question.
<
> My knowledge is at the "blackbox" level, interfaces and parameters visible to outside world, rather than internal operations.
<
> > One of the advantages of eDRAM over "commodity DRAM" is
> > that one could, at least in theory, configure the bank sizes to meet
> > your needs. Given that the communication is all on chip, so number of
> > wires isn't a big issue, couldn't you configure the row size to be equal
> > to the cache line size such that once you get the row into the buffers,
> > you can transfer the whole row in one clock, essentially eliminating the
> > whole CAS part of the access time and transferring a whole cache line at
> > once instead of the eight clocks typically used for external DRAM? Does
> > this work?
<
> I think that more that, say, 8K rows per bank wouldn't work.
<
At equivalent word line speed a DRAM can have 2× as many bits on the word line
as an SRAM.
<
Modern fast SRAMs have no more than 128bits on a word line and no more than
64 words on a bit line (otherwise the leakage can be sufficient to drown out the
driving cell compared to the leakage of all the unselected cells.
<
DRAM cells have a bit less than ½ the bit line drive capability.
<
So the modern SRAM macro will have 1KB-2KB and so with the DRAM macros.
<
Let us postulate that the DRAM cell is ¼ the size of the SRAM cell and that the
SRAM overhead is 20% of the array size. This means that three (3) DRAM macros
are the size of an SRAM macro.
<
> But a lot of small banks, say 8K rows * 128B * 256 banks for total size of 256 MB, does not seem unreasonable if one is willing to sacrifice a bit of density (more banks==more sense amplifiers).
<
Yes, lots of banks (maybe as many as 128 or 256) to keep the "same bank" problem to
a minimum. {Somebody prior to the DRAM cache should have already filtered out any
"I need the same line as him" conflicts"}
<
> I wouldn't be surprised if that's how things were done in POWER9 except that the total was only 120 MB
> > --
> > - Stephen Fuld
> > (e-mail address disguised to prevent spam)

> CPU architects (like me) have advocated for DRAM caches since caches
> got ECC. Test and marketing guys wave red flags at the people with money.

But DRAM _is_ only a cache for disk. At least in COMA.

On 6/11/2021 1:34 PM, MitchAlsup wrote:
> On Friday, June 11, 2021 at 2:43:23 PM UTC-5, Chris M. Thomasson wrote:
>> On 6/11/2021 6:03 AM, Terje Mathisen wrote:
>>> Quadibloc wrote:
>>>> On Thursday, June 10, 2021 at 1:59:19 PM UTC-6, MitchAlsup wrote:
>>>>> On Thursday, June 10, 2021 at 1:26:57 AM UTC-5, Thomas Koenig wrote:
>>>>
>>>>>> It's certainly an idea. I could see a vector processor integrated
>>>>>> tightly with the RAM so it can bypass all of the caches etc and
>>>>>> only the end result of some calculation needs to reach the CPU.
>>>>>>
>>>>>> However, it hasn't caught on, and there are several reasons why this
>>>>>> could be the case.
>>>>> <
>>>>> FFTs cannot be performed on such an architecture. FFTs have the
>>>>> aspect that sooner (Decimation in time) or later (decimation in
>>>>> frequency) you are touching memory that is very far apart. That is
>>>>> it is not suitable for bringing the compute to memory architecture.
>>>>
>>>> Yes.
>>>>
>>>> I would have used *matrix multiplications* as my example, and once
>>>> again you have to touch memory far apart to do them.
>>>>
>>>> That's not to say that processing in memory integrated with DRAM
>>>> isn't of _some_ use; *other* kinds of dumb calculations that don't
>>>> require accessing far-apart memory could at least be done massively.
>>>>
>>>> However, my concept of doing processing in memory is based on not
>>>> just one, but *two*, historic failures in the field of computing.
>>>>
>>>> The Illiac IV and the PDP-8/S.
>>>>
>>>> The Illiac IV was SIMD instead of MIMD, and that was why it failed;
>>>> unlike modern "MIMD" computing, it was constructed in such a way
>>>> that the cost savings from making it SIMD instead of MIMD were
>>>> modest - even though the loss in usability and power was large.
>>>>
>>>> The PDP-8/S had a serial ALU, and that made it slow.
>>>>
>>>> The idea is:
>>>>
>>>> You have to make the DRAM chips 64 bits wide to do calculations on
>>>> 64 bit values. That's unfortunate. However, since today's DRAM only
>>>> works at maximum efficiency when you pump out 16 consecutive values
>>>> at a time... you could leave the pins at 4 bits wide, and just
>>>> re-organize
>>>> how values are stored in memory.
>>>>
>>>> The downside is now that you _always_ have to read the memory in the
>>>> maximum efficiency value, and can never read just one value if that's
>>>> all you need.
>>>>
>>>> Each DRAM chip gets one, or a bunch of, seriously minimal CPUs.
>>>>
>>>> These consist of:
>>>>
>>>> A 64-bit shift register or three...
>>>>
>>>> A serial ALU that is controlled from an external pin of the DRAM chip,
>>>>
>>>> Storage for a handful of logic flags.
>>>>
>>>> So this adds a _minimal_ amount of circuitry to the DRAM chip, on the
>>>> basis that usually most systems will seldom if ever use this capability,
>>>> so providing it can't threaten cost-competitiveness.
>>>>
>>>> When it is used...
>>>>
>>>> The central CPU steps a range of memory through a calculation. Say it
>>>> wants
>>>> to replace the 64-bit floating-point numbers in an enormous swath of
>>>> memory
>>>> with their sines.
>>>>
>>>> It steps the serial ALUs through the calculation. One shift and add at
>>>> a time.
>>>>
>>>> Sometimes, there will be "test" instructions that set a logic flag,
>>>> and sometimes
>>>> code sequences will be predicated on one of those logic flags. (The
>>>> Illiac IV
>>>> did that kind of stuff.)
>>>>
>>>> You can actually manage to do 64-bit floating arithmetic with a
>>>> single-bit ALU.
>>>> It just takes time. But if you have thousands of them all working at
>>>> once, it's
>>>> not too bad.
>>>
>>> What would the power efficiency be of such a system, i.e. compared to
>>> streaming the same data past a GPU array?
> <
>> If it can get 60fps rendering a hardcore scene, it should be okay.
> <
> <
> Well GPU tends to have embarisingly large amounts of parallelism, so maybe
> 1,000,000,000 1-bit CPUs at 1 GHz would be sufficient.

That would allow for a 31622^2 resolution to have a cpu per pixel. Nice!

> <
>> Shadertoy is a nice friend to have:
>>
>> https://www.shadertoy.com/view/Ms2SD1
>>
>> https://www.shadertoy.com/view/Nt2GRW
>>
>> ;^)