Rocksolid Light

Welcome to novaBBS (click a section below)

mail  files  register  newsreader  groups  login

Message-ID:  

"Turn on, tune up, rock out." -- Billy Gibbons


devel / comp.arch / Re: Versatile Cache from IBM

SubjectAuthor
* Versatile Cache from IBMQuadibloc
+* Re: Versatile Cache from IBMTerje Mathisen
|+- Re: Versatile Cache from IBMEricP
|`* Re: Versatile Cache from IBMQuadibloc
| `- Re: Versatile Cache from IBMTerje Mathisen
`* Re: Versatile Cache from IBMAnton Ertl
 +* Re: Versatile Cache from IBMStefan Monnier
 |`* Re: Versatile Cache from IBMJohn Levine
 | `* Re: Versatile Cache from IBMQuadibloc
 |  `* Re: Versatile Cache from IBMJohn Dallman
 |   +- Re: Versatile Cache from IBMAnne & Lynn Wheeler
 |   +* Re: Versatile Cache from IBMStephen Fuld
 |   |`* Re: Versatile Cache from IBMJohn Dallman
 |   | `* Re: Versatile Cache from IBMStephen Fuld
 |   |  `* Re: Versatile Cache from IBMJohn Dallman
 |   |   `- Re: Versatile Cache from IBMStephen Fuld
 |   `* Re: Versatile Cache from IBMAl Grant
 |    `- Re: Versatile Cache from IBMThomas Koenig
 `- Re: Versatile Cache from IBMStephen Fuld

1
Versatile Cache from IBM

<70130c3b-ecb9-4098-86be-7fc7e5506e3bn@googlegroups.com>

 copy mid

https://www.novabbs.com/devel/article-flat.php?id=20255&group=comp.arch#20255

 copy link   Newsgroups: comp.arch
X-Received: by 2002:a0c:90d0:: with SMTP id p74mr10350982qvp.29.1630905765781;
Sun, 05 Sep 2021 22:22:45 -0700 (PDT)
X-Received: by 2002:a9d:7110:: with SMTP id n16mr9624256otj.94.1630905765551;
Sun, 05 Sep 2021 22:22:45 -0700 (PDT)
Path: i2pn2.org!i2pn.org!weretis.net!feeder6.news.weretis.net!news.misty.com!border2.nntp.dca1.giganews.com!nntp.giganews.com!news-out.google.com!nntp.google.com!postnews.google.com!google-groups.googlegroups.com!not-for-mail
Newsgroups: comp.arch
Date: Sun, 5 Sep 2021 22:22:45 -0700 (PDT)
Injection-Info: google-groups.googlegroups.com; posting-host=2001:56a:f39d:2c00:e8c3:cfaf:11c7:a0ad;
posting-account=1nOeKQkAAABD2jxp4Pzmx9Hx5g9miO8y
NNTP-Posting-Host: 2001:56a:f39d:2c00:e8c3:cfaf:11c7:a0ad
User-Agent: G2/1.0
MIME-Version: 1.0
Message-ID: <70130c3b-ecb9-4098-86be-7fc7e5506e3bn@googlegroups.com>
Subject: Versatile Cache from IBM
From: jsav...@ecn.ab.ca (Quadibloc)
Injection-Date: Mon, 06 Sep 2021 05:22:45 +0000
Content-Type: text/plain; charset="UTF-8"
Lines: 15
 by: Quadibloc - Mon, 6 Sep 2021 05:22 UTC

Just read this:

https://www.anandtech.com/show/16924/did-ibm-just-preview-the-future-of-caches

IBM's new Telum chip, for its next generation of Z-series mainframes,
has 8 processors per die, each one with 32 megabytes of L2 cache.

There is no L3 or L4 cache... in the usual sense.

However, processors on the same die may make some of their L2
cache available as L3 cache for other processors on the die, and
processors on other dies on the same multi-chip module may even
provide some of their L2 cache as L4 cache for processors on other
dies.

John Savard

Re: Versatile Cache from IBM

<sh4cp4$j7p$1@gioia.aioe.org>

 copy mid

https://www.novabbs.com/devel/article-flat.php?id=20258&group=comp.arch#20258

 copy link   Newsgroups: comp.arch
Path: i2pn2.org!i2pn.org!aioe.org!Liunnst7X9VOeBBqqVtBCw.user.46.165.242.91.POSTED!not-for-mail
From: terje.ma...@tmsw.no (Terje Mathisen)
Newsgroups: comp.arch
Subject: Re: Versatile Cache from IBM
Date: Mon, 6 Sep 2021 08:34:12 +0200
Organization: Aioe.org NNTP Server
Message-ID: <sh4cp4$j7p$1@gioia.aioe.org>
References: <70130c3b-ecb9-4098-86be-7fc7e5506e3bn@googlegroups.com>
Mime-Version: 1.0
Content-Type: text/plain; charset=ISO-8859-1; format=flowed
Content-Transfer-Encoding: 7bit
Injection-Info: gioia.aioe.org; logging-data="19705"; posting-host="Liunnst7X9VOeBBqqVtBCw.user.gioia.aioe.org"; mail-complaints-to="abuse@aioe.org";
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:60.0) Gecko/20100101
Firefox/60.0 SeaMonkey/2.53.9
X-Notice: Filtered by postfilter v. 0.9.2
 by: Terje Mathisen - Mon, 6 Sep 2021 06:34 UTC

Quadibloc wrote:
> Just read this:
>
> https://www.anandtech.com/show/16924/did-ibm-just-preview-the-future-of-caches
>
> IBM's new Telum chip, for its next generation of Z-series mainframes,
> has 8 processors per die, each one with 32 megabytes of L2 cache.
>
> There is no L3 or L4 cache... in the usual sense.
>
> However, processors on the same die may make some of their L2
> cache available as L3 cache for other processors on the die, and
> processors on other dies on the same multi-chip module may even
> provide some of their L2 cache as L4 cache for processors on other
> dies.

Isn't this how many/most multi-CPU slot machines work?

All the CPUs look at memory traffic from other chips, and update their
own cache state correspondingly, flushing dirty lines to RAM or a common
L3/L4. Sending an updated line directly to the requesting core is a
common optimization.

If the IBM idea is an OS-level feature where one CPU can reserve some of
its own L2 cache space for another CPU, presumably to serve as a victim
cache, then that is new I guess?

Terje

--
- <Terje.Mathisen at tmsw.no>
"almost all programming can be viewed as an exercise in caching"

Re: Versatile Cache from IBM

<dNnZI.26805$2B4.18346@fx04.iad>

 copy mid

https://www.novabbs.com/devel/article-flat.php?id=20263&group=comp.arch#20263

 copy link   Newsgroups: comp.arch
Path: i2pn2.org!i2pn.org!paganini.bofh.team!news.dns-netz.com!news.freedyn.net!newsreader4.netcologne.de!news.netcologne.de!peer01.ams1!peer.ams1.xlned.com!news.xlned.com!peer01.iad!feed-me.highwinds-media.com!news.highwinds-media.com!fx04.iad.POSTED!not-for-mail
From: ThatWoul...@thevillage.com (EricP)
User-Agent: Thunderbird 2.0.0.24 (Windows/20100228)
MIME-Version: 1.0
Newsgroups: comp.arch
Subject: Re: Versatile Cache from IBM
References: <70130c3b-ecb9-4098-86be-7fc7e5506e3bn@googlegroups.com> <sh4cp4$j7p$1@gioia.aioe.org>
In-Reply-To: <sh4cp4$j7p$1@gioia.aioe.org>
Content-Type: text/plain; charset=ISO-8859-1; format=flowed
Content-Transfer-Encoding: 7bit
Lines: 49
Message-ID: <dNnZI.26805$2B4.18346@fx04.iad>
X-Complaints-To: abuse@UsenetServer.com
NNTP-Posting-Date: Mon, 06 Sep 2021 12:25:45 UTC
Date: Mon, 06 Sep 2021 08:25:45 -0400
X-Received-Bytes: 2615
 by: EricP - Mon, 6 Sep 2021 12:25 UTC

Terje Mathisen wrote:
> Quadibloc wrote:
>> Just read this:
>>
>> https://www.anandtech.com/show/16924/did-ibm-just-preview-the-future-of-caches
>>
>>
>> IBM's new Telum chip, for its next generation of Z-series mainframes,
>> has 8 processors per die, each one with 32 megabytes of L2 cache.
>>
>> There is no L3 or L4 cache... in the usual sense.
>>
>> However, processors on the same die may make some of their L2
>> cache available as L3 cache for other processors on the die, and
>> processors on other dies on the same multi-chip module may even
>> provide some of their L2 cache as L4 cache for processors on other
>> dies.
>
> Isn't this how many/most multi-CPU slot machines work?
>
> All the CPUs look at memory traffic from other chips, and update their
> own cache state correspondingly, flushing dirty lines to RAM or a common
> L3/L4. Sending an updated line directly to the requesting core is a
> common optimization.
>
> If the IBM idea is an OS-level feature where one CPU can reserve some of
> its own L2 cache space for another CPU, presumably to serve as a victim
> cache, then that is new I guess?
>
> Terje
>

The article didn't say anything about reserve - it sounded dynamic.
They eliminated L3 & L4, and allow a remote nodes' L2 to be used
as another nodes victim cache.

Where a traditional coherence protocol only allows a core to
pull a line from a remote nodes' cache, this one additionally
allows it to push a line into a remote nodes' cache.

If you have idle cores and the remote L2 would otherwise be unused
this is a win.

But it makes matching application working set to cache size a bit fuzzy.

I can also see the potential for system wide thrashing if the total
working set size exceeds capacity, similar to old Unix systems with
their global working set management.

Re: Versatile Cache from IBM

<2021Sep6.152856@mips.complang.tuwien.ac.at>

 copy mid

https://www.novabbs.com/devel/article-flat.php?id=20266&group=comp.arch#20266

 copy link   Newsgroups: comp.arch
Path: i2pn2.org!i2pn.org!eternal-september.org!reader02.eternal-september.org!.POSTED!not-for-mail
From: ant...@mips.complang.tuwien.ac.at (Anton Ertl)
Newsgroups: comp.arch
Subject: Re: Versatile Cache from IBM
Date: Mon, 06 Sep 2021 13:28:56 GMT
Organization: Institut fuer Computersprachen, Technische Universitaet Wien
Lines: 44
Message-ID: <2021Sep6.152856@mips.complang.tuwien.ac.at>
References: <70130c3b-ecb9-4098-86be-7fc7e5506e3bn@googlegroups.com>
Injection-Info: reader02.eternal-september.org; posting-host="03f3240762f4be93a744c0efcd918770";
logging-data="16435"; mail-complaints-to="abuse@eternal-september.org"; posting-account="U2FsdGVkX1+8TsITWfRK9NXquKw004zS"
Cancel-Lock: sha1:vgKBDBOjJ9gIhyb5kyUtGZHNWtk=
X-newsreader: xrn 10.00-beta-3
 by: Anton Ertl - Mon, 6 Sep 2021 13:28 UTC

Quadibloc <jsavard@ecn.ab.ca> writes:
>Just read this:
>
>https://www.anandtech.com/show/16924/did-ibm-just-preview-the-future-of-caches
>
>IBM's new Telum chip, for its next generation of Z-series mainframes,
>has 8 processors per die, each one with 32 megabytes of L2 cache.
>
>There is no L3 or L4 cache... in the usual sense.
>
>However, processors on the same die may make some of their L2
>cache available as L3 cache for other processors on the die, and
>processors on other dies on the same multi-chip module may even
>provide some of their L2 cache as L4 cache for processors on other
>dies.

Unfortunately, both the Hot Chips slides and the article (that is
supposed to be written after additional Q&A with the IBM guys;
apparently the IBM guys were not allowed to reveal anything more) miss
pretty much all the interesting details. Basically, the only thing we
learn ist that other CPUs L2 caches can be used as victim caches.

Questions that come to my mind:

How does a line become "unused" and available for other CPUs? EricP
suggests that cache lines on idle cores are unused, but a currently
idle core could continue on the same task the very next microsecond,
so that does not sound like a good strategy. Cache lines belonging to
no-longer-allocated ASIDs might be a better approach; it probably
requires some scrubbing (probably in hardware), though. Or maybe a
long-unaccessed cache line that is still accessible could also be
categorized as "unused".

Does this system employ a directory or snooping. With hundreds of L2
caches, snooping seems inefficient, but what do I know. A directory
might be based on the memory controller for the physical RAM that
holds the memory if not in cache; i.e., send the request to the memory
controller, and it will send it on to RAM or the appropriate L2 cache
based on the directory.

- anton
--
'Anyone trying for "industrial quality" ISA should avoid undefined behavior.'
Mitch Alsup, <c17fcd89-f024-40e7-a594-88a85ac10d20o@googlegroups.com>

Re: Versatile Cache from IBM

<jwvsfyh7pcj.fsf-monnier+comp.arch@gnu.org>

 copy mid

https://www.novabbs.com/devel/article-flat.php?id=20270&group=comp.arch#20270

 copy link   Newsgroups: comp.arch
Path: i2pn2.org!i2pn.org!weretis.net!feeder8.news.weretis.net!eternal-september.org!reader02.eternal-september.org!.POSTED!not-for-mail
From: monn...@iro.umontreal.ca (Stefan Monnier)
Newsgroups: comp.arch
Subject: Re: Versatile Cache from IBM
Date: Mon, 06 Sep 2021 10:27:50 -0400
Organization: A noiseless patient Spider
Lines: 6
Message-ID: <jwvsfyh7pcj.fsf-monnier+comp.arch@gnu.org>
References: <70130c3b-ecb9-4098-86be-7fc7e5506e3bn@googlegroups.com>
<2021Sep6.152856@mips.complang.tuwien.ac.at>
Mime-Version: 1.0
Content-Type: text/plain
Injection-Info: reader02.eternal-september.org; posting-host="9fd737a8d44ae1880b4a37cdc160d617";
logging-data="24307"; mail-complaints-to="abuse@eternal-september.org"; posting-account="U2FsdGVkX19pZW2plYwLG8N9WnIzpBXJ"
User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/28.0.50 (gnu/linux)
Cancel-Lock: sha1:MKPgsbG8Y7iIHrnVEeMKRrZPhQo=
sha1:zyICIML7gRzHhHmVYwsNJ4Oi5Fs=
 by: Stefan Monnier - Mon, 6 Sep 2021 14:27 UTC

> How does a line become "unused" and available for other CPUs?

Is there any reason to think this isn't just decided by the OS?

Stefan

Re: Versatile Cache from IBM

<2b8598ec-6dfd-435c-b1c7-3f60ddb76237n@googlegroups.com>

 copy mid

https://www.novabbs.com/devel/article-flat.php?id=20271&group=comp.arch#20271

 copy link   Newsgroups: comp.arch
X-Received: by 2002:ac8:44aa:: with SMTP id a10mr11299064qto.63.1630940664532; Mon, 06 Sep 2021 08:04:24 -0700 (PDT)
X-Received: by 2002:a4a:e1a4:: with SMTP id 4mr14214560ooy.14.1630940664134; Mon, 06 Sep 2021 08:04:24 -0700 (PDT)
Path: i2pn2.org!i2pn.org!usenet.goja.nl.eu.org!2.eu.feeder.erje.net!feeder.erje.net!newsfeed.xs4all.nl!newsfeed7.news.xs4all.nl!tr2.eu1.usenetexpress.com!feeder.usenetexpress.com!tr2.iad1.usenetexpress.com!border1.nntp.dca1.giganews.com!nntp.giganews.com!news-out.google.com!nntp.google.com!postnews.google.com!google-groups.googlegroups.com!not-for-mail
Newsgroups: comp.arch
Date: Mon, 6 Sep 2021 08:04:23 -0700 (PDT)
In-Reply-To: <sh4cp4$j7p$1@gioia.aioe.org>
Injection-Info: google-groups.googlegroups.com; posting-host=2001:56a:f39d:2c00:cda1:52ef:a6dc:d771; posting-account=1nOeKQkAAABD2jxp4Pzmx9Hx5g9miO8y
NNTP-Posting-Host: 2001:56a:f39d:2c00:cda1:52ef:a6dc:d771
References: <70130c3b-ecb9-4098-86be-7fc7e5506e3bn@googlegroups.com> <sh4cp4$j7p$1@gioia.aioe.org>
User-Agent: G2/1.0
MIME-Version: 1.0
Message-ID: <2b8598ec-6dfd-435c-b1c7-3f60ddb76237n@googlegroups.com>
Subject: Re: Versatile Cache from IBM
From: jsav...@ecn.ab.ca (Quadibloc)
Injection-Date: Mon, 06 Sep 2021 15:04:24 +0000
Content-Type: text/plain; charset="UTF-8"
Lines: 11
 by: Quadibloc - Mon, 6 Sep 2021 15:04 UTC

On Monday, September 6, 2021 at 12:34:15 AM UTC-6, Terje Mathisen wrote:

> Isn't this how many/most multi-CPU slot machines work?
>
> All the CPUs look at memory traffic from other chips, and update their
> own cache state correspondingly, flushing dirty lines to RAM or a common
> L3/L4.

That's just maintaining cache coherency. Actually using the caches of other
processors as a higher-level cache is something different.

John Savard

Re: Versatile Cache from IBM

<sh5nr0$2fui$1@gal.iecc.com>

 copy mid

https://www.novabbs.com/devel/article-flat.php?id=20297&group=comp.arch#20297

 copy link   Newsgroups: comp.arch
Path: i2pn2.org!i2pn.org!weretis.net!feeder6.news.weretis.net!news.misty.com!news.iecc.com!.POSTED.news.iecc.com!not-for-mail
From: joh...@taugh.com (John Levine)
Newsgroups: comp.arch
Subject: Re: Versatile Cache from IBM
Date: Mon, 6 Sep 2021 18:49:04 -0000 (UTC)
Organization: Taughannock Networks
Message-ID: <sh5nr0$2fui$1@gal.iecc.com>
References: <70130c3b-ecb9-4098-86be-7fc7e5506e3bn@googlegroups.com> <2021Sep6.152856@mips.complang.tuwien.ac.at> <jwvsfyh7pcj.fsf-monnier+comp.arch@gnu.org>
Injection-Date: Mon, 6 Sep 2021 18:49:04 -0000 (UTC)
Injection-Info: gal.iecc.com; posting-host="news.iecc.com:2001:470:1f07:1126:0:676f:7373:6970";
logging-data="81874"; mail-complaints-to="abuse@iecc.com"
In-Reply-To: <70130c3b-ecb9-4098-86be-7fc7e5506e3bn@googlegroups.com> <2021Sep6.152856@mips.complang.tuwien.ac.at> <jwvsfyh7pcj.fsf-monnier+comp.arch@gnu.org>
Cleverness: some
X-Newsreader: trn 4.0-test77 (Sep 1, 2010)
Originator: johnl@iecc.com (John Levine)
 by: John Levine - Mon, 6 Sep 2021 18:49 UTC

According to Stefan Monnier <monnier@iro.umontreal.ca>:
>> How does a line become "unused" and available for other CPUs?
>
>Is there any reason to think this isn't just decided by the OS?

Seems unlikely. IBM systems run a lot of legacy code including old systems in
virtual machines and if this is going to be useful it won't depend on OS upgrades.

I suppose the physical OS knows when it's reclaimed a page frame but that seems
rather coarse granularity for a cache flush.

--
Regards,
John Levine, johnl@taugh.com, Primary Perpetrator of "The Internet for Dummies",
Please consider the environment before reading this e-mail. https://jl.ly

Re: Versatile Cache from IBM

<42fd1c31-424a-4ec8-9a9c-ce470a8b5064n@googlegroups.com>

 copy mid

https://www.novabbs.com/devel/article-flat.php?id=20304&group=comp.arch#20304

 copy link   Newsgroups: comp.arch
X-Received: by 2002:a37:ea0c:: with SMTP id t12mr12438461qkj.509.1630959647519;
Mon, 06 Sep 2021 13:20:47 -0700 (PDT)
X-Received: by 2002:a05:6808:2114:: with SMTP id r20mr602407oiw.110.1630959647091;
Mon, 06 Sep 2021 13:20:47 -0700 (PDT)
Path: i2pn2.org!i2pn.org!weretis.net!feeder6.news.weretis.net!news.misty.com!border2.nntp.dca1.giganews.com!nntp.giganews.com!news-out.google.com!nntp.google.com!postnews.google.com!google-groups.googlegroups.com!not-for-mail
Newsgroups: comp.arch
Date: Mon, 6 Sep 2021 13:20:46 -0700 (PDT)
In-Reply-To: <sh5nr0$2fui$1@gal.iecc.com>
Injection-Info: google-groups.googlegroups.com; posting-host=2001:56a:f39d:2c00:cda1:52ef:a6dc:d771;
posting-account=1nOeKQkAAABD2jxp4Pzmx9Hx5g9miO8y
NNTP-Posting-Host: 2001:56a:f39d:2c00:cda1:52ef:a6dc:d771
References: <70130c3b-ecb9-4098-86be-7fc7e5506e3bn@googlegroups.com>
<2021Sep6.152856@mips.complang.tuwien.ac.at> <jwvsfyh7pcj.fsf-monnier+comp.arch@gnu.org>
<sh5nr0$2fui$1@gal.iecc.com>
User-Agent: G2/1.0
MIME-Version: 1.0
Message-ID: <42fd1c31-424a-4ec8-9a9c-ce470a8b5064n@googlegroups.com>
Subject: Re: Versatile Cache from IBM
From: jsav...@ecn.ab.ca (Quadibloc)
Injection-Date: Mon, 06 Sep 2021 20:20:47 +0000
Content-Type: text/plain; charset="UTF-8"
Lines: 10
 by: Quadibloc - Mon, 6 Sep 2021 20:20 UTC

On Monday, September 6, 2021 at 12:49:07 PM UTC-6, John Levine wrote:

> Seems unlikely. IBM systems run a lot of legacy code including old systems in
> virtual machines and if this is going to be useful it won't depend on OS upgrades.

It's not clear to me why an upgraded OS conflicts with legacy applications.

I mean, one could even have the cache allocated at the hypervisor level if you wanted
to run each legacy application under its own legacy operating system.

John Savard

Re: Versatile Cache from IBM

<sh5tkc$g95$1@gioia.aioe.org>

 copy mid

https://www.novabbs.com/devel/article-flat.php?id=20305&group=comp.arch#20305

 copy link   Newsgroups: comp.arch
Path: i2pn2.org!i2pn.org!aioe.org!ppYixYMWAWh/woI8emJOIQ.user.46.165.242.91.POSTED!not-for-mail
From: terje.ma...@tmsw.no (Terje Mathisen)
Newsgroups: comp.arch
Subject: Re: Versatile Cache from IBM
Date: Mon, 6 Sep 2021 22:27:55 +0200
Organization: Aioe.org NNTP Server
Message-ID: <sh5tkc$g95$1@gioia.aioe.org>
References: <70130c3b-ecb9-4098-86be-7fc7e5506e3bn@googlegroups.com>
<sh4cp4$j7p$1@gioia.aioe.org>
<2b8598ec-6dfd-435c-b1c7-3f60ddb76237n@googlegroups.com>
Mime-Version: 1.0
Content-Type: text/plain; charset=ISO-8859-1; format=flowed
Content-Transfer-Encoding: 7bit
Injection-Info: gioia.aioe.org; logging-data="16677"; posting-host="ppYixYMWAWh/woI8emJOIQ.user.gioia.aioe.org"; mail-complaints-to="abuse@aioe.org";
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:60.0) Gecko/20100101
Firefox/60.0 SeaMonkey/2.53.9
X-Notice: Filtered by postfilter v. 0.9.2
 by: Terje Mathisen - Mon, 6 Sep 2021 20:27 UTC

Quadibloc wrote:
> On Monday, September 6, 2021 at 12:34:15 AM UTC-6, Terje Mathisen wrote:
>
>> Isn't this how many/most multi-CPU slot machines work?
>>
>> All the CPUs look at memory traffic from other chips, and update their
>> own cache state correspondingly, flushing dirty lines to RAM or a common
>> L3/L4.
>
> That's just maintaining cache coherency. Actually using the caches of other
> processors as a higher-level cache is something different.

Which was in my next (snipped) paragraph. Why didn't you respond to that
part instead?

Terje

--
- <Terje.Mathisen at tmsw.no>
"almost all programming can be viewed as an exercise in caching"

Re: Versatile Cache from IBM

<memo.20210906214600.9608A@jgd.cix.co.uk>

 copy mid

https://www.novabbs.com/devel/article-flat.php?id=20308&group=comp.arch#20308

 copy link   Newsgroups: comp.arch
Path: i2pn2.org!i2pn.org!eternal-september.org!reader02.eternal-september.org!.POSTED!not-for-mail
From: jgd...@cix.co.uk (John Dallman)
Newsgroups: comp.arch
Subject: Re: Versatile Cache from IBM
Date: Mon, 6 Sep 2021 21:46 +0100 (BST)
Organization: A noiseless patient Spider
Lines: 20
Message-ID: <memo.20210906214600.9608A@jgd.cix.co.uk>
References: <42fd1c31-424a-4ec8-9a9c-ce470a8b5064n@googlegroups.com>
Reply-To: jgd@cix.co.uk
Injection-Info: reader02.eternal-september.org; posting-host="367c5b4b1e493bf7b36b3c1a9c7f3e3e";
logging-data="28970"; mail-complaints-to="abuse@eternal-september.org"; posting-account="U2FsdGVkX1+tn35fOm9Js2J27ZuRF8begyZlbA13VFk="
Cancel-Lock: sha1:4iTxEsRAdbMUHiz/BhFBtpbw1pQ=
 by: John Dallman - Mon, 6 Sep 2021 20:46 UTC

In article <42fd1c31-424a-4ec8-9a9c-ce470a8b5064n@googlegroups.com>,
jsavard@ecn.ab.ca (Quadibloc) wrote:

> It's not clear to me why an upgraded OS conflicts with legacy
> applications.

Some aspects of those operating systems are surprisingly primitive. As in
data structures expected to be accessed at absolute addresses, for
example. There are some very baroque sets of registers for remapping
memory in weird ways.

The "Principles of Operation" manual is free, available here:
<https://www.ibm.com/support/pages/zarchitecture-principles-operation>

I read some sections of it recently, wondering what the programming
environment was like. It's a weird mixture of primitive and sophisticated.
My conclusion was "Stick to Linux, if I ever have to deal with this
platform."

John

Re: Versatile Cache from IBM

<87mtopqu1y.fsf@localhost>

 copy mid

https://www.novabbs.com/devel/article-flat.php?id=20309&group=comp.arch#20309

 copy link   Newsgroups: comp.arch
Path: i2pn2.org!i2pn.org!eternal-september.org!reader02.eternal-september.org!.POSTED!not-for-mail
From: lyn...@garlic.com (Anne & Lynn Wheeler)
Newsgroups: comp.arch
Subject: Re: Versatile Cache from IBM
Date: Mon, 06 Sep 2021 11:21:29 -1000
Organization: Wheeler&Wheeler
Lines: 49
Message-ID: <87mtopqu1y.fsf@localhost>
References: <42fd1c31-424a-4ec8-9a9c-ce470a8b5064n@googlegroups.com>
<memo.20210906214600.9608A@jgd.cix.co.uk>
Mime-Version: 1.0
Content-Type: text/plain
Injection-Info: reader02.eternal-september.org; posting-host="7bf7e32309ea1fb2a7430a981a923556";
logging-data="11206"; mail-complaints-to="abuse@eternal-september.org"; posting-account="U2FsdGVkX1/82dX/51qKaMyqCwWI+mVPfLHZw7taFg4="
User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/27.2 (gnu/linux)
Cancel-Lock: sha1:N+Y/rjlWMQ7s6Wz1Z7ZRNXZ8dy0=
sha1:oEcBBRJsPT41PmTAsS6SEioRN48=
 by: Anne & Lynn Whee - Mon, 6 Sep 2021 21:21 UTC

jgd@cix.co.uk (John Dallman) writes:
> The "Principles of Operation" manual is free, available here:
> <https://www.ibm.com/support/pages/zarchitecture-principles-operation>
>
> I read some sections of it recently, wondering what the programming
> environment was like. It's a weird mixture of primitive and sophisticated.
> My conclusion was "Stick to Linux, if I ever have to deal with this
> platform."

access registers (multiple concurrent, active address spaces) & program
call started out as part of 811 (for 370/xa arch. documents dated
nov1978) for MVS. OS/360 was heavily pointer passing API. MVT (real
storage) was initially mapped to a single 16mbyte virtual address space
for VS2/R1 ... then for MVS, VS2/R2 each application and subsystem was
giving its own 16mbyte virtual address space. However, for the pointer
passing APIs, they mapped an 8mbyte image of the kernel into every
application virtual adddress space (leaving 8mbytes for application).
Then for subsystem calls (each in its own private 16mbyte virtual
address space), they created the common segment (1mbyte) mapped into
every 16mbyte virtual address space, parameter list/returns storage can
be allocated in the common segment area and the pointer passed to the
called subsystem.

Common area size requirement is somewhat proportional to number of
concurrent applications and number of subsystems ... by 3033 (before
370/xa & 31bit addressing, still 24bit/16mbytes)) installations were
requiring 5-6 mbytes for the common area (renamed from common segment
area, CSA, to common system area, CSA) leaving 2-3mbytes for
applications ... but threatening to increase to 8mbytes ... leaving zero
bytes for applications.

Come 3081, there is both 370 mode and 370/xa mode ... however customers
weren't migrating to MVS/XA as expected which was increasingly putting
enormous pressure for MVS operation as environments scaled up with more
concurrent applications executing and more running subsystems.

With 370/xa, program call and access registers ... there is privilege
("MVS") kernel table of subsystems and address space pointers.
Subsystem calls reference a specific entry in that table and the
hardware moves the caller's address space pointer into secondary (access
register) and loads the subsystem address space pointer as primarry and
transfers to the subsystem. The subsystem can now directly access the
caller's parameter list in its secondary address space (eliminating the
enormous pressure on the CSA as well as kernel call software overhead
for the switch). The return instruction restores the applications
address space to primary and returns to the caller

--
virtualization experience starting Jan1968, online at home since Mar1970

Re: Versatile Cache from IBM

<sh63lq$v79$1@dont-email.me>

 copy mid

https://www.novabbs.com/devel/article-flat.php?id=20310&group=comp.arch#20310

 copy link   Newsgroups: comp.arch
Path: i2pn2.org!i2pn.org!aioe.org!feeder1.feed.usenet.farm!feed.usenet.farm!newsfeed.xs4all.nl!newsfeed8.news.xs4all.nl!news-out.netnews.com!news.alt.net!fdc2.netnews.com!news.mixmin.net!eternal-september.org!reader02.eternal-september.org!.POSTED!not-for-mail
From: sfu...@alumni.cmu.edu.invalid (Stephen Fuld)
Newsgroups: comp.arch
Subject: Re: Versatile Cache from IBM
Date: Mon, 6 Sep 2021 15:11:03 -0700
Organization: A noiseless patient Spider
Lines: 29
Message-ID: <sh63lq$v79$1@dont-email.me>
References: <42fd1c31-424a-4ec8-9a9c-ce470a8b5064n@googlegroups.com>
<memo.20210906214600.9608A@jgd.cix.co.uk>
Mime-Version: 1.0
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Transfer-Encoding: 7bit
Injection-Date: Mon, 6 Sep 2021 22:11:07 -0000 (UTC)
Injection-Info: reader02.eternal-september.org; posting-host="7cc6d863c83299a204796e743ef84ac0";
logging-data="31977"; mail-complaints-to="abuse@eternal-september.org"; posting-account="U2FsdGVkX19JX0VwEKQ+dhyNt5/DFduCfJOFbaxi04A="
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
Thunderbird/78.13.0
Cancel-Lock: sha1:yh/pnPdEPl7CF6bJwb8S9hrQ4sU=
In-Reply-To: <memo.20210906214600.9608A@jgd.cix.co.uk>
Content-Language: en-US
 by: Stephen Fuld - Mon, 6 Sep 2021 22:11 UTC

On 9/6/2021 1:45 PM, John Dallman wrote:
> In article <42fd1c31-424a-4ec8-9a9c-ce470a8b5064n@googlegroups.com>,
> jsavard@ecn.ab.ca (Quadibloc) wrote:
>
>> It's not clear to me why an upgraded OS conflicts with legacy
>> applications.
>
> Some aspects of those operating systems are surprisingly primitive. As in
> data structures expected to be accessed at absolute addresses, for
> example. There are some very baroque sets of registers for remapping
> memory in weird ways.
>
> The "Principles of Operation" manual is free, available here:
> <https://www.ibm.com/support/pages/zarchitecture-principles-operation>
>
> I read some sections of it recently, wondering what the programming
> environment was like. It's a weird mixture of primitive and sophisticated.
> My conclusion was "Stick to Linux, if I ever have to deal with this
> platform."

You are not being exactly fair. First of all, what you see in the POO
is a lot of the "cruft" remaining from 60 years of development history,
always maintaining backward compatibility. Second, you are comparing
hardware as shown in the manual versus software (Linux).

--
- Stephen Fuld
(e-mail address disguised to prevent spam)

Re: Versatile Cache from IBM

<4e9b1d6c-5ccc-40d6-b5d1-13f47cddfbaen@googlegroups.com>

 copy mid

https://www.novabbs.com/devel/article-flat.php?id=20316&group=comp.arch#20316

 copy link   Newsgroups: comp.arch
X-Received: by 2002:a05:620a:4001:: with SMTP id h1mr14840513qko.454.1631002232480;
Tue, 07 Sep 2021 01:10:32 -0700 (PDT)
X-Received: by 2002:a54:470c:: with SMTP id k12mr2133876oik.78.1631002232103;
Tue, 07 Sep 2021 01:10:32 -0700 (PDT)
Path: i2pn2.org!i2pn.org!weretis.net!feeder6.news.weretis.net!news.misty.com!border2.nntp.dca1.giganews.com!nntp.giganews.com!news-out.google.com!nntp.google.com!postnews.google.com!google-groups.googlegroups.com!not-for-mail
Newsgroups: comp.arch
Date: Tue, 7 Sep 2021 01:10:31 -0700 (PDT)
In-Reply-To: <memo.20210906214600.9608A@jgd.cix.co.uk>
Injection-Info: google-groups.googlegroups.com; posting-host=217.140.106.13; posting-account=NkMcCAoAAABs23KArpYPXpvZx_NTJUQe
NNTP-Posting-Host: 217.140.106.13
References: <42fd1c31-424a-4ec8-9a9c-ce470a8b5064n@googlegroups.com> <memo.20210906214600.9608A@jgd.cix.co.uk>
User-Agent: G2/1.0
MIME-Version: 1.0
Message-ID: <4e9b1d6c-5ccc-40d6-b5d1-13f47cddfbaen@googlegroups.com>
Subject: Re: Versatile Cache from IBM
From: algrant...@gmail.com (Al Grant)
Injection-Date: Tue, 07 Sep 2021 08:10:32 +0000
Content-Type: text/plain; charset="UTF-8"
Lines: 33
 by: Al Grant - Tue, 7 Sep 2021 08:10 UTC

On Monday, September 6, 2021 at 9:46:03 PM UTC+1, John Dallman wrote:
> I read some sections of it recently, wondering what the programming
> environment was like. It's a weird mixture of primitive and sophisticated.
> My conclusion was "Stick to Linux, if I ever have to deal with this
> platform."

The programming environment is very usable. It's true that
some data is located in the 4K block located at 0x0, but that
page is more like an extended set of system registers
(the sort that on other architectures are accessed using
special instructions) - it's mapped to a different physical
address per CPU. Almost everything else is accessed via
pointer chaining - it's like Linux, but with documentation.

Linux runs on z/Series, and if new features need OS support,
they might show up in the upstream kernel.

I wrote a lot of 370 assembler in the 90s and read a lot
more. Code looked pretty much like it did in the 1960s.
Looking at the output of GCC for z/Series recently, I barely
recognized it. There are a lot of new instructions - and this
is the regular userspace ISA (or "problem state"). So the
userspace ISA has changed far more in its last 30 years
than in its first 30 years. Something must be using those
new instructions! The compilers and the major applications
(CICS and DB2 etc.) will be, at least. So, they may have
added support for the new cache features. My guess is
it would involve the "Next Instruction Access Intent"
instruction, which gives hints on cache placement for
whatever data is touched by the next instruction.

There may be a lot of legacy assembler code, but it was
written to run in small footprints and slow CPUs, so
performance on modern machines is less of an issue.

Re: Versatile Cache from IBM

<memo.20210907094936.9608B@jgd.cix.co.uk>

 copy mid

https://www.novabbs.com/devel/article-flat.php?id=20317&group=comp.arch#20317

 copy link   Newsgroups: comp.arch
Path: i2pn2.org!i2pn.org!eternal-september.org!reader02.eternal-september.org!.POSTED!not-for-mail
From: jgd...@cix.co.uk (John Dallman)
Newsgroups: comp.arch
Subject: Re: Versatile Cache from IBM
Date: Tue, 7 Sep 2021 09:49 +0100 (BST)
Organization: A noiseless patient Spider
Lines: 15
Message-ID: <memo.20210907094936.9608B@jgd.cix.co.uk>
References: <sh63lq$v79$1@dont-email.me>
Reply-To: jgd@cix.co.uk
Injection-Info: reader02.eternal-september.org; posting-host="c1ea565c8de55cadc454487146cd664f";
logging-data="29670"; mail-complaints-to="abuse@eternal-september.org"; posting-account="U2FsdGVkX19greGAn58onryWIWZL92QA9a5MvOky4iY="
Cancel-Lock: sha1:Qo6ZGu7RAv2XSVWZB2n6FqIINiQ=
 by: John Dallman - Tue, 7 Sep 2021 08:49 UTC

In article <sh63lq$v79$1@dont-email.me>, sfuld@alumni.cmu.edu.invalid
(Stephen Fuld) wrote:

> You are not being exactly fair. First of all, what you see in the
> POO is a lot of the "cruft" remaining from 60 years of development
> history, always maintaining backward compatibility. Second, you
> are comparing hardware as shown in the manual versus software
> (Linux).

Indeed. Part of an operating system's job is to hide hardware complexity
from the application programmer. However, since the traditional OSes for
Z architecture have grown with the hardware design, they seem to conceal
rather less than Linux.

John

Re: Versatile Cache from IBM

<sh7dn4$amu$1@newsreader4.netcologne.de>

 copy mid

https://www.novabbs.com/devel/article-flat.php?id=20318&group=comp.arch#20318

 copy link   Newsgroups: comp.arch
Path: i2pn2.org!i2pn.org!weretis.net!feeder8.news.weretis.net!newsreader4.netcologne.de!news.netcologne.de!.POSTED.2001-4dd7-c002-0-7285-c2ff-fe6c-992d.ipv6dyn.netcologne.de!not-for-mail
From: tkoe...@netcologne.de (Thomas Koenig)
Newsgroups: comp.arch
Subject: Re: Versatile Cache from IBM
Date: Tue, 7 Sep 2021 10:08:36 -0000 (UTC)
Organization: news.netcologne.de
Distribution: world
Message-ID: <sh7dn4$amu$1@newsreader4.netcologne.de>
References: <42fd1c31-424a-4ec8-9a9c-ce470a8b5064n@googlegroups.com>
<memo.20210906214600.9608A@jgd.cix.co.uk>
<4e9b1d6c-5ccc-40d6-b5d1-13f47cddfbaen@googlegroups.com>
Injection-Date: Tue, 7 Sep 2021 10:08:36 -0000 (UTC)
Injection-Info: newsreader4.netcologne.de; posting-host="2001-4dd7-c002-0-7285-c2ff-fe6c-992d.ipv6dyn.netcologne.de:2001:4dd7:c002:0:7285:c2ff:fe6c:992d";
logging-data="10974"; mail-complaints-to="abuse@netcologne.de"
User-Agent: slrn/1.0.3 (Linux)
 by: Thomas Koenig - Tue, 7 Sep 2021 10:08 UTC

Al Grant <algrant109@gmail.com> schrieb:

> I wrote a lot of 370 assembler in the 90s and read a lot
> more. Code looked pretty much like it did in the 1960s.
> Looking at the output of GCC for z/Series recently, I barely
> recognized it. There are a lot of new instructions - and this
> is the regular userspace ISA (or "problem state"). So the
> userspace ISA has changed far more in its last 30 years
> than in its first 30 years. Something must be using those
> new instructions!

The main change was the switch from 31 to 64 bits, with
new registers, new instructions to access them, new whatnot.

It is not surprising that you recognize few instructions.

Re: Versatile Cache from IBM

<sh82bo$86n$1@dont-email.me>

 copy mid

https://www.novabbs.com/devel/article-flat.php?id=20324&group=comp.arch#20324

 copy link   Newsgroups: comp.arch
Path: i2pn2.org!i2pn.org!eternal-september.org!reader02.eternal-september.org!.POSTED!not-for-mail
From: sfu...@alumni.cmu.edu.invalid (Stephen Fuld)
Newsgroups: comp.arch
Subject: Re: Versatile Cache from IBM
Date: Tue, 7 Sep 2021 09:00:53 -0700
Organization: A noiseless patient Spider
Lines: 37
Message-ID: <sh82bo$86n$1@dont-email.me>
References: <sh63lq$v79$1@dont-email.me>
<memo.20210907094936.9608B@jgd.cix.co.uk>
Mime-Version: 1.0
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Transfer-Encoding: 7bit
Injection-Date: Tue, 7 Sep 2021 16:00:56 -0000 (UTC)
Injection-Info: reader02.eternal-september.org; posting-host="7cc6d863c83299a204796e743ef84ac0";
logging-data="8407"; mail-complaints-to="abuse@eternal-september.org"; posting-account="U2FsdGVkX19Uq+dkIvnSRF5Jo7ml0G7QbsJVnDNKFw0="
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
Thunderbird/78.13.0
Cancel-Lock: sha1:qZ//qQ3/U9sPMLMG/WQgM5vIaj0=
In-Reply-To: <memo.20210907094936.9608B@jgd.cix.co.uk>
Content-Language: en-US
 by: Stephen Fuld - Tue, 7 Sep 2021 16:00 UTC

On 9/7/2021 1:48 AM, John Dallman wrote:
> In article <sh63lq$v79$1@dont-email.me>, sfuld@alumni.cmu.edu.invalid
> (Stephen Fuld) wrote:
>
>> You are not being exactly fair. First of all, what you see in the
>> POO is a lot of the "cruft" remaining from 60 years of development
>> history, always maintaining backward compatibility. Second, you
>> are comparing hardware as shown in the manual versus software
>> (Linux).
>
> Indeed. Part of an operating system's job is to hide hardware complexity
> from the application programmer. However, since the traditional OSes for
> Z architecture have grown with the hardware design, they seem to conceal
> rather less than Linux.

That's true, but I think the big difference is that Linux, indeed Unix,
was from the start written in a higher level language, i.e. not
assembler. Thus, while it was not the initial intent, it was fairly
easy to port to new architectures. The IBM OSs, in contrast, were
written entirely in assembler, so were, pretty much by definition,
hardware specific. By the time most of them had been rewritten in a
higher level language (presumably to aid development and maintenance) it
was too late to consider them to be portable.

Thus the Unix specific model of an OS, and Linux in particular, became
the lingua franca of a programming environment. You may not realize how
much the decisions of the original Unix developers pervades what we
think of as an OS, but there were many other decisions that they could
have made that would make things different. That those seem odd to many
people is an indicator of my point.

--
- Stephen Fuld
(e-mail address disguised to prevent spam)

Re: Versatile Cache from IBM

<memo.20210907173548.9608D@jgd.cix.co.uk>

 copy mid

https://www.novabbs.com/devel/article-flat.php?id=20325&group=comp.arch#20325

 copy link   Newsgroups: comp.arch
Path: i2pn2.org!i2pn.org!eternal-september.org!reader02.eternal-september.org!.POSTED!not-for-mail
From: jgd...@cix.co.uk (John Dallman)
Newsgroups: comp.arch
Subject: Re: Versatile Cache from IBM
Date: Tue, 7 Sep 2021 17:35 +0100 (BST)
Organization: A noiseless patient Spider
Lines: 16
Message-ID: <memo.20210907173548.9608D@jgd.cix.co.uk>
References: <sh82bo$86n$1@dont-email.me>
Reply-To: jgd@cix.co.uk
Injection-Info: reader02.eternal-september.org; posting-host="265cb6b7726570e4f2e0a5d7a1ed88a8";
logging-data="24760"; mail-complaints-to="abuse@eternal-september.org"; posting-account="U2FsdGVkX18lmHospxuEyBaovVi34w+eQYTWzJGdDAw="
Cancel-Lock: sha1:cP/A17DUAWFcImL/1az1eo2x+Oc=
 by: John Dallman - Tue, 7 Sep 2021 16:35 UTC

In article <sh82bo$86n$1@dont-email.me>, sfuld@alumni.cmu.edu.invalid
(Stephen Fuld) wrote:

> Thus the Unix specific model of an OS, and Linux in particular,
> became the lingua franca of a programming environment. You may not
> realize how much the decisions of the original Unix developers
> pervades what we think of as an OS, but there were many other
> decisions that they could have made that would make things
> different.

I've done enough VMS, MacOS Classic and CP/M to have some idea. I've also
seen the Unix ideas pushed too far, in HeliOS, which really wasn't
practical. The Unix abstractions are not perfect, but they are pretty
good.

John

Re: Versatile Cache from IBM

<sh85mn$23f$1@dont-email.me>

 copy mid

https://www.novabbs.com/devel/article-flat.php?id=20328&group=comp.arch#20328

 copy link   Newsgroups: comp.arch
Path: i2pn2.org!i2pn.org!eternal-september.org!reader02.eternal-september.org!.POSTED!not-for-mail
From: sfu...@alumni.cmu.edu.invalid (Stephen Fuld)
Newsgroups: comp.arch
Subject: Re: Versatile Cache from IBM
Date: Tue, 7 Sep 2021 09:57:59 -0700
Organization: A noiseless patient Spider
Lines: 34
Message-ID: <sh85mn$23f$1@dont-email.me>
References: <sh82bo$86n$1@dont-email.me>
<memo.20210907173548.9608D@jgd.cix.co.uk>
Mime-Version: 1.0
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Transfer-Encoding: 7bit
Injection-Date: Tue, 7 Sep 2021 16:58:00 -0000 (UTC)
Injection-Info: reader02.eternal-september.org; posting-host="7cc6d863c83299a204796e743ef84ac0";
logging-data="2159"; mail-complaints-to="abuse@eternal-september.org"; posting-account="U2FsdGVkX1+w2lAWQgrInCgOQxslKsFXUFMOtnzjNIM="
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
Thunderbird/78.13.0
Cancel-Lock: sha1:uRLAIaM54TvEV5YRKQkdSigz4GY=
In-Reply-To: <memo.20210907173548.9608D@jgd.cix.co.uk>
Content-Language: en-US
 by: Stephen Fuld - Tue, 7 Sep 2021 16:57 UTC

On 9/7/2021 9:34 AM, John Dallman wrote:
> In article <sh82bo$86n$1@dont-email.me>, sfuld@alumni.cmu.edu.invalid
> (Stephen Fuld) wrote:
>
>> Thus the Unix specific model of an OS, and Linux in particular,
>> became the lingua franca of a programming environment. You may not
>> realize how much the decisions of the original Unix developers
>> pervades what we think of as an OS, but there were many other
>> decisions that they could have made that would make things
>> different.
>
> I've done enough VMS, MacOS Classic and CP/M to have some idea.

OK, but those are relatively recent compared to mainframe OSs such as
OS/360. The differences are even bigger.

> I've also
> seen the Unix ideas pushed too far, in HeliOS, which really wasn't
> practical.

While I have no knowledge of that, I certainly can believe you.

> The Unix abstractions are not perfect, but they are pretty
> good.

Agreed. I think the biggest omission in original Unix (since addressed)
was native threads. Other people probably have others things.

--
- Stephen Fuld
(e-mail address disguised to prevent spam)

Re: Versatile Cache from IBM

<sh87np$hnr$1@dont-email.me>

 copy mid

https://www.novabbs.com/devel/article-flat.php?id=20329&group=comp.arch#20329

 copy link   Newsgroups: comp.arch
Path: i2pn2.org!i2pn.org!eternal-september.org!reader02.eternal-september.org!.POSTED!not-for-mail
From: sfu...@alumni.cmu.edu.invalid (Stephen Fuld)
Newsgroups: comp.arch
Subject: Re: Versatile Cache from IBM
Date: Tue, 7 Sep 2021 10:32:39 -0700
Organization: A noiseless patient Spider
Lines: 49
Message-ID: <sh87np$hnr$1@dont-email.me>
References: <70130c3b-ecb9-4098-86be-7fc7e5506e3bn@googlegroups.com>
<2021Sep6.152856@mips.complang.tuwien.ac.at>
Mime-Version: 1.0
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Transfer-Encoding: 7bit
Injection-Date: Tue, 7 Sep 2021 17:32:42 -0000 (UTC)
Injection-Info: reader02.eternal-september.org; posting-host="7cc6d863c83299a204796e743ef84ac0";
logging-data="18171"; mail-complaints-to="abuse@eternal-september.org"; posting-account="U2FsdGVkX19yxeLiOcMgqOUnkTklgnNWaIe0e7dHrF8="
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
Thunderbird/78.13.0
Cancel-Lock: sha1:a7JR0afjvviIqVf+tsGOJKXELik=
In-Reply-To: <2021Sep6.152856@mips.complang.tuwien.ac.at>
Content-Language: en-US
 by: Stephen Fuld - Tue, 7 Sep 2021 17:32 UTC

On 9/6/2021 6:28 AM, Anton Ertl wrote:

snip

> Unfortunately, both the Hot Chips slides and the article (that is
> supposed to be written after additional Q&A with the IBM guys;
> apparently the IBM guys were not allowed to reveal anything more) miss
> pretty much all the interesting details. Basically, the only thing we
> learn ist that other CPUs L2 caches can be used as victim caches.

IBM is masterful at giving you lots of information, giving the
impression that they are very open, but not actually giving you the
details you want. They have been doing that for decades! :-(

> Questions that come to my mind:
>
> How does a line become "unused" and available for other CPUs? EricP
> suggests that cache lines on idle cores are unused, but a currently
> idle core could continue on the same task the very next microsecond,
> so that does not sound like a good strategy. Cache lines belonging to
> no-longer-allocated ASIDs might be a better approach; it probably
> requires some scrubbing (probably in hardware), though. Or maybe a
> long-unaccessed cache line that is still accessible could also be
> categorized as "unused".
>
> Does this system employ a directory or snooping. With hundreds of L2
> caches, snooping seems inefficient, but what do I know. A directory
> might be based on the memory controller for the physical RAM that
> holds the memory if not in cache; i.e., send the request to the memory
> controller, and it will send it on to RAM or the appropriate L2 cache
> based on the directory.

You and I agree that we don't know. However, perhaps we can take some
hints from the following quotation. From the article

> In the Q&A following the session, Dr. Christian Jacobi (Chief Architect of Z) said that the system is designed to keep track of data on a cache miss, uses broadcasts, and memory state bits are tracked for broadcasts to external chips. These go across the whole system, and when data arrives it makes sure it can be used and confirms that all other copies are invalidated before working on the data. In the slack channel as part of the event, he also stated that lots of cycle counting goes on!

So perhaps they are using cycle counts since last reference as an
indicator of "oldness"? They broadcast the cycle count of the to be
evicted line to see if any other cache has an older one that is
preferable to evict? Of course, this is pure speculation. :-(

--
- Stephen Fuld
(e-mail address disguised to prevent spam)

1
server_pubkey.txt

rocksolid light 0.9.7
clearnet tor