EricP <ThatWouldBeTelling@thevillage.com> writes:
[z15]
>This 6 wide zillion stage pipeline (which hence forth shall be called
>a super-duper pipeline) makes sense if they HW multi-thread's the uOps
>so that there are 6 uOps from 1 thread or 1 uOp from 6 threads,
>or whatever in between.
>
>So yes, a particular HW thread takes a branch miss and stalls,
>but 5 others are stuffing uOps into the pipeline stages
>so it really never drains.

I expect that even with a single thread, the branch predictor does not
let the pipeline drain. As soon as a misprediction is found the
correct target address is fed into the first stage of the pipeline for
fetching from there. Of course there will be a lot of stuff in flight
that is then on the mispredicted path, but I expect that these
instructions will run through the pipeline until retirement and be
thrown away there, instead of canceling them wherever they are.

As for SMT, each thread will have a shorter predicted path, and
therefore less to throw away when a misprediction appears; also,
independent threads provide more independent instructions, increasing
utilization. Well, at least that's the theory. What I have seen on
Intel Skylake and AMD Zen has been disappointing.

- anton
--
'Anyone trying for "industrial quality" ISA should avoid undefined behavior.'
Mitch Alsup, <c17fcd89-f024-40e7-a594-88a85ac10d20o@googlegroups.com>

On Friday, May 28, 2021 at 3:45:20 AM UTC-5, Anton Ertl wrote:
> EricP <ThatWould...@thevillage.com> writes:
> [z15]
> >This 6 wide zillion stage pipeline (which hence forth shall be called
> >a super-duper pipeline) makes sense if they HW multi-thread's the uOps
> >so that there are 6 uOps from 1 thread or 1 uOp from 6 threads,
> >or whatever in between.
> >
> >So yes, a particular HW thread takes a branch miss and stalls,
> >but 5 others are stuffing uOps into the pipeline stages
> >so it really never drains.
<
> I expect that even with a single thread, the branch predictor does not
> let the pipeline drain. As soon as a misprediction is found the
> correct target address is fed into the first stage of the pipeline for
> fetching from there. Of course there will be a lot of stuff in flight
> that is then on the mispredicted path, but I expect that these
> instructions will run through the pipeline until retirement and be
> thrown away there, instead of canceling them wherever they are.
<
There is a 6 stage branch predictor and there is a 2 stage branch predictor
(the 2 stage predictor is new compared to Z14).
<
But also note it takes another 15 stage pipeline to perform retirement !!
{15 to get instructions to decode, 15 to retire instructions, and at least
a 15 stage pipeline to perform calculations and Dcache access} They
fetch an entire cache line in a single access and the branch predictor
has to predict at least 5 branches per fetch.
<
On the other hand there are dedicated units to perform IBM floating point,
IEEE floating point, Decimal Floating Point, Decimal Arithmetic,.....
<
On final note, the whole thing is water cooled and has 17 layers of metal
and is 24mm^2. The water cooling is probably allowing for 25% frequency
gain (4GHz->5GHz).
<
So from all of the above, and not mentioned anywhere in the IBM literature,
one can conclude that the thing is indeed SMT, nothing else makes sense.
Maybe not Niagra-like SMT, but a significant number of threads are in
progress at any clock tick of the machine.
>
> As for SMT, each thread will have a shorter predicted path, and
> therefore less to throw away when a misprediction appears; also,
> independent threads provide more independent instructions, increasing
> utilization. Well, at least that's the theory. What I have seen on
> Intel Skylake and AMD Zen has been disappointing.
<
Yes, this is the throughput side of their (IBM) argument.
<
I find it interesting that one can buy a machine with 240 cores, yet applications
only have access to 190 of these, leaving 30 cores to perform I/O !!! and
other system "things".
<
> - anton
> --
> 'Anyone trying for "industrial quality" ISA should avoid undefined behavior.'
> Mitch Alsup, <c17fcd89-f024-40e7...@googlegroups.com>

MitchAlsup <MitchAlsup@aol.com> schrieb:

> So from all of the above, and not mentioned anywhere in the IBM literature,
> one can conclude that the thing is indeed SMT, nothing else makes sense.
> Maybe not Niagra-like SMT, but a significant number of threads are in
> progress at any clock tick of the machine.

Their Red Book, IBM z15 (8561) Technical Guide, says

# The Integrated Facility for Linux (IFL) and IBM Z Integrated
# Information Processor (zIIP) processor units on the z15 server can
# be configured to run two simultaneous threads per clock cycle in
# a single processor (SMT). This feature increases the capacity of
# these processors with 25% in average over processors that are
# running single thread.

Two processes per clock cycle?

> But also note it takes another 15 stage pipeline to perform retirement !!
> {15 to get instructions to decode, 15 to retire instructions, and at least
> a 15 stage pipeline to perform calculations and Dcache access} They
> fetch an entire cache line in a single access and the branch predictor
> has to predict at least 5 branches per fetch.
[...]
> I find it interesting that one can buy a machine with 240 cores, yet applications
> only have access to 190 of these, leaving 30 cores to perform I/O !!! and
> other system "things".

Hmm... any chance these "240 cores" are really 16 cores with 15 threads
each, or something like that? The available literature suggests it's
not the case, but the numbers seem to hint in that direction ;-)

Stefan

In article <jwveedpczhj.fsf-monnier+comp.arch@gnu.org>,
monnier@iro.umontreal.ca (Stefan Monnier) wrote:

> Hmm... any chance these "240 cores" are really 16 cores with 15
> threads each, or something like that? The available literature
> suggests it's not the case, but the numbers seem to hint in that
> direction ;-)

Not plausibly. The machines are very modular, built up from "Single-Chip
Modules" ("SCM") which hold variable numbers of "Processor Units" ("PU").
It's possible to upgrade the machines very incrementally, and the
customers will be very aware of the cooling requirements.

https://www.redbooks.ibm.com/abstracts/sg248850.html

John

According to Stefan Monnier <monnier@iro.umontreal.ca>:
>> But also note it takes another 15 stage pipeline to perform retirement !!
>> {15 to get instructions to decode, 15 to retire instructions, and at least
>> a 15 stage pipeline to perform calculations and Dcache access} They
>> fetch an entire cache line in a single access and the branch predictor
>> has to predict at least 5 branches per fetch.
>[...]
>> I find it interesting that one can buy a machine with 240 cores, yet applications
>> only have access to 190 of these, leaving 30 cores to perform I/O !!! and
>> other system "things".

Some of them are hot spares. Z series can switch cores on the fly if one of them fails.
They really really do not want their systems to crash.

--
Regards,
John Levine, johnl@taugh.com, Primary Perpetrator of "The Internet for Dummies",
Please consider the environment before reading this e-mail. https://jl.ly

Stefan Monnier <monnier@iro.umontreal.ca> writes:
>> I find it interesting that one can buy a machine with 240 cores, yet applications
>> only have access to 190 of these, leaving 30 cores to perform I/O !!! and
>> other system "things".

240-190=50 but see below.

>Hmm... any chance these "240 cores" are really 16 cores with 15 threads
>each, or something like that?

No. Each CPU has 12 cores, and the max190 has 5 drawers with 43 PUs
(cores) per drawer (that would be 215 PUs, so 25 PUs are for internal
things); I guess they have 4 CPUs per drawer, and 1-2 cores per CPU
are disabled for yield reasons. Each core supports 2-way SMT.

Sources:
https://en.wikipedia.org/wiki/IBM_z15_%28microprocessor%29
https://www.redbooks.ibm.com/redbooks/pdfs/sg248851.pdf
https://www.ibm.com/downloads/cas/NN7GBPJ1

- anton
--
'Anyone trying for "industrial quality" ISA should avoid undefined behavior.'
Mitch Alsup, <c17fcd89-f024-40e7-a594-88a85ac10d20o@googlegroups.com>

MitchAlsup wrote:
> On Friday, May 28, 2021 at 3:45:20 AM UTC-5, Anton Ertl wrote:
>> EricP <ThatWould...@thevillage.com> writes:
>> [z15]
>>> This 6 wide zillion stage pipeline (which hence forth shall be called
>>> a super-duper pipeline) makes sense if they HW multi-thread's the uOps
>>> so that there are 6 uOps from 1 thread or 1 uOp from 6 threads,
>>> or whatever in between.
>>>
>>> So yes, a particular HW thread takes a branch miss and stalls,
>>> but 5 others are stuffing uOps into the pipeline stages
>>> so it really never drains.
> <
>> I expect that even with a single thread, the branch predictor does not
>> let the pipeline drain. As soon as a misprediction is found the
>> correct target address is fed into the first stage of the pipeline for
>> fetching from there. Of course there will be a lot of stuff in flight
>> that is then on the mispredicted path, but I expect that these
>> instructions will run through the pipeline until retirement and be
>> thrown away there, instead of canceling them wherever they are.
> <
> There is a 6 stage branch predictor and there is a 2 stage branch predictor
> (the 2 stage predictor is new compared to Z14).
> <
> But also note it takes another 15 stage pipeline to perform retirement !!
> {15 to get instructions to decode, 15 to retire instructions, and at least
> a 15 stage pipeline to perform calculations and Dcache access} They
> fetch an entire cache line in a single access and the branch predictor
> has to predict at least 5 branches per fetch.
> <
> On the other hand there are dedicated units to perform IBM floating point,
> IEEE floating point, Decimal Floating Point, Decimal Arithmetic,.....
> <
> On final note, the whole thing is water cooled and has 17 layers of metal
> and is 24mm^2. The water cooling is probably allowing for 25% frequency
> gain (4GHz->5GHz).
> <
> So from all of the above, and not mentioned anywhere in the IBM literature,
> one can conclude that the thing is indeed SMT, nothing else makes sense.
> Maybe not Niagra-like SMT, but a significant number of threads are in
> progress at any clock tick of the machine.
>>
>> As for SMT, each thread will have a shorter predicted path, and
>> therefore less to throw away when a misprediction appears; also,
>> independent threads provide more independent instructions, increasing
>> utilization. Well, at least that's the theory. What I have seen on
>> Intel Skylake and AMD Zen has been disappointing.
> <
> Yes, this is the throughput side of their (IBM) argument.
> <
> I find it interesting that one can buy a machine with 240 cores, yet applications
> only have access to 190 of these, leaving 30 cores to perform I/O !!! and
> other system "things".

240 - 190 = 50 "system reserved" cores?

Terje

--
- <Terje.Mathisen at tmsw.no>
"almost all programming can be viewed as an exercise in caching"

MitchAlsup wrote:
> On Friday, May 28, 2021 at 3:45:20 AM UTC-5, Anton Ertl wrote:
>> EricP <ThatWould...@thevillage.com> writes:
>> [z15]
>>> This 6 wide zillion stage pipeline (which hence forth shall be called
>>> a super-duper pipeline) makes sense if they HW multi-thread's the uOps
>>> so that there are 6 uOps from 1 thread or 1 uOp from 6 threads,
>>> or whatever in between.
>>>
>>> So yes, a particular HW thread takes a branch miss and stalls,
>>> but 5 others are stuffing uOps into the pipeline stages
>>> so it really never drains.
> <
>> I expect that even with a single thread, the branch predictor does not
>> let the pipeline drain. As soon as a misprediction is found the
>> correct target address is fed into the first stage of the pipeline for
>> fetching from there. Of course there will be a lot of stuff in flight
>> that is then on the mispredicted path, but I expect that these
>> instructions will run through the pipeline until retirement and be
>> thrown away there, instead of canceling them wherever they are.
> <
> There is a 6 stage branch predictor and there is a 2 stage branch predictor
> (the 2 stage predictor is new compared to Z14).
> <
> But also note it takes another 15 stage pipeline to perform retirement !!
> {15 to get instructions to decode, 15 to retire instructions, and at least
> a 15 stage pipeline to perform calculations and Dcache access} They
> fetch an entire cache line in a single access and the branch predictor
> has to predict at least 5 branches per fetch.

That paper on the z15 branch predictor says that the minimum delay
for a branch mispredict is 26 cycles, and the statistical penalty
is 35 cycles because of other queue disruptions.
An I$L1 miss with L2 hit is a minimum of 8 extra clocks.

So a fair portion of that pipeline drains on a branch mispredict.

> <
> On the other hand there are dedicated units to perform IBM floating point,
> IEEE floating point, Decimal Floating Point, Decimal Arithmetic,.....
> <
> On final note, the whole thing is water cooled and has 17 layers of metal
> and is 24mm^2. The water cooling is probably allowing for 25% frequency
> gain (4GHz->5GHz).
> <
> So from all of the above, and not mentioned anywhere in the IBM literature,
> one can conclude that the thing is indeed SMT, nothing else makes sense.
> Maybe not Niagra-like SMT, but a significant number of threads are in
> progress at any clock tick of the machine.

The branch paper says it is SMT and refers to "SMT2" mode
and the text implies there are 2 HW threads.

If a single branch mispredict can drain 35 clocks*6 = 210 instructions,
and they say instructions average 5 bytes each so thats 1kB lost,
I would have expected more than 1 alternate thread fetch buffer would be
required to keep the pipeline busy providing an alternate 1kB instructions.

>> As for SMT, each thread will have a shorter predicted path, and
>> therefore less to throw away when a misprediction appears; also,
>> independent threads provide more independent instructions, increasing
>> utilization. Well, at least that's the theory. What I have seen on
>> Intel Skylake and AMD Zen has been disappointing.
> <
> Yes, this is the throughput side of their (IBM) argument.
> <
> I find it interesting that one can buy a machine with 240 cores, yet applications
> only have access to 190 of these, leaving 30 cores to perform I/O !!! and
> other system "things".
> <
>> - anton
>> --
>> 'Anyone trying for "industrial quality" ISA should avoid undefined behavior.'
>> Mitch Alsup, <c17fcd89-f024-40e7...@googlegroups.com>