With their new z16 series and the Telum processor, IBM has put
a lot of half precision fmas into their mainframes - apparently,
they want to do AI inference in realtime on their new mainframes
(credit card fraud is cited).

Some information is at
https://www.hpcwire.com/2021/08/23/ibms-upcoming-z-series-chip-gains-on-chip-ai-acceleration-and-new-name-telum/

According to one slide, they have a new "Neural Network Processing
Assist" instruction, which is a memory-to-memory CISC instruction
doing matrix multiplication, convolution and activation functions
They also prefetch data into the L2 cache.

This certainly solves the "graphics card is too far from the CPU"
problem.

They have special fma nodes (128 processor tiles with 8-way
FP-16 SIMD) and 32 nodes specialized for different activation
functions, either in 16 or 32-bit floating point.

Interesting.

It also shows that half precision is moving towards the mainstream,
or should I say mainframe?

On 4/16/2022 1:26 AM, Thomas Koenig wrote:
> With their new z16 series and the Telum processor, IBM has put
> a lot of half precision fmas into their mainframes - apparently,
> they want to do AI inference in realtime on their new mainframes
> (credit card fraud is cited).
>
> Some information is at
> https://www.hpcwire.com/2021/08/23/ibms-upcoming-z-series-chip-gains-on-chip-ai-acceleration-and-new-name-telum/
>
> According to one slide, they have a new "Neural Network Processing
> Assist" instruction, which is a memory-to-memory CISC instruction
> doing matrix multiplication, convolution and activation functions
> They also prefetch data into the L2 cache.
>
> This certainly solves the "graphics card is too far from the CPU"
> problem.
>
> They have special fma nodes (128 processor tiles with 8-way
> FP-16 SIMD) and 32 nodes specialized for different activation
> functions, either in 16 or 32-bit floating point.
>
> Interesting.
>
> It also shows that half precision is moving towards the mainstream,
> or should I say mainframe?

A little more information from IBM's Hot Chips presentation

https://hc33.hotchips.org/assets/program/conference/day1/HC2021.C1.3%20IBM%20Cristian%20Jacobi%20Final.pdf

--
- Stephen Fuld
(e-mail address disguised to prevent spam)

Stephen Fuld <sfuld@alumni.cmu.edu.invalid> schrieb:
> On 4/16/2022 1:26 AM, Thomas Koenig wrote:
>> With their new z16 series and the Telum processor, IBM has put
>> a lot of half precision fmas into their mainframes - apparently,
>> they want to do AI inference in realtime on their new mainframes
>> (credit card fraud is cited).
>>
>> Some information is at
>> https://www.hpcwire.com/2021/08/23/ibms-upcoming-z-series-chip-gains-on-chip-ai-acceleration-and-new-name-telum/
>>
>> According to one slide, they have a new "Neural Network Processing
>> Assist" instruction, which is a memory-to-memory CISC instruction
>> doing matrix multiplication, convolution and activation functions
>> They also prefetch data into the L2 cache.
>>
>> This certainly solves the "graphics card is too far from the CPU"
>> problem.
>>
>> They have special fma nodes (128 processor tiles with 8-way
>> FP-16 SIMD) and 32 nodes specialized for different activation
>> functions, either in 16 or 32-bit floating point.
>>
>> Interesting.
>>
>> It also shows that half precision is moving towards the mainstream,
>> or should I say mainframe?
>
> A little more information from IBM's Hot Chips presentation
>
> https://hc33.hotchips.org/assets/program/conference/day1/HC2021.C1.3%20IBM%20Cristian%20Jacobi%20Final.pdf

Ah, that was the presentation I was missing.

In the talk itself, there's a little bit more info:
https://www.youtube.com/watch?v=fUqOdu2ympk starting at 17:10.