Came across this item:

https://www.nextplatform.com/2021/07/12/gutting-decades-of-architecture-to-build-a-new-kind-of-processor/

about an attempt to design a processor that does away with instruction
sets and all those instructions that just move data around.

John Savard

On Monday, July 12, 2021 at 4:17:29 PM UTC-5, Quadibloc wrote:
> Came across this item:
>
> https://www.nextplatform.com/2021/07/12/gutting-decades-of-architecture-to-build-a-new-kind-of-processor/
>
> about an attempt to design a processor that does away with instruction
> sets and all those instructions that just move data around.
>
> John Savard
<
Sounds like a cross between the Transputer and an systolic array.

On Monday, July 12, 2021 at 10:17:29 PM UTC+1, Quadibloc wrote:

> about an attempt to design a processor that does away with instruction
> sets and all those instructions that just move data around.

reminds me of Elixent's stuff. they did NSEW neighbour processing.
fascinating design: 4-bit ALUs. tens of thousands of them. programming
this style of processor is an absolute pig.

l.

In article <9b868d91-8bb9-41b8-ba0d-a02aceeb888dn@googlegroups.com>,
luke.leighton@gmail.com () wrote:

> reminds me of Elixent's stuff. they did NSEW neighbour processing.
> fascinating design: 4-bit ALUs. tens of thousands of them.
> programming this style of processor is an absolute pig.

I once had to respond to a pitch from a company who wanted to build big
grids of 8-bit processors along similar lines. They claimed to be able to
compile C and C++ code into highly optimised programs for this hardware,
and were trying to get ISVs to commit to supporting it to help their case
for venture capital.

It looked rather hard to debug on, which they admitted. I asked how they
were planning to offer support on bugs in their compiler and the response
"Why would there be any?" doomed the pitch.

John

On 12/07/2021 23:17, Quadibloc wrote:
> Came across this item:
>
> https://www.nextplatform.com/2021/07/12/gutting-decades-of-architecture-to-build-a-new-kind-of-processor/
>
> about an attempt to design a processor that does away with instruction
> sets and all those instructions that just move data around.
>
> John Savard
>

Didn't the Itanium demonstrate that "magic compilers" don't work?
Compilers (and other tools) along with massive arrays of limited cores
are fine for code that has a lot of calculations but little variation
and few conditionals - so they are useful for graphics, AI, physical
simulations, etc. But they fall apart as soon as you are do a few "if"
statements and you get beyond the possibility of running all paths at
once and doing a conditional move at the end.

On 2021-07-13, David Brown wrote:
> On 12/07/2021 23:17, Quadibloc wrote:
>> Came across this item:
>>
>> https://www.nextplatform.com/2021/07/12/gutting-decades-of-architecture-to-build-a-new-kind-of-processor/
>>
>> about an attempt to design a processor that does away with instruction
>> sets and all those instructions that just move data around.
>>
>> John Savard
>>
>
> Didn't the Itanium demonstrate that "magic compilers" don't work?
> Compilers (and other tools) along with massive arrays of limited cores
> are fine for code that has a lot of calculations but little variation
> and few conditionals - so they are useful for graphics, AI, physical
> simulations, etc. But they fall apart as soon as you are do a few "if"
> statements and you get beyond the possibility of running all paths at
> once and doing a conditional move at the end.
>

If at first you don't succeed...?

I'm personally quite skeptical about relying too much on compiler
technology advancements. Itanium proved it's a bad strategy. Auto-
vectorization for SIMD ISA:s proved it's a bad strategy.

OTOH we have the end of Moore's Law around the corner, and we've been
trying to crack the problem of massive parallelism for decades now, so
I really hope that we'll see some (useful) paradigm shifts in the near
future.

To me it seems that it's more of a programming problem than a HW
problem, and it feels like we're using the wrong tools to describe
solutions to our problems. At some point in time serial & branchy
instruction streams derived from traditional programming languages
will be harder and more expensive to construct and run than to simply
throw massively parallel compute at the problem.

I work on a product where we use a mix of hand-made algorithms and
deep learning, so I'm front row witnessing the transition from well
defined logic & algebra to fuzzy neural networks. And how successful
that transition is. And it's frankly quite scary IMO.

Or maybe we should just accept the end of Moore's Law and require
programmers to got back to being great again, like they were a few
decades ago. ;-) I wonder how many modern day programmers would pull
off something like a graphical action/adventure game with a 1 MHz
8-bit CPU and 128 bytes of RAM (e.g. Atari 2600 Pitfall).

On 13/07/2021 13:25, Marcus wrote:
> On 2021-07-13, David Brown wrote:
>> On 12/07/2021 23:17, Quadibloc wrote:
>>> Came across this item:
>>>
>>> https://www.nextplatform.com/2021/07/12/gutting-decades-of-architecture-to-build-a-new-kind-of-processor/
>>>
>>>
>>> about an attempt to design a processor that does away with instruction
>>> sets and all those instructions that just move data around.
>>>
>>> John Savard
>>>
>>
>> Didn't the Itanium demonstrate that "magic compilers" don't work?
>> Compilers (and other tools) along with massive arrays of limited cores
>> are fine for code that has a lot of calculations but little variation
>> and few conditionals - so they are useful for graphics, AI, physical
>> simulations, etc.  But they fall apart as soon as you are do a few "if"
>> statements and you get beyond the possibility of running all paths at
>> once and doing a conditional move at the end.
>>
>
> If at first you don't succeed...?
>
> I'm personally quite skeptical about relying too much on compiler
> technology advancements. Itanium proved it's a bad strategy. Auto-
> vectorization for SIMD ISA:s proved it's a bad strategy.
>
> OTOH we have the end of Moore's Law around the corner, and we've been
> trying to crack the problem of massive parallelism for decades now, so
> I really hope that we'll see some (useful) paradigm shifts in the near
> future.
>
> To me it seems that it's more of a programming problem than a HW
> problem, and it feels like we're using the wrong tools to describe
> solutions to our problems. At some point in time serial & branchy
> instruction streams derived from traditional programming languages
> will be harder and more expensive to construct and run than to simply
> throw massively parallel compute at the problem.
>
> I work on a product where we use a mix of hand-made algorithms and
> deep learning, so I'm front row witnessing the transition from well
> defined logic & algebra to fuzzy neural networks. And how successful
> that transition is. And it's frankly quite scary IMO.
>
> Or maybe we should just accept the end of Moore's Law and require
> programmers to got back to being great again, like they were a few
> decades ago. ;-) I wonder how many modern day programmers would pull
> off something like a graphical action/adventure game with a 1 MHz
> 8-bit CPU and 128 bytes of RAM (e.g. Atari 2600 Pitfall).

What seems to be missing here (at least, /I/ have missed it) is a
discusion about what people actually want to do with computing power.
Is it fair to suggest that most tasks that people currently want to run
faster, are actually reasonably parallel? And that most tasks that
people want to run a /lot/ faster are /very/ parallel?

For the majority of users, about the only thing that needs to go faster
than today's machines is games. Moore's Law is not a problem for them -
graphics cards that do more in parallel make the games faster. Some
aspects are still cpu-bound, but often multiple cpu cores will scale as
well as single-threaded performance. For more professional work, 3D
CAD, software builds, modelling work, simulations, etc., - much of it
can use multiple cpu cores just as much as fast single cores.

On servers, it's usually lots of semi-independent tasks that can be
spread amongst many cores. For HPC, massively parallel is the norm.

And for the current fad of AI, it's inherently very parallel.

So I agree with you that it is a primarily a programming problem, and
the answer lies in better tools, training, languages, etc., aimed at
parallelisation of code.

Where hardware can help is to make multi-core processors that have
features to aid parallel coding and synchronisation. Instead of the
absurdly inefficient and complicated memory models, barriers, fences,
software synchronisation, and overly general bus locks, cache snooping
and the rest of it, we need hardware that supports key OS features.
Cores should know the id of the thread they are running. Processors
should have a shared block of locks, immediately accessible by all cores
rather than putting locks in main memory - so taking a lock will be a
few cycles if it is uncontested. Processors should support mailboxes
for rapid messaging. It is time for processors to support
multi-threading OS's, instead of the OS having to deal with awkward
processors.

David Brown <david.brown@hesbynett.no> wrote:
> What seems to be missing here (at least, /I/ have missed it) is a
> discusion about what people actually want to do with computing power.
> Is it fair to suggest that most tasks that people currently want to run
> faster, are actually reasonably parallel? And that most tasks that
> people want to run a /lot/ faster are /very/ parallel?

I think the question to ask is: what tasks are we /not/ doing, because we
don't have the computing power to handle them?

Artificial neural networks were invented a long time ago, but were not
feasible to deploy because they needed what was, at the time, infeasible
amounts of computing power. Now the computing power has caught up and
they're feasible - and so we're using them as a hammer to hit every problem.
The hardware is hugely worse in size and power efficiency than the neural
network between our ears, but we pay that cost.

So what fields have dismissed problems because there is at present
insufficient computing resource?

I don't buy the 'throw infinite parallelism at it' argument BTW - even if
you build such a processor you still have to feed it from a memory that
represents the shared state of the system (and manage its coherence).
Again, we can cream off those problems with a limited amount of shared state
that can be parcelled up to individual cores or computers that interact
minimally (the classic scale out) and those are the problems that have seen
the most attention. But I wonder about the problems nobody's trying because
they're too hard to do in that way.

Theo

On Mon, 12 Jul 2021 15:28:23 -0700 (PDT), "luke.l...@gmail.com"
<luke.leighton@gmail.com> wrote:

>On Monday, July 12, 2021 at 10:17:29 PM UTC+1, Quadibloc wrote:
>
>> about an attempt to design a processor that does away with instruction
>> sets and all those instructions that just move data around.
>
>reminds me of Elixent's stuff. they did NSEW neighbour processing.
>fascinating design: 4-bit ALUs. tens of thousands of them. programming
>this style of processor is an absolute pig.
>
>l.

Dunno. I programmed Connection Machines. Admittedly a problem had to
be embarrassingly parallel to fit with the hardware ... but given
that, the CM was quite easy to work with.

George

On Tuesday, July 13, 2021 at 6:25:54 AM UTC-5, Marcus wrote:
> On 2021-07-13, David Brown wrote:
> > On 12/07/2021 23:17, Quadibloc wrote:
> >> Came across this item:
> >>
> >> https://www.nextplatform.com/2021/07/12/gutting-decades-of-architecture-to-build-a-new-kind-of-processor/
> >>
> >> about an attempt to design a processor that does away with instruction
> >> sets and all those instructions that just move data around.
> >>
> >> John Savard
> >>
> >
> > Didn't the Itanium demonstrate that "magic compilers" don't work?
> > Compilers (and other tools) along with massive arrays of limited cores
> > are fine for code that has a lot of calculations but little variation
> > and few conditionals - so they are useful for graphics, AI, physical
> > simulations, etc. But they fall apart as soon as you are do a few "if"
> > statements and you get beyond the possibility of running all paths at
> > once and doing a conditional move at the end.
> >
> If at first you don't succeed...?
>
> I'm personally quite skeptical about relying too much on compiler
> technology advancements. Itanium proved it's a bad strategy. Auto-
> vectorization for SIMD ISA:s proved it's a bad strategy.
>
> OTOH we have the end of Moore's Law around the corner, and we've been
> trying to crack the problem of massive parallelism for decades now, so
> I really hope that we'll see some (useful) paradigm shifts in the near
> future.
>
> To me it seems that it's more of a programming problem than a HW
> problem, and it feels like we're using the wrong tools to describe
> solutions to our problems.
<
A) Bingo:: this has proven to be a SW problem
B) we do not yet have a "Touring" model of interacting threads
C) One big problem is that HW does not provide proper/adequate primitives
(instructions or streams of instructions) that deliver the kind of ATOMICITY
SW requires.
<
As to C:: My 66000 (and apparently MILL) provide multiple location ATOMICs
With significantly better semantics than DCAS or LL/SC........
<
> At some point in time serial & branchy
> instruction streams derived from traditional programming languages
> will be harder and more expensive to construct and run than to simply
> throw massively parallel compute at the problem.
>
> I work on a product where we use a mix of hand-made algorithms and
> deep learning, so I'm front row witnessing the transition from well
> defined logic & algebra to fuzzy neural networks. And how successful
> that transition is. And it's frankly quite scary IMO.
>
> Or maybe we should just accept the end of Moore's Law and require
> programmers to got back to being great again, like they were a few
> decades ago. ;-) I wonder how many modern day programmers would pull
> off something like a graphical action/adventure game with a 1 MHz
> 8-bit CPU and 128 bytes of RAM (e.g. Atari 2600 Pitfall).
<
1.238% of them.

MitchAlsup wrote:
> On Monday, July 12, 2021 at 4:17:29 PM UTC-5, Quadibloc wrote:
>> Came across this item:
>>
>> https://www.nextplatform.com/2021/07/12/gutting-decades-of-architecture-to-build-a-new-kind-of-processor/
>>
>> about an attempt to design a processor that does away with instruction
>> sets and all those instructions that just move data around.
>>
>> John Savard
> <
> Sounds like a cross between the Transputer and an systolic array.
>
The fun part for me is that the founders are all Norwegian (until they
brought in Peter Toley), which is interesting...

Terje

--
- <Terje.Mathisen at tmsw.no>
"almost all programming can be viewed as an exercise in caching"

According to Marcus <m.delete@this.bitsnbites.eu>:
>I'm personally quite skeptical about relying too much on compiler
>technology advancements. Itanium proved it's a bad strategy. Auto-
>vectorization for SIMD ISA:s proved it's a bad strategy.

I dunno, the IBM 801 and PL.8 proved it is a good strategy, at least if your name is John Cocke or Fram Allen.

Perhaps the lesson is to be realistic in your plan for how much hardware ugliness you can paper over with
compiler cleverness, and to remeber that the tradeoffs change over time.

Itanium was based on work at Multiflow in the 1980s, when the amount of stuff you could do in hardware was a lot
less than it was a decade or two later. The compiler did what it could to schedule memory references statically,
but once you could do that in hardware, dynamic hardware scheduling worked a lot better.

--
Regards,
John Levine, johnl@taugh.com, Primary Perpetrator of "The Internet for Dummies",
Please consider the environment before reading this e-mail. https://jl.ly

On Tuesday, July 13, 2021 at 6:13:13 AM UTC-6, David Brown wrote:

> What seems to be missing here (at least, /I/ have missed it) is a
> discusion about what people actually want to do with computing power.
> Is it fair to suggest that most tasks that people currently want to run
> faster, are actually reasonably parallel? And that most tasks that
> people want to run a /lot/ faster are /very/ parallel?

Unfortunately, no, it isn't.

Many of the tasks that people want to run faster are parallel, but some of
them are not.

Also, it's at least possible that *some* of the tasks people want to run
faster... are perhaps more parallel than people realize, and some new
programming paradigm might enable this parallelism to be brought to
light. At least that's the hope that fuels attempts to design compilers
that will bring out extra parallelism that's not obvious to human programmers.

John Savard

Quadibloc <jsavard@ecn.ab.ca> wrote:
> Came across this item:
>
> https://www.nextplatform.com/2021/07/12/gutting-decades-of-architecture-to-build-a-new-kind-of-processor/
>
> about an attempt to design a processor that does away with instruction
> sets and all those instructions that just move data around.

I'm curious about this:

"We can run 700,000 lines of code, which includes standard C libraries used
in SPEC, and we compile that and run that on our FPGA testbed, which is not
the full architecture, but a big chunk of it, and get functionally correct
results. We have a full symbolic debugger and other infrastructure to
actually make something like that work."

I wonder what's in those 700KLOC. Because frequently existing codebases are
very CPU-oriented. For example, lots of control flow, doing memory
allocations all over the place because that's what CPUs do, but things that
don't lend themselves to parallelism. And then those codebases expect to do
things like fopen() and printf() and mmap() and other things which don't sit
very well in a data-oriented machine. So I'm curious as to what OS support
they have, or whether they're mining some codebase that intentionally
doesn't have all the pesky I/O and control flow that today's real software
does.

Theo

According to Theo Markettos <theom+news@chiark.greenend.org.uk>:
>Quadibloc <jsavard@ecn.ab.ca> wrote:
>> https://www.nextplatform.com/2021/07/12/gutting-decades-of-architecture-to-build-a-new-kind-of-processor/
>>
>> about an attempt to design a processor that does away with instruction
>> sets and all those instructions that just move data around.
>
>I'm curious about this:
>
>"We can run 700,000 lines of code, which includes standard C libraries used
>in SPEC, and we compile that and run that on our FPGA testbed, which is not
>the full architecture, but a big chunk of it, and get functionally correct
>results. We have a full symbolic debugger and other infrastructure to
>actually make something like that work."
>
>I wonder what's in those 700KLOC. Because frequently existing codebases are
>very CPU-oriented.

Standard C libraries are written to be portable and to abstract away the details of
the machine into macros and parameter definitions.

If I were looking for test code for a wacky architecture, it would be a good choice.
One thing that's non-negotiable is that data has to be 8 bit byte addressable. That's
built in too many places.

--
Regards,
John Levine, johnl@taugh.com, Primary Perpetrator of "The Internet for Dummies",
Please consider the environment before reading this e-mail. https://jl.ly

On 07/13/21 22:58, Quadibloc wrote:
> On Tuesday, July 13, 2021 at 6:13:13 AM UTC-6, David Brown wrote:
>
>> What seems to be missing here (at least, /I/ have missed it) is a
>> discusion about what people actually want to do with computing power.
>> Is it fair to suggest that most tasks that people currently want to run
>> faster, are actually reasonably parallel? And that most tasks that
>> people want to run a /lot/ faster are /very/ parallel?
>
> Unfortunately, no, it isn't.
>
> Many of the tasks that people want to run faster are parallel, but some of
> them are not.
>
> Also, it's at least possible that *some* of the tasks people want to run
> faster... are perhaps more parallel than people realize, and some new
> programming paradigm might enable this parallelism to be brought to
> light. At least that's the hope that fuels attempts to design compilers
> that will bring out extra parallelism that's not obvious to human programmers.
>
> John Savard

A convergence with ai techniques, perhaps ?...

On Tuesday, July 13, 2021 at 3:25:45 PM UTC-5, John Levine wrote:
> According to Marcus <m.de...@this.bitsnbites.eu>:
> >I'm personally quite skeptical about relying too much on compiler
> >technology advancements. Itanium proved it's a bad strategy. Auto-
> >vectorization for SIMD ISA:s proved it's a bad strategy.
> I dunno, the IBM 801 and PL.8 proved it is a good strategy, at least if your name is John Cocke or Fram Allen.
>
> Perhaps the lesson is to be realistic in your plan for how much hardware ugliness you can paper over with
> compiler cleverness, and to remeber that the tradeoffs change over time.
>
> Itanium was based on work at Multiflow in the 1980s, when the amount of stuff you could do in hardware was a lot
> less than it was a decade or two later. The compiler did what it could to schedule memory references statically,
> but once you could do that in hardware, dynamic hardware scheduling worked a lot better.
<
Except for that power thing............
>
> --
> Regards,
> John Levine, jo...@taugh.com, Primary Perpetrator of "The Internet for Dummies",
> Please consider the environment before reading this e-mail. https://jl.ly

On Tuesday, July 13, 2021 at 5:12:09 PM UTC-5, Theo Markettos wrote:
> Quadibloc <jsa...@ecn.ab.ca> wrote:
> > Came across this item:
> >
> > https://www.nextplatform.com/2021/07/12/gutting-decades-of-architecture-to-build-a-new-kind-of-processor/
> >
> > about an attempt to design a processor that does away with instruction
> > sets and all those instructions that just move data around.
> I'm curious about this:
>
> "We can run 700,000 lines of code, which includes standard C libraries used
> in SPEC, and we compile that and run that on our FPGA testbed, which is not
> the full architecture, but a big chunk of it, and get functionally correct
> results. We have a full symbolic debugger and other infrastructure to
> actually make something like that work."
<
Ask them about their SPECint score ??
>
>
> I wonder what's in those 700KLOC. Because frequently existing codebases are
> very CPU-oriented. For example, lots of control flow, doing memory
> allocations all over the place because that's what CPUs do, but things that
> don't lend themselves to parallelism. And then those codebases expect to do
> things like fopen() and printf() and mmap() and other things which don't sit
> very well in a data-oriented machine. So I'm curious as to what OS support
> they have, or whether they're mining some codebase that intentionally
> doesn't have all the pesky I/O and control flow that today's real software
> does.
>
> Theo

On 2021-07-13, Theo Markettos wrote:

[snip]

> The hardware is hugely worse in size and power efficiency than the neural
> network between our ears, but we pay that cost.
>

I personally think that the proper implementation of a power efficient
neural network requires two things:

1) Memory (signals and weights) should be co-located with the ALU:s (or
distributed across the compute matrix, if you will).

2) Compute cells should only be active when activated by a signal.
Perhaps the design should not be clocked by a global clock at all?

And if the speed and power efficiency advantage is big enough, some may
even accept a design with non-deterministic output (example: the neural
network between our ears) - e.g. as a result of an asynchronous design.
That could further open up for things like cheaper / denser memory
structures where bit-flips could be accepted (or maybe even desired?),
etc.

That would be a massive paradigm shift.

/Marcus

On 14/07/2021 08:01, Marcus wrote:
> On 2021-07-13, Theo Markettos wrote:
>
> [snip]
>
>> The hardware is hugely worse in size and power efficiency than the neural
>> network between our ears, but we pay that cost.
>>
>
> I personally think that the proper implementation of a power efficient
> neural network requires two things:
>
> 1) Memory (signals and weights) should be co-located with the ALU:s (or
>    distributed across the compute matrix, if you will).
>
> 2) Compute cells should only be active when activated by a signal.
>    Perhaps the design should not be clocked by a global clock at all?
>

I agree on both accounts. (I haven't looked much at neural networks
since university, but I assume the principles haven't changed.)

A biological neuron encompasses its own memory (weights), its own
processing, its own IO, its own learning system. To make really
powerful artificial neural networks, the component parts need that too.
Then you can scale the whole thing by adding more of the same.

> And if the speed and power efficiency advantage is big enough, some may
> even accept a design with non-deterministic output (example: the neural
> network between our ears) - e.g. as a result of an asynchronous design.
> That could further open up for things like cheaper / denser memory
> structures where bit-flips could be accepted (or maybe even desired?),
> etc.
>
> That would be a massive paradigm shift.
>

Accepting non-deterministic output, or imperfect results, would be a big
change. It would not be suitable for general computing - but could be
fine for some specialised tasks. A good neural network architecture
could be vastly more efficient for vision processing, just as a good
quantum architecture could be efficient for some kinds of optimisation
problems - but neither would be any use for a Usenet client!

On 2021-07-14 09:01, David Brown wrote:
> On 14/07/2021 08:01, Marcus wrote:
>> On 2021-07-13, Theo Markettos wrote:
>>
>> [snip]
>>
>>> The hardware is hugely worse in size and power efficiency than the neural
>>> network between our ears, but we pay that cost.
>>>
>>
>> I personally think that the proper implementation of a power efficient
>> neural network requires two things:
>>
>> 1) Memory (signals and weights) should be co-located with the ALU:s (or
>>    distributed across the compute matrix, if you will).
>>
>> 2) Compute cells should only be active when activated by a signal.
>>    Perhaps the design should not be clocked by a global clock at all?
>>
>
> I agree on both accounts. (I haven't looked much at neural networks
> since university, but I assume the principles haven't changed.)
>
> A biological neuron encompasses its own memory (weights), its own
> processing, its own IO, its own learning system. To make really
> powerful artificial neural networks, the component parts need that too.
> Then you can scale the whole thing by adding more of the same.

Exactly. You'll not be bounded by memory bandwidth or similar - it's a
truly distributed system that should scale very well.

>
>> And if the speed and power efficiency advantage is big enough, some may
>> even accept a design with non-deterministic output (example: the neural
>> network between our ears) - e.g. as a result of an asynchronous design.
>> That could further open up for things like cheaper / denser memory
>> structures where bit-flips could be accepted (or maybe even desired?),
>> etc.
>>
>> That would be a massive paradigm shift.
>>
>
> Accepting non-deterministic output, or imperfect results, would be a big
> change. It would not be suitable for general computing - but could be
> fine for some specialised tasks. A good neural network architecture
> could be vastly more efficient for vision processing, just as a good
> quantum architecture could be efficient for some kinds of optimisation
> problems - but neither would be any use for a Usenet client!
>

I think it's kind of like lossless vs. lossy compression. Once you
accept imperfection, you get orders of magnitude wins. For some
applications this will be fine. For some applications where we currently
think that determinism is required, it will be fine too. But for most
of the software that we're used to (OS:es, compilers, Usenet readers,
text editors, Web browsers etc) it will be of little use.

/Marcus

On 14/07/2021 09:57, Marcus wrote:
> On 2021-07-14 09:01, David Brown wrote:
>> On 14/07/2021 08:01, Marcus wrote:
>>> On 2021-07-13, Theo Markettos wrote:
>>>
>>> [snip]
>>>
>>>> The hardware is hugely worse in size and power efficiency than the
>>>> neural
>>>> network between our ears, but we pay that cost.
>>>>
>>>
>>> I personally think that the proper implementation of a power efficient
>>> neural network requires two things:
>>>
>>> 1) Memory (signals and weights) should be co-located with the ALU:s (or
>>>     distributed across the compute matrix, if you will).
>>>
>>> 2) Compute cells should only be active when activated by a signal.
>>>     Perhaps the design should not be clocked by a global clock at all?
>>>
>>
>> I agree on both accounts.  (I haven't looked much at neural networks
>> since university, but I assume the principles haven't changed.)
>>
>> A biological neuron encompasses its own memory (weights), its own
>> processing, its own IO, its own learning system.  To make really
>> powerful artificial neural networks, the component parts need that too.
>>   Then you can scale the whole thing by adding more of the same.
>
> Exactly. You'll not be bounded by memory bandwidth or similar - it's a
> truly distributed system that should scale very well.
>
>>
>>> And if the speed and power efficiency advantage is big enough, some may
>>> even accept a design with non-deterministic output (example: the neural
>>> network between our ears) - e.g. as a result of an asynchronous design.
>>> That could further open up for things like cheaper / denser memory
>>> structures where bit-flips could be accepted (or maybe even desired?),
>>> etc.
>>>
>>> That would be a massive paradigm shift.
>>>
>>
>> Accepting non-deterministic output, or imperfect results, would be a big
>> change.  It would not be suitable for general computing - but could be
>> fine for some specialised tasks.  A good neural network architecture
>> could be vastly more efficient for vision processing, just as a good
>> quantum architecture could be efficient for some kinds of optimisation
>> problems - but neither would be any use for a Usenet client!
>>
>
> I think it's kind of like lossless vs. lossy compression. Once you
> accept imperfection, you get orders of magnitude wins. For some
> applications this will be fine. For some applications where we currently
> think that determinism is required, it will be fine too. But for most
> of the software that we're used to (OS:es, compilers, Usenet readers,
> text editors, Web browsers etc) it will be of little use.
>

That's a good analogy.

But I also think there is scope for big improvements even within
"normal" deterministic code. As I mentioned elsewhere in the thread, I
think there are features that code be added to current processors that
could greatly improve and simplify parallel coding, and thereby make
better use of the architectures we have.

On 2021-07-13 15:14, Theo Markettos wrote:
> David Brown <david.brown@hesbynett.no> wrote:
>> What seems to be missing here (at least, /I/ have missed it) is a
>> discusion about what people actually want to do with computing power.
>> Is it fair to suggest that most tasks that people currently want to run
>> faster, are actually reasonably parallel? And that most tasks that
>> people want to run a /lot/ faster are /very/ parallel?
>
> I think the question to ask is: what tasks are we /not/ doing, because we
> don't have the computing power to handle them?
>
> Artificial neural networks were invented a long time ago, but were not
> feasible to deploy because they needed what was, at the time, infeasible
> amounts of computing power. Now the computing power has caught up and
> they're feasible - and so we're using them as a hammer to hit every problem.
> The hardware is hugely worse in size and power efficiency than the neural
> network between our ears, but we pay that cost.
>
> So what fields have dismissed problems because there is at present
> insufficient computing resource?

I think that the most interesting problems & solutions are the ones that
most of us never thought of, until there was adequate hardware to
implement them.

A lot of research went in to image denoising a few years ago (I happened
to work at Autodesk when they implemented a couple of denoising
algorithms for ray tracers), and then suddenly there was a GPU that was
capable of doing it in real time using neural networks. I think that
very few people expected that.

To quote John Carmack (regarding ray tracing in games), [1]:

"One significant thing I didn't have on my radar back then [2013] was
neural network denoising / image enhancement."

/Marcus

[1] https://twitter.com/ID_AA_Carmack/status/1098687168443240450

> 2) Compute cells should only be active when activated by a signal.
> Perhaps the design should not be clocked by a global clock at all?

I recentlyish saw an article (in ACM Communications maybe?) about the
use of delay to encode values, in order to significantly lower power
consumption. The data representation is analog but it uses standard
digital logic elements (e.g. the AND gate performs the `min` operation
and the OR gate performs the `max` operation).

Stefan

On Wednesday, July 14, 2021 at 1:01:05 AM UTC-5, Marcus wrote:
> On 2021-07-13, Theo Markettos wrote:
>
> [snip]
> > The hardware is hugely worse in size and power efficiency than the neural
> > network between our ears, but we pay that cost.
> >
> I personally think that the proper implementation of a power efficient
> neural network requires two things:
>
> 1) Memory (signals and weights) should be co-located with the ALU:s (or
> distributed across the compute matrix, if you will).
>
> 2) Compute cells should only be active when activated by a signal.
> Perhaps the design should not be clocked by a global clock at all?
<
The calculations are signaled by the arrival of the data. No clock, local or
global.
>
> And if the speed and power efficiency advantage is big enough, some may
> even accept a design with non-deterministic output (example: the neural
> network between our ears) - e.g. as a result of an asynchronous design.
> That could further open up for things like cheaper / denser memory
> structures where bit-flips could be accepted (or maybe even desired?),
> etc.
>
> That would be a massive paradigm shift.
>
> /Marcus