Working out the details for memory management of core called rfPhoenix. The core is 32-bit and a 10-bit ASID is part of the address, so a 42-bit virtual address results. If the page size used is 4kB then a 41-bit virtual address can easily be mapped: 12+9+10+10 (PTEs are 64-bit, PDEs are 32-bit) But 42-bits need to be mapped. So how to handle the extra bit? My thought is to handle the extra bit by using a PDE that is only 16-bits in size for the first level table. Then things could be mapped 12+9+10+11. There is a nice sequence to those numbers. It would work with a valid bit and 15 bits for the page directory address. The upper bits would be assumed to be zero. This means the page directories at the second level must be within the first 27 bits (12+15) of the physical address space. For the third level pages could be located anywhere.
There are seven bits left over in the 64-bit PTE.
Another alternative is to simply use a larger page size. 8kB pages could map 13+10+11+11 or a 45-bit address space. I prefer the 4kB page size or smaller though.

On Saturday, September 3, 2022 at 3:31:25 AM UTC-5, robf...@gmail.com wrote:
> Working out the details for memory management of core called rfPhoenix. The core is 32-bit and a 10-bit ASID is part of the address, so a 42-bit virtual address results. If the page size used is 4kB then a 41-bit virtual address can easily be mapped: 12+9+10+10 (PTEs are 64-bit, PDEs are 32-bit) But 42-bits need to be mapped. So how to handle the extra bit? My thought is to handle the extra bit by using a PDE that is only 16-bits in size for the first level table. Then things could be mapped 12+9+10+11. There is a nice sequence to those numbers. It would work with a valid bit and 15 bits for the page directory address. The upper bits would be assumed to be zero. This means the page directories at the second level must be within the first 27 bits (12+15) of the physical address space. For the third level pages could be located anywhere.
> There are seven bits left over in the 64-bit PTE.
> Another alternative is to simply use a larger page size. 8kB pages could map 13+10+11+11 or a 45-bit address space. I prefer the 4kB page size or smaller though.
<
When I faced a similar problem, I wanted to go with 4K but reality drove me to 8K (where it lives today.) But, 13+10+10+10 gives the progression 8K->8M->8G->8E and 1 more bit to play with in PTE/PTP encoding.
<

robf...@gmail.com wrote:
> Working out the details for memory management of core called rfPhoenix. The core is 32-bit and a 10-bit ASID is part of the address, so a 42-bit virtual address results. If the page size used is 4kB then a 41-bit virtual address can easily be mapped: 12+9+10+10 (PTEs are 64-bit, PDEs are 32-bit) But 42-bits need to be mapped. So how to handle the extra bit? My thought is to handle the extra bit by using a PDE that is only 16-bits in size for the first level table. Then things could be mapped 12+9+10+11. There is a nice sequence to those numbers. It would work with a valid bit and 15 bits for the page directory address. The upper bits would be assumed to be zero. This means the page directories at the second level must be within the first 27 bits (12+15) of the physical address space. For the third level pages could be located anywhere.
> There are seven bits left over in the 64-bit PTE.
> Another alternative is to simply use a larger page size. 8kB pages could map 13+10+11+11 or a 45-bit address space. I prefer the 4kB page size or smaller though.

There are multiple possible meanings to what you said.
Are you saying
1) that the ASID is visible and accessible to software as the high
10 bits of 42-bit virtual address, similar to x86 segment number

2) or the ISA virtual address is 32 bits and inside the MMU
an ASID is pre-pended as high address bits creating
a 42-bit single virtual address space,
(so that the ASID is part of the translation tables but
not directly visible to most OS or application software)

3) or the ISA virtual address space is 32 bits and an ASID is
used as a tag in the TLB to optimize address space switches
(in which case it is not a 42-bit virtual address,
it is 32 bits as the ASID bits are not part of the page tables).

I suspect you mean #2 which raises a bigger question as to why
you need a 42-bit virtual address space for a 32-bit processor?
And where did the number 42 come from (aside from HGTTG)?
Because if the ASID bits are only in the MMU then it can be any size
and would be MMU-model dependent.

How big is the physical address? The usual reason for 64-bit PTEs is
to hold a large physical address. How would this fit into 32-bit PDEs
along with the management bits (valid, cache control, OS-reserved, etc)?

Is compatibility with existing OS software a consideration,
or are you writing everything yourself?

My general advice would be design the MMU that you would want to
write the management software for. Don't do weird shit (like different
sized PDEs that can only exist in certain address ranges) because you
will pay for it 100 times over trying to compensate later.

On Saturday, September 3, 2022 at 10:26:25 AM UTC-4, EricP wrote:
> robf...@gmail.com wrote:
> > Working out the details for memory management of core called rfPhoenix. The core is 32-bit and a 10-bit ASID is part of the address, so a 42-bit virtual address results. If the page size used is 4kB then a 41-bit virtual address can easily be mapped: 12+9+10+10 (PTEs are 64-bit, PDEs are 32-bit) But 42-bits need to be mapped. So how to handle the extra bit? My thought is to handle the extra bit by using a PDE that is only 16-bits in size for the first level table. Then things could be mapped 12+9+10+11. There is a nice sequence to those numbers. It would work with a valid bit and 15 bits for the page directory address. The upper bits would be assumed to be zero. This means the page directories at the second level must be within the first 27 bits (12+15) of the physical address space. For the third level pages could be located anywhere.
> > There are seven bits left over in the 64-bit PTE.
> > Another alternative is to simply use a larger page size. 8kB pages could map 13+10+11+11 or a 45-bit address space. I prefer the 4kB page size or smaller though.
> There are multiple possible meanings to what you said.
> Are you saying
> 1) that the ASID is visible and accessible to software as the high
> 10 bits of 42-bit virtual address, similar to x86 segment number
>
> 2) or the ISA virtual address is 32 bits and inside the MMU
> an ASID is pre-pended as high address bits creating
> a 42-bit single virtual address space,
> (so that the ASID is part of the translation tables but
> not directly visible to most OS or application software)
>
> 3) or the ISA virtual address space is 32 bits and an ASID is
> used as a tag in the TLB to optimize address space switches
> (in which case it is not a 42-bit virtual address,
> it is 32 bits as the ASID bits are not part of the page tables).
>
> I suspect you mean #2 which raises a bigger question as to why
> you need a 42-bit virtual address space for a 32-bit processor?
> And where did the number 42 come from (aside from HGTTG)?
> Because if the ASID bits are only in the MMU then it can be any size
> and would be MMU-model dependent.
>
> How big is the physical address? The usual reason for 64-bit PTEs is
> to hold a large physical address. How would this fit into 32-bit PDEs
> along with the management bits (valid, cache control, OS-reserved, etc)?
>
> Is compatibility with existing OS software a consideration,
> or are you writing everything yourself?
>
> My general advice would be design the MMU that you would want to
> write the management software for. Don't do weird shit (like different
> sized PDEs that can only exist in certain address ranges) because you
> will pay for it 100 times over trying to compensate later.

You are correct, I meant #2.

I planned on providing a 40-bit physical address space. So, 28 bits were
required to represent the page number. I also store the privilege level and
access rights in the PTE so a 64-bit PTE was needed. PDEs can fit a 28-bit
page number in 32-bits.
It is not really a 42-bit virtual address space required. The software sees
only a 32-bit virtual address space. The ASID is 10-bits tacked onto a 32-bit
address creates a 42-bit virtual address. It is not HGGTG related, just a
coincidence. The ASID is used to separate address spaces for different apps
/ threads. I thought about trimming some of the virtual address space. But
it is somewhat ulgy to say the address space is only 30-bits for instance.
I have decided to go with 8kB page tables and PDEs with the same layout as
PTEs following your sage advice.

On Monday, September 5, 2022 at 2:06:06 AM UTC-4, robf...@gmail.com wrote:
> On Saturday, September 3, 2022 at 10:26:25 AM UTC-4, EricP wrote:
> > robf...@gmail.com wrote:
> > > Working out the details for memory management of core called rfPhoenix. The core is 32-bit and a 10-bit ASID is part of the address, so a 42-bit virtual address results. If the page size used is 4kB then a 41-bit virtual address can easily be mapped: 12+9+10+10 (PTEs are 64-bit, PDEs are 32-bit) But 42-bits need to be mapped. So how to handle the extra bit? My thought is to handle the extra bit by using a PDE that is only 16-bits in size for the first level table. Then things could be mapped 12+9+10+11. There is a nice sequence to those numbers. It would work with a valid bit and 15 bits for the page directory address. The upper bits would be assumed to be zero.. This means the page directories at the second level must be within the first 27 bits (12+15) of the physical address space. For the third level pages could be located anywhere.
> > > There are seven bits left over in the 64-bit PTE.
> > > Another alternative is to simply use a larger page size. 8kB pages could map 13+10+11+11 or a 45-bit address space. I prefer the 4kB page size or smaller though.
> > There are multiple possible meanings to what you said.
> > Are you saying
> > 1) that the ASID is visible and accessible to software as the high
> > 10 bits of 42-bit virtual address, similar to x86 segment number
> >
> > 2) or the ISA virtual address is 32 bits and inside the MMU
> > an ASID is pre-pended as high address bits creating
> > a 42-bit single virtual address space,
> > (so that the ASID is part of the translation tables but
> > not directly visible to most OS or application software)
> >
> > 3) or the ISA virtual address space is 32 bits and an ASID is
> > used as a tag in the TLB to optimize address space switches
> > (in which case it is not a 42-bit virtual address,
> > it is 32 bits as the ASID bits are not part of the page tables).
> >
> > I suspect you mean #2 which raises a bigger question as to why
> > you need a 42-bit virtual address space for a 32-bit processor?
> > And where did the number 42 come from (aside from HGTTG)?
> > Because if the ASID bits are only in the MMU then it can be any size
> > and would be MMU-model dependent.
> >
> > How big is the physical address? The usual reason for 64-bit PTEs is
> > to hold a large physical address. How would this fit into 32-bit PDEs
> > along with the management bits (valid, cache control, OS-reserved, etc)?
> >
> > Is compatibility with existing OS software a consideration,
> > or are you writing everything yourself?
> >
> > My general advice would be design the MMU that you would want to
> > write the management software for. Don't do weird shit (like different
> > sized PDEs that can only exist in certain address ranges) because you
> > will pay for it 100 times over trying to compensate later.
> You are correct, I meant #2.
>
> I planned on providing a 40-bit physical address space. So, 28 bits were
> required to represent the page number. I also store the privilege level and
> access rights in the PTE so a 64-bit PTE was needed. PDEs can fit a 28-bit
> page number in 32-bits.
> It is not really a 42-bit virtual address space required. The software sees
> only a 32-bit virtual address space. The ASID is 10-bits tacked onto a 32-bit
> address creates a 42-bit virtual address. It is not HGGTG related, just a
> coincidence. The ASID is used to separate address spaces for different apps
> / threads. I thought about trimming some of the virtual address space. But
> it is somewhat ulgy to say the address space is only 30-bits for instance..
> I have decided to go with 8kB page tables and PDEs with the same layout as
> PTEs following your sage advice.

In FPGA land, it turns out the minimum size that the data cache can be made
is 256kB as this is a minimal configuration of block RAMs for a 4-way set
associative cache with 64B line size. That leads ultimately to a 16-bit cache
index to fetch data. If the cache is virtually indexed that means the virtual
index needs to be 16 bits, leading to a page size of 64kB. Otherwise it cost
at least an extra cycle to physically index the cache with a smaller page size.

On Monday, September 5, 2022 at 7:37:36 AM UTC-5, robf...@gmail.com wrote:
> On Monday, September 5, 2022 at 2:06:06 AM UTC-4, robf...@gmail.com wrote:
> > On Saturday, September 3, 2022 at 10:26:25 AM UTC-4, EricP wrote:
> > > robf...@gmail.com wrote:
> > > > Working out the details for memory management of core called rfPhoenix. The core is 32-bit and a 10-bit ASID is part of the address, so a 42-bit virtual address results. If the page size used is 4kB then a 41-bit virtual address can easily be mapped: 12+9+10+10 (PTEs are 64-bit, PDEs are 32-bit) But 42-bits need to be mapped. So how to handle the extra bit? My thought is to handle the extra bit by using a PDE that is only 16-bits in size for the first level table. Then things could be mapped 12+9+10+11. There is a nice sequence to those numbers. It would work with a valid bit and 15 bits for the page directory address. The upper bits would be assumed to be zero. This means the page directories at the second level must be within the first 27 bits (12+15) of the physical address space. For the third level pages could be located anywhere.
> > > > There are seven bits left over in the 64-bit PTE.
> > > > Another alternative is to simply use a larger page size. 8kB pages could map 13+10+11+11 or a 45-bit address space. I prefer the 4kB page size or smaller though.
> > > There are multiple possible meanings to what you said.
> > > Are you saying
> > > 1) that the ASID is visible and accessible to software as the high
> > > 10 bits of 42-bit virtual address, similar to x86 segment number
> > >
> > > 2) or the ISA virtual address is 32 bits and inside the MMU
> > > an ASID is pre-pended as high address bits creating
> > > a 42-bit single virtual address space,
> > > (so that the ASID is part of the translation tables but
> > > not directly visible to most OS or application software)
> > >
> > > 3) or the ISA virtual address space is 32 bits and an ASID is
> > > used as a tag in the TLB to optimize address space switches
> > > (in which case it is not a 42-bit virtual address,
> > > it is 32 bits as the ASID bits are not part of the page tables).
> > >
> > > I suspect you mean #2 which raises a bigger question as to why
> > > you need a 42-bit virtual address space for a 32-bit processor?
> > > And where did the number 42 come from (aside from HGTTG)?
> > > Because if the ASID bits are only in the MMU then it can be any size
> > > and would be MMU-model dependent.
> > >
> > > How big is the physical address? The usual reason for 64-bit PTEs is
> > > to hold a large physical address. How would this fit into 32-bit PDEs
> > > along with the management bits (valid, cache control, OS-reserved, etc)?
> > >
> > > Is compatibility with existing OS software a consideration,
> > > or are you writing everything yourself?
> > >
> > > My general advice would be design the MMU that you would want to
> > > write the management software for. Don't do weird shit (like different
> > > sized PDEs that can only exist in certain address ranges) because you
> > > will pay for it 100 times over trying to compensate later.
> > You are correct, I meant #2.
> >
> > I planned on providing a 40-bit physical address space. So, 28 bits were
> > required to represent the page number. I also store the privilege level and
> > access rights in the PTE so a 64-bit PTE was needed. PDEs can fit a 28-bit
> > page number in 32-bits.
> > It is not really a 42-bit virtual address space required. The software sees
> > only a 32-bit virtual address space. The ASID is 10-bits tacked onto a 32-bit
> > address creates a 42-bit virtual address. It is not HGGTG related, just a
> > coincidence. The ASID is used to separate address spaces for different apps
> > / threads. I thought about trimming some of the virtual address space. But
> > it is somewhat ulgy to say the address space is only 30-bits for instance.
> > I have decided to go with 8kB page tables and PDEs with the same layout as
> > PTEs following your sage advice.
<
> In FPGA land, it turns out the minimum size that the data cache can be made
> is 256kB as this is a minimal configuration of block RAMs for a 4-way set
> associative cache with 64B line size. That leads ultimately to a 16-bit cache
> index to fetch data. If the cache is virtually indexed that means the virtual
> index needs to be 16 bits, leading to a page size of 64kB. Otherwise it cost
> at least an extra cycle to physically index the cache with a smaller page size.
<
SRAMs come with 2 critical dimensions: number of words deep, and number of
bits wide. Ignoring the correction of faults, both numbers are powers of 2. The
typical sizes of an SRAM macro today is 1K Bytes and 2KBytes (64 words ×
128-bits = 1K Byte)
<
With 1KByte SRAMs and a 64 Byte cache line, the cache tag can contain 64 words
of 3 tags each (44-bit address space) making for 12KByte 3-way set associative
cache {under certain circumstances, on can also make 16KByte 4-way SA cache..
<
FPGA-land may be different.

robf...@gmail.com wrote:
> On Monday, September 5, 2022 at 2:06:06 AM UTC-4, robf...@gmail.com wrote:
>> On Saturday, September 3, 2022 at 10:26:25 AM UTC-4, EricP wrote:
>>> robf...@gmail.com wrote:
>>>> Working out the details for memory management of core called rfPhoenix. The core is 32-bit and a 10-bit ASID is part of the address, so a 42-bit virtual address results. If the page size used is 4kB then a 41-bit virtual address can easily be mapped: 12+9+10+10 (PTEs are 64-bit, PDEs are 32-bit) But 42-bits need to be mapped. So how to handle the extra bit? My thought is to handle the extra bit by using a PDE that is only 16-bits in size for the first level table. Then things could be mapped 12+9+10+11. There is a nice sequence to those numbers. It would work with a valid bit and 15 bits for the page directory address. The upper bits would be assumed to be zero.. This means the page directories at the second level must be within the first 27 bits (12+15) of the physical address space. For the third level pages could be located anywhere.
>>>> There are seven bits left over in the 64-bit PTE.
>>>> Another alternative is to simply use a larger page size. 8kB pages could map 13+10+11+11 or a 45-bit address space. I prefer the 4kB page size or smaller though.
>>>
>>> 2) or the ISA virtual address is 32 bits and inside the MMU
>>> an ASID is pre-pended as high address bits creating
>>> a 42-bit single virtual address space,
>>> (so that the ASID is part of the translation tables but
>>> not directly visible to most OS or application software)
>>>
>> You are correct, I meant #2.
>>
>> I planned on providing a 40-bit physical address space. So, 28 bits were
>> required to represent the page number. I also store the privilege level and
>> access rights in the PTE so a 64-bit PTE was needed. PDEs can fit a 28-bit
>> page number in 32-bits.
>> It is not really a 42-bit virtual address space required. The software sees
>> only a 32-bit virtual address space. The ASID is 10-bits tacked onto a 32-bit
>> address creates a 42-bit virtual address. It is not HGGTG related, just a
>> coincidence. The ASID is used to separate address spaces for different apps
>> / threads. I thought about trimming some of the virtual address space. But
>> it is somewhat ulgy to say the address space is only 30-bits for instance..
>> I have decided to go with 8kB page tables and PDEs with the same layout as
>> PTEs following your sage advice.
>
> In FPGA land, it turns out the minimum size that the data cache can be made
> is 256kB as this is a minimal configuration of block RAMs for a 4-way set
> associative cache with 64B line size. That leads ultimately to a 16-bit cache
> index to fetch data. If the cache is virtually indexed that means the virtual
> index needs to be 16 bits, leading to a page size of 64kB. Otherwise it cost
> at least an extra cycle to physically index the cache with a smaller page size.

So the SRAM block for L1 line data is configured as 1024 rows * 2048 bits,
and each lines' status and tag is stored elsewhere in a separate SRAM.
Are other SRAM dimension options available?
How many such SRAM blocks do you have?

And I guess you use a virtually indexed cache so that it can
look up L1D cache in parallel with a TLB lookup.

As this is FPGA there are no CAMs for the TLB so do you intend to
build a set assoc TLB, or a small fully assoc TLB?

I understand the problem with the virtually index cache.
The cache index is 10 bits. The virtual address bits [5:0] are the
line byte offset, bits [12:6] are the 7-bit line number in an 8kB page,
and bits [31:13] are the virtual page number, [41:32] the ASID.
So the 10 cache index bits would cross 3 bits into the virtual page number
which causes aliasing issues.

I had a left-field idea on how your virtual indexed cache might work:
Don't use the virtual page number bits [15:13] in the cache index [9:7].

Instead we assign the cache index bits [9:7] to a register using
a 3 bit number which is assigned by the address space switch software
from a table of the most recent 8 ASIDs to have run.
Call this 3 bit number the Cache Address Space Area (CASA)
and it is the high order 3 bits of the 10-bit cache index.

At address space switch, scan an 8 entry ASID table and if your ASID
is present then that slot is your 3 bit CASA, otherwise assign the next
slot in circular order. That 3 bit number goes in the CASA register.
(In effect, that is a fully assoc. lookup in software but it only
happens during address space switch so that is negligible cost.)

So the cache index [9:0] is made up of CASA [2:0] pre-pended to
virtual address bits [12:6].
The line status and tag SRAM would have the same indexing and
the way tag would match virtual address bits [31:13].
(The ASID bits [41:32] aren't in the tag match because they were
matched when the CASA slot was assigned. However you do need to
invalidate all the match tags in one CASA area when a slot is reassigned.
I'm not sure how that can be done in an FPGA.)

(It also means that the virtually index cache has to be write through
so a cache area invalidate doesn't have a mass of lines to write back.)

In effect, the 256kB cache separates into eight 32kB virtual address
space cache areas for the 8 most recently run address spaces.
Each area is 128 rows, from the 7 virtual address bit [12:6],
with 4 ways of 64 byte lines, so 7+2+6 = 15 bits = 32kB.
Which is a pretty good L1 data cache size for an FPGA.
And it could give lightning quick address space switches because
recently run processes could restart with their cache area warm.

....just musing out loud.

EricP wrote:
>
> I had a left-field idea on how your virtual indexed cache might work:
> ...
>
> In effect, the 256kB cache separates into eight 32kB virtual address
> space cache areas for the 8 most recently run address spaces.
> Each area is 128 rows, from the 7 virtual address bit [12:6],
> with 4 ways of 64 byte lines, so 7+2+6 = 15 bits = 32kB.
> Which is a pretty good L1 data cache size for an FPGA.
> And it could give lightning quick address space switches because
> recently run processes could restart with their cache area warm.

Hmmm... a bit too left field. This doesn't work for pages shared between
address spaces, and IO DMA would have trouble invalidating cache lines
at a physical address because it has 8 rows where a line could be.

For a virtual index cache with 8kB pages and 64B lines requires
it have 128 rows, which are the 7 line address bits in common between
virtual and physical, with as many ways in each row as you like,
and each way physically tagged so you need the TLB lookup result
but later in the cache access cycle. A 256kB cache would have 32 ways.

On Tuesday, September 6, 2022 at 10:21:33 AM UTC-4, EricP wrote:
> EricP wrote:
> >
> > I had a left-field idea on how your virtual indexed cache might work:
> > ...
> >
> > In effect, the 256kB cache separates into eight 32kB virtual address
> > space cache areas for the 8 most recently run address spaces.
> > Each area is 128 rows, from the 7 virtual address bit [12:6],
> > with 4 ways of 64 byte lines, so 7+2+6 = 15 bits = 32kB.
> > Which is a pretty good L1 data cache size for an FPGA.
> > And it could give lightning quick address space switches because
> > recently run processes could restart with their cache area warm.
> Hmmm... a bit too left field. This doesn't work for pages shared between
> address spaces, and IO DMA would have trouble invalidating cache lines
> at a physical address because it has 8 rows where a line could be.
>
> For a virtual index cache with 8kB pages and 64B lines requires
> it have 128 rows, which are the 7 line address bits in common between
> virtual and physical, with as many ways in each row as you like,
> and each way physically tagged so you need the TLB lookup result
> but later in the cache access cycle. A 256kB cache would have 32 ways.

I have used the way of the cache as extra address bits to the block RAM to allow the RAM to do the multiplexing so there is no need to multiplex a 2048-bit wide collection of cache line. It does mean the way is needed to form an address before the RAM is accessed. I suspect wide busses not to route as well as narrow ones. The FPGA has a max bus width of the something like 4k bits. So, there are 128x4 (four ways) or 512 rows of 584 bits each making up a 32kB cache. The extra bits beyond 512 are an overflow area for instructions plus postfixes spanning cache lines.
I have the tags and valid bits in separate memories, the way is determined from these. The tags and valid bits have one cycle of latency while the data bits are two cycles. It ends up taking three cycle to get data from the cache, but it is pipelined. The cache is not on the critical timing path so maybe the latency could be reduced. Time to experiment again.
The block RAM in FPGA part is 4kB. I just select a 512x584 bit block RAM in the ip catalog tool and let the tool work out how many RAMs and the connections between the RAMs. If I choose a 128x584 bit RAM it uses the same number of RAMs as a 512x584 bit RAM, so I chose the max size that does not use any more RAMs. The tool must max out the width of a RAM block which I think is 64-bits, 72-bits if parity bit is used. It is using eight RAMs to get 512-bits wide regardless of how deep the memory is. The I$ uses 8.5 block RAMs. 32kB is a good size for L1.

On Tuesday, September 6, 2022 at 7:00:29 PM UTC-5, robf...@gmail.com wrote:
> On Tuesday, September 6, 2022 at 10:21:33 AM UTC-4, EricP wrote:
> > EricP wrote:

> I have used the way of the cache as extra address bits to the block RAM to allow the RAM to do the multiplexing so there is no need to multiplex a 2048-bit wide collection of cache line. It does mean the way is needed to form an address before the RAM is accessed. I suspect wide busses not to route as well as narrow ones. The FPGA has a max bus width of the something like 4k bits. So, there are 128x4 (four ways) or 512 rows of 584 bits each making up a 32kB cache. The extra bits beyond 512 are an overflow area for instructions plus postfixes spanning cache lines.
<
When I went to Samsung in 2012 to do a GPU, I was still of the mindset established at AMD in
the Opteron era. Busses are same width as register file, and you multi-beat to the length of a
cache line. But we (all of the previous CPU designers) quickly learned that the GPUs we might
be competing with had 8-32 shader cores, and each shader core was connected to the RAM
hierarchy with 1024-bits in and 1024-bits out PER CYCLE !! AND they had up to 32 of these
critters (in our 7nm target process with 10-layers of metal).
<
As I digested this kind of BW and how to best utilize it, I quickly came to the realization that
wires had become free (as long as they don't cross a pin-boundary, and as long as you can
signal-strength repeat and clock-cycle buffer them.)
<
My current 1-wide machine is looking to use 512-bit wide interconnect running at CPU frequency
as its main bus. Whether it is 512-in and 512-out is yet to be determined.
<
I generally quote 10 cycle context switches, but on my 1-wide machine this will end up being
20-cycles at the core and 5 cycles on the "bus". Larger more agressive machines are likely
to achieve 10-cycle context switches (not control transfers--from running thread[j] under
GuestOS[i] to running thread[n] under GuestOS[m] {and of course under different priority and
affinity.} )
<
This kind of BW should enable 16-cores to have enough BW at reasonable latencies and
without doing anything that has not been in GPU practice for a decade.
<
> I have the tags and valid bits in separate memories, the way is determined from these. The tags and valid bits have one cycle of latency while the data bits are two cycles. It ends up taking three cycle to get data from the cache, but it is pipelined. The cache is not on the critical timing path so maybe the latency could be reduced. Time to experiment again.
<
I got 3 tags, LRU, and V<->P alias-bits in a single SRAM Macro supporting a 44-bit physical
address space.
<
> The block RAM in FPGA part is 4kB. I just select a 512x584 bit block RAM in the ip catalog tool and let the tool work out how many RAMs and the connections between the RAMs. If I choose a 128x584 bit RAM it uses the same number of RAMs as a 512x584 bit RAM, so I chose the max size that does not use any more RAMs. The tool must max out the width of a RAM block which I think is 64-bits, 72-bits if parity bit is used. It is using eight RAMs to get 512-bits wide regardless of how deep the memory is. The I$ uses 8.5 block RAMs. 32kB is a good size for L1.

In trying to allow more than 4GB physical address space in MMU tables as compactly
as possible, I note there are unused bit in the page directory entries. By using these bits,
it would be possible to get a 36-bit physical address using 32-bit PTEs and PDEs. The
unused bits could be split between the PDE and the high order address bits for the PTEs
within the PDEs area. If a 32-bit PTE/PDE is used there are about eight bits in the PDE
which would allow more address bits for both the PDE and PTE higher order bits. The
drawback is that all the PTEs in the PTE table pointed to by the PDE would need to
reside in the same 4GB area of physical memory. The benefit is that twice as many
PTE/PDE entries fit into a page than a 64-bit structure would allow. If PDEs are
restricted to a 4GB area of memory, then all the extra bits in the PDE could be used as
high-order bits for the PTE entries. It would be possible to map a 40-bit physical address
range then using 32-bit PTEs.

On Wednesday, October 5, 2022 at 1:04:45 PM UTC-5, robf...@gmail.com wrote:
> In trying to allow more than 4GB physical address space in MMU tables as compactly
> as possible, I note there are unused bit in the page directory entries. By using these bits,
<
Are we to assume that you are using 2-level paging {PTP and PTE} ?
Are we to assume 4096 Byte pages ?
<
Given the above:: you have 12-bits at PTP and 12-bits at PTE::
How are you using these bits at the PTP level ?
How are you using these bits at the PTE level ?
<
> it would be possible to get a 36-bit physical address using 32-bit PTEs and PDEs. The
<
If you place all access rights and permissions at the PTE, you can project a 32-VA into
a 48-bit PA by simply placing PA<47..32> = PTP<11:0>.
<
Then:: PA<31:12> = PTE<31:12> and PA<11:0> = VA<11:0>
<
> unused bits could be split between the PDE and the high order address bits for the PTEs
> within the PDEs area. If a 32-bit PTE/PDE is used there are about eight bits in the PDE
> which would allow more address bits for both the PDE and PTE higher order bits. The
<
As I illustrated, you can use all 12 of these bits in the PTP.
<
> drawback is that all the PTEs in the PTE table pointed to by the PDE would need to
> reside in the same 4GB area of physical memory.
<
This problem is solved by having a doubleword Root pointer. The table has to be
contiguous in some 4GB area, but not necessarily the zero'th, or perhaps 4 non-
contiguous 1GB areas.
<
> The benefit is that twice as many
> PTE/PDE entries fit into a page than a 64-bit structure would allow. If PDEs are
> restricted to a 4GB area of memory, then all the extra bits in the PDE could be used as
> high-order bits for the PTE entries. It would be possible to map a 40-bit physical address
> range then using 32-bit PTEs.
<
once again:: 48 = 32 + 12.

On Wednesday, October 5, 2022 at 3:40:07 PM UTC-4, MitchAlsup wrote:
> On Wednesday, October 5, 2022 at 1:04:45 PM UTC-5, robf...@gmail.com wrote:
> > In trying to allow more than 4GB physical address space in MMU tables as compactly
> > as possible, I note there are unused bit in the page directory entries. By using these bits,
> <
> Are we to assume that you are using 2-level paging {PTP and PTE} ?

For the current project the table is two-level. The upper level is stored in a
dedicated 4kB SRAM which is at a fixed address. There are only six
address bits plus two bits for thread selection that need translating.
However, I may use the same hardware in another project mapping a
64-bit address which requires a four-level table plus a number of table-
pointers per thread.

> Are we to assume 4096 Byte pages ?

Pages are 16kB in size. Chosen to allow virtual indexing of the L1 cache.

> <
> Given the above:: you have 12-bits at PTP and 12-bits at PTE::
> How are you using these bits at the PTP level ?

For the page directory / PTP there is just a page number and a level
field. So, there are 11 or 12 bits available.

> How are you using these bits at the PTE level ?

The access rights including a cacheable indicator are stored at the
PTE level along with modified, accessed, and global translation bits.
There are 18 bits for the page number. Then there are three bits for
the table level and a valid bit. It occurs to me the valid bit can be
removed by making assumptions depending on the value of other
bits. That bit is just a convenience. Three bits extra are reserved for
software use.

> <
> > it would be possible to get a 36-bit physical address using 32-bit PTEs and PDEs. The
> <
> If you place all access rights and permissions at the PTE, you can project a 32-VA into
> a 48-bit PA by simply placing PA<47..32> = PTP<11:0>.
> <
> Then:: PA<31:12> = PTE<31:12> and PA<11:0> = VA<11:0>
> <
> > unused bits could be split between the PDE and the high order address bits for the PTEs
> > within the PDEs area. If a 32-bit PTE/PDE is used there are about eight bits in the PDE
> > which would allow more address bits for both the PDE and PTE higher order bits. The
> <
> As I illustrated, you can use all 12 of these bits in the PTP.

> <
> > drawback is that all the PTEs in the PTE table pointed to by the PDE would need to
> > reside in the same 4GB area of physical memory.
> <
> This problem is solved by having a doubleword Root pointer. The table has to be
> contiguous in some 4GB area, but not necessarily the zero'th, or perhaps 4 non-
> contiguous 1GB areas.
> <
> > The benefit is that twice as many
> > PTE/PDE entries fit into a page than a 64-bit structure would allow. If PDEs are
> > restricted to a 4GB area of memory, then all the extra bits in the PDE could be used as
> > high-order bits for the PTE entries. It would be possible to map a 40-bit physical address
> > range then using 32-bit PTEs.
> <
> once again:: 48 = 32 + 12.

The new math.

Hmm, if a two-bit level field were used instead of a three-bit field another bit could be gained.
Tables would be limited to only four level then, but that should still be enough to map a 64-bit
address. Would need to assume an address of all zeros, or all ones? indicates an invalid entry,
making the page of memory inaccessible. Making 16kB of the boot ROM inaccessible at the
high end of memory after a boot may be okay.

That would give a 19+12+14 or 45-bit physical address. I wonder how valuable to software bits
are, and if more than a single bit is required. I would think that only a single bit is needed to
indicate the entire record is for software use. That would get an additional bit or two.

The whole scheme complicates software. Having to locate pages with the same high order address
bits. But less of the cache would be consumed with page table data over 64-bit entries.

On Wednesday, October 5, 2022 at 6:14:15 PM UTC-4, robf...@gmail.com wrote:
> On Wednesday, October 5, 2022 at 3:40:07 PM UTC-4, MitchAlsup wrote:
> > On Wednesday, October 5, 2022 at 1:04:45 PM UTC-5, robf...@gmail.com wrote:
> > > In trying to allow more than 4GB physical address space in MMU tables as compactly
> > > as possible, I note there are unused bit in the page directory entries. By using these bits,
> > <
> > Are we to assume that you are using 2-level paging {PTP and PTE} ?
> For the current project the table is two-level. The upper level is stored in a
> dedicated 4kB SRAM which is at a fixed address. There are only six
> address bits plus two bits for thread selection that need translating.
> However, I may use the same hardware in another project mapping a
> 64-bit address which requires a four-level table plus a number of table-
> pointers per thread.
> > Are we to assume 4096 Byte pages ?
> Pages are 16kB in size. Chosen to allow virtual indexing of the L1 cache.
> > <
> > Given the above:: you have 12-bits at PTP and 12-bits at PTE::
> > How are you using these bits at the PTP level ?
> For the page directory / PTP there is just a page number and a level
> field. So, there are 11 or 12 bits available.
> > How are you using these bits at the PTE level ?
> The access rights including a cacheable indicator are stored at the
> PTE level along with modified, accessed, and global translation bits.
> There are 18 bits for the page number. Then there are three bits for
> the table level and a valid bit. It occurs to me the valid bit can be
> removed by making assumptions depending on the value of other
> bits. That bit is just a convenience. Three bits extra are reserved for
> software use.
> > <
> > > it would be possible to get a 36-bit physical address using 32-bit PTEs and PDEs. The
> > <
> > If you place all access rights and permissions at the PTE, you can project a 32-VA into
> > a 48-bit PA by simply placing PA<47..32> = PTP<11:0>.
> > <
> > Then:: PA<31:12> = PTE<31:12> and PA<11:0> = VA<11:0>
> > <
> > > unused bits could be split between the PDE and the high order address bits for the PTEs
> > > within the PDEs area. If a 32-bit PTE/PDE is used there are about eight bits in the PDE
> > > which would allow more address bits for both the PDE and PTE higher order bits. The
> > <
> > As I illustrated, you can use all 12 of these bits in the PTP.
>
> > <
> > > drawback is that all the PTEs in the PTE table pointed to by the PDE would need to
> > > reside in the same 4GB area of physical memory.
> > <
> > This problem is solved by having a doubleword Root pointer. The table has to be
> > contiguous in some 4GB area, but not necessarily the zero'th, or perhaps 4 non-
> > contiguous 1GB areas.
> > <
> > > The benefit is that twice as many
> > > PTE/PDE entries fit into a page than a 64-bit structure would allow. If PDEs are
> > > restricted to a 4GB area of memory, then all the extra bits in the PDE could be used as
> > > high-order bits for the PTE entries. It would be possible to map a 40-bit physical address
> > > range then using 32-bit PTEs.
> > <
> > once again:: 48 = 32 + 12.
> The new math.
>
> Hmm, if a two-bit level field were used instead of a three-bit field another bit could be gained.
> Tables would be limited to only four level then, but that should still be enough to map a 64-bit
> address. Would need to assume an address of all zeros, or all ones? indicates an invalid entry,
> making the page of memory inaccessible. Making 16kB of the boot ROM inaccessible at the
> high end of memory after a boot may be okay.
>
> That would give a 19+12+14 or 45-bit physical address. I wonder how valuable to software bits
> are, and if more than a single bit is required. I would think that only a single bit is needed to
> indicate the entire record is for software use. That would get an additional bit or two.
>
> The whole scheme complicates software. Having to locate pages with the same high order address
> bits. But less of the cache would be consumed with page table data over 64-bit entries.

For cores the author is working on, PMA, physical memory attributes, checking is accomplished
using a region table. The author was inspired by researching RISCV, and found a region table in a
few other architectures (Itanium and an HP risc machine). The region table would be setup when
the system is configured, and otherwise not altered.

The memory region table has been updated to use 48-bit physical address pointers. The region
table which stores the attributes for regions of memory, the type of memory and in the author’s
case also contains pointers for any card tables needed for garbage collection, and pointers for
the memory page information of the pages in the region. There are only eight regions supported
as the region ranges are searched in parallel hardware to determine region attributes. The region
table has defaults for only four of eight regions which are ROM, IO, DRAM, and scratchpad RAM.
Attributes from the region table are gated with attributes from the TLB to ensure proper access.
For instance, if the region table indicates memory is read-only it will be only read accessible even
if the TLB output indicates a writable area. The region table can be updated programmatically but
has a lock feature to lock region characteristics to prevent accidental updates. The author has the
region table memory mapped as an array of 32-bit registers. This will work for either 32-bit or
64-bit architectures. As access to the region table by software is infrequent a 32-bit interface
seems adequate.

MMU support for my little (12-bit) 6809 system consisted of a couple of block RAMs used
to map pages of memory and provide some simple access rights (read-write-execute)
protection. 8kB of mapping RAM was enough to handle many mega-bytes of memory.

So, I think memory mapping and protection for small embedded systems is very feasible.

robf...@gmail.com <robfi680@gmail.com> wrote:
> MMU support for my little (12-bit) 6809 system consisted of a couple of
> block RAMs used to map pages of memory and provide some simple access
> rights (read-write-execute) protection. 8kB of mapping RAM was enough to
> handle many mega-bytes of memory.
>
> So, I think memory mapping and protection for small embedded systems is very feasible.

I think it boils down to what is a 'small embedded system' and where is
there a niche where it's reasonable to spend area on SRAM, but yet not be
running software that needs a full MMU.

Four problems:

1. SRAM is expensive. A typical 8 bit MCU has a few KiB of onboard SRAM.
On FPGA BRAM is 'free' (because you're already paying tens or hundreds of
dollars for the chip), which is very much not the case for a regular chip.
Anything that interposes between CPU and main memory needs to be fast, which
basically forces the table to be in SRAM. Any system with off-chip DRAM would
much rather spend that area on cache, since that directly improves
performance.

2. Unless you can fit the whole page table in SRAM (see point 1), you need
to store page tables in DRAM and have a local cache (ie a TLB). TLB misses
involve DRAM cycles, which are expensive.

3. A static mapping table imposes a hard limit on memory size. For
example, the first ARM's memory controller (MEMC1) had a static 128 entry
mapping table (addressed as a CAM). With 4KiB pages that could only address
512KiB of DRAM. Because they couldn't add more entries if you upgraded the
RAM, they instead increase the page size - to 32KiB for 4MiB DRAM. This
made the system pretty hard to write OS software for because you never knew
what your page size was going to be. You couldn't go beyond 4MiB of DRAM
because you ran out of mapping table space.
(although you could go up to 16MiB via a cunning address line hack by muxing
in more MEMC chips, only possible because it was a separate chip).

4. Depends what you mean by 'embedded system', but many of the OS tricks we
use today (eg demand paging, shared libraries, ASLR, memory-mapped files)
require a much bigger virtual address space than we have physical memory.
Those 8-bit MCUs aren't doing those tricks, but bigger systems designed for
MMUs do.

So if you're in an FPGA niche, why not spend the cheap BRAM on mapping
tables? But that's not where low-end systems are today.

Theo

On 10/21/2022 2:15 AM, Theo wrote:
> robf...@gmail.com <robfi680@gmail.com> wrote:
>> MMU support for my little (12-bit) 6809 system consisted of a couple of
>> block RAMs used to map pages of memory and provide some simple access
>> rights (read-write-execute) protection. 8kB of mapping RAM was enough to
>> handle many mega-bytes of memory.
>>
>> So, I think memory mapping and protection for small embedded systems is very feasible.
>
> I think it boils down to what is a 'small embedded system' and where is
> there a niche where it's reasonable to spend area on SRAM, but yet not be
> running software that needs a full MMU.
>
> Four problems:
>
> 1. SRAM is expensive. A typical 8 bit MCU has a few KiB of onboard SRAM.
> On FPGA BRAM is 'free' (because you're already paying tens or hundreds of
> dollars for the chip), which is very much not the case for a regular chip.
> Anything that interposes between CPU and main memory needs to be fast, which
> basically forces the table to be in SRAM. Any system with off-chip DRAM would
> much rather spend that area on cache, since that directly improves
> performance.
>
> 2. Unless you can fit the whole page table in SRAM (see point 1), you need
> to store page tables in DRAM and have a local cache (ie a TLB). TLB misses
> involve DRAM cycles, which are expensive.

Since such systems typically don't have an external storage device to
keep currently unused pages, why do you have a page table at all? It is
an unneeded complication. There are only physical addresses so no
virtual to physical mapping is required. You do need some kind of
protection mechanism, but certainly don't need pages for that.

--
- Stephen Fuld
(e-mail address disguised to prevent spam)

Stephen Fuld <sfuld@alumni.cmu.edu.invalid> writes:
>Since such systems typically don't have an external storage device to
>keep currently unused pages, why do you have a page table at all? It is
>an unneeded complication. There are only physical addresses so no
>virtual to physical mapping is required. You do need some kind of
>protection mechanism, but certainly don't need pages for that.

Virtual memory may have started out as a way to use a disk drive for
main memory, but it has grown far beyond that. These days we
configure our systems without swap space, yet we would not disable
virtual memory.

A major benfit of an MMU is that each process can have it's own
address space. Another benefit is that the physical memory is not
limited by the virtual address bits.

- anton
--
'Anyone trying for "industrial quality" ISA should avoid undefined behavior.'
Mitch Alsup, <c17fcd89-f024-40e7-a594-88a85ac10d20o@googlegroups.com>

In article <2022Oct21.182833@mips.complang.tuwien.ac.at>,
anton@mips.complang.tuwien.ac.at (Anton Ertl) wrote:

> A major benfit of an MMU is that each process can have its own
> address space.

And when you're dealing with systems that have 1-16 GB of physical memory,
no swap space, and 50+ processes running at minimum, that's a big deal.
It allows you to avoid most of the problems with fragmentation of
physical memory. That's really important for systems that are supposed to
stay running for months at a time, and don't have anyone to do system
administration on them.

I'm talking about Android and iOS, of course. They are vastly better than
the previous generation of mass-market mobile operating systems; MMUs are
one of the underlying reasons for that.

John

Stephen Fuld <sfuld@alumni.cmu.edu.invalid> writes:
>On 10/21/2022 2:15 AM, Theo wrote:
>> robf...@gmail.com <robfi680@gmail.com> wrote:
>>> MMU support for my little (12-bit) 6809 system consisted of a couple of
>>> block RAMs used to map pages of memory and provide some simple access
>>> rights (read-write-execute) protection. 8kB of mapping RAM was enough to
>>> handle many mega-bytes of memory.
>>>
>>> So, I think memory mapping and protection for small embedded systems is very feasible.
>>
>> I think it boils down to what is a 'small embedded system' and where is
>> there a niche where it's reasonable to spend area on SRAM, but yet not be
>> running software that needs a full MMU.
>>
>> Four problems:
>>
>> 1. SRAM is expensive. A typical 8 bit MCU has a few KiB of onboard SRAM.
>> On FPGA BRAM is 'free' (because you're already paying tens or hundreds of
>> dollars for the chip), which is very much not the case for a regular chip.
>> Anything that interposes between CPU and main memory needs to be fast, which
>> basically forces the table to be in SRAM. Any system with off-chip DRAM would
>> much rather spend that area on cache, since that directly improves
>> performance.
>>
>> 2. Unless you can fit the whole page table in SRAM (see point 1), you need
>> to store page tables in DRAM and have a local cache (ie a TLB). TLB misses
>> involve DRAM cycles, which are expensive.
>
>Since such systems typically don't have an external storage device to
>keep currently unused pages, why do you have a page table at all?

They've always been about protection and flexibility with or
without backing store.

Being able to write-protect pages is quite useful. Being able
to prevent instruction fetches from a data page is very useful.
Being able to allocate in page-sized chunks, thus eliminating the
need for large contiguous physical address space chunks is very
useful.

On Friday, October 21, 2022 at 12:00:02 PM UTC-5, John Dallman wrote:
> In article <2022Oct2...@mips.complang.tuwien.ac.at>,
> an...@mips.complang.tuwien.ac.at (Anton Ertl) wrote:
>
> > A major benfit of an MMU is that each process can have its own
> > address space.
>
> And when you're dealing with systems that have 1-16 GB of physical memory,
> no swap space, and 50+ processes running at minimum, that's a big deal.
<
How many processes will a front door IoT lock need ?
{1, 2, 3, 4} ?
<
> It allows you to avoid most of the problems with fragmentation of
> physical memory. That's really important for systems that are supposed to
> stay running for months at a time, and don't have anyone to do system
> administration on them.
>
> I'm talking about Android and iOS, of course. They are vastly better than
> the previous generation of mass-market mobile operating systems; MMUs are
> one of the underlying reasons for that.
>
> John

On 10/21/2022 9:58 AM, John Dallman wrote:
> In article <2022Oct21.182833@mips.complang.tuwien.ac.at>,
> anton@mips.complang.tuwien.ac.at (Anton Ertl) wrote:
>
>> A major benfit of an MMU is that each process can have its own
>> address space.
>
> And when you're dealing with systems that have 1-16 GB of physical memory,
> no swap space, and 50+ processes running at minimum, that's a big deal.
> It allows you to avoid most of the problems with fragmentation of
> physical memory. That's really important for systems that are supposed to
> stay running for months at a time, and don't have anyone to do system
> administration on them.
>
> I'm talking about Android and iOS, of course. They are vastly better than
> the previous generation of mass-market mobile operating systems; MMUs are
> one of the underlying reasons for that.

I think we have a terminology problem here. Even though smart phones,
etc. are called embedded systems, I want to distinguish them from things
like SCADA systems, factory automation, code running in disk drives,
TVs, etc.

To me, the distinguishing factor is to what extent the ultimate user
can/does run arbitrary code, especially stuff he didn't write. Once you
get to Android and iOS, (which I wouldn't call embedded systems) I agree
you need all the stuff you mentioned. In that sense, they are closer to
desktops than to "traditional" embedded systems.

My point is that, for what I call embedded systems, while you certainly
benefit from various protection mechanisms, you don't need paging to get
those benefits.

BTW, I haven't seen any recent figures, especially since the explosion
of smart phones, but at one point in history, such systems vastly
outnumbered desktops, etc.

--
- Stephen Fuld
(e-mail address disguised to prevent spam)

Anton Ertl <anton@mips.complang.tuwien.ac.at> wrote:
> Stephen Fuld <sfuld@alumni.cmu.edu.invalid> writes:
> >Since such systems typically don't have an external storage device to
> >keep currently unused pages, why do you have a page table at all? It is
> >an unneeded complication. There are only physical addresses so no
> >virtual to physical mapping is required. You do need some kind of
> >protection mechanism, but certainly don't need pages for that.
>
> Virtual memory may have started out as a way to use a disk drive for
> main memory, but it has grown far beyond that. These days we
> configure our systems without swap space, yet we would not disable
> virtual memory.
>
> A major benfit of an MMU is that each process can have it's own
> address space. Another benefit is that the physical memory is not
> limited by the virtual address bits.

Again, it boils down to general system complexity. In a simple
physical-address-only system, dynamic memory allocation is restricted
because sooner or later your physical memory gets fragmented and you can't
make any further allocations of the size you want. Various allocator
strategies mitigate that, but it's not a problem with an MMU because you
can back contiguous virtual address space with any jumble of pages. But
maybe that is fine for simple 'embedded' workloads, where dynamic allocation
is forbidden for predictability reasons.

In a PA-only system, if you aren't using the MMU for protection, you need
something to interpose on memory accesses to determine if they're allowed or
not. That means either an MPU-style thing (have a small number of ranges
that are allowed, and hardware to check an access against the ranges), or a
similar kind of lookup table structure, or make the check against some
property of the pointer (segments, pointer capabilities, out of band
metadata), at which point you need to worry about forgeability.

As to why you would care, typically you want to limit damage. Your chip
might only be running a single 'app' (ie whatever embedded task it's
programmed for), rather than downloading apps from the store, but you want
to compartmentalise within the software pieces. For example, suppose you
get the IP stack, or the wifi code, or the LTE stack or Bluetooth, etc etc
from a third party. You want to ensure that both the device is not
vulnerable to an over-the-air attack (whose goal is to inject code to be run
on the device), but also that the third-party stack from $vendor should not
be able to attack your application code.

An MMU is a well-understood way to do this, but it's quite expensive in
terms of area and in performance. That doesn't mean people who can't afford
an MMU don't care (or if they don't, they should).

This story is for iPhones, but you can think of these wifi processors as an
example of an embedded system - one which didn't put much effort into
protection, with predictable results:
https://googleprojectzero.blogspot.com/2017/04/over-air-exploiting-broadcoms-wi-fi_4.html
https://googleprojectzero.blogspot.com/2017/09/over-air-vol-2-pt-1-exploiting-wi-fi.html

Theo

In article <364aaeda-907d-4187-b2b3-3c6238b4ad9en@googlegroups.com>,
MitchAlsup@aol.com (MitchAlsup) wrote:
> On Friday, October 21, 2022 at 12:00:02 PM UTC-5, John Dallman
> wrote:
> > And when you're dealing with systems that have 1-16 GB of
> > physical memory, no swap space, and 50+ processes running at
> > minimum, that's a big deal.
> How many processes will a front door IoT lock need ?
> {1, 2, 3, 4} ?

OK, I'd lost track of the context of the discussion. However, I'd be
surprised if we don't see Android-based IoT devices becoming commonplace
within a few years.

The appeal to low-end manufacturers would be that the OS is free, and
it's really easy to hire Android programmers. Those manufacturers don't
do much software support or updating, having found that it doesn't
increase sales, so are unlikely to worry about Android's larger attack
surface.

In higher-end embedded systems, there is substantial use of Embedded
Windows, mostly because the software development tools are decent and
it's easy to produce familiar GUIs. There's an obvious analogy for use of
Android further down the price scale.

John

On 10/22/2022 6:35 AM, John Dallman wrote:
> In article <364aaeda-907d-4187-b2b3-3c6238b4ad9en@googlegroups.com>,
> MitchAlsup@aol.com (MitchAlsup) wrote:
>> On Friday, October 21, 2022 at 12:00:02 PM UTC-5, John Dallman
>> wrote:
>>> And when you're dealing with systems that have 1-16 GB of
>>> physical memory, no swap space, and 50+ processes running at
>>> minimum, that's a big deal.
>> How many processes will a front door IoT lock need ?
>> {1, 2, 3, 4} ?
>
> OK, I'd lost track of the context of the discussion. However, I'd be
> surprised if we don't see Android-based IoT devices becoming commonplace
> within a few years.
>
> The appeal to low-end manufacturers would be that the OS is free, and
> it's really easy to hire Android programmers. Those manufacturers don't
> do much software support or updating, having found that it doesn't
> increase sales, so are unlikely to worry about Android's larger attack
> surface.
>
> In higher-end embedded systems, there is substantial use of Embedded
> Windows, mostly because the software development tools are decent and
> it's easy to produce familiar GUIs. There's an obvious analogy for use of
> Android further down the price scale.

Of course, there is a wide range. Below Android, someone here (I
apologize that I forgot who) mentioned using Free RTOS, which is
probably more appropriate for an IOT doorknob.

https://en.wikipedia.org/wiki/FreeRTOS

--
- Stephen Fuld
(e-mail address disguised to prevent spam)