Experimenting with 34-bit instructions for ANY1 and looking for caveats.
15 34-bit instructions fit into a 512-bit cache line nicely with only two unused bits. It is not too bad to get the hardware implemented for 34-bits. I had changes done in about an hour. It is a 15-to-1 34-bit wide mux from the cache line to extract an instruction. Very similar to a 32-bit instruction. For addressing, we pretend that the IP is addressing 32-bit words. It only makes a difference for the last instruction on a cache line which is not present, so the IP increment skips over that word while incrementing. The hardware part seems easy, but the software is a bit more challenging. Assembling programs takes a bit of work.

On Thursday, June 17, 2021 at 1:21:51 AM UTC-6, robf...@gmail.com wrote:
> 15 34-bit instructions fit into a 512-bit cache line nicely with only two unused bits.

That makes sense, because 512 is equal to two plus 17 times 30. Or 34 times 15.

In Concertina II, I've been working with 34-bit instructions myself, but since they're
one of several alternatives within an elaborate header structure, based on blocks
of 256 bits in length, it is less efficient. I do have one efficient choice where there are
four unused bits, plus seven 36-bit instructions. Or fourteen 18-bit instructions.

John Savard

robf...@gmail.com <robfi680@gmail.com> wrote:
> 15 34-bit instructions fit into a 512-bit cache line nicely with only two
> unused bits. It is not too bad to get the hardware implemented for
> 34-bits. I had changes done in about an hour. It is a 15-to-1 34-bit
> wide mux from the cache line to extract an instruction. Very similar to a
> 32-bit instruction. For addressing, we pretend that the IP is addressing
> 32-bit words. It only makes a difference for the last instruction on a
> cache line which is not present, so the IP increment skips over that word
> while incrementing. The hardware part seems easy, but the software is a
> bit more challenging. Assembling programs takes a bit of work.

I suppose the question is: do you force 512 bit alignment and where does
that not make sense?

For example, how does the program counter work - do you perform a rounding
operation such that base+4 is assumed to be the address really starting at
base+4.25? Are there instances where the ambiguity between a PC of +4 (ie
+4.25) and a data pointer of +4 (ie +4.0) gets you into trouble?

What happens if you have to relocate code? For example when linking do you
end up with lots of basic blocks that are force-aligned to 512 bits?

Theo

On Thursday, June 17, 2021 at 9:41:51 AM UTC-4, Theo wrote:
> robf...@gmail.com <robf...@gmail.com> wrote:
> > 15 34-bit instructions fit into a 512-bit cache line nicely with only two
> > unused bits. It is not too bad to get the hardware implemented for
> > 34-bits. I had changes done in about an hour. It is a 15-to-1 34-bit
> > wide mux from the cache line to extract an instruction. Very similar to a
> > 32-bit instruction. For addressing, we pretend that the IP is addressing
> > 32-bit words. It only makes a difference for the last instruction on a
> > cache line which is not present, so the IP increment skips over that word
> > while incrementing. The hardware part seems easy, but the software is a
> > bit more challenging. Assembling programs takes a bit of work.
> I suppose the question is: do you force 512 bit alignment and where does
> that not make sense?
>
> For example, how does the program counter work - do you perform a rounding
> operation such that base+4 is assumed to be the address really starting at
> base+4.25? Are there instances where the ambiguity between a PC of +4 (ie
> +4.25) and a data pointer of +4 (ie +4.0) gets you into trouble?
>
> What happens if you have to relocate code? For example when linking do you
> end up with lots of basic blocks that are force-aligned to 512 bits?
>
> Theo

Just thinking about this some more, it may be easier to have bit-pair addressable memory rather than forcing 512-bit alignment. It makes the cache more interesting, but it gets rid of several other issues. This would really affect the branch instructions because there is only a 10-bit displacement field. Still 8 whole bits is probably enough 90% of the time. Branches can also have a modifier that adds 12 more displacement bits. So, it could be spit out all the time and left to the linker to compress. Ambiguity between the IP and data pointers could be resolved with bit-pair addressable memory. For data, the fractional portion of the address would always be zero. Unless trying to reference an instruction. With a 64-bit machine it is probably okay to use two address bits to represent a bit-pair address.

On Thursday, June 17, 2021 at 2:21:51 AM UTC-5, robf...@gmail.com wrote:
> Experimenting with 34-bit instructions for ANY1 and looking for caveats.
<
I have come to the conclusion that the ideal size for RISC-like ISAs with 32 registers is 36-bits.
<
> 15 34-bit instructions fit into a 512-bit cache line nicely with only two unused bits. It is not too bad to get the hardware implemented for 34-bits.. I had changes done in about an hour. It is a 15-to-1 34-bit wide mux from the cache line to extract an instruction. Very similar to a 32-bit instruction. For addressing, we pretend that the IP is addressing 32-bit words. It only makes a difference for the last instruction on a cache line which is not present, so the IP increment skips over that word while incrementing. The hardware part seems easy, but the software is a bit more challenging. Assembling programs takes a bit of work.

On Thursday, June 17, 2021 at 9:12:08 AM UTC-6, MitchAlsup wrote:

> I have come to the conclusion that the ideal size for RISC-like ISAs with 32 registers is 36-bits.

I'm inclined to agree. However, Concertina II is (or at least includes) an attempt to live with 32 bits,
even if it is less than ideal. But it offers more as an alternative, as well.

John Savard

Quadibloc <jsavard@ecn.ab.ca> wrote:
> On Thursday, June 17, 2021 at 9:12:08 AM UTC-6, MitchAlsup wrote:
>
> > I have come to the conclusion that the ideal size for RISC-like ISAs with 32 registers is 36-bits.
>
> I'm inclined to agree. However, Concertina II is (or at least includes) an attempt to live with 32 bits,
> even if it is less than ideal. But it offers more as an alternative, as well.

A slightly offbeat suggestion might be to keep the in-memory representation
of instructions to be 32 bits, but have the extra bits in a shadow space
somehow. For example you could have instructions at n, n+4, n+8 and so on.
And then the other 4 bits of the instructions live at m, m+0.5, m+1, etc.

PC still refers to n so branching etc are all fine, but there's implicit
behaviour that changing n also changes m in some relation. For example you
could declare m=n+4GB so that loading a program means populating 32 bits of
instructions around PC and then populating the remaining bits at 4GB+PC/4.
You would likely do some page table work to map in pages into both spaces at
the same time when you load binaries.

Lots of ways this could get exciting, not least atomicity. But it might be
feasible to make this work with enough effort.

Theo

On Thursday, June 17, 2021 at 11:40:24 AM UTC-5, Theo wrote:
> Quadibloc <jsa...@ecn.ab.ca> wrote:
> > On Thursday, June 17, 2021 at 9:12:08 AM UTC-6, MitchAlsup wrote:
> >
> > > I have come to the conclusion that the ideal size for RISC-like ISAs with 32 registers is 36-bits.
> >
> > I'm inclined to agree. However, Concertina II is (or at least includes) an attempt to live with 32 bits,
> > even if it is less than ideal. But it offers more as an alternative, as well.
> A slightly offbeat suggestion might be to keep the in-memory representation
> of instructions to be 32 bits, but have the extra bits in a shadow space
> somehow. For example you could have instructions at n, n+4, n+8 and so on.
> And then the other 4 bits of the instructions live at m, m+0.5, m+1, etc.
<
That is basically what I do in My 66000 with CARRY and PRED instruction-modifiers.
<
low end machines can execute these instructions, higher end machines take
the semantic contents of these and "decorate" instructions in their shadows
so that both sets of semantics are performed in a single calculation.
>
> PC still refers to n so branching etc are all fine, but there's implicit
> behaviour that changing n also changes m in some relation. For example you
> could declare m=n+4GB so that loading a program means populating 32 bits of
> instructions around PC and then populating the remaining bits at 4GB+PC/4.
> You would likely do some page table work to map in pages into both spaces at
> the same time when you load binaries.
>
> Lots of ways this could get exciting, not least atomicity. But it might be
> feasible to make this work with enough effort.
>
> Theo

On 6/17/2021 9:40 AM, Theo wrote:
> Quadibloc <jsavard@ecn.ab.ca> wrote:
>> On Thursday, June 17, 2021 at 9:12:08 AM UTC-6, MitchAlsup wrote:
>>
>>> I have come to the conclusion that the ideal size for RISC-like ISAs with 32 registers is 36-bits.
>>
>> I'm inclined to agree. However, Concertina II is (or at least includes) an attempt to live with 32 bits,
>> even if it is less than ideal. But it offers more as an alternative, as well.
>
> A slightly offbeat suggestion might be to keep the in-memory representation
> of instructions to be 32 bits, but have the extra bits in a shadow space
> somehow. For example you could have instructions at n, n+4, n+8 and so on.
> And then the other 4 bits of the instructions live at m, m+0.5, m+1, etc.
>
> PC still refers to n so branching etc are all fine, but there's implicit
> behaviour that changing n also changes m in some relation. For example you
> could declare m=n+4GB so that loading a program means populating 32 bits of
> instructions around PC and then populating the remaining bits at 4GB+PC/4.
> You would likely do some page table work to map in pages into both spaces at
> the same time when you load binaries.
>
> Lots of ways this could get exciting, not least atomicity. But it might be
> feasible to make this work with enough effort.
>
> Theo
>

CHERI and other caps systems have tried this to add caps tags to memory.
There are practical problems: extra IOIs, bigger working sets, and
keeping stores and loads atomic. The problems are less severe if you
restrict the extra bits to code, because you can assume atomicity, but
the rest remain. The only real solution seems to be to talk Samsung into
making nine bit bytes.

On Thursday, June 17, 2021 at 11:31:20 AM UTC-6, Ivan Godard wrote:
> The only real solution seems to be to talk Samsung into
> making nine bit bytes.

Good news. The upcoming DDR5 standard will change memory from being
64 bits wide with an optional 8 bits for ECC to being two channels, each 32
bits wide, with an optional 8 bits for ECC.

So you could store 72-bit words in memory and still have 8 bits available for
ECC, so chips with a 36-bit word are now practical, although they would require
ECC DRAM.

John Savard

On 6/17/2021 3:25 PM, Quadibloc wrote:
> On Thursday, June 17, 2021 at 11:31:20 AM UTC-6, Ivan Godard wrote:
>> The only real solution seems to be to talk Samsung into
>> making nine bit bytes.
>
> Good news. The upcoming DDR5 standard will change memory from being
> 64 bits wide with an optional 8 bits for ECC to being two channels, each 32
> bits wide, with an optional 8 bits for ECC.
>
> So you could store 72-bit words in memory and still have 8 bits available for
> ECC, so chips with a 36-bit word are now practical, although they would require
> ECC DRAM.
>

Hmm, makes me wonder about a RAM module which rather than having a bunch
of DDRn modules ganged together, had a collection of individually
addressable LPDDR modules (using LPDDR mostly to reduce pin count). So,
say, rather than having 1 or 2 channels at 64 or 32 bits, it has 4
channels each being 16 bits wide (with ECC adding a 5th channel).

Then, the DDR controller splits cache-lines based on address between the
4 chips and potentially addresses multiple chips in parallel.

Decided to go with 36-bit instructions. There is very little difference in code density due to the increased number of instructions that can be encoded in 36 bits over 34 bits. Memory is now nybble addressable. Displacements for branches, jumps and calls are in terms of nybbles. All the addressing recognizes nybble addressability although load and store instructions are limited to byte aligned addresses (the low bit of the address is always zero).. Data pointers and code pointers now have the same meaning. Code is relocatable on nybble boundaries. The machine is limited to 2^63 bytes of memory, probably a non-issue.
For debug display purposes addresses are shown as byte addresses with a fractional bit representing which nybble in the byte is addressed.
ANY1 v3 has gone from 40-bits to 34-bits and back to 36-bit instructions in the last three or four days. So, limited stability in the ISA yet.
Toying with the idea of allowed three source register ports for some instructions. There are a few useful instructions that need three ports, which currently must use modifiers.

On Thursday, June 17, 2021 at 2:25:33 PM UTC-6, Quadibloc wrote:
> On Thursday, June 17, 2021 at 11:31:20 AM UTC-6, Ivan Godard wrote:

> > The only real solution seems to be to talk Samsung into
> > making nine bit bytes.

> Good news. The upcoming DDR5 standard will change memory from being
> 64 bits wide with an optional 8 bits for ECC to being two channels, each 32
> bits wide, with an optional 8 bits for ECC.

> So you could store 72-bit words in memory and still have 8 bits available for
> ECC, so chips with a 36-bit word are now practical, although they would require
> ECC DRAM.

Of course, what I've been doing instead is this:

I've used 32-bit instructions as the default.

Instructions are in aligned 256-bit blocks; a bit of the 32-bit
instruction opcode space is taken for block headers.

Longer instructions can be composed of bits from the header
plus the 32 bits of the instruction proper.

There's a program counter that keeps track of the current
block, and is advanced by being incremented. Moving through
instructions within a block depends on what instructions are
in the block, and where they are located. So that's handled
between decoding the instructions and putting them on the
pipeline.

Of course, though, there is a "program counter" that is seen
by the programmer, which is based on where the current
instruction came from, but that's something that's basically
reconstituted when needed, it doesn't actually exist as a
counter register. Since all the instructions in a block are
decoded in parallel, after all, a traditional program counter
doesn't really have a function - it just needs to have a virtual
existence for things like program-relative addressing and
subroutine calls.

So things like using end-around carry to convert block
number times 7 into a binary number quickly aren't relevant
here either.

John Savard

On 6/17/2021 9:12 PM, robf...@gmail.com wrote:
> Decided to go with 36-bit instructions. There is very little difference in code density due to the increased number of instructions that can be encoded in 36 bits over 34 bits. Memory is now nybble addressable. Displacements for branches, jumps and calls are in terms of nybbles.

If all instructions are fixed length of 36 bits, have you considered
making such displacements in terms of "instructions" rather than
nybbles? That would increase the range of values in such instructions
by a factor of nine (for 36 bit instructions). It adds a little cost to
the address calculation, but only an extra short add. It also makes
things "nice" for the assembler programmer - a jump over say three
instructions (e.g. to skip to the else part of an if-then-else) is coded
as current IP plus 4 instead of having to calculate nybbles.

Loads and stores would still have to be nybble addresses.

--
- Stephen Fuld
(e-mail address disguised to prevent spam)

On Friday, June 18, 2021 at 11:10:42 AM UTC-5, Stephen Fuld wrote:
> On 6/17/2021 9:12 PM, robf...@gmail.com wrote:
> > Decided to go with 36-bit instructions. There is very little difference in code density due to the increased number of instructions that can be encoded in 36 bits over 34 bits. Memory is now nybble addressable. Displacements for branches, jumps and calls are in terms of nybbles.
<
> If all instructions are fixed length of 36 bits, have you considered
> making such displacements in terms of "instructions" rather than
> nybbles?
<
Wise !
<
> That would increase the range of values in such instructions
> by a factor of nine (for 36 bit instructions). It adds a little cost to
> the address calculation, but only an extra short add. It also makes
> things "nice" for the assembler programmer - a jump over say three
> instructions (e.g. to skip to the else part of an if-then-else) is coded
> as current IP plus 4 instead of having to calculate nybbles.
>
> Loads and stores would still have to be nybble addresses.
<
Only if they need to read/write 36-bit items--which is doubtful.
Excepting for the JIT only the FETCH unit should know the size of
instructions.
>
>
> --
> - Stephen Fuld
> (e-mail address disguised to prevent spam)

On 6/18/2021 7:23 AM, Quadibloc wrote:
> On Thursday, June 17, 2021 at 2:25:33 PM UTC-6, Quadibloc wrote:
>> On Thursday, June 17, 2021 at 11:31:20 AM UTC-6, Ivan Godard wrote:
>
>>> The only real solution seems to be to talk Samsung into
>>> making nine bit bytes.
>
>> Good news. The upcoming DDR5 standard will change memory from being
>> 64 bits wide with an optional 8 bits for ECC to being two channels, each 32
>> bits wide, with an optional 8 bits for ECC.
>
>> So you could store 72-bit words in memory and still have 8 bits available for
>> ECC, so chips with a 36-bit word are now practical, although they would require
>> ECC DRAM.
>
> Of course, what I've been doing instead is this:
>
> I've used 32-bit instructions as the default.
>

I have also ended up with a 32-bit default (in BJX2):
32-bits: Pretty much the whole core ISA;
16-bits: Subset, most ops limited to R0..R15.

Encoding using a mix of 16 and 32 bit ops is good for code-density, but
isn't really the highest performance option in my ISA (it is mostly
limited to scalar operation).

There are a few larger encodings:
48-bits, Jumbo-Op48: Still unused (*1)
64-bits, Jumbo-Op64: Recently added (*2).
64-bits, Jumbo+Op32 (Jumbo64): Larger immediate values.
96-bits, Jumbo(2x)+Op32 (Jumbo96): Large immediate values (*3).

*1: Due to technical reasons, would be partly incompatible with the use
of WEX encodings. Would need to make non-trivial modifications to both
the instruction decoder and C compiler to resolve this. Having 48-bit
ops which can only be used in certain contexts would be "kinda lame".
Some similar restrictions also apply to 16-bit ops.

*2: At present, these mostly apply to cases involving predicated ops
which also use R32..R63, encoding SR.S predicated ops, ...
Some other cases, like "MOV Imm33s, Rn" (for R32..R63) also involve Op64
encodings.

*3: These remain roughly the same as before, just with the XGPR
extension making a few functional changes to Jumbo96 which mostly
effected a few unused encodings. This tweak also extends Jumbo96 "Imm64,
Rn" encodings to also cover R32..R63.

Currently, there are no encodings to load either Imm16 or Binary16 into
R32..R63, one will need to make due instead with Imm33s or Binary32
constant-loads (64-bit encodings).

> Instructions are in aligned 256-bit blocks; a bit of the 32-bit
> instruction opcode space is taken for block headers.
>
> Longer instructions can be composed of bits from the header
> plus the 32 bits of the instruction proper.
>
> There's a program counter that keeps track of the current
> block, and is advanced by being incremented. Moving through
> instructions within a block depends on what instructions are
> in the block, and where they are located. So that's handled
> between decoding the instructions and putting them on the
> pipeline.
>
> Of course, though, there is a "program counter" that is seen
> by the programmer, which is based on where the current
> instruction came from, but that's something that's basically
> reconstituted when needed, it doesn't actually exist as a
> counter register. Since all the instructions in a block are
> decoded in parallel, after all, a traditional program counter
> doesn't really have a function - it just needs to have a virtual
> existence for things like program-relative addressing and
> subroutine calls.
>
> So things like using end-around carry to convert block
> number times 7 into a binary number quickly aren't relevant
> here either.
>

This part differs.

FWIW, In general, I have some doubts about the viability of a
non-power-of-2 encoding scheme given typical RAM deals with power-of-2
blocks; and 8-bit bytes + power-of-2 memory words has been pretty much
standard for the last 50 years or so...

An ISA built around 40 or 48 bit encodings would at least remain
compatible with 8 bit bytes or 16 bit words, or a 5x25 or 3x42 bit
encoding with 128-bit blocks.

Though, 7x36 in 256-bits could work, though with the wonkiness of mod-7
addressing.

I could potentially (if I wanted) design a fork of my core which
supported RISC-V or similar, though apart from better compiler maturity,
it would probably "kinda suck".

Something like x86 could be more useful, but making an x86 emulation
layer that "doesn't suck" seems non-trivial. Supporting x86 in hardware
is basically no-go.

Ivan Godard <ivan@millcomputing.com> wrote:
> CHERI and other caps systems have tried this to add caps tags to memory.
> There are practical problems: extra IOIs, bigger working sets, and
> keeping stores and loads atomic. The problems are less severe if you
> restrict the extra bits to code, because you can assume atomicity, but
> the rest remain. The only real solution seems to be to talk Samsung into
> making nine bit bytes.

The difference in what we do with CHERI is the tags aren't user accessible.
So we put a tag cache in from of DRAM that presents the memory as
essentially 129 bits wide (or some multiple thereof) but, while the 129th
bit is carried through caches etc, it isn't addressable directly by
regular instructions. So software still thinks memory is 128 bits wide and
there are no awkward addressing problems. You can get at the 1-bit tag
using different instructions which are deliberately restricted in what they
can do with tags.

You could do a similar trick with instruction fetch and have the extra bits
come out of the shadow space. But the trouble is that software wants to
manipulate instructions - loading binaries, dynamic linking, JIT. And then
things get interesting because you have to manipulate the main space and the
shadow space at the same time. Not to say it's not doable, but existing
toolchain is not written this way.

Putting extra metadata in ECC bits is another way, but a) people want to use
them for ECC (due to rowhammer and friends) and b) in CHERI having efficient
bulk access to tags is quite important (eg for revocation) and that's harder
if you have to fetch every DRAM word to get at them.

Going to inline 36 bit instructions is probably more practical from an
implementation point of view, since a byte is then only shared between two
instructions. It's still work in the toolchain but less so. I'm curious
how to handle PC though still it would have to go 0, 4, 9, 13, 18 or 0, 5,
9, 14, 18. Which is rather inconvenient for calculating addresses.

You would probably have to force alignment, effectively making instruction
words 72 bits and packing two instructions per word. That means you could
only branch into the first instruction of a word and might mean you'd have
to pad with NOPs in some circumstances. That would probably be doable
though, and the tradeoff might be worth it.

Theo

On Saturday, June 19, 2021 at 3:30:55 AM UTC-6, Theo wrote:

> You would probably have to force alignment, effectively making instruction
> words 72 bits and packing two instructions per word. That means you could
> only branch into the first instruction of a word and might mean you'd have
> to pad with NOPs in some circumstances. That would probably be doable
> though, and the tradeoff might be worth it.

I can see that alignment to byte boundaries is important, but why that would
prevent branching to the second instruction is unclear.

How to tell, when branching to a byte, if it is the first or second instruction in
72 bits that starts there? Also align to multiples of nine bytes, not just to the
byte, even though that is not a physical memory issue.

John Savard

On Friday, June 18, 2021 at 12:10:42 PM UTC-4, Stephen Fuld wrote:
> On 6/17/2021 9:12 PM, robf...@gmail.com wrote:
> > Decided to go with 36-bit instructions. There is very little difference in code density due to the increased number of instructions that can be encoded in 36 bits over 34 bits. Memory is now nybble addressable. Displacements for branches, jumps and calls are in terms of nybbles.
> If all instructions are fixed length of 36 bits, have you considered
> making such displacements in terms of "instructions" rather than
> nybbles? That would increase the range of values in such instructions
> by a factor of nine (for 36 bit instructions). It adds a little cost to
> the address calculation, but only an extra short add. It also makes
> things "nice" for the assembler programmer - a jump over say three
> instructions (e.g. to skip to the else part of an if-then-else) is coded
> as current IP plus 4 instead of having to calculate nybbles.
>
> Loads and stores would still have to be nybble addresses.
>
>
> --
> - Stephen Fuld
> (e-mail address disguised to prevent spam)
Yes, I thought of using just whole instructions for addressing. But that creates issues with things like alignment. It also makes data pointers and instruction pointers different. With nybble addressing incrementing the IP is simple, just add 9. The linker would have to align instructions on proper 36-bit boundaries beginning at address zero. It would need to know the nybble address. It might be challenging to pick load addresses that are multiples of 36 bits. It is just easier to use nybble addressing. The start address is defined as byte address $FFFC0300. Is that aligned to an even 36 bits or not? In the boot code there are a bunch of ‘.align’ directives to align things to the start of a cache line for small routines (align 64 bytes). There is no guarantee that the line starts at an even 36-bit address. Branches support a 12-bit displacement. Since it is nybble addressing that is +-1kB. Still an adequate branch range for 95+% of branches.
The instruction aligner must shift the cache line by multiples of 36 bits, beginning at an arbitrary nybble position. It must do this whether or not instructions are aligned on multiples of 36-bits. So, there is no real savings in following 36-bit multiples.

On Saturday, June 19, 2021 at 10:49:32 PM UTC-5, robf...@gmail.com wrote:
> On Friday, June 18, 2021 at 12:10:42 PM UTC-4, Stephen Fuld wrote:
> > On 6/17/2021 9:12 PM, robf...@gmail.com wrote:
> > > Decided to go with 36-bit instructions. There is very little difference in code density due to the increased number of instructions that can be encoded in 36 bits over 34 bits. Memory is now nybble addressable. Displacements for branches, jumps and calls are in terms of nybbles.
> > If all instructions are fixed length of 36 bits, have you considered
> > making such displacements in terms of "instructions" rather than
> > nybbles? That would increase the range of values in such instructions
> > by a factor of nine (for 36 bit instructions). It adds a little cost to
> > the address calculation, but only an extra short add. It also makes
> > things "nice" for the assembler programmer - a jump over say three
> > instructions (e.g. to skip to the else part of an if-then-else) is coded
> > as current IP plus 4 instead of having to calculate nybbles.
> >
> > Loads and stores would still have to be nybble addresses.
> >
> >
> > --
> > - Stephen Fuld
> > (e-mail address disguised to prevent spam)
> Yes, I thought of using just whole instructions for addressing. But that creates issues with things like alignment. It also makes data pointers and instruction pointers different. With nybble addressing incrementing the IP is simple, just add 9. The linker would have to align instructions on proper 36-bit boundaries beginning at address zero. It would need to know the nybble address. It might be challenging to pick load addresses that are multiples of 36 bits. It is just easier to use nybble addressing. The start address is defined as byte address $FFFC0300. Is that aligned to an even 36 bits or not? In the boot code there are a bunch of ‘.align’ directives to align things to the start of a cache line for small routines (align 64 bytes). There is no guarantee that the line starts at an even 36-bit address. Branches support a 12-bit displacement. Since it is nybble addressing that is +-1kB. Still an adequate branch range for 95+% of branches.
> The instruction aligner must shift the cache line by multiples of 36 bits, beginning at an arbitrary nybble position. It must do this whether or not instructions are aligned on multiples of 36-bits. So, there is no real savings in following 36-bit multiples.
<
Multiply by 9 (x<<3+x) and divide by 8 (x>>3) are easy enough.
Then you fetch wide, and use the bits that fall off the down shift as an index into the block.
<
Not even hard............
<
You just keep instruction pointers in data (registers+memory) as indexes into 36-bit containers.

On 6/19/2021 8:49 PM, robf...@gmail.com wrote:
> On Friday, June 18, 2021 at 12:10:42 PM UTC-4, Stephen Fuld wrote:
>> On 6/17/2021 9:12 PM, robf...@gmail.com wrote:
>>> Decided to go with 36-bit instructions. There is very little difference in code density due to the increased number of instructions that can be encoded in 36 bits over 34 bits. Memory is now nybble addressable. Displacements for branches, jumps and calls are in terms of nybbles.
>> If all instructions are fixed length of 36 bits, have you considered
>> making such displacements in terms of "instructions" rather than
>> nybbles? That would increase the range of values in such instructions
>> by a factor of nine (for 36 bit instructions). It adds a little cost to
>> the address calculation, but only an extra short add. It also makes
>> things "nice" for the assembler programmer - a jump over say three
>> instructions (e.g. to skip to the else part of an if-then-else) is coded
>> as current IP plus 4 instead of having to calculate nybbles.
>>
>> Loads and stores would still have to be nybble addresses.
>>
>>
>> --
>> - Stephen Fuld
>> (e-mail address disguised to prevent spam)

> Yes, I thought of using just whole instructions for addressing.

Of course, your architecture, so your decision. But I think you might
have misunderstood some things.

> But that creates issues with things like alignment.

No, it doesn't. All alignment is exactly the same as it is under your
plan. Only the contents of the displacement field in jumps, calls etc.
instructions are changed.

> It also makes data pointers and instruction pointers different.

Yes, it does that.

> With nybble addressing incrementing the IP is simple, just add 9.

With instruction addressing, incrementing the IP is just add 1.

> The linker would have to align instructions on proper 36-bit boundaries beginning at address zero.

No. Instructions in memory are exactly the same as under your proposal.
And you don't have to start at zero any more than you have to start
nybble addressing at zero.

> It would need to know the nybble address.

No, it is easily computed.

> It might be challenging to pick load addresses that are multiples of 36 bits.

I'm not sure what you mean here. Addresses in Load instructions are
unaffected.

It is just easier to use nybble addressing. The start address is defined
as byte address $FFFC0300. Is that aligned to an even 36 bits or not?

Who cares? The hardware transparently converts instruction address to
nybble address upon a control transfer instruction.

In the boot code there are a bunch of ‘.align’ directives to align
things to the start of a cache line for small routines (align 64 bytes).
There is no guarantee that the line starts at an even 36-bit address.

Again, no one is proposing changing the memory layout, only the
displacements in control transfer instructions.

Branches support a 12-bit displacement. Since it is nybble addressing
that is +-1kB. Still an adequate branch range for 95+% of branches.

But not enough for many/most call type instructions.

> The instruction aligner must shift the cache line by multiples of 36 bits, beginning at an arbitrary nybble position. It must do this whether or not instructions are aligned on multiples of 36-bits. So, there is no real savings in following 36-bit multiples.

Again, you seem to be assuming that was not proposed. Actual placement
in memory is not changed.

--
- Stephen Fuld
(e-mail address disguised to prevent spam)

Stephen Fuld wrote:
> On 6/19/2021 8:49 PM, robf...@gmail.com wrote:
>> On Friday, June 18, 2021 at 12:10:42 PM UTC-4, Stephen Fuld wrote:
>>> On 6/17/2021 9:12 PM, robf...@gmail.com wrote:
>>>> Decided to go with 36-bit instructions. There is very little
>>>> difference in code density due to the increased number of
>>>> instructions that can be encoded in 36 bits over 34 bits. Memory is
>>>> now nybble addressable. Displacements for branches, jumps and calls
>>>> are in terms of nybbles.
>>> If all instructions are fixed length of 36 bits, have you considered
>>> making such displacements in terms of "instructions" rather than
>>> nybbles? That would increase the range of values in such instructions
>>> by a factor of nine (for 36 bit instructions). It adds a little cost to
>>> the address calculation, but only an extra short add. It also makes
>>> things "nice" for the assembler programmer - a jump over say three
>>> instructions (e.g. to skip to the else part of an if-then-else) is coded
>>> as current IP plus 4 instead of having to calculate nybbles.
>>>
>>> Loads and stores would still have to be nybble addresses.
>>>
>>>
>>> --
>>> - Stephen Fuld
>>> (e-mail address disguised to prevent spam)
>
>> Yes, I thought of using just whole instructions for addressing.
>
> Of course, your architecture, so your decision. But I think you might
> have misunderstood some things.
>
>> But that creates issues with things like alignment.
>
> No, it doesn't. All alignment is exactly the same as it is under your
> plan. Only the contents of the displacement field in jumps, calls etc.
> instructions are changed.
>
>> It also makes data pointers and instruction pointers different.
>
> Yes, it does that.
>
>> With nybble addressing incrementing the IP is simple, just add 9.
>
> With instruction addressing, incrementing the IP is just add 1.

Right, everything is done with absolute or relative instruction numbers.
Just like if instructions were fixed size 32 bits.

Only the fetch unit has to know the program counter maps to byte addresses
by *4.5 (IP << 3 + IP) => 33 bit address with the upper bits [32:1]
being the byte address and lsb selecting lower or upper nibble.

>> The linker would have to align instructions on proper 36-bit
>> boundaries beginning at address zero.
>
> No. Instructions in memory are exactly the same as under your proposal.
> And you don't have to start at zero any more than you have to start
> nybble addressing at zero.
>
>
>> It would need to know the nybble address.
>
> No, it is easily computed.
>
>
>> It might be challenging to pick load addresses that are multiples of
>> 36 bits.
>
> I'm not sure what you mean here. Addresses in Load instructions are
> unaffected.
>
>
> It is just easier to use nybble addressing. The start address is defined
> as byte address $FFFC0300. Is that aligned to an even 36 bits or not?
>
> Who cares? The hardware transparently converts instruction address to
> nybble address upon a control transfer instruction.
>
> In the boot code there are a bunch of ‘.align’ directives to align
> things to the start of a cache line for small routines (align 64 bytes).
> There is no guarantee that the line starts at an even 36-bit address.
>
> Again, no one is proposing changing the memory layout, only the
> displacements in control transfer instructions.
>
>
> Branches support a 12-bit displacement. Since it is nybble addressing
> that is +-1kB. Still an adequate branch range for 95+% of branches.

The program counter is a 30-bit absolute instruction number
in the range 0.. 2^32/4.5 = 0..964,437,176.

Relative branches would be +/- 12-bit instruction number offset.
The number is just added to the current PC,
then the fetch unit takes care of mapping to bytes (PC << 3 + PC).

Absolute jump addresses would be a 30 bit instruction number.
As in "JMP reg" would only look at bits reg[29:0].

At PC = 964,437,177 the PC*4.5 wraps around in the byte address
space and probably does something random,
so probably best not to overflow the program counter.
But that's no different than usual.

> But not enough for many/most call type instructions.
>
>
>> The instruction aligner must shift the cache line by multiples of 36
>> bits, beginning at an arbitrary nybble position. It must do this
>> whether or not instructions are aligned on multiples of 36-bits. So,
>> there is no real savings in following 36-bit multiples.
>
>
> Again, you seem to be assuming that was not proposed. Actual placement
> in memory is not changed.

Linker needs to know about the mapping of instruction number to bytes
but that is no different from 32-bit instructions.

You can get an instruction that straddles a page boundary
but that is no different than byte aligned instructions.

On Sunday, June 20, 2021 at 1:33:06 PM UTC-5, EricP wrote:
> Stephen Fuld wrote:
> > On 6/19/2021 8:49 PM, robf...@gmail.com wrote:
> >> On Friday, June 18, 2021 at 12:10:42 PM UTC-4, Stephen Fuld wrote:
> >>> On 6/17/2021 9:12 PM, robf...@gmail.com wrote:
> >>>> Decided to go with 36-bit instructions. There is very little
> >>>> difference in code density due to the increased number of
> >>>> instructions that can be encoded in 36 bits over 34 bits. Memory is
> >>>> now nybble addressable. Displacements for branches, jumps and calls
> >>>> are in terms of nybbles.
> >>> If all instructions are fixed length of 36 bits, have you considered
> >>> making such displacements in terms of "instructions" rather than
> >>> nybbles? That would increase the range of values in such instructions
> >>> by a factor of nine (for 36 bit instructions). It adds a little cost to
> >>> the address calculation, but only an extra short add. It also makes
> >>> things "nice" for the assembler programmer - a jump over say three
> >>> instructions (e.g. to skip to the else part of an if-then-else) is coded
> >>> as current IP plus 4 instead of having to calculate nybbles.
> >>>
> >>> Loads and stores would still have to be nybble addresses.
> >>>
> >>>
> >>> --
> >>> - Stephen Fuld
> >>> (e-mail address disguised to prevent spam)
> >
> >> Yes, I thought of using just whole instructions for addressing.
> >
> > Of course, your architecture, so your decision. But I think you might
> > have misunderstood some things.
> >
> >> But that creates issues with things like alignment.
> >
> > No, it doesn't. All alignment is exactly the same as it is under your
> > plan. Only the contents of the displacement field in jumps, calls etc.
> > instructions are changed.
> >
> >> It also makes data pointers and instruction pointers different.
> >
> > Yes, it does that.
> >
> >> With nybble addressing incrementing the IP is simple, just add 9.
> >
> > With instruction addressing, incrementing the IP is just add 1.
> Right, everything is done with absolute or relative instruction numbers.
> Just like if instructions were fixed size 32 bits.
>
> Only the fetch unit has to know the program counter maps to byte addresses
> by *4.5 (IP << 3 + IP) => 33 bit address with the upper bits [32:1]
> being the byte address and lsb selecting lower or upper nibble.
> >> The linker would have to align instructions on proper 36-bit
> >> boundaries beginning at address zero.
> >
> > No. Instructions in memory are exactly the same as under your proposal.
> > And you don't have to start at zero any more than you have to start
> > nybble addressing at zero.
> >
> >
> >> It would need to know the nybble address.
> >
> > No, it is easily computed.
> >
> >
> >> It might be challenging to pick load addresses that are multiples of
> >> 36 bits.
> >
> > I'm not sure what you mean here. Addresses in Load instructions are
> > unaffected.
> >
> >
> > It is just easier to use nybble addressing. The start address is defined
> > as byte address $FFFC0300. Is that aligned to an even 36 bits or not?
> >
> > Who cares? The hardware transparently converts instruction address to
> > nybble address upon a control transfer instruction.
> >
> > In the boot code there are a bunch of ‘.align’ directives to align
> > things to the start of a cache line for small routines (align 64 bytes)..
> > There is no guarantee that the line starts at an even 36-bit address.
> >
> > Again, no one is proposing changing the memory layout, only the
> > displacements in control transfer instructions.
> >
> >
> > Branches support a 12-bit displacement. Since it is nybble addressing
> > that is +-1kB. Still an adequate branch range for 95+% of branches.
<
A 12-bit displacement enables ±2047*instructions = ±9,211 bytes..
<
> The program counter is a 30-bit absolute instruction number
> in the range 0.. 2^32/4.5 = 0..964,437,176.
<
Why not a 32-bit (or 64-bit) IP that indexes 4G instructions 19Gbytes.
Then keep the residual in another "place".
>
> Relative branches would be +/- 12-bit instruction number offset.
> The number is just added to the current PC,
> then the fetch unit takes care of mapping to bytes (PC << 3 + PC).
>
> Absolute jump addresses would be a 30 bit instruction number.
> As in "JMP reg" would only look at bits reg[29:0].
>
> At PC = 964,437,177 the PC*4.5 wraps around in the byte address
> space and probably does something random,
> so probably best not to overflow the program counter.
> But that's no different than usual.
> > But not enough for many/most call type instructions.
> >
> >
> >> The instruction aligner must shift the cache line by multiples of 36
> >> bits, beginning at an arbitrary nybble position. It must do this
> >> whether or not instructions are aligned on multiples of 36-bits. So,
> >> there is no real savings in following 36-bit multiples.
> >
> >
> > Again, you seem to be assuming that was not proposed. Actual placement
> > in memory is not changed.
> Linker needs to know about the mapping of instruction number to bytes
> but that is no different from 32-bit instructions.
>
> You can get an instruction that straddles a page boundary
> but that is no different than byte aligned instructions.
<
I would suggest you detect page crossings and not allow instructions to straddle
page boundaries. You get 910 instructions per page losing out on 1 byte but not
having a myriad of "other problems".

I was not trying to suggest that program loads must occur at address zero. I was also referring to program loads, not the load instruction. I meant only that things must be relative to the instruction address in terms of modular arithmetic if whole instructions are used as a reference point. Potential load addresses would be 0, 4.5, 9.0, 13.5, 18.0, …

I still think there could be alignment issues. (4.5 * IP) mod 16384 may not be zero assuming one wants to load program code at page boundaries. Moving code around is restrictive if it must be aligned to (4.5 * instruction address). I am not real fond of nybble addressing. In dumps etc. I have things expressed as byte addresses with a fraction of a byte. The IP is really a byte address with a fractional bit, so 4.5 bytes are being added to the IP. I do not see this as different from adding 4 bytes to the IP for a classic machine. It is just fixed-point arithmetic with one point. How much more hardware is it to add 4.5 instead of 1? It keeps data pointers and instruction pointers the same. To me, keeping the pointers the same may be valuable. What if one wants to place read-only data in a code page, then reference the data with IP relative addresses?

9x the branch range is appealing for capturing that extra 5% of branches. I may change this as suggested because within a subroutine addresses may be viewed in terms of sequential instructions - instruction counts, and conditional branches are almost exclusively used in subroutines. Unless one does something obtuse like insert an align statement to a boundary that is not a whole instruction away, there should be no issues with using instruction counts for branches.
Branches may also use a modifier that adds another 14 bits to the displacement range in addition to allowing the branch to act as a conditional subroutine call. The branch-and-link instruction has a 30-bit displacement packed into a single instruction.
I may leave jump and call instructions as fractional byte addressable so that code may be aligned at any half-byte address. So, code has the option of always starting at the start of a 16kB page. They have 30-bit displacement/address fields. “Wasting” a couple of bits for half-byte addressing probably is not an issue. The instructions are also extendable with instruction modifiers to have more addressing bits, for that 0.1% case where 30-bits is not enough.
I agree that it is relatively easy to convert the instruction address to a nybble address in memory.
The instruction pointer is currently 33 bits only because there’s only 512MB ram in the FPGA test system so, it does not make sense to use much over 1GB range for the test system. It saves a little bit of hardware if all the addresses are only 32 bits, but that is just for testing. But the IP register is spec’d as 64-bit. It is a 64-bit machine. (The size of an address is spec’d in one single config file location so it may be easily modified.). Only the order 24-bits of the IP increment. The rest of the bits are loadable.
I had similar thoughts about page crossing. The machine is setup to use 16kB pages as the smallest page size. 3640 instructions. 16kB is also the memory base register alignment. Currently the I$ is setup to load 548 bits per line, requiring a burst of five 128-bit accesses. This does not fit evenly in a memory page, and so cache loads do not cross page boundaries, I was thinking of restricting code addresses at the tail end of a page. I was not wanting to need to process a TLB miss in the middle of a cache load.

On Sunday, June 20, 2021 at 4:53:58 PM UTC-6, MitchAlsup wrote:

> I would suggest you detect page crossings and not allow instructions to straddle
> page boundaries. You get 910 instructions per page losing out on 1 byte but not
> having a myriad of "other problems".

I know that some machines were designed so as to take care to handle all the
problems that might arise. Thus, the IBM System/360 had instructions that were
16, 32, and 48 bits long - and page boundaries were essentially invisible to the
programmer.

Of course, it wasn't until the Model 85 that cache memory was introduced to
the line - and this was what forced IBM, in order to maintain compatibility, to
do whatever it took to ensure that if there was a page fault in the middle of
fetching an instruction... it was handled correctly.

John Savard