On Sunday, October 10, 2021 at 3:19:02 PM UTC-5, John Dallman wrote:
> In article <sjvesb$a36$1...@dont-email.me>, cr8...@gmail.com (BGB) wrote:
>
> > The design of the PBO ABI puts the responsibility of the GBR reload
> > onto the called function, which in turn is only needed in certain
> > cases: ...
> > Function pointers which are also non-leaf and/or access global
> > variables.
> How does the compiler know, when compiling a function, if it will be
> called via a pointer?
<
More importantly, why should it even know ?
<
double *trans_table[4] = {sin, cos, tan, atan};
....
x = trans_tasble[i](arg);
>
> John

MitchAlsup <MitchAlsup@aol.com> schrieb:
> On Sunday, October 10, 2021 at 1:39:45 PM UTC-5, Thomas Koenig wrote:
>> John Dallman <j...@cix.co.uk> schrieb:
>> > In article <sjv4u6$37u$1...@dont-email.me>, cr8...@gmail.com (BGB) wrote:
>> >
>> >> 32-bits, could comfortably fit one program
>> >> Maybe several of them, if they are "fairly modest"
>> >> 48 or 64 bits, could likely fit thousands of program instances.
>> >
>> >> This is excluding programs which reserve absurdly large address
>> >> ranges "because they can", but I consider this to be an aberration
>> >> rather than something which should be encouraged.
>> >
>> > Some programs do need to be able to handle sets of data, in RAM, that are
>> > too large to fit into 32-bit addressing. There aren't huge numbers of
>> > them, but they definitely exist.
>> Compuational fluid dynamics.
>>
>> With today's problems, with 32-bit addressing (well, usually less
>> than that) you can just go home.
><
> A decade ago (can it be that long) I was doing 1D CFD on intake and
> exhaust systems for automotive engines.

That's hardly CFD these days :-) 1D unsteady, I assume?

> These applications would
> sometimes run for several days (24/7) on my 6 core system, but
> never used "that much memory". The operating mesh had hundreds of
> thousands of boundary constraints (mostly pipe walls).

Most of the CPU time was spent doing some wildly non-linear
things, I assume?

><
> Perhaps 2D CFD and 3D CFD exponentiate my experience.......

Most serious CFD is 3D. A good rule of thumb for finite volume
is that you need 4 to 8 GB per core these days. A 32-core system
will then need between 64 and 128 GB.

The workstation below my desk at work has 512 GB main memory, but
I rarely use that. This is for a finite element code which has
has an option for direct solvers. They are far more stable than
the iterative ones, but _very_ memory hungry if the problem
is big. Still, 256GB would in all probability have been enough.

On 10/10/2021 3:18 PM, John Dallman wrote:
> In article <sjvesb$a36$1@dont-email.me>, cr88192@gmail.com (BGB) wrote:
>
>> The design of the PBO ABI puts the responsibility of the GBR reload
>> onto the called function, which in turn is only needed in certain
>> cases: ...
>> Function pointers which are also non-leaf and/or access global
>> variables.
>
> How does the compiler know, when compiling a function, if it will be
> called via a pointer?
>

In this case, it is because the code-generator does a graph-walk and
marks the function as having had its address taken before this point.

This in turn works because my compiler does all of the code generation
for a program at the same time in the back-end, rather than using object
files and a linker.

This has the advantage that, at code-generation time, it is possible to
check whether or not something is reachable via searching a global graph
structure representing the program (and whether or not it can be
potentially accessible from outside the current binary).

For separate compilation and static libraries, it uses a format I call
RIL3, which is basically a stack-based bytecode IR (along vaguely
similar lines to .NET bytecode).

The RIL3 format is essentially a single linear stack machine, which
itself builds all of the metadata via stack operations. By high-level
analogy, it is sort of like if PostScript was used to drive an OpenGL
style API which in turn builds a bunch of compiler metadata, ...

In effect, the RIL3 loader is a fairly naive bytecode interpreter
(albeit, at this stage, the bytecode format is not Turing complete).

I had at one point considered a fancier RIL variant that would have
added PostScript style block-structuring and a Turing-complete execution
model, but didn't go that direction. In the years since then, I have
come to the opinion that having a Turing complete interpreter as a
compiler IR loader would probably not be a particularly great idea (my
younger self was more into this sorta stuff though).

So, Front End:
Preprocess;
Parse;
Translate AST -> RIL3;
Optionally dump RIL3 to a file.

Back End:
Load RIL3 modules, translating function bytecode into a 3AC format;
Determine what is reachable, ...
Pass to determine memory model (estimates section sizes, ...)
Several mock-up code-generation passes;
Needed to sort out things like branch displacements and reg alloc;
Final code-generation pass (generates actual binary code);
Layout sections within binary image;
Apply symbolic relocations, generate base reloc table;
Generate a PE/COFF (PEL) image (albeit lacking the MZ stub);
LZ compress PE image;
Store output binary.

Some things could be better, some amount is from historical vestigial
structures.

Note that some things, such as ASM files and inline ASM, will typically
be passed through the RIL3 stage essentially as big ASCII text blobs (or
string literals), with a bytecode op that is like "Yeah, that big blob
of text on the top of the stack here, that is a blob of ASM code..." (at
the top-level, the ASM blob is popped off the stack and added as its own
"global object", or inside a function, emits a 3AC IR op that invokes
the ASM parser on the given string literal).

....

On 10/10/2021 3:59 PM, MitchAlsup wrote:
> On Sunday, October 10, 2021 at 3:19:02 PM UTC-5, John Dallman wrote:
>> In article <sjvesb$a36$1...@dont-email.me>, cr8...@gmail.com (BGB) wrote:
>>
>>> The design of the PBO ABI puts the responsibility of the GBR reload
>>> onto the called function, which in turn is only needed in certain
>>> cases: ...
>>> Function pointers which are also non-leaf and/or access global
>>> variables.
>> How does the compiler know, when compiling a function, if it will be
>> called via a pointer?
> <
> More importantly, why should it even know ?
> <
> double *trans_table[4] = {sin, cos, tan, atan};
> ...
> x = trans_tasble[i](arg);

I guess this points out a potential drawback of the design:
Taking the address of a function in one place may cause it to get slower
everywhere else...

>>
>> John

On Sunday, October 10, 2021 at 4:27:30 PM UTC-5, Thomas Koenig wrote:
> MitchAlsup <Mitch...@aol.com> schrieb:
> > On Sunday, October 10, 2021 at 1:39:45 PM UTC-5, Thomas Koenig wrote:
> >> John Dallman <j...@cix.co.uk> schrieb:
> >> > In article <sjv4u6$37u$1...@dont-email.me>, cr8...@gmail.com (BGB) wrote:
> >> >
> >> >> 32-bits, could comfortably fit one program
> >> >> Maybe several of them, if they are "fairly modest"
> >> >> 48 or 64 bits, could likely fit thousands of program instances.
> >> >
> >> >> This is excluding programs which reserve absurdly large address
> >> >> ranges "because they can", but I consider this to be an aberration
> >> >> rather than something which should be encouraged.
> >> >
> >> > Some programs do need to be able to handle sets of data, in RAM, that are
> >> > too large to fit into 32-bit addressing. There aren't huge numbers of
> >> > them, but they definitely exist.
> >> Compuational fluid dynamics.
> >>
> >> With today's problems, with 32-bit addressing (well, usually less
> >> than that) you can just go home.
> ><
> > A decade ago (can it be that long) I was doing 1D CFD on intake and
> > exhaust systems for automotive engines.
> That's hardly CFD these days :-) 1D unsteady, I assume?
<
I certainly had to use many-many surfaces to adequately model velocity
stacks--so something 9" long had 45 individual set of diameters.
<
> > These applications would
> > sometimes run for several days (24/7) on my 6 core system, but
> > never used "that much memory". The operating mesh had hundreds of
> > thousands of boundary constraints (mostly pipe walls).
> Most of the CPU time was spent doing some wildly non-linear
> things, I assume?
> ><
> > Perhaps 2D CFD and 3D CFD exponentiate my experience.......
<
> Most serious CFD is 3D.
<
I can certainly agree with this notion, however, I had Free ($0.00) access
to a 1D modeler that was already setup to do "about" what I wanted to do
(including the combustion physics); so I used what was easily available.
<
> A good rule of thumb for finite volume
> is that you need 4 to 8 GB per core these days. A 32-core system
> will then need between 64 and 128 GB.
>
> The workstation below my desk at work has 512 GB main memory, but
> I rarely use that. This is for a finite element code which has
> has an option for direct solvers. They are far more stable than
> the iterative ones, but _very_ memory hungry if the problem
> is big. Still, 256GB would in all probability have been enough.

>> >> > Some programs do need to be able to handle sets of data, in RAM, that are
>> >> > too large to fit into 32-bit addressing. There aren't huge numbers of
>> >> > them, but they definitely exist.

I find this discussion quite surprising. On the one hand, I'm still
using the i386 port of Debian on most/all of my machines, so clearly all
my applications fit within that 32bit address space, but I'd have expected it's
quite common for numerical simulations to need a fair bit more.

After all, my Firefox process often gets in the GB range, my Emacs needs
at least as much RAM as the size of the files it edits (and log files >4GB
aren't that rare), and even smartphones come with 8GB these days.

So is it really still rare(ish) to pass the 4GB limit?

Stefan

On 10/10/2021 3:07 PM, Stefan Monnier wrote:
>>>>>> Some programs do need to be able to handle sets of data, in RAM, that are
>>>>>> too large to fit into 32-bit addressing. There aren't huge numbers of
>>>>>> them, but they definitely exist.
>
> I find this discussion quite surprising. On the one hand, I'm still
> using the i386 port of Debian on most/all of my machines, so clearly all
> my applications fit within that 32bit address space, but I'd have expected it's
> quite common for numerical simulations to need a fair bit more.
>
> After all, my Firefox process often gets in the GB range, my Emacs needs
> at least as much RAM as the size of the files it edits (and log files >4GB
> aren't that rare), and even smartphones come with 8GB these days.
>
> So is it really still rare(ish) to pass the 4GB limit?
>

Volumetric renders on very large stacks can be very memory intensive.
Say a 2048^3 volume.

On Sunday, October 10, 2021 at 5:07:25 PM UTC-5, Stefan Monnier wrote:
> >> >> > Some programs do need to be able to handle sets of data, in RAM, that are
> >> >> > too large to fit into 32-bit addressing. There aren't huge numbers of
> >> >> > them, but they definitely exist.
> I find this discussion quite surprising. On the one hand, I'm still
> using the i386 port of Debian on most/all of my machines, so clearly all
> my applications fit within that 32bit address space, but I'd have expected it's
> quite common for numerical simulations to need a fair bit more.
>
> After all, my Firefox process often gets in the GB range, my Emacs needs
> at least as much RAM as the size of the files it edits (and log files >4GB
> aren't that rare), and even smartphones come with 8GB these days.
>
> So is it really still rare(ish) to pass the 4GB limit?
<
When virtual memory is working well, 4GB can hold a lot of processes.
<
We used to run cycle-accurate CPU simulations giving the CPU 4GB
of physical memory on machines with ¼GB of actual memory running
a full UNIX workload on top of the simulations we ran.
<
On the other hand, many workloads have "workarounds" that are perfectly
adequate for the application at hand. CRAY used to get CPU/machine
orders for systems where the numeric performance was behind NEC
supercomputers, but where the CRAY I/O system could sustain more
I/O (double buffered data stream I/O) and thus attack larger problems
than the NEC machines.
>
>
> Stefan

On 10/10/2021 2:27 PM, Thomas Koenig wrote:
> MitchAlsup <MitchAlsup@aol.com> schrieb:
>> On Sunday, October 10, 2021 at 1:39:45 PM UTC-5, Thomas Koenig wrote:
>>> John Dallman <j...@cix.co.uk> schrieb:
>>>> In article <sjv4u6$37u$1...@dont-email.me>, cr8...@gmail.com (BGB) wrote:
>>>>
>>>>> 32-bits, could comfortably fit one program
>>>>> Maybe several of them, if they are "fairly modest"
>>>>> 48 or 64 bits, could likely fit thousands of program instances.
>>>>
>>>>> This is excluding programs which reserve absurdly large address
>>>>> ranges "because they can", but I consider this to be an aberration
>>>>> rather than something which should be encouraged.
>>>>
>>>> Some programs do need to be able to handle sets of data, in RAM, that are
>>>> too large to fit into 32-bit addressing. There aren't huge numbers of
>>>> them, but they definitely exist.
>>> Compuational fluid dynamics.
>>>
>>> With today's problems, with 32-bit addressing (well, usually less
>>> than that) you can just go home.
>> <
>> A decade ago (can it be that long) I was doing 1D CFD on intake and
>> exhaust systems for automotive engines.
>
> That's hardly CFD these days :-) 1D unsteady, I assume?
>
>> These applications would
>> sometimes run for several days (24/7) on my 6 core system, but
>> never used "that much memory". The operating mesh had hundreds of
>> thousands of boundary constraints (mostly pipe walls).
>
> Most of the CPU time was spent doing some wildly non-linear
> things, I assume?
>
>> <
>> Perhaps 2D CFD and 3D CFD exponentiate my experience.......
>
> Most serious CFD is 3D. A good rule of thumb for finite volume
> is that you need 4 to 8 GB per core these days. A 32-core system
> will then need between 64 and 128 GB.
>
> The workstation below my desk at work has 512 GB main memory, but
> I rarely use that. This is for a finite element code which has
> has an option for direct solvers. They are far more stable than
> the iterative ones, but _very_ memory hungry if the problem
> is big. Still, 256GB would in all probability have been enough.
>

That's what- 40 bits? So a 64-bit shared address space can accommodate
2^24 of such apps, concurrently?

Clearly inadequate.

On 10/10/2021 11:54 AM, MitchAlsup wrote:
> On Sunday, October 10, 2021 at 1:33:59 PM UTC-5, John Dallman wrote:
>> In article <sjv4u6$37u$1...@dont-email.me>, cr8...@gmail.com (BGB) wrote:
>>
>>> 32-bits, could comfortably fit one program
>>> Maybe several of them, if they are "fairly modest"
>>> 48 or 64 bits, could likely fit thousands of program instances.
>>> This is excluding programs which reserve absurdly large address
>>> ranges "because they can", but I consider this to be an aberration
>>> rather than something which should be encouraged.
> <
>> Some programs do need to be able to handle sets of data, in RAM, that are
>> too large to fit into 32-bit addressing. There aren't huge numbers of
>> them, but they definitely exist.
> <
> In a 64-bit virtual address space, the user should be able to create holes
> in the virtual address space as large as 2^62 (I might be argued down to
> 2^60). Certainly no OS needs to grow bigger than 2^53......

Why?

On 10/10/2021 11:47 AM, MitchAlsup wrote:
> On Sunday, October 10, 2021 at 12:56:25 PM UTC-5, Ivan Godard wrote:
>> On 10/10/2021 9:57 AM, MitchAlsup wrote:
>>> On Sunday, October 10, 2021 at 10:10:31 AM UTC-5, Ivan Godard wrote:
>>>> On 10/10/2021 7:31 AM, EricP wrote:
>>>>> Ivan Godard wrote:
>>>>>> On 10/8/2021 11:25 PM, Stephen Fuld wrote:
>>>>>>>
>>>>>>> OK. This is generally called "Single Address Space". This is used
>>>>>>> by such systems as the IBM System i, and the Mill.
>>>>>>>
>>>>>>>> Granted, this does have the (fairly obvious) property that most
>>>>>>>> traditional schemes for memory protection, ..., kinda go out the
>>>>>>>> window when one has multiple logical processes within the same
>>>>>>>> address space (as do the traditional distinctions between threads
>>>>>>>> and processes).
>>>>>>>
>>>>>>> Not necessarily. Just because multiple logical processes exist
>>>>>>> within the same address space, doesn't mean that each process has
>>>>>>> legal access to all the addresses within that space.
>>>>>>
>>>>>> An address space tells you what you can name; a protection domain
>>>>>> tells you whether naming it does you any good.
>>>>>
>>>>> What exactly is the distinction between a single address space
>>>>> with page protection that allows multiple threads in that address space
>>>>> access to some of those pages, and multiple address spaces with multiple
>>>>> threads that allows access to some of those pages.
>>>>>
>>>>> Because it sure sounds like the difference is just whether or not there is
>>>>> a page-table-base pointer change on process switch, and if the TLB
>>>>> supports ASID's Address Space ID's then there is no performance difference.
>>>>>
>>>>> In other words, one page table with PTE's with fancy per-thread protection
>>>>> keys that change when threads switch, is the same as multiple page tables
>>>>> with PTE's with simple protection that map different memory sections.
>>>>>
>>>>> Where is the functional distinction?
>>>>> What can one do that the other can't?
>>> <
>>>> A single address space permits VMAX (typically 2^64) of virtual total,
>>>> system wide, for all processes. A multiple address space permits TMAX
>>>> (typically 2^16 or so)*VMAX total of virtual, system wide.
>>> <
>>> Mc 88200 supported 2×VMAX {code and data could be in different VASs.}
>>> Nobody used it that way.
>>> Everybody used it as ½×VMAX {lower ½ was user upper ½ was supervisor}.
>>>>
>>>> That's the historical reason that mapping is used: so a 16-bit system
>>>> could have more that 16 bits worth of combined system and appp code
>>>> running. 64-bit spaces are not (yet) constraining enough to need
>>>> multiple spaces..
>>> <
>>> PDP-11/70 had 16-bit user code+data and 16-bit supervisor code+data VASs.
>>> And LD-From and ST-To instructions access.
>>> <
>>> I am coming around to this means of accessing another VAS; indeed, this seems
>>> to be the cleanest way to implement ADA rendezvous where the caller and the
>>> acceptor can be in different address spaces, running at different priorities, on
>>> different cores, possibly different machines (i.e., over a network).
>>> <
>>> Acceptor has access to the callers VAS+arguments and can write acceptor
>>> results back to callers VAS+results.
>>> <
>>> About the only thing this does not provide is access across different virtual
>>> machines--at least directly.
>>>
>> How do you bound the access to only the argument (var. result) regions?
> <
> I am expecting ADA to be reasonable (i.e., ADA), here. During argument evaluation
> ADA will not create a dope vector that exceeds the actual passed data area (checked
> on the way out). During use of argument the acceptor will not violate (checked on the
> way in) the dope vector constraints.

Not asking what *will* get checked; asking what *can avoid* being
checked, inadvertently or maliciously. What prevents accessing (in or
out) out side what is described by the dope vector? Or are you assuming
that nothing but correctly compiled code can run, a la Unisys?
>>
>> How do you pass an object by reference? (e.g. an out-of-regs shared buffer)
> <
> Argument passed as in normal ABI (other than the caller knowing the call is to an
> accept), the caller evaluates arguments just like any other call. The process of
> rendezvous passes a Root pointer and a pointer to the <now stored> caller register
> file in places (control registers) only an acceptor is taught where to look (by the
> compiler). I may make access to these control registers raise exceptions if not
> in an Accept (it is not hard to tell). (I probably should........)
>

Is there both root and upper bound? How does the extent get into the TLB
for a bounds check? What is the granularity? How do I pass the buffer
pointed to by an argument? How do I pass all 10k nodes of a tree rooted
in a pointer passed as an argument?

Detailed description please.

On 10/10/2021 5:46 PM, Ivan Godard wrote:
> On 10/10/2021 2:27 PM, Thomas Koenig wrote:
>> MitchAlsup <MitchAlsup@aol.com> schrieb:
>>> On Sunday, October 10, 2021 at 1:39:45 PM UTC-5, Thomas Koenig wrote:
>>>> John Dallman <j...@cix.co.uk> schrieb:
>>>>> In article <sjv4u6$37u$1...@dont-email.me>, cr8...@gmail.com (BGB)
>>>>> wrote:
>>>>>
>>>>>> 32-bits, could comfortably fit one program
>>>>>> Maybe several of them, if they are "fairly modest"
>>>>>> 48 or 64 bits, could likely fit thousands of program instances.
>>>>>
>>>>>> This is excluding programs which reserve absurdly large address
>>>>>> ranges "because they can", but I consider this to be an aberration
>>>>>> rather than something which should be encouraged.
>>>>>
>>>>> Some programs do need to be able to handle sets of data, in RAM,
>>>>> that are
>>>>> too large to fit into 32-bit addressing. There aren't huge numbers of
>>>>> them, but they definitely exist.
>>>> Compuational fluid dynamics.
>>>>
>>>> With today's problems, with 32-bit addressing (well, usually less
>>>> than that) you can just go home.
>>> <
>>> A decade ago (can it be that long) I was doing 1D CFD on intake and
>>> exhaust systems for automotive engines.
>>
>> That's hardly CFD these days :-)  1D unsteady, I assume?
>>
>>> These applications would
>>> sometimes run for several days (24/7) on my 6 core system, but
>>> never used "that much memory". The operating mesh had hundreds of
>>> thousands of boundary constraints (mostly pipe walls).
>>
>> Most of the CPU time was spent doing some wildly non-linear
>> things, I assume?
>>
>>> <
>>> Perhaps 2D CFD and 3D CFD exponentiate my experience.......
>>
>> Most serious CFD is 3D.   A good rule of thumb for finite volume
>> is that you need 4 to 8 GB per core these days.  A 32-core system
>> will then need between 64 and 128 GB.
>>
>> The workstation below my desk at work has 512 GB main memory, but
>> I rarely use that.  This is for a finite element code which has
>> has an option for direct solvers.  They are far more stable than
>> the iterative ones, but _very_ memory hungry if the problem
>> is big.  Still, 256GB would in all probability have been enough.
>>
>
> That's what- 40 bits? So a 64-bit shared address space can accommodate
> 2^24 of such apps, concurrently?
>
> Clearly inadequate.

Yeah, even the current 48-bit addressing scheme in BJX2 would be able to
accommodate 100 program instances this size.

For "typical sized" programs that would fit within a 32-bit machine, one
could reasonably fit upward of 120,000 of them in a single address
space, or roughly about 8 million Doom instances...

On 10/10/2021 3:46 PM, Ivan Godard wrote:
> On 10/10/2021 2:27 PM, Thomas Koenig wrote:
>> MitchAlsup <MitchAlsup@aol.com> schrieb:
>>> On Sunday, October 10, 2021 at 1:39:45 PM UTC-5, Thomas Koenig wrote:
>>>> John Dallman <j...@cix.co.uk> schrieb:
>>>>> In article <sjv4u6$37u$1...@dont-email.me>, cr8...@gmail.com (BGB)
>>>>> wrote:
>>>>>
>>>>>> 32-bits, could comfortably fit one program
>>>>>> Maybe several of them, if they are "fairly modest"
>>>>>> 48 or 64 bits, could likely fit thousands of program instances.
>>>>>
>>>>>> This is excluding programs which reserve absurdly large address
>>>>>> ranges "because they can", but I consider this to be an aberration
>>>>>> rather than something which should be encouraged.
>>>>>
>>>>> Some programs do need to be able to handle sets of data, in RAM,
>>>>> that are
>>>>> too large to fit into 32-bit addressing. There aren't huge numbers of
>>>>> them, but they definitely exist.
>>>> Compuational fluid dynamics.
>>>>
>>>> With today's problems, with 32-bit addressing (well, usually less
>>>> than that) you can just go home.
>>> <
>>> A decade ago (can it be that long) I was doing 1D CFD on intake and
>>> exhaust systems for automotive engines.
>>
>> That's hardly CFD these days :-)  1D unsteady, I assume?
>>
>>> These applications would
>>> sometimes run for several days (24/7) on my 6 core system, but
>>> never used "that much memory". The operating mesh had hundreds of
>>> thousands of boundary constraints (mostly pipe walls).
>>
>> Most of the CPU time was spent doing some wildly non-linear
>> things, I assume?
>>
>>> <
>>> Perhaps 2D CFD and 3D CFD exponentiate my experience.......
>>
>> Most serious CFD is 3D.   A good rule of thumb for finite volume
>> is that you need 4 to 8 GB per core these days.  A 32-core system
>> will then need between 64 and 128 GB.
>>
>> The workstation below my desk at work has 512 GB main memory, but
>> I rarely use that.  This is for a finite element code which has
>> has an option for direct solvers.  They are far more stable than
>> the iterative ones, but _very_ memory hungry if the problem
>> is big.  Still, 256GB would in all probability have been enough.
>>
>
> That's what- 40 bits? So a 64-bit shared address space can accommodate
> 2^24 of such apps, concurrently?
>
> Clearly inadequate.

I don't think the question of whether 64 bits is adequate (it is), but
for how long into the future will it be adequate, and secondarily, when
it does become inadequate, how big a change, hardware and software, will
be required to deal with it?

Remember, not allowing a large enough address space is considered by
many to be the biggest mistake in computer architecture. :-(

--
- Stephen Fuld
(e-mail address disguised to prevent spam)

> I don't think the question of whether 64 bits is adequate (it is), but for
> how long into the future will it be adequate, and secondarily, when it does
> become inadequate, how big a change, hardware and software, will be required
> to deal with it?
> Remember, not allowing a large enough address space is considered by many to
> be the biggest mistake in computer architecture. :-(

I wonder, indeed. It seems that the amount of RAM per CPU has been
growing fairly slowly of late. Even for those applications which
require >4GB of memory, many of them could still fairly easily be
reorganized into separate processes that each fit within 4GB (tho the
effort would make no sense currently).

I suspect that we should be able to live within 64bit per-process for
a very long time to come. The question being mostly whether the
convenience of a single address space shared between all those
processes/threads will be enough to justify moving to longer pointers,
or if instead compilers will at that time be able to hide easily enough
those details and offer the illusion of a single shared memory space
while dividing the application into a set of separate processes running
in separate address spaces.

I sometimes have dreams of machines that evolved back into systems with
far pointers (128bit, or even just URLs) and near pointers (e.g 32bit,
just to refer to the last-level cache, everything above that is accessed
as "disk blocks").

Stefan

On Sunday, October 10, 2021 at 6:00:40 PM UTC-5, Ivan Godard wrote:
> On 10/10/2021 11:47 AM, MitchAlsup wrote:
> > On Sunday, October 10, 2021 at 12:56:25 PM UTC-5, Ivan Godard wrote:
> >> On 10/10/2021 9:57 AM, MitchAlsup wrote:

> >>> I am coming around to this means of accessing another VAS; indeed, this seems
> >>> to be the cleanest way to implement ADA rendezvous where the caller and the
> >>> acceptor can be in different address spaces, running at different priorities, on
> >>> different cores, possibly different machines (i.e., over a network).
> >>> <
> >>> Acceptor has access to the callers VAS+arguments and can write acceptor
> >>> results back to callers VAS+results.
> >>> <
> >>> About the only thing this does not provide is access across different virtual
> >>> machines--at least directly.
> >>>
> >> How do you bound the access to only the argument (var. result) regions?
> > <
> > I am expecting ADA to be reasonable (i.e., ADA), here. During argument evaluation
> > ADA will not create a dope vector that exceeds the actual passed data area (checked
> > on the way out). During use of argument the acceptor will not violate (checked on the
> > way in) the dope vector constraints.
<
> Not asking what *will* get checked; asking what *can avoid* being
> checked, inadvertently or maliciously. What prevents accessing (in or
> out) out side what is described by the dope vector? Or are you assuming
> that nothing but correctly compiled code can run, a la Unisys?
<
When an argument to a subroutine arrives in a register, there is a way to move
that data from where it is stored in memory to a register in the accept call-point.
<
When data is not scalar, ADA builds a dope vector describing the access
characteristics of the non-scalar data on the way out.
<
When data is not scalar ADA uses the dope vector properly, and much of the
time, the way good ADA is written, the checks are not necessary, just like if
there was no call-accept sitting between the semantics of the ASCII text
and the compiled code.
<
> >>
> >> How do you pass an object by reference? (e.g. an out-of-regs shared buffer)
> > <
> > Argument passed as in normal ABI (other than the caller knowing the call is to an
> > accept), the caller evaluates arguments just like any other call. The process of
> > rendezvous passes a Root pointer and a pointer to the <now stored> caller register
> > file in places (control registers) only an acceptor is taught where to look (by the
> > compiler). I may make access to these control registers raise exceptions if not
> > in an Accept (it is not hard to tell). (I probably should........)
> >
> Is there both root and upper bound?
<
There is a dope vector into the foreign address space, and there is a root for the
translation tables in the foreign address space. The dope vector contains a base
pointer and other information used to determine the extent of the non-scalar data.
<
> How does the extent get into the TLB
> for a bounds check?
<
The TLB merely performs the translation and gives the acceptor the same access
rights as the caller. The Dope vector and ADA generated instructions perform the
bounds checking. My 66000 has an efficient bounds check in the CMP instruction.
<
< What is the granularity?
Byte.
> How do I pass the buffer
> pointed to by an argument?
<
In a dope vector which contains a pointer (base) and a bounds.
<
< How do I pass all 10k nodes of a tree rooted
> in a pointer passed as an argument?
<
Pass a pointer to a type of tree-node.
>
> Detailed description please.

On Sunday, October 10, 2021 at 6:31:22 PM UTC-5, Stephen Fuld wrote:
> On 10/10/2021 3:46 PM, Ivan Godard wrote:
> > On 10/10/2021 2:27 PM, Thomas Koenig wrote:
> >> MitchAlsup <Mitch...@aol.com> schrieb:
> >>> On Sunday, October 10, 2021 at 1:39:45 PM UTC-5, Thomas Koenig wrote:
> >>>> John Dallman <j...@cix.co.uk> schrieb:
> >>>>> In article <sjv4u6$37u$1...@dont-email.me>, cr8...@gmail.com (BGB)
> >>>>> wrote:
> >>>>>
> >>>>>> 32-bits, could comfortably fit one program
> >>>>>> Maybe several of them, if they are "fairly modest"
> >>>>>> 48 or 64 bits, could likely fit thousands of program instances.
> >>>>>
> >>>>>> This is excluding programs which reserve absurdly large address
> >>>>>> ranges "because they can", but I consider this to be an aberration
> >>>>>> rather than something which should be encouraged.
> >>>>>
> >>>>> Some programs do need to be able to handle sets of data, in RAM,
> >>>>> that are
> >>>>> too large to fit into 32-bit addressing. There aren't huge numbers of
> >>>>> them, but they definitely exist.
> >>>> Compuational fluid dynamics.
> >>>>
> >>>> With today's problems, with 32-bit addressing (well, usually less
> >>>> than that) you can just go home.
> >>> <
> >>> A decade ago (can it be that long) I was doing 1D CFD on intake and
> >>> exhaust systems for automotive engines.
> >>
> >> That's hardly CFD these days :-) 1D unsteady, I assume?
> >>
> >>> These applications would
> >>> sometimes run for several days (24/7) on my 6 core system, but
> >>> never used "that much memory". The operating mesh had hundreds of
> >>> thousands of boundary constraints (mostly pipe walls).
> >>
> >> Most of the CPU time was spent doing some wildly non-linear
> >> things, I assume?
> >>
> >>> <
> >>> Perhaps 2D CFD and 3D CFD exponentiate my experience.......
> >>
> >> Most serious CFD is 3D. A good rule of thumb for finite volume
> >> is that you need 4 to 8 GB per core these days. A 32-core system
> >> will then need between 64 and 128 GB.
> >>
> >> The workstation below my desk at work has 512 GB main memory, but
> >> I rarely use that. This is for a finite element code which has
> >> has an option for direct solvers. They are far more stable than
> >> the iterative ones, but _very_ memory hungry if the problem
> >> is big. Still, 256GB would in all probability have been enough.
> >>
> >
> > That's what- 40 bits? So a 64-bit shared address space can accommodate
> > 2^24 of such apps, concurrently?
> >
> > Clearly inadequate.
<
> I don't think the question of whether 64 bits is adequate (it is), but
> for how long into the future will it be adequate,
<
First notice that the hard limit only applies to single address space architectures.
<
Secondarily notice: I am only worried about "the rest of my lifetime" time line.
<
> and secondarily, when
> it does become inadequate, how big a change, hardware and software, will
> be required to deal with it?
<
I think the transition from 64-bits to whatever will be greater than the
the transition from 32-bits to 64-bits. The next power of 2 is way bigger
than necessary 2^128 (10^38.5) while there are only 10^49 atoms on the
planet.
<
Non-powers of 2 are problematic; but I don't think I have to worry about this.
<
Main memory growth seems to have slowed.........a bit......
>
> Remember, not allowing a large enough address space is considered by
> many to be the biggest mistake in computer architecture. :-(
<
I remember, and I knew the person who first stated that.
>
>
>
>
> --
> - Stephen Fuld
> (e-mail address disguised to prevent spam)

On Sunday, October 10, 2021 at 6:51:09 PM UTC-5, Stefan Monnier wrote:
> > I don't think the question of whether 64 bits is adequate (it is), but for
> > how long into the future will it be adequate, and secondarily, when it does
> > become inadequate, how big a change, hardware and software, will be required
> > to deal with it?
> > Remember, not allowing a large enough address space is considered by many to
> > be the biggest mistake in computer architecture. :-(
<
> I wonder, indeed. It seems that the amount of RAM per CPU has been
> growing fairly slowly of late. Even for those applications which
> require >4GB of memory, many of them could still fairly easily be
> reorganized into separate processes that each fit within 4GB (tho the
> effort would make no sense currently).
>
> I suspect that we should be able to live within 64bit per-process for
> a very long time to come. The question being mostly whether the
> convenience of a single address space shared between all those
> processes/threads will be enough to justify moving to longer pointers,
<
Note:: something like 98% of all applications only need 32-bits as of today.
<
Also note:: both of the big architectures are not SaS.
<
> or if instead compilers will at that time be able to hide easily enough
> those details and offer the illusion of a single shared memory space
> while dividing the application into a set of separate processes running
> in separate address spaces.
<
DataBase experience says no.
>
> I sometimes have dreams of machines that evolved back into systems with
> far pointers (128bit, or even just URLs) and near pointers (e.g 32bit,
> just to refer to the last-level cache, everything above that is accessed
> as "disk blocks").
<
URLs are the "solves everything" except performance hammer.
>
>
> Stefan

BGB wrote:
> On 10/10/2021 2:00 PM, John Levine wrote:
>> According to Ivan Godard <ivan@millcomputing.com>:
>>>> Where is the functional distinction?
>>>> What can one do that the other can't?
>>>
>>> A single address space permits VMAX (typically 2^64) of virtual total,
>>> system wide, for all processes. A multiple address space permits TMAX
>>> (typically 2^16 or so)*VMAX total of virtual, system wide.
>>>
>>> That's the historical reason that mapping is used: so a 16-bit system
>>> could have more that 16 bits worth of combined system and appp code
>>> running. 64-bit spaces are not (yet) constraining enough to need
>>> multiple spaces..
>>
>> That's one reason, the other flips it around, so the application
>> address space could be
>> bigger than physical memory, and you fake it with paging. You
>> probably remember static
>> shared libraries, which loaded at the same place in every process,
>> something that needs
>> an address space per process to work.
>>
>> It is my impression that once you have enough hardware to handle
>> paging in one application
>> address space, multiple address spaces aren't much harder.
>>
>
> Harder... No.
> Slower... Yes.
>
> As noted, in my case there is little stopping the MMU from being used
> for multiple address spaces. It is a little more awkward for an OS
> kernel, but doesn't really effect the HW all that much.
>
> However, the more obvious issue is that, as soon as one switches address
> spaces, none of the contents of the TLB are relevant anyone. So, every
> task switch is very likely to be followed by potentially a few thousand
> cycles worth of TLB reloads.
>
> If it is a single address space, then it is likely that any shared
> contents (such as OS provided libraries, ...) will still be in the TLB
> following a task switch.
>
>
>
> An ASID could help, if one assumes that some of the old contents remain
> in the TLB by the time the scheduler cycles back around.
>
> Another possible option though could be a hierarchical ASID, say:
> ASID: 6b=AGID, 10b=APID
> Where a TLBE:
> ASID=0, Globally Valid
> AGID!=0, APID=0: Valid so long as AGID matches.
> AGID!=0, APID!=0: Valid if ASID's are equal.
>
> With the ASID being in the same encoding space as VUGID.
>
> The ASID bits are partly co-opted in the split TLBE scheme for 96-bit
> addressing though, so it is likely that in 96-bit mode the MMU would not
> have ASID's (well, along with some debate as to whether it would reduce
> TLB associativity, or add a separate smaller TLB for matching address
> bits 95:48...).
>

The ASID doesn't have to be very large because they can be assigned
on a per-core basis, as opposed to globally unique ASIDs,
and you can recycle the numbers for each core.

Say it has 4 kB pages and full 64 bit address so the TLB
has a fully assoc table with 52 bit virtual addresses.

A 4 bit ASID tags the TLB VA entries for the most recent 16 address
spaces mapped on that core. When the OS recycles an ASID it purges
that cores' TLB entries of just that ASID number before reusing.

When a process is mapped in a core, its page table base address is set
in the MMU base register, and the assigned ASID is set in a ID register.
Each time a PTE is loaded, if it is a per-process PTE then the current
ID register is copied to the new TLB entry ASID field.

OS needs a little code to track which ASID's for each process
were assigned on each core but nothing complicated.

According to BGB <cr88192@gmail.com>:
>Harder... No.
>Slower... Yes.
>
>As noted, in my case there is little stopping the MMU from being used
>for multiple address spaces. It is a little more awkward for an OS
>kernel, but doesn't really effect the HW all that much.
>
>However, the more obvious issue is that, as soon as one switches address
>spaces, none of the contents of the TLB are relevant anyone. So, every
>task switch is very likely to be followed by potentially a few thousand
>cycles worth of TLB reloads. ...

There are approaches to this issue going back at least to the 1980s.
They're variations on putting a some sort of process or segment ID
in the TLB so for the TLB it's one big address space, even though
for programs they are separate.

The IBM ROMP and I think the early POWER chips divided the 32 bit
address into a 4 bit segment number and 28 bit offset, with the segment
number being in index into a much larger set of segments, 12 or 15 bits.
On a process switch it'd change the 16 segment map entries but it
didn't need to flush the TLB like thing its reverse map had.

--
Regards,
John Levine, johnl@taugh.com, Primary Perpetrator of "The Internet for Dummies",
Please consider the environment before reading this e-mail. https://jl.ly

John Levine wrote:
> According to John Dallman <jgd@cix.co.uk>:
>>> This is excluding programs which reserve absurdly large address
>>> ranges "because they can", but I consider this to be an aberration
>>> rather than something which should be encouraged.
>>
>> Some programs do need to be able to handle sets of data, in RAM, that are
>> too large to fit into 32-bit addressing. There aren't huge numbers of
>> them, but they definitely exist.
>
> Databases will buffer as much as they can in RAM. If your machine can give
> your database ten gig of RAM, it can do useful things with it, assuming
> you have databases that big which these days you probably do.

10GB is a small DB these days!

More interestingly a 10GB RAM buffer often means that you can take a 100
GB regular DB and convert it to compressed column store layout which
will then fit everything in memory.

Terje

--
- <Terje.Mathisen at tmsw.no>
"almost all programming can be viewed as an exercise in caching"

MitchAlsup <MitchAlsup@aol.com> schrieb:
> On Sunday, October 10, 2021 at 4:27:30 PM UTC-5, Thomas Koenig wrote:
>> MitchAlsup <Mitch...@aol.com> schrieb:

>> > A decade ago (can it be that long) I was doing 1D CFD on intake and
>> > exhaust systems for automotive engines.
>> That's hardly CFD these days :-) 1D unsteady, I assume?
><
> I certainly had to use many-many surfaces to adequately model velocity
> stacks--so something 9" long had 45 individual set of diameters.
><
>> > These applications would
>> > sometimes run for several days (24/7) on my 6 core system, but
>> > never used "that much memory". The operating mesh had hundreds of
>> > thousands of boundary constraints (mostly pipe walls).
>> Most of the CPU time was spent doing some wildly non-linear
>> things, I assume?
>> ><
>> > Perhaps 2D CFD and 3D CFD exponentiate my experience.......
><
>> Most serious CFD is 3D.
><
> I can certainly agree with this notion, however, I had Free ($0.00) access
> to a 1D modeler that was already setup to do "about" what I wanted to do
> (including the combustion physics); so I used what was easily available.

Ah, that is the "wildly nonlinear" part I was looking for.
The Arrhenius equation for the rate of chemical reactions
has two nasty properties for numerics:

k = A * exp(-Ea/(R*T))

is

a) exponential, so you need to evaluate an expensive function
(a nod to your implementation there - are you closer to having
it implemented by a chip designer?)

b) highly nonlinear, so you need a large number of iterations,
especially if you have many of them coupled.

so it is possible to burn almost arbitrary amounts of CPU time
on such a problem.

Such a problem is usually CPU bound, not memory bound like "normal"
CFD. For a 1D problem, I would normally not classify it as CFD;
for me, that starts at 2D, but that is a definition by me
and my colleagues. You would also use different programs for this.

Interestingly enough, even a program like Ansys Fluent does not
really solve 2D CFD problems. Instead, it turns a 2D problem into
a 3D problem on a slab with a thickness of 1m (by default).

Stefan Monnier <monnier@iro.umontreal.ca> schrieb:
>>> >> > Some programs do need to be able to handle sets of data, in RAM, that are
>>> >> > too large to fit into 32-bit addressing. There aren't huge numbers of
>>> >> > them, but they definitely exist.
>
> I find this discussion quite surprising. On the one hand, I'm still
> using the i386 port of Debian on most/all of my machines, so clearly all
> my applications fit within that 32bit address space, but I'd have expected it's
> quite common for numerical simulations to need a fair bit more.
>
> After all, my Firefox process often gets in the GB range, my Emacs needs
> at least as much RAM as the size of the files it edits (and log files >4GB
> aren't that rare), and even smartphones come with 8GB these days.
>
> So is it really still rare(ish) to pass the 4GB limit?

Most processes do not pass 4GB, but there are important ones that do.

If you're on a x86, you also have fewer registers in 32-bit mode,
unless you use the x32 ABI (is that still alive?)

SPARC compilers will give you 32 bit code by default to save
on code size. If you want to run a cryptographic hash, then
of course 64 bit is better :-)

MitchAlsup <MitchAlsup@aol.com> schrieb:

> I think the transition from 64-bits to whatever will be greater than the
> the transition from 32-bits to 64-bits. The next power of 2 is way bigger
> than necessary 2^128 (10^38.5) while there are only 10^49 atoms on the
> planet.

My personal favorite would be 72 bits (bring back the 36-bit REAL of the
IBM 704 and the PDP-10!), but of course
><
> Non-powers of 2 are problematic; but I don't think I have to worry about this.

this is true.

MitchAlsup <MitchAlsup@aol.com> schrieb:

> On the other hand, many workloads have "workarounds" that are perfectly
> adequate for the application at hand. CRAY used to get CPU/machine
> orders for systems where the numeric performance was behind NEC
> supercomputers, but where the CRAY I/O system could sustain more
> I/O (double buffered data stream I/O) and thus attack larger problems
> than the NEC machines.

Cray I/O was quite good, I once heard (anecdotally) that the
computing center at Stuttgart got a Cray for database applications
because it had a better price than the equivalent IBM equipment.
This was in the 1990s.

However, regarding CFD, this is a thing of the past. If you get
into paging (or swapping) with a modern CFD, you can usually forget
about getting a result in a reasonable time.

On Sat, 9 Oct 2021 14:28:22 -0700 (PDT), MitchAlsup
<MitchAlsup@aol.com> wrote:

>Question: in a 64-bit virtual address space, is there ever a reason that
>the "heap" cannot live in a contiguous subsection of that space so
>base and bounds could tell if the pointer should be considered for
>GC (or not)?

I can't think of one offhand. 64 bits gives plenty of space for lots
of programs to have huge contiguous heaps.

GCs for uncooperative languages (like C) do check potential pointers -
i.e. properly aligned values - against the heap limits as a first
step. But what the GC really needs to know is if a pointer refers to
an actual allocated block. Determining /that/ is a lot harder because
the value must be range checked against every allocation, not just
against the heap limits.

There are various data structures that facilitate interval / range
searching, but maintaining them is extra overhead in the allocator and
collector, and potentially is heap space unavailable to the program.
It's a lot more effective to require that the compiler retain original
pointers and always work on copies of them.

George