On 28/05/2021 20:33, Branimir Maksimovic wrote:
> On 2021-05-28, Bart <bc@freeuk.com> wrote:

>> My languages ONLY interface to external libraries via DLL files.
>>
>> Those [DLL] files already contain the names of exported functions
>> and variables. What they don't contain are all the relevant types. Nor
>> exported user-types, structs, and enums.
>>
>> Imagine if a compiler, with a simple switch, added such information when
>> generating a DLL file.
>>
>> Now C header files would be practically obsolete! Only C would still
>> need them. Languages that are more with it would just say:
>>
>> importd lib # import lib.dll directly
>>
>> and all the declarations necessary to use the library would be instantly
>> available, in machine-readable format.
>>
>> Wouldn't that be brilliant? No, not according to the luddites here; who
>> on earth would want such a sensible feature? What would they do with all
>> those wonderful tools?

> It's too late to change that. You would need different dll format as
> well. Only way you can write type arguments is via symbols in functions,
> which is not portable even from compiler to compiler...

It doesn't need a DLL format change. (I think SWIG inserts stuff into a
DLL file that CPython can pick up.)

It can be done by exporting certain tables (as variables) to DLL, or
providing a special function that can be interrogated to provide extra
information.

In fact I planned such a feature last year (not for C but for a private
language), which is where 'importd' comes from.

The was the next step after 'importx' which works now. 'importx' uses a
discrete exports file (there is LIB.DLL, and LIB.MX which is loaded here
by importx). LIB.MX, in this language, is automatically generated when
compilng the source LIB.M into LIB.DLL.

Also generated, optionally, is a Docs file LIB.TXT which lists the
exported functions together with any doc-strings that are associated
with it in the source.

Where a C compiler would have a problem, is deciding exactly what to
export (either to LIB.DLL, or LIB.EXP if it went that route; even SWIG
already requires user input in the form of LIB.I to provide some of thay
info).

In fact there is already a problem when generating an ordinary DLL/so
now, of distinguishing between a function exported from one module to be
used by others, but not intended to be visible outside the program, and
exporting a function from the program, ie. from the DLL/so in that case.

But the idea can still be advantageous, assuming the info stored in the
DLL, or any explicit exports file, is not allowed to use C source
features such as typedefs and macros and includes. It'll be purely
table-based, and therefore more likely to be language-neutral.

(I believe C++ encodes some type info in DLLs, in the form of a
'decorated' name, which have some similarities, but in my version, the
function names are unmolested.)

Bart <bc@freeuk.com> writes:
[...]
> How about someone finally admitting that all these issues with C do
> actually make all the above harder?

OK, I admit it.

> Then I'll stop.

I guess we'll see.

> It's people
> pretending it's all really so wonderful and that there's nothing wrong
> with it, it's just me moaning for no readon, that winds me up.)

Stop exaggerating. I don't believe I've ever used the word
"wonderful" or the phrase "nothing wrong" with respect to C.
Most of what I do here in comp.lang.c is *describing* C as it is,
with all its warts, with the goal of showing off^H^H^H^H^H^H^H^H^H^H
helping others use it effectively.

Yes, interfacing to C from other languages can be difficult.
It's not something I've done a lot, so I don't know as much about
the issues as some others might. There are existing solutions,
which may have their own problems.

But sometimes I can help out by, for example, showing how to write
a C program that displays the relevant characteristics of C's
predefined types without getting hung up on the quirks like types
with the same sizes still being distinct.

--
Keith Thompson (The_Other_Keith) Keith.S.Thompson+u@gmail.com
Working, but not speaking, for Philips Healthcare
void Void(void) { Void(); } /* The recursive call of the void */

On 2021-05-28, Bart <bc@freeuk.com> wrote:
> On 28/05/2021 19:49, Kaz Kylheku wrote:
>> On 2021-05-28, David Brown <david.brown@hesbynett.no> wrote:
>>> On 28/05/2021 19:12, Bart wrote:
>>>>
>>>> Here's a challenge for you: write a self-contained program up to 1000
>>>> lines that can take an API expressed in the form of C headers such as
>>>> sdl.h (remember there are 80 sub-headers), and output a machine-readable
>>>> summary of:
>>>
>>> Here's a challenge for /you/ - stop making stupid artificial
>>> restrictions that serve no purpose but to fuel your paranoia about how
>>> everything related to C is designed specifically to annoy you.
>>
>> I think that in a parallel universe in which the Unix world switched
>> to Modula-2 in 1982 all the same issues exist.
>>
>> In this parallel universe, there are Modula-2 compilers in which long
>> and regular integers have been made the same, due to the bigs witch to
>> 32 bit computing.
>>
>> Size-specific types have been introduced into the language.
>>
>> Multiple implementations of POSIX from different vendors use slightly
>> different versions of Modula-2 records with differently sized members
>> and different quantities and sizes of undocumented members, making
>> FFI tricky.
>>
>> In this parallel universe, Modula-2 doesn't have header files with
>> macros, but .def have come to specify inline functions which are,
>> of course, in Modula-2 syntax. Non-inline versions of ithese functions
>> are often missing from the compiled objects, so everyone must parse
>> Modula-2 syntax to re-create them or else roll them by hand.
>>
>> There is an active comp.lang.modula2 newsgroup, which is trolled by
>> people who want to change the language into something else, while they
>> completely ignore comp.std.modula2 where that is topical
>
> Alternatively there could be a parallel universe in which APIs look like
> this:

So the Modula-2 universe wouldn't be good enough?

> clang proc "glAccum" (u32,r32)
> clang proc "glAlphaFunc" (u32,r32)
> clang function "glAreTexturesResident" (i32,ref u32,ref
> byte)byte

What is "ref u32"? Is that just one u32 or a whole array?

Can it be null or does it have to point to a u32?

Is it dynamic memory, and if so, does gAreTexturesResident take
ownership of this, and free it?

Or is it static memory that nobody has to free, and the API can keep
referring to it after the function returns?

> Wouldn't that be brilliant? No, not according to the luddites here; who
> on earth would want such a sensible feature? What would they do with all
> those wonderful tools?

I would be surprised if the luddites here were not familiar with at
least two interface description languages for remote procedure calls.

Why don't you publish the specification of this interface definition
language, and some reference tooling?

--
TXR Programming Language: http://nongnu.org/txr
Cygnal: Cygwin Native Application Library: http://kylheku.com/cygnal

On 2021-05-28, Bart <bc@freeuk.com> wrote:
> On 28/05/2021 20:33, Branimir Maksimovic wrote:
>> On 2021-05-28, Bart <bc@freeuk.com> wrote:
>
>>> My languages ONLY interface to external libraries via DLL files.
>>>
>>> Those [DLL] files already contain the names of exported functions
>>> and variables. What they don't contain are all the relevant types. Nor
>>> exported user-types, structs, and enums.
>>>
>>> Imagine if a compiler, with a simple switch, added such information when
>>> generating a DLL file.
>>>
>>> Now C header files would be practically obsolete! Only C would still
>>> need them. Languages that are more with it would just say:
>>>
>>> importd lib # import lib.dll directly
>>>
>>> and all the declarations necessary to use the library would be instantly
>>> available, in machine-readable format.
>>>
>>> Wouldn't that be brilliant? No, not according to the luddites here; who
>>> on earth would want such a sensible feature? What would they do with all
>>> those wonderful tools?
>
>> It's too late to change that. You would need different dll format as
>> well. Only way you can write type arguments is via symbols in functions,
>> which is not portable even from compiler to compiler...
>
> It doesn't need a DLL format change. (I think SWIG inserts stuff into a
> DLL file that CPython can pick up.)
>
> It can be done by exporting certain tables (as variables) to DLL, or
> providing a special function that can be interrogated to provide extra
> information.
>
> In fact I planned such a feature last year (not for C but for a private
> language), which is where 'importd' comes from.
>
> The was the next step after 'importx' which works now. 'importx' uses a
> discrete exports file (there is LIB.DLL, and LIB.MX which is loaded here
> by importx). LIB.MX, in this language, is automatically generated when
> compilng the source LIB.M into LIB.DLL.
>
> Also generated, optionally, is a Docs file LIB.TXT which lists the
> exported functions together with any doc-strings that are associated
> with it in the source.
>
> Where a C compiler would have a problem, is deciding exactly what to
> export (either to LIB.DLL, or LIB.EXP if it went that route; even SWIG
> already requires user input in the form of LIB.I to provide some of thay
> info).
>
> In fact there is already a problem when generating an ordinary DLL/so
> now, of distinguishing between a function exported from one module to be
> used by others, but not intended to be visible outside the program, and
> exporting a function from the program, ie. from the DLL/so in that case.
>
> But the idea can still be advantageous, assuming the info stored in the
> DLL, or any explicit exports file, is not allowed to use C source
> features such as typedefs and macros and includes. It'll be purely
> table-based, and therefore more likely to be language-neutral.
>
> (I believe C++ encodes some type info in DLLs, in the form of a
> 'decorated' name, which have some similarities, but in my version, the
> function names are unmolested.)

Look C is widelly used to interface all languages. Changes like that
wouldn't be welcome nowhere ;)
All new languages implement things like you say, but those languages
are not used to write cross language libraries, rather C.
Call it inertia or sand of time, but someone will standardize those
things and then you will be happy ;)

--
current job title: senior software engineer
skills: x86 aasembler,c++,c,rust,go,nim,haskell...

press any key to continue or any other to quit...

On 28/05/2021 22:43, Kaz Kylheku wrote:
> On 2021-05-28, Bart <bc@freeuk.com> wrote:

>> clang function "glAreTexturesResident" (i32,ref u32,ref
>> byte)byte
>
> What is "ref u32"? Is that just one u32 or a whole array?
>
> Can it be null or does it have to point to a u32?
>
> Is it dynamic memory, and if so, does gAreTexturesResident take
> ownership of this, and free it?

This is auto-translated from the original C which is:

WINGDIAPI GLboolean APIENTRY glAreTexturesResident(GLsizei n,const
GLuint *textures,GLboolean *residences);

As to what it means and how it's used, you need to see the docs which
say that is an array. (Which is another issue with C which I hardly dare
bring up, in that you can't tell if a T* type is a pointer to un
individual T object, or is the idiom for passing an array of T.)

If I was to go through these 500 functions and tweak for more apt
parameter types, I would probably change that to one of:

ref[]u32 textures
[]u32 &textures

(Given the presence of 'const', I might mark the parameter as 'in'
although I don't yet support that; it's anyway more useful for the
callee than caller.)

Defined like this, my language will only let me pass a pointer to array;
as ref u32, it will only let me pass a pointer to u32.

> Or is it static memory that nobody has to free, and the API can keep
> referring to it after the function returns?
>
>> Wouldn't that be brilliant? No, not according to the luddites here; who
>> on earth would want such a sensible feature? What would they do with all
>> those wonderful tools?
>
> I would be surprised if the luddites here were not familiar with at
> least two interface description languages for remote procedure calls.
>
> Why don't you publish the specification of this interface definition
> language, and some reference tooling?

It's not a remote procedure call. It is simply bundling information into
a DLL.

If some binary form rather than text was used, it might be able to get
away from at least some C sources dependencies, by forcing the 'cryptic'
C code (shall we say) to be rendered into a more specific representation.

Bear in mind that a DLL binary is already specific to a particular
target and platform, and already contains 'rendered' or compiled code of
that same library.

So there is no longer a need for a representation that can configured
umpteen ways; another advantage.

(My original idea for this, for my own use, was to just incorporate an
entire .mx exports text file into the matching and /working/ DLL, which
would be returned as a string from a suitable function. Just to make
things tidy.

But that won't do for C as you'd still have header files, but now it's
just harder to access them!)

On 2021-05-28, Bart <bc@freeuk.com> wrote:
> On 28/05/2021 22:43, Kaz Kylheku wrote:
>> On 2021-05-28, Bart <bc@freeuk.com> wrote:
>
>
>>> clang function "glAreTexturesResident" (i32,ref u32,ref
>>> byte)byte
>>
>> What is "ref u32"? Is that just one u32 or a whole array?
>>
>> Can it be null or does it have to point to a u32?
>>
>> Is it dynamic memory, and if so, does gAreTexturesResident take
>> ownership of this, and free it?
>
> This is auto-translated from the original C which is:
>
> WINGDIAPI GLboolean APIENTRY glAreTexturesResident(GLsizei n,const
> GLuint *textures,GLboolean *residences);
>
> As to what it means and how it's used, you need to see the docs which
> say that is an array. (Which is another issue with C which I hardly dare
> bring up, in that you can't tell if a T* type is a pointer to un
> individual T object, or is the idiom for passing an array of T.)

That's not a C problem! A routine coded in assembly language could
have a pointer parameter which points to a single item,
or which points to the base of an array, which could be
dynamic memory whose ownership is passed down or else not and so on.

A C declaration written for the assembly code will give you
some useful information, like types, but not everything.

Remember, you dohn't like C; you're proposing a different declaration
language.

Let's assume that the target is assembly code which conforms to
the calling conventons so it's safely callable from C; arguments
are passed in the right way, returning is handled in the
right way, the right things are saved and restored.

So, there is no C in the picture at all; just your declarative
language and a block of machine code made from assembly which
has no declarations at all.

You need the nuance in this interface language to capture all the common
memory passing protocols, or it's good for next to nothing.

>> Why don't you publish the specification of this interface definition
>> language, and some reference tooling?
>
> It's not a remote procedure call. It is simply bundling information into
> a DLL.

Remote procedure call interface languages can support intra-process
call, and have done so. The same language is used, just the
implementation can drop the marshaling.

So for that reason, there is overlap between describing interfaces
for the purpose of RPC and for the purpose of FFI.

Broadly speaking, if you have enough semantics in the description
language for FFI, you can generate marshaling/unmarshaling code for
remote procedure calls.

--
TXR Programming Language: http://nongnu.org/txr
Cygnal: Cygwin Native Application Library: http://kylheku.com/cygnal

On 5/28/2021 7:00 PM, Kaz Kylheku wrote:
> On 2021-05-28, Bart <bc@freeuk.com> wrote:
>> On 28/05/2021 22:43, Kaz Kylheku wrote:
>>> On 2021-05-28, Bart <bc@freeuk.com> wrote:
>>
>>

You are my best reviewer.

I totally understand the Halting Problem Diagonalization Argument.
I addressed your post on the link and in comp.theory.

Refutation of Halting Problem Diagonalization Argument
DOI: 10.13140/RG.2.2.34446.28483

https://www.researchgate.net/publication/351947980_Refutation_of_Halting_Problem_Diagonalization_Argument

--
Copyright 2021 Pete Olcott

"Great spirits have always encountered violent opposition from mediocre
minds." Einstein

[Sorry, original may have been sent to your email by mistake]

On 29/05/2021 01:00, Kaz Kylheku wrote:
> On 2021-05-28, Bart <bc@freeuk.com> wrote:
>> On 28/05/2021 22:43, Kaz Kylheku wrote:
>>> On 2021-05-28, Bart <bc@freeuk.com> wrote:
>>
>>
>>>> clang function "glAreTexturesResident" (i32,ref
u32,ref
>>>> byte)byte
>>>
>>> What is "ref u32"? Is that just one u32 or a whole array?
>>>
>>> Can it be null or does it have to point to a u32?
>>>
>>> Is it dynamic memory, and if so, does gAreTexturesResident take
>>> ownership of this, and free it?
>>
>> This is auto-translated from the original C which is:
>>
>> WINGDIAPI GLboolean APIENTRY glAreTexturesResident(GLsizei n,const
>> GLuint *textures,GLboolean *residences);
>>
>> As to what it means and how it's used, you need to see the docs which
>> say that is an array. (Which is another issue with C which I hardly dare
>> bring up, in that you can't tell if a T* type is a pointer to un
>> individual T object, or is the idiom for passing an array of T.)
>
> That's not a C problem! A routine coded in assembly language could
> have a pointer parameter which points to a single item,
> or which points to the base of an array, which could be
> dynamic memory whose ownership is passed down or else not and so on.
>
> A C declaration written for the assembly code will give you
> some useful information, like types, but not everything.
>
> Remember, you dohn't like C; you're proposing a different declaration
> language.

The problem with C arrays is one I'd forgotten about. They would indeed
cause an issue with using a C-like type system for a more
language-neutral API.

One solution would be to use actual arrays in C:

const GLuint (*)[]textures

But that generates others (when used with C, it would change the
necessary code on both sides of the function, which also needs more
unwieldy syntax for array accessing).

> Let's assume that the target is assembly code which conforms to
> the calling conventons so it's safely callable from C; arguments
> are passed in the right way, returning is handled in the
> right way, the right things are saved and restored.
>
> So, there is no C in the picture at all; just your declarative
> language and a block of machine code made from assembly which
> has no declarations at all.
>
> You need the nuance in this interface language to capture all the common
> memory passing protocols, or it's good for next to nothing.

That's defined by the ABI.

It might be good if this interface scheme included facilities for:

* Value arrays (not possible in C)

* Pointers to arrays (possible in C but hardly ever used in actual APIs)

If these are added, people might argue for also supporting the kinds of
arrays used in C++, Go, Rust and so on.

But I feel that these are less transparent and do not belong in such a
lower-level interface.

Value arrays ([] types) and pointers to arrays ((*)[] in C parlance) do
belong in a lower-level type system.

It's just that former are usually expunged from use in C expressions,
and the latter are never used because the C syntax for it stinks.

This lower-level type system I would define as follows:

* Scalar types of 8/16/32/64 bits (ints and floats)

* Raw pointers to any of these types

* Structs of any set of these types including structs/arrays

* Fixed-length arrays of any of these types including structs/arrays,
with no restrictons on passing value-arrays

* Pointers to unbounded arrays of any of these types

Anything else can be emulated on top of these.

Bart <bc@freeuk.com> writes:
>[Sorry, original may have been sent to your email by mistake]

> > You need the nuance in this interface language to capture all the common
> > memory passing protocols, or it's good for next to nothing.
>
>That's defined by the ABI.
>
>It might be good if this interface scheme included facilities for:
>
>* Value arrays (not possible in C)

What possible use is there for value arrays? Pointless data movement.

On 29/05/2021 18:23, Scott Lurndal wrote:
> Bart <bc@freeuk.com> writes:
>> [Sorry, original may have been sent to your email by mistake]
>
>>> You need the nuance in this interface language to capture all the common
>>> memory passing protocols, or it's good for next to nothing.
>>
>> That's defined by the ABI.
>>
>> It might be good if this interface scheme included facilities for:
>>
>> * Value arrays (not possible in C)
>
> What possible use is there for value arrays? Pointless data movement.
>

About the same as there is for value structs, especially structs
containing arrays, which is what you have to do to emulate value arrays.

In any case, with any sizeable arrays, ABIs may demand they are handled
via references. And with small ones of 1,2,4,8 bytes, they can be
handled like scalars.

Here's what you can do, expressed as C syntax:

enum {N=50};
int A[N];
int B[N];
int (*P)[N];

A=B; // instead of memcpy(A, B, sizeof(A)), and compiler
// can check the sizes

if (A==B) // instead of memcmp(A, B, sizeof(A))==0

A=*P;

A=F(B);

or instead of doing it all with structs.

I'm not suggesting they should be added to C, I'm suggesting it would be
silly to exclude them from a slightly more universal type system where
there would be a great big hole where they would clearly go.

On 2021-05-29, Scott Lurndal <scott@slp53.sl.home> wrote:
> Bart <bc@freeuk.com> writes:
>>[Sorry, original may have been sent to your email by mistake]
>
>> > You need the nuance in this interface language to capture all the common
>> > memory passing protocols, or it's good for next to nothing.
>>
>>That's defined by the ABI.
>>
>>It might be good if this interface scheme included facilities for:
>>
>>* Value arrays (not possible in C)
>
> What possible use is there for value arrays? Pointless data movement.

Value arrays are not possible in the C type system, but at the ABI
level, this kind of thing is not distinguishable from an array:

struct array_wrapper {
int array[10];
};

This structure has the size and alignment requirements of an
"int array[10]".

I have designed a FFI type system which allows this function:

int foo(struct array_wrapper a);

to be specified like this:

(typedef arr (array 10 int))
(deffi foo "foo" int (arr))

The type system simply has "by value" arrays, without structure
wrapping. In the FFI call, these are placed into the argument space the
same way as a one-member structure. It can be thought of as a
"transparent struct".

This is advantageous because thus defined, the function may be called
with a vector argument, as:

(foo #(1 2 3 4 5 6 7 8 9 10))

whereas an array wrapped in structure would require a Lisp structure
holding an array as a slot.

--
TXR Programming Language: http://nongnu.org/txr
Cygnal: Cygwin Native Application Library: http://kylheku.com/cygnal

On 29/05/2021 01:00, Kaz Kylheku wrote:
> On 2021-05-28, Bart <bc@freeuk.com> wrote:

>> As to what it means and how it's used, you need to see the docs which
>> say that is an array. (Which is another issue with C which I hardly dare
>> bring up, in that you can't tell if a T* type is a pointer to un
>> individual T object, or is the idiom for passing an array of T.)
>
> That's not a C problem! A routine coded in assembly language could
> have a pointer parameter which points to a single item,
> or which points to the base of an array, which could be
> dynamic memory whose ownership is passed down or else not and so on.

ASM doesn't have a type system which is of much use to even lower-level
HLLs.

At best it knows about type sizes, so will have these:

d8
d16
d32
d64
dN (value-block)

Type-awareness is imposed by the programmer when they choose the
instructions to use; that requires extra info not present here (not in a
simple assembler anyway; there are some hairier ones that pretend to be
a HLL).

I use, internally on a some projects (compiler backends), a type system
one step up from this, but that is not enough either.

The one I'm talking about here which I've outlined elsewhere, is a
rationalised version of the one commonly used by C-style APIs.

Higher-level type systems exist (some languages seem to do nothing but
mess around tying themselves up in knots with them), but I'm only
interested here in the ones used in the APIs of commonly used libraries,
for example:

Early WinAPI (before they switched to C++ and beyond)
GMP
OpenGL, SDL, Cairo, GTK, Raylib, STB

> Let's assume that the target is assembly code which conforms to
> the calling conventons so it's safely callable from C;

This is little to do with call-conventions. Such a type system as above
should be largely independent of target, other than having some types
whose widths depend on characteristics of the target.

I don't believe the ABI can tell you how structs are laid out either.
There is a particular way that C organises them, but any language
(including C) can also choose whatever layout it likes.

What's important here is telling the layout from the information
available (if using a third party library), or imparting that info if
you are writing the library.

> So, there is no C in the picture at all;

That sounds attractive, but actually it is probably good that C is in
the picture, to keep things grounded and avoid a free-for-all. But the
model it uses can be improved and be less C-centric with no macros or
whatever that do not belong here.

On 2021-05-30, Bart <bc@freeuk.com> wrote:
> On 29/05/2021 01:00, Kaz Kylheku wrote:
>> On 2021-05-28, Bart <bc@freeuk.com> wrote:
>
>>> As to what it means and how it's used, you need to see the docs which
>>> say that is an array. (Which is another issue with C which I hardly dare
>>> bring up, in that you can't tell if a T* type is a pointer to un
>>> individual T object, or is the idiom for passing an array of T.)
>>
>> That's not a C problem! A routine coded in assembly language could
>> have a pointer parameter which points to a single item,
>> or which points to the base of an array, which could be
>> dynamic memory whose ownership is passed down or else not and so on.
>
>
> ASM doesn't have a type system which is of much use to even lower-level
> HLLs.

An API writtten in assembly language can conform to an ABI. Its
interface can be described with documentation that refers to ABI
concepts. ABI's have type systems; typically based on C concepts.

> At best it knows about type sizes, so will have these:
>
> d8
> d16
> d32
> d64
> dN (value-block)
>
> Type-awareness is imposed by the programmer when they choose the

"Type awareness" of an assembly language routine is imposed by
its documentation.

It's not uncommon for assembly language routines to be declared in a
higher level language.

>> Let's assume that the target is assembly code which conforms to
>> the calling conventons so it's safely callable from C;
>
> This is little to do with call-conventions.

If I mention calling conventions, it is for a good reason.

Assembly language routines can specify arbitrary calling conventions
like "place the input value into register R19 before calling, and
afterward pull three words off the stack; and, by the way, all
registers may be clobbered except the stack and instruction pointer".

I simply wanted to restrict the discusssion to assembly language that
is callable at all using the conventions expected by the FFI
mechanism and the platform ABI. Non-ABI-conforming machine code will be
hard to use, and is a strawman.

> I don't believe the ABI can tell you how structs are laid out either.

That's in fact one of the crucial pieces of information that platform
ABIs must necessarily contain.

> There is a particular way that C organises them, but any language
> (including C) can also choose whatever layout it likes.

That is obviously false if the intent is being able to directly pass
those structures to other components. Then their layout has to conform
to the ABI in all details: sizes of members and their alignment so that
the offsets and padding are all correct.

(It's possible that the ABI might not have all the details. E.g. if no
API in some platform uses bitfields, then there is no reason to say
anything about them in the ABI.)

An ABI might refer to C concepts to describe types, but it's not C.

>> So, there is no C in the picture at all;
>
> That sounds attractive, but actually it is probably good that C is in
> the picture, to keep things grounded and avoid a free-for-all. But the
> model it uses can be improved and be less C-centric with no macros or
> whatever that do not belong here.

API's sometimmes use macros for inline functions. Any API in any
language, if it provides /only/ an inline function for some operation,
such that the function exists only at the compile time of that language
and not in the compiled image, is being hostile to interoperability.

The C standard sets an example for avoiding this: whenever a standard
function may be a macro, a real function must also exist. (If the header
file is included which defines that macro, #undef can be used to reveal
the real function, e.g. #undef putc). The implication for FFI users is
that the can always bind to a real putc function from a standard C
library, if such C library is available for such use.

--
TXR Programming Language: http://nongnu.org/txr
Cygnal: Cygwin Native Application Library: http://kylheku.com/cygnal

muta...@gmail.com <mutazilah@gmail.com> wrote:
> On Tuesday, May 18, 2021 at 4:54:40 PM UTC+10, David Brown wrote:
>
> > For a new language, there
> > are many possibilities but calling things "short" or "long" is unlikely
> > to be a good choice.
>
> Why do you say that?
>
> What are the first principles you are working from?
> (that Ritchie was apparently not aware of - and I
> certainly aren't)

There are two "kinds" of integers which may appear in program.
One kind have size determined by problem, in such case in
low level language (like C) you want fixed sized types.
Theoretically nicest way is to specify range, several languages
have range types. Or you may specify number of bits, again
several languages allow this. Or you may provide small
number of predefined types, like <stdint.h>, again several
languages has this.

Other kind of integers have size depending on environment,
main example being integers used as array indices and
integer used to represent size of pieces of memory.

Original C design did not provide portable fixed size integers,
from modern poit of view this is a flaw. Concerning integers
used for addressing and memory sizes existing practice relatively
quickly led to confusion. Modern C practice handles this
almost reasonably, but modern treatment uses different
types than "short", "long" or "int".

As Bart noted in C there is problem that standard printf specifiers,
integer literals and promotion rules are tied to ""short", "int"
and "long" (and their unsigned variant), so you can not ignore
old types and need to look at their releations to modern types.

--
Waldek Hebisch

David Brown <david.brown@hesbynett.no> wrote:
> On 18/05/2021 12:23, Bart wrote:
> > On 18/05/2021 09:14, muta...@gmail.com wrote:
> >
> >> But int is almost always 32-bits "these days". How do you
> >> reconcile that?
> >
> > Because most existing C programs would likely break if 'int' was
> > suddenly 64 bits.
>
> Would they? That is not my impression at all. Some would have trouble
> if it was changed to 16-bit on their systems, because people assume
> "int" is big enough to support numbers larger than 32767. But it will
> rarely be a problem that "int" can store larger numbers.

If you look at lines of code, then number of affected lines of
code is likely to be small. But typical programs are large
and are likely to contains some "bad" spots. Potential problem:

- overlaying structures defined in terms of bytes with one
defined in terms of "int". Major variation of this is
describing layout of file via C structs
- depending on wraparound. In some cases wraparound is UB
and program could be declared as broken. In other cases
wraparound is on store which is "kosher". There were a
study looking at several programs which found that in significant
number of cases when overflow led to wraparound that wraparound
was actually expected by program.
- preprocessing magic which may get badly confused by unexpected
system parameters.

--
Waldek Hebisch

muta...@gmail.com <mutazilah@gmail.com> wrote:
> On Monday, May 24, 2021 at 4:14:42 AM UTC+10, muta...@gmail.com wrote:
>
> > > MVS was useful in its day. We've moved beyond it. Leave it in the past.
>
> > I don't consider it to be remotely in the past. It remains
> > to be seen if it can be made into a competitor to z/OS.
> > It already is to some extent.
>
> Note that IBM actually have in their license agreement
> that z/OS can't be used for production workloads in
> an emulated environment, because they know damned
> well that it can be. Computers are that fast.
>
> I doubt that you are familiar enough with the mainframe
> to know the difference between MVS 3.8J and z/OS to
> be able to assess that a modified MVS 3.8J is not
> viable.

AFAICS one does not need to look very deep to why MVS 3.8J
is not viable:

- limitation to 24-bit address space
- you essentially get bare OS, without tools expected
by users

AFAIK 24-bit address space is quite serious limitation, because
large part of address space is used by OS and normally
there is a lot of libraries and data loaded into common
area. Your hack helps for one program, but is does almost
nothing towards solving general problem.

Concerning tools, MVS 3.8J available on the net has Cobol
and PL/I compilers from OS360. Both languages changed
a lot, I do not think anybody using z/OS would like
to go back to old compilers. Even assembler changed
compared to MVS time, so it would be not so trivial
effort to assemble modern files with MVS assembler.
I saw people praising KICKS, but is is not clear if
it can handle loads that CICS can handle. Serious
user of z/OS is likely to use some major database, do
you have replacement for MVS 3.8J? And that is just
few largish items, there is much more extras and
most of them are likely to cause trouble.

I appreciate effort some people spent on MVS 3.8J. However,
given that after several years (MVS on Hercules was about
20 years ago) there are no _real_ sources (AFAIK nobody
was able to reproduce system from "source material"),
chances for significant progress are close to 0.

Add to that that z/OS usere tend to be conservative, they
use z/OS because that (in their opinion) minimizes
chance of disraption. Going back to old system is
the last thing they want.

--
Waldek Hebisch

muta...@gmail.com <mutazilah@gmail.com> wrote:
> There is a S/3X0 flavor of PDOS (already) and those
> programs above will not run.
>
> Your C90-compliant "hello, world" will run though.
> You can even add a C90-compliant "int" in it, and
> it will still work.
>
> I am only concerned about getting C90-compliant
> programs to run, on all environments (8086, S/370
> etc). PDOS/386 and a future PDOS/x64 are just
> one more environment.

Hmm, gcc on s390 handles 'short' fine. Note that C
has integer promotion rules: everything in expressions
is promoted at least to 'int'. So for smaller types
all you need is load and stores. 16-bit operations
can gain you some efficiency if immediately after store
you use result of store. Or when code contains casts.
But this is not different than other optimizations,
some machines can do work using single instruction,
other may need two or three. Anyway, it would be
rather exceptional program if using 'short' causes
measurable slowdown on mainfraime processors.

--
Waldek Hebisch

On 30/05/2021 16:20, Kaz Kylheku wrote:
> On 2021-05-30, Bart <bc@freeuk.com> wrote:

>> I don't believe the ABI can tell you how structs are laid out either.
>
> That's in fact one of the crucial pieces of information that platform
> ABIs must necessarily contain.

IME my problems with C libraries are little to do with ABIs. Those are
just the necessary details needed if you are involved in writing compilers.

Somebody could be a /user/ of a compiler for a new language, and wants
to know how to express types and so on in that language to match the
types used in an API. They don't need to know anything about the ABI.

But they may need to know the target if conditional types need to be
made concrete.

I also need to use such APIs from interpreted code. There, the notion of
an ABI is meaningless. It's involved in the special mechanisms needed to
call an arbitrary, only-known-at-runtime function in a dynamic library,
but it's a small part of making that tricky task work.

(Harder also in that case is working with callback functions, but I
can't blame C for those.)

>> There is a particular way that C organises them, but any language
>> (including C) can also choose whatever layout it likes.
>
> That is obviously false if the intent is being able to directly pass
> those structures to other components. Then their layout has to conform
> to the ABI in all details: sizes of members and their alignment so that
> the offsets and padding are all correct.

The Win64 ABI makes recommendations about alignments inside structs, but
doesn't enforce them. Mainly it's about how they are passed: in
registers for sizes of 1,2,4,8 bytes, else by implicit reference.

A few WinAPI structs use misaligned elements anyway, example:

#include <pshpack2.h>
typedef struct tagBITMAPFILEHEADER {
WORD bfType;
DWORD bfSize;
WORD bfReserved1;
WORD bfReserved2;
DWORD bfOffBits;
} BITMAPFILEHEADER,*LPBITMAPFILEHEADER,*PBITMAPFILEHEADER;
#include <poppack.h>

(The bfSize member follows immediately from bfType without the normal 2
bytes of padding.) The docs for this struct show the same layout, but
without the pack includes. You would not know if working from this.
Those special includes contain:

#if !(defined(lint) || defined(RC_INVOKED))
#pragma pack(push,2)
#endif

This really isn't stuff you want to see anywhere outside of the C
language. (I define this particular one like this:

global type ws_bitmapfileheader = struct
wt_word typex
wt_dword size
wt_word res1, res2
wt_dword offbits
end

My structs use pack(1) by default unless a 'Caligned' attribute is used.)

>
> (It's possible that the ABI might not have all the details. E.g. if no
> API in some platform uses bitfields, then there is no reason to say
> anything about them in the ABI.)
>
> An ABI might refer to C concepts to describe types, but it's not C.
>
>>> So, there is no C in the picture at all;
>>
>> That sounds attractive, but actually it is probably good that C is in
>> the picture, to keep things grounded and avoid a free-for-all. But the
>> model it uses can be improved and be less C-centric with no macros or
>> whatever that do not belong here.
>
> API's sometimmes use macros for inline functions. Any API in any
> language, if it provides /only/ an inline function for some operation,
> such that the function exists only at the compile time of that language
> and not in the compiled image, is being hostile to interoperability.
>
> The C standard sets an example for avoiding this: whenever a standard
> function may be a macro, a real function must also exist. (If the header
> file is included which defines that macro, #undef can be used to reveal
> the real function, e.g. #undef putc). The implication for FFI users is
> that the can always bind to a real putc function from a standard C
> library, if such C library is available for such use.

Here's the first of 400 macros from processing sdl.h:

global macro SDL_LoadBMP(file) =
SDL_LoadBMP_RW(SDL_RWFromFile(file,"rb"),1)

In simplified form, it does this:

macro F(x) = G(H(x,"rb"),1)

The RHS is C syntax verbatim. At least function call syntax is fairly
universal, but it is still necessary to process such macros. Here's
another that is not as simple:

global macro SDL_stack_alloc(type,count) =
(type*)SDL_malloc(sizeof(type)*(count))

On Friday, May 14, 2021 at 5:01:07 AM UTC+8, muta...@gmail.com wrote:
> When hard disks started exceeding 2 GiB,
> but processors were still 32-bit, did anyone
> create a compiler for the 80386 that did
> 64-bit longs while everything else (pointers,
> size_t) was still 32-bits?
>
Don't know about 80386, that is too old now. But for newer CPU,
gcc has -mx32 option that creates 64-bit code with 32-bit addresses.
However, it seems to be no longer maintained, and it is hard to find all the libraries one needs built with mx32

On 26/05/2021 20:54, Lew Pitcher wrote:
> On Wed, 26 May 2021 20:39:48 +0200, David Brown wrote:
>
>> On 26/05/2021 18:36, Bart wrote:
> [snip]
>>> First tell me why C calls the floating point type next afer 'float' a
>>> 'double'.
>>
>> I don't know the history of the names. I do know that "double" is not
>> necessarily double the size of "float" (either in terms of bits in the
>> representation, range, precision, or any other feature).
>
> At the time that Kernighan, et al, created "K&R" C, there were two

Kernighan wrote much of the K&R book about it, if that's what you mean,
but I don't think he ever claimed to have created the language. Rather,
C's creator was Dennis Ritchie based on languages by Ken Thompson and
Martin Richards, AIUI.

--
James Harris

In article <s9lnqb$is4$1@dont-email.me>,
James Harris <james.harris.1@gmail.com> wrote:
>On 26/05/2021 20:54, Lew Pitcher wrote:
>> [snip]
>> At the time that Kernighan, et al, created "K&R" C, there were two
>
>Kernighan wrote much of the K&R book about it, if that's what you mean,
>but I don't think he ever claimed to have created the language. Rather,
>C's creator was Dennis Ritchie based on languages by Ken Thompson and
>Martin Richards, AIUI.

Indeed. He has explicitly *disclaimed* invention of the language
every time I've spoken with him. He gives all credit to Dennis,
and Dennis credits Thompson for B and Richards for BCPL.

- Dan C.

On 5/19/2021 5:52 PM, Joe Pfeiffer wrote:
> "Chris M. Thomasson" <chris.m.thomasson.1@gmail.com> writes:
>>
>> I am wondering what volumetric rendering software she is using? Is it
>> an expensive DICOM renderer for viewing 3d medical images? It almost
>> has to be.
>
> Her work is on the retina, so it's pretty close to all 2D. I don't know
> what volumetric or other rendering software she's using

Probably a shit load of 2d images, that can create a volume.

https://fractalforums.org/index.php?action=gallery;sa=view;id=5945

> ; last I talked
> to her she was mainly running clustering algorithms classifying cells
> and identifying conections (her research has been on macular
> degeneration, in particular the breakdown and restructuring of the
> connections between neurons during the process).
>

Excellent!

On 5/20/2021 9:13 AM, John Dill wrote:
> On Wednesday, May 19, 2021 at 5:44:01 PM UTC-4, Chris M. Thomasson wrote:
>> On 5/19/2021 7:18 AM, Joe Pfeiffer wrote:
>>> Malcolm McLean <malcolm.ar...@gmail.com> writes:
>>>
>>>> On Tuesday, 18 May 2021 at 09:14:37 UTC+1, muta...@gmail.com wrote:
>>>>>
>>>>>> There are "just a number" uses - a counter, an index, etc. The type for
>>>>>> this needs to be big enough that it is not going to overflow - the
>>>>>> programmer can treat it as though it were an unlimited size mathematical
>>>>>> integer. These days, that really means 64-bit - or an unlimited integer
>>>>>> type that grows as needed. The type name here should reflect that -
>>>>>> "number" would be good. In C, this is "int".
>>>>> But int is almost always 32-bits "these days". How do you
>>>>> reconcile that?
>>>>>
>>>> You only rarely have 2 billion data points. A 1024 * 1024 image is quite large,
>>>> for example, but it's only a million pixels.
>>>> Whist modern CPU operations with 64 bits will be as fast as 32 bit
>>>> operations, you
>>>> can store more 32 bit integers in the cache, and it's cache mises which are
>>>> the main determiner of performance.
>>>
>>> My son does deep learning at Microsoft. He tells me one of their recent
>>> projects gets 1GB of data per hour.
>>>
>>> My daughter does retina research at U Utah. I asked what sort of image
>>> sizes she works with (EM photos of retinas). The conversation went
>>>
>>> Me: I'm trying to remember -- how big (in bytes) did you say the images
>>> you're dealing with are? Came across someone saying a 2GB image is
>>> rare.
>>>
>>> Her: Lol
>>> Our volumes (stacks of images) are in the terabytes
>>> Off the scope I want to say a standard image (montage of tiles) is
>>> around 50, but that is super off the cuff. I'd have to actually
>>> look to know a real number.
>>>
>>> But 50gb images aren't unusual these days.
>>>
>>
>> Ohhh yeah, your daughter knows! A large volumetric rendering can be
>> 8192^3. For what its worth, I have some code that generates a "special"
>> Mandelbulb I created in the form of a volume. Its not contained in a
>> single DICOM file, but the volume is a stack of images. Here is my crude
>> code:
>>
>> https://pastebin.com/raw/07TWQQYF
>>
>> I take the image stack and view it using a volumetric renderer. ImageJ
>> in this case:
>>
>> https://www.fractalforums.com/index.php?action=gallery;sa=view;id=17187
>>
>> I am wondering what volumetric rendering software she is using? Is it an
>> expensive DICOM renderer for viewing 3d medical images? It almost has to be.
>
> It depends, DICOM is primarily used when patient data is associated with
> the image data in hospital type environments, but it's not uncommon to use
> simpler formats for image processing research to anonymize them. I've
> been out of medical image processing for 10-15 years now, so I don't know
> what the current university setting is like in the field.
>
> Most of the rendering software at the university when I was there was
> either in-house (we had a team of 3-4 professors that had a shared student
> developed image processing library), VTK or ITK derived software, or we
> had Analyze that supported a variety of cookie cutter volume rendering
> algorithms.
>
> As part of my MS thesis project, I had made a tool with VTK and our
> in-house library that created a 3-dimension renderer that could show 3
> planes of image data that you could slide along each axis, combined
> with a volume renderer that would take a medical doctor's hand contoured
> image volumes and create translucent surface volume rendering's
> colorized with radiation dose profiles to show how much calculated
> dose was applied to the anatomy, with options to zoom and rotate the
> orientation.
>
> You could probably get something working with a combination of ITK
> and VTK. You may even be able to find a sample program that works
> out of the box or gets you pretty close. You've typically got to do a
> deep dive into the algorithms/software in the field if you want have
> some kind application that implements some kind of first person
> navigation of a surface or volume rendering of a 3D volume of medical
> anatomy or in your case, a surface or volume rendering of a fractal.

I agree. Actually, create a stack of images for a volume is not really
ideal for fractals. Using a distance estimator can work fine, but I am
not sure how that can fit into a medical basis where they need data, not
an estimation?

https://www.shadertoy.com/view/fdBSzm