David Brown <david.brown@hesbynett.no> writes:
>On 26/05/2021 23:27, Bart wrote:

>> But the next one I know about that is supported by hardware is Intel's
>> 80-bit float. double double would be misleading.
>>
>
>80-bit floats are a hangover from the days of external FPU coprocessors.
> 128-bit quad precision floating point is the next big thing. They are
>not yet supported in the x86 world, but are used on POWER9 and a few
>other big processors.

Oddly, the big new thing are 16-bit floats. Really. ARM, AMD, Intel
all have been adding FP16 support. Primarily for machine learning applications.

https://community.arm.com/developer/ip-products/processors/b/ml-ip-blog/posts/bfloat16-processing-for-neural-networks-on-armv8_2d00_a

scott@slp53.sl.home (Scott Lurndal) writes:

> David Brown <david.brown@hesbynett.no> writes:
>>On 26/05/2021 23:27, Bart wrote:
>
>>> But the next one I know about that is supported by hardware is Intel's
>>> 80-bit float. double double would be misleading.
>>>
>>
>>80-bit floats are a hangover from the days of external FPU coprocessors.
>> 128-bit quad precision floating point is the next big thing. They are
>>not yet supported in the x86 world, but are used on POWER9 and a few
>>other big processors.
>
> Oddly, the big new thing are 16-bit floats. Really. ARM, AMD, Intel
> all have been adding FP16 support. Primarily for machine learning
> applications.

(FP16 sometimes indicates IEEE's 16-bit float format using 1:5:10 bits
which does not seem to be what's being added to these processors.)

> https://community.arm.com/developer/ip-products/processors/b/ml-ip-blog/posts/bfloat16-processing-for-neural-networks-on-armv8_2d00_a

Didn't know about the BF16 (1:8:7) format. I can see the advantages
(it's just the first two bytes of IEEE's 32-bit format) but it seems very
constrained in other respects.

I thought real-time 3D rendering was one proposed use of 16-bit floats,
but maybe support for the in the "main" CPU is no longer an issue.

--
Ben.

On 27/05/2021 14:33, Bart wrote:
> On 27/05/2021 13:02, David Brown wrote:
>> On 27/05/2021 12:46, Bart wrote:
>
>> When people make libraries that are likely to be useful from many other
>> languages, they will often make interface libraries, headers, wrappers
>> or whatever - a C header, a Pascal unit, a Python extension library, a
>> TCL interface, etc.  Whatever makes sense for languages that are likely
>> to be used in conjunction with the library.  Often that will be done
>> using tools like SWIG which dramatically cut down on the effort involved.

>
> unix % swig -python example.i
> unix % gcc -c example.c example_wrap.c \
>        -I/usr/local/include/python2.7
> unix % ld -shared example.o example_wrap.o -o _example.so
> ----------------------------
>
> Then, you magically just do 'import example' in Python. Can Python
> directly import DLL/so files? Is there extra magic involved? Since no
> other files are produced according to this.

I tried this. My SWIG version was the wrong one; I redownloaded for
Windows. It was smaller at 30MB, but just as many files.

From the original 18-line example.c file, and the 14-line example.i
file I copied from the tutorial, the program generated example_wrap.c of
3700 lines, and example.py of 75 lines. With gcc, linking with
python38.dll not shown in the tutorial, I got a DLL of 359KB (summarised
below).

However when I tried to run a Python program of 'import example', it
failed because example.py tried to import a further module '_example',
which doesn't exist as .py, and it doesn't look at _example.dll.

So the program is not straightforward and not foolproof.

I then tried the same example using my script language. Here I got bcc
to generate example.dll from the C, which was only 2.5KB. The test file
in my language was just this:

import clib

importdll example= # equivalent of example.i
clang function fact(int32)int32
clang function my_mod(int32 x,y)int32
clang function get_time()ref byte
end

function getstring(p)=
return makestr(p,strlen(p))
end

proc start=
println fact(12)
println my_mod(100,17)
println getstring(get_time())
end

Only one issue: it doesn't directly deal with 'char*' return types (that
ought to work with 'string', which I'll fix), so a helper function
getstring() was needed.

When I run this, the output is:

479001600
15
Thu May 27 17:29:12 2021

So, a comparison:

CPython/SWIG My Q Language

example.c (18 lines) (18 lines)
example.i 14 lines 5 lines (importdll block)
example_wrap.c 3780 lines ---
example.py 75 lines --- (see below)
_example.dll 359 KB 2.5 KB
Python headers? Yes ---
python38.dll? Yes
SWIG 30 MB ---
SWIG files 5680 files ---
WORKING RESULT No** Yes

BTW, everything involved in getting my example to work (C compiler,
creating the DLL, the interpreter, the implementation language for all
those, the implementation of the implemen, was 100% my own work).

Now, you might just begin to understand what my stuff is about. Which of
course I fully expect you to denigrate.

** I assume SWIG can be made to work, I just couldn't today.

I did one further test which was to get my C compiler to automatically
create the importdll block. The process is imperfect, but with minor
tweaks it worked. (I don't support DLL-exported variables ATM.)

On 27/05/2021 15:33, Bart wrote:
> On 27/05/2021 13:02, David Brown wrote:
>> On 27/05/2021 12:46, Bart wrote:
>
>> When people make libraries that are likely to be useful from many other
>> languages, they will often make interface libraries, headers, wrappers
>> or whatever - a C header, a Pascal unit, a Python extension library, a
>> TCL interface, etc.  Whatever makes sense for languages that are likely
>> to be used in conjunction with the library.  Often that will be done
>> using tools like SWIG which dramatically cut down on the effort involved.
>>
>> So who /actually/ finds it a problem that there are typedefs or macros
>> involved in determining the details of a function call in a C library?
>> Those who are using such unpopular (or new) languages that there are no
>> wrappers or bindings for their language, and who in addition cannot use
>> existing tools like SWIG or otherwise directly connect to C bindings.
>>
>> For other programmers, having types like "time_t", "uid_t", "uintptr_t",
>> and all the rest that you complain about, is a big advantage for
>> readability and portability.
>>
>> Should the rest of the programming world take a step backwards just to
>> make /your/ life a little easier?  Or should you learn that the
>> programming world does not revolve around you, and perhaps learn to use
>> tools like SWIG ?
>
> I had a quick look at SWIG, and it doesn't seem to me to be any sort of
> solution. I mean, not for the problems /I/ have. This tutorial
> (http://www.swig.org/tutorial.html) gives this example of C functions to
> be shared by scripting languages:

I am by no means an expert on SWIG. I mention it simply because I know
it is very popular and has been used to generate language bindings for a
wide variety of target languages and source libraries. So if /I/ had my
own language and wanted to get interfaces or bindings to existing
libraries, that would be where I would start looking.

>
> ----------------------------
>  #include <time.h>
>  double My_variable = 3.0;
>
>  int fact(int n) {
>      if (n <= 1) return 1;
>      else return n*fact(n-1);
>  }
>
>  int my_mod(int x, int y) {
>      return (x%y);
>  }
>     
>  char *get_time()
>  {
>      time_t ltime;
>      time(&ltime);
>      return ctime(&ltime);
>  }
>
> ----------------------------
>
> It suggests creating an interface file to SWIG that just duplicates the
> declarations needed. In practice this will be for a third party library
> with 100s of declarations.
>
> After this, you have to go through a process like this, this one for
> Python:
>
> ----------------------------
>
>  unix % swig -python example.i
>  unix % gcc -c example.c example_wrap.c \
>         -I/usr/local/include/python2.7
>  unix % ld -shared example.o example_wrap.o -o _example.so
> ----------------------------
>
> Then, you magically just do 'import example' in Python. Can Python
> directly import DLL/so files? Is there extra magic involved? Since no
> other files are produced according to this.
>

Yes, Python can import DLL/so files like that, I believe - /if/ they
contain the necessary Python-specific functions and data needed.
Presumably those are included in the file SWIG generates.

> But as I said, this doesn't help me. My starting point is already an
> existing DLL/so file, and an interface typically specified as a C
> header. Apart from which, SWIG supports only certain languages, not mine.

You start with a C header file that declares all the types and functions
you want from the library, and also includes any other headers needed
for those definitions. Then you use SWIG to make wrappers.

I don't know what SWIG outputs would suit you best, as it obviously
won't support your language directly. Perhaps it can generate a
simplified C interface that you can use, and it can certainly generate
an XML file that you could then process into your own language.

But if SWIG doesn't help (and if it is 80 MB then it is presumably
written off already by you as being too big to be useful to anyone),
don't use it. It was merely a suggestion on a tool that might help you.

>
>
>>> I tried this, you will find yourself typed up in knots when working with
>>> char and byte (uint8_t) types. My 'char' type is slightly different from
>>> 'byte', but I had to specially allow char* and byte* types to be
>>> compatible otherwise there'd be casts everywhere.
>>>
>>> (For example, you have a function read a file into memory and return a
>>> pointer to the data. But does it return char* or uint8_t*?)
>>>
>>
>> It would return a data8_t pointer.
>
> Unless you make data8_t* compatible with both char* and uint8_t*, then
> you will have problems when trying to use it as either of those latter two.
>

That is the point about strong typing - you are /supposed/ to have
trouble when faffing about mixing types. The type designer (whether it
is the language designer for built-in types, the library designer, or
the application programmer) decides what type conversions are allowed,
and how they are intended to work.

> It might be worth bearing mind that memory and disk capacities are
> generally measured in bytes, so that is already considered a de facto
> type for generic data.

data8_t /would/ be a byte.

On 27/05/2021 17:41, Scott Lurndal wrote:
> David Brown <david.brown@hesbynett.no> writes:
>> On 26/05/2021 23:27, Bart wrote:
>
>>> But the next one I know about that is supported by hardware is Intel's
>>> 80-bit float. double double would be misleading.
>>>
>>
>> 80-bit floats are a hangover from the days of external FPU coprocessors.
>> 128-bit quad precision floating point is the next big thing. They are
>> not yet supported in the x86 world, but are used on POWER9 and a few
>> other big processors.
>
> Oddly, the big new thing are 16-bit floats. Really. ARM, AMD, Intel
> all have been adding FP16 support. Primarily for machine learning applications.
>
> https://community.arm.com/developer/ip-products/processors/b/ml-ip-blog/posts/bfloat16-processing-for-neural-networks-on-armv8_2d00_a
>

Yes indeed. There seems to be a certain level of disagreement about the
format for 16-bit floats, but they are very efficient when you are
dealing with lots of them and doing massive SIMD vector operations.

On 27/05/2021 19:35, David Brown wrote:
> On 27/05/2021 15:33, Bart wrote:

>> I had a quick look at SWIG, and it doesn't seem to me to be any sort of
>> solution. I mean, not for the problems /I/ have. This tutorial
>> (http://www.swig.org/tutorial.html) gives this example of C functions to
>> be shared by scripting languages:
>
> I am by no means an expert on SWIG. I mention it simply because I know
> it is very popular and has been used to generate language bindings for a
> wide variety of target languages and source libraries. So if /I/ had my
> own language and wanted to get interfaces or bindings to existing
> libraries, that would be where I would start looking.

My follow-up posts showed that I made a serious attempt at trying the
example in the tutorial for Python, and it didn't work.

But giving it the benefit of a doubt, and assuming it would have done,
it's a far too heavyweight and elaborate a tool to perform what should
be a simple task.

I mentioned in my post the 1000s of lines of code and 100s of KB of DLL
code needed, but my own solution was basically 5 lines of code (and it
worked). The DLL was also 100x smaller (I don't know how much of that is
support code, and how much link options.)

So my own solution is simple and effortless. Except for one thing,
generating those 5 lines of code, comprising 3 declarations.

It's here that my problem lies, and the fact that an automatic solution
really requires implementing much of a C compiler. Even my automatic
tool is based on my C compiler.

What I suggested somewhere in the thread was an API format simple and
cross-platform enough to process with a simple script.

Note that most of those script languages that SWIG supports don't have
a built-in FFI for C-style APIs; you need loads of add-ons or tools lke
SWIG. My own script languages DO have that built-in, as well as the
ability to directly work with C-style structs, C-style arrays, C-style
pointers, and C-style primitives.

THAT is why SWIG is needed for those.

So the interface side is not the issue for me. It is the precise
information about the functions, types and so on that are exported from
a library, usually buried in C headers and C-style docs.

On 2021-05-27, Bart <bc@freeuk.com> wrote:
> On 27/05/2021 19:35, David Brown wrote:
>> On 27/05/2021 15:33, Bart wrote:
>
>>> I had a quick look at SWIG, and it doesn't seem to me to be any sort of
>>> solution. I mean, not for the problems /I/ have. This tutorial
>>> (http://www.swig.org/tutorial.html) gives this example of C functions to
>>> be shared by scripting languages:
>>
>> I am by no means an expert on SWIG. I mention it simply because I know
>> it is very popular and has been used to generate language bindings for a
>> wide variety of target languages and source libraries. So if /I/ had my
>> own language and wanted to get interfaces or bindings to existing
>> libraries, that would be where I would start looking.
>
> My follow-up posts showed that I made a serious attempt at trying the
> example in the tutorial for Python, and it didn't work.

I'm currently translating Simon Tatham's puzzles from C to TXR Lisp.

I'm doing the Win32 front end.

This is entirely written in C, using no UI libs at all: just Win32
calls.

The FFI handling is a complete breeze. I'm just through the windows.c
file from top to bottom, translating mostly function by function and
adding missing definitions to the FFI.

So far, the most tricky thing I had to translate is this:

char *geterrstr(void)
{ LPVOID lpMsgBuf;
DWORD dw = GetLastError();
char *ret;

FormatMessage(
FORMAT_MESSAGE_ALLOCATE_BUFFER |
FORMAT_MESSAGE_FROM_SYSTEM,
NULL,
dw,
MAKELANGID(LANG_NEUTRAL, SUBLANG_DEFAULT),
(LPTSTR) &lpMsgBuf,
0, NULL );

ret = dupstr(lpMsgBuf);

LocalFree(lpMsgBuf);

return ret;
}

The translation is this:

(defun geterrstr ()
(let* ((msgbuf (carray-blank 1 (ffi LPTSTR)))
(voidbuf (carray-pun msgbuf (ffi LPVOID)))
(dw (GetLastError)))
(FormatMessage (logior FORMAT_MESSAGE_ALLOCATE_BUFFER
FORMAT_MESSAGE_FROM_SYSTEM)
NULL dw 0
msgbuf
0 NULL)
(prog1 [msgbuf 0] (LocalFree [voidbuf 0]))))

There is no SWIG or anything; everything is manual, but the FFI is very
ergonomic and pleasant to work with. Adding missing Win32 functions is a very
small overhead compared to the rest of the work. Things like:

(with-dyn-lib "gdi32.dll"
(deffi DeleteObject "DeleteObject" BOOL (HGDIOBJ))
(deffi CreateCompatibleBitmap "CreateCompatibleBitmap" HBITMAP (HDC int int))
(deffi CreateCompatibleDC "CreateCompatibleDC" HDC (HDC))
(deffi SelectObject "SelectObject" HGDIOBJ (HDC HGDIOBJ))
(deffi BitBlt "BitBlt" BOOL (HDC int int int int HDC int int DWORD))
(deffi DeleteDC "DeleteDC" int (HDC))
(deffi SetTextColor "SetTextColor" COLORREF (HDC COLORREF))
(deffi CreateSolidBrush "CreateSolidBrush" HBRUSH (COLORREF))
(deffi CreateHatchBrush "CreateHatchBrush" HBRUSH (int COLORREF)))

The FORMAT_MESSAGE_ALLOCATE_BUFFER behavior of FormatMessage
simulates by-reference passing. The lpMsgBuf argument is a pointer to
a pointer which will receive the allocated buffer. We cannot just do that in
Lisp: cannot take the address of a variable and pass it. So we bind msgbuf
to a "C array" object of the right element type, of length 1. We also create
an alias whose element types is LPVOID.

The expression:

(prog1 [msgbuf 0] (LocalFree [voidbuf 0]))

means evaluate both [msgbuf ...] and (LocalFree ...) and then return the value
of the former. [voidbuf 0] produces a LPVOID object (alias for a "C pointer"
type), which is compatible with the call to LocalFree. [msgbuf 0] produces a
Lisp string.

The analog of this bit of logic:

ret = dupstr(lpMsgBuf);

is performed internally by [msgbuf 0]. The carray object's element type
LPTSTR ffi type is a typedef alias for the wstr type, which converts from
C to Lisp by allocating a Lisp string, without taking ownership.
The FFI is loaded with intelligent malloc-free handling with declarative
support for all common ownership transfer protocols.

LocalFree cannot be invoked on [msgbuf 0], because that's a string:

1> (LocalFree "abc")
** ffi-call: "abc" isn't a cptr

Test of geterrstr:

2> (progn (SetLastError 123) (geterrstr))
"The filename, directory name, or volume label syntax is incorrect.\r\n"
3> (progn (SetLastError 124) (geterrstr))
"The system call level is not correct.\r\n"
>
> What I suggested somewhere in the thread was an API format simple and
> cross-platform enough to process with a simple script.

There is such a thing. The Gnome object's "GObject Introspection".

> So the interface side is not the issue for me. It is the precise
> information about the functions, types and so on that are exported from
> a library, usually buried in C headers and C-style docs.

Gnome's GObject Introspection (GI) addresses this problem. Each binary library
(shared object) is accompanied by an XML description of its API: all the types
and functions. The description includes semantics that cannot be inferred from
the header file, like that a function allocates a certain pointer that the
caller must free.

GI only used by the Gnome project, as far as I know,
though; it's not a wide-spread standard. Still, there it is.

If you have a language with FFI, you can just process the GI XML and generate
bindings from that.

--
TXR Programming Language: http://nongnu.org/txr
Cygnal: Cygwin Native Application Library: http://kylheku.com/cygnal

On 27/05/2021 21:21, Kaz Kylheku wrote:
> On 2021-05-27, Bart <bc@freeuk.com> wrote:
>> On 27/05/2021 19:35, David Brown wrote:
>>> On 27/05/2021 15:33, Bart wrote:
>>
>>>> I had a quick look at SWIG, and it doesn't seem to me to be any sort of
>>>> solution. I mean, not for the problems /I/ have. This tutorial
>>>> (http://www.swig.org/tutorial.html) gives this example of C functions to
>>>> be shared by scripting languages:
>>>
>>> I am by no means an expert on SWIG. I mention it simply because I know
>>> it is very popular and has been used to generate language bindings for a
>>> wide variety of target languages and source libraries. So if /I/ had my
>>> own language and wanted to get interfaces or bindings to existing
>>> libraries, that would be where I would start looking.
>>
>> My follow-up posts showed that I made a serious attempt at trying the
>> example in the tutorial for Python, and it didn't work.
>
> I'm currently translating Simon Tatham's puzzles from C to TXR Lisp.
>
> I'm doing the Win32 front end.
>
> This is entirely written in C, using no UI libs at all: just Win32
> calls.
>
> The FFI handling is a complete breeze. I'm just through the windows.c
> file from top to bottom, translating mostly function by function and
> adding missing definitions to the FFI.

You mean windows.h? Depending on which version you have (belonging to
which compiler) it can be anything from 20,000 to 200,000 lines. How
many months did it take?

> So far, the most tricky thing I had to translate is this:
>
> char *geterrstr(void)
> {
> LPVOID lpMsgBuf;
> DWORD dw = GetLastError();
> char *ret;
>
> FormatMessage(
> FORMAT_MESSAGE_ALLOCATE_BUFFER |
> FORMAT_MESSAGE_FROM_SYSTEM,
> NULL,
> dw,
> MAKELANGID(LANG_NEUTRAL, SUBLANG_DEFAULT),
> (LPTSTR) &lpMsgBuf,
> 0, NULL );
>
> ret = dupstr(lpMsgBuf);
>
> LocalFree(lpMsgBuf);
>
> return ret;
> }
>
> The translation is this:
>
> (defun geterrstr ()
> (let* ((msgbuf (carray-blank 1 (ffi LPTSTR)))
> (voidbuf (carray-pun msgbuf (ffi LPVOID)))
> (dw (GetLastError)))
> (FormatMessage (logior FORMAT_MESSAGE_ALLOCATE_BUFFER
> FORMAT_MESSAGE_FROM_SYSTEM)
> NULL dw 0
> msgbuf
> 0 NULL)
> (prog1 [msgbuf 0] (LocalFree [voidbuf 0]))))
>
> There is no SWIG or anything; everything is manual, but the FFI is very
> ergonomic and pleasant to work with. Adding missing Win32 functions is a very
> small overhead compared to the rest of the work. Things like:

Working with Win32 is not so difficult, generally the docs are
comprehensive, but there's so much of it. The simplest task seems to
require dozens of functions, types, structs, constants, and macros, all
specific to the task.

I had a go at this one, for dynamic code which is usually a little more
challenging, and came up with this:

-------------------------------
import clib
import winapi

importdll kernel32 =
windows function "FormatMessageA" as formatmessage (
wt_dword flags,
wt_ptr source,
wt_dword messageid, languageid,
wt_string buffer,
wt_dword size,
...)wt_dword
windows function "LocalFree"(wt_handle)wt_handle
end

const fm_alloc_buffer = 0x100
const fm_from_system = 0x1000

function geterrstr=
msgbuf:=nil
error:=getlasterror()

formatmessage(
fm_alloc_buffer+fm_from_system,
nil, error, nil,
&msgbuf,
0, nil)

s:=makestr(msgbuf,strlen(msgbuf))
localfree(msgbuf)

return s
end

proc start=
println geterrstr()
end
-------------------------------

It displays:

The operation completed successfully.

(Of course, in an actual error situation, in between detecting an error,
and ending up calling this function, a script language especially could
end up calling multiple other WinAPI functions before it gets to call
GetLastError here. For example, if the message is to be displayed in a
pop-up.)

> The FORMAT_MESSAGE_ALLOCATE_BUFFER behavior of FormatMessage
> simulates by-reference passing. The lpMsgBuf argument is a pointer to
> a pointer which will receive the allocated buffer. We cannot just do that in
> Lisp: cannot take the address of a variable and pass it.

This is where my script language, although dynamic, is specially
designed to also work effortlessly with lower-level types.

So we bind msgbuf
> to a "C array" object of the right element type, of length 1. We also create
> an alias whose element types is LPVOID.
>
> The expression:
>
> (prog1 [msgbuf 0] (LocalFree [voidbuf 0]))
>
> means evaluate both [msgbuf ...] and (LocalFree ...) and then return the value
> of the former.

Which is a neat feature but which I've never found a decent syntax for.

>> What I suggested somewhere in the thread was an API format simple and
>> cross-platform enough to process with a simple script.

>
> There is such a thing. The Gnome object's "GObject Introspection".

>> So the interface side is not the issue for me. It is the precise
>> information about the functions, types and so on that are exported from
>> a library, usually buried in C headers and C-style docs.
>
> Gnome's GObject Introspection (GI) addresses this problem. Each binary library
> (shared object) is accompanied by an XML description of its API: all the types
> and functions. The description includes semantics that cannot be inferred from
> the header file, like that a function allocates a certain pointer that the
> caller must free.
>
> GI only used by the Gnome project, as far as I know,
> though; it's not a wide-spread standard. Still, there it is.
>
> If you have a language with FFI, you can just process the GI XML and generate
> bindings from that.

I'm sure there are plenty of things like this, but they also tend to be
monstrous. (In your link, I read as far as "XML, Python package,
scanner, typelib, a compiler to compile the typelib, a C library
'libgirepository'". That would be the rest of 2021 taken care of!)

All such solutions do not reduce the complexity, they just add extra
layers of complexity.

In any case, I usually strive to be self-sufficient, self-contained and
with minimum dependencies.

Ben Bacarisse <ben.usenet@bsb.me.uk> writes:
>scott@slp53.sl.home (Scott Lurndal) writes:
>
>> David Brown <david.brown@hesbynett.no> writes:
>>>On 26/05/2021 23:27, Bart wrote:
>>
>>>> But the next one I know about that is supported by hardware is Intel's
>>>> 80-bit float. double double would be misleading.
>>>>
>>>
>>>80-bit floats are a hangover from the days of external FPU coprocessors.
>>> 128-bit quad precision floating point is the next big thing. They are
>>>not yet supported in the x86 world, but are used on POWER9 and a few
>>>other big processors.
>>
>> Oddly, the big new thing are 16-bit floats. Really. ARM, AMD, Intel
>> all have been adding FP16 support. Primarily for machine learning
>> applications.
>
>(FP16 sometimes indicates IEEE's 16-bit float format using 1:5:10 bits
>which does not seem to be what's being added to these processors.)

Yeah, typo, I meant BF16. IEEE FP16 is an optional feature on
ARMv8 processors.

On 27/05/2021 16:39, Scott Lurndal wrote:
> Bart <bc@freeuk.com> writes:
>> On 27/05/2021 07:52, David Brown wrote:
>
>>> I can tell you want /I/ used when I wanted specific sized types before
>>> C99. I used typedefs for standard types. So I had code like:
>>>
>>> typedef unsigned char byte;
>>> typedef unsigned short int word;
>>> typedef unsigned long int dword;
>>> typedef signed char tiny;
>>> typedef signed short int sint;
>>> typedef signed long int lint;
>>>
>>>
>
>>
>> Your own choices are much better than anything in offical pre-C99 C.
>
> no, they're not. There is no consensus on the size of a word,
> much less a double word or quad word. Particularly in 1976 when
> common word sizes included 12, 16, 36 and 48 bits.
They're better first because unsigned types finally have their own
identity: 'word' rather than some category of 'int'.

Second because they can be expressed as a single token (keeping them
succinct, and reducing the ways they can be written)

As for there being no consensus, there is as much as there is for 'int'.
So if you know int's size, then you know word's size. And if you don't,
then you're probably not worried.

They are also very familiar to users of certain assemblers, where it is
long established that 'word' is 16 bits with dword and qword on top.

Admittedly they still don't give you guaranteed size-specific types. (I
didn't have those for a while either; I didn't write portable code so
sizes were implicit for a certain target. However if something was to
supposed to be wider or narrower, then it was.)

David Brown <david.brown@hesbynett.no> writes:
>On 27/05/2021 15:33, Bart wrote:

>>
>> I had a quick look at SWIG, and it doesn't seem to me to be any sort of
>> solution. I mean, not for the problems /I/ have. This tutorial
>> (http://www.swig.org/tutorial.html) gives this example of C functions to
>> be shared by scripting languages:
>
>I am by no means an expert on SWIG. I mention it simply because I know
>it is very popular and has been used to generate language bindings for a
>wide variety of target languages and source libraries. So if /I/ had my
>own language and wanted to get interfaces or bindings to existing
>libraries, that would be where I would start looking.

Our threaded SoC simulator is built as a shared object; it can be linked with
a simple main, or it can be linked with other applications, or using
a SWIG wrapper, can be used from python (subject to python's threading
limitations).

The SWIG wrapper is quite simple, and I think I had it working within
an hour of first installing the SWIG package.

scott@slp53.sl.home (Scott Lurndal) writes:

> Ben Bacarisse <ben.usenet@bsb.me.uk> writes:
>>scott@slp53.sl.home (Scott Lurndal) writes:
>>
>>> David Brown <david.brown@hesbynett.no> writes:
>>>>On 26/05/2021 23:27, Bart wrote:
>>>
>>>>> But the next one I know about that is supported by hardware is Intel's
>>>>> 80-bit float. double double would be misleading.
>>>>>
>>>>
>>>>80-bit floats are a hangover from the days of external FPU coprocessors.
>>>> 128-bit quad precision floating point is the next big thing. They are
>>>>not yet supported in the x86 world, but are used on POWER9 and a few
>>>>other big processors.
>>>
>>> Oddly, the big new thing are 16-bit floats. Really. ARM, AMD, Intel
>>> all have been adding FP16 support. Primarily for machine learning
>>> applications.
>>
>>(FP16 sometimes indicates IEEE's 16-bit float format using 1:5:10 bits
>>which does not seem to be what's being added to these processors.)
>
> Yeah, typo, I meant BF16. IEEE FP16 is an optional feature on
> ARMv8 processors.

Again, interesting. Thanks.

--
Ben.

On 27/05/2021 23:40, Bart wrote:
> On 27/05/2021 16:39, Scott Lurndal wrote:
>> Bart <bc@freeuk.com> writes:
>>> On 27/05/2021 07:52, David Brown wrote:
>>
>>>> I can tell you want /I/ used when I wanted specific sized types before
>>>> C99.  I used typedefs for standard types.  So I had code like:
>>>>
>>>> typedef unsigned char byte;
>>>> typedef unsigned short int word;
>>>> typedef unsigned long int dword;
>>>> typedef signed char tiny;
>>>> typedef signed short int sint;
>>>> typedef signed long int lint;
>>>>
>>>>
>>
>>>
>>> Your own choices are much better than anything in offical pre-C99 C.

These are written in "official pre-C99 C". Typedef is part of the language.

>>
>> no, they're not.  There is no consensus on the size of a word,
>> much less a double word or quad word.  Particularly in 1976 when
>> common word sizes included 12, 16, 36 and 48 bits.

Agreed - that's why I changed to use the <stdint.h> names as soon as I
saw them, even when working in C90.

>  They're better first because unsigned types finally have their own
> identity: 'word' rather than some category of 'int'.

It is not a better name - precisely because, as Scott says, people might
assume "word" is a certain size but that size varies by system.

Putting the size in the name is better when you want size-specific
types. About the only type you could reasonably say has a fixed, known
and agreed size would be "byte" - but even that is problematic if you
work with some kinds of DSPs. (And I have done so.)

You could reasonably work with "word8_t", "word16_t", "word32_t" if you
like the name "word" rather than "uint", and I presume you would want to
omit the "_t" suffix.

However, since these types here /are/ a kind of integer - unsigned
integer, an arithmetic type - "uint" seems entirely appropriate. As I
mentioned in other posts, if they were pure raw data, then "data<n>_t"
might be a better choice.

>
> Second because they can be expressed as a single token (keeping them
> succinct, and reducing the ways they can be written)
>

There I agree - I've never liked "unsigned" or "long" as they stand,
because to me they read as an adjective without a noun. And there's no
doubt that "unsigned long int" is wordy and there are occasions where
its multiple token nature is inconvenient. But that is in no way
different for "uint32_t" and "dword".

> As for there being no consensus, there is as much as there is for 'int'.
> So if you know int's size, then you know word's size. And if you don't,
> then you're probably not worried.
>

It is precisely because there is no consensus that /explicitly/ sized
types are better. Thus "uint32_t" is better than "dword" as a type name.

> They are also very familiar to users of certain assemblers, where it is
> long established that 'word' is 16 bits with dword and qword on top.
>

I did a lot more of my work in assembly in the "old days" - for the
small and cheap microcontrollers, C compilers were usually very poor and
very expensive. And that no doubt influenced by naming choices - but in
assembly, you are never in any doubt about the sizes you are dealing
with! However, I prefer to look forward more than looking back - I
don't see similarity with tools I mostly left behind 20 years ago as
being helpful.

> Admittedly they still don't give you guaranteed size-specific types. (I
> didn't have those for a while either; I didn't write portable code so
> sizes were implicit for a certain target. However if something was to
> supposed to be wider or narrower, then it was.)

On 28/05/2021 08:01, David Brown wrote:
> On 27/05/2021 23:40, Bart wrote:
>> On 27/05/2021 16:39, Scott Lurndal wrote:
>>> Bart <bc@freeuk.com> writes:
>>>> On 27/05/2021 07:52, David Brown wrote:
>>>
>>>>> I can tell you want /I/ used when I wanted specific sized types before
>>>>> C99.  I used typedefs for standard types.  So I had code like:
>>>>>
>>>>> typedef unsigned char byte;
>>>>> typedef unsigned short int word;
>>>>> typedef unsigned long int dword;
>>>>> typedef signed char tiny;
>>>>> typedef signed short int sint;
>>>>> typedef signed long int lint;
>>>>>
>>>>>
>>>
>>>>
>>>> Your own choices are much better than anything in offical pre-C99 C.
>
> These are written in "official pre-C99 C". Typedef is part of the language.

I was refering to your choices of user-identifiers. But you knew that of
course.

On 28/05/2021 11:04, Bart wrote:
> On 28/05/2021 08:01, David Brown wrote:
>> On 27/05/2021 23:40, Bart wrote:
>>> On 27/05/2021 16:39, Scott Lurndal wrote:
>>>> Bart <bc@freeuk.com> writes:
>>>>> On 27/05/2021 07:52, David Brown wrote:
>>>>
>>>>>> I can tell you want /I/ used when I wanted specific sized types
>>>>>> before
>>>>>> C99.  I used typedefs for standard types.  So I had code like:
>>>>>>
>>>>>> typedef unsigned char byte;
>>>>>> typedef unsigned short int word;
>>>>>> typedef unsigned long int dword;
>>>>>> typedef signed char tiny;
>>>>>> typedef signed short int sint;
>>>>>> typedef signed long int lint;
>>>>>>
>>>>>>
>>>>
>>>>>
>>>>> Your own choices are much better than anything in offical pre-C99 C.
>>
>> These are written in "official pre-C99 C".  Typedef is part of the
>> language.
>
> I was refering to your choices of user-identifiers. But you knew that of
> course.
>

Yes, I know that. And we both know that the only good reasons for
having user-defined size-specific general integer types in C are
compatibility/consistency with existing code, and that these type names
were not standardised before C99. (It's fine to have different names
for specific purposes, of course.) But it is still good to make it
clear that there is nothing in C90 hindering size-specific type names,
whether they are explicit (like "word16_t") or implicit (like "dword"),
or matching <stdint.h> names like "uint16_t".

On 28/05/2021 11:04, David Brown wrote:
> On 28/05/2021 11:04, Bart wrote:
>> On 28/05/2021 08:01, David Brown wrote:
>>> On 27/05/2021 23:40, Bart wrote:
>>>> On 27/05/2021 16:39, Scott Lurndal wrote:
>>>>> Bart <bc@freeuk.com> writes:
>>>>>> On 27/05/2021 07:52, David Brown wrote:
>>>>>
>>>>>>> I can tell you want /I/ used when I wanted specific sized types
>>>>>>> before
>>>>>>> C99.  I used typedefs for standard types.  So I had code like:
>>>>>>>
>>>>>>> typedef unsigned char byte;
>>>>>>> typedef unsigned short int word;
>>>>>>> typedef unsigned long int dword;
>>>>>>> typedef signed char tiny;
>>>>>>> typedef signed short int sint;
>>>>>>> typedef signed long int lint;
>>>>>>>
>>>>>>>
>>>>>
>>>>>>
>>>>>> Your own choices are much better than anything in offical pre-C99 C.
>>>
>>> These are written in "official pre-C99 C".  Typedef is part of the
>>> language.
>>
>> I was refering to your choices of user-identifiers. But you knew that of
>> course.
>>
>
> Yes, I know that. And we both know that the only good reasons for
> having user-defined size-specific general integer types in C are
> compatibility/consistency with existing code, and that these type names
> were not standardised before C99.

Look at my thread on FFI bindings, where I show 40+ lines of code that
SDL uses to ensure uint32_t etc are properly defined (which doesn't
always work).

And the first thing they do with those types? It's typedefs like this:

typedef uint32_t Uint32;

Why would they do that? Even if Uint32 was used pre-C99, it's a simple
global replacement to change them to uint32_t.

2021 (SDL was a fresh download today) and it's still a palaver in C just
to get a u32 type, or even to predict what any application might choose
to use.

In Rust, it's "u32" ALWAYS; in C, it's anyone's guess!

On 28/05/2021 12:17, Bart wrote:
> On 28/05/2021 11:04, David Brown wrote:
>> On 28/05/2021 11:04, Bart wrote:
>>> On 28/05/2021 08:01, David Brown wrote:
>>>> On 27/05/2021 23:40, Bart wrote:
>>>>> On 27/05/2021 16:39, Scott Lurndal wrote:
>>>>>> Bart <bc@freeuk.com> writes:
>>>>>>> On 27/05/2021 07:52, David Brown wrote:
>>>>>>
>>>>>>>> I can tell you want /I/ used when I wanted specific sized types
>>>>>>>> before
>>>>>>>> C99.  I used typedefs for standard types.  So I had code like:
>>>>>>>>
>>>>>>>> typedef unsigned char byte;
>>>>>>>> typedef unsigned short int word;
>>>>>>>> typedef unsigned long int dword;
>>>>>>>> typedef signed char tiny;
>>>>>>>> typedef signed short int sint;
>>>>>>>> typedef signed long int lint;
>>>>>>>>
>>>>>>>>
>>>>>>
>>>>>>>
>>>>>>> Your own choices are much better than anything in offical pre-C99 C.
>>>>
>>>> These are written in "official pre-C99 C".  Typedef is part of the
>>>> language.
>>>
>>> I was refering to your choices of user-identifiers. But you knew that of
>>> course.
>>>
>>
>> Yes, I know that.  And we both know that the only good reasons for
>> having user-defined size-specific general integer types in C are
>> compatibility/consistency with existing code, and that these type names
>> were not standardised before C99.
>
>
> Look at my thread on FFI bindings, where I show 40+ lines of code that
> SDL uses to ensure uint32_t etc are properly defined (which doesn't
> always work).
>
> And the first thing they do with those types? It's typedefs like this:
>
>   typedef uint32_t Uint32;
>
> Why would they do that? Even if Uint32 was used pre-C99, it's a simple
> global replacement to change them to uint32_t.

Why don't you ask the SDL folks? /I/ didn't write it. I could guess
that it is because the history of the SDL code goes back to pre-C99, and
because you don't do global replacement in code that is established and
working. You can do it in little one-person projects, but you don't do
it in serious multi-person projects - it screws up everyone's copy of
the code, their modified branches, the version control history,
comparisons between versions to see what has changed, etc.

>
> 2021 (SDL was a fresh download today) and it's still a palaver in C just
> to get a u32 type, or even to predict what any application might choose
> to use.
>
> In Rust, it's "u32" ALWAYS; in C, it's anyone's guess!

On 28/05/2021 11:49, David Brown wrote:
> On 28/05/2021 12:17, Bart wrote:
>> On 28/05/2021 11:04, David Brown wrote:
>>> On 28/05/2021 11:04, Bart wrote:
>>>> On 28/05/2021 08:01, David Brown wrote:
>>>>> On 27/05/2021 23:40, Bart wrote:
>>>>>> On 27/05/2021 16:39, Scott Lurndal wrote:
>>>>>>> Bart <bc@freeuk.com> writes:
>>>>>>>> On 27/05/2021 07:52, David Brown wrote:
>>>>>>>
>>>>>>>>> I can tell you want /I/ used when I wanted specific sized types
>>>>>>>>> before
>>>>>>>>> C99.  I used typedefs for standard types.  So I had code like:
>>>>>>>>>
>>>>>>>>> typedef unsigned char byte;
>>>>>>>>> typedef unsigned short int word;
>>>>>>>>> typedef unsigned long int dword;
>>>>>>>>> typedef signed char tiny;
>>>>>>>>> typedef signed short int sint;
>>>>>>>>> typedef signed long int lint;
>>>>>>>>>
>>>>>>>>>
>>>>>>>
>>>>>>>>
>>>>>>>> Your own choices are much better than anything in offical pre-C99 C.
>>>>>
>>>>> These are written in "official pre-C99 C".  Typedef is part of the
>>>>> language.
>>>>
>>>> I was refering to your choices of user-identifiers. But you knew that of
>>>> course.
>>>>
>>>
>>> Yes, I know that.  And we both know that the only good reasons for
>>> having user-defined size-specific general integer types in C are
>>> compatibility/consistency with existing code, and that these type names
>>> were not standardised before C99.
>>
>>
>> Look at my thread on FFI bindings, where I show 40+ lines of code that
>> SDL uses to ensure uint32_t etc are properly defined (which doesn't
>> always work).
>>
>> And the first thing they do with those types? It's typedefs like this:
>>
>>   typedef uint32_t Uint32;
>>
>> Why would they do that? Even if Uint32 was used pre-C99, it's a simple
>> global replacement to change them to uint32_t.
>
> Why don't you ask the SDL folks?

Why just SDL? It's symptomatic of a lot of C programs I see. A lot of
mucking about before you even get to the same start point as other
languages.

> /I/ didn't write it. I could guess
> that it is because the history of the SDL code goes back to pre-C99, and
> because you don't do global replacement in code that is established and
> working.

Um, yes you can. Except I don't know how much types like Uint32 actually
need to appear as types (rather than as literals, or ordinary,
compatible uint32_t's) in applications that use the library. Even then,
that is easily managed with one small, deprecated compatibility block.

You have a new, lean, streamlined interface for new projects, and
require a compatibility header for old ones.

> You can do it in little one-person projects, but you don't do
> it in serious multi-person projects - it screws up everyone's copy of
> the code, their modified branches, the version control history,
> comparisons between versions to see what has changed, etc.

How much screwing up would getting rid of those 40 lines of
initialisation code and just using stdint.h entail? It won't work on a
compiler from 1994?

I'm not surprised so much C code is such a patchword mess of #ifdef
blocks to suport a dozen obsolete C implementations if this is the
general attitude.

(BTW those SDL headers repeat that same 40-line block I think 4 times in
all, in different, platform-specific headers. Plus there are a bunch of
types like these:

typedef uint32_t khronos_uint32_t;

People sure do like creating their own types!)

On 28/05/2021 14:02, Bart wrote:
> On 28/05/2021 11:49, David Brown wrote:
>> On 28/05/2021 12:17, Bart wrote:
>>> On 28/05/2021 11:04, David Brown wrote:
>>>> On 28/05/2021 11:04, Bart wrote:
>>>>> On 28/05/2021 08:01, David Brown wrote:
>>>>>> On 27/05/2021 23:40, Bart wrote:
>>>>>>> On 27/05/2021 16:39, Scott Lurndal wrote:
>>>>>>>> Bart <bc@freeuk.com> writes:
>>>>>>>>> On 27/05/2021 07:52, David Brown wrote:
>>>>>>>>
>>>>>>>>>> I can tell you want /I/ used when I wanted specific sized types
>>>>>>>>>> before
>>>>>>>>>> C99.  I used typedefs for standard types.  So I had code like:
>>>>>>>>>>
>>>>>>>>>> typedef unsigned char byte;
>>>>>>>>>> typedef unsigned short int word;
>>>>>>>>>> typedef unsigned long int dword;
>>>>>>>>>> typedef signed char tiny;
>>>>>>>>>> typedef signed short int sint;
>>>>>>>>>> typedef signed long int lint;
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>
>>>>>>>>>
>>>>>>>>> Your own choices are much better than anything in offical
>>>>>>>>> pre-C99 C.
>>>>>>
>>>>>> These are written in "official pre-C99 C".  Typedef is part of the
>>>>>> language.
>>>>>
>>>>> I was refering to your choices of user-identifiers. But you knew
>>>>> that of
>>>>> course.
>>>>>
>>>>
>>>> Yes, I know that.  And we both know that the only good reasons for
>>>> having user-defined size-specific general integer types in C are
>>>> compatibility/consistency with existing code, and that these type names
>>>> were not standardised before C99.
>>>
>>>
>>> Look at my thread on FFI bindings, where I show 40+ lines of code that
>>> SDL uses to ensure uint32_t etc are properly defined (which doesn't
>>> always work).
>>>
>>> And the first thing they do with those types? It's typedefs like this:
>>>
>>>    typedef uint32_t Uint32;
>>>
>>> Why would they do that? Even if Uint32 was used pre-C99, it's a simple
>>> global replacement to change them to uint32_t.
>>
>> Why don't you ask the SDL folks?
>
> Why just SDL? It's symptomatic of a lot of C programs I see. A lot of
> mucking about before you even get to the same start point as other
> languages.
>

Your cherry-picking is getting tedious. You've made up some crazy rules
about what you alone think is something bad in C coding, simply because
they don't match your personal ideas about what would fit your personal
language, your personal coding style, your personal needs. Then you
find examples of code that breaks these rules of yours, and throw your
toys out your pram over it.

At what point in your life did you decide it was better to hate
programming and devote yourself to complaining and finding ways to make
life difficult for yourself, instead of being constructive and using
languages (flawed or not) for creating something?

I've tried constructive dialogue, discussing suggestions, my own
opinions, ideas that might be helpful to you. But in the end your
negativity is breaking down my desire to help.

On 28/05/2021 17:11, David Brown wrote:
> On 28/05/2021 14:02, Bart wrote:

>> Why just SDL? It's symptomatic of a lot of C programs I see. A lot of
>> mucking about before you even get to the same start point as other
>> languages.
>>
>
> Your cherry-picking is getting tedious. You've made up some crazy rules
> about what you alone think is something bad in C coding, simply because
> they don't match your personal ideas about what would fit your personal
> language, your personal coding style, your personal needs. Then you
> find examples of code that breaks these rules of yours, and throw your
> toys out your pram over it.
>
> At what point in your life did you decide it was better to hate
> programming and devote yourself to complaining and finding ways to make
> life difficult for yourself, instead of being constructive and using
> languages (flawed or not) for creating something?
>
> I've tried constructive dialogue, discussing suggestions, my own
> opinions, ideas that might be helpful to you.

I'm devoting myself to making things simpler, not harder!

Yet you are determinated to always take the opposite view.

C APis /are/ too C-centric when it comes to using those from arbitrary
languages. That makes it harder than necessary to use either manual or
automatic methods to create FFI bindings.

One solution is to take a different approach:

* Use a standard, well-defined set of types in APIs

* Avoid gratuituous typedefs

* Avoid macros

* Avoid conditional code

* Avoid nested includes

* And in the case of SDL, avoid code that only favours certain compilers

* In short, avoid /having/ to use a C compiler, or preprocessor or
parser simply in order to extract the information out of an API.

Of course, this is never going to happen, because nobody cares. Either
they use C anyway, or somebody else has to do all the hard work. Or has
devised some super-heavy-weight approach such as SWIG or that GObject
system.

If C-centric APIs represent a mountain of difficulties, then rather than
suggest turning that into a mere hill, you propose using a tool which is
an even bigger mountain!

As for SWIG, I notice you conveniently ignored the fact that, while
using SWIG for that tutorial involved:

* A 30MB SWIG installation
* Having to manually enumerate all the entities that need exporting
* Generating nearly 4000 lines of C and Python code
* Generating a 350KB DLL (and in the end it didn't work)

my solution required only 5 lines of code and only a 2.5KB DLL (and it
worked).

Instead a mountain, it's only a molehill!

I'd like to know on which planet such a solution would make things
harder rather than simpler.

However the main obstacle to a fully automatic approach is still those
issues I listed. C /as it is normally used/ is a terrible way to present
those APIs.

C with certain strict guidelines would be better.

I can only assume that you have no idea about this stuff because you've
never had to do it.

-------------------------------------

Here's a challenge for you: write a self-contained program up to 1000
lines that can take an API expressed in the form of C headers such as
sdl.h (remember there are 80 sub-headers), and output a machine-readable
summary of:

* exported functions, with parameter and return types all expressed as
plain size-specific sizes (to simplify, you can assume a specific target)

* exported variables

* exported macros, ie, ones that are need to be exposed for use in
applications

* exported user-types and structs

* exported enums

The program should not output the contents of system headers such as
stdlib.h or windows.h.

If you think that's not possible in 1000 lines, then see if you can
devise a simpler format (either a new syntax, or restricted, stylised C)
to express that information, where it IS possible.

At that point, you might finally get what this is about.

(Note that with C headers, the language cannot distinguish between
exported and non-exported elements. It can only see static and
non-static variable and functions, and static ones normally wouldn't be
in a header.)

> But in the end your
> negativity is breaking down my desire to help.

How about someone finally admitting that all these issues with C do
actually make all the above harder? Then I'll stop. It's people
pretending it's all really so wonderful and that there's nothing wrong
with it, it's just me moaning for no readon, that winds me up.)

On 28/05/2021 19:12, Bart wrote:
>
> Here's a challenge for you: write a self-contained program up to 1000
> lines that can take an API expressed in the form of C headers such as
> sdl.h (remember there are 80 sub-headers), and output a machine-readable
> summary of:

Here's a challenge for /you/ - stop making stupid artificial
restrictions that serve no purpose but to fuel your paranoia about how
everything related to C is designed specifically to annoy you.

Why would anybody write such a program, when they could start with SWIG
and get the information? Who cares if SWIG is big? It does a lot - but
if it also does what is needed here, then it does not matter if it does
everything else.

And if SWIG doesn't work, why not use clang/llvm as a library to do the
C parsing? Or gcc, looking at the pre-processor output, or debug
output, or generating new programs full of printf sizeof lines and run them?

There is a time and place for compact and efficient code. When you are
writing a computer game, or a microcontroller program, saving run-time
and code space gives a better game or a cheaper device. But for
something like this, it does not matter in the slightest if you have to
install a complete virtual machine, or you have a concoction of scripts
in six different languages that takes 5 GB of memory and half an hour to
run - it is saving /developer/ time and effort.

And you could probably have written a Python script in less than 1000
lines that do all you want, in a fraction of the time you've spent
crying about how evil C is. Of course, it might require you to be able
to run a proper C compiler.

On 2021-05-28, David Brown <david.brown@hesbynett.no> wrote:
> On 28/05/2021 19:12, Bart wrote:
>>
>> Here's a challenge for you: write a self-contained program up to 1000
>> lines that can take an API expressed in the form of C headers such as
>> sdl.h (remember there are 80 sub-headers), and output a machine-readable
>> summary of:
>
> Here's a challenge for /you/ - stop making stupid artificial
> restrictions that serve no purpose but to fuel your paranoia about how
> everything related to C is designed specifically to annoy you.

I think that in a parallel universe in which the Unix world switched
to Modula-2 in 1982 all the same issues exist.

In this parallel universe, there are Modula-2 compilers in which long
and regular integers have been made the same, due to the bigs witch to
32 bit computing.

Size-specific types have been introduced into the language.

Multiple implementations of POSIX from different vendors use slightly
different versions of Modula-2 records with differently sized members
and different quantities and sizes of undocumented members, making
FFI tricky.

In this parallel universe, Modula-2 doesn't have header files with
macros, but .def have come to specify inline functions which are,
of course, in Modula-2 syntax. Non-inline versions of ithese functions
are often missing from the compiled objects, so everyone must parse
Modula-2 syntax to re-create them or else roll them by hand.

There is an active comp.lang.modula2 newsgroup, which is trolled by
people who want to change the language into something else, while they
completely ignore comp.std.modula2 where that is topical.

On 28/05/2021 18:29, David Brown wrote:
> On 28/05/2021 19:12, Bart wrote:
>>
>> Here's a challenge for you: write a self-contained program up to 1000
>> lines that can take an API expressed in the form of C headers such as
>> sdl.h (remember there are 80 sub-headers), and output a machine-readable
>> summary of:
>
> Here's a challenge for /you/ - stop making stupid artificial
> restrictions that serve no purpose but to fuel your paranoia about how
> everything related to C is designed specifically to annoy you.
>
> Why would anybody write such a program, when they could start with SWIG
> and get the information? Who cares if SWIG is big? It does a lot - but
> if it also does what is needed here, then it does not matter if it does
> everything else.
>
> And if SWIG doesn't work, why not use clang/llvm as a library to do the
> C parsing? Or gcc, looking at the pre-processor output, or debug
> output, or generating new programs full of printf sizeof lines and run them?
>
> There is a time and place for compact and efficient code. When you are
> writing a computer game, or a microcontroller program, saving run-time
> and code space gives a better game or a cheaper device. But for
> something like this, it does not matter in the slightest if you have to
> install a complete virtual machine, or you have a concoction of scripts
> in six different languages that takes 5 GB of memory and half an hour to
> run - it is saving /developer/ time and effort.

Apparently the people who created SWIG cared. That's why they created
the product, to cut down on boilerplate and make things simpler and easier.

Good thing they didn't have you to advise and tell them they were
wasting their time!

We'll all trying to make things a litle better (obviously not you; you
just use off-the-shelf the tools that others have created to make your
life easier, people who identified short-comings).

Anyway, SWIG solves a different set of problems than what I'm interested in:

* It's managing the unwieldly interfaces from scripting languages to C/C++

* It doesn't try and do 100% of the job of extracting exported info; it
expects the user to create an interface file listing what is to be exported

* It seems to be designed for people writing the actual libraries, since
it expects to be able to build them from sources into a DLL/so.

* Those people will be familiar with the ins and outs of their libraries

My requirement is different:

* I'm working with third party libraries, with existing binary DLL/so files

* I only have C header files for the libraries and possibly some docs

* I don't need help with interfacing my language to a low-level API and
low-level data types; I just need the signatures of the functions ...

* ... plus the types, structs, enums and named constants - NOT IN THE
FORM OF C MACROS - which are necessary to be able to use the library.
Since I'm not the author, I won't know what they are.

This may seem a trivial point, but to give an idea the scale of the
problem, these are the number of lines of export info that my own tool
extracts from these header files:

sdl.h 2800 lines
windows.h 18000 (based on one of the smallest windows.h headers)
gtk.h 25000

> And you could probably have written a Python script in less than 1000
> lines that do all you want, in a fraction of the time you've spent
> crying about how evil C is.

Yeah, sure. That's why SWIG is such a tiny, simple program, one that
took 25 years to write.

For one thing, any such tool will need to incorporate a C preprocessor.
You can't just use a C compiler to preprocess either, since that wil
destroy #defines that must be exported.

I'm coming to the conclusion that you don't know what you're talking about.

On 28/05/2021 19:49, Kaz Kylheku wrote:
> On 2021-05-28, David Brown <david.brown@hesbynett.no> wrote:
>> On 28/05/2021 19:12, Bart wrote:
>>>
>>> Here's a challenge for you: write a self-contained program up to 1000
>>> lines that can take an API expressed in the form of C headers such as
>>> sdl.h (remember there are 80 sub-headers), and output a machine-readable
>>> summary of:
>>
>> Here's a challenge for /you/ - stop making stupid artificial
>> restrictions that serve no purpose but to fuel your paranoia about how
>> everything related to C is designed specifically to annoy you.
>
> I think that in a parallel universe in which the Unix world switched
> to Modula-2 in 1982 all the same issues exist.
>
> In this parallel universe, there are Modula-2 compilers in which long
> and regular integers have been made the same, due to the bigs witch to
> 32 bit computing.
>
> Size-specific types have been introduced into the language.
>
> Multiple implementations of POSIX from different vendors use slightly
> different versions of Modula-2 records with differently sized members
> and different quantities and sizes of undocumented members, making
> FFI tricky.
>
> In this parallel universe, Modula-2 doesn't have header files with
> macros, but .def have come to specify inline functions which are,
> of course, in Modula-2 syntax. Non-inline versions of ithese functions
> are often missing from the compiled objects, so everyone must parse
> Modula-2 syntax to re-create them or else roll them by hand.
>
> There is an active comp.lang.modula2 newsgroup, which is trolled by
> people who want to change the language into something else, while they
> completely ignore comp.std.modula2 where that is topical

Alternatively there could be a parallel universe in which APIs look like
this:

clang proc "glAccum" (u32,r32)
clang proc "glAlphaFunc" (u32,r32)
clang function "glAreTexturesResident" (i32,ref u32,ref
byte)byte
clang proc "glArrayElement" (i32)
clang proc "glBegin" (u32)

A format almost simple enough you could process in a read-line loop
(nowhere near 1000 lines either). And it could be made simpler still.

Oh, wait, that format already exists in /this/ universe.

My languages ONLY interface to external libraries via DLL files.

Those interface files already contain the names of exported functions
and variables. What they don't contain are all the relevant types. Nor
exported user-types, structs, and enums.

Imagine if a compiler, with a simple switch, added such information when
generating a DLL file.

Now C header files would be practically obsolete! Only C would still
need them. Languages that are more with it would just say:

importd lib # import lib.dll directly

and all the declarations necessary to use the library would be instantly
available, in machine-readable format.

Wouldn't that be brilliant? No, not according to the luddites here; who
on earth would want such a sensible feature? What would they do with all
those wonderful tools?

On 2021-05-28, Bart <bc@freeuk.com> wrote:
> On 28/05/2021 19:49, Kaz Kylheku wrote:
>> On 2021-05-28, David Brown <david.brown@hesbynett.no> wrote:
>>> On 28/05/2021 19:12, Bart wrote:
>>>>
>>>> Here's a challenge for you: write a self-contained program up to 1000
>>>> lines that can take an API expressed in the form of C headers such as
>>>> sdl.h (remember there are 80 sub-headers), and output a machine-readable
>>>> summary of:
>>>
>>> Here's a challenge for /you/ - stop making stupid artificial
>>> restrictions that serve no purpose but to fuel your paranoia about how
>>> everything related to C is designed specifically to annoy you.
>>
>> I think that in a parallel universe in which the Unix world switched
>> to Modula-2 in 1982 all the same issues exist.
>>
>> In this parallel universe, there are Modula-2 compilers in which long
>> and regular integers have been made the same, due to the bigs witch to
>> 32 bit computing.
>>
>> Size-specific types have been introduced into the language.
>>
>> Multiple implementations of POSIX from different vendors use slightly
>> different versions of Modula-2 records with differently sized members
>> and different quantities and sizes of undocumented members, making
>> FFI tricky.
>>
>> In this parallel universe, Modula-2 doesn't have header files with
>> macros, but .def have come to specify inline functions which are,
>> of course, in Modula-2 syntax. Non-inline versions of ithese functions
>> are often missing from the compiled objects, so everyone must parse
>> Modula-2 syntax to re-create them or else roll them by hand.
>>
>> There is an active comp.lang.modula2 newsgroup, which is trolled by
>> people who want to change the language into something else, while they
>> completely ignore comp.std.modula2 where that is topical
>
> Alternatively there could be a parallel universe in which APIs look like
> this:
>
> clang proc "glAccum" (u32,r32)
> clang proc "glAlphaFunc" (u32,r32)
> clang function "glAreTexturesResident" (i32,ref u32,ref
> byte)byte
> clang proc "glArrayElement" (i32)
> clang proc "glBegin" (u32)
>
>
> A format almost simple enough you could process in a read-line loop
> (nowhere near 1000 lines either). And it could be made simpler still.
>
> Oh, wait, that format already exists in /this/ universe.
>
>
> My languages ONLY interface to external libraries via DLL files.
>
> Those interface files already contain the names of exported functions
> and variables. What they don't contain are all the relevant types. Nor
> exported user-types, structs, and enums.
>
> Imagine if a compiler, with a simple switch, added such information when
> generating a DLL file.
>
> Now C header files would be practically obsolete! Only C would still
> need them. Languages that are more with it would just say:
>
> importd lib # import lib.dll directly
>
> and all the declarations necessary to use the library would be instantly
> available, in machine-readable format.
>
> Wouldn't that be brilliant? No, not according to the luddites here; who
> on earth would want such a sensible feature? What would they do with all
> those wonderful tools?
It's too late to change that. You would need different dll format as
well. Only way you can write type arguments is via symbols in functions,
which is not portable even from compiler to compiler...
>

--
current job title: senior software engineer
skills: x86 aasembler,c++,c,rust,go,nim,haskell...

press any key to continue or any other to quit...