Kaz Kylheku <433-929-6894@kylheku.com> writes:
> On 2023-12-29, Bart <bc@freeuk.cm> wrote:
[...]
>> (I guess this is something that is harder on Linux because there, many
>> standard headers are not part of a specific C compiler, but are a
>> resource shared by all C compilers, or tools that need to process C
>> headers.)
>
> Headers on GNU/Linux systems tend to assume GCC. Clang would not be
> usable did it not have GCC compatibility.

On my Ubuntu 22.04 system, tcc manages to use the system headers, which
are mostly provided by glibc. In a quick glance at /usr/include/stdio.h,
I see some #ifdefs for symbols like __GNUC__ (which is predefined by gcc
and clang but not by tcc) and __USE_GNU (I haven't bothered to look into
how that's defined).

--
Keith Thompson (The_Other_Keith) Keith.S.Thompson+u@gmail.com
Will write code for food.
void Void(void) { Void(); } /* The recursive call of the void */

Janis Papanagnou <janis_papanagnou+ng@hotmail.com> writes:
[...]
> BTW, is 'inline' meanwhile C standard? (I know that from C++ but
> haven't done much C for long now.)

C added inline in C99 (the 1999 edition of the ISO C standard, the same
one that removed implicit int).

I think C and C++ have subtly different semantics for inline.

--
Keith Thompson (The_Other_Keith) Keith.S.Thompson+u@gmail.com
Will write code for food.
void Void(void) { Void(); } /* The recursive call of the void */

On 2023-12-29, Keith Thompson <Keith.S.Thompson+u@gmail.com> wrote:
> Kaz Kylheku <433-929-6894@kylheku.com> writes:
>> On 2023-12-29, Bart <bc@freeuk.cm> wrote:
> [...]
>>> (I guess this is something that is harder on Linux because there, many
>>> standard headers are not part of a specific C compiler, but are a
>>> resource shared by all C compilers, or tools that need to process C
>>> headers.)
>>
>> Headers on GNU/Linux systems tend to assume GCC. Clang would not be
>> usable did it not have GCC compatibility.
>
> On my Ubuntu 22.04 system, tcc manages to use the system headers, which
> are mostly provided by glibc. In a quick glance at /usr/include/stdio.h,
> I see some #ifdefs for symbols like __GNUC__ (which is predefined by gcc
> and clang but not by tcc) and __USE_GNU (I haven't bothered to look into
> how that's defined).

__GNUC__ would be a definite ever-present signal from the compiler.

__USE_GNU is the internal feature selector corresponding to when you
use the externally documented _GNU_SOURCE.

--
TXR Programming Language: http://nongnu.org/txr
Cygnal: Cygwin Native Application Library: http://kylheku.com/cygnal
Mastodon: @Kazinator@mstdn.ca
NOTE: If you use Google Groups, I don't see you, unless you're whitelisted.

On 2023-12-29, Keith Thompson <Keith.S.Thompson+u@gmail.com> wrote:
> David Brown <david.brown@hesbynett.no> writes:
>> A useful tool that someone might like to write for this particular
>> situation would be a partial C preprocessor, letting you choose what
>> gets handled. You could choose to expand the code here for, say,
>> _GNU_SOURCE and _BSD_SOURCE - any use of these in #ifdef's and
>> conditional compilation would be expanded according to whether you
>> have defined the symbols or not, leaving an output that is easier to
>> understand while keeping most of the pre-processor stuff unchanged (so
>> not affecting #includes, and leaving #define'd macros and constants
>> untouched and therefore more readable).
>
> The unifdef tool does some of this. (I haven't used it much.)

GNU cpp has an option which is something like this: -fdirectives-only.
It causes it not to expand macros.

However, I don't think there is any way to prevent the removal of
comments, which could be a deal breaker.

--
TXR Programming Language: http://nongnu.org/txr
Cygnal: Cygwin Native Application Library: http://kylheku.com/cygnal
Mastodon: @Kazinator@mstdn.ca
NOTE: If you use Google Groups, I don't see you, unless you're whitelisted.

Keith Thompson <Keith.S.Thompson+u@gmail.com> writes:
>Janis Papanagnou <janis_papanagnou+ng@hotmail.com> writes:
>[...]
>> BTW, is 'inline' meanwhile C standard? (I know that from C++ but
>> haven't done much C for long now.)
>
>C added inline in C99 (the 1999 edition of the ISO C standard, the same
>one that removed implicit int).
>
>I think C and C++ have subtly different semantics for inline.

Mostly related to symbol visibility, IIRC.

On 29/12/2023 20:23, Kaz Kylheku wrote:
> On 2023-12-29, Keith Thompson <Keith.S.Thompson+u@gmail.com> wrote:
>> David Brown <david.brown@hesbynett.no> writes:
>>> A useful tool that someone might like to write for this particular
>>> situation would be a partial C preprocessor, letting you choose what
>>> gets handled. You could choose to expand the code here for, say,
>>> _GNU_SOURCE and _BSD_SOURCE - any use of these in #ifdef's and
>>> conditional compilation would be expanded according to whether you
>>> have defined the symbols or not, leaving an output that is easier to
>>> understand while keeping most of the pre-processor stuff unchanged (so
>>> not affecting #includes, and leaving #define'd macros and constants
>>> untouched and therefore more readable).
>>
>> The unifdef tool does some of this. (I haven't used it much.)
>
> GNU cpp has an option which is something like this: -fdirectives-only.
> It causes it not to expand macros.

It flattens include files, processes conditionals, and keeps #defines
unchanged.

However, it turns gcc's sys/stat.h from 300 lines into 3000 lines.

If I apply it to my stat.h (also my stddef.h which it includes), which
are 110 lines together, it produces 900 lines. Most of that consists of
lots of built-in #defines with __ prefixes (each complete with a line
saying it is built-in).

When I use my own conversion tool (designed to turn C headers that
define APIs into declarations in my language), the output is 65 lines.

The gcc option does not expand typedefs or macros. So if there is a
declaration using a type which uses both, that is unchanged, which is
not helpful. (At least not if trying to create bindings for your FFI.)

gcc with just -E will expand macros but still keep typedefs.

So, for producing a streamlined standard header, it still leaves a lot
to be desired. And for trying to flatten layers of macros and typedefs,
to reveal the underlying types, it's not that great either.

Purpose-built tools are always better, but dealing with C is not trivial
anyway, and dealing with gnu- and gcc-specific features is even harder.

On Fri, 29 Dec 2023 22:40:47 +0000, Bart wrote:

> It flattens include files, processes conditionals, and keeps #defines
> unchanged.
>
> However, it turns gcc's sys/stat.h from 300 lines into 3000 lines.

Still, merging all the stuff into fewer files likely means it loads
faster.

Includes files are kludge, and all these attempts to improve them are a
kludge on top of a kludge. This is why better-designed languages have a
proper module system that solves the whole issue.

On 30/12/2023 01:28, Lawrence D'Oliveiro wrote:
> On Fri, 29 Dec 2023 22:40:47 +0000, Bart wrote:
>
>> It flattens include files, processes conditionals, and keeps #defines
>> unchanged.
>>
>> However, it turns gcc's sys/stat.h from 300 lines into 3000 lines.
>
> Still, merging all the stuff into fewer files likely means it loads
> faster.
>
> Includes files are kludge, and all these attempts to improve them are a
> kludge on top of a kludge. This is why better-designed languages have a
> proper module system that solves the whole issue.

Flattening hierarchies of include files is not popular in C
applications. But this option doesn't go far enough anyway.

Loading files is not a bottleneck in compilation; it is repeated
processing of huge amounts of content.

For example a GTK2 user will include "gtk.h", but that involves over
1000 other #includes, 550 unique, totaling 350K lines of declarations
across a dozen folders.

All that processing is repeated for each module that includes it.

When I applied my tool to GTK2, I found that all information could be
reduced to a single, flat 25K line representation in my syntax. (My
language /has/ a module scheme; that interface file is processed once
per build.)

It could just as well have generated a 25K line C header file. That's
93% smaller than all those separate headers, and one #include instead of
1000.

So, why don't the vendors of the library do that exercise? End-users
don't need 550 separate header files.

By contrast, all 30 or so standard C headers have 3-5K lines in total.
The problem there is that it is just very messy.

Bart wrote:

> Why not have a dedicated header file that is the specific to a
> particular version of a C compiler for a given platform? That it can be
> streamlined for that purpose.

In fact, this is similar to exactly what Plan 9 did: for include files
that had arch-dependent content, it'd have a separate version of them for
each arch, with an arch's versions of headers like these all stored in
their own directory.

--
Blue-Maned_Hawk│shortens to Hawk│/blu.mɛin.dʰak/│he/him/
his/himself/Mr.
blue-maned_hawk.srht.site
Of course, further simplicifications would be possible by killing off all
but one or two arches.

On Fri, 29 Dec 2023 13:31:37 +0000, Bart wrote:

> Why not have a dedicated header file that is the specific to a
> particular version of a C compiler for a given platform? That it can be
> streamlined for that purpose.

The GCC compilers already work this way. For example, on my Debian
system I have directories /usr/lib/gcc/x86_64-linux-gnu/12/ and
/usr/lib/gcc/x86_64-linux-gnu/13/. And when I look to see what
packages have put stuff in them, I find quite a lot:

ldo@theon:~> dpkg-query -S /usr/lib/gcc/x86_64-linux-gnu/12
libgcc-12-dev:amd64, gcc-12, libgfortran-12-dev:amd64, g++-12, cpp-12, gfortran-12, libstdc++-12-dev:amd64, gnat-12: /usr/lib/gcc/x86_64-linux-gnu/12
ldo@theon:~> dpkg-query -S /usr/lib/gcc/x86_64-linux-gnu/13
libstdc++-13-dev:amd64, libgcc-13-dev:amd64, libobjc-13-dev:amd64, gfortran-13, gcc-13, libgfortran-13-dev:amd64: /usr/lib/gcc/x86_64-linux-gnu/13

On 12/30/2023 12:51 AM, Blue-Maned_Hawk wrote:
> Bart wrote:
>
>> Why not have a dedicated header file that is the specific to a
>> particular version of a C compiler for a given platform? That it can be
>> streamlined for that purpose.
>
> In fact, this is similar to exactly what Plan 9 did: for include files
> that had arch-dependent content, it'd have a separate version of them for
> each arch, with an arch's versions of headers like these all stored in
> their own directory.
>

Yeah, we don't actually need big monolithic C libraries, nor big
monolithic C compilers, ...

Many people's assumption, that one needs "one true C compiler" (with
people debating GCC cs Clang, mostly ignoring any other possibilities)
and then using the same C library, etc, everywhere, may potentially
actually be doing the world a disservice.

Though, it seems like a different kind of strategy could be possible:
Split compilers into frontends and backends (which may exist as
semi-independent projects).

Frontend deals with source languages, compiles down to a common IR (with
the IR taking the place of object files);
Backend compiles this to the actual machine code, and does any linking,
before emitting the actual binary.

Possibly, separate frontends and backends could exist, merely agreeing
on a common IR.

The frontend could be more target neutral, trying to reduce
target-specific features mostly to "general parameters" (realistically,
a C frontend will still need to know things like the sizes of various
types, various other major architectural features, etc...). But, mostly,
it will care about input languages.

Though, in my compiler I had added an extension for an "__ifarch()" feature:
__ifarch(feature) some_decl
Where, used like an attribute, the declaration is only enabled (in the
backend) if the corresponding feature is present (though the features
may be combined using logical expressions, so "!feature",
"feature1&&!feature2", ...).

But also for blocks of code:
__ifarch(feature)
{
// only emitted if feature is present.
...
}
But, unlike "#ifdef", all the code needs to be syntactically valid, etc.
Partly, this is because the front-end still needs to be able to compile it.

The backend will care about target machine, but should not need to care
(too much) about the language, though will likely need to care about
some things outside the scope of plain C (such as namespaces, objects,
and exceptions, ...), mostly to be able to deal effectively with "OO
languages" and similar (though, I have noted that some simplifications
are possible relative to something like C++, where it is possible in
premise to mimic C++ style multiple inheritance on top of a
single-inheritance object model by treating MI classes as aggregates, ...).

In my experience, the "generally least bad" option for an IR seems to be
for a stack-machine like model (similar to JVM or .NET).

While a backend will most likely operate in terms of a TAC or SSA like form:
RPN -> TAC or SSA is fairly easy.
TAC -> SSA is more involved;
Using SSA directly exposes piles of hair across the interface.

For similar reasons to not work with raw SSA, it also seems like "not a
good thing" to expose bare ASTs (an AST will expose a whole lot of
design choices and tradeoffs within the frontend; while also being
"comparably expensive" in terms of the memory needed to work with them).

Though, one can note that RPN does need around 40-60% more operations
than TAC or SSA, but in practice, this seems to not be too big of an
issue (most of these extra stack operations "magically disappear" if one
treats the stack more as a holding space for variable IDs, than as an
"actual location that stores stuff").

Note that the use of the stack will be fairly constrained in such an IR
to use this model (unlike Forth, it is not free-form relative to control
flow). However, given the existing VMs also impose similar constraints,
it seems likely they also work in this way.

Some major tradeoffs within a stack IR:
Explicit types in opcodes (like JVM) or implicit (like .NET);
Explicit is better for a naive interpreter;
Implicit makes more sense for a compiler or JIT.
Though, better here is to leave type-promotions to the frontend.
Whether to fuse variable stores, etc, into the ops.
LOAD x; LOAD #1; ADD; STORE y
Vs:
LOAD x; LOAD #1; ADD_ST y
...

Well, along with tradeoffs for how to package the bytecode within a
file, and how to best express the various metadata (such as "a few big
tables holding everything", like in JVM, or splitting stuff into a large
number of interconnected tables like in .NET, ...).

For my compiler, it was:
Stack based IR, implicit types, with variable store merged with various ops;
Packaging was a big linear blob of bytecode (with metadata itself
expressed in the bytecode), but better likely would have been to use a
more structured TLV-style format and/or a WAD-file variant;
Metadata was split into two major tables:
One held everything that was a global variable of function declaration
or similar;
The other table held anything that was more like a value or type
declaration (so, structs/unions/classes/etc, value lists for initialized
arrays, etc).

In my case, sort of like PostScript, there is an explicit MARK operator
for function calls, eg:
dst=foo(args...);
Becomes:
MARK; args...; CALL foo; STORE dst;

....

For C library, there can be per platform libraries, or possibly ones
with a partial split between "generic C library stuff" and "stuff that
needs to be platform specific" (based on OS, target architecture, ...).

But, this is likely its own topic...

On Sat, 30 Dec 2023 16:16:07 -0600, BGB wrote:

> Many people's assumption, that one needs "one true C compiler" (with
> people debating GCC cs Clang, mostly ignoring any other possibilities)
> and then using the same C library, etc, everywhere, may potentially
> actually be doing the world a disservice.

Actually, we have quite a choice of C libraries, even on the same Linux
platform. I once used one of them (I think it was musl) just to prove I
could build a small executable† that would run in just 20kB of RAM on a
modern 64-bit Linux machine.

†It did nothing more than take an integer command-line argument and sleep
for that number of seconds. That way the process would hang around long
enough for me to confirm how much/little RAM it was using.

On 12/30/2023 5:21 PM, Lawrence D'Oliveiro wrote:
> On Sat, 30 Dec 2023 16:16:07 -0600, BGB wrote:
>
>> Many people's assumption, that one needs "one true C compiler" (with
>> people debating GCC cs Clang, mostly ignoring any other possibilities)
>> and then using the same C library, etc, everywhere, may potentially
>> actually be doing the world a disservice.
>
> Actually, we have quite a choice of C libraries, even on the same Linux
> platform. I once used one of them (I think it was musl) just to prove I
> could build a small executable† that would run in just 20kB of RAM on a
> modern 64-bit Linux machine.
>
> †It did nothing more than take an integer command-line argument and sleep
> for that number of seconds. That way the process would hang around long
> enough for me to confirm how much/little RAM it was using.

On Linux, I think yeah, glibc, newlib, and musl, are the main options.
Seemingly, a majority of stuff assumes glibc though.

With main popular compilers being GCC and LLVM/Clang.
There also exist variants of, say: lcc, tcc, pcc, etc...

They exist, but most people seem to ignore them...

For my custom hobby OS + CPU ISA, I am using:
BGBCC, full custom C compiler, though it is mostly based on code that
has been beating around in my projects for the past 20 years or so
(originally starting as a fork off an early version of an interpreter I
had written for a JavaScript-like language).

The main alternatives at the time were:
Modify GCC to support my designs, which looked like a massive pain;
LLVM, which even trying to recompile it one-off "totally owned" my PC at
the time;
LCC, which looked possible, but didn't offer much obvious advantage over
the code I already had laying around;
....

With the C library being a heavily modified version of PDPCLIB that had
also grown a lot of OS style appendages (the OS kernel, C library, and
shell, are all sort of a conjoined mess that probably needs some amount
of redesign).

Did eventually get around to make it so that at least, normal user-mode
binaries will omit much of the kernel related code from the binary
(though, does mean these binaries can no longer be launched "bare metal").

Had started trying to write a new C library at one point, but effort on
this kinda fizzled.

Internal architecture gets a little weird, with various parts of the
runtime libraries and OS interfacing via structures resembling COM
objects (with a C like API wrapping the internal COM style interfaces).
Along with some funkiness like using COM style interfaces to call
services running inside different tasks.

Ironically, the system call mechanism could itself almost be a COM
object, apart from the fact that it was instead implemented as a "magic
function pointer" (which can be used to perform some system calls
directly, and to fetch objects for some other OS API interfaces).

Though, it is not exactly the same:
There is no IDL tool;
There is more variability in terms of method signatures and naming;
Interfaces are generally identified as pairs of FOURCC or EIGHTCC values
rather than GUIDs, but there is the idea that the EIGHTCC pair could
easily be used for a GUID as well (though, likely limiting GUIDs to
"private interfaces", with FOURCC/EIGHTCC making more sense for OS
provided APIs and services).

Part of this was in-turn because providing inter-task interfaces via a
"GetProcAddress" style mechanism would have resulted in considerably
higher overheads (an full object has roughly the same overhead, but can
provide a bunch of different functions all at the same time).

The handling for DLL's and the C library is kind of backwards compared
with the strategy Windows uses:

Windows: Each binary/DLL has a C library linked in, which calls back to
shared DLLs for the OS API.

My case: Main EXE has a statically-linked C library, which provides an
interface that the DLLs can use;
Each DLL has a stripped down C library, which uses (yet another)
COM-style interface to call back into the C library provided by the main
binary (which may then redirect calls across the system-call interface
or similar).

It was done this way mostly for "technical reasons" (partly because,
among the available options, this did not preclude bare metal programs
or the kernel from being able to use DLLs).

Note that (unlike ELF on Linux), it is not currently possible to
directly share global variables across DLL boundaries.

Though, the effect this would have on the C library are reduced as
stdin/stdout/errno/etc were already implemented as macros wrapping calls
to getter functions.

Eg:
errno_t *__get_errno();
#define errno (*(__get_errno()))

....

All still kind of a mess though...

Similarly, my compiler can at least sort of mimic the POSIX style
command-line interface (though was written originally with a more
MSVC-like CLI);
But, still falls short of being able to satisfy "./configure" and
similar that it is a valid cross compiler.

Then again, not like there is much hope of getting stuff ported when in
general, nearly all of the Linux software seems to take a "you need the
whole jungle to get a single banana" approach to software dependencies
(and then much of the code tends to be riddled with GCC'isms).

Often almost easier to port DOS era or older Unix-era code (it is still
not usually going to work "out of the box" but at least tends to be more
straightforward for how one is to go about porting it).

Well, at least excluding some "obviously nasty" code...

....

On Sat, 30 Dec 2023 19:14:55 -0600, BGB wrote:

> Note that (unlike ELF on Linux), it is not currently possible to
> directly share global variables across DLL boundaries.

Windows is broken in so many ways ... when you achieve something, it’s
like getting a bear to dance: it’s not that it dances badly, but that it
dances at all.

On Sat, 30 Dec 2023 01:58:53 +0000, Bart wrote:

> So, why don't the vendors of the library do that exercise?

Maybe because most of the “vendors” of proprietary libraries have gone
extinct. What we have now is “developers” and “contributors” to open-
source projects. And if you have a bright idea for how they can do things
better, you are free to contribute it.

On 31/12/2023 01:36, Lawrence D'Oliveiro wrote:
> On Sat, 30 Dec 2023 01:58:53 +0000, Bart wrote:
>
>> So, why don't the vendors of the library do that exercise?
>
> Maybe because most of the “vendors” of proprietary libraries have gone
> extinct. What we have now is “developers” and “contributors” to open-
> source projects. And if you have a bright idea for how they can do things
> better, you are free to contribute it.

I have plenty of ideas, but people are generally not interested. Even if
some were, how do you persuade the creators of 100s of libraries to do
things differently?

So I use my ideas in my own languages and in my own compilers, including
one for C. There, the standard headers /are/ specific to that platform,
although I do only support one. (If another comes along, it will have
its own set!)

On 31/12/2023 01:34, Lawrence D'Oliveiro wrote:
> On Sat, 30 Dec 2023 19:14:55 -0600, BGB wrote:
>
>> Note that (unlike ELF on Linux), it is not currently possible to
>> directly share global variables across DLL boundaries.
>
> Windows is broken in so many ways ... when you achieve something, it’s
> like getting a bear to dance: it’s not that it dances badly, but that it
> dances at all.
I think that that limitation was specific to BGB's handling of DLLs; it
was not made clear.

If it is an actual issue on Windows, then someone would first have to
explain what it means, and why it happens, as I can't see it.

Each DLL exports certain symbols such as the addresses of functions and
variables. So no reason you can't access a variable exported from any
DLL, unless perhaps multiple instances of the same DLL have to share the
same static data, but that sounds very unlikely, as little would work.

>Windows is broken in so many ways

Other examples? I find rather the opposite. For example I did like this
remark of BGB's:

>Then again, not like there is much hope of getting stuff ported when
in general, nearly all of the Linux software seems to take a "you need
the whole jungle to get a single banana" approach to software
dependencies (and then much of the code tends to be riddled with GCC'isms)

This somes my experience of software originating in Linux. This is why
Windows had to acquire CYGWIN then MSYS then WSL. You can't build the
simplest program without involving half of Linux.

On 12/30/2023 8:18 PM, Bart wrote:
> On 31/12/2023 01:34, Lawrence D'Oliveiro wrote:
>> On Sat, 30 Dec 2023 19:14:55 -0600, BGB wrote:
>>
>>> Note that (unlike ELF on Linux), it is not currently possible to
>>> directly share global variables across DLL boundaries.
>>
>> Windows is broken in so many ways ... when you achieve something, it’s
>> like getting a bear to dance: it’s not that it dances badly, but that it
>> dances at all.
> I think that that limitation was specific to BGB's handling of DLLs; it
> was not made clear.
>

Yes.

In my case (for my custom target) I am using a modified version of
PE/COFF (with some tweaks, *), but it has the issue that there is not
currently (any) mechanism for sharing variables across DLL boundaries
apart from getter/setter functions or similar.

*1: Major differences from Windows' PE/COFF:
Removing the MZ header
File typically starts directly at the 'PE' magic;
For my uses, the MZ stub/header was entirely useless.
The loaders will still handle it if encountered;
But, having MZ also disables some of the other format tweaks.
Adding LZ4 compression
'PEL4' magic: Everything past the first 1K is LZ4 compressed.
This is an optional extension, mostly to help with loader speed.
The loader is mostly IO bound, and LZ4 can make loading faster.
The resource section was replaced with a modified WAD2 variant.
Albeit, with different headers, and offsets encoded in RVA space.
The original format seemed needlessly complicated and awkward.
IOW: Based on the format used for textures in Quake and Half-Life.
I also went over to a different checksum algorithm and similar.
The original algorithm was very weak against some types of errors.
...

However, much of its structure still has more in common with PE/COFF
than some of the other COFF variants (such as ECOFF and XCOFF).

Note, original algorithm was something like (from memory):
u32 PE_Checksum(void *buf, int sz)
{
uint16_t *cs, *cse;
uint32_t sum;
cs=buf; cse=cs+(sz>>1); sum=0;
while(cs<cse)
{ sum+=*cs++; }
sum=((uint16_t)sum)+(sum>>16);
sum=((uint16_t)sum)+(sum>>16);
return(sum);
}
Had replaced it with:
uint32_t TKPE_CalculateImagePel4BChecksum(byte *buf, int size)
{
byte *cs, *cse;
uint32_t v, v0, v1, v2, v3;
uint64_t acc_lo, acc_hi;
uint32_t csum;
cs=buf; cse=cs+size;
acc_lo=1; acc_hi=0;
while(cs<cse)
{
v0=((uint32_t *)cs)[0]; v1=((uint32_t *)cs)[1];
v2=((uint32_t *)cs)[2]; v3=((uint32_t *)cs)[3];
acc_lo=acc_lo+v0; acc_hi=acc_hi+acc_lo;
acc_lo=acc_lo+v1; acc_hi=acc_hi+acc_lo;
acc_lo=acc_lo+v2; acc_hi=acc_hi+acc_lo;
acc_lo=acc_lo+v3; acc_hi=acc_hi+acc_lo;
cs+=16;
}
acc_lo=((uint32_t )acc_lo)+(acc_lo>>32);
acc_lo=((uint32_t )acc_lo)+(acc_lo>>32);
acc_hi=((uint32_t )acc_hi)+(acc_hi>>32);
acc_hi=((uint32_t )acc_hi)+(acc_hi>>32);
csum=(uint32_t )(acc_lo^acc_hi);
return(csum);
}

Where in this case, my ISA has enough registers that is is generally
faster to do the accumulation 4-wide than 1-wide.

The use of checksums in this case mostly to verify that the program has
been loaded intact and the binary is not corrupt.

> If it is an actual issue on Windows, then someone would first have to
> explain what it means, and why it happens, as I can't see it.
>
> Each DLL exports certain symbols such as the addresses of functions and
> variables. So no reason you can't access a variable exported from any
> DLL, unless perhaps multiple instances of the same DLL have to share the
> same static data, but that sounds very unlikely, as little would work.
>

Much past roughly Win9x or so, it has been possible to use
"__declspec(dllimport)" on global variables in Windows (in an earlier
era, it was not possible to use the __declspec's, but instead necessary
to manage DLL import/exports by writing out lists in ".DEF" files).

It isn't entirely transparent, but yes, on actual Windows, it is very
much possible to share global variables across DLL boundaries.

Just, this feature is not (yet) supported by my compiler. Personally, I
don't see this as a huge loss (even if it did work; I personally see it
as "poor coding practice").

But, otherwise, I am using the newer __declspec dllimport/dllexport
syntax; and technically using exclusively import/export by name (the use
of ordinal numbers is not supported either by my compiler or PE loader).

But, yeah, making shared global variables work is on the "eventual TODO"
list, just not an immediate priority.

Note that, like with the MS tools, unless a function is marked as
dllexport, its visibility is purely local to a given EXE or DLL.

This differs slightly from Cygwin, which seems to use an "export
everything" (and implicitly import everything) strategy, likely in an
attempt to mimic ELF behavior.

Though, there are some differences:
In ELF based systems, it is possible to leave symbols absent in the
SO's, with them resolved to a symbol exported from the main binary;
This strategy doesn't really work with DLLs (where the import dependency
tree needs to be acyclic).

> >Windows is broken in so many ways
>
> Other examples? I find rather the opposite. For example I did like this
> remark of BGB's:
>
> >Then again, not like there is much hope of getting stuff ported when
> in general, nearly all of the Linux software seems to take a "you need
> the whole jungle to get a single banana" approach to software
> dependencies (and then much of the code tends to be riddled with GCC'isms)
>
> This somes my experience of software originating in Linux. This is why
> Windows had to acquire CYGWIN then MSYS then WSL. You can't build the
> simplest program without involving half of Linux.

Yes, and it is really annoying sometimes.

For the most part, Linux software builds and works fairly well... if one
is using a relatively mainline and relatively up-to-date Linux distro...

But, if one is not trying to build in or for a typical Linux style / GNU
based userland; it is straight up pain...

Like, typically either the "./configure" script is going to go down in a
crap-storm of error messages (say, if the shell is not "bash", or some
commands it tries to use are absent or don't accept the same
command-line arguments, etc); or libraries are going to be missing; or
the build just ends up dying due to compiler errors (say, which headers
exist are different, or their contents are different, ...).

Within the code itself, it often doesn't take much looking to find one of:
Pointer arithmetic on "void *";
Various GCC specific "__attribute__((whatever))" modifiers;
Blobs of GAS specific inline ASM;
...

Whereas in more cross-platform code, one will usually find stuff like:
#ifdef __GNUC__
... GCC specific stuff goes here ...
#endif
#ifdef _MSC_VER
... MSVC specific stuff goes here ...
#endif
....

And, different ways of doing stuff, say:
Some stuff that works on MSVC will break on GCC;
Some stuff that performs well on GCC will perform like garbage on MSVC;
....

Sometimes it makes sense to write a bunch of wrapper code or macros for
various tasks which differ between compilers and targets.

Though, have noted that sometimes programs will work at one optimization
level, but break at another (so, "-O3" or more so "-Ofast" is "playing
with fire" with GCC, as it is with "/O2" in MSVC; with GCC one often
needing "-fno-strict-aliasing -fwrapv" and similar for some older code
to work correctly).

My compiler uses sort of an intermediate C dialect, but is more
conservative by default in some areas, such as treating things like TBAA
as "opt-in" features, rather than "opt-out", ...

Though, I did designate various cases as "no consistent or sensible
behavior exists", so "whatever happens, happens". Separating out cases
that are "technically undefined, but has a conventionally accepted
behavior" (such as using pointer casts for type punning, etc), vs "no
accepted behavior and any behavior that may result is effectively a dice
roll..." (a lot of cases involving out-of-bounds memory access, etc).

Some amount of the extensions have more MSVC-like syntax (albeit the ASM
syntax itself is more derived from GAS style ASM syntax than Intel style
syntax). Though, in particular, it is derived from "GAS SuperH" (which
falls into a similar category as M68K and PDP-11 ASM syntax):
R4 //this is a register
@R4 //memory with address in R4
(R4) //same as @R4
(R4,32) //displacement
32(R4) //same as (R4,32)

On 29/12/2023 23:18, Scott Lurndal wrote:
> Keith Thompson <Keith.S.Thompson+u@gmail.com> writes:
>> Janis Papanagnou <janis_papanagnou+ng@hotmail.com> writes:
>> [...]
>>> BTW, is 'inline' meanwhile C standard? (I know that from C++ but
>>> haven't done much C for long now.)
>>
>> C added inline in C99 (the 1999 edition of the ISO C standard, the same
>> one that removed implicit int).
>>
>> I think C and C++ have subtly different semantics for inline.
>
> Mostly related to symbol visibility, IIRC.

In C, the "inline" qualifier is pretty much a message from the
programmer saying "I think the resulting code would be more efficient if
this is inlined by the optimiser". Optimising compilers mostly ignore
it. In C++, the "inline" qualifier is a message from the programmer
saying "I might define this thing in multiple translation units, and I
promise I'll do it the same way each time".

You are most likely to see issues if some calls can't be inlined (or are
not inlined by the compiler). If you have code that might need to be
compiled as C or C++ (such as if the function is in a header), the best
method is to declare it "static inline" rather than plain "inline". And
in a C or C++ implementation file, any function that you'd consider
declaring "inline" is likely to be "static" anyway. (Or in an anonymous
namespace in C++, which amounts to the same thing.) "static inline"
works, AFAIK, in exactly the same way in C and C++.

On 29/12/2023 23:40, Bart wrote:
> On 29/12/2023 20:23, Kaz Kylheku wrote:
>> On 2023-12-29, Keith Thompson <Keith.S.Thompson+u@gmail.com> wrote:
>>> David Brown <david.brown@hesbynett.no> writes:
>>>> A useful tool that someone might like to write for this particular
>>>> situation would be a partial C preprocessor, letting you choose what
>>>> gets handled.  You could choose to expand the code here for, say,
>>>> _GNU_SOURCE and _BSD_SOURCE - any use of these in #ifdef's and
>>>> conditional compilation would be expanded according to whether you
>>>> have defined the symbols or not, leaving an output that is easier to
>>>> understand while keeping most of the pre-processor stuff unchanged (so
>>>> not affecting #includes, and leaving #define'd macros and constants
>>>> untouched and therefore more readable).
>>>
>>> The unifdef tool does some of this.  (I haven't used it much.)
>>
>> GNU cpp has an option which is something like this: -fdirectives-only.
>> It causes it not to expand macros.
>
> It flattens include files, processes conditionals, and keeps #defines
> unchanged.
>
> However, it turns gcc's sys/stat.h from 300 lines into 3000 lines.
>
> If I apply it to my stat.h (also my stddef.h which it includes), which
> are 110 lines together, it produces 900 lines. Most of that consists of
> lots of built-in #defines with __ prefixes (each complete with a line
> saying it is built-in).
>
> When I use my own conversion tool (designed to turn C headers that
> define APIs into declarations in my language), the output is 65 lines.
>
> The gcc option does not expand typedefs or macros. So if there is a
> declaration using a type which uses both, that is unchanged, which is
> not helpful. (At least not if trying to create bindings for your FFI.)
>
> gcc with just -E will expand macros but still keep typedefs.
>

Note that typedefs are part of the core C language, not the
preprocessor, so there could not possibly be a cpp option to do anything
with typedefs (the phrase "expand typedefs" is entirely wrong).

I realise that you (and possibly others) might find it useful for a tool
to replace typedef identifiers with their definitions, but it could only
be done for some cases, and is not as simple as macro substitution.

> So, for producing a streamlined standard header, it still leaves a lot
> to be desired. And for trying to flatten layers of macros and typedefs,
> to reveal the underlying types, it's not that great either.
>
> Purpose-built tools are always better, but dealing with C is not trivial
> anyway, and dealing with gnu- and gcc-specific features is even harder.
>
>

On 31/12/2023 05:46, BGB wrote:
> On 12/30/2023 8:18 PM, Bart wrote:
>> On 31/12/2023 01:34, Lawrence D'Oliveiro wrote:
>>> On Sat, 30 Dec 2023 19:14:55 -0600, BGB wrote:
>>>
>>>> Note that (unlike ELF on Linux), it is not currently possible to
>>>> directly share global variables across DLL boundaries.
>>>
>>> Windows is broken in so many ways ... when you achieve something, it’s
>>> like getting a bear to dance: it’s not that it dances badly, but that it
>>> dances at all.
>> I think that that limitation was specific to BGB's handling of DLLs;
>> it was not made clear.
>>
>
> Yes.
>
> In my case (for my custom target) I am using a modified version of
> PE/COFF (with some tweaks, *), but it has the issue that there is not
> currently (any) mechanism for sharing variables across DLL boundaries
> apart from getter/setter functions or similar.

I find PE/COFF format a nightmare. I did eventually get my tools to
generate first OBJ then EXE files. But when it came to DLL, there was
something wrong in the files I generated that I couldn't fix.

So for a couple of years, I created my own shared library format,
utterly different from PE+, and about 10 times simpler.

The shared library files were called ML, and I extended it to standalone
executables called MX files.

However ML libraries could only be used from my languages, and MX files
needed a small conventional EXE loader to get started.

Eventually I fixed the problems with DLL. (Partly it was that my
generated code wasn't fully position-independent and only ran in
low-memory below 2GB, but there was also a bug in the base-reloc tables.)

Now, sadly, I will probably drop my ML/MX files, even though my MLs have
advantages over DLLs. (Eg. they have the same environment as the host
apps, so that they can share things like pointers to allocated memory
and file handles. With DLL, a pointer malloc-ed in the host cannot be
freed within the DLL and vice versa.)

>> Each DLL exports certain symbols such as the addresses of functions
>> and variables. So no reason you can't access a variable exported from
>> any DLL, unless perhaps multiple instances of the same DLL have to
>> share the same static data, but that sounds very unlikely, as little
>> would work.
>>
>
> Much past roughly Win9x or so, it has been possible to use
> "__declspec(dllimport)" on global variables in Windows (in an earlier
> era, it was not possible to use the __declspec's, but instead necessary
> to manage DLL import/exports by writing out lists in ".DEF" files).
>
> It isn't entirely transparent, but yes, on actual Windows, it is very
> much possible to share global variables across DLL boundaries.
>
>
> Just, this feature is not (yet) supported by my compiler. Personally, I
> don't see this as a huge loss (even if it did work; I personally see it
> as "poor coding practice").

This is a language issue. Or, in C, it is compiler related.

I've never been quite sure how you tell a C compiler to export a certain
symbol when creating a DLL. Sometimes it just works; I think it just
exports everything that is not static (it may depend on a compiler
option too).

And some compilers may need this __declspec business, but I've never
bothered with it.

Mine just exports all not-static names. So this program:

int abc;
static int def;

void F(void) {}
static void G(void) {}

if compiled as: 'mcc -dll prog', produces a file prog.dll which, if I
dump it, shows this export table:

Export Directory

0 00000000 0 Fun F
1 00000000 0 Var abc

(There's something in it that distinguishes functions from variables,
but I can't remember the details.)

In any case, in C it can be hit and miss. In my own language, it is more
controlled: I used an 'export' prefix to export symbols from a program.

(It also conventiently creates interface files to be able to use the DLL
library from a program. The equivalent of prog.h for my example
containing the API needed to use it. Rolling that out to C is not
practical however as my 'export' applies also to things like types and
enums.)

>> This [somes] my experience of software originating in Linux. This is why
>> Windows had to acquire CYGWIN then MSYS then WSL. You can't build the
>> simplest program without involving half of Linux.
>
> Yes, and it is really annoying sometimes.
>
>
> For the most part, Linux software builds and works fairly well... if one
> is using a relatively mainline and relatively up-to-date Linux distro...
>
>
> But, if one is not trying to build in or for a typical Linux style / GNU
> based userland; it is straight up pain...
>
> Like, typically either the "./configure" script is going to go down in a
> crap-storm of error messages (say, if the shell is not "bash", or some
> commands it tries to use are absent or don't accept the same
> command-line arguments, etc); or libraries are going to be missing; or
> the build just ends up dying due to compiler errors (say, which headers
> exist are different, or their contents are different, ...).

../configure is an abomination anyway; I've seen 30,000-line scripts
which take forever to run, and test things like whether 'printf' is
supported.

But the biggest problem with them is when someone expects a Windows user
to use that same build process. Of course, ./configure is a Bash script
using Linux utilities.

It's like someone providing a .BAT file and expecting Linux users to do
something with it.

>
> Within the code itself, it often doesn't take much looking to find one of:
>   Pointer arithmetic on "void *";
>   Various GCC specific "__attribute__((whatever))" modifiers;
>   Blobs of GAS specific inline ASM;
>   ...
>
>
> Whereas in more cross-platform code, one will usually find stuff like:
> #ifdef __GNUC__
>   ... GCC specific stuff goes here ...
> #endif
> #ifdef _MSC_VER
>   ... MSVC specific stuff goes here ...
> #endif
> ...

Those conditional blocks never list my compiler, funnily enough. (#ifdef
__MCC__ will do it.)

> My compiler uses sort of an intermediate C dialect, but is more
> conservative by default in some areas, such as treating things like TBAA
> as "opt-in" features, rather than "opt-out", ...
>
> Though, I did designate various cases as "no consistent or sensible
> behavior exists", so "whatever happens, happens". Separating out cases
> that are "technically undefined, but has a conventionally accepted
> behavior" (such as using pointer casts for type punning, etc), vs "no
> accepted behavior and any behavior that may result is effectively a dice
> roll..." (a lot of cases involving out-of-bounds memory access, etc).
>
> Some amount of the extensions have more MSVC-like syntax (albeit the ASM
> syntax itself is more derived from GAS style ASM syntax than Intel style
> syntax). Though, in particular, it is derived from "GAS SuperH" (which
> falls into a similar category as M68K and PDP-11 ASM syntax):
>   R4  //this is a register
>   @R4  //memory with address in R4
>   (R4)  //same as @R4
>   (R4,32)  //displacement
>   32(R4)   //same as (R4,32)

My C compiler is from 2017. Eventually I decided it was too
non-conforming and buggy to be a serious tool. It became a private one
(for example one special feature is being able to turn library APIs
defined as C headers, into bindings for either of my own languages,
although that can only do 90% of the work).

Last autumn I upgraded it with a new backend. But I also got rid of all
experimental features I'd played with.

It may be a poor compiler but CLI-wise it's much better than gcc which
continues to be a pain to use (still generating a.exe), and has odd bugs
in its CLI.

So, mine is still satisfying to have. I just call it a C-subset or
C-dialect compiler to get over the non-conformance. But for building my
own C code, it's my first choice.

On 31/12/2023 15:25, David Brown wrote:
> On 29/12/2023 23:40, Bart wrote:
>> On 29/12/2023 20:23, Kaz Kylheku wrote:
>>> On 2023-12-29, Keith Thompson <Keith.S.Thompson+u@gmail.com> wrote:
>>>> David Brown <david.brown@hesbynett.no> writes:
>>>>> A useful tool that someone might like to write for this particular
>>>>> situation would be a partial C preprocessor, letting you choose what
>>>>> gets handled.  You could choose to expand the code here for, say,
>>>>> _GNU_SOURCE and _BSD_SOURCE - any use of these in #ifdef's and
>>>>> conditional compilation would be expanded according to whether you
>>>>> have defined the symbols or not, leaving an output that is easier to
>>>>> understand while keeping most of the pre-processor stuff unchanged (so
>>>>> not affecting #includes, and leaving #define'd macros and constants
>>>>> untouched and therefore more readable).
>>>>
>>>> The unifdef tool does some of this.  (I haven't used it much.)
>>>
>>> GNU cpp has an option which is something like this: -fdirectives-only.
>>> It causes it not to expand macros.
>>
>> It flattens include files, processes conditionals, and keeps #defines
>> unchanged.
>>
>> However, it turns gcc's sys/stat.h from 300 lines into 3000 lines.
>>
>> If I apply it to my stat.h (also my stddef.h which it includes), which
>> are 110 lines together, it produces 900 lines. Most of that consists
>> of lots of built-in #defines with __ prefixes (each complete with a
>> line saying it is built-in).
>>
>> When I use my own conversion tool (designed to turn C headers that
>> define APIs into declarations in my language), the output is 65 lines.
>>
>> The gcc option does not expand typedefs or macros. So if there is a
>> declaration using a type which uses both, that is unchanged, which is
>> not helpful. (At least not if trying to create bindings for your FFI.)
>>
>> gcc with just -E will expand macros but still keep typedefs.
>>
>
> Note that typedefs are part of the core C language, not the
> preprocessor, so there could not possibly be a cpp option to do anything
> with typedefs (the phrase "expand typedefs" is entirely wrong).
>
> I realise that you (and possibly others) might find it useful for a tool
> to replace typedef identifiers with their definitions, but it could only
> be done for some cases, and is not as simple as macro substitution.

Take this program, which uses two nested typedefs and one macro:

typedef short T;
typedef T U;
#define V U

typedef struct R {
V a, b, c;
} S;

Passed through 'gcc -E', it manages to expand the V in the struct with
U. (-fdirectives-only doesn't even do that).

So what are the types of 'a, b, c'? Across 1000s of line of code, they
may need tracking down. At least, for someone not using your super-duper
tools.

If I use my compiler with 'mcc -mheaders', I get an output file that
includes this:

record R = $caligned
i16 a
i16 b
i16 c
end

It gives all the information I might need. Including the fact that it
uses default C alignment rules.

Notice however the name of the record is R not S; here it needs a
struct-tag to avoid an anonymous name. The typedef name is harder to use
as it is replaced early on in compilation.

Here, I'm effectively expanding a typedef. The output could just as
equally have been C source code.

On 2023-12-31, Bart <bc@freeuk.cm> wrote:
> and file handles. With DLL, a pointer malloc-ed in the host cannot be
> freed within the DLL and vice versa.)

What??? All DLLs are in the same address space. malloc and free are
sister functions that typically live in the same DLL, and don't
care what calls them.

Maybe you're referring to one DLL's free not being able to handle
pointers produced by another DLL's malloc.

(That's not a problem caused by the DLL mechanism itself and would not
go away if those were somehow statically linked together.)

--
TXR Programming Language: http://nongnu.org/txr
Cygnal: Cygwin Native Application Library: http://kylheku.com/cygnal
Mastodon: @Kazinator@mstdn.ca
NOTE: If you use Google Groups, I don't see you, unless you're whitelisted.

On 12/31/23 08:40, David Brown wrote:
....
> In C, the "inline" qualifier is pretty much a message from the
> programmer saying "I think the resulting code would be more efficient
> if this is inlined by the optimiser".

Actually, what the C standard says is "Making a function an
inline function suggests that calls to the function be as fast as
possible". The standard does not specify how this is to be achieved, it
merely imposes some requirements that constrain how it could be
achieved. Inlining a function call is just one way to do that.

Bart <bc@freeuk.cm> writes:
>On 31/12/2023 01:36, Lawrence D'Oliveiro wrote:
>> On Sat, 30 Dec 2023 01:58:53 +0000, Bart wrote:
>>
>>> So, why don't the vendors of the library do that exercise?
>>
>> Maybe because most of the “vendors” of proprietary libraries have gone
>> extinct. What we have now is “developers” and “contributors” to open-
>> source projects. And if you have a bright idea for how they can do things
>> better, you are free to contribute it.
>
>I have plenty of ideas, but people are generally not interested.

Perhaps they are not very good ideas, then....

Frankly, your obsession with header files is puzzling. 99.9%
percent of C/C++ programmers don't care.