scott@slp53.sl.home (Scott Lurndal) writes:
> Keith Thompson <Keith.S.Thompson+u@gmail.com> writes:
>>Michael S <already5chosen@yahoo.com> writes:
>>[...]
>>> Is there any chance at all that on typical Linux machine (i.e. the one
>>> configured to overcommit virtual memory) strdup() returns NULL?
>>> If it was malloc() I'd say - no chance. For strdup() I'm less sure.
>>
>>strdup() calls malloc(), so strdup() can return NULL if and only if
>>malloc() can return NULL -- but with the additional constraint that you
>>first need to have a string argument to strdup() that's long enough to
>>cause a suffiently large argument to be passed to malloc().
>>
>>One data point: On my Ubuntu system, malloc(SIZE_MAX) returns NULL for
>>very large arguments (over about 23 GiB in my quick and dirty test).
>
> That may be due to resource limits (setrlimit/getrlimit/ulimit) on
> the size of the address space.

I don't see anything relevant in the output of `ulimit -a`. In
particular, "data seg size" is unlimited.

--
Keith Thompson (The_Other_Keith) Keith.S.Thompson+u@gmail.com
Working, but not speaking, for Medtronic
void Void(void) { Void(); } /* The recursive call of the void */

Kaz Kylheku <433-929-6894@kylheku.com> writes:
> Scott Lurndal <scott@slp53.sl.home> wrote:
>> Malcolm McLean <malcolm.arthur.mclean@gmail.com> writes:
>>>On 10/03/2024 18:47, Scott Lurndal wrote:
>>>> Kaz Kylheku <433-929-6894@kylheku.com> writes:
>>>>> Not take up space in every application for a common library routine.
>>>>
>>>> It's a form of lazy programming. I've seen a lot of open source
>>>> code that uses strdup without checking for failure and frequently
>>>> "forgetting" to free the result.
>>>
>>>And it is probably more likely that machine with many gigabytes of RAM
>>
>> Actually, your assumptions that:
>> 1) strdup is the only allocation function used by an application
>> 2) all strings are "short"
>
> Even if a string is long enough to need its own mmap request,
> that will still return valid memory that later fails to commit.

Since strdup necessarily writes to almost all the memory it allocates,
you’d expect the failure to be immediate.

--
https://www.greenend.org.uk/rjk/

Richard Kettlewell <invalid@invalid.invalid> writes:
> Kaz Kylheku <433-929-6894@kylheku.com> writes:
>> Scott Lurndal <scott@slp53.sl.home> wrote:
>>> Malcolm McLean <malcolm.arthur.mclean@gmail.com> writes:
>>>>On 10/03/2024 18:47, Scott Lurndal wrote:
>>>>> Kaz Kylheku <433-929-6894@kylheku.com> writes:
>>>>>> Not take up space in every application for a common library routine.
>>>>>
>>>>> It's a form of lazy programming. I've seen a lot of open source
>>>>> code that uses strdup without checking for failure and frequently
>>>>> "forgetting" to free the result.
>>>>
>>>>And it is probably more likely that machine with many gigabytes of RAM
>>>
>>> Actually, your assumptions that:
>>> 1) strdup is the only allocation function used by an application
>>> 2) all strings are "short"
>>
>> Even if a string is long enough to need its own mmap request,
>> that will still return valid memory that later fails to commit.
>
> Since strdup necessarily writes to almost all the memory it allocates,
> you’d expect the failure to be immediate.

Yes, but that write happens *after* the malloc() call, so it won't
necessarily be reflected in the value returned by strdup(). It could
crash the current program, or in the worst case it could cause the OoM
killer to kill some other process.

--
Keith Thompson (The_Other_Keith) Keith.S.Thompson+u@gmail.com
Working, but not speaking, for Medtronic
void Void(void) { Void(); } /* The recursive call of the void */

On Mon, 11 Mar 2024 10:13:12 -0700
Keith Thompson <Keith.S.Thompson+u@gmail.com> wrote:
> One data point: On my Ubuntu system, malloc(SIZE_MAX) returns NULL for
> very large arguments (over about 23 GiB in my quick and dirty test).

You lost me there.

On Mon, 11 Mar 2024 19:35:11 +0200
Michael S <already5chosen@yahoo.com> wrote:
> Some people would say that writing code (a handler for allocation
> returning NULL) that either can't be tested in principle or can be
> tested only in principle, but most certainly not tested in
> practice, isn't a sign of a good programmer.

It can be tested in practice by explicitly inserting failure code , perhaps
activated by a macro :

pointer = malloc(...) ;
if (pointer == 0 || testN) { ... handle error ... } ;

It clutters the code but it's testable.

> Myself, I still tend to code this checks, but
> (1) my main targets are not Linux with overcommit, so the
> chance of allocation returning NULL could be estimated like "not going
> to happen" rather than "can't happen".
> (2) I am old full that like his unreasonable old habits

What's the practical difference between "not going to happen" and
"can't happen" ? Practically , you can never know that it can't happen
because overcommit is a matter of configuration of the Linux system.

Spiros Bousbouras <spibou@gmail.com> writes:
> On Mon, 11 Mar 2024 10:13:12 -0700
> Keith Thompson <Keith.S.Thompson+u@gmail.com> wrote:
>> One data point: On my Ubuntu system, malloc(SIZE_MAX) returns NULL for
>> very large arguments (over about 23 GiB in my quick and dirty test).
>
> You lost me there.

Sorry, editing error. I initially tried malloc(SIZE_MAX) and got a NULL
result. I then tried smaller values to try to find out where the cutoff
is, and didn't correctly edit the post.

Corrected version:
One data point: On my Ubuntu system, malloc() returns NULL for
very large arguments (over about 23 GiB in my quick and dirty test).

--
Keith Thompson (The_Other_Keith) Keith.S.Thompson+u@gmail.com
Working, but not speaking, for Medtronic
void Void(void) { Void(); } /* The recursive call of the void */

On Fri, 8 Mar 2024 09:01:21 +0100, David Brown wrote:

> It seems much more appropriate for Ada (though Pascal also had stricter
> checking and stronger types than most other popular languages had when
> Pascal was developed).

That’s why Ada was built on Pascal: if you want something intended for
high-reliability, safety-critical applications, why not build it on a
foundation that was already the most, shall we say, anal-retentive, among
well-known languages of the time?

On Fri, 8 Mar 2024 21:36:14 -0800, Ross Finlayson wrote:

> What I'd like to know about is who keeps dialing the "harmonization"
> efforts, which really must give grouse to the "harmonisation"
> spellers ...

Some words came from French and had “-ize”, others did not and had “-ise”.
Some folks in Britain decided to change the former to the latter.

“Televise”, “merchandise”, “advertise” -- never any “-ize” form.

“Synchronize”, “harmonize”, “apologize” -- “-ize” originally.

On 11/03/2024 17:00, Scott Lurndal wrote:
> Malcolm McLean <malcolm.arthur.mclean@gmail.com> writes:
>> On 10/03/2024 18:47, Scott Lurndal wrote:
>>> Kaz Kylheku <433-929-6894@kylheku.com> writes:
>>>> On 2024-03-10, Michael S <already5chosen@yahoo.com> wrote:
>>>>> On Sat, 09 Mar 2024 16:37:19 -0800
>>>>> Keith Thompson <Keith.S.Thompson+u@gmail.com> wrote:
>>>>>> strdup() and strndup() are being added to the C23 standard.
>>>>>>
>>>>>
>>>>> What is justification?
>>>>
>>>> strdup is required by POSIX already and thus widely implemented.
>>>> Many programmers who are not into standards already assume it's in C.
>>>>
>>>> For decades, portable programs have been doing things like this:
>>>>
>>>> #if HAVE_STRDUP
>>>> #define xstrdup(s) strdup(s)
>>>> #else
>>>> char *xstrdup(const char *); // own definition
>>>> #endif
>>>>
>>>>> What strdup() can do better, for any chosen value of better, than
>>>>> strlen()+malloc()+memcpy() ?
>>>>
>>>> Not take up space in every application for a common library routine.
>>>
>>> It's a form of lazy programming. I've seen a lot of open source
>>> code that uses strdup without checking for failure and frequently
>>> "forgetting" to free the result.
>>
>> And it is probably more likely that machine with many gigabytes of RAM
>
> Actually, your assumptions that:
> 1) strdup is the only allocation function used by an application
> 2) all strings are "short"
>
> are both flawed.
>
The bank has many billions. But there is a banking crisis on and it is
about to fail. And someone, somewhere, will be that man who tries to
withdraw some money and is told "no". But how likely is that man to be
you with your 20 pounds at the cashpoint? How likely is it to be another
bank making a cash call for a 100 million or so?

And how often do banks fail, actually, and how often does government
take action when it's heading that way, but nowhere near failing yet?

--
Check out Basic Algorithms and my other books:
https://www.lulu.com/spotlight/bgy1mm

Malcolm McLean <malcolm.arthur.mclean@gmail.com> writes:
> On 11/03/2024 17:00, Scott Lurndal wrote:
>> Malcolm McLean <malcolm.arthur.mclean@gmail.com> writes:
>>> On 10/03/2024 18:47, Scott Lurndal wrote:
>>>> Kaz Kylheku <433-929-6894@kylheku.com> writes:
>>>>> On 2024-03-10, Michael S <already5chosen@yahoo.com> wrote:
>>>>>> On Sat, 09 Mar 2024 16:37:19 -0800
>>>>>> Keith Thompson <Keith.S.Thompson+u@gmail.com> wrote:
>>>>>>> strdup() and strndup() are being added to the C23 standard.
>>>>>>>
>>>>>>
>>>>>> What is justification?
>>>>>
>>>>> strdup is required by POSIX already and thus widely implemented.
>>>>> Many programmers who are not into standards already assume it's in C.
>>>>>
>>>>> For decades, portable programs have been doing things like this:
>>>>>
>>>>> #if HAVE_STRDUP
>>>>> #define xstrdup(s) strdup(s)
>>>>> #else
>>>>> char *xstrdup(const char *); // own definition
>>>>> #endif
>>>>>
>>>>>> What strdup() can do better, for any chosen value of better, than
>>>>>> strlen()+malloc()+memcpy() ?
>>>>>
>>>>> Not take up space in every application for a common library routine.
>>>>
>>>> It's a form of lazy programming. I've seen a lot of open source
>>>> code that uses strdup without checking for failure and frequently
>>>> "forgetting" to free the result.
>>>
>>> And it is probably more likely that machine with many gigabytes of RAM
>> Actually, your assumptions that:
>> 1) strdup is the only allocation function used by an application
>> 2) all strings are "short"
>> are both flawed.
>>
> The bank has many billions. But there is a banking crisis on and it is
> about to fail. And someone, somewhere, will be that man who tries to
> withdraw some money and is told "no". But how likely is that man to be
> you with your 20 pounds at the cashpoint? How likely is it to be
> another bank making a cash call for a 100 million or so?

If I withdraw 20 pounds from my bank, I'll bet you that 20 pounds that
the bank still checks whether it has the money.

I'd rather write correct code than code that almost certainly happens to
work. Sure, strdup() is unlikely to fail-- but I'm going to check the
result.

> And how often do banks fail, actually, and how often does government
> take action when it's heading that way, but nowhere near failing yet?

The government isn't going to intervene when your laptop is running low
on memory.

--
Keith Thompson (The_Other_Keith) Keith.S.Thompson+u@gmail.com
Working, but not speaking, for Medtronic
void Void(void) { Void(); } /* The recursive call of the void */

On 03/11/2024 05:07 PM, Lawrence D'Oliveiro wrote:
> On Fri, 8 Mar 2024 21:36:14 -0800, Ross Finlayson wrote:
>
>> What I'd like to know about is who keeps dialing the "harmonization"
>> efforts, which really must give grouse to the "harmonisation"
>> spellers ...
>
> Some words came from French and had “-ize”, others did not and had “-ise”.
> Some folks in Britain decided to change the former to the latter.
>
> “Televise”, “merchandise”, “advertise” -- never any “-ize” form.
>
> “Synchronize”, “harmonize”, “apologize” -- “-ize” originally.
>

Hey thanks that's something I hadn't thought,
that the harmonization was coming from this
side of the pond besides vice-versa, with regards
to that "harmonization" is an effort in controlled
languages in terms of natural languages which
are organic though of course subject their extended
memory the written corpi, which I write corpi, not corpora.

It's like when the dictionary adds new words,
the old words are still words, in, the "Wortbuch",
an abstract dictionary of all the words, that I read
about in Curme. (I'm a fan of Tesniere and Curme.)

About parsing and re-writing systems, I'm really wondering
a lot about, compilation units, lines, spacing and indentation,
blocks, comments, quoting, punctuation, identifiers,
brackets, commas, and stops, how to write grammars
for all sorts usual source language in those, and result,
a novel sort of linear data structure above those,
in whatever languages so recognized in those,
and any sections it doesn't as the source text.

I looked around a bit and after re-writing on the Wiki
and "multi-pass parser" there are some sorts ideas,
usually in terms of fungible intermediate languages
for targeting those to whatever languages, here
though mostly to deal with a gamut of existing code,
there are lots of syntax recognizers and highlighters
and this kind of thing, "auto-detect" in the static
analysis toolkit, the languages, then as with regards to
that a given compilation unit is only gonna be one or
a few languages in it, with regards for example to
"code in text" or "text in code", about comments,
sections, blocks, or "language integrated code"
or "convenience code", "sugar modes", you know,
about what the _grammar_ specifications would be,
and the lexical and syntax the specifications, to
arrive at a multi-pass parser, that compiles a whole
bunch of language specs, finds which ones apply
where to the compilation unit, then starts building
them up "lifting" them above the character sequence,
building an "abstract syntax sequence" (yeah I know)
above that, then building a model of the productions
directly above that, that happens to be exactly derived
from the grammar productions, with the same sort
of structure as the grammar productions.

(Order, loop, optional, a superset of eBNF, to support
syntaxes with bracket blocks like C-style and syntaxes
with indent blocks though I'm not into that, the various
inversions of comments and code, the various interpolations
of quoting, brackets and grouping and precedence,
commas and joining and separating, and because SQL
doesn't really comport itself to BNF, these kinds of things.)

Of course it's obligatory that this would be about C/C++
and as with regards to Java which of course is in the
same style, or that its derivative, is for example that
M4/C/C++ code is already to a multi-pass parser, and,
Java at some point added language features which
fundamentally require a multi-pass parser, so it's not
like the entire resources of the mainframe has to fit
a finite-state-machine on the read-head, in fact at
compile-time specifically there's "it's fair to consider
a concatenation of the compilation units as a linear
input in space", then figuring the "liftings" are linear
in that, in space, then that the productions whence
derived are as concise as the productions a minimal
model, thus discardable the intermediate bit, is for
introducing a sort of common model of language
representation, source language, for reference
implementations of the grammars, then to make
the act of ingestion of sources in languages as a
first-class kind of thing, I'm looking for one of those,
and that's about as much I've figured out it is.

It's such a usual idea I must imagine that it's
commonplace, as it's just the very most simple
act of the model of iterating these things and
reading them out.

I probably might not care about it but getting
to where it takes a parser that can parse SQL
for example, or, you know, when there are lots
of source formats but it's just data and definitions,
yeah if you know that there's like a very active
open project in that I'd be real interested in a
sort of "source/object/relational mapping", ...,
as it were, "source/grammatical-production mapping",
what results you identify grammars and pick sources
and it prints out the things.

I'm familiar with the traditional approaches,
and intend to employ them. I figure this
must be a very traditional approach if
nobody's heard of it.

On 12/03/2024 01:20, Keith Thompson wrote:
> Malcolm McLean <malcolm.arthur.mclean@gmail.com> writes:
>> On 11/03/2024 17:00, Scott Lurndal wrote:
>>> Malcolm McLean <malcolm.arthur.mclean@gmail.com> writes:
>>>> On 10/03/2024 18:47, Scott Lurndal wrote:
>>>>> Kaz Kylheku <433-929-6894@kylheku.com> writes:
>>>>>> On 2024-03-10, Michael S <already5chosen@yahoo.com> wrote:
>>>>>>> On Sat, 09 Mar 2024 16:37:19 -0800
>>>>>>> Keith Thompson <Keith.S.Thompson+u@gmail.com> wrote:
>>>>>>>> strdup() and strndup() are being added to the C23 standard.
>>>>>>>>
>>>>>>>
>>>>>>> What is justification?
>>>>>>
>>>>>> strdup is required by POSIX already and thus widely implemented.
>>>>>> Many programmers who are not into standards already assume it's in C.
>>>>>>
>>>>>> For decades, portable programs have been doing things like this:
>>>>>>
>>>>>> #if HAVE_STRDUP
>>>>>> #define xstrdup(s) strdup(s)
>>>>>> #else
>>>>>> char *xstrdup(const char *); // own definition
>>>>>> #endif
>>>>>>
>>>>>>> What strdup() can do better, for any chosen value of better, than
>>>>>>> strlen()+malloc()+memcpy() ?
>>>>>>
>>>>>> Not take up space in every application for a common library routine.
>>>>>
>>>>> It's a form of lazy programming. I've seen a lot of open source
>>>>> code that uses strdup without checking for failure and frequently
>>>>> "forgetting" to free the result.
>>>>
>>>> And it is probably more likely that machine with many gigabytes of RAM
>>> Actually, your assumptions that:
>>> 1) strdup is the only allocation function used by an application
>>> 2) all strings are "short"
>>> are both flawed.
>>>
>> The bank has many billions. But there is a banking crisis on and it is
>> about to fail. And someone, somewhere, will be that man who tries to
>> withdraw some money and is told "no". But how likely is that man to be
>> you with your 20 pounds at the cashpoint? How likely is it to be
>> another bank making a cash call for a 100 million or so?
>
> If I withdraw 20 pounds from my bank, I'll bet you that 20 pounds that
> the bank still checks whether it has the money.
>
> I'd rather write correct code than code that almost certainly happens to
> work. Sure, strdup() is unlikely to fail-- but I'm going to check the
> result.
>
>> And how often do banks fail, actually, and how often does government
>> take action when it's heading that way, but nowhere near failing yet?
>
> The government isn't going to intervene when your laptop is running low
> on memory.
>
No, it's an analogy. You run lots of apps gobbling lots of memory, and
maybe a program won't launch or thngs start to slow or a warning comes
up, and you realise that soon the program of interest might run out of
memory, and so you shut other things down so that that doesn't happen.
And so it's pretty rare to actually run out of memory unless the program
isn't correct and there is a leak.

--
Check out Basic Algorithms and my other books:
https://www.lulu.com/spotlight/bgy1mm

Keith Thompson <Keith.S.Thompson+u@gmail.com> writes:
>Malcolm McLean <malcolm.arthur.mclean@gmail.com> writes:
>> On 11/03/2024 17:00, Scott Lurndal wrote:
>>> Malcolm McLean <malcolm.arthur.mclean@gmail.com> writes:
>>>> On 10/03/2024 18:47, Scott Lurndal wrote:
>>>>> Kaz Kylheku <433-929-6894@kylheku.com> writes:
>>>>>> On 2024-03-10, Michael S <already5chosen@yahoo.com> wrote:
>>>>>>> On Sat, 09 Mar 2024 16:37:19 -0800
>>>>>>> Keith Thompson <Keith.S.Thompson+u@gmail.com> wrote:
>>>>>>>> strdup() and strndup() are being added to the C23 standard.
>>>>>>>>
>>>>>>>
>>>>>>> What is justification?
>>>>>>
>>>>>> strdup is required by POSIX already and thus widely implemented.
>>>>>> Many programmers who are not into standards already assume it's in C.
>>>>>>
>>>>>> For decades, portable programs have been doing things like this:
>>>>>>
>>>>>> #if HAVE_STRDUP
>>>>>> #define xstrdup(s) strdup(s)
>>>>>> #else
>>>>>> char *xstrdup(const char *); // own definition
>>>>>> #endif
>>>>>>
>>>>>>> What strdup() can do better, for any chosen value of better, than
>>>>>>> strlen()+malloc()+memcpy() ?
>>>>>>
>>>>>> Not take up space in every application for a common library routine.
>>>>>
>>>>> It's a form of lazy programming. I've seen a lot of open source
>>>>> code that uses strdup without checking for failure and frequently
>>>>> "forgetting" to free the result.
>>>>
>>>> And it is probably more likely that machine with many gigabytes of RAM
>>> Actually, your assumptions that:
>>> 1) strdup is the only allocation function used by an application
>>> 2) all strings are "short"
>>> are both flawed.
>>>
>> The bank has many billions. But there is a banking crisis on and it is
>> about to fail. And someone, somewhere, will be that man who tries to
>> withdraw some money and is told "no". But how likely is that man to be
>> you with your 20 pounds at the cashpoint? How likely is it to be
>> another bank making a cash call for a 100 million or so?
>
>If I withdraw 20 pounds from my bank, I'll bet you that 20 pounds that
>the bank still checks whether it has the money.

And as for "how likely it is that the bank will not be able
to honor withdrawal requests", I refer Malcolm to 1931.

"In 1930, 1,352 banks held more than $853 million in
deposits; in 1931, one year later, 2,294 banks failed
with nearly $1.7 billion in deposits."

On 06/03/2024 04:43, Mr. Man-wai Chang wrote:
> On 5/3/2024 9:51 pm, Mr. Man-wai Chang wrote:
>> On 3/3/2024 7:13 am, Lynn McGuire wrote:
>>>
>>> "The Biden administration backs a switch to more memory-safe programming
>>> languages. The tech industry sees their point, but it won't be easy."
>>>
>>> No.  The feddies want to regulate software development very much.  They
>>> have been talking about it for at least 20 years now.  This is a very
>>> bad thing.
>>
>> A responsible, good progreammer or a better C/C++ pre-processor can
>> avoid a lot of problems!!
>
> Or maybe A.I.-assisted code analyzer?? But there are still blind spots...

I think AI could be used and give goods result but it is not ideal.
The advantage of AI it could understand patterns. Like the names init
and destroy could work as tips or patterns.

However, I think programming needs a formal language for contracts and
the static analysis needs to check them.
Also ideally is better contracts for the interface rather having to see
the body of the functions.

On 06/03/2024 04:43, Mr. Man-wai Chang wrote:
> On 5/3/2024 9:51 pm, Mr. Man-wai Chang wrote:
>> On 3/3/2024 7:13 am, Lynn McGuire wrote:
>>>
>>> "The Biden administration backs a switch to more memory-safe programming
>>> languages. The tech industry sees their point, but it won't be easy."
>>>
>>> No.  The feddies want to regulate software development very much.  They
>>> have been talking about it for at least 20 years now.  This is a very
>>> bad thing.
>>
>> A responsible, good progreammer or a better C/C++ pre-processor can
>> avoid a lot of problems!!
>
> Or maybe A.I.-assisted code analyzer?? But there are still blind spots...

I think AI could be used and give goods result but it is not ideal.
The advantage of AI it could understand patterns. Like the names init
and destroy could work as tips or patterns.

However, I think programming needs a formal language for contracts and
the static analysis needs to check them.
Also ideally is better contracts for the interface rather having to see
the body of the functions.

Malcolm McLean <malcolm.arthur.mclean@gmail.com> writes:
> On 12/03/2024 01:20, Keith Thompson wrote:
>> Malcolm McLean <malcolm.arthur.mclean@gmail.com> writes:
[...]
>> If I withdraw 20 pounds from my bank, I'll bet you that 20 pounds
>> that the bank still checks whether it has the money. I'd rather
>> write correct code than code that almost certainly happens to work.
>> Sure, strdup() is unlikely to fail-- but I'm going to check the
>> result.
>>
>>> And how often do banks fail, actually, and how often does government
>>> take action when it's heading that way, but nowhere near failing yet?
>> The government isn't going to intervene when your laptop is running
>> low on memory.
>>
> No, it's an analogy. You run lots of apps gobbling lots of memory, and
> maybe a program won't launch or thngs start to slow or a warning comes
> up, and you realise that soon the program of interest might run out of
> memory, and so you shut other things down so that that doesn't happen.
> And so it's pretty rare to actually run out of memory unless the
> program isn't correct and there is a leak.

Are you relying in a person sitting in front of the computer and
noticing that memory is running low? The vast majority of software
doesn't have someone watching it.

I agree that an allocation failure in strdup() is unlikely. Are you
suggesting that that means you needn't bother checking the result?

--
Keith Thompson (The_Other_Keith) Keith.S.Thompson+u@gmail.com
Working, but not speaking, for Medtronic
void Void(void) { Void(); } /* The recursive call of the void */

On 12/03/2024 15:55, Scott Lurndal wrote:
> Keith Thompson <Keith.S.Thompson+u@gmail.com> writes:
>> Malcolm McLean <malcolm.arthur.mclean@gmail.com> writes:
>>> On 11/03/2024 17:00, Scott Lurndal wrote:
>>>> Malcolm McLean <malcolm.arthur.mclean@gmail.com> writes:
>>>>> On 10/03/2024 18:47, Scott Lurndal wrote:
>>>>>> Kaz Kylheku <433-929-6894@kylheku.com> writes:
>>>>>>> On 2024-03-10, Michael S <already5chosen@yahoo.com> wrote:
>>>>>>>> On Sat, 09 Mar 2024 16:37:19 -0800
>>>>>>>> Keith Thompson <Keith.S.Thompson+u@gmail.com> wrote:
>>>>>>>>> strdup() and strndup() are being added to the C23 standard.
>>>>>>>>>
>>>>>>>>
>>>>>>>> What is justification?
>>>>>>>
>>>>>>> strdup is required by POSIX already and thus widely implemented.
>>>>>>> Many programmers who are not into standards already assume it's in C.
>>>>>>>
>>>>>>> For decades, portable programs have been doing things like this:
>>>>>>>
>>>>>>> #if HAVE_STRDUP
>>>>>>> #define xstrdup(s) strdup(s)
>>>>>>> #else
>>>>>>> char *xstrdup(const char *); // own definition
>>>>>>> #endif
>>>>>>>
>>>>>>>> What strdup() can do better, for any chosen value of better, than
>>>>>>>> strlen()+malloc()+memcpy() ?
>>>>>>>
>>>>>>> Not take up space in every application for a common library routine.
>>>>>>
>>>>>> It's a form of lazy programming. I've seen a lot of open source
>>>>>> code that uses strdup without checking for failure and frequently
>>>>>> "forgetting" to free the result.
>>>>>
>>>>> And it is probably more likely that machine with many gigabytes of RAM
>>>> Actually, your assumptions that:
>>>> 1) strdup is the only allocation function used by an application
>>>> 2) all strings are "short"
>>>> are both flawed.
>>>>
>>> The bank has many billions. But there is a banking crisis on and it is
>>> about to fail. And someone, somewhere, will be that man who tries to
>>> withdraw some money and is told "no". But how likely is that man to be
>>> you with your 20 pounds at the cashpoint? How likely is it to be
>>> another bank making a cash call for a 100 million or so?
>>
>> If I withdraw 20 pounds from my bank, I'll bet you that 20 pounds that
>> the bank still checks whether it has the money.
>
> And as for "how likely it is that the bank will not be able
> to honor withdrawal requests", I refer Malcolm to 1931.
>
> "In 1930, 1,352 banks held more than $853 million in
> deposits; in 1931, one year later, 2,294 banks failed
> with nearly $1.7 billion in deposits."
>
So 2,294 Americans in 1932 had the experience "you want to withdraw
cash? OK, here we go, oh sorry, that just took us over the edge and the
bank is now closed". Everyone else, either "here's your money", or
"sorry, the bank has now closed, no more withdrawal requests".
--
Check out Basic Algorithms and my other books:
https://www.lulu.com/spotlight/bgy1mm

Malcolm McLean <malcolm.arthur.mclean@gmail.com> writes:
>On 12/03/2024 15:55, Scott Lurndal wrote:
>> Keith Thompson <Keith.S.Thompson+u@gmail.com> writes:
>>> Malcolm McLean <malcolm.arthur.mclean@gmail.com> writes:
>>>> On 11/03/2024 17:00, Scott Lurndal wrote:
>>>>> Malcolm McLean <malcolm.arthur.mclean@gmail.com> writes:
>>>>>> On 10/03/2024 18:47, Scott Lurndal wrote:
>>>>>>> Kaz Kylheku <433-929-6894@kylheku.com> writes:
>>>>>>>> On 2024-03-10, Michael S <already5chosen@yahoo.com> wrote:
>>>>>>>>> On Sat, 09 Mar 2024 16:37:19 -0800
>>>>>>>>> Keith Thompson <Keith.S.Thompson+u@gmail.com> wrote:
>>>>>>>>>> strdup() and strndup() are being added to the C23 standard.
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>> What is justification?
>>>>>>>>
>>>>>>>> strdup is required by POSIX already and thus widely implemented.
>>>>>>>> Many programmers who are not into standards already assume it's in C.
>>>>>>>>
>>>>>>>> For decades, portable programs have been doing things like this:
>>>>>>>>
>>>>>>>> #if HAVE_STRDUP
>>>>>>>> #define xstrdup(s) strdup(s)
>>>>>>>> #else
>>>>>>>> char *xstrdup(const char *); // own definition
>>>>>>>> #endif
>>>>>>>>
>>>>>>>>> What strdup() can do better, for any chosen value of better, than
>>>>>>>>> strlen()+malloc()+memcpy() ?
>>>>>>>>
>>>>>>>> Not take up space in every application for a common library routine.
>>>>>>>
>>>>>>> It's a form of lazy programming. I've seen a lot of open source
>>>>>>> code that uses strdup without checking for failure and frequently
>>>>>>> "forgetting" to free the result.
>>>>>>
>>>>>> And it is probably more likely that machine with many gigabytes of RAM
>>>>> Actually, your assumptions that:
>>>>> 1) strdup is the only allocation function used by an application
>>>>> 2) all strings are "short"
>>>>> are both flawed.
>>>>>
>>>> The bank has many billions. But there is a banking crisis on and it is
>>>> about to fail. And someone, somewhere, will be that man who tries to
>>>> withdraw some money and is told "no". But how likely is that man to be
>>>> you with your 20 pounds at the cashpoint? How likely is it to be
>>>> another bank making a cash call for a 100 million or so?
>>>
>>> If I withdraw 20 pounds from my bank, I'll bet you that 20 pounds that
>>> the bank still checks whether it has the money.
>>
>> And as for "how likely it is that the bank will not be able
>> to honor withdrawal requests", I refer Malcolm to 1931.
>>
>> "In 1930, 1,352 banks held more than $853 million in
>> deposits; in 1931, one year later, 2,294 banks failed
>> with nearly $1.7 billion in deposits."
>>
>So 2,294 Americans in 1932 had the experience "you want to withdraw
>cash? OK, here we go, oh sorry, that just took us over the edge and the
>bank is now closed". Everyone else, either "here's your money", or
>"sorry, the bank has now closed, no more withdrawal requests".

I'm pretty sure the quote above from Wikipedia is written in
a form of English that even the English understand, so I don't see
how you came up with the idea that only 2,294 Americans were
affected when 2,294 banks failed.

Nor to I see the relevence to checking the return value
of strdup(3).

On 3/12/24 19:50, Scott Lurndal wrote:
> Malcolm McLean <malcolm.arthur.mclean@gmail.com> writes:
....
>> So 2,294 Americans in 1932 had the experience "you want to withdraw
>> cash? OK, here we go, oh sorry, that just took us over the edge and the
>> bank is now closed". Everyone else, either "here's your money", or
>> "sorry, the bank has now closed, no more withdrawal requests".
>
> I'm pretty sure the quote above from Wikipedia is written in
> a form of English that even the English understand, so I don't see
> how you came up with the idea that only 2,294 Americans were
> affected when 2,294 banks failed.

Every bank fails at some particular time, when it runs out of money.
There is always one particular claim on that bank's resources that
caused it to run out and for some reason he's focusing on those claims.
Other Americans also had problems due to the bank having already failed,
but at most 2294 Americans made claims that actually triggered that
failure. I haven't a clue as to why he considers that fact important,
but he has correctly made that distinction.

Actually, it may be that in some cases a bank closed immediately as soon
as a successful withdrawal request left it with too little funds, so the
number might actually be less that 2294.

> Nor to I see the relevence to checking the return value
> of strdup(3).

Agreed.

On 12/03/2024 22:31, Keith Thompson wrote:
> Malcolm McLean <malcolm.arthur.mclean@gmail.com> writes:
>> On 12/03/2024 01:20, Keith Thompson wrote:
>>> Malcolm McLean <malcolm.arthur.mclean@gmail.com> writes:
> [...]
>>> If I withdraw 20 pounds from my bank, I'll bet you that 20 pounds
>>> that the bank still checks whether it has the money. I'd rather
>>> write correct code than code that almost certainly happens to work.
>>> Sure, strdup() is unlikely to fail-- but I'm going to check the
>>> result.
>>>
>>>> And how often do banks fail, actually, and how often does government
>>>> take action when it's heading that way, but nowhere near failing yet?
>>> The government isn't going to intervene when your laptop is running
>>> low on memory.
>>>
>> No, it's an analogy. You run lots of apps gobbling lots of memory, and
>> maybe a program won't launch or thngs start to slow or a warning comes
>> up, and you realise that soon the program of interest might run out of
>> memory, and so you shut other things down so that that doesn't happen.
>> And so it's pretty rare to actually run out of memory unless the
>> program isn't correct and there is a leak.
>
> Are you relying in a person sitting in front of the computer and
> noticing that memory is running low? The vast majority of software
> doesn't have someone watching it.
>
The server runs out of memory once. Sometimes yes, it's "these things
happen, just get it back up". But more often. it's "Can't have that
happening again. Shut something else down we don;t really need, or maybe
buy an extra 2GB. Only ten dollars after all."

> I agree that an allocation failure in strdup() is unlikely. Are you
> suggesting that that means you needn't bother checking the result?
>
If it's genuinely the case that an electrical faiure is more likely, is
there a really a point? But for my code, the test would go in, because
it usually is intended to last for a very long time, and small memory
machines might come back into fashion (eg smaller but very cheap and
very fast, or maybe it will be used in an embedded system)
--
Check out Basic Algorithms and my other books:
https://www.lulu.com/spotlight/bgy1mm

On 13/03/2024 08:46, James Kuyper wrote:
> On 3/12/24 19:50, Scott Lurndal wrote:
>> Malcolm McLean <malcolm.arthur.mclean@gmail.com> writes:
> ...
>>> So 2,294 Americans in 1932 had the experience "you want to withdraw
>>> cash? OK, here we go, oh sorry, that just took us over the edge and the
>>> bank is now closed". Everyone else, either "here's your money", or
>>> "sorry, the bank has now closed, no more withdrawal requests".
>>
>> I'm pretty sure the quote above from Wikipedia is written in
>> a form of English that even the English understand, so I don't see
>> how you came up with the idea that only 2,294 Americans were
>> affected when 2,294 banks failed.
>
> Every bank fails at some particular time, when it runs out of money.
> There is always one particular claim on that bank's resources that
> caused it to run out and for some reason he's focusing on those claims.
> Other Americans also had problems due to the bank having already failed,
> but at most 2294 Americans made claims that actually triggered that
> failure. I haven't a clue as to why he considers that fact important,
> but he has correctly made that distinction.
>
> Actually, it may be that in some cases a bank closed immediately as soon
> as a successful withdrawal request left it with too little funds, so the
> number might actually be less that 2294.
>

There are all kinds of different scenarios possible, leading to all
sorts of different numbers for the people caught in exceptional
circumstances during bank failure. I think the least likely conceivable
possibility is to imagine that the failure of a bank is an instantaneous
event that happens in the middle of a single withdrawal.

The analogue with software would be a program with thousands of threads
running at once, across dozens of cores, on a system that is running
many such programs. Each thread will be allocating and deallocating
memory asynchronously. The system will have virtual memory, and lots of
other IO going on that affects the speed of the VM backing. To imagine
that a /single/ normal allocation attempt turns that from a fully
working system to a fully failed system is clearly absurd. And the same
applies to banks.

>> Nor to I see the relevence to checking the return value
>> of strdup(3).
>
> Agreed.

Also agreed.

Analogies are fine if they shed some light on the topic under
discussion. This one started out badly (it seems to have been a
suggestion that some bad things only occur very rarely, so you should
pretend they won't happen to you), and it got rapidly less realistic and
less relevant.

Michael S <already5chosen@yahoo.com> writes:

> On Tue, 5 Mar 2024 22:58:10 -0000 (UTC)
> Lawrence D'Oliveiro <ldo@nz.invalid> wrote:
>
>> On Tue, 5 Mar 2024 11:11:03 +0200, Michael S wrote:
>>
>>> On Tue, 5 Mar 2024 01:54:46 -0000 (UTC)
>>> Lawrence D'Oliveiro <ldo@nz.invalid> wrote:
>>>
>>>> Discord did some benchmarking of its back-end servers, which had
>>>> been using Go, and decided that switching to Rust offered better
>>>> performance.
>>>
>>> - for big and complex real-world back-end processing, writing
>>> working solution in go will take 5 time less man hours than
>>> writing it in Rust
>>
>> Nevertheless, they found the switch to Rust worthwhile.
>
> I read a little more about it.
> https://discord.com/blog/why-discord-is-switching-from-go-to-rust
>
> Summary: performance of one of Discord's most heavy-duty servers
> suffered from weakness in implementation of Go garbage collector.
> [...]
>
> I have few questions about the story, most important one is
> whether the weakness of this sort is specific to GC of Go, due
> to its relative immaturity or more general and applies equally
> to most mature GCs on the market, i.e. J2EE and .NET.

After reading the article, it seems clear that the design of the
GC used in Go is not a good fit to the Discord server workload.
That is not to say that Go's GC would be a bad fit for other
workloads, only that it's not a good choice for the Discord
server. Also it seems clear that other approaches to GC design
(meant in the sense of already having been thought of and tried)
would be just fine for the Discord server. Of course whether
such schemes would be a good fit for the Go environment is
a separate question (and one about which I have nothing to
offer since I know very little about Go).

> Another question is whether the problem is specific to GC-style
> of automatic memory management (AMM) or applies, at least to
> some degree, to other forms of AMM, most importantly, to AMMs
> based on Reference Counting used by Swift and also popular in
> C++.

It's very hard to make a blanket statement that applies to all
the different approaches to garbage collection, or to other
automatic memory management schemes (with reference counting as a
specific example), in a general way. My experience with garbage
collected environments is that they are perfectly usable both for
long-term use and for interactive use. Reference counting too:
typically RC gives a smoother feel, but that comes at a cost,
because RC does not, by itself, reclaim circular structures.
That means that either, one, code must be written to break the
circularity of such structures (so memory management is not fully
automated); or two, periodically some sort of more general GC
method must be invoked to reclaim them; or three, eventually
more and more memory is used to where the application must be
rebooted (like the memory usage patterns of some popular web
browsers). The question is what kind of workloads need to be
supported - some schemes are good for typical interactive, but
short-term, applications, other schemes are better for the sort
of long-term server processes like the Discord example. The
space of already existing techniques for doing GC is pretty
large - if a particular kind of workload needs to be supported,
there is a very good chance that an appropriate GC scheme can
be found without much difficulty.

Incidentally, there is no such thing as a fully automatic memory
management system. Even full garbage collection sometimes needs
help from the programmer so all memory is eventually reclaimed.
(If you're feeling brave try doing a web search for "ephemeron".)
The point of all AMM schemes is not to reduce the amount of
programmer effort to zero but simply to greatly reduce it (and
hopefully reduce it to zero in many of the most common cases).
But for complicated programs it's almost inevitable that some
programmer-written code needs to be included to help whatever
automated mechanisms are used.

> Of course, I don't expected that my questions will be answered
> fully on comp.lang.c, but if some knowledgeable posters will try
> to answer I would appreciate.

The questions presented were somewhat general, and so the answers
given are also rather general. In spite of that I hope you found
what you're looking for.

Blue-Maned_Hawk:

> --
> Blue-Maned_Hawk│shortens to Hawk│/blu.mɛin.dʰak/│he/him/his/himself/Mr.
> blue-maned_hawk.srht.site
> TEST

This one is readable, but I still hate UTF-8 whenever plain old ASCII
or a bilingual 8-bit encodinng would suffice.

By the way, I liked your website, including the color scheme. Pray
to not call yourself a mediocre writer: leave the observation to the
reader.

--
() ascii ribbon campaign -- against html e-mail
/\ www.asciiribbon.org -- against proprietary attachments

On Fri, 8 Mar 2024 15:32:22 +0100, David Brown wrote:

> And it will be tough on the cache as everything has to be copied and
> moved.

I think all kinds of garbage collector end up being tough on the cache.
Because remember, they are doing things with lots of blocks of memory that
haven’t been accessed recently, and therefore are not likely to be in the
cache.

On Fri, 8 Mar 2024 14:41:16 +0200, Paavo Helde wrote:

> AFAIK CPython uses reference counting ...

Combination of reference-counting as a first resort, with full garbage
collection to deal with those less common cases where you have reference
cycles. Trying to get the best of both worlds.

The trouble with reference-counting is it impacts multithreading
performance. However, the CPython developers have a scheme to deal with
this, by making the reference counts a little less deterministic (i.e.
there may be a slight delay before they become fully correct). I think
this is a complicated idea, and it may take them some time to get it fully
implemented.