On 11/10/2021 9:34 PM, antispam@math.uni.wroc.pl wrote:
> Don Y <blockedofcourse@foo.invalid> wrote:

>>> Classic trick it to parametrize. However even if you
>>> parametrize there are hundreds of design decisions going
>>> into relatively small piece of code. If you expose all
>>> design decisions then user as well may write his/her own
>>> code because complexity will be similar. So normaly
>>> parametrization is limited and there will be users who
>>> find hardcoded desion choices inadequate.
>>>
>>> Another things is that current tools are rather weak
>>> at supporting parametrization.
>>
>> Look at a fleshy UART driver and think about how you would decompose
>> it into N different variants that could be "compile time configurable".
>> You'll be surprised as to how easy it is. Even if the actual UART
>> hardware differs from instance to instance.
>
> UART-s are simple. And yet some things are tricky: in C to have
> "compile time configurable" buffer size you need to use macros.
> Works, but in a sense UART implementation "leaks" to user code.

You can configure using manifest constants, conditional compilation,
or even run-time switches. Or, by linking against different
"support" routines. How and where the configuration "leaks"
into user code is a function of the configuration mechanisms that
you decide to employ.

E.g., You'd likely NOT design your network stack to be tightly integrated
with your choice of NIC (all else being equal) -- simply because you'd
want to be able to reuse the stack with some *other* NIC without having
to rewrite it.

OTOH, it's not unexpected to want to isolate the caching of ARP results
in an "application specific" manner as you'll likely know the sorts (and
number!) of clients/services with which the device in question will be
connecting. So, that (sub)module can be replaced with something most
appropriate to the application yet with a "standardized" interface to
the stack itself (*YOU* define that standard)

All of these require decisions up-front; you can't expect to be able to
retrofit an existing piece of code (cheaply) to support a more
modular/configurable implementation in the future.

But, personal experience teaches you what you are likely to need
by way of flexibility/configurability. Most folks tend to eork
in a very narrow set of application domains. Chances are, the
network stack you design for an embedded product will be considerably
different than one for a desktop OS. If you plan to straddle
both domains, then the configurability challenge is greater!

>> There are costs to both approaches. If I dedicate resource to
>> ensuring I don't miss anything, then some other aspect of the
>> design will bear that cost. If I rely on detecting missed
>> messages, then I have to put a figure on their relative
>> likelihood so my device doesn't fail to provide its desired
>> functionality (because it is always missing one or two characters
>> out of EVERY message -- and, thus, sees NO messages).
>
> My thinking goes toward using relatively short messages and
> buffer big enough for two messages.

You can also design with the intent of parsing messages before they are
complete and "reducing" them along the way. This is particularly
important if messages can have varying length *or* there is a possibility
for the ass end of a message to get dropped (how do you know when the
message is complete? Imagine THE USER misconfiguring your device
to expect CRLFs and the traffic only contains newlines; the terminating
CRLF never arrives!)

[At the limit case, a message reduces to a concept -- that is represented
in some application specific manner: "Start the motor", "Clear the screen",
etc.]

Barcodes are messages (character sequences) of a sort. I typically
process a barcode at several *concurrent* levels:
- an ISR that captures the times of transitions (black->white->black)
- a task that reduces the data captured by the ISR into "bar widths"
- a task that aggregates bar widths to form characters
- a task that parses character sequences to determine valid messages
- an application layer interpretation (or discard) of that message
This allows each layer to decide when the data on which it relies
does not represent a valid barcode and discard some (or all) of it...
without waiting for a complete message to be present. So, the
resources that were consumed by that (partial?) message are
freed earlier.

As such, there is never a "start time" nor "end time" for a barcode
message -- because you don't want the user to have to "do something"
to tell you that he is now going to scan a barcode (otherwise, the
efficiency of using barcodes is subverted).

[Think about the sorts of applications that use barcodes; how many
require the user to tell the device "here comes a barcode, please start
your decoder algorithm NOW!"]

As users can abuse the barcode reader (there is nothing preventing them
from continuously scanning barcodes, in violation of any "protocol"
that the product may *intend*), you have to tolerate the case where
the data arrives faster than it can be consumed. *Knowing* where
(in the event stream) you may have "lost" some data (transitions,
widths, characters or messages) lets you resync to a less pathological
event stream later (when the user starts "behaving properly")

> If there is need for
> high speed I would go for continous messages and DMA
> transfers (using break interrupt to discover end of message
> in case of variable length messages). So device should
> be able to get all messages and in case of excess message
> trafic whole message could be dropped (possibly looking
> first for some high priority messages). Of course, there
> may be some externaly mandated message format and/or
> communitation protocal making DMA inappropriate.
> Still, assuming interrupts, all characters should reach
> interrupt handler, causing possibly some extra CPU
> load. The only possiblity of unnoticed loss of characters
> would be blocking interrupts too long. If interrupts can
> be blocked for too long, then I would expect loss of whole
> messages. In such case protocol should have something like
> "dont talk to me for next 100 miliseconds, I will be busy"
> to warn other nodes and request silence. Now, if you
> need to faithfully support sillyness like Modbus RTU timeouts,
> then I hope that you are adequatly paid...

>> IMO, the advantages of writing in a multitasking environment so
>> far outweigh the "costs" of an MTOS that it behooves one to consider
>> how to shoehorn that functionality into EVERY design.
>>
>> When writing in a HLL, there are complications that impose
>> constraints on how the MTOS provides its services. But, for small
>> projects written in ASM, you can gain the benefits of an MTOS
>> for very few bytes of code (and effectively zero RAM).
>
> Well, looking at books and articles I did not find convincing
> argument/example showing that one really need multitasking for
> small systems.

The advantages of multitasking lie in problem decomposition.
Smaller problems are easier to "get right", in isolation.
The *challenge* of multitasking is coordinating the interactions
between these semi-concurrent actors. Experience teaches you how
to partition a "job".

I want to blink a light at 1 Hz and check for a button to be
pressed which will start some action that may be lengthy. I
can move the light blink into an ISR (which GENERALLY is a ridiculous
use of that "resource") to ensure the 1Hz timeliness is maintained
regardless of what the "lengthy" task may be doing, at the time.

Or, I can break the lengthy task into smaller chunks that
are executed sequentially with "peeks" at the "light timer"
between each of those segments.

sequence1 := sequence2 := sequence3 := sequence4 := 0;
while (FOREVER) {
task1:
case sequence1++ {
0 => do_task1_step0;
1 => do_task1_step1;
2 => do_task1_step2;
...
}

do_light;

task2:
case sequence2++ {
0 => do_task2_step0;
1 => do_task2_step1;
2 => do_task2_step2;
...
}

do_light;

task3:
switch sequence3++ {
0 => do_task3_step0;
1 => do_task3_step1;
2 => do_task3_step2;
...
}

do_light;

...
}

When you need to do seven (or fifty) other "lengthy actions"
concurrently (each of which may introduce other "blinking
lights" or timeliness constraints), its easier (less brittle)
to put a structure in place that lets those competing actions
share the processor without requiring the developer to
micromanage at this level.

[50 tasks isn't an unusual load in a small system; video arcade
games from the early 80's -- 8 bit processors, kilobytes of
ROM+RAM -- would typically treat each object on the screen
(including bullets!) as a separate process]

The above example has low overhead for the apparent concurrency.
But, pushes all of the work onto the developer's lap. He has
to carefully size each "step" of each "task" to ensure the
overall system is responsive.

A nicer approach is to just let an MTOS handle the switching
between tasks. But, this comes at a cost of additional run-time
overhead (e.g., arbitrary context switches).

> I tend to think rather in terms of collection
> of coupled finite state machines (or if you prefer Petri net).
> State machines transition in response to events and may generate
> events. Each finite state machine could be a task. But it is
> not clear if it should. Some transitions are simple and should
> be fast and that I would do in interrupt handlers. Some
> other are triggered in regular way from other machines and
> are naturally handled by function calls. Some need queues.
> The whole thing fits resonably well in "super loop" paradigm.

I use FSMs for UIs and message parsing. They let the structure
of the code "rise to the top" where it is more visible (to another
developer) instead of burying it in subroutines and function calls.

"Event sources" create events which are consumed by FSMs, as
needed. So, a "power monitor" could generate POWER_FAIL, LOW_BATTERY,
POWER_RESTORED, etc. events while a "keypad decoder" could put out
ENTER, CLEAR, ALPHA_M, NUMERIC_5, etc. events.

Because there is nothing *special* about an "event", *ANY* piece of
code can generate them. Their significance assigns based on where
they are "placed" (in memory) and who/what can "see" them. So,
you can use an FSM to parse a message (using "received characters"
as an ordered stream of events) and "signal" MESSAGE_COMPLETE to
another FSM that is awaiting "messages" (along with a pointer to the
completed message)

>>>> From "KLEE: Unassisted and Automatic Generation of High-Coverage
>>>> Tests for Complex Systems Programs":
>>>>
>>>> KLEE finds important errors in heavily-tested code. It
>>>> found ten fatal errors in COREUTILS (including three
>>>> that had escaped detection for 15 years), which account
>>>> for more crashing bugs than were reported in 2006, 2007
>>>> and 2008 combined. It further found 24 bugs in BUSYBOX, 21
>>>> bugs in MINIX, and a security vulnerability in HISTAR? a
>>>> total of 56 serious bugs.
>>>>
>>>> Ooops! I wonder how many FOSS *eyes* missed those errors?
>>>
>>> Open source folks tend to be more willing to talk about bugs.
>>> And the above nicely shows that there is a lot of bugs, most
>>> waiting to by discovered.
>>
>> Part of the problem is ownership of the codebase. You are
>> more likely to know where your own bugs lie -- and, more
>> willing to fix them ("pride of ownership"). When a piece
>> of code is shared, over time, there seems to be less incentive
>> for folks to tackle big -- often dubious -- issues as the
>> "reward" is minimal (i.e., you may not own the code when the bug
>> eventually becomes a problem)
>
> Ownership may cause problems: there is tendency to "solve"
> problems locally, that is in code that given person "owns".
> This is good if there is easy local solution. However, this
> may also lead to ugly workarounds that really do not work
> well, while problem is easily solvable in different part
> ("owned" by different programmer). I have seen such thing
> several times, looking at whole codebase after some effort
> it was possible to do simple fix, while there were workarounds
> in different ("wrong") places. I had no contact with
> original authors, but it seems that workarounds were due to
> "ownership".

You are *always* at the mercy of the code's owner. Just as folks
are at YOUR mercy for the code that you (currently) exert ownership
over. The best compliments you'll receive are from folks who
inherit your codebase and can appreciate its structure and
consistency. Conversely, your worst nightmares will be inheriting
a codebase that was "hacked together", willy-nilly, by some number
of predecessors with no real concern over their "product" (code).

E.g., For FOSS projects, ownership isn't just a matter of who takes
"responsibility" for coordinating/merging diffs into the
codebase but, also, who has a compatible "vision" for the
codebase, going forward. You'd not want a radically different
vision from one owner to the next as this leads to gyrations in
the codebase that will be seen as instability by its users
(i.e., other developers).

I use PostgreSQL in my current design. I have no desire to
*develop* the RDBMS software -- let folks who understand that
sort of thing work their own magic on the codebase. I can add
value *elsewhere* in my designs.

But, I eventually have to take ownership of *a* version of the
software as I can't expect the "real owners" to maintain some
version that *I* find useful, possibly years from now. Once
I assume ownership of that chosen release, it will be my
priorities and skillset that drive how it evolves. I can
choose to cherry pick "fixes" from the main branch and back-port
them into the version I've adopted. Or, decide to live with
some particular set of problems/bugs/shortcomings.

If I am prudent, I will attempt to adopt the "style" of the
original developers in fitting any changes that I make to
that codebase. I'd want my changes to "blend in" and seem
consistent with that which preceded them.

Folks following the main distribution would likely NOT be interested
in the changes that I choose to embrace as they'll likely have
different goals than I. But that doesn't say my ownership is
"toxic", just that it doesn't suit the needs of (most) others.

---

I've got to bow out of this conversation. I made a commitment to
release 6 designs to manufacturing before year end. As it stands,
now, it looks like I'll only have time enough for four of them as
I got "distracted", spending the past few weeks gallavanting (but
it was wicked fun!).

OTOH, It won't be fun starting the new year two weeks "behind"... :<

[Damn holidays eat into my work time. And, no excuse on my part;
it's not like I didn't KNOW they were coming!! :< ]