On 10/31/2021 3:54 PM, antispam@math.uni.wroc.pl wrote:
>> Aren't THEY components?
>
> Well, some folks expect more from components than from
> traditional libraries. Some evan claim to deliver.
> However, libraries have limitations and ATM I see nothing
> that fundamentally change situation.

A component is something that you can use as a black box,
without having to reinvent it. It is the epitome of reuse.

>> How often do you reimplement
>> ?printf() to avoid all of the bloat that typically accompanies it?
>
> I did that once (for OS kernel where standard library would not
> work). If needed I can reuse it. On PC-s I am not worried by
> bloat due to printf. OTOH, on MCU-s I am not sure if I ever used
> printf. Rather, printing was done by specialized routines
> either library provided or my own.

You can also create a ?printf() that you can configure at build time to
support the modifiers and specifiers that you know you will need.

Just like you can configure a UART driver to support a FIFO size defined
at configuration, hardware handshaking, software flowcontrol, the
high and low water marks for each of those (as they can be different),
the character to send to request the remote to stop transmitting,
the character you send to request resumption of transmission, which
character YOU will recognize as requesting your Tx channel to pause,
the character (or condition) you will recognize to resume your Tx,
whether or not you will sample the condition codes in the UART, how
you read/write the data register, how you read/write the status register,
etc.

While these sound like lots of options, they are all relatively
trivial additions to the code.

>> (when was the last time you needed ALL of those format specifiers
>> in an application? And modifiers?
>>
>>> There are some things which are reusable
>>> if you accept modest to severe overhead.
>>
>> What you need is components with varying characteristics.
>> You can buy diodes with all sorts of current carrying capacities,
>> PIVs, package styles, etc. But, they all still perform the
>> same function. Why so many different part numbers? Why not
>> just use the biggest, baddest diode in ALL your circuits?
>
> I heard such electronic analogies many times. But they miss
> important point: there is no way for me to make my own diode,

Sure there is! It is just not an efficient way of spending your
resources when you have so many OTS offerings available.

You can design your own processor. Why do you "settle" for an
OTS device (ANS: because there is so little extra added value
you will typically gain from rolling your own vs. the "inefficiency"
of using a COTS offering)

> I am stuck with what is available on the market. And diode
> is logically pretty simple component, yet we need many kinds.
>
>> I.e., we readily accept differences in "standard components"
>> in other disciplines; why not when it comes to software
>> modules?
>
> Well, software is _much_ more compilcated than physical
> engineering artifacts. Physical thing may have 10000 joints,
> but if joints are identical, then this is moral equivalent of
> simple loop that just iterates fixed number of times.

This is the argument in favor of components. You'd much rather
read a comprehensive specification ("datasheet") for a software
component than have to read through all of the code that implements
it. What if it was implemented in some programming language in
which you aren't expert? What if it was a binary "BLOB" and
couldn't be inspected?

> At software level number of possible pre-composed blocks
> is so large that it is infeasible to deliver all of them.

You don't have to deliver all of them. When you wire a circuit,
you still have to *solder* connections, don't you? The
components don't magically glue themselves together...

> Classic trick it to parametrize. However even if you
> parametrize there are hundreds of design decisions going
> into relatively small piece of code. If you expose all
> design decisions then user as well may write his/her own
> code because complexity will be similar. So normaly
> parametrization is limited and there will be users who
> find hardcoded desion choices inadequate.
>
> Another things is that current tools are rather weak
> at supporting parametrization.

Look at a fleshy UART driver and think about how you would decompose
it into N different variants that could be "compile time configurable".
You'll be surprised as to how easy it is. Even if the actual UART
hardware differs from instance to instance.

>>> For example things tends
>>> to compose nicely if you dynamically allocate everything and use
>>> garbage collection. But performace cost may be substantial.
>>> And in embedded setting garbage collection may be unacceptable.
>>> In some cases I have found out that I can get much better
>>> speed joing things that could be done as composition of library
>>> operations into single big routine.
>>
>> Sure, but now you're tuning a solution to a specific problem.
>> I've designed custom chips to solve particular problems.
>> But, they ONLY solve those particular problems! OTOH,
>> I use lots of OTC components in my designs because those have
>> been designed (for the most part) with an eye towards
>> meeting a variety of market needs.
>
> Maybe I made wrong impression, I think some explanation is in
> place here. I am trying to make my code reusable. For my
> problems performance is important part of reusablity: our
> capability to solve problem is limited by performance and with
> better perfomance users can solve bigger problems. I am
> re-using code that I can and I would re-use more if I could
> but there there are technical obstacles. Also, while I am
> trying to make my code reusable, there are intrusive
> design decision which may interfere with your possiobility
> and willingness to re-use.

If you don't know where the design is headed, then you can't
pick the components that it will need.

I approach a design from the top (down) and bottom (up). This
lets me gauge the types of information that I *may* have
available from the hardware -- so I can sort out how to
approach those limitations from above. E.g., if I can't
control the data rate of a comm channel, then I either have
to ensure I can catch every (complete) message *or* design a
protocol that lets me detect when I've missed something.

There are costs to both approaches. If I dedicate resource to
ensuring I don't miss anything, then some other aspect of the
design will bear that cost. If I rely on detecting missed
messages, then I have to put a figure on their relative
likelihood so my device doesn't fail to provide its desired
functionality (because it is always missing one or two characters
out of EVERY message -- and, thus, sees NO messages).

> In slightly different spirit: in another thread you wrote
> about accessing disc without OS file cache. Here I
> normaly depend on OS and OS file caching is big thing.
> It is not perfect, but OS (OK, at least Linux) is doing
> this resonably well I have no temptation to avoid it.
> And I appreciate that with OS cache performance is
> usually much better that would be "without cache".
> OTOH, I routinly avoid stdio for I/O critical things
> (so no printf in I/O critical code).

My point about the cache was that it is of no value in my case;
I'm not going to revisit a file once I've seen it the first
time (so why hold onto that data?)

>>> In other cases I fixed
>>> bugs by replacing composition of library routines by a single
>>> routine: there were interactions making simple composition
>>> incorrect. Correct alterantive was single routine.
>>>
>>> As I wrote my embedded programs are simple and small. But I
>>> use almost no external libraries. Trying some existing libraries
>>> I have found out that some produce rather large programs, linking
>>> in a lot of unneeded stuff.
>>
>> Because they try to address a variety of solution spaces without
>> trying to be "optimal" for any. You trade flexibility/capability
>> for speed/performance/etc.
>
> I think that this is more subtle: libraries frequently force some
> way of doing things. Which may be good if you try to quickly roll
> solution and are within capabilities of library. But if you
> need/want different design, then library may be too inflexible
> to deliver it.

Use a different diode.

>>> Of course, writing for scratch
>>> will not scale to bigger programs. OTOH, I feel that with
>>> proper tooling it would be possible to retain efficiency and
>>> small code size at least for large class of microntroller
>>> programs (but existing tools and libraries do not support this).
>>
>> Templates are an attempt in this direction. Allowing a class of
>> problems to be solved once and then tailored to the specific
>> application.
>
> Yes, templates could help. But they also have problems. One
> of them is that (among others) I would like to target STM8
> and I have no C++ compiler for STM8. My idea is to create
> custom "optimizer/generator" for (annotated) C code.
> ATM it is vapourware, but I think it is feasible with
> reasonable effort.
>
>> But, personal experience is where you win the most. You write
>> your second or third UART driver and start realizing that you
>> could leverage a previous design if you'd just thought it out
>> more fully -- instead of tailoring it to the specific needs
>> of the original application.
>>
>> And, as you EXPECT to be reusing it in other applications (as
>> evidenced by the fact that it's your third time writing the same
>> piece of code!), you anticipate what those *might* need and
>> think about how to implement those features "economically".
>>
>> It's rare that an application is *so* constrained that it can't
>> afford a couple of extra lines of code, here and there. If
>> you've considered efficiency in the design of your algorithms,
>> then these little bits of inefficiency will be below the noise floor.
>
> Well, I am not talking about "couple of extra lines". Rather
> about IMO substantial fixed overhead. As I wrote, one of my
> targets is STM8 with 8k flash, another is MSP430 with 16k flash,
> another is STM32 with 16k flash (there are also bigger targets).
> One of libraries/frameworks for STM32 after activating few featurs
> pulled in about 16k code, this is substantial overhead given
> how little features I needed. Other folks reported that for
> trivial programs vendor supplied frameworks pulled close to 30k

A "framework" is considerably more than a set of individually
selectable components. I've designed products with 2KB of code and
128 bytes of RAM. The "components" were ASM modules instead of
HLL modules. Each told me how big it was, how much RAM it required,
how deep the stack penetration when invoked, how many T-states
(worst case) to execute, etc.

So, before I designed the hardware, I knew what I would need
by way of ROM/RAM (before the days of FLASH) and could commit
the hardware to foil without fear of running out of "space" or
"time".

> code. That may be fine if you have bigger device and need features,
> but for smaller MCU-s it may be difference between not fitting into
> device or (without library) having plenty of free space.

Sure. But a component will have a datasheet that tells you what
it provides and at what *cost*.

> When I tried it Free RTOS for STM32 needed about 8k flash. Which
> is fine if you need RTOS. But ATM my designs run without RTOS.

RTOS is a commonly misused term. Many are more properly called
MTOSs (they provide no real timeliness guarantees, just multitasking
primitives).

IMO, the advantages of writing in a multitasking environment so
far outweigh the "costs" of an MTOS that it behooves one to consider
how to shoehorn that functionality into EVERY design.

When writing in a HLL, there are complications that impose
constraints on how the MTOS provides its services. But, for small
projects written in ASM, you can gain the benefits of an MTOS
for very few bytes of code (and effectively zero RAM).

> I have found libopencm3 to have small overhead. But is routines
> are doing so little that direct register access may give simpler
> code.
>
>>>> Unlikely that this code will describe itself as "works well enough
>>>> SOME of the time..."
>>>>
>>>> And, when/if you stumble on such faults, good luck explaining to
>>>> your customer why it's going to take longer to fix and retest the
>>>> *existing* codebase before you can get on with your modifications...
>>>
>>> Commercial vendors like to say how good their progam are. But
>>> market reality is that program my be quite bad and still sell.
>>
>> The same is true of FOSS -- despite the claim that many eyes (may)
>> have looked at it (suggesting that bugs would have been caught!)
>>
>> From "KLEE: Unassisted and Automatic Generation of High-Coverage
>> Tests for Complex Systems Programs":
>>
>> KLEE finds important errors in heavily-tested code. It
>> found ten fatal errors in COREUTILS (including three
>> that had escaped detection for 15 years), which account
>> for more crashing bugs than were reported in 2006, 2007
>> and 2008 combined. It further found 24 bugs in BUSYBOX, 21
>> bugs in MINIX, and a security vulnerability in HISTAR? a
>> total of 56 serious bugs.
>>
>> Ooops! I wonder how many FOSS *eyes* missed those errors?
>
> Open source folks tend to be more willing to talk about bugs.
> And the above nicely shows that there is a lot of bugs, most
> waiting to by discovered.

Part of the problem is ownership of the codebase. You are
more likely to know where your own bugs lie -- and, more
willing to fix them ("pride of ownership"). When a piece
of code is shared, over time, there seems to be less incentive
for folks to tackle big -- often dubious -- issues as the
"reward" is minimal (i.e., you may not own the code when the bug
eventually becomes a problem)

>> Every time you reinvent a solution, you lose much of the benefit
>> of the previous TESTED solution.
>
> TESTED part works for simple repeatable tasks. But if you have
> complex task it is quite likely that you will be the first
> person with given use case. gcc is borderline case: if you
> throw really new code at it you can expect to see bugs.
> gcc user community it large and there is resonable chance that
> sombody wrote earlier code which is sufficiently similar to
> yours to catch troubles. But there are domains that are at
> least as complicated as compilation and have much smaller
> user community. You may find out that there are _no_ code
> that could be reasonably re-used. Were you ever in situation
> when you looked how some "standard library" solves a tricky
> problem and realized that in fact library does not solve
> the problem?

As I said, your *personal* experience tells you where YOU will
likely benefit. I did a stint with a company that manufactured
telecommunications kit. We had all sorts of bizarre interface
protocols with which we had to contend (e.g., using RLSD as
a hardware "pacing" signal). So, it was worthwhile to spend
time developing a robust UART driver (and handler, above it)
as you *knew* the next project would likely have need of it,
in some form or other.

If you're working free-lance and client A needs a BITBLTer
for his design, you have to decide how likely client B
(that you haven't yet met) will be to need the same sort
of module/component.

For example, I've never (until recently) needed to interface
to a disk controller in a product. So, I don't have a
ready-made "component" in my bag-of-tricks. When I look
at a new project, I "take inventory" of what I am likely to
need... and compare that to what I know I have "in stock".
If there's a lot of overlap, then my confidence in my bid
goes up. If there'a a lot of new ground that I'll have to
cover, then it goes down (and the price goes up!).

Reuse helps you better estimate new projects, especially as
projects grow in complexity.

[There's nothing worse than having to upgrade someone else's
design that didn't plan for the future. It's as if you
have to redesign the entire product from scratch --- despite
the fact that it *seems* to work, "as is" (but, not "as desired"!]