Rocksolid Light

Welcome to novaBBS (click a section below)

mail  files  register  newsreader  groups  login

Message-ID:  

You had mail. Paul read it, so ask him what it said.


devel / comp.arch.embedded / Re: Embedding a Checksum in an Image File

SubjectAuthor
* Embedding a Checksum in an Image FileRick C
+* Re: Embedding a Checksum in an Image FileNiklas Holsti
|`- Re: Embedding a Checksum in an Image FileRick C
+- Re: Embedding a Checksum in an Image FilePeter Heitzer
+* Re: Embedding a Checksum in an Image Filedalai lamah
|`- Re: Embedding a Checksum in an Image FileRick C
+- Re: Embedding a Checksum in an Image FileDavid Brown
+* Re: Embedding a Checksum in an Image FileGeorge Neuner
|+* Re: Embedding a Checksum in an Image FileRick C
||+* Re: Embedding a Checksum in an Image FileDavid Brown
|||`* Re: Embedding a Checksum in an Image FileUlf Samuelsson
||| `- Re: Embedding a Checksum in an Image FileUlf Samuelsson
||`* Re: Embedding a Checksum in an Image FileGeorge Neuner
|| +* Re: Embedding a Checksum in an Image FileDon Y
|| |`* Re: Embedding a Checksum in an Image FileDavid Brown
|| | `* Re: Embedding a Checksum in an Image FileDon Y
|| |  `* Re: Embedding a Checksum in an Image FileDavid Brown
|| |   `* Re: Embedding a Checksum in an Image FileDon Y
|| |    `* Re: Embedding a Checksum in an Image FileDavid Brown
|| |     `* Re: Embedding a Checksum in an Image FileDon Y
|| |      `* Re: Embedding a Checksum in an Image FileDavid Brown
|| |       `* Re: Embedding a Checksum in an Image FileDon Y
|| |        `* Re: Embedding a Checksum in an Image FileDavid Brown
|| |         +* Re: Embedding a Checksum in an Image FileUlf Samuelsson
|| |         |`* Re: Embedding a Checksum in an Image FileDavid Brown
|| |         | `* Re: Embedding a Checksum in an Image FileUlf Samuelsson
|| |         |  `- Re: Embedding a Checksum in an Image FileDon Y
|| |         `- Re: Embedding a Checksum in an Image FileDon Y
|| `- Re: Embedding a Checksum in an Image FileStefan Reuther
|`* Re: Embedding a Checksum in an Image FileTauno Voipio
| `- Re: Embedding a Checksum in an Image FileGeorge Neuner
+* Re: Embedding a Checksum in an Image FileRichard Damon
|`* Re: Embedding a Checksum in an Image FileRick C
| `- Re: Embedding a Checksum in an Image FileRichard Damon
+* Re: Embedding a Checksum in an Image FileBrian Cockburn
|`* Re: Embedding a Checksum in an Image FileRick C
| +* Re: Embedding a Checksum in an Image FileDavid Brown
| |+* Re: Embedding a Checksum in an Image FileBrian Cockburn
| ||`- Re: Embedding a Checksum in an Image FileDavid Brown
| |`* Re: Embedding a Checksum in an Image FileRick C
| | +* Re: Embedding a Checksum in an Image FileDavid Brown
| | |`* Re: Embedding a Checksum in an Image FileRick C
| | | `* Re: Embedding a Checksum in an Image FileDavid Brown
| | |  +* Re: Embedding a Checksum in an Image FileGrant Edwards
| | |  |`* Re: Embedding a Checksum in an Image FileDavid Brown
| | |  | `* Re: Embedding a Checksum in an Image FileGrant Edwards
| | |  |  `* Re: Embedding a Checksum in an Image FileDavid Brown
| | |  |   `* Re: Embedding a Checksum in an Image FileRichard Damon
| | |  |    `- Re: Embedding a Checksum in an Image FileDavid Brown
| | |  +- Re: Embedding a Checksum in an Image FileboB
| | |  `* Re: Embedding a Checksum in an Image FileRick C
| | |   `* Re: Embedding a Checksum in an Image FileDavid Brown
| | |    `* Re: Embedding a Checksum in an Image FileRick C
| | |     `* Re: Embedding a Checksum in an Image FileDavid Brown
| | |      `- Re: Embedding a Checksum in an Image FileRick C
| | `* Re: Embedding a Checksum in an Image FileUlf Samuelsson
| |  `- Re: Embedding a Checksum in an Image FileUlf Samuelsson
| `* Re: Embedding a Checksum in an Image FileBrian Cockburn
|  `* Re: Embedding a Checksum in an Image FileRick C
|   `* Re: Embedding a Checksum in an Image FileBrian Cockburn
|    +- Re: Embedding a Checksum in an Image FileRichard Damon
|    `- Re: Embedding a Checksum in an Image FileRick C
+* Re: Embedding a Checksum in an Image FileUlf Samuelsson
|+* Re: Embedding a Checksum in an Image FileRick C
||+* Re: Embedding a Checksum in an Image FileNiklas Holsti
|||+- Re: Embedding a Checksum in an Image FileNiklas Holsti
|||`* Re: Embedding a Checksum in an Image FileUlf Samuelsson
||| `* Re: Embedding a Checksum in an Image FileUlf Samuelsson
|||  `* Re: Embedding a Checksum in an Image FileDavid Brown
|||   `* Re: Embedding a Checksum in an Image FileUlf Samuelsson
|||    `* Re: Embedding a Checksum in an Image FileDavid Brown
|||     `* Re: Embedding a Checksum in an Image FileUlf Samuelsson
|||      `- Re: Embedding a Checksum in an Image FileDavid Brown
||`* Re: Embedding a Checksum in an Image FileUlf Samuelsson
|| `* Re: Embedding a Checksum in an Image FileNiklas Holsti
||  `- Re: Embedding a Checksum in an Image FileUlf Samuelsson
|`* Re: Embedding a Checksum in an Image FileUlf Samuelsson
| `* Re: Embedding a Checksum in an Image FileDavid Brown
|  `- Re: Embedding a Checksum in an Image FileUlf Samuelsson
`- Re: Embedding a Checksum in an Image FileUlf Samuelsson

Pages:1234
Re: Embedding a Checksum in an Image File

<99aaf8df-1f0e-4911-9706-0bac770e3d1cn@googlegroups.com>

 copy mid

https://www.novabbs.com/devel/article-flat.php?id=1487&group=comp.arch.embedded#1487

 copy link   Newsgroups: comp.arch.embedded
X-Received: by 2002:ae9:e40c:0:b0:74e:1090:8906 with SMTP id q12-20020ae9e40c000000b0074e10908906mr2296820qkc.0.1682271241337;
Sun, 23 Apr 2023 10:34:01 -0700 (PDT)
X-Received: by 2002:a05:622a:1793:b0:3e3:8172:ff21 with SMTP id
s19-20020a05622a179300b003e38172ff21mr3889351qtk.8.1682271241051; Sun, 23 Apr
2023 10:34:01 -0700 (PDT)
Path: i2pn2.org!i2pn.org!weretis.net!feeder8.news.weretis.net!proxad.net!feeder1-2.proxad.net!209.85.160.216.MISMATCH!news-out.google.com!nntp.google.com!postnews.google.com!google-groups.googlegroups.com!not-for-mail
Newsgroups: comp.arch.embedded
Date: Sun, 23 Apr 2023 10:34:00 -0700 (PDT)
In-Reply-To: <u2171f$3cbcu$1@dont-email.me>
Injection-Info: google-groups.googlegroups.com; posting-host=63.114.57.174; posting-account=I-_H_woAAAA9zzro6crtEpUAyIvzd19b
NNTP-Posting-Host: 63.114.57.174
References: <116ff07e-5e25-469a-90a0-9474108aadd3n@googlegroups.com>
<66e9fb2a-b9e6-4597-9afa-6572caaa8ca1n@googlegroups.com> <a4495c87-c68d-4c87-94aa-701c21bdd19cn@googlegroups.com>
<u1u8hu$2ps79$1@dont-email.me> <ef5ad4e6-57ed-4950-baed-3a6746b9d16en@googlegroups.com>
<u20tin$3alrj$1@dont-email.me> <3986aac8-aec3-4cde-8bb8-8a61c20084c9n@googlegroups.com>
<u2171f$3cbcu$1@dont-email.me>
User-Agent: G2/1.0
MIME-Version: 1.0
Message-ID: <99aaf8df-1f0e-4911-9706-0bac770e3d1cn@googlegroups.com>
Subject: Re: Embedding a Checksum in an Image File
From: gnuarm.d...@gmail.com (Rick C)
Injection-Date: Sun, 23 Apr 2023 17:34:01 +0000
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable
 by: Rick C - Sun, 23 Apr 2023 17:34 UTC

On Saturday, April 22, 2023 at 1:55:01 PM UTC-4, David Brown wrote:
> On 22/04/2023 18:56, Rick C wrote:
> > On Saturday, April 22, 2023 at 11:13:32 AM UTC-4, David Brown wrote:
> >> On 22/04/2023 05:14, Rick C wrote:
> >>> On Friday, April 21, 2023 at 11:02:28 AM UTC-4, David Brown wrote:
> >>>> On 21/04/2023 14:12, Rick C wrote:
> >>>>>
> >>>>> This is simply to be able to say this version is unique,
> >>>>> regardless of what the version number says. Version numbers are
> >>>>> set manually and not always done correctly. I'm looking for
> >>>>> something as a backup so that if the checksums are different, I
> >>>>> can be sure the versions are not the same.
> >>>>>
> >>>>> The less work involved, the better.
> >>>>>
> >>>> Run a simple 32-bit crc over the image. The result is a hash of
> >>>> the image. Any change in the image will show up as a change in the
> >>>> crc.
> >>>
> >>> No one is trying to detect changes in the image. I'm trying to label
> >>> the image in a way that can be read in operation. I'm using the
> >>> checksum simply because that is easy to generate. I've had problems
> >>> with version numbering in the past. It will be used, but I want it
> >>> supplemented with a number that will change every time the design
> >>> changes, at least with a high probability, such as 1 in 64k.
> >>>
> >> Again - use a CRC. It will give you what you want.
> >
> > Again - as will a simple addition checksum.
> A simple addition checksum might be okay much of the time, but it
> doesn't have the resolving power of a CRC. If the source code changes
> "a = 1; b = 2;" to "a = 2; b = 1;", the addition checksum is likely to
> be exactly the same despite the change in the source. In general, you
> will have much higher chance of collisions, though I think it would be
> very hard to quantify that.
>
> Maybe it will be good enough for you. Simple checksums were popular
> once, and can still make sense if you are very short on program space.
> But there are good reasons why they fell out of favour in many uses.
> >
> >
> >> You might want to go for 32-bit CRC rather than a 16-bit CRC, depending
> >> on the kind of program, how often you build it, and what consequences a
> >> hash collision could have. With a 16-bit CRC, you have a 5% chance of a
> >> collision after 82 builds. If collisions only matter for releases, and
> >> you only release a couple of updates, fine - but if they matter during
> >> development builds, you are getting a more significant risk. Since a
> >> 32-bit CRC is quick and easy, it's worth using.
> >
> > Or, I might want to go with a simple checksum.
> >
> > Thanks for your comments.
> >
> It's your choice (obviously). I only point out the weaknesses in case
> anyone else is listening in to the thread.
>
> If you like, I can post code for a 32-bit CRC. It's a table, and a few
> lines of C code.

You know nothing of the project I am working on or those that I typically work on. But thanks for the advice.

--

Rick C.

+-- Get 1,000 miles of free Supercharging
+-- Tesla referral code - https://ts.la/richard11209

Re: Embedding a Checksum in an Image File

<105b1cab-9e5f-91b3-1de6-99586246ecbd@gmail.com>

 copy mid

https://www.novabbs.com/devel/article-flat.php?id=1503&group=comp.arch.embedded#1503

 copy link   Newsgroups: comp.arch.embedded
Path: i2pn2.org!i2pn.org!eternal-september.org!news.eternal-september.org!.POSTED!not-for-mail
From: ulf.r.sa...@gmail.com (Ulf Samuelsson)
Newsgroups: comp.arch.embedded
Subject: Re: Embedding a Checksum in an Image File
Date: Thu, 27 Apr 2023 18:26:37 +0200
Organization: eMagii
Lines: 36
Message-ID: <105b1cab-9e5f-91b3-1de6-99586246ecbd@gmail.com>
References: <116ff07e-5e25-469a-90a0-9474108aadd3n@googlegroups.com>
Reply-To: ulf.r.samuelsson@gmail.com
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 7bit
Injection-Info: dont-email.me; posting-host="e06ed78c1495380a4807241f34669d85";
logging-data="2102917"; mail-complaints-to="abuse@eternal-september.org"; posting-account="U2FsdGVkX1+hOH7r/DwxPJ/+V9g8uNKr"
User-Agent: Mozilla/5.0 (Windows NT 6.1; Win64; x64; rv:102.0) Gecko/20100101
Thunderbird/102.8.0
Cancel-Lock: sha1:S8i0biu0Co06e1yQkf4sfVXUyx8=
Content-Language: sv
In-Reply-To: <116ff07e-5e25-469a-90a0-9474108aadd3n@googlegroups.com>
 by: Ulf Samuelsson - Thu, 27 Apr 2023 16:26 UTC

Den 2023-04-20 kl. 04:06, skrev Rick C:
> This is a bit of the chicken and egg thing. If you want a embed a checksum in a code module to report the checksum, is there a way of doing this? It's a bit like being your own grandfather, I think.
>
The proper way to do this is to have a directive in the linker.
This reserves space for the CRC and defines the area where the CRC is
calculated.
I am not aware of any linker which support this.

Two months ago, I added the DIGEST directive to binutils aka the GNU
linker. It was committed, but then people realized that I had not signed
an agreement with Free Software Foundation.
Since part of the code I pushed was from a third party which released
their code under MIT, the licensing has not been resolved yet
but the patch is in binutils git, but reverted.

You would write (IIRC):
DIGEST "CRC64-ECMA", (from, to)
and the linker would reserve 8 bytes which is filled with the CRC in the
final link stage.

/Ulf

> I'm not thinking anything too fancy, like a CRC, but rather a simple modulo N addition, maybe N being 2^16.
>
> I keep thinking of using a placeholder, but that doesn't seem to work out in any useful way. Even if you try to anticipate the impact of adding the checksum, that only gives you a different checksum, that you then need to anticipate further... ad infinitum.
>
> I'm not thinking of any special checksum generator that excludes the checksum data. That would be too messy.
>
> I keep thinking there is a different way of looking at this to achieve the result I want...
>
> Maybe I can prove it is impossible. Assume the file checksums to X when the checksum data is zero. The goal would then be to include the checksum data value Y in the file, that would change X to Y. Given the properties of the module N checksum, this would appear to be impossible for the general case, unless... Add another data value, called, checksum normalizer. This data value checksums with the original checksum to give the result zero. Then, when the checksum is also added, the resulting checksum is, in fact, the checksum. Another way of looking at this is to add a value that combines with the added checksum, to be zero, leaving the original checksum intact.
>
> This might be inordinately hard for a CRC, but a simple checksum would not be an issue, I think. At least, this could work in software, where data can be included in an image file as itself. In a device like an FPGA, it might not be included in the bit stream file so directly... but that might depend on where in the device it is inserted. Memory might have data that is stored as itself. I'll need to look into that.
>

Re: Embedding a Checksum in an Image File

<u2e7p3$205k5$2@dont-email.me>

 copy mid

https://www.novabbs.com/devel/article-flat.php?id=1504&group=comp.arch.embedded#1504

 copy link   Newsgroups: comp.arch.embedded
Path: i2pn2.org!i2pn.org!eternal-september.org!news.eternal-september.org!.POSTED!not-for-mail
From: ulf.r.sa...@gmail.com (Ulf Samuelsson)
Newsgroups: comp.arch.embedded
Subject: Re: Embedding a Checksum in an Image File
Date: Thu, 27 Apr 2023 18:27:11 +0200
Organization: eMagii
Lines: 36
Message-ID: <u2e7p3$205k5$2@dont-email.me>
References: <116ff07e-5e25-469a-90a0-9474108aadd3n@googlegroups.com>
Reply-To: ulf.r.samuelsson@gmail.com
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 7bit
Injection-Date: Thu, 27 Apr 2023 16:27:15 -0000 (UTC)
Injection-Info: dont-email.me; posting-host="e06ed78c1495380a4807241f34669d85";
logging-data="2102917"; mail-complaints-to="abuse@eternal-september.org"; posting-account="U2FsdGVkX19Qgg2DKm7UqNyJfkofPW/R"
User-Agent: Mozilla/5.0 (Windows NT 6.1; Win64; x64; rv:102.0) Gecko/20100101
Thunderbird/102.8.0
Cancel-Lock: sha1:cEXgUQaOtrYa5yrZx+83VSJgLFM=
In-Reply-To: <116ff07e-5e25-469a-90a0-9474108aadd3n@googlegroups.com>
Content-Language: sv
 by: Ulf Samuelsson - Thu, 27 Apr 2023 16:27 UTC

Den 2023-04-20 kl. 04:06, skrev Rick C:
> This is a bit of the chicken and egg thing. If you want a embed a checksum in a code module to report the checksum, is there a way of doing this? It's a bit like being your own grandfather, I think.
>
The proper way to do this is to have a directive in the linker.
This reserves space for the CRC and defines the area where the CRC is
calculated.
I am not aware of any linker which support this.

Two months ago, I added the DIGEST directive to binutils aka the GNU
linker. It was committed, but then people realized that I had not signed
an agreement with Free Software Foundation.
Since part of the code I pushed was from a third party which released
their code under MIT, the licensing has not been resolved yet
but the patch is in binutils git, but reverted.

You would write (IIRC):
DIGEST "CRC64-ECMA", (from, to)
and the linker would reserve 8 bytes which is filled with the CRC in the
final link stage.

/Ulf

> I'm not thinking anything too fancy, like a CRC, but rather a simple modulo N addition, maybe N being 2^16.
>
> I keep thinking of using a placeholder, but that doesn't seem to work out in any useful way. Even if you try to anticipate the impact of adding the checksum, that only gives you a different checksum, that you then need to anticipate further... ad infinitum.
>
> I'm not thinking of any special checksum generator that excludes the checksum data. That would be too messy.
>
> I keep thinking there is a different way of looking at this to achieve the result I want...
>
> Maybe I can prove it is impossible. Assume the file checksums to X when the checksum data is zero. The goal would then be to include the checksum data value Y in the file, that would change X to Y. Given the properties of the module N checksum, this would appear to be impossible for the general case, unless... Add another data value, called, checksum normalizer. This data value checksums with the original checksum to give the result zero. Then, when the checksum is also added, the resulting checksum is, in fact, the checksum. Another way of looking at this is to add a value that combines with the added checksum, to be zero, leaving the original checksum intact.
>
> This might be inordinately hard for a CRC, but a simple checksum would not be an issue, I think. At least, this could work in software, where data can be included in an image file as itself. In a device like an FPGA, it might not be included in the bit stream file so directly... but that might depend on where in the device it is inserted. Memory might have data that is stored as itself. I'll need to look into that.
>

Re: Embedding a Checksum in an Image File

<u2e8ba$205k5$3@dont-email.me>

 copy mid

https://www.novabbs.com/devel/article-flat.php?id=1505&group=comp.arch.embedded#1505

 copy link   Newsgroups: comp.arch.embedded
Path: i2pn2.org!i2pn.org!eternal-september.org!news.eternal-september.org!.POSTED!not-for-mail
From: ulf.r.sa...@gmail.com (Ulf Samuelsson)
Newsgroups: comp.arch.embedded
Subject: Re: Embedding a Checksum in an Image File
Date: Thu, 27 Apr 2023 18:36:55 +0200
Organization: eMagii
Lines: 88
Message-ID: <u2e8ba$205k5$3@dont-email.me>
References: <116ff07e-5e25-469a-90a0-9474108aadd3n@googlegroups.com>
<pvl24i57aef4vc7bbdk9mvj7sic9dsh64t@4ax.com>
<f0afa198-e735-4da1-a16a-82764af3de4dn@googlegroups.com>
<u1s76b$o11h$1@dont-email.me>
Reply-To: ulf.r.samuelsson@gmail.com
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 8bit
Injection-Date: Thu, 27 Apr 2023 16:36:59 -0000 (UTC)
Injection-Info: dont-email.me; posting-host="e06ed78c1495380a4807241f34669d85";
logging-data="2102917"; mail-complaints-to="abuse@eternal-september.org"; posting-account="U2FsdGVkX1/FJFHaoXs4r2srjC8L7jnZ"
User-Agent: Mozilla/5.0 (Windows NT 6.1; Win64; x64; rv:102.0) Gecko/20100101
Thunderbird/102.8.0
Cancel-Lock: sha1:+Gw4Q69iZ+bJy9jSbtnXiVUoVC8=
Content-Language: sv
In-Reply-To: <u1s76b$o11h$1@dont-email.me>
 by: Ulf Samuelsson - Thu, 27 Apr 2023 16:36 UTC

Den 2023-04-20 kl. 22:26, skrev David Brown:
> On 20/04/2023 18:45, Rick C wrote:
>> On Thursday, April 20, 2023 at 11:33:28 AM UTC-4, George Neuner
>> wrote:
>>> On Wed, 19 Apr 2023 19:06:33 -0700 (PDT), Rick C
>>> <gnuarm.del...@gmail.com> wrote:
>>>
>>>> This is a bit of the chicken and egg thing. If you want a embed
>>>> a checksum in a code module to report the checksum, is there a
>>>> way of doing this? It's a bit like being your own grandfather, I
>>>> think.
>>> Take a look at the old xmodem/ymodem CRC. It was designed such
>>> that when the CRC was sent immediately following the data, a
>>> receiver computing CRC over the whole incoming packet (data and CRC
>>> both) would get a result of zero.
>>>
>>> But AFAIK it doesn't work with CCITT equation(s) - you have to use
>>> xmodem/ymodem.
>>>> I'm not thinking anything too fancy, like a CRC, but rather a
>>>> simple modulo N addition, maybe N being 2^16.
>>> Sorry, I don't know a way to do it with a modular checksum. YMMV,
>>> but I think 16-bit CRC is pretty simple.
>>>
>>> George
>>
>> CRC is not complicated, but I would not know how to calculate an
>> inserted value to force the resulting CRC to zero.  How do you do
>> that?
>
> You "insert" the value at the end.  Anything else is insane.

In all projects I have been involved with, the application binary starts
with a header looking like this.

MAGIC WORD 1
CRC
Entry Point
Size
other info...
MAGIC WORD 2
APPLICATION_START
....
APPLICATION_END (aligned with flash sector)

The bootloader first checks the two magic words.
It then computes CRC on the header (from Entry Point) to APPLICATION_END

I ported the IAR ielftool (open source) to Linux at
https://github.com/emagii/ielftool

This can insert the CRC in the ELF file, but needs tweaks to work
with an ELF file generated by the GNU tools.

/Ulf

>
> CRC's are quite good hashes, for suitable sized data.  There are perhaps
> some special cases, but basically you'd be doing trial-and-error
> searches to find an inserted value that gives you a zero CRC overall.
> 2^16 is not an overwhelming search space, but the whole idea is pointless.
>
>>
>> Even so, I'm not trying to validate the file.  I'm trying to come up
>> with a substitute for a time stamp or version number.  I don't want
>> to have to rely on my consistency in handling the version number
>> correctly.  This would be a backup in case there was more than one
>> version released, even only within the "lab", that were different.  A
>> checksum that could be read by the controlling software would do the
>> job.
>
> A CRC is fine for that.
>
>>
>> I have run into this before, where the version number was not a 100%
>> indication of the uniqueness of an executable.  The checksum would be
>> a second indicator.
>>
>> I should mention that I'm not looking for a solution that relies on
>> any specific details of the tools.
>>
>
> A table-based CRC is easy, runs quickly, and can be quickly ported to
> pretty much any language (the C and Python code, for example, is almost
> the same).

Re: Embedding a Checksum in an Image File

<u2e8lu$205k5$4@dont-email.me>

 copy mid

https://www.novabbs.com/devel/article-flat.php?id=1506&group=comp.arch.embedded#1506

 copy link   Newsgroups: comp.arch.embedded
Path: i2pn2.org!i2pn.org!eternal-september.org!news.eternal-september.org!.POSTED!not-for-mail
From: ulf.r.sa...@gmail.com (Ulf Samuelsson)
Newsgroups: comp.arch.embedded
Subject: Re: Embedding a Checksum in an Image File
Date: Thu, 27 Apr 2023 18:42:34 +0200
Organization: eMagii
Lines: 24
Message-ID: <u2e8lu$205k5$4@dont-email.me>
References: <116ff07e-5e25-469a-90a0-9474108aadd3n@googlegroups.com>
<66e9fb2a-b9e6-4597-9afa-6572caaa8ca1n@googlegroups.com>
<a4495c87-c68d-4c87-94aa-701c21bdd19cn@googlegroups.com>
<u1u8hu$2ps79$1@dont-email.me>
<ef5ad4e6-57ed-4950-baed-3a6746b9d16en@googlegroups.com>
Reply-To: ulf.r.samuelsson@gmail.com
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 8bit
Injection-Date: Thu, 27 Apr 2023 16:42:38 -0000 (UTC)
Injection-Info: dont-email.me; posting-host="e06ed78c1495380a4807241f34669d85";
logging-data="2102917"; mail-complaints-to="abuse@eternal-september.org"; posting-account="U2FsdGVkX1/wLpFi/2JtwbACh6JXQ0+Q"
User-Agent: Mozilla/5.0 (Windows NT 6.1; Win64; x64; rv:102.0) Gecko/20100101
Thunderbird/102.8.0
Cancel-Lock: sha1:+UMGBEB2Y/mhEmmHerH4KLbICV0=
In-Reply-To: <ef5ad4e6-57ed-4950-baed-3a6746b9d16en@googlegroups.com>
Content-Language: sv
 by: Ulf Samuelsson - Thu, 27 Apr 2023 16:42 UTC

Den 2023-04-22 kl. 05:14, skrev Rick C:
> On Friday, April 21, 2023 at 11:02:28 AM UTC-4, David Brown wrote:
>> On 21/04/2023 14:12, Rick C wrote:
>>>
>>> This is simply to be able to say this version is unique, regardless
>>> of what the version number says. Version numbers are set manually
>>> and not always done correctly. I'm looking for something as a backup
>>> so that if the checksums are different, I can be sure the versions
>>> are not the same.
>>>
>>> The less work involved, the better.
>>>
>> Run a simple 32-bit crc over the image. The result is a hash of the
>> image. Any change in the image will show up as a change in the crc.
>
> No one is trying to detect changes in the image. I'm trying to label the image in a way that can be read in operation. I'm using the checksum simply because that is easy to generate. I've had problems with version numbering in the past. It will be used, but I want it supplemented with a number that will change every time the design changes, at least with a high probability, such as 1 in 64k.
>

Another thing I added (and was later removed) was a timestamp directive.
A 64 bit integer with the number of seconds since 1970-01-01 00:00.

/Ulf

Re: Embedding a Checksum in an Image File

<b62a77f7-3a13-4d76-b7cb-c42902595751n@googlegroups.com>

 copy mid

https://www.novabbs.com/devel/article-flat.php?id=1507&group=comp.arch.embedded#1507

 copy link   Newsgroups: comp.arch.embedded
X-Received: by 2002:ac8:7d13:0:b0:3ef:3204:5158 with SMTP id g19-20020ac87d13000000b003ef32045158mr814703qtb.7.1682615367544;
Thu, 27 Apr 2023 10:09:27 -0700 (PDT)
X-Received: by 2002:a05:622a:15c1:b0:3ef:499a:dd95 with SMTP id
d1-20020a05622a15c100b003ef499add95mr739328qty.7.1682615367241; Thu, 27 Apr
2023 10:09:27 -0700 (PDT)
Path: i2pn2.org!i2pn.org!weretis.net!feeder6.news.weretis.net!news.misty.com!border-2.nntp.ord.giganews.com!nntp.giganews.com!news-out.google.com!nntp.google.com!postnews.google.com!google-groups.googlegroups.com!not-for-mail
Newsgroups: comp.arch.embedded
Date: Thu, 27 Apr 2023 10:09:27 -0700 (PDT)
In-Reply-To: <105b1cab-9e5f-91b3-1de6-99586246ecbd@gmail.com>
Injection-Info: google-groups.googlegroups.com; posting-host=65.207.89.54; posting-account=I-_H_woAAAA9zzro6crtEpUAyIvzd19b
NNTP-Posting-Host: 65.207.89.54
References: <116ff07e-5e25-469a-90a0-9474108aadd3n@googlegroups.com> <105b1cab-9e5f-91b3-1de6-99586246ecbd@gmail.com>
User-Agent: G2/1.0
MIME-Version: 1.0
Message-ID: <b62a77f7-3a13-4d76-b7cb-c42902595751n@googlegroups.com>
Subject: Re: Embedding a Checksum in an Image File
From: gnuarm.d...@gmail.com (Rick C)
Injection-Date: Thu, 27 Apr 2023 17:09:27 +0000
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable
Lines: 41
 by: Rick C - Thu, 27 Apr 2023 17:09 UTC

On Thursday, April 27, 2023 at 12:26:47 PM UTC-4, Ulf Samuelsson wrote:
> Den 2023-04-20 kl. 04:06, skrev Rick C:
> > This is a bit of the chicken and egg thing. If you want a embed a checksum in a code module to report the checksum, is there a way of doing this? It's a bit like being your own grandfather, I think.
> >
> The proper way to do this is to have a directive in the linker.
> This reserves space for the CRC and defines the area where the CRC is
> calculated.

That assumes there is a linker. How does the application access this information?

> I am not aware of any linker which support this.
>
> Two months ago, I added the DIGEST directive to binutils aka the GNU
> linker. It was committed, but then people realized that I had not signed
> an agreement with Free Software Foundation.
> Since part of the code I pushed was from a third party which released
> their code under MIT, the licensing has not been resolved yet
> but the patch is in binutils git, but reverted.
>
> You would write (IIRC):
> DIGEST "CRC64-ECMA", (from, to)
> and the linker would reserve 8 bytes which is filled with the CRC in the
> final link stage.

You are making a lot of assumptions about the tools. I'm pretty sure they don't apply to my case. I'm not at all clear how this is workable, anyway. Adding the checksum to the file, changes the checksum, which is where this conversation started... unless I'm missing something significant.

--

Rick C.

+++ Get 1,000 miles of free Supercharging
+++ Tesla referral code - https://ts.la/richard11209

Re: Embedding a Checksum in an Image File

<kavt7iFlvc5U1@mid.individual.net>

 copy mid

https://www.novabbs.com/devel/article-flat.php?id=1509&group=comp.arch.embedded#1509

 copy link   Newsgroups: comp.arch.embedded
Path: i2pn2.org!i2pn.org!news.swapon.de!fu-berlin.de!uni-berlin.de!individual.net!not-for-mail
From: niklas.h...@tidorum.invalid (Niklas Holsti)
Newsgroups: comp.arch.embedded
Subject: Re: Embedding a Checksum in an Image File
Date: Thu, 27 Apr 2023 21:29:05 +0300
Organization: Tidorum Ltd
Lines: 56
Message-ID: <kavt7iFlvc5U1@mid.individual.net>
References: <116ff07e-5e25-469a-90a0-9474108aadd3n@googlegroups.com>
<105b1cab-9e5f-91b3-1de6-99586246ecbd@gmail.com>
<b62a77f7-3a13-4d76-b7cb-c42902595751n@googlegroups.com>
Mime-Version: 1.0
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 8bit
X-Trace: individual.net VWgrIikAZ6SOAqu1P9O3eQfTr9E30UUyfNRkeCnId/pRyHL0K3
Cancel-Lock: sha1:u3Z3LidXklZL4GUaT8bEYJ00P/E=
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.14; rv:102.0)
Gecko/20100101 Thunderbird/102.6.1
Content-Language: en-US
In-Reply-To: <b62a77f7-3a13-4d76-b7cb-c42902595751n@googlegroups.com>
 by: Niklas Holsti - Thu, 27 Apr 2023 18:29 UTC

On 2023-04-27 20:09, Rick C wrote:
> On Thursday, April 27, 2023 at 12:26:47 PM UTC-4, Ulf Samuelsson wrote:
>> Den 2023-04-20 kl. 04:06, skrev Rick C:
>>> This is a bit of the chicken and egg thing. If you want a embed a checksum in a code module to report the checksum, is there a way of doing this? It's a bit like being your own grandfather, I think.
>>>
>> The proper way to do this is to have a directive in the linker.
>> This reserves space for the CRC and defines the area where the CRC is
>> calculated.
>
> That assumes there is a linker.

Almost all toolchains have a linker.

> How does the application access this information?

In Ulf's suggestion, it seems the DIGEST directive emits 8 bytes of
checksum at the current point (usually the linker "." symbol). I assume
one can give that point in the image a linkage symbol, perhaps like

_checksum DIGEST "CRC64-ECMA", (from, to)

or like

_checksum EQU. .
DIGEST "CRC64-ECMA", (from, to)

(This is schematic linker code, not necessarily proper syntax.)

One can then from the application code access the "checksum" location as
an externally defined object, say:

extern uint8[8] checksum;

The linker will connect that C identifier to the actual address of the
DIGEST checksum. Here I assumed that the C compiler mangles C
identifiers into linkage symbols by prefixing an underscore; YMMV.

> You are making a lot of assumptions about the tools. I'm pretty sure
> they don't apply to my case. I'm not at all clear how this is
> workable, anyway. Adding the checksum to the file, changes the
> checksum, which is where this conversation started... unless I'm
> missing something significant.

But you have insisted that your "checksum" is for the purpose of
identifying the version of the program, not for checking the integrity
of the memory image. If so, that checksum does not have to be the
checksum of the whole memory image, as long as it is the checksum of the
part of the image that contains the actual code and constant data, and
so will change according to changes in those parts of the image.

Re: Embedding a Checksum in an Image File

<kavtrnFlvc5U2@mid.individual.net>

 copy mid

https://www.novabbs.com/devel/article-flat.php?id=1510&group=comp.arch.embedded#1510

 copy link   Newsgroups: comp.arch.embedded
Path: i2pn2.org!i2pn.org!news.swapon.de!fu-berlin.de!uni-berlin.de!individual.net!not-for-mail
From: niklas.h...@tidorum.invalid (Niklas Holsti)
Newsgroups: comp.arch.embedded
Subject: Re: Embedding a Checksum in an Image File
Date: Thu, 27 Apr 2023 21:39:51 +0300
Organization: Tidorum Ltd
Lines: 18
Message-ID: <kavtrnFlvc5U2@mid.individual.net>
References: <116ff07e-5e25-469a-90a0-9474108aadd3n@googlegroups.com>
<105b1cab-9e5f-91b3-1de6-99586246ecbd@gmail.com>
<b62a77f7-3a13-4d76-b7cb-c42902595751n@googlegroups.com>
<kavt7iFlvc5U1@mid.individual.net>
Mime-Version: 1.0
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 8bit
X-Trace: individual.net ekm21ivTAVX/vvduGtIMEQliEE2UBYjofDepnVE7//FkornHxb
Cancel-Lock: sha1:WMfQJUvmfnWp+errq4JyWE1yzG0=
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.14; rv:102.0)
Gecko/20100101 Thunderbird/102.6.1
Content-Language: en-US
In-Reply-To: <kavt7iFlvc5U1@mid.individual.net>
 by: Niklas Holsti - Thu, 27 Apr 2023 18:39 UTC

Um, just noting some typos in my last, with apologies:

On 2023-04-27 21:29, Niklas Holsti wrote:

>   _checksum  EQU.   .

should be

_checksum EQU .

(Thunderbird inserted an extra period out of "friendliness"...)

>    extern uint8[8] checksum;

should be (my C is rusty):

extern uint8 checksum[8];

Re: Embedding a Checksum in an Image File

<u2emd2$22pgk$1@dont-email.me>

 copy mid

https://www.novabbs.com/devel/article-flat.php?id=1511&group=comp.arch.embedded#1511

 copy link   Newsgroups: comp.arch.embedded
Path: i2pn2.org!i2pn.org!eternal-september.org!news.eternal-september.org!.POSTED!not-for-mail
From: ulf.r.sa...@gmail.com (Ulf Samuelsson)
Newsgroups: comp.arch.embedded
Subject: Re: Embedding a Checksum in an Image File
Date: Thu, 27 Apr 2023 22:36:46 +0200
Organization: eMagii
Lines: 56
Message-ID: <u2emd2$22pgk$1@dont-email.me>
References: <116ff07e-5e25-469a-90a0-9474108aadd3n@googlegroups.com>
<105b1cab-9e5f-91b3-1de6-99586246ecbd@gmail.com>
<b62a77f7-3a13-4d76-b7cb-c42902595751n@googlegroups.com>
Reply-To: ulf.r.samuelsson@gmail.com
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 8bit
Injection-Date: Thu, 27 Apr 2023 20:36:50 -0000 (UTC)
Injection-Info: dont-email.me; posting-host="e06ed78c1495380a4807241f34669d85";
logging-data="2188820"; mail-complaints-to="abuse@eternal-september.org"; posting-account="U2FsdGVkX19vzTgA/F1SKYptnraCucFe"
User-Agent: Mozilla/5.0 (Windows NT 6.1; Win64; x64; rv:102.0) Gecko/20100101
Thunderbird/102.8.0
Cancel-Lock: sha1:WsC6OMnIwYccgHrOIafB9RFoJlY=
Content-Language: sv
In-Reply-To: <b62a77f7-3a13-4d76-b7cb-c42902595751n@googlegroups.com>
 by: Ulf Samuelsson - Thu, 27 Apr 2023 20:36 UTC

Den 2023-04-27 kl. 19:09, skrev Rick C:
> On Thursday, April 27, 2023 at 12:26:47 PM UTC-4, Ulf Samuelsson wrote:
>> Den 2023-04-20 kl. 04:06, skrev Rick C:
>>> This is a bit of the chicken and egg thing. If you want a embed a checksum in a code module to report the checksum, is there a way of doing this? It's a bit like being your own grandfather, I think.
>>>
>> The proper way to do this is to have a directive in the linker.
>> This reserves space for the CRC and defines the area where the CRC is
>> calculated.
>
> That assumes there is a linker. How does the application access this information?
>
>
Linker command file
public CRC64; start, stop
HEADER = .;
QUAD(MAGIC);
CRC64 = .;
DIGEST "CRC64-ECMA", (start, stop)
start = .;
# Your data to be protected
...
stop = .;

C source code.

extern uint64_t CRC64;
extern char* start;
extern char* stop;

uint64_t crc;

crc64 = calc_crc64_ecma(start, stop);
if (crc64 == CRC64) {
/* everything is OK */
}

>> I am not aware of any linker which support this.
>>
>> Two months ago, I added the DIGEST directive to binutils aka the GNU
>> linker. It was committed, but then people realized that I had not signed
>> an agreement with Free Software Foundation.
>> Since part of the code I pushed was from a third party which released
>> their code under MIT, the licensing has not been resolved yet
>> but the patch is in binutils git, but reverted.
>>
>> You would write (IIRC):
>> DIGEST "CRC64-ECMA", (from, to)
>> and the linker would reserve 8 bytes which is filled with the CRC in the
>> final link stage.
>
> You are making a lot of assumptions about the tools. I'm pretty sure they don't apply to my case. I'm not at all clear how this is workable, anyway. Adding the checksum to the file, changes the checksum, which is where this conversation started... unless I'm missing something significant.
>
I am assuming that no tool support this off the shelg, but the patches
are inside binutils, but reverted.

/Ulf

Re: Embedding a Checksum in an Image File

<u2emsa$22pgk$2@dont-email.me>

 copy mid

https://www.novabbs.com/devel/article-flat.php?id=1512&group=comp.arch.embedded#1512

 copy link   Newsgroups: comp.arch.embedded
Path: i2pn2.org!i2pn.org!eternal-september.org!news.eternal-september.org!.POSTED!not-for-mail
From: ulf.r.sa...@gmail.com (Ulf Samuelsson)
Newsgroups: comp.arch.embedded
Subject: Re: Embedding a Checksum in an Image File
Date: Thu, 27 Apr 2023 22:44:54 +0200
Organization: eMagii
Lines: 77
Message-ID: <u2emsa$22pgk$2@dont-email.me>
References: <116ff07e-5e25-469a-90a0-9474108aadd3n@googlegroups.com>
<105b1cab-9e5f-91b3-1de6-99586246ecbd@gmail.com>
<b62a77f7-3a13-4d76-b7cb-c42902595751n@googlegroups.com>
<kavt7iFlvc5U1@mid.individual.net>
Reply-To: ulf.r.samuelsson@gmail.com
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 8bit
Injection-Date: Thu, 27 Apr 2023 20:44:58 -0000 (UTC)
Injection-Info: dont-email.me; posting-host="e06ed78c1495380a4807241f34669d85";
logging-data="2188820"; mail-complaints-to="abuse@eternal-september.org"; posting-account="U2FsdGVkX1/YDFZfcHtKsjHpKszyWSu3"
User-Agent: Mozilla/5.0 (Windows NT 6.1; Win64; x64; rv:102.0) Gecko/20100101
Thunderbird/102.8.0
Cancel-Lock: sha1:xjp9Tvt+8AX62EmWnugJNp8h6+8=
Content-Language: sv
In-Reply-To: <kavt7iFlvc5U1@mid.individual.net>
 by: Ulf Samuelsson - Thu, 27 Apr 2023 20:44 UTC

Den 2023-04-27 kl. 20:29, skrev Niklas Holsti:
> On 2023-04-27 20:09, Rick C wrote:
>> On Thursday, April 27, 2023 at 12:26:47 PM UTC-4, Ulf Samuelsson wrote:
>>> Den 2023-04-20 kl. 04:06, skrev Rick C:
>>>> This is a bit of the chicken and egg thing. If you want a embed a
>>>> checksum in a code module to report the checksum, is there a way of
>>>> doing this? It's a bit like being your own grandfather, I think.
>>>>
>>> The proper way to do this is to have a directive in the linker.
>>> This reserves space for the CRC and defines the area where the CRC is
>>> calculated.
>>
>> That assumes there is a linker.
>
>
> Almost all toolchains have a linker.
>
>
>> How does the application access this information?
>
>
> In Ulf's suggestion, it seems the DIGEST directive emits 8 bytes of
> checksum at the current point (usually the linker "." symbol). I assume
> one can give that point in the image a linkage symbol, perhaps like
>
>   _checksum  DIGEST "CRC64-ECMA", (from, to)
>
> or like
>
>   _checksum  EQU.   .
>              DIGEST "CRC64-ECMA", (from, to)
>
>
> (This is schematic linker code, not necessarily proper syntax.)
>
> One can then from the application code access the "checksum" location as
> an externally defined object, say:
>
>    extern uint8[8] checksum;
>
> The linker will connect that C identifier to the actual address of the
> DIGEST checksum. Here I assumed that the C compiler mangles C
> identifiers into linkage symbols by prefixing an underscore; YMMV.
>

Yes, that is more or less it.

>
>> You are making a lot of assumptions about the tools.  I'm pretty sure
>> they don't apply to my case.  I'm not at all clear how this is
>> workable, anyway.  Adding the checksum to the file, changes the
>> checksum, which is where this conversation started... unless I'm
>> missing something significant.
No, you reserve room for the checksum, but that needs to be outside
the checked area.
The address of the checksum needs to be known to the application.
Also the limits of the checked area.
That is why the application has a header in front in my projects.
The application is started by the bootloader, which checks
a number of things before the application is started.
The application can read the header as well to allow checking
the code area at runtime.

>
>
> But you have insisted that your "checksum" is for the purpose of
> identifying the version of the program, not for checking the integrity
> of the memory image. If so, that checksum does not have to be the
> checksum of the whole memory image, as long as it is the checksum of the
> part of the image that contains the actual code and constant data, and
> so will change according to changes in those parts of the image.
>

/Ulf

Re: Embedding a Checksum in an Image File

<kb0a5rFlvc6U1@mid.individual.net>

 copy mid

https://www.novabbs.com/devel/article-flat.php?id=1515&group=comp.arch.embedded#1515

 copy link   Newsgroups: comp.arch.embedded
Path: i2pn2.org!i2pn.org!weretis.net!feeder8.news.weretis.net!lilly.ping.de!fu-berlin.de!uni-berlin.de!individual.net!not-for-mail
From: niklas.h...@tidorum.invalid (Niklas Holsti)
Newsgroups: comp.arch.embedded
Subject: Re: Embedding a Checksum in an Image File
Date: Fri, 28 Apr 2023 01:10:02 +0300
Organization: Tidorum Ltd
Lines: 55
Message-ID: <kb0a5rFlvc6U1@mid.individual.net>
References: <116ff07e-5e25-469a-90a0-9474108aadd3n@googlegroups.com>
<105b1cab-9e5f-91b3-1de6-99586246ecbd@gmail.com>
<b62a77f7-3a13-4d76-b7cb-c42902595751n@googlegroups.com>
<u2emd2$22pgk$1@dont-email.me>
Mime-Version: 1.0
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 8bit
X-Trace: individual.net Z1r6UbirE2t7ernDCxvkBQf+t0t1Nrpc1s8BFw/w31c6TiaQpQ
Cancel-Lock: sha1:VCYE5glX6KfVIrCOi4EMzWmL0yI=
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.14; rv:102.0)
Gecko/20100101 Thunderbird/102.6.1
Content-Language: en-US
In-Reply-To: <u2emd2$22pgk$1@dont-email.me>
 by: Niklas Holsti - Thu, 27 Apr 2023 22:10 UTC

On 2023-04-27 23:36, Ulf Samuelsson wrote:
> Den 2023-04-27 kl. 19:09, skrev Rick C:
>> On Thursday, April 27, 2023 at 12:26:47 PM UTC-4, Ulf Samuelsson wrote:
>>> Den 2023-04-20 kl. 04:06, skrev Rick C:
>>>> This is a bit of the chicken and egg thing. If you want a embed a
>>>> checksum in a code module to report the checksum, is there a way of
>>>> doing this? It's a bit like being your own grandfather, I think.
>>>>
>>> The proper way to do this is to have a directive in the linker.
>>> This reserves space for the CRC and defines the area where the CRC is
>>> calculated.
>>
>> That assumes there is a linker.  How does the application access this
>> information?
>>
>>
> Linker command file
>         public CRC64; start, stop
>         HEADER = .;
>         QUAD(MAGIC);
>     CRC64 = .;
>         DIGEST "CRC64-ECMA", (start, stop)
>         start = .;
>         # Your data to be protected
>         ...
>         stop = .;
>
> C source code.
>
> extern uint64_t CRC64;
> extern char* start;
> extern char* stop;
>
> uint64_t crc;
>
> crc64 = calc_crc64_ecma(start, stop);
> if (crc64 == CRC64) {
>    /* everything is OK */
> }

I'm nit-picking, but that C code does not look right to me. The extern
declarations for "start" and "stop" claim them to be names of memory
locations that contain addresses, but the linker file just places them
at the starting and one-past-end locations of the block to be protected.
So the "start" variable contains the first bytes of the "data to be
protected", and the contents of the "stop" variable are not defined
because it is placed after the "data to be protected", where no code or
data is loaded (it seems).

It seems to me that the call to calc_crc64_ecma should get the addresses
of "start" and "stop" as arguments (&start, &stop), instead of their
values. But perhaps calc_crc64_ecma is not a function, but a macro that
can itself take the addresses of its parameters.

Re: Embedding a Checksum in an Image File

<u2g0gc$2cdfn$1@dont-email.me>

 copy mid

https://www.novabbs.com/devel/article-flat.php?id=1526&group=comp.arch.embedded#1526

 copy link   Newsgroups: comp.arch.embedded
Path: i2pn2.org!i2pn.org!eternal-september.org!news.eternal-september.org!.POSTED!not-for-mail
From: ulf.r.sa...@gmail.com (Ulf Samuelsson)
Newsgroups: comp.arch.embedded
Subject: Re: Embedding a Checksum in an Image File
Date: Fri, 28 Apr 2023 10:35:20 +0200
Organization: eMagii
Lines: 107
Message-ID: <u2g0gc$2cdfn$1@dont-email.me>
References: <116ff07e-5e25-469a-90a0-9474108aadd3n@googlegroups.com>
<pvl24i57aef4vc7bbdk9mvj7sic9dsh64t@4ax.com>
<f0afa198-e735-4da1-a16a-82764af3de4dn@googlegroups.com>
<u1s76b$o11h$1@dont-email.me> <u2e8ba$205k5$3@dont-email.me>
<u2frll$2bncf$1@dont-email.me>
Reply-To: ulf.r.samuelsson@gmail.com
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 8bit
Injection-Date: Fri, 28 Apr 2023 08:35:24 -0000 (UTC)
Injection-Info: dont-email.me; posting-host="b452f82109ea9c6d09e4f26752b75ae5";
logging-data="2504183"; mail-complaints-to="abuse@eternal-september.org"; posting-account="U2FsdGVkX18Rop+CdHSXpc4b5RruiBGT"
User-Agent: Mozilla/5.0 (Windows NT 6.1; Win64; x64; rv:102.0) Gecko/20100101
Thunderbird/102.8.0
Cancel-Lock: sha1:zV6lqFMXu/UZbGroZNPtp2pDaIU=
Content-Language: sv
In-Reply-To: <u2frll$2bncf$1@dont-email.me>
 by: Ulf Samuelsson - Fri, 28 Apr 2023 08:35 UTC

Den 2023-04-28 kl. 09:12, skrev David Brown:
> On 27/04/2023 18:36, Ulf Samuelsson wrote:
>> Den 2023-04-20 kl. 22:26, skrev David Brown:
>>> On 20/04/2023 18:45, Rick C wrote:
>>>> On Thursday, April 20, 2023 at 11:33:28 AM UTC-4, George Neuner
>>>> wrote:
>>>>> On Wed, 19 Apr 2023 19:06:33 -0700 (PDT), Rick C
>>>>> <gnuarm.del...@gmail.com> wrote:
>>>>>
>>>>>> This is a bit of the chicken and egg thing. If you want a embed
>>>>>> a checksum in a code module to report the checksum, is there a
>>>>>> way of doing this? It's a bit like being your own grandfather, I
>>>>>> think.
>>>>> Take a look at the old xmodem/ymodem CRC. It was designed such
>>>>> that when the CRC was sent immediately following the data, a
>>>>> receiver computing CRC over the whole incoming packet (data and CRC
>>>>> both) would get a result of zero.
>>>>>
>>>>> But AFAIK it doesn't work with CCITT equation(s) - you have to use
>>>>> xmodem/ymodem.
>>>>>> I'm not thinking anything too fancy, like a CRC, but rather a
>>>>>> simple modulo N addition, maybe N being 2^16.
>>>>> Sorry, I don't know a way to do it with a modular checksum. YMMV,
>>>>> but I think 16-bit CRC is pretty simple.
>>>>>
>>>>> George
>>>>
>>>> CRC is not complicated, but I would not know how to calculate an
>>>> inserted value to force the resulting CRC to zero.  How do you do
>>>> that?
>>>
>>> You "insert" the value at the end.  Anything else is insane.
>>
>> In all projects I have been involved with, the application binary starts
>> with a header looking like this.
>>
>>
>> MAGIC WORD 1
>> CRC
>> Entry Point
>> Size
>> other info...
>> MAGIC WORD 2
>> APPLICATION_START
>> ...
>> APPLICATION_END (aligned with flash sector)
>>
>>
>> The bootloader first checks the two magic words.
>> It then computes CRC on the header (from Entry Point) to APPLICATION_END
>>
>> I ported the IAR ielftool (open source) to Linux at
>> https://github.com/emagii/ielftool
>>
>> This can insert the CRC in the ELF file, but needs tweaks to work
>> with an ELF file generated by the GNU tools.
>>
>> /Ulf
>
> That can work for some microcontrollers, but is unsuitable for others -
> it depends on how the flash is organised.  For an msp430, for example,
> it would be fine, as the interrupt vectors (including the reset vector)
> are at the end of flash.  But for most ARM Cortex M devices, it would
> not be suitable - they expect the reset vector and initial stack pointer
> at the start of the flash image.  Some devices have a boot ROM, and then
> you have to match their specifics for the header - or you can have your
> own boot program, and make the header how ever you like.

All projects I am involved with have a custom bootloader.
If there is a problem with the reset vector, then the program will fail
immediately.
The CRC is right after the initial vector table.
The bootloader application contains a copy of the vector table.

THe first thing the bootloader does is to check the CRC from right after
the CRC. Then it compares the vector table with the copy.

The header is only for the application.

>
> I am absolutely a fan of having some kind of header like this (and
> sometimes even a human-readable copyright notice, identifier and version
> information).  And having it as near the beginning as possible is good.
> But for many microcontrollers, having it at the start is not feasible.
> And if you can't put the CRC at the start like you do, you have to put
> it at the end of the image.
>
>
> I've never really thought about trying to inject a CRC into an elf file.
>  I use elfs (or should that be "elves" ?) for debugging, not flash
> programming.  And usually the main concern for having a CRC at the end
> of the image is when you have an online update of some kind, to check
> that nothing has gone wrong during the transfer or in-field update.
>
>

The last bootloader I wrote download using Y-Modem which has CRC
checking. Since it had more RAM than internal flash, the whole
application was downloaded to RAM first, and then when everything is OK,
the flash can be programmed. Finally, the header is analyzed and the
flash contents checked. There is absolutely no need to have the CRC at
the end since the CRC result is stored in a known location.

/Ulf

Re: Embedding a Checksum in an Image File

<u2g10k$2cdfn$2@dont-email.me>

 copy mid

https://www.novabbs.com/devel/article-flat.php?id=1528&group=comp.arch.embedded#1528

 copy link   Newsgroups: comp.arch.embedded
Path: i2pn2.org!i2pn.org!eternal-september.org!news.eternal-september.org!.POSTED!not-for-mail
From: ulf.r.sa...@gmail.com (Ulf Samuelsson)
Newsgroups: comp.arch.embedded
Subject: Re: Embedding a Checksum in an Image File
Date: Fri, 28 Apr 2023 10:44:00 +0200
Organization: eMagii
Lines: 58
Message-ID: <u2g10k$2cdfn$2@dont-email.me>
References: <116ff07e-5e25-469a-90a0-9474108aadd3n@googlegroups.com>
<66e9fb2a-b9e6-4597-9afa-6572caaa8ca1n@googlegroups.com>
<a4495c87-c68d-4c87-94aa-701c21bdd19cn@googlegroups.com>
<u1u8hu$2ps79$1@dont-email.me>
<ef5ad4e6-57ed-4950-baed-3a6746b9d16en@googlegroups.com>
<u2e8lu$205k5$4@dont-email.me> <u2fs41$2bnd0$1@dont-email.me>
Reply-To: ulf.r.samuelsson@gmail.com
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 8bit
Injection-Date: Fri, 28 Apr 2023 08:44:04 -0000 (UTC)
Injection-Info: dont-email.me; posting-host="b452f82109ea9c6d09e4f26752b75ae5";
logging-data="2504183"; mail-complaints-to="abuse@eternal-september.org"; posting-account="U2FsdGVkX19S+n3MAHB5ub5K9M+q8HnL"
User-Agent: Mozilla/5.0 (Windows NT 6.1; Win64; x64; rv:102.0) Gecko/20100101
Thunderbird/102.8.0
Cancel-Lock: sha1:0pFiUjt1aD+IuWHAbkZGyLVaAeI=
Content-Language: sv
In-Reply-To: <u2fs41$2bnd0$1@dont-email.me>
 by: Ulf Samuelsson - Fri, 28 Apr 2023 08:44 UTC

Den 2023-04-28 kl. 09:20, skrev David Brown:
> On 27/04/2023 18:42, Ulf Samuelsson wrote:
>> Den 2023-04-22 kl. 05:14, skrev Rick C:
>>> On Friday, April 21, 2023 at 11:02:28 AM UTC-4, David Brown wrote:
>>>> On 21/04/2023 14:12, Rick C wrote:
>>>>>
>>>>> This is simply to be able to say this version is unique, regardless
>>>>> of what the version number says. Version numbers are set manually
>>>>> and not always done correctly. I'm looking for something as a backup
>>>>> so that if the checksums are different, I can be sure the versions
>>>>> are not the same.
>>>>>
>>>>> The less work involved, the better.
>>>>>
>>>> Run a simple 32-bit crc over the image. The result is a hash of the
>>>> image. Any change in the image will show up as a change in the crc.
>>>
>>> No one is trying to detect changes in the image.  I'm trying to label
>>> the image in a way that can be read in operation.  I'm using the
>>> checksum simply because that is easy to generate.  I've had problems
>>> with version numbering in the past.  It will be used, but I want it
>>> supplemented with a number that will change every time the design
>>> changes, at least with a high probability, such as 1 in 64k.
>>>
>>
>> Another thing I added (and was later removed) was a timestamp directive.
>> A 64 bit integer with the number of seconds since 1970-01-01 00:00.
>>
>
> Timestamping a build in some way (as part of the "make", using __DATE__
> or __TIME__ in source code, or some feature of a revision control
> system) is very tempting, and can be helpful for tracking exactly what
> code you have on the system.
>
> However, IMHO having reproducible builds is much more valuable.  I am
> not happy with a project build until I am getting identical binaries
> built on multiple hosts (Windows and Linux).  That's how you can be
> absolutely sure of what code went into a particular binary, even years
> or decades later.

With the timestamp located in the header, you can simply compare the
non-header area.
Make with __DATE__ or __TIME__ will tell you when that module is
compiled, not when the program is generated.
That is why TIMESTAMP is best generated in the linker.

/Ulf

>
> A compromise that can work is to distinguish development builds and
> production builds, and have timestamping in development builds.  That
> also reduces the rate at which your minor version number or build number
> goes up, and avoids endless changes to your "version.h" include file.
>
>
>
>

Re: Embedding a Checksum in an Image File

<u2g1bu$2cdfn$3@dont-email.me>

 copy mid

https://www.novabbs.com/devel/article-flat.php?id=1529&group=comp.arch.embedded#1529

 copy link   Newsgroups: comp.arch.embedded
Path: i2pn2.org!i2pn.org!eternal-september.org!news.eternal-september.org!.POSTED!not-for-mail
From: ulf.r.sa...@gmail.com (Ulf Samuelsson)
Newsgroups: comp.arch.embedded
Subject: Re: Embedding a Checksum in an Image File
Date: Fri, 28 Apr 2023 10:50:02 +0200
Organization: eMagii
Lines: 52
Message-ID: <u2g1bu$2cdfn$3@dont-email.me>
References: <116ff07e-5e25-469a-90a0-9474108aadd3n@googlegroups.com>
<105b1cab-9e5f-91b3-1de6-99586246ecbd@gmail.com>
<b62a77f7-3a13-4d76-b7cb-c42902595751n@googlegroups.com>
<kavt7iFlvc5U1@mid.individual.net> <u2emsa$22pgk$2@dont-email.me>
<u2ft59$2bnd0$4@dont-email.me>
Reply-To: ulf.r.samuelsson@gmail.com
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 8bit
Injection-Date: Fri, 28 Apr 2023 08:50:06 -0000 (UTC)
Injection-Info: dont-email.me; posting-host="b452f82109ea9c6d09e4f26752b75ae5";
logging-data="2504183"; mail-complaints-to="abuse@eternal-september.org"; posting-account="U2FsdGVkX1+YocL/nJ3L6oAzng3lOL1I"
User-Agent: Mozilla/5.0 (Windows NT 6.1; Win64; x64; rv:102.0) Gecko/20100101
Thunderbird/102.8.0
Cancel-Lock: sha1:jiccs+8MzRpAHBUcfIivTuQPuzA=
In-Reply-To: <u2ft59$2bnd0$4@dont-email.me>
Content-Language: sv
 by: Ulf Samuelsson - Fri, 28 Apr 2023 08:50 UTC

Den 2023-04-28 kl. 09:38, skrev David Brown:
> On 27/04/2023 22:44, Ulf Samuelsson wrote:
>> Den 2023-04-27 kl. 20:29, skrev Niklas Holsti:
>>> On 2023-04-27 20:09, Rick C wrote:
>
>>
>>>
>>>> You are making a lot of assumptions about the tools.  I'm pretty sure
>>>> they don't apply to my case.  I'm not at all clear how this is
>>>> workable, anyway.  Adding the checksum to the file, changes the
>>>> checksum, which is where this conversation started... unless I'm
>>>> missing something significant.
>> No, you reserve room for the checksum, but that needs to be outside
>> the checked area.
>> The address of the checksum needs to be known to the application.
>
> The address here could have a symbol, and then declared "extern" in the
> C code - it would not have to be a known numerical address.  But if the
> image is checked or started from another program (such as a boot
> program), you need an absolute address somewhere to chain this all
> together.

The header is declared as a struct.

>
>> Also the limits of the checked area.
>> That is why the application has a header in front in my projects.
>> The application is started by the bootloader, which checks
>> a number of things before the application is started.
>> The application can read the header as well to allow checking
>> the code area at runtime.
>>
>
> Or for my preferences, the CRC "DIGEST" would be put at the end of the
> image, rather than near the start.  Then the "from, to" range would
> cover the entire image except for the final CRC.  But I'd have a similar
> directive for the length of the image at a specific area near the start.
>

I really do not see a benefit of splitting the meta information about
the image to two separate locations.

The bootloader uses the struct for all checks.
It is a much simpler implementation once the tools support it.

You might find it easier to write a tool which adds the CRC at the end,
but that is a different issue.

Occam's Razor!

/Ulf

Re: Embedding a Checksum in an Image File

<u2g1jk$2cdfn$4@dont-email.me>

 copy mid

https://www.novabbs.com/devel/article-flat.php?id=1530&group=comp.arch.embedded#1530

 copy link   Newsgroups: comp.arch.embedded
Path: i2pn2.org!i2pn.org!eternal-september.org!news.eternal-september.org!.POSTED!not-for-mail
From: ulf.r.sa...@gmail.com (Ulf Samuelsson)
Newsgroups: comp.arch.embedded
Subject: Re: Embedding a Checksum in an Image File
Date: Fri, 28 Apr 2023 10:54:08 +0200
Organization: eMagii
Lines: 61
Message-ID: <u2g1jk$2cdfn$4@dont-email.me>
References: <116ff07e-5e25-469a-90a0-9474108aadd3n@googlegroups.com>
<105b1cab-9e5f-91b3-1de6-99586246ecbd@gmail.com>
<b62a77f7-3a13-4d76-b7cb-c42902595751n@googlegroups.com>
<u2emd2$22pgk$1@dont-email.me> <kb0a5rFlvc6U1@mid.individual.net>
Reply-To: ulf.r.samuelsson@gmail.com
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 8bit
Injection-Date: Fri, 28 Apr 2023 08:54:12 -0000 (UTC)
Injection-Info: dont-email.me; posting-host="b452f82109ea9c6d09e4f26752b75ae5";
logging-data="2504183"; mail-complaints-to="abuse@eternal-september.org"; posting-account="U2FsdGVkX19+gJ5+j/om5fhl5gD9GEFX"
User-Agent: Mozilla/5.0 (Windows NT 6.1; Win64; x64; rv:102.0) Gecko/20100101
Thunderbird/102.8.0
Cancel-Lock: sha1:6cheOsxzqTeyTcjgLG2BZk4IFI8=
In-Reply-To: <kb0a5rFlvc6U1@mid.individual.net>
Content-Language: sv
 by: Ulf Samuelsson - Fri, 28 Apr 2023 08:54 UTC

Den 2023-04-28 kl. 00:10, skrev Niklas Holsti:
> On 2023-04-27 23:36, Ulf Samuelsson wrote:
>> Den 2023-04-27 kl. 19:09, skrev Rick C:
>>> On Thursday, April 27, 2023 at 12:26:47 PM UTC-4, Ulf Samuelsson wrote:
>>>> Den 2023-04-20 kl. 04:06, skrev Rick C:
>>>>> This is a bit of the chicken and egg thing. If you want a embed a
>>>>> checksum in a code module to report the checksum, is there a way of
>>>>> doing this? It's a bit like being your own grandfather, I think.
>>>>>
>>>> The proper way to do this is to have a directive in the linker.
>>>> This reserves space for the CRC and defines the area where the CRC is
>>>> calculated.
>>>
>>> That assumes there is a linker.  How does the application access this
>>> information?
>>>
>>>
>> Linker command file
>>          public CRC64; start, stop
>>          HEADER = .;
>>          QUAD(MAGIC);
>>      CRC64 = .;
>>          DIGEST "CRC64-ECMA", (start, stop)
>>          start = .;
>>          # Your data to be protected
>>          ...
>>          stop = .;
>>
>> C source code.
>>
>> extern uint64_t CRC64;
>> extern char* start;
>> extern char* stop;
>>
>> uint64_t crc;
>>
>> crc64 = calc_crc64_ecma(start, stop);
>> if (crc64 == CRC64) {
>>     /* everything is OK */
>> }
>
>
> I'm nit-picking, but that C code does not look right to me. The extern
> declarations for "start" and "stop" claim them to be names of memory
> locations that contain addresses, but the linker file just places them
> at the starting and one-past-end locations of the block to be protected.
> So the "start" variable contains the first bytes of the "data to be
> protected", and the contents of the "stop" variable are not defined
> because it is placed after the "data to be protected", where no code or
> data is loaded (it seems).
>
> It seems to me that the call to calc_crc64_ecma should get the addresses
> of "start" and "stop" as arguments (&start, &stop), instead of their
> values. But perhaps calc_crc64_ecma is not a function, but a macro that
> can itself take the addresses of its parameters.
>
Whatever,
I did not put a lot of thought into that, and certainly did not check
it. The important thing is that you can declare labels in the linker
and use them in the code through extern declarations.
/Ulf

Re: Embedding a Checksum in an Image File

<u2g1n5$2cdfn$5@dont-email.me>

 copy mid

https://www.novabbs.com/devel/article-flat.php?id=1532&group=comp.arch.embedded#1532

 copy link   Newsgroups: comp.arch.embedded
Path: i2pn2.org!i2pn.org!eternal-september.org!news.eternal-september.org!.POSTED!not-for-mail
From: ulf.r.sa...@gmail.com (Ulf Samuelsson)
Newsgroups: comp.arch.embedded
Subject: Re: Embedding a Checksum in an Image File
Date: Fri, 28 Apr 2023 10:56:01 +0200
Organization: eMagii
Lines: 42
Message-ID: <u2g1n5$2cdfn$5@dont-email.me>
References: <116ff07e-5e25-469a-90a0-9474108aadd3n@googlegroups.com>
<105b1cab-9e5f-91b3-1de6-99586246ecbd@gmail.com>
<u2fsb1$2bnd0$2@dont-email.me>
Reply-To: ulf.r.samuelsson@gmail.com
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 8bit
Injection-Date: Fri, 28 Apr 2023 08:56:05 -0000 (UTC)
Injection-Info: dont-email.me; posting-host="b452f82109ea9c6d09e4f26752b75ae5";
logging-data="2504183"; mail-complaints-to="abuse@eternal-september.org"; posting-account="U2FsdGVkX1+ClFOGGow2nIRl9jaAZ1kt"
User-Agent: Mozilla/5.0 (Windows NT 6.1; Win64; x64; rv:102.0) Gecko/20100101
Thunderbird/102.8.0
Cancel-Lock: sha1:pXRNbyPMGdFH6u3IG3zzARTuiw4=
In-Reply-To: <u2fsb1$2bnd0$2@dont-email.me>
Content-Language: sv
 by: Ulf Samuelsson - Fri, 28 Apr 2023 08:56 UTC

Den 2023-04-28 kl. 09:24, skrev David Brown:
> On 27/04/2023 18:26, Ulf Samuelsson wrote:
>> Den 2023-04-20 kl. 04:06, skrev Rick C:
>>> This is a bit of the chicken and egg thing.  If you want a embed a
>>> checksum in a code module to report the checksum, is there a way of
>>> doing this?  It's a bit like being your own grandfather, I think.
>>>
>> The proper way to do this is to have a directive in the linker.
>> This reserves space for the CRC and defines the area where the CRC is
>> calculated.
>> I am not aware of any linker which support this.
>>
>> Two months ago, I added the DIGEST directive to binutils aka the GNU
>> linker. It was committed, but then people realized that I had not signed
>> an agreement with Free Software Foundation.
>> Since part of the code I pushed was from a third party which released
>> their code under MIT, the licensing has not been resolved yet
>> but the patch is in binutils git, but reverted.
>>
>> You would write (IIRC):
>>     DIGEST "CRC64-ECMA", (from, to)
>> and the linker would reserve 8 bytes which is filled with the CRC in
>> the final link stage.
>>
>> /Ulf
>>
>
> I like that.  Thanks for doing that work.
>
> Is there also a way to get the length of the final link, and insert it
> near the beginning of the image?  I suppose that would be another kind
> of DIGEST where the algorithm is simply (to - from).  (I assume that
> "to" and "from" may be linker symbols.)
>
>

app_size = .;
LONG(to-from);

should work using the GNU linker.
/Ulf

Re: Embedding a Checksum in an Image File

<u2gg9d$2ejus$3@dont-email.me>

 copy mid

https://www.novabbs.com/devel/article-flat.php?id=1538&group=comp.arch.embedded#1538

 copy link   Newsgroups: comp.arch.embedded
Path: i2pn2.org!i2pn.org!eternal-september.org!news.eternal-september.org!.POSTED!not-for-mail
From: david.br...@hesbynett.no (David Brown)
Newsgroups: comp.arch.embedded
Subject: Re: Embedding a Checksum in an Image File
Date: Fri, 28 Apr 2023 15:04:45 +0200
Organization: A noiseless patient Spider
Lines: 61
Message-ID: <u2gg9d$2ejus$3@dont-email.me>
References: <116ff07e-5e25-469a-90a0-9474108aadd3n@googlegroups.com>
<105b1cab-9e5f-91b3-1de6-99586246ecbd@gmail.com>
<b62a77f7-3a13-4d76-b7cb-c42902595751n@googlegroups.com>
<kavt7iFlvc5U1@mid.individual.net> <u2emsa$22pgk$2@dont-email.me>
<u2ft59$2bnd0$4@dont-email.me> <u2g1bu$2cdfn$3@dont-email.me>
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 8bit
Injection-Date: Fri, 28 Apr 2023 13:04:45 -0000 (UTC)
Injection-Info: dont-email.me; posting-host="3d1097169b2902b2a75ddccb5ac1bd12";
logging-data="2576348"; mail-complaints-to="abuse@eternal-september.org"; posting-account="U2FsdGVkX1+qk6YOHkMYF9IEi8NcQDXbnwiCu39M/Ro="
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101
Thunderbird/102.9.0
Cancel-Lock: sha1:FysSK/VC/MoPhY/s4gjIo+OMXDk=
Content-Language: en-GB
In-Reply-To: <u2g1bu$2cdfn$3@dont-email.me>
 by: David Brown - Fri, 28 Apr 2023 13:04 UTC

On 28/04/2023 10:50, Ulf Samuelsson wrote:
> Den 2023-04-28 kl. 09:38, skrev David Brown:
>> On 27/04/2023 22:44, Ulf Samuelsson wrote:
>>> Den 2023-04-27 kl. 20:29, skrev Niklas Holsti:
>>>> On 2023-04-27 20:09, Rick C wrote:
>>
>>>
>>>>
>>>>> You are making a lot of assumptions about the tools.  I'm pretty sure
>>>>> they don't apply to my case.  I'm not at all clear how this is
>>>>> workable, anyway.  Adding the checksum to the file, changes the
>>>>> checksum, which is where this conversation started... unless I'm
>>>>> missing something significant.
>>> No, you reserve room for the checksum, but that needs to be outside
>>> the checked area.
>>> The address of the checksum needs to be known to the application.
>>
>> The address here could have a symbol, and then declared "extern" in
>> the C code - it would not have to be a known numerical address.  But
>> if the image is checked or started from another program (such as a
>> boot program), you need an absolute address somewhere to chain this
>> all together.
>
> The header is declared as a struct.
>
>>
>>> Also the limits of the checked area.
>>> That is why the application has a header in front in my projects.
>>> The application is started by the bootloader, which checks
>>> a number of things before the application is started.
>>> The application can read the header as well to allow checking
>>> the code area at runtime.
>>>
>>
>> Or for my preferences, the CRC "DIGEST" would be put at the end of the
>> image, rather than near the start.  Then the "from, to" range would
>> cover the entire image except for the final CRC.  But I'd have a
>> similar directive for the length of the image at a specific area near
>> the start.
>>
>
> I really do not see a benefit of splitting the meta information about
> the image to two separate locations.
>
> The bootloader uses the struct for all checks.
> It is a much simpler implementation once the tools support it.
>
> You might find it easier to write a tool which adds the CRC at the end,
> but that is a different issue.
>
> Occam's Razor!
>

There are different needs for different projects - and more than one way
to handle them. I find adding a CRC at the end of the image works best
for me, but I have no problem appreciating that other people have
different solutions.

Re: Embedding a Checksum in an Image File

<u2gghc$2ejus$4@dont-email.me>

 copy mid

https://www.novabbs.com/devel/article-flat.php?id=1539&group=comp.arch.embedded#1539

 copy link   Newsgroups: comp.arch.embedded
Path: i2pn2.org!i2pn.org!eternal-september.org!news.eternal-september.org!.POSTED!not-for-mail
From: david.br...@hesbynett.no (David Brown)
Newsgroups: comp.arch.embedded
Subject: Re: Embedding a Checksum in an Image File
Date: Fri, 28 Apr 2023 15:09:00 +0200
Organization: A noiseless patient Spider
Lines: 48
Message-ID: <u2gghc$2ejus$4@dont-email.me>
References: <116ff07e-5e25-469a-90a0-9474108aadd3n@googlegroups.com>
<105b1cab-9e5f-91b3-1de6-99586246ecbd@gmail.com>
<u2fsb1$2bnd0$2@dont-email.me> <u2g1n5$2cdfn$5@dont-email.me>
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 8bit
Injection-Date: Fri, 28 Apr 2023 13:09:00 -0000 (UTC)
Injection-Info: dont-email.me; posting-host="3d1097169b2902b2a75ddccb5ac1bd12";
logging-data="2576348"; mail-complaints-to="abuse@eternal-september.org"; posting-account="U2FsdGVkX193nLNa17iy9ZrJoabE3lnaFZBufiv+OLc="
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101
Thunderbird/102.9.0
Cancel-Lock: sha1:IYTKG2QznxFoJkzTc48qTEITnVw=
Content-Language: en-GB
In-Reply-To: <u2g1n5$2cdfn$5@dont-email.me>
 by: David Brown - Fri, 28 Apr 2023 13:09 UTC

On 28/04/2023 10:56, Ulf Samuelsson wrote:
> Den 2023-04-28 kl. 09:24, skrev David Brown:
>> On 27/04/2023 18:26, Ulf Samuelsson wrote:
>>> Den 2023-04-20 kl. 04:06, skrev Rick C:
>>>> This is a bit of the chicken and egg thing.  If you want a embed a
>>>> checksum in a code module to report the checksum, is there a way of
>>>> doing this?  It's a bit like being your own grandfather, I think.
>>>>
>>> The proper way to do this is to have a directive in the linker.
>>> This reserves space for the CRC and defines the area where the CRC is
>>> calculated.
>>> I am not aware of any linker which support this.
>>>
>>> Two months ago, I added the DIGEST directive to binutils aka the GNU
>>> linker. It was committed, but then people realized that I had not signed
>>> an agreement with Free Software Foundation.
>>> Since part of the code I pushed was from a third party which released
>>> their code under MIT, the licensing has not been resolved yet
>>> but the patch is in binutils git, but reverted.
>>>
>>> You would write (IIRC):
>>>     DIGEST "CRC64-ECMA", (from, to)
>>> and the linker would reserve 8 bytes which is filled with the CRC in
>>> the final link stage.
>>>
>>> /Ulf
>>>
>>
>> I like that.  Thanks for doing that work.
>>
>> Is there also a way to get the length of the final link, and insert it
>> near the beginning of the image?  I suppose that would be another kind
>> of DIGEST where the algorithm is simply (to - from).  (I assume that
>> "to" and "from" may be linker symbols.)
>>
>>
>
>    app_size = .;
>    LONG(to-from);
>
> should work using the GNU linker.

Will that work when placed earlier in the link than the definition of
"to" ? I had assumed - perhaps completely incorrectly - that the linker
would have to have established the value of "to" before its use in such
an expression.

Re: Embedding a Checksum in an Image File

<u2k0ks$33teb$1@dont-email.me>

 copy mid

https://www.novabbs.com/devel/article-flat.php?id=1545&group=comp.arch.embedded#1545

 copy link   Newsgroups: comp.arch.embedded
Path: i2pn2.org!i2pn.org!eternal-september.org!news.eternal-september.org!.POSTED!not-for-mail
From: ulf.r.sa...@gmail.com (Ulf Samuelsson)
Newsgroups: comp.arch.embedded
Subject: Re: Embedding a Checksum in an Image File
Date: Sat, 29 Apr 2023 23:02:15 +0200
Organization: eMagii
Lines: 61
Message-ID: <u2k0ks$33teb$1@dont-email.me>
References: <116ff07e-5e25-469a-90a0-9474108aadd3n@googlegroups.com>
<105b1cab-9e5f-91b3-1de6-99586246ecbd@gmail.com>
<u2fsb1$2bnd0$2@dont-email.me> <u2g1n5$2cdfn$5@dont-email.me>
<u2gghc$2ejus$4@dont-email.me>
Reply-To: ulf.r.samuelsson.invalid@gmail.com
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 8bit
Injection-Date: Sat, 29 Apr 2023 21:02:20 -0000 (UTC)
Injection-Info: dont-email.me; posting-host="9df99c9307656e0fa7d1da40a76efb0c";
logging-data="3274187"; mail-complaints-to="abuse@eternal-september.org"; posting-account="U2FsdGVkX1+oSkKJgtSc5AEmLcbsdW15"
User-Agent: Mozilla/5.0 (Windows NT 6.1; Win64; x64; rv:102.0) Gecko/20100101
Thunderbird/102.8.0
Cancel-Lock: sha1:KMKdW+7k3ATRAMuELQGiyY47Ri0=
Content-Language: sv
In-Reply-To: <u2gghc$2ejus$4@dont-email.me>
 by: Ulf Samuelsson - Sat, 29 Apr 2023 21:02 UTC

Den 2023-04-28 kl. 15:09, skrev David Brown:
> On 28/04/2023 10:56, Ulf Samuelsson wrote:
>> Den 2023-04-28 kl. 09:24, skrev David Brown:
>>> On 27/04/2023 18:26, Ulf Samuelsson wrote:
>>>> Den 2023-04-20 kl. 04:06, skrev Rick C:
>>>>> This is a bit of the chicken and egg thing.  If you want a embed a
>>>>> checksum in a code module to report the checksum, is there a way of
>>>>> doing this?  It's a bit like being your own grandfather, I think.
>>>>>
>>>> The proper way to do this is to have a directive in the linker.
>>>> This reserves space for the CRC and defines the area where the CRC
>>>> is calculated.
>>>> I am not aware of any linker which support this.
>>>>
>>>> Two months ago, I added the DIGEST directive to binutils aka the GNU
>>>> linker. It was committed, but then people realized that I had not
>>>> signed
>>>> an agreement with Free Software Foundation.
>>>> Since part of the code I pushed was from a third party which
>>>> released their code under MIT, the licensing has not been resolved yet
>>>> but the patch is in binutils git, but reverted.
>>>>
>>>> You would write (IIRC):
>>>>     DIGEST "CRC64-ECMA", (from, to)
>>>> and the linker would reserve 8 bytes which is filled with the CRC in
>>>> the final link stage.
>>>>
>>>> /Ulf
>>>>
>>>
>>> I like that.  Thanks for doing that work.
>>>
>>> Is there also a way to get the length of the final link, and insert
>>> it near the beginning of the image?  I suppose that would be another
>>> kind of DIGEST where the algorithm is simply (to - from).  (I assume
>>> that "to" and "from" may be linker symbols.)
>>>
>>>
>>
>>     app_size = .;
>>     LONG(to-from);
>>
>> should work using the GNU linker.
>
> Will that work when placed earlier in the link than the definition of
> "to" ?  I had assumed - perhaps completely incorrectly - that the linker
> would have to have established the value of "to" before its use in such
> an expression.

Yes, the GNU linker creates a list of statements with a known size
updating the location counter for each statement.

The expressions are evaluated in a later stage
so you can add a DIGEST statement and compute the SIZE by
"LONG(to-from);" before "to" and "from" are declared.

/Ulf

>
>

Re: Embedding a Checksum in an Image File

<u2k0mp$33teb$2@dont-email.me>

 copy mid

https://www.novabbs.com/devel/article-flat.php?id=1546&group=comp.arch.embedded#1546

 copy link   Newsgroups: comp.arch.embedded
Path: i2pn2.org!i2pn.org!eternal-september.org!news.eternal-september.org!.POSTED!not-for-mail
From: ulf.r.sa...@gmail.com (Ulf Samuelsson)
Newsgroups: comp.arch.embedded
Subject: Re: Embedding a Checksum in an Image File
Date: Sat, 29 Apr 2023 23:03:16 +0200
Organization: eMagii
Lines: 64
Message-ID: <u2k0mp$33teb$2@dont-email.me>
References: <116ff07e-5e25-469a-90a0-9474108aadd3n@googlegroups.com>
<105b1cab-9e5f-91b3-1de6-99586246ecbd@gmail.com>
<b62a77f7-3a13-4d76-b7cb-c42902595751n@googlegroups.com>
<kavt7iFlvc5U1@mid.individual.net> <u2emsa$22pgk$2@dont-email.me>
<u2ft59$2bnd0$4@dont-email.me> <u2g1bu$2cdfn$3@dont-email.me>
<u2gg9d$2ejus$3@dont-email.me>
Reply-To: ulf.r.samuelsson.invalid@gmail.com
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 8bit
Injection-Date: Sat, 29 Apr 2023 21:03:21 -0000 (UTC)
Injection-Info: dont-email.me; posting-host="9df99c9307656e0fa7d1da40a76efb0c";
logging-data="3274187"; mail-complaints-to="abuse@eternal-september.org"; posting-account="U2FsdGVkX18ZzaBjgCozPdtUJ5OFmxhm"
User-Agent: Mozilla/5.0 (Windows NT 6.1; Win64; x64; rv:102.0) Gecko/20100101
Thunderbird/102.8.0
Cancel-Lock: sha1:3hUXOc+JQNJTL6lwogD0AfuNo50=
In-Reply-To: <u2gg9d$2ejus$3@dont-email.me>
Content-Language: sv
 by: Ulf Samuelsson - Sat, 29 Apr 2023 21:03 UTC

Den 2023-04-28 kl. 15:04, skrev David Brown:
> On 28/04/2023 10:50, Ulf Samuelsson wrote:
>> Den 2023-04-28 kl. 09:38, skrev David Brown:
>>> On 27/04/2023 22:44, Ulf Samuelsson wrote:
>>>> Den 2023-04-27 kl. 20:29, skrev Niklas Holsti:
>>>>> On 2023-04-27 20:09, Rick C wrote:
>>>
>>>>
>>>>>
>>>>>> You are making a lot of assumptions about the tools.  I'm pretty sure
>>>>>> they don't apply to my case.  I'm not at all clear how this is
>>>>>> workable, anyway.  Adding the checksum to the file, changes the
>>>>>> checksum, which is where this conversation started... unless I'm
>>>>>> missing something significant.
>>>> No, you reserve room for the checksum, but that needs to be outside
>>>> the checked area.
>>>> The address of the checksum needs to be known to the application.
>>>
>>> The address here could have a symbol, and then declared "extern" in
>>> the C code - it would not have to be a known numerical address.  But
>>> if the image is checked or started from another program (such as a
>>> boot program), you need an absolute address somewhere to chain this
>>> all together.
>>
>> The header is declared as a struct.
>>
>>>
>>>> Also the limits of the checked area.
>>>> That is why the application has a header in front in my projects.
>>>> The application is started by the bootloader, which checks
>>>> a number of things before the application is started.
>>>> The application can read the header as well to allow checking
>>>> the code area at runtime.
>>>>
>>>
>>> Or for my preferences, the CRC "DIGEST" would be put at the end of
>>> the image, rather than near the start.  Then the "from, to" range
>>> would cover the entire image except for the final CRC.  But I'd have
>>> a similar directive for the length of the image at a specific area
>>> near the start.
>>>
>>
>> I really do not see a benefit of splitting the meta information about
>> the image to two separate locations.
>>
>> The bootloader uses the struct for all checks.
>> It is a much simpler implementation once the tools support it.
>>
>> You might find it easier to write a tool which adds the CRC at the
>> end, but that is a different issue.
>>
>> Occam's Razor!
>>
>
> There are different needs for different projects - and more than one way
> to handle them.  I find adding a CRC at the end of the image works best
> for me, but I have no problem appreciating that other people have
> different solutions.
>
>
>
>
I'd be curious to know WHY it works best for you.
/Ulf

Re: Embedding a Checksum in an Image File

<u2ltdt$3i28u$1@dont-email.me>

 copy mid

https://www.novabbs.com/devel/article-flat.php?id=1548&group=comp.arch.embedded#1548

 copy link   Newsgroups: comp.arch.embedded
Path: i2pn2.org!i2pn.org!eternal-september.org!news.eternal-september.org!.POSTED!not-for-mail
From: david.br...@hesbynett.no (David Brown)
Newsgroups: comp.arch.embedded
Subject: Re: Embedding a Checksum in an Image File
Date: Sun, 30 Apr 2023 16:19:41 +0200
Organization: A noiseless patient Spider
Lines: 80
Message-ID: <u2ltdt$3i28u$1@dont-email.me>
References: <116ff07e-5e25-469a-90a0-9474108aadd3n@googlegroups.com>
<105b1cab-9e5f-91b3-1de6-99586246ecbd@gmail.com>
<b62a77f7-3a13-4d76-b7cb-c42902595751n@googlegroups.com>
<kavt7iFlvc5U1@mid.individual.net> <u2emsa$22pgk$2@dont-email.me>
<u2ft59$2bnd0$4@dont-email.me> <u2g1bu$2cdfn$3@dont-email.me>
<u2gg9d$2ejus$3@dont-email.me> <u2k0mp$33teb$2@dont-email.me>
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 8bit
Injection-Date: Sun, 30 Apr 2023 14:19:41 -0000 (UTC)
Injection-Info: dont-email.me; posting-host="bfae2ae3a249c0269fba5b17fdef3f24";
logging-data="3737886"; mail-complaints-to="abuse@eternal-september.org"; posting-account="U2FsdGVkX1/CXw35zKwjWLwpVSU/7B0xNwc2wTNz9H8="
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101
Thunderbird/102.7.1
Cancel-Lock: sha1:hgHKhjfYctqxYhW1h0kyqOr40ZE=
In-Reply-To: <u2k0mp$33teb$2@dont-email.me>
Content-Language: en-GB
 by: David Brown - Sun, 30 Apr 2023 14:19 UTC

On 29/04/2023 23:03, Ulf Samuelsson wrote:
> Den 2023-04-28 kl. 15:04, skrev David Brown:
>> On 28/04/2023 10:50, Ulf Samuelsson wrote:
>>> Den 2023-04-28 kl. 09:38, skrev David Brown:

>>>>
>>>> Or for my preferences, the CRC "DIGEST" would be put at the end of
>>>> the image, rather than near the start.  Then the "from, to" range
>>>> would cover the entire image except for the final CRC.  But I'd have
>>>> a similar directive for the length of the image at a specific area
>>>> near the start.
>>>>
>>>
>>> I really do not see a benefit of splitting the meta information about
>>> the image to two separate locations.
>>>
>>> The bootloader uses the struct for all checks.
>>> It is a much simpler implementation once the tools support it.
>>>
>>> You might find it easier to write a tool which adds the CRC at the
>>> end, but that is a different issue.
>>>
>>> Occam's Razor!
>>>
>>
>> There are different needs for different projects - and more than one
>> way to handle them.  I find adding a CRC at the end of the image works
>> best for me, but I have no problem appreciating that other people have
>> different solutions.
>>
>>
>>
>>
> I'd be curious to know WHY it works best for you.
> /Ulf

I regularly do not have a bootloader - I am not free to put a CRC at the
start of the image. And if the bootloader itself needs to be updatable,
it is again impossible to have the CRC (or any other metadata) at the
start of the image. I want most of the metadata to be at a fixed
location as close to the start as reasonably practical (such as after
the vector table, or other microcontroller-specific information that
might be used for flash security, early chip setup, etc.). If I am to
have one single checksum for the image, which is what I prefer, then it
has to be at the end of the image. For example, there might be :

0x00000000 : vectors
0x00000400 : external flash configuration block
0x00000600 : program info metadata
0x00001000 : main program
: CRC

There is no way to have the metadata or CRC at the start of the image,
so the CRC goes at the end.

It would be possible to have two CRCs - one that covers the vectors,
configuration information, and metadata and is placed second last in the
metadata block. A second CRC placed last in the metadata block would
cover the main program - everything after the CRCs. That would let me
have a single metadata block and no CRC at the end of the image.
However, it would mean splitting the check in two, rather than one check
for the whole image. I don't see that as a benefit.

When making images that are started from a bootloader, I certainly
/could/ put the CRC at the start. But I see no particular reason to do
so - it makes a lot more sense to keep a similar format.

(Bootloaders don't often have to check their own CRC - after all, even
if the CRC fails there is usually little you can do about it, except
charge on and hope for the best. But if the bootloader is updatable in
system, then you want a CRC during the download procedure to check that
you have got a good download copy before updating the flash.)

Re: Embedding a Checksum in an Image File

<u2t1mh$16mg5$1@dont-email.me>

 copy mid

https://www.novabbs.com/devel/article-flat.php?id=1549&group=comp.arch.embedded#1549

 copy link   Newsgroups: comp.arch.embedded
Path: i2pn2.org!i2pn.org!eternal-september.org!news.eternal-september.org!.POSTED!not-for-mail
From: blockedo...@foo.invalid (Don Y)
Newsgroups: comp.arch.embedded
Subject: Re: Embedding a Checksum in an Image File
Date: Wed, 3 May 2023 00:15:27 -0700
Organization: A noiseless patient Spider
Lines: 426
Message-ID: <u2t1mh$16mg5$1@dont-email.me>
References: <116ff07e-5e25-469a-90a0-9474108aadd3n@googlegroups.com>
<pvl24i57aef4vc7bbdk9mvj7sic9dsh64t@4ax.com>
<f0afa198-e735-4da1-a16a-82764af3de4dn@googlegroups.com>
<36534il81ipvnhog6980r9ln9tdqn5cbh6@4ax.com> <u1t7eb$10gmu$3@dont-email.me>
<u1tpcr$1377f$1@dont-email.me> <u1tsm3$13ook$1@dont-email.me>
<u1u7ro$2poss$1@dont-email.me> <u1v9q8$2v4d5$1@dont-email.me>
<u20sli$3ag7l$1@dont-email.me> <u25bas$8fh0$2@dont-email.me>
<u2646q$clg2$1@dont-email.me>
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 8bit
Injection-Date: Wed, 3 May 2023 07:15:29 -0000 (UTC)
Injection-Info: dont-email.me; posting-host="20dc3c74eeb35238b7b9d38da8ff32e5";
logging-data="1268229"; mail-complaints-to="abuse@eternal-september.org"; posting-account="U2FsdGVkX18IFf23wNrDaU5ONK5BbKlJ"
User-Agent: Mozilla/5.0 (Windows NT 6.1; Win64; x64; rv:102.0) Gecko/20100101
Thunderbird/102.2.2
Cancel-Lock: sha1:Pz2wizWn1B6bxzuWMeWn/F15uDc=
Content-Language: en-US
In-Reply-To: <u2646q$clg2$1@dont-email.me>
 by: Don Y - Wed, 3 May 2023 07:15 UTC

On 4/24/2023 7:37 AM, David Brown wrote:
> On 24/04/2023 09:32, Don Y wrote:
>> On 4/22/2023 7:57 AM, David Brown wrote:
>>>>> However, in almost every case where CRC's might be useful, you have
>>>>> additional checks of the sanity of the data, and an all-zero or all-one
>>>>> data block would be rejected.  For example, Ethernet packets use CRC for
>>>>> integrity checking, but an attempt to send a packet type 0 from MAC
>>>>> address 00:00:00:00:00:00 to address 00:00:00:00:00:00, of length 0, would
>>>>> be rejected anyway.
>>>>
>>>> Why look at "data" -- which may be suspect -- and *then* check its CRC?
>>>> Run the CRC first.  If it fails, decide how you are going to proceed
>>>> or recover.
>>>
>>> That is usually the order, yes.  Sometimes you want "fail fast", such as
>>> dropping a packet that was not addressed to you (it doesn't matter if it was
>>> received correctly but for someone else, or it was addressed to you but the
>>> receiver address was corrupted - you are dropping the packet either way).
>>> But usually you will run the CRC then look at the data.
>>>
>>> But the order doesn't matter - either way, you are still checking for valid
>>> data, and if the data is invalid, it does not matter if the CRC only passed
>>> by luck or by all zeros.
>>
>> You're assuming the CRC is supposed to *vouch* for the data.
>> The CRC can be there simply to vouch for the *transport* of a
>> datagram.
>
> I am assuming that the CRC is there to determine the integrity of the data in
> the face of possible unintentional errors.  That's what CRC checks are for.
> They have nothing to do with the content of the data, or the type of the data
> package or image.

Exactly. And, a CRC on *a* protocol can use ANY ALGORITHM that the protocol
defines. Not some "canned one-size fits all" approach.

> As an example of the use of CRC's in messaging, look at Ethernet frames:
>
> <https://en.wikipedia.org/wiki/Ethernet_frame>
>
> The CRC  does not care about the content of the data it protects.

AND, if the packet yielded an incorrect CRC, you can assume the
data was corrupt... OR, you are looking at a different protocol
and MISTAKING it for something that you *think* it might be.

If I produce a stream of data, can you tell me what the checksum
for THAT stream *should* be? You have to either be told what
it is (and have a way of knowing what the checksum SHOULD be)
*or* have to make some assumptions about it.

If you have assumed wrong *or* if the data has been corrupt, then
the CRC should fail. You don't care why it failed -- because you
can't do anything about it. You just know that you can't use the data
in the way you THOUGHT it could be used.

>> So, use a version-specific CRC on the packet.  If it fails, then
>> either the data in the packet has been corrupted (which could just
>> as easily have involved an embedded "interface version" parameter);
>> or the packet was formed with the wrong CRC.
>>
>> If the CRC is correct FOR THAT VERSION OF THE PROTOCOL, then
>> why bother looking at a "protocol version" parameter?  Would
>> you ALSO want to verify all the rest of the parameters?
>
> I'm sorry, I simply cannot see your point.  Identifying the version of a
> protocol, or other protocol type information, is a totally orthogonal task to
> ensuring the integrity of the data.  The concepts should be handled separately.

It is. A packet using protocol XYZ is delivered to port ABC.
Port ABC *only* handles protocol XYZ. Anything else arriving there,
with a potentially different checksum, is invalid. Even if, for example,
byte number 27 happens to have the correct "magic number" for that
protocol.

Because the message doesn't obey the rules defined by the protocol
FOR THAT PORT. What do I gain by insisting that byte number 27 must
be 0x5A that the CRC doesn't already tell me?

You are assuming the CRC has to identify the protocol. I didn't say that.
All I said was the CRC has to be correct for THAT protocol.

You likely don't use the same algorithm to compute the checksum of
a boot image as you do to verify the integrity of a ethernet datagram.
So, if you were presented with a stream of data, you wouldn't
arbitrarily decide to try different CRCs to see which yielded correct
results and, from that, *infer* the nature of the message.

Why would you think I wouldn't expect *a* particular protocol to use
a particular CRC?

>>>> What term would you have me use to indicate a "bias" applied to a CRC
>>>> algorithm?
>>>
>>> Well, first I'd note that any kind of modification to the basic CRC
>>> algorithm is pointless from the viewpoint of its use as an integrity check.
>>> (There have been, mostly historically, some justifications in terms of
>>> implementation efficiency.  For example, bit and byte re-ordering could be
>>> done to suit hardware bit-wise implementations.)
>>>
>>> Otherwise I'd say you are picking a specific initial value if that is what
>>> you are doing, or modifying the final value (inverting it or xor'ing it with
>>> a fixed value).  There is, AFAIK, no specific terms for these - and I don't
>>> see any benefit in having one.  Misusing the term "salt" from cryptography
>>> is certainly not helpful.
>>
>> Salt just ensures that you can differentiate between functionally identical
>> values.  I.e., in a CRC, it differentiates between the "0x0000" that CRC-1
>> generates from the "0x0000" that CRC-2 generates.
>
> Can we agree that this is called an "initial value", not "salt" ?

It depends on how you implement it. The point is to produce
different results for the same polynmomial.

>> You don't see the parallel to ensuring that *my* use of "Passw0rd" is
>> encoded in a different manner than *your* use of "Passw0rd"?
>
> No.  They are different things.
>
> An important difference is that adding "salt" to a password hash is an
> important security feature.  Picking a different initial value for a CRC
> instead of having appropriate protocol versioning in the data (or a surrounding
> envelope) is a misfeature.

And you don't see that verifying that a packet of data received at
port ABC that should only see the checksum associated with protocol
XYZ as being similarly related?

Why not just assume the lower level protocols are sufficient to
guarantee reliable delivery and, if something arrives at port ABC
then, by definition, it must be intact (not corrupt) and, as
nothing other than protocol XYZ *should* target that port, why
even bother checking magic numbers in a protocol packet?

You build these *superfluous* tests into products to ensure their
integrity -- by catching ANYTHING that "can't happen" (yet
somehow does)

> The second difference is the purpose of the hashing.  The CRC here is for data
> integrity - spotting mistakes in the data during transfer or storage.  The hash
> in a password is for security, avoiding the password ever being transmitted or
> stored in plain text.
>
> Any coincidence in the the way these might be implemented is just that -
> coincidence.
>
>>>> See the RMI desciption.
>>>
>>> I'm sorry, I have no idea what "RMI" is or where it is described. You've
>>> mentioned that abbreviation twice, but I can't figure it out.
>>
>> <https://en.wikipedia.org/wiki/RMI>
>> <https://en.wikipedia.org/wiki/OCL>
>>
>> Nothing magical with either term.
>
> I looked up RMI on Wikipedia before asking, and saw nothing of relevance to
> CRC's or checksums.

How do you think the marshalled arguments get from device A to (remote)
device B? And, the result(s) from device B back to device A?

Obviously *some* form of communication medium. So, some potential for
data to be corrupted (or altered!) in transit. Along with other
data streams to compete for those endpoints.

Imagine invoking a function and, between the actual construction of the
stack frame and the first line of code in the targeted function, "something"
can interfere with the data you're trying to pass (and results you're
hoping to eventually receive) as well as the actual function being targeted!

You don't worry about this because the compiler handles all of the machinery
AND it relies on the CPU being well-behaved; nothing can sneak in and
disturb the address/data -busses or alter register contents during this
process.


Click here to read the complete article
Re: Embedding a Checksum in an Image File

<u2tl7k$19ji0$1@dont-email.me>

 copy mid

https://www.novabbs.com/devel/article-flat.php?id=1550&group=comp.arch.embedded#1550

 copy link   Newsgroups: comp.arch.embedded
Path: i2pn2.org!i2pn.org!eternal-september.org!news.eternal-september.org!.POSTED!not-for-mail
From: david.br...@hesbynett.no (David Brown)
Newsgroups: comp.arch.embedded
Subject: Re: Embedding a Checksum in an Image File
Date: Wed, 3 May 2023 14:48:52 +0200
Organization: A noiseless patient Spider
Lines: 229
Message-ID: <u2tl7k$19ji0$1@dont-email.me>
References: <116ff07e-5e25-469a-90a0-9474108aadd3n@googlegroups.com>
<pvl24i57aef4vc7bbdk9mvj7sic9dsh64t@4ax.com>
<f0afa198-e735-4da1-a16a-82764af3de4dn@googlegroups.com>
<36534il81ipvnhog6980r9ln9tdqn5cbh6@4ax.com> <u1t7eb$10gmu$3@dont-email.me>
<u1tpcr$1377f$1@dont-email.me> <u1tsm3$13ook$1@dont-email.me>
<u1u7ro$2poss$1@dont-email.me> <u1v9q8$2v4d5$1@dont-email.me>
<u20sli$3ag7l$1@dont-email.me> <u25bas$8fh0$2@dont-email.me>
<u2646q$clg2$1@dont-email.me> <u2t1mh$16mg5$1@dont-email.me>
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 8bit
Injection-Date: Wed, 3 May 2023 12:48:52 -0000 (UTC)
Injection-Info: dont-email.me; posting-host="ea85f716dba8ceaa535642ec1fc343a2";
logging-data="1363520"; mail-complaints-to="abuse@eternal-september.org"; posting-account="U2FsdGVkX1+KQA++eRqVvV4PPtTIaxq+Pi6CYZBT00s="
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101
Thunderbird/102.9.0
Cancel-Lock: sha1:sypVD7aXXD8zvq9ALl+uo101p1w=
Content-Language: en-GB
In-Reply-To: <u2t1mh$16mg5$1@dont-email.me>
 by: David Brown - Wed, 3 May 2023 12:48 UTC

On 03/05/2023 09:15, Don Y wrote:
> On 4/24/2023 7:37 AM, David Brown wrote:
>> On 24/04/2023 09:32, Don Y wrote:
>>> On 4/22/2023 7:57 AM, David Brown wrote:
>>>>>> However, in almost every case where CRC's might be useful, you
>>>>>> have additional checks of the sanity of the data, and an all-zero
>>>>>> or all-one data block would be rejected.  For example, Ethernet
>>>>>> packets use CRC for integrity checking, but an attempt to send a
>>>>>> packet type 0 from MAC address 00:00:00:00:00:00 to address
>>>>>> 00:00:00:00:00:00, of length 0, would be rejected anyway.
>>>>>
>>>>> Why look at "data" -- which may be suspect -- and *then* check its
>>>>> CRC?
>>>>> Run the CRC first.  If it fails, decide how you are going to proceed
>>>>> or recover.
>>>>
>>>> That is usually the order, yes.  Sometimes you want "fail fast",
>>>> such as dropping a packet that was not addressed to you (it doesn't
>>>> matter if it was received correctly but for someone else, or it was
>>>> addressed to you but the receiver address was corrupted - you are
>>>> dropping the packet either way). But usually you will run the CRC
>>>> then look at the data.
>>>>
>>>> But the order doesn't matter - either way, you are still checking
>>>> for valid data, and if the data is invalid, it does not matter if
>>>> the CRC only passed by luck or by all zeros.
>>>
>>> You're assuming the CRC is supposed to *vouch* for the data.
>>> The CRC can be there simply to vouch for the *transport* of a
>>> datagram.
>>
>> I am assuming that the CRC is there to determine the integrity of the
>> data in the face of possible unintentional errors.  That's what CRC
>> checks are for. They have nothing to do with the content of the data,
>> or the type of the data package or image.
>
> Exactly.  And, a CRC on *a* protocol can use ANY ALGORITHM that the
> protocol
> defines.  Not some "canned one-size fits all" approach.

It makes sense to use an 8-bit CRC on small telegrams, 16-bit CRC on
bigger things, 32-bit CRC on flash images, and 64-bit CRC when you want
to use the CRC as an identifying hash (and malicious tampering is
non-existent). There can also be benefits of particular choices of CRC
for particular use-cases, in terms of detection of certain error
patterns for certain lengths of data.

What I don't see any point in is using variations, such as different
initial values. I've already said why I think pathological cases such
as all zero data are normally irrelevant - but I can accept that there
may be occasions when they could happen, and thus a /single/ non-zero
initial value would be useful.

>
>> As an example of the use of CRC's in messaging, look at Ethernet frames:
>>
>> <https://en.wikipedia.org/wiki/Ethernet_frame>
>>
>> The CRC  does not care about the content of the data it protects.
>
> AND, if the packet yielded an incorrect CRC, you can assume the
> data was corrupt... OR, you are looking at a different protocol
> and MISTAKING it for something that you *think* it might be.

If the CRC does not match, you reject the packet or data. End of story.
You don't know or care /why/ - because you cannot be sure of any reason.

>
> If I produce a stream of data, can you tell me what the checksum
> for THAT stream *should* be?  You have to either be told what
> it is (and have a way of knowing what the checksum SHOULD be)
> *or* have to make some assumptions about it.

If you are transmitting some data then both sides need to agree on the
CRC algorithm (size, polynomial, initial value, etc.), and on whether a
check is "CRC of everything gives 0" or "CRC of everything except the
pre-calculated CRC equals the transmitted pre-calculated CRC".

>
> If you have assumed wrong *or* if the data has been corrupt, then
> the CRC should fail.  You don't care why it failed -- because you
> can't do anything about it.  You just know that you can't use the data
> in the way you THOUGHT it could be used.
>

Well, yes. Obviously.

If you are making incorrect assumptions here, someone is doing a pretty
poor job at designing, describing or implementing the communications
system. It is just like getting the baud rate wrong on a UART link.

>>> So, use a version-specific CRC on the packet.  If it fails, then
>>> either the data in the packet has been corrupted (which could just
>>> as easily have involved an embedded "interface version" parameter);
>>> or the packet was formed with the wrong CRC.
>>>
>>> If the CRC is correct FOR THAT VERSION OF THE PROTOCOL, then
>>> why bother looking at a "protocol version" parameter?  Would
>>> you ALSO want to verify all the rest of the parameters?
>>
>> I'm sorry, I simply cannot see your point.  Identifying the version of
>> a protocol, or other protocol type information, is a totally
>> orthogonal task to ensuring the integrity of the data.  The concepts
>> should be handled separately.
>
> It is.  A packet using protocol XYZ is delivered to port ABC.
> Port ABC *only* handles protocol XYZ.  Anything else arriving there,
> with a potentially different checksum, is invalid.  Even if, for example,
> byte number 27 happens to have the correct "magic number" for that
> protocol.
>
> Because the message doesn't obey the rules defined by the protocol
> FOR THAT PORT.  What do I gain by insisting that byte number 27 must
> be 0x5A that the CRC doesn't already tell me?
>

A CRC failure doesn't tell you that the telegram type is wrong. It
tells you that the data is corrupted.

If there can be different protocols, or telegram types, or whatever,
then identify them. Stop playing silly buggers with abuse of different
concepts that have different roles in the communication system.

>>> Salt just ensures that you can differentiate between functionally
>>> identical
>>> values.  I.e., in a CRC, it differentiates between the "0x0000" that
>>> CRC-1
>>> generates from the "0x0000" that CRC-2 generates.
>>
>> Can we agree that this is called an "initial value", not "salt" ?
>
> It depends on how you implement it.  The point is to produce
> different results for the same polynmomial.

It is called an "initial value" - it is not "salt". It doesn't matter
if you want to pick different initial values for your CRC, or why you
want to do that. You are still not talking about salt.

If you insist on using your own terminology, you will be left talking to
yourself.

>
>>> You don't see the parallel to ensuring that *my* use of "Passw0rd" is
>>> encoded in a different manner than *your* use of "Passw0rd"?
>>
>> No.  They are different things.
>>
>> An important difference is that adding "salt" to a password hash is an
>> important security feature.  Picking a different initial value for a
>> CRC instead of having appropriate protocol versioning in the data (or
>> a surrounding envelope) is a misfeature.
>
> And you don't see that verifying that a packet of data received at
> port ABC that should only see the checksum associated with protocol
> XYZ as being similarly related?

No. They are different things.

Look, I /do/ understand what you are doing, and I appreciate that you
think it is a good idea. To me, it is an unpleasant mix of orthogonal
concepts that needlessly complicates things. Just because something is
/possible/, does not mean it is a good idea.

>>
>>>>> See the RMI desciption.
>>>>
>>>> I'm sorry, I have no idea what "RMI" is or where it is described.
>>>> You've mentioned that abbreviation twice, but I can't figure it out.
>>>
>>> <https://en.wikipedia.org/wiki/RMI>
>>> <https://en.wikipedia.org/wiki/OCL>
>>>
>>> Nothing magical with either term.
>>
>> I looked up RMI on Wikipedia before asking, and saw nothing of
>> relevance to CRC's or checksums.
>

I've snipped the ramblings that have nothing to do with the question I
asked. I assume you don't want to answer me.

>
>> I noticed no mention of "OCL" in your posts, and looking
>
> You need to read more carefully.


Click here to read the complete article
Re: Embedding a Checksum in an Image File

<u3e3oa$bcui$1@dont-email.me>

 copy mid

https://www.novabbs.com/devel/article-flat.php?id=1583&group=comp.arch.embedded#1583

 copy link   Newsgroups: comp.arch.embedded
Path: i2pn2.org!i2pn.org!eternal-september.org!news.eternal-september.org!.POSTED!not-for-mail
From: ulf.r.sa...@gmail.com (Ulf Samuelsson)
Newsgroups: comp.arch.embedded
Subject: Re: Embedding a Checksum in an Image File
Date: Tue, 9 May 2023 20:34:42 +0200
Organization: eMagii
Lines: 99
Message-ID: <u3e3oa$bcui$1@dont-email.me>
References: <116ff07e-5e25-469a-90a0-9474108aadd3n@googlegroups.com>
<105b1cab-9e5f-91b3-1de6-99586246ecbd@gmail.com>
<b62a77f7-3a13-4d76-b7cb-c42902595751n@googlegroups.com>
<kavt7iFlvc5U1@mid.individual.net> <u2emsa$22pgk$2@dont-email.me>
<u2ft59$2bnd0$4@dont-email.me> <u2g1bu$2cdfn$3@dont-email.me>
<u2gg9d$2ejus$3@dont-email.me> <u2k0mp$33teb$2@dont-email.me>
<u2ltdt$3i28u$1@dont-email.me>
Reply-To: ulf.r.samuelsson.invalid@gmail.com
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 8bit
Injection-Date: Tue, 9 May 2023 18:34:50 -0000 (UTC)
Injection-Info: dont-email.me; posting-host="4d44c97a992e8e1295ecd6a473e79b59";
logging-data="373714"; mail-complaints-to="abuse@eternal-september.org"; posting-account="U2FsdGVkX188q153wP7RGxoteSpE0Zjk"
User-Agent: Mozilla/5.0 (Windows NT 6.1; Win64; x64; rv:102.0) Gecko/20100101
Thunderbird/102.10.0
Cancel-Lock: sha1:8erY66PwxtRhzPZq5NQRHTrcp94=
In-Reply-To: <u2ltdt$3i28u$1@dont-email.me>
Content-Language: sv
 by: Ulf Samuelsson - Tue, 9 May 2023 18:34 UTC

Den 2023-04-30 kl. 16:19, skrev David Brown:
> On 29/04/2023 23:03, Ulf Samuelsson wrote:
>> Den 2023-04-28 kl. 15:04, skrev David Brown:
>>> On 28/04/2023 10:50, Ulf Samuelsson wrote:
>>>> Den 2023-04-28 kl. 09:38, skrev David Brown:
>
>>>>>
>>>>> Or for my preferences, the CRC "DIGEST" would be put at the end of
>>>>> the image, rather than near the start.  Then the "from, to" range
>>>>> would cover the entire image except for the final CRC.  But I'd
>>>>> have a similar directive for the length of the image at a specific
>>>>> area near the start.
>>>>>
>>>>
>>>> I really do not see a benefit of splitting the meta information
>>>> about the image to two separate locations.
>>>>
>>>> The bootloader uses the struct for all checks.
>>>> It is a much simpler implementation once the tools support it.
>>>>
>>>> You might find it easier to write a tool which adds the CRC at the
>>>> end, but that is a different issue.
>>>>
>>>> Occam's Razor!
>>>>
>>>
>>> There are different needs for different projects - and more than one
>>> way to handle them.  I find adding a CRC at the end of the image
>>> works best for me, but I have no problem appreciating that other
>>> people have different solutions.
>>>
>>>
>>>
>>>
>> I'd be curious to know WHY it works best for you.
>> /Ulf
>
> I regularly do not have a bootloader - I am not free to put a CRC at the
> start of the image.  And if the bootloader itself needs to be updatable,
> it is again impossible to have the CRC (or any other metadata) at the
> start of the image.  I want most of the metadata to be at a fixed
> location as close to the start as reasonably practical (such as after
> the vector table, or other microcontroller-specific information that
> might be used for flash security, early chip setup, etc.).  If I am to
> have one single checksum for the image, which is what I prefer, then it
> has to be at the end of the image.  For example, there might be :
>
> 0x00000000 : vectors
> 0x00000400 : external flash configuration block
> 0x00000600 : program info metadata
> 0x00001000 : main program
>            : CRC
>
> There is no way to have the metadata or CRC at the start of the image,
> so the CRC goes at the end.

For the Bootloader, I keep the CRC right after the vectors.
I keep a copy of the vectors right after the CRC, and compare
the two vector tables.
This is to always know the location of the CRC.

>
> It would be possible to have two CRCs - one that covers the vectors,
> configuration information, and metadata and is placed second last in the
> metadata block.  A second CRC placed last in the metadata block would
> cover the main program - everything after the CRCs.  That would let me
> have a single metadata block and no CRC at the end of the image.
> However, it would mean splitting the check in two, rather than one check
> for the whole image.  I don't see that as a benefit.
>
>
> When making images that are started from a bootloader, I certainly
> /could/ put the CRC at the start.  But I see no particular reason to do
> so - it makes a lot more sense to keep a similar format.
>
You want more metadata like entry point and length, as well as text
information about the image. Putting things in a header means that
location is fixed.
There are a number of checks in my bootloader to ensure that the
information in the header makes sense.

> (Bootloaders don't often have to check their own CRC - after all, even
> if the CRC fails there is usually little you can do about it, except
> charge on and hope for the best.  But if the bootloader is updatable in
> system, then you want a CRC during the download procedure to check that
> you have got a good download copy before updating the flash.)

In functional safety applications you regularily check the flash
contents and refuse to boot if there is a mismatch.

/Ulf
>
>
>
>
>
>
>

Re: Embedding a Checksum in an Image File

<u3e46p$bcui$2@dont-email.me>

 copy mid

https://www.novabbs.com/devel/article-flat.php?id=1584&group=comp.arch.embedded#1584

 copy link   Newsgroups: comp.arch.embedded
Path: i2pn2.org!i2pn.org!eternal-september.org!news.eternal-september.org!.POSTED!not-for-mail
From: ulf.r.sa...@gmail.com (Ulf Samuelsson)
Newsgroups: comp.arch.embedded
Subject: Re: Embedding a Checksum in an Image File
Date: Tue, 9 May 2023 20:42:25 +0200
Organization: eMagii
Lines: 241
Message-ID: <u3e46p$bcui$2@dont-email.me>
References: <116ff07e-5e25-469a-90a0-9474108aadd3n@googlegroups.com>
<pvl24i57aef4vc7bbdk9mvj7sic9dsh64t@4ax.com>
<f0afa198-e735-4da1-a16a-82764af3de4dn@googlegroups.com>
<36534il81ipvnhog6980r9ln9tdqn5cbh6@4ax.com> <u1t7eb$10gmu$3@dont-email.me>
<u1tpcr$1377f$1@dont-email.me> <u1tsm3$13ook$1@dont-email.me>
<u1u7ro$2poss$1@dont-email.me> <u1v9q8$2v4d5$1@dont-email.me>
<u20sli$3ag7l$1@dont-email.me> <u25bas$8fh0$2@dont-email.me>
<u2646q$clg2$1@dont-email.me> <u2t1mh$16mg5$1@dont-email.me>
<u2tl7k$19ji0$1@dont-email.me>
Reply-To: ulf.r.samuelsson.invalid@gmail.com
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 8bit
Injection-Date: Tue, 9 May 2023 18:42:33 -0000 (UTC)
Injection-Info: dont-email.me; posting-host="4d44c97a992e8e1295ecd6a473e79b59";
logging-data="373714"; mail-complaints-to="abuse@eternal-september.org"; posting-account="U2FsdGVkX19KXLxBUC6+BS6PNrWDnKnn"
User-Agent: Mozilla/5.0 (Windows NT 6.1; Win64; x64; rv:102.0) Gecko/20100101
Thunderbird/102.10.0
Cancel-Lock: sha1:kT1jMw5+R4ydMiXR0nxQ7uXJzDA=
Content-Language: sv
In-Reply-To: <u2tl7k$19ji0$1@dont-email.me>
 by: Ulf Samuelsson - Tue, 9 May 2023 18:42 UTC

Den 2023-05-03 kl. 14:48, skrev David Brown:
> On 03/05/2023 09:15, Don Y wrote:
>> On 4/24/2023 7:37 AM, David Brown wrote:
>>> On 24/04/2023 09:32, Don Y wrote:
>>>> On 4/22/2023 7:57 AM, David Brown wrote:
>>>>>>> However, in almost every case where CRC's might be useful, you
>>>>>>> have additional checks of the sanity of the data, and an all-zero
>>>>>>> or all-one data block would be rejected.  For example, Ethernet
>>>>>>> packets use CRC for integrity checking, but an attempt to send a
>>>>>>> packet type 0 from MAC address 00:00:00:00:00:00 to address
>>>>>>> 00:00:00:00:00:00, of length 0, would be rejected anyway.
>>>>>>
>>>>>> Why look at "data" -- which may be suspect -- and *then* check its
>>>>>> CRC?
>>>>>> Run the CRC first.  If it fails, decide how you are going to proceed
>>>>>> or recover.
>>>>>
>>>>> That is usually the order, yes.  Sometimes you want "fail fast",
>>>>> such as dropping a packet that was not addressed to you (it doesn't
>>>>> matter if it was received correctly but for someone else, or it was
>>>>> addressed to you but the receiver address was corrupted - you are
>>>>> dropping the packet either way). But usually you will run the CRC
>>>>> then look at the data.
>>>>>
>>>>> But the order doesn't matter - either way, you are still checking
>>>>> for valid data, and if the data is invalid, it does not matter if
>>>>> the CRC only passed by luck or by all zeros.
>>>>
>>>> You're assuming the CRC is supposed to *vouch* for the data.
>>>> The CRC can be there simply to vouch for the *transport* of a
>>>> datagram.
>>>
>>> I am assuming that the CRC is there to determine the integrity of the
>>> data in the face of possible unintentional errors.  That's what CRC
>>> checks are for. They have nothing to do with the content of the data,
>>> or the type of the data package or image.
>>
>> Exactly.  And, a CRC on *a* protocol can use ANY ALGORITHM that the
>> protocol
>> defines.  Not some "canned one-size fits all" approach.
>
> It makes sense to use an 8-bit CRC on small telegrams, 16-bit CRC on
> bigger things, 32-bit CRC on flash images, and 64-bit CRC when you want
> to use the CRC as an identifying hash (and malicious tampering is
> non-existent).  There can also be benefits of particular choices of CRC
> for particular use-cases, in terms of detection of certain error
> patterns for certain lengths of data.

Flash images larger than X kB may need a 64-bit CRC.
I don't remember exactly when to start considering it,
but something between 64kB-256kB is probably correct.

It is all to do with Hamming Distance, and this is also affected by the
polynome.
/Ulf

>
>
> What I don't see any point in is using variations, such as different
> initial values.  I've already said why I think pathological cases such
> as all zero data are normally irrelevant - but I can accept that there
> may be occasions when they could happen, and thus a /single/ non-zero
> initial value would be useful.
>
>>
>>> As an example of the use of CRC's in messaging, look at Ethernet frames:
>>>
>>> <https://en.wikipedia.org/wiki/Ethernet_frame>
>>>
>>> The CRC  does not care about the content of the data it protects.
>>
>> AND, if the packet yielded an incorrect CRC, you can assume the
>> data was corrupt... OR, you are looking at a different protocol
>> and MISTAKING it for something that you *think* it might be.
>
> If the CRC does not match, you reject the packet or data.  End of story.
>  You don't know or care /why/ - because you cannot be sure of any reason.
>
>>
>> If I produce a stream of data, can you tell me what the checksum
>> for THAT stream *should* be?  You have to either be told what
>> it is (and have a way of knowing what the checksum SHOULD be)
>> *or* have to make some assumptions about it.
>
> If you are transmitting some data then both sides need to agree on the
> CRC algorithm (size, polynomial, initial value, etc.), and on whether a
> check is "CRC of everything gives 0" or "CRC of everything except the
> pre-calculated CRC equals the transmitted pre-calculated CRC".
>
>>
>> If you have assumed wrong *or* if the data has been corrupt, then
>> the CRC should fail.  You don't care why it failed -- because you
>> can't do anything about it.  You just know that you can't use the data
>> in the way you THOUGHT it could be used.
>>
>
> Well, yes.  Obviously.
>
> If you are making incorrect assumptions here, someone is doing a pretty
> poor job at designing, describing or implementing the communications
> system.  It is just like getting the baud rate wrong on a UART link.
>
>
>>>> So, use a version-specific CRC on the packet.  If it fails, then
>>>> either the data in the packet has been corrupted (which could just
>>>> as easily have involved an embedded "interface version" parameter);
>>>> or the packet was formed with the wrong CRC.
>>>>
>>>> If the CRC is correct FOR THAT VERSION OF THE PROTOCOL, then
>>>> why bother looking at a "protocol version" parameter?  Would
>>>> you ALSO want to verify all the rest of the parameters?
>>>
>>> I'm sorry, I simply cannot see your point.  Identifying the version
>>> of a protocol, or other protocol type information, is a totally
>>> orthogonal task to ensuring the integrity of the data.  The concepts
>>> should be handled separately.
>>
>> It is.  A packet using protocol XYZ is delivered to port ABC.
>> Port ABC *only* handles protocol XYZ.  Anything else arriving there,
>> with a potentially different checksum, is invalid.  Even if, for example,
>> byte number 27 happens to have the correct "magic number" for that
>> protocol.
>>
>> Because the message doesn't obey the rules defined by the protocol
>> FOR THAT PORT.  What do I gain by insisting that byte number 27 must
>> be 0x5A that the CRC doesn't already tell me?
>>
>
> A CRC failure doesn't tell you that the telegram type is wrong.  It
> tells you that the data is corrupted.
>
> If there can be different protocols, or telegram types, or whatever,
> then identify them.  Stop playing silly buggers with abuse of different
> concepts that have different roles in the communication system.
>
>
>>>> Salt just ensures that you can differentiate between functionally
>>>> identical
>>>> values.  I.e., in a CRC, it differentiates between the "0x0000" that
>>>> CRC-1
>>>> generates from the "0x0000" that CRC-2 generates.
>>>
>>> Can we agree that this is called an "initial value", not "salt" ?
>>
>> It depends on how you implement it.  The point is to produce
>> different results for the same polynmomial.
>
> It is called an "initial value" - it is not "salt".  It doesn't matter
> if you want to pick different initial values for your CRC, or why you
> want to do that.  You are still not talking about salt.
>
> If you insist on using your own terminology, you will be left talking to
> yourself.
>
>>
>>>> You don't see the parallel to ensuring that *my* use of "Passw0rd" is
>>>> encoded in a different manner than *your* use of "Passw0rd"?
>>>
>>> No.  They are different things.
>>>
>>> An important difference is that adding "salt" to a password hash is
>>> an important security feature.  Picking a different initial value for
>>> a CRC instead of having appropriate protocol versioning in the data
>>> (or a surrounding envelope) is a misfeature.
>>
>> And you don't see that verifying that a packet of data received at
>> port ABC that should only see the checksum associated with protocol
>> XYZ as being similarly related?
>
> No.  They are different things.
>
> Look, I /do/ understand what you are doing, and I appreciate that you
> think it is a good idea.  To me, it is an unpleasant mix of orthogonal
> concepts that needlessly complicates things.  Just because something is
> /possible/, does not mean it is a good idea.
>
>
>>>
>>>>>> See the RMI desciption.
>>>>>
>>>>> I'm sorry, I have no idea what "RMI" is or where it is described.
>>>>> You've mentioned that abbreviation twice, but I can't figure it out.
>>>>
>>>> <https://en.wikipedia.org/wiki/RMI>
>>>> <https://en.wikipedia.org/wiki/OCL>
>>>>
>>>> Nothing magical with either term.
>>>
>>> I looked up RMI on Wikipedia before asking, and saw nothing of
>>> relevance to CRC's or checksums.
>>
>
> I've snipped the ramblings that have nothing to do with the question I
> asked.  I assume you don't want to answer me.
>
>>
>>> I noticed no mention of "OCL" in your posts, and looking
>>
>> You need to read more carefully.
>
> I've looked.  You did not mention "OCL" anywhere before giving the URL
> to the wikipedia page.  You only mentioned it /afterwards/ - without any
> context that suggests what you meant.  (Here's a hint for you - if you
> want to refer to a wikipedia page, put a link to the /relevant/ page.)
>
>
> Presumably "RMI" and "OCL" have particular meanings that are relevant
> for projects you work on, and are so familiar to you that they are part
> of your language.  No one else knows or cares what they are, and they
> are irrelevant in this thread.  So let's leave them there.
>
>>
>>>> Give me the sources for Windows (Linux, *BSD, etc.) and I can
>>>> subvert all the state-of-the-art digital signing used to ensure
>>>> binaries aren't altered.  Nothing *outside* the box is involved
>>>> so, by definition, everything I need has to reside *in* the box.
>>>
>>> No, you can't.  The sources for Linux and *BSD /are/ all freely
>>> available.  The private signing keys used by, for example, Red Hat or
>>> Debian, are /not/ freely available.  You cannot make changes to a Red
>>> Hat or Debian package that will pass the security checks - you are
>>> unable to sign the packages.
>>
>> Sure I can!  If you are just signing a package to verify that it hasn't
>> been tampered with BUT THE CONTENTS ARE NOT ENCRYPTED, then all you have
>> to do is remove the signature check -- leaving the signature in the
>> (unchecked) executable.
>
> Woah, you /really/ don't understand this stuff, do you?  Here's a clue -
> ask yourself what is being signed, and what is doing the checking.
>
> Perhaps also ask yourself if /all/ the people involved in security for
> Linux or BSD - all the companies such as Red Hat, IBM, Intel, etc., -
> ask if /all/ of them have got it wrong, and only /you/ realise that
> digital signatures on open source software is useless?  /Very/
> occasionally, there is a lone genius that understands something while
> all the other experts are wrong - but in most cases, the loner is the
> one that is wrong.
>


Click here to read the complete article
Pages:1234
server_pubkey.txt

rocksolid light 0.9.7
clearnet tor