On 21/04/2023 13:39, Don Y wrote:
> On 4/21/2023 3:43 AM, David Brown wrote:
>>> Note that you want to choose a polynomial that doesn't
>>> give you a "win" result for "obviously" corrupt data.
>>> E.g., if data is all zeros or all 0xFF (as these sorts of
>>> conditions can happen with hardware failures) you probably
>>> wouldn't want a "success" indication!
>>
>> No, that is pointless for something like a code image.  It just adds
>> needless complexity to your CRC algorithm.
>
> Perhaps you've forgotten that you don't just use CRCs (secure hashes, etc.)
> on "code images"?

No - but "code images" is the topic here.

However, in almost every case where CRC's might be useful, you have
additional checks of the sanity of the data, and an all-zero or all-one
data block would be rejected. For example, Ethernet packets use CRC for
integrity checking, but an attempt to send a packet type 0 from MAC
address 00:00:00:00:00:00 to address 00:00:00:00:00:00, of length 0,
would be rejected anyway.

I can't think of any use-cases where you would be passing around a block
of "pure" data that could reasonably take absolutely any value, without
any type of "envelope" information, and where you would think a CRC
check is appropriate.

>
>> You should already have checks that would eliminate an all-zero image
>> or other "obviously corrupt" data.  You'll be checking the image for a
>> key or "magic number" that identifies the image as "program image for
>> board X, project Y". You'll be checking version numbers.  You'll be
>> reading the length of the image so you know the range for your CRC
>> function, and where to find the appended CRC check.  You might not
>> have all of these in a given system, but you'll have some kind of
>> check which would fail on an all-zero image.
>
> See above.

See above.

>
>>> You can also "salt" the calculation so that the residual
>>> is deliberately nonzero.  So, for example, "success" is
>>> indicated by a residual of 0x474E.  :>
>>
>> Again, pointless.
>>
>> Salt is important for security-related hashes (like password hashes),
>> not for integrity checks.
>
> You've missed the point.  The correct "sum" can be anything.
> Why is "0" more special than any other value?  As the value is
> typically meaningless to anything other than the code that verifies
> it, you couldn't look at an image (or the output of the verifier)
> and gain anything from seeing that obscure value.

Do you actually know what is meant by "salt" in the context of hashes,
and why it is useful in some circumstances? Do you understand that
"salt" is added (usually prepended, or occasionally mixed in in some
other way) to the data /before/ the hash is calculated?

I have not given the slightest indication to suggest that "0" is a
special value. I fully agree that the value you get from the checking
algorithm does not have to be 0 - I already suggested it could be
compared to the stored value. I.e., your build your image file as "data
++ crc(data)", at check it by re-calculating "crc(data)" on the received
image and comparing the result to the received crc. There is no
necessity or benefit in having a crc run calculated over the received
data plus the received crc being 0.

"Salt" is used in cases where the original data must be kept secret, and
only the hashes are transmitted or accessible - by adding salt to the
original data before hashing it, you avoid a direct correspondence
between the hash and the original data. The prime use-case is to stop
people being able to figure out a password by looking up the hash in a
list of pre-computed hashes of common passwords.

>
> OTOH, if the CRC yields something familiar -- or useful -- then
> it can tell you something about the image.  E.g., salt the algorithm
> with the product code, version number, your initials, 0xDEADBEEF, etc.
>

You are making no sense at all. Are you suggesting that it would be a
good idea to add some value to the start of the image so that the
resulting crc calculation gives a nice recognisable product code? This
"salt" would be different for each program image, and calculated by
trial and error. If you want a product code, version number, etc., in
the program image (and it's a good idea), just put these in the program
image!

>>>> So now you have a new extended block   |....data....|crc|
>>>>
>>>> Now if you compute a new CRC on the extended block, the resulting
>>>> value /should/ come out to zero. If it doesn't, either your data or
>>>> the original CRC value appended to it has been changed/corrupted.
>>>
>>> As there is usually a lack of originality in the algorithms
>>> chosen, you have to consider if you are also hoping to use
>>> this to safeguard the *integrity* of your image (i.e.,
>>> against intentional modification).
>>
>> "Integrity" has nothing to do with the motivation for change.
>> /Security/ is concerned with intentional modifications that
>> deliberately attempt to defeat /integrity/ checks.  Integrity is about
>> detecting any changes.
>>
>> If you are concerned about the possibility of intentional malicious
>> changes,
>
> Changes don't have to be malicious.

Accidental changes (such as human error, noise during data transfer,
memory cell errors, etc.) do not pass integrity tests unnoticed. To be
more accurate, the chances of them passing unnoticed are of the order of
1 in 2^n, for a good n-bit check such as a CRC check. Certain types of
error are always detectable, such as single and double bit errors. That
is the point of using a checksum or hash for integrity checking.

/Intentional/ changes are a different matter. If a hacker changes the
program image, they can change the transmitted hash to their own
calculated hash. Or for a small CRC, they could change a different part
of the image until the original checksum matched - for a 16-bit CRC,
that only takes 65,535 attempts in the worst case.

That is why you need to distinguish between the two possibilities. If
you don't have to worry about malicious attacks, a 32-bit CRC takes a
dozen lines of C code and a 1 KB table, all running extremely
efficiently. If security is an issue, you need digital signatures - an
RSA-based signature system is orders of magnitude more effort in both
development time and in run time.

> I altered the test procedure for a
> piece of military gear we were building simply to skip some lengthy
> tests that I *knew* would pass (I don't want to inject an extra 20
> minutes of wait time
> just to get through a lengthy test I already know works before I can get
> to the test of interest to me, now.
>
> I failed to undo the change before the official signoff on the device.
>
> The only evidence of this was the fact that I had also patched the
> startup message to say "Go for coffee..." -- which remained on the
> screen for the duration of the lengthy (even with the long test
> elided) procedure...
>
> ..which alerted folks to the fact that this *probably* wasn't the
> original image.  (The computer running the test suite on the DUT had
> no problem accepting my patched binary)

And what, exactly, do you think that anecdote tells us about CRC checks
for image files? It reminds us that we are all fallible, but does no
more than that.

>
>> CRC's alone are useless.  All the attacker needs to do after modifying
>> the image is calculate the CRC themselves, and replace the original
>> checksum with their own.
>
> That assumes the "alterer" knows how to replace the checksum, how it
> is computed, where it is embedded in the image, etc.  I modified the Compaq
> portable mentioned without ever knowing where the checksum was store
> or *if* it was explicitly stored.  I had no desire to disassemble the
> BIOS ROMs (though could obviously do so as there was no "proprietary
> hardware" limiting access to their contents and the instruction set of
> the processor is well known!).
>
> Instead, I did this by *guessing* how they would implement such a check
> in a bit of kit from that era (ERPOMs aren't easily modified by malware
> so it wasn't likely that they would go to great lengths to "protect" the
> image).  And, if my guess had been incorrect, I could always reinstall
> the original EPROMs -- nothing lost, nothing gained.
>
> Had much experience with folks counterfeiting your products and making
> "simple" changes to the binaries?  Like changing the copyright notice
> or splash screen?
>
> Then, bringing the (accused) counterfeit of YOUR product into a courtroom
> and revealing the *hidden* checksum that the counterfeiter wasn't aware of?
>
> "Gee, why does YOUR (alleged) device have *my* name in it -- in addition
> to behaving exactly like mine??"
>
> [I guess obscurity has its place!]

Security by obscurity is not security. Having a hidden signature or
other mark can be useful for proving ownership (making an intentional
mistake is another common tactic - such as commercial maps having a few
subtle spelling errors). But that is not security.

>
> Use a non-secret approach and you invite folks to alter it, as well.
>
>> Using non-standard algorithms for security is a simple way to get
>> things completely wrong.  "Security by obscurity" is very rarely the
>> right answer.  In reality, good security algorithms, and good
>> implementations, are difficult and specialised tasks, best left to
>> people who know what they are doing.
>>
>> To make something secure, you have to ensure that the check algorithms
>> depend on a key that you know, but that the attacker does not have.
>> That's the basis of digital signatures (though you use a secure hash
>> algorithm rather than a simple CRC).
>
> If you can remove the check, then what value the key's secrecy?  By your
> criteria, the adversary KNOWS how you are implementing your security
> so he knows exactly what to remove to bypass your checks and allow his
> altered image to operate in its place.
>
> Ever notice how manufacturers don't PUBLICLY disclose their security
> hooks (without an NDA)?  If "security by obscurity" was not important,
> they would publish these details INVITING challenges (instead of
> trying to limit the knowledge to people with whom they've officially
> contracted).
>

Any serious manufacturer /does/ invite challenges to their security.

There are multiple reasons why a manufacturer (such as a semiconductor
manufacturer) might be guarded about the details of their security
systems. They can be avoiding giving hints to competitors. Maybe they
know their systems aren't really very secure, because their keys are too
short or they can be read out in some way.

But I think the main reasons are often:

They want to be able to change the details, and that's far easier if
there are only a few people who have read the information.

They don't want endless support questions from amateurs.

They are limited by idiotic government export restrictions made by
ignorant politicians who don't understand cryptography.

Some things benefit from being kept hidden, or under restricted access.
The details of the CRC algorithm you use to catch accidental errors in
your image file is /not/ one of them. If you think hiding it has the
remotest hint of a benefit, you are doing things wrong - you need a
/security/ check, not a simple /integrity/ check.

And then once you have switched to a security check - a digital
signature - there's no need to keep that choice hidden either, because
it is the /key/ that is important, not the type of lock.