Janis Papanagnou <janis_papanagnou+ng@hotmail.com> wrote:
> The 'od' tool allows displaying binary data in various formats, but
> it works on a whole data stream (not on individual fields).
> Are there any tools that support a more flexible inspection of binary
> data?
>
> I was thinking of some data specification and a tool to work with that
> specification and binary data files. My current experimental hack has
> a data specification of a form as shown in this example

If I'm following you, then this sounds like a description of
something like GNU Poke:

http://www.jemarch.net/poke

Not something I've had a use for myself since finding out about it
recently, but it seems like a comprehensive solution to the
problem.

--
__ __
#_ < |\| |< _#

On 02.09.2023 00:53, Computer Nerd Kev wrote:
> Janis Papanagnou <janis_papanagnou+ng@hotmail.com> wrote:
>> The 'od' tool allows displaying binary data in various formats, but
>> it works on a whole data stream (not on individual fields).
>> Are there any tools that support a more flexible inspection of binary
>> data?
>>
>> I was thinking of some data specification and a tool to work with that
>> specification and binary data files. My current experimental hack has
>> a data specification of a form as shown in this example
>
> If I'm following you, then this sounds like a description of
> something like GNU Poke:
>
> http://www.jemarch.net/poke
>
> Not something I've had a use for myself since finding out about it
> recently, but it seems like a comprehensive solution to the
> problem.

This is really overwhelming! - Indeed it seems to cover what I was
looking for, but yet much much more; a complete programming language
with control constructs and exception handling, just to name one big
part of the package. So I'm not quite decided that it's what I'd use.
I certainly don't want to write a program[*] to extract some data,
for my purpose the advertised declarative approach[**] would be it.
I'll have to work through the docs to see whether some basic features
are actually supported (e.g. I'm not sure whether simple fixed length
strings (without \0 termination) are supported; I suppose they are,
but some statement I read in the docs made me cautious, so I'll have
to see). - All in all an interesting tool, so thanks for the link!

BTW, in the poke docs I saw examples WRT endian'ness, like the spec
little int a;
big int b;
int c;
In the past I've assumed that endian'ness is a machine characteristic
and would not change within a protocol element. The example taken from
the poke docs suggests that there may be different elements. Of course
we can think about different payload data in a single protocol element,
but is that usual? - I'm coming from the ITU-T ASN.1/BER perspective,
where the ASN.1 data spec is agnostic and endian'ness should happen
in the encoding and decoding process for a specific source and target
architecture. - The answer would lead either to a data spec (like in
poke) to specify that property separately with every data element, or
as a single parameter for the processing.

Janis

[*] An example can be found in the poke docs:
http://www.jemarch.net/poke-3.3-manual/poke.html#elfextractor

[**] http://www.jemarch.net/poke-3.3-manual/poke.html#Motivation

Janis Papanagnou <janis_papanagnou+ng@hotmail.com> wrote:
> BTW, in the poke docs I saw examples WRT endian'ness, like the spec
> little int a;
> big int b;
> int c;
> In the past I've assumed that endian'ness is a machine characteristic
> and would not change within a protocol element. The example taken from
> the poke docs suggests that there may be different elements. Of course
> we can think about different payload data in a single protocol element,
> but is that usual?

I can't speak to what's "usual" in a general sense, but one example
that comes to mind is working on a firmware file that's intended to
be programmed to a device by another system. It could have
information for the programming system stored in that system's byte
order, while the actual data to be written will use the endianness
of the device (or whatever reads it later).

--
__ __
#_ < |\| |< _#

On 2023-09-02, Janis Papanagnou <janis_papanagnou+ng@hotmail.com> wrote:
> BTW, in the poke docs I saw examples WRT endian'ness, like the spec
> little int a;
> big int b;

"big endian int" is nonsensical, which detracts from the example.

Endian specifications only make sense on exact sized types like int16,
uint32 or int64.

"int" is a local concept: matching this system's principal compiler's
"int", which is referenced in the system ABI.

If we are dealing with external data---which we must be, if we are
concerned with byte order---that data doesn't care what our local "int"
is. We don't want the extraction code to break with a different "int".

Tus, if we're commiting to a byte order, we should commit to the number
of bytes which constitute that order.

--
TXR Programming Language: http://nongnu.org/txr
Cygnal: Cygwin Native Application Library: http://kylheku.com/cygnal
Mastodon: @Kazinator@mstdn.ca

On 03.09.2023 08:25, Kaz Kylheku wrote:
> On 2023-09-02, Janis Papanagnou <janis_papanagnou+ng@hotmail.com> wrote:
>> BTW, in the poke docs I saw examples WRT endian'ness, like the spec
>> little int a;
>> big int b;
>
> "big endian int" is nonsensical, which detracts from the example.
>
> Endian specifications only make sense on exact sized types like int16,
> uint32 or int64.

I cannot speak for the 'poke' package; maybe 'int' is just a shortcut
for the 'int' type of the concrete machine where it is running.
Similar to the "int c" declaration (from my quote upthread) that is
assuming (as far as I recall) some default endian'ness on the machine.

Janis

> [...]