I'm thinking of writing a routine to convert JSON to C structures in C source
code. (It would go in the Baby X resource compiler as the first use case).

So it takes JSON and spits out a C source file with a structure declaration
and initial elements in a following definition.

I can see some obvious difficulties, for instance JSON allows for arrays
of mixed types. Then a JSON object has an identier, but not a type name,
whilst C structures require type name.

Anyone any ideas about how to proceed?

In article <8d514df1-31eb-42ae-a1e1-d204b98d89b0n@googlegroups.com>,
Malcolm McLean <malcolm.arthur.mclean@gmail.com> wrote:
>I'm thinking of writing a routine to convert JSON to C structures in C source
>code. (It would go in the Baby X resource compiler as the first use case).
>
>So it takes JSON and spits out a C source file with a structure declaration
>and initial elements in a following definition.
>
>I can see some obvious difficulties, for instance JSON allows for arrays
>of mixed types. Then a JSON object has an identier, but not a type name,
>whilst C structures require type name.
>
>Anyone any ideas about how to proceed?

An obvious suggestion at this point would be to ask if you've looked at the
"jq" program. I'm guessing that "jq" does something pretty similar to what
you're suggesting. Sounds like a good starting point to me.

--
The randomly chosen signature file that would have appeared here is more than 4
lines long. As such, it violates one or more Usenet RFCs. In order to remain
in compliance with said RFCs, the actual sig can be found at the following URL:
http://user.xmission.com/~gazelle/Sigs/TedCruz

On 2022-04-01, Malcolm McLean <malcolm.arthur.mclean@gmail.com> wrote:
> I'm thinking of writing a routine to convert JSON to C structures in C source
> code. (It would go in the Baby X resource compiler as the first use case).
>
> So it takes JSON and spits out a C source file with a structure declaration
> and initial elements in a following definition.
>
> I can see some obvious difficulties, for instance JSON allows for arrays
> of mixed types. Then a JSON object has an identier, but not a type name,
> whilst C structures require type name.
>
> Anyone any ideas about how to proceed?

C99 designated initializers handle unions

jnode j0023 = { .type = jnode_num, .val = { .num = 3.14 ] };
jnode j0024 = { .type = jnode_str, .val = { .str = "abc" ] };
jnode j0025 = { .type = jnode_true };
jnode j0026 = { .type = jnode_false };

jnode *ja0027[4] = { &j0023, &j0024, &j0025, &j0026 };

jnode j0027 = { .type = jnode_vec,
.val = { .vec = { .data = ja0027, .len = 4 } } };

where jnode is something like

enum jnode_type { jnode_false, jnode_true, ..., jnode_num, ...}

typedef struct jnode {
enum jnode_type type;
union {
double num;
char *str;
struct {
struct jnode *data;
size_t len;
} vec;
} val;
} jnode;

--
TXR Programming Language: http://nongnu.org/txr
Cygnal: Cygwin Native Application Library: http://kylheku.com/cygnal

On Friday, 1 April 2022 at 16:06:56 UTC+1, Kaz Kylheku wrote:
> On 2022-04-01, Malcolm McLean <malcolm.ar...@gmail.com> wrote:
> > I'm thinking of writing a routine to convert JSON to C structures in C source
> > code. (It would go in the Baby X resource compiler as the first use case).
> >
> > So it takes JSON and spits out a C source file with a structure declaration
> > and initial elements in a following definition.
> >
> > I can see some obvious difficulties, for instance JSON allows for arrays
> > of mixed types. Then a JSON object has an identier, but not a type name,
> > whilst C structures require type name.
> >
> > Anyone any ideas about how to proceed?
> C99 designated initializers handle unions
>
> jnode j0023 = { .type = jnode_num, .val = { .num = 3.14 ] };
> jnode j0024 = { .type = jnode_str, .val = { .str = "abc" ] };
> jnode j0025 = { .type = jnode_true };
> jnode j0026 = { .type = jnode_false };
>
> jnode *ja0027[4] = { &j0023, &j0024, &j0025, &j0026 };
>
> jnode j0027 = { .type = jnode_vec,
> .val = { .vec = { .data = ja0027, .len = 4 } } };
>
>
> where jnode is something like
>
> enum jnode_type { jnode_false, jnode_true, ..., jnode_num, ...}
>
> typedef struct jnode {
> enum jnode_type type;
> union {
> double num;
> char *str;
> struct {
> struct jnode *data;
> size_t len;
> } vec;
> } val;
> } jnode;
>
That's to build a JSON parser in C. I already have one of those.

What I was thinking of was a program along the lines of

input
{ "name" : "Fred Bloggs",
"salary": 32000.0,
"id": 1234,
"hours_worked" : [8, 8, 7, 8, 10]
}

output

typedef struct
{ char *name;
double salary;
int id;
int hours_worked[5];
} EMPLOYEE;

EMPLOYEE fred {"Fred Blogs", 32000.0, 1234, 8, 8,7, 8, 10]};

Obviously the first issue is that there is no way a computer can get from that JSON to "EMPLOYEE".
The other issue is that whilst the scalar fields are simple, the array fields start opening up more and
more difficulties.

On 4/1/2022 5:19 PM, Malcolm McLean wrote:
> On Friday, 1 April 2022 at 16:06:56 UTC+1, Kaz Kylheku wrote:
>> On 2022-04-01, Malcolm McLean <malcolm.ar...@gmail.com> wrote:
>>> I'm thinking of writing a routine to convert JSON to C structures in C source
>>> code. (It would go in the Baby X resource compiler as the first use case).
>>>
>>> So it takes JSON and spits out a C source file with a structure declaration
>>> and initial elements in a following definition.
>>>
>>> I can see some obvious difficulties, for instance JSON allows for arrays
>>> of mixed types. Then a JSON object has an identier, but not a type name,
>>> whilst C structures require type name.
>>>
>>> Anyone any ideas about how to proceed?
>> C99 designated initializers handle unions
>>
>> jnode j0023 = { .type = jnode_num, .val = { .num = 3.14 ] };
>> jnode j0024 = { .type = jnode_str, .val = { .str = "abc" ] };
>> jnode j0025 = { .type = jnode_true };
>> jnode j0026 = { .type = jnode_false };
>>
>> jnode *ja0027[4] = { &j0023, &j0024, &j0025, &j0026 };
>>
>> jnode j0027 = { .type = jnode_vec,
>> .val = { .vec = { .data = ja0027, .len = 4 } } };
>>
>>
>> where jnode is something like
>>
>> enum jnode_type { jnode_false, jnode_true, ..., jnode_num, ...}
>>
>> typedef struct jnode {
>> enum jnode_type type;
>> union {
>> double num;
>> char *str;
>> struct {
>> struct jnode *data;
>> size_t len;
>> } vec;
>> } val;
>> } jnode;
>>
> That's to build a JSON parser in C. I already have one of those.
>
> What I was thinking of was a program along the lines of
>
> input
> {
> "name" : "Fred Bloggs",
> "salary": 32000.0,
> "id": 1234,
> "hours_worked" : [8, 8, 7, 8, 10]
> }
>
> output
>
> typedef struct
> {
> char *name;
> double salary;
> int id;
> int hours_worked[5];
> } EMPLOYEE;
>
> EMPLOYEE fred {"Fred Blogs", 32000.0, 1234, 8, 8,7, 8, 10]};
>
> Obviously the first issue is that there is no way a computer can get from that JSON to "EMPLOYEE".

Well, in fact there is, it's about dynamic types and introspection,
which is easy for interpreted languages, but not as much in C.

I believe one thing to consider is /how/ "EMPLOYEE" is going to be used
in the rest of the code, and how the rest of the code is supposed to
know about this type.
Answers to these questions will probably shed light on how this type
should be generated as well.

> The other issue is that whilst the scalar fields are simple, the array fields start opening up more and
> more difficulties.

I believe this part is a different perspective on the same problem as
the first one.

On Friday, 1 April 2022 at 17:13:44 UTC+1, Manfred wrote:
> On 4/1/2022 5:19 PM, Malcolm McLean wrote:
> > On Friday, 1 April 2022 at 16:06:56 UTC+1, Kaz Kylheku wrote:
> >> On 2022-04-01, Malcolm McLean <malcolm.ar...@gmail.com> wrote:
> >>> I'm thinking of writing a routine to convert JSON to C structures in C source
> >>> code. (It would go in the Baby X resource compiler as the first use case).
> >>>
> >>> So it takes JSON and spits out a C source file with a structure declaration
> >>> and initial elements in a following definition.
> >>>
> >>> I can see some obvious difficulties, for instance JSON allows for arrays
> >>> of mixed types. Then a JSON object has an identier, but not a type name,
> >>> whilst C structures require type name.
> >>>
> >>> Anyone any ideas about how to proceed?
> >> C99 designated initializers handle unions
> >>
> >> jnode j0023 = { .type = jnode_num, .val = { .num = 3.14 ] };
> >> jnode j0024 = { .type = jnode_str, .val = { .str = "abc" ] };
> >> jnode j0025 = { .type = jnode_true };
> >> jnode j0026 = { .type = jnode_false };
> >>
> >> jnode *ja0027[4] = { &j0023, &j0024, &j0025, &j0026 };
> >>
> >> jnode j0027 = { .type = jnode_vec,
> >> .val = { .vec = { .data = ja0027, .len = 4 } } };
> >>
> >>
> >> where jnode is something like
> >>
> >> enum jnode_type { jnode_false, jnode_true, ..., jnode_num, ...}
> >>
> >> typedef struct jnode {
> >> enum jnode_type type;
> >> union {
> >> double num;
> >> char *str;
> >> struct {
> >> struct jnode *data;
> >> size_t len;
> >> } vec;
> >> } val;
> >> } jnode;
> >>
> > That's to build a JSON parser in C. I already have one of those.
> >
> > What I was thinking of was a program along the lines of
> >
> > input
> > {
> > "name" : "Fred Bloggs",
> > "salary": 32000.0,
> > "id": 1234,
> > "hours_worked" : [8, 8, 7, 8, 10]
> > }
> >
> > output
> >
> > typedef struct
> > {
> > char *name;
> > double salary;
> > int id;
> > int hours_worked[5];
> > } EMPLOYEE;
> >
> > EMPLOYEE fred {"Fred Blogs", 32000.0, 1234, 8, 8,7, 8, 10]};
> >
> > Obviously the first issue is that there is no way a computer can get from that JSON to "EMPLOYEE".
> Well, in fact there is, it's about dynamic types and introspection,
> which is easy for interpreted languages, but not as much in C.
>
> I believe one thing to consider is /how/ "EMPLOYEE" is going to be used
> in the rest of the code, and how the rest of the code is supposed to
> know about this type.
> Answers to these questions will probably shed light on how this type
> should be generated as well.
> > The other issue is that whilst the scalar fields are simple, the array fields start opening up more and
> > more difficulties.
> I believe this part is a different perspective on the same problem as
> the first one.
>
The Baby X resource compiler is basically a way of getting images into C programs, by
dumping them as RGBA quads in C source. However it does other things as well, for
example there's an audio resampler in there, and it can escape strings to create
text resources.
One thing someone might want to do is import non-image data. So the question is,
what format would that data come in, and in what format should it be output to
make it easy for someone writing a program to integrate the data into their
source code?
One format for data is JSON. A generic JSON structure with nodes would work, but it
wouldn't be easy to use. I want a flat array of C structures with easy to access fields.

On 2022-04-01, Malcolm McLean <malcolm.arthur.mclean@gmail.com> wrote:
> On Friday, 1 April 2022 at 16:06:56 UTC+1, Kaz Kylheku wrote:
>> On 2022-04-01, Malcolm McLean <malcolm.ar...@gmail.com> wrote:
>> > I'm thinking of writing a routine to convert JSON to C structures in C source
>> > code. (It would go in the Baby X resource compiler as the first use case).
>> >
>> > So it takes JSON and spits out a C source file with a structure declaration
>> > and initial elements in a following definition.
>> >
>> > I can see some obvious difficulties, for instance JSON allows for arrays
>> > of mixed types. Then a JSON object has an identier, but not a type name,
>> > whilst C structures require type name.
>> >
>> > Anyone any ideas about how to proceed?
>> C99 designated initializers handle unions
>>
>> jnode j0023 = { .type = jnode_num, .val = { .num = 3.14 ] };
>> jnode j0024 = { .type = jnode_str, .val = { .str = "abc" ] };
>> jnode j0025 = { .type = jnode_true };
>> jnode j0026 = { .type = jnode_false };
>>
>> jnode *ja0027[4] = { &j0023, &j0024, &j0025, &j0026 };
>>
>> jnode j0027 = { .type = jnode_vec,
>> .val = { .vec = { .data = ja0027, .len = 4 } } };
>>
>>
>> where jnode is something like
>>
>> enum jnode_type { jnode_false, jnode_true, ..., jnode_num, ...}
>>
>> typedef struct jnode {
>> enum jnode_type type;
>> union {
>> double num;
>> char *str;
>> struct {
>> struct jnode *data;
>> size_t len;
>> } vec;
>> } val;
>> } jnode;
>>
> That's to build a JSON parser in C. I already have one of those.

Well, no; the idea was to just convert a JSON data structure
to something statically declared in C, without there necessarily
being any parser in the program.

> What I was thinking of was a program along the lines of
>
> input
> {
> "name" : "Fred Bloggs",
> "salary": 32000.0,
> "id": 1234,
> "hours_worked" : [8, 8, 7, 8, 10]
> }

Right, so that just means JSON objects are becoming structs.

> output
>
> typedef struct
> {
> char *name;
> double salary;
> int id;
> int hours_worked[5];
> } EMPLOYEE;

I see, so a string or number can directly go to char * or double.

This EMPLOYEE identifier has to come from somewhere; the Json
has no such thing because objects are anonymous. You need a
convention like:

input
{
"class" : "EMPLOYEE"
...

}

and if that property is missing, then a machine-generated
name is used.

> EMPLOYEE fred {"Fred Blogs", 32000.0, 1234, 8, 8,7, 8, 10]};
>
> Obviously the first issue is that there is no way a computer can get
> from that JSON to "EMPLOYEE".

> The other issue is that whilst the
> scalar fields are simple, the array fields start opening up more and
> more difficulties.

JSON is dynamically typed; an array can have a mixtur eof any kind
of object. An array can contain a number, string, EMPLOYEE,
CUSTOMER, ... all at the same time.

That is why in my proposal, I have objects with type fields and
discriminated unions. The example shows an array being made of
heterogeneous objects.

One approach would be that the compiler for this adds all of the
structure names into the type enumeration.

Say that an EMPLOYEE and CUSTOMER types occur in the input.

Then:

enum jnode_type {
// the basic types
jnode_false, jnode_true, ..., jnode_num, jnode_vec,
// custom types ferreted out from data
jnode_EMPLOYEE,
jnode_CUSTOMER,
...
}

Then:

typedef struct jnode {
enum jnode_type type;
union {
double num;
char *str;

struct {
struct jnode *data;
size_t len;
} vec;

struct EMPLOYEE { // type == jnode_EMPLOYEE
char *name;
double salary;
int id;
int hours_worked[5];
} EMPLOYEE;

struct CUSTOMER { // type == jnode_CUSTOMER
...
} CUSTOMER;
} val;
} jnode;

Or these could be pointers, to keep the union small:

struct EMPLOYEE *EMPLOYEE; type == jnode_EMPLOYEE
struct CUSTOMER *CUSTOMER; type == jnode_CUSTOMER;

S then if you have a pointer to a node P, and you know that N->type is
jnode_EMPLOYEE, you can access P->val->EMPLOYEE.salary.

The hours_worked array could be int hours_worked[5] because the compiler
has identified that all the numbers which occur in the data for every
instance of employee are integers.

(The size of the array is the maximum across all the instances of
EMPLOYEE, and there should be a machine-generated size field for the
actual number present in the given instance, like n_hours_worked.)

If the compiler detects that the array is heterogeneous, then it
emits something else;

struct EMPLOYEE {
char *name;
double salary;
int id;
jnode hours_worked; // vector
} EMPLOYEE;

hours_worked becomes a jnode of type jnode_vec, and the code
has to use more complicated access to get into it.

(Since writing the code which accesses this structure is left
as an exercise to the application, this could be knocked off
in an afternoon.)

--
TXR Programming Language: http://nongnu.org/txr
Cygnal: Cygwin Native Application Library: http://kylheku.com/cygnal

Malcolm McLean <malcolm.arthur.mclean@gmail.com> writes:

> What I was thinking of was a program along the lines of
>
> input
> {
> "name" : "Fred Bloggs",
> "salary": 32000.0,
> "id": 1234,
> "hours_worked" : [8, 8, 7, 8, 10]
> }
>
> output
>
> typedef struct
> {
> char *name;
> double salary;
> int id;
> int hours_worked[5];
> } EMPLOYEE;

There a small wrinkle here. JSON has only number values, not int and
double, so unless the JSON is coming from somewhere you can influence
(for example a user who is given a convention to use) you won't be able
to tell that salary is a double and id is an int.

--
Ben.

On Friday, 1 April 2022 at 21:44:24 UTC+1, Ben Bacarisse wrote:
> Malcolm McLean <malcolm.ar...@gmail.com> writes:
>
> > What I was thinking of was a program along the lines of
> >
> > input
> > {
> > "name" : "Fred Bloggs",
> > "salary": 32000.0,
> > "id": 1234,
> > "hours_worked" : [8, 8, 7, 8, 10]
> > }
> >
> > output
> >
> > typedef struct
> > {
> > char *name;
> > double salary;
> > int id;
> > int hours_worked[5];
> > } EMPLOYEE;
> There a small wrinkle here. JSON has only number values, not int and
> double, so unless the JSON is coming from somewhere you can influence
> (for example a user who is given a convention to use) you won't be able
> to tell that salary is a double and id is an int.
>
If you've only got a few values, it's impossible to tell. But if you've got a large
number of values, and they are all written without decimal points, then you
can be fairly sure that the field is inherently integral.
The Baby X resource compiler takes an XML file which lists the input files.
It can be used to supply data that can't be inferred automatically.
However the feature needs to be lightly used. If the XML becomes too
complicated, it starts to defeat the point.

Malcolm McLean <malcolm.arthur.mclean@gmail.com> writes:

> On Friday, 1 April 2022 at 21:44:24 UTC+1, Ben Bacarisse wrote:
>> Malcolm McLean <malcolm.ar...@gmail.com> writes:
>>
>> > What I was thinking of was a program along the lines of
>> >
>> > input
>> > {
>> > "name" : "Fred Bloggs",
>> > "salary": 32000.0,
>> > "id": 1234,
>> > "hours_worked" : [8, 8, 7, 8, 10]
>> > }
>> >
>> > output
>> >
>> > typedef struct
>> > {
>> > char *name;
>> > double salary;
>> > int id;
>> > int hours_worked[5];
>> > } EMPLOYEE;
>> There a small wrinkle here. JSON has only number values, not int and
>> double, so unless the JSON is coming from somewhere you can influence
>> (for example a user who is given a convention to use) you won't be able
>> to tell that salary is a double and id is an int.
>>
> If you've only got a few values, it's impossible to tell. But if
> you've got a large number of values, and they are all written without
> decimal points, then you can be fairly sure that the field is
> inherently integral.

That'll often work, but your example wanted double for salary. My
annual salary never had fractional pounds.

> The Baby X resource compiler takes an XML file
> which lists the input files. It can be used to supply data that can't
> be inferred automatically. However the feature needs to be lightly
> used. If the XML becomes too complicated, it starts to defeat the
> point.

--
Ben.

On Saturday, 2 April 2022 at 01:26:31 UTC+3, Malcolm McLean wrote:
> On Friday, 1 April 2022 at 21:44:24 UTC+1, Ben Bacarisse wrote:
> > Malcolm McLean <malcolm.ar...@gmail.com> writes:
> >
> > > What I was thinking of was a program along the lines of
> > >
> > > input
> > > {
> > > "name" : "Fred Bloggs",
> > > "salary": 32000.0,
> > > "id": 1234,
> > > "hours_worked" : [8, 8, 7, 8, 10]
> > > }
> > >
> > > output
> > >
> > > typedef struct
> > > {
> > > char *name;
> > > double salary;
> > > int id;
> > > int hours_worked[5];
> > > } EMPLOYEE;
> > There a small wrinkle here. JSON has only number values, not int and
> > double, so unless the JSON is coming from somewhere you can influence
> > (for example a user who is given a convention to use) you won't be able
> > to tell that salary is a double and id is an int.
> >
> If you've only got a few values, it's impossible to tell. But if you've got a large
> number of values, and they are all written without decimal points, then you
> can be fairly sure that the field is inherently integral.
> The Baby X resource compiler takes an XML file which lists the input files.
> It can be used to supply data that can't be inferred automatically.
> However the feature needs to be lightly used. If the XML becomes too
> complicated, it starts to defeat the point.

Real data always comes from somewhere and so conforms to some
kind of schema of layout that is specified by that source of data.
That specification is usually simple to code as C, and then data
file contents are also usually simple to transform into values of
that C program.

JSON has loose rules for to keep it terse. That does not bother us
when we know the schema but makes it impossible to guess the
schema automatically on general case. For example in specification
it is easy to say that when member "unit" is missing from JSON
object then it is "meter". How you guess that from data?

On 4/1/2022 6:36 PM, Malcolm McLean wrote:
> On Friday, 1 April 2022 at 17:13:44 UTC+1, Manfred wrote:
>> On 4/1/2022 5:19 PM, Malcolm McLean wrote:
>>> On Friday, 1 April 2022 at 16:06:56 UTC+1, Kaz Kylheku wrote:
>>>> On 2022-04-01, Malcolm McLean <malcolm.ar...@gmail.com> wrote:
>>>>> I'm thinking of writing a routine to convert JSON to C structures in C source
>>>>> code. (It would go in the Baby X resource compiler as the first use case).
>>>>>
>>>>> So it takes JSON and spits out a C source file with a structure declaration
>>>>> and initial elements in a following definition.
>>>>>
>>>>> I can see some obvious difficulties, for instance JSON allows for arrays
>>>>> of mixed types. Then a JSON object has an identier, but not a type name,
>>>>> whilst C structures require type name.
>>>>>
>>>>> Anyone any ideas about how to proceed?
>>>> C99 designated initializers handle unions
>>>>
>>>> jnode j0023 = { .type = jnode_num, .val = { .num = 3.14 ] };
>>>> jnode j0024 = { .type = jnode_str, .val = { .str = "abc" ] };
>>>> jnode j0025 = { .type = jnode_true };
>>>> jnode j0026 = { .type = jnode_false };
>>>>
>>>> jnode *ja0027[4] = { &j0023, &j0024, &j0025, &j0026 };
>>>>
>>>> jnode j0027 = { .type = jnode_vec,
>>>> .val = { .vec = { .data = ja0027, .len = 4 } } };
>>>>
>>>>
>>>> where jnode is something like
>>>>
>>>> enum jnode_type { jnode_false, jnode_true, ..., jnode_num, ...}
>>>>
>>>> typedef struct jnode {
>>>> enum jnode_type type;
>>>> union {
>>>> double num;
>>>> char *str;
>>>> struct {
>>>> struct jnode *data;
>>>> size_t len;
>>>> } vec;
>>>> } val;
>>>> } jnode;
>>>>
>>> That's to build a JSON parser in C. I already have one of those.
>>>
>>> What I was thinking of was a program along the lines of
>>>
>>> input
>>> {
>>> "name" : "Fred Bloggs",
>>> "salary": 32000.0,
>>> "id": 1234,
>>> "hours_worked" : [8, 8, 7, 8, 10]
>>> }
>>>
>>> output
>>>
>>> typedef struct
>>> {
>>> char *name;
>>> double salary;
>>> int id;
>>> int hours_worked[5];
>>> } EMPLOYEE;
>>>
>>> EMPLOYEE fred {"Fred Blogs", 32000.0, 1234, 8, 8,7, 8, 10]};
>>>
>>> Obviously the first issue is that there is no way a computer can get from that JSON to "EMPLOYEE".
>> Well, in fact there is, it's about dynamic types and introspection,
>> which is easy for interpreted languages, but not as much in C.
>>
>> I believe one thing to consider is /how/ "EMPLOYEE" is going to be used
>> in the rest of the code, and how the rest of the code is supposed to
>> know about this type.
>> Answers to these questions will probably shed light on how this type
>> should be generated as well.
>>> The other issue is that whilst the scalar fields are simple, the array fields start opening up more and
>>> more difficulties.
>> I believe this part is a different perspective on the same problem as
>> the first one.
>>
> The Baby X resource compiler is basically a way of getting images into C programs, by
> dumping them as RGBA quads in C source. However it does other things as well, for
> example there's an audio resampler in there, and it can escape strings to create
> text resources.
> One thing someone might want to do is import non-image data. So the question is,
> what format would that data come in, and in what format should it be output to
> make it easy for someone writing a program to integrate the data into their
> source code?
> One format for data is JSON. A generic JSON structure with nodes would work, but it
> wouldn't be easy to use. I want a flat array of C structures with easy to access fields.

So, resource compiler. Which means out of line processing of data. Also,
the user of such compiled types is the application developer, not some
code framework. In this context, probably Kaz's suggestion elsethread
would do (or at least I can't find a better solution at this time)

On Saturday, 2 April 2022 at 01:14:00 UTC+1, Ben Bacarisse wrote:
> Malcolm McLean <malcolm.ar...@gmail.com> writes:
>
> > On Friday, 1 April 2022 at 21:44:24 UTC+1, Ben Bacarisse wrote:
> >> Malcolm McLean <malcolm.ar...@gmail.com> writes:
> >>
> >> > What I was thinking of was a program along the lines of
> >> >
> >> > input
> >> > {
> >> > "name" : "Fred Bloggs",
> >> > "salary": 32000.0,
> >> > "id": 1234,
> >> > "hours_worked" : [8, 8, 7, 8, 10]
> >> > }
> >> >
> >> > output
> >> >
> >> > typedef struct
> >> > {
> >> > char *name;
> >> > double salary;
> >> > int id;
> >> > int hours_worked[5];
> >> > } EMPLOYEE;
> >> There a small wrinkle here. JSON has only number values, not int and
> >> double, so unless the JSON is coming from somewhere you can influence
> >> (for example a user who is given a convention to use) you won't be able
> >> to tell that salary is a double and id is an int.
> >>
> > If you've only got a few values, it's impossible to tell. But if
> > you've got a large number of values, and they are all written without
> > decimal points, then you can be fairly sure that the field is
> > inherently integral.
> That'll often work, but your example wanted double for salary. My
> annual salary never had fractional pounds.
>
My CSV file parser solves this problem by returning all numerical fields as
doubles. But C programmers expect values that are constrained to be integers
to be an integer type. And C won't allow you to use a floating point type as
an array index.
So there isn't a good answer.

Malcolm McLean <malcolm.arthur.mclean@gmail.com> writes:

> On Saturday, 2 April 2022 at 01:14:00 UTC+1, Ben Bacarisse wrote:
>> Malcolm McLean <malcolm.ar...@gmail.com> writes:
>>
>> > On Friday, 1 April 2022 at 21:44:24 UTC+1, Ben Bacarisse wrote:
>> >> Malcolm McLean <malcolm.ar...@gmail.com> writes:
>> >>
>> >> > What I was thinking of was a program along the lines of
>> >> >
>> >> > input
>> >> > {
>> >> > "name" : "Fred Bloggs",
>> >> > "salary": 32000.0,
>> >> > "id": 1234,
>> >> > "hours_worked" : [8, 8, 7, 8, 10]
>> >> > }
>> >> >
>> >> > output
>> >> >
>> >> > typedef struct
>> >> > {
>> >> > char *name;
>> >> > double salary;
>> >> > int id;
>> >> > int hours_worked[5];
>> >> > } EMPLOYEE;
>> >> There a small wrinkle here. JSON has only number values, not int and
>> >> double, so unless the JSON is coming from somewhere you can influence
>> >> (for example a user who is given a convention to use) you won't be able
>> >> to tell that salary is a double and id is an int.
>> >>
>> > If you've only got a few values, it's impossible to tell. But if
>> > you've got a large number of values, and they are all written without
>> > decimal points, then you can be fairly sure that the field is
>> > inherently integral.
>>
>> That'll often work, but your example wanted double for salary. My
>> annual salary never had fractional pounds.
>>
> My CSV file parser solves this problem by returning all numerical fields as
> doubles. But C programmers expect values that are constrained to be integers
> to be an integer type. And C won't allow you to use a floating point type as
> an array index.
> So there isn't a good answer.

I'd use a richer syntax than JSON. The last time I did something
similar (more than 30 years go mind) there was no JSON so I had to
invent my own. With just a bit more info you can distinguish numeric
types (the syntax forced doubles to have either a . or an exponent) and
also distinguish between char * and char [] fields. Of course, if your
data comes from somewhere else that uses JSON, you are stuck with JSON.

--
Ben.

No one needs a static JSON to C data structure converter.
People need dyanmic converters that convert the data
structures at runtime.

On Sunday, 3 April 2022 at 18:12:39 UTC+1, Bonita Montero wrote:
> No one needs a static JSON to C data structure converter.
> People need dyanmic converters that convert the data
> structures at runtime.
>
I already have a JSON parser to accept JSON at runtime and extract
the data as C native variables.
The Baby X resource compiler takes in data and converts it to compileable
C source. But there might not really be a use case for JSON. Or it might
be too difficult to specify all the additional information needed to convert the
JSON to acceptable C.
Scratching the idea is a definite possibility.

Am 03.04.2022 um 23:01 schrieb Malcolm McLean:

> But there might not really be a use case for JSON.

There are many usecases for JSON.
JSON often replaces XML because it's simpler
(look at this XML Schema bloat).

On Tuesday, 5 April 2022 at 14:12:05 UTC+1, Bonita Montero wrote:
> Am 03.04.2022 um 23:01 schrieb Malcolm McLean:
>
> > But there might not really be a use case for JSON.
> There are many usecases for JSON.
> JSON often replaces XML because it's simpler
> (look at this XML Schema bloat).
>
JSON has been accepted as a format for data exchange.

The question is whether there is a use case for it in a C source generator.
Would people take data in in JSON and want it converted to compileable
C structures?

Am 05.04.2022 um 15:22 schrieb Malcolm McLean:
> On Tuesday, 5 April 2022 at 14:12:05 UTC+1, Bonita Montero wrote:
>> Am 03.04.2022 um 23:01 schrieb Malcolm McLean:
>>
>>> But there might not really be a use case for JSON.
>> There are many usecases for JSON.
>> JSON often replaces XML because it's simpler
>> (look at this XML Schema bloat).
>>
> JSON has been accepted as a format for data exchange.
>
> The question is whether there is a use case for it in a C source generator.
> Would people take data in in JSON and want it converted to compileable
> C structures?

No, and that's while I'm asking who needs such a converter.

On Tuesday, 5 April 2022 at 15:45:14 UTC+1, Bonita Montero wrote:
> Am 05.04.2022 um 15:22 schrieb Malcolm McLean:
>
> > The question is whether there is a use case for it in a C source generator.
> > Would people take data in in JSON and want it converted to compileable
> > C structures?
> No, and that's while I'm asking who needs such a converter.
>
You often want to hardcode data rather than reading it in from a file. In
the embedded world, there might be no filesystem from which you can
read in the data. In the hosted world, it often makes the executable
easier to distribute if it doesn't depend on external data files for support,
and it's more robust.
Theres definitely a case for facilities to package image files. The question
uis whether anyone would actually use a similar facility for JSON.

On 05/04/2022 14:22, Malcolm McLean wrote:
> On Tuesday, 5 April 2022 at 14:12:05 UTC+1, Bonita Montero wrote:
>> Am 03.04.2022 um 23:01 schrieb Malcolm McLean:
>>
>>> But there might not really be a use case for JSON.
>> There are many usecases for JSON.
>> JSON often replaces XML because it's simpler
>> (look at this XML Schema bloat).
>>
> JSON has been accepted as a format for data exchange.
>
> The question is whether there is a use case for it in a C source generator.
> Would people take data in in JSON and want it converted to compileable
> C structures?

To take an arbitrary JSON and produce a C source file with hard-coded
struct types is problematic.

AIUI, JSON contains two structuring mechanisms:

[...] arrays of mixed/arbitrary types
{...} objects of string:value pairs

So the first problem with turning this into C, is that each element of
the array may be a number, bool, string, object or nested array.

The next, if intending to turn {...} into a struct, is that each {...}
object could represent a different struct. Maybe they represent the same
record, but it is up to the translator to decide if two objects are the
same struct or not:

{ "age":10, "name":"bart", "colour":0xFFFF00}
{ "name":lisa, "age":8}

?

It would be very easy to end up with 1000s of different struct types.

You need to step back and look at a selection of actual use-cases to see
whether such a tool as you propose could do anything useful, and what
extra data or meta-data would be needed.

For example, you can take a C program containing an initialised array of
structs, and write some code to serialise that data into a JSON file.

Then you know that that data /could/ be turned back into C (which
defines a populated array of structs) using a suitable tool, but it
might need some extra hints.

Once that's done, you can have an application capable of writing out
data via JSON, and reading it back in. But it will not cope with any
random JSON file downloaded from somewhere.

If I take this example from Wikipedia (where I got my recently acquired
knowledge):

{
"firstName": "John",
"lastName": "Smith",
"isAlive": true,
"age": 27,
"address": {
"streetAddress": "21 2nd Street",
"city": "New York",
"state": "NY",
"postalCode": "10021-3100"
},
"phoneNumbers": [
{
"type": "home",
"number": "212 555-1234"
},
{
"type": "office",
"number": "646 555-4567"
}
],
"children": [],
"spouse": null
}

This is data that can be represented within any scripting language (I
believe JSON format is supposed to be a subset of JavaScript).

I can even paste that into my own scripting language, and run it after
making these substitutions:

{...} => [...] A dict constructor when key:value pairs present
[...] => (...) A list constructor
null => nil

data:=
[
"firstName": "John",
"lastName": "Smith",
"isAlive": true,
"age": 27,
"address": [
"streetAddress": "21 2nd Street",
"city": "New York",
"state": "NY",
"postalCode": "10021-3100"
],
"phoneNumbers": (
[
"type": "home",
"number": "212 555-1234"
],
[
"type": "office",
"number": "646 555-4567"
]
),
"children": (),
"spouse": nil
]

println data

This outputs (unformatted):

[age:27,children:(),phoneNumbers:([type:home,number:212
555-1234],[type:office,number:646 555-4567])
,firstName:John,spouse:nil,address:[state:NY,postalCode:10021-3100,city:New
York,streetAddress:21 2n
d Street],isAlive:1,lastName:Smith]

The point is, JSON format suits dynamic, arbitrarily structured data. It
doesn't suit the fixed data of a C program, unless you specifically
write a C program that represents arbitrary data. Which means not
hard-coding structs for that data.

On Tuesday, 5 April 2022 at 16:25:43 UTC+1, Bart wrote:
> On 05/04/2022 14:22, Malcolm McLean wrote:
> > On Tuesday, 5 April 2022 at 14:12:05 UTC+1, Bonita Montero wrote:
> >> Am 03.04.2022 um 23:01 schrieb Malcolm McLean:
> >>
> >>> But there might not really be a use case for JSON.
> >> There are many usecases for JSON.
> >> JSON often replaces XML because it's simpler
> >> (look at this XML Schema bloat).
> >>
> > JSON has been accepted as a format for data exchange.
> >
> > The question is whether there is a use case for it in a C source generator.
> > Would people take data in in JSON and want it converted to compileable
> > C structures?
> To take an arbitrary JSON and produce a C source file with hard-coded
> struct types is problematic.
>
> AIUI, JSON contains two structuring mechanisms:
>
> [...] arrays of mixed/arbitrary types
> {...} objects of string:value pairs
>
> So the first problem with turning this into C, is that each element of
> the array may be a number, bool, string, object or nested array.
>
> The next, if intending to turn {...} into a struct, is that each {...}
> object could represent a different struct. Maybe they represent the same
> record, but it is up to the translator to decide if two objects are the
> same struct or not:
>
> { "age":10, "name":"bart", "colour":0xFFFF00}
> { "name":lisa, "age":8}
>
> ?
>
> It would be very easy to end up with 1000s of different struct types.
>
> You need to step back and look at a selection of actual use-cases to see
> whether such a tool as you propose could do anything useful, and what
> extra data or meta-data would be needed.
>
> For example, you can take a C program containing an initialised array of
> structs, and write some code to serialise that data into a JSON file.
>
> Then you know that that data /could/ be turned back into C (which
> defines a populated array of structs) using a suitable tool, but it
> might need some extra hints.
>
> Once that's done, you can have an application capable of writing out
> data via JSON, and reading it back in. But it will not cope with any
> random JSON file downloaded from somewhere.
>
>
> If I take this example from Wikipedia (where I got my recently acquired
> knowledge):
>
> {
> "firstName": "John",
> "lastName": "Smith",
> "isAlive": true,
> "age": 27,
> "address": {
> "streetAddress": "21 2nd Street",
> "city": "New York",
> "state": "NY",
> "postalCode": "10021-3100"
> },
> "phoneNumbers": [
> {
> "type": "home",
> "number": "212 555-1234"
> },
> {
> "type": "office",
> "number": "646 555-4567"
> }
> ],
> "children": [],
> "spouse": null
> }
>
> This is data that can be represented within any scripting language (I
> believe JSON format is supposed to be a subset of JavaScript).
>
> I can even paste that into my own scripting language, and run it after
> making these substitutions:
>
> {...} => [...] A dict constructor when key:value pairs present
> [...] => (...) A list constructor
> null => nil
>
> data:=
> [
> "firstName": "John",
> "lastName": "Smith",
> "isAlive": true,
> "age": 27,
> "address": [
> "streetAddress": "21 2nd Street",
> "city": "New York",
> "state": "NY",
> "postalCode": "10021-3100"
> ],
> "phoneNumbers": (
> [
> "type": "home",
> "number": "212 555-1234"
> ],
> [
> "type": "office",
> "number": "646 555-4567"
> ]
> ),
> "children": (),
> "spouse": nil
> ]
>
> println data
>
> This outputs (unformatted):
>
> [age:27,children:(),phoneNumbers:([type:home,number:212
> 555-1234],[type:office,number:646 555-4567])
> ,firstName:John,spouse:nil,address:[state:NY,postalCode:10021-3100,city:New
> York,streetAddress:21 2n
> d Street],isAlive:1,lastName:Smith]
>
> The point is, JSON format suits dynamic, arbitrarily structured data. It
> doesn't suit the fixed data of a C program, unless you specifically
> write a C program that represents arbitrary data. Which means not
> hard-coding structs for that data.
>
I've already got a JSON parser. So I can take in a JSON string from an external
file, or hardcode it as a string, then query it for the keys and datatype, and
extract the data.
But it's rare to really need that flexibility. In my real life use of JSON, I use
JSON to store user settings. So I know the keys and the type of data they
hold beforehand. I just set a field to a default value if a key is missing or
holds invalid data.

An example use case might be a football management game. The data for the
players is held in some editing tool which stores the players' statistics and
history as a big array of JSON objects. You don't want the data to be visible
to the user in the final product, so you hardcode it to a C array. You know what
the keys are when you write the main program, because it has to access
fields of the struct.
The question is whether the JSON to C converter can be generic, or if it has
to be bespoke.

Am 05.04.2022 um 17:03 schrieb Malcolm McLean:
> On Tuesday, 5 April 2022 at 15:45:14 UTC+1, Bonita Montero wrote:
>> Am 05.04.2022 um 15:22 schrieb Malcolm McLean:
>>
>>> The question is whether there is a use case for it in a C source generator.
>>> Would people take data in in JSON and want it converted to compileable
>>> C structures?
>> No, and that's while I'm asking who needs such a converter.
>>
> You often want to hardcode data rather than reading it in from a file.

And you never have JSON as a raw material for that.
Rest from you text unread.

On 2022-04-05, Malcolm McLean <malcolm.arthur.mclean@gmail.com> wrote:
> On Tuesday, 5 April 2022 at 14:12:05 UTC+1, Bonita Montero wrote:
>> Am 03.04.2022 um 23:01 schrieb Malcolm McLean:
>>
>> > But there might not really be a use case for JSON.
>> There are many usecases for JSON.
>> JSON often replaces XML because it's simpler
>> (look at this XML Schema bloat).
>>
> JSON has been accepted as a format for data exchange.
>
> The question is whether there is a use case for it in a C source generator.
> Would people take data in in JSON and want it converted to compileable
> C structures?

People would, but mainly those who don't know Lisp.

--
TXR Programming Language: http://nongnu.org/txr
Cygnal: Cygwin Native Application Library: http://kylheku.com/cygnal