Rocksolid Light

Welcome to novaBBS (click a section below)

mail  files  register  newsreader  groups  login

Message-ID:  

Witch! Witch! They'll burn ya! -- Hag, "Tomorrow is Yesterday", stardate unknown


devel / comp.compilers / Re: Are there "compiler generators"?

SubjectAuthor
* Are there "compiler generators"?Roger L Costello
+- Re: Are there "compiler generators"?Robin Vowels
+- Re: Are there "compiler generators"?Jan Ziak
+- Re: Are there "compiler generators"?Anton Ertl
+* Re: Are there "compiler generators"?Thomas Koenig
|`* Re: Are there "compiler generators"?Hans-Peter Diettrich
| `* Re: Are there "compiler generators"?Hans-Peter Diettrich
|  `* Re: Are there "compiler generators"?gah4
|   `- RE: Are there compiler generators?Christopher F Clark
+* Re: Are there "compiler generators"?Martin Ward
|`- Re: Are there "compiler generators"?Anton Ertl
+- Re: Are there "compiler generators"?Fernando
+* Re: Are there "compiler generators"?gah4
|`- Re: Are there "compiler generators"?mac
`- Re: Are there "compiler generators"?Kaz Kylheku

1
Are there "compiler generators"?

<22-05-054@comp.compilers>

 copy mid

https://www.novabbs.com/devel/article-flat.php?id=409&group=comp.compilers#409

 copy link   Newsgroups: comp.compilers
Path: i2pn2.org!i2pn.org!weretis.net!feeder6.news.weretis.net!news.misty.com!news.iecc.com!.POSTED.news.iecc.com!nerds-end
From: coste...@mitre.org (Roger L Costello)
Newsgroups: comp.compilers
Subject: Are there "compiler generators"?
Date: Sat, 28 May 2022 22:27:53 +0000
Organization: Compilers Central
Lines: 30
Sender: news@iecc.com
Approved: comp.compilers@iecc.com
Message-ID: <22-05-054@comp.compilers>
Mime-Version: 1.0
Content-Type: text/plain; charset="us-ascii"
Content-Transfer-Encoding: 8bit
Injection-Info: gal.iecc.com; posting-host="news.iecc.com:2001:470:1f07:1126:0:676f:7373:6970";
logging-data="91449"; mail-complaints-to="abuse@iecc.com"
Keywords: question, comment
Posted-Date: 28 May 2022 22:16:18 EDT
X-submission-address: compilers@iecc.com
X-moderator-address: compilers-request@iecc.com
X-FAQ-and-archives: http://compilers.iecc.com
Content-Language: en-US
 by: Roger L Costello - Sat, 28 May 2022 22:27 UTC

Hi Folks,

There are lexer generators. Flex is a lexer generator.

There are parser generators. Bison is a parser generator.

Are there compiler generators?

Page 52 of the book "Crafting a Compiler with C" says this in the chapter
titled "Scanning--Theory and Practice":

Programming a scanner generator is an example of nonprocedural programming.
That is, unlike ordinary programming, which we call procedural, we do not tell
a scanner generator "how" do scan but simply "what" we want scanned. This is a
higher-level approach and in many ways a more natural one. Much recent
research in computer science is directed toward nonprocedural programming
styles. (Database query languages and Prolog, a "logic" programming language,
are nonprocedural.) Nonprocedural programming is most successful in limited
domains, such as scanning, where the range of implementation decisions that
must be automatically made is limited. Nonetheless, a long-standing (and as
yet unrealized) goal of computer scientists is to generate an entire compiler
from a specification of the properties of the source language and target
computer.

That was written in 1991. Is it still true in 2022--there are no compiler
generators?

/Roger
[There are certainly programs that will generate a combined lexer and parser
but there's a lot more to a compiler. -John]

Re: Are there "compiler generators"?

<22-05-055@comp.compilers>

 copy mid

https://www.novabbs.com/devel/article-flat.php?id=410&group=comp.compilers#410

 copy link   Newsgroups: comp.compilers
Path: i2pn2.org!i2pn.org!weretis.net!feeder6.news.weretis.net!news.misty.com!news.iecc.com!.POSTED.news.iecc.com!nerds-end
From: robi...@dodo.com.au (Robin Vowels)
Newsgroups: comp.compilers
Subject: Re: Are there "compiler generators"?
Date: Sun, 29 May 2022 13:34:28 +1000
Organization: Compilers Central
Lines: 13
Sender: news@iecc.com
Approved: comp.compilers@iecc.com
Message-ID: <22-05-055@comp.compilers>
References: <22-05-054@comp.compilers>
Mime-Version: 1.0
Content-Type: text/plain; format=flowed; charset="iso-8859-1"; reply-type=original
Content-Transfer-Encoding: 8bit
Injection-Info: gal.iecc.com; posting-host="news.iecc.com:2001:470:1f07:1126:0:676f:7373:6970";
logging-data="53115"; mail-complaints-to="abuse@iecc.com"
Keywords: tools
Posted-Date: 29 May 2022 18:13:39 EDT
X-submission-address: compilers@iecc.com
X-moderator-address: compilers-request@iecc.com
X-FAQ-and-archives: http://compilers.iecc.com
 by: Robin Vowels - Sun, 29 May 2022 03:34 UTC

"There's "A Compiler Generator" by McKeeman, Horning, and Wortman.

----- Original Message -----
From: "Roger L Costello" <costello@mitre.org>
Sent: Sunday, May 29, 2022 8:27 AM

> There are lexer generators. Flex is a lexer generator.
>
> There are parser generators. Bison is a parser generator.
>
> Are there compiler generators?

Re: Are there "compiler generators"?

<22-05-056@comp.compilers>

 copy mid

https://www.novabbs.com/devel/article-flat.php?id=411&group=comp.compilers#411

 copy link   Newsgroups: comp.compilers
Path: i2pn2.org!i2pn.org!weretis.net!feeder6.news.weretis.net!news.misty.com!news.iecc.com!.POSTED.news.iecc.com!nerds-end
From: 0xe2.0x9...@gmail.com (Jan Ziak)
Newsgroups: comp.compilers
Subject: Re: Are there "compiler generators"?
Date: Sat, 28 May 2022 23:52:54 -0700 (PDT)
Organization: Compilers Central
Lines: 18
Sender: news@iecc.com
Approved: comp.compilers@iecc.com
Message-ID: <22-05-056@comp.compilers>
References: <22-05-054@comp.compilers>
Mime-Version: 1.0
Content-Type: text/plain; charset="UTF-8"
Injection-Info: gal.iecc.com; posting-host="news.iecc.com:2001:470:1f07:1126:0:676f:7373:6970";
logging-data="53374"; mail-complaints-to="abuse@iecc.com"
Keywords: tools
Posted-Date: 29 May 2022 18:14:13 EDT
X-submission-address: compilers@iecc.com
X-moderator-address: compilers-request@iecc.com
X-FAQ-and-archives: http://compilers.iecc.com
In-Reply-To: <22-05-054@comp.compilers>
 by: Jan Ziak - Sun, 29 May 2022 06:52 UTC

On Sunday, May 29, 2022 at 4:16:21 AM UTC+2, Roger L Costello wrote:
> Hi Folks,
>
> There are lexer generators. Flex is a lexer generator.
>
> There are parser generators. Bison is a parser generator.
>
> Are there compiler generators? ...

If you like reading about something which might be practical in a
hundred years: https://people.idsia.ch/~juergen/goedelmachine.html
(1st paper in the series: https://arxiv.org/abs/cs/0309048)

Some practical examples available today, having some similarity to the above method, are PyPy and GraalVM.

There is also http://www.general-game-playing.de for example.

-atom

Re: Are there "compiler generators"?

<22-05-057@comp.compilers>

 copy mid

https://www.novabbs.com/devel/article-flat.php?id=412&group=comp.compilers#412

 copy link   Newsgroups: comp.compilers
Path: i2pn2.org!i2pn.org!weretis.net!feeder6.news.weretis.net!news.misty.com!news.iecc.com!.POSTED.news.iecc.com!nerds-end
From: ant...@mips.complang.tuwien.ac.at (Anton Ertl)
Newsgroups: comp.compilers
Subject: Re: Are there "compiler generators"?
Date: Sun, 29 May 2022 06:45:21 GMT
Organization: Institut fuer Computersprachen, Technische Universitaet Wien
Lines: 38
Sender: news@iecc.com
Approved: comp.compilers@iecc.com
Message-ID: <22-05-057@comp.compilers>
References: <22-05-054@comp.compilers>
Injection-Info: gal.iecc.com; posting-host="news.iecc.com:2001:470:1f07:1126:0:676f:7373:6970";
logging-data="53684"; mail-complaints-to="abuse@iecc.com"
Keywords: tools
Posted-Date: 29 May 2022 18:14:55 EDT
X-submission-address: compilers@iecc.com
X-moderator-address: compilers-request@iecc.com
X-FAQ-and-archives: http://compilers.iecc.com
 by: Anton Ertl - Sun, 29 May 2022 06:45 UTC

Roger L Costello <costello@mitre.org> writes:
>Are there compiler generators?

Apart from scanner generators and parser generators, there are also:

* generators of attribute grammar evaluators (e.g., Ox). They use an
attribute grammar as input.

* generators of instruction selectors (e.g., burg and friends). Burg
uses a tree grammar as input.

* generators of virtual machine interpreters and other code dealing
with virtual machine instructions (e.g., vmgen). Vmgen uses a
description of the virtual machine instructions as inputs.

* There is also some work on generating type checkers.

All of these could be classified as non-procedural, although they tend
to include some procedural code.

You need to add substantial amounts of glue code to integrate the
pieces generated by these generators into a compiler, especially for
languages that do not quite fit in the generators' molds. Plus, there
are pieces such as symbol tables and register allocation that are not
covered by these generators.

I have also seen a paper by a French group (don't remember the names)
in the early 1990s where a generator could generate a complete Pascal
compiler from a specification. My impression, however, was that the
generator could only generate compilers for languages that are
relatively close to Pascal, and I saw no good way to make it much more
flexible.

- anton
--
M. Anton Ertl
anton@mips.complang.tuwien.ac.at
http://www.complang.tuwien.ac.at/anton/

Re: Are there "compiler generators"?

<22-05-058@comp.compilers>

 copy mid

https://www.novabbs.com/devel/article-flat.php?id=413&group=comp.compilers#413

 copy link   Newsgroups: comp.compilers
Path: i2pn2.org!i2pn.org!weretis.net!feeder6.news.weretis.net!news.misty.com!news.iecc.com!.POSTED.news.iecc.com!nerds-end
From: tkoe...@netcologne.de (Thomas Koenig)
Newsgroups: comp.compilers
Subject: Re: Are there "compiler generators"?
Date: Sun, 29 May 2022 09:14:51 -0000 (UTC)
Organization: news.netcologne.de
Lines: 46
Sender: news@iecc.com
Approved: comp.compilers@iecc.com
Message-ID: <22-05-058@comp.compilers>
References: <22-05-054@comp.compilers>
Injection-Info: gal.iecc.com; posting-host="news.iecc.com:2001:470:1f07:1126:0:676f:7373:6970";
logging-data="54101"; mail-complaints-to="abuse@iecc.com"
Keywords: tools
Posted-Date: 29 May 2022 18:16:02 EDT
X-submission-address: compilers@iecc.com
X-moderator-address: compilers-request@iecc.com
X-FAQ-and-archives: http://compilers.iecc.com
 by: Thomas Koenig - Sun, 29 May 2022 09:14 UTC

Roger L Costello <costello@mitre.org> schrieb:
> Is it still true in 2022--there are no compiler
> generators?

Consider what a compiler does can be roughly subdivided
into the follwing tasks:

a) Lexing, resulting in a stream of tokens
b) Parsing, resulting in an abstract syntax tree
c) Conversion into intermediate form, resulting in
SSA (usually, these days)
d) Optimization of the SSA, resulting in modified SSA
e) Conversion into machine language, resulting in some
internl representation of the machine language
f) Optimization of the machine language
g) Output to later stages (in assembler or in intermediate
form)

Roughly speaking a)-c) is the front end, d) is the middle end and
e)-f) is the back end.

As our moderator writes,

> [There are certainly programs that will generate a combined lexer and parser
> but there's a lot more to a compiler. -John]

a) and b) are well covered. For c), you would need a machine-
implementable description of the language semantics, and there is
no such thing in widespread use. d) is a main component of today's
compilers, which is hand-crafted code. e)-g) can be covered by
machine descriptions, that gcc and LLVM have (it is debatable if
these can be called generators).

So, assuming you can use gcc or LLVM, the main gap is c) - you
would need a formal semantic description, and it would have to
be written in a way that it can be translated automatically.
That might be doable if you design your own langue with that goal.

Defining a format for language semantics which is useful for a
reasonable subset of the multitude of languages in current use and
machine-translatable (a input to a "semantic front end generator",
if you will) will be _extremely_ hard. Maybe starting with the
formal definition of Modula-2 and then thinking in two directions,
how could a semantic front end generator be built from that
description and and how could that format be extended to cover
other languages, could be the start of an approach.

Re: Are there "compiler generators"?

<22-05-059@comp.compilers>

 copy mid

https://www.novabbs.com/devel/article-flat.php?id=414&group=comp.compilers#414

 copy link   Newsgroups: comp.compilers
Path: i2pn2.org!i2pn.org!weretis.net!feeder6.news.weretis.net!news.misty.com!news.iecc.com!.POSTED.news.iecc.com!nerds-end
From: mar...@gkc.org.uk (Martin Ward)
Newsgroups: comp.compilers
Subject: Re: Are there "compiler generators"?
Date: Sun, 29 May 2022 12:00:12 +0100
Organization: Compilers Central
Lines: 54
Sender: news@iecc.com
Approved: comp.compilers@iecc.com
Message-ID: <22-05-059@comp.compilers>
References: <22-05-054@comp.compilers>
Mime-Version: 1.0
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Transfer-Encoding: 8bit
Injection-Info: gal.iecc.com; posting-host="news.iecc.com:2001:470:1f07:1126:0:676f:7373:6970";
logging-data="54404"; mail-complaints-to="abuse@iecc.com"
Keywords: tools, theory
Posted-Date: 29 May 2022 18:16:48 EDT
X-submission-address: compilers@iecc.com
X-moderator-address: compilers-request@iecc.com
X-FAQ-and-archives: http://compilers.iecc.com
In-Reply-To: <22-05-054@comp.compilers>
Content-Language: en-GB
 by: Martin Ward - Sun, 29 May 2022 11:00 UTC

On 28/05/2022 23:27, Roger L Costello wrote:
> There are lexer generators. Flex is a lexer generator.
>
> There are parser generators. Bison is a parser generator.
>
> Are there compiler generators?

Suppose a program F(x, y) takes two inputs and produces
an output. Suppose input x is known at compile time.
Then we can optimise the program F(x, .) by precomputing
all the computations that depend on x to give a program
G(.) which can be applied to y to produce the output F(x, y):

G(y) = F(x, y)

This optimisation process is called "partial evaluation"
or "specialisation".

An interpreter is a program F(p, d) which takes
a program p and some input data d and produces
the result of executing p on d.

In theory a partial evaluator (a "specialiser") can be applied
to an interpreter and the source code for a program to give
an executable which is a specialised version of the interpreter
which only runs the given source code, does not require
the source code to be applied and runs faster than
the original combination of interpreter and source code.

To take this to the next level: specialising the specialiser
for the interpreter generates a compiler for the interpreted
language. So this is a type of "compiler generator".

Finally, specialising the specialiser for itself gives a tool
that can convert any interpreter into an equivalent compiler.

https://en.wikipedia.org/wiki/Partial_evaluation

Futamura, Y. (1999). "Partial Evaluation of Computation Process—An
Approach to a Compiler-Compiler". Higher-Order and Symbolic Computation.
12 (4): 381–391. CiteSeerX 10.1.1.10.2747. doi:10.1023/A:1010095604496.
S2CID 12673078.

In practice, there is a lot more to writing a compiler than just
partially evaluating an interpreter.

(The difference between theory and practice is that in theory there
is no difference between theory and practice, but in practice there is.)

--
Martin

Dr Martin Ward | Email: martin@gkc.org.uk | http://www.gkc.org.uk
G.K.Chesterton site: http://www.gkc.org.uk/gkc | Erdos number: 4

Re: Are there "compiler generators"?

<22-05-060@comp.compilers>

 copy mid

https://www.novabbs.com/devel/article-flat.php?id=415&group=comp.compilers#415

 copy link   Newsgroups: comp.compilers
Path: i2pn2.org!i2pn.org!weretis.net!feeder6.news.weretis.net!news.misty.com!news.iecc.com!.POSTED.news.iecc.com!nerds-end
From: prone...@gmail.com (Fernando)
Newsgroups: comp.compilers
Subject: Re: Are there "compiler generators"?
Date: Sun, 29 May 2022 05:00:47 -0700 (PDT)
Organization: Compilers Central
Lines: 26
Sender: news@iecc.com
Approved: comp.compilers@iecc.com
Message-ID: <22-05-060@comp.compilers>
References: <22-05-054@comp.compilers>
Mime-Version: 1.0
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
Injection-Info: gal.iecc.com; posting-host="news.iecc.com:2001:470:1f07:1126:0:676f:7373:6970";
logging-data="54654"; mail-complaints-to="abuse@iecc.com"
Keywords: tools, theory
Posted-Date: 29 May 2022 18:17:18 EDT
X-submission-address: compilers@iecc.com
X-moderator-address: compilers-request@iecc.com
X-FAQ-and-archives: http://compilers.iecc.com
In-Reply-To: <22-05-054@comp.compilers>
 by: Fernando - Sun, 29 May 2022 12:00 UTC

Hi Roger.

At least in theory, the Second Futamura Projection (of which there are three
of them) would generate a compiler. That's a way to do partial program
evaluation (https://en.wikipedia.org/wiki/Partial_evaluation). There have been
research papers about it, demonstrating that one can indeed generate compilers
out of partial evaluation, eg.:

* An experiment in partial evaluation: the generation of a compiler generator.
ND Jones et al. 1985
* A compiler generator produced by a self-applicable specializer can have a
surprisingly natural and understandable structure. SA Romanenko, 1988
* Practical second Futamura projection: partial evaluation for
high-performance language interpreters. F Latifi, 2019
* Etc

Regards,

Fernando

On Saturday, May 28, 2022 at 11:16:21 PM UTC-3, Roger L Costello wrote:
> Hi Folks,
>
> There are lexer generators. Flex is a lexer generator.
> There are parser generators. Bison is a parser generator.
> Are there compiler generators?

Re: Are there "compiler generators"?

<22-05-061@comp.compilers>

 copy mid

https://www.novabbs.com/devel/article-flat.php?id=416&group=comp.compilers#416

 copy link   Newsgroups: comp.compilers
Path: i2pn2.org!i2pn.org!weretis.net!feeder6.news.weretis.net!news.misty.com!news.iecc.com!.POSTED.news.iecc.com!nerds-end
From: gah...@u.washington.edu (gah4)
Newsgroups: comp.compilers
Subject: Re: Are there "compiler generators"?
Date: Sun, 29 May 2022 23:29:58 -0700 (PDT)
Organization: Compilers Central
Lines: 22
Sender: news@iecc.com
Approved: comp.compilers@iecc.com
Message-ID: <22-05-061@comp.compilers>
References: <22-05-054@comp.compilers>
Mime-Version: 1.0
Content-Type: text/plain; charset="UTF-8"
Injection-Info: gal.iecc.com; posting-host="news.iecc.com:2001:470:1f07:1126:0:676f:7373:6970";
logging-data="71262"; mail-complaints-to="abuse@iecc.com"
Keywords: tools
Posted-Date: 30 May 2022 14:47:26 EDT
X-submission-address: compilers@iecc.com
X-moderator-address: compilers-request@iecc.com
X-FAQ-and-archives: http://compilers.iecc.com
In-Reply-To: <22-05-054@comp.compilers>
 by: gah4 - Mon, 30 May 2022 06:29 UTC

On Saturday, May 28, 2022 at 7:16:21 PM UTC-7, Roger L Costello wrote:

> There are lexer generators. Flex is a lexer generator.

> There are parser generators. Bison is a parser generator.

> Are there compiler generators?

My old favorite, and probably still favorite, compiler book is:

"Retargetable C Compiler, A: Design and Implementation"

(It is more likely to be your favorite if you are interested in a C compiler.)

LCC uses, more or less, a code generator generator. You supply the
instruction combinations to do operations that are needed, and it
uses dynamic programming to select the optimal code.

It is usual to generate assembly code for an assembler.

Mostly that leaves the middle end, especially for optimization,
that doesn't have its own generator.

Re: Are there "compiler generators"?

<22-05-062@comp.compilers>

 copy mid

https://www.novabbs.com/devel/article-flat.php?id=417&group=comp.compilers#417

 copy link   Newsgroups: comp.compilers
Path: i2pn2.org!i2pn.org!weretis.net!feeder6.news.weretis.net!news.misty.com!news.iecc.com!.POSTED.news.iecc.com!nerds-end
From: ant...@mips.complang.tuwien.ac.at (Anton Ertl)
Newsgroups: comp.compilers
Subject: Re: Are there "compiler generators"?
Date: Mon, 30 May 2022 07:35:16 GMT
Organization: Institut fuer Computersprachen, Technische Universitaet Wien
Lines: 22
Sender: news@iecc.com
Approved: comp.compilers@iecc.com
Message-ID: <22-05-062@comp.compilers>
References: <22-05-054@comp.compilers> <22-05-059@comp.compilers>
Injection-Info: gal.iecc.com; posting-host="news.iecc.com:2001:470:1f07:1126:0:676f:7373:6970";
logging-data="71575"; mail-complaints-to="abuse@iecc.com"
Keywords: tools, theory
Posted-Date: 30 May 2022 14:48:11 EDT
X-submission-address: compilers@iecc.com
X-moderator-address: compilers-request@iecc.com
X-FAQ-and-archives: http://compilers.iecc.com
 by: Anton Ertl - Mon, 30 May 2022 07:35 UTC

Martin Ward <martin@gkc.org.uk> writes:
>In practice, there is a lot more to writing a compiler than just
>partially evaluating an interpreter.

In the 1990s I heard several talks about compiler generation in that
way, and it was an old topic by then, with still no practical results,
so my impression was that it never was going to be practical.

But recently I heard great things about optimizing Truffle
interpreters which eventually results in a compiler for the
interpreted language. You probably still have to hold it right, but
it's certainly much better than what I expected.

I asked one of the people involved if there had been a breakthrough,
but he could not name one, and said that there were just a lot of
little problems to be solved.

- anton
--
M. Anton Ertl
anton@mips.complang.tuwien.ac.at
http://www.complang.tuwien.ac.at/anton/

Re: Are there "compiler generators"?

<22-05-063@comp.compilers>

 copy mid

https://www.novabbs.com/devel/article-flat.php?id=418&group=comp.compilers#418

 copy link   Newsgroups: comp.compilers
Path: i2pn2.org!i2pn.org!weretis.net!feeder6.news.weretis.net!news.misty.com!news.iecc.com!.POSTED.news.iecc.com!nerds-end
From: DrDiettr...@netscape.net (Hans-Peter Diettrich)
Newsgroups: comp.compilers
Subject: Re: Are there "compiler generators"?
Date: Mon, 30 May 2022 14:53:59 +0200
Organization: Compilers Central
Lines: 17
Sender: news@iecc.com
Approved: comp.compilers@iecc.com
Message-ID: <22-05-063@comp.compilers>
References: <22-05-054@comp.compilers> <22-05-058@comp.compilers>
Mime-Version: 1.0
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Transfer-Encoding: 8bit
Injection-Info: gal.iecc.com; posting-host="news.iecc.com:2001:470:1f07:1126:0:676f:7373:6970";
logging-data="71936"; mail-complaints-to="abuse@iecc.com"
Keywords: tools, comment
Posted-Date: 30 May 2022 14:49:42 EDT
X-submission-address: compilers@iecc.com
X-moderator-address: compilers-request@iecc.com
X-FAQ-and-archives: http://compilers.iecc.com
In-Reply-To: <22-05-058@comp.compilers>
 by: Hans-Peter Diettrich - Mon, 30 May 2022 12:53 UTC

On 5/29/22 11:14 AM, Thomas Koenig wrote:
> Roughly speaking a)-c) is the front end, d) is the middle end and
> e)-f) is the back end.

According to CoCo/R a "compiler" can be anything that deals with formal
input. A generator of binary programs is only one kind of a compiler,
another one is a pretty printer.

I mention this because IMO there is much more demand for non-programming
language processors (HTML, XML...) than for programming language
compilers. I'd appreciate a word of our esteemed mod about the term
"compiler" in this group.

DoDi
[We chose the name comp.compilers in the 1980s and it's not going to change,
but I've always considered anything related to analysis or translation of computer
languages to be on topic. -John]

Re: Are there "compiler generators"?

<22-05-064@comp.compilers>

 copy mid

https://www.novabbs.com/devel/article-flat.php?id=419&group=comp.compilers#419

 copy link   Newsgroups: comp.compilers
Path: i2pn2.org!i2pn.org!weretis.net!feeder6.news.weretis.net!news.misty.com!news.iecc.com!.POSTED.news.iecc.com!nerds-end
From: 480-992-...@kylheku.com (Kaz Kylheku)
Newsgroups: comp.compilers
Subject: Re: Are there "compiler generators"?
Date: Mon, 30 May 2022 20:20:44 -0000 (UTC)
Organization: A noiseless patient Spider
Lines: 49
Sender: news@iecc.com
Approved: comp.compilers@iecc.com
Message-ID: <22-05-064@comp.compilers>
References: <22-05-054@comp.compilers>
Injection-Info: gal.iecc.com; posting-host="news.iecc.com:2001:470:1f07:1126:0:676f:7373:6970";
logging-data="94264"; mail-complaints-to="abuse@iecc.com"
Keywords: code, design
Posted-Date: 30 May 2022 16:27:52 EDT
X-submission-address: compilers@iecc.com
X-moderator-address: compilers-request@iecc.com
X-FAQ-and-archives: http://compilers.iecc.com
 by: Kaz Kylheku - Mon, 30 May 2022 20:20 UTC

On 2022-05-28, Roger L Costello <costello@mitre.org> wrote:
> Hi Folks,
>
> There are lexer generators. Flex is a lexer generator.
>
> There are parser generators. Bison is a parser generator.
>
> Are there compiler generators?

Writing a compiler isn't something that, as a whole, requires code
generation.

The reason is that the bulk of it can be handled by libraries.

Say you want to write a working compiler with minimal effort.

You have some input syntax and so for that there is scanner and parser
generation with which you write your front end.

What helps you avoid writing the middle, and the back-end from scratch?
More code generation of some kind?

Quite simply, some compiler construction components which have API's
that you use. Ideally speaking, here is something like how it goes: in
your parser's actions, you use the API's to generate the right
intermediate representation of the code, and then hand it off to some
function that handles the remaining processing. The resulting compiler
then fits into an existing toolchain framework as a first-class
citizen.

There isn't any part of that process which screams "this needs code
generation". Really, doing the pattern matching to handle your source
syntax and translate it into the intermediate representation is the main
part of the system that benefits from code generation, and
that's about it.

What could help you would be a specialized parser construction language
which has some constructs that can be used in the actions to ease the
translations to the intermediate form. That would then be closer to a
"compiler generator", perhaps: something that not only writes the
parser, where you write procedural code for every phrase structure rule,
but something which has a rewrite notation that lets write the
source->intermediate translation scheme declaratively, using higher
level concepts from that particular intermediate rep, rather than, say,
low C language calls into its API.

--
TXR Programming Language: http://nongnu.org/txr
Cygnal: Cygwin Native Application Library: http://kylheku.com/cygnal

Re: Are there "compiler generators"?

<22-05-065@comp.compilers>

 copy mid

https://www.novabbs.com/devel/article-flat.php?id=420&group=comp.compilers#420

 copy link   Newsgroups: comp.compilers
Path: i2pn2.org!i2pn.org!weretis.net!feeder6.news.weretis.net!news.misty.com!news.iecc.com!.POSTED.news.iecc.com!nerds-end
From: DrDiettr...@netscape.net (Hans-Peter Diettrich)
Newsgroups: comp.compilers
Subject: Re: Are there "compiler generators"?
Date: Tue, 31 May 2022 12:57:18 +0200
Organization: Compilers Central
Lines: 11
Sender: news@iecc.com
Approved: comp.compilers@iecc.com
Message-ID: <22-05-065@comp.compilers>
References: <22-05-054@comp.compilers> <22-05-058@comp.compilers> <22-05-063@comp.compilers>
Mime-Version: 1.0
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Transfer-Encoding: 8bit
Injection-Info: gal.iecc.com; posting-host="news.iecc.com:2001:470:1f07:1126:0:676f:7373:6970";
logging-data="59967"; mail-complaints-to="abuse@iecc.com"
Keywords: design, comment
Posted-Date: 31 May 2022 11:20:05 EDT
X-submission-address: compilers@iecc.com
X-moderator-address: compilers-request@iecc.com
X-FAQ-and-archives: http://compilers.iecc.com
 by: Hans-Peter Diettrich - Tue, 31 May 2022 10:57 UTC

On 5/30/22 2:53 PM, Hans-Peter Diettrich wrote:

> analysis or translation of computer languages to be on topic. -John]

What are those "computer" languages? I'd prefer "formal" languages
(Chomsky...) instead. E.g. Meta§ also was used for DNA analysis.

DoDi
[I was going to say artificial languages but I don't think we have anything
useful to say about Esperanto. In practice it hasn't been very hard to keep
discussions more or less on topic. -John]

Re: Are there "compiler generators"?

<22-05-066@comp.compilers>

 copy mid

https://www.novabbs.com/devel/article-flat.php?id=421&group=comp.compilers#421

 copy link   Newsgroups: comp.compilers
Path: i2pn2.org!i2pn.org!weretis.net!feeder6.news.weretis.net!news.misty.com!news.iecc.com!.POSTED.news.iecc.com!nerds-end
From: gah...@u.washington.edu (gah4)
Newsgroups: comp.compilers
Subject: Re: Are there "compiler generators"?
Date: Tue, 31 May 2022 16:55:22 -0700 (PDT)
Organization: Compilers Central
Lines: 30
Sender: news@iecc.com
Approved: comp.compilers@iecc.com
Message-ID: <22-05-066@comp.compilers>
References: <22-05-054@comp.compilers> <22-05-058@comp.compilers> <22-05-063@comp.compilers> <22-05-065@comp.compilers>
Mime-Version: 1.0
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
Injection-Info: gal.iecc.com; posting-host="news.iecc.com:2001:470:1f07:1126:0:676f:7373:6970";
logging-data="67424"; mail-complaints-to="abuse@iecc.com"
Keywords: design, comment
Posted-Date: 31 May 2022 22:21:19 EDT
X-submission-address: compilers@iecc.com
X-moderator-address: compilers-request@iecc.com
X-FAQ-and-archives: http://compilers.iecc.com
In-Reply-To: <22-05-065@comp.compilers>
 by: gah4 - Tue, 31 May 2022 23:55 UTC

On Tuesday, May 31, 2022 at 8:20:08 AM UTC-7, Hans-Peter Diettrich wrote:
> On 5/30/22 2:53 PM, Hans-Peter Diettrich wrote:

> > analysis or translation of computer languages to be on topic. -John]
> What are those "computer" languages? I'd prefer "formal" languages
> (Chomsky...) instead. E.g. Meta§ also was used for DNA analysis.

> [I was going to say artificial languages but I don't think we have anything
> useful to say about Esperanto. In practice it hasn't been very hard to keep
> discussions more or less on topic. -John]

Definitely interpreters and macro processors have been discussed,
though some might not call them compilers. Also text processors
like TeX.

I am wondering, though, about (human) language translators.
It seems that many use non-deterministic AI systems, and so are
fundamentally different from most of what is discussed here.

If you parse Esperanto with a Flex/Bison parser then it should
be fine here. If you parse Fortran with a deep neural net, then
maybe not.
[That's essentially what I've been thinking. In the 1950s and 1960s
there was a lot of work trying to do human language translation using
formal methods and it worked very poorly, sort of adequate for
translating technical manuals, not for anything else. The breakthrough
was when someone at Google realized that the Candian parliament's
Hansard, the transcript of debates, had high quality parallel
French/English translations going back a century and they fed it
into their machine learning systems. -John]

RE: Are there compiler generators?

<22-06-002@comp.compilers>

 copy mid

https://www.novabbs.com/devel/article-flat.php?id=423&group=comp.compilers#423

 copy link   Newsgroups: comp.compilers
Path: i2pn2.org!i2pn.org!weretis.net!feeder6.news.weretis.net!news.misty.com!news.iecc.com!.POSTED.news.iecc.com!nerds-end
From: christop...@compiler-resources.com (Christopher F Clark)
Newsgroups: comp.compilers
Subject: RE: Are there compiler generators?
Date: Wed, 1 Jun 2022 14:07:46 +0300
Organization: Compilers Central
Lines: 63
Sender: news@iecc.com
Approved: comp.compilers@iecc.com
Message-ID: <22-06-002@comp.compilers>
References: <22-05-054@comp.compilers> <22-05-058@comp.compilers> <22-05-063@comp.compilers> <22-05-065@comp.compilers> <22-05-066@comp.compilers>
Mime-Version: 1.0
Content-Type: text/plain; charset="UTF-8"
Injection-Info: gal.iecc.com; posting-host="news.iecc.com:2001:470:1f07:1126:0:676f:7373:6970";
logging-data="18088"; mail-complaints-to="abuse@iecc.com"
Keywords: design
Posted-Date: 01 Jun 2022 13:17:41 EDT
X-submission-address: compilers@iecc.com
X-moderator-address: compilers-request@iecc.com
X-FAQ-and-archives: http://compilers.iecc.com
 by: Christopher F Clark - Wed, 1 Jun 2022 11:07 UTC

Although there are many wonderful examples in replies already. I want
to mention the Eli "system"
http://eli-project.sourceforge.net/

It was (is) an attempt to do exactly that. It did so by cobbling
together different tools that different parts of a compiler. One in
particular is worth mentioning (Oil) which handled static type systems
and which types were compatible and when coercions were applicable. I
never found the overall system to be that usable because it didn't
really unify the tools to work well together, so that the notations
tended to stay divergent.

In fact, that is partially why in our (Compiler Resources) version of
Yacc++ we integrated the lexer and used Yacc-like notation for it. We
actually did it, because using one notation simply makes sense, but
seeing a tool which used multiple non-integrated notations simply
reinforced that opinion.

-----

But, the main point is that compilers are a big area with lots of
little parts, some of them have tools and others don't. More
importantly, much of the development in compilers since 1991 has been
in hand-written parts. I don't think there is a single production
quality compiler for C++ using anything but a hard-written parser.
That to me is a shame. What we have instead of tools are more
"frameworks" e.g. gcc and the gnu-compiler-collection, CLANG, even the
MIPS suite (Chow's work). LLVM (and possiblu MLIR) are bright spots
that the trend might be reversing. It's a pendulum, swinging one way
than the other.

------

And finally to Dr Dietrich's and our moderators point. I think that
compilers hand-written or tool generated are all appropriate,
including things like TeX and HTML etc. That would also include IDEs.
However, AI systems that aren't self explanatory and which we don't
know exactly what they recognize are probably not. That's a different
technology. Fortunately, it sounds like there are AI researchers
attempting to make AI systems that do "explain" what their reasoning
is. I might include them especially if they help us advance our own
understanding. The following article talks about it:

https://www.quantamagazine.org/machine-scientists-distill-the-laws-of-physics-from-raw-data-20220510/

------

Sorry, that I glommed three replies into one. It's just that I am
still in a rush to get some software finished for my "day" job
(despite having once retired but then un-retired to do interesting
stuff and some not so interesting) and taking time to write these
replies definitely impacts that.

Best regards,
Chris

--
******************************************************************************
Chris Clark email: christopher.f.clark@compiler-resources.com
Compiler Resources, Inc. Web Site: http://world.std.com/~compres
23 Bailey Rd voice: (508) 435-5016
Berlin, MA 01503 USA twitter: @intel_chris
------------------------------------------------------------------------------

Re: Are there "compiler generators"?

<22-06-022@comp.compilers>

 copy mid

https://www.novabbs.com/devel/article-flat.php?id=442&group=comp.compilers#442

 copy link   Newsgroups: comp.compilers
Path: i2pn2.org!rocksolid2!i2pn.org!weretis.net!feeder6.news.weretis.net!news.misty.com!news.iecc.com!.POSTED.news.iecc.com!nerds-end
From: acol...@efunct.com (mac)
Newsgroups: comp.compilers
Subject: Re: Are there "compiler generators"?
Date: Thu, 9 Jun 2022 14:12:51 -0000 (UTC)
Organization: A noiseless patient Spider
Lines: 8
Sender: news@iecc.com
Approved: comp.compilers@iecc.com
Message-ID: <22-06-022@comp.compilers>
References: <22-05-054@comp.compilers> <22-05-061@comp.compilers>
Mime-Version: 1.0
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: 8bit
Injection-Info: gal.iecc.com; posting-host="news.iecc.com:2001:470:1f07:1126:0:676f:7373:6970";
logging-data="68081"; mail-complaints-to="abuse@iecc.com"
Keywords: tools, history
Posted-Date: 09 Jun 2022 12:49:58 EDT
X-submission-address: compilers@iecc.com
X-moderator-address: compilers-request@iecc.com
X-FAQ-and-archives: http://compilers.iecc.com
 by: mac - Thu, 9 Jun 2022 14:12 UTC

> Are there compiler generators?

Our esteemed moderator no doubt remembers PQCC
(https://en.wikipedia.org/wiki/PQCC) the Production-Quality-Compiler
Compiler back in the 80s.

Our idea of a production-quality compiler has evolved since then, both in
language semantics and in code quality

1
server_pubkey.txt

rocksolid light 0.9.7
clearnet tor