Hello everyone.
i'm currently trying to build an executable using GForth's built in assembler, but apparently it, and most of the assemblers that are written in forth only assemble the code to be used as a word. Is it possible to somehow redirect the assemblers or something similar to write the machine code to an allocated chunk of memory instead of the forth vocabulary?

In article <93b46356-bb52-4da7-bf0b-1a8231c9d2edn@googlegroups.com>,
Krzysztof Jeszke <krzysztofjeszke0@gmail.com> wrote:
>Hello everyone.
>i'm currently trying to build an executable using GForth's built in assembler, but apparently it, and most of the assemblers that are written in forth only
>assemble the code to be used as a word. Is it possible to somehow redirect the assemblers or something similar to write the machine code to an allocated
>chunk of memory instead of the forth vocabulary?

First of all, the choice of assembler matters. The following is about
Intel 386.

If you want to tie Forth to assembler, your are better off with a Forth
written in assembler, where the correspondance is clear.
If you compile in ciforth :
lina -c hello.frt
you generate an executable `hello',
with an intermediate step that identifies
the part of memory that is written to the executable.
It is not hard to extend lina.asm with the assembler code
that result in hello.asm, provided that hello is small.

There is a reverse engineering assembler written in ciforth:
https://github.com/albertvanderhorst/ciasdis
These can generate an assembler file from an executable, and then
regenerate the executable. That second part you could port to gforth.
Examples are part of the tests.
A prestigious example is to generate version 4.0.5 of lina 32 itself,
analysing the code and generating labels found in the executable.
Expect to write a Forth code plug in to extract these labels.

Likewise a 64 bit lina is reverse engineered and reassembled, but this
is not transferred to github yet.
Part of the Makefile:

lina530.asm : ciasdis lina530crawl.cul testcmp/lina530
ciasdis -d testcmp/lina530 lina530crawl.cul >$@
ciasdis -a $@ lina530
$(DIFF_BIN) lina530
$(DIFF_TXT) $@

So the lina530 in testcmp is disassembled (-d) generating lina530.asm.
Then this source is assembler (-a) to a lina530.
The files are identical to the ones in testcmp, or the test fails.

Groetjes Albert
--
"in our communism country Viet Nam, people are forced to be
alive and in the western country like US, people are free to
die from Covid 19 lol" duc ha
albert@spe&ar&c.xs4all.nl &=n http://home.hccnet.nl/a.w.m.van.der.horst

On 28 Jul 2022 at 22:04:14 CEST, "Krzysztof Jeszke"
<krzysztofjeszke0@gmail.com> wrote:

> Hello everyone.
> i'm currently trying to build an executable using GForth's built in assembler,
> but apparently it, and most of the assemblers that are written in forth only
> assemble the code to be used as a word. Is it possible to somehow redirect the
> assemblers or something similar to write the machine code to an allocated
> chunk of memory instead of the forth vocabulary?

You may find this easier to do with a native Forth system such as VFX.

Nearly all Forth systems defined one or more pairs of words to turn the
assembler on and off. These deal with placing names in the right place, and to
put the dictionary pointer HERE in the right place. It's not magic. What you
want to do is (probably) to put the names in the Forth and the assembled code
at HERE. To do this you will need to know how to change the address returned
by HERE.

The other way to do this is to use a cross compiler. Most Forth cross
compilers are undocumented and user-hostile. The only ones with real
documentation and a focus on usability are IMHO the ones from the commercial
vendors.

Stephen
--
Stephen Pelc, stephen@vfxforth.com
MicroProcessor Engineering, Ltd. - More Real, Less Time
133 Hill Lane, Southampton SO15 5AF, England
tel: +44 (0)23 8063 1441, +44 (0)78 0390 3612,
+34 649 662 974
http://www.mpeforth.com - free VFX Forth downloads

Hi Krzysztof!

Stephen Pelc's 64-bit VFX Forth not only has the best code generator on the market, but also the best assembler / disassembler due to the syntax close to Intel syntax, as well as adding encoding when disassembling in the comment, which allows you to use the result as a source. And it's made available to the community for free if you don't intend to make money.

I will show you and others how to assemble in VFX Forth in a selected block of memory.

First, let's prepare a memory block (for example 4KB) and write its address in word a:
$1000 allocate drop value a

Second, we will prepare the h word to store the current HERE value so that we can restore the state it was before changing the place for assembly:
here value h

Now we can start defining the word in assembly language. While defining it, we will change the assembly target place. We will remove this word afterwards.
code krzysztof
here to h
a h - allot

Now we can assemble from address a with the convenient 64-bit VFX Forth assembler, for example:
mov rax, rbx
mov r11, rax
inc r11
inc rbx
nop
It is important that the last instruction will not be placed in the memory block provided by us because VFX Forth will put it in the dictionary when terminating the word definition with END-CODE. Therefore, we add an unnecessary nop at the end.

Now we're going to bring the HERE pointer back to its pre-assembly state and finish defining our helper word:
h here - allot
end-code

You can see what has been assembled from address a (first 30 bytes) as follows:
a 30 disasm/al
Likewise, you probably want to see out of curiosity what our temporary word looks like:
' krzysztof 30 disasm/al
Finally, you can delete the temporary word:
forget krzysztof
Your clean assembly code is in your block from address a :)
Cheers!

In article <6c5f6b33-ddf3-47ab-aa77-958f9972e783n@googlegroups.com>,
Micha� Kasprzak <siarczek83@gmail.com> wrote:
>Hi Krzysztof!
>
> Stephen Pelc's 64-bit VFX Forth not only has the best code generator
> on the market, but also the best assembler / disassembler due to the
> syntax close to Intel syntax, as well as adding encoding when
> disassembling in the comment, which allows you to use the result as a
> source. And it's made available to the community for free if you don't
> intend to make money.

I will agree that the VFX is one of the best choices of assembling 64
bit Intel in Forth. Bluntly asserting that it is the best, cannot fly,
because there can be numerous considerations valid, applicable to
any sort of situation.

A requirement that ciasdis observes is that a object can be disassembled
and then be assembled to the exact same object.
A litmus test is that the assembler makes a distinction between the
two short forms of the instruction:

MOV AX BX \ or EAX or RAX

(not counting longer forms of the same instruction.)
In ciasdis this is either of the two forms:
MOV, X| T| BX'| R| AX| \ AX --> BX to
MOV, X| F| AX'| R| BX| \ AX --> BX from

If you investigate viruses, you will need this.

<SNIP>
>mov rax, rbx
<NIP>
>Cheers!

Groetjes Albert
--
"in our communism country Viet Nam, people are forced to be
alive and in the western country like US, people are free to
die from Covid 19 lol" duc ha
albert@spe&ar&c.xs4all.nl &=n http://home.hccnet.nl/a.w.m.van.der.horst

> You may find this easier to do with a native Forth system such as VFX.

But from what I see on „download” page VFX is family of „hosted” Forths,
not „native” at all?

Krzysztof Jeszke <krzysztofjeszke0@gmail.com> writes:
>Hello everyone.
>i'm currently trying to build an executable using GForth's built in assembl=
>er, but apparently it, and most of the assemblers that are written in forth=
> only assemble the code to be used as a word.

Yes, that's the primary purpose.

>Is it possible to somehow red=
>irect the assemblers or something similar to write the machine code to an a=
>llocated chunk of memory instead of the forth vocabulary?

The most straightforward way is to define a section (in a recent
development snapshot of Gforth), maybe something like

1024 1024 * extra-section mycode
' section# mycode #>section
assember
.... \ assemble away

For executables you also need to store the meta-information of the
exeutable file formats, and you will have to find out how to handle
that yourself.

- anton
--
M. Anton Ertl http://www.complang.tuwien.ac.at/anton/home.html
comp.lang.forth FAQs: http://www.complang.tuwien.ac.at/forth/faq/toc.html
New standard: https://forth-standard.org/
EuroForth 2022: http://www.euroforth.org/ef22/cfp.html

On Thursday, July 28, 2022 at 3:04:16 PM UTC-5, Krzysztof Jeszke wrote:
> Hello everyone.
> i'm currently trying to build an executable using GForth's built in assembler, but apparently it, and most of the assemblers that are written in forth only assemble the code to be used as a word. Is it possible to somehow redirect the assemblers or something similar to write the machine code to an allocated chunk of memory instead of the forth vocabulary?

I think you are asking for a Forth assembler to just lay down code
instead of laying down some minimal Forth and adapting the Forth to
the application. Typical from what I have seen is the latter and have
seen comments from noted experts as why anyone would want the former.
Whatever, I'm not interested in such a debate.

In past on DOS I made some small comm programs just compiling, "C," ,
machine code. Today, hosted by Linux, I can send code to its assembler
and have the results compiled in a word. Haven't really used it other
than seeing it can be done. Also, why bring Forth into it, just do the
assembly.

Another possibility having an assembly based Forth is just use the
assembler and its Forth primitives, perhaps with some adjustment, to
lay agnostic code. Haven't tried it but long back I saw someone's
assembler that had a set of Forth macros to just do that or so I
thought.
--
me

On 29 Jul 2022 at 14:21:18 CEST, "Zbig" <zbigniew2011@gmail.com> wrote:

>> You may find this easier to do with a native Forth system such as VFX.
>
> But from what I see on „download” page VFX is family of „hosted” Forths,
> not „native” at all?

As opposed to being written in C or some other language.
I meant a Forth written in Forth. Sorry for the confusion.

Stephen
--
Stephen Pelc, stephen@vfxforth.com
MicroProcessor Engineering, Ltd. - More Real, Less Time
133 Hill Lane, Southampton SO15 5AF, England
tel: +44 (0)23 8063 1441, +44 (0)78 0390 3612,
+34 649 662 974
http://www.mpeforth.com - free VFX Forth downloads

Friday, July 29, 2022 at 17:20:06 UTC+2 S Jack wrote:

> I think you are asking for a Forth assembler to just lay down code
> instead of laying down some minimal Forth and adapting the Forth to
> the application. Typical from what I have seen is the latter and have
> seen comments from noted experts as why anyone would want the former.
> Whatever, I'm not interested in such a debate.

I'm aiming to write a metacompiled forth in a combination of forth and assembly, which the goal of is it being a minimal bootable forth for multiple platforms. One can't simply make an executable without any assembly.

On Friday, July 29, 2022 at 5:30:19 PM UTC+2, Krzysztof Jeszke wrote:
> Friday, July 29, 2022 at 17:20:06 UTC+2 S Jack wrote:
>
> > I think you are asking for a Forth assembler to just lay down code
> > instead of laying down some minimal Forth and adapting the Forth to
> > the application. Typical from what I have seen is the latter and have
> > seen comments from noted experts as why anyone would want the former.
> > Whatever, I'm not interested in such a debate.
> I'm aiming to write a metacompiled forth in a combination of forth and assembly,
> which the goal of is it being a minimal bootable forth for multiple platforms.
> One can't simply make an executable without any assembly.

I have done it that way since the MSDOS days, and before that
on Motorola's 68k, Inmos transputer, and Z80.

It is always a hassle when the processor architecture upgrades
wordsize (16->32->64bits->...), or when changing from one processor
to another (e.g. from 68K to Intel to M1). Given the current state-of-the-art,
I don't think I'll need to write a 128 or 256bit Forth for a radically new
processor that is orders of magnitude better than the ice-giants, anytime soon.

-marcel

Krzysztof Jeszke schrieb am Freitag, 29. Juli 2022 um 17:30:19 UTC+2:
> Friday, July 29, 2022 at 17:20:06 UTC+2 S Jack wrote:
>
> > I think you are asking for a Forth assembler to just lay down code
> > instead of laying down some minimal Forth and adapting the Forth to
> > the application. Typical from what I have seen is the latter and have
> > seen comments from noted experts as why anyone would want the former.
> > Whatever, I'm not interested in such a debate.
> I'm aiming to write a metacompiled forth in a combination of forth and assembly, which the goal of is it being a minimal bootable forth for multiple platforms. One can't simply make an executable without any assembly.

We had been doing that in the eighties and nineties for programmable controllers,
and with every new CPU or controller generation - or sometimes by customer requirement -
we had to reengineer and reimplement and test and document a new kernel variant.
What a hassle - particularly for maintenance !!!!

Then we switched from assembly to C and never looked back.

On Thursday, July 28, 2022 at 4:04:16 PM UTC-4, Krzysztof Jeszke wrote:
> Hello everyone.
> i'm currently trying to build an executable using GForth's built in assembler, but apparently it, and most of the assemblers that are written in forth only assemble the code to be used as a word. Is it possible to somehow redirect the assemblers or something similar to write the machine code to an allocated chunk of memory instead of the forth vocabulary?

This is totally obvious to the old hands here but since it wasn't mentioned specifically
I will stick my neck out because I didn't understand this years ago either..
When I realized how simple it was I was kind of shocked. :-)

You can make a Forth cross-Assembler with special pointable versions of comma,
w-comma, c-comma and whatever other number compilers your architecture needs.
You also need a new HERE and ALLOT for the target memory space.

Once you have those, you just re-compile the assembler with those "pointable"
versions of comma replacing the generic versions and replacing HERE with THERE.

From a quick scan of GForth source it looks like the various asm.fs files are available.
This might be more trouble than it's worth for you needs, but it does work.

Here is how my cross-assembler begins for a 16 bit machine:

VARIABLE TDP \ "target dictionary point"

\ set where the Cross-assembler puts its code
: ORG ( addr -- ) TDP ! ;

\ Target versions of HERE and ALLOT
: THERE ( -- addr) TDP @ ;
: TALLOT ( n -- ) TDP +! ;

\ integer and byte "Target" compilers
: T, ( n -- ) THERE ! 2 TALLOT ;
: TC, ( c -- ) THERE C! 1 TALLOT ;