After some research, for a project I have used "https://wiki.tcl-lang.org/page/Pure-tcl+readline2" to make an interactive tcl shell. The absence of compilation was the driving force for me.
I am running this through "expect", not TclX. It is working well, but I am observing a huge performance hit when running anything in it VS pure tcl.

Specifically, in interactive mode if I do:
source a_noticeably_long_operation.tcl
VS
tclsh a_noticeably_long_operation.tcl

I get a 10x perf degradation in the interactive session, through "source". This took me a bit by surprise since I am using 'source' - so I should be 'out' of the interactive wrapper.

Digging a bit into the code of the pure tcl readline, I am not clear what is causing the issue, and if any 'trick' could make it work better. I observe that a lot happens to stdin and stdout, but I don't see how that can relates to the perf of say 'source'.

I walked around the tcl wiki on this subject of 'pure tcl' readlines alternative, but could not find anything usefull on performance.

Does anyone understanding a bit how this pure tcl wrapper works "in details" could point me where they might be slow ? If not known, I will have to profile that and this looks like a daunting task ...

Thanks in for any pointers,

Best regards,

Sebastien.

Le mardi 7 février 2023 à 14:51:16 UTC+1, Sebastien Marchal a écrit :
> After some research, for a project I have used "https://wiki.tcl-lang.org/page/Pure-tcl+readline2" to make an interactive tcl shell. The absence of compilation was the driving force for me.
> I am running this through "expect", not TclX. It is working well, but I am observing a huge performance hit when running anything in it VS pure tcl.
>
> Specifically, in interactive mode if I do:
> source a_noticeably_long_operation.tcl
> VS
> tclsh a_noticeably_long_operation.tcl
>
> I get a 10x perf degradation in the interactive session, through "source".. This took me a bit by surprise since I am using 'source' - so I should be 'out' of the interactive wrapper.
>
> Digging a bit into the code of the pure tcl readline, I am not clear what is causing the issue, and if any 'trick' could make it work better. I observe that a lot happens to stdin and stdout, but I don't see how that can relates to the perf of say 'source'.
>
> I walked around the tcl wiki on this subject of 'pure tcl' readlines alternative, but could not find anything usefull on performance.
>
> Does anyone understanding a bit how this pure tcl wrapper works "in details" could point me where they might be slow ? If not known, I will have to profile that and this looks like a daunting task ...
>
> Thanks in for any pointers,
>
> Best regards,
>
> Sebastien.
I have inspected further the problem.

It 'thickens' on my side, and the problem is not 'that generic'. I created some dummy long operations, and pure tcl run the same. Those are other things - specific to me - that takes much longer. Those are tcl extension in C++ provided by others, and because it is C++ I assumed could not be them.
I will continue the profiling, but my request was a bit premature ... If I find the issue the culprit and it can be of general interest, I'll update the thread.

Sebastien.

Le jeudi 9 février 2023 à 14:51:26 UTC+1, Sebastien Marchal a écrit :
> Le mardi 7 février 2023 à 14:51:16 UTC+1, Sebastien Marchal a écrit :
> > After some research, for a project I have used "https://wiki.tcl-lang.org/page/Pure-tcl+readline2" to make an interactive tcl shell. The absence of compilation was the driving force for me.
> > I am running this through "expect", not TclX. It is working well, but I am observing a huge performance hit when running anything in it VS pure tcl.
> >
> > Specifically, in interactive mode if I do:
> > source a_noticeably_long_operation.tcl
> > VS
> > tclsh a_noticeably_long_operation.tcl
> >
> > I get a 10x perf degradation in the interactive session, through "source". This took me a bit by surprise since I am using 'source' - so I should be 'out' of the interactive wrapper.
> >
> > Digging a bit into the code of the pure tcl readline, I am not clear what is causing the issue, and if any 'trick' could make it work better. I observe that a lot happens to stdin and stdout, but I don't see how that can relates to the perf of say 'source'.
> >
> > I walked around the tcl wiki on this subject of 'pure tcl' readlines alternative, but could not find anything usefull on performance.
> >
> > Does anyone understanding a bit how this pure tcl wrapper works "in details" could point me where they might be slow ? If not known, I will have to profile that and this looks like a daunting task ...
> >
> > Thanks in for any pointers,
> >
> > Best regards,
> >
> > Sebastien.
> I have inspected further the problem.
>
> It 'thickens' on my side, and the problem is not 'that generic'. I created some dummy long operations, and pure tcl run the same. Those are other things - specific to me - that takes much longer. Those are tcl extension in C++ provided by others, and because it is C++ I assumed could not be them.
> I will continue the profiling, but my request was a bit premature ... If I find the issue the culprit and it can be of general interest, I'll update the thread.
>
> Sebastien.

Continuing my own monolog, and I found the issue.

Indeed, the 'pure tcl' wrapper include a cutomized 'unkown' to enable calling aliases or shell commands. And it does does that for anything - ie it first check for this before calling the standard unkown.
In my case, the C++ code extension uses an 'objected oriented' still for calling thing with a new procedure for every object created so that you can do $object call_this_method instead of call_this_method $object. Not 100% sure of what is going in here, but clearly disabling the special unkown of the pure tcl readline returns performance to the same level in interactive VS batch.

So, I need to see how to improve the tcl readline unkown or the c++ thing to avoid calling unkown over and over.

On Thursday, February 9, 2023 at 6:17:45 PM UTC+1, Sebastien Marchal wrote:
> Le jeudi 9 février 2023 à 14:51:26 UTC+1, Sebastien Marchal a écrit :
> > Le mardi 7 février 2023 à 14:51:16 UTC+1, Sebastien Marchal a écrit :
> Continuing my own monolog, and I found the issue.

While you are the lone writer, be assured that the community can benefit! I was reading :-)

> Indeed, the 'pure tcl' wrapper include a cutomized 'unkown' to enable calling aliases or shell commands. And it does does that for anything - ie it first check for this before calling the standard unkown.
> In my case, the C++ code extension uses an 'objected oriented' still for calling thing with a new procedure for every object created so that you can do $object call_this_method instead of call_this_method $object. Not 100% sure of what is going in here, but clearly disabling the special unkown of the pure tcl readline returns performance to the same level in interactive VS batch.
>
> So, I need to see how to improve the tcl readline unkown or the c++ thing to avoid calling unkown over and over.

If that extension is using unknown and introspection for the "new procedure for every object", then aliases could be the way to go.