Den 2023-04-13 kl. 15:24, skrev Arne Vajhøj:
> On 4/13/2023 8:26 AM, Simon Clubley wrote:
>> On 2023-04-13, Arne Vajhøj <arne@vajhoej.dk> wrote:
>>> On 4/13/2023 6:44 AM, Neil Rieck wrote:
>>>> And although python is used to do a lot of admin scripting on Linux
>>>> platforms (tools like "yum" and "firewall-cmd" are written in
>>>> python), python does a really good job supporting server-side
>>>> scripting under apache cgi-bin. I don't know how this is possible,
>>>> but python programs are faster than compiled DEC-BASIC programs
>>>> started via apache.
>>>
>>> That is unexpected. If both are run via CGI mechanism then
>>> script activation should be the same. And compiled code should be
>>> faster than interpreted code. And if libraries used are also the
>>> same then it is a mystery.
>>>
>>
>> Maybe. Maybe not. Based purely on some comments from people here who
>> know/use DEC Basic, it's quite possible that DEC Basic has a RTL which
>> imposes some serious runtime overheads.
>
> VMS Basic is a language with a high level of abstraction
> so more "happens under the hood" than in C/Fortran/Cobol.
>
> There are some overhead due to that.
>
> Obvious example is strings. VMS Basic use dynamic
> strings. The RTL does some work there.
>
> But Python is even more high level and does similar
> work - including the equivalent of dynamic strings.
>
> But it is still interpreted (for standard Python
> implementation) while VMS Basic get compiled.
>
> Only thing in Pythons favor is that there must
> be spent 1000 times more effort optimizing Python
> libraries than optimizing VMS Basic RTL.
>
> Arne
>
>

Are my post about standard-CGI vs. "loadable modules" lost?

On 4/13/2023 10:50 AM, Jan-Erik Söderholm wrote:
> Den 2023-04-13 kl. 15:24, skrev Arne Vajhøj:
>> On 4/13/2023 8:26 AM, Simon Clubley wrote:
>>> On 2023-04-13, Arne Vajhøj <arne@vajhoej.dk> wrote:
>>>> On 4/13/2023 6:44 AM, Neil Rieck wrote:
>>>>> And although python is used to do a lot of admin scripting on Linux
>>>>> platforms (tools like "yum" and "firewall-cmd" are written in
>>>>> python), python does a really good job supporting server-side
>>>>> scripting under apache cgi-bin. I don't know how this is possible,
>>>>> but python programs are faster than compiled DEC-BASIC programs
>>>>> started via apache.
>>>>
>>>> That is unexpected. If both are run via CGI mechanism then
>>>> script activation should be the same. And compiled code should be
>>>> faster than interpreted code. And if libraries used are also the
>>>> same then it is a mystery.
>>>
>>> Maybe. Maybe not. Based purely on some comments from people here who
>>> know/use DEC Basic, it's quite possible that DEC Basic has a RTL which
>>> imposes some serious runtime overheads.
>>
>> VMS Basic is a language with a high level of abstraction
>> so more "happens under the hood" than in C/Fortran/Cobol.
>>
>> There are some overhead due to that.
>>
>> Obvious example is strings. VMS Basic use dynamic
>> strings. The RTL does some work there.
>>
>> But Python is even more high level and does similar
>> work - including the equivalent of dynamic strings.
>>
>> But it is still interpreted (for standard Python
>> implementation) while VMS Basic get compiled.
>>
>> Only thing in Pythons favor is that there must
>> be spent 1000 times more effort optimizing Python
>> libraries than optimizing VMS Basic RTL.
>
> Are my post about standard-CGI vs. "loadable modules" lost?

No. It is there. And it is all true. CGI is not an efficient
method to run code.

But Neil wrote "python does a really good job supporting server-side
scripting under apache cgi-bin" and I assumed that to be correct
with "If both are run via CGI mechanism then script activation
should be the same". The assumption for all of the above is that
Python is being run via CGI similar to Basic.

If that is not the case, then we have a very good explanation.

Arne

On 4/13/2023 8:18 AM, Jan-Erik Söderholm wrote:
> Den 2023-04-13 kl. 12:44, skrev Neil Rieck:
>> And although python is used to do a lot of admin scripting on Linux
>> platforms (tools like "yum" and "firewall-cmd" are written in python),
>> python does a really good job supporting server-side scripting under
>> apache cgi-bin. I don't know how this is possible, but python programs
>> are faster than compiled DEC-BASIC programs started via apache.
>
> The performance depends a lot on if you are using standard Apache CGI
> or Apache "loadable modules". If you are just writing your own server
> apps, I guess they run as standard CGI (one process started per call).
> And I would expect Python to be available as a "loadable module". And
> that makes a big differnce, in particular on VMS with it's process
> creation overhead.
>
> Here are some figures for WASD and Apache.
> http://wasd.vsm.com.au/wasd_root/wasdoc/features/#11.2.scripting

It is well known that CGI is not suited for high performance. And has
been known so for 25+ years.

Note though that the numbers in that link are awfully low per modern
standards. Today tens of thousands of requests per second is expected.

(it is even possible to find benchmarks showing hundreds of thousands
and even millions of requests per second, but those are very special
cases)

> We are using Python 2 in the form of PyRTE (persisting processes) and
> have OK performance. First call (starting the Python process) does have
> a 1-2 sec overhead, but following calls are processes in fractions of a
> second (incl. Rdb processing).
>
> We are currently looking at writing our own persistent server processes
> to see if we can tweek the performance a bit more by using compiled
> code (Cobol, in our case).
>
> Using a test C application (just returns a timestamp to the browser, the
> reply time including starting the VMS process is 172 ms. Additional
> calls to the server with the process started are around 60 ms. As
> reported by
> the performance tools in Firefox.

Why not try something like Python Flask?

Arne

On 2023-04-13, Arne Vajhøj <arne@vajhoej.dk> wrote:
>
> I remember back when Perl, Python and Tcl was 3 equivalent choices
> for scripting.
>
> They evolved very different - Perl stayed where it was, Python became
> a widely used language and Tcl is practically dead.
>
> Why did Python become such a success while the other did not?
>
> My guess:
> - Python managed to get into education - not just CS like
> Perl but much broader
> - Python web approaches with Django etc. turned more future oriented
> than Perl CGI
> - Python got on the big data / analytics / ML / AI train
> and when that took off it pulled Python with it
>

Add:

- Python is an excellent way to add automation/scripting capabilities to
a wide range of applications.

Try doing the following with Perl: :-)

https://docs.blender.org/api/current/info_overview.html

Another example:

https://www.panda3d.org/

In this example, you can use either C++ and Python depending on your
requirements.

To compare the Python and C++ options:

https://docs.panda3d.org/1.10/python/index

Python has turned out to be a _really_ good way to provide a very
wide range of scripting functionality across the computing world
in general.

Simon.

--
Simon Clubley, clubley@remove_me.eisner.decus.org-Earth.UFP
Walking destinations on a map are further away than they appear.

On 4/14/23 7:19 AM, Simon Clubley wrote:
> On 2023-04-13, Arne Vajhøj <arne@vajhoej.dk> wrote:
>>
>> I remember back when Perl, Python and Tcl was 3 equivalent choices
>> for scripting.
>>
>> They evolved very different - Perl stayed where it was, Python became
>> a widely used language and Tcl is practically dead.
>>
>> Why did Python become such a success while the other did not?
>>
>> My guess:
>> - Python managed to get into education - not just CS like
>> Perl but much broader

The P in Perl stands for "Practical." Larry Wall is a linguist, not a
computer scientist, and computer scientists can rarely be accused of
being practical :-). There have been books on such things as functional
programming in Perl but I don't think it was ever big with computer
scientists.

>> - Python web approaches with Django etc. turned more future oriented
>> than Perl CGI

The use of a web framework like Django has nothing to do with the
presence or absence of CGI. You can in fact run Django using CGI and
people used to do that. But for some time now you wouldn't because WSGI
is better. It's no different in the Perl world. While you can run Perl
scripts via CGI, today you'd be more like to use PSGI.

>> - Python got on the big data / analytics / ML / AI train
>> and when that took off it pulled Python with it
>>
>
> Add:
>
> - Python is an excellent way to add automation/scripting capabilities to
> a wide range of applications.
>
> Try doing the following with Perl: :-)
>
> https://docs.blender.org/api/current/info_overview.html

I saw the smiley, but what exactly is your point? That you can embed a
Python interpreter in another application? Perl has that capability
too, and has had for decades.

I'm not trying to start language war and I have a lot of respect for the
accomplishments of the Python community. But the capabilities of Python
aren't as unique as some people seem to think and probably haven't
played that big a role in its market dominance.

On 2023-04-14, Craig A. Berry <craigberry@nospam.mac.com> wrote:
>
> On 4/14/23 7:19 AM, Simon Clubley wrote:
>>
>> Add:
>>
>> - Python is an excellent way to add automation/scripting capabilities to
>> a wide range of applications.
>>
>> Try doing the following with Perl: :-)
>>
>> https://docs.blender.org/api/current/info_overview.html
>
> I saw the smiley, but what exactly is your point? That you can embed a
> Python interpreter in another application? Perl has that capability
> too, and has had for decades.
>
> I'm not trying to start language war and I have a lot of respect for the
> accomplishments of the Python community. But the capabilities of Python
> aren't as unique as some people seem to think and probably haven't
> played that big a role in its market dominance.
>

My point is that Python has a major thing going for it in that it is
relatively easy to learn and use by people who are not professional
programmers or professional sysadmins. At the same time, it's also
a very powerful tool for people who _are_ experts in these areas.

The two examples I gave were not by accident, especially Blender.

These are examples of programs used by people who are skilled in their
own areas of expertise, but who are not professional programmers or
sysadmins. Python is a relatively easy language for these people to
learn, which is one of the reasons it has really taken off in terms
of this application-level integration, and has also really helped to
drive Python use in general.

Simon.

--
Simon Clubley, clubley@remove_me.eisner.decus.org-Earth.UFP
Walking destinations on a map are further away than they appear.

On 4/14/2023 1:46 PM, Simon Clubley wrote:
>
>
> My point is that Python has a major thing going for it in that it is
> relatively easy to learn and use by people who are not professional
> programmers or professional sysadmins.

And if all there was to programming was syntax that might be a good
thing. But it isn't and having all these totally unqualified asses
writing "programs" is certainly not a good thing.

bill

On 4/14/2023 8:19 AM, Simon Clubley wrote:
> On 2023-04-13, Arne Vajhøj <arne@vajhoej.dk> wrote:
>> Why did Python become such a success while the other did not?
>>
>> My guess:
>> - Python managed to get into education - not just CS like
>> Perl but much broader
>> - Python web approaches with Django etc. turned more future oriented
>> than Perl CGI
>> - Python got on the big data / analytics / ML / AI train
>> and when that took off it pulled Python with it
>
> Add:
>
> - Python is an excellent way to add automation/scripting capabilities to
> a wide range of applications.
>
> Try doing the following with Perl: :-)
>
> https://docs.blender.org/api/current/info_overview.html

It is possible to embed Perl as well.

I would say Perl and Python is similar in that regard.

https://docs.python.org/3/extending/embedding.html

https://perldoc.perl.org/perlembed

More prefer Python than Perl today though.

> Another example:
>
> https://www.panda3d.org/
>
> In this example, you can use either C++ and Python depending on your
> requirements.
>
> To compare the Python and C++ options:
>
> https://docs.panda3d.org/1.10/python/index

Having a scripting language utilize a C/C++ library
is also very standard.

Arne

On 4/14/2023 11:48 AM, Craig A. Berry wrote:
>
> On 4/14/23 7:19 AM, Simon Clubley wrote:
>> On 2023-04-13, Arne Vajhøj <arne@vajhoej.dk> wrote:
>>>
>>> I remember back when Perl, Python and Tcl was 3 equivalent choices
>>> for scripting.
>>>
>>> They evolved very different - Perl stayed where it was, Python became
>>> a widely used language and Tcl is practically dead.
>>>
>>> Why did Python become such a success while the other did not?
>>>
>>> My guess:
>>> - Python managed to get into education - not just CS like
>>>     Perl but much broader
>
> The P in Perl stands for "Practical." Larry Wall is a linguist, not a
> computer scientist, and computer scientists can rarely be accused of
> being practical :-).  There have been books on such things as functional
> programming in Perl but I don't think it was ever big with computer
> scientists.

Maybe not big with Knuth style CS.

But it is my impression that practically all Perl users are
IT professionals. A lot of them with a CS degree even though
they may be more practical than academic.

And I would also call Larry Wall for a programmer even though
he did not study CS.

>>> - Python web approaches with Django etc. turned more future oriented
>>>     than Perl CGI
>
> The use of a web framework like Django has nothing to do with the
> presence or absence of CGI.  You can in fact run Django using CGI and
> people used to do that.  But for some time now you wouldn't because WSGI
> is better. It's no different in the Perl world.  While you can run Perl
> scripts via CGI, today you'd be more like to use PSGI.

I am not an expert in Django but a bit of googling indicate
that Django has never supported CGI and that they
deprecated/removed FastCGI support in 2014/2015. When Django
was invented 2005-2008 then CGI was already an obsolete technology.
All Django tutorials I can find do not use CGI or FastCGI.

I am sure there are other options than CGI for Perl (even I have
heard about mod_perl). And I am also sure that modern web
frameworks similar to Django has been created for Perl (Google
find Mojolicious).

But if you look at the web landscape today then Django and Flask
(on Python and not using CGI) are widely used and Perl is niche - and
a good chunk of that niche is 20+ years old CGI scripts.

Could Mojolicious and Perl had become widely used with Django and
Python as niche? Probably, but it did not happen.

Arne

On 4/17/2023 8:15 AM, Simon Clubley wrote:
> On 2023-04-14, bill <bill.gunshannon@gmail.com> wrote:
>> On 4/14/2023 1:46 PM, Simon Clubley wrote:
>>>
>>>
>>> My point is that Python has a major thing going for it in that it is
>>> relatively easy to learn and use by people who are not professional
>>> programmers or professional sysadmins.
>>
>> And if all there was to programming was syntax that might be a good
>> thing. But it isn't and having all these totally unqualified asses
>> writing "programs" is certainly not a good thing.
>>
>
> That is a totally out of touch and _extremely_ elitist thing to say Bill.

Really? Do you apply the same opinion to doctors? Lawyers?
The investment bankers handling your retirement money? How
about the mechanic who's going to work on your Tesla or Porsche?

There is much more to programming than just knowing the syntax
of a language.

>
> We are talking about applications and areas that have skilled domain
> knowledge experts, but who are not computer programmers. What you
> appear to be saying is that these people are not allowed to use their
> expertise until they become C or C++ experts.

No, what I am saying is they should be coordinating with someone
skilled in programming to get the job done. Remember a language
called Pilot? How about Hypertalk? These were going to take the
task of programming out of the hands of programmer and put it in
the hands of elementary school teachers. How'd that work out?

>
> Python is being used to allow these people to make full of their
> specialised knowledge without having to learn C or C++.

So you would be OK with the guy who fixes lawnmowers using his
"specialized skills" to fix your Cadilac Escalade? After all,
their both just devices with ICE.

>
> Also, given the interactive nature of many of these applications, Python is
> way more suited than C or C++ anyway, at least when it comes to initially
> exploring a problem.

Yeah, well, I don;'t agree with that either, but that is another
problem entirely.

>
> Besides, how would you even _use_ C or C++ as a scripting language
> in (for example) Blender anyway ?

Any task you can do with a scripting language you can do with a real
language. Picking the right language for the job is part of what used
to be software engineering. But then, if your not going to have an
engineer design and build your software. Let's apply that to other
tasks like bridges. I can build nice bridges with Lego blocks so I
should be able to design and build the next big highway bridge.

I really miss the days before software engineering when we had
domain specific languages and program were actually designed to
accomplish specific tasks. Oh wait, we have Agile now and there
is no longer a need for a design because reaching an endpoint is
no longer required.

bill

Den 2023-04-17 kl. 14:39, skrev bill:
> On 4/17/2023 8:15 AM, Simon Clubley wrote:
>> On 2023-04-14, bill <bill.gunshannon@gmail.com> wrote:
>>> On 4/14/2023 1:46 PM, Simon Clubley wrote:
>>>>
>>>>
>>>> My point is that Python has a major thing going for it in that it is
>>>> relatively easy to learn and use by people who are not professional
>>>> programmers or professional sysadmins.
>>>
>>> And if all there was to programming was syntax that might be a good
>>> thing.  But it isn't and having all these totally unqualified asses
>>> writing "programs" is certainly not a good thing.
>>>
>>
>> That is a totally out of touch and _extremely_ elitist thing to say Bill.
>
> Really?  Do you apply the same opinion to doctors?  Lawyers?
> The investment bankers handling your retirement money?  How
> about the mechanic who's going to work on your Tesla or Porsche?
>
> There is much more to programming than just knowing the syntax
> of a language.
>
>>
>> We are talking about applications and areas that have skilled domain
>> knowledge experts, but who are not computer programmers. What you
>> appear to be saying is that these people are not allowed to use their
>> expertise until they become C or C++ experts.
>
> No, what I am saying is they should be coordinating with someone
> skilled in programming to get the job done.  Remember a language
> called Pilot?  How about Hypertalk?  These were going to take the
> task of programming out of the hands of programmer and put it in
> the hands of elementary school teachers.  How'd that work out?
>
>>
>> Python is being used to allow these people to make full of their
>> specialised knowledge without having to learn C or C++.
>
> So you would be OK with the guy who fixes lawnmowers using his
> "specialized skills" to fix your Cadilac Escalade?  After all,
> their both just devices with ICE.
>
>>
>> Also, given the interactive nature of many of these applications, Python is
>> way more suited than C or C++ anyway, at least when it comes to initially
>> exploring a problem.
>
> Yeah, well, I don;'t agree with that either, but that is another
> problem entirely.
>
>>
>> Besides, how would you even _use_ C or C++ as a scripting language
>> in (for example) Blender anyway ?
>
> Any task you can do with a scripting language you can do with a real
> language.  Picking the right language for the job is part of what used
> to be software engineering.  But then, if your not going to have an
> engineer design and build your software.  Let's apply that to other
> tasks like bridges.  I can build nice bridges with Lego blocks so I
> should be able to design and build the next big highway bridge.
>
>
> I really miss the days before software engineering when we had
> domain specific languages and program were actually designed to
> accomplish specific tasks.  Oh wait, we have Agile now and there
> is no longer a need for a design because reaching an endpoint is
> no longer required.
>
> bill
>

Thinking that there exist an "end-point" is out of touch with
anything today. Applications are living creaturea and changes
all the time. There might be check-points that you reach and
pass on the way to the next requirements on the applications
but they will usually never be completely "finished".

This is also one reason that development and maintenance is now
going back from outsourcing companies to own resources. It is
easier to get a continuous maintenance with no specific endpoint.

On 4/17/2023 8:39 AM, bill wrote:
> On 4/17/2023 8:15 AM, Simon Clubley wrote:
>> On 2023-04-14, bill <bill.gunshannon@gmail.com> wrote:
>>> On 4/14/2023 1:46 PM, Simon Clubley wrote:
>>>>
>>>>
>>>> My point is that Python has a major thing going for it in that it is
>>>> relatively easy to learn and use by people who are not professional
>>>> programmers or professional sysadmins.
>>>
>>> And if all there was to programming was syntax that might be a good
>>> thing. But it isn't and having all these totally unqualified asses
>>> writing "programs" is certainly not a good thing.
>>>
>>
>> That is a totally out of touch and _extremely_ elitist thing to say Bill.
>
> Really? Do you apply the same opinion to doctors? Lawyers?
> The investment bankers handling your retirement money? How
> about the mechanic who's going to work on your Tesla or Porsche?
>
> There is much more to programming than just knowing the syntax
> of a language.
>
>>
>> We are talking about applications and areas that have skilled domain
>> knowledge experts, but who are not computer programmers. What you
>> appear to be saying is that these people are not allowed to use their
>> expertise until they become C or C++ experts.
>
> No, what I am saying is they should be coordinating with someone
> skilled in programming to get the job done. Remember a language
> called Pilot? How about Hypertalk? These were going to take the
> task of programming out of the hands of programmer and put it in
> the hands of elementary school teachers. How'd that work out?
>
>>
>> Python is being used to allow these people to make full of their
>> specialised knowledge without having to learn C or C++.
>
> So you would be OK with the guy who fixes lawnmowers using his
> "specialized skills" to fix your Cadilac Escalade? After all,
> their both just devices with ICE.
>
>>
>> Also, given the interactive nature of many of these applications, Python is
>> way more suited than C or C++ anyway, at least when it comes to initially
>> exploring a problem.
>
> Yeah, well, I don;'t agree with that either, but that is another
> problem entirely.
>
>>
>> Besides, how would you even _use_ C or C++ as a scripting language
>> in (for example) Blender anyway ?
>
> Any task you can do with a scripting language you can do with a real
> language. Picking the right language for the job is part of what used
> to be software engineering. But then, if your not going to have an
> engineer design and build your software. Let's apply that to other
> tasks like bridges. I can build nice bridges with Lego blocks so I
> should be able to design and build the next big highway bridge.
>
>
> I really miss the days before software engineering when we had
> domain specific languages and program were actually designed to
> accomplish specific tasks. Oh wait, we have Agile now and there
> is no longer a need for a design because reaching an endpoint is
> no longer required.
>
> bill
>

Gotta say, I'm 100% with Bill on this subject. Yes, the user knows what is
needed. But the user may not understand subtle issues. Case in point.

In CODIS we've provided what I'll call web services. May not meet Jan Erik's
definition of a web services, but they do the job. One allowed a socket
connection to inquire about inventory availability. Someone setting up a web
based "shopping cart" could invoke the service to get the availability of
products. Now, there is a bit of overhead in the inquiry. Ask for a socket
connection, do the handshake stuff, send the inquiry, receive the data, and then
break down the connection. One web "programmer" managed to just about drag the
entire application to a halt. He was looking for the availability of thousands
of products. And so issued thousands of connections. Lots of overhead. The
designer of the web service understood what could be needed, and set up the
design to allow thousands of products to be requested in one connection. Note,
this was well documented. However, the user didn't do the research, just did
what he needed, and didn't worry about the overall task. That's what happens
when some hacker who doesn't understand, or care about, the overall task throws
together something.

--
David Froble Tel: 724-529-0450
Dave Froble Enterprises, Inc. E-Mail: davef@tsoft-inc.com
DFE Ultralights, Inc.
170 Grimplin Road
Vanderbilt, PA 15486

Den 2023-04-17 kl. 16:17, skrev Dave Froble:
> On 4/17/2023 8:39 AM, bill wrote:
>> On 4/17/2023 8:15 AM, Simon Clubley wrote:
>>> On 2023-04-14, bill <bill.gunshannon@gmail.com> wrote:
>>>> On 4/14/2023 1:46 PM, Simon Clubley wrote:
>>>>>
>>>>>
>>>>> My point is that Python has a major thing going for it in that it is
>>>>> relatively easy to learn and use by people who are not professional
>>>>> programmers or professional sysadmins.
>>>>
>>>> And if all there was to programming was syntax that might be a good
>>>> thing.  But it isn't and having all these totally unqualified asses
>>>> writing "programs" is certainly not a good thing.
>>>>
>>>
>>> That is a totally out of touch and _extremely_ elitist thing to say Bill.
>>
>> Really?  Do you apply the same opinion to doctors?  Lawyers?
>> The investment bankers handling your retirement money?  How
>> about the mechanic who's going to work on your Tesla or Porsche?
>>
>> There is much more to programming than just knowing the syntax
>> of a language.
>>
>>>
>>> We are talking about applications and areas that have skilled domain
>>> knowledge experts, but who are not computer programmers. What you
>>> appear to be saying is that these people are not allowed to use their
>>> expertise until they become C or C++ experts.
>>
>> No, what I am saying is they should be coordinating with someone
>> skilled in programming to get the job done.  Remember a language
>> called Pilot?  How about Hypertalk?  These were going to take the
>> task of programming out of the hands of programmer and put it in
>> the hands of elementary school teachers.  How'd that work out?
>>
>>>
>>> Python is being used to allow these people to make full of their
>>> specialised knowledge without having to learn C or C++.
>>
>> So you would be OK with the guy who fixes lawnmowers using his
>> "specialized skills" to fix your Cadilac Escalade?  After all,
>> their both just devices with ICE.
>>
>>>
>>> Also, given the interactive nature of many of these applications, Python is
>>> way more suited than C or C++ anyway, at least when it comes to initially
>>> exploring a problem.
>>
>> Yeah, well, I don;'t agree with that either, but that is another
>> problem entirely.
>>
>>>
>>> Besides, how would you even _use_ C or C++ as a scripting language
>>> in (for example) Blender anyway ?
>>
>> Any task you can do with a scripting language you can do with a real
>> language.  Picking the right language for the job is part of what used
>> to be software engineering.  But then, if your not going to have an
>> engineer design and build your software.  Let's apply that to other
>> tasks like bridges.  I can build nice bridges with Lego blocks so I
>> should be able to design and build the next big highway bridge.
>>
>>
>> I really miss the days before software engineering when we had
>> domain specific languages and program were actually designed to
>> accomplish specific tasks.  Oh wait, we have Agile now and there
>> is no longer a need for a design because reaching an endpoint is
>> no longer required.
>>
>> bill
>>
>
> Gotta say, I'm 100% with Bill on this subject.  Yes, the user knows what is
> needed.  But the user may not understand subtle issues.  Case in point.
>
> In CODIS we've provided what I'll call web services.  May not meet Jan
> Erik's definition of a web services, but they do the job.  One allowed a
> socket connection to inquire about inventory availability.  Someone setting
> up a web based "shopping cart" could invoke the service to get the
> availability of products.  Now, there is a bit of overhead in the inquiry.
> Ask for a socket connection, do the handshake stuff, send the inquiry,
> receive the data, and then break down the connection.  One web "programmer"
> managed to just about drag the entire application to a halt.  He was
> looking for the availability of thousands of products.  And so issued
> thousands of connections.  Lots of overhead.  The designer of the web
> service understood what could be needed, and set up the design to allow
> thousands of products to be requested in one connection.  Note, this was
> well documented.  However, the user didn't do the research, just did what
> he needed, and didn't worry about the overall task.  That's what happens
> when some hacker who doesn't understand, or care about, the overall task
> throws together something.
>

OK. But that can happen no matter what tools the server was built with.
It is not that you should *only* use one or the other. Both compiled
apps for the high volume/high performance tasks and scripted things
does have there roles to play.

We have for many years used Python scripts for our web backend services.
Right now, I'm looking at, for one new and specific application, using
compiled Cobol apps instead. Just becuse we'd like to get the best
throughput possible for this specific application.

First POCs points to approx 50-60 ms roundtrip from the browser,
through the web server itself, to the separate web service process
and back again to the browser. That is an "empty" application without
the business logic just to check the overhead.

And yes, we got exactly the same figures with C and Cobol (as expected).

On 2023-04-17, bill <bill.gunshannon@gmail.com> wrote:
> On 4/17/2023 8:15 AM, Simon Clubley wrote:
>>
>> Besides, how would you even _use_ C or C++ as a scripting language
>> in (for example) Blender anyway ?
>
> Any task you can do with a scripting language you can do with a real
> language. Picking the right language for the job is part of what used
> to be software engineering. But then, if your not going to have an
> engineer design and build your software. Let's apply that to other
> tasks like bridges. I can build nice bridges with Lego blocks so I
> should be able to design and build the next big highway bridge.
>

Bill, there is a vast, vast, world out there that you are very clearly
unaware of. The act of typing a program into an editor, and then
compiling it into executable architecture-specific machine code from
a command line is only a small part of the computing world.

Do you even know what Blender is (for example) ?

Here is a link I posted previously:

https://docs.blender.org/api/current/info_overview.html

How exactly would you use C or C++ to do scripting in such an environment ?

>
> I really miss the days before software engineering when we had
> domain specific languages and program were actually designed to
> accomplish specific tasks. Oh wait, we have Agile now and there
> is no longer a need for a design because reaching an endpoint is
> no longer required.
>

In many ways, Python is effectively a domain specific language, but
due to the way the domain knowledge is implemented in Python, the
same language can be used in multiple areas.

From the above page:

|Here is a simple example which moves a vertex attached to an object named
|"Cube":
| |import bpy
|bpy.data.objects["Cube"].data.vertices[0].co.x += 1.0
| |This modifies Blender's internal data directly. When you run this in the
|interactive console you will see the 3D Viewport update.

How exactly is that not effectively a domain specific language ?

The same is true for all the other applications that have integrated
Python into them as a scripting and automation language.

Simon.

--
Simon Clubley, clubley@remove_me.eisner.decus.org-Earth.UFP
Walking destinations on a map are further away than they appear.

Simon Clubley <clubley@remove_me.eisner.decus.org-Earth.UFP> wrote:
>On 2023-04-14, bill <bill.gunshannon@gmail.com> wrote:
>> On 4/14/2023 1:46 PM, Simon Clubley wrote:
>>>
>>> My point is that Python has a major thing going for it in that it is
>>> relatively easy to learn and use by people who are not professional
>>> programmers or professional sysadmins.
>>
>> And if all there was to programming was syntax that might be a good
>> thing. But it isn't and having all these totally unqualified asses
>> writing "programs" is certainly not a good thing.
>>
>
>That is a totally out of touch and _extremely_ elitist thing to say Bill.
>
>We are talking about applications and areas that have skilled domain
>knowledge experts, but who are not computer programmers. What you
>appear to be saying is that these people are not allowed to use their
>expertise until they become C or C++ experts.

I don't think it is elitist or out of touch at all.

25 years ago, those skilled domain knowledge experts would be sitting down
with a programmer and telling the programmer what they wanted, and the
programmer would be saying things like "Would it be okay if we did it in
THIS order because it would be faster?" and things like "If this value is
zero, it's not going to work, so will this ever be zero?" There would be
a team with the programmer and the subject matter expert.

Now more likely we have SMEs writing code and I hate to say it but this is
often a very very bad idea. "It's okay, I can use Matlab for the database
engine!" I wish I were joking about that but I am not.
--scott

--
"C'est un Nagra. C'est suisse, et tres, tres precis."

On 4/17/2023 3:00 PM, Scott Dorsey wrote:
> Simon Clubley <clubley@remove_me.eisner.decus.org-Earth.UFP> wrote:
>> On 2023-04-14, bill <bill.gunshannon@gmail.com> wrote:
>>> On 4/14/2023 1:46 PM, Simon Clubley wrote:
>>>> My point is that Python has a major thing going for it in that it is
>>>> relatively easy to learn and use by people who are not professional
>>>> programmers or professional sysadmins.
>>>
>>> And if all there was to programming was syntax that might be a good
>>> thing. But it isn't and having all these totally unqualified asses
>>> writing "programs" is certainly not a good thing.
>>
>> That is a totally out of touch and _extremely_ elitist thing to say Bill.
>>
>> We are talking about applications and areas that have skilled domain
>> knowledge experts, but who are not computer programmers. What you
>> appear to be saying is that these people are not allowed to use their
>> expertise until they become C or C++ experts.
>
> I don't think it is elitist or out of touch at all.
>
> 25 years ago, those skilled domain knowledge experts would be sitting down
> with a programmer and telling the programmer what they wanted, and the
> programmer would be saying things like "Would it be okay if we did it in
> THIS order because it would be faster?" and things like "If this value is
> zero, it's not going to work, so will this ever be zero?" There would be
> a team with the programmer and the subject matter expert.
>
> Now more likely we have SMEs writing code and I hate to say it but this is
> often a very very bad idea. "It's okay, I can use Matlab for the database
> engine!" I wish I were joking about that but I am not.

"shadow IT" is a real problem. Adhoc hacked code with no
development process that is impossible to maintain, contains security
vulnerabilities, performs poorly etc.etc..

But it is not all types of programming that requires a formal
development process.

Sure for the major application that are doing something important
and are expected to live for 10-20-30-40 years years, then it
should be a given.

But there are other types of programming:
* the finance person that need to automate some stuff
in Excel and do it in VBA
* the finance person that want to see how various
interest scenarios impact the budget and do it
in Python
* the sys admin that need to automate some processes
and do it in shell/Python
* the sys admin that need to move a lot of stuff around
as a one time thing and do it in shell/Python
* the data person that need to export a lot of data from
some data sources and load them into a DWH and do it
in Python
* the data science person that need to try out hundreds
of different forecast models to find the best model and
do it in Python/R/Matlab
etc.

In those cases the formal development process does not
make any sense. It takes too long time and cost too much
money.

Arne

On 4/17/2023 1:46 PM, Simon Clubley wrote:
> On 2023-04-17, bill <bill.gunshannon@gmail.com> wrote:
>> On 4/17/2023 8:15 AM, Simon Clubley wrote:
>>> Besides, how would you even _use_ C or C++ as a scripting language
>>> in (for example) Blender anyway ?
>>
>> Any task you can do with a scripting language you can do with a real
>> language. Picking the right language for the job is part of what used
>> to be software engineering. But then, if your not going to have an
>> engineer design and build your software. Let's apply that to other
>> tasks like bridges. I can build nice bridges with Lego blocks so I
>> should be able to design and build the next big highway bridge.
>
> Bill, there is a vast, vast, world out there that you are very clearly
> unaware of. The act of typing a program into an editor, and then
> compiling it into executable architecture-specific machine code from
> a command line is only a small part of the computing world.

True.

> Do you even know what Blender is (for example) ?
>
> Here is a link I posted previously:
>
> https://docs.blender.org/api/current/info_overview.html
>
> How exactly would you use C or C++ to do scripting in such an environment ?

I don't think there is anything preventing such a thing.

Someone would need to develop a C interpreter or use a
callable C compiler.

I have never heard about such for C, but I cannot see any technical
reasons why it would be impossible.

The reason that it has not been done is that C would be a terrible
language for that purpose.

An embedded scripting language:
* should be very high level / powerful aka lot of functionality
per line of code
* should not require any "how computers actually work" stuff, but
let the developers just express the desired logic without worrying
about data types, memory allocation and deallocation etc.

That is not C.

That is Python.

Arne

On 4/17/2023 8:39 AM, bill wrote:
> On 4/17/2023 8:15 AM, Simon Clubley wrote:
>> Besides, how would you even _use_ C or C++ as a scripting language
>> in (for example) Blender anyway ?
>
> Any task you can do with a scripting language you can do with a real
> language.  Picking the right language for the job is part of what used
> to be software engineering.  But then, if your not going to have an
> engineer design and build your software.  Let's apply that to other
> tasks like bridges.  I can build nice bridges with Lego blocks so I
> should be able to design and build the next big highway bridge.

"Picking the right language for the job" is exactly what it is
all about.

And the bridge analogy is perfect.

If you are to build a highway bridge, then you want an engineer to
design it, you want concrete that match some well defined specs,
you will use heavy equipment, you want skilled people to
operate the equipment etc..

But what if your wife ask you to make a cute 3 ft long 1 ft wide
bridge over a pond in your backyard.

Do you send for an engineer, trucks with concrete, heavy equipment and
a large crew?

Or do you drive to Home Depot or Lowes, pick up some lumber +
some stones + a few bags of concrete and do it yourself?

> I really miss the days before software engineering when we had
> domain specific languages and program were actually designed to
> accomplish specific tasks.  Oh wait, we have Agile now and there
> is no longer a need for a design because reaching an endpoint is
> no longer required.

Agile requires both architecture and design.

But only what is needed/warranted by the problem - not
what is mandated by some process.

Arne

On 4/17/2023 10:17 AM, Dave Froble wrote:
> Gotta say, I'm 100% with Bill on this subject.  Yes, the user knows what
> is needed.  But the user may not understand subtle issues.  Case in point.
>
> In CODIS we've provided what I'll call web services.  May not meet Jan
> Erik's definition of a web services, but they do the job.

In standard IT industry terminology "web service" is used for
text payloads transported over HTTP(S).

But I don't think your point depends on whether it is text over HTTP(S)
or binary over plain TCP.

>   One allowed a
> socket connection to inquire about inventory availability.  Someone
> setting up a web based "shopping cart" could invoke the service to get
> the availability of products.  Now, there is a bit of overhead in the
> inquiry.  Ask for a socket connection, do the handshake stuff, send the
> inquiry, receive the data, and then break down the connection.  One web
> "programmer" managed to just about drag the entire application to a
> halt.  He was looking for the availability of thousands of products.
> And so issued thousands of connections.  Lots of overhead.  The designer
> of the web service understood what could be needed, and set up the
> design to allow thousands of products to be requested in one
> connection.  Note, this was well documented.  However, the user didn't
> do the research, just did what he needed, and didn't worry about the
> overall task.  That's what happens when some hacker who doesn't
> understand, or care about, the overall task throws together something.

I have a very different take on that example.

If the service documentation said that an argument X
should be 10 characters, the client developer did not read
it and send 12 characters and as a result the server
crashed - then I assume we would agree that it was a
bad design/implementation of the server, because it
should check the length and handle invalid input.

But the overload situation is really the same. The
server should be prepared for a high request rate and
limit the request rate so the service could not bring
down the server.

If the first 10 of those 1000 requests took 50 ms each
and the rest took 500 ms each, then the client developer
would quickly start reading docs or ask someone.

So I think this is a case of the server side being
bad design.

There is a little extra twist here. If the service
had been a standard web service, then it would not
have been necessary to implement the request rate
limit in the service itself. Instead a setup like:

client--API gateway--service

could have been used and the API gateway could
do rate limiting defined in configuration (it could
also do caching, access control and other useful
stuff all defined in configuration).

Arne

On 4/17/2023 3:14 PM, Dave Froble wrote:
> On 4/17/2023 3:00 PM, Scott Dorsey wrote:
>> 25 years ago, those skilled domain knowledge experts would be sitting
>> down
>> with a programmer and telling the programmer what they wanted, and the
>> programmer would be saying things like "Would it be okay if we did it in
>> THIS order because it would be faster?" and things like "If this value is
>> zero, it's not going to work, so will this ever be zero?"  There would be
>> a team with the programmer and the subject matter expert.
>
> This is most of the problem.  The hackers, Ok, SME, don't consider
> anything except what they want to do.
>
> Another simple example.  A third party accepts and stores credit card
> information, so vendors don't have to.  Some web designer didn't know
> how to query if the credit card info was already on file, so he just
> created a new credit card entry for every usage.  No problem for him.
> But, when a capable person did an inquiry to see if the card was already
> on file, back came thousands of the same CC #.  Caused real problems,
> but not for the hacker.

Obviously a disaster design/implementation.

But usually it is the SME's problem if business rules are
not properly implemented.

Arne

PS: Having the credit card information even transient should be
sufficient to trigger PCI-DSS compliance requirements even
if persisted elsewhere.

On 4/17/2023 9:43 PM, Arne Vajhøj wrote:
> On 4/17/2023 3:14 PM, Dave Froble wrote:
>> On 4/17/2023 3:00 PM, Scott Dorsey wrote:
>>> 25 years ago, those skilled domain knowledge experts would be sitting down
>>> with a programmer and telling the programmer what they wanted, and the
>>> programmer would be saying things like "Would it be okay if we did it in
>>> THIS order because it would be faster?" and things like "If this value is
>>> zero, it's not going to work, so will this ever be zero?" There would be
>>> a team with the programmer and the subject matter expert.
>>
>> This is most of the problem. The hackers, Ok, SME, don't consider anything
>> except what they want to do.
>>
>> Another simple example. A third party accepts and stores credit card
>> information, so vendors don't have to. Some web designer didn't know how to
>> query if the credit card info was already on file, so he just created a new
>> credit card entry for every usage. No problem for him. But, when a capable
>> person did an inquiry to see if the card was already on file, back came
>> thousands of the same CC #. Caused real problems, but not for the hacker.
>
> Obviously a disaster design/implementation.
>
> But usually it is the SME's problem if business rules are
> not properly implemented.

But the SME is only worried about what affects him. That's not enough.

> Arne
>
> PS: Having the credit card information even transient should be
> sufficient to trigger PCI-DSS compliance requirements even
> if persisted elsewhere.

This example was about a vendor selling PCI type of services. Why they let
multiple copies of a CC#, that's another story.

--
David Froble Tel: 724-529-0450
Dave Froble Enterprises, Inc. E-Mail: davef@tsoft-inc.com
DFE Ultralights, Inc.
170 Grimplin Road
Vanderbilt, PA 15486

On 4/17/2023 7:25 PM, Arne Vajhøj wrote:
> On 4/17/2023 3:00 PM, Scott Dorsey wrote:
>> Simon Clubley <clubley@remove_me.eisner.decus.org-Earth.UFP> wrote:
>>> On 2023-04-14, bill <bill.gunshannon@gmail.com> wrote:
>>>> On 4/14/2023 1:46 PM, Simon Clubley wrote:
>>>>> My point is that Python has a major thing going for it in that it is
>>>>> relatively easy to learn and use by people who are not professional
>>>>> programmers or professional sysadmins.
>>>>
>>>> And if all there was to programming was syntax that might be a good
>>>> thing. But it isn't and having all these totally unqualified asses
>>>> writing "programs" is certainly not a good thing.
>>>
>>> That is a totally out of touch and _extremely_ elitist thing to say Bill.
>>>
>>> We are talking about applications and areas that have skilled domain
>>> knowledge experts, but who are not computer programmers. What you
>>> appear to be saying is that these people are not allowed to use their
>>> expertise until they become C or C++ experts.
>>
>> I don't think it is elitist or out of touch at all.
>>
>> 25 years ago, those skilled domain knowledge experts would be sitting down
>> with a programmer and telling the programmer what they wanted, and the
>> programmer would be saying things like "Would it be okay if we did it in
>> THIS order because it would be faster?" and things like "If this value is
>> zero, it's not going to work, so will this ever be zero?" There would be
>> a team with the programmer and the subject matter expert.
>>
>> Now more likely we have SMEs writing code and I hate to say it but this is
>> often a very very bad idea. "It's okay, I can use Matlab for the database
>> engine!" I wish I were joking about that but I am not.
>
> "shadow IT" is a real problem. Adhoc hacked code with no
> development process that is impossible to maintain, contains security
> vulnerabilities, performs poorly etc.etc..
>
> But it is not all types of programming that requires a formal
> development process.
>
> Sure for the major application that are doing something important
> and are expected to live for 10-20-30-40 years years, then it
> should be a given.
>
> But there are other types of programming:
> * the finance person that need to automate some stuff
> in Excel and do it in VBA
> * the finance person that want to see how various
> interest scenarios impact the budget and do it
> in Python
> * the sys admin that need to automate some processes
> and do it in shell/Python
> * the sys admin that need to move a lot of stuff around
> as a one time thing and do it in shell/Python
> * the data person that need to export a lot of data from
> some data sources and load them into a DWH and do it
> in Python
> * the data science person that need to try out hundreds
> of different forecast models to find the best model and
> do it in Python/R/Matlab
> etc.
>
> In those cases the formal development process does not
> make any sense. It takes too long time and cost too much
> money.
>
> Arne

But what's the problem of running ideas past some decent analysts? That assumes
said analysts will provide the time.

--
David Froble Tel: 724-529-0450
Dave Froble Enterprises, Inc. E-Mail: davef@tsoft-inc.com
DFE Ultralights, Inc.
170 Grimplin Road
Vanderbilt, PA 15486

On 4/17/2023 8:00 PM, Arne Vajhøj wrote:
> On 4/17/2023 10:17 AM, Dave Froble wrote:
>> Gotta say, I'm 100% with Bill on this subject. Yes, the user knows what is
>> needed. But the user may not understand subtle issues. Case in point.
>>
>> In CODIS we've provided what I'll call web services. May not meet Jan Erik's
>> definition of a web services, but they do the job.
>
> In standard IT industry terminology "web service" is used for
> text payloads transported over HTTP(S).
>
> But I don't think your point depends on whether it is text over HTTP(S)
> or binary over plain TCP.
>
>> One allowed a
>> socket connection to inquire about inventory availability. Someone setting up
>> a web based "shopping cart" could invoke the service to get the availability
>> of products. Now, there is a bit of overhead in the inquiry. Ask for a
>> socket connection, do the handshake stuff, send the inquiry, receive the data,
>> and then break down the connection. One web "programmer" managed to just
>> about drag the entire application to a halt. He was looking for the
>> availability of thousands of products. And so issued thousands of
>> connections. Lots of overhead. The designer of the web service understood
>> what could be needed, and set up the design to allow thousands of products to
>> be requested in one connection. Note, this was well documented. However, the
>> user didn't do the research, just did what he needed, and didn't worry about
>> the overall task. That's what happens when some hacker who doesn't
>> understand, or care about, the overall task throws together something.
>
> I have a very different take on that example.
>
> If the service documentation said that an argument X
> should be 10 characters, the client developer did not read
> it and send 12 characters and as a result the server
> crashed - then I assume we would agree that it was a
> bad design/implementation of the server, because it
> should check the length and handle invalid input.

That wasn't the issue.

> But the overload situation is really the same. The
> server should be prepared for a high request rate and
> limit the request rate so the service could not bring
> down the server.

The server performed adequately. It was the hacker who complained, and then we
had to look at what he was doing. Then we had to point him to the
documentation, which he hadn't read throughly. It was his stuff that was slow.

> If the first 10 of those 1000 requests took 50 ms each
> and the rest took 500 ms each, then the client developer
> would quickly start reading docs or ask someone.

You got it. Now try to get some of the hackers to actually read the docs.

> So I think this is a case of the server side being
> bad design.

No,it worked quite well.

> There is a little extra twist here. If the service
> had been a standard web service, then it would not
> have been necessary to implement the request rate
> limit in the service itself. Instead a setup like:

Guess you got a problem with apps that work quite well?

> client--API gateway--service
>
> could have been used and the API gateway could
> do rate limiting defined in configuration (it could
> also do caching, access control and other useful
> stuff all defined in configuration).

Same result, the hacker would have had poor performance.

--
David Froble Tel: 724-529-0450
Dave Froble Enterprises, Inc. E-Mail: davef@tsoft-inc.com
DFE Ultralights, Inc.
170 Grimplin Road
Vanderbilt, PA 15486

On 2023-04-14, bill <bill.gunshannon@gmail.com> wrote:
> On 4/14/2023 1:46 PM, Simon Clubley wrote:
>>
>>
>> My point is that Python has a major thing going for it in that it is
>> relatively easy to learn and use by people who are not professional
>> programmers or professional sysadmins.
>
> And if all there was to programming was syntax that might be a good
> thing. But it isn't and having all these totally unqualified asses
> writing "programs" is certainly not a good thing.
>

That is a totally out of touch and _extremely_ elitist thing to say Bill.

We are talking about applications and areas that have skilled domain
knowledge experts, but who are not computer programmers. What you
appear to be saying is that these people are not allowed to use their
expertise until they become C or C++ experts.

Python is being used to allow these people to make full of their
specialised knowledge without having to learn C or C++.

Also, given the interactive nature of many of these applications, Python is
way more suited than C or C++ anyway, at least when it comes to initially
exploring a problem.

Besides, how would you even _use_ C or C++ as a scripting language
in (for example) Blender anyway ?

Simon.

--
Simon Clubley, clubley@remove_me.eisner.decus.org-Earth.UFP
Walking destinations on a map are further away than they appear.

On 4/17/2023 3:00 PM, Scott Dorsey wrote:
> Simon Clubley <clubley@remove_me.eisner.decus.org-Earth.UFP> wrote:
>> On 2023-04-14, bill <bill.gunshannon@gmail.com> wrote:
>>> On 4/14/2023 1:46 PM, Simon Clubley wrote:
>>>>
>>>> My point is that Python has a major thing going for it in that it is
>>>> relatively easy to learn and use by people who are not professional
>>>> programmers or professional sysadmins.
>>>
>>> And if all there was to programming was syntax that might be a good
>>> thing. But it isn't and having all these totally unqualified asses
>>> writing "programs" is certainly not a good thing.
>>>
>>
>> That is a totally out of touch and _extremely_ elitist thing to say Bill.
>>
>> We are talking about applications and areas that have skilled domain
>> knowledge experts, but who are not computer programmers. What you
>> appear to be saying is that these people are not allowed to use their
>> expertise until they become C or C++ experts.
>
> I don't think it is elitist or out of touch at all.
>
> 25 years ago, those skilled domain knowledge experts would be sitting down
> with a programmer and telling the programmer what they wanted, and the
> programmer would be saying things like "Would it be okay if we did it in
> THIS order because it would be faster?" and things like "If this value is
> zero, it's not going to work, so will this ever be zero?" There would be
> a team with the programmer and the subject matter expert.

This is most of the problem. The hackers, Ok, SME, don't consider anything
except what they want to do.

Another simple example. A third party accepts and stores credit card
information, so vendors don't have to. Some web designer didn't know how to
query if the credit card info was already on file, so he just created a new
credit card entry for every usage. No problem for him. But, when a capable
person did an inquiry to see if the card was already on file, back came
thousands of the same CC #. Caused real problems, but not for the hacker.

> Now more likely we have SMEs writing code and I hate to say it but this is
> often a very very bad idea. "It's okay, I can use Matlab for the database
> engine!" I wish I were joking about that but I am not.
> --scott

Can be a very, very, very bad idea ...

--
David Froble Tel: 724-529-0450
Dave Froble Enterprises, Inc. E-Mail: davef@tsoft-inc.com
DFE Ultralights, Inc.
170 Grimplin Road
Vanderbilt, PA 15486