Re: Python recompile

On Fri, 14 Mar 2025 00:37:48 -0000 (UTC)
antispam@fricas.org (Waldek Hebisch) wrote:

bart <bc@freeuk.com> wrote:

On 11/03/2025 01:33, Waldek Hebisch wrote: 

bart <bc@freeuk.com> wrote: 

On 10/03/2025 10:58, Waldek Hebisch wrote: 

bart <bc@freeuk.com> wrote: 

>
I think nobody does. There's always been some sort of mystique
surrounding 'gcc' on Windows.
>
'MinGW' supposedly 'Minimalist Gnu on Windows'. In that case I
wouldn't like to see the full-scale one.. 

>
"Minimalist" is not about size of the compiler.  Rather, it is
about possible support routines.  For "hosted implementation" C
mandates presence of C library and there is a lot of functions
not in C standard, but included in libraries of C compilers.
There is also question of operating system support, complicated
by fact that Windows is different than other systems.  Cygwin
solved those issues by offering Posix emulation and a sizable
collection os libraries.  MinGW is minimalist in the sense
that it provides very little own libraries and mainly uses
what is provide by Windows. 

>
I still don't get this stuff.
>
I get the impression that a port of gcc to Windows is not simply
about building C programs, but building C programs that use a lot
of features from Linux. 

 
You apparently do not get fact that people want tools to
automate various routine tasks. 

 
What routine task is this? I'm talking exclusively about turning a
bunch of source files in some language (here it is C) into an
executable binary.
 
This task can be done with a program called a 'compiler'. 

 
You ignore fact that people are developing programs.  And developement
is much more than "turning a bunch of source files into an
executable". GPL says "The source code for a work means the preferred
form of the work for making modifications to it".  And that nicely
captures the idea: sources are in form that is convenient for
developer and may require several steps before one gets the
executable.
 

However, what I'm arguing about is that this simple task has become
unnecessarily elaborate on OSes like Linux, by introducing
makefiles, OS-specific scripts, and OS-specific utilities. 

 
There is not much OS-specific in "Linux tools".  'make' is
problematic on zOS, because files have no timestamps there,
but most normal OS-es have timestamps (and IIUC even zOS
has USS where timestamps are available).  You need ability
to invoke a program from a different program, exact form
of this is OS-dependent, but ability is there in any
semi-reasonable general purpose OS.  You need ability
to get exit status of a program, again exact form is
OS-dependent but ability is there.
 
FYI, is started using "Linux tools" on MS-DOS, before I
heard about Linux.  And tools are widely available,
IIUC classic zOS is problematic, but I think that USS
is OK.  Some systems may be prone to breakage, when
there is small number of _active_ users and nobody bothers
to timely report real problems.
 
Coming back to developement, in many cases it is desirable
to generate some C files.  C preprocessor can do some
transformations, but is may be convenient to transform
source in a way hard to do via proprocessor.  Also,
this is C group, but big programs frequently have parts
in different languages, an then it is good to have
language-neutral tools.  Simple tools to transform
or generate files are 'sed' and 'awk'.  Some people
use 'perl' or 'python' for such purpose and in principle
there is much more possiblities.
 
More specialised are compiler-writing tools like 'bison'
and 'flex'.  You can write a parser by hand, but using
'bison' one does not need to know much to write working
parser.  'flex' helps in writing scanners, but IMO
is quite useful for text processing task (its support
for regular expressions is nicer than Perl or Python
offer, and speed frequently is much better too).
 
Users beside binary need also documentation.  In principle
documentation could be provided as plain text, but there
is now tendency to offer nicely formatted documentation
and to offer multiple formats.  Which means that there is
need for formatting tools.  Old ones are Texinfo and TeX.
And some moment there was push towards SGML and formats
like DocBook.  This needs appropriate tools.  Some people
now prefer tools like 'sphinx' (which needs Python with
several extention packages).  There is 'doxygen' which
extract information from source files and presents nicely
formated info about API (I am not a fan of 'doxygen' but
is seems to be widely used).
 
Beside building program and documentation one wants some
automatic way of running tests.  And people want to
automate more tasks, for example removal of generated
files or installation.
 
Automation requires some scripting language.  For authors
of free programs important point is free availability
(preferably with sources) of choosen language.  That
ruled out things like MS Basic or 'command.com' (and
'command.com' is quite limited).  Unix shell was
available as port of OS on Unix systems and quite early
there were free ports of Unix shell to other systems.
So there were natural preference to Unix shell, simply
because of wide availability.  Similarly with other
classic Unix tools.  One can argue that those classic
tools are now dated, but there is stong inertia in
action: tools are widely used, widely availeble, there
is knowldege how to use them so people keep using them.
 
When one thinks about possible replacements, it is not an
easy job.  Namely, Unix tools evolved and are designed
to work together.  Different tool either must cooperate
nicely with existing tools, which forces similar
behaviour to current tools, or must provide replacement
for the whole toolkit which is much more complicated
than replacing a single tool.
 
When you look at bloat, I expect competing tools to be
much more bloated than classic Unix tools.  Namely,
Unix tools were designed to be simple and power comes
from possibility to compose them.  It is hard to
imagine way of composing tools that is both simple
and offers more possibilities than Unix way.  So
one can have simpler/smaller tools but less powerful.
Or one can increase fuctionality of each tools separately
(to some degree this is happening with Unix tools), but
this increases complexity of each tool.  One can try
to make one tool that replaces the whole toolkit, but
experiance suggest that such a single tool would be
much more complicated.  To summarize, more powerful
tools are likely to be more complicated, hence more
bloated than classic Unix tools.  Given that classic
Unix tools are quite small compared to modern systems
(they can be implemented in few megabytes of code)
incentive to make smaller tools is rather low.
 

This is done even on smaller, simple applications, and also on
those that are supposedly cross-platform that are to be built on
the target. 

 
You can have basic toolkit running on rather wide set of systems.
So this is not a big burden for recipients, in particuar since
the same toolkit may be used to build many programs.
 

If scripts are going to be used, then use them at the developer
site only, and make the script generate the streamlined set of
files for the particular platform of interest. 

 
Current free software ecosystem is colaborative work of many people.
In particular, there are people creating binary packages and if
all you want is running binary you may be better off getting a
package from them.  Now, people creating binaries want to do
this in an automated way.  For them having to give a list of files
to compiler each time when they build a package is too mach work.
They have scripts that fetch sources from the net, check checksums
(to verify authenticity), run 'configure' and 'make'.  They are
willing to install tools which are needed for this work.
 

It should not rely on anything that is not native to the target
platform. 

 
Why not?  You already admited that one need a compiler.  While
a bunch of tiny tools should be a blocker?
 

 And people do not want
to reinvent the wheel, they prefer to use code written
by others. 

 
What wheels are these that are being reinvented? I'm simply arguing
for using either 2 or 4 wheels, not 18!
 
And I'm not suggesting reinventing those 2/4 either, just using the
ones I have on my OS. 

 
Well, one trouble is that things you have on your preferred OS are
not free, one can not use them legally on a different OS.  And when
it comes to developement tools what cames with basic install of
your OS seem to be quite limited.  No law prevents you from installing
free tools and that is simplest solution.
 
Alternative is extra work for developers to reproduce what tool do
within program, that would be reinventing the wheel.  Or some
intermedate stages between binary and source, which require extra
work and there seem to be small interest in such things (say
hundreds of people among milions using free software).

 

 Folks at Bell Labs had several good ideas
and shared them with others.  Those ideas are not limited
to Linux or Unix, you can look at old "Software tools in ..."
books to understand what I am talking about.  MinGW
_distribution_ bundles several tools.  This allows you
to compile programs when build process uses those tools.
You can also use the tools for your purpose.  Aparently
you have very strong "not invented here" syndrom and
insist on usig your own tools. 

 
That's another aspect of it which is not that relevant, other than
it shows just how simple things could be.
 
To build open source projects, I'm happy to use an existing C
compiler. I'm NOT happy about bending over backwards to use CYGWIN,
MSYS2 or WSL because the developers insist on forcing their Linux
dependencies down my throat. 

 
As I explained they want to use tools.  Some tools may be problematic,
but I see no reason to reject base tools.  For most programs you do
not need to CYGWIN or WSL.  Concerning MSYS2, IIUC they reuse small
part of Cygwin to support developement tools.  If for some reason
you find this problematic, you may invest some time in making
the tools more pure, that is using native Windows calls instead
of Posix emulation provided by Cygwin.  Some of this was done
to improve performance, but for people using MSYS2 performance
seem to be OK and clearly using native calls Windows calls on
Windows means that there need to be special variant for Windows
which adds extra work, so they have little motivation to remove
all traces of Cygwin.  If you are concerned about bloat, then
you should take into account that base tools can be quite
small (of order of few megabytes for whole toolkit).
 
If you object using tools that behave the same on Linux and
Windows, then really developers insist on this.  Simply,
using common tools is basic survival strategy.  Otherwise
they would have to spend effort on incompatible "Windows
native toolkit" which would fragment developement effort.
 
Using no tools may work for very simple projects, but
means too much extra work in other cases.
 
Extra remark: in one case I asked people using a program
on Windows which way is most convenient for them.  Rather
old answer was "And Linux" (which was a package allowing
to run Linux binaries on Windows), modern answer was WSL.
You see, people were interested in shortest way to get
running program, not about purity of approach.
 

(Those developers also seem to think that the only alternative to
MSYS2, configure scripts etc is to use the monstrosity that is MS
Visual Studio and all it comprises.) 

 
You see, people that use Windows also want to use tools.  And
when they do not like or can not use simpler tools they go to
Visual Studio.
 

If an application is written in C, then a C compiler should suffice.
 

 That is fine, as long
as you keep this to yourselfs.  But refusing other
folks right to use tools they want to use is not
fair. 

 
Developers can do what they like. But they shouldn't inflict their
choices on other people, especially those using other OSes. 

 
Well, I use Linux and can maintain X based GUI.  Since I do
not have Windows I can not create native Windows GUI.  And
while there were several significant contributors to the
program (some working on Windows), nobody volunteered to
create Windows GUI.  So ATM for this program the only way
to get GUI on Windows is to use Cygwin and X library provided
by Cygwin or run Linux version under WSL.  AFAIK related
approach was used by several commercial vendors providing
both Unix and Windows versions of their programs: they used
X based GUI, on Windows linked with commercial X library and
connected to available X server.  So, simply since "cost" of
providing native GUI is too high, programs used X based GUI.
 
I do not know why you develop software.  For me significant
part is fun, in particular that I can do "interesting"
things with moderate effort.  But all that fun disappears
if somebody want we to do a lot of work for no good
reason (beyond "I want it that way").  There were cases
when I wanted things some way and spent effort to make
them work that way.  I do not complain that other
developers did not want to do work for me.
 

 Or, to put it differently, when you refuse
the tools _you_ need to do by hand needed work.
You may be able to extract from work of others
something that fits your taste (like list of files
to compile), but that is your business. 

 
Fine. In that case tell me what the files are! Lua actually does
this (but it's not obvious, it's also unexpected since the majority
of apps are unhelpful in such matters).
 
When I tried to build A68G, the whole build process, under
VirtualBox Linux, took 5 minutes. I couldn't build under Windows
because of a 30,000-line bash script. 

 
IME shell scripts usually were no problem, except for possibly
requiring some patience (they run significantly slower than on
Linux).  In one case 'configure' script was really problematic,
that is for about 2 weeks Windows NT 4.0 kept crashing, guy
wanting to compile program restarted Windows, 'configure' made
some progress and then Windows crashed again.  After 2 weeks
the guy gave up, run configure on Linux (I think it was the
same hardware), 'configure' needed 2 hours on Linux, but
went fine.  Then some extra hours to compile the program.
Note that there were _system_ crashes running otherwise ordinary
program.  However, Windows now seem to be much more reliable,
so I do not expect trouble like this.  And the mentioned 'configure'
script on Linux run exceptionally long.
 

However, with some effort, I was able to isolate the C files
needed, and fortunately there was no synthesised code. Given that
list of 12 files (!) building the app on Windows took my compiler
one second.

If so, then that's the wrong approach. There are various lesser
compilers which know nothing about mingw or posix of anything of
the sort. (For example, lccwin32, DMC, PellesC, I think Tiny C; I
can't speak for MSVC.) 

 
You may be able to do some things using just a C compiler.
But requesting people to do other things by hand is as
wise as requesting them to code in assembler 

 
What things, what assembly? I'm talking about building a program
written in a HLL using an existing compiler.
 
That seems reasonable enough, yes?
 
The only info needed is the list of files to submit to said
compiler. 

 
If thats to the only needed info than Makefile.in (thing used as
a template for Makefile) should contain something like:
 
SRCS = file1.c file2.c  fileN.c
 
and list of files would be easy to extract from Makefile.in
(just look for list of files).  But usually there is more to
this, like compiler flags, generated headers, etc.  And the
list of files usually depends on OS and possibly other
factors.
 

 (assembler 

may be converted to binary faster than C, and with enough
effort resulting binary will run faster than result of
C compilation :).  Another thing are libraries and
bigger applications.  Some folks want "true Windows look
and feel", some say that proper Windows DLL must be
compiled by MSVC 

 
Any C compiler for Windows can generate DLL files. 

 
I am not an expert on Windows features.  But one things was
that they wanted "components".  Another was exception handling.
So "just a DLL" was too little for them.
 

(otherwise is lack some features that
they consider essential).  But there are people who
had to or wanted to use Windows, but also wanted to
use Unix applications.  Cygwin was addressed to such
folks: it allows you to compile substantial Unix
application so that it runs on Windows.  If you have
no interest in such applications, than fine, Cygwin
is probably not for you.  Today WSL may be better
for such a goal, but in the past Cygwin provided
easiest way to port Unix application to Windows.
If easy port was the goal and application was
substantial, then Cygwin usually was right choice. 

 
As I understand it, POSIX is about 80 header files, of which ~30
are the standard C headers, leaving 50 POSIX headers that do not
exist on Windows.
 
If your Linux app uses those headers, and you want to port it to
Windows, then you have problems to solve.
 
But I would question whether the apps I want to build need all
that. How does POSIX figure in the GMP library for example? 

 
Well, GMP needs some way to signal division by 0.  Which in principle
do not require system stuff, but maybe they use something to
do it nicer.

From GMP manual:
"Division is undefined if the divisor is zero. Passing a zero divisor to
the division or modulo functions (including the modular powering
functions mpz_powm and mpz_powm_ui) will cause an intentional division
by zero. This lets a program handle arithmetic exceptions in these
functions the same way as for normal C int arithmetic."

So, they didn't find reason to complicate it with unnecessary
dependencies.

GMP need memory allocation, which can be done via
'malloc' but also my be dan via system calls.
 

I am pretty sure that they don't use system calls.