Re: question about linker

On 03/12/2024 14:34, David Brown wrote:

On 02/12/2024 22:53, Bart wrote:

So, how would you have organised the 16-module example I posted elsewhere? (Not a C project, these are 16 source files, so no headers etc.)
>
Because two posters here have suggested my organisation is poor, but without knowing how big, small, or complex my projects are.

 No one (as far as I have noticed) have said that your organisation /is/ poor - they have said it /sounds/ poor from the way you describe it. The difference is very significant.
 For file organisation, I'd likely have all the modules in one directory unless there is a particular reason to split them up.  I would not have any non-project files in that directory.
 But the questions raised about your organisation was not a matter of where you store your files, or how they are divided in directories.  It is about how you organise the code and split functionality between files (or directories, for bigger projects).
 What you have described is modules that have far too much in one file, modules with little or no structure as to where things are in the file,

Because it doesn't really matter. (My language allows out-of-order everything. So all module-scope variables could go at the end of the file, or file-scope variables at the end of the function! However I don't do that.
But I really don't want to care about whether function F precedes G in the file, or follows it. Any more than I would care whether a file "F" is stored before or after file "G" in a directory! The ordering could be sorted in different ways for display; perhaps an editor could do the same. (I guess your IDE does that.)

little to no structure or control over which modules use facilities from which other modules, and completely random inter-module dependencies which can happily be circular.

They can be circular when it makes sense. They are after all part of the same program!
In C, if you have 100 modules, but modules 23 and 87 need to share some variable or function, it can be visible to the other 98 too, or can clash with the same name thaty 17 and 26 want to share. Or with a name that module 72 forgot to make static.
Or module 49 exports variable 'abc' as int, but 53 imports it as 'char*', then fun and games follow. C has a lot worse problems!
Being able to split a too-large module into two or more files without worrying about hierarchy is a good thing; isn't it? If the original function exported F G H, accessed as X.F, X.G, X.H, do you then have to change all the calls to X.F, Y.G, Z.G because of how the original file has been split up?
My scheme just makes this stuff very easy and effortless.

These opinions are formed from how you describe your code and your language.

Below I've given the actual set of modules from my C compiler project, partly annotated. It comprises three libraries whose sources files are compiled into one executable.
The 'PCL' one (pc_ files) will probably be split up too at some point; right now there's no need.
The average module size is 1000 lines, but some are larger. For example, pc_genmcl, which is a set of 150 handler functions for the 150 opcodes of my IL.
And pc_run, which is the interpreter for the IL, whose core is an 1100-line function containing a special kind of looping switch which is implemented as a computed-goto (different from a normal switch since there are N dispatch points rather than 1).
So a module is larger when it needs to encapsulate functions that do part of the same job. There are several modules which have to dispatch on an IL bytecode, or AST tag, or native code instruction.

 

>
BTW your project doesn't sound that big, especially if you have a penchant for having a larger number of smaller files.
>
A line-count would give a better idea.
>

 For the whole project (relevant to things like the build and the organisation):
 -----------------------------------------------------------------------
Language             files          blank        comment           code
-----------------------------------------------------------------------
C                      155          13849          34547          80139
C/C++ Header           284          13719          62329          61056
C++                     43           2447           1230          13009
-----------------------------------------------------------------------
SUM:                   482          30015          98106         154204
-----------------------------------------------------------------------

So about 600 lines per file.

 For the project-specific code, rather than libraries, SDK, etc.:
 -----------------------------------------------------------------------
Language             files          blank        comment           code
-----------------------------------------------------------------------
C++                     39           2217           1020          11820
C/C++ Header            44           1078            798           2696
C                        3            259            237           1152
-----------------------------------------------------------------------
SUM:                    86           3554           2055          15668
------------------------------------------------------------------------

And here about 250 lines per file. Total line count isn't that different from my 1990s CAD app, which was about 150Kloc, more than half consisting of scripted modules.
This doesn't include the add-on scripted modules that OEMs would write (plus alls sorts of data files, libraries of parts etc) to form the final product.
Anyway, below are my C compiler modules as promised.
Here, there is a hierarchy. None of the second two libraries can access anything from the main application. Both the first two can can access imported entities from the third library.
-----------------------------------
# Main application
project =
     module cc_cli
# Global Data and Tables
     module cc_decls
     module cc_tables
# Lexing and Parsing
     module cc_lex
     module cc_parse
# Generate PCL
     module cc_genpcl
     module cc_blockpcl
     module cc_libpcl
# General
     module cc_lib
     module cc_support
# Bundled headers
     module cc_headers
#   module cc_headersx
# Diagnostics
     module cc_show
#   module cc_showdummy
# IL Backend
     $sourcepath "c:/px/"
     import pcl
end
# Backend library
project =
     module pc_decls
# API to generate IL from a front-end compiler
     module pc_api
# IL Diagnostics
     module pc_diags
#   module pc_diags_dummy
     module pc_reduce         # experimental IL optimiser
# PCL (IL) Interpreter
     module pc_run
     module pc_runaux
# Tables (eg. types and IL opcodes)
     module pc_tables
# x64 backend (pcl -> mcl)
     module mc_GenMCL         # Main loop scanning PCL code
     module mc_AuxMCL
     module mc_LibMCL         # API for building MCL (native code) representation
     module mc_StackMCL       # Deal with converting stack-based IL to reg-based native code
# Generate SS tables (MCL -> SS; SS is binary native code)
     module mc_GenSS
     module mc_Decls as md    # (eg. x64 opcode enums)
     module mc_OBJdecls       # Describe PE-format data structures
# Write ASM (currently only one at a time is compiled in)
     module mc_WriteASM       # MCL -> ASM source
#   module mc_WriteNASM      # MCL -> NASM-format source
# Write EXE/DLL/OBJ (from SS)
     module mc_WriteEXE       # Create PE image and write EXE or DLL
     module mc_WriteOBJ
# SS diagnostic display
#   module mc_writess
#   module mc_disasm         # x64 disassembler for code segments
     module mc_writess_dummy
# SS -> MCU -> MCX/MCB (Runnable in-memory code; or MX binary file)
     module mx_decls
     module mx_run
     module mx_lib
     module mx_write          # Write MX-format binary
end
# Standard library of my language (this is imported by default)
project =
     module msys          # Language support
     module mlib          # The std library
     module mclib         # Import some C std library functions
     module mwindows      # OS-specific functions
     module mwindll       # provides LIBFFI-like capabilities
end
-----------------------------------
When I used to support Linux targets, then mwindows was replaced by 'mlinux', or 'mnos' for an OS-neutral version. This module provides a common set of wrapper functions over some OS-specfic functions
For a C target (of the compiler for this project, then a target of this project), then mwindll is replaced mwindllc, which provides some limited functionality using HLL code only (so that it can be transpiled to C)