Misc (semi OT): Well, distractions...

So, recently, have been distracted on other stuff.
My ISA project has mostly stabilized, and the main things remaining going on here are basically going right up the side of a mountain in terms of difficulty curve. Like, as quickly as "low hanging fruit" tasks get done, whatever is left becomes progressively more difficult (and to push into "more than just a toy" areas will likely require far more work than I can reasonably pull off in a reasonable timeframe).
I was also starting to miss tinkering around with 3D engine stuff some, so, yeah...
But, a recent line of fiddling has gone in an odd direction.
   I felt a need for a basic script language for some tasks;
   I wanted to keep code footprint low;
     My past script interpreters had a lot more code footprint.
   I didn't want any external dependencies.
     Using something like Lua would mean a dependency on a library.
     Despite claims of being "small", Lua is still quite large...
       Like, it is "small" if compared with Python or V8 or something.
       But, still like around twice as much code as the Doom engine...
So, I went for a BASIC variant:
It is possible to implement a BASIC interpreter with fairly little code.
Or, at least, for an 80s style dialect.
There was some stuff in early BASIC's that did not make sense for use in a 3D engine, so I left it out.
The core dialect was a fairly limited one:
   i=0
   label:
   i=i+1
   if i<10 goto label  //THEN is optional with GOTO
So, no real loops or other block-structured control-flow, as these would essentially require a more complex interpreter.
I ended up making 'THEN' the optional keyword here, rather than 'GOTO' as "IF cond GOTO label" is unambiguous, whereas "IF cond THEN label" requires figuring out whether it is a label or a command.
For now, if one wants a block structured IF/THEN/ELSE/ENDIF, they need to build it themselves with GOTO.
There is GOSUB/RETURN, but this was essentially a GOTO that remembers the return path on a small internal stack. So RETURN would effectively go to the line following the location where GOSUB was called.
General interpreter structure works by reading in the code, and breaking it into lines and tokens.
Interpreter then walks code line-by-line, directly driving logic based on the tokens it encounters.
Not really efficient, but smaller code footprint.
I was torn between "tokenize then interpret", or "leave code as big blob of ASCII text and then parse each line one at a time". I ended up going with pre-tokenization as it added a few perks, is slightly less slow, and possibly even helped overall regarding code footprint (while the initial loading step is more code, the rest of the parsing logic is less code).
Initially, I was using global-only scoping, but it doesn't take long to realize that only having global variables raises problems:
Every piece of code needs to use its own variable names to not stomp code elsewhere;
Recursive code is essentially impossible.
Most lazy solution to this:
   Add dynamic scoping.
     TEMP i=3
   Creates 'i' only within the scope of the current GOSUB.
   RETURN happens, variable goes away.
   Inner scopes mask outer scopes, without interfering;
   This allows recursion, and some amount of sanity regarding variables.
Dynamic scoping is, however, not the direction that the other BASIC's went. For example, QBASIC had went over to block-structuring and lexical scoping. However, for these, would likely need a "real" parser, and not just something that works by reading lines one at a time with an option of GOTO.
The other option would have been a LOCAL keyword being used for dynamic variables.
But, then I realized I wanted return values in some cases, so:
   x = GOSUB label
   ...
   label:
   ...
   RETURN expr
Works, but is unorthodox (also breaks the function/subroutine distinction that is present in most other BASICs; but mostly absent in C family languages).
However, the GOSUB here is not a true expression, so:
   x = (GOSUB label) + 1
Is not valid.
Well, and then another bit of wonk:
   t = GOSUB label x=3, y=4
Which works by creating x and y within the callee's frame, so can pass arguments. Also: was the laziest option given the existing implementation. But, is kinda weird...
Namely, caller doesn't need to fetch the callee function or know anything about its argument list, as it is merely binding each variable within the callee's dynamic environment frame.
Dynamic scoping was less code in effect because it was merely a stack, and call/return marks off the stack position (along with the internal line number and similar).
Well, and added vectors:
   v0 = (vec 1,2,3)
Which, possibly, don't really fit in with BASIC.
But, interpreter is still around 1500 lines of C, and would likely have been bigger if I did this stuff "properly".
Though, now I am starting to second guess myself and wonder if it might have ended up being less code at this point to implement a small version of a language similar to Emacs Lisp.
Well, since at some point "parse tokens into S-Expressions and then walk the S-Expression lists" becomes less code than "read line, break into tokens, and walk over tokens and dispatch logic based on said tokens" once one moves past a certain level of triviality.
And, with "vec", the difference between "(vec 1,2,3)" and (vec 1 2 3) is not exactly large. I had also been half tempted to use [1,2,3] syntax, as it was going the route (like in OpenSCAD) that arrays and vectors are basically the same thing.
In this case, interpreter infers if you do:
   vec3=vec1+vec2
And, vec1 and vec2 are arrays of the same length, then to do a vector operation.
...
Well, and I end up trying to implement an 80s style BASIC and within a week already starts mutating into some sort of weird hybrid of 80s style BASIC and Emacs Lisp...
Well, and to add to this, I had also considered adding CSG operations, to allow using it in a vaguely similar way to OpenSCAD, for 3D modeling uses.
   box1=(csgaabb (vec -10,-10,0),(vec 10,10,4))
   box2=(csgaabb (vec -1,-1,4),(vec 1,1,40))
   base1=(csgunion box1,box2)
   ...
But, again, this wouldn't be that different from, say:
   (let box1 (csgaabb (vec -10 -10 0) (vec 10 10 4)))
   (let box2 (csgaabb (vec -1 -1 4) (vec 1 1 40)))
   (let base1 (csgunion box1 box2))
Though, with a Lisp variant, would have likely ended up with more proper functions by this point:
   (defun func (x y) (+ x y))
...
But, errm...
Though, can note that in G-Code, there was a trick that can be used to encode block-structuring without the need to break from line-by-line parsing; namely matching source/end labels.
Where a similar idea could maybe be adapted as:
   SUB label
   ENDSUB label
By relying on label matching, though the ENDSUB label could be omitted if one assumes that since SUB may not nest, it is sufficient (if encountering SUB) to merely scan forwards until one encounters the next ENDSUB (at which point, things continue line-by-line).
So, say:
If SUB is encountered, it is understood as a label, but then interpreter immediately scans forward to find the ENDSUB and then jumps to there. Could maybe keep track of an argument list with said label, which may then be used if "CALL sub(1,2,3)" or similar is used.
Could use similar for IF-THEN/ELSE/ENDIF, but would disallow nested IF (well, unless we keep track of IF/ENDIF levels).
Could in theory use a count-trick to allow for this to allow nested IF, but then if one mixes it with GOTO, a horrible mess may result.
Well, and faking a block-structured BASIC by using a nesting count strategy (like I had used in my C preprocessor) is not likely to scale very well.
Well, would be kind of a pile of crap, but could maybe get up to something closer to 90s era BASIC levels still without needing the relative added complexity of first needing to parse it into an AST.
I guess, if I could get it up closer to QBASIC levels while still staying below 2 kLOC for the interpreter, via wonky tricks, this would be kind of funny in a way.
Though, it is either this, or redo things partly and jump to a "slightly more scalable" design, like BASIC-style syntax over a core more like that of Emacs Lisp (possibly jumping over to CONS lists and a list-walking interpreter).
While at present, the 3D engine does already have access to BGBCC's AST code (being used mostly for XML and similar), I am not particularly inclined to use it for an AST walking interpreter (I have been down this path before... Would be both bulkier and slower than either token-walking or cons-list walking; like the merit of XML ASTs here being structural flexibility, but ease or performance of working with them, not so much...).
Then again, can note that I did end up putting the type-handling stuff in a different C file (from the parser/interpreter), which is another 0.6 kLOC at the moment.
Decided to leave of writing about the typesystem. It is a similar "64-bit type-tagged pointers" system similar to my previous VMs. No objects yet, if objects were added, would probably be JavaScript style. At present, the interpreter leaks memory if using any complex types as values, may need to add reference counting or similar.
...
So, within the 3D engine:
Have been experimenting with trying to shift over to 16 color artwork, but with the issue that I may not have the art skills to pull off 16 color and have it not look like crap.
I had saw a game recently ("The Crimson Diamond") where the creator had used 16 color to good effect. They were in turn copying the style of some of the 1980s Sierra games (apparently, most directly, "The Colonel's Bequest").
Seemingly, the newer game had used more detailed scenery though, as the backgrounds appeared to be "full bitmap" whereas the 1980s Sierra games were more of a "line graphics and flood fill" style. So, a lot more big flat shapes and not so much use of dithering. Apparently, these games were mostly using bitmap graphics for sprites, but for the backgrounds the engine would draw line graphics and then flood-fill between the lines ("vector graphics" style).
Admittedly, I had not saw that whole era when it was new, as by the time I started using computers everyone had mostly already moved over to 256 colors and basic 3D (like, my childhood had Wolf3D, Doom, and Quake; So had sort of missed out on the era of 16 color text adventure games...).
But, these older 80s games are sort of interesting in an aesthetic and mechanics sense.
I was trying to integrate some text-adventure mechanics into one of my 3D engines. Internally, it already had a Quake-style "console" system, and in this case the "text adventure" parts were partially glued onto the console system. I ended up partly borrowing a mechanic similar to Minecraft's UI, where hitting '/' brings up a line interface where one can type stuff. This works OK for both command entry and for text-adventure stuff, and is less intrusive here than using the "~ to pull down console" strategy that was used in Quake and similar.
In this case, I am also partly using a text-based interface for things like inventory management, but as of yet there are no items for the character's inventory (opening the inventory will keep the command-bar open, since all one can really do in there is type commands and see the results).
Previously, there was a partial menu-driven dialog system (glued onto the menu system). So, commands like "look" or "talk to" or similar in this case will leverage the dialog menus for interaction.
Unclear if it will stick with a menu-driven interface (press number of option you want), or go to an approach more like "Crimson Diamond" where the only option is either advancing linearly or typing commands. Actual mechanics aren't much different though in my case, as if there is only a single option, it is understood more as "press any key to use this option" (generally either advances to the next blob of text or closes the dialog box).
For the 3D parts, the 3D engine is mostly using voxel terrain (similar to Minecraft). Main change recently is attempting to design and use 16 color texture art, and trying to make it look "not terrible" (vs the usual roughly RGB555 color I would have otherwise).
Block textures mostly use a texture atlas, with 64x64 pixels for each texture block. The basic layout of the atlas was carried over from my prior BGBTech2 engine, as were most of the original textures (which were in turn mostly either originally drawn, or partly repurposed artwork mostly from FreeDoom; which is sort of where a lot of the sound effects came from as well, though with some editing).
For both BGBTech2 and BGBTech3, had thus far mostly been using sprite artwork. This was in contrast to my first engine, which had used 3D models.
I had partly abandoned 3D models after the first engine mostly because of a detail vs effort tradeoff:
   3D models: More effort relative to detail detail;
   Sprites: Less effort and more detail.
Downside of sprites being that they are essentially flat. They work OK for some things, but for other things are a bit lacking.
So, had started looking into reintroducing some 3D models into my uses.
Probably, would stick to sprites for things like mobs and NPCs, but thinking of 3D models mostly for scene objects.
Say, if one puts a chair or table or similar in a room, a 3D model would work well, but using a sprite here looks pretty bad.
One issue though, is 3D modeling software:
Most of the semi OK FOSS 3D modelers, don't do texture maps, nor much of the time even support colors.
But, the main option that does support textures, and animations, is Blender. Big downside of Blender being that it is effectively monolithic, and there isn't really a good way to get anything one makes in Blender out of Blender into some form you can actually use (want to try to pull stuff out of a ".blend" file, well, good luck there...).
One of the more usable approaches to 3D modeling I have found was using OpenSCAD and then exporting STL files. Downside: No textures here.
Have experimented with loading STLs and projecting textures onto them, which probably works in the simple case, but is fairly limited (if you want a plain wood chair or table, probably fine; if one wants different textures on different parts of the model, this is a problem).
One option could have been to write a parser and interpreter for the openSCAD language, but with extensions to support textures and animations...
Though, less effort being to just sort of kludge CSG stuff onto my BASIC dialect. Then I can make 3D models this way, and then project textures onto them.
A few of the more unorthodox features, namely the "GOSUB with return value" and similar, being intended primarily in the context of making the language more suitable for CSG.
It will differ some from OpenSCAD in that CSG solids will be first class values, rather than implicit.
Though, likely, the return value of running a CSG model script would need to be the generated 3D model.
Granted, still wonky...
Besides the option of running CSG stuff in Basic and/or importing STLs, had started work on a basic 3D model format to use for in-engine use.
It aims to be "not horribly complicated" (downside of many existing formats), where:
   STL: Limited (no textures or colors), but widely supported.
   OBJ, AC3D: ASCII based, kinda bulky.
   DAE, glTF: Horribly complicated
     Trying to "address everyone's problems";
     More designed around the choices of big 3D modeling tools...
   3DS: Old proprietary binary format.
     Might be more useful if more widely supported by tools.
     Still more a format for the 3D modeler tool, than for in-engine.
I had designed a basic format along vaguely similar lines to the Quake and Half-Life model formats, though slightly different:
   Coords are block-floating-point rather than bytes.
     Quake had used a per-model scale field (and 3 bytes);
     IIRC, for HL, the coords were interpolated within a bounding box.
     Mine uses 3x 9-bit fractions and a 5 bit exponent.
       Though, this is less precise than what Quake3's MD3 had used.
   Normals are encoded, expressed in an A-Law like format
     (S.E3.F4, unit range, "FP8A")
     Ended up having the max value be 1.937,
       as while this wastes some dynamic range,
       it is better for normals and quaternions.
   S/T coordinates are fixed point,
     12 or 16 bit depending on vertex format.
     With 11 or 12 bits for fractional part.
       12b: Covers range of -0.5 to 1.5, to deal with seams.
   Vertex colors are supported, encoded via a 256-color fixed palette.
   Encodes an optional skeleton and animation loops.
     Bone rotations are encoded as quaternions in FP8A form.
       Vs HL, which had used 3-byte Euler angles.
     Translations, if used, use the block-scaled form.
     Animations only encode bone positions in keyframes
       Similar to the HL models.
       Does mean loader needs to interpolate animations though.
   ...
Vertices are 8/12/16 bytes, depending on features needed.
Triangles are 4 or 8 bytes, depending on the number of vertices in the mesh (4 byte triangles limited to fewer than 1024 vertices in a mesh).
Initial tests give models that are around 1/8 to 1/10 the size of corresponding STL models, while also having the relevant features (though, the need to unpack stuff to Binary32 floats in RAM does mean the in-RAM format is a lot more bulky).
For static models, the size is what it is.
For animated models, there will be a sort of cache system for transformed vertex arrays.
Fully decoding every pose in advance would eat too much RAM;
Recalculating positions every time the model is drawn would have a lot of performance overhead.
Though, ironically, I am thus far mostly using a design for 3D character models "inspired by" the "Money for Nothing" music video (and Weird Al's parody of said video).
Namely, geometric and mostly flat colors (following a 16-color palette). Though, ironically, this was a difference between the original and Weird Al's parody: The original was doing 3D with model base colors mostly in accordance with a 16-color palette; whereas Weird Al's version used a more free-form palette (skin tones, multiple shades of brown, etc, that don't really map to 4-bit RGBI).
Though, in either case, due to lighting, there will be more colors in the final render.
Not that I couldn't do better, but for humanoid characters, there is this large "valley of potato" between simpler geometric designs, and realistic humanoids, where venturing into this space (but not making it all the way to the other side) results in humanoid characters that look like they are made of misshapen potatoes and thus, rather ugly (semi-common in early/mid 90s CGI whenever they tried to do "realistic" humanoids).
Where, say:
   "Money for Nothing" designs.
     Or, "Lawnmower Man", where they screwed it up...
       So, still ugly, despite simplistic graphics.
   "ReBoot": Pushing it, but still unnatural enough,
   "Valley of potato" / "uncanny valley"
     Most "realistic" CGI humanoids.
     Particularly 90s stuff, and student art stuff.
   "Good": Professional VFX
     Not readily achieved without a pile of money.
   "Stylized": Professional or semi-professional
     Make the characters cartoony, avoiding the ugly.
     Common in most CGI shows that people actually bother to watch.
Well, then shows like "Miraculous Ladybug" are on the stylized side, but trying to push a little too close enough to "realism" for comfort. But, at least (unlike 90s CGI) manages to avoid the issue of characters looking like they have potatoes for heads.
Though, I am left to partly suspect part of the "potato" look may result from, say:
Trying to define humanoid features first using coarse geometry, with exaggerated features, and then throwing NURBS or similar on top of it.
...