Re: neos Universal Compiler and FORTRAN IV

On 3/28/2025 8:14 AM, Mr Flibble wrote:

On Fri, 28 Mar 2025 14:01:40 +0200, Mikko wrote:
 

On 2025-03-19 16:11:18 +0000, Mikko said:
>

On 2025-03-19 14:32:30 +0000, Mr Flibble said:
>

On Wed, 19 Mar 2025 12:26:20 +0200, Mikko wrote:
>

On 2025-03-18 14:08:50 +0000, Mr Flibble said:
>

On Tue, 18 Mar 2025 15:59:45 +0200, Mikko wrote:
>

On 2025-03-17 16:53:01 +0000, Mr Flibble said:
>

On Mon, 17 Mar 2025 11:21:05 +0200, Mikko wrote:
>

On 2025-03-16 18:40:42 +0000, Mr Flibble said:
>

On Sun, 16 Mar 2025 12:28:24 +0200, Mikko wrote:
>

On 2025-03-15 15:08:47 +0000, Mr Flibble said:
>

On Sat, 15 Mar 2025 11:55:52 +0200, Mikko wrote:
>

On 2025-03-15 04:00:52 +0000, Mr Flibble said:
>

Hi!
>
neos universal compiler (that can compile any programming
language)
is successfully running the tokenization stage tokenizing a
program written in the neos reference language. #cpp #coding
#compiler #compsci #gamedev
>
/Flibble

>
Can it tokenize FORTRAN 60 or FORTRAN IV ?

>
ANY programming language.
>
/Flibble

>
How is neos configured to tokenize FORTRAN 60 ?

>
The same way you would configure it for any other programming
language.

>
If it is configured the same for each programming language then
how does it know how to tokenize?

>
You configure it by providing a language specific neosBNF schema
(grammar)
file (an input to the compilalation process).

>
Is there a neosBNF schema that describes the tokens of FORtRAN 66
or Algol 60?

>
Not yet.

>
The definition of string literal of Algol 60 would be a good example
of something that cannot be defined with a regular expression and is
therefore impossible or at least complicated with an ordinary
tokenizer.

>
neos does not use regular expressions and the neos grammar is context
sensitive (i.e. not context free).

>
A context free grammar should be enough for tokenizing strings of Algol
60.
>
Tokenizing statements of FORTRAN 66 is a harder problem.

>
Some test cases for a FORTRAN IV tokenizer:
>
         REALITY IS, AS I MAY LOGICALLY SAY, REAL LOGICAL = WHAT I SAY
>
Note that these lines begin with 8 spaces.
>
Tokens on the first line:
keyword REAL identifier ITYIS comma identifier ASIMAY end of statement
>
Tokens on the second line:
keyword LOGICAL identifier LYSAY comma identifier REAL end of statement
>
Tokens on the third line:
identifier LOGICAL assignment symbol identifier WHAT I SAY
>
Parser needs to know whether an instance of REAL or LOGICAL is an
identifier or keyword.
>
I don't understand how neos can be configred to tokenize the above test
lines and other FORTRAN IV statements.

 If a native FORTRAN IV compiler can parse it then neos can parse it as
long as it is somehow expressable in the neos attribute grammar.

But that was the question: Can it be parsed in a neos expressible grammar? Note that those early Fortran grammars were 1) not finite state, 2) not context free, and 3) inherently ambiguous.It was possible, for example, to write a "format" and an "assignment" statement that were character to character identical. As an aside, a friend who wrote an early Fortran compiler that compiled such a chimera so that it would do an assignment if executed and could be referenced as a format from an IO statement.
--
Jeff Barnett