Re: neos Universal Compiler and FORTRAN IV

On 3/28/2025 1:27 PM, Mr Flibble wrote:

On Fri, 28 Mar 2025 13:05:10 -0600, Jeff Barnett wrote:
 

On 3/28/2025 8:14 AM, Mr Flibble wrote:

On Fri, 28 Mar 2025 14:01:40 +0200, Mikko wrote:
>

On 2025-03-19 16:11:18 +0000, Mikko said:
>

On 2025-03-19 14:32:30 +0000, Mr Flibble said:
>

On Wed, 19 Mar 2025 12:26:20 +0200, Mikko wrote:
>

On 2025-03-18 14:08:50 +0000, Mr Flibble said:
>

On Tue, 18 Mar 2025 15:59:45 +0200, Mikko wrote:
>

On 2025-03-17 16:53:01 +0000, Mr Flibble said:
>

On Mon, 17 Mar 2025 11:21:05 +0200, Mikko wrote:
>

On 2025-03-16 18:40:42 +0000, Mr Flibble said:
>

On Sun, 16 Mar 2025 12:28:24 +0200, Mikko wrote:
>

On 2025-03-15 15:08:47 +0000, Mr Flibble said:
>

On Sat, 15 Mar 2025 11:55:52 +0200, Mikko wrote:
>

On 2025-03-15 04:00:52 +0000, Mr Flibble said:
>

Hi!
>
neos universal compiler (that can compile any programming
language)
is successfully running the tokenization stage tokenizing
a program written in the neos reference language. #cpp
#coding #compiler #compsci #gamedev
>
/Flibble

>
Can it tokenize FORTRAN 60 or FORTRAN IV ?

>
ANY programming language.
>
/Flibble

>
How is neos configured to tokenize FORTRAN 60 ?

>
The same way you would configure it for any other programming
language.

>
If it is configured the same for each programming language then
how does it know how to tokenize?

>
You configure it by providing a language specific neosBNF schema
(grammar)
file (an input to the compilalation process).

>
Is there a neosBNF schema that describes the tokens of FORtRAN 66
or Algol 60?

>
Not yet.

>
The definition of string literal of Algol 60 would be a good
example of something that cannot be defined with a regular
expression and is therefore impossible or at least complicated with
an ordinary tokenizer.

>
neos does not use regular expressions and the neos grammar is
context sensitive (i.e. not context free).

>
A context free grammar should be enough for tokenizing strings of
Algol 60.
>
Tokenizing statements of FORTRAN 66 is a harder problem.

>
Some test cases for a FORTRAN IV tokenizer:
>
          REALITY IS, AS I MAY LOGICALLY SAY, REAL LOGICAL = WHAT I SAY
>
Note that these lines begin with 8 spaces.
>
Tokens on the first line:
keyword REAL identifier ITYIS comma identifier ASIMAY end of statement
>
Tokens on the second line:
keyword LOGICAL identifier LYSAY comma identifier REAL end of
statement
>
Tokens on the third line:
identifier LOGICAL assignment symbol identifier WHAT I SAY
>
Parser needs to know whether an instance of REAL or LOGICAL is an
identifier or keyword.
>
I don't understand how neos can be configred to tokenize the above
test lines and other FORTRAN IV statements.

>
If a native FORTRAN IV compiler can parse it then neos can parse it as
long as it is somehow expressable in the neos attribute grammar.

>
But that was the question: Can it be parsed in a neos expressible
grammar? Note that those early Fortran grammars were 1) not finite
state, 2) not context free, and 3) inherently ambiguous.It was possible,
for example, to write a "format" and an "assignment" statement that were
character to character identical. As an aside, a friend who wrote an
early Fortran compiler that compiled such a chimera so that it would do
an assignment if executed and could be referenced as a format from an IO
statement.

 The tokenisation step is entirely optional in neos which instead deploys a
configurable multi-stage parsing "pipeline" so again there should be no
problem if FORTRAN IV is expressable in the neos attribute grammar given
the problem is itself not intractable as FORTRAN IV compilers already
exist.

I REPEAT -- But that was the question: Can it be parsed in a neos expressible grammar?
--
Jeff Barnett