Re: Alternative for long parsing words

On 2024-08-05 18:33, Anton Ertl wrote:

Ruvim <ruvim.pinka@gmail.com> writes:

On 2024-08-05 14:04, albert@spenarnc.xs4all.nl wrote:

In article <v8nrb0$3vbpv$3@dont-email.me>,
Ruvim  <ruvim.pinka@gmail.com> wrote:

But if you hate parsing words in principle (just because they do
parsing), why not hate such long parsing words like `[if]`, `\`, the
construct "]]...[[", etc? What is an alternative for them?

...

I meant the word `[IF]` by itself, without connection with `WANT`.

 Not necessarily a parsing word.  Could also be treated as something
like another state (i.e., the text interpreter does the parsing, but
does something different with the words than interpretation state or
compile state.

[...]

 \ parses, but apart from interactions like above it looks fine to me.
 

The word `]]` is also a parsing word (in a standard-compliant
implementation).

 You mean that implementing ]] as a standard program requires parsing.
That's true, but the usual implementation in systems is as another
state-like thing.  And you also get interesting interactions if you
combine it with [if], or parsing words like \ or (.  The sequences
inside ]]...[[ are short enough that this is rarely necessary, and if
it is, you can do, e.g.
 ]] bla [[ ( x y ) ]] blub [[
 instead of
 ]] bla ( x y ) blub [[
 Not great, but has not been a significant issue yet.
 

How to implement such functionality without active parsing the input stream?

 How does :NONAME or ] implement its functionality?  Do you also
consider it a parsing word?  Note that in some Forth, Inc. Forth
systems ] parses on its own rather than using a state of the ordinary
text interpreter.
 

Yes, I mean that in a standard program the only approach available to implement such functionality is the active parsing approach (at the moment).
The Recognizer API allows to replace short parsing words with syntactically recognized forms (limited by one lexeme). Actually, it is a generalization of numeric literals.
But to implement string literals or string templates (for string interpolation), we still need active parsing. And we need it even if we implement the beginning of such a literal as a recognizable form.
An illustration of this two approaches:
- a parsing word `s"`:
   s" lorem ipsum dolor"
- a recognizable form, a lexeme that starts with `"`:
   "lorem ipsum dolor"
If a recognizer has no side effects and returns a token translator on success, then, for a string literal, the returned translator *parses* the input buffer (or the input stream) till `"` [1,2].
So, in general, parsing is *inevitable*.  Whether it is a parsing word or a parsing translator — it does not matter.

But if such long parsing words like `[IF]` and `]]` are acceptable, why
the words that parses a single lexeme are not acceptable? (apart from
naming and terminator visibility problems)?

 A problem with parsing words like ' and ['] is that code using them
usually cannot be copied and pasted between interpreted and compiled
code. 

I would define interpretation semantics for `[']` to be the same as for `'`, as follows:
   : compilation ( -- flag ) state @ 0<> ;
   : tt-lit ( x -- x | ) compilation if lit, then ; immediate
   : ['] ( "name" -- xt | ) '  tt-lit ; immediate
So, this problem can be easily solved.

A problem with parsing words like S" is that implementing them
properly requires mechanisms beyond what has been traditionally
available.

It does not require mechanisms beyond the Forth-2012.
If you say that `postpone` behaves incorrectly for such words, you now my answer — it is a problem of a particular implementation of `postpone`, and this problem can be easily fixed [3].
[1] String literals via Recognizer API <https://gist.github.com/ruv/67134d5eb0493969520f77d3f89f85d5#file-file-01-string-literal-via-recognizer-v5-fth>
[2] A recognizer for multi-line string literal <https://gist.github.com/ruv/60450afc33a7ad0f614bedefe4841ddb>
[3] A polyfill for `postpone` <https://github.com/ForthHub/discussion/discussions/103#user-content-solution>
--
Ruvim