On 2024-08-06 16:19, Ruvim wrote:
On 2024-08-05 18:33, Anton Ertl wrote:
Ruvim <ruvim.pinka@gmail.com> writes:
On 2024-08-05 14:04, albert@spenarnc.xs4all.nl wrote:
In article <v8nrb0$3vbpv$3@dont-email.me>,
Ruvim <ruvim.pinka@gmail.com> wrote:
But if you hate parsing words in principle (just because they do
parsing), why not hate such long parsing words like `[if]`, `\`, the
construct "]]...[[", etc? What is an alternative for them?
...
I meant the word `[IF]` by itself, without connection with `WANT`.
>
Not necessarily a parsing word. Could also be treated as something
like another state (i.e., the text interpreter does the parsing, but
does something different with the words than interpretation state or
compile state.
[...]
>
\ parses, but apart from interactions like above it looks fine to me.
>
The word `]]` is also a parsing word (in a standard-compliant
implementation).
>
You mean that implementing ]] as a standard program requires parsing.
That's true, but the usual implementation in systems is as another
state-like thing.
[...]
>
How to implement such functionality without active parsing the input stream?
>
How does :NONAME or ] implement its functionality? Do you also
consider it a parsing word? Note that in some Forth, Inc. Forth
systems ] parses on its own rather than using a state of the ordinary
text interpreter.
>
Yes, I mean that in a standard program the only approach available to implement such functionality is the active parsing approach (at the moment).
The Recognizer API allows to replace short parsing words with syntactically recognized forms (limited by one lexeme). Actually, it is a generalization of numeric literals.
But to implement string literals or string templates (for string interpolation), we still need active parsing. And we need it even if we implement the beginning of such a literal as a recognizable form.
An illustration of this two approaches:
- a parsing word `s"`:
s" lorem ipsum dolor"
- a recognizable form, a lexeme that starts with `"`:
"lorem ipsum dolor"
If a recognizer has no side effects and returns a token translator on success, then, for a string literal, the returned translator *parses* the input buffer (or the input stream) till `"` [1,2].
So, in general, parsing is *inevitable*. Whether it is a parsing word or a parsing translator — it does not matter.
One difference between parsing words and a recognizable syntactic forms is visual — the latter does not require a space before the enclosed content (this makes sense when whitespaces matter, like in string literals).
Another difference is related to API: `[']` and `postpone` are not applicable to the latter (but are applicable to the former).
NB: applying `postpone` to `s"` is not the same as applying `postpone` to a particular string literal.
So, a parsing word can be used by itself to reuse its functionality. But to reuse functionality of some recognizable syntactic form, additional words should be provided. It's notable that the Recognizers API encourages (or even forces) us to provide such additional words.
One of features that Recognizer API provides is the *ability* to reuse the system's Forth text interpreter loop without nesting.
An example:
\ Common data types that are related to the Recognizer API:
\ DataType: recognizer ⇒ xt
\ DataType: tt ⇒ xt
\ DataType: token ⇒ ( S: i*x F: j*k )
\ DataType: qt ⇒ ( token tt | 0 )
\ Functional data types qualifications:
\ DataType: recognizer = ( sd.lexeme -- qt )
\ DataType: tt = ( i*x token -- j*x )
wordlist constant foo-wid
here constant foo-magic
\ DataType: foo-sys ⇒ ( recognizer.prev x.foo-magic )
: end-foo ( foo-sys -- )
foo-magic <> -22 and throw \ "control structure mismatch"
set-perceptor \ restore the perceptor state
;
: recognize-foo ( sd.lexeme -- tt | 0 )
2dup "}foo" equals if 2drop ['] end-foo exit then
foo-wid search-wordlist if exit then 0
;
: begin-foo ( -- foo-sys )
\ save the perceptor state
perceptor foo-magic ( recognizer.prev x.foo-magic ) ( foo-sys )
\ set the system to use our recognizer
['] recognize-foo set-perceptor
;
: foo{ ( -- foo-sys )
begin-foo
; immediate
\ NB: "}foo" is not a word, but just a terminator.
\ create some test words
get-current foo-wid set-current
: test ." (test passed)" ;
: n1 1 ; : n2 2 ; : + + ; : . . ;
set-current
\ run some tests
foo{ test }foo \ should print "(test passed)"
foo{ n1 n2 + . }foo \ should print "3"
t{ foo{ n1 n2 + }foo -> 3 }t
Voilà! The third test fails with error -22. Because of foo-sys, we cannot freely consume or produce stack parameters from/to outside of the foo{ }foo structure.
NB: this example is not intended to show how to implement this dummy functionality, but to show how the Recognizer API is used.
This functionality can be also implemented as a parsing word, using the same Recognizer API, as follows:
: recognize-foo ( sd.lexeme -- tt | 0 )
foo-wid search-wordlist if exit then 0
;
: foo{
[: ( sd.lexeme -- qt )
2dup "}foo" equals 0= if recognize-foo exit then
2drop ['] unnest-translation
;] translate-input-with
; immediate
Where the following common factors are used:
: extract-lexeme ( -- sd.lexeme | 0 0 )
begin parse-name dup if exit then 2drop refill 0= until 0 0
;
: unnest-translation ( -- ⊥ )
true abort" unnest-translation is not handled"
;
: translate-input-till-unnest ( i*x -- j*x )
begin extract-lexeme
dup 0= -39 and throw \ "unexpected end of file"
perceive dup 0= -13 and throw \ "unrecognized"
dup ['] unnest-translation <> while execute
repeat drop
;
: translate-input-with ( i*x recognizer -- j*x )
perceptor >r set-perceptor
['] translate-input-till-unnest catch
r> set-perceptor throw
;
This implementation for "foo{" does not leave foo-sys on the stack, so the following works as expected:
t{ foo{ n1 n2 + }foo -> 3 }t
Also, in this implementation the perceptor state is restored on error.
Thus, with Recognizer API, in some cases we have the choice of using the non-parsing approach or the parsing approach.
The advantages of the parsing approach are:
- no additional items on the data stack;
- restoring the system's state on error;
- the API for reuse the implemented functionality does not impose a particular terminator (and the provided recognizer is not polluted by a particular terminator);
Now the question is: why should one prefer the non-parsing approach to implement some functionality?
-- Ruvim
Operator overloading?
Re: Operator overloading?
