I don’t think a tokenizer like that is a good idea for Forth. You’ve got to read the next space separated thing, find out if it’s supposed to be compiled or run.
Eg the naive tokenizer would probably not work for .” for example.
I agree. The tokenizer in the article completely misses the point of how Forth works: the tokenization is supposed to be driven by the words themselves, i.e. ." is looking for a " delimiter, ( is looking for a ).
Not to mention that the rest of the compiler also misses the point of how Forth works. This compiles a fixed subset of Forth and entirely misses out on the extensibility of the language.
microtherion|4 months ago
Not to mention that the rest of the compiler also misses the point of how Forth works. This compiles a fixed subset of Forth and entirely misses out on the extensibility of the language.