Re: New routine in the development version : Tokenize()

[Benoît Minisini <benoit.minisini@xxxxxxxxxxxxxxxx>]

Le 03/10/2025 à 09:43, Benoît Minisini a écrit :
> Hi,
>
> I have just added a general-purpose 'Tokenize()' routine to Gambas, that 
> splits a string into tokens (identifiers, numbers, strings...), 
> according to a few arguments.
>
> The documentation is at: https://gambaswiki.org/wiki/lang/tokenize
>
> You must login to get the last version that fixes many typos.
>
> The aim of that routine is being useful without taking too many 
> arguments - as a real dynamic parser is like 'gb.highlight', it needs a 
> compiler!
>
> Please tell me what you think about 'Tokenize()', if you find it useful, 
> if you think some options can be added or removed, and so on.
>
> Regards,

Maybe 'Tokenize()' will be renamed as 'Parse()'. I'm not sure yet.

--
Benoît Minisini.