Function biodivine_sketchbook::algorithms::fo_logic::tokenizer::try_tokenize_recursive

fn try_tokenize_recursive(
    input_chars: &mut Peekable<Chars<'_>>,
    top_level: bool,
    top_fn_level: bool,
) -> Result<(Vec<FolToken>, char), String>

Process a peekable iterator of characters into a vector of FolTokens. This function is used for both tokenizing a top-level formula an expression that is given as fn’s argument.

Returns a vector of (nested) tokens, and a last character. The last character is important when we are parsing function arguments (to find out if another argument is expected or we already processed the closing parenthesis). When parsing the top-level formula expression (not a function argument), we simply return ‘$’.

top_fn_level is used in case we are processing an expression passed as argument to some function symbol (then ‘,’ is valid delimiter).