Skip to content

Highlighting: keywords are not necessarily properly tokenized #22

@ymeine

Description

@ymeine

There is a rule (Word) in the highlighter which is able to parse a portion of a source code considering a set of alternative words. For instance:

  • true or false for boolean tokens
  • +, -, *, /, etc. for operator tokens

The technique this rule uses for parsing is not perfect (or maybe the problem comes from the model of rules), since it doesn't consider white spaces. In some cases, it's fine:

  • var a = false;: we indeed want false to be detected as a boolean, even though there is no white space after
  • var a = 2+2: same thing for this plus operator

However, in some cases, this is not suitable. Take for instance any identifier whose name begins with false, like false_assertion. The part false in the identifier will be tokenized as being a boolean.

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions