Second+ statement will not be parsed (as DefaultContext) if first statement is not correct

[surister]

When parsing a query, if it has more than one statement, the statement of index 1+ has to start with a correct keyword to be correctly identified as an statement and therefore parsed.

For these examples the ExceptionError listener is turned off.

Ex:

sqlparse("""
    ELECT 1; -- wrong
    Begin 2; -- correct
    ELECT 2; -- wrong
""")

>>> [Statement<ELECT 1>, Statement<Begin>]

sqlparse("""
    ELECT 1; -- wrong
    Begin 2; -- correct
    SELECT 2; -- correct
""")
>>> [Statement<ELECT 1>, Statement<Begin>, Statement<SELECT 2>] 

 sqlparse("""
    ELECT 1; -- wrong
    egin 2; -- wrong
    SELECT 2; -- correct
""")

>>> [Statement<ELECT 1>]

Note in the last example how it needs a correct first keyword to keep parsing statements, even if the third 'statement' is correct, this only seems to apply from the first statement onward.

Definition of statements: https://github.com/crate/crate/blob/302c91871ca5c6b34dac07c01ef03f6f2e0017ee/libs/sql-parser/src/main/antlr/io/crate/sql/parser/antlr/SqlBaseParser.g4#L26

[surister]

cratedb-sqlparse/cratedb_sqlparse_py/cratedb_sqlparse/parser.py

Line 110 in cfa11be

statements = list(filter(lambda children: isinstance(children, SqlBaseParser.StatementContext), tree.children))

This only seems to happen when second+ statement is wrong.

For:

 -- This is a comment
SELECT A, B, C, D FROM tbl1;
-- This is a 2nd comment
SELECT 1; -- This is a 3rd comment
SELECT D, A FROM tbl1 WHERE;

tree.children is [[244], ;, [246], ;, [246], ;, <EOF>] where they are DefaultContext

For:

 -- This is a comment
SELECT A, B, C, D FROM tbl1;
-- This is a 2nd comment
SELEC 1; -- This is a 3rd comment
SELECT D, A FROM tbl1 WHERE;

tree.children is [[244], ;, SELEC, 1, ;, SELECT, D, ,, A, FROM, tbl1, WHERE, ;] where only the first one is DefaultContext and the rest are ErrorNodeImpl

[surister]

sqlparse(
"""
    ELECT 1;
    Begin 2;
    SELECT 2;
"""
)

Generates:

    query=e.ctx.parser.getTokenStream().getText(e.ctx.start, e.offendingToken.tokenIndex),
          ^^^^^
AttributeError: 'NoneType' object has no attribute 'ctx'

It triggers an error. ExceptionCollectorListener.syntaxError is called but antlr4 passes a None as error e