Add native parameter types to Token constructor

[MauricioFauth]

Ensures that token are always a string and that flags are always an integer.

Using an empty string for the last delimiter has the same effect as using null.

• Related to Add native param types #521

[codecov]

Codecov Report

All modified and coverable lines are covered by tests ✅

Comparison is base (94f2db2) 96.45% compared to head (16389b2) 96.45%.

Additional details and impacted files

@@            Coverage Diff            @@
##             master     #526   +/-   ##
=========================================
  Coverage     96.45%   96.45%           
  Complexity     2152     2152           
=========================================
  Files            64       64           
  Lines          5019     5019           
=========================================
  Hits           4841     4841           
  Misses          178      178           

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[MauricioFauth]

@kamil-tekiela What do you think?

[kamil-tekiela]

On line 256 in Lexer you still assign null to $delimiter property which is then used on line 1013. Isn't this a problem still?

[kamil-tekiela]

I hope that this ok, but I feel like the phpdoc should be updated.

[MauricioFauth]

Token::$value type should definitely be updated. That value is only created by Lexer line 285.

sql-parser/src/Lexer.php

Lines 284 to 285 in 94f2db2

	// Adding a final delimiter to mark the ending.
	$list->tokens[$list->count++] = new Token(null, TokenType::Delimiter);

[MauricioFauth]

On line 256 in Lexer you still assign null to $delimiter property which is then used on line 1013. Isn't this a problem still?

Lexer::$delimiter should only be string. But this is not an issue because it gets coerced to string right after.