There may not be an existing SQLite parser we can use from Go but for simple queries we can use a PostgreSQL parser, see here for a good one and example of use.
The way this would work is that it would attempt to parse the query. If it can parse the query and the query consists of only syntax that we support, return all fields in the query. Then we pass this list of fields to the SQLiteWriter. If this list is set in the SQLiteWriter then when we write fields to SQLite we only write the ones in this list.
For a first pass I'd suggest supporting:
SELECT x FROM {} WHERE y = 1 where this returns ['x', 'y']
Additional ones that won't be too bad:
SELECT COUNT(x) FROM {} WHERE y = 2 returns ['x', 'y']SELECT x FROM {} GROUP BY z returns ['x', 'z']
Harder but reasonable examples:
SELECT a.x FROM {0} a JOIN {1} b ON a.id = b.json_id returns {'a': ['x', 'id'], 'b': ['json_id']}
Examples this must fail on (this is not a comprehensive list):
SELECT x, * FROM {} (because of the star operatorSELECT x FROM {0} JOIN {1} ON id=json_id (ambiguous where x, id, and json_id come from; also requires supporting different columns for different tables)
There may not be an existing SQLite parser we can use from Go but for simple queries we can use a PostgreSQL parser, see here for a good one and example of use.
The way this would work is that it would attempt to parse the query. If it can parse the query and the query consists of only syntax that we support, return all fields in the query. Then we pass this list of fields to the SQLiteWriter. If this list is set in the SQLiteWriter then when we write fields to SQLite we only write the ones in this list.
For a first pass I'd suggest supporting:
SELECT x FROM {} WHERE y = 1where this returns['x', 'y']Additional ones that won't be too bad:
SELECT COUNT(x) FROM {} WHERE y = 2returns['x', 'y']SELECT x FROM {} GROUP BY zreturns['x', 'z']Harder but reasonable examples:
SELECT a.x FROM {0} a JOIN {1} b ON a.id = b.json_idreturns{'a': ['x', 'id'], 'b': ['json_id']}Examples this must fail on (this is not a comprehensive list):
SELECT x, * FROM {}(because of the star operatorSELECT x FROM {0} JOIN {1} ON id=json_id(ambiguous where x, id, and json_id come from; also requires supporting different columns for different tables)