feat: Add prqlc lex command to the CLI

[max-sixty]

It's not perfect, but I think good enough for now + clearly indicated where lacking

Useful for #4397

[aljazerzen]

Great!

There is a few other stages within the resolver that I could also expose. But they not really useful to the user of the compiler but would be of a bit of help for debugging. I was thinking about setting up a general framework that would take a normal compilation flow and have "probes" injected at certain points. These "probes" would take the current representation of the query and convert it to a string that could be piped out into log file/terminal.

Currently my workflow is adding a dbg!() when I need it or even a permanent log::debug!("ast here {expr:#?}"), this is not ideal and does mean more re-compiling just because I've added a debug stmt.

---

Another thing I've been thinking about is, what prqlc interfaces do we really want to expose? Because we expose the first AST, PL and RQ trough library functions and CLI, but we change them constantly so I couldn't recommend them to be depended upon. For this reason I've added the debug subcommand to the cli, but I feel more and more that most of the cli commands should be under debug :D

[max-sixty]

There is a few other stages within the resolver that I could also expose. But they not really useful to the user of the compiler but would be of a bit of help for debugging. I was thinking about setting up a general framework that would take a normal compilation flow and have "probes" injected at certain points. These "probes" would take the current representation of the query and convert it to a string that could be piped out into log file/terminal.

I was thinking of a similar goal. I was more thinking about a "compiler dump" that would dump everything for a query and could be read in an easy way — maybe that's a markdown with <details> drop down.

(or maybe it's some cool HTML output that shows details about any token you hover over it, etc... IIRC there are examples from clang...)

In the meantime log::debug!("ast here {expr:#?}") seems pretty reasonable (or trace...)

---

More minor, but one other issue our API has as we increase the number of stages — if we want a function from any stage to any later stage, that's O(n^2) functions. So maybe we try and make this more abstract one day

---

For this reason I've added the debug subcommand to the cli, but I feel more and more that most of the cli commands should be under debug :D

Yes very reasonable!

(though I think it's also OK to just not have guarantees about compatibility with the AST for the moment...)