Move TypeHint into a Python expression AST

[Tpt]

Implement a subset of Python expr AST

Adds to the "name" construct a kind to distinguish global (global name that does not need to be resolved against the current module) vs local (names that needs to be resolved).
This consolidates the previous builtin/local/module construct by taking into account that modules can be relative (for example if in a #[pyclass] the user set module= to the submodule name)

Adds also the support for Callable[[int], float] and a beginning of constructs to represent constants (currently only None but will be useful for typing.Literal support)

The interesting part of this PR is the inspect/mod.rs file

[davidhewitt]

Thanks for working on this.

My initial reaction reading the code is that PyStaticExpr is the right data format but the API for for type hints specifically is less nice, e.g. compare the union constructor we had before with the bit_or constructor we have now.

Due to the limitations of const fn (no allocations etc) I think the only way to get the ergonomics back would be a macro in that case, e.g. type_hint_union!(&a, &b, &c) instead of PyStaticExpr(&a, &PyStaticExpr::bit_or(&b, &c)).

Maybe for others like TypeHint::module_attr it might be able to have a normal const fn if we wanted to keep that API.

I'm also not sure if we should keep struct TypeHint(PyStaticExpr) as a way to have a namespace for type hint constructors specifically.

Wonder what you think of those kinds of points?

[davidhewitt]

String(&'static str) would also be interesting because it can be used as an escape hatch for forward-reference style annotations for things we can't parse.

e.g. we could have "tuple[()]" already as a type hint until we can support it natively (with a PyStaticCall ast node, I guess).

[Tpt]

yes for String(&'static str)! I am planning to do it in a follow up if it's fine with you (to get it done properly, it needs a JSON escaping implementation working in const...).

For tuple[()] it's possible with this MR with the ::Tuple(&[]) variant.

[Tpt]

Thank you!

Due to the limitations of const fn (no allocations etc) I think the only way to get the ergonomics back would be a macro in that case, e.g. type_hint_union!(&a, &b, &c) instead of PyStaticExpr(&a, &PyStaticExpr::bit_or(&b, &c)).

Yes! It might make sense to have them for usual operations (union and subscript). Will add them.

I'm also not sure if we should keep struct TypeHint(PyStaticExpr) as a way to have a namespace for type hint constructors specifically.

If we had to end up with macros anyway, then the "namespace for constructors" usecase seems less interesting to me. What I can do is experiment a bit with macros and see what it looks like?

[davidhewitt]

👍 sounds good to me, thanks!

[Tpt]

@davidhewitt Experiment done with macros. I have replaced the const constructor with them. Does it looks good to you? If yes, I will move toward the next steps (same kind of changes in macro and introspection code)

[davidhewitt]

The macro constructors seem to work nicely to me 👍

[davidhewitt]

This seems justifiable to extract into a separate PR.

[Tpt]

The macro constructors seem to work nicely to me 👍

Amazing! Will finish this MR then.

[Tpt]

I don't think we are fully there yet in term of imports generation in the stubs: if the module argument of #[pyclass] is not absolute we might still generate wrong imports. A better approach might be to include in the PyTypeInfo::TYPE_HINT of classes generated by #[pyclass] the introspection identifier: this way the stub generator can build proper import paths for all of them with full knowledge of the module layout. We can assume all other type hints from traits to be absolute paths. I plan to do it in a follow up.

Then for custom type hints, I guess the best approach is to ask the user to write imports from them in the #[pymodule] macro. This way the imports generation algorithm is:

1. For type hints from #[pyclass] generate proper module path from the library layout and edit the type hint
2. Start from import from #[pymodule]
3. For all type hints that are not from the user, generate an import for them if they do not conflict with existing imports and rewrite the type hint to leverage these imports.

This way we should hopefully get something correct. And PyStaticNameKind::Global means "in the python intrepreter root (either a module or a built-in)" and PyStaticNameKind::Local means "relative to importswritten #[pymodule]". The third case "with module path written by the user but maybe relative or wrong" does not exist anymore.