Handle overload generic callable#2472
Closed
fangyi-zhou wants to merge 9 commits intofacebook:mainfrom
Closed
Conversation
This comment has been minimized.
This comment has been minimized.
1 similar comment
This comment has been minimized.
This comment has been minimized.
When an overloaded function is passed through a generic function like `copy[A, B](c: Callable[[A], B]) -> Callable[[A], B]`, the result should preserve the overloaded type. Currently pyrefly arbitrarily resolves to the first overload, causing false errors. Tracks facebook#2470 Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
When a generic function like `copy[A, B](c: Callable[[A], B]) -> Callable[[A], B]` is called with an overloaded argument, apply the call once per overload signature and return an overloaded result. This preserves the full overloaded type rather than collapsing to whichever signature the type solver happens to pick first. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Also apply the per-signature expansion when a class with an overloaded constructor is passed to a generic Callable parameter. For example, `accepts_callable(Class7)` where `Class7.__init__` is overloaded now correctly returns `Overload[(x: int) -> Class7[int], (x: str) -> Class7[str]]` instead of collapsing to the first overload. Fixes the conformance test at constructors_callable.py:167. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- Extend call_generic_with_overloaded_arg to scan keyword args (e.g. copy_kw(c=foo)) in addition to positional args, so overloaded callables passed by keyword are also preserved through generic calls. - Introduce a local OverloadSource enum (Arg/Keyword) to track which argument position held the overload, used in the expansion loop to substitute into modified_args or modified_kws accordingly. - Extract the overload-extraction logic into a shared find_overload_in_type closure to avoid repeating the Type::Overload / Type::ClassDef match. - Update docstring to document positional + keyword scanning, state the "first overloaded arg only" limitation with rationale (cartesian product would be required for multiple overloaded args), and remove stale "Returns None if expansion is not applicable" phrasing. - Add pre-evaluation comment at call site explaining why args are always typed before entering the generic-callable overload path. - Add unit tests: test_overload_constructor_through_generic_callable and test_overload_through_generic_callable_keyword. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Three tests needed updates after the rebase:
- test_constructor_callable_conversion: Class7's overloaded constructor
through a generic Callable is now correctly handled by the
overload-through-generic fix, so remove the two stale # E: markers
that documented the previously incorrect behavior.
- test_context_lambda_paramspec: The lambda parameter `x` is now typed
as Unknown (rather than int) when contextual typing fails due to a
parameter name mismatch (z vs y). Update the expected error text.
- test_typed_dict_hint_in_typevar_bound: is_subset_anonymous_typed_dict
now correctly recognizes {"x": 0} as assignable to TypedDict bound TD,
so remove the stale # E: marker.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
… callables
The previous `if tparams.is_some()` guard was too broad — it fired for
every generic function call, causing two problems:
1. Unnecessary pre-evaluation overhead for non-callable-param generics
2. Lambda contextual-typing regression: lambdas were eagerly evaluated
without a hint, so ParamSpec-inferred params came back as `Unknown`
Two targeted fixes:
- Narrow the guard to `tparams.is_some() && has_callable_param`, where
`has_callable_param` is true only when the function has a
`Callable`-typed param or a `ParamSpec` params list. This skips
pre-evaluation for `identity[T]`, `max[T]`, etc.
- Add `Expr::Lambda` to the keep-as-Expr list in `CallWithTypes`, so
lambdas are never eagerly pre-evaluated even when the guard fires.
Lambdas can never be overloaded, so this is always safe.
Also mark `test_typed_dict_hint_in_typevar_bound` with a bug marker: the
test was accidentally passing due to the broad pre-evaluation guard, and
the narrowing reverts that side effect. The underlying issue (dict literal
not contextually typed as a TypedDict inside a tuple against a bounded
typevar) is a separate bug.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
fangyi-zhou
force-pushed
the
fix-overload-generic-callable
branch
from
ed0c968 to
7f4b261
Compare
|
Diff from mypy_primer, showing the effect of this PR on open source code: Expression (https://github.com/cognitedata/Expression)
- ERROR tests/test_array.py:47:31-51: No matching overload found for function `expression.core.pipe.pipe` called with arguments: (TypedArray[int], (TypedArray[object]) -> TypedArray[str]) [no-matching-overload]
+ ERROR tests/test_array.py:47:31-51: No matching overload found for function `expression.core.pipe.pipe` called with arguments: (TypedArray[int], Overload[
+ (TypedArray[object]) -> TypedArray[str]
+ (TypedArray[Buffer]) -> TypedArray[str]
+ ]) [no-matching-overload]
zulip (https://github.com/zulip/zulip)
- ERROR zerver/lib/display_recipient.py:82:40-84:10: No matching overload found for function `list.__init__` called with arguments: (QuerySet[UserProfile, dict[str, Any]]) [no-matching-overload]
+ ERROR zerver/lib/display_recipient.py:82:24-84:10: Argument `(ids: Unknown) -> list[dict[str, Any]]` is not assignable to parameter `query_function` with type `(list[int]) -> Iterable[UserDisplayRecipient]` in function `zerver.lib.cache.bulk_cached_fetch` [bad-argument-type]
pydantic (https://github.com/pydantic/pydantic)
- ERROR pydantic/_internal/_mock_val_ser.py:137:25-142:6: `MockValSer[PluggableSchemaValidator]` is not assignable to attribute `validator` with type `PluggableSchemaValidator | SchemaValidator` [bad-assignment]
+ ERROR pydantic/_internal/_mock_val_ser.py:137:25-142:6: `MockValSer[@_]` is not assignable to attribute `validator` with type `PluggableSchemaValidator | SchemaValidator` [bad-assignment]
- ERROR pydantic/_internal/_mock_val_ser.py:141:44-67: Argument `(ta: TypeAdapter[Unknown]) -> PluggableSchemaValidator | SchemaValidator` is not assignable to parameter `attr_fn` with type `(TypeAdapter[Unknown]) -> PluggableSchemaValidator | None` in function `attempt_rebuild_fn` [bad-argument-type]
- ERROR pydantic/_internal/_mock_val_ser.py:143:26-148:6: `MockValSer[SchemaSerializer]` is not assignable to attribute `serializer` with type `SchemaSerializer` [bad-assignment]
+ ERROR pydantic/_internal/_mock_val_ser.py:143:26-148:6: `MockValSer[@_]` is not assignable to attribute `serializer` with type `SchemaSerializer` [bad-assignment]
+ ERROR pydantic/_internal/_mock_val_ser.py:176:34-181:6: `MockValSer[@_]` is not assignable to attribute `__pydantic_validator__` with type `PluggableSchemaValidator | SchemaValidator` [bad-assignment]
- ERROR pydantic/_internal/_mock_val_ser.py:176:34-181:6: `MockValSer[PluggableSchemaValidator]` is not assignable to attribute `__pydantic_validator__` with type `PluggableSchemaValidator | SchemaValidator` [bad-assignment]
- ERROR pydantic/_internal/_mock_val_ser.py:180:44-78: Argument `(c: type[BaseModel]) -> PluggableSchemaValidator | SchemaValidator` is not assignable to parameter `attr_fn` with type `(type[BaseModel]) -> PluggableSchemaValidator | None` in function `attempt_rebuild_fn` [bad-argument-type]
- ERROR pydantic/_internal/_mock_val_ser.py:182:35-187:6: `MockValSer[SchemaSerializer]` is not assignable to attribute `__pydantic_serializer__` with type `SchemaSerializer` [bad-assignment]
+ ERROR pydantic/_internal/_mock_val_ser.py:182:35-187:6: `MockValSer[@_]` is not assignable to attribute `__pydantic_serializer__` with type `SchemaSerializer` [bad-assignment]
+ ERROR pydantic/_internal/_mock_val_ser.py:217:34-222:6: `MockValSer[@_]` is not assignable to attribute `__pydantic_validator__` with type `PluggableSchemaValidator | SchemaValidator` [bad-assignment]
- ERROR pydantic/_internal/_mock_val_ser.py:217:34-222:6: `MockValSer[PluggableSchemaValidator]` is not assignable to attribute `__pydantic_validator__` with type `PluggableSchemaValidator | SchemaValidator` [bad-assignment]
- ERROR pydantic/_internal/_mock_val_ser.py:221:44-78: Argument `(c: type[PydanticDataclass]) -> PluggableSchemaValidator | SchemaValidator` is not assignable to parameter `attr_fn` with type `(type[PydanticDataclass]) -> PluggableSchemaValidator | None` in function `attempt_rebuild_fn` [bad-argument-type]
- ERROR pydantic/_internal/_mock_val_ser.py:223:35-228:6: `MockValSer[SchemaSerializer]` is not assignable to attribute `__pydantic_serializer__` with type `SchemaSerializer` [bad-assignment]
+ ERROR pydantic/_internal/_mock_val_ser.py:223:35-228:6: `MockValSer[@_]` is not assignable to attribute `__pydantic_serializer__` with type `SchemaSerializer` [bad-assignment]
meson (https://github.com/mesonbuild/meson)
- ERROR mesonbuild/backend/ninjabackend.py:1942:81-93: Object of class `NoneType` has no attribute `startswith` [missing-attribute]
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Fixes #2470
AI coded the solution for the fix, but I haven't had a look -- will mark as ready to review after I tidy things up.
Test Plan