Skip to content

Load JSON using pydantic directly in HTTP client#1122

Open
Fogapod wants to merge 1 commit intoqdrant:devfrom
Fogapod:ee/pydantic-load-json
Open

Load JSON using pydantic directly in HTTP client#1122
Fogapod wants to merge 1 commit intoqdrant:devfrom
Fogapod:ee/pydantic-load-json

Conversation

@Fogapod
Copy link

@Fogapod Fogapod commented Nov 20, 2025

Partially addresses #714

There were 2 attempts at this: #756 and #902 but they add extra dependency.
Pydantic json parsing is pretty fast. They claim being faster than orjson. Even if they are not there is a benefit of not crossing pyo3 bridge twice.

I used TypeAdapter instead of hoops with obj property although I'm not sure why this extra layer is needed at all. I checked a few types this is called with and they are all BaseModel already so you could just model_validate_json on them. If there are some non-pydantic types i guess something like this could be done:

    if issubclass(type_, BaseModel):
        return type_

My local pyright shows 8738 errors, 0 warnings, 0 informations not sure what to think of that. I did run poetry install

All Submissions:

  • Contributions should target the dev branch. Did you create your branch from dev?
  • Have you followed the guidelines in our Contributing document?
  • Have you checked to ensure there aren't other open Pull Requests for the same update/change?

New Feature Submissions:

  1. Does your submission pass tests?
  2. Have you installed pre-commit with pip3 install pre-commit and set up hooks with pre-commit install?

Changes to Core Features:

  • Have you added an explanation of what your changes do and why you'd like us to include them?
  • Have you written new tests for your core changes, as applicable?
  • Have you successfully ran tests with your changes locally?

@netlify
Copy link

netlify bot commented Nov 20, 2025

Deploy Preview for poetic-froyo-8baba7 ready!

Name Link
🔨 Latest commit 89cb79c
🔍 Latest deploy log https://app.netlify.com/projects/poetic-froyo-8baba7/deploys/691f0509edfa360008fe2235
😎 Deploy Preview https://deploy-preview-1122--poetic-froyo-8baba7.netlify.app
📱 Preview on mobile
Toggle QR Code...

QR Code

Use your smartphone camera to open QR code link.

To edit notification comments on pull requests, go to your Netlify project configuration.

@coderabbitai
Copy link

coderabbitai bot commented Nov 20, 2025

📝 Walkthrough

Walkthrough

The changes refactor the response parsing mechanism in the API client by replacing dynamic Pydantic model creation with TypeAdapter-based validation. The implementation shifts response deserialization from response.json() to response.content (bytes), introduces a new _wrap_in_pydantic_model function that returns a TypeAdapter, updates the parse_as_type function signature to accept bytes directly, and removes the previous _get_parsing_type function and dynamic create_model dependency. Both synchronous and asynchronous code paths are aligned to use the new parsing flow.

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~20–25 minutes

  • TypeAdapter integration: Verify that TypeAdapter-based validation produces equivalent behavior to the previous dynamic model creation approach, including error handling and edge cases.
  • Bytes vs. object handling: Ensure the shift from response.json() to response.content correctly handles the raw bytes data and doesn't introduce serialization/encoding issues.
  • Sync and async consistency: Confirm both code paths are symmetrically updated and maintain the same validation semantics.
  • ValidationError handling: Check that error propagation and exception handling remain appropriate after the refactoring.

Pre-merge checks and finishing touches

❌ Failed checks (1 warning)
Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 0.00% which is insufficient. The required threshold is 80.00%. You can run @coderabbitai generate docstrings to improve docstring coverage.
✅ Passed checks (2 passed)
Check name Status Explanation
Title check ✅ Passed The title directly describes the main change: replacing JSON parsing approach in the HTTP client to use pydantic's TypeAdapter instead of previous methods.
Description check ✅ Passed The PR description directly addresses the changeset by explaining the motivation (issue #714), the rationale for using pydantic's TypeAdapter instead of orjson, and implementation details of the JSON parsing refactor.
✨ Finishing touches
  • 📝 Generate docstrings
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Post copyable unit tests in a comment

Tip

📝 Customizable high-level summaries are now available in beta!

You can now customize how CodeRabbit generates the high-level summary in your pull requests — including its content, structure, tone, and formatting.

  • Provide your own instructions using the high_level_summary_instructions setting.
  • Format the summary however you like (bullet lists, tables, multi-section layouts, contributor stats, etc.).
  • Use high_level_summary_in_walkthrough to move the summary from the description to the walkthrough section.

Example instruction:

"Divide the high-level summary into five sections:

  1. 📝 Description — Summarize the main change in 50–60 words, explaining what was done.
  2. 📓 References — List relevant issues, discussions, documentation, or related PRs.
  3. 📦 Dependencies & Requirements — Mention any new/updated dependencies, environment variable changes, or configuration updates.
  4. 📊 Contributor Summary — Include a Markdown table showing contributions:
    | Contributor | Lines Added | Lines Removed | Files Changed |
  5. ✔️ Additional Notes — Add any extra reviewer context.
    Keep each section concise (under 200 words) and use bullet or numbered lists for clarity."

Note: This feature is currently in beta for Pro-tier users, and pricing will be announced later.


Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

Copy link

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 0

Caution

Some comments are outside the diff and can’t be posted inline due to platform limitations.

⚠️ Outside diff range comments (2)
qdrant_client/http/api_client.py (2)

111-130: Missing handler for type_=None case.

The method signature allows type_ to be None (see overloads at lines 73-79), but line 127 unconditionally calls parse_as_type(response.content, type_). When type_ is None, this would attempt to create TypeAdapter(None), which is invalid.

Apply this diff to handle the None case:

     if response.status_code in [200, 201, 202]:
+        if type_ is None:
+            return None
         try:
             return parse_as_type(response.content, type_)
         except ValidationError as e:

200-219: Missing handler for type_=None case in async path.

Same issue as the synchronous send method: line 216 unconditionally calls parse_as_type even when type_ is None (see overloads at lines 160-169).

Apply this diff to handle the None case:

     if response.status_code in [200, 201, 202]:
+        if type_ is None:
+            return None
         try:
             return parse_as_type(response.content, type_)
         except ValidationError as e:
🧹 Nitpick comments (1)
qdrant_client/http/api_client.py (1)

253-255: Consider optimization for BaseModel types.

The caching approach with lru_cache is good for performance. However, as noted in the PR description, most target types are already BaseModel. For these cases, you could potentially optimize further by using model_validate_json directly, which may be faster than going through TypeAdapter.

Consider this optimization:

@lru_cache(maxsize=None)
def _wrap_in_pydantic_model(type_: T) -> TypeAdapter[T]:
    return TypeAdapter(type_)


def parse_as_type(data: bytes, type_: Type[T]) -> T:
    # Optimize for BaseModel subclasses
    try:
        if isinstance(type_, type) and issubclass(type_, BaseModel):
            return type_.model_validate_json(data)
    except TypeError:
        pass
    return _wrap_in_pydantic_model(type_).validate_json(data)

You would need to import BaseModel:

from pydantic import BaseModel, TypeAdapter, ValidationError
📜 Review details

Configuration used: CodeRabbit UI

Review profile: CHILL

Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between 9ce7346 and 89cb79c.

📒 Files selected for processing (1)
  • qdrant_client/http/api_client.py (4 hunks)
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (3)
  • GitHub Check: Redirect rules - poetic-froyo-8baba7
  • GitHub Check: Header rules - poetic-froyo-8baba7
  • GitHub Check: Pages changed - poetic-froyo-8baba7
🔇 Additional comments (2)
qdrant_client/http/api_client.py (2)

258-259: No issues found - TypeAdapter implementation is correct.

Pydantic's TypeAdapter.validate_json() is the recommended approach for validating arbitrary types in pydantic v2. The code properly handles all edge cases mentioned:

  • Union types: Fully supported (the codebase contains 75+ Union types including Union[datetime, date] and Union[List[X], X])
  • Optional types: Fully supported (extensively used in response models)
  • Custom types: Fully supported (all Pydantic BaseModel subclasses are handled correctly)

The change from response.json() with dynamic create_model() wrapper to response.content with direct TypeAdapter.validate_json() is a correct performance optimization using pydantic v2's intended pattern.


7-7: The original review comment is incorrect.

The dependency constraint pydantic = ">=1.10.8,!=2.0.*,!=2.1.*,!=2.2.0" permits pydantic 2.2.1 and above. The poetry.lock file confirms pydantic 2.12.4 is currently in use, and TypeAdapter is available in this version. The import is valid and compatible.

Likely an incorrect or invalid review comment.

@Fogapod
Copy link
Author

Fogapod commented Nov 21, 2025

Looks like TypeAdapter was added in pydantic 2 so this won't work if v1 compatibility is still needed

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant