Add builtin varchar array codec#1321
Open
anton-oai wants to merge 1 commit into
Open
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Add a builtin binary codec for PostgreSQL
varchar[]/pg_catalog._varchar.asyncpgalready has builtin support for scalarvarcharand fortext[], but not forvarchar[]. As a result, queries that return or bindvarchar[]require asyncpg to run its recursive typeinfo introspection query on each new connection.This change registers
varchar[]as a builtin array codec. It reuses text element encode/decode behavior, while encoding arrays withVARCHAROIDas the element OID so boundvarchar[]parameters match PostgreSQL's expected array element type.Motivation
Without an explicit builtin codec for
varchar[], asyncpg runs its typeinfo introspection query for every new connection that sees this type.In isolation, the cost of one introspection query is low in our measurements, around ~200us when a single query is executed. However, during reconnect storm events with thousands of concurrent reconnects, this introspection query becomes a major CPU hit and can materially degrade overall database performance.
For schemas with many
varchar[]columns, especially on large tables where changing column types is not desirable, supporting this native PostgreSQL type out of the box is more practical than requiring applications to avoidvarchar[]or add query-level casts.Test Plan
Added
test_varchar_array_does_not_introspect, which uses a custom connection class that counts_introspect_types()calls and verifies that both parameter and resultvarchar[]paths work without triggering type introspection.Locally run:
PYTHONPATH=. .venv/bin/python -m pytest tests/test_introspection.py -k varchar_array_does_not_introspect -q