⚡️ Speed up method PSBaseParser._parse_number by 47%

[codeflash-ai]

📄 47% (0.47x) speedup for PSBaseParser._parse_number in pdfminer/psparser.py

⏱️ Runtime : 488 microseconds → 332 microseconds (best of 250 runs)

📝 Explanation and details

The optimized code achieves a 46% speedup by eliminating Python overhead in two hot paths within _parse_number and _parse_float:

Key Optimizations:

1. Replaced contextlib.suppress(ValueError) with explicit try/except/else blocks

   • The contextlib.suppress context manager adds significant overhead (creating a context manager object, entering/exiting it) on every invocation
   • Direct try/except/else is Python's native control flow and much faster
   • Line profiler shows the with contextlib.suppress(ValueError): line consumed 28.6% of _parse_number runtime in the original (1.14ms), while the optimized try/except structure only takes 4.2% (126μs) for the try statement itself
   • The actual conversion and token addition happens in the else branch only on success, avoiding unnecessary stack unwinding

2. Avoided single-byte slice creation for dot comparison

   • Changed c = s[j : j + 1]; if c == b"." to if s[j] == 46 (where 46 is ord(b"."))
   • Indexing bytes directly returns an integer, avoiding the allocation of a one-byte bytes object
   • While individually small (~7% reduction in that specific check's time from 410ns to 366ns per hit), this adds up over 391 hits per test run

Why This Works:

• PDF parsing involves tokenizing thousands of numbers in typical documents. The parser calls _parse_number repeatedly in tight loops
• Context managers in Python have non-trivial overhead: they require __enter__/__exit__ method calls and exception handling setup even when no exception occurs
• The explicit try/except/else pattern lets CPython's bytecode optimizer handle the common "no exception" path more efficiently
• Single-byte slice allocation is wasteful when we can directly access the byte value as an integer

Performance Profile:

The annotated tests show consistent 28-51% speedups across diverse numeric inputs:

• Simple integers: 34-49% faster
• Large integers (150+ digits): 28-32% faster
• Sequential parsing of multiple numbers: 47-51% faster
• Edge cases (lone signs, invalid conversions): 28-44% faster

The optimization particularly shines in the "large scale" test with 300 sequential numbers: 51.8% faster (311μs → 205μs), demonstrating the cumulative benefit when _parse_number is called repeatedly—exactly the real-world PDF parsing scenario.

✅ Correctness verification report:

Test	Status
⚙️ Existing Unit Tests	🔘 None Found
🌀 Generated Regression Tests	✅ 438 Passed
⏪ Replay Tests	🔘 None Found
🔎 Concolic Coverage Tests	🔘 None Found
📊 Tests Coverage	100.0%

🌀 Click to see Generated Regression Tests

import io

import pytest  # used for our unit tests
from pdfminer.psparser import PSBaseParser

# NOTE:
# The tests below exercise the PSBaseParser._parse_number implementation
# exactly as provided in the codebase. They set up the parser internal
# state as the real tokenizer would (setting _curtoken and calling the
# internal parsing function with appropriate buffer and index).
#
# Each test is small, deterministic, and focuses on a single behavior
# of _parse_number: successful integer parsing, handling of lone sign
# characters, transition to float parsing when '.' is encountered,
# behavior when no END_NUMBER is found (incomplete buffer), and the
# suppression of a ValueError when the bytes do not represent an int.
#
# The "large scale" test constructs many numeric invocations (<= 1000)
# to ensure the parser appends many integer tokens correctly and
# efficiently.

# helper to create a fresh parser instance backed by a BytesIO
def _new_parser():
    # Create a parser around an in-memory byte stream; the stream isn't
    # read by these unit tests, but PSBaseParser.__init__ expects a
    # BinaryIO and calls seek(0), so BytesIO is appropriate.
    return PSBaseParser(io.BytesIO(b""))

def test_parse_integer_basic():
    # Basic integer parsing:
    # Simulate that the parser has already consumed the first digit '1'
    # (this is how the real tokenizer calls _parse_number) and that the
    # remaining buffer contains '23 ' starting at index 1.
    p = _new_parser()
    p._curtoken = b"1"  # initial fragment already read
    p._curtokenpos = 0  # position is recorded but not essential for value
    # Buffer: "123 " - call with i=1 so bytes[1:3] ('23') get appended.
    s = b"123 "
    codeflash_output = p._parse_number(s, 1); ret = codeflash_output # 2.62μs -> 1.91μs (37.2% faster)
    pos, token = p._tokens[0]

def test_parse_plus_sign_and_negative_sign_only_behavior():
    # When only a sign character is present in _curtoken and there are no
    # digits after it, int() will raise ValueError; the implementation
    # suppresses the ValueError and no token should be added.
    p = _new_parser()
    # Case: lone '-' sign followed by delimiter (space)
    p._curtoken = b"-"
    s = b"- "
    # i=1 means we are at the position after the '-' in the buffer.
    codeflash_output = p._parse_number(s, 1); ret = codeflash_output # 4.82μs -> 3.76μs (28.2% faster)

    # Case: plus sign followed by a valid number should succeed.
    p2 = _new_parser()
    p2._curtoken = b"+"
    s2 = b"+42 "  # i=1 -> remaining bytes '42' will be appended
    codeflash_output = p2._parse_number(s2, 1); ret2 = codeflash_output # 1.93μs -> 1.36μs (41.9% faster)

def test_parse_dot_transitions_to_float_parser():
    # If the character immediately following the parsed number fragment
    # is a dot ('.'), _parse_number should append '.' to the token and
    # set the next parsing function to _parse_float without adding an int token.
    p = _new_parser()
    p._curtoken = b"123"
    # Buffer where '.' is immediately after the digits: "123.45 "
    s = b"123.45 "
    # i=3 points to the '.' after '123'.
    codeflash_output = p._parse_number(s, 3); ret = codeflash_output # 1.67μs -> 1.68μs (0.595% slower)

def test_parse_incomplete_buffer_appends_and_returns_len():
    # If END_NUMBER.search doesn't find a delimiter (end of supplied buffer),
    # _parse_number should append the remainder of the buffer to _curtoken and
    # return len(s) so the caller knows more data is needed.
    p = _new_parser()
    p._curtoken = b"9"
    # Buffer with no delimiter at all - only digits
    s = b"99"  # Suppose we started at i=1: there's '9' remaining, but no delimiter
    codeflash_output = p._parse_number(s, 1); ret = codeflash_output # 1.01μs -> 980ns (3.06% faster)

def test_parse_scientific_not_added_as_int_but_no_exception():
    # Numbers like '1e3' are valid floats but not ints; _parse_number will
    # attempt an int conversion and suppress ValueError. No token should be
    # added by _parse_number in such a case; the parser state should still
    # recover to _parse_main.
    p = _new_parser()
    p._curtoken = b"1"
    s = b"1e3 "
    codeflash_output = p._parse_number(s, 1); ret = codeflash_output # 2.72μs -> 1.89μs (43.9% faster)

def test_large_scale_many_numbers():
    # Large-scale test: parse a sequence of many integers (well under 1000).
    # We simulate repeated calls to _parse_number as if the stream was tokenized
    # one token at a time. For each integer n we set _curtoken to the first
    # byte of the representation and provide the full representation followed
    # by a space. This mirrors how the tokenizer feeds fragments to _parse_number.
    count = 300  # comfortably under the 1000 element limit
    p = _new_parser()
    for n in range(count):
        s = (str(n) + " ").encode("ascii")
        # Simulate that the first byte was already consumed and stored in _curtoken
        p._curtoken = s[:1]  # first character of the number
        # Choose a unique position so tokens' stored positions are distinguishable
        p._curtokenpos = n
        # Call the parser as the tokenizer would: start parsing at index 1
        codeflash_output = p._parse_number(s, 1); ret = codeflash_output # 311μs -> 205μs (51.8% faster)
    # Verify tokens are the integers we expect, in order
    tokens = [t for pos, t in p._tokens]
    # Verify that the positions recorded are the ones we set
    positions = [pos for pos, t in p._tokens]
# codeflash_output is used to check that the output of the original code is the same as that of the optimized code.

import io

import pytest
from pdfminer.psparser import PSBaseParser

class TestPSBaseParserParseNumber:
    """Comprehensive test suite for PSBaseParser._parse_number method."""
    
    # ============================================================================
    # BASIC TEST CASES
    # ============================================================================
    
    def test_parse_simple_positive_integer(self):
        """Test parsing a simple positive integer."""
        fp = io.BytesIO(b"42 ")
        parser = PSBaseParser(fp)
        parser._curtoken = b"4"
        parser._parse_number(b"2 ", 0) # 3.51μs -> 2.41μs (45.6% faster)
    
    def test_parse_simple_negative_integer(self):
        """Test parsing a negative integer."""
        fp = io.BytesIO(b"-42 ")
        parser = PSBaseParser(fp)
        parser._curtoken = b"-4"
        parser._parse_number(b"2 ", 0) # 3.00μs -> 2.18μs (37.6% faster)
    
    def test_parse_positive_sign_integer(self):
        """Test parsing an integer with explicit positive sign."""
        fp = io.BytesIO(b"+42 ")
        parser = PSBaseParser(fp)
        parser._curtoken = b"+4"
        parser._parse_number(b"2 ", 0) # 2.67μs -> 1.99μs (34.2% faster)
    
    def test_parse_single_digit_zero(self):
        """Test parsing the number zero."""
        fp = io.BytesIO(b"0 ")
        parser = PSBaseParser(fp)
        parser._curtoken = b"0"
        parser._parse_number(b" ", 0) # 2.74μs -> 1.93μs (42.0% faster)
    
    def test_parse_single_digit_nine(self):
        """Test parsing a single digit number."""
        fp = io.BytesIO(b"9 ")
        parser = PSBaseParser(fp)
        parser._curtoken = b"9"
        parser._parse_number(b" ", 0) # 2.59μs -> 1.84μs (40.8% faster)
    
    def test_parse_number_followed_by_space(self):
        """Test that parsing stops at whitespace."""
        fp = io.BytesIO(b"123 ")
        parser = PSBaseParser(fp)
        parser._curtoken = b"12"
        codeflash_output = parser._parse_number(b"3 ", 0); result = codeflash_output # 2.69μs -> 1.87μs (43.9% faster)
    
    def test_parse_number_followed_by_newline(self):
        """Test that parsing stops at newline."""
        fp = io.BytesIO(b"456\n")
        parser = PSBaseParser(fp)
        parser._curtoken = b"45"
        parser._parse_number(b"6\n", 0) # 2.52μs -> 1.88μs (34.0% faster)
    
    def test_parse_number_followed_by_comma(self):
        """Test that parsing stops at punctuation."""
        fp = io.BytesIO(b"789,")
        parser = PSBaseParser(fp)
        parser._curtoken = b"78"
        parser._parse_number(b"9,", 0) # 2.49μs -> 1.82μs (36.8% faster)
    
    def test_parse_transition_to_float(self):
        """Test that decimal point triggers float parsing mode."""
        fp = io.BytesIO(b"42.5 ")
        parser = PSBaseParser(fp)
        parser._curtoken = b"42"
        codeflash_output = parser._parse_number(b".5 ", 0); result = codeflash_output # 1.80μs -> 1.70μs (5.88% faster)
    
    def test_parse_large_positive_integer(self):
        """Test parsing a large positive integer."""
        fp = io.BytesIO(b"999999999 ")
        parser = PSBaseParser(fp)
        parser._curtoken = b"99999999"
        parser._parse_number(b"9 ", 0) # 2.63μs -> 1.77μs (48.6% faster)
    
    def test_parse_large_negative_integer(self):
        """Test parsing a large negative integer."""
        fp = io.BytesIO(b"-999999999 ")
        parser = PSBaseParser(fp)
        parser._curtoken = b"-99999999"
        parser._parse_number(b"9 ", 0) # 2.54μs -> 1.84μs (38.0% faster)
    
    # ============================================================================
    # EDGE TEST CASES
    # ============================================================================
    
    def test_parse_number_at_end_of_buffer(self):
        """Test parsing when number reaches end of buffer without delimiter."""
        fp = io.BytesIO(b"123")
        parser = PSBaseParser(fp)
        parser._curtoken = b"12"
        codeflash_output = parser._parse_number(b"3", 0); result = codeflash_output # 1.09μs -> 1.10μs (0.909% slower)
    
    def test_parse_number_empty_continuation(self):
        """Test parsing with empty continuation string."""
        fp = io.BytesIO(b"42")
        parser = PSBaseParser(fp)
        parser._curtoken = b"42"
        codeflash_output = parser._parse_number(b"", 0); result = codeflash_output # 1.16μs -> 1.09μs (6.42% faster)
    
    def test_parse_single_minus_sign(self):
        """Test that a single minus sign is attempted as integer conversion."""
        fp = io.BytesIO(b"- ")
        parser = PSBaseParser(fp)
        parser._curtoken = b"-"
        parser._parse_number(b" ", 0) # 5.15μs -> 3.90μs (32.1% faster)
    
    def test_parse_single_plus_sign(self):
        """Test that a single plus sign is attempted as integer conversion."""
        fp = io.BytesIO(b"+ ")
        parser = PSBaseParser(fp)
        parser._curtoken = b"+"
        parser._parse_number(b" ", 0) # 4.01μs -> 2.96μs (35.5% faster)
    
    def test_parse_number_with_multiple_delimiters(self):
        """Test that parsing stops at first delimiter."""
        fp = io.BytesIO(b"42/abc")
        parser = PSBaseParser(fp)
        parser._curtoken = b"42"
        parser._parse_number(b"/abc", 0) # 2.67μs -> 1.93μs (38.3% faster)
    
    def test_parse_negative_zero(self):
        """Test parsing negative zero."""
        fp = io.BytesIO(b"-0 ")
        parser = PSBaseParser(fp)
        parser._curtoken = b"-0"
        parser._parse_number(b" ", 0) # 2.59μs -> 1.85μs (40.0% faster)
    
    def test_parse_number_with_leading_zeros(self):
        """Test parsing integer with leading zeros."""
        fp = io.BytesIO(b"00042 ")
        parser = PSBaseParser(fp)
        parser._curtoken = b"0004"
        parser._parse_number(b"2 ", 0) # 2.63μs -> 1.90μs (38.4% faster)
    
    def test_parse_number_decimal_point_at_start(self):
        """Test transition to float when encountering decimal point."""
        fp = io.BytesIO(b"0.5 ")
        parser = PSBaseParser(fp)
        parser._curtoken = b"0"
        codeflash_output = parser._parse_number(b".5 ", 0); result = codeflash_output # 1.64μs -> 1.65μs (0.606% slower)
    
    def test_parse_number_start_position_nonzero(self):
        """Test parsing with non-zero starting position in buffer."""
        fp = io.BytesIO(b"xx123 ")
        parser = PSBaseParser(fp)
        parser._curtoken = b"12"
        codeflash_output = parser._parse_number(b"xx123 ", 2); result = codeflash_output # 2.71μs -> 2.00μs (35.5% faster)
    
    def test_parse_number_with_parenthesis_delimiter(self):
        """Test that parenthesis acts as delimiter."""
        fp = io.BytesIO(b"42(")
        parser = PSBaseParser(fp)
        parser._curtoken = b"42"
        parser._parse_number(b"(", 0) # 2.51μs -> 1.74μs (44.3% faster)
    
    def test_parse_number_with_angle_bracket_delimiter(self):
        """Test that angle bracket acts as delimiter."""
        fp = io.BytesIO(b"42<")
        parser = PSBaseParser(fp)
        parser._curtoken = b"42"
        parser._parse_number(b"<", 0) # 2.47μs -> 1.75μs (41.1% faster)
    
    def test_parse_number_returns_correct_position(self):
        """Test that function returns correct position after parsing."""
        fp = io.BytesIO(b"42 rest")
        parser = PSBaseParser(fp)
        parser._curtoken = b"42"
        codeflash_output = parser._parse_number(b" rest", 0); result = codeflash_output # 2.48μs -> 1.75μs (41.6% faster)
    
    def test_parse_number_with_tab_delimiter(self):
        """Test that tab character acts as delimiter."""
        fp = io.BytesIO(b"99\t")
        parser = PSBaseParser(fp)
        parser._curtoken = b"99"
        parser._parse_number(b"\t", 0) # 2.43μs -> 1.70μs (42.9% faster)
    
    def test_parse_number_with_carriage_return_delimiter(self):
        """Test that carriage return acts as delimiter."""
        fp = io.BytesIO(b"88\r")
        parser = PSBaseParser(fp)
        parser._curtoken = b"88"
        parser._parse_number(b"\r", 0) # 2.49μs -> 1.72μs (44.8% faster)
    
    def test_parse_number_invalid_conversion_suppressed(self):
        """Test that ValueError from invalid int conversion is suppressed."""
        fp = io.BytesIO(b"not_a_number ")
        parser = PSBaseParser(fp)
        parser._curtoken = b"not_a_numbe"
        parser._parse_number(b"r ", 0) # 4.94μs -> 3.71μs (33.2% faster)
    
    def test_parse_number_mode_reset_after_parsing(self):
        """Test that parse mode is reset to _parse_main after parsing."""
        fp = io.BytesIO(b"42 ")
        parser = PSBaseParser(fp)
        parser._curtoken = b"42"
        parser._parse_number(b" ", 0) # 2.45μs -> 1.81μs (35.4% faster)
    
    # ============================================================================
    # LARGE SCALE TEST CASES
    # ============================================================================
    
    def test_parse_very_large_integer(self):
        """Test parsing a very large integer (100+ digits)."""
        large_num_str = "9" * 150
        large_num_bytes = large_num_str.encode() + b" "
        fp = io.BytesIO(large_num_bytes)
        parser = PSBaseParser(fp)
        parser._curtoken = large_num_str[:-1].encode()
        parser._parse_number(large_num_str[-1:].encode() + b" ", 0) # 3.06μs -> 2.39μs (28.0% faster)
    
    def test_parse_very_large_negative_integer(self):
        """Test parsing a very large negative integer."""
        large_num_str = "-" + "9" * 150
        large_num_bytes = large_num_str.encode() + b" "
        fp = io.BytesIO(large_num_bytes)
        parser = PSBaseParser(fp)
        parser._curtoken = large_num_str[:-1].encode()
        parser._parse_number(large_num_str[-1:].encode() + b" ", 0) # 2.95μs -> 2.23μs (32.3% faster)
    
    def test_parse_multiple_sequential_numbers(self):
        """Test parsing multiple numbers in sequence."""
        fp = io.BytesIO(b"1 2 3 4 5")
        parser = PSBaseParser(fp)
        # Simulate parsing numbers
        for num_val in [1, 2, 3, 4, 5]:
            parser._curtoken = str(num_val).encode()
            parser._parse_number(b" ", 0) # 7.91μs -> 5.39μs (46.8% faster)
        for i, num_val in enumerate([1, 2, 3, 4, 5]):
            pass
    
    def test_parse_number_with_long_continuation(self):
        """Test parsing with very long continuation string."""
        long_continuation = b"0" * 500 + b" "
        fp = io.BytesIO(long_continuation)
        parser = PSBaseParser(fp)
        parser._curtoken = b"1"
        parser._parse_number(long_continuation, 0) # 5.89μs -> 5.13μs (14.8% faster)
    
    def test_parse_range_of_integers(self):
        """Test parsing a range of different integer values."""
        fp = io.BytesIO(b"")
        parser = PSBaseParser(fp)
        # Test a range of values from -500 to 500
        for num in range(-500, 501, 50):
            parser._tokens = []
            parser._curtoken = str(num).encode()
            parser._parse_number(b" ", 0) # 24.7μs -> 16.6μs (49.1% faster)
    
    def test_parse_boundary_integers(self):
        """Test parsing integers at common boundary values."""
        fp = io.BytesIO(b"")
        parser = PSBaseParser(fp)
        # Test powers of 2 and related values
        boundary_values = [
            2**10,      # 1024
            2**16,      # 65536
            2**31 - 1,  # Max 32-bit signed int
            -(2**31),   # Min 32-bit signed int
            10**9,      # 1 billion
            10**18,     # 1 quintillion
        ]
        for num in boundary_values:
            parser._tokens = []
            parser._curtoken = str(num).encode()
            parser._parse_number(b" ", 0) # 8.64μs -> 5.87μs (47.2% faster)
    
    def test_parse_number_buffer_transition(self):
        """Test number parsing across multiple buffer fills."""
        # This tests the case where curtoken accumulates data
        fp = io.BytesIO(b"123456789 ")
        parser = PSBaseParser(fp)
        # Simulate accumulation as if buffer was filled multiple times
        parser._curtoken = b"123456"
        codeflash_output = parser._parse_number(b"789 ", 0); result = codeflash_output # 2.66μs -> 1.91μs (39.3% faster)
    
    def test_parse_all_single_digits(self):
        """Test parsing all single digit numbers (0-9)."""
        fp = io.BytesIO(b"")
        parser = PSBaseParser(fp)
        for digit in range(10):
            parser._tokens = []
            parser._curtoken = str(digit).encode()
            parser._parse_number(b" ", 0) # 12.8μs -> 8.44μs (51.2% faster)
    
    def test_parse_all_negative_single_digits(self):
        """Test parsing all negative single digit numbers (-1 to -9)."""
        fp = io.BytesIO(b"")
        parser = PSBaseParser(fp)
        for digit in range(1, 10):
            parser._tokens = []
            parser._curtoken = b"-" + str(digit).encode()
            parser._parse_number(b" ", 0) # 11.6μs -> 7.67μs (50.7% faster)
    
    def test_parse_number_consistency_across_formats(self):
        """Test that same number in different formats parses to same value."""
        fp = io.BytesIO(b"")
        parser = PSBaseParser(fp)
        test_cases = [
            (b"42", 42),
            (b"+42", 42),
            (b"042", 42),
            (b"+042", 42),
            (b"-42", -42),
            (b"-042", -42),
        ]
        for token, expected in test_cases:
            parser._tokens = []
            parser._curtoken = token
            parser._parse_number(b" ", 0) # 8.55μs -> 5.95μs (43.7% faster)
# codeflash_output is used to check that the output of the original code is the same as that of the optimized code.

To edit these changes git checkout codeflash/optimize-PSBaseParser._parse_number-mkqzioxm and push.