Skip to content

Enhance benchmark with comprehensive SET/GET/DELETE testing#79

Merged
bisho merged 1 commit into
mainfrom
03-11-improve_benchmark_to_test_set_get_and_delete_and_populate_values
Mar 26, 2026
Merged

Enhance benchmark with comprehensive SET/GET/DELETE testing#79
bisho merged 1 commit into
mainfrom
03-11-improve_benchmark_to_test_set_get_and_delete_and_populate_values

Conversation

@bisho
Copy link
Copy Markdown
Member

@bisho bisho commented Mar 11, 2026

Enhanced benchmark script with comprehensive testing and realistic data patterns

This PR significantly improves the benchmark script to provide more realistic and comprehensive performance testing:

What changed:

  • Replaced mock socket implementation with real server connections only (removed FakeSocket and related fake connection logic, as it was too synthetic and unrealistic)
  • Added comprehensive benchmark phases: SET, GET, and DELETE operations with detailed performance reporting
  • Implemented realistic data patterns with mixed value sizes (95% small values 80-250B, 5% large values 100KB)
  • Enhanced the benchmark to pre-populate 80% of keyspace for realistic hit/miss ratios during GET operations
  • Switched from MixedSerializer to ZstdSerializer with compression effectively disabled for cleaner performance measurement
  • Added detailed GC tracking across all benchmark phases to monitor memory behavior
  • Made server parameter required since mock functionality was removed
  • Updated meta-memcache-socket dependency from >= 0.1.3 to >= 0.1.7

Why these changes:

  • The previous benchmark only tested GET operations against mocked responses, which doesn't reflect real-world performance
  • Mixed value sizes better represent actual cache usage patterns where most values are small but some are significantly larger
  • Multi-phase testing (SET/GET/DELETE) provides a complete picture of cache client performance
  • Real server connections give accurate performance metrics instead of artificial mock results
  • Enhanced GC tracking helps identify memory leaks or excessive object creation during operations

The benchmark now provides actionable performance insights for all major cache operations under realistic conditions.

Copy link
Copy Markdown
Member Author

bisho commented Mar 11, 2026

@bisho bisho changed the title Enhance benchmark with comprehensive SET/GET/DELETE testing and ZstdSerializer Enhance benchmark with comprehensive SET/GET/DELETE testing Mar 11, 2026
@bisho bisho marked this pull request as ready for review March 11, 2026 14:56
@bisho bisho force-pushed the 03-11-improve_benchmark_to_test_set_get_and_delete_and_populate_values branch from 04a0c2a to abc3e7c Compare March 13, 2026 14:43
@bisho bisho requested a review from a team March 24, 2026 17:00
@bisho bisho force-pushed the 03-11-improve_benchmark_to_test_set_get_and_delete_and_populate_values branch from abc3e7c to 135ffbb Compare March 25, 2026 12:38
Comment thread benchmark.py Outdated
Comment thread benchmark.py Outdated
Comment thread benchmark.py Outdated
Comment thread benchmark.py Outdated
Comment thread benchmark.py Outdated
Comment thread benchmark.py Outdated
@bisho bisho force-pushed the 03-11-improve_benchmark_to_test_set_get_and_delete_and_populate_values branch from 135ffbb to b420700 Compare March 26, 2026 14:11
Copy link
Copy Markdown

@xmartinez xmartinez left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LG!

Copy link
Copy Markdown
Member Author

bisho commented Mar 26, 2026

Merge activity

  • Mar 26, 3:31 PM UTC: A user started a stack merge that includes this pull request via Graphite.
  • Mar 26, 3:32 PM UTC: @bisho merged this pull request with Graphite.

@bisho bisho merged commit 4cbfff4 into main Mar 26, 2026
3 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants