[SAP] modify select_datastore_by_name to use a cache#322
Merged
hemna merged 1 commit intostable/2023.1-m3from Apr 1, 2026
Merged
[SAP] modify select_datastore_by_name to use a cache#322hemna merged 1 commit intostable/2023.1-m3from
hemna merged 1 commit intostable/2023.1-m3from
Conversation
Optimize the select_datastore_by_name method to significantly reduce vCenter API load during high-volume operations like boot-from-volume. Problem: The original implementation called _get_datastores() which fetches ALL datastores with their 'host' and 'summary' properties. With ~40 datastores and ~50 hosts each, this transferred ~500KB-2MB of data per call. Under high load (e.g., 96 volume creates in 5 minutes), this caused vCenter connection pool exhaustion and cascading timeouts averaging 530 seconds per _select_ds_for_volume call. Solution: 1. Add _get_datastore_by_name() method that fetches properties for only the specific datastore needed, reducing data transfer to ~10-20KB. 2. Add a 5-minute TTL cache for datastore name -> moref mappings in get_ds_ref_by_name(). Since volume creates are bursty and typically target the same datastores, this eliminates repeated vCenter queries for the lightweight name lookup on cache hits. 3. The host availability check still queries vCenter on every call to ensure we always have fresh data about which hosts are connected and not in maintenance mode. Performance impact: - First call: 1 lightweight query (names only) + 1 targeted query - Subsequent calls (within 5 min): 1 targeted query only (cache hit) - Original: 1 heavy query fetching all datastores with all host mounts Change-Id: If9d46fe833418b67393535384af075a95f2ca4cb
Carthaca
approved these changes
Apr 1, 2026
sumitarora2786
approved these changes
Apr 1, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Optimize the select_datastore_by_name method to significantly reduce vCenter API load during high-volume operations like boot-from-volume.
Problem:
The original implementation called _get_datastores() which fetches ALL datastores with their 'host' and 'summary' properties. With ~40 datastores and ~50 hosts each, this transferred ~500KB-2MB of data per call. Under high load (e.g., 96 volume creates in 5 minutes), this caused vCenter connection pool exhaustion and cascading timeouts averaging 530 seconds per _select_ds_for_volume call.
Solution:
Add _get_datastore_by_name() method that fetches properties for only the specific datastore needed, reducing data transfer to ~10-20KB.
Add a 5-minute TTL cache for datastore name -> moref mappings in get_ds_ref_by_name(). Since volume creates are bursty and typically target the same datastores, this eliminates repeated vCenter queries for the lightweight name lookup on cache hits.
The host availability check still queries vCenter on every call to ensure we always have fresh data about which hosts are connected and not in maintenance mode.
Performance impact:
Change-Id: If9d46fe833418b67393535384af075a95f2ca4cb