Inconsistent price caching TTLs and permanent storage of current prices

[SuaveIV]

The problem

There are inconsistencies in how price data is cached, and current Steam prices are stored permanently when they shouldn't be.

1. Current Steam prices are cached permanently

cache_game_details() is called with permanent=True almost everywhere:

• populate_prices.py, populate_prices_optimized.py, populate_prices_async.py all use permanent=True
• steam_helpers.py uses permanent=True
• plugin_admin_actions.py uses permanent=True
• admin_commands.py uses permanent=True

The game_details_cache table stores price_overview which contains current price and discount percentage. This data changes frequently, especially during Steam sales. With permanent=True, the bot shows stale prices forever unless someone manually runs --force-refresh.

The default TTL is 168 hours (1 week) from config, but it's almost never used because every caller passes permanent=True.

2. ITAD historical low caching is inconsistent

Different code paths use different TTLs for ITAD prices:

Code path	TTL	Permanent?
populate_prices*.py scripts	permanent	Yes
get_lowest_price() in utils.py	14 days (336h)	No
prefetch_itad_prices() in utils.py	14 days (336h)	No
Failed lookups in utils.py	3 days (72h)	No

Running just populate-prices-turbo gives you permanent ITAD cache, but using !deals (which calls get_lowest_price()) only caches for 14 days.

3. Game metadata and prices share the same cache entry

The game_details_cache table stores both:

• Game metadata: name, type, categories, is_multiplayer, is_coop, is_family_shared
• Price data: price_overview (current price, discount, original price)

You can't have permanent game metadata while having short-lived price data because they're in the same row.

What should change

Current Steam prices (price_overview):

• Should NOT be permanent
• TTL should be 24-48 hours (configurable)
• During Steam sales, --refresh-current should be run more frequently

ITAD historical lows:

• These only go down over time, never up
• Could reasonably be permanent or 14-30 days
• Should be consistent across all code paths

Game metadata (name, categories, multiplayer flags):

• Can be permanent - these rarely change

Possible fixes

Option A: Don't use permanent for price data
Change cache_game_details() callers to not pass permanent=True when price data is present. Use the existing GAME_DETAILS_CACHE_TTL (1 week default, configurable). Simple but means game metadata also expires.

Option B: Separate price cache from metadata cache
Create a new current_price_cache table for price_overview with its own TTL. Keep game_details_cache permanent for metadata. More work but cleaner.

Option C: Split the row
Store metadata and prices in separate columns with different TTLs. Complex but most correct.

Recommendation

Option A is the simplest and addresses the main issue. Change the populate scripts and other callers to use permanent=False when caching game details with price data. The GAME_DETAILS_CACHE_TTL default of 1 week is reasonable for most cases, and users can configure it lower if they want more frequent updates.

For ITAD, standardize on 14 days everywhere (or make it configurable via itad_cache_ttl_hours in config.yml).

Files affected

• scripts/populate_prices.py - change permanent=True to permanent=False
• scripts/populate_prices_optimized.py - same
• scripts/populate_prices_async.py - same
• src/familybot/lib/steam_helpers.py - same
• src/familybot/lib/plugin_admin_actions.py - same
• src/familybot/lib/admin_commands.py - same
• src/familybot/lib/utils.py - standardize ITAD TTL to 14 days
• config-template.yml - document the TTL settings