Skip to content

feat: Centralize logging setup with structlog integration#179

Open
khvn26 wants to merge 15 commits intomainfrom
feat/logging-improvements
Open

feat: Centralize logging setup with structlog integration#179
khvn26 wants to merge 15 commits intomainfrom
feat/logging-improvements

Conversation

@khvn26
Copy link
Member

@khvn26 khvn26 commented Mar 13, 2026

Closes #31

What changed

Logging setup moves from Django settings into flagsmith-common, running before Django loads. Both stdlib and structlog now route through a single ProcessorFormatter, so all output (app code, structlog, Gunicorn error logs) shares the same format and ISO timestamps.

Key additions

  • setup_logging() in common.core.logging — configures stdlib via dictConfig, then wires structlog through stdlib using LoggerFactory (not PrintLoggerFactory). Accepts application_loggers for per-package level control and extra_foreign_processors for extending the foreign_pre_chain.
  • _SentryFriendlyProcessorFormatter — subclass of ProcessorFormatter that snapshots and restores record.msg/record.args across formatting, so Sentry's LoggingIntegration (which reads these in a finally block after handlers run) sees the original message template for proper event grouping.
  • sentry_processor in common.core.sentry — structlog processor that sets Sentry context from event dicts. Hardwired into the structlog processor chain. Since structlog routes through stdlib, Sentry's LoggingIntegration automatically captures ERROR+ logs.
  • application_loggers allowlist — root logger stays at WARNING to suppress third-party noise; listed app loggers get the requested level. DEBUG overrides everything. Configurable via APPLICATION_LOGGERS env var (comma-separated).
  • make_gunicorn_access_processor() in common.gunicorn.processorsforeign_pre_chain processor that extracts structured fields (path, method, status, duration, extra items) from Gunicorn access log records. Configurable via ACCESS_LOG_EXTRA_ITEMS env var.
  • Gunicorn error logs propagate to root — no separate handler/formatter needed.
  • Gunicorn access logs — JSON mode replaces the formatter on Gunicorn's handler with root's ProcessorFormatter (respecting ACCESS_LOG_LOCATION); generic mode keeps Gunicorn's own CLF formatter.
  • TTY-aware colorsConsoleRenderer only enables ANSI colors when stdout is a TTY.
  • sentry-sdk added as an explicit dependency in the common-core group.
  • environs used in ensure_cli_env for env var parsing.

What it removes

  • GunicornAccessLogJsonFormatter and GunicornAccessLogJsonRecord — replaced by the foreign_pre_chain processor.
  • Separate formatter management for Gunicorn error logs.
  • LOGGING_DEFAULT_GENERIC_FORMAT constant — ConsoleRenderer handles generic formatting.
  • PROMETHEUS_HISTOGRAM_BUCKETS and PROMETHEUS_HTTP_SERVER_RESPONSE_SIZE_HISTOGRAM_BUCKETS Django settings — histogram buckets now use library defaults directly, removing import-time Django settings access that prevented pre-Django logging setup.

What it keeps

  • JsonFormatter and JsonRecordJsonRecord is still the schema definition used by map_event_to_json_record.
  • Gunicorn access log CLF format in generic mode.
  • Gunicorn constants (WSGI_EXTRA_SUFFIX_TO_CATEGORY, wsgi_extra_key_regex) in their original location.

Migration guide for flagsmith/flagsmith

1. Disable Django's logging configuration in settings/common.py

# Remove the entire LOGGING = { ... } block and the structlog.configure block.
# Replace with:
LOGGING_CONFIG = None

LOGGING_CONFIG = None tells Django to skip its dictConfig() call during django.setup(), preserving the handlers and logger levels configured by setup_logging().

2. Set env vars

APPLICATION_LOGGERS=app,features,integrations,task_processor,app_analytics,webhooks,gunicorn
ACCESS_LOG_EXTRA_ITEMS={flagsmith.route}e
ACCESS_LOG_LOCATION=-

3. Delete api/util/logging.py

It duplicates JsonFormatter which now lives in common.core.logging.

4. Remove PROMETHEUS_HISTOGRAM_BUCKETS setting

No longer consumed by flagsmith-common.

@khvn26 khvn26 requested a review from a team as a code owner March 13, 2026 19:28
@khvn26 khvn26 requested review from Zaimwa9 and removed request for a team March 13, 2026 19:28
@codecov-commenter
Copy link

codecov-commenter commented Mar 13, 2026

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 96.48%. Comparing base (398c83c) to head (394df59).

Additional details and impacted files
@@            Coverage Diff             @@
##             main     #179      +/-   ##
==========================================
+ Coverage   96.12%   96.48%   +0.36%     
==========================================
  Files          92       97       +5     
  Lines        3222     3552     +330     
==========================================
+ Hits         3097     3427     +330     
  Misses        125      125              

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@Zaimwa9 Zaimwa9 requested a review from emyller March 23, 2026 13:27
Copy link
Contributor

@emyller emyller left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks great! Left a few questions.

@khvn26 khvn26 requested a review from emyller March 26, 2026 18:11
Copy link
Contributor

@emyller emyller left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Logging improvements

3 participants