feat: Add Reports Agents (Weekly, Enterprise, Daily) #33
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,228 @@ | ||
| --- | ||
| name: daily-reports | ||
| description: Generate automated daily reports on key metrics. Includes day-over-day comparisons, anomaly detection, and quick health checks. | ||
| tags: [reports, daily, monitoring, alerts] | ||
| --- | ||
|
|
||
| # Daily Reports | ||
|
|
||
| ## When to use | ||
|
|
||
| - "Generate daily report" | ||
| - "Daily metrics summary" | ||
| - "Yesterday's performance" | ||
| - "Daily health check" | ||
| - "Morning metrics report" | ||
|
|
||
| ## What it generates | ||
|
|
||
| - **Core metrics**: DAU, generation volumes (image/video), downloads, new signups | ||
| - **Day-over-day comparison**: % change vs prior day and 7-day average | ||
| - **Anomaly alerts**: Metrics that deviate >2 std devs from baseline | ||
| - **Quick wins**: Notable improvements or milestones | ||
| - **Red flags**: Concerning drops or issues | ||
| - **Segment breakdown**: Key metrics by tier (if significant variance) | ||
|
|
||
| ## Steps | ||
|
|
||
| ### 1. Gather Requirements | ||
|
|
||
| Ask the user: | ||
| - **Report date**: Yesterday (default) or specific date? | ||
| - **Audience**: Product team, engineering, leadership? | ||
| - **Delivery**: Automated daily post or on-demand? | ||
| - **Threshold**: What % change should trigger "red flag"? (Default: >15% drop) | ||
| - **Format**: Slack, email, or dashboard? | ||
|
|
||
| ### 2. Read Shared Knowledge | ||
|
|
||
| Before generating report: | ||
| - **`shared/bq-schema.md`** — ltxstudio_user_all_actions table | ||
| - **`shared/metric-standards.md`** — DAU calculation, generation metrics | ||
|
|
||
| ### 3. Define Report Period | ||
|
|
||
| Calculate date ranges: | ||
| - **Report date**: Yesterday (`DATE_SUB(CURRENT_DATE(), INTERVAL 1 DAY)`) | ||
| - **Comparison date**: Day before yesterday | ||
| - **Baseline**: 7-day average (days -8 to -2) | ||
|
Collaborator
There was a problem hiding this comment. let's increase this to 14 or 30 |
||
|
|
||
| ### 4. Generate SQL for Each Section | ||
|
|
||
| **Core Metrics (Yesterday):** | ||
| ```sql | ||
| SELECT | ||
| COUNT(DISTINCT lt_id) AS dau, | ||
| COUNT(CASE WHEN action_name = 'generate_image' THEN 1 END) AS image_generations, | ||
| COUNT(CASE WHEN action_name = 'generate_video' AND action_category = 'generations' THEN 1 END) AS video_generations, | ||
|
Collaborator
There was a problem hiding this comment. action_name_detailed? |
||
| COUNT(CASE WHEN action_name = 'download_image' THEN 1 END) AS image_downloads, | ||
| COUNT(CASE WHEN action_name = 'download_video' THEN 1 END) AS video_downloads | ||
| FROM `ltx-dwh-prod-processed.web.ltxstudio_user_all_actions` | ||
| WHERE date(action_ts) = DATE_SUB(CURRENT_DATE(), INTERVAL 1 DAY) | ||
| ``` | ||
|
|
||
| **Day-over-Day Comparison:** | ||
| ```sql | ||
| WITH yesterday AS ( | ||
| SELECT | ||
| COUNT(DISTINCT lt_id) AS dau, | ||
| COUNT(CASE WHEN action_name = 'generate_image' THEN 1 END) AS image_gens | ||
| FROM `ltx-dwh-prod-processed.web.ltxstudio_user_all_actions` | ||
| WHERE date(action_ts) = DATE_SUB(CURRENT_DATE(), INTERVAL 1 DAY) | ||
| ), | ||
| prior_day AS ( | ||
| SELECT | ||
| COUNT(DISTINCT lt_id) AS dau, | ||
| COUNT(CASE WHEN action_name = 'generate_image' THEN 1 END) AS image_gens | ||
| FROM `ltx-dwh-prod-processed.web.ltxstudio_user_all_actions` | ||
| WHERE date(action_ts) = DATE_SUB(CURRENT_DATE(), INTERVAL 2 DAY) | ||
| ) | ||
| SELECT | ||
| y.dau AS yesterday_dau, | ||
| p.dau AS prior_day_dau, | ||
| ROUND(SAFE_DIVIDE(y.dau - p.dau, p.dau) * 100, 1) AS dau_dod_change_pct | ||
| FROM yesterday y, prior_day p | ||
| ``` | ||
|
|
||
| **Anomaly Detection:** | ||
| ```sql | ||
|
Collaborator
There was a problem hiding this comment. I'd increase it to 14 or 30 days |
||
| WITH daily AS ( | ||
| SELECT | ||
| date(action_ts) AS dt, | ||
| COUNT(DISTINCT lt_id) AS dau | ||
| FROM `ltx-dwh-prod-processed.web.ltxstudio_user_all_actions` | ||
| WHERE date(action_ts) >= DATE_SUB(CURRENT_DATE(), INTERVAL 9 DAY) | ||
| AND date(action_ts) < CURRENT_DATE() | ||
| GROUP BY dt | ||
| ), | ||
| baseline AS ( | ||
| SELECT | ||
| AVG(dau) AS avg_dau_7d, | ||
| STDDEV(dau) AS std_dau_7d | ||
| FROM daily | ||
| WHERE dt BETWEEN DATE_SUB(CURRENT_DATE(), INTERVAL 8 DAY) | ||
| AND DATE_SUB(CURRENT_DATE(), INTERVAL 2 DAY) | ||
| ) | ||
| SELECT | ||
| d.dau AS yesterday_dau, | ||
| b.avg_dau_7d, | ||
| ROUND(SAFE_DIVIDE(d.dau - b.avg_dau_7d, NULLIF(b.std_dau_7d, 0)), 1) AS z_score, | ||
| CASE WHEN ABS(SAFE_DIVIDE(d.dau - b.avg_dau_7d, NULLIF(b.std_dau_7d, 0))) > 2 | ||
| THEN 'ANOMALY' ELSE 'NORMAL' END AS status | ||
| FROM daily d, baseline b | ||
| WHERE d.dt = DATE_SUB(CURRENT_DATE(), INTERVAL 1 DAY) | ||
| ``` | ||
|
|
||
| ### 5. Analyze Results | ||
|
|
||
| **Identify key insights:** | ||
| - **Green flags**: Metrics up >10% vs prior day or 7-day avg | ||
| - **Red flags**: Metrics down >15% vs prior day or 7-day avg | ||
| - **Anomalies**: Z-score > 2 (statistical outlier) | ||
| - **Stability**: Metrics within ±10% (business as usual) | ||
|
|
||
| **Check segmentation if red flag:** | ||
| - If DAU drops >15%, break down by tier to find which segment dropped | ||
| - If generations drop, check if it's image, video, or both | ||
| - If specific tier drops, investigate potential cause (billing issue, product bug, competition) | ||
|
|
||
| ### 6. Format Report | ||
|
|
||
| **Structure:** | ||
| ``` | ||
| # Daily Report: [Day of Week], [MM/DD/YYYY] | ||
|
|
||
| ## 📊 Core Metrics | ||
|
Collaborator
There was a problem hiding this comment. let's do it per segment, and maybe we should add registered and churned?
Collaborator
There was a problem hiding this comment. and token consumption |
||
| | Metric | Yesterday | Prior Day | Change | vs 7d Avg | | ||
| |--------|-----------|-----------|--------|-----------| | ||
| | DAU | X | Y | ±Z% | ±A% | | ||
| | Image Gens | X | Y | ±Z% | ±A% | | ||
| | Video Gens | X | Y | ±Z% | ±A% | | ||
| | Downloads | X | Y | ±Z% | ±A% | | ||
|
|
||
| ## 🚨 Alerts | ||
| [If no alerts: ✅ No anomalies detected — all metrics within normal range] | ||
|
|
||
| [If alerts:] | ||
| - 🔴 **DAU dropped 25%** vs prior day (Z-score: -3.2) — investigate immediately | ||
| - 🟡 **Video generations down 12%** vs 7d avg — monitor trend | ||
|
|
||
| ## 💚 Quick Wins | ||
| - Image generations up 15% (new high for the week!) | ||
| - Enterprise DAU up 8% (strong enterprise engagement) | ||
|
|
||
| ## 📈 Trends to Watch | ||
| - DAU has been declining for 3 consecutive days (-5%, -8%, -15%) | ||
| - Video generation adoption improving (+18% week-to-date) | ||
|
|
||
| ## 🎯 Action Items | ||
| - [If red flag: "Investigate DAU drop — check logs for errors or outages"] | ||
| - [If trend: "Schedule product review meeting to discuss declining DAU"] | ||
| - [If stable: "No action needed — metrics healthy"] | ||
| ``` | ||
|
|
||
| **Slack format (concise):** | ||
| ``` | ||
| 📊 *Daily Report: [MM/DD]* | ||
|
|
||
| *Core Metrics:* | ||
| • DAU: X (↓15% vs prior day) 🔴 | ||
| • Image Gens: X (↑5% vs 7d avg) ✅ | ||
| • Video Gens: X (stable) ✅ | ||
|
|
||
| *Alerts:* | ||
| 🔴 DAU anomaly detected (Z-score: -3.2) | ||
|
|
||
| *Action:* Investigate DAU drop immediately | ||
| ``` | ||
|
|
||
| ### 7. Deliver Report | ||
|
|
||
| - **Slack**: Post to #product-daily or #data-alerts at 8am daily | ||
| - **Email**: Send to distribution list for daily digest | ||
| - **Dashboard**: Update live dashboard (Hex, Looker) with latest data | ||
| - **On-call**: Trigger PagerDuty/Slack alert for critical red flags (>20% drop) | ||
|
|
||
| ## Reference Files | ||
|
|
||
| | File | Read when | | ||
| |------|-----------| | ||
| | `shared/bq-schema.md` | ltxstudio_user_all_actions table schema | | ||
| | `shared/metric-standards.md` | DAU calculation, generation metrics | | ||
|
Collaborator
There was a problem hiding this comment. we had it earlier in the doc |
||
|
|
||
| ## Rules | ||
|
|
||
| ### Query Best Practices | ||
|
|
||
| - **DO** use yesterday's complete data (never include today) | ||
| - **DO** filter on `date(action_ts)` partition column for performance | ||
| - **DO** calculate Z-score using 7-day baseline (excludes yesterday) | ||
| - **DO** use NULLIF when dividing by potentially-zero denominators | ||
| - **DO** round percentages to 1 decimal place | ||
|
|
||
| ### Report Content | ||
|
|
||
| - **DO** lead with summary table of core metrics | ||
| - **DO** highlight anomalies prominently (🔴/🟡/✅ indicators) | ||
| - **DO** include both day-over-day AND vs 7-day avg comparisons | ||
| - **DO** provide quick wins to balance red flags | ||
| - **DO** include actionable recommendations | ||
| - **DO NOT** overload with detail — keep it scannable | ||
| - **DO NOT** report metrics that are always stable (noise) | ||
|
|
||
| ### Alerting | ||
|
|
||
| - **DO** set clear thresholds for red flags (>15% drop) | ||
| - **DO** escalate critical anomalies (Z-score > 2.5 or drop >25%) | ||
| - **DO** include context for anomalies (holiday, product launch, known issue) | ||
| - **DO** distinguish between concerning drops vs expected variance | ||
| - **DO NOT** alert on weekends (lower usage is normal) | ||
| - **DO NOT** cry wolf — only alert on actionable issues | ||
|
|
||
| ### Automation | ||
|
|
||
| - **DO** schedule daily report generation for 8am (after yesterday is complete) | ||
| - **DO** use BigQuery scheduled queries for consistency | ||
| - **DO** cache baseline calculations (7-day avg, stddev) for performance | ||
| - **DO** implement fail-safes (if query fails, send notification) | ||
| - **DO** version control report SQL (track changes over time) | ||
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
let's add more shared knowledge