-
Notifications
You must be signed in to change notification settings - Fork 26
ci: daily LoC report to Slack (daily) and Telegram (weekly) #368
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
Changes from all commits
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,152 @@ | ||
| #!/usr/bin/env bash | ||
| # | ||
| # Counts Rust lines of code in the ethlambda workspace and produces report | ||
| # files for Slack, Telegram, and the GitHub Actions step summary. | ||
| # | ||
| # Inputs (optional): | ||
| # loc_report.json.old Previous run's report — used to compute deltas. | ||
| # | ||
| # Outputs: | ||
| # loc_report.json Machine-readable report for caching. | ||
| # loc_report_slack.json Slack Block Kit payload (daily message). | ||
| # loc_report_telegram.txt Telegram HTML body (weekly message). | ||
| # loc_report_github.txt Plain-text block for the workflow step summary. | ||
|
|
||
| set -euo pipefail | ||
|
|
||
| OLD_REPORT="loc_report.json.old" | ||
| NEW_REPORT="loc_report.json" | ||
|
|
||
| count_loc() { | ||
| # Count Rust lines of code under $1. Excludes common non-product folders. | ||
| # If the path has no Rust files, returns 0. | ||
| # `-t Rust` (short form) is accepted by tokei v12 and v14. | ||
| tokei "$1" -t Rust --output json \ | ||
| -e tests -e benches -e examples 2>/dev/null \ | ||
| | jq '.Rust.code // 0' | ||
| } | ||
|
|
||
| # Enumerate workspace members through cargo so the list stays in sync | ||
| # with Cargo.toml automatically. | ||
| CRATE_DIRS=$( | ||
| cargo metadata --no-deps --format-version 1 \ | ||
| | jq -r '.packages[] | .manifest_path | sub("/Cargo.toml$"; "")' \ | ||
| | sort | ||
| ) | ||
|
|
||
| CRATES_JSON='[]' | ||
| TOTAL=0 | ||
| while IFS= read -r dir; do | ||
| [[ -z "$dir" ]] && continue | ||
| rel="${dir#"$PWD/"}" | ||
| src="${dir}/src" | ||
| if [[ -d "$src" ]]; then | ||
| loc=$(count_loc "$src") | ||
| else | ||
| loc=0 | ||
| fi | ||
| TOTAL=$((TOTAL + loc)) | ||
| CRATES_JSON=$(jq --arg path "$rel" --argjson loc "$loc" \ | ||
| '. + [{path: $path, loc: $loc}]' <<< "$CRATES_JSON") | ||
| done <<< "$CRATE_DIRS" | ||
|
|
||
| CRATES_JSON=$(jq 'sort_by(-.loc)' <<< "$CRATES_JSON") | ||
|
|
||
| jq -n --argjson total "$TOTAL" --argjson crates "$CRATES_JSON" \ | ||
| '{total: $total, crates: $crates}' > "$NEW_REPORT" | ||
|
|
||
| # Resolve previous totals (defaulting to current → zero deltas on first run). | ||
| OLD_TOTAL=$TOTAL | ||
| OLD_CRATES_JSON=$CRATES_JSON | ||
| if [[ -f "$OLD_REPORT" ]]; then | ||
| OLD_TOTAL=$(jq '.total' "$OLD_REPORT") | ||
| OLD_CRATES_JSON=$(jq '.crates' "$OLD_REPORT") | ||
| fi | ||
|
|
||
| format_diff() { | ||
| local cur=$1 old=$2 | ||
| if (( cur > old )); then echo "(+$((cur - old)))" | ||
| elif (( cur < old )); then echo "(-$((old - cur)))" | ||
| else echo "" | ||
| fi | ||
| } | ||
|
|
||
| TOTAL_DIFF=$(format_diff "$TOTAL" "$OLD_TOTAL") | ||
| COMMIT_SHA=${GITHUB_SHA:-$(git rev-parse HEAD)} | ||
| SHORT_SHA=${COMMIT_SHA:0:7} | ||
| DATE_UTC=$(date -u +"%Y-%m-%d") | ||
|
|
||
| # Build per-crate annotated rows once and reuse for every format. | ||
| ROWS_JSON=$(jq --argjson old "$OLD_CRATES_JSON" ' | ||
| map( | ||
| . as $c | ||
| | ($old | map(select(.path == $c.path)) | .[0].loc // 0) as $old_loc | ||
| | . + { | ||
| old_loc: $old_loc, | ||
| diff: ($c.loc - $old_loc) | ||
| } | ||
| ) | ||
| ' <<< "$CRATES_JSON") | ||
|
|
||
| format_diff_jq=' | ||
| def diff_str: | ||
| if . > 0 then "(+" + (. | tostring) + ")" | ||
| elif . < 0 then "(-" + ((. | -.) | tostring) + ")" | ||
| else "" end; | ||
| ' | ||
|
|
||
| # GitHub step summary (plain text inside a code block). | ||
| { | ||
| echo '```' | ||
| echo "ethlambda lines of code (${DATE_UTC}, ${SHORT_SHA})" | ||
| echo "============================================" | ||
| echo "Total Rust LoC: ${TOTAL} ${TOTAL_DIFF}" | ||
| echo | ||
| echo "Per-crate" | ||
| echo "---------" | ||
| jq -r "$format_diff_jq"' | ||
| .[] | "\(.path): \(.loc) \(.diff | diff_str)" | ||
| ' <<< "$ROWS_JSON" | ||
| echo | ||
| echo "Excluded folders: tests/, benches/, examples/" | ||
| echo '```' | ||
| } > loc_report_github.txt | ||
|
|
||
| # Slack Block Kit payload. | ||
| CRATES_MRKDWN=$(jq -r "$format_diff_jq"' | ||
| map("*\(.path)*: \(.loc) \(.diff | diff_str)") | join("\n") | ||
| ' <<< "$ROWS_JSON") | ||
|
|
||
| SUMMARY_TEXT=$(printf '*Total Rust LoC:* %s %s\n_Date:_ %s • _Commit:_ `%s`' \ | ||
| "$TOTAL" "$TOTAL_DIFF" "$DATE_UTC" "$SHORT_SHA") | ||
|
|
||
| jq -n \ | ||
| --arg summary "$SUMMARY_TEXT" \ | ||
| --arg crates "$CRATES_MRKDWN" \ | ||
| '{ | ||
| blocks: [ | ||
| { type: "header", text: { type: "plain_text", text: "Daily ethlambda LoC Report" } }, | ||
| { type: "divider" }, | ||
| { type: "section", text: { type: "mrkdwn", text: $summary } }, | ||
| { type: "header", text: { type: "plain_text", text: "Per-crate" } }, | ||
| { type: "section", text: { type: "mrkdwn", text: $crates } }, | ||
| { type: "context", elements: [ | ||
| { type: "mrkdwn", text: "_Excluded folders: tests/, benches/, examples/_" } | ||
| ]} | ||
| ] | ||
| }' > loc_report_slack.json | ||
|
|
||
| # Telegram (HTML parse mode). | ||
| { | ||
| echo "<b>Weekly ethlambda LoC Report</b>" | ||
| echo "Date: ${DATE_UTC} • Commit: <code>${SHORT_SHA}</code>" | ||
| echo | ||
| echo "<b>Total Rust LoC:</b> ${TOTAL} ${TOTAL_DIFF}" | ||
| echo | ||
| echo "<b>Per-crate</b>" | ||
| jq -r "$format_diff_jq"' | ||
| .[] | "<b>\(.path)</b>: \(.loc) \(.diff | diff_str)" | ||
| ' <<< "$ROWS_JSON" | ||
| echo | ||
| echo "<i>Excluded folders: tests/, benches/, examples/</i>" | ||
| } > loc_report_telegram.txt | ||
| Original file line number | Diff line number | Diff line change | ||||||||||||||||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| @@ -0,0 +1,19 @@ | ||||||||||||||||||||||||||||||
| #!/usr/bin/env bash | ||||||||||||||||||||||||||||||
| # | ||||||||||||||||||||||||||||||
| # POSTs a Slack Block Kit payload to an incoming webhook. | ||||||||||||||||||||||||||||||
| # | ||||||||||||||||||||||||||||||
| # Usage: publish_slack.sh <webhook_url> <payload_file> | ||||||||||||||||||||||||||||||
|
|
||||||||||||||||||||||||||||||
| set -euo pipefail | ||||||||||||||||||||||||||||||
|
|
||||||||||||||||||||||||||||||
| WEBHOOK_URL="${1:?webhook URL required}" | ||||||||||||||||||||||||||||||
| PAYLOAD_FILE="${2:?payload file required}" | ||||||||||||||||||||||||||||||
|
|
||||||||||||||||||||||||||||||
| if [[ -z "$WEBHOOK_URL" ]]; then | ||||||||||||||||||||||||||||||
| echo "::error::Slack webhook URL resolved to an empty value — check the secret configured for this trigger (scheduled vs manual)" | ||||||||||||||||||||||||||||||
| exit 1 | ||||||||||||||||||||||||||||||
| fi | ||||||||||||||||||||||||||||||
|
Comment on lines
+9
to
+15
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
Suggested change
Prompt To Fix With AIThis is a comment left during a code review.
Path: .github/scripts/publish_slack.sh
Line: 9-15
Comment:
The `[[ -z "$WEBHOOK_URL" ]]` guard on lines 12-15 is dead code and can never be reached. The `${1:?webhook URL required}` expansion on line 9 already causes the script to exit with an error if `$1` is empty or unset — so by the time execution reaches the `if` block, `WEBHOOK_URL` is guaranteed to be non-empty. The informative `::error::` annotation with the secrets-troubleshooting message is therefore never surfaced to the operator.
```suggestion
WEBHOOK_URL="${1}"
PAYLOAD_FILE="${2:?payload file required}"
if [[ -z "$WEBHOOK_URL" ]]; then
echo "::error::Slack webhook URL resolved to an empty value — check the secret configured for this trigger (scheduled vs manual)"
exit 1
fi
```
How can I resolve this? If you propose a fix, please make it concise. |
||||||||||||||||||||||||||||||
|
|
||||||||||||||||||||||||||||||
| curl --fail-with-body -X POST "$WEBHOOK_URL" \ | ||||||||||||||||||||||||||||||
| -H 'Content-Type: application/json; charset=utf-8' \ | ||||||||||||||||||||||||||||||
| --data @"$PAYLOAD_FILE" | ||||||||||||||||||||||||||||||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,28 @@ | ||
| #!/usr/bin/env bash | ||
| # | ||
| # POSTs the contents of a file as an HTML-formatted Telegram message. | ||
| # | ||
| # Required env: | ||
| # TELEGRAM_BOT_TOKEN Bot token used to authenticate the request. | ||
| # TELEGRAM_ETHLAMBDA_CHAT_ID Destination chat ID. | ||
| # | ||
| # Usage: publish_telegram.sh <message_file> | ||
|
|
||
| set -euo pipefail | ||
|
|
||
| MESSAGE_FILE="${1:?message file required}" | ||
|
|
||
| if [[ -z "${TELEGRAM_BOT_TOKEN:-}" ]]; then | ||
| echo "::error::TELEGRAM_BOT_TOKEN secret is not set — skipping Telegram post" | ||
| exit 1 | ||
| fi | ||
|
|
||
| if [[ -z "${TELEGRAM_ETHLAMBDA_CHAT_ID:-}" ]]; then | ||
| echo "::error::TELEGRAM_ETHLAMBDA_CHAT_ID resolved to an empty value — check that the appropriate secret is configured for this trigger (scheduled vs manual)" | ||
| exit 1 | ||
| fi | ||
|
|
||
| curl --fail-with-body -X POST "https://api.telegram.org/bot${TELEGRAM_BOT_TOKEN}/sendMessage" \ | ||
| -d chat_id="$TELEGRAM_ETHLAMBDA_CHAT_ID" \ | ||
| -d parse_mode=HTML \ | ||
| --data-urlencode text="$(cat "$MESSAGE_FILE")" |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,104 @@ | ||
| name: Daily Lines of Code Report | ||
|
|
||
| on: | ||
| schedule: | ||
| # Every day at UTC midnight (Slack daily, Telegram on Monday only) | ||
| - cron: "0 0 * * *" | ||
| workflow_dispatch: | ||
| inputs: | ||
| target: | ||
| description: "Where to post (test channel/chat or prod)" | ||
| required: true | ||
| default: "test" | ||
| type: choice | ||
| options: | ||
| - test | ||
| - prod | ||
| post_telegram: | ||
| description: "Also post to Telegram on this manual run" | ||
| required: false | ||
| default: false | ||
| type: boolean | ||
|
|
||
| permissions: | ||
| contents: read | ||
| actions: write | ||
|
|
||
| env: | ||
| CARGO_NET_GIT_FETCH_WITH_CLI: "true" | ||
| CARGO_NET_RETRY: "10" | ||
|
|
||
| jobs: | ||
| loc: | ||
| name: Count ethlambda LoC and publish report | ||
| runs-on: ubuntu-latest | ||
| steps: | ||
| - name: Checkout sources | ||
| uses: actions/checkout@v6 | ||
|
|
||
| - name: Setup Rust | ||
| uses: dtolnay/rust-toolchain@master | ||
| with: | ||
| toolchain: "1.92.0" | ||
|
|
||
| - name: Setup cache | ||
| uses: Swatinem/rust-cache@v2 | ||
|
|
||
| - name: Install tokei | ||
| run: cargo install tokei --locked --version 12.1.2 | ||
|
|
||
| - name: Restore previous LoC report | ||
| id: cache-loc-report | ||
| uses: actions/cache/restore@v5 | ||
| with: | ||
| path: loc_report.json | ||
| key: loc-report-${{ github.ref_name }}-${{ github.run_id }} | ||
| restore-keys: | | ||
| loc-report-${{ github.ref_name }}- | ||
|
|
||
| - name: Stash previous report as .old for delta computation | ||
| if: steps.cache-loc-report.outputs.cache-hit != '' | ||
| run: mv loc_report.json loc_report.json.old | ||
|
|
||
| - name: Generate LoC report | ||
| run: bash .github/scripts/generate_loc_report.sh | ||
|
|
||
| - name: Save new LoC report to cache | ||
| if: success() | ||
| uses: actions/cache/save@v5 | ||
| with: | ||
| path: loc_report.json | ||
| key: loc-report-${{ github.ref_name }}-${{ github.run_id }} | ||
|
|
||
| - name: Post results to workflow summary | ||
| run: cat loc_report_github.txt >> "$GITHUB_STEP_SUMMARY" | ||
|
|
||
| - name: Post to Slack | ||
| env: | ||
| SLACK_WEBHOOK: >- | ||
| ${{ (github.event_name == 'schedule' || inputs.target == 'prod') | ||
| && secrets.ETHLAMBDA_GENERAL_SLACK_WEBHOOK | ||
| || secrets.ETHLAMBDA_TEST_SLACK_WEBHOOK }} | ||
| run: bash .github/scripts/publish_slack.sh "$SLACK_WEBHOOK" loc_report_slack.json | ||
|
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
Prompt To Fix With AIThis is a comment left during a code review.
Path: .github/workflows/daily_loc_report.yml
Line: 82
Comment:
**Webhook URL passed as positional CLI argument**
`$SLACK_WEBHOOK` is expanded into the command line, making it visible in `ps` output for the duration of the `curl` call. Passing it via an environment variable (as is already done for `TELEGRAM_BOT_TOKEN`) keeps it out of the process argument list. The script already has `SLACK_WEBHOOK` as an env var, so you could read it directly from there instead of forwarding it as `$1`.
How can I resolve this? If you propose a fix, please make it concise. |
||
|
|
||
| - name: Post to Telegram (weekly, or manual opt-in) | ||
| env: | ||
| TELEGRAM_BOT_TOKEN: ${{ secrets.TELEGRAM_BOT_TOKEN }} | ||
| TELEGRAM_ETHLAMBDA_CHAT_ID: >- | ||
| ${{ (github.event_name == 'schedule' || inputs.target == 'prod') | ||
| && secrets.TELEGRAM_ETHLAMBDA_CHAT_ID | ||
| || secrets.TELEGRAM_ETHLAMBDA_TEST_CHAT_ID }} | ||
| run: | | ||
| # Scheduled runs only post to Telegram on Monday (UTC). | ||
| # Manual runs require post_telegram=true to opt in. | ||
| if [[ "${{ github.event_name }}" == "schedule" ]]; then | ||
| day_of_week=$(date -u +%u) # 1=Monday .. 7=Sunday | ||
| if [[ "$day_of_week" != "1" ]]; then | ||
| echo "Skipping Telegram post (scheduled run, only sent on Monday)" | ||
| exit 0 | ||
| fi | ||
| elif [[ "${{ inputs.post_telegram }}" != "true" ]]; then | ||
| echo "Skipping Telegram post (manual run, post_telegram not enabled)" | ||
| exit 0 | ||
| fi | ||
| bash .github/scripts/publish_telegram.sh loc_report_telegram.txt | ||
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
src/subdirectory onlyCrates that place Rust source in directories other than
src/— for example, aproc-macrosub-crate that uses a flat layout, or a virtual workspace member with code directly at the crate root — will be counted as 0 LoC. The tokei call would still find their.rsfiles if pointed at the crate root ($dir), but the current code skips any crate where$dir/srcdoes not exist. This is worth a clarifying comment if the workspace layout is always conventional.Prompt To Fix With AI