Skip to content

Add BrainLayer newsyslog rotation#299

Merged
EtanHey merged 1 commit into
mainfrom
chore/newsyslog
May 18, 2026
Merged

Add BrainLayer newsyslog rotation#299
EtanHey merged 1 commit into
mainfrom
chore/newsyslog

Conversation

@EtanHey
Copy link
Copy Markdown
Owner

@EtanHey EtanHey commented May 18, 2026

Summary

  • Add launchd/newsyslog.d/brainlayer.conf covering BrainLayer LaunchAgent logs under ~/Library/Logs/brainlayer/.
  • Add launchd/install-newsyslog.sh to install the drop-in into /etc/newsyslog.d/brainlayer.conf and normalize existing log ownership/mode.
  • Document the root:admin/root-owned replacement file footgun and why entries pin etanheyman:staff.

Rotation policy

  • brainbar, watch, drain, maintenance logs: 7 archives, 1024 KB threshold.
  • enrichment: 7 archives, 5120 KB threshold.
  • backup-daily: 4 archives, 512 KB threshold.
  • All entries use mode 644, flags JN, and owner/group etanheyman:staff so user LaunchAgents can keep appending after rotation.

Tests

  • bash -n launchd/install-newsyslog.sh
  • PYTHONPATH=src pytest tests/test_newsyslog_config.py -q
  • ruff check src/ tests/ && ruff format --check src/ tests/
  • git diff --check

Verification limitation

  • newsyslog -nv -f launchd/newsyslog.d/brainlayer.conf was attempted and confirmed macOS requires root privileges.
  • sudo newsyslog -nv -f launchd/newsyslog.d/brainlayer.conf could not run in this non-interactive shell because sudo requires a password/TTY.

Note

Medium Risk
Adds a root-run installer that writes to /etc/newsyslog.d and changes on-disk log ownership/permissions, so mistakes could affect system log rotation or file access. Scope is contained to BrainLayer log paths and includes validation/tests to reduce risk.

Overview
Adds a newsyslog drop-in (launchd/newsyslog.d/brainlayer.conf) to rotate finite BrainLayer LaunchAgent *.out.log/*.err.log files with explicit owner:group, mode 644, and JN flags so rotated files remain writable by user daemons.

Introduces launchd/install-newsyslog.sh to render the template config for a chosen BRAINLAYER_LOG_OWNER/GROUP/DIR, pre-fix existing log directory/file ownership+mode, validate via newsyslog -nv, and install into /etc/newsyslog.d/brainlayer.conf.

Documents the macOS root-owned rotated file footgun and intentionally excludes long-running/held-open logs; adds Python tests asserting config coverage, exclusions, and installer behavior.

Reviewed by Cursor Bugbot for commit a0cbf99. Bugbot is set up for automated code reviews on this repo. Configure here.

Note

Add newsyslog log rotation config and installer for BrainLayer LaunchAgent logs

  • Adds brainlayer.conf, a newsyslog drop-in config that rotates finite-lifetime LaunchAgent logs (backup-daily, decay, wal-checkpoint, index, repair-fts) with mode 644, JN flags, and explicit owner:group to avoid root-owned rotated files.
  • Adds install-newsyslog.sh, a bash installer that resolves the invoking user's home directory via dscl, renders the template config with sed, repairs ownership/permissions on existing log files, validates with newsyslog -nv, and installs to /etc/newsyslog.d/brainlayer.conf.
  • Long-running jobs (brainbar, enrichment, watch, drain) are intentionally excluded due to missing post-rotate hook and copy-truncate support on macOS.
  • Adds a test suite that validates config coverage, ownership, flags, exclusions, and installer behavior.
  • Risk: installer must run as root and explicitly rejects log paths containing whitespace (dscl home resolution fails silently on some edge cases).

Macroscope summarized a0cbf99.

@coderabbitai
Copy link
Copy Markdown

coderabbitai Bot commented May 18, 2026

Warning

Rate limit exceeded

@EtanHey has exceeded the limit for the number of commits that can be reviewed per hour. Please wait 23 minutes and 1 second before requesting another review.

You’ve run out of usage credits. Purchase more in the billing tab.

⌛ How to resolve this issue?

After the wait time has elapsed, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.

We recommend that you space out your commits to avoid hitting the rate limit.

🚦 How do rate limits work?

CodeRabbit enforces hourly rate limits for each developer per organization.

Our paid plans have higher rate limits than the trial, open-source and free plans. In all cases, we re-allow further reviews after a brief timeout.

Please see our FAQ for further information.

ℹ️ Review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: ASSERTIVE

Plan: Pro

Run ID: 8f661fad-080b-4f20-86c0-eba889902b96

📥 Commits

Reviewing files that changed from the base of the PR and between 4f0b2a2 and a0cbf99.

📒 Files selected for processing (4)
  • launchd/install-newsyslog.sh
  • launchd/newsyslog.d/README.md
  • launchd/newsyslog.d/brainlayer.conf
  • tests/test_newsyslog_config.py
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch chore/newsyslog

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

Copy link
Copy Markdown

@greptile-apps greptile-apps Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Your free trial has ended. If you'd like to continue receiving code reviews, you can add a payment method here.

@EtanHey
Copy link
Copy Markdown
Owner Author

EtanHey commented May 18, 2026

@codex review

@EtanHey
Copy link
Copy Markdown
Owner Author

EtanHey commented May 18, 2026

@cursor @BugBot review

@EtanHey
Copy link
Copy Markdown
Owner Author

EtanHey commented May 18, 2026

@greptileai review

@EtanHey
Copy link
Copy Markdown
Owner Author

EtanHey commented May 18, 2026

@coderabbitai review

@cursor
Copy link
Copy Markdown

cursor Bot commented May 18, 2026

You need to increase your spend limit or enable usage-based billing to run background agents. Go to Cursor

@coderabbitai
Copy link
Copy Markdown

coderabbitai Bot commented May 18, 2026

✅ Actions performed

Review triggered.

Note: CodeRabbit is an incremental review system and does not re-review already reviewed commits. This command is applicable only when automatic reviews are paused.

Copy link
Copy Markdown

@greptile-apps greptile-apps Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Your free trial has ended. If you'd like to continue receiving code reviews, you can add a payment method here.

Comment thread launchd/install-newsyslog.sh
Copy link
Copy Markdown

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 60781526b5

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread launchd/install-newsyslog.sh Outdated
Comment on lines +24 to +25
chown "$OWNER:$GROUP" "$log"
chmod 0644 "$log"
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Preserve the user when repairing logs

When the installer is run as the normal LaunchAgent user, the root-owned replacement-file case described in the README leaves existing logs owned by root/admin, but this loop uses plain chown/chmod under set -e; the first root-owned *.log aborts the script instead of fixing the broken append path. Running the whole script with sudo is not a safe workaround either, because OWNER and LOG_DIR are then derived from the root environment. Use privileged chown/chmod while targeting the invoking user’s log directory.

Useful? React with 👍 / 👎.

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed. Same change: the script no longer relies on root HOME when invoked via sudo, and privileged ownership repair is explicit on each existing log file.

Comment thread launchd/install-newsyslog.sh Outdated
Comment thread launchd/install-newsyslog.sh Outdated
@EtanHey EtanHey force-pushed the chore/newsyslog branch from 6078152 to a7e049f Compare May 18, 2026 10:36
Copy link
Copy Markdown

@greptile-apps greptile-apps Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Your free trial has ended. If you'd like to continue receiving code reviews, you can add a payment method here.

@EtanHey
Copy link
Copy Markdown
Owner Author

EtanHey commented May 18, 2026

@codex review

Copy link
Copy Markdown

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: a7e049ff21

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread launchd/install-newsyslog.sh Outdated

mkdir -p "$LOG_DIR"
sudo mkdir -p /etc/newsyslog.d
sudo install -o root -g wheel -m 0644 "$SRC" "$DST"
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Render newsyslog config for the selected owner

When this installer is run by any account other than etanheyman (or with BRAINLAYER_LOG_OWNER/BRAINLAYER_LOG_DIR set), it computes the intended owner and log directory but then copies the literal config unchanged, so newsyslog still watches /Users/etanheyman/... and creates files as etanheyman:staff. The launchd plists are rendered to the invoking user's $HOME, so those users' logs will not be rotated and the root-owned replacement-file problem this script is meant to prevent remains.

Useful? React with 👍 / 👎.

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in 99938f7. The installer now renders the checked-in config through a temp file, replacing both the canonical log directory and owner:group with the selected BRAINLAYER_LOG_DIR and OWNER:GROUP before installing /etc/newsyslog.d/brainlayer.conf.

Comment thread launchd/install-newsyslog.sh
Comment thread launchd/install-newsyslog.sh Outdated
Comment thread launchd/install-newsyslog.sh Outdated
Comment thread launchd/install-newsyslog.sh Outdated
@EtanHey EtanHey force-pushed the chore/newsyslog branch from a7e049f to 99938f7 Compare May 18, 2026 10:42
Copy link
Copy Markdown

@greptile-apps greptile-apps Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Your free trial has ended. If you'd like to continue receiving code reviews, you can add a payment method here.

@EtanHey
Copy link
Copy Markdown
Owner Author

EtanHey commented May 18, 2026

@codex review
@cursor review
@BugBot review

Copy link
Copy Markdown

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 99938f70c8

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread launchd/newsyslog.d/brainlayer.conf Outdated
# BrainLayer user LaunchAgent logs.
# owner:group is explicit so rotated files remain writable by user-level daemons.
# logfilename owner:group mode count size when flags
/Users/etanheyman/Library/Logs/brainlayer/brainbar.out.log etanheyman:staff 644 7 1024 * JN
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Restart launchd jobs when rotating held-open logs

For long-running LaunchAgents such as brainbar, watch, drain, and enrichment, their StandardOutPath/StandardErrorPath file descriptors stay open after newsyslog renames the file. The N flag explicitly means no process is signaled, so after a size-triggered rotation these daemons can keep writing to the rotated or unlinked inode instead of the new .log, defeating the rotation and potentially continuing disk growth. This config needs a post-rotate restart/reopen mechanism for the launchd jobs, or it should not rotate held-open logs this way.

Useful? React with 👍 / 👎.

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in 93b55d3. The newsyslog drop-in no longer rotates held-open long-running StandardOutPath/StandardErrorPath logs. It now covers finite scheduled jobs only, and the README/config document that BrainBar, watch, enrichment, and drain need a coupled launchd restart or pid-file signal path before they can be safely added.

Comment thread launchd/install-newsyslog.sh Outdated

sudo mkdir -p "$LOG_DIR"
sudo mkdir -p /etc/newsyslog.d
sudo install -o root -g wheel -m 0644 "$RENDERED_CONFIG" "$DST"
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Validate rendered config before installing it

If a runtime substitution is invalid, for example BRAINLAYER_LOG_GROUP is misspelled, this writes /etc/newsyslog.d/brainlayer.conf before the later chown/newsyslog -nv failure stops the script. Because macOS newsyslog reads drop-ins from /etc/newsyslog.d, the machine is left with a broken system rotation config even though the installer exits unsuccessfully; validate the rendered file and owner/group before replacing the live drop-in.

Useful? React with 👍 / 👎.

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in 93b55d3. The installer now validates the rendered temp config with sudo newsyslog -nv before creating/replacing /etc/newsyslog.d/brainlayer.conf, after creating and ownership-validating the target log dir.

Comment thread launchd/install-newsyslog.sh
@EtanHey EtanHey force-pushed the chore/newsyslog branch from 99938f7 to 93b55d3 Compare May 18, 2026 10:55
Copy link
Copy Markdown

@greptile-apps greptile-apps Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Your free trial has ended. If you'd like to continue receiving code reviews, you can add a payment method here.

@EtanHey
Copy link
Copy Markdown
Owner Author

EtanHey commented May 18, 2026

@codex review
@cursor review
@BugBot review

Copy link
Copy Markdown

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 93b55d3063

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

OWNER_GROUP_ESCAPED="$(escape_sed_replacement "$OWNER:$GROUP")"

sed \
-e "s#/Users/etanheyman/Library/Logs/brainlayer#$LOG_DIR_ESCAPED#g" \
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Reject whitespace log directories before rendering

When the invoking user's home or BRAINLAYER_LOG_DIR contains whitespace, this substitution writes the path into newsyslog.conf as a bare field. newsyslog.conf lines are whitespace-delimited, so a path like /Users/Jane Doe/Library/Logs/brainlayer is parsed as /Users/Jane plus shifted columns, making the installer fail validation instead of supporting the space-preserving home lookup that the new test/README advertise. Please reject such paths with a clear error or render them in a format newsyslog actually accepts.

Useful? React with 👍 / 👎.

@EtanHey EtanHey force-pushed the chore/newsyslog branch from 93b55d3 to 18042ed Compare May 18, 2026 10:59
@EtanHey
Copy link
Copy Markdown
Owner Author

EtanHey commented May 18, 2026

@codex review
@cursor review
@BugBot review

Copy link
Copy Markdown

@greptile-apps greptile-apps Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Your free trial has ended. If you'd like to continue receiving code reviews, you can add a payment method here.

Copy link
Copy Markdown

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 18042edd14

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment on lines +47 to +49
for log in "$LOG_DIR"/*.log; do
[ -e "$log" ] || continue
sudo chown "$OWNER:$GROUP" "$log"
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Avoid following user-controlled log symlinks

When this installer is run by an admin for a BRAINLAYER_LOG_OWNER whose log directory already exists or is user-writable, a *.log symlink in that directory is expanded by the glob and passed to sudo chown without -h; the macOS chown(8) docs indicate -h is the option that changes the link itself, so this follows the link and can transfer ownership/chmod of arbitrary files outside the log directory. Please skip symlinks or operate only on verified regular files before doing privileged chown/chmod.

Useful? React with 👍 / 👎.

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in a958d49. The ownership repair loop now skips symlinks and non-regular paths before any sudo chown/chmod, so user-controlled log symlinks are not followed.

OWNER_GROUP_ESCAPED="$(escape_sed_replacement "$OWNER:$GROUP")"

sed \
-e "s#/Users/etanheyman/Library/Logs/brainlayer#$LOG_DIR_ESCAPED#g" \
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Escape whitespace in rendered newsyslog paths

When BRAINLAYER_LOG_DIR or the resolved OWNER_HOME contains spaces, this substitution writes the raw path into the config. macOS newsyslog.conf(5) documents that each config line's fields are separated with whitespace, so sudo newsyslog -nv -f "$RENDERED_CONFIG" will parse the path as multiple fields and reject or misconfigure the install despite the added test claiming home paths with spaces are preserved. Render a newsyslog-safe escaped path or reject such paths explicitly.

Useful? React with 👍 / 👎.

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in a958d49. The installer now rejects whitespace-containing LOG_DIR values before rendering because newsyslog.conf is whitespace-delimited; README and tests document that explicit limitation.

@EtanHey EtanHey force-pushed the chore/newsyslog branch from 18042ed to a958d49 Compare May 18, 2026 11:09
Copy link
Copy Markdown

@greptile-apps greptile-apps Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Your free trial has ended. If you'd like to continue receiving code reviews, you can add a payment method here.

@EtanHey
Copy link
Copy Markdown
Owner Author

EtanHey commented May 18, 2026

@codex review
@cursor review
@BugBot review

@chatgpt-codex-connector
Copy link
Copy Markdown

Codex Review: Didn't find any major issues. 🎉

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

@EtanHey EtanHey force-pushed the chore/newsyslog branch from a958d49 to a0cbf99 Compare May 18, 2026 11:15
@EtanHey
Copy link
Copy Markdown
Owner Author

EtanHey commented May 18, 2026

@codex review
@cursor review
@BugBot review

Copy link
Copy Markdown

@greptile-apps greptile-apps Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Your free trial has ended. If you'd like to continue receiving code reviews, you can add a payment method here.

@chatgpt-codex-connector
Copy link
Copy Markdown

Codex Review: Didn't find any major issues. Hooray!

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

@EtanHey EtanHey merged commit 8bf7b75 into main May 18, 2026
7 checks passed
Copy link
Copy Markdown

@cursor cursor Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cursor Bugbot has reviewed your changes and found 1 potential issue.

Fix All in Cursor

❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.

Reviewed by Cursor Bugbot for commit a0cbf99. Configure here.

SRC="$SCRIPT_DIR/newsyslog.d/brainlayer.conf"
DST="/etc/newsyslog.d/brainlayer.conf"
OWNER="${BRAINLAYER_LOG_OWNER:-${SUDO_USER:-$(id -un)}}"
GROUP="${BRAINLAYER_LOG_GROUP:-staff}"
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

GROUP variable not validated for whitespace like LOG_DIR

Low Severity

BRAINLAYER_LOG_GROUP is never validated for whitespace or existence. The script explicitly rejects whitespace in LOG_DIR because "newsyslog.conf is whitespace-delimited," but the same reasoning applies to the owner:group field. Unlike OWNER, which is implicitly validated via id -u (which rejects non-existent users), GROUP passes through unchecked. A whitespace-containing value would produce a malformed newsyslog config line where field boundaries shift.

Additional Locations (1)
Fix in Cursor Fix in Web

Reviewed by Cursor Bugbot for commit a0cbf99. Configure here.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant