Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
18 changes: 17 additions & 1 deletion Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -160,9 +160,21 @@ update-deps: ## Update dependencies
fi

# Docker
DOCKER_BASE_IMAGE := ctxos/ctxai-base
DOCKER_BASE_TAG := latest
DOCKER_PLATFORMS := linux/amd64,linux/arm64

docker-base: ## Build base Docker image for current platform
@echo "$(BLUE)Building base Docker image $(DOCKER_BASE_IMAGE):$(DOCKER_BASE_TAG)...$(NC)"
docker buildx build --builder mybuilder -f docker/base/Dockerfile -t $(DOCKER_BASE_IMAGE):$(DOCKER_BASE_TAG) --load .

docker-base-push: ## Build and push multi-arch base Docker image
@echo "$(BLUE)Building and pushing multi-arch base Docker image $(DOCKER_BASE_IMAGE):$(DOCKER_BASE_TAG)...$(NC)"
docker buildx build --builder mybuilder -f docker/base/Dockerfile -t $(DOCKER_BASE_IMAGE):$(DOCKER_BASE_TAG) --platform $(DOCKER_PLATFORMS) --push .
Comment on lines +169 to +173
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

suggestion: Avoid hard-coding the buildx builder name to reduce setup friction.

These targets currently depend on a preconfigured BuildKit builder named mybuilder, so they’ll fail on machines where that builder doesn’t exist.

Consider either:

  • dropping the --builder flag and using the default builder, or
  • making the builder name configurable (e.g. DOCKER_BUILDER ?= mybuilder) and documenting the requirement, or
  • having the Makefile create/configure the builder if it’s missing.

That would make these targets more portable across different environments.

Suggested implementation:

docker-base: ## Build base Docker image for current platform
	@echo "$(BLUE)Building base Docker image $(DOCKER_BASE_IMAGE):$(DOCKER_BASE_TAG)...$(NC)"
	docker buildx build -f docker/base/Dockerfile -t $(DOCKER_BASE_IMAGE):$(DOCKER_BASE_TAG) --load .

docker-base-push: ## Build and push multi-arch base Docker image
	@echo "$(BLUE)Building and pushing multi-arch base Docker image $(DOCKER_BASE_IMAGE):$(DOCKER_BASE_TAG)...$(NC)"
	docker buildx build -f docker/base/Dockerfile -t $(DOCKER_BASE_IMAGE):$(DOCKER_BASE_TAG) --platform $(DOCKER_PLATFORMS) --push .

Comment on lines +169 to +173
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟡 Minor

Avoid hardcoding buildx builder name.

Using --builder mybuilder assumes local pre-configuration and makes these targets fail in clean CI/dev environments.

🔧 Proposed fix
-docker buildx build --builder mybuilder -f docker/base/Dockerfile -t $(DOCKER_BASE_IMAGE):$(DOCKER_BASE_TAG) --load .
+docker buildx build -f docker/base/Dockerfile -t $(DOCKER_BASE_IMAGE):$(DOCKER_BASE_TAG) --load .

-docker buildx build --builder mybuilder -f docker/base/Dockerfile -t $(DOCKER_BASE_IMAGE):$(DOCKER_BASE_TAG) --platform $(DOCKER_PLATFORMS) --push .
+docker buildx build -f docker/base/Dockerfile -t $(DOCKER_BASE_IMAGE):$(DOCKER_BASE_TAG) --platform $(DOCKER_PLATFORMS) --push .
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
docker buildx build --builder mybuilder -f docker/base/Dockerfile -t $(DOCKER_BASE_IMAGE):$(DOCKER_BASE_TAG) --load .
docker-base-push: ## Build and push multi-arch base Docker image
@echo "$(BLUE)Building and pushing multi-arch base Docker image $(DOCKER_BASE_IMAGE):$(DOCKER_BASE_TAG)...$(NC)"
docker buildx build --builder mybuilder -f docker/base/Dockerfile -t $(DOCKER_BASE_IMAGE):$(DOCKER_BASE_TAG) --platform $(DOCKER_PLATFORMS) --push .
docker buildx build -f docker/base/Dockerfile -t $(DOCKER_BASE_IMAGE):$(DOCKER_BASE_TAG) --load .
docker-base-push: ## Build and push multi-arch base Docker image
`@echo` "$(BLUE)Building and pushing multi-arch base Docker image $(DOCKER_BASE_IMAGE):$(DOCKER_BASE_TAG)...$(NC)"
docker buildx build -f docker/base/Dockerfile -t $(DOCKER_BASE_IMAGE):$(DOCKER_BASE_TAG) --platform $(DOCKER_PLATFORMS) --push .
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@Makefile` around lines 169 - 173, The Makefile hardcodes "--builder
mybuilder" in the docker buildx invocations (see the top docker buildx build
line and the docker-base-push target), which breaks on clean CI/dev hosts;
change these invocations to use a configurable variable (e.g.
DOCKER_BUILDX_BUILDER) or omit the flag entirely so the default builder is used,
and only pass "--builder $(DOCKER_BUILDX_BUILDER)" when that variable is set (or
document/ensure builder creation separately); update the docker-base-push and
the other build rule to reference the variable instead of the literal
"mybuilder".


docker-build: ## Build Docker image
@echo "$(BLUE)Building Docker image $(DOCKER_IMAGE):$(DOCKER_TAG)...$(NC)"
docker build -t $(DOCKER_IMAGE):$(DOCKER_TAG) .
docker build -f DockerfileLocal -t $(DOCKER_IMAGE):$(DOCKER_TAG) .

docker-run: ## Run CtxAI with Docker
@echo "$(BLUE)Running CtxAI with Docker...$(NC)"
Expand All @@ -177,6 +189,10 @@ docker-stop: ## Stop Docker Compose services
@echo "$(BLUE)Stopping Docker Compose services...$(NC)"
cd docker/run && docker-compose down

docker-test: ## Run tests inside Docker container
@echo "$(BLUE)Running tests in Docker container...$(NC)"
docker run --rm --entrypoint python $(DOCKER_IMAGE):$(DOCKER_TAG) -m pytest -v

# Documentation
docs: ## Generate documentation
@echo "$(BLUE)Generating documentation...$(NC)"
Expand Down
14 changes: 4 additions & 10 deletions api/message.py
Original file line number Diff line number Diff line change
Expand Up @@ -28,10 +28,8 @@ async def communicate(self, input: dict, request: Request):
attachments = request.files.getlist("attachments")
attachment_paths = []

upload_folder_int = "/ctx0/usr/uploads"
upload_folder_ext = files.get_abs_path(
"usr/uploads"
) # for development environment
upload_folder_int = "/ctx0/usr/uploads"
upload_folder_ext = files.get_abs_path("usr/uploads") # for development environment

if attachments:
os.makedirs(upload_folder_ext, exist_ok=True)
Expand Down Expand Up @@ -60,9 +58,7 @@ async def communicate(self, input: dict, request: Request):

# call extension point, alow it to modify data
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nitpick (typo): Fix the typo in the comment for clarity.

Change alow to allow in the comment:

# call extension point, allow it to modify data
Suggested change
# call extension point, alow it to modify data
# call extension point, allow it to modify data

data = {"message": message, "attachment_paths": attachment_paths}
await extension.call_extensions_async(
"user_message_ui", agent=context.get_agent(), data=data
)
await extension.call_extensions_async("user_message_ui", agent=context.get_agent(), data=data)
message = data.get("message", "")
attachment_paths = data.get("attachment_paths", [])

Expand All @@ -73,7 +69,5 @@ async def communicate(self, input: dict, request: Request):
mq.log_user_message(context, message, attachment_paths, message_id)

return context.communicate(
UserMessage(
message=message, attachments=attachment_paths, id=message_id or ""
)
UserMessage(message=message, attachments=attachment_paths, id=message_id or "")
), context
8 changes: 5 additions & 3 deletions docker/base/Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -5,8 +5,10 @@ FROM kalilinux/kali-rolling
# Set locale to en_US.UTF-8 and timezone to UTC
RUN apt-get update && DEBIAN_FRONTEND=noninteractive apt-get install -y locales tzdata
RUN sed -i -e 's/# \(en_US\.UTF-8 .*\)/\1/' /etc/locale.gen && \
dpkg-reconfigure --frontend=noninteractive locales && \
update-locale LANG=en_US.UTF-8 LANGUAGE=en_US:en LC_ALL=en_US.UTF-8
locale-gen && \
echo 'LANG=en_US.UTF-8' > /etc/default/locale && \
echo 'LANGUAGE=en_US:en' >> /etc/default/locale && \
echo 'LC_ALL=en_US.UTF-8' >> /etc/default/locale
RUN ln -sf /usr/share/zoneinfo/UTC /etc/localtime
RUN echo "UTC" > /etc/timezone
RUN dpkg-reconfigure -f noninteractive tzdata
Expand All @@ -16,7 +18,7 @@ ENV LC_ALL=en_US.UTF-8
ENV TZ=UTC

# Copy contents of the project to /
COPY ./fs/ /
COPY docker/base/fs/ /

# install packages software (split for better cache management)
RUN bash /ins/install_base_packages1.sh
Expand Down
2 changes: 1 addition & 1 deletion docker/run/fs/etc/supervisor/conf.d/supervisord.conf
Original file line number Diff line number Diff line change
Expand Up @@ -60,7 +60,7 @@ killasgroup=true

[program:run_ui]
command=/exe/run_CTX0.sh
environment=
environment=CTX0_DEBUG="true",CTX0_WS_DEBUG="true",LOGLEVEL="DEBUG",UVICORN_LOG_LEVEL="debug"
user=root
stopwaitsecs=60
stdout_logfile=/dev/stdout
Expand Down
51 changes: 18 additions & 33 deletions docker/run/fs/exe/self_update_manager.py
Original file line number Diff line number Diff line change
Expand Up @@ -33,12 +33,8 @@
"CTX0_SELF_UPDATE_HEALTH_URL",
"http://127.0.0.1:80/api/health",
)
DEFAULT_HEALTH_TIMEOUT_SECONDS = int(
os.environ.get("CTX0_SELF_UPDATE_HEALTH_TIMEOUT_SECONDS", "120")
)
DEFAULT_HEALTH_POLL_INTERVAL_SECONDS = float(
os.environ.get("CTX0_SELF_UPDATE_HEALTH_POLL_INTERVAL_SECONDS", "2")
)
DEFAULT_HEALTH_TIMEOUT_SECONDS = int(os.environ.get("CTX0_SELF_UPDATE_HEALTH_TIMEOUT_SECONDS", "120"))
DEFAULT_HEALTH_POLL_INTERVAL_SECONDS = float(os.environ.get("CTX0_SELF_UPDATE_HEALTH_POLL_INTERVAL_SECONDS", "2"))
DEFAULT_BACKUP_DIR = "/root/update-backups"
DEFAULT_BACKUP_CONFLICT_POLICY = "rename"
BACKUP_CONFLICT_POLICIES = {"rename", "overwrite", "fail"}
Expand Down Expand Up @@ -216,8 +212,7 @@ def get_latest_same_major_tag(
]
if not same_major_tags:
raise RuntimeError(
f"No v{current_major}.x release tags are reachable from branch "
f"{branch_ref.rsplit('/', 1)[-1]}."
f"No v{current_major}.x release tags are reachable from branch {branch_ref.rsplit('/', 1)[-1]}."
)
return sort_selector_supported_tags(same_major_tags)[0]

Expand All @@ -238,8 +233,7 @@ def ensure_latest_target_matches_current_major(
target_major = parse_major_version(target_version)
if target_major is None or not is_supported_selector_tag(target_version):
raise RuntimeError(
f"Could not resolve latest on branch {branch} to a supported vX.Y release. "
"Use an explicit tag instead."
f"Could not resolve latest on branch {branch} to a supported vX.Y release. Use an explicit tag instead."
)

if target_major != current_major:
Expand Down Expand Up @@ -398,8 +392,7 @@ def run_command(
logger.log_block("stderr", completed.stderr)
if completed.returncode != 0:
raise RuntimeError(
error_message
or f"Command failed with exit code {completed.returncode}: {' '.join(command)}"
error_message or f"Command failed with exit code {completed.returncode}: {' '.join(command)}"
)
return completed

Expand Down Expand Up @@ -439,8 +432,7 @@ def create_rollback_stash(repo_dir: Path, logger: AttemptLogger) -> str | None:
if not stash_ref or stash_ref == previous_top:
raise RuntimeError("Failed to create the pre-update rollback stash.")
logger.log(
f"Saved local tracked/untracked changes into {stash_ref}. "
"Ignored files stay in place and are not stashed."
f"Saved local tracked/untracked changes into {stash_ref}. Ignored files stay in place and are not stashed."
)
return stash_ref

Expand Down Expand Up @@ -471,9 +463,7 @@ def apply_stash(repo_dir: Path, stash_ref: str, logger: AttemptLogger) -> None:
try:
drop_stash(repo_dir, stash_ref, logger)
except Exception as exc:
logger.log(
f"Rollback stash {stash_ref} was restored but could not be dropped automatically: {exc}"
)
logger.log(f"Rollback stash {stash_ref} was restored but could not be dropped automatically: {exc}")


def clean_repo_worktree(
Expand Down Expand Up @@ -598,9 +588,7 @@ def resolve_requested_target(
current_version=current_version,
target_version=head_short_tag,
)
logger.log(
f"Resolved latest on branch {branch} to commit {head_commit[:7]} ({head_describe})"
)
logger.log(f"Resolved latest on branch {branch} to commit {head_commit[:7]} ({head_describe})")
return {
"requested_tag": LATEST_SELECTOR_TAG,
"effective_tag": head_short_tag,
Expand Down Expand Up @@ -692,7 +680,8 @@ def launch_ui_process(repo_dir: Path, logger: AttemptLogger) -> subprocess.Popen
[
sys.executable,
str(repo_dir / "run_ui.py"),
"--dockerized=true",
"--dockerized=false",
"--development=true",
Comment on lines +683 to +684
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟠 Major

🧩 Analysis chain

🏁 Script executed:

#!/bin/bash
set -euo pipefail

# Verify how docker/development flags are interpreted and consumed
rg -n -C3 'def is_dockerized|def is_development|def initialize|get_arg\(' helpers/runtime.py
rg -n -C3 '--dockerized|--development|is_dockerized\(|is_development\(' run_ui.py docker/run/fs/exe/self_update_manager.py

Repository: FortiShield/CtxAI

Length of output: 1353


🏁 Script executed:

#!/bin/bash
set -euo pipefail

# Check exact lines 683-684 in self_update_manager.py
echo "=== Lines 680-690 in self_update_manager.py ==="
sed -n '680,690p' docker/run/fs/exe/self_update_manager.py | cat -n

# Search for flag usage patterns separately
echo -e "\n=== Flag usage in self_update_manager.py ==="
rg -n 'dockerized|development' docker/run/fs/exe/self_update_manager.py

# Search for how this is invoked from run_ui or other places
echo -e "\n=== References to self_update_manager ==="
rg -n 'self_update_manager' --type py

Repository: FortiShield/CtxAI

Length of output: 3658


🏁 Script executed:

#!/bin/bash
set -euo pipefail

# Get broader context around lines 674 and 683
echo "=== Context around line 674 (prepare_script) ==="
sed -n '670,680p' docker/run/fs/exe/self_update_manager.py | cat -n

echo -e "\n=== Context around line 683 (run_ui) ==="
sed -n '675,695p' docker/run/fs/exe/self_update_manager.py | cat -n

# Search for function definition containing these lines
echo -e "\n=== Function containing these lines ==="
rg -n -B20 'run_command.*prepare_script.*--dockerized=true' docker/run/fs/exe/self_update_manager.py | head -40

# Check run_ui.py to see how it uses these flags
echo -e "\n=== run_ui.py usage of dockerized and development ==="
rg -n 'dockerized|development' run_ui.py | head -20

Repository: FortiShield/CtxAI

Length of output: 2651


🏁 Script executed:

#!/bin/bash
set -euo pipefail

# Check where launch_ui_process is called from
echo "=== Where is launch_ui_process called? ==="
rg -n 'launch_ui_process' docker/run/fs/exe/self_update_manager.py

# Check the broader context of the docker-run-ui command
echo -e "\n=== docker-run-ui command context ==="
rg -n -B5 -A10 'docker-run-ui' docker/run/fs/exe/self_update_manager.py | head -50

# Check if there are any other UI launches and how they set these flags
echo -e "\n=== Other run_ui.py invocations ==="
rg -n 'run_ui.py' --type py

# Check run_ui.py itself to see how it determines if it should run in dev mode
echo -e "\n=== run_ui.py initialization of runtime args ==="
head -50 run_ui.py

Repository: FortiShield/CtxAI

Length of output: 3509


🏁 Script executed:

#!/bin/bash
set -euo pipefail

# Get the full docker_run_ui function to understand the overall flow
echo "=== docker_run_ui function ==="
rg -n -A50 'def docker_run_ui\(' docker/run/fs/exe/self_update_manager.py | head -70

# Check if there are comments or documentation about these flags
echo -e "\n=== Documentation/comments about dockerized flag ==="
rg -n -B2 -A2 'dockerized' docker/run/fs/exe/self_update_manager.py

# Check if run_ui.py is ever launched with --dockerized=true
echo -e "\n=== All run_ui.py invocations with flags ==="
rg -n -A10 'run_ui.py' docker/run/fs/exe/self_update_manager.py

Repository: FortiShield/CtxAI

Length of output: 3381


Set --dockerized=true to match the actual runtime environment.

The self_update_manager runs inside a docker container (as indicated by its path in docker/run/fs/exe/). Setting --dockerized=false when launching the UI creates a runtime mismatch that can break environment-specific behavior, path handling, and feature detection. Additionally, line 674 correctly uses --dockerized=true for prepare.py, but line 683 contradicts this for the UI launcher.

Since is_development() in runtime.py returns not is_dockerized(), the explicit --development=true at line 684 is redundant once the dockerized flag is fixed.

🔧 Proposed fix
-            "--dockerized=false",
-            "--development=true",
+            "--dockerized=true",
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
"--dockerized=false",
"--development=true",
"--dockerized=true",
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@docker/run/fs/exe/self_update_manager.py` around lines 683 - 684, The UI
launcher in self_update_manager.py currently passes "--dockerized=false" and
"--development=true"; change "--dockerized=false" to "--dockerized=true" to
reflect the container runtime (matching the prepare.py invocation) and remove
the redundant "--development=true" flag because is_development() derives its
value from is_dockerized(); update the argument list where the UI is launched so
it passes "--dockerized=true" and omit "--development=true".

"--port=80",
"--host=0.0.0.0",
Comment on lines 680 to 686
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

issue (bug_risk): Re-evaluate --dockerized=false in a Docker-managed UI process; it likely should stay dockerized.

Passing --dockerized=false here will make runtime.is_dockerized() return false for a process that’s actually running in a container, which can change paths and config (e.g., upload/base directories) and potentially break container-specific behavior. If the goal is just to enable dev logging, keep --dockerized=true and only add --development=true so the process still behaves as containerized code.

],
Comment on lines 680 to 687
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

security (python.lang.security.audit.dangerous-subprocess-use-audit): Detected subprocess function 'Popen' without a static string. If this data can be controlled by a malicious actor, it may be an instance of command injection. Audit the use of this call to ensure it is not controllable by an external resource. You may consider using 'shlex.escape()'.

Source: opengrep

Expand Down Expand Up @@ -873,7 +862,10 @@ def execute_pending_update(
"Git checkout completed but the repository commit does not match the requested target. "
f"Expected {resolved_target['expected_commit']}, got {current_info['commit']}."
)
if resolved_target.get("expected_short_tag") and current_info["short_tag"] != resolved_target["expected_short_tag"]:
if (
resolved_target.get("expected_short_tag")
and current_info["short_tag"] != resolved_target["expected_short_tag"]
):
raise RuntimeError(
"Git checkout completed but the repository version does not match the requested tag. "
f"Expected {resolved_target['expected_short_tag']}, got {current_info['short_tag']}."
Expand Down Expand Up @@ -908,9 +900,7 @@ def execute_pending_update(
try:
drop_stash(REPO_DIR, stash_ref, logger)
except Exception as exc:
logger.log(
f"Temporary rollback stash {stash_ref} could not be dropped automatically: {exc}"
)
logger.log(f"Temporary rollback stash {stash_ref} could not be dropped automatically: {exc}")
return updated_process

logger.log(f"Updated UI failed health check, rolling back: {details}")
Expand Down Expand Up @@ -939,8 +929,7 @@ def execute_pending_update(
record_result(
status="rolled_back",
message=(
"Updated version failed its health check and the previous version was restored. "
f"Reason: {details}"
f"Updated version failed its health check and the previous version was restored. Reason: {details}"
),
request_data=request_data,
source_info=source_info,
Expand All @@ -955,9 +944,7 @@ def execute_pending_update(
terminate_process(rollback_process)
record_result(
status="rollback_failed",
message=(
"Updated version failed its health check and rollback also failed to become healthy."
),
message=("Updated version failed its health check and rollback also failed to become healthy."),
request_data=request_data,
source_info=source_info,
current_version=source_info["short_tag"],
Expand Down Expand Up @@ -1159,9 +1146,7 @@ def docker_run_ui() -> int:
requested_branch=requested_branch,
requested_tag=requested_tag,
):
logger.log(
"Requested tag already matches the installed version, skipping file replacement."
)
logger.log("Requested tag already matches the installed version, skipping file replacement.")
record_result(
status="skipped",
message="Requested tag already matches the installed version.",
Expand Down
Loading