Skip to content

Winston/agents#403

Open
ElasticBottle wants to merge 10 commits intomainfrom
winston/agents
Open

Winston/agents#403
ElasticBottle wants to merge 10 commits intomainfrom
winston/agents

Conversation

@ElasticBottle
Copy link
Collaborator

No description provided.

Copilot AI review requested due to automatic review settings March 2, 2026 00:57
@pkg-pr-new
Copy link

pkg-pr-new bot commented Mar 2, 2026

Open in StackBlitz

pnpm add https://pkg.pr.new/@rectangular-labs/emails@403
pnpm add https://pkg.pr.new/@rectangular-labs/loro-file-system@403
pnpm add https://pkg.pr.new/@rectangular-labs/result@403

commit: daabe76

@gemini-code-assist
Copy link

Summary of Changes

Hello, I'm Gemini Code Assist1! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!

This pull request introduces a significant architectural shift in the AI agent system, moving towards a more robust and modular design. The core change involves implementing an orchestrator agent that intelligently delegates tasks to specialized subagents, such as a Strategy Advisor and a Writer. This refactor aims to improve the clarity, maintainability, and scalability of AI-driven workflows. Concurrently, a new evaluation framework has been integrated, allowing for systematic assessment and scoring of AI-generated content and strategies against predefined fixtures. This enhancement provides a structured approach to measure and improve agent performance, ensuring higher quality and more consistent outputs.

Highlights

  • AI Agent Architecture Refactor: The AI agent system has undergone a significant architectural overhaul, transitioning from a monolithic strategist/writer agent to a modular orchestrator-subagent model. This introduces a dedicated orchestrator agent that delegates tasks to specialized subagents (Strategy Advisor, Writer) and handles interactive tools directly.
  • New AI Agent Evaluation Framework: A comprehensive evaluation framework has been implemented for AI agent outputs. This includes new admin routes and backend logic for defining content and strategy fixtures, generating outputs from these fixtures, and scoring them using a combination of deterministic checks and LLM-as-a-Judge (G-Eval pattern).
  • Consolidated Data Access and Tooling: Multiple individual AI tools for file management, strategy details, and content creation have been consolidated into a new data-access-tool. This streamlines data interaction for AI agents and improves consistency.
  • Enhanced Writer Agent Pipeline: The Writer agent now operates through a structured pipeline (research, planning, writing, internal links, images, review loop), with detailed instructions and improved handling of external/internal linking, image generation, and self-correction based on review feedback.
  • Removal of Planner Workflow: The dedicated seo-plan-keyword task and its associated workflow (SeoPlannerWorkflow) have been removed, as planning capabilities are now integrated directly into the new Writer agent's pipeline.
  • Dependency Updates: Several core dependencies, including @t3-oss/env-core and @types/node, have been updated across various applications and packages to their latest versions.

🧠 New Feature in Public Preview: You can now enable Memory to help Gemini Code Assist learn from your team's feedback. This makes future code reviews more consistent and personalized to your project's style. Click here to enable Memory in your admin console.

Changelog
  • .cursor/rules/base-rules.mdc
    • Removed the base rules documentation file.
  • AGENTS.md
    • Updated the 'Common Commands' section to direct users to the root package.json.
    • Removed the 'Repository Layout' section.
    • Added a new code convention for AI tool definitions, specifying inline schema usage.
    • Added a new code convention advising against exporting code out of a file by default.
    • Removed the 'Domain Context' and 'Package Generation' sections.
  • apps/seo-contact/package.json
    • Updated @t3-oss/env-core dependency from ^0.13.8 to ^0.13.10.
    • Updated @types/node dependency from ^24.10.1 to ^25.2.3.
  • apps/seo-www/package.json
    • Updated @t3-oss/env-core dependency from ^0.13.8 to ^0.13.10.
    • Updated @types/node dependency from ^24.10.1 to ^25.2.3.
  • apps/seo/package.json
    • Updated @t3-oss/env-core dependency from ^0.13.8 to ^0.13.10.
    • Updated @types/node dependency from ^24.10.1 to ^25.2.3.
  • apps/seo/src/routeTree.gen.ts
    • Refactored the admin route from a layout route (_authed/admin/route) to an index route (_authed/admin/index).
    • Updated imports and route definitions to reflect the change in the admin route structure.
  • apps/seo/src/routes/_authed/$organizationSlug/-components/project-chat-panel.tsx
    • Removed imports for Queue, DialogDrawer, and Task components, simplifying the UI dependencies.
    • Replaced CreatePlanToolPart with DeleteDataToolPart to handle data deletion requests.
    • Integrated addToolApprovalResponse for managing tool approval flows within the chat.
    • Removed todoSnapshot state and related effects, streamlining chat state management.
  • apps/seo/src/routes/_authed/admin/index.tsx
    • Added a new admin panel component for managing AI agent evaluation tasks.
  • apps/seo/src/routes/_authed/admin/route.tsx
    • Removed the admin route component.
  • apps/seo/src/server.ts
    • Removed the export of SeoPlannerWorkflow.
  • apps/seo/wrangler.jsonc
    • Removed seo-planner workflow bindings from both default and production environments.
  • apps/www/package.json
    • Updated @t3-oss/env-core dependency from ^0.13.8 to ^0.13.10.
    • Updated @types/node dependency from ^24.10.1 to ^25.2.3.
  • packages/api-core/package.json
    • Updated @types/node dependency from ^24.10.1 to ^25.2.3.
  • packages/api-seo/package.json
    • Updated @t3-oss/env-core dependency from ^0.13.8 to ^0.13.10.
    • Updated @types/node dependency from ^24.10.1 to ^25.2.3.
  • packages/api-seo/src/context.ts
    • Removed seoPlannerWorkflow from the createApiContext function.
  • packages/api-seo/src/eval/content/fixtures/index.ts
    • Added a new file defining content evaluation fixtures for testing AI writer outputs.
  • packages/api-seo/src/eval/content/score.ts
    • Added a new file implementing content evaluation scoring logic, including deterministic checks and LLM-as-a-Judge.
  • packages/api-seo/src/eval/strategy/fixtures/index.ts
    • Added a new file defining strategy evaluation fixtures for testing AI strategy advisor outputs.
  • packages/api-seo/src/eval/strategy/score.ts
    • Added a new file implementing strategy evaluation scoring logic using LLM-as-a-Judge.
  • packages/api-seo/src/eval/types.ts
    • Added a new file defining shared types for the AI agent evaluation framework.
  • packages/api-seo/src/lib/ai/agents/orchestrator.ts
    • Added a new file defining the AI orchestrator agent, responsible for delegating tasks to subagents.
  • packages/api-seo/src/lib/ai/agents/strategy-advisor.ts
    • Added a new file defining the AI strategy advisor agent, specializing in SEO/GEO analysis and planning.
  • packages/api-seo/src/lib/ai/agents/writer.ts
    • Added a new file defining the AI writer agent and its multi-phase content generation pipeline.
  • packages/api-seo/src/lib/ai/arktype-json-schema.ts
    • Removed the utility file for transforming ArkType schemas to AI JSON schemas.
  • packages/api-seo/src/lib/ai/instructions/orchestrator.ts
    • Added a new file defining the system instructions for the orchestrator agent.
  • packages/api-seo/src/lib/ai/instructions/strategy-advisor.ts
    • Added a new file defining the system instructions for the strategy advisor agent.
  • packages/api-seo/src/lib/ai/instructions/writer.ts
    • Added a new file defining the system instructions for the writer agent, including detailed content philosophy and structural standards.
  • packages/api-seo/src/lib/ai/tools/ask-question-tool.ts
    • Added a new file defining a tool for AI agents to ask clarification questions to users.
  • packages/api-seo/src/lib/ai/tools/create-article-tool.ts
    • Removed the tool for creating article drafts.
  • packages/api-seo/src/lib/ai/tools/data-access-tool.ts
    • Added a new file defining tools for listing, searching, reading, updating, and soft-deleting strategies and content drafts.
  • packages/api-seo/src/lib/ai/tools/data-analysis-agent-tool.ts
    • Removed the data analysis agent tool.
  • packages/api-seo/src/lib/ai/tools/dataforseo-tool.ts
    • Updated input schemas for DataForSEO tools to use jsonSchema directly.
    • Added inputExamples to DataForSEO tool definitions for better clarity and usage guidance.
  • packages/api-seo/src/lib/ai/tools/file-tool.ts
    • Removed the file management tools (ls, cat, rm, write_file).
  • packages/api-seo/src/lib/ai/tools/google-search-console-tool.ts
    • Updated input schema for the Google Search Console query tool to use jsonSchema directly.
    • Added inputExamples to the GSC tool definition for better clarity and usage guidance.
  • packages/api-seo/src/lib/ai/tools/image-tools.ts
    • Updated input schemas for image generation, stock image search, and screenshot capture tools to use jsonSchema directly.
    • Added inputExamples to image tool definitions.
    • Refactored image media type normalization logic.
  • packages/api-seo/src/lib/ai/tools/internal-links-tool.ts
    • Updated input schema for the internal links tool to use jsonSchema directly.
    • Added inputExamples to the internal links tool definition.
  • packages/api-seo/src/lib/ai/tools/planner-tools.ts
    • Removed the planner tools (ask_questions, create_plan).
  • packages/api-seo/src/lib/ai/tools/screenshot-script.ts
    • Removed the screenshot script file.
  • packages/api-seo/src/lib/ai/tools/settings-tools.ts
    • Updated input schemas for settings management tools to use jsonSchema directly.
    • Added inputExamples to settings tool definitions.
  • packages/api-seo/src/lib/ai/tools/skill-tools.ts
    • Removed the skill management tools (read_skills, use_skills).
  • packages/api-seo/src/lib/ai/tools/strategy-tools.ts
    • Removed the strategy details tool.
  • packages/api-seo/src/lib/ai/tools/todo-tool.ts
    • Removed the todo management tool.
  • packages/api-seo/src/lib/ai/tools/utils.ts
    • Removed general AI tool utility functions and types.
  • packages/api-seo/src/lib/ai/tools/web-tools.ts
    • Updated input schemas for web fetch and web search tools to use jsonSchema directly.
    • Added inputExamples to web tool definitions.
  • packages/api-seo/src/lib/ai/utils/agent-telemetry.ts
    • Added a new file for summarizing AI agent usage, token counts, and estimated costs.
  • packages/api-seo/src/lib/ai/utils/auth-init.ts
    • Added a new utility for initializing authentication handlers for workflows.
  • packages/api-seo/src/lib/ai/utils/format-business-background.ts
    • Adjusted the formatting of business background information.
  • packages/api-seo/src/lib/ai/utils/project-context.ts
    • Added a new utility for building standardized project context blocks for AI system prompts.
  • packages/api-seo/src/lib/ai/utils/review.ts
    • Added a new file implementing LLM-as-a-Judge based article review logic.
  • packages/api-seo/src/lib/ai/utils/store-optimized-or-original-image.ts
    • Updated the getImagesBinding function to handle potential import errors and return null if the binding is unavailable.
  • packages/api-seo/src/lib/ai/utils/wrapped-language-model.ts
    • Added a new utility for wrapping language models with middleware to inject tool input examples.
  • packages/api-seo/src/lib/ai/writer-agent.ts
    • Removed the writer agent file, replaced by the new agents/writer.ts.
  • packages/api-seo/src/lib/content/write-content-draft.ts
    • Removed the logic that created an seo-plan-keyword task when a draft was set to 'suggested' status.
  • packages/api-seo/src/lib/dataforseo/utils.ts
    • Exported new types for DataForSEO fetch arguments (FetchKeywordSuggestionsArgs, FetchKeywordsOverviewArgs, FetchRankedKeywordsForSiteArgs, FetchRankedPagesForSiteArgs, FetchSerpArgs).
  • packages/api-seo/src/lib/task.ts
    • Removed the seo-plan-keyword task type from the createTask function.
  • packages/api-seo/src/lib/workspace/constants.ts
    • Updated the default custom instructions for writing settings to an empty string.
  • packages/api-seo/src/lib/workspace/workflow.constant.ts
    • Significantly expanded and refined the ARTICLE_TYPE_TO_WRITER_RULE definitions for various article types, including target length, intent, tone, structure, and key rules.
    • Updated the DEFAULT_BRAND_VOICE to be more concise and focused on SEO/GEO optimization.
  • packages/api-seo/src/routes/admin.ts
    • Added new admin routes for managing AI agent evaluation fixtures, generation, and scoring.
    • Implemented runEvalGenerationJob to execute content and strategy generation tasks for evaluation.
    • Added putEvalGenerationStatus and getEvalGenerationKey for managing evaluation job status in KV storage.
    • Included toJsonSafe utility for serializing potentially problematic data types in JSON outputs.
    • Added assertFluidpostsEmail for access control to admin routes.
  • packages/api-seo/src/routes/chat.sendMessage.ts
    • Refactored chat message handling to use the new createOrchestrator agent.
    • Removed direct calls to createStrategistAgent and createWriterAgent.
    • Integrated summarizeAgentStep for logging orchestrator step telemetry.
    • Removed todoSnapshot related logic from the chat panel.
  • packages/api-seo/src/routes/task.ts
    • Removed the seo-plan-keyword task type from the task status retrieval logic.
  • packages/api-seo/src/types.ts
    • Updated SeoChatMessage type definition to use InferAgentUIMessage with createOrchestrator.
    • Removed seoPlannerWorkflow from the InitialContext interface.
  • packages/api-seo/src/workflows/index.ts
    • Removed seoPlannerWorkflow binding from the createWorkflows function.
  • packages/api-seo/src/workflows/onboarding-workflow.ts
    • Updated createWebToolsWithMetadata import to createWebTools.
    • Modified schema handling for businessBackgroundSchema and brandVoice extraction to use jsonSchema directly.
  • packages/api-seo/src/workflows/planner-workflow.ts
    • Removed the entire planner workflow file.
  • packages/api-seo/src/workflows/strategy-phase-generation-workflow.ts
    • Refactored to use createStrategyAdvisorAgent for generating phase suggestions.
    • Integrated summarizeAgentInvocation for logging agent telemetry.
    • Updated GSC integration to use createWorkflowAuth for authentication.
    • Modified schema handling for strategy phase suggestions to use jsonSchema directly.
  • packages/api-seo/src/workflows/strategy-suggestions-workflow.ts
    • Refactored to use createStrategyAdvisorAgent for generating strategy suggestions.
    • Integrated summarizeAgentInvocation for logging agent telemetry.
    • Updated GSC integration to use createWorkflowAuth for authentication.
    • Modified schema handling for strategy suggestions to use jsonSchema directly.
  • packages/api-seo/src/workflows/writer-workflow.ts
    • Refactored to use the new createWriterPipeline for article generation.
    • Removed direct calls to generateText and manual review loop logic.
    • Integrated summarizeAgentInvocation for logging agent telemetry.
    • Simplified article type inference logic.
  • packages/api-user-vm/package.json
    • Updated @t3-oss/env-core dependency from ^0.13.8 to ^0.13.10.
    • Updated @types/node dependency from ^24.10.1 to ^25.2.3.
  • packages/auth/package.json
    • Updated @t3-oss/env-core dependency from ^0.13.8 to ^0.13.10.
    • Updated @types/node dependency from ^24.10.1 to ^25.2.3.
  • packages/content/package.json
    • Updated @types/node dependency from ^24.10.1 to ^25.2.3.
  • packages/core/src/schema/arktype-json-schema-transformer.ts
    • Removed the ArkType JSON schema transformer utility.
  • packages/core/src/schemas/project-parsers.ts
    • Added JsonSchema exports for businessBackgroundSchema, imageSettingsSchema, writingSettingsSchema, publishingSettingsSchema, and authorSettingsSchema.
  • packages/core/src/schemas/task-parsers.ts
    • Removed seoPlanKeywordTaskInputSchema and seoPlanKeywordTaskOutputSchema.
  • packages/dataforseo/package.json
    • Updated @t3-oss/env-core dependency from ^0.13.8 to ^0.13.10.
    • Updated @types/node dependency from ^24.10.1 to ^25.2.3.
  • packages/dataforseo/src/index.ts
    • Exported new types for DataForSEO fetch arguments (FetchRankedKeywordsForSiteArgs, FetchRankedPagesForSiteArgs, FetchKeywordSuggestionsArgs, FetchKeywordsOverviewArgs, FetchSerpArgs).
  • packages/db/package.json
    • Updated @t3-oss/env-core dependency from ^0.13.8 to ^0.13.10.
    • Updated @types/node dependency from ^24.10.1 to ^25.2.3.
  • packages/db/src/operations/seo/content-operations.ts
    • Added softDeleteDraft function to mark content drafts as deleted.
  • packages/db/src/operations/seo/strategy-operations.ts
    • Added softDeleteStrategy function to mark strategies as dismissed and unlink associated content drafts.
  • packages/emails/package.json
    • Updated @types/node dependency from ^24.10.1 to ^25.2.3.
  • packages/google-apis/package.json
    • Updated @t3-oss/env-core dependency from ^0.13.8 to ^0.13.10.
    • Updated @types/node dependency from ^24.10.1 to ^25.2.3.
  • packages/task/package.json
    • Updated @t3-oss/env-core dependency from ^0.13.8 to ^0.13.10.
    • Updated @types/node dependency from ^24.10.1 to ^25.2.3.
Activity
  • No human activity has been recorded on this pull request yet.
Using Gemini Code Assist

The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.

Invoking Gemini

You can request assistance from Gemini at any point by creating a comment using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands on the current page.

Feature Command Description
Code Review /gemini review Performs a code review for the current pull request in its current state.
Pull Request Summary /gemini summary Provides a summary of the current pull request in its current state.
Comment @gemini-code-assist Responds in comments when explicitly tagged, both in pull request comments and review comments.
Help /gemini help Displays a list of available commands.

Customization

To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.

Limitations & Feedback

Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for Github and other Google products, sign up here.

You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension.

Footnotes

  1. Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution.

Copy link

@gemini-code-assist gemini-code-assist bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request introduces a significant and well-executed refactoring of the AI agent architecture, moving from a monolithic agent to a more structured, hierarchical system with an orchestrator and specialized sub-agents. Key changes include the removal of the SeoPlannerWorkflow and the introduction of a sophisticated, multi-phase writer agent pipeline. A comprehensive evaluation framework for content and strategy generation has also been added, which is a great step towards ensuring AI output quality. The tool definitions have been updated to use the ai SDK's jsonSchema helper, improving consistency. My review has identified a bug in a utility function and a potential UI improvement for data refetching after a destructive action. Overall, this is a high-quality refactoring that greatly improves the agent system's design and maintainability.

.join("")}`
}
- Sample Competitor's site: ${background.competitorsWebsites.length === 0 ? "None Present" : `${background.competitorsWebsites.length} Present`}
- Competitor websites: ${background.competitorsWebsites.length === 0 ? "None Present" : `${background.competitorsWebsites.map((website, index) => `\n - ${index + 1}. ${website}`).join("")}`}

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

high

The competitorsWebsites array contains objects of shape { url: string }. The current implementation ...map((website, index) => ... ${website}) will result in [object Object] being rendered. You should access the url property of the website object.

Suggested change
- Competitor websites: ${background.competitorsWebsites.length === 0 ? "None Present" : `${background.competitorsWebsites.map((website, index) => `\n - ${index + 1}. ${website}`).join("")}`}
- Competitor websites: ${background.competitorsWebsites.length === 0 ? "None Present" : `${background.competitorsWebsites.map((website, index) => `\n - ${index + 1}. ${website.url}`).join("")}`}

- `pnpm db:mp:preview` - Apply migrations to preview database
- `pnpm db:mp:production` - Apply migrations to production database
- `pnpm db:studio` - Open Drizzle Studio
check the root level package.json for the list of supported commands

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

While pointing to package.json for commands is more maintainable, removing the explicit list of common commands, database commands, and repository layout makes this document less helpful for developers trying to quickly get an overview of the project. Consider restoring some of the key commands and the repository layout summary to improve developer onboarding and quick reference.

Comment on lines +306 to 422
function DeleteDataToolPart({
part,
onApprove,
onReject,
onRespond,
}: {
part: ChatToolPart;
onApprove: () => void;
onReject: () => void;
part: DeleteDataToolPart;
onRespond: (args: { id: string; approved: boolean; reason?: string }) => void;
}) {
const [open, setOpen] = useState(false);
if (
part.type !== "tool-create_plan" ||
part.state === "input-streaming" ||
part.state === "output-error"
)
return null;
const plan = part.input;
const title = plan.name?.trim() || "Proposed plan";
const overview = plan.overview?.trim();
const markdown = plan.plan ?? "";
const [denialReason, setDenialReason] = useState("");

return (
<div className="mb-4 w-full rounded-md border bg-background p-4">
<DialogDrawer
onOpenChange={setOpen}
open={open}
trigger={
<button
className={cn(
"w-full text-left",
"rounded-md focus-visible:outline-none focus-visible:ring-2 focus-visible:ring-ring",
)}
type="button"
>
<div className="flex items-start justify-between gap-3">
<div className="min-w-0">
<div className="truncate font-medium text-sm">{title}</div>
{overview ? (
<div className="mt-1 line-clamp-2 text-muted-foreground text-xs">
{overview}
</div>
) : (
<div className="mt-1 text-muted-foreground text-xs">
Click to view details
</div>
)}
</div>
<div className="shrink-0 text-muted-foreground text-xs">
{part.state === "output-available" ? "Ready" : "Running"}
</div>
</div>
</button>
}
>
<DialogDrawerHeader className="space-y-1">
<DialogDrawerTitle>{title}</DialogDrawerTitle>
{overview ? (
<div className="text-muted-foreground text-sm">{overview}</div>
) : null}
</DialogDrawerHeader>
if (part.type !== "tool-delete_existing_data") return null;

<div className="max-h-[60vh] overflow-auto px-4 pb-4">
<Response>{markdown}</Response>
</div>
const input = part.input;
if (!input) {
return (
<div className="mb-4 w-full rounded-md border bg-background p-4 text-sm">
Delete request is missing input parameters.
</div>
);
}

<DialogDrawerFooter className="gap-2">
<Button
disabled={part.state !== "output-available"}
onClick={() => {
setOpen(false);
onReject();
}}
size="sm"
type="button"
variant="outline"
>
Reject
</Button>
<Button
disabled={part.state !== "output-available"}
onClick={() => {
setOpen(false);
onApprove();
}}
size="sm"
type="button"
>
Approve
</Button>
</DialogDrawerFooter>
</DialogDrawer>
const entityLabel =
input.entityType === "strategy" ? "strategy" : "content draft";
const isApprovalRequested = part.state === "approval-requested";

<div className="mt-3 flex justify-end gap-2">
<Button
disabled={part.state !== "output-available"}
onClick={onReject}
size="sm"
type="button"
variant="outline"
>
Reject
</Button>
<Button
disabled={part.state !== "output-available"}
onClick={onApprove}
size="sm"
type="button"
>
Approve
</Button>
if (part.state === "input-streaming" || part.state === "input-available") {
return (
<Shimmer className="text-sm" key={part.toolCallId}>
Preparing delete request
</Shimmer>
);
}

return (
<div className="mb-4 w-full rounded-md border bg-background p-4">
<div className="space-y-2">
<div className="font-medium text-sm">Delete {entityLabel}</div>
<div className="text-muted-foreground text-sm">
ID: <code>{input.id}</code>
</div>
{input.reason ? (
<div className="text-muted-foreground text-sm">
Reason: {input.reason}
</div>
) : null}
{input.entityType === "strategy" ? (
<div className="text-amber-700 text-sm">
This only removes the strategy. Content previously linked to it will
remain in your project.
</div>
) : (
<div className="text-amber-700 text-sm">
Deleting this content also unpublishes it from your site.
</div>
)}

{part.state === "output-available" &&
part.output &&
typeof part.output === "object" &&
"success" in part.output &&
part.output.success === true ? (
<div className="text-green-700 text-sm">
Deletion approved and completed.
</div>
) : null}
{part.state === "output-error" ? (
<div className="text-destructive text-sm">
Delete failed: {part.errorText}
</div>
) : null}
{part.state === "output-denied" ? (
<div className="text-orange-700 text-sm">
Delete request denied.
{part.approval.reason ? ` Reason: ${part.approval.reason}` : ""}
</div>
) : null}
</div>

{isApprovalRequested ? (
<div className="mt-4 space-y-2">
<Input
onChange={(event) => setDenialReason(event.target.value)}
placeholder="Optional denial reason"
value={denialReason}
/>
<div className="flex justify-end gap-2">
<Button
onClick={() =>
onRespond({
id: part.approval.id,
approved: false,
reason: denialReason.trim() || undefined,
})
}
size="sm"
type="button"
variant="outline"
>
Deny
</Button>
<Button
onClick={() =>
onRespond({
id: part.approval.id,
approved: true,
})
}
size="sm"
type="button"
>
Approve delete
</Button>
</div>
</div>
) : null}
</div>
);
}

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

After a successful deletion via the DeleteDataToolPart, the UI should reflect this change by refetching the relevant data (e.g., the list of strategies or content drafts). Currently, the component shows a success message but doesn't seem to trigger any data invalidation. Consider using useQueryClient from @tanstack/react-query to invalidate the queries for the deleted entity type upon successful deletion. This will ensure the UI stays in sync with the backend state.

Copy link

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Refactors the SEO product’s AI/chat and workflow architecture toward an orchestrator + subagent model, while removing the legacy “planner/plan-keyword” workflow path. Also tightens AI tool schema wiring around jsonSchema(...), adds new evaluation scaffolding, introduces soft-delete DB operations, and bumps shared dependency versions across the monorepo.

Changes:

  • Replace legacy strategist/writer agent + planner workflow with a ToolLoopAgent orchestrator and a Strategy Advisor subagent, and update workflows/routes accordingly.
  • Introduce JSON schema exports for project settings (for AI tool validation) and remove ArkType→JSON-schema transformer utilities.
  • Add new DB soft-delete operations for SEO strategies/drafts and add eval framework files for content/strategy scoring.

Reviewed changes

Copilot reviewed 82 out of 83 changed files in this pull request and generated 8 comments.

Show a summary per file
File Description
packages/task/package.json Bumps @types/node and @t3-oss/env-core.
packages/google-apis/package.json Bumps @types/node and @t3-oss/env-core.
packages/emails/package.json Bumps @types/node.
packages/db/src/operations/seo/strategy-operations.ts Adds softDeleteStrategy DB operation.
packages/db/src/operations/seo/content-operations.ts Adds softDeleteDraft DB operation.
packages/db/package.json Bumps @types/node and @t3-oss/env-core.
packages/dataforseo/src/index.ts Extracts exported request arg types for DataForSEO helpers.
packages/dataforseo/package.json Bumps @types/node and @t3-oss/env-core.
packages/core/src/schemas/task-parsers.ts Removes seo-plan-keyword task schemas from unions.
packages/core/src/schemas/project-parsers.ts Adds explicit JSON Schemas for settings; adjusts ArkType validators.
packages/core/src/schema/arktype-json-schema-transformer.ts Deletes ArkType JSON schema transformer utility.
packages/content/package.json Bumps @types/node.
packages/auth/package.json Bumps @types/node and @t3-oss/env-core.
packages/api-user-vm/package.json Bumps @types/node and @t3-oss/env-core.
packages/api-seo/src/workflows/strategy-suggestions-workflow.ts Switches suggestion generation to Strategy Advisor agent + telemetry/auth helper.
packages/api-seo/src/workflows/onboarding-workflow.ts Moves AI outputs to jsonSchema(...) and updates web tools factory usage.
packages/api-seo/src/workflows/index.ts Removes planner workflow binding/export.
packages/api-seo/src/types.ts Updates chat message typing to agent-based message inference; removes planner workflow typing.
packages/api-seo/src/routes/task.ts Removes planner workflow instance lookup for seo-plan-keyword.
packages/api-seo/src/routes/chat.sendMessage.ts Switches chat streaming to orchestrator agent; updates step logging.
packages/api-seo/src/lib/workspace/constants.ts Adjusts default writing settings (custom instructions now empty).
packages/api-seo/src/lib/task.ts Removes seo-plan-keyword workflow creation.
packages/api-seo/src/lib/dataforseo/utils.ts Reuses exported DataForSEO request arg types to reduce duplication.
packages/api-seo/src/lib/content/write-content-draft.ts Removes automatic seo-plan-keyword task creation on “suggested” status.
packages/api-seo/src/lib/ai/writer-agent.ts Deletes legacy writer agent implementation.
packages/api-seo/src/lib/ai/utils/wrapped-language-model.ts Adds wrapper for injecting tool input examples via middleware.
packages/api-seo/src/lib/ai/utils/store-optimized-or-original-image.ts Makes Cloudflare binding resolution dynamic/optional.
packages/api-seo/src/lib/ai/utils/review.ts Adds Gemini-based article review utility with structured output.
packages/api-seo/src/lib/ai/utils/project-context.ts Adds shared project-context formatter utility.
packages/api-seo/src/lib/ai/utils/format-business-background.ts Updates business background formatting for prompts.
packages/api-seo/src/lib/ai/utils/auth-init.ts Adds workflow auth initialization helper.
packages/api-seo/src/lib/ai/utils/agent-telemetry.ts Adds token/cost telemetry summarizers for agent invocations.
packages/api-seo/src/lib/ai/tools/web-tools.ts Reworks tool schemas to jsonSchema(...) + input examples; renames factory.
packages/api-seo/src/lib/ai/tools/utils.ts Deletes tool-definition/skills formatting helper.
packages/api-seo/src/lib/ai/tools/todo-tool.ts Deletes legacy todo tool.
packages/api-seo/src/lib/ai/tools/strategy-tools.ts Deletes legacy strategy tools wrapper.
packages/api-seo/src/lib/ai/tools/skill-tools.ts Deletes legacy “skills” meta-tools.
packages/api-seo/src/lib/ai/tools/settings-tools.ts Reworks schemas to jsonSchema(...) and adds wrapped OpenAI model usage.
packages/api-seo/src/lib/ai/tools/screenshot-script.ts Deletes local screenshot script.
packages/api-seo/src/lib/ai/tools/planner-tools.ts Deletes legacy planner tools.
packages/api-seo/src/lib/ai/tools/internal-links-tool.ts Reworks internal-links tool to jsonSchema(...) and project-derived locale.
packages/api-seo/src/lib/ai/tools/image-tools.ts Reworks image tools schemas; normalizes media types; input examples.
packages/api-seo/src/lib/ai/tools/google-search-console-tool.ts Reworks GSC tool schema to jsonSchema(...) + input examples.
packages/api-seo/src/lib/ai/tools/file-tool.ts Deletes legacy file workspace tool.
packages/api-seo/src/lib/ai/tools/dataforseo-tool.ts Reworks DataForSEO tools to jsonSchema(...) + examples + defaults.
packages/api-seo/src/lib/ai/tools/data-analysis-agent-tool.ts Deletes legacy data analysis subagent tool.
packages/api-seo/src/lib/ai/tools/create-article-tool.ts Deletes legacy create-article tool.
packages/api-seo/src/lib/ai/tools/ask-question-tool.ts Adds ask-questions tool using jsonSchema(...) + examples.
packages/api-seo/src/lib/ai/strategist-agent.ts Deletes legacy strategist agent.
packages/api-seo/src/lib/ai/instructions/strategy-advisor.ts Adds Strategy Advisor system prompt/instructions.
packages/api-seo/src/lib/ai/instructions/orchestrator.ts Adds orchestrator system prompt/instructions.
packages/api-seo/src/lib/ai/arktype-json-schema.ts Deletes ArkType→AI JSON schema adapter.
packages/api-seo/src/lib/ai/agents/strategy-advisor.ts Adds ToolLoopAgent-based Strategy Advisor agent construction.
packages/api-seo/src/lib/ai/agents/orchestrator.ts Adds ToolLoopAgent-based orchestrator with advise/write delegation tools.
packages/api-seo/src/eval/types.ts Adds shared eval framework types.
packages/api-seo/src/eval/strategy/score.ts Adds LLM-judge strategy scorer + pairwise comparison.
packages/api-seo/src/eval/strategy/fixtures/index.ts Adds strategy eval fixtures.
packages/api-seo/src/eval/content/score.ts Adds mixed deterministic + LLM-judge content scorer.
packages/api-seo/src/context.ts Removes planner workflow from API context typing.
packages/api-seo/package.json Bumps @types/node and @t3-oss/env-core.
packages/api-core/package.json Bumps @types/node.
apps/www/package.json Bumps @types/node and @t3-oss/env-core.
apps/seo/wrangler.jsonc Removes planner workflow bindings across envs.
apps/seo/src/server.ts Removes planner workflow export wiring.
apps/seo/src/routes/_authed/admin/route.tsx Deletes old admin route file.
apps/seo/src/routeTree.gen.ts Updates generated routes for admin index route location.
apps/seo/package.json Bumps @types/node and @t3-oss/env-core.
apps/seo-www/package.json Bumps @types/node and @t3-oss/env-core.
apps/seo-contact/package.json Bumps @types/node and @t3-oss/env-core.
AGENTS.md Updates repo guidance, including AI tool schema convention.
.cursor/rules/base-rules.mdc Removes Cursor base rules doc.
Files not reviewed (1)
  • pnpm-lock.yaml: Language not supported

Comment on lines +20 to +27
export function buildStrategyAdvisorInstructions(args: {
project: typeof schema.seoProject.$inferSelect;
}): string {
const projectContext = buildProjectContext(args.project);

return `<role>
You are a world-class SEO/GEO strategist, analyst, and diagnostician for ${args.project.name ?? args.project.websiteUrl}.

Copy link

Copilot AI Mar 2, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same as orchestrator: buildProjectContext() returns a string[] but ${projectContext} will stringify with commas. Join with \n (or have buildProjectContext return a single string) so the Strategy Advisor prompt stays readable and consistent.

Copilot uses AI. Check for mistakes.
## Tool Selection Guide

- **Keyword research**: Start with get_keyword_suggestions for expanding a seed term, then get_keywords_overview to compare volume/difficulty/intent across candidates.
- **SERP analysis**: Use gxist, and what content format wins. This is the most underused and met_serp_for_keyword to see who actually ranks, what SERP features eost valuable tool — use it liberally.
Copy link

Copilot AI Mar 2, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This instruction line looks corrupted/garbled ("Use gxist... met_serp_for_keyword... eost valuable tool") and references a non-existent tool name. Since this text is part of the agent's system prompt, it can materially harm tool usage. Please rewrite it to clearly reference the actual tool (get_serp_for_keyword) and remove the stray words.

Suggested change
- **SERP analysis**: Use gxist, and what content format wins. This is the most underused and met_serp_for_keyword to see who actually ranks, what SERP features eost valuable tool use it liberally.
- **SERP analysis**: Use get_serp_for_keyword to see who actually ranks, which SERP features appear, and what content format wins for a query. This is one of the most valuable tools use it liberally.

Copilot uses AI. Check for mistakes.

<project-context>
${projectContext}
- Google Search Console: ${args.gscConnected ? `Connected (${args.gscDomain})` : "Not connected. If the the user asks for performance/decay/CTR, prioritize connecting via `manage_integrations`"}
Copy link

Copilot AI Mar 2, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There is a duplicated word in the GSC status line ("If the the user asks..."). Since this is system-prompt text, please fix the typo to keep prompts clean.

Suggested change
- Google Search Console: ${args.gscConnected ? `Connected (${args.gscDomain})` : "Not connected. If the the user asks for performance/decay/CTR, prioritize connecting via `manage_integrations`"}
- Google Search Console: ${args.gscConnected ? `Connected (${args.gscDomain})` : "Not connected. If the user asks for performance/decay/CTR, prioritize connecting via `manage_integrations`"}

Copilot uses AI. Check for mistakes.
Comment on lines +22 to 23
- Competitor websites: ${background.competitorsWebsites.length === 0 ? "None Present" : `${background.competitorsWebsites.map((website, index) => `\n - ${index + 1}. ${website}`).join("")}`}
`;
Copy link

Copilot AI Mar 2, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

background.competitorsWebsites is an array of objects ({ url: string } per businessBackgroundSchema), but this formatter interpolates each item as ${website}, which will render as [object Object] and degrade prompt/context quality. Map to website.url (and consider formatting as plain lines without nested template literals for readability).

Copilot uses AI. Check for mistakes.
Comment on lines +103 to +112
competitorsWebsites: {
type: "array",
description: "List of URLs of direct competitors. Leave blank if none.",
items: {
type: "object",
additionalProperties: false,
required: ["url"] satisfies string[],
properties: {
url: { type: "string" },
},
Copy link

Copilot AI Mar 2, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

businessBackgroundSchema now validates competitorsWebsites.url as string.url, but businessBackgroundJsonSchema defines it as just { type: "string" } (no format: "uri"). This makes the AI-output validation weaker than the runtime validator and can let invalid URLs through when using jsonSchema(...). Align the JSON schema with the ArkType schema by adding a URI format constraint.

Copilot uses AI. Check for mistakes.
Comment on lines 48 to 52
const serpResult = await fetchSerp({
keyword: input.query,
keyword: `${input.query} site:${targetUrl}`,
locationName,
languageCode: input.languageCode,
targetUrl,
languageCode,
});
Copy link

Copilot AI Mar 2, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

internal_links currently constrains results by appending site:${targetUrl} where targetUrl is project.websiteUrl (likely includes protocol and possibly a path). This can lead to poor/incorrect SERP constraints. Prefer constraining via DataForSEO's targetUrl param (supported by fetchSerp) or at least parse project.websiteUrl to a hostname before using the site: operator.

Copilot uses AI. Check for mistakes.
Comment on lines 96 to +98
description:
"Run a live web search for up-to-date information. This tool uses the web to find the latest information on the given queries.",
inputSchema: webSearchInputSchema,
"Run a live web search for up-to-date information. This tool uses the web to find the latest information based off the the given instruction and queries.",
inputSchema: jsonSchema<{
Copy link

Copilot AI Mar 2, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Tool description has a grammatical typo: "based off the the" (double "the"). Please fix the string to avoid confusing agents/users and to keep logs/docs clean.

Copilot uses AI. Check for mistakes.
</core-behavior>

<project-context>
${projectContext}
Copy link

Copilot AI Mar 2, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

buildProjectContext() returns a string[], but it's interpolated directly (${projectContext}) which will stringify with commas between items. This makes the system prompt harder to read and can reduce agent performance. Join the array with newlines (or change buildProjectContext to return a single newline-joined string).

Suggested change
${projectContext}
${projectContext.join("\n")}

Copilot uses AI. Check for mistakes.
@github-actions
Copy link
Contributor

github-actions bot commented Mar 2, 2026

Cloudflare Preview URL for WWW 🎈 : https://pr-403.rectangularlabs.com (custom domain)
Cloudflare Preview URL for SEO 🎈 : https://pr-403.fluidposts.com (custom domain)
Cloudflare Preview URL for SEO WWW 🎈 : https://www-pr-403.fluidposts.com (custom domain)
Cloudflare Preview URL for SEO Contact 🎈 : https://contact-pr-403.fluidposts.com (custom domain)

Copy link

@chatgpt-codex-connector chatgpt-codex-connector bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 4b81f7aad9

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment on lines +134 to +136
schema: jsonSchema<
Omit<typeof businessBackgroundSchema.infer, "version">
>(businessBackgroundJsonSchema),

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Keep settings version when applying AI updates

The manage_settings update path now generates businessBackground, imageSettings, and writingSettings with schemas typed as Omit<..., "version">, then writes that object back to seoProject. This means any update from this tool can drop the required version: "v1" field, and subsequent settings reads that validate against businessBackgroundSchema/imageSettingsSchema/writingSettingsSchema will fail for those projects. This is a data-corrupting regression for any user who asks chat to update project settings.

Useful? React with 👍 / 👎.

status: "completed",
type: "content",
fixtureId: fixture.id,
output: draft.text,

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Extract markdown before storing content eval output

runEvalGenerationJob saves draft.text directly as the generated article, but the writer agent now returns text as a JSON stringified object ({ markdown, heroImage, heroImageCaption }). As a result, content eval scoring and downloaded .md output operate on serialized JSON instead of raw markdown, which skews deterministic checks (word count, headings, keyword placement) and corrupts experiment results.

Useful? React with 👍 / 👎.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants