[Bug] GenerateScript stage fails with AssertionError due to file path mismatch

[xucailiang]

Environment

• Python Version: 3.12.12
• ms-agent Version: Installed from source (main branch)
• Model Provider: ModelScope API Inference
• Model Name: Qwen/Qwen3-235B-A22B-Instruct-2507
• OS: macOS (darwin)

Bug Description

When running the generate_script stage of the singularity_cinema project, the LLM automatically creates a semantically-named subdirectory based on the user's input topic (e.g., spongebob_birthday) and writes script.txt, title.txt, etc. into that subdirectory. However, the on_task_end method checks for files at output_video/script.txt (without the subdirectory), causing an assertion failure.

Steps to Reproduce

1. Navigate to the project directory:

   cd projects/singularity_cinema

2. Run the command:

   PYTHONPATH=../.. ms-agent run --config . \
     --query "SpongeBob and Patrick celebrate birthday together" \
     --trust_remote_code true \
     --llm.service modelscope \
     --llm.modelscope_api_key <your_api_key> \
     --llm.modelscope_base_url https://api-inference.modelscope.cn/v1/ \
     --llm.model Qwen/Qwen3-235B-A22B-Instruct-2507

3. Wait for the generate_script stage to complete

Expected Behavior

Task completes successfully and proceeds to the next workflow stage.

Actual Behavior

Throws AssertionError and terminates the task.

Error Logs

# LLM creates subdirectory
[INFO:ms_agent] [generate_script] [tool_calling]:
{
    "arguments": {"path": "spongebob_birthday"},
    "tool_name": "file_system---create_directory"
}
[INFO:ms_agent] [generate_script] Directory: <output_video/spongebob_birthday> was created.

# LLM writes files to subdirectory
[INFO:ms_agent] [generate_script] [tool_calling]:
{
    "arguments": {
        "path": "spongebob_birthday/script.txt",
        "content": "Bikini Bottom is especially lively today..."
    },
    "tool_name": "file_system---write_file"
}
[INFO:ms_agent] [generate_script] Save file <.../output_video/spongebob_birthday/script.txt> successfully.

# Assertion fails
[WARNING:ms_agent] Traceback (most recent call last):
  File ".../ms_agent/agent/llm_agent.py", line 1081, in run_loop
    await self.on_task_end(messages)
  File ".../projects/singularity_cinema/generate_script/agent.py", line 39, in on_task_end
    assert os.path.isfile(script)
AssertionError

Full Stack Trace

Traceback (most recent call last):
  File ".../ms-agent/.venv/bin/ms-agent", line 10, in <module>
    sys.exit(run_cmd())
  File ".../ms_agent/cli/cli.py", line 31, in run_cmd
    cmd.execute()
  File ".../ms_agent/cli/run.py", line 150, in execute
    return self._execute_with_config()
  File ".../ms_agent/cli/run.py", line 209, in _execute_with_config
    asyncio.run(engine.run(self.args.query))
  File ".../ms_agent/workflow/chain_workflow.py", line 102, in run
    outputs = await engine.run(inputs)
  File ".../projects/singularity_cinema/generate_script/agent.py", line 53, in run
    inputs = await super().run(messages, **kwargs)
  File ".../ms_agent/agent/llm_agent.py", line 1118, in run
    async for chunk in self.run_loop(messages=messages, **kwargs):
  File ".../ms_agent/agent/llm_agent.py", line 1099, in run_loop
    raise e
  File ".../ms_agent/agent/llm_agent.py", line 1081, in run_loop
    await self.on_task_end(messages)
  File ".../projects/singularity_cinema/generate_script/agent.py", line 39, in on_task_end
    assert os.path.isfile(script)
AssertionError

Root Cause Analysis

Expected Path	Actual Path
output_video/script.txt	output_video/spongebob_birthday/script.txt
output_video/title.txt	output_video/spongebob_birthday/title.txt

The on_task_end method has hardcoded file paths:

def on_task_end(self, messages: List[Message]):
    script = os.path.join(self.work_dir, 'script.txt')  # output_video/script.txt
    assert os.path.isfile(script)  # Fails! File is in subdirectory

Proposed Fix (Recommended)

The optimal solution is to combine both prompt modification and code-level fallback:

1. Update system prompt in agent.yaml to explicitly instruct the LLM not to create subdirectories:

   **IMPORTANT: Create these files directly in the root directory. Do NOT create any subdirectories.**
   

2. Add recursive file search in agent.py as a fallback mechanism:

   import glob
   
   def on_task_end(self, messages: List[Message]):
       # Try direct path first
       script = os.path.join(self.work_dir, 'script.txt')
       title = os.path.join(self.work_dir, 'title.txt')
       
       # Fallback to recursive search if not found
       if not os.path.isfile(script):
           script_files = glob.glob(os.path.join(self.work_dir, '**/script.txt'), recursive=True)
           assert len(script_files) > 0, f"script.txt not found in {self.work_dir}"
           script = script_files[0]
       
       if not os.path.isfile(title):
           title_files = glob.glob(os.path.join(self.work_dir, '**/title.txt'), recursive=True)
           assert len(title_files) > 0, f"title.txt not found in {self.work_dir}"
           title = title_files[0]
       
       return super().on_task_end(messages)

This approach ensures:

• The prompt guides the LLM to follow the expected behavior
• The code gracefully handles cases where the LLM doesn't follow instructions

Related Files

• projects/singularity_cinema/agent.yaml
• projects/singularity_cinema/generate_script/agent.py

---

I'm willing to work on a PR for this. Please let me know if this approach works, or if you would recommend a different solution.

[suluyana]

I’ve run into the same issue before. This tends to happen with some older models. As a general recommendation, could you try a newer model (e.g., qwen3.5-plus) and see if it resolves the problem?

Root cause: the workflow requires the model to place all generated intermediate files under work_dir, but the model sometimes generates files in a different path.

Your proposed fix can address the current step (the script file path mismatch). However, I’m a bit concerned that with the same model, subsequent generations (audio/image/video) might also be written to incorrect paths, leading to similar issues later in the pipeline.

[xucailiang]

I’ve run into the same issue before. This tends to happen with some older models. As a general recommendation, could you try a newer model (e.g., qwen3.5-plus) and see if it resolves the problem?

Root cause: the workflow requires the model to place all generated intermediate files under work_dir, but the model sometimes generates files in a different path.

Your proposed fix can address the current step (the script file path mismatch). However, I’m a bit concerned that with the same model, subsequent generations (audio/image/video) might also be written to incorrect paths, leading to similar issues later in the pipeline.

I tested with qwen3.5-plus and the issue still reproduces. The model also created a subdirectory spongebob_birthday/, resulting in the same AssertionError.

Also worth noting: the default Qwen3-235B-A22B-Instruct-2507 is already a quite large and advanced model (235B parameters). So this doesn't seem to be a model capability issue — rather, models generally tend to create semantically meaningful subdirectories based on user input.

[suluyana]

That’s a bit surprising — in my earlier test with qwen3.5-plus I didn’t hit this behavior. I’ll re-test on my side and evaluate a more complete fix for this issue.

In the meantime, you can apply the modification you mentioned locally and continue using it as a workaround. At the very least, it should fix the script-generation step itself (i.e., the current script path mismatch / AssertionError).

[suluyana]

I re-tested with qwen3.5-plus, and I wasn’t able to reproduce the extra subdirectory being created under a relative path. So it might be query-dependent?

I also tried Qwen3-235B-A22B-Instruct-2507 and considered making the pipeline more robust by searching for intermediate artifacts (scripts, audio, images, etc.) under work_dir. Unfortunately, that model sometimes generates absolute paths outside work_dir, so it doesn’t seem feasible to reliably support it with the current project assumptions.

In fact, later stages involve generating and iteratively fixing animation code, which places fairly higher demands on the model. That’s why we strongly recommend using the models mentioned in the README or chinese models released within the last 1–2 months, as they generally deliver a significantly higher success rate.