Skip to content

Conversation

@vikas-gits-good
Copy link

@vikas-gits-good vikas-gits-good commented Jan 14, 2026

Summary

  1. Fixed arun_many not working with DeepCrawlStrategy.
  2. Fixed a silent bug where dispatch_result attribute of CrawlResult wasn't being updated correctly.

Fixes #1277

List of files changed and why

  1. DeepCrawlDecorator in crawl4ai.deep_crawling.base_strategy was changed as it was returning a List[CrawlResultContainer] to arun_many which is an incompatible datatype. The data was flattened as explained in the issue comment.
  2. arun_many in crawl4ai.async_webcrawler.AsyncWebCrawler was changed as the datatype needed further modification.
  3. Also discovered that dispatch_result attribute of CrawlResult was not being updated. Fixed that as well.

How Has This Been Tested?

  1. arun with deepcrawl=True | Status=working
  2. arun with deepcrawl=False | Status=working
  3. arun_many with dispatcher=MAD, deepcrawl=True | Status=working
  4. arun_many with dispatcher=SD, deepcrawl=True | Status=working
  5. arun_many with dispatcher=MAD, deepcrawl=False | Status=working
  6. arun_many with dispatcher=SD, deepcrawl=False | Status=working
  7. Haven't tested streaming combo. But that shouldn't matter as stream as a different pipeline.

Use this file to check.

Checklist:

  • My code follows the style guidelines of this project
  • I have performed a self-review of my own code
  • I have commented my code, particularly in hard-to-understand areas
  • I have made corresponding changes to the documentation
  • I have added/updated unit tests that prove my fix is effective or that my feature works
  • New and existing unit tests pass locally with my changes

@vikas-gits-good
Copy link
Author

can you review it once?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Bug]: arun_many Unable to get results at runtime

1 participant