diff --git a/docs/FRICTIONLESS_SETUP_PLAN.md b/docs/FRICTIONLESS_SETUP_PLAN.md new file mode 100644 index 000000000..c1439a412 --- /dev/null +++ b/docs/FRICTIONLESS_SETUP_PLAN.md @@ -0,0 +1,490 @@ +# Frictionless Setup Implementation Plan + +## Executive Summary + +Transform Codeflash from a "config-required" tool to a "smart-defaults with optional config" tool, reducing setup friction while maintaining enterprise-grade configurability. + +**Key Principle: Use native config files only (pyproject.toml, package.json) - no new config formats.** + +--- + +## Current State vs Target State + +| Aspect | Current State | Target State | +|--------|--------------|--------------| +| First run | Error: "Run codeflash init" | Auto-detect → Quick confirm → Run | +| Config requirement | Mandatory for Python | Optional (smart defaults) | +| Setup questions | 5-10 questions | 1 confirmation (or 0 with `--yes`) | +| Config storage | pyproject.toml / package.json | Same - native files only | +| `codeflash init` | Required for all users | Optional - for GitHub Actions setup | + +--- + +## User Flows + +### Flow 1: First-Time User (Zero Friction) +``` +$ codeflash --file src/utils.py --function calculate + +⚡ Welcome to Codeflash! + +I auto-detected your project: +┌─────────────────────────────────────────┐ +│ Language: Python │ +│ Module root: src/ │ +│ Test runner: pytest │ +│ Formatter: ruff (from pyproject.toml)│ +│ Ignoring: __pycache__, .venv, dist │ +└─────────────────────────────────────────┘ + +? Proceed with these settings? [Y/n/customize] +> Y + +✅ Settings saved to codeflash.yaml +🔑 Enter API key: cf-xxxxx +✅ Running optimization... +``` + +### Flow 2: Subsequent Runs (No Prompts) +``` +$ codeflash --file src/utils.py --function calculate + +⚡ Running optimization... +``` + +### Flow 3: Enterprise Setup (`codeflash init`) +``` +$ codeflash init + +⚡ Codeflash Enterprise Setup + +[Full wizard with all options] +[GitHub Actions integration] +[Team-wide config] +``` + +### Flow 4: CI/CD (Config Required) +``` +# In CI, config must exist - no interactive prompts +$ codeflash --all + +# If no config: clear error with instructions +``` + +--- + +## Architecture + +### New Components + +``` +codeflash/ +├── setup/ # NEW: Setup & detection module +│ ├── __init__.py +│ ├── detector.py # Universal auto-detection engine +│ ├── first_run.py # First-run experience handler +│ ├── config_schema.py # Universal config schema (Pydantic) +│ └── config_writer.py # Write config to codeflash.yaml +│ +├── code_utils/ +│ ├── config_parser.py # MODIFY: Add codeflash.yaml support +│ └── config_js.py # KEEP: JS-specific detection +│ +├── cli_cmds/ +│ ├── cmd_init.py # MODIFY: Enterprise-focused +│ └── init_javascript.py # DEPRECATE: Merge into setup/ +│ +└── main.py # MODIFY: Add first-run check +``` + +### Config File Strategy (Native Files) + +Use **native config files** for each language ecosystem: + +| Language | Config File | Section | +|----------|-------------|---------| +| Python | `pyproject.toml` | `[tool.codeflash]` | +| JavaScript | `package.json` | `{ "codeflash": {} }` | +| TypeScript | `package.json` | `{ "codeflash": {} }` | +| Rust (future) | `Cargo.toml` | `[package.metadata.codeflash]` | +| Go (future) | `codeflash.yaml` | Root level (Go has no standard) | + +#### Python Config (pyproject.toml) +```toml +[tool.codeflash] +module-root = "src" +tests-root = "tests" +ignore-paths = ["vendor/", "migrations/"] +formatter-cmds = ["ruff check --fix $file", "ruff format $file"] +# Optional +git-remote = "origin" +disable-telemetry = false +benchmarks-root = "benchmarks" +``` + +#### JavaScript/TypeScript Config (package.json) +```json +{ + "codeflash": { + "moduleRoot": "src", + "ignorePaths": ["dist/", "coverage/"], + "formatterCmds": ["npx prettier --write $file"] + } +} +``` + +### Why Native Files? + +1. **No extra file clutter** - Uses existing config files +2. **Ecosystem conventions** - Developers expect config in standard locations +3. **No tracking needed** - Project root always has pyproject.toml or package.json +4. **Tool discovery** - Other tools already know where to look + +### Config Discovery Flow + +``` +find_project_and_config(): + 1. Walk up from current directory + 2. Find project root (.git, package.json, pyproject.toml) + 3. Detect language from project root files + 4. Read codeflash config from appropriate native file + 5. If no codeflash section → first run experience +``` + +--- + +## Implementation Phases + +## Phase 1: Universal Auto-Detection Engine +**Goal**: Create a single detection engine that works for all languages + +### Task 1.1: Create Detection Module Structure +- [ ] Create `codeflash/setup/` directory +- [ ] Create `codeflash/setup/__init__.py` +- [ ] Create `codeflash/setup/detector.py` + +### Task 1.2: Implement Universal Project Detector +**File**: `codeflash/setup/detector.py` +```python +@dataclass +class DetectedProject: + language: str # python | javascript | typescript + project_root: Path + module_root: Path + tests_root: Path | None + test_runner: str # pytest | jest | vitest | mocha + formatter: list[str] | None + ignore_paths: list[str] + confidence: float # 0.0 - 1.0 + +def detect_project(path: Path | None = None) -> DetectedProject: + """Auto-detect all project settings.""" + ... +``` + +### Task 1.3: Implement Language Detection +- [ ] Detect from file extensions count +- [ ] Detect from config files (pyproject.toml, package.json, tsconfig.json) +- [ ] Detect from lockfiles (poetry.lock, yarn.lock, etc.) +- [ ] Return confidence score + +### Task 1.4: Implement Module Root Detection +- [ ] Python: Look for `__init__.py`, src/ convention, pyproject.toml name +- [ ] JS/TS: Look for package.json exports/main, src/ convention +- [ ] Generic: Find directory with most source files + +### Task 1.5: Implement Tests Root Detection +- [ ] Look for tests/, test/, __tests__/ directories +- [ ] Look for *_test.py, *.test.js patterns +- [ ] Check pytest.ini, jest.config.js locations + +### Task 1.6: Implement Test Runner Detection +- [ ] Python: Check for pytest, unittest in dependencies/config +- [ ] JS/TS: Check for jest, vitest, mocha in devDependencies +- [ ] Parse test scripts in package.json + +### Task 1.7: Implement Formatter Detection +- [ ] Python: Check ruff.toml, pyproject.toml [tool.ruff], black config +- [ ] JS/TS: Check .prettierrc, .eslintrc, devDependencies +- [ ] Return formatter commands list + +### Task 1.8: Implement Ignore Paths Detection +- [ ] Parse .gitignore +- [ ] Add language-specific defaults (node_modules, __pycache__, etc.) +- [ ] Merge with any existing config + +--- + +## Phase 2: Universal Config Schema +**Goal**: Single config format that works for all languages + +### Task 2.1: Create Config Schema +**File**: `codeflash/setup/config_schema.py` +```python +from pydantic import BaseModel + +class CodeflashConfig(BaseModel): + version: int = 1 + language: str + module_root: str + tests_root: str | None = None + ignore_paths: list[str] = [] + formatter_commands: list[str] = [] + test_runner: str | None = None + git_remote: str = "origin" + disable_telemetry: bool = False + benchmarks_root: str | None = None +``` + +### Task 2.2: Create Config Writer +**File**: `codeflash/setup/config_writer.py` +- [ ] Write to codeflash.yaml (primary) +- [ ] Optionally write to pyproject.toml (Python projects) +- [ ] Optionally write to package.json (JS/TS projects) + +### Task 2.3: Update Config Parser +**File**: `codeflash/code_utils/config_parser.py` +- [ ] Add codeflash.yaml as primary config source +- [ ] Maintain backward compatibility with pyproject.toml +- [ ] Maintain backward compatibility with package.json +- [ ] Priority: codeflash.yaml > pyproject.toml > package.json + +### Task 2.4: Create Config Finder +- [ ] `find_config_file()` - Search for any valid config +- [ ] Walk up directory tree to find project root +- [ ] Return config file path and type + +--- + +## Phase 3: First-Run Experience +**Goal**: Seamless first-run with auto-detection and quick confirm + +### Task 3.1: Create First-Run Handler +**File**: `codeflash/setup/first_run.py` +```python +def is_first_run() -> bool: + """Check if this is first run (no config exists).""" + ... + +def handle_first_run(args: Namespace) -> Namespace: + """Handle first-run experience with auto-detection.""" + ... +``` + +### Task 3.2: Implement First-Run Detection +- [ ] Check for codeflash.yaml in project root +- [ ] Check for [tool.codeflash] in pyproject.toml +- [ ] Check for codeflash section in package.json +- [ ] Return True if none found + +### Task 3.3: Implement First-Run UI +- [ ] Show welcome message +- [ ] Display auto-detected settings in table +- [ ] Quick confirm prompt: Y/n/customize +- [ ] Handle `--yes` flag to skip prompt + +### Task 3.4: Implement API Key Prompt (First Run Only) +- [ ] Check for CODEFLASH_API_KEY env var +- [ ] If missing, prompt for API key +- [ ] Offer OAuth login option +- [ ] Save to shell rc file + +### Task 3.5: Implement Config Auto-Save +- [ ] After confirmation, save config to codeflash.yaml +- [ ] Show saved location +- [ ] Continue with optimization + +### Task 3.6: Integrate with Main Entry Point +**File**: `codeflash/main.py` +- [ ] Check `is_first_run()` before loading config +- [ ] If first run, call `handle_first_run()` +- [ ] If not first run, proceed normally + +--- + +## Phase 4: Refactor `codeflash init` for Enterprise +**Goal**: Make init command focused on enterprise/CI setup + +### Task 4.1: Simplify Init Command +**File**: `codeflash/cli_cmds/cmd_init.py` +- [ ] Remove redundant questions (auto-detect instead) +- [ ] Focus on: API key, GitHub Actions, advanced overrides +- [ ] Add `--enterprise` flag for full wizard + +### Task 4.2: Create Enterprise Wizard +- [ ] All current questions available +- [ ] Team-wide config options +- [ ] GitHub Actions setup +- [ ] CI/CD validation mode + +### Task 4.3: Merge JS/TS Init into Main Init +- [ ] Remove `init_javascript.py` as separate module +- [ ] Use universal detector for all languages +- [ ] Same flow regardless of language + +### Task 4.4: Update Help Text +- [ ] `codeflash init` - Quick setup with auto-detection +- [ ] `codeflash init --enterprise` - Full enterprise wizard +- [ ] Document that init is optional for most users + +--- + +## Phase 5: CLI Updates +**Goal**: Support new flow with appropriate flags + +### Task 5.1: Add New CLI Flags +**File**: `codeflash/cli_cmds/cli.py` +```python +# New flags +--yes, -y # Skip confirmation prompts +--no-save # Don't save config (one-off run) +--show-config # Show detected/current config and exit +--reset-config # Delete saved config, re-detect +``` + +### Task 5.2: Update Main Entry Point +**File**: `codeflash/main.py` +- [ ] Add first-run check +- [ ] Handle --yes flag +- [ ] Handle --no-save flag +- [ ] Handle --show-config flag + +### Task 5.3: CI/CD Mode Detection +- [ ] Detect CI environment (CI=true, GITHUB_ACTIONS, etc.) +- [ ] In CI: Require config, no interactive prompts +- [ ] Show clear error if config missing in CI + +### Task 5.4: Update Error Messages +- [ ] Replace "Run codeflash init" with helpful auto-detection message +- [ ] Show what was detected vs what's missing +- [ ] Suggest specific fixes + +--- + +## Phase 6: Backend Updates (If Needed) +**Goal**: Ensure backend supports new flow + +### Task 6.1: Review API Requirements +- [ ] Check if API needs config sent inline +- [ ] Verify API key validation flow +- [ ] Check telemetry expectations + +### Task 6.2: Update Telemetry Events +- [ ] Add "first_run_auto_detect" event +- [ ] Add "config_saved" event +- [ ] Track detection confidence scores + +### Task 6.3: API Key Validation +- [ ] Validate API key before first optimization +- [ ] Clear error message for invalid keys +- [ ] Support OAuth flow + +--- + +## Phase 7: Testing & Documentation +**Goal**: Ensure reliability and clear documentation + +### Task 7.1: Unit Tests +- [ ] Test detector.py for all languages +- [ ] Test config_schema.py validation +- [ ] Test config_writer.py output +- [ ] Test first_run.py flow + +### Task 7.2: Integration Tests +- [ ] Test first-run on Python project +- [ ] Test first-run on JS project +- [ ] Test first-run on TS project +- [ ] Test CI mode (no interactive) +- [ ] Test --yes flag + +### Task 7.3: E2E Tests +- [ ] Full flow: fresh project → first run → optimization +- [ ] Full flow: existing config → optimization +- [ ] Full flow: codeflash init → GitHub Actions + +### Task 7.4: Update Documentation +- [ ] Update README with new quick start +- [ ] Update init command docs +- [ ] Add config file reference +- [ ] Add migration guide from old config + +--- + +## Migration Strategy + +### For Existing Users +1. Existing pyproject.toml/package.json configs continue to work +2. On first run after update, offer to migrate to codeflash.yaml +3. No breaking changes - backward compatible + +### For New Users +1. First run auto-detects everything +2. Quick confirm saves to codeflash.yaml +3. Optional: run `codeflash init` for enterprise setup + +--- + +## Success Metrics + +| Metric | Current | Target | +|--------|---------|--------| +| Time to first optimization | 5-10 min (init required) | < 1 min | +| Setup questions asked | 5-10 | 1 (confirm) or 0 (--yes) | +| Config file required | Always | Optional | +| Languages supported uniformly | 2 (Python, JS) | All (universal flow) | + +--- + +## Risk Mitigation + +| Risk | Mitigation | +|------|------------| +| Auto-detection wrong | Show detected values, allow override | +| Breaking existing configs | Full backward compatibility | +| CI/CD breaks | Detect CI mode, require explicit config | +| API key exposure | Never log API keys, use env vars | + +--- + +## File Changes Summary + +| File | Action | Description | +|------|--------|-------------| +| `codeflash/setup/__init__.py` | CREATE | New module | +| `codeflash/setup/detector.py` | CREATE | Universal detection engine | +| `codeflash/setup/first_run.py` | CREATE | First-run experience | +| `codeflash/setup/config_schema.py` | CREATE | Pydantic config model | +| `codeflash/setup/config_writer.py` | CREATE | Config file writer | +| `codeflash/main.py` | MODIFY | Add first-run check | +| `codeflash/cli_cmds/cli.py` | MODIFY | Add new flags | +| `codeflash/cli_cmds/cmd_init.py` | MODIFY | Enterprise focus | +| `codeflash/code_utils/config_parser.py` | MODIFY | Add codeflash.yaml | +| `codeflash/cli_cmds/init_javascript.py` | DEPRECATE | Merge into setup/ | + +--- + +## Timeline Estimate + +| Phase | Tasks | Complexity | +|-------|-------|------------| +| Phase 1: Detection Engine | 8 tasks | Medium | +| Phase 2: Config Schema | 4 tasks | Low | +| Phase 3: First-Run Experience | 6 tasks | Medium | +| Phase 4: Refactor Init | 4 tasks | Medium | +| Phase 5: CLI Updates | 4 tasks | Low | +| Phase 6: Backend Updates | 3 tasks | Low-Medium | +| Phase 7: Testing & Docs | 4 tasks | Medium | + +--- + +## Next Steps + +1. Review and approve this plan +2. Create GitHub issues for each phase +3. Start with Phase 1 (Detection Engine) as foundation +4. Iterate based on feedback + +--- + +*Plan created: 2024* +*Author: Claude (Product Manager & Engineering Lead)* \ No newline at end of file diff --git a/docs/JS_PROMPT_PARITY_RECOMMENDATIONS.md b/docs/JS_PROMPT_PARITY_RECOMMENDATIONS.md new file mode 100644 index 000000000..22c736014 --- /dev/null +++ b/docs/JS_PROMPT_PARITY_RECOMMENDATIONS.md @@ -0,0 +1,151 @@ +# JavaScript Test Generation Prompt - Parity Recommendations + +This document outlines gaps between Python and JavaScript test generation prompts and provides recommendations for improving the JavaScript prompts. + +## Current State Comparison + +### Python Prompt (76 lines) - Comprehensive sections: +1. **PRESERVE ORIGINAL FUNCTION** - Don't modify function being tested +2. **USE REAL CLASSES - NO STUBS OR FAKES** - Import actual domain classes +3. **HANDLING INSTANCE METHODS** - Proper instantiation patterns +4. **USE CONFTEST.PY FIXTURES** - Leverage existing test fixtures +5. **DO NOT MOCK THE FUNCTION UNDER TEST** - Critical rule +6. **IMPORT CLASSES FROM THEIR REAL MODULES** - Proper import sources +7. **IMPORT EVERYTHING YOU USE** - No missing imports +8. **ONLY IMPORT WHAT YOU USE** - No unused imports +9. **USE CORRECT IMPORT SOURCES** - Match context provided +10. **DO NOT USE MOCK OBJECTS FOR DOMAIN CLASSES** - Real instances +11. **USE CORRECT CONSTRUCTOR SIGNATURES** - Proper instantiation +12. **VALID PYTHON STRING LITERALS** - Escape sequences, raw strings + +### JavaScript Prompt (44 lines) - Current sections: +1. Basic/Edge/Large Scale test structure ✓ +2. DO NOT MOCK THE FUNCTION UNDER TEST ✓ +3. IMPORT FROM REAL MODULES ✓ +4. HANDLE ASYNC PROPERLY ✓ +5. IMPORT PATH RULES (no extensions) ✓ +6. MOCKING RULES (Jest vs Vitest) ✓ + +## Gap Analysis + +| Python Section | JS Status | Priority | +|----------------|-----------|----------| +| PRESERVE ORIGINAL FUNCTION | Missing | High | +| USE REAL CLASSES - NO STUBS | Missing | High | +| HANDLING INSTANCE METHODS | Missing | High | +| USE CONFTEST.PY FIXTURES | N/A (JS uses different patterns) | - | +| IMPORT EVERYTHING YOU USE | Missing | Medium | +| ONLY IMPORT WHAT YOU USE | Missing | Medium | +| USE CORRECT IMPORT SOURCES | Missing | High | +| USE CORRECT CONSTRUCTOR SIGNATURES | Missing | High | +| VALID STRING LITERALS | Missing | Medium | + +## Recommended Additions to JavaScript Prompt + +### Core Sections (Port from Python) + +```markdown +**CRITICAL: PRESERVE ORIGINAL FUNCTION**: +- Do NOT modify or rewrite the function being tested. +- Your task is ONLY to write tests for the function as given. + +**CRITICAL: USE REAL CLASSES - NO STUBS OR FAKES**: +- When the function uses custom classes/types, import and use the REAL class. +- **WRONG**: Creating inline stub classes or interfaces +- **CORRECT**: `import { UserProfile } from '../models/UserProfile'` + +**CRITICAL: HANDLING CLASS METHODS**: +- For instance methods, properly instantiate the class first. +- Use the constructor signature shown in the context. +- Example: + ```javascript + const processor = new DataProcessor(config); + const result = processor.processData(input); + ``` + +**CRITICAL: IMPORT EVERYTHING YOU USE**: +- Every class, type, function, or constant used in tests MUST be imported. +- Do not assume anything is globally available. + +**CRITICAL: ONLY IMPORT WHAT YOU USE**: +- Do not add unused imports - they cause TypeScript/linting errors. + +**CRITICAL: USE CORRECT IMPORT SOURCES**: +- Import from the exact module paths shown in the provided context. +- Do not guess or infer import paths. + +**CRITICAL: USE CORRECT CONSTRUCTOR SIGNATURES**: +- Check the class definition for required constructor parameters. +- Do not omit required parameters or add non-existent ones. + +**CRITICAL: VALID STRING LITERALS**: +- Use proper escaping for special characters in strings. +- For multiline strings, use template literals (`backticks`). +- Escape backslashes properly: `\\n` for literal `\n`. +``` + +### JS/TS-Specific Sections (New) + +```markdown +**CRITICAL: HANDLE TYPESCRIPT TYPES**: +- If testing TypeScript code, ensure test file uses `.test.ts` patterns. +- Respect type annotations - don't pass wrong types to functions. +- Use type assertions (`as Type`) only when necessary. + +**CRITICAL: HANDLE PROMISES AND CALLBACKS**: +- For callback-based APIs, use promisify or done() callback. +- Never leave floating promises - always await or return. +- Use `expect.assertions(n)` for async error testing. + +**CRITICAL: ES MODULES VS COMMONJS**: +- Check if project uses ESM (`import/export`) or CJS (`require/module.exports`). +- Match the import style to the project configuration. + +**CRITICAL: HANDLE THIS BINDING**: +- Arrow functions don't have their own `this` context. +- For methods that use `this`, test with proper binding: + ```javascript + // Correct - preserves this context + const instance = new MyClass(); + expect(instance.method()).toBe(expected); + + // Wrong - loses this context + const { method } = new MyClass(); + expect(method()).toBe(expected); // May fail! + ``` + +**CRITICAL: NULL VS UNDEFINED**: +- JavaScript distinguishes between `null` and `undefined`. +- Test both cases when function handles optional values. +- Use `toBe(null)` vs `toBeUndefined()` appropriately. +``` + +## Implementation Priority + +1. **Phase 1 (High Priority)**: Add core sections ported from Python + - PRESERVE ORIGINAL FUNCTION + - USE REAL CLASSES + - HANDLING CLASS METHODS + - USE CORRECT IMPORT SOURCES + - USE CORRECT CONSTRUCTOR SIGNATURES + +2. **Phase 2 (Medium Priority)**: Add import management + - IMPORT EVERYTHING YOU USE + - ONLY IMPORT WHAT YOU USE + - VALID STRING LITERALS + +3. **Phase 3 (JS-Specific)**: Add JavaScript-specific guidance + - HANDLE TYPESCRIPT TYPES + - HANDLE PROMISES AND CALLBACKS + - ES MODULES VS COMMONJS + - HANDLE THIS BINDING + - NULL VS UNDEFINED + +## Metrics to Track + +After implementing these changes, track: +- Test generation success rate (tests that compile) +- Test execution pass rate (tests that run without errors) +- Import error frequency +- Type error frequency (TypeScript projects) +- Mock-related failures \ No newline at end of file diff --git a/docs/codeflash-concepts/how-codeflash-works.mdx b/docs/codeflash-concepts/how-codeflash-works.mdx index 4456cd3d0..b9ab9a060 100644 --- a/docs/codeflash-concepts/how-codeflash-works.mdx +++ b/docs/codeflash-concepts/how-codeflash-works.mdx @@ -3,25 +3,31 @@ title: "How Codeflash Works" description: "Understand Codeflash's generate-and-verify approach to code optimization and correctness verification" icon: "gear" sidebarTitle: "How It Works" -keywords: ["architecture", "verification", "correctness", "testing", "optimization", "LLM", "benchmarking"] +keywords: ["architecture", "verification", "correctness", "testing", "optimization", "LLM", "benchmarking", "javascript", "typescript", "python"] --- # How Codeflash Works Codeflash follows a "generate and verify" approach to optimize code. It uses LLMs to generate optimizations, then it rigorously verifies if those optimizations are indeed faster and if they have the same behavior. The basic unit of optimization is a function—Codeflash tries to speed up the function, and tries to ensure that it still behaves the same way. This way if you merge the optimized code, it simply runs faster without breaking any functionality. +Codeflash supports **Python**, **JavaScript**, and **TypeScript** projects. + ## Analysis of your code Codeflash scans your codebase to identify all available functions. It locates existing unit tests in your projects and maps which functions they test. When optimizing a function, Codeflash runs these discovered tests to verify nothing has broken. +For Python, code analysis uses `libcst` and `jedi`. For JavaScript/TypeScript, it uses `tree-sitter` for AST parsing. + #### What kind of functions can Codeflash optimize? Codeflash works best with self-contained functions that have minimal side effects (like communicating with external systems or sending network requests). Codeflash optimizes a group of functions - consisting of an entry point function and any other functions it directly calls. -Codeflash supports optimizing async functions. +Codeflash supports optimizing async functions in all supported languages. #### Test Discovery -Codeflash currently only runs tests that directly call the target function in their test body. To discover tests that indirectly call the function, you can use the Codeflash Tracer. The Tracer analyzes your test suite and identifies all tests that eventually call a function. +Codeflash discovers tests that directly call the target function in their test body. For Python, it finds pytest and unittest tests. For JavaScript/TypeScript, it finds Jest and Vitest test files. + +To discover tests that indirectly call the function, you can use the Codeflash Tracer. The Tracer analyzes your test suite and identifies all tests that eventually call a function. ## Optimization Generation @@ -48,12 +54,12 @@ We recommend manually reviewing the optimized code since there might be importan Codeflash generates two types of tests: -- LLM Generated tests - Codeflash uses LLMs to create several regression test cases that cover typical function usage, edge cases, and large-scale inputs to verify both correctness and performance. -- Concolic coverage tests - Codeflash uses state-of-the-art concolic testing with an SMT Solver (a theorem prover) to explore execution paths and generate function arguments. This aims to maximize code coverage for the function being optimized. Codeflash runs the resulting test file to verify correctness. Currently, this feature only supports pytest. +- **LLM Generated tests** - Codeflash uses LLMs to create several regression test cases that cover typical function usage, edge cases, and large-scale inputs to verify both correctness and performance. This works for Python, JavaScript, and TypeScript. +- **Concolic coverage tests** - Codeflash uses state-of-the-art concolic testing with an SMT Solver (a theorem prover) to explore execution paths and generate function arguments. This aims to maximize code coverage for the function being optimized. Currently, this feature only supports Python (pytest). ## Code Execution -Codeflash runs tests for the target function using either pytest or unittest frameworks. The tests execute on your machine, ensuring access to the Python environment and any other dependencies associated to let Codeflash run your code properly. Running on your machine also ensures accurate performance measurements since runtime varies by system. +Codeflash runs tests for the target function on your machine. For Python, it uses pytest or unittest. For JavaScript/TypeScript, it uses Jest or Vitest. Running on your machine ensures access to your environment and dependencies, and provides accurate performance measurements since runtime varies by system. #### Performance benchmarking diff --git a/docs/configuration.mdx b/docs/configuration.mdx index 3f388a531..29506b952 100644 --- a/docs/configuration.mdx +++ b/docs/configuration.mdx @@ -1,83 +1,26 @@ --- title: "Manual Configuration" -description: "Configure Codeflash for your project with pyproject.toml settings and advanced options" +description: "Configure Codeflash for your project" icon: "gear" sidebarTitle: "Manual Configuration" keywords: [ "configuration", - "pyproject.toml", "setup", "settings", - "pytest", - "formatter", ] --- # Manual Configuration Codeflash is installed and configured on a per-project basis. -`codeflash init` should guide you through the configuration process, but if you need to manually configure Codeflash or set advanced settings, you can do so by editing the `pyproject.toml` file in the root directory of your project. - -## Configuration Options - -Codeflash config looks like the following - -```toml -[tool.codeflash] -module-root = "my_module" -tests-root = "tests" -formatter-cmds = ["black $file"] -# optional configuration -benchmarks-root = "tests/benchmarks" # Required when running with --benchmark -ignore-paths = ["my_module/build/"] -pytest-cmd = "pytest" -disable-imports-sorting = false -disable-telemetry = false -git-remote = "origin" -override-fixtures = false -``` - -All file paths are relative to the directory of the `pyproject.toml` file. - -Required Options: - -- `module-root`: The Python module you want Codeflash to optimize going forward. Only code under this directory will be optimized. It should also have an `__init__.py` file to make the module importable. -- `tests-root`: The directory where your tests are located. Codeflash will use this directory to discover existing tests as well as generate new tests. - -Optional Configuration: - -- `benchmarks-root`: The directory where your benchmarks are located. Codeflash will use this directory to discover existing benchmarks. Note that this option is required when running with `--benchmark`. -- `ignore-paths`: A list of paths within the `module-root` to ignore when optimizing code. Codeflash will not optimize code in these paths. Useful for ignoring build directories or other generated code. You can also leave this empty if not needed. -- `pytest-cmd`: The command to run your tests. Defaults to `pytest`. You can specify extra commandline arguments here for pytest. -- `formatter-cmds`: The command line to run your code formatter or linter. Defaults to `["black $file"]`. In the command line `$file` refers to the current file being optimized. The assumption with using tools here is that they overwrite the same file and returns a zero exit code. You can also specify multiple tools here that run in a chain as a toml array. You can also disable code formatting by setting this to `["disabled"]`. - - `ruff` - A recommended way to run ruff linting and formatting is `["ruff check --exit-zero --fix $file", "ruff format $file"]`. To make `ruff check --fix` return a 0 exit code please add a `--exit-zero` argument. -- `disable-imports-sorting`: By default, codeflash uses isort to organize your imports before creating suggestions. You can disable this by setting this field to `true`. This could be useful if you don't sort your imports or while using linters like ruff that sort imports too. -- `disable-telemetry`: Disable telemetry data collection. Defaults to `false`. Set this to `true` to disable telemetry data collection. Codeflash collects anonymized telemetry data to understand how users are using Codeflash and to improve the product. Telemetry does not collect any code data. -- `git-remote`: The git remote to use for pull requests. Defaults to `"origin"`. -- `override-fixtures`: Override pytest fixtures during optimization. Defaults to `false`. - -## Example Configuration - -Here's an example project with the following structure: - -```text -acme-project/ -|- foo_module/ -| |- __init__.py -| |- foo.py -| |- main.py -|- tests/ -| |- __init__.py -| |- test_script.py -|- pyproject.toml -``` - -Here's a sample `pyproject.toml` file for the above project: - -```toml -[tool.codeflash] -module-root = "foo_module" -tests-root = "tests" -ignore-paths = [] -``` +`codeflash init` should guide you through the configuration process, but if you need to manually configure Codeflash or set advanced settings, follow the guide for your language: + + + + Configure via `pyproject.toml` + + + Configure via `package.json` + + \ No newline at end of file diff --git a/docs/configuration/javascript.mdx b/docs/configuration/javascript.mdx new file mode 100644 index 000000000..a789a0428 --- /dev/null +++ b/docs/configuration/javascript.mdx @@ -0,0 +1,110 @@ +--- +title: "JavaScript / TypeScript Configuration" +description: "Configure Codeflash for JavaScript and TypeScript projects using package.json" +icon: "js" +sidebarTitle: "JavaScript / TypeScript" +keywords: + [ + "configuration", + "package.json", + "javascript", + "typescript", + "jest", + "vitest", + "prettier", + "eslint", + ] +--- + +# JavaScript / TypeScript Configuration + +Codeflash stores its configuration in `package.json` under the `"codeflash"` key. + +## Full Reference + +```json +{ + "name": "my-project", + "codeflash": { + "moduleRoot": "src", + "testsRoot": "tests", + "testRunner": "jest", + "formatterCmds": ["prettier --write $file"], + "ignorePaths": ["src/generated/"], + "disableTelemetry": false, + "gitRemote": "origin" + } +} +``` + +All file paths are relative to the directory containing `package.json`. + + +Codeflash auto-detects most settings from your project structure. If `codeflash init` produces the right config, you may not need any manual configuration. + + +## Required Options + +- `moduleRoot`: The source directory to optimize. Only code under this directory will be optimized. +- `testsRoot`: The directory where your tests are located. Codeflash discovers existing tests and generates new ones here. + +## Optional Options + +- `testRunner`: Test framework to use. Auto-detected from your dependencies. Supported values: `"jest"`, `"vitest"`. +- `formatterCmds`: Formatter commands. `$file` refers to the file being optimized. Disable with `["disabled"]`. + - **Prettier**: `["prettier --write $file"]` + - **ESLint + Prettier**: `["eslint --fix $file", "prettier --write $file"]` + - **Biome**: `["biome check --write $file"]` +- `ignorePaths`: Paths within `moduleRoot` to skip during optimization. +- `disableTelemetry`: Disable anonymized telemetry. Defaults to `false`. +- `gitRemote`: Git remote for pull requests. Defaults to `"origin"`. + +## Module Systems + +Codeflash handles both ES Modules and CommonJS automatically. It detects the module system from your `package.json`: + +```json +{ + "type": "module" +} +``` + +- `"type": "module"` — Files are treated as ESM (`import`/`export`) +- `"type": "commonjs"` or omitted — Files are treated as CommonJS (`require`/`module.exports`) + +No additional configuration is needed. Codeflash respects `.mjs`/`.cjs` extensions as well. + +## TypeScript + +TypeScript projects work out of the box. Codeflash detects TypeScript from the presence of `tsconfig.json` and handles `.ts`/`.tsx` files automatically. + +No separate configuration is needed for TypeScript vs JavaScript. + +## Test Framework Support + +| Framework | Auto-detected from | Notes | +|-----------|-------------------|-------| +| **Jest** | `jest` in dependencies | Default for most projects | +| **Vitest** | `vitest` in dependencies | ESM-native support | + +## Example + +```text +my-app/ +|- src/ +| |- utils.js +| |- index.js +|- tests/ +| |- utils.test.js +|- package.json +``` + +```json +{ + "name": "my-app", + "codeflash": { + "moduleRoot": "src", + "testsRoot": "tests" + } +} +``` \ No newline at end of file diff --git a/docs/configuration/python.mdx b/docs/configuration/python.mdx new file mode 100644 index 000000000..765a7ac82 --- /dev/null +++ b/docs/configuration/python.mdx @@ -0,0 +1,80 @@ +--- +title: "Python Configuration" +description: "Configure Codeflash for Python projects using pyproject.toml" +icon: "python" +sidebarTitle: "Python" +keywords: + [ + "configuration", + "pyproject.toml", + "python", + "pytest", + "formatter", + "ruff", + "black", + ] +--- + +# Python Configuration + +Codeflash stores its configuration in `pyproject.toml` under the `[tool.codeflash]` section. + +## Full Reference + +```toml +[tool.codeflash] +# Required +module-root = "my_module" +tests-root = "tests" + +# Optional +formatter-cmds = ["black $file"] +benchmarks-root = "tests/benchmarks" +ignore-paths = ["my_module/build/"] +pytest-cmd = "pytest" +disable-imports-sorting = false +disable-telemetry = false +git-remote = "origin" +override-fixtures = false +``` + +All file paths are relative to the directory of the `pyproject.toml` file. + +## Required Options + +- `module-root`: The Python module to optimize. Only code under this directory will be optimized. It should have an `__init__.py` file to make the module importable. +- `tests-root`: The directory where your tests are located. Codeflash discovers existing tests and generates new ones here. + +## Optional Options + +- `benchmarks-root`: Directory for benchmarks. Required when running with `--benchmark`. +- `ignore-paths`: Paths within `module-root` to skip. Useful for build directories or generated code. +- `pytest-cmd`: Command to run your tests. Defaults to `pytest`. You can add extra arguments here. +- `formatter-cmds`: Formatter/linter commands. `$file` refers to the file being optimized. Disable with `["disabled"]`. + - **ruff** (recommended): `["ruff check --exit-zero --fix $file", "ruff format $file"]` + - **black**: `["black $file"]` +- `disable-imports-sorting`: Disable isort import sorting. Defaults to `false`. +- `disable-telemetry`: Disable anonymized telemetry. Defaults to `false`. +- `git-remote`: Git remote for pull requests. Defaults to `"origin"`. +- `override-fixtures`: Override pytest fixtures during optimization. Defaults to `false`. + +## Example + +```text +acme-project/ +|- foo_module/ +| |- __init__.py +| |- foo.py +| |- main.py +|- tests/ +| |- __init__.py +| |- test_script.py +|- pyproject.toml +``` + +```toml +[tool.codeflash] +module-root = "foo_module" +tests-root = "tests" +ignore-paths = [] +``` \ No newline at end of file diff --git a/docs/index.mdx b/docs/index.mdx index 24d0d1561..b94258ed3 100644 --- a/docs/index.mdx +++ b/docs/index.mdx @@ -1,27 +1,38 @@ --- -title: "Codeflash is an AI performance optimizer for Python code" +title: "Codeflash is an AI performance optimizer for your code" icon: "rocket" sidebarTitle: "Overview" -keywords: ["python", "performance", "optimization", "AI", "code analysis", "benchmarking"] +keywords: ["python", "javascript", "typescript", "performance", "optimization", "AI", "code analysis", "benchmarking"] --- -Codeflash speeds up any Python code by figuring out the best way to rewrite it while verifying that the behavior of the code is unchanged, and verifying real speed -gains through performance benchmarking. +Codeflash speeds up your code by figuring out the best way to rewrite it while verifying that the behavior is unchanged, and verifying real speed +gains through performance benchmarking. It supports **Python**, **JavaScript**, and **TypeScript**. The optimizations Codeflash finds are generally better algorithms, opportunities to remove wasteful compute, better logic, utilizing caching and utilization of more efficient library methods. Codeflash does not modify the system architecture of your code, but it tries to find the most efficient implementation of your current architecture. +### Get Started + + + + Install via pip, uv, or poetry + + + Install via npm, yarn, pnpm, or bun + + + ### How to use Codeflash - Target and optimize individual Python functions for maximum performance gains. + Target and optimize individual functions for maximum performance gains. ```bash - codeflash --file path.py --function my_function + codeflash --file path/to/file --function my_function ``` - + Automatically find optimizations for Pull Requests with GitHub Actions integration. ```bash codeflash init-actions @@ -29,7 +40,7 @@ does not modify the system architecture of your code, but it tries to find the m - End-to-end optimization of entire Python workflows with execution tracing. + End-to-end optimization of entire workflows with execution tracing. ```bash codeflash optimize myscript.py ``` @@ -42,7 +53,6 @@ does not modify the system architecture of your code, but it tries to find the m ``` - ### How does Codeflash verify correctness? diff --git a/docs/optimizing-with-codeflash/benchmarking.mdx b/docs/optimizing-with-codeflash/benchmarking.mdx index ade7eb023..f373cce81 100644 --- a/docs/optimizing-with-codeflash/benchmarking.mdx +++ b/docs/optimizing-with-codeflash/benchmarking.mdx @@ -1,6 +1,6 @@ --- title: "Optimize Performance Benchmarks with every Pull Request" -description: "Configure and use pytest-benchmark integration for performance-critical code optimization" +description: "Configure and use benchmark integration for performance-critical code optimization" icon: "chart-line" sidebarTitle: Setup Benchmarks to Optimize keywords: @@ -26,6 +26,10 @@ It will then try to optimize the new code for the benchmark and calculate the im ## Using Codeflash in Benchmark Mode + + Benchmark mode currently supports Python projects using pytest-benchmark. JavaScript/TypeScript benchmark support is coming soon. + + 1. **Create a benchmarks root:** Create a directory for benchmarks if it does not already exist. @@ -44,7 +48,7 @@ It will then try to optimize the new code for the benchmark and calculate the im 2. **Define your benchmarks:** - Currently, Codeflash only supports benchmarks written as pytest-benchmarks. Check out the [pytest-benchmark](https://pytest-benchmark.readthedocs.io/en/stable/index.html) documentation for more information on syntax. + Codeflash supports benchmarks written as pytest-benchmarks. Check out the [pytest-benchmark](https://pytest-benchmark.readthedocs.io/en/stable/index.html) documentation for more information on syntax. For example: @@ -58,7 +62,7 @@ It will then try to optimize the new code for the benchmark and calculate the im Note that these benchmarks should be defined in such a way that they don't take a long time to run. - The pytest-benchmark format is simply used as an interface. The plugin is actually not used - Codeflash will run these benchmarks with its own pytest plugin + The pytest-benchmark format is simply used as an interface. The plugin is actually not used - Codeflash will run these benchmarks with its own pytest plugin. 3. **Run and Test Codeflash:** @@ -74,7 +78,7 @@ It will then try to optimize the new code for the benchmark and calculate the im codeflash --file test_file.py --benchmark --benchmarks-root path/to/benchmarks ``` -4. **Run Codeflash :** +4. **Run Codeflash with GitHub Actions:** Benchmark mode is best used together with Codeflash as a GitHub Action. This way, Codeflash will trace through your benchmark and optimize the functions modified in your Pull Request to speed up the benchmark. diff --git a/docs/optimizing-with-codeflash/codeflash-all.mdx b/docs/optimizing-with-codeflash/codeflash-all.mdx index 92a232e67..7749817c7 100644 --- a/docs/optimizing-with-codeflash/codeflash-all.mdx +++ b/docs/optimizing-with-codeflash/codeflash-all.mdx @@ -3,13 +3,13 @@ title: "Optimize Your Entire Codebase" description: "Automatically optimize all codepaths in your project with Codeflash's comprehensive analysis" icon: "database" sidebarTitle: "Optimize Entire Codebase" -keywords: ["codebase optimization", "all functions", "batch optimization", "github app", "checkpoint", "recovery"] +keywords: ["codebase optimization", "all functions", "batch optimization", "github app", "checkpoint", "recovery", "javascript", "typescript", "python"] --- # Optimize your entire codebase Codeflash can optimize your entire codebase by analyzing all the functions in your project and generating optimized versions of them. -It iterates through all the functions in your codebase and optimizes them one by one. +It iterates through all the functions in your codebase and optimizes them one by one. This works for Python, JavaScript, and TypeScript projects. To optimize your entire codebase, run the following command in your project directory: @@ -30,15 +30,27 @@ codeflash --all path/to/dir ``` - If your project has a good number of unit tests, we can trace those to achieve higher quality results. - The following approach is recommended instead: + If your project has a good number of unit tests, tracing them achieves higher quality results. + + + ```bash codeflash optimize --trace-only -m pytest tests/ ; codeflash --all ``` - This will run your test suite, trace all the code covered by your tests, ensuring higher correctness guarantees - and better performance benchmarking, and help create optimizations for code where the LLMs struggle to generate and run tests. + + + ```bash + codeflash optimize --trace-only --jest ; codeflash --all + # or for Vitest projects + codeflash optimize --trace-only --vitest ; codeflash --all + ``` + + + + This runs your test suite, traces all the code covered by your tests, ensuring higher correctness guarantees + and better performance benchmarking, and helps create optimizations for code where the LLMs struggle to generate and run tests. - Even though `codeflash --all` discovers any existing unit tests. It currently can only discover any test that directly calls the + `codeflash --all` discovers any existing unit tests, but it currently can only discover tests that directly call the function under optimization. Tracing all the tests helps ensure correctness for code that may be indirectly called by your tests. diff --git a/docs/optimizing-with-codeflash/codeflash-github-actions.mdx b/docs/optimizing-with-codeflash/codeflash-github-actions.mdx index b8da4ebac..7cd5a5fe5 100644 --- a/docs/optimizing-with-codeflash/codeflash-github-actions.mdx +++ b/docs/optimizing-with-codeflash/codeflash-github-actions.mdx @@ -26,9 +26,9 @@ We highly recommend setting this up, since once you set it up all your new code ✅ A Codeflash API key from the [Codeflash Web App](https://app.codeflash.ai/) -✅ Completed [local installation](/getting-started/local-installation) with `codeflash init` +✅ Completed local installation with `codeflash init` ([Python](/getting-started/local-installation) or [JavaScript/TypeScript](/getting-started/javascript-installation)) -✅ A Python project with a configured `pyproject.toml` file +✅ A configured project (`pyproject.toml` for Python, `package.json` for JavaScript/TypeScript) ## Setup Options @@ -113,7 +113,7 @@ jobs: -Customize the dependency installation based on your Python package manager: +Customize the dependency installation based on your package manager: The workflow will need to be set up in such a way the Codeflash can create and run tests for functionality and speed, so the stock YAML may need to be altered to @@ -121,7 +121,7 @@ suit the specific codebase. Typically the setup steps for a unit test workflow c be copied. -```yaml Poetry +```yaml Poetry (Python) - name: Install Project Dependencies run: | python -m pip install --upgrade pip @@ -129,11 +129,11 @@ be copied. poetry install --with dev - name: Run Codeflash to optimize code run: | - poetry env use python + poetry env use python poetry run codeflash ``` -```yaml uv +```yaml uv (Python) - uses: astral-sh/setup-uv@v6 with: enable-cache: true @@ -142,7 +142,7 @@ be copied. run: uv run codeflash ``` -```yaml pip +```yaml pip (Python) - name: Install Project Dependencies run: | python -m pip install --upgrade pip @@ -151,6 +151,16 @@ be copied. - name: Run Codeflash to optimize code run: codeflash ``` + +```yaml npm (JavaScript/TypeScript) +- uses: actions/setup-node@v4 + with: + node-version: '18' +- name: Install Project Dependencies + run: npm ci +- name: Run Codeflash to optimize code + run: npx codeflash +``` diff --git a/docs/optimizing-with-codeflash/one-function.mdx b/docs/optimizing-with-codeflash/one-function.mdx index 8f60db0a7..194531198 100644 --- a/docs/optimizing-with-codeflash/one-function.mdx +++ b/docs/optimizing-with-codeflash/one-function.mdx @@ -1,6 +1,6 @@ --- title: "Optimize a Single Function" -description: "Target and optimize individual Python functions for maximum performance gains" +description: "Target and optimize individual functions for maximum performance gains" icon: "bullseye" sidebarTitle: "Optimize Single Function" keywords: @@ -10,6 +10,9 @@ keywords: "class methods", "performance", "targeted optimization", + "javascript", + "typescript", + "python", ] --- @@ -24,23 +27,55 @@ your mileage may vary. ## How to optimize a function -To optimize a function, you can run the following command in your project: +To optimize a function, run the following command in your project: + + ```bash codeflash --file path/to/your/file.py --function function_name ``` + + +```bash +codeflash --file path/to/your/file.js --function functionName +``` + + +```bash +codeflash --file path/to/your/file.ts --function functionName +``` + + If you have installed the GitHub App to your repository, the above command will open a pull request with the optimized function. -If you want to optimize a function locally, you can add a `--no-pr` argument as follows: +If you want to optimize a function locally, add a `--no-pr` argument: + + ```bash codeflash --file path/to/your/file.py --function function_name --no-pr ``` + + +```bash +codeflash --file path/to/your/file.ts --function functionName --no-pr +``` + + ### Optimizing class methods -To optimize a method `method_name` in a class `ClassName`, you can run the following command: +To optimize a method `methodName` in a class `ClassName`: + + ```bash codeflash --file path/to/your/file.py --function ClassName.method_name ``` + + +```bash +codeflash --file path/to/your/file.ts --function ClassName.methodName +``` + + diff --git a/docs/optimizing-with-codeflash/trace-and-optimize.mdx b/docs/optimizing-with-codeflash/trace-and-optimize.mdx index 3f4e23465..fb62ea1c1 100644 --- a/docs/optimizing-with-codeflash/trace-and-optimize.mdx +++ b/docs/optimizing-with-codeflash/trace-and-optimize.mdx @@ -1,6 +1,6 @@ --- title: "Trace & Optimize E2E Workflows" -description: "End-to-end optimization of entire Python workflows with execution tracing" +description: "End-to-end optimization of entire workflows with execution tracing" icon: "route" sidebarTitle: "Optimize E2E Workflows" keywords: @@ -11,28 +11,50 @@ keywords: "end-to-end", "script optimization", "context manager", + "javascript", + "typescript", + "jest", + "vitest", ] --- -Codeflash can optimize an entire Python script end-to-end by tracing the script's execution and generating Replay Tests. -Tracing follows the execution of a script, profiles it and captures inputs to all functions it called, allowing them to be replayed during optimization. -Codeflash uses these Replay Tests to optimize the most important functions called in the script, delivering the best performance for your workflow. +Codeflash can optimize an entire script or test suite end-to-end by tracing its execution and generating Replay Tests. +Tracing follows the execution of your code, profiles it and captures inputs to all functions it called, allowing them to be replayed during optimization. +Codeflash uses these Replay Tests to optimize the most important functions called in the workflow, delivering the best performance. ![Function Optimization](/images/priority-order.png) -To optimize a script, `python myscript.py`, simply replace `python` with `codeflash optimize` and run the following command: + + +To optimize a script, `python myscript.py`, simply replace `python` with `codeflash optimize`: ```bash codeflash optimize myscript.py ``` -You can also optimize code called by pytest tests that you could normally run like `python -m pytest tests/`, this provides for a good workload to optimize. Run this command: +You can also optimize code called by pytest tests: ```bash codeflash optimize -m pytest tests/ ``` + + +To trace and optimize your Jest or Vitest tests: -The powerful `codeflash optimize` command creates high-quality optimizations, making it ideal when you need to optimize a workflow or script. The initial tracing process can be slow, so try to limit your script's runtime to under 1 minute for best results. If your workflow is longer, consider tracing it into smaller sections by using the Codeflash tracer as a context manager (point 3 below). +```bash +# Jest +codeflash optimize --jest + +# Vitest +codeflash optimize --vitest + +# Or trace a specific script +codeflash optimize --language javascript script.js +``` + + + +The `codeflash optimize` command creates high-quality optimizations, making it ideal when you need to optimize a workflow or script. The initial tracing process can be slow, so try to limit your script's runtime to under 1 minute for best results. The generated replay tests and the trace file are for the immediate optimization use, don't add them to git. @@ -61,6 +83,9 @@ This way you can be _sure_ that the optimized function causes no changes of beha ## Using codeflash optimize + + + Codeflash script optimizer can be used in three ways: 1. **As an integrated command** @@ -100,10 +125,10 @@ Codeflash script optimizer can be used in three ways: - `--timeout`: The maximum time in seconds to trace the entire workflow. Default is indefinite. This is useful while tracing really long workflows. -3. **As a Context Manager -** +3. **As a Context Manager** - To trace only specific sections of your code, You can also use the Codeflash Tracer as a context manager. - You can wrap the code you want to trace in a `with` statement as follows - + To trace only specific sections of your code, you can use the Codeflash Tracer as a context manager. + You can wrap the code you want to trace in a `with` statement as follows: ```python from codeflash.tracer import Tracer @@ -128,3 +153,46 @@ Codeflash script optimizer can be used in three ways: - `output`: The file to save the trace to. Default is `codeflash.trace`. - `config_file_path`: The path to the `pyproject.toml` file which stores the Codeflash config. This is auto-discovered by default. You can also disable the tracer in the code by setting the `disable=True` option in the `Tracer` constructor. + + + + +The JavaScript tracer uses Babel instrumentation to capture function calls during your test suite execution. + +1. **Trace your test suite** + + ```bash + # Jest projects + codeflash optimize --jest + + # Vitest projects + codeflash optimize --vitest + + # Trace a specific script + codeflash optimize --language javascript src/main.js + ``` + +2. **Trace specific functions only** + + ```bash + codeflash optimize --jest --only-functions processData,transformInput + ``` + +3. **Trace and optimize as two separate steps** + + ```bash + # Step 1: Create trace file + codeflash optimize --trace-only --jest --output trace_file.sqlite + + # Step 2: Optimize with replay tests + codeflash --replay-test /path/to/test_replay_test_0.test.js + ``` + + More Options: + + - `--timeout`: Maximum tracing time in seconds. + - `--max-function-count`: Maximum traces per function (default: 256). + - `--only-functions`: Comma-separated list of function names to trace. + + + diff --git a/packages/codeflash/package-lock.json b/packages/codeflash/package-lock.json index aaffaef6a..ea4b361f6 100644 --- a/packages/codeflash/package-lock.json +++ b/packages/codeflash/package-lock.json @@ -1,12 +1,12 @@ { "name": "codeflash", - "version": "0.8.0", + "version": "0.9.0", "lockfileVersion": 3, "requires": true, "packages": { "": { "name": "codeflash", - "version": "0.8.0", + "version": "0.9.0", "hasInstallScript": true, "license": "MIT", "dependencies": { diff --git a/packages/codeflash/package.json b/packages/codeflash/package.json index dfd3abdf1..a7f712157 100644 --- a/packages/codeflash/package.json +++ b/packages/codeflash/package.json @@ -1,6 +1,6 @@ { "name": "codeflash", - "version": "0.8.0", + "version": "0.9.0", "description": "Codeflash - AI-powered code optimization for JavaScript and TypeScript", "main": "runtime/index.js", "types": "runtime/index.d.ts",