Skip to content

README.md

Latest commit

 

History

History
260 lines (183 loc) · 8.86 KB

README.md

File metadata and controls

260 lines (183 loc) · 8.86 KB

🚀 Prompt Optimization

⚠️ The Core Problem

Traditional code editing approaches like toolName('old_string', 'new_string') are inefficient because:

  • Token Waste: 80-95% of tokens are spent on unchanged code
  • Accuracy Problems: LLM might match wrong string in large files, causing incorrect changes
  • Pattern Learning Failure: AI can't learn editing patterns from full file dumps
  • Streaming Inefficiency: Large contexts prevent real-time suggestion streaming
  • Cost Explosion: Paying for redundant information that doesn't help predictions

✅ Optimized Approach

🔧 Unified Diff Format for Pattern Learning

LLM can use custom unified diff format to enable:

  • Sequential pattern recognition - AI learns from your editing sequence
  • Real-time streaming - Suggestions appear as you type
  • Minimal context - Only send what's necessary for predictions
Custom Diff Format:
@@ filepath:lineNumber
- old content
+ new content

📚 For detailed unified diff format explanation, see the main README

📊 Token Efficiency Math

Efficiency = (Context + Changes + Metadata) / Full_File_Tokens × 100%

Typical Results:
- Full file approach: 1000+ tokens (100% waste on unchanged code)
- Custom diff approach: 50-100 tokens (90%+ token reduction)
- Pattern learning: Enabled (vs disabled with full files)

🎯 Accuracy Problems with Full File Matching

When using toolName('old_string', 'new_string') with large files:

Accuracy_Risk = Duplicate_Strings / Total_Strings × 100%

Example Problem:
File has 3 identical functions:
- function add(a, b) { return a + b; }
- function add(a, b) { return a + b; }  // Duplicate!
- function add(a, b) { return a + b; }  // Duplicate!

LLM might change the wrong one!

Worse Problem - Code Duplication:
User asks: "Add error handling to add function"
LLM response: Adds error handling + duplicates function + adds popular patterns
Result: Messy code with duplications and junk!

Common Accuracy Issues:

  • AI not reading full prompt: LLM skips important instructions in long prompts
  • Context Confusion: LLM picks wrong occurrence
  • Partial Matches: Matches substring instead of full context
  • Duplicate Code: Same function appears multiple times
  • Similar Patterns: return a + b; appears in 10+ places
  • Code Duplication: AI adds same function multiple times when editing
  • Junk Code: AI inserts popular/unwanted patterns that don't belong

🧠 Pattern Learning Strategy

🔄 Sequential Edit Detection

LLM learns patterns by analyzing your editing sequence:

Pattern Learning = Analyze(Edit_Sequence) → Predict(Next_Edit)

Example Sequence:
1. Add JSDoc to function A
2. Add JSDoc to function B
3. AI predicts: "Add JSDoc to function C"

⚙️ Context Optimization

Context = {
  recent_edits: last_5_changes,
  file_structure: function_signatures,
  cursor_position: current_location,
  editing_pattern: detected_behavior
}

Token_Count = 50-100 (vs 1000+ for full file)

💡 Real Editing Examples

❌ Traditional Approach (Breaks AI Learning + Accuracy Issues)

// PROBLEM: Sends entire file - AI can't learn patterns + might match wrong string
const prompt = `
... Messy rules
... Context
... Over-engineered prompt

Here's the full file:
${entireFileContent}

Please change or add some features
`;

// RISK: If file has duplicate functions, LLM might change the wrong one!
// Example: File has 3 identical "add" functions, LLM changes wrong occurrence
// Result: No pattern learning, 1000+ tokens wasted, accuracy problems

✅ Simple Optimized Approach (Enables Learning + Perfect Accuracy)

// SOLUTION: Sends only changes with exact line targeting
const prompt = `
... Explain formating
... Minimal context

Recent edits:
@@ src/calculator.ts:12
-  return a + b;
+  return a + b; // Added comment

@@ src/calculator.ts:18  
-  return a - b;
+  return a - b; // Added comment

Predict next edit for similar functions...
`;

// BENEFIT: Exact line targeting prevents wrong matches
// Result: Pattern learning enabled, 80 tokens, 10x faster, 100% accuracy

📈 Performance Math

🎯 Pattern Learning Efficiency

Learning_Efficiency = Pattern_Matches / Total_Suggestions × 100%
Accuracy_Rate = Correct_Changes / Total_Changes × 100%

With Custom Diff:

  • Pattern recognition: 70-90% accuracy (optimal conditions)
  • Change accuracy: 95%+ (exact line targeting)
  • Token efficiency: 85%+ savings (realistic usage)
  • Streaming speed: 5-8x faster (with filtering)

With Full File:

  • Pattern recognition: 0-10% (minimal learning)
  • Change accuracy: 60-80% (string matching issues)
  • Token efficiency: 0% (all waste)
  • Streaming speed: 1x (baseline)

💰 Cost-Benefit for Custom Diff

ROI = (Time_Savings + Token_Savings + Productivity_Gains) / Development_Cost

Where:
- Time_Savings = (Response_Time_Old - Response_Time_New) × Usage_Frequency × Time_Value
- Token_Savings = (Tokens_Old - Tokens_New) × API_Cost_Per_Token × Usage_Frequency
- Productivity_Gains = Improved_Suggestion_Accuracy × Developer_Hourly_Rate × Time_Saved
- Development_Cost = Implementation_Hours × Developer_Hourly_Rate

Example Calculation (Realistic):
- Time_Savings = (8s - 2s) × 50_daily_requests × $50/hour = $5/day
- Token_Savings = (800 - 120) × $0.002/token × 50_daily = $68/day
- Productivity_Gains = 10% × $50/hour × 1.5_hours = $7.50/day
- Development_Cost = 40_hours × $50/hour = $2,000

ROI = ($5 + $68 + $7.50) × 30_days / $2,000 = 121% monthly ROI

Note: These are realistic performance metrics. Results may vary based on model complexity, context quality, and implementation.

📋 Decision Matrix

Scenario Change Density File Size Use Custom Diff? Reason
Refactoring < 30% > 100 lines ✅ Yes Targeted changes with pattern learning
Adding comments < 20% > 200 lines ✅ Yes Perfect for pattern learning
Bug fixes < 30% Any size ✅ Yes Learn from similar fixes
New features < 40% Any size ✅ Yes Easy for identified & added features on specific lines
Major rewrite > 50% Any size ❌ No Too many changes, use full file approach
Complex changes 40-50% Any size ⚠️ Maybe Depends on change complexity and file size

🚀 Real Implementation

This algorithm has been successfully implemented in my private workspace development and works exceptionally well with free models. I'm using semantic research as described here: [Code indexing examples]

💡 Key Insights from Production Use:

  • Bottom Line: Not sending full context to the LLM significantly increases accuracy.
  • The Solution: Provide correct context & clear explanations of what the LLM needs. If users can't explain their requirements clearly, let the LLM learn from their workspace patterns instead.

⚡ Why This Works:

  • Free models perform better with focused, relevant context
  • Pattern learning from workspace behavior is more effective than verbose prompts
  • Semantic understanding of code structure beats raw file dumps
  • Targeted suggestions based on actual usage patterns

🏗️ Technical Architecture

The implementation leverages several key components:

1. Semantic Code Analysis

  • Uses simple data structures to understand code structure and relationships
  • Implements repository mapping for intelligent context selection
  • Maintains lightweight snapshots of project architecture

2. Pattern Recognition

  • Tracks sequential editing behaviors across multiple files
  • Learns from user acceptance/rejection patterns
  • Filters unrelated context based on active workspace
  • Maintains sliding window of recent changes (5-10 edits)
  • Implements intelligent caching for repeated patterns

3. Real-time Streaming

  • Processes suggestions as they're generated
  • Implements abort mechanisms for off-target responses
  • Provides immediate feedback loops for learning

📚 References