Skip to content

Human in Loop Design

Jump to bottom
iAmGiG edited this page Nov 15, 2025 · 2 revisions

Human-in-Loop Design

AutoTrader-AgentEdge is designed for human oversight, not autonomous AI trading. The system assists traders by providing validated signals, but humans make the final decisions.

Core Philosophy

"Pure mathematical indicators + human decision making > complex AI sentiment analysis"

Why Human-in-Loop?

  1. Accountability: Humans are responsible for trading decisions
  2. Context: Humans understand market conditions, news, and edge cases
  3. Risk Management: Humans can override signals based on portfolio needs
  4. Regulatory: Many jurisdictions require human oversight for trading
  5. Learning: Humans improve by understanding why signals work or fail

Current Implementation

VoterAgent: Signal Generator

The VoterAgent generates recommendations, not orders:

result = voter.evaluate_voting('SPY', market_data)
# Returns: {'action': 'BUY', 'confidence': 0.85, 'reasoning': '...'}

Human Decision Point: Review the signal, confidence, and reasoning before executing.

main.py CLI: Manual Execution

All main.py commands require human initiation:

# Human reviews output before any action
python main.py test-voter

# Human decides whether to execute suggested actions
python main.py paper-trade SPY

Paper Trading Cycle

The paper-trade command performs automated checks but requires human review:

  1. System reconciles state
  2. System reviews positions
  3. System calculates recommended adjustments
  4. System displays recommendations
  5. Human reviews output
  6. System only executes if explicitly confirmed

Currently, the system displays recommendations but doesn't require explicit confirmation (Issue #308).

Planned Enhancements (Issue #308)

Interactive CLI for Trade Approval

Target: Q4 2025

Features:

python main.py --interactive

# Output:
📊 VoterAgent Recommendation:
   Symbol: SPY
   Action: BUY
   Confidence: 85%
   Reasoning: Strong consensus - MACD and RSI both bullish
   Current Price: $450.25
   Position Size: 100 shares
   Risk: $2,250 (5% stop loss)

❓ Approve this trade? [y/N/m(odify)/s(kip)]:

Capabilities:

  • Review each recommendation individually
  • Modify position size or parameters
  • Skip recommendations without penalty
  • Add manual notes and reasoning
  • Track approval/rejection patterns

Enhanced Decision Support

Features:

  • Display recent performance metrics
  • Show portfolio impact simulation
  • Risk assessment summary
  • Historical similar signals and outcomes
  • Market condition context

Override Mechanisms

Human Control Points:

  1. Pre-approval: Review before any order placement
  2. Modification: Adjust size, entry price, or risk parameters
  3. Rejection: Skip signals without affecting future recommendations
  4. Emergency Stop: Immediately halt all automated checks
  5. Manual Orders: Place orders outside of VoterAgent signals

What the System Does NOT Do

❌ Autonomous Trading

The system does NOT:

  • Place orders without human initiation
  • Make trading decisions independently
  • Override human decisions
  • Execute trades on a schedule automatically
  • Learn or adapt trading strategy based on outcomes

✅ What It DOES Do

The system DOES:

  • Generate validated trading signals
  • Calculate risk parameters
  • Monitor position health
  • Suggest stop-loss adjustments
  • Provide comprehensive analysis
  • Track performance metrics

Design Principles

1. Transparency

All decisions are explainable:

  • MACD histogram value
  • RSI level
  • Voting logic reasoning
  • Position sizing calculation
  • Risk parameters

2. Determinism

Same inputs → same outputs:

  • Pure mathematical calculations
  • No randomness or LLM uncertainty
  • Reproducible signals
  • Auditable decision logic

3. Reversibility

All actions can be undone:

  • Positions can be closed manually
  • Orders can be cancelled
  • System state can be reset
  • No irreversible automated actions

4. Visibility

Complete system state is accessible:

  • Current positions
  • Open orders
  • Recent decisions
  • Performance history
  • Risk exposure

Comparison: Autonomous vs Human-in-Loop

Feature Autonomous AI Human-in-Loop (This System)
Decision Making AI decides Human decides
Order Execution Automatic Manual approval
Risk Management AI-controlled Human-controlled
Accountability Unclear Clear (human)
Adaptability AI learns Human learns
Regulation Complex Straightforward
Trust Black box Transparent

Future Enhancements

Planned Features

  1. Trade Approval Dashboard (Issue #308)

    • Web-based interface
    • Mobile notifications
    • Batch approval workflow

  2. Decision Analytics

    • Track human approval patterns
    • Measure signal vs execution performance
    • Identify learning opportunities

  3. Portfolio Context

    • Show overall portfolio impact
    • Display correlation with existing positions
    • Highlight concentration risks

  4. Smart Defaults

    • Learn from human preferences
    • Suggest position sizes based on past approvals
    • Pre-filter low-confidence signals

NOT Planned

  • Autonomous execution: Will always require human approval
  • LLM decision-making: Pure math stays pure math
  • Adaptive strategies: Parameters stay validated and locked

Why NOT Full Automation?

Historical Context

The project tested extensive LLM-based sentiment analysis (V0-V4 framework) and found:

  • Pure MACD+RSI (0.856 Sharpe) > LLM sentiment (~60% accuracy)
  • Mathematical signals more reliable than AI reasoning
  • Simpler systems easier to understand and debug
  • Human oversight adds value that AI cannot

Design Decision

Rather than pursue complex autonomous AI trading, the project focuses on:

  • Validated pure math signals (proven 0.856 Sharpe)
  • Human decision intelligence (context, judgment, risk tolerance)
  • Cost-efficient execution (90% fewer API calls)
  • Transparent operation (explainable decisions)

Getting Started

Review Signals

# Generate signal for review
python main.py test-voter

Monitor Positions

# Check current positions daily
python main.py check-positions

Trading Cycle

# Run daily checks (morning and evening)
python main.py paper-trade SPY

Coming Soon: Interactive Mode

# Interactive approval workflow (Issue #308)
python main.py --interactive

Best Practices

Daily Routine

  1. Morning (9:20 AM): Run paper-trade to review overnight changes
  2. Midday: Check positions if significant market moves
  3. Evening (3:50 PM): Run paper-trade for EOD review
  4. Weekly: Analyze performance and adjust as needed

Decision Framework

  1. Review Signal: Understand VoterAgent reasoning
  2. Check Context: Look at news, market conditions, portfolio
  3. Assess Risk: Consider position sizing and stops
  4. Make Decision: Approve, modify, or reject
  5. Execute: Place order manually or via system
  6. Monitor: Track position performance

Risk Controls

  1. Position Limits: Max 3 positions, 33% each
  2. Stop Losses: 5% default (balanced strategy)
  3. Take Profits: 8% default (balanced strategy)
  4. Review Frequency: At least twice daily
  5. Override Authority: Human can always override

Next Steps

Design Philosophy | Human Oversight Required | Issue #308 for Interactive CLI

Clone this wiki locally