Personal Assistant Notetaking Application

A local, web-based personal assistant that serves as an intelligent notetaking application. The system automatically classifies user input as questions or statements, provides answers to questions using a vector database, and stores statements as notes for future reference.

Features

🤖 Intelligent Input Classification

• Question Detection: Automatically identifies questions using NLP techniques
• Statement Recognition: Captures facts, observations, and personal notes
• Real-time Classification: Shows input type as you type

💬 Question Answering System

• Vector Database Integration: Uses ChromaDB for semantic search
• Local LLM Integration: Powered by Ollama for privacy
• Context-Aware Answers: Provides relevant source references
• Metadata Filtering: Filter by source types and tags

📝 Note Management

• Tag-based Organization: AI-powered tag suggestions
• Search & Filter: Find notes by content or tags
• Export Functionality: Export in Markdown, JSON, or plain text
• Local Storage: All data stored locally for privacy

⚙️ Model Management

• Multiple Models: Support for Phi-3 Mini, Mistral 7B, Llama 3.1, and more
• Dynamic Switching: Change models during runtime
• Model Optimization: Automatic quantization and memory management
• Comprehensive Model Information: Display context window size, memory usage, inference speed, and capabilities
• Expandable Model Cards: Detailed view with architecture, quantization, and license information

🧠 AI Prompt Engineering

• Optimized System Prompts: Carefully crafted prompts for better AI responses
• Context-Aware Processing: Intelligent handling of conversation history and knowledge base context
• Human-Readable Tool Output: Clear, natural language descriptions of tool operations
• Intent Classification Integration: Seamless integration of user intent analysis into response generation

Technology Stack

Backend

• FastAPI: Modern Python web framework
• ChromaDB: Local vector database
• Ollama: Local LLM management
• Sentence Transformers: Text embeddings
• Pydantic: Data validation

Frontend

• React 18: Modern UI framework
• TypeScript: Type safety
• Tailwind CSS: Utility-first styling
• Vite: Fast build tool
• Lucide React: Beautiful icons

System Requirements

• Memory: 24GB RAM (for optimal performance with 7B models)
• Storage: 10GB free space
• OS: macOS, Linux, or Windows
• Python: 3.8+
• Node.js: 18+

Installation

1. Clone the Repository

git clone <repository-url>
cd vision

2. Install Ollama

First, install Ollama from ollama.ai

3. Install Backend Dependencies

cd backend
pip install -r requirements.txt

4. Install Frontend Dependencies

cd frontend
npm install

5. Download Models

# Install the default model (Phi-3 Mini)
ollama pull phi3-mini

# Optional: Install additional models
ollama pull mistral:7b-instruct
ollama pull llama3.1:8b

Running the Application

Quick Start (Recommended)

# From the project root directory
./run_tests.sh    # Run all tests
./start.sh        # Start both backend and frontend

Manual Start

1. Start the Backend

cd backend
python3 main.py

The backend will be available at http://localhost:8000

2. Start the Frontend

cd frontend
npm run dev

The frontend will be available at http://localhost:3000

3. Access the Application

Open your browser and navigate to http://localhost:3000

Usage

Asking Questions

1. Type your question in the input area
2. The system will automatically classify it as a question
3. Click "Ask Question" or press Enter
4. View the AI-generated answer with source references

Writing Notes

1. Type your statement or observation
2. The system will classify it as a statement
3. Review suggested tags and select relevant ones
4. Click "Save Note" to store it

Managing Notes

• View Notes: Navigate to the Notes tab
• Search: Use the search bar to find specific notes
• Filter: Click on tags to filter notes
• Export: Use the Settings tab to export notes

Model Management

• View Models: Check the Settings tab for available models
• Switch Models: Click "Activate" on any model
• Monitor Usage: View system information and model status

API Endpoints

Questions

• POST /api/v1/question - Ask a question and get an answer

Notes

• POST /api/v1/notes - Create a new note
• GET /api/v1/notes - Get all notes

Tags

• POST /api/v1/suggest-tags - Get tag suggestions for content

Models

• GET /api/v1/models - List available models
• POST /api/v1/models/{model_name}/activate - Activate a model

Export

• POST /api/v1/export - Export notes in specified format

Configuration

Backend Configuration

Edit backend/app/core/config.py to modify:

• Database paths
• Ollama settings
• Model configurations
• API settings

Frontend Configuration

Edit frontend/src/services/api.ts to change:

• API base URL
• Request headers
• Timeout settings

Testing

Run All Tests

# From the project root directory
./run_tests.sh

This script will:

• Activate the virtual environment
• Run all backend tests using python3
• Provide detailed test results and coverage

Individual Test Files

cd backend
source venv/bin/activate

# Run specific tests
python3 test_similarity_threshold.py
python3 test_story_generation.py

# Or use the test runner directly
python3 run_tests.py

Development

Backend Development

cd backend
# Install development dependencies
pip install -r requirements.txt

# Run with auto-reload
uvicorn main:app --reload --host 0.0.0.0 --port 8000

Frontend Development

cd frontend
# Install dependencies
npm install

# Start development server
npm run dev

# Build for production
npm run build

Troubleshooting

Common Issues

1. Ollama Connection Error

   • Ensure Ollama is running: ollama serve
   • Check if models are installed: ollama list

2. Memory Issues

   • Use smaller models (Phi-3 Mini instead of 7B models)
   • Close other applications to free memory
   • Consider model quantization

3. ChromaDB Errors

   • Check if the data directory exists and is writable
   • Restart the backend to reinitialize the database

4. Frontend Build Issues

   • Clear node_modules and reinstall: rm -rf node_modules && npm install
   • Check Node.js version compatibility

Performance Optimization

1. Model Selection

   • Use Phi-3 Mini for real-time interactions
   • Use larger models for complex reasoning tasks

2. Memory Management

   • Monitor memory usage with Activity Monitor/Task Manager
   • Restart the application if memory usage is high

3. Database Optimization

   • Regularly export and backup notes
   • Clean up old embeddings if needed

Contributing

1. Fork the repository
2. Create a feature branch
3. Make your changes
4. Add tests if applicable
5. Submit a pull request

License

This project is licensed under the MIT License - see the LICENSE file for details.

Support

For issues and questions:

1. Check the troubleshooting section
2. Review the API documentation at http://localhost:8000/docs
3. Open an issue on GitHub

---

Note: This application runs completely locally for maximum privacy. All data is stored on your machine and no information is sent to external services.