add long term memory

[YuchengZhou821]

This pull request introduces a new long-term memory subsystem to the Fuser application, enabling persistent storage and retrieval of facts and daily interactions. The changes add support for reading and writing memory, searching daily logs using embeddings, and configuring memory via the runtime configuration. The most important changes are grouped below:

Long-Term Memory Subsystem Implementation:

• Added MemoryReader and MemoryWriter classes to provide reading, searching, and writing capabilities for persistent memory, including support for a MEMORY.md file (for facts) and daily markdown logs (for interactions and summaries). (src/fuser/memory_base/reader.py, src/fuser/memory_base/writer.py) [1] [2]
• Introduced MemoryIndex for in-memory embedding-based search over memory chunks, with support for batch indexing, cosine similarity search, and daily log parsing. (src/fuser/memory_base/indexer.py)

Integration with Fuser Core:

• Updated src/fuser/__init__.py to initialize and use MemoryReader and MemoryWriter based on new configuration options, and to inject relevant memory context into the fused prompt during operation. [1] [2] [3] [4] [5]

Configuration Enhancements:

• Extended RuntimeConfig and ModeSystemConfig to support a new memory configuration block, allowing memory features to be enabled or disabled via config files. Updated config loading and serialization logic accordingly. (src/runtime/config.py) [1] [2] [3] [4] [5]

These changes lay the foundation for persistent, queryable long-term memory, improving the application's ability to recall and leverage past interactions and facts.

[Copilot]

Pull request overview

Adds a new long-term memory subsystem to Fuser, enabling persistent “facts” (MEMORY.md) and embedding-based retrieval over daily interaction logs, wired through runtime configuration and prompt fusion.

Changes:

• Introduces MemoryReader/MemoryWriter for reading/searching and writing memory files.
• Adds MemoryIndex and supporting parsing/index population for embedding-based daily-log search.
• Extends runtime config/conversion and integrates memory context injection and interaction logging into the runtime tick.

Reviewed changes

Copilot reviewed 7 out of 8 changed files in this pull request and generated 8 comments.

Show a summary per file

File	Description
src/runtime/cortex.py	Appends Voice interactions to memory after actions execute.
src/runtime/converter.py	Carries memory config through the single→multi conversion global section.
src/runtime/config.py	Adds memory to RuntimeConfig/ModeSystemConfig and (de)serialization.
src/fuser/__init__.py	Initializes memory components and injects memory context into the fused prompt.
src/fuser/memory_base/indexer.py	Implements in-memory embedding index, daily parsing, and index population/retention behavior.
src/fuser/memory_base/reader.py	Implements reading MEMORY.md, searching daily logs, and formatting memory context.
src/fuser/memory_base/writer.py	Implements writing daily interactions/summaries and appending facts to MEMORY.md.
src/fuser/memory_base/__init__.py	Adds the new memory_base package.

[codecov]

❌ 5 Tests Failed:

Tests completed	Failed	Passed	Skipped
3772	5	3767	6

View the top 3 failed test(s) by shortest run time

tests/fuser/memory_base/test_summarizer.py::TestExtractReviewableFacts::test_empty_memory

Stack Traces | 0.001s run time

self = <tests.fuser.memory_base.test_summarizer.TestExtractReviewableFacts object at 0x7f072ba4e120>

    def test_empty_memory(self):
>       assert MemorySummarizer._extract_reviewable_facts("# Memory\n") == []
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
E       AttributeError: 'function' object has no attribute '_extract_reviewable_facts'

.../fuser/memory_base/test_summarizer.py:454: AttributeError

tests/fuser/memory_base/test_summarizer.py::TestExtractReviewableFacts::test_extracts_preferences_and_facts

Stack Traces | 0.001s run time

self = <tests.fuser.memory_base.test_summarizer.TestExtractReviewableFacts object at 0x7f072ba4cfe0>

    def test_extracts_preferences_and_facts(self):
        content = (
            "# Memory\n\n"
            "## Identity\n- User is Alice\n\n"
            "## Preferences\n- User likes coffee <!-- expired: 0 -->\n\n"
            "## Facts\n- User lives in SF <!-- expired: 2 -->\n"
        )
>       result = MemorySummarizer._extract_reviewable_facts(content)
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
E       AttributeError: 'function' object has no attribute '_extract_reviewable_facts'

.../fuser/memory_base/test_summarizer.py:438: AttributeError

tests/fuser/memory_base/test_summarizer.py::TestExtractReviewableFacts::test_handles_no_marker

Stack Traces | 0.001s run time

self = <tests.fuser.memory_base.test_summarizer.TestExtractReviewableFacts object at 0x7f072ba4c260>

    def test_handles_no_marker(self):
        content = "## Facts\n- plain fact\n"
>       result = MemorySummarizer._extract_reviewable_facts(content)
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
E       AttributeError: 'function' object has no attribute '_extract_reviewable_facts'

.../fuser/memory_base/test_summarizer.py:450: AttributeError

tests/fuser/memory_base/test_summarizer.py::TestExtractReviewableFacts::test_no_reviewable_sections

Stack Traces | 0.001s run time

self = <tests.fuser.memory_base.test_summarizer.TestExtractReviewableFacts object at 0x7f072ba4e450>

    def test_no_reviewable_sections(self):
        content = "# Memory\n\n## Identity\n- User is Bob\n"
>       assert MemorySummarizer._extract_reviewable_facts(content) == []
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
E       AttributeError: 'function' object has no attribute '_extract_reviewable_facts'

.../fuser/memory_base/test_summarizer.py:458: AttributeError

tests/fuser/memory_base/test_summarizer.py::TestExtractReviewableFacts::test_strips_expired_marker

Stack Traces | 0.001s run time

self = <tests.fuser.memory_base.test_summarizer.TestExtractReviewableFacts object at 0x7f072ba4c800>

    def test_strips_expired_marker(self):
        content = "## Facts\n- some fact <!-- expired: 3 -->\n"
>       result = MemorySummarizer._extract_reviewable_facts(content)
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
E       AttributeError: 'function' object has no attribute '_extract_reviewable_facts'

.../fuser/memory_base/test_summarizer.py:445: AttributeError

To view more test analytics, go to the Test Analytics Dashboard
_{📋 Got 3 mins? Take this short survey to help us improve Test Analytics.}

[Copilot]

Pull request overview

Copilot reviewed 7 out of 8 changed files in this pull request and generated 10 comments.

Comments suppressed due to low confidence (1)

src/fuser/init.py:114

• query_text is derived from the voice input in the newly added block, but the knowledge-base query block re-fetches Voice and resets query_text to None again. This duplicates IOProvider calls and makes it easier for KB/memory to diverge on which utterance they query. Consider computing query_text once per tick and reusing it for both KB and memory queries.

        query_text = None
        voice_input = self.io_provider.get_input("Voice")
        if voice_input and voice_input.input and self.io_provider.tick_counter == voice_input.tick:
            query_text = voice_input.input.strip()

        # Query the knowledge base if configured and if there are inputs to query with
        kb_context = ""
        if self.knowledge_base and inputs_fused:
            try:
                query_text = None
                voice_input = self.io_provider.get_input("Voice")
                if voice_input and voice_input.input and self.io_provider.tick_counter == voice_input.tick:
                    query_text = voice_input.input.strip()