Local Memory Mechanism in QonQrete

QonQrete implements a sophisticated local memory mechanism designed to provide its AI agents (including Qwen) with both short-term operational awareness and structured, long-term knowledge about the codebase. This approach aims to keep AI models grounded in relevant information while optimizing token usage.

The local memory is managed across several key directories and file types, orchestrated by qrane/paths.py and utilized by various worqer agents.

Components of Local Memory

1. reqap.d/ (Operational Memory / Feedback Loop)

• Purpose: Stores “reQap” files (e.g., cyqleXXX_reqap.md), which capture the output and assessment of an agent’s execution within a given cycle. This acts as a short-term, dynamic memory of recent actions and their outcomes.
• Mechanism:
  • After an agent runs, its output and a summary are saved into a reQap file.
  • At “cheqpoints” (qrane.py::handle_cheqpoint), this reQap file is reviewed. In autonomous mode, it’s automatically promoted. In user-gated mode, the user can inspect and even modify it.
  • The content of the reQap is then used to generate the tasq.md for the next cycle, effectively feeding the results of one step into the directive of the subsequent step.
• How it works as memory: It provides a continuous, updated record of task progression and agent performance, allowing the system to learn from its immediate past.

2. tasq.d/ (Directive Memory / Current Task)

• Purpose: Stores tasq files (e.g., cyqleXXX_tasq.md), which contain the current directive or task for the agents in a given cycle.
• Mechanism:
  • A tasq.md for cyqle N+1 is typically generated by promoting the reQap from cyqle N.
  • It includes critical instructions and often the assessment of the previous cycle.
• How it works as memory: It defines the immediate objective and provides historical context of the task at hand.

3. qontext.d/ (Semantic Memory / Context Cache)

• Purpose: Stores structured YAML summaries and symbol information for each processed file (e.g., file.py.q.yaml). This directory holds the “semantic context” of the codebase.
• Mechanism:
  • The worqer/qontextor.py agent is responsible for creating and updating these files. It operates in two modes based on the provider setting in config.yaml:
    • Local Mode (provider: local): Performs a deterministic, high-speed analysis of Python files using a sophisticated stack of local tools, including AST, Jedi for type inference, and PyCG for call graph analysis. This is the default and recommended mode. It can be further configured for 'fast' or 'complex' analysis (the latter including deep semantic embeddings).
    • AI Mode (provider: [ai_provider]): Uses an LLM (via worqer/lib_ai.py) to analyze qompressed code, documentation, and configuration files. It extracts key information like function/class names, signatures, purposes, and dependencies.
  • qontextor runs an initial scan and also update scans based on changes detected (e.g., from construqtor’s output).
• How it works as memory: This is a form of long-term, high-level memory about the codebase’s functionality and structure, accessible to any agent that needs to understand specific components semantically.

4. bloq.d/ (Structural Memory / Skeleton Cache)

• Purpose: Stores “qompressed” (skeletonized) code, representing the architectural structure of the codebase without implementation details.
• Mechanism:
  • The worqer/qompressor.py agent creates these files by processing the raw code from qodeyard/.
  • It uses AST parsing for Python and regex for other languages to achieve efficient body stripping.
• How it works as memory: This acts as a highly token-efficient structural memory of the codebase. It allows agents or LLMs to understand the architecture, available APIs, and code organization without being overwhelmed by full code content.

5. exeq.d/ (Execution Summaries)

• Purpose: Stores summaries of agent executions within each cycle (e.g., cyqleXXX_summary.md).
• Mechanism:
  • Individual agents might write their own summaries, and qrane.py manages the paths.
• How it works as memory: Provides a historical log of executed steps and their high-level outcomes, useful for review and debugging.

6. QONQ_PREVIOUS_LOG Environment Variable (Ephemeral Memory)

• Purpose: Temporarily holds the path to the qonsole.log of the immediately preceding agent.
• Mechanism:
  • Set by qrane.py before calling run_agent.
  • Used by worqer/lib_ai.py in _build_prompt to inject the previous agent’s output as “FALLBACK CONTEXT” into the current LLM prompt.
• How it works as memory: This provides a very short-term, direct channel of communication between sequentially executed agents, ensuring continuity of thought within a pipeline segment.

Local Memory Management Philosophy

The QonQrete memory system is designed around several principles:

• Tiered Approach: Different levels of detail and longevity (ephemeral, short-term, long-term).
• Token Efficiency: qompressor and qontextor work in tandem to reduce the raw code’s token cost, making more information accessible to LLMs within their context window.
• Structured Knowledge: qontext.d provides machine-readable (YAML) summaries, enabling agents to query and utilize semantic information programmatically.
• Feedback Integration: reqap.d and tasq.d create a strong feedback loop, allowing the system to iterate and refine its actions based on past results.
• Modularity: Memory components are distinct but interconnected, allowing specialized agents to manage specific aspects of knowledge.