# NOTES.md ## ✅ What Was AI-Generated and What Wasn't - **AI-Generated:** - Initial boilerplate for `index.ts` setup. - Basic structure for `agent.ts` route logic. - Draft implementations for `rag.ts`, `plugins.ts`, and `llm.ts` using Gemini. - Prompt engineering template for dynamic prompt construction. - README and deployment guidelines. - **Human-Written / Edited:** - Integration and debugging of Gemini API calls. - Environment configuration using `dotenv` for ES modules. - TypeScript types and refinements for session memory and plugins. - Adjustments for async loading of document embeddings. - Error handling and fallback logic for plugins. - Build and deployment scripts in `package.json`. --- ## ✅ Bugs Faced and How They Were Solved 1. **`__dirname` Not Defined in ES Modules** - Issue: Node.js ES modules do not support `__dirname` by default. - Solution: Used `fileURLToPath(import.meta.url)` and `path.dirname()` to create a custom `__dirname`. 2. **Dotenv Not Loading in ES Module Context** - Issue: `dotenv.config()` failed in ES module setup. - Solution: Replaced with `import 'dotenv/config';` for auto-loading. 3. **Cosine Similarity Function** - Issue: Manual implementation was error-prone and unnecessary. - Solution: Installed `compute-cosine-similarity` and replaced custom logic. 4. **Async Loading of Document Embeddings** - Issue: Using `Array.forEach` with async calls caused incomplete embeddings. - Solution: Replaced with `for...of` loop and `await` to ensure sequential loading. 5. **Weather Plugin API Errors (404)** - Issue: Incorrect city string and interpolation caused invalid requests. - Solution: Used Axios `params` object and `.trim()` on city names for safe encoding. --- ## ✅ How the Agent Routes Plugin Calls + Embeds Memory + Context ### 1. **Routing Plugin Calls** - Inside `/agent/message`: - Message is analyzed for keywords. - `detectAndExecutePlugins(message)`: - If `message` contains "weather in", call `weatherPlugin`. - If `message` has digits/operators, call `mathPlugin`. - Plugin results are returned as an array of strings. ### 2. **Memory Embedding** - Session memory stored in `sessions` Map: - Each `session_id` stores an array of `{ role, content }`. - Last 2 messages retrieved via `getLastMessages(session_id, 2)`. ### 3. **Context via RAG** - Query embedding generated with Gemini `embedding-001`. - Cosine similarity computed with all document chunks. - Top 3 chunks retrieved and included in the system prompt. ### 4. **Prompt Composition** - System prompt dynamically includes: - **Instructions** (agent behavior rules) - **Memory summary** (last 2 messages) - **Retrieved context** (from RAG) - **Plugin outputs** (if available) - Final prompt passed to Gemini LLM for response generation. --- ### ✅ End Result - User sends a message → Agent enriches it with memory, RAG, and plugins → Generates an accurate, context-aware response via Gemini.