ms-agent

Central LLMAgent class orchestrates execution loops across multiple LLM providers (OpenAI, Anthropic, local models via Ollama) through a unified interface. The framework abstracts provider-specific APIs into a common message-passing protocol, enabling agents to switch backends without code changes. Configuration-driven provider selection allows runtime binding of LLM endpoints.

Implements MCP-compliant tool registration and invocation through a schema-based function registry. Tools are defined with JSON schemas describing parameters, return types, and descriptions; the framework automatically marshals function calls from LLM outputs into executable tool invocations with type validation. Supports both built-in tools and external MCP servers.

Web UI layer built with Gradio provides interactive interface for agent execution, project management, and workflow visualization. Implements agent runner subprocess management for isolated execution, project discovery for loading agent configurations from filesystem or registry, and real-time execution monitoring with streaming output.

Specialized Singularity Cinema workflow generates short videos (~5 minutes) from text prompts through multi-step composition: script generation from prompt, scene planning with visual descriptions, and video synthesis using text-to-video models. Manages video artifacts and enables iterative refinement of generated videos.

Configuration system uses YAML files to define agents, tools, workflows, and LLM providers without code. Supports configuration inheritance, variable substitution, and environment-based overrides. AgentLoader factory class parses configurations and instantiates agents/workflows with dependency injection, enabling configuration-driven agent construction.

Message flow architecture implements callback hooks at key execution points (before/after LLM calls, tool execution, task completion) enabling custom event processing without modifying core agent logic. Callbacks receive message context and can modify behavior through return values. Supports both synchronous and asynchronous callbacks.

Specialized workflow (Agentic Insight v2) that decomposes research tasks into iterative exploration phases. The agent autonomously generates follow-up questions, adapts search breadth based on information density, and synthesizes findings into structured reports. Uses web search integration and document processing to gather and analyze information across multiple sources.

Specialized Code Genesis workflow decomposes code generation into three distinct phases: Design (architecture planning), Coding (implementation), and Refine (testing and optimization). Each phase uses targeted prompts and tool calls to produce artifacts (design docs, code files, test cases). The framework maintains artifact state across phases and enables iterative refinement based on execution feedback.

DagWorkflow engine executes tasks as directed acyclic graphs, enabling complex multi-step workflows with conditional branches, parallel execution, and data flow between tasks. Tasks are defined in YAML configuration with dependencies, conditions, and parameter mappings. The engine handles task scheduling, error propagation, and state management across the DAG.

Implements the Anthropic Agent Skills protocol for progressive loading of tool/skill definitions based on task context. Rather than loading all available skills upfront, the framework analyzes the task and loads only relevant skill schemas, reducing token overhead. Skills are organized hierarchically with metadata enabling semantic matching to task requirements.

Memory system maintains conversation history with configurable retention policies (sliding window, summarization, or full history). Supports multiple memory backends (in-memory, Redis, database) and implements automatic summarization of old messages to maintain context while reducing token usage. Memory is scoped per agent instance with optional shared memory for multi-agent coordination.

Framework implements automatic error detection and recovery through configurable retry policies, fallback LLM providers, and error-specific recovery handlers. When a task fails (tool execution error, LLM timeout, validation failure), the agent automatically attempts recovery using strategies like retrying with modified prompts, switching to fallback providers, or decomposing the task into simpler subtasks.

Specialized FinResearch workflow orchestrates multiple agents for financial market research: a quantitative analysis agent processes financial data and metrics, a sentiment analysis agent evaluates news and social signals, and a synthesis agent combines findings into investment recommendations. Uses DAG-based composition with data flow between agents.

Document processing pipeline ingests PDFs, web pages, and other documents, extracts text with OCR support, chunks content into semantic units, generates embeddings, and stores in vector database for retrieval-augmented generation (RAG). Supports both dense retrieval (semantic similarity) and sparse retrieval (keyword matching) with configurable ranking strategies.