agents-towards-production

Implements complex task routing and state management using LangGraph's StateGraph and MemorySaver primitives, enabling agents to maintain conversation context across multiple turns while supporting human intervention checkpoints. The system uses a directed acyclic graph (DAG) pattern where each node represents a discrete agent action or decision point, with edges defining conditional routing logic based on agent output and external signals. State is persisted between invocations, allowing agents to resume interrupted workflows and maintain audit trails for compliance.

Combines short-term working memory (Redis-backed state store) with long-term semantic memory (vector database with embeddings) to enable agents to recall relevant historical context without token bloat. Short-term memory stores recent conversation turns and task state as structured JSON, while long-term memory indexes past interactions as embeddings, allowing semantic similarity search to retrieve relevant prior conversations. The system uses a retrieval-augmented generation (RAG) pattern where the agent queries long-term memory based on current context, then synthesizes retrieved memories into the prompt.

Provides Infrastructure-as-Code (IaC) templates (Terraform, CloudFormation, or Pulumi) for deploying agents to cloud platforms (AWS, GCP, Azure) with all supporting infrastructure (databases, monitoring, networking). The system defines agent deployment as code, enabling version control, reproducible deployments, and easy scaling. Templates include best practices for security (IAM roles, secrets management), networking (VPCs, load balancers), and monitoring (CloudWatch, Datadog).

Provides utilities for fine-tuning LLMs on agent-specific tasks (instruction following, tool use, output formatting) using training data collected from agent interactions. The system includes data collection (logging agent interactions), data preparation (filtering, formatting), and fine-tuning orchestration (calling OpenAI, Anthropic, or local fine-tuning APIs). Fine-tuned models can be deployed as drop-in replacements for base models, improving accuracy and reducing costs.

Provides a structured tutorial system where each production capability is taught through hands-on, runnable Jupyter notebooks and Python scripts. Each tutorial follows a standardized pattern: conceptual explanation, code walkthrough, and a working example that developers can execute locally. Tutorials are organized by production layer (orchestration, memory, tools, security, deployment), enabling developers to learn incrementally from prototype to production.

Implements OAuth2-based permission scoping for agent tool invocations, ensuring agents can only call APIs on behalf of authenticated users with appropriate authorization. The system uses an ArcadeTool abstraction that wraps external APIs (Slack, GitHub, Google Workspace) with auth_callback hooks, intercepting tool calls to validate user credentials and enforce scope restrictions before execution. Each tool invocation is tagged with the calling user's identity and permission set, enabling fine-grained access control and audit logging.

Integrates real-time search capabilities (via Tavily Search API) as a callable tool within agent workflows, enabling agents to fetch current web information and incorporate it into reasoning. The system wraps search queries in a TavilySearchResults tool that returns ranked, deduplicated results with source attribution, which the agent can then synthesize into its response. Search results are cached briefly to avoid redundant queries within the same conversation turn, and the agent can iteratively refine searches based on initial results.

Implements multi-layer security guardrails using LlamaFirewall and QualifireGuard to detect and block prompt injection attacks and personally identifiable information (PII) leakage. The system operates at two checkpoints: (1) input validation filters user messages for injection patterns and PII before they reach the agent, and (2) output validation filters agent responses to prevent PII from being returned to users. Guardrails use pattern matching, regex, and LLM-based classification to identify threats, with configurable severity levels (block, redact, warn).

Provides deployment abstractions (BedrockAgentCoreApp, @app.entrypoint decorator) that enable agents to run on serverless platforms (AWS Bedrock, Lambda) without managing infrastructure. The system handles request routing, state persistence, and scaling automatically, allowing developers to define agents as simple Python functions decorated with @app.entrypoint. The runtime manages cold starts, timeout handling, and integration with cloud logging/monitoring services.

Implements a standardized JSON-RPC communication protocol (A2AProtocol) for agents to invoke each other, enabling complex multi-agent workflows where specialized agents collaborate on tasks. Each agent is registered as an AgentCard with metadata (name, capabilities, input/output schema), and agents can discover and invoke other agents through a central registry. Communication is asynchronous with request/response tracking, allowing agents to wait for results or handle timeouts gracefully.

Provides structured logging and monitoring infrastructure that captures agent execution traces (state transitions, tool calls, LLM invocations) in a queryable format. The system logs each step of agent execution with timestamps, input/output, latency, and error information, enabling developers to debug issues, analyze performance, and detect anomalies. Logs are exported to cloud monitoring services (CloudWatch, Datadog, New Relic) for centralized analysis and alerting.

Provides a testing framework for evaluating agent behavior against defined criteria (accuracy, latency, cost, safety). The system allows developers to define test cases with expected outputs, run agents against test suites, and measure performance metrics. Evaluation supports both deterministic assertions (output matches expected value) and probabilistic metrics (accuracy across multiple runs, cost per invocation). Results are aggregated and compared across agent versions to track improvements.

Provides Docker containerization templates and best practices for packaging agents with all dependencies, enabling reproducible deployment across environments (development, staging, production). The system includes Dockerfile templates optimized for agent workloads (minimal base images, multi-stage builds, layer caching), and docker-compose configurations for local development with supporting services (Redis, vector DB, monitoring). Containers can be deployed to Kubernetes, ECS, or other container orchestration platforms.