Vision for Copilot Preview

Enables users to attach images directly to chat messages in VS Code's chat panel via clipboard paste, drag-and-drop, or workspace file selection. The extension processes the image data and passes it as multimodal context to the configured vision-capable LLM provider (OpenAI, Anthropic, Gemini, or Azure OpenAI), allowing the AI to analyze visual content and respond with insights, explanations, or code suggestions based on the image content.

Provides quick-fix code actions in markdown, HTML, JSX, and TSX files to automatically generate or refine alt text for images. When triggered, the extension sends the image file and surrounding document context to the configured vision LLM, which analyzes the image content and returns descriptive alt text that can be inserted directly into the code. This improves accessibility compliance without manual effort.

Provides a 'Copilot Vision: Troubleshoot' command that captures the current VS Code window state as a screenshot and automatically sends it to the chat panel with the configured vision LLM. Users can then ask the AI to diagnose issues, explain error messages, or suggest fixes based on what's visible in the editor. This enables rapid troubleshooting without manual screenshot tools or context-switching.

Allows users to configure and switch between multiple vision-capable LLM providers (OpenAI, Anthropic, Gemini, Azure OpenAI) and their respective models through VS Code settings and commands. The extension manages API keys per provider, validates configuration, and routes vision requests to the selected provider's API. Users can set different providers for different use cases or switch providers based on cost, latency, or model capabilities.

Provides commands to securely store, update, and remove API keys for each configured vision provider. The extension uses VS Code's secure credential storage mechanism (via the VS Code Secret Storage API) to manage API keys without exposing them in plain text in settings files. Users can set or update keys via the 'Copilot Vision: Set Current Model's API Key' command and remove them via 'Copilot Vision: Remove Current Model's API Key' command.

Registers the vision extension as a chat participant in VS Code's chat panel, allowing users to invoke vision capabilities through natural chat interactions. The extension hooks into the chat participant API to intercept messages, detect image attachments, and route them to the configured vision LLM provider. This enables a conversational interface where users can ask questions about images, request alt text generation, or seek troubleshooting help without leaving the chat UI.

Allows users to select and attach image files directly from their workspace to chat messages or vision commands. The extension provides a file picker UI that filters for image formats (JPEG, PNG, GIF, WebP) and enables users to browse the workspace directory structure to find and attach images without manual file path entry. Selected images are read from disk and passed to the vision LLM provider.

When generating alt text or analyzing images, the extension passes surrounding document context (code structure, file type, semantic meaning) to the vision LLM alongside the image data. This allows the AI to generate alt text that is semantically appropriate for the specific context (e.g., alt text for a diagram in technical documentation differs from alt text for a UI mockup in a design system). The extension extracts relevant code snippets and document metadata to enrich the vision request.

Abstracts the differences between multiple vision API providers (OpenAI, Anthropic, Gemini, Azure OpenAI) behind a unified interface. The extension handles provider-specific API request formatting, response parsing, and error handling, allowing users to switch providers without changing their workflow. This abstraction layer translates generic vision requests (e.g., 'analyze this image') into provider-specific API calls with appropriate parameters, model names, and authentication.

The extension is documented as being deprecated in favor of built-in image flow in GitHub Copilot Chat. This capability represents the extension's current status and the planned migration path for users. The extension continues to function but is positioned as a temporary solution until GitHub Copilot Chat's native vision features reach feature parity. Users are implicitly encouraged to migrate to GitHub Copilot Chat for long-term vision support.