Dominik Kundel

This commit **integrates a new image generation skill** directly into the system's built-in capabilities, making it a core component of the `codex-rs/skills` module. It introduces a comprehensive **image generation feature**, including a Python script (`image_gen.py`) that provides a fallback command-line interface for generating and editing images using GPT Image models. The work involves adding extensive **documentation**, configuration (`openai.yaml`), and sample prompts, significantly expanding the platform's **multimodal functionality** for users.

This commit introduces **two significant new capabilities** to the agent framework: a `shell` tool for executing arbitrary shell commands and an `apply_patch` tool for applying V4A diffs to file content. The **`agents-core`** package is extensively updated to define these tools, manage their execution, handle state serialization, and includes a new `applyDiff` utility. Support for these tools is integrated into **`agents-openai`** for model interaction and response processing, while **`agents-extensions`** is updated to handle them within its AI SDK. Comprehensive **documentation** has been added and updated across multiple languages to reflect these new tools and the concept of execution plans, greatly expanding the agents' ability to interact with the system and modify code.

This commit **introduces significant new capabilities** to the agent system by adding the **`ShellTool` and `ApplyPatchTool`**. The **`ShellTool` empowers agents to execute arbitrary shell commands**, while the **`ApplyPatchTool` enables programmatic application of diffs** to files, enhancing code modification workflows. This **feature development** involved extensive changes across the `src/agents` module, including new data structures, a dedicated `apply_diff` utility, and core execution logic within `src/agents/_run_impl.py` to process these new tool calls. These additions substantially **expand the agents' ability to interact with the operating system and perform complex code changes**, making them more versatile for advanced tasks.

feat(realtime-twilio): add MCP demo tools (#427)

This commit introduces a **feature enhancement** and **bug fix** to significantly improve **`gpt-oss` model compatibility** within the `codex-rs/core` library. It addresses a critical issue where `gpt-oss` models would lose context regarding tool call reasoning across subsequent chat completions by implementing new logic in the **`chat_completions` module** to automatically attach this reasoning to assistant messages and tool calls. This change, affecting functions like `stream_chat_completions` and `process_chat_sse`, ensures models maintain necessary conversational state for reliable tool usage. The update also includes minor **refactoring** and comprehensive **new tests** to validate the correct handling of reasoning in both payload generation and streaming scenarios.

This commit **fixes** an **incorrect JSON payload structure** within the `curl` examples provided in the **real-time voice agents quickstart guide** documentation. Specifically, it corrects the example for creating a real-time voice agent session, addressing issue #406. This **documentation fix** applies to both the English (`quickstart.mdx`) and **Japanese** (`ja/quickstart.mdx`) versions of the guide. The change ensures that users can accurately follow the provided instructions, significantly improving the **usability and correctness** of the quickstart documentation for developers integrating real-time voice agents.

This commit introduces **General Availability (GA) support for Realtime**, significantly enhancing the `agents-realtime` package. It enables **Multi-Cloud Platform (MCP) tool management**, allowing for background tool results and new approval workflows, alongside the addition of **image input** capabilities. Extensive **refactoring** was performed across core Realtime components, including session configuration, event handling, and transport layers, to align with the new Realtime API endpoints and model names. This **feature** release also updates the `TwilioRealtimeTransport` and relevant **documentation**, providing developers with advanced features and a more robust Realtime experience.

This commit introduces a **new feature** to the **`agents-core` package** by exposing a `history` getter on the `RunState` class. This new getter provides a public API to retrieve the complete sequence of inputs and generated items from an agent's execution, significantly enhancing observability and debuggability. **New tests** have been added to `runState.test.ts` to ensure the `history` getter functions correctly and preserves data across serialization. This allows developers to easily inspect and understand the full lifecycle of an agent's run.

This commit introduces a **new capability** to process and extract **reasoning content** from chat completions, specifically for **third-party AI providers** integrated through the OpenAI agent. It extends the core protocol by adding a `ReasoningText` type and `rawContent` field to `ReasoningItem` in `packages/agents-core/src/types/protocol.ts`. The `packages/agents-openai` module is updated to convert, extract, and stream this reasoning content, modifying `itemsToMessages`, `getResponse`, and `convertChatCompletionsStreamToResponses` to correctly handle these new data types. This enhancement allows the system to capture and utilize the internal thought processes or explanations provided by AI models, improving the depth of AI interactions.

This commit performs a **maintenance fix** by correcting the version bump type for the `@openai/agents-realtime` package. It updates the `.changeset/realtime-url-configurable.md` file to change the intended version increment from `minor` to `patch`. This **chore** ensures that the upcoming release of the **`@openai/agents-realtime`** module will correctly reflect a **patch-level change**, aligning with semantic versioning principles and preventing an accidental minor version bump.

This commit introduces a **new capability** to the **`@openai/agents-realtime`** package, enabling users to **configure the WebSocket URL** for realtime connections. The `OpenAIRealtimeWebSocketOptions` now accepts a `url` parameter, which is then utilized by the `OpenAIRealtimeWebSocket` constructor and `connect` method to establish connections to a custom endpoint. Furthermore, the `RealtimeSession` component has been updated to correctly forward this `url` option to its underlying transport layer. This enhancement provides greater flexibility for deploying and connecting to alternative WebSocket services for **realtime communication**.

This commit **fixes a minor grammatical error** within the project's documentation by **adding a missing comma**. Specifically, it targets the **main index page** located at `docs/src/content/docs/index.mdx`. This **documentation fix** improves the overall **readability and grammar** of the introductory content, ensuring a more polished and professional presentation for users consulting the project's documentation.

This commit introduces a **critical bug fix** to prevent **accidental cancellation errors** in the **realtime WebSocket transport** across the `agents-extensions` and `agents-realtime` packages. The `_interrupt` method signature has been updated in `TwilioRealtimeTransport.ts` and `openaiRealtimeWebsocket.ts` to explicitly manage ongoing response cancellations via a new `cancelOngoingResponse` parameter. This ensures that real-time agent interactions are not prematurely terminated, significantly improving the **stability and reliability** of the system. Additionally, related refactoring in `openaiRealtimeWebsocket.ts` corrects audio length calculation and `openaiRealtimeBase.ts` enhances session configuration capture to support this more robust interruption handling.

This commit introduces a **bug fix** to the **Twilio real-time transport** within the `agents-extensions` package. It addresses an issue where input and output audio formats were not consistently preserved during session configuration updates or handoffs. A new method, `updateSessionConfig`, has been added to `TwilioRealtimeTransport.ts` to explicitly ensure that audio formats are maintained, preventing potential audio quality degradation or compatibility problems. This change improves the robustness and reliability of audio handling for Twilio integrations, with a new test case verifying the correct behavior of `updateSessionConfig`.

This commit **enhances the tracing documentation** by adding a new section detailing **external trace providers**. It updates both the English (`docs/src/content/docs/guides/tracing.mdx`) and **Japanese (`docs/src/content/docs/ja/guides/tracing.mdx`) tracing guides**, providing users with a comprehensive list and links to relevant external tracing processors. This **documentation improvement** helps users discover and integrate various tracing solutions, making the system's observability features more accessible and understandable. The Japanese version also includes refinements to existing text for improved clarity.

This commit **fixes a bug** in the **AI SDK extension** where `providerMetadata` was being unnecessarily wrapped during message conversion and model response processing. It ensures that `providerMetadata` is now passed directly within functions like `itemsToLanguageV1Messages` and `getResponse` in `packages/agents-extensions/src/aiSdk.ts`, preventing data integrity issues. This **improves the reliability** of data exchange with AI providers and is accompanied by **documentation updates** in `docs/src/content/docs/extensions/ai-sdk.mdx` to guide users on correct `providerMetadata` usage. New test cases have been added to verify this corrected behavior, ensuring proper integration for users leveraging AI SDK capabilities.

The commit **introduces debug logging** to the **AI SDK integration** within the `agents-extensions` package. This **new feature** specifically enhances the `AiSdkModel` by logging detailed requests and responses for its `getResponse` and `getStreamedResponse` methods. This improves **observability** and aids in **diagnosing issues** by providing clearer insights into the interactions with AI models.

This commit introduces **critical bug fixes** and **improvements** to the **Human-In-The-Loop** functionality within the `packages/agents-core` module. It **resolves an infinite loop** that could occur during interruption handling and **prevents double counting of tool calls** when processing approvals, ensuring more reliable agent execution. A new `getInterruptions` method is also added to `RunState` to facilitate better management and retrieval of active interruptions, enhancing the overall robustness of the system.

This commit delivers a **bug fix** for the **`agents-extensions`** package, resolving an issue where `NaN` values could incorrectly appear in token usage calculations. It modifies the `AiSdkModel` within `packages/agents-extensions/src/aiSdk.ts` to gracefully handle non-numeric `promptTokens` and `completionTokens`, defaulting them to `0` for both direct `getResponse` and streamed `getStreamedResponse` AI responses. This ensures accurate and reliable reporting of token consumption, preventing potential downstream issues in analytics or billing systems. New test cases have been added to thoroughly validate this corrected behavior.

This commit **enhances event reporting** by adding `toolCall` details to the `agent_tool_start` and `agent_tool_end` events. It introduces a new `details` property containing `toolCall` information within the `AgentHookEvents`, `RunHookEvents`, and `RealtimeSessionEventTypes` across the **`agents-core` and `agents-realtime` packages**. This **new capability** provides richer context about the specific tool being executed, improving observability and debugging for agent operations. The changes affect how tool execution events are emitted and consumed, ensuring that downstream systems can access comprehensive tool call data. Test expectations were also updated to verify the inclusion of these new details.