Tony Murphy

This commit **refactors** the **Container Runtime** by modifying the `loadRuntime2` function to return an object `{ runtime: ContainerRuntime }` instead of directly returning the runtime, a **preparatory change** for future extensions such as staging mode controls. Concurrently, it introduces a **new capability** with `loadContainerRuntimeAlpha`, which provides a specialized `ContainerRuntimeBaseAlpha` for upcoming **staging mode APIs**. The existing `loadRuntime` function is adapted to seamlessly integrate with the new `loadRuntime2` signature, preserving its public contract. This work enables advanced **staging mode functionality** within the runtime and includes necessary **test updates** to reflect the API changes.

This commit performs a **refactoring and maintenance** task by **removing the `DisableFlushBeforeProcess` feature flag** from the **Fluid Framework `ContainerRuntime`**. Having been validated in production, this flag is now deprecated, leading to the **hardcoding of the default flush-before-process behavior** and the removal of the `skipSafetyFlushDuringProcessStack` member in `containerRuntime.ts`. Additionally, it **eliminates the `flushPartialBatches` configuration option** from `IOutboxConfig` and simplifies the `maybeFlushPartialBatch()` method in the **`Outbox`** component, streamlining the operation processing and batch flushing logic. This change simplifies the codebase, removes conditional logic, and ensures a consistent, validated operation flow within the runtime.

This commit introduces a **bug fix** to the **`@fluidframework/task-manager`** package, specifically within the `TaskManagerClass`. It ensures that the `abandoned` flag is correctly set to `true` when a task's subscription ends, such as during `unsubscribe` or `checkIfRolledBack` operations. This crucial update improves the reliability of **task lifecycle management** by accurately reflecting the abandonment status of tasks. By preventing tasks from being incorrectly considered active, this change enhances the overall correctness of the task system.

This commit **updates the default setting** for `readonlyInStagingMode` to `false` within the **`@fluidframework/datastore` runtime**, specifically impacting **staging mode**. This **configuration change** allows data stores to be **writable by default** when operating in staging mode, significantly aiding the prototyping and development experience. As a **feature enhancement**, it simplifies initial setup for developers, though it represents a breaking change given staging mode's alpha status. Additionally, the commit includes **documentation updates** in `IFluidDataStorePolicies` within `@fluidframework/runtime-definitions` to clearly reflect this new default behavior.

This commit introduces a **new capability** by exposing the `activeLocalOperationActivity` property on the **`IFluidDataStoreRuntime` interface** and implementing its getter in the **`FluidDataStoreRuntime` class**. This property allows consumers to detect when the **Fluid DataStore Runtime** is performing internal local operations, such as applying stashed operations or rolling back uncommitted changes. The implementation ensures this state is accurately reflected during these activities and reset afterward, providing crucial insight into the runtime's internal processing. This **API enhancement** improves observability for downstream consumers, enabling them to differentiate between user-initiated and runtime-initiated state modifications. API reports for the **`@fluidframework/datastore-definitions`** and **`@fluidframework/datastore`** packages have been updated to reflect this change.

This commit introduces a **new capability** by adding the `PendingLocalStateStore` utility class to the **`container-loader`** package, designed to efficiently manage and deduplicate pending local states for attached containers, thereby optimizing memory usage. It **refactors** the **`local-server-stress-tests`** harness to integrate this store, simplifying client restart and cloning logic by consolidating state management into the add/remove client flow. This change provides a more robust and realistic simulation of client lifecycle events in stress tests, while also improving the overall efficiency of handling container states.

This commit **refactors** and **evolves the Staging Mode API** by unifying its exposed interfaces under `@legacy @alpha`. The deprecated `IFluidDataStoreRuntimeExperimental` interface is replaced with `IFluidDataStoreRuntimeAlpha` in **`@fluidframework/datastore-definitions`**, making `inStagingMode` and `isDirty` properties non-optional. This change impacts **`@fluidframework/container-runtime`**, **`@fluidframework/datastore`**, and **`@fluidframework/runtime-utils`** by updating all internal usages to the new alpha interface and introducing the `asLegacyAlpha` helper for type conversions. The work ensures a consistent and more stable (though still experimental) interface for staging mode, with updated API extraction and reporting to reflect these changes.

This commit introduces a significant **refactoring** to the **container loader's snapshot blob management** within `packages/loader/container-loader`, primarily focusing on **memory optimization** and consistency. It updates the `ContainerStorageAdapter` and `ISerializedStateManagerDocumentStorageService` to exclusively use `ArrayBuffer` for blob contents, removing legacy string encoding and introducing a new `cacheSnapshotBlobs` method. The `combineSnapshotTreeAndSnapshotBlobs` utility is also refactored to directly handle `ArrayBuffer`s, eliminating redundant string-to-buffer conversions. This work streamlines snapshot handling, improves type safety, and ensures buffers are reused across the system, leading to **reduced memory consumption** and a more robust blob management architecture.

This commit **enables `getPendingLocalState` by default** within the **`container-loader`** package, fundamentally **optimizing snapshot management** and **improving memory efficiency**. It refactors the in-memory representation of snapshots from `SnapshotWithBlobs` to the more efficient `ISnapshot` format, which aligns with native driver output and stores blobs as `ArrayBuffers`, thereby eliminating redundant data storage and reducing memory overhead for large documents. This **performance enhancement** also defers blob retrieval and `SnapshotWithBlobs` conversion until `getPendingLocalState` is explicitly called, ensuring a faster load path and maintaining compatibility with older drivers. Consequently, various tests across the codebase are updated to remove the now-obsolete explicit configuration for offline load.

This commit introduces a significant **refactoring** to the **`SerializedStateManager`** and **`Container`** classes, primarily by moving the `pendingLocalState` from being an internal member variable to an explicit parameter passed to the `fetchSnapshot` method. This change **improves memory management** by allowing `pendingLocalState` to be garbage collected after it's no longer needed, as it's converted to buffers. The **`container.ts`** and **`serializedStateManager.ts`** files are updated to reflect this new data flow, along with corresponding **test updates** in **`serializedStateManager.spec.ts`**. This refactoring clarifies the lifecycle of local state during container loading and snapshot operations, ensuring resources are released promptly.

This commit addresses a **timer leak bug** in the **container loader's** `SerializedStateManager` by implementing robust **lifecycle management**. The `SerializedStateManager` now implements `IDisposable`, ensuring its snapshot refresh timer is properly cleared upon disposal and preventing invalid operations on a disposed instance. The **Container** is updated to explicitly dispose of its `SerializedStateManager` during shutdown, which, combined with centralized offline snapshot refresh configuration, prevents **resource leaks** and enhances **runtime stability**, particularly when **offline snapshot refresh** is enabled.

This commit **refactors** the **container attachment and snapshot handling** within the `packages/loader/container-loader` package to **standardize the in-memory snapshot format**. It consistently uses the `ISnapshot` type, replacing older custom snapshot structures and removing the `offlineLoadEnabled` flag to simplify the attachment workflow. This **refactoring** improves **type safety**, reduces code complexity, and enhances the maintainability of snapshot retrieval and processing utilities. The `SerializedStateManager` and various snapshot conversion utilities are updated to align with this new, unified approach, ensuring a more consistent and robust system.

This commit **refactors** the `stringToBuffer` utility functions within the **`client-utils` package** to explicitly return an `ArrayBuffer` type. This change **narrows the return type** from `ArrayBufferLike` to `ArrayBuffer` in both browser (`bufferBrowser.ts`) and Node.js (`bufferNode.ts`) environments. The primary purpose is to enhance **type compatibility and interoperability** with the `ISnapshot` interface, which mandates `ArrayBuffer` for its blob data. This **refactoring** ensures consistent data handling across the system and is not expected to impact existing consumers, as `ArrayBuffer` was already the actual type being returned.

This commit **refactors** the **configuration management** for **offline mode** and **duplicate batch detection** within the Fluid Framework, introducing more granular feature flags. It updates the **`Container`** and **`SerializedStateManager`** to use `Fluid.Container.enableOfflineFull` for comprehensive offline enablement, while **`ContainerRuntime`** now utilizes `Fluid.ContainerRuntime.enableOfflineBatchDetection` for dedicated control over duplicate batch detection. This **maintenance** effort improves **maintainability** and allows for more explicit and flexible **feature toggling** of offline capabilities. The changes are reflected across various test suites, ensuring consistency with the new configuration logic.

This commit **optimizes the container loading process** within the **`container-loader` package** by **refactoring** how snapshot blob contents are retrieved. It **defers blocking work**, specifically the retrieval of all snapshot blobs via `getBlobContentsFromTree`, to the background, integrating it with the existing snapshot refresh mechanism to prevent performance degradation during **offline container loads**. This **performance fix** also unifies container attribute retrieval, ensuring it's fetched only once per load. The changes significantly improve the **responsiveness and efficiency of container initialization**, particularly in scenarios involving offline state.

This commit **corrects the connection logic** within the **`container-loader`** package, specifically in `container.ts`, to ensure the **`Container`** connects as **'read' even when there is pending local state**, unless explicitly forced to 'write'. This **bug fix** addresses a previous incorrect assumption that `savedOps` implied local changes, which could lead to premature 'write' connections. The change prevents unintended write connections and is crucial for upcoming **offline and staging mode features**, allowing for scenarios where local changes might be discarded without forcing an immediate write connection. A related test in `localServerStress.spec.ts` is also adjusted.

This commit introduces a **new capability** for **`FrozenContainer`** instances within the **`container-loader`** module, enabling them to **read blobs from the network**. The `FrozenDocumentService` and `FrozenDocumentStorageService` classes are enhanced to accept an optional underlying document service, which facilitates `readBlob` operations by fetching blob data from the network. This **feature enhancement** ensures that `loadFrozenContainerFromPendingState` can successfully rehydrate containers even when local pending state does not fully capture all blobs, improving robustness for online rehydration scenarios. The change impacts the `createAndLoadContainerUtils.ts` and `frozenServices.ts` files, along with corresponding test updates.

This commit **modifies the default behavior of the StagingMode feature** within the `container-runtime` package, changing it from readonly to non-readonly. Specifically, it adjusts the `isReadOnly` condition in `packages/runtime/container-runtime/src/dataStoreContext.ts` to ensure that staging mode is non-readonly by default unless explicitly configured otherwise. This **feature enhancement** simplifies the setup and prototyping experience for the **alpha StagingMode feature**, making it easier for developers to explore its capabilities without extensive initial configuration. As this feature is not yet in production use, this change is expected to have no negative downstream impact.

This commit introduces a **new capability** to the **ContainerLoader** by enabling the loading of Fluid containers in a "frozen" (read-only, storage-only) mode, specifically from pending local state. It achieves this through the addition of new **`FrozenDocumentServiceFactory`** and **`FrozenDeltaStream`** classes, which replace the previous `NoDeltaStream` implementation, and exposes a new API `loadFrozenContainerFromPendingState` for this purpose. This **refactoring** in `connectionManager.ts` and the new utility in `createAndLoadContainerUtils.ts` facilitate this mode, ensuring that containers loaded this way only interact with storage and disallow delta stream operations. **Test updates** in `local-server-stress-tests` validate the consistency of DDSs after loading in frozen mode, enhancing the robustness of client restart scenarios, while a minor **bug fix** addresses the `attached` getter in `StressDataObject`.

This commit **refactors** the exposure of experimental APIs within the **`ContainerLoader`** package by introducing a new **`ContainerAlpha`** interface and an `asLegacyAlpha` helper. This change provides a safer and more forward-compatible mechanism for accessing experimental features, such as `getPendingLocalState`, replacing the deprecated `IContainerExperimental` approach. The **`Container`** class is updated to implement this new interface, and its adoption is reflected across numerous **test suites** in `dds/tree`, `local-server-stress-tests`, and `test-end-to-end-tests`. Additionally, **API extractor configurations** are updated to properly lint and report on the new alpha interface, establishing a clearer standard for experimental API management.