Sonali Deshpande

This commit **temporarily disables** the `'e2e - odsp-client with ODSP service'` stage within the `test-service-clients.yml` pipeline. This **maintenance** change acts as a workaround to prevent ongoing CI failures and alerts, as these `odsp-client` e2e tests have been consistently failing since December due to new MFA requirements. The stage will remain skipped until the associated user accounts are updated, impacting the **CI/CD pipeline** by pausing validation for the `odsp-client` against the ODSP service.

This commit introduces a new `cleared` event in the **`SharedDirectory` DDS** within the **`@fluidframework/map` package**, providing a `path` parameter to specify which subdirectory was cleared. This **enhances the API** by resolving a limitation where event listeners lacked context on the specific cleared path. As part of this **API evolution**, the existing `clear` event in `ISharedDirectoryEvents` is now **deprecated**, with its removal planned for future major versions. This change provides more granular event information and prepares for a **breaking change** by guiding consumers to adopt the new event.

This commit introduces a **bug fix** to the **DDS Map** (`packages/dds/map`) by ensuring that `valueChanged` events are consistently emitted for each key deleted when the `clear()` method is called locally. Previously, this event emission only occurred for remote operations, leading to inconsistent behavior between local and remote state changes. The `mapKernel.ts` file was updated to implement this change, modifying the `clear` function to emit these events whether the map is attached or unattached. Associated test files (`mapOracle.ts`, `map.rollback.spec.ts`) were adjusted to reflect the new eventing, improving the predictability and consistency of eventing for consumers of the DDS Map.

This commit delivers a **bug fix** for the **`directory` DDS** within `packages/dds/map`, specifically addressing an issue where events were incorrectly emitted for disposed or unreachable directories. It introduces an `isNotDisposedAndReachable` check, which is integrated into the event emission logic for remote operations such as `processRemoteSet` and `processRemoteDelete`. This prevents the **`directory` DDS** from notifying consumers about directories that are optimistically deleted or have pending ancestor deletions, ensuring that only valid and active directory states trigger events. The change significantly improves the consistency and reliability of the **`directory` DDS**'s eventing mechanism.

This commit introduces a **bug fix** to the **DDS Map**'s eventing mechanism, specifically addressing an issue where a remote `clear` operation, when combined with pending local `set` operations, would emit an unreliable `clear` event while keys still persisted. The `clear` method in `packages/dds/map/src/mapKernel.ts` is modified to now **emit individual `valueChanged` events** for each key that is effectively deleted during such a remote clear. This change significantly improves the **predictability and accuracy** of map state updates, allowing consumers to reliably track changes via `valueChanged` events without needing to re-read the entire map after a `clear`. Associated tests in `packages/dds/map/src/test/mapOracle.ts` and `packages/dds/map/src/test/mocha/map.spec.ts` were updated to validate this new event emission behavior and simplify oracle logic.

This commit introduces a **new `SharedDirectoryOracle`** within **`packages/dds/map`** to significantly enhance the **testing and validation** of `ISharedDirectory` event correctness and API contracts. This **new capability** maintains two distinct internal models, tracking directory state via `valueChanged` and `containedValueChanged` events, ensuring consistent reporting across different event streams. The oracle validates event properties like `previousValue`, accounting for optimistic operations and post-clear scenarios. Integrated into existing fuzz tests, this **testing infrastructure** improves the **robustness and reliability** of the `SharedDirectory` implementation by catching inconsistencies.

This commit introduces a **new testing capability** for `ISequenceIntervalCollection` by adding an `IntervalCollectionOracle`. This oracle listens to all interval-related events, maintains an independent state snapshot, and validates that events are emitted correctly according to their **API contracts**, significantly enhancing the **robustness of interval collection testing**. Concurrently, it includes a **bug fix** in `packages/dds/sequence/src/intervalCollection.ts` to ensure the `changed` event correctly passes an empty object for `propertyDeltas` instead of `undefined`, resolving an **API contract violation**. The new oracle is integrated into the **fuzz testing infrastructure** to improve consistency checks and ensure reliable behavior of the `SharedString`'s interval management.

This commit delivers a **bug fix** for the **`intervalCollection`** DDS, addressing two issues discovered during oracle testing. It corrects the event emission logic to ensure `deleteInterval` events are properly emitted when intervals slide off during rebasing, preventing inconsistencies. Furthermore, it prevents events from being emitted with `undefined` intervals for already deleted intervals, upholding the API contract for `interval` never being `undefined`. These changes enhance the reliability and correctness of the `intervalCollection`'s eventing system, particularly in collaborative scenarios involving remote operations and rebasing.

This commit introduces a **new `clearInternal` event** within the **`SharedDirectory`** DDS, specifically in the **`@fluidframework/map`** package, to address a critical limitation in event tracking. Previously, the standard `clear` event did not provide the absolute path of the cleared subdirectory, forcing external systems like test oracles to clear all known directories. This **new capability** emits the absolute path when a directory is cleared, enabling more precise state management and **fixing** an ambiguity in event handling. The `clearInternal` event is marked as internal and defined in the `ISharedDirectoryEventsInternal` interface, significantly improving the granularity of directory state tracking.

This commit introduces a **new capability** by adding a **`SharedMatrixOracle`** to the **`matrix`** DDS, specifically for **event testing** and **consistency validation** during fuzz tests. The oracle tracks `conflicts` events within a `SharedMatrix`, intelligently adjusting its state to account for row and column insertions, deletions, and shifts. It captures the conflict winner, conflicting value, and actual cell value, enabling robust validation of the matrix's state and conflict resolution. This enhancement significantly improves the **testing infrastructure** for `SharedMatrix`, ensuring greater reliability and correctness under complex, concurrent operations.

This commit implements a **temporary workaround** by **conditionally skipping several end-to-end presence tests** within the `packages/service-clients/end-to-end-tests/azure-client/src/test/multiprocess/presenceTest.spec.ts` file. This **maintenance** action addresses **intermittent failures** observed in the **Azure Client**'s e2e pipelines, preventing them from blocking CI. The tests are specifically skipped when running against Azure, acknowledging an open repair item to fix the underlying issues. This ensures pipeline stability while the root cause of the test failures is investigated and resolved.

This commit introduces the **`SharedMapOracle` test helper** to significantly enhance the **consistency validation for the `SharedMap` DDS**. The `SharedMapOracle` maintains an independent, synchronized view of a `SharedMap`'s state by listening to `valueChanged` and `clear` events, rebuilding its internal oracle on clear to prevent mismatches. This **new testing capability** is integrated into the **`SharedMap` fuzz testing framework** via `fuzzUtils.ts` and `map.fuzz.spec.ts`, allowing for robust state comparison via its `validate()` method. By providing a definitive "ground truth" for `SharedMap` state, this work **improves the reliability and thoroughness of `SharedMap` tests**, particularly for complex event-driven scenarios.

This commit introduces a **new testing capability** for the **`SharedString` DDS** by adding a `SharedStringOracle`. This oracle maintains a lightweight local copy of the `SharedString`'s state, listening to incoming deltas and applying equivalent changes to its internal model using `splice` for **consistency validation** during **fuzz testing**. It integrates into the **DDS fuzz testing framework** via `ddsFuzzHarness.ts` to register and dispose oracles, ensuring robust verification of `SharedString` behavior without affecting its operational state. This enhancement significantly improves the reliability of `SharedString` by providing a dedicated mechanism to detect state discrepancies.

This commit delivers a crucial **bug fix** for the **restart-from-pending operation**, enhancing the reliability of client state management. It addresses an issue where clients could remain in staging mode during a restart by ensuring the `sourceClient` **exits staging mode** if active, preventing incorrect state retrieval within the **container runtime**. Furthermore, the **pending state manager**'s `replayPendingMessages` logic is corrected to only clear `savedOps` when not committing staged batches, preventing erroneous data loss. These changes significantly improve the robustness of client restarts and resolve several previously failing stress test seeds.

This commit introduces a **new capability** to the **local server stress tests** by adding a `clientRestartFromPendingProbability` option to the client join configuration. This allows testers to define the probability of a client restarting from a pending state, thereby enhancing the realism and configurability of stress test scenarios. The change primarily affects the `localServerStressHarness.ts` file, where the new option is integrated into the test harness configuration and utilized by the `mixinRestartClientFromPendingState` logic. This **feature addition** improves the testing infrastructure by enabling more comprehensive evaluation of client resilience and state management during various restart conditions.

This commit introduces a **new capability** to the **local server stress model** within the `local-server-stress-tests` package. It adds support for a `restartClientFromPendingState` operation, enabling the simulation of clients restarting from their pending local state with a configurable probability. This **feature enhancement** modifies the operation generator to occasionally produce this new operation and updates the reducer logic to correctly dispose of the old client and load a new client from the pending state. This significantly improves the realism and coverage of **stress tests**, helping to identify issues related to client state recovery and resilience.

This commit **introduces a rollback mechanism** into the **DDS fuzz testing harness**, enabling more robust and complex state-reversion scenarios for distributed data structures. A new `rollbackProbability` option is added to the fuzz suite configuration via `mixinRollback` in `ddsFuzzHarness.ts`, which is initially set to `0` across various **DDS fuzz test suites** (e.g., `array`, `sharedString`, `shared-tree`) to temporarily disable rollback and stabilize existing tests. This **feature addition** also includes an **enhancement to `MockContainerRuntime`** in `test-runtime-utils` with a new `runWithManualFlush` method, providing finer control over flush behavior during testing. The overall **maintenance and feature work** lays the foundation for advanced fuzz testing while ensuring current test stability, with plans to re-enable full rollback testing as issues are addressed.

This commit **fixes a critical security vulnerability** in the **Historian service's blob cache** by implementing proper **tenant isolation**. Previously, the cache was shared across tenants, allowing unauthorized access to blob contents if a SHA1 was known. The change modifies the `getBlob` method in `restGitService.ts` to accept and incorporate `tenantId` into cache keys, ensuring that blob data is strictly scoped per tenant. Calls to `getBlob` from functions like `create` in `blobs.ts`, `createBlob`, and `getTree` are updated to pass the `tenantId`, thereby preventing cross-tenant data leakage and enhancing the overall security of the **Historian** component.

This commit introduces a **maintenance change** to the **Riddler service** within the `historian-base` package, specifically addressing unit consistency for token lifetimes. It **converts the token lifetime** obtained from `validateTokenClaimsExpiration` from milliseconds to seconds within the `createToken` method in `server/historian/packages/historian-base/src/services/riddlerService.ts`. This ensures that the token expiration values are consistently represented in seconds, preventing potential misinterpretations or incorrect time calculations for downstream components relying on these values. The change improves the **robustness and predictability** of token generation and validation logic.

This commit implements a **bug fix** to ensure that `tenantId` and `documentId` are consistently captured and logged, even when server requests result in timeouts or 499 errors. It modifies the **logging middleware** in `server/routerlicious/packages/services-utils/src/morganLoggerMiddleware.ts` to retrieve these identifiers from `res.locals`. To facilitate this, the `app.ts` files within **server components** like `gitrest-base`, `historian-base`, and `routerlicious-base` (specifically alfred, nexus, and riddler) are updated to store `tenantId` and `documentId` in `res.locals`. This change significantly enhances the **observability** and diagnostic capabilities for troubleshooting specific tenant and document-related server failures.