Subham Sinha

This commit **fixes a regression** in the **synchronous Google Cloud Spanner client** by **restoring implicit database dialect reloading**. Specifically, the `Database.database_dialect` property in `google/cloud/spanner_v1/database.py` will now correctly trigger a blocking `self.reload()` call when accessed and found to be `DATABASE_DIALECT_UNSPECIFIED`. This behavior, which was lost during the introduction of AsyncIO support via CrossSync, is re-enabled through a conditional guard in `_async/database.py` that ensures the synchronous client maintains its lazy-loading mechanism. The **bug fix** ensures **strict backwards compatibility** for existing synchronous libraries that rely on this implicit dialect retrieval.

This commit **updates documentation** across the **AsyncIO APIs** for the `google-cloud-spanner` library, specifically within the `_async` subpackage. It adds explicit experimental warnings to the docstrings of numerous core components, including `Client`, `Database`, `Instance`, `Session`, `Snapshot`, and various session pool classes like `AbstractSessionPool`. This **documentation update** serves as a **maintenance** task to inform users that these **asynchronous Spanner APIs** are currently in a **beta phase** and are **subject to change**, guiding their usage and managing expectations regarding API stability.

This commit delivers a **critical bug fix** addressing a **deadlock issue** in the **asynchronous multiplexed session manager** within the `google-cloud-spanner` client library. The problem arose from using a synchronous `threading.Lock` around `await` calls in `_async/database_sessions_manager.py`, which could block the entire `asyncio` event loop. The solution involves refactoring the `_async/database_sessions_manager.py` to correctly employ an **asynchronous lock** (`asyncio.Lock`) for `_get_multiplexed_session` and `_maintain_multiplexed_session`, alongside similar adjustments to the synchronous `database_sessions_manager.py`. This significantly enhances the **stability and reliability** of asynchronous Spanner clients by preventing application unresponsiveness during concurrent session acquisition and maintenance.

This commit **fixes a critical thread and memory leak** within the `google-cloud-spanner` client's **metrics initialization** when built-in metrics are enabled. It addresses the issue where new OpenTelemetry `MeterProvider` and `PeriodicExportingMetricReader` instances were unconditionally created, spawning unmanaged background threads that could lead to `RuntimeError` or OOM crashes. The **bug fix** implements a **singleton pattern for the `MeterProvider`** and refactors tracer management using `contextvars.ContextVar` to ensure **thread-safe, isolated metric tracing**. Furthermore, it **resolves a regression in `Batch.commit` idempotency** by correctly initializing retry counters, preventing spurious `AlreadyExists` errors. This significantly enhances the **stability and resource management** of the client library, particularly in high-concurrency environments.

This commit **fixes a critical error handling issue** within the **Google Cloud Spanner client library's snapshot streaming mechanism**, specifically in the `_restart_on_unavailable` function. Previously, if re-initialization of a stream iterator failed after a `ServiceUnavailable` or resumable `InternalServerError`, the exception would propagate unhandled, breaking the retry loop. The **bug fix** modifies the logic to reset the iterator to `None` and continue the loop, ensuring that subsequent re-initialization attempts are properly retried within the `try` block. This significantly improves the **robustness and reliability** of stream processing for Spanner users, with new unit tests added to validate the corrected behavior.

This commit **fixes critical issues** related to both **testing infrastructure** and **SQL parsing logic** within the `google-cloud-spanner` library. It resolves prerelease dependency failures by adding `cryptography` and `cffi` to `noxfile.py`'s `prerel_deps`, ensuring `google-auth` functions correctly during test runs. Concurrently, the **`spanner_dbapi`'s SQL parsing** is improved by modifying `parse_utils.py` to bypass `sqlparse` for `RUN PARTITION` commands, which prevents "Maximum grouping depth exceeded" errors with long partition IDs and **enhances system test stability**. Unit tests for `classify_statement` were also added to cover these `RUN PARTITION` scenarios.

This commit **enhances the Google Cloud Spanner client library** by **inverting the default behavior for built-in OpenTelemetry metrics**, making them **enabled by default** for improved out-of-the-box observability. This is a **breaking change** where metrics will now automatically collect and export unless explicitly disabled. Users must now set the new `SPANNER_DISABLE_BUILTIN_METRICS` environment variable to `true` to opt-out, deprecating the old `ENABLE_SPANNER_METRICS_ENV_VAR`. The change affects the **metrics configuration logic** within `client.py` and is validated by updated unit and system tests.

This commit introduces **new observability features** to the **Google Cloud Spanner client library** by enhancing **OpenTelemetry tracing**. It adds the `cloud.region` attribute to all spans for better geographical context, propagates `transaction_tag` from `Transaction` objects to `Commit` requests and corresponding spans for improved end-to-end traceability. Additionally, it supports `request_tag` on individual Spanner operations like `read`, `execute_sql`, and `execute_update`, adding it as a `spanner.request_tag` attribute to their respective spans. This **new capability** significantly improves the granularity and context available for performance analysis and debugging of Spanner operations.

This commit **enables OpenTelemetry metrics and tracing by default** across the `google-cloud-spanner` client library, significantly enhancing its **observability features**. This **new capability** removes previous conditional checks in `_opentelemetry_tracing.py` and `session.py`, ensuring that tracing is always active for relevant operations. Specifically, tracing is now applied to the `ping` method, and conditional span checks are removed from `exists` and `_do_transaction`. This change provides out-of-the-box telemetry, simplifying the monitoring and debugging experience for users of the Spanner client.

This commit **resolves a `TypeError`** that could occur during the **initialization of built-in metrics** within the **`google-cloud-spanner` client library**. It introduces **robust exception handling** in `client.py` and `spanner_metrics_tracer_factory.py` to gracefully manage failures during GCP resource location detection, preventing application crashes and logging warnings instead. This **bug fix** ensures the Spanner client remains stable even when metrics initialization encounters unexpected issues, improving the reliability of both the client and its metrics reporting. New unit tests were added to verify this improved error handling for the `Client` constructor and the `_get_location` method.

This commit introduces a **new feature** to the **Google Cloud Spanner client library**, enabling **lazy decoding** for **partitioned queries and reads**. It modifies the `google.cloud.spanner_v1.database` module to accept a `lazy_decode` parameter in methods like `run_partitioned_query` and `process_partitioned_read`, and implements the core lazy decoding logic within the `google.cloud.spanner_v1.merged_result_set.MergedResultSet` class. Furthermore, this change adds **context manager support for `BatchSnapshot`**, simplifying resource management. This enhancement allows users to defer data type conversion until rows are accessed, potentially improving performance and memory efficiency for large result sets.