Sushan Bhattarai

This commit introduces a **new feature** to the **Bigtable client library's connection pooling** mechanism. It modifies the `checkIfDirectAccessCompatible` function in `bigtable/internal/transport/connpool.go` to specifically handle `PermissionDenied` errors encountered during the initial `pingandwarm` phase of a Direct Access connection. Instead of immediately failing, the system will now proceed to check for **ALTS (Application Layer Transport Security)** compatibility as a fallback. This enhancement improves the **resilience of Bigtable client connections** by providing an automatic alternative transport mechanism when Direct Access is denied, ensuring more robust connectivity.

This commit **enhances the Bigtable direct access mechanism** by introducing new diagnostic capabilities and integrating with Traffic Director. It adds **new functionality** to fetch xDS endpoints from **Traffic Director** via `FetchXdsEndpoints` and includes checks for local IPv4/IPv6 **UDP routes** using `CheckLocalIPv4Routes` and `CheckLocalIPv6Routes`. These new checks, along with ALTS probing, are integrated into the `investigateDirectAccessFailure` function within the **connection pool** (`connpool.go`) to provide more granular reasons for direct access failures. Debug logging is also improved in `gcp.go` and `metadata_server.go` for better environment and metadata server diagnostics. This work significantly improves the **reliability and troubleshootability** of direct access connections for Bigtable clients.

This commit introduces a new **t4t7 latency tracking mechanism** within the **Bigtable client library**, enabling more accurate measurement of direct access operations. It defines a `t4t7Tracker` and integrates it into the `gaxInvokeWithRecorder` path and the existing metrics system, where it serves as a fallback for server latency. This **feature** also effectively **fixes previously broken client blocking latency measurements** by providing this robust fallback. The changes significantly improve the **observability of Bigtable operations** by ensuring more reliable and comprehensive latency data.

This commit introduces a **new capability** allowing users to explicitly disable **Bigtable direct path** by setting the `CBT_ENABLE_DIRECTPATH` environment variable to `'false'`. It significantly **enhances the direct access compatibility checks** within the `bigtable/internal/transport` module by adding detailed logic to verify the operating environment, including GCP detection, loopback interface status, and metadata server reachability. The `bigtable/client` module's `NewClientWithConfig` now respects this user preference, and the connection pooling logic includes **improved failure investigation and reporting** for direct access attempts. This ensures greater control and transparency over the client's network configuration and direct access eligibility.

This commit introduces a **new load balancing penalty mechanism** within the **Bigtable client's internal transport layer**. It enhances the **connection pool** (`connpool.go`) by implementing an `applyErrorPenalty` method, which dynamically increases a connection's perceived load when RPCs encounter errors. This **feature** modifies the `calculateConnLoad` logic and updates error handling in `Invoke`, `SendMsg`, and `RecvMsg` to apply these penalties, ensuring that problematic connections are less likely to be chosen for subsequent RPCs. The change aims to improve the client's resilience and distribute RPC traffic more effectively by de-prioritizing unreliable connections.

This commit introduces a **new capability** for the **Bigtable client**, enabling it to **dynamically derive its connection pool size from a resolver** through `NewClientWithConfig`. This significantly improves resource management by allowing the client to adapt its connection pool based on resolved configurations, with a default fallback. The change also includes a **refactoring** to add debug logging for connection pool creation within `NewBigtableChannelPool` and a **test fix** in `TestIntegration_Aggregates` to correct error pattern assertions. Overall, this enhances the client's adaptability and the reliability of its integration tests.

This commit introduces a **new feature** to the **Bigtable client library's dynamic channel pool** by adding a `MaxScaleUpPercentage` configuration option. This new setting, defined in `bigtable/internal/option/option.go` with a default of 30%, allows users to cap the rate at which the connection pool can increase its connections. The capping logic is implemented and validated within `bigtable/internal/transport/dynamic_scale_monitor.go`, ensuring controlled resource allocation. This enhancement provides finer control over the **dynamic channel pool's scaling behavior**, preventing overly aggressive connection increases and improving system stability.

This commit introduces a **new feature** to the **Bigtable client library's dynamic channel pooling mechanism**, making **downscaling less aggressive**. It modifies the `DynamicScaleMonitor` to only trigger a downscale operation after a configurable number of consecutive runs indicate a need for reduction, preventing premature or erratic resource adjustments. A new configuration option, `ContinuousDownscaleRunsThreshold`, is added to `DynamicChannelPoolConfig` to control this behavior, ensuring **more stable and predictable resource management** for Bigtable connections. This enhancement aims to improve overall client performance by reducing unnecessary fluctuations in connection resources.

This commit introduces a **new feature** to the **Bigtable** client library, establishing `PowerOfTwoLeastInFlight` as the default load balancing policy. It modifies the internal options parsing, specifically within `bigtable/internal/option/option.go` and the `parseLoadBalancingStrategy` function, to apply this strategy when no explicit policy is configured or an unknown value is provided. This change ensures that Bigtable clients will automatically utilize a potentially more efficient load distribution mechanism by default, improving performance and resource utilization without requiring manual configuration.

This commit **refactors** the **Bigtable client library's direct access check** logic to explicitly force internal gRPC options for **DirectPath** and **ALTS**. This **fix** removes the previous dependency on the `CBT_ENABLE_DIRECTPATH` environment variable, which was used to enable direct access. The change, primarily within the `CheckDirectAccessSupported` function, streamlines the configuration and ensures more reliable enablement of direct access for Bigtable connections. This improves the robustness and clarity of how direct access is managed for the **Bigtable client**.

This commit introduces a **new feature** and a **refactoring** to the **Bigtable client library's internal transport layer**. It **enhances the direct access compatibility checker** by populating the IP protocol, which is then included in the `reportDirectAccessMetric` for more granular connection insights. Additionally, the method `checkIfDirectAccessCompatible` within the `connpool.go` module has been **renamed for conciseness**, improving code clarity. This work primarily affects the **connection pooling mechanism** and the **diagnostic capabilities** related to direct access for Bigtable connections.

This commit performs an **internal upgrade** within the **Bigtable client library's transport layer**, migrating its random number generation from the legacy `math/rand` package to the more modern `math/rand/v2`. Specifically, the **connection recycler** (`conn_recycler.go`) and the **connection pool's least loaded selection logic** (`connpool.go`) have been refactored to utilize the new random number generator. This **refactoring** enhances the internal mechanisms for connection management, ensuring more robust and potentially more performant random selections for tasks like connection recycling and load balancing within the client's internal operations. The change is internal and does not affect the public API of the Bigtable client.

This commit **introduces direct access capabilities** for the **Bigtable client**, allowing it to establish more direct connections when supported. It significantly modifies the **Bigtable connection pool** (`bigtable/internal/transport/connpool.go`) to incorporate a new **direct access dialer** and associated configuration options, including logic to check and report eligibility. This **new capability** aims to optimize connection paths by bypassing certain layers, with the `NewClientWithConfig` function in `bigtable/client.go` updated to support this method. The feature is currently **guarded by the `CBT_BIGTABLE_CONN_POOL` environment variable** and has initial limitations, such as lacking PeerInfo checks and retry logic for Prime.

This commit introduces **support for `GAUGE` metrics**, also known as longupdown metrics, within the **Bigtable metrics monitoring exporter**. This **new capability** allows the exporter to correctly handle and send a broader range of metric types, such as `subchannel/open_connections`, to monitoring systems. The implementation in `bigtable/metrics_monitoring_exporter.go` involves introducing a metric kind parameter and adjusting time interval calculation logic, particularly within functions like `recordToTimeSeriesPb`. Corresponding tests in `metrics_monitoring_exporter_test.go` have been updated and extended to validate the new `GAUGE` metric handling.

This commit introduces a new **Pacemaker component** into the **Bigtable client library's connection pool** (`connpool.go`). This **new feature** is designed to monitor runtime scheduling delays within the transport layer and report them as **OpenTelemetry metrics**, significantly enhancing observability. The `Pacemaker` is integrated during the `NewBigtableChannelPool` initialization, with comprehensive unit tests (`pacemaker_test.go`) ensuring accurate metric collection. This provides valuable insights into the performance characteristics and potential bottlenecks of the Bigtable client's internal operations.

This commit introduces a **new feature** to the **Bigtable client library**: a **preemptive connection recycler**. The `ConnectionRecycler` (implemented in `bigtable/internal/transport/conn_recycler.go`) monitors the age of Bigtable connections and automatically refreshes long-lived ones to prevent potential issues with stale connections. The `bigtable/client.go` module integrates this functionality, allowing users to configure its behavior via `ConnectionRecycleConfig` when creating a new client with `NewClientWithConfig`. This enhancement improves the overall **reliability and stability** of Bigtable client connections by proactively managing their lifecycle.

This commit introduces a **new feature** to the **Bigtable** client library by integrating **gRPC subchannel metrics** into its **OpenTelemetry observability** capabilities. It specifically modifies `bigtable/otel_metrics.go` to include these new metrics, enhancing the monitoring of underlying gRPC connection behavior. This allows users to gain deeper insights into the performance and health of their Bigtable client connections.

This commit performs a significant **refactoring** of the **Bigtable client library** by decomposing the central `bigtable.go` file into more specialized components. It extracts functionalities such as **client initialization**, **table operations**, **query execution**, and **bulk mutations** into dedicated files like `client.go`, `table.go`, `query.go`, and `apply_bulk.go` respectively. This **improves code organization and maintainability** within the `bigtable` package, making its internal structure more modular and easier to navigate. The change primarily impacts the **internal architecture** of the library, enhancing clarity without altering the external API or behavior for users.

This commit **introduces a new `connectionFactory`** within the **Bigtable client's internal transport layer** to abstract and manage the creation and priming of connections. This **new feature** integrates the factory into the `BigtableChannelPool` defined in `bigtable/internal/transport/connpool.go`, enhancing the robustness of connection management. It also includes updates to test helpers in `bigtable/internal/transport/connpool_helper_test.go` and new test suites in `bigtable/internal/transport/connpool_test.go` to validate the new retry logic and ping error handling. This change improves the reliability and flexibility of Bigtable client connections by centralizing connection lifecycle logic.

This commit **fixes** the logic for determining `hostName` and `region` attributes within the **Bigtable client's OpenTelemetry metrics**. It introduces robust fallback mechanisms that leverage Kubernetes and cloud availability zone information, ensuring these attributes align with OpenTelemetry semantic conventions. This **bug fix** improves the accuracy and consistency of metric reporting for the Bigtable client, especially in diverse deployment environments. New test cases have been added to validate the updated `hostName` and `region` derivation logic across various cloud and Kubernetes scenarios.