Garrett Wu

This commit performs a **refactoring** of the **`bigframes.testing` module** to improve its internal structure and resolve import issues, particularly in `google3` environments. It **removes direct exports of assertion functions** from `bigframes/bigframes/testing/__init__.py`, relocating their primary exposure to `bigframes.testing.utils`. Consequently, all **system and unit tests** across various `bigframes` components, including `test_dataframe`, `test_groupby`, and `test_ml`, are updated to import these assertion functions from their new, explicit location within `bigframes.testing.utils`. This **maintenance** effort ensures consistent and correct test execution by standardizing import paths for testing utilities.

This commit introduces a **new capability** to the **BigQuery ML integration** within the `bigframes` library. It adds the `bigframes.bigquery.ml.get_insights` function, allowing users to programmatically retrieve insights from BigQuery ML models. This function generates the necessary SQL for the BigQuery `ML.GET_INSIGHTS` statement, enabling direct analysis of model explanations like contribution analysis. Comprehensive unit and system tests have been added to ensure the correct functionality and SQL generation of this new feature, enhancing the analytical power available to `bigframes` users.

This commit **introduces new integration tests** for **multimodal data operations** within the **`bigframes` package**. Specifically, it adds the `test_multimodal_example` function to `packages/bigframes/samples/snippets/multimodal_test.py`, designed to verify code snippets from the Google Cloud BigQuery multimodal data SQL tutorial. This **enhances the reliability** of `bigframes`'s multimodal features by ensuring that documented examples function correctly, serving as a **maintenance and validation effort**.

This commit **refactors** the **developer documentation** for the `bigframes` package by **modularizing** the monolithic `GEMINI.md` file. It breaks down comprehensive guidelines into several smaller, topic-specific markdown files located under `packages/bigframes/.gemini/`, covering areas such as contribution constraints, documentation standards, scalar operator implementation, and various testing and styling instructions. This **improves documentation organization and maintainability**, allowing for easier navigation and selective import of instructions, particularly for automated agents. Additionally, `GEMINI.md` is added to `.gitignore` to prevent conflicts and manage its state post-modularization.

This commit **refactors and updates the LLM test suite** within the `bigframes` package, primarily focusing on **test maintenance** and **modernization**. It **removes tests for the deprecated `gemini-2.0-flash-exp` model** from `test_llm.py` and `test_multimodal_llm.py`, ensuring the test suite remains relevant. Concurrently, new tests are introduced for **text embedding**, **multimodal embedding**, **structured output**, and **retry logic**, significantly enhancing the coverage of current LLM functionalities. Additionally, an outdated `generative_ai` notebook is removed, and another has its kernel information updated.

This commit introduces a **FutureWarning** in **BigFrames** to alert users when a default BigQuery connection is implicitly accessed, encouraging explicit connection management. A new `bq_connection` property was added to the `bigframes.session` module, which emits this warning upon its first access. This **feature addition** is complemented by **refactoring** internal functions like `_resolve_connection_id` and `resolve_bq_connection_id` across `bigframes.bigquery._operations.ai`, `bigframes.functions._function_session`, and `bigframes.operations.blob` to consistently utilize this new property. The change ensures that users are proactively informed about reliance on default connections, improving future compatibility and best practices.

This commit **fixes a formatting bug** within the **User-Defined Function (UDF) generation mechanism** of the `bigframes` library. Specifically, it adjusts the string formatting logic in the `generate_managed_function_code` function located in `packages/bigframes/bigframes/functions/function_template.py`. The change ensures that the **generated UDF handler code** is now **correctly dedented**, resolving previous indentation issues. This **bug fix** improves the reliability and readability of automatically generated UDFs, preventing potential syntax errors or unexpected behavior due to incorrect spacing.

This commit introduces a **new capability** to the **BigQuery AI operations** within the `bigframes` library. It **enhances** the `bigquery.ai.generate_table` function by allowing its `output_schema` parameter to accept a `Mapping` type, such as a dictionary, in `packages/bigframes/bigframes/bigquery/_operations/ai.py`. This change provides greater flexibility for users to define the schema of AI-generated tables, simplifying complex schema specifications. **New unit and system tests** have been added to `test_ai.py` files to ensure the correct processing and functionality of this updated parameter.

This commit provides a **documentation fix** for the **BigQuery AI operations** within `bigframes`. It updates the docstrings for the `generate_text` and `generate_table` functions in `packages/bigframes/bigframes/bigquery/_operations/ai.py`. The change ensures that the input column is correctly specified as `prompt` instead of `content`, improving clarity and correctness for users interacting with these AI capabilities.

This commit introduces a **new capability** by adding the `generate_table` function to the **`bigframes.bigquery.ai` module**. This function allows users to **generate tables using BigQuery ML models**, specifying an output schema and optional parameters for advanced data generation. The core logic is implemented in `bigframes/bigquery/_operations/ai.py` and then exposed via `bigframes/bigquery/ai.py`. This feature is thoroughly validated with new unit and system tests, ensuring robust integration. This significantly enhances the `bigframes` library's programmatic interaction with BigQuery ML for data synthesis and analysis.

This commit **refactors** the import and usage of the `cast` function across several **BigQuery DataFrames ML modules** to improve code consistency. It updates files like `bigframes/ml/base.py` and `bigframes/ml/llm.py` to explicitly use `typing.cast` instead of `from typing import cast`. This **code style improvement** standardizes the approach to type casting, which helps ensure the internal codebase accurately reflects how the `cast` method is presented in public documentation. The change affects core ML components, enhancing overall maintainability and readability within the `bigframes.ml` package.

This commit **adds new system tests** for the `bigquery.ml.create_model` function within the `bigframes` library, specifically targeting the `packages/bigframes/tests/system/large/bigquery/test_ml.py` file. These tests validate the correct behavior of creating **linear regression models** and applying various **transform options** using the BigQuery ML integration. This **testing enhancement** significantly improves the reliability and robustness of the `bigframes` package by ensuring critical machine learning functionalities operate as expected.

This commit introduces a **new capability** to the **BigFrames library**, providing a `bigframes.bigquery.load_data` function. This function allows users to programmatically **load data into BigQuery tables** by generating and executing BigQuery `LOAD DATA` DDL statements. The change primarily affects the `bigframes.bigquery` and `bigframes.core.sql` modules, with the new `load_data` function exposed via `bigframes.bigquery.__init__.py` and its underlying DDL generation handled by `bigframes.core.sql.io.load_data_ddl`. This feature significantly enhances the data ingestion capabilities of BigFrames, simplifying the process of moving external data sources into BigQuery within a BigFrames workflow.

This commit performs a **code cleanup** by **removing redundant assertions** from the codebase. This **maintenance** task primarily affects the **internal testing and validation logic**, streamlining the code by eliminating unnecessary checks that either duplicated existing ones or were no longer relevant. The change improves **code readability** and maintainability without altering any user-facing functionality or core logic.

This commit performs an **internal refactoring** within the **`bigframes.bigquery` module** by relocating the `table.py` file to `bigquery/_operations/table.py`. This **maintenance** task reorganizes the codebase, specifically affecting the `create_external_table` function's internal import path. Corresponding unit tests in `test_table.py` were updated to reflect the new module structure, ensuring no functional changes or regressions. The change primarily impacts the **internal code organization** and **maintainability** of the `bigframes` library without altering its public API or user-facing behavior.

This commit introduces a **new feature** by adding the `bigquery.ml.generate_embedding` function to the `bigframes` library. This function allows users to **generate text embeddings** directly from dataframes using BigQuery ML models, simplifying natural language processing tasks. It enhances the **BigQuery ML integration** by providing a high-level API that translates into the appropriate `ML.GENERATE_EMBEDDING` SQL statements. The change impacts the `bigframes.bigquery.ml` module and its core SQL generation logic, and is thoroughly validated with new **system and unit tests**.

This commit introduces a **new capability** to the `bigframes` library by adding the `bigquery.create_external_table` method. This method allows users to programmatically create **BigQuery external tables**, simplifying the process of integrating external data sources directly into BigQuery. The functionality is exposed through the `bigframes.bigquery` module and implemented in `bigframes.bigquery.table.py`, with SQL DDL generation handled by `bigframes.core.sql.table.py`. This enhancement provides a direct API for managing external data sources within BigQuery via BigFrames. Comprehensive unit and system tests are included to ensure the correctness and reliability of this new feature.

This commit introduces a **new capability** to the **BigQuery DataFrames library**, enabling users to perform **text generation** using BigQuery ML models. It adds a `generate_text` function to the `bigframes.bigquery.ml` module, which internally handles the execution and generates the appropriate `ML.GENERATE_TEXT` SQL via `bigframes.core.sql.ml`. This enhancement significantly expands the **BigQuery ML integration**, allowing for advanced natural language processing tasks directly within the BigQuery DataFrames environment. The change includes new unit tests and SQL snapshots to ensure correct functionality and SQL generation for various options.

This commit introduces a **new feature** by adding a `transform` function to the **BigQuery ML integration** within `bigframes`. This function enables users to perform BigQuery ML model transformations on input dataframes, abstracting the underlying `ML.TRANSFORM` SQL statement generation. It significantly enhances the `bigframes.bigquery.ml` module by providing a direct and simplified API for applying trained ML models to transform data. This addition streamlines machine learning inference workflows, allowing users to easily integrate model predictions into their data processing pipelines. Unit tests and SQL snapshots are included to ensure the correct functionality and SQL generation of the new `transform` operation.

This commit performs **maintenance** and **test cleanup** by removing references to the deprecated `claude-3-opus` LLM model. It **removes `claude-3-opus` from test parameterizations and conditional logic** within the `test_llm.py` system tests, ensuring the test suite no longer validates a non-supported model. Concurrently, the `packages/bigframes/bigframes/ml/llm.py` module is updated to **mark `claude-3-opus` as deprecated** in its internal documentation. This change streamlines the `bigframes.ml` subsystem's test suite and clarifies the status of the `claude-3-opus` model.