elrakadm

This commit **refactors** the **SQL module validation logic** by **centralizing** the `check_banned_patterns` functionality into a shared utility. The `check_banned_patterns` function is now located in `python/generators/sql_processing/utils.py`, making it a reusable component for various SQL processing tasks. As part of this **enhancement**, the underlying `match_pattern` function was updated to **ignore case** during pattern matching, improving the robustness of banned SQL statement detection. This change ensures more consistent and comprehensive enforcement of SQL coding standards across the project by preventing circumvention through case variations.

This commit **reverts** a previous change (commit `663d2fdc`) that introduced a **regression** in the **Trace Processor**'s structured query analysis. The original change, which aimed to refactor the `AnalyzeStructuredQueries` API to accept multiple queries and simplify shared query handling, inadvertently caused `TraceSummary` queries to fail with a "table already exist" error. By reverting these modifications, this commit **restores the functionality** of `TraceSummary` queries and prevents the internal error during query execution. This impacts the **Trace Processor backend**'s query generation and execution, as well as the **UI's ExplorePage** query builder, which had been adapted to the multi-query API.

This commit **reverts** the previous introduction of a `slices` module for Critical User Journeys (CUJs) into the standard library, addressing a **bug** where the `android_jank_cuj_draw_frame_slice` table was missing a `duration` column. As part of this **revert**, the **Android Jank metrics** subsystem within the **trace processor** undergoes **refactoring**. SQL functions and table/view definitions, including `vsync_from_name` and `gpu_completion_fence_id_from_name`, are moved back from the deleted `slices.sql` module into `relevant_slices.sql`. Additionally, `cujs_boundaries.sql` is updated to correctly reference CUJ boundaries, ensuring the integrity and proper calculation of **Android Jank traces**.

This commit **reverts** a previous change that modified the **Trace Processor's structured query analysis API** (`AnalyzeStructuredQuery`) to process a single query identified within a `TraceSummary`. The revert is crucial as the original change introduced **failures in downstream tests** due to incorrect handling of **shared queries**. Consequently, the **Trace Processor API**, its **RPC protocol**, and the **structured query generator** are restored to their prior state, now accepting multiple structured queries directly instead of relying on a single ID within a summary. This **bug fix** undoes the problematic refactoring, ensuring the stability and correctness of structured query processing. The change also impacts the **UI's Explore Page query builder**, which reverts its logic to align with the restored API, removing special handling for query IDs.

This commit **enhances** the **Perfetto metrics v2 system** by introducing additional units for measurement. It specifically adds `Celsius`, `MicroAmpHours`, and `TimeMins` to the `MetricUnit` enum within the `TraceMetricV2Spec` defined in `protos/perfetto/trace_summary/v2_metric.proto`. This **feature enhancement** expands the capabilities of the **trace summary protobuf definition**, allowing for more precise and varied data representation. Consequently, it enables better analysis of temperature, power consumption, and time-based measurements within **Perfetto's trace summary metrics**.

This commit **enhances the data model** for **trace summary metrics** by introducing additional metadata fields. It specifically adds `display_name`, `display_help`, and `doc_link` to the `DimensionSpec` and `ValueColumnSpec` messages in `protos/perfetto/trace_summary/v2_metric.proto`. This **feature enhancement** provides richer descriptive information for dimensions and value columns. This will improve their clarity and user understanding in downstream applications and UIs that consume these metric specifications.

This commit introduces a **performance optimization** to the **trace processor's metric bundle output**, ensuring that only interned dimension keys actually utilized by metrics are emitted. Previously, the system could output a full list of potential dimensions, leading to unnecessary proto output size. This **refinement** to the `trace_summary` module, specifically within `src/trace_processor/trace_summary/summary.cc`'s `WriteInternedDimensionBundles` function, significantly **reduces the size of the generated proto output** for metric bundles. An integration test was added to verify this behavior, and documentation for `InternedDimensionSpec` in `protos/perfetto/trace_summary/v2_metric.proto` was updated to reflect this optimization.

This commit **introduces a new boolean dimension type** to the **metric summary specification**, enhancing the flexibility of metric categorization. It **extends the `trace_summary` protobuf schema** by adding a `BOOLEAN` enum value and a corresponding `bool_value` field, allowing metrics to be grouped by true/false dimensions. The **`trace_processor`'s summary generation logic** is updated to correctly handle and validate these new boolean dimensions within `WriteDimension` and `WriteInternedDimensionValue`. An **integration test** was also updated to ensure proper functionality and verification of this **new capability**, enabling more expressive metric summarization.

This commit introduces a **new capability** to the **`trace_processor`'s `trace_summary` output**, specifically for **metric bundles**. It now includes a **top-level `bundle_id`** within the metric bundle, allowing users to easily identify the associated metric specification or group without needing to consult external documentation. This **enhancement** improves the interpretability of the metric output, making it clearer which metrics are being presented. The change primarily affects the `CreateQueriesAndComputeMetrics` function in `src/trace_processor/trace_summary/summary.cc` and is reflected in updated integration and diff tests to accommodate the new field.

This commit introduces **interned dimensions** to the **Trace Processor's Trace Summary subsystem**, enabling more efficient and expandable metadata for generated metrics. This **new feature** allows metric dimensions, such as process UPIDs or package names, to be linked to a separate bundle of richer metadata without duplicating all associated details for every metric entry. The change involves defining new protobuf structures like `InternedDimensionSpec` and `InternedDimensionBundle` in `v2_metric.proto`, and implementing the core processing, validation, and serialization logic within `src/trace_processor/trace_summary/summary.cc`. This significantly enhances the expressiveness and storage efficiency of **trace summary metrics**, with integration tests ensuring correct functionality.

This commit **reverts** the previous introduction of **specialized storage for small value equality** within the `trace_processor/dataframe` module, as the feature caused a **regression in the `power_rails_summary_metric`**. This **bug fix** undoes the complex logic across several files, including `adhoc_dataframe_builder.h`, `bytecode_instructions.h`, `bytecode_interpreter_impl.h`, `query_plan.cc`, and `types.h`. Specifically, it removes the specialized storage structures, related bytecode instructions, interpreter logic, and query planning preferences. The revert ensures the stability and correct functioning of the trace processor by eliminating the identified performance issue.

This commit **reverts** a previous change that introduced performance regressions in the **Trace Processor's Android Java heap metrics**. It specifically undoes the integration of the `heap_graph.class_relationship` module and the use of the `android_heap_graph_class_find_descendants` macro within `src/trace_processor/metrics/sql/android/java_heap_histogram.sql`. This **bug fix** replaces the problematic logic for `android_special_classes` with a recursive CTE, effectively **preventing timeouts in trace metrics** and restoring the stability of Android heap analysis.

This commit **fixes a data reporting issue** within the **trace processor's metrics calculation** by ensuring the `sched_duration_ns` metric always returns a numerical value. Specifically, the SQL logic in `src/trace_processor/metrics/sql/trace_metadata.sql` is updated to emit `0` instead of `NULL` when the `sched` table is empty. This **bug fix** prevents incorrect `NULL` values from appearing in 24-hour traces from wear devices, leading to more robust and accurate scheduling duration metrics.

This commit **improves the accuracy** of scheduling duration measurements within the **trace processor's metrics generation**. It **fixes** the calculation of `sched_duration_ns` in the `src/trace_processor/metrics/sql/trace_metadata.sql` metric by switching to the `TO_MONOTONIC` function. This ensures that scheduling durations are measured using a **monotonic clock**, preventing inaccuracies caused by system time adjustments or non-monotonic clock sources. Consequently, any analysis relying on `sched_duration_ns` will now benefit from more reliable and consistent timing data, as reflected in updated test outputs for `trace_metadata`.

This commit **reverts a previous revert** (`ec96e82e`) that introduced a **regression** in the `android_jank_cuj` script. It effectively **restores the original changes** that added SF and HWUI callback logic and modified dependencies on the `android.frames.timeline` module within the `src/trace_processor/metrics/sql/android/jank/` SQL definitions. This **bug fix** re-introduces direct definitions for tables such as `vsync_missed_callback`, `android_jank_cuj_app_to_sf_vsync_match`, and `android_jank_cuj_sf_process`. The primary impact is resolving a critical issue affecting **Android jank analysis metrics** by ensuring the `android_jank_cuj` script functions correctly.

This commit introduces **explicit dimension specification** for **metrics v2** within the `trace_processor`, allowing users to define the expected type of dimensions in `protos/perfetto/trace_summary/v2_metric.proto`. This **new capability** ensures `summary.cc` correctly reads columns based on specified dimension/metric names rather than order, and accepts nulls for all dimension types. Crucially, it **fixes critical bugs** that led to incorrect output when dimension orders differed or metric columns were not last, significantly enhancing the **robustness and correctness** of `metrics v2` data extraction. New tests have been added to validate this improved functionality.

This commit **fixes** an issue in the **systrace parser** by removing non-UTF-8 characters from `print` ftrace events, which are incompatible with SQLite's UTF-8 string expectation. It introduces a new utility function, `CheckAsciiAndRemoveInvalidUTF8`, within `perfetto/ext/base/string_utils.h` and `src/base/string_utils.cc` to sanitize strings. The `src/trace_processor/importers/systrace/systrace_parser.cc` is updated to utilize this function, specifically in `ParsePrintEvent`, ensuring data integrity. This **maintenance** work prevents potential errors during trace processing and improves the **reliability** of `systrace` data ingestion into the trace processor.

This commit introduces a **new capability** to **Perfetto's Android process metadata** by adding an `is_kernel_task` field to the `AndroidProcessMetadata` protobuf message. It includes new SQL logic in `src/trace_processor/perfetto_sql/stdlib/android/process_metadata.sql` to identify and populate this field, allowing users to distinguish kernel processes in trace analysis. This enhancement impacts the `AndroidProcessMetadata` protobuf and various **Trace Processor metrics**, requiring extensive **test updates** across numerous Android-related diff tests to reflect the new output. Ultimately, this provides more detailed process classification for **Android traces**, improving the accuracy of performance analysis.

This commit **enhances the Android slice name standardization** within `trace_processor` by refining the logic for removing numerical patterns. It **modifies the intrinsic `StripHex` function** to correctly identify and remove standalone sequences of two or more repeated digits from slice names. This **new capability** is then applied to several **Android SQL standardization functions** like `android_standardize_slice_name`, ensuring more consistent and cleaner trace data. The change improves the accuracy of slice name aggregation and analysis, with corresponding **unit and diff tests updated** to reflect the new expected output.

This commit introduces a **bug fix** to the **`trace_processor`'s `perfetto_sql` standard library for Android**, specifically addressing how **thread names** are parsed. It modifies the thread name processing logic in `src/trace_processor/perfetto_sql/stdlib/android/thread.sql` to **remove numerical characters** from thread names, but only when these numbers are preceded by a non-alphanumeric character. This is achieved by applying the `__intrinsic_strip_hex` function after an initial string splitting operation. The change improves the reliability and consistency of thread identification within Android traces by preventing misleading numerical suffixes from appearing in processed thread names.