Samuel Wu

This commit **improves the robustness** of the **Wattson plugin** by changing how it retrieves CPU information. Previously, Wattson relied on the `SchedPlugin`, which could inaccurately report CPUs only if they had tasks scheduled, leading to incomplete data. Now, the plugin directly queries the more reliable `cpu` table to identify all available CPUs on a device, ensuring comprehensive data even for WALT-based schedulers where CPUs might be idle. This **bug fix** (Bug: 495918677) provides a more accurate and complete view of CPU topology for analysis within the **UI**, making Wattson's insights more reliable.

This commit **refactors** the **Trace Processor's Android metrics subsystem** by updating how **Wattson metrics** access Android Jank CUJ (Common User Journey) data. It modifies `src/trace_processor/metrics/sql/android/wattson_atrace_apps_rails.sql` and `src/trace_processor/metrics/sql/android/wattson_atrace_apps_threads.sql` to **include the `android.cujs.base` module**. This change is a **cleanup** that leverages the `stdlib` for the `android_jank_cuj` table, which was recently moved there. The update improves **maintainability** and **consistency** by using a standardized module for common data, affecting the overall structure of Android-related metric definitions.

This commit **refactors** the **Wattson plugin's CPU support availability check** within the UI, specifically targeting the `hasWattsonCpuSupport` function. It **simplifies the underlying SQL queries** to directly utilize `cpu_counter_track`, thereby removing the need for complex `stdlib` table inclusions. This **performance improvement** results in a **faster availability check** for Wattson CPU support, enhancing UI responsiveness. The change also **reduces unnecessary dependencies** by eliminating unused module imports, contributing to a more efficient **Wattson integration**.

This commit introduces a **new capability** to the **Perfetto UI**, enabling the automatic pinning of **Wattson power tracks** for top power-consuming threads when analyzing **SysUI jank metrics**. The `PinAndroidPerfMetrics` plugin now integrates with the `Wattson` plugin, calling `updateWindowsOfInterest` to identify and pin relevant threads within a specified window of interest without automatically scrolling the view. This enhancement, requested by the SysUI team, significantly improves the **usability** for performance engineers by automatically highlighting critical power consumption data associated with jank events. Minor **maintenance** updates to UI screenshot hashes are also included.

This commit **introduces a new feature** to the **Wattson plugin** in the UI, allowing users to **pin per-thread Wattson power tracks**. This new capability enables the automatic pinning of top threads by power within specified windows of interest, such as app startup or jank windows, when activated via URL parameters. The change provides a **more informative view** for performance analysis by highlighting potentially problematic threads' power consumption. It primarily affects `ui/src/plugins/org.kernel.Wattson/index.ts` and `ui/src/plugins/org.kernel.Wattson/wattson_thread_utils.ts`, enhancing the **UI's diagnostic capabilities** for power-related issues.

This commit performs a **maintenance refactoring** within the **`Wattson` CPU analysis module** of the `trace_processor/perfetto_sql` `stdlib`. It **optimizes memory and performance** by replacing `SPAN_OUTER_JOIN` with `_interval_intersect()` for `arm_dsu` tables and merging intermediate tables for general CPU statistics calculations. This change reduces the number of internal tables, leading to more efficient data processing for L3 hit/miss rates and overall CPU statistics. The `test/trace_processor/diff_tests/stdlib/wattson/tests.py` file is updated to align with these refactored queries.

This commit **enhances the presubmit system's error reporting** by preventing silent failures from helper scripts. It modifies the `tools/run_presubmit` script to **print the standard output of subprocess commands** when they fail, providing immediate context for issues. This **maintenance** update specifically impacts checks such as `CheckSqlModules`, `CheckIncludeViolations`, and `CheckProtoComments`, ensuring developers receive **clearer and more actionable error messages**. The change significantly **improves the developer experience** by making presubmit failures easier to diagnose and resolve.

This commit **expands the `interval_intersect` intrinsic** within **Trace Processor**'s **Perfetto SQL** engine, increasing its capacity to intersect up to 10 tables from the previous limit of 5. This **feature enhancement** is immediately **leveraged to refactor Wattson's CPU pivot SQL**, consolidating multiple `_interval_intersect` calls into a single, more efficient operation, thereby **improving memory and performance** for CPU statistics and DSU frequency analysis. Additionally, a **minor fix** adjusts the return type of `__intrinsic_table_ptr_bind` in the SQL engine. This change allows for more complex interval analysis and optimizes key performance metrics within the **Wattson** module.

This commit delivers significant **performance optimizations** and **refactoring** for the **Wattson** analysis module within the `trace_processor` `stdlib`. It improves runtime by up to ~6% by streamlining SQL queries, specifically by removing unnecessary `LEFT JOIN`s and moving static calculations to be performed only once per unique configuration. The changes primarily affect **CPU idle state detection** in `cpu/idle.sql` and **CPU pivot calculations** in `cpu/pivot.sql`, simplifying CTEs and optimizing data processing. This work enhances the efficiency and speed of Wattson's power and performance analysis.

This commit introduces **support for the MT6897 device** within the **Perfetto `stdlib`'s Wattson power estimation component**. This **new capability** provides specific 1D and 2D CPU power curve data for the MT6897, along with its device information and CPU policy mappings. Build files are updated to include the new `mt6897_2d.sql` curve definition, which is then integrated into the main 2D CPU curves. This enables the trace processor to perform **accurate power estimations** for the MT6897 device, significantly expanding the range of hardware for which Wattson can provide detailed power analysis.

This commit introduces a **new feature** to the **Wattson plugin** in the UI, enabling the visualization of **per-thread power estimates**. It enhances the `WattsonThreadSelectionAggregator` to support filtering by UTID and adds a 'Show' button to display individual thread power tracks, supported by new utility functions like `addWattsonThreadTrack` and updated SQL queries. This allows users to more easily analyze and debug power consumption at a granular thread level, significantly improving performance insights. A minor **refactoring** also renames `hasWattsonSufficientCPUConfigs` to `missingWattsonCpuConfigs` for improved clarity within the plugin's warning system.

This commit **refactors** the **Perfetto metrics system** by changing the `is_crude_estimate` field within **Android Wattson power metrics protobuf schemas** from a boolean to an `int32`. This **compatibility fix** addresses a limitation where the `PerfettoGenericPostProcessor` could not correctly process string representations of boolean values (e.g., 'true'/'false'). The change impacts `wattson_in_time_period.proto`, `wattson_tasks_attribution.proto`, and `perfetto_merged_metrics.proto`, along with the Python `trace_processor`'s `metrics.descriptor`. This ensures **correct data processing and interpretation** of Wattson metrics, preventing parsing errors in downstream tools and required updates to several metric test outputs.

This commit **refactors and optimizes Wattson's SQL analysis queries** to improve performance and reduce table sizes. It **simplifies computations** by removing redundant JOINs for default policy and frequency, calculating them per unique system state instead of per time slice, which also reduces the column width of `_w_independent_cpus_calc`. Additionally, **QUERY PLAN-informed simplifications** are applied to heavy-hitting tables like `_unified_idle_state` and the `_cpu_stats_subquery` macro within the **`stdlib/wattson/cpu`** module. These **performance optimizations** result in equal or slight improvements across various Wattson trace analyses by reducing query complexity and table footprint.

This commit introduces a significant **performance optimization** for **Wattson power estimation** within the `perfetto_sql/stdlib` module by implementing a **caching mechanism**. It refactors the CPU power calculation logic, particularly in `estimates.sql` and `pivot.sql`, to compute estimates only once for each **unique CPU configuration** (based on frequency and idle states) rather than for every individual time slice. This **refactoring**, which includes the introduction of a `config_hash` to identify unique configurations, then re-joins these cached results to the original trace. The change leads to a **substantial speedup** in Wattson computations, with performance improvements ranging from 10% to 70% across various benchmarks, making trace processing significantly faster.

This commit introduces a **performance optimization** to the **Wattson** subsystem by **refactoring** the SQL logic within `src/trace_processor/perfetto_sql/stdlib/wattson/cpu/pivot.sql`. The change consolidates a table and moves the filtering of non-active entries earlier in the processing pipeline, thereby reducing the dataset size for subsequent `CROSS JOIN` operations. This strategic **refactoring** significantly improves the efficiency of **Wattson calculations**, resulting in **2-4% faster** trace processing across various metrics, including `wattson_dsu_devfreq_system_state` and `wattson_atrace_apps_threads_output`.

This commit performs a **refactoring** within the **Perfetto SQL standard library** by **renaming Wattson curve lookup table (LUT) SQL files**. The redundant `device_` prefix has been removed from these filenames to improve clarity and reduce unnecessary verbosity. This change primarily affects the **Wattson module** within `stdlib`, requiring updates to build configurations in `Android.bp`, `BUILD`, and `BUILD.gn` files to correctly reference the new paths. Additionally, the `src/trace_processor/perfetto_sql/stdlib/wattson/curves/utils.sql` file's `INCLUDE PERFETTO MODULE` statements were adjusted to reflect the updated file names, ensuring continued functionality without any user-facing impact.

This commit introduces a significant **performance optimization** and **refactoring** to the **`intrinsics`** module by improving the `interval_intersect` function. The changes specifically target reducing heap allocations within `interval_intersect` by replacing a dynamic vector with a fixed-size array for indices and optimizing allocation patterns in `PushPartition` and `Run`. This optimization primarily benefits **Wattson** and other standard library tables that frequently use this intrinsic, leading to an 8-10% improvement in `interval_intersect` compute times. Consequently, various **Wattson** benchmarks, such as `wattson_dsu_devfreq_system_state` and `wattson_idle_attribution`, show an overall performance improvement of 1-2%.

This commit performs **maintenance** by ensuring the **`TrackEvent` plugin** correctly includes its dependencies. It **adds the `intervals.intersect` module** to the SQL query string within the `computeTrackEventCallstackFlamegraph` function in `ui/src/plugins/dev.perfetto.TrackEvent/index.ts`. This change is crucial for the proper functioning of `TrackEvent`'s callstack flamegraph computation, allowing it to correctly utilize interval intersection logic.

This commit introduces a **performance optimization** and **refactoring** for **Wattson** continuous interval calculations within the `trace_processor`'s `stdlib`. It modifies the `_get_continuous_estimates` macro in `continuous_estimates.sql` to utilize the `counter_leading_intervals!()` function from the `counters.intervals` module, replacing a less efficient SQL-based method. This change significantly **improves performance** by making the calculation approximately 30-40% faster across various use cases. While introducing negligible differences at the tail end of counters, the updated logic is reflected in the `android/wattson_trace_threads.out` test expectations.

This commit **enhances the Wattson UI plugin** by integrating a direct link to the official Wattson documentation within the CPU warnings, improving user access to detailed information. It also **clarifies the naming of performance tracks** by updating "estimate" tracks to "crude estimates" when the swapper is utilized for idle state calculations, providing more accurate context for the displayed data. This **feature enhancement** primarily affects the `org.kernel.Wattson` plugin and its associated tests, ensuring users of the Perfetto UI have a better understanding of the displayed metrics and easy access to external resources.