Simon MacMullen

This commit introduces a **new capability** to the **trace summary analysis** by allowing the `ValueColumnSpec` to be extended. A new `extensions` field has been added to the `ValueColumnSpec` message within `protos/perfetto/trace_summary/v2_metric.proto`, enabling **vendor-specific configurations** for aggregation across traces. This **feature enhancement** provides greater flexibility for defining custom metric aggregation logic. Corresponding **documentation** in `docs/analysis/trace-summary.md` has been updated to explain how to utilize these new extensions.

This commit **enhances the diagnostic capabilities** of the **trace processor** by explicitly recording an error when **empty power rail data packets** are encountered. The `android_probes_module.cc` now utilizes a new `power_rail_empty_packet` statistic, defined in `stats.h`, to track these occurrences during the `TokenizePacket` process. This **maintenance improvement** provides crucial visibility into potential data acquisition unreliability for power rails, allowing developers to identify and address issues that might otherwise go unnoticed. It helps determine if Perfetto experiences similar data integrity problems as observed in other systems.

This commit introduces a **new capability** to the **Perfetto SQL engine** by adding the `regexp_replace_simple` function, enabling **regular expression-based string replacement** directly within SQL queries. This **feature** enhances the **Trace Processor's** data manipulation capabilities by providing a powerful tool for advanced text transformation. The implementation involves a new `RegexpReplaceSimple` intrinsic function, a `Replace` method in the `util/regex` utility, and its definition within the SQL standard library's `functions.sql`. This addition significantly expands the string processing power available to users of Perfetto SQL.

This commit **enhances** the **Day Explorer plugin** by fundamentally changing its data source from generic "behaviors" to specific **device usage data**. The function `addDayExplorerBehaviors` in `ui/src/plugins/com.android.DayExplorer/index.ts` was renamed to `addDayExplorerUsage` to reflect this semantic shift, ensuring the plugin now queries and displays more precise activity information. This **feature update** provides users with a more accurate representation of device activity within the Day Explorer interface. Additionally, minor **capitalization issues** in track and group names within the plugin's data handling are **fixed**, improving overall consistency.

This commit delivers substantial **UI enhancements** and a **bug fix** for the **CPU-per-UID** analysis in the tracing UI. It **optimizes performance** by preventing the materialization of counter data in `QueryCounterTrack`, significantly reducing loading times. New capabilities include **summary and filtered views** for CPU usage, providing clearer insights into major users and aggregated app/system activity, alongside improved track labeling within the **`com.android.CpuPerUid` plugin**. Furthermore, a **critical bug** in `src/trace_processor/perfetto_sql/stdlib/android/cpu/cpu_per_uid.sql` is resolved, ensuring correct package name attachment for non-base users.

This commit significantly **improves the performance** of the `android.kernel_wakelocks` calculation within the `trace_processor/perfetto_sql/stdlib` module. It **refactors** the SQL logic in `src/trace_processor/perfetto_sql/stdlib/android/kernel_wakelocks.sql` to leverage `counter_leading_intervals()` and an intermediate table. This **optimization** addresses a known bug and reduces the processing time for typical long traces from 9.4 seconds to 1.6 seconds. The change provides a much faster analysis of kernel wakelock data, enhancing the overall efficiency of Android trace processing.

This commit **fixes** a **bug** in **Trace Processor's SQL `counter_leading_intervals()` function** to correctly handle counter transitions, specifically when values remain at zero or transition from zero. Previously, these "zero deltas" were not accurately processed, leading to incorrect interval calculations for counter data. The **`CounterTrackPartition`** in `counter.h` is **refactored** to store `last_equal_id`, `last_equal_ts`, and `last_equal_val`, enabling the aggregation logic in `type_builders.cc` to correctly emit intervals even when values are identical. This ensures more **accurate analysis of counter data** in Trace Processor and is validated by new and updated **diff tests** in `tests.py`, `memory.py`, and `tests_power_rails.py`.

This commit addresses a **data accuracy bug** within the **Perfetto SQL standard library** by correcting the calculation of CPU usage metrics. Specifically, it **fixes** the `android/cpu/cpu_per_uid.sql` module, ensuring that `diff_ms` and `cpu_ratio` are accurately derived using `next_value - value` instead of the previously lagging `delta_value`. This change improves the reliability of **CPU per UID metrics** for Android traces. Furthermore, **documentation is updated** in `counters/intervals.sql` to clarify the schema and behavior of `delta_value` within the `counter_leading_intervals` macro, enhancing understanding for future users.

This commit introduces **CPU usage per UID analysis** to Perfetto, enabling detailed insights into process-level CPU consumption. It adds a new `cpu_per_uid.sql` module to the **trace processor's SQL standard library**, providing `android_cpu_per_uid_track` and `android_cpu_per_uid_counter` tables for querying this data. Concurrently, a new **Perfetto UI plugin** (`com.android.CpuPerUid`) is added to visualize this information, allowing users to easily inspect CPU usage broken down by individual UIDs directly within the UI. This **new capability** significantly enhances **Android performance debugging** by providing granular CPU attribution.

This commit **refactors** the monolithic `AndroidLongBatteryTracing` plugin by splitting its extensive functionality into six more specialized plugins to improve modularity and maintainability. **New plugins** `com.android.Bluetooth`, `com.android.ContainedTraces`, `com.android.DayExplorer`, and `com.android.RilModem` now handle specific Android tracing aspects like Bluetooth events, contained traces, Day Explorer data, and RIL modem information, respectively. A new `com.android.AndroidLongBatterySupport` plugin provides shared utilities and group state management, while the original `com.android.AndroidLongBatteryTracing` retains the remaining core logic. Most of these **new capabilities** are **enabled by default** in `ui/src/core/default_plugins.ts`, with `com.android.RilModem` being an opt-in exception due to its complexity and performance impact. This significant architectural change enhances the organization and control over **Android tracing capabilities** within the UI.

This commit introduces a **feature enhancement** to the **AndroidLongBatteryTracing plugin**, improving how job data is presented in the UI. It implements **conditional logic** within `ui/src/plugins/com.android.AndroidLongBatteryTracing/index.ts` to fetch job information from `statsd atoms` when the `atom.scheduled_job_state_changed` and `google3` features are enabled. This change prioritizes a potentially more granular data source, falling back to the existing `battery stats` if the new `statsd` atoms are not available. The update aims to provide more accurate and detailed job state insights for Android long battery traces by leveraging a richer data source.

This commit introduces a **new capability** to the **Trace Processor** by adding **support for processing Android CPU-per-UID trace data**. It implements a dedicated `AndroidCpuPerUidModule` within the `proto` importers, complete with its state management (`AndroidCpuPerUidState`), to parse and interpret this specific data source. This **enhances the Trace Processor's diagnostic capabilities for Android system performance**, allowing for detailed analysis of CPU usage attributed to individual user IDs. Build configurations across `Android.bp`, `BUILD`, and `BUILD.gn` are updated, and new `diff_tests` are included to validate the correct parsing and processing of this new data.

This commit introduces an **"on use" mode** for the **`packages_list` data source**, allowing it to only emit package information for UIDs that are actively consuming CPU. This **new capability** is enabled via a `only_write_on_cpu_use_every_ms` configuration option in `PackagesListConfig`, which triggers polling for CPU-per-UID usage. To facilitate this, a substantial **refactoring** extracts CPU time collection and incremental state management into a new reusable utility class, `AndroidCpuPerUidPoller`, which is now utilized by both `packages_list` and `android_cpu_per_uid` probes. This change significantly improves tracing efficiency by reducing the overhead and trace size associated with the `packages_list` probe when only active processes are of interest.

This commit introduces a **significant enhancement** to the **`trace_processor`'s `metatrace` subsystem** by making its internal state **thread-local**. It refactors the `g_enabled_categories` variable and the `RingBuffer` instance, accessed via `GetInstance()`, to operate on a per-thread basis instead of globally. This **enables `metatrace` to function correctly and safely in multi-threaded environments**, resolving a prior limitation that prevented its use in concurrent contexts. The change allows for more robust performance profiling and debugging in concurrent applications, provided each `trace processor` instance is managed per thread.

This commit implements a **bug fix** by **removing an unnecessary and problematic call** to `DynamicLibLoader::EnsureCpuTimesAvailable` from the `Start` method of the **Android CPU per UID data source** (`android_cpu_per_uid_data_source.cc`). The system server already handles the underlying `startTrackingUidTimes()` functionality, and the `traced` process lacks the required permissions to perform this operation directly. This change **simplifies the data source startup** by eliminating a redundant and failing permission check. Consequently, it ensures the **`android_cpu_per_uid` probe** can initialize correctly and reliably collect CPU usage data per UID without encountering permission-related failures.

This commit introduces a **new capability** to the **Perfetto tracing system** by adding the `AndroidCpuPerUidDataSource`. This new data source is responsible for polling and recording **CPU time per UID and per cluster** on Android devices, utilizing the `cputimeinstate` library. The change involves significant updates to **proto definitions** for both configuration (`CpuPerUidConfig`) and trace data (`CpuPerUidData`), alongside comprehensive modifications to the **build system** to integrate this new probe. This enhancement allows for more granular performance analysis by providing detailed CPU usage statistics for individual applications and CPU core types within traces.

This commit implements a **defensive bug fix** for the **Android kernel wakelock tracing** subsystem, specifically addressing occasional implausibly large wakelock values reported by `SuspendControlService`. The `android_kernel_wakelocks` probe in `traced` now includes logic to detect these erroneous values, **ignoring** them from processing and instead setting an error flag (`kKernelWakelockErrorImplausiblyLargeValue`). A new statistic, `kernel_wakelock_implausibly_large_value_reported`, is also introduced and incremented within the **trace processor** to track the frequency of these occurrences. This change prevents corrupted or misleading wakelock data from appearing in traces, significantly improving the reliability and accuracy of **kernel wakelock analysis**.

This commit introduces a new **diff test** for **Android kernel wakelocks** within the **trace processor's standard library**. It adds a new test data file, `android_kernel_wakelocks.textproto`, and integrates a corresponding test case, `test_android_kernel_wakelocks`, into `tests.py`. This **maintenance** work enhances the **testing infrastructure** by ensuring the accurate processing and output of Android kernel wakelock information. The new test improves the reliability of **Android performance analysis** by preventing regressions in wakelock data interpretation.

This commit **fixes a regression** in the **Perfetto SQL standard library**'s **Android kernel wakelocks** analysis. It **corrects a bug** in the `src/trace_processor/perfetto_sql/stdlib/android/kernel_wakelocks.sql` query, specifically within the `_android_kernel_wakelocks_joined` CTE. The `GROUP BY` clause was mistakenly applied to a pre-renamed `ts` column, rather than the intended `original_ts` after a recent cleanup. This ensures **accurate aggregation** and **correct data analysis** for **Android kernel wakelock traces**, preventing erroneous results in Perfetto's SQL interface.

This commit **fixes a bug** in the **Perfetto SQL standard library** by correcting the `duration` calculation within the `android_kernel_wakelocks` table. Previously, an erroneous `WHERE` clause in `src/trace_processor/perfetto_sql/stdlib/android/kernel_wakelocks.sql` caused the `dur` column to only reflect "awake time," leading to inaccurate total duration reporting for kernel wakelocks. The change removes this incorrect filter, ensuring `dur` now represents the true total duration. Additionally, a new `awake_dur` column is introduced to explicitly expose the "awake time" duration, providing more granular data for **Android power management trace analysis**.