Lalit Maganti

This commit introduces a **new plugin system** for the **trace processor**, establishing a foundational **new capability** to decouple components from its core. It implements a self-registering mechanism where plugins, derived from `PluginBase`, are automatically discovered, topologically sorted by dependencies, and instantiated, integrating into key lifecycle stages within `trace_processor_impl` like importer registration and SQL engine initialization. This significant architectural change enhances the **extensibility** and **modularity** of the trace processor, laying the groundwork for future component migration and easier integration of new features. While no existing components are converted to plugins in this change, it provides the necessary infrastructure for future development.

This commit **rewrites** the **PerfettoSQL getting-started guide** (`docs/analysis/perfetto-sql-getting-started.md`) to significantly improve the onboarding experience for new users. As a **maintenance** task, it introduces new sections covering core concepts like scheduling, stack sampling, heap profiling, and heap graphs, complete with practical query examples. The update clarifies distinctions between UI and trace processor "tracks," promotes `stdlib` views (`thread_slice`, `process_slice`) for adding context to slices, and adds a crucial best practices section. This effort also removes confusing or misleading content, such as the scheduling-as-track example and ancestor/descendant slice sections. The overall scope is to make the **PerfettoSQL documentation** more accessible and easier to learn for new users.

This commit introduces **`TreeColumns`**, a new, optimized columnar storage type for tree-structured data within the **Trace Processor's tree processing subsystem**. It also adds **`TreeColumnsBuilder`** to efficiently construct this data, leveraging a new `BuildRaw()` method in `RuntimeDataframeBuilder` to bypass full `Dataframe` creation. The **`TreeTransformer`** and related **SQL intrinsics** like `TreeFromTable` are **refactored** to utilize `TreeColumns`, simplifying data handling and improving performance by avoiding unnecessary data conversions. This **new capability** streamlines tree data management, setting the stage for more efficient tree transformations.

This commit performs a significant **refactoring and maintenance** effort to **decouple the `PerfTracker` module from direct dependencies on ETM (Embedded Trace Macrocell) code** within the **Trace Processor**. It achieves this by introducing a new registration mechanism where ETM-related components now register their auxiliary tokenizer factories with `PerfTracker` via `TraceProcessorContext`, rather than `PerfTracker` having explicit ETM knowledge. Key architectural changes include `PerfTracker` accepting a registry of callbacks, the introduction of a pure virtual `OnEventsFullyExtracted()` method in `AuxDataTokenizer` for event finalization, and `EtmV4StreamDemultiplexer` now owning its `EtmTracker`. This improves modularity, making `PerfTracker` more generic and extensible for handling diverse auxiliary trace data streams.

This commit introduces a **new internal Google App Engine server** within the `infra/gae-internal` subsystem, designed to serve static content directly from a Google Cloud Storage bucket. This **new capability** establishes a dedicated Flask application (`main.py`) that handles file serving, including path normalization and CORS configuration, for internal Google extensions. The server is configured for deployment on GAE using Python 3.12 and Gunicorn, streamlining the delivery of internal UI assets. This provides a simple and efficient mechanism for hosting internal tools or extensions.

This commit **refactors** the **UI's core functionality** by migrating the `isInternalUser` detection and certain **analytics** dimension initialization to the **`dev.perfetto.ExtensionServers` plugin**. This change centralizes the logic for identifying internal users and dynamically adding analytics dimensions, deferring analytics setup until plugins are active. It also updates the extension server configuration to support **embedder-managed extension servers** via an `origin` enum, replacing a simpler `locked` boolean. This significant **architectural migration** is a key step towards removing the legacy script loader, enhancing the modularity and extensibility of the UI's telemetry and user-specific features.

This commit **fixes a bug** in the **Trace Processor's `descendant_slice` table function** that caused instant children occurring precisely at the parent slice's end timestamp to be erroneously omitted from results. Previously, the upper timestamp bound used a strict less-than comparison, but it now correctly uses less-than-or-equal and verifies ancestry for boundary candidates, mirroring the start-boundary logic in `src/trace_processor/perfetto_sql/intrinsics/table_functions/descendant.cc`. This **bug fix** ensures that **all valid descendant slices**, including those at critical boundary conditions, are accurately identified, improving the completeness of trace analysis. A **regression test** has been added to `test/trace_processor/diff_tests/stdlib/dynamic_tables/tests.py` to prevent future regressions for this specific scenario.

This commit **refactors** the caching mechanism for **UI extension server requests**, relocating it from the service worker directly into the `extension_server.ts` code. It introduces a **client-side caching strategy** using a network-first approach with Cache API fallback, integrated into the `fetchJson` and `fetchWithCache` logic. This change **simplifies the architecture** by eliminating the need for complex CORS workarounds previously required when caching in the service worker. The **UI's data fetching performance** from extensions is improved, leading to a more robust and efficient user experience by leveraging direct browser caching capabilities.

This commit introduces a **new configuration option**, `disable_extension_descriptors`, to the **Perfetto tracing service**, allowing users to control the emission of extension descriptors. This option, added to `BuiltinDataSource` in the trace configuration, enables the `src/tracing/service/tracing_service_impl.cc` to conditionally omit these descriptors from the trace. This **new capability** primarily serves to **reduce trace size**, which is critical for **reporter CTS tests**. Specifically, it resolves a failing test in `reporter_test_cts.cc` by preventing excessive trace data, thereby improving test stability and efficiency.

This commit **fixes a UI rendering bug** by incorporating `yMode` and `yDisplay` into the query slot invalidation parameters for the **`BaseCounterTrack` component**. Previously, changes to these display settings for counter tracks would not trigger a canvas rerender, leading to stale visualizations. This **maintenance fix** ensures that the **UI accurately reflects the selected display mode and range** for counter data, providing a consistent and responsive user experience.

This commit **enhances the UI for counter details panels** to accurately reflect the selected counter display mode, resolving previous inconsistencies between hover and click views. It introduces new utility functions in `ui/src/components/tracks/base_counter_track.ts` for calculating and formatting counter values based on the display mode. The **`CounterDetailsPanel`** in `ui/src/plugins/dev.perfetto.TraceProcessorTrack/counter_details_panel.ts` is updated to dynamically display delta and rate values, leveraging these new mode-aware formatting capabilities. This **`grow` enhancement** provides users with consistent and meaningful counter information, significantly improving the overall debugging experience for performance counters.

This commit introduces a **new capability** to visualize **Android GPU counter events** within the UI. A new plugin, `com.android.AndroidGpu`, is implemented to process these events, specifically converting raw delta values into per-second rates for clearer performance analysis. This **enhancement** to the **GPU counter visualization** system allows users to better understand Android GPU workload performance, with the new plugin enabled by default. Additionally, existing GPU counter tracks now pass more descriptive metadata, and UI screenshot checksums were updated to reflect related changes.

This commit **enhances** the **Trace Processor** by improving the **formatting of GPU counter track units**. It introduces a new utility function, `MeasureUnitToString`, within `src/trace_processor/importers/proto/gpu_event_parser.cc` to convert internal GPU counter unit enums into **human-readable strings**, replacing previously opaque numerical representations. This **new capability** ensures that users will now see properly formatted and easily understandable units for GPU counters in trace outputs. The change significantly improves the clarity and interpretability of performance data and is validated by new and updated diff tests in `test/trace_processor/diff_tests/parser/graphics/`.

This commit introduces a **usability enhancement** to the **Perfetto UI's Power Rails plugin**, addressing user feedback about manual track expansion. It modifies the `onTraceLoad` function in `ui/src/plugins/dev.perfetto.PowerRails/index.ts` to set the `collapsed` property to `false` for new subsystem group `TrackNode` instances. Consequently, **power rail subsystem group tracks will now be expanded by default** when a trace is loaded. This **new feature** significantly improves **productivity** by providing an immediate overview of power rail data, eliminating the need for users to individually expand tracks.

This commit **temporarily disables caching** for the **UI's extension server functionality** within the `dev.perfetto.ExtensionServers` core plugin. It addresses an issue where caching was not working correctly on `ui.perfetto.dev` by removing the `X-Perfetto-Extension` header and its associated logic from `extension_server.ts`. This is a **temporary fix** or **maintenance** action to ensure correct behavior, with plans to re-enable caching once the root cause is understood and resolved. The immediate impact is that extension server responses will not be cached, potentially affecting performance until the caching mechanism is properly restored.

This commit **refactors** the **Perfetto UI** by **separating the "Expand all" and "Collapse all" buttons** within the `timeline_toolbar.ts` module. Previously, a single button toggled between these states, often requiring users to press it twice to achieve the desired outcome. This **UI improvement** provides distinct and clear controls for **track management**, enhancing the user experience by eliminating ambiguity and making track expansion and collapse operations more intuitive.

This commit introduces a **bug fix** within the **UI's ExtensionServers plugin** to refine how extension headers are managed. It modifies the `extensionHeaders` function in `ui/src/core_plugins/dev.perfetto.ExtensionServers/extension_server.ts` to **only set the extension header when the service worker controller is in an 'activated' state**. This change prevents premature or incorrect header application, thereby improving the **robustness and correctness** of the UI's network interactions and communication with browser extensions or service workers.

This commit **enables the extension server plugin by default** within the **Perfetto UI**, making its functionality immediately available to all users. It modifies `ui/src/core/default_plugins.ts` to include `'dev.perfetto.ExtensionServers'` in the list of automatically loaded plugins. This change represents a **new capability** for the UI, ensuring that the **extension server functionality** is active without requiring manual configuration.

This commit introduces a **new capability** to **cache extension server requests** within the **Perfetto UI's service worker**, significantly enhancing offline user experience. It modifies `ui/src/core_plugins/dev.perfetto.ExtensionServers/extension_server.ts` to include a special header (`extensionHeaders`) in fetch requests, signaling the service worker for interception. The `ui/src/service_worker/service_worker.ts` module now contains robust logic, including `handleExtensionRequest` and new error types like `TimeoutError`, to manage caching and ensure reliable access to extension data even without a network connection. This **grow** feature improves the overall resilience and responsiveness of the UI's extension ecosystem.

This commit introduces a new `NullBitvector` struct within the **trace_processor** to unify the null bitvector and its prefix popcount cache into a single register, replacing separate register allocations. This **refactoring** significantly reduces register bloat and treats the bitvector and popcount as an atomic unit across all bytecode instructions, similar to `TreeState`. The change primarily impacts the **bytecode interpreter** and **dataframe query planning** subsystems, simplifying argument lists for functions like `TranslateSparseNullIndices` and `PrefixPopcount` and updating how `SparseNull` and `DenseNull` columns are managed. This **internal architectural improvement** streamlines null value processing, making the system more robust and easier to maintain by consolidating related data structures.