Primiano Tucci

This commit **improves the testing infrastructure** by modifying the `ReadTraceData` function within `test/test_helper.cc`. It updates the **trace data processing logic** to explicitly ignore trace packets that contain an `extension_descriptor`. This change ensures that tests correctly handle trace data by filtering out these specific packet types, which might otherwise cause unexpected behavior or failures in the test suite. Ultimately, this makes the **test suite more robust** and reliable when processing diverse trace data.

This commit performs **maintenance** by **updating the Perfetto `TrackEvent` extension configuration** to reserve a specific identifier range for Android vendor atoms. It modifies `protos/perfetto/trace/track_event/track_event_extensions.json` to define the new `android_vendor` range as 5000-8999, reflecting an upstream change. This ensures that **Android vendors have a dedicated and non-conflicting range for their custom `TrackEvent` atoms**, improving the organization and future extensibility of the tracing system. The change also adjusts the 'unallocated' range to accommodate this new reservation within the **`TrackEvent` subsystem**.

This commit **enhances the debugging capabilities** of the **`base` watchdog subsystem** by introducing a new crash key. When a time-based watchdog fires, its configured duration is now recorded in this crash key, providing vital context for diagnosing issues. This **observability improvement** specifically targets the POSIX watchdog implementation, helping to understand and resolve cases where relative deadlines might be too short. The change involves adding a `duration_ms` field to `Watchdog::TimerData` in `include/perfetto/ext/base/watchdog_posix.h` and implementing the recording logic in `src/base/watchdog_posix.cc`. This provides crucial data in crash reports to aid in diagnosing **watchdog-related problems**.

This commit **adds a new documentation file** `sdk/MOVED_TO_GITHUB_RELEASES_ARTIFACTS.md` to inform users about a significant change in the project's distribution strategy. This **documentation update** clarifies that the **amalgamated tracing SDK** has been moved to **GitHub Releases artifacts** for future distribution. The primary goal of this **maintenance task** is to proactively communicate this change, thereby reducing user surprise and potential confusion regarding the SDK's location in upcoming releases.

This commit **updates the prebuilt binaries** for several Perfetto tools and their Python library manifests to version v54.0. It refreshes the manifests for `trace_processor_shell`, `tracebox`, and `traceconv` within the **Perfetto Python library prebuilts** (`python/perfetto/prebuilts/manifests/`). Additionally, various **Perfetto command-line tools** such as `cpu_profile`, `heap_profile`, `record_android_trace`, `trace_processor`, `tracebox`, and `traceconv` now embed the updated v54.0 manifest data for their respective dependencies. This **maintenance update** ensures that users leveraging these Perfetto components will automatically fetch and utilize the **latest stable versions**, incorporating any recent bug fixes or enhancements from the v54.0 release.

This commit delivers **bug fixes** and **workflow improvements** to the project's build and release tooling. The `tools/gen_amalgamated` script is updated to explicitly use the **hermetic `clang++` compiler** from `buildtools` on Linux, preventing build failures that could occur if a system `clang` was not installed. Concurrently, the `tools/release/roll-prebuilts` script now correctly determines the `ROOT_DIR` and automatically triggers `gen_amalgamated_python_tools` and `format-sources` after prebuilts are rolled. These changes enhance **build reliability** and **hermeticity**, while also **streamlining the prebuilt rolling process** by automating previously manual or hook-dependent steps.

This commit **refactors** the **Perfetto tracing proto extension registry** to introduce a more flexible and future-proof JSON schema. The `protos/perfetto/trace/track_event/track_event_extensions.json` schema is updated to include an `'extensions'` root array, allowing multiple extension definitions for different trace messages like `TrackEvent` and `TracePacket` within a single file. Correspondingly, the `src/tools/tracing_proto_extensions.cc` parsing logic was rewritten, introducing `ParseRegistryFile` and adapting `CollectProtosFromRegistry` to correctly process this new structure. This **maintenance** change improves the extensibility of the tracing system by providing a robust mechanism for defining new proto extensions.

This **maintenance** commit **updates the Windows SDK paths** within the **LUCI build system** to align with recent CIPD package changes. It modifies the `windows_sdk` recipe module (`api.py`) to correctly load environment variables and test data from the new SDK location. Consequently, the **Perfetto Windows CI recipe** (`ci_win.json`) is also updated to reflect these new paths in its expected output. This ensures the **Windows CI builds continue to function correctly** by resolving issues caused by outdated SDK references.

This commit **updates the default Windows SDK version** utilized by the **LUCI `windows_sdk` recipe module** from `2019-09-06` to `2024-01-11`. This crucial **maintenance update** acts as a **bug fix**, addressing recent build failures observed in Perfetto's continuous integration on Windows, which were likely caused by an outdated SDK. By upgrading the SDK, the change aims to improve build stability and prevent compatibility issues for **Perfetto's CI Windows builds**. Additionally, the expected output for the `perfetto.expected/ci_win.json` recipe is updated to reflect this new SDK version.

This commit primarily **fixes Windows build failures** by addressing issues related to **protobuf compilation and linking**. It updates the **GN build system** by changing the `protobuf_lite` dependency from `deps` to `public_deps` in `gn/standalone/proto_library.gni`, ensuring correct transitive dependency visibility for projects using `proto_library`. Additionally, `buildtools/BUILD.gn` is modified to add specific clang compiler flags to suppress warnings and link `shell32.lib` for protobuf on the **Windows platform**. A minor **maintenance fix** also adds a missing `#include <array>` to `src/base/circular_queue_unittest.cc` to ensure the unit tests compile correctly.

This commit performs **maintenance** by updating the project's **CHANGELOG** file. It specifically adds and organizes the release notes for the upcoming **version 54**, documenting all significant changes, new features, and bug fixes included in this release. This update ensures that users and developers have a clear, chronological record of modifications, aiding in understanding the evolution and new capabilities of the software.

This commit introduces a **new capability** to the **Perfetto tracing system**, enabling the loading and processing of **out-of-tree protobuf extension descriptors**. It specifically allows the `traced` service on **Android** to mmap and load gzipped descriptor sets from `/etc` and `/vendor/etc` paths, which are then serialized and emitted into trace packets. The **trace format** is updated to accommodate these gzipped descriptors, and the **Trace Processor** is enhanced to decompress and parse them from incoming traces. This feature significantly improves the extensibility of Perfetto, allowing custom protobuf messages to be used in traces without requiring Perfetto recompilation.

This commit **introduces a new capability** to the **Android build system** by exposing a binary tool designed to handle out-of-tree protobuf definitions. It adds the `tracing_proto_extensions` host binary target and the `etc_system_tracing_proto_descriptors` phony target to `Android.bp` and `Android.bp.extras`. The `tools/gen_android_bp` script is updated to include this new tool in default builds and to add a new dependency for the `traced` target. This **new feature** facilitates the use of custom, external tracing protobufs within the Android environment, significantly enhancing the **extensibility of the tracing subsystem**.

This commit **improves the granularity and accuracy of trace buffer statistics** within the **tracing service's `TraceBufferV1WithV2Shadow` component**. It **refactors** the packet hash tracking and statistics calculation logic to maintain separate counters for V1 and V2 packets, addressing a previous issue where statistics were losing too much information. A new `stats_version` field is added to the `ShadowBufferStats` protobuf, allowing for future compatibility and indicating the improved calculation method. This **enhancement** provides **more detailed and reliable observability** for trace buffer activity, accepting a trade-off of slightly increased memory usage for the statistics.

This commit introduces a **new `tracing_proto_extensions` tool** to manage and validate **out-of-tree protobuf extensions** within the **Perfetto tracing system**. It **refactors** the core `perfetto_trace.proto` and `track_event.proto` definitions by consolidating `TrackEvent` extension ranges and updates comments to reference a new `track_event_extensions.json` file, which serves as a central allocation registry. The new tool parses this JSON registry, validates extension allocations, and generates `FileDescriptorSets`, providing a robust mechanism for **extensibility** and **preventing conflicts** in custom trace event definitions. This **new capability** significantly enhances the **maintainability** and **scalability** of Perfetto's tracing capabilities by formalizing the extension process.

This commit **refactors** the project by **removing the external `jsoncpp` dependency**, standardizing JSON parsing to the project's **internal `json_value` utility**. Specifically, the **`bigtrace/worker/repository_policies` module** now uses `json_value` for parsing GCS access tokens within `gcs_trace_processor_loader.cc`, eliminating `jsoncpp` from core logic. Additionally, several **`trace_processor` unit tests**, such as `export_json_unittest.cc` and `storage_minimal_smoke_test.cc`, have been updated to utilize the internal utility for JSON output validation. This change streamlines the build process by updating `BUILD.gn` files and reduces the overall external dependency footprint of the project.

This commit performs a significant **refactoring** by **extracting the `MultiFileErrorCollectorImpl` class** into its own dedicated header (`src/protozero/multifile_error_collector.h`) and implementation (`src/protozero/multifile_error_collector.cc`) files. This centralizes the logic for recording errors and warnings across multiple files, which was previously duplicated in various locations. The change primarily affects the **`protozero` library** and its associated tools like `filter_util` and `proto_merger`, which now include the new standalone component. This improves code reusability and maintainability by consolidating a common utility, preparing for its increased usage. Build configurations across **Android.bp, BUILD, and `BUILD.gn` files** have been updated to reflect this new module.

This commit performs a **critical maintenance update** to the **CI infrastructure** by upgrading the **GitHub Actions self-hosted runner** version within the `infra/ci/sandbox/Dockerfile` to `2.331.0`. This update directly addresses an issue where GitHub was incorrectly identifying auto-updating runners as outdated, ensuring the **CI system** remains compliant and recognized. The change primarily affects the **GitHub Actions self-hosted runner sandbox environment**, preventing potential disruptions related to perceived outdated runner versions. Additionally, the `infra/ci/Makefile` is updated with a helpful authentication message for the `push` target, improving the **developer experience** for CI image management.

This commit **increases the Cloud Build timeout** for the **`ui.perfetto.dev` infrastructure** from 30 minutes to 60 minutes. This **maintenance update** addresses potential build failures due to prolonged execution times, ensuring more robust and reliable UI deployments. Specifically, it modifies the `cloudbuild.yaml` and `cloudbuild_release.yaml` configurations. Additionally, it **updates comments** within these files to reflect the current GitHub-based trigger configuration, removing obsolete instructions. This change improves the stability of the Perfetto UI build process and clarifies its configuration.

This commit introduces a **build performance optimization** by significantly reducing the inclusion of the heavy `<cmath>` header. It **refactors** the frequently included `perfetto/ext/base/murmur_hash.h` to avoid `<cmath>` by utilizing compiler built-ins like `__builtin_isnan` through a new `PERFETTO_HAS_BUILTIN` macro in `compiler.h`. To further isolate `<cmath>` usage, the `BytesToGB` function's implementation is moved from `machine_tracker.h` to `machine_tracker.cc`, where `<cmath>` is now exclusively included. This change primarily affects the **Trace Processor** and its base utilities, resulting in a noticeable **speed-up in compilation times**, particularly for targets like `trace_processor_shell`.