Chris Hopman

This commit **adds comprehensive documentation** for **Buck2's link groups system**, introducing a new file `link_groups_explained.md`. This detailed guide provides a top-to-bottom explanation of the entire linking pipeline, from parsing `link_group_map` through final shared library linking, including all traversal types and special labels like `MATCH_ALL`, `NO_MATCH`, and `MATCH_DIRECT_DEPS`. This **documentation improvement** clarifies complex aspects such as preferred linkage, force-static handling, and symlink tree assembly, significantly enhancing developer understanding of this critical **linking infrastructure**. This **maintenance** effort aims to improve the clarity and maintainability of the **Buck2 build system** for users and contributors working with linking configurations.

This commit introduces a **new capability** to the **critical path visualization** by making **anonymous target analysis visible**, which was previously hidden within parent analysis durations. It achieves this by splitting analysis nodes that depend on anon targets into two parts (before and after resolution) and introducing dedicated `AnonTargetKey` nodes to represent the anon target's own execution time. This enhancement to `buck2_analysis` and `buck2_build_signals_impl` provides a more accurate and detailed view of build dependencies, showing a clear flow like `analysis[part1] → anon_analysis → analysis[part2]`. The change also supports nested anonymous targets and required updates to `buck2_data` protos and ingress processing for critical path data.

This commit introduces a **soft error** within the **critical path proto enhancer** in the `buck2_build_signals_impl` module. Specifically, the `add_entry` function now detects and reports instances where consecutive critical path nodes have **overlapping time spans**. This **bug fix** prevents incorrect critical path calculations and double counting of time, which previously occurred when a node claimed to start before its predecessor finished. The new error logging, which includes details of both overlapping nodes, significantly aids in **debugging** and ensures the accuracy of performance metrics.

This commit **fixes a bug** in **Buck2's materialization logic** concerning **critical path dependency tracking** for **tset projections**. Previously, when an artifact from a tset was materialized, the system incorrectly only recognized a dependency on the action producing that specific artifact, rather than the entire `ArtifactGroup` (tset) it originated from. The change propagates the top-level `ArtifactGroup` being materialized through the `ArtifactMaterializer` and `BuildSignals` components, specifically in `app/buck2_build_api/src/actions/artifact/materializer.rs` and `app/buck2_build_signals_impl/src/lib.rs`. This ensures that the full tset is correctly recognized as a dependency during **final materialization**, improving the accuracy of **dependency resolution** and preventing potential build failures.

This commit introduces a **new capability** by adding the `EnsureTransitiveSetProjection` protobuf message to the **Buck2 data model** (`buck2_data/data.proto`). This new proto is integrated into `CriticalPathEntry2`, enabling the capture of specific data related to "transitive sets" for enhanced build analysis. The change updates conversion logic within `buck2_build_signals_impl` and adds display support in `buck2_event_observer` for this new critical path entry type. This provides more granular insights into **critical path analysis**, allowing for better understanding and optimization of build performance related to transitive sets.

This commit **enhances critical path analysis** by accurately measuring and recording the execution duration of `EnsureTransitiveSetProjectionKey` evaluations within the **DICE evaluation system**. Previously, these nodes showed zero duration, but now they capture the `TimeSpan` for post-dependency synchronization work, specifically the construction of `ArtifactGroupValues`. This **new capability** involves introducing `EnsureTransitiveSetProjectionKeyActivationData` in `buck2_build_api` and integrating its tracking into `BuildSignalSender` in `buck2_build_signals_impl`. The change provides more precise **build performance insights** by ensuring proper duration data is available in the critical path graph for these crucial build steps.

This commit introduces a **new capability**, `ResultMaybeCompatible<T>`, a three-variant enum designed to flatten `Result<MaybeCompatible<T>>` into `Compatible(T) | Incompatible(...) | Err(...)`, simplifying error and incompatibility propagation. By implementing the `Try` trait, it allows the `?` operator to handle both errors and incompatibilities, significantly reducing boilerplate in **target configuration and analysis** code. This change involves a substantial **refactoring** effort, migrating numerous functions like `get_configured_target_node` across core modules such as `buck2_analysis`, `buck2_build_api`, and `buck2_configured` to use this new type. The overall impact is improved **developer experience** and cleaner error handling for compatibility checks throughout Buck2.

This commit **refactors** the **attribute configuration API** by changing `CoercedAttr::configure` and `CoercedAttrFull::configure` to return `ResultMaybeCompatible<ConfiguredAttr>`. This modification is **preparation for the `select_incompatible()` feature**, enabling the build system to propagate target incompatibility as a distinct state rather than an error during configuration. It impacts **configured node computation** and **transition resolution** within `buck2_configured`, allowing for more nuanced handling of incompatible configurations. All existing call sites are updated to either explicitly `.require_compatible()` where incompatibility should be an error, or `.expect_compatible()` in contexts where it's not anticipated. This fundamentally alters how **incompatible configurations** are managed throughout the build graph resolution process.

This commit introduces the **new Starlark function `select_incompatible()`** to the **Buck2 build system**, providing a mechanism for **conditional target incompatibility** within `select` statements. Unlike `select_fail()`, this function causes targets to be **gracefully skipped** when a specific configuration branch is selected, mirroring the behavior of `target_compatible_with`. This **new capability** enhances flexibility in attribute definitions, allowing users to express that a target is incompatible with certain configurations directly inline, and this incompatibility correctly propagates through the dependency graph. The change involves updates across the **Starlark interpreter**, **attribute configuration**, and **incompatibility handling** subsystems, including new `CoercedAttr` variants and updated JSON reporting.

This commit introduces a **new capability** for the **materializer fast rollout** feature by upgrading its configuration mechanism within the **Buck2 server context**. It replaces the simple boolean `materializer_fast_rollout` buckconfig key with a new `materializer_fast_rollout_v2` key, which now parses as a `RolloutPercentage`. This enhancement enables **per-build randomized rollout**, allowing for more granular control over feature activation, such as enabling it for a specific percentage of builds (e.g., `daemon:0.5`). Additionally, **logging tags** are updated to `rollout_enabled` and `rollout_disabled` to clearly differentiate the new rollout states from the old `enabled`/`disabled` tags.

This commit **enhances the visualization of critical and slowest paths** within **Buck2's Chrome trace output** by introducing a **hierarchical rendering structure**. Previously flat spans are now organized such that non-waiting entries, such as analysis or action, become parent spans that encompass their preceding waiting periods and execution times as nested child spans. This **feature enhancement** in the `buck2_cmd_debug_client` module, specifically affecting functions like `write_critical_path` and `write_critical_path_hierarchical`, significantly **improves the clarity and interpretability of performance traces**. It makes it much easier for users to identify and understand the impact of waiting times on specific operations by clearly associating them with their respective actions.

This commit introduces **new utility functions** to the **`AbsPath`** and **`AbsPathBuf`** types within the **`buck2_fs`** module. Specifically, it adds `with_extension`, `with_added_extension`, and `add_extension` to enable robust manipulation of file extensions. This **feature enhancement** addresses a common pain point where dereferencing to `Path` could inadvertently cause the loss of the absolute path property. By providing these dedicated methods, the commit ensures that absolute path integrity is maintained during common path operations, significantly improving the usability and reliability of path handling.

This commit introduces a **new feature** to the **Buck2 server's profiling subsystem**, enabling the **merging of multiple Starlark profiles** collected using the `--profile-patterns` option. Specifically, the `app/buck2_server/src/profile_patterns.rs` module now includes logic, via functions like `finalize` and `handle_profile_collected`, to combine individual profile data into a single, consolidated `merged.profile` file. This enhancement provides a unified view of Starlark performance across various patterns, significantly simplifying performance analysis and debugging. The change is validated by updates to golden test files, reflecting the new `merged.profile` and merged flamegraph outputs.

This commit introduces a **new capability** to the **`buck2_server`** application, enabling the automatic rendering of **flamegraphs as SVG files** for any profile mode that generates flamegraph data, specifically when using **profile patterns**. It leverages `inferno` to create these visual representations, enhancing the analysis of profiling data. This **feature addition** improves the user experience by providing immediate, rendered flamegraphs, making performance analysis more accessible and intuitive. The change primarily affects the **profiling system** and is supported by new test cases and a golden file for verification.

This commit introduces a **refactoring** within the **Starlark runtime profiler** by changing the `gen_flame_data` function to return an `Option<String>`. This modification explicitly signals whether flame graph data is available, removing reliance on hardcoded assumptions about which profile variants produce output. Consequently, the **Buck2 profile application**'s `write_starlark_profile` function is updated to correctly handle this new `Option` return type. This improves the robustness and clarity of the **flame graph generation logic** by making the absence of data explicit, accompanied by necessary **test updates** across the profiling module.

This commit introduces a **new integration test**, `test_profile_patterns`, to validate the functionality of **Buck2's statement-level profiling** via the `--profile-patterns` option. The test builds targets with specific profiling patterns enabled and asserts that the expected profile output files are correctly generated and contain data. This enhances the robustness of the **profiling system** by ensuring that pattern-based profiling produces valid and non-empty results, improving confidence in the accuracy of performance analysis.

This commit **refactors** the **`buck2_profile`** module by extracting the flamegraph writing logic into a dedicated, reusable function. Specifically, the functionality for generating and writing flamegraphs, previously embedded within `write_starlark_profile`, has been moved into a new function named `write_starlark_flamegraph`. This **refactoring** improves **code reusability** and modularity within the **profiling subsystem**, making the flamegraph writing capability easily invokable from other parts of the application. The original `write_starlark_profile` function is updated to call this new helper, ensuring consistent behavior while reducing code duplication.

This commit **introduces a new capability** to the `pagable` crate by defining the `PagableDeserializeOwned` trait within `pagable/src/traits.rs`. This trait is analogous to `serde::DeserializeOwned` and is **automatically implemented** for any type that implements `PagableDeserialize<'de>` for all lifetimes. Its primary purpose is to **simplify the expression of ownership requirements** in generic code that performs pagable deserialization, thereby improving the ergonomics and flexibility of the **pagable deserialization system**. The new trait is also exported and documented in `pagable/src/lib.rs`.

This commit **enhances the `pagable` crate** by **exposing key data types** and **enabling memory tracking**. Specifically, it makes `DataKey` and `OptionalDataKey` from the `storage::data` module **publicly accessible** via `pagable/src/lib.rs`, allowing external consumers to interact with these fundamental identifiers. Concurrently, the `Allocative` trait is **implemented for both `DataKey` and `OptionalDataKey`** in `pagable/src/storage/data.rs`, providing crucial support for **memory profiling and analysis** of these data structures within the system. This **new feature** improves the introspection and usability of the `pagable` crate's core components.

This commit introduces a new **`PagablePanic` derive macro** within the `pagable_derive` crate, which automatically implements the `PagableSerialize` and `PagableDeserialize` traits with `unimplemented!()` bodies. This **new capability** serves as a **refactoring aid** to facilitate the incremental addition of required trait bounds across the codebase. By applying this macro, developers can satisfy new trait requirements without immediate full implementations, enabling a **staged migration** to stricter type enforcement within the **`pagable` system**. The macro's implementation is in `pagable_derive/src/derive_pagable_panic.rs`, and its functionality is verified by new tests in `pagable/src/test/derive.rs`.