Rishi Arora

This commit **enhances the Python packaging build system** by ensuring that **runtime artifacts are correctly materialized at build time when operating in 'copy' mode**. It modifies the `_make_py_package_live` function within `prelude/python/make_py_package.bzl` to explicitly include `hidden = <manifest>.artifacts` for outplace builds. This **resolves issues where Python packages built in 'copy' mode might be missing necessary files**, as this mode requires actual file presence for copying, unlike 'inplace' mode which uses symlinks. The change improves the reliability and correctness of **Python package builds** when using the 'copy' strategy.

This commit introduces a **new capability** to the **Python packaging system** by adding a `copy` parameter to the `_make_py_package_live` function within `prelude/python/make_py_package.bzl`. This enhancement allows for the generation of **outplace Python Archive (PAR) files**, meaning the package contents are fully copied rather than symlinked. The modification to the execution logic based on this parameter provides greater flexibility, enabling `rust_make_par` to produce self-contained PARs that do not rely on symlinks, which can be crucial for certain deployment scenarios.

This commit performs **code cleanup** and **refactoring** by removing the deprecated non-incremental `rust_make_par` action from the **Python package build logic**. Specifically, it targets `prelude/python/make_py_package.bzl` and the `_make_py_package_live` function, eliminating conditional logic that previously allowed for a non-incremental build path. This change standardizes the build process to exclusively use the **incremental Rust build**, which is now fully rolled out and the default. The removal of this **dead code** simplifies the codebase without impacting functionality, as all Rust builds within Python packages already leverage the incremental approach.

This commit **enhances the `prelude` module's `_live_par_generated_files` function** by introducing a new `linktree_suffix` parameter. This parameter, defaulting to `#link-tree`, is now **forwarded as `--linktree-suffix` to the `gen_lpar_bootstrap` tool** when generating `_bootstrap.sh` and `.par` scripts. The **feature enhancement** allows the bootstrap generation process to use a **customizable link tree suffix**, providing greater flexibility in the generated package structure. This **maintenance improvement** ensures that the `gen_lpar_bootstrap` process can be more precisely controlled from the `prelude` configuration, impacting how `.par` scripts are built.

This commit introduces a **new feature** to the **`gen_lpar_bootstrap.py` tool** by adding the `--linktree-suffix` command-line argument. This argument allows users to configure the name of the sibling directory used for the link tree, which was previously hardcoded as `#link-tree` within `.par` bash scripts. The change provides crucial flexibility, enabling support for different operational modes such as "outplace (copy mode)" which requires a `#copy-tree` directory. The default value remains `#link-tree` to ensure backward compatibility for existing `.par` package generation.

This commit **enables the `rust_make_par` macro for all operating systems** within the `fbcode` build environment, removing a previous restriction that limited its use to Linux. Specifically, it **removes the Windows OS limitation** for `rust_make_par` when the `package_style` is set to `inplace` for Python packages. By modifying the `_make_py_package_wrapper` function in `prelude/python/make_py_package.bzl`, this **feature enablement** broadens the applicability of Rust-based Python extensions. This change ensures that `rust_make_par` can be consistently utilized across various platforms, improving the build experience for Python projects integrating Rust components.

This commit **enables `rust_make_par`** for **macOS** within the **Python packaging system**. It **modifies the condition** for using `_make_py_package_live` in `prelude/python/make_py_package.bzl`, specifically excluding only Windows from this optimization. This **enhancement** allows `rust_make_par` to be utilized on macOS, improving the build process and **performance** for Python packages on that platform. The change primarily affects the `_make_py_package_wrapper` symbol and the overall build system's platform compatibility.

This commit **introduces Windows-specific bootstrapping logic** within the **Python package build system**. It adds the `_live_par_windows_bootstrap` function and integrates it into `_make_py_package_live` in `prelude/python/make_py_package.bzl`. This **preparatory change** is a **no-op** for now, laying the essential groundwork for future **Windows support for live Python packages**. It addresses previous symlink issues encountered on Windows, paving the way for full cross-platform compatibility for these packages.

This commit introduces a **bug fix** to the **Python packaging system**, specifically addressing an issue with cache interactions on Windows. It prevents cache uploads for local Windows inplace actions by modifying the `_make_py_package_live` rule within `prelude/python/make_py_package.bzl`. The change conditionally sets `allow_cache_upload` to false, ensuring that local builds in **Windows development environments** do not attempt to upload to the cache. This **maintenance** improves the efficiency and correctness of the build process by avoiding unnecessary or problematic cache operations for inplace builds.

This commit **expands the `rust_make_par` build system's compatibility** to **Windows and macOS** within the `fbcode` environment. It introduces **platform-specific Python package bootstrapping** by adding a new function, `_live_par_windows_bootstrap`, to `prelude/python/make_py_package.bzl`. Existing Python package creation functions, such as `_make_py_package_wrapper` and `_make_py_package_live`, are updated to conditionally invoke this new Windows-specific logic. This **new capability** enables developers to utilize `rust_make_par` for projects targeting these operating systems, significantly broadening its applicability.

This commit introduces a **performance optimization** for the **`py_dbg_source_db` action**, which is crucial for incremental Python builds. It **refactors** the mechanism for passing dependency manifests, moving from thousands of individual `--dependency` arguments to a single file containing all manifest paths. This change, primarily affecting `prelude/python/source_db.bzl` and the `make_source_db.py` script, **significantly reduces command-line parsing overhead**. Consequently, it **improves build times** when Python files are added or removed, enhancing the overall efficiency of the build system.

This commit introduces a **hybrid execution strategy** for **incremental build actions** within the **build system**, specifically addressing how `state.json` is handled. Previously, actions that only output `state.json` were always treated as non-incremental on **remote execution (RE)**, leading to slower builds. The change **optimizes build performance** by allowing these incremental actions to run locally where it's significantly faster, while still leveraging RE for efficient caching of cold builds. This **performance improvement** ensures quicker feedback loops during iterative development by intelligently choosing the most efficient execution environment.

This commit introduces a **performance optimization** by **refactoring** the `compute_mappings` function within `prelude/cxx/groups.bzl`. The **target-to-group mapping logic** now performs a **single graph iteration** for non-root link group mappings, rather than iterating the entire graph for each mapping. This change significantly **reduces analysis time for large binaries** by streamlining the group assignment process within the **C++ build system**. The `_update_target_to_group_mapping` function is utilized to ensure target ordering is preserved.

This commit **refactors** the `_create_precompile_cmd` function within the **C++ compilation prelude** (`prelude/cxx/compile.bzl`) to significantly **improve performance**. It addresses an inefficiency where the **regex for header grouping** was being redundantly re-compiled inside a loop on every iteration. By moving this regex compilation outside the loop, the change **optimizes the C++ build process** by eliminating unnecessary overhead. This is a **performance optimization** targeting the build system's C++ tooling, leading to more efficient compilation.

This commit **fixes a caching bug** affecting the `rust_make_par_incremental` action within the **Python package build process**. Previously, a generated state file was not being downloaded during cache hits on Sandcastle, leading to incomplete or incorrect cached outputs. The change modifies the `_make_py_package_live` function in `prelude/python/make_py_package.bzl` to **include the state file in `runtime_files`**, ensuring it is properly retrieved when a build action is served from cache. This **improves the reliability of cached builds** by guaranteeing all necessary artifacts are present.

This commit introduces **support for resource directories** within the **Python wheel packaging tool**. Previously, specifying a directory as a resource would lead to a failure; now, the `write` method in `prelude/python/tools/wheel.py` has been **enhanced to recursively add all files within such directories** to the generated wheel archive. This **new capability** prevents build failures and simplifies resource management for Python packages, allowing users to define entire directories as resources.

This commit **resolves a build state corruption issue** affecting the **Python build rules** (`make_py_package`). It **fixes** a problem where switching between **incremental and standard `make_par` builds** for a target could leave the **link-tree** in an inconsistent state. The change modifies `prelude/python/make_py_package.bzl` to ensure a state file is always generated, even an empty one for standard builds, which then correctly triggers incremental builds to delete the old link-tree. This **improves the reliability and correctness** of the build system by preventing inconsistent build artifacts when developers switch between build strategies.

This commit implements a **performance optimization** within the **Python build system** by refactoring how module manifests are handled. Previously, the `make_py_package_manifest_module.py` script received thousands of individual `--module-manifest` flags, leading to slow argument parsing. Now, the `_generate_manifest_module` rule in `prelude/python/make_py_package.bzl` passes a single file containing all manifest paths, which the script's `parse_args` and `main` functions are updated to process. This **refactoring** significantly **reduces execution time** for large targets, improving build performance from 14 seconds to approximately 1-2 seconds.

This commit introduces a **performance optimization** to the **Python package manifest generation tool**, specifically within `prelude/python/tools/make_py_package_manifest_module.py`. It refactors the `main` function's logic to **skip processing parent directories that have already been added** to the module dictionary from previous package paths. This change prevents redundant work during manifest creation, leading to a notable **reduction in execution time** for large targets, cutting processing time from approximately 4 seconds to 1-2 seconds. This **refactoring** improves the efficiency of the build system's manifest generation step.

This commit introduces a **new capability** by adding the `use_rust_make_par_incremental` attribute to the `_python_executable_attrs` function within the **Python rules definition** in `prelude/decls/python_rules.bzl`. This attribute serves as a **rollout flag** to enable the development and eventual adoption of **incremental `rust_make_par`** for Python executables. By providing this configuration option, the change facilitates future improvements in build performance for Python projects by allowing them to opt into incremental build actions. This is a foundational step towards a more efficient build system.