Jiawei Lü

This commit **optimizes the Android build system** by **enabling cache upload** for various `aapt2` actions. It configures `aapt2 link`, `aapt2 compile`, resource filtering, and unzip operations to upload their generated artifacts to the build cache. This **maintenance enhancement** significantly improves build efficiency and **reduces build times** for Android projects by allowing subsequent builds to retrieve these outputs from the cache instead of re-executing the actions. The changes are applied within the `prelude/android` build rules, specifically impacting the `get_aapt2_link` and `aapt2_compile` functions.

This commit introduces a **new capability** to the **Android build toolchain** by implementing **64K-limit-aware secondary DEX splitting**. It modifies the `D8ExecutableMain.java` to correctly sum method, field, and type reference counts from `.dex.jar` files, which are then utilized by the Python `sort_pre_dexed_files.py` script and integrated via `dex_rules.bzl` to intelligently split DEX files. This enhancement ensures that secondary DEX files adhere to the 64K method limit, improving the robustness and compliance of Android application builds by preventing runtime issues related to DEX size constraints.

This commit **reverts** the functionality introduced by D96347117, effectively removing the **DEX ref count-based splitting mechanism** for secondary DEX files. It **refactors** the **Android DEX splitting logic** by eliminating the Python implementation in `sort_pre_dexed_files.py` and the associated Buck build rules in `dex_rules.bzl` that used method, field, and type reference counts. Specifically, the `_sort_pre_dexed_files`, `assign_pre_dexed_classes_to_secondary_dex`, and `_organize_lib` functions are modified to remove this logic. Additionally, the `readDexRefCountsFromJar` method and related processing are removed from `D8ExecutableMain.java`, ensuring the build system no longer relies on this specific strategy for 64K-limit-aware DEX splitting. This **revert** impacts how pre-dexed classes are assigned to secondary DEX files, reverting to the previous splitting methodology.

This commit introduces a **new capability** to enhance **Android multi-DEX splitting** by implementing **64K method limit enforcement** using accurate reference counts. It modifies the **DEX reference count calculation** in `D8ExecutableMain.java` to sum counts from all DEX files within a JAR, ensuring comprehensive tracking of method, field, and type references. This refined counting logic is then integrated into the `_sort_pre_dexed_files` process within `prelude/android/dex_rules.bzl` and `prelude/android/tools/sort_pre_dexed_files.py` to intelligently assign and organize pre-dexed files. This improvement helps prevent 64K limit issues in large Android applications by making the secondary DEX splitting process more robust and limit-aware.

This commit **introduces a new capability** to the **D8 compilation process** by allowing the extraction of exact method, field, and type reference counts directly from the DEX file header. A new `--ref-count-path` flag in `D8ExecutableMain` writes these critical 64K DEX limit values to a specified file. This enhancement provides precise metrics for **64K-limit-aware secondary DEX splitting**, improving upon previous estimation methods. It correctly handles both `.dex.jar` and bare `.dex` outputs, including empty JARs, and is supported by new unit tests for the `DexRefCounts` utility.

This commit **wires DEX reference counts** through the **pre-dex pipeline**, establishing the infrastructure to track and expose these metrics. It **adds a `ref_count` field** to the `DexLibraryInfo` provider in `dex.bzl`, enabling the generation of `_ref_count.txt` artifacts and passing `--ref-count-path` to D8. The `filter_dex.py` tool is **enhanced to accept and parse these ref count files**, subsequently including `method_ref_count`, `field_ref_count`, and `type_ref_count` in its JSON output. This **new capability** in the Android build system makes reference count data available in the `filter_dex` output, laying the groundwork for future optimizations like improved DEX splitting decisions.

This commit **integrates the `sort_pre_dexed_files` Python tool into the Android dex merge pipeline** as a separate, cacheable build action. This **performance optimization** allows the `merge_pre_dexed_libs` lambda to consume a pre-computed dex plan, significantly reducing `DeferredEvaluation` time during the dex merging process. The change primarily affects the **Android build system's dexing rules** (`dex_rules.bzl`) and **toolchain definitions**, providing a fast path while retaining a Starlark fallback. Gated by the `[android].use_sort_pre_dexed_files_tool` buckconfig flag, this enhancement delivers a substantial **reduction in build times** for Android applications.

This commit introduces a **new Python tool**, `sort_pre_dexed_files.py`, designed to **optimize the Android build system's dexing process**. This **performance optimization** addresses a critical bottleneck in the `merge_pre_dexed_libs` dynamic lambda by replacing a slow, single-threaded Starlark implementation of `_sort_pre_dexed_files`. The new tool efficiently reads `filter_dex` outputs, library metadata, and the module graph to produce a `dex_plan.json`, significantly reducing the time required to sort pre-dexed libraries into dex groups. This change drastically cuts down critical path build times for Android applications like `ig4a`, and is thoroughly validated by **comprehensive unit and integration tests** for the new `prelude/android/tools` component.

This commit introduces a significant **performance optimization** for **Android resource rewriting** within the `ExoResourcesRewriter`. It enables a raw pass-through mechanism where unmodified zip entries (everything except `resources.arsc` and compiled XMLs) are copied directly from the input to the output zip without costly decompression and re-compression. This required **enhancements to the core zip infrastructure** (`CustomZipEntry`, `EntryAccounting`, `DeterministicZipBuilder`, `ResourcesZipBuilder`) to support efficient raw byte copying. The change drastically **reduces processing time and CPU usage** for large APKs by avoiding unnecessary work on the majority of entries, improving overall build efficiency when `ResourceProcessingConfig.isOptimizationsEnabled()` is true.

This commit introduces a crucial **memory optimization** within the **`ExoResourcesRewriter.ApkZip`** component, addressing an unbounded cache that previously stored the full decompressed bytes of every zip entry. For large APKs, this change significantly **reduces memory consumption** by replacing the `entryContents` cache with on-demand access when optimizations are enabled. Transformed entries are now serialized from in-memory objects, while pass-through entries are read directly from the ZipFile, preventing the entire decompressed APK from residing in memory simultaneously. This **performance enhancement** improves resource processing efficiency, particularly for Android builds, without any functional impact on the generated artifacts.

This commit **optimizes resource processing** within the **`ExoResourcesRewriter`** by **lowering the compression level** for recompressed entries, such as `resources.arsc` and modified XMLs. When optimizations are enabled, it now uses compression level 6 instead of `Deflater.BEST_COMPRESSION` (level 9) to improve performance. This **refactoring** introduces a new `getCompressionLevel()` method to dynamically determine the compression level, resulting in 2-3x faster recompression with a negligible increase in file size. The change aims to **speed up Android application build times** by making resource rewriting more efficient.

This commit implements a **bug fix** within the **ZIP file generation** utility `EntryAccounting.java`. It addresses a latent issue where the `compressed size` field in the **ZIP local file header** and **ZIP64 extra field** was erroneously written using the uncompressed size (`entry.getSize()`) instead of the actual compressed size (`entry.getCompressedSize()`). This correction ensures the **integrity of generated ZIP archives**, preventing potential corruption or incorrect size reporting. The fix is crucial for upcoming `RAW_COPY` entry types that will utilize non-data-descriptor DEFLATED entries, where compressed and uncompressed sizes will significantly differ. Specifically, the `writeLocalFileHeader` and `writeZip64ExtraField` methods were updated to reflect this change.

This commit introduces a **performance optimization** by **skipping the ZipAlign step for Android exo resources** when build optimizations are enabled. Previously, these resources were temporarily written and then processed by `zipalign -f 4`, but this is now bypassed, allowing direct writing to the final output path. This change in the **resource processing pipeline** improves build times, as ZipAlign is unnecessary for exo resources loaded via `AssetManager.addAssetPath()` and often a no-op for their DEFLATED format. The modification is implemented within `ExoResourcesRewriterExecutableMain.java` and is conditional on `ResourceProcessingConfig.isOptimizationsEnabled()`.

This commit introduces a **new capability** for **performance optimization** within the **Android resource processing** pipeline by establishing a gated optimization infrastructure. It adds a `ResourceProcessingConfig` class and a `--optimized-processing` CLI flag, integrated into build rules like `android_binary_resources_rules.bzl`, all controlled by the `android.optimized_resource_processing` feature flag. Two initial optimizations are included: **disabling redundant debug assertions** in `ResourcesXml.transformReferences()` and **eliminating an inefficient `TreeSet.contains()` lookup** in `UsedResourcesFinder`. These changes aim to **reduce overhead and improve build times** for Android applications when the feature flag is enabled, without affecting current behavior as it defaults to off.

This commit introduces a **performance optimization** and **refactoring** within the **Android resource processing** module, specifically affecting `ResTableType` and `ResourcesXml`. It replaces Stream-based attribute sorting in methods like `sortAttributesAt()` and `fixupAttrs()` with a more efficient `Arrays.sort()` approach when optimizations are enabled. This change reduces object allocation and Stream overhead during resource compilation, while also including a **bug fix** to prevent potential integer overflow in `AttrRef.compareTo()` by using `Integer.compare()`. The modification improves resource processing efficiency without altering the final output, ensuring bit-identical results.

This commit **refactors** the **Android resource parsing logic** within the `ResTableType` class by replacing hardcoded byte offsets with named constants. This **maintenance** effort significantly improves the readability and maintainability of methods like `getEntrySizeAtOffset` and `transformEntryDataOffsets`, which are crucial for traversing and manipulating binary resource table entries. By explicitly documenting the resource table entry format through these constants, the code becomes easier to audit and modify, ensuring more robust handling of Android resources without altering its functional behavior.

This commit introduces a **performance optimization** to the **Android resource processing** subsystem, specifically within the `MergeAssetsUtils.addAllAssets()` method. It **eliminates a double file read** for asset files by, when optimizations are enabled, reading each file once into memory to compute its CRC32 hash and then streaming from that in-memory buffer. This **refactoring** improves efficiency by reducing redundant I/O operations during asset merging. The change is gated by `ResourceProcessingConfig.isOptimizationsEnabled()` and ensures **bit-identical output** compared to the previous approach, primarily impacting the speed of asset compilation.

This commit introduces a **performance optimization** within the **Android resource processing** module, specifically in `ResTablePackage`. It **refactors** the `getTypeSpec()` method to replace an O(n) linear scan with an O(1) array lookup for retrieving type specifications. This change significantly speeds up resource lookup operations, particularly benefiting `UsedResourcesFinder.iterateArsc()` and `computePrimaryApkClosure()`, by using a pre-built array when optimizations are enabled. The modification ensures bit-identical output, focusing solely on improving read-path efficiency without altering functionality.

This commit introduces dedicated **visit-only traversal methods** in **`ResTableType`** and **`ResourcesXml`** to improve performance within the Android resource processing tools. Previously, read-only `visitXxx()` methods unnecessarily invoked `transformXxx()` with an identity lambda, incurring overhead from write-back path logic. By adding pure read methods like `visitEntryDataOffset()`, this **refactoring** significantly **reduces processing overhead** for operations like closure computation in `UsedResourcesFinder` when optimizations are enabled. This **performance optimization** ensures bit-identical output while providing faster resource traversal.

This commit performs a significant **maintenance** and **refactoring** effort by **removing all references to the `on_unused_dependencies` build attribute** across the entire codebase, as this attribute is no longer functional. The removal impacts various **build system components**, including macro wrappers for `aloha`, `portal-sdk`, and other projects, the `do_unused_deps_detection` function, a hardcoded override in `fb_core_android_library`, and numerous Buck2 prelude rule definitions for Java, Android, Groovy, Scala, and Kotlin. This cleanup simplifies the build configuration by eliminating a defunct attribute, ensuring greater consistency and reducing potential confusion. Notably, the previous ABI generation mode for `aloha` and `portal-sdk` targets is now explicitly preserved through `abi_generation_mode = "class"`.