Tom Davies

This commit significantly **improves the performance** of the **test_runner** by optimizing the test grouping logic from O(n²) to **O(n)** in the `get_requested_tests` function. It also **fixes a bug** in the `getCtHook` function, ensuring all `ct_hooks` entries are correctly collected by using `lists:partition/2` instead of `lists:splitwith/2`. This **performance optimization** and **bug fix** enhance the efficiency of test execution setup within the `test_runner.erl` module. The change ensures faster test preparation and accurate configuration of Common Test hooks, even when entries are non-contiguous.

This commit introduces a **refactoring** to the **Erlang common_test daemon core**, specifically within the `ct_daemon_core.erl` module. It **inlines a nested list comprehension** inside the `list/1` function, which significantly **simplifies the code structure**. This optimization also **removes a redundant list allocation**, thereby improving memory efficiency and overall performance within the test execution subsystem.

This commit performs a **configuration update** by adding the `test_application` label to the `erlang_application` definition for the **Erlang common test CLI library** (`test_cli_lib`). Located in the `prelude/erlang/common_test/test_cli_lib/BUCK` file, this change explicitly marks the library as test-related within the **build system metadata**. The primary impact is on **downstream tooling**, enabling build systems and other development tools to correctly identify and appropriately process this testing utility. This ensures proper handling of test-specific components, improving **build system efficiency** and integration.

This commit **fixes** a **bug** in the **Erlang toolchain rule** that prevented **utility modules** from being accessible when executing **toolchain scripts**. Specifically, it modifies the `_gen_toolchain_script` function within `prelude/erlang/erlang_toolchain.bzl` to correctly add these utility modules to the toolchain script paths. This ensures that all necessary modules are properly included and available at runtime, resolving issues where toolchain scripts would fail due to missing dependencies. The change improves the reliability and functionality of the **Erlang toolchain**, allowing scripts to leverage their intended utilities without error.

This commit performs a significant **refactoring** within the **Erlang toolchain** by centralizing common filename conversion logic. A new utility module, `prelude/erlang/toolchain/dependency_utils.erl`, is introduced with a `chars_to_binary` function to encapsulate this functionality. Consequently, modules such as `app_src_builder`, `dependency_analyzer`, `dependency_finalizer`, and `dependency_merger` are updated to utilize `dependency_utils:chars_to_binary`, replacing their previously duplicated `to_binary_filename` or `filename_to_binary` implementations. This change enhances code maintainability and reduces redundancy across the core dependency processing components.

This commit **resolves a Dialyzer warning** within the **Erlang common_test framework** by correcting a type specification and performing a minor **refactoring**. Specifically, it updates the type for the `listing_to_testnames` function in `test_binary.erl` from `binary` to `string`, aligning it with actual usage. This correction allowed for the removal of the now-unnecessary helper function `binary_to_list_string`, improving code clarity and reducing complexity. The change enhances the static analysis correctness and maintainability of the `test_binary` module.

This commit introduces a significant **performance optimization** by **refactoring** I/O operations across the **Erlang toolchain's dependency analysis and merging components**. The `dependency_analyzer`, `dependency_finalizer`, and `dependency_merger` modules now directly utilize the `prim_file` module and eagerly convert filenames to binaries, which **reduces memory copying and garbage collection overhead**. Additionally, `dependency_merger` switches to `erlang:term_to_iovec` for writing terms, and `dependency_finalizer` includes an update to `merge_deps` arity and optimized key construction. These changes collectively improve the efficiency and speed of the dependency processing pipeline.

This commit introduces a **refactoring** within the **Erlang common_test hooks** module, specifically affecting `prelude/erlang/common_test/cth_hooks/src/cth_tpx_test_tree.erl`. The `get_result` function has been updated to utilize **list comprehensions** for traversing collections and gathering results, thereby **simplifying its implementation**. This change enhances the **readability and maintainability** of the code by adopting a more idiomatic Erlang pattern for result collection, without altering its functional behavior.

This commit provides a significant **bug fix** and **enhancement** for **unicode support** within the **Erlang Common Test integration** and related toolchain components. It addresses previous crashes when non-ASCII characters were present in test names, particularly during **test listing**. The changes involve updating numerous `io_lib:format` calls from `~s` to `~ts` specifiers, adding the `binary` option to `file:write_file` calls, and introducing helper functions like `unicode_characters_to_binary` and `list_string_to_binary` to correctly handle unicode in string manipulations, output formatting, and regular expressions (`re:compile`). This work improves the ability to list tests with unicode names and ensures proper encoding in various outputs and file writes, though comprehensive end-to-end support for running such tests still requires further effort.

This commit **optimizes** the **Erlang `.app` file builder** in `prelude/erlang/toolchain/app_src_builder.erl` by significantly reducing file I/O overhead through less copying and serialization, and by minimizing garbage production. It introduces **unicode support for filenames**, enhancing compatibility for diverse project structures. Furthermore, the commit **refactors** the internal logic for diff construction and application normalization, leading to more efficient and robust `.app` file generation. This work improves the overall performance and reliability of the build process.

This commit **fixes an incorrect type specification** within the **Erlang common_test module**, specifically for the `listing_to_testnames` function in `test_binary.erl`. The previous specification incorrectly declared the function to return a list of strings, when its actual return type is a **list of binaries**. This **maintenance fix** ensures the type system accurately reflects the function's runtime behavior, improving code correctness and preventing potential type-related issues for any code consuming this function's output.

This commit implements a significant **performance optimization** by **refactoring** numerous file I/O operations to utilize **raw file access**, thereby **bypassing the file server** which was identified as a bottleneck. The changes primarily impact the **`common_test` framework**, including components like `test_artifact_directory`, `test_binary`, `test_runner`, and `ct_executor`, as well as parts of the **Erlang `shell` utilities** and **`toolchain` components**. Functions such as `write_output`, `load_from_file`, `execute_test_suite`, and various directory/file checks are updated to leverage this direct I/O. This work aims to **improve I/O performance** and **reduce execution times** for tests and build processes where the overhead of the file server's remote access features is unnecessary.

This commit introduces **UTF-8 printing support** across the **Erlang Common Test framework** to prevent crashes when non-ASCII characters appear in output. Previously, the printer would fail when encountering Unicode characters in error messages, assertion output, or log entries. This **bug fix** updates numerous `io_lib:format` calls in modules like `ct_error_printer.erl`, `buck_ct_parser.erl`, and `test_logger.erl` to use `~tp` and `~ts` format specifiers. As a result, the test system can now robustly display and log **Unicode characters** in error reports, assertion failures, and generated log filenames, significantly improving its internationalization capabilities.

This commit performs a comprehensive **refactoring and cleanup** of the **Erlang build rules** across numerous `prelude/erlang/*.bzl` files. It introduces a new `erlang_paths.bzl` module to centralize common path manipulation utilities, which are then adopted throughout the existing rules for consistency. Key changes include factoring out common sub-expressions, simplifying intermediate values, standardizing string formatting, and explicitly passing build directories and application names for improved clarity. This work significantly enhances the **maintainability, readability, and robustness** of the Erlang build system by reducing duplication and improving modularity in how artifacts and paths are handled.

This commit introduces a **performance optimization** within the **Erlang toolchain's application source builder**. The `do_parse_app_info_file` function in `prelude/erlang/toolchain/app_src_builder.erl` has been **refactored** to utilize raw file I/O when parsing `AppInfoFile`s. This change is expected to provide **considerably faster** file processing for the `app_src_builder` component, as the features of the non-raw fileserver mode are not necessary for this specific operation. This enhancement aims to improve the overall efficiency of build processes that rely on `AppInfoFile` parsing.

This commit **enrolls specific Erlang source files into automatic formatting**, a **maintenance** task aimed at improving code consistency. It achieves this by adding a formatting comment to the top of `common_util.erl` and `json_interfacer.erl` within the `prelude/erlang/common_test` module. This change ensures a more uniform syntactic style across the codebase, which is crucial for **developer collaboration** and **enabling future codemod applications** that rely on consistent formatting. The overall scope is to enhance the **code quality and maintainability** of the Erlang components.

This commit performs a significant **refactoring** and **cleanup** of the **Erlang Language Platform (ELP) BXL script** (`elp.bxl`). It **removes dead code** by deleting several unused functions and updates the script to utilize **recommended BXL APIs**. Furthermore, the logic for handling absolute paths and includes within the script is consolidated, improving its clarity and maintainability. This work modernizes the `elp.bxl` script, streamlining its operations and preparing it for future enhancements.

This commit introduces a **performance tuning** change by configuring the execution environment for **Erlang escripts**. Specifically, it sets the `MALLOC_ARENA_MAX` environment variable to `2` when escripts are executed via the `_run_escript` function in `prelude/erlang/erlang_build.bzl`. This adjustment aims to **optimize memory management** by limiting the number of glibc malloc arenas, thereby balancing memory consumption and ensuring more consistent performance for these scripts.

This commit **optimizes the Erlang VM arguments** used by the `dependency_analyzer` tool within the **Erlang build system**. It achieves this by **refactoring** the `_get_deps_file` function in `prelude/erlang/erlang_build.bzl` to invoke the `dependency_analyzer` via the standard `_run_escript` helper. This change ensures the **dependency analysis process** now benefits from previously established VM argument optimizations, leading to more efficient and potentially faster build times for Erlang projects.

This commit applies a series of **performance optimizations** and **refactoring** to several Erlang escripts within the `prelude/erlang/toolchain`. It enhances the efficiency of **boot script generation** (`boot_script_builder.escript`), **dependency finalization** (`dependency_finalizer.escript`), **escript building** (`escript_builder.escript`), and **release variable creation** (`release_variables_builder.escript`). Key changes include utilizing raw binary file modes for faster I/O, refactoring parallel worker management, and extracting dependency collection logic into new helper functions. These updates collectively aim to **reduce build times** and improve the overall performance of the Erlang build process.