Conner Nilsen

This commit introduces a **new telemetry capability** to capture and log the exit code of the build system process (e.g., Buck2) when querying the **source database**. This **feature addition** modifies the `pyrefly_build` module to classify and store the process's exit reason, updating the `query_source_db` function to extract and assign this information. The `TelemetrySourceDbRebuildInstanceStats` in `pyrefly_util` is extended with an `exit_reason` field to record this data. This significantly enhances **observability** by providing detailed insights into source DB query outcomes, enabling better classification of failures (e.g., infrastructure errors, user errors, timeouts) and confirming successful operations.

This commit introduces **new telemetry events** to provide detailed **observability into the LSP startup process**. It adds an `LspStartup` event kind in `crates/pyrefly_util/src/telemetry.rs` and modifies the **LSP server** (`pyrefly/lib/lsp/non_wasm/server.rs`) to capture the true `lsp_start_time` before the handshake. This ensures accurate timing for all subsequent events and tracks the `initial_client_config_wait_ms` duration. This **new capability** will help diagnose and optimize LSP initialization performance by providing critical insights into client configuration delays.

This commit performs a significant **refactoring** of the **thread pool management** within the `pyrefly` project. It **removes the global `static THREADS` state** and the `init_thread_pool()` function, ensuring that thread pool configuration is always explicit. Consequently, the `State::with_thread_count()` and `ThreadPool::with_thread_count()` constructors have been **renamed to `new()`**, requiring callers in modules like `pyrefly/lib/commands`, `pyrefly/lib/lsp`, and various `pyrefly/lib/test` utilities to pass `ThreadCount` directly. This change improves the clarity and predictability of thread pool initialization by eliminating implicit global dependencies, making the system more robust.

This commit **refactors** the **test infrastructure** by migrating all test callers of `State::new()` to explicitly initialize `State` objects with a defined thread count using `State::with_thread_count(config_finder, TEST_THREAD_COUNT)`. This **maintenance** change affects numerous **test cases** across various modules, including `error/suppress.rs`, `lsp/non_wasm/external_provider.rs`, `state/errors.rs`, `stubgen/mod.rs`, and the core `test/incremental.rs`, `test/state.rs`, and `test/util.rs` helpers. The update ensures consistency and explicitness in how `State` objects are created within tests, improving the clarity and maintainability of the **testing environment**.

This commit performs a significant **refactoring** by explicitly threading the `thread_count: ThreadCount` parameter through the entire **CLI dispatch chain** and all production callers. It updates the initialization of `State` across various **commands** (e.g., `buck_check`, `check`, `init`), the **LSP and TSP servers**, the **query engine**, and the **playground**. Specifically, all instances of `State::new(config_finder)` are replaced with `State::with_thread_count(config_finder, thread_count)`. This change ensures that thread management is consistently and explicitly configured for core operations, improving the clarity and control over concurrency settings throughout the application. Test callers were also updated to maintain compilation and reflect the new API.

This commit introduces **foundational changes to decouple thread pool state** from global variables within the `pyrefly` system. It adds a new constructor, `State::with_thread_count()`, enabling explicit thread count configuration for `State` instances, alongside a `CommonGlobalArgs::thread_count()` accessor. Furthermore, a `TEST_THREAD_COUNT` constant is introduced in `crates/pyrefly_util/src/thread_pool.rs` and immediately used to update test initialization in `pyrefly/lib/test/util.rs`. This is a **refactoring and new capability** effort, preparing the `pyrefly` system for more flexible thread pool management, though existing production code continues to use global state for now.

This commit **refactors** the **PyRefly** project's **ad-hoc transaction telemetry** recording mechanism to use the more robust `SubTaskTelemetry` object. Previously, a custom closure was used, which lacked the guarantees provided by the standard subtask telemetry approach. This change primarily impacts the **LSP server's `handle_solve_request`** and the **transaction state management** within `Transaction` and `TransactionData` structs, where the custom recorder is replaced with `SubTaskTelemetry`. By standardizing on `SubTaskTelemetry`, this **maintenance** effort improves telemetry reliability and **prepares the system for future requirements**, such as mandatory telemetry event IDs.

This commit introduces an **enhancement to transaction management** within `pyrefly`, specifically addressing how **state lock blocking durations** are tracked. Previously, when a transaction failed to restore from saved state, the time spent waiting for the state lock was discarded. Now, the `TransactionData::restore` method in `pyrefly/lib/state/state.rs` is updated to return this blocking duration upon failure, which is then captured by the `non_committable_transaction` function in `pyrefly/lib/lsp/non_wasm/transaction_manager.rs`. This ensures that the **blocked time is correctly attributed** to the newly created transaction, significantly improving the **accuracy of performance metrics** related to transaction processing and state restoration failures.

This commit **enhances the reliability and reproducibility of LSP interaction tests** by exposing the thread pool size configuration. The `LspInteraction` constructors (`new`, `new_with_args`) and the `init_test` function within the `pyrefly/lib/test/lsp/lsp_interaction` module now accept an optional `ThreadCount` parameter. This **maintenance** change allows developers to explicitly control the thread pool size during test setup, directly addressing issues where incorrect or implicitly set thread counts led to unreliable test outcomes and difficulties in reproducing LSP-related bugs. The primary impact is on **developer experience** and the **stability of the LSP testing infrastructure**.

This commit introduces a **new capability** to the **LSP integration testing framework** by adding `TestTelemetry` and `RecordedTelemetryEvent` structs. These new components enable the recording and subscription of all logged telemetry events directly within **LSP integration tests**. By modifying `LspInteraction` in `pyrefly/lib/test/lsp/lsp_interaction/object_model.rs` to accept a custom telemetry implementation, this change provides deeper insight into event timing and duration. This enhancement facilitates more thorough analysis for tests and benchmarks, significantly improving the ability to understand and optimize Pyrefly's LSP behavior during testing.

This commit performs a **refactoring** of the **PyreFly LSP server's telemetry system** to streamline data collection for sub-tasks. It **eliminates the redundant `TelemetryTaskContext` struct** in `crates/pyrefly_util/src/telemetry.rs`, opting to directly use `TelemetryEvent` for context in `SubTaskTelemetry::new`. This change impacts the **LSP server's heavy task queue** (`pyrefly/lib/lsp/non_wasm/queue.rs`) and the **LSP server implementation** (`pyrefly/lib/lsp/non_wasm/server.rs`), where function signatures like `run`, `queue_task`, and `find_all_symbols` are updated to reflect the simplified telemetry context. The primary goal is to **reduce telemetry overhead** by removing unused fields and simplifying the API for performance-critical "heavy tasks."

This commit **refactors** the **telemetry system** to make the `task_id` field **mandatory** for `TelemetryEvent` and `SubTaskTelemetry` structs. The `set_task_id` method is removed from `crates/pyrefly_util/src/telemetry.rs`, requiring the `task_id` to be provided directly during event construction. Consequently, the **LSP** and **TSP server loops** (`pyrefly/lib/lsp/non_wasm/server.rs`, `pyrefly/lib/tsp/server.rs`) and the **LSP heavy task queue** (`pyrefly/lib/lsp/non_wasm/queue.rs`) are updated to generate and pass the `task_id` when creating new telemetry events. This change **enforces stricter data integrity** by ensuring all task-related telemetry events are consistently associated with a `task_id` from their inception.

This commit introduces a **new feature** by enhancing the **telemetry system** within the `pyrefly_util` crate. The `SubTaskTelemetry` struct now includes a `file_stats` field, which is populated during its construction and included when converting telemetry data to an event. This change allows the **LSP server's request handling logic** in `pyrefly/lib/lsp/non_wasm/server.rs` to capture and report detailed file-level statistics for each subtask. The update provides richer insights into the performance and characteristics of individual operations by extending the scope of collected telemetry data.

This commit **adjusts telemetry logging** for the **`quickfix` and `fix-all` code actions** within the **PyreFly LSP server**. Previously, telemetry events for these actions were only logged if they were enabled and produced results, creating an inconsistency with other code actions. This **maintenance fix** ensures that telemetry is now consistently recorded whenever the computation for these actions is performed, even if no fixes are ultimately suggested. This change improves the accuracy and completeness of usage and performance data for these critical code assistance features, providing better insights into their performance regardless of outcome.

This commit **refactors the telemetry system** for **code actions** within the **LSP server** to improve data consistency and automation. It transitions from manually creating subtasks for code action telemetry to utilizing the `SubTaskTelemetry` object, which automatically links these events back to their originating LSP requests. Specifically, the `pyrefly/lib/lsp/non_wasm/server.rs` file is updated to remove the `record_code_action_telemetry` helper and modify the `code_action` function to use `SubTaskTelemetry`. This **refactoring** enhances the **accuracy and consistency of telemetry data** by ensuring relevant fields are automatically copied, setting the stage for more comprehensive future tracking of code action usage.

This commit **enhances the LSP server's telemetry capabilities** by integrating recording for two specific code actions. It **adds telemetry** to the `pyrefly/lib/lsp/non_wasm/server.rs` module for the `convert_module_package` and `safe_delete_file` code actions. This **maintenance** work provides crucial visibility into the usage and performance of these recently introduced features. The change ensures that the project gains better insights into how developers interact with these particular **code actions**, aiding future development and prioritization.

This commit introduces a **dedicated thread pool for Language Server Protocol (LSP) operations** within the `pyrefly` LSP server. Previously, LSP queries like `search_exports` for completions and code actions shared a single thread pool with background and foreground rechecks, leading to **contention and blocking of LSP requests** when rechecks consumed all available threads. By adding a separate `ThreadPool` to the `Server` struct in `pyrefly/lib/lsp/non_wasm/server.rs`, this **new feature** significantly improves the **responsiveness and reliability of LSP features** by ensuring they are no longer starved of threads during intensive recheck operations. This change primarily affects the **LSP subsystem**, enhancing the user experience for developers relying on `pyrefly`'s language server capabilities.

This commit introduces a **significant performance improvement** for **Language Server Protocol (LSP) operations** by allowing LSP-related transactions to utilize a dedicated thread pool. Previously, concurrent transactions could block the LSP, leading to extended durations for operations like `didOpen`. The change involves **refactoring core transaction execution functions** in `pyrefly/lib/state/state.rs` to accept an optional custom thread pool, and modifying **LSP server functions** in `pyrefly/lib/lsp/non_wasm/server.rs` to pass this pool for validation. Numerous other files are updated to pass `None` for the custom pool, aligning with the new concurrency model. This optimization drastically **reduces LSP operation durations**, with `didOpen` times dropping from approximately 2 seconds to 40 milliseconds, enhancing the responsiveness and user experience of the LSP.

This commit introduces a significant **performance optimization** by **refactoring** the `pyrefly` LSP server to offload computationally intensive tasks to a dedicated LSP thread pool. Specifically, **LSP operations** such as `code_action` generation, `completion` suggestions, and `workspace_symbols` searches are now executed in `lsp_threads` rather than the main LSP loop. This involved modifying core functions like `local_quickfix_code_actions_sorted`, `search_exports_fuzzy`, and `completion` within `pyrefly/lib/state/lsp.rs` to accept an optional `ThreadPool` argument. The change primarily affects the **LSP server's responsiveness**, ensuring a smoother user experience during code editing by preventing blocking operations.

This commit **refactors** the internal **PyTorch configuration system** by migrating specific settings into a new, dedicated configurer wrapper. Previously, these internal settings were managed through hardcoded `if cfg!(fbcode_build)` conditional checks, leading to less maintainable and more complex code. The change centralizes the management of these configurations, **simplifying the codebase** and eliminating the need for scattered build-specific conditionals. This enhancement primarily affects the **core configuration logic**, improving its clarity and making it easier to manage environment-dependent settings.