Jason Fried

This commit **refactors** the **Buck2 build system's Cython integration** by migrating `cython_library` and `cython_static_extension` from Starlark macros to native rules. This change significantly **simplifies the build graph** and **improves performance** by replacing the expansion into numerous sub-targets with direct `ctx.actions.run()` for Cython transpilation and `cxx_library_parameterized()` for C++ compilation. New files under `prelude/python/cython/` define the core logic, providers, and rule implementations, along with a new `cython_toolchain`. This **enhancement** streamlines the compilation process for projects leveraging Cython, making its integration more efficient and robust within Buck2.

This commit provides a **bug fix** for **native Python binaries** to prevent `PyInit` symbols for C extensions from being dropped by the linker. By emitting `-u` flags for these symbols and passing them to the linker, it ensures that necessary object files are pulled from archives, even when link groups or `--gc-sections` are active. This resolves issues like the `regex._regex` import failure in omnibus-linked binaries, guaranteeing **correct symbol resolution** and proper loading of C extensions in complex dependency environments. The change is implemented in `prelude/python/linking/native.bzl`, specifically within the `_compute_cxx_executable_info` logic.

This commit performs a **refactoring** across the **Thrift Python asynchronous codebase** to migrate from the deprecated `asyncio.get_event_loop()` to `asyncio.get_running_loop()`. Specifically, it updates core server components like `thrift/lib/py/server/TAsyncioServer.py` and example servers, along with numerous **Thrift Python test files** to align with modern `asyncio` practices. This **maintenance** effort ensures compatibility with newer Python versions, preventing potential runtime warnings or errors, and improves the long-term robustness of **Thrift's asynchronous server components**.

This commit **fixes memory leaks** identified in the **Thrift Python FastProto protocol** implementation. It addresses these issues by adding **Python reference count decrements** and implementing **module state clearing and traversal mechanisms** within `fastproto.cpp`. Specifically, changes were made to functions like `encode_impl` and `decode_struct` to prevent leaks during encoding and decoding, and new functions such as `fastproto_traverse` and `fastproto_clear` were introduced for proper module state management. This ensures more robust memory handling and prevents resource exhaustion when using the **FastProto** protocol in Python applications.

This commit introduces an explicit `ensure()` function to the **`folly.request_context`** Python module, providing a **safer and more explicit mechanism** for installing critical callbacks. Previously, these callbacks were installed implicitly as a side effect of importing the module, which could lead to issues if the import was optimized away or removed. The new `request_context.ensure()` function now explicitly loads the module and guarantees callback installation, preventing potential breakage for systems relying on `task_factories`. This **maintenance improvement** ensures the continued functionality of the request context's callback system by making its initialization explicit and testable, with a new test case verifying its behavior.

This commit performs **maintenance** and **refactoring** on the **`thrift-py3` test suite** to ensure compatibility with **Python 3.14**. It primarily addresses changes in `asyncio` event loop management by migrating numerous test classes within `thrift/lib/py3/test/auto_migrate` to use `unittest.IsolatedAsyncioTestCase`, converting their methods to be asynchronous. Additionally, the problematic `test_queue_timeout` test was removed due to its instability, and `asyncio.get_event_loop()` calls were updated to `asyncio.new_event_loop()` where appropriate. This work ensures the **`thrift-py3` asyncio tests** run correctly and reliably on the latest Python versions, preventing failures related to event loop handling.

This commit **refactors** the **`folly::RequestContext` watcher mechanism** within the **Python bindings** (`folly/python/request_context.pyx`) to resolve critical deadlocks. It specifically removes problematic GIL grabbing logic that caused issues in multi-threaded edge cases, replacing it with a more robust **thread-local current Python context** and refined GIL state checks. This **bug fix** significantly enhances the reliability of `RequestContext` when used across different threads and with asynchronous operations. New test cases have been added to validate correct behavior in these complex multi-threading scenarios, ensuring stable context propagation.

This commit provides a **bug fix** for the **Thrift Python server**'s `request_context` module, specifically addressing a compatibility issue with Cython 3.1.x. It resolves a problem where Cython's generated C++ code for `std::optional.value()` could lead to **AddressSanitizer errors** (stack-use-after-scope) due to incorrect variable scoping. By redefining `std::optional` within `thrift/lib/python/server/request_context.pxd`, this **compatibility fix** ensures that Cython generates correct and safe C++ code, preventing runtime crashes. This change is crucial for maintaining stability when using newer Cython versions with the Thrift Python server.

This commit **refactors** the **`folly::python::request_context`** module to enhance the robustness and correctness of the `setContext` watcher mechanism. It introduces a **thread-local variable** (`set_curr_PyContext`) to track the current Python context, allowing the watcher to gracefully ignore callbacks if no Python context is active, and **simplifies `PyState` checks** for post-finalization scenarios. The change also ensures **GIL acquisition** during context variable updates, safeguarding against potential issues in mixed C++/Python threading environments. A new test case in `folly/python/test/request_context.py` validates the improved cross-language context propagation.

This commit implements a **compatibility fix** for the **`folly/python/request_context` module**, specifically targeting issues encountered on `aarch64` architectures. It introduces an **environment variable check** within `folly/python/request_context.pyx` to serve as an "escape hatch." This mechanism allows users to **conditionally disable the module's internal watchers**, thereby resolving compatibility problems and ensuring the `request_context` module can function correctly on `aarch64` systems. This change provides a crucial workaround to prevent runtime failures in `aarch64` environments.

This commit introduces a significant **performance optimization** within the **`folly/python/request_context`** module by implementing **thread-local caching** for the last set context. This **refactoring** reduces the overhead of `setContext` operations in Python threads by minimizing expensive `PyContext` reads and Python C-API interactions. Specifically, it prevents the `_setContextWatcher` from redundantly setting an already active context, leading to **faster context management** and improved efficiency. The change includes new internal functions like `get_last_set_context` and `set_last_set_context`, and is accompanied by a **test update** to verify `request_context.save()` functionality.

This commit **fixes a memory leak** within the **`folly/python/request_context`** module by **refactoring** how `RequestContext` objects are stored and managed. Previously, `RequestContext` `shared_ptr`s were wrapped in raw `PyCapsule` objects, leading to incorrect reference counting and leaks, especially when multiple context copies or tasks were involved. The internal representation is now reworked to embed the `shared_ptr` directly into the `Context` Cython class, replacing the problematic `PyCapsule` usage. This change ensures proper memory management and simplifies the API, with new tests added to verify the `use_count` functionality and confirm the **resolution of the memory leak**.

This commit implements a **BREAKFIX** within the **`folly/python/request_context`** module, specifically addressing the conditions for installing a Python context watcher. It **expands the allowed environments** by permitting watcher installation when `sys.version` contains either `+fb` or `+cinder`. This ensures that **request context propagation** functions correctly in Meta and Cinder-specific Python builds, which were previously blocked from utilizing this essential functionality. The change rectifies a recent regression, restoring expected behavior for these specialized environments.

This commit introduces a **bug fix** within the **Folly Python integration** to correctly manage the `Folly::RequestContext` during thread teardown. Specifically, the `_watcher` function in `folly/python/request_context.pyx` is modified to **unset the Folly RequestContext** by setting it to `nullptr` when the associated `PyContext` is `None`. This ensures proper cleanup and prevents potential issues with dangling context pointers when Python threads exit, aligning the C++ `Folly::RequestContext` state with expected behavior during thread destruction. The change improves the robustness of **cross-language request context management** by ensuring consistent context handling.

This commit provides a **bug fix** for **Folly's Python bindings**, specifically within the `folly/python/request_context` module. It addresses an issue where Conda wheel builds would segfault at runtime due to a mismatch with `cpython+meta` headers. A conditional check is introduced in `folly/python/request_context.pyx` to prevent the problematic call to `FOLLY_PYTHON_PyContext_AddWatcher` when running in incompatible environments like Conda. This **maintenance** change significantly improves **runtime stability** for Python users deploying Folly's bindings within **Conda environments**.

This commit **refactors** the **Python bindings for Folly's request context** to **safeguard** the `_RequestContext` `ContextVar`. Specifically, it modifies `folly/python/request_context.pyx` by changing the declaration of `_RequestContext` to `cdef object`. This **limits its control surface** and **prevents external setting**, thereby **restricting direct modification** of the `ContextVar`. The change is a **security and stability enhancement**, preventing unintended side effects or incorrect usage in unit tests or everyday application code, mirroring the protection level of `thrift-python`'s `CPP2RequestContext` holder.

This commit introduces a **new capability** by adding the `get_PyContext` function to the **`folly/python/request_context`** module. This function provides a controlled mechanism to retrieve the `PyCapsule` stored within an internal `ContextVar`, which is managed by Cython. The primary purpose is to prevent direct, potentially "undesirable" mutations of the `ContextVar` from Python code, thereby enhancing the **integrity and stability** of the request context management. This ensures that `PyCapsules` can be safely accessed without exposing the underlying `ContextVar` to arbitrary changes, improving the robustness of context handling.

This commit introduces a **performance optimization** to the **Thrift Python server** by significantly accelerating `ContextVar` handling. It refactors the `combinedHandler` within `thrift/lib/python/server/python_async_processor.pyx` to leverage faster C-API functions, specifically `PyContextVar_Set` and `PyContextVar_Reset`. This change replaces the slower standard Python runtime loop methods, resulting in **substantially faster** `ContextVar` operations, estimated to be up to 15x quicker. The improvement primarily benefits RPC `create_task` operations, enhancing the overall efficiency of asynchronous RPC processing.

This commit introduces a **new context watcher mechanism** within the **`folly/python/request_context`** module, making `_setContextWatcher` the exclusive interface for managing the `PyContext` variable. This **refactoring** ensures that Python context variables are reliably synchronized with underlying C++ `RequestContext` changes, replacing direct manipulation methods like `set_PyContext` and `reset_PyContext`. The change improves consistency by centralizing how the Python context observes and reacts to C++ state, with new tests validating this synchronization. An efficiency optimization was also applied to `RequestContextToPyCapsule` using `std::move`.

This commit introduces a **new feature** to the `folly::RequestContext` mechanism by implementing a **Watcher system**. This system allows external components to register callbacks that are invoked whenever the active `folly::RequestContext` is changed via `setContext()` or `setShallowCopyContext()`. The primary goal is to improve **cross-language context integration**, enabling better synchronization with foreign context systems like Python's `PyContext`. This enhancement to the `folly/io/async` module provides a crucial mechanism for propagating context changes across different runtime environments, with new declarations in `Request.h` and implementation in `Request.cpp`, alongside dedicated unit tests.