cd rubin

This commit introduces **new Semgrep rules** to enforce **directory entry validations**, ensuring files like `.semgrep/directory-entry-validation.yaml` have correct extensions, an `id` field, and a valid ID format. It includes a **bug fix** for `src/content/directory/cloudflare-agent.yaml` by adding a unique `id` to comply with these new standards. Furthermore, the `tools/semgrep-repo-rules` helper script is **enhanced** to be more flexible, allowing scans of all files or specific paths. This work represents a **new capability** for code quality enforcement and a **maintenance fix**, improving the consistency and maintainability of directory entries across the project.

This commit **improves the project's documentation** by adding new content to the **style guide**. It introduces detailed usage instructions, props, and behavior for the `ProductAvailabilityText` component, ensuring developers understand its proper implementation. Additionally, it clarifies how 'Beta' badges are automatically displayed within the **sidebar** based on product availability data, enhancing consistency in feature presentation. This **documentation update** aims to provide clearer guidance for UI component usage and feature badging.

This commit performs a **maintenance update** by adjusting the `id` field within the `src/content/directory/cloudflare-tunnel-sase.yaml` file. Specifically, it modifies the identifier for the **Cloudflare Tunnel SASE directory entry** to ensure it conforms to established naming conventions or internal standards. This `chore` improves **data consistency** and internal referencing for this particular content entry. The change ensures proper identification of the **Cloudflare Tunnel SASE** service within the directory structure, with no direct impact on user-facing functionality.

This commit performs a significant **refactoring** and **maintenance** effort to streamline data management for product availability and sidebar beta badges across the documentation site. It **removes the deprecated `wid` field** from numerous documentation page frontmatter and the `sidebar` utility's schema, centralizing data sourcing. The `ProductAvailabilityText` component and **sidebar badge logic** are updated to consistently query the `src/content/directory` collection, improving the maintainability and accuracy of these UI elements. This change standardizes how product status and beta badge information is retrieved, cleaning up outdated metadata and simplifying future content updates.

This commit primarily **expands the project's documentation** by adding 26 new YAML files to the `src/content/directory` module, introducing entries for a wide array of product features such as 'Firewall for AI', 'WAF Sensitive Data Detection', and 'Workers Smart Placement'. Concurrently, it includes a **tooling maintenance update** to the `tools/directory-entry-ids` script, specifically improving its **macOS environment path resolution** for better reliability. This work enhances the discoverability and information available for numerous features, providing more comprehensive content for users and improving developer experience for macOS users of the script. Additionally, an unnecessary 'ethereum' directory entry was removed, streamlining the content.

This commit introduces a **new capability** by adding a unique `id` field to all **directory entry YAML files** located within `src/content/directory/`. This standardization enhances the programmatic identification and referencing of individual directory entries across the system. A new helper script, `tools/directory-entry-ids`, is also included to **generate and manage these unique IDs**, streamlining content maintenance workflows. Furthermore, some directory entry files undergo a **refactoring** to standardize their file extension from `.yml` to `.yaml`. This work improves data consistency and provides better tooling for managing the content directory.

This commit introduces **granular control definitions** within the `granular-control-applications` content collection by updating its Zod schema to include new control items and adjusting the data source URL. This **new capability** allows for more detailed configuration of application content. Concurrently, the **data loading utilities** in `src/util/custom-loaders.ts` have been **refactored** to improve download handling. This involves extracting download logic into a new utility function, `downloadToDotTempIfNotPresent`, and updating `middlecacheLoader` to utilize it, enhancing the robustness and organization of data fetching processes.

This commit **fixes a bug** related to the **display of category titles** by correcting a data retrieval path. Specifically, the `lookupProductTitle` function within `src/util/sidebar.ts` was updated to fetch the product title from `entry.title` instead of the previously incorrect `product.title`. This **corrects the semantic data lookup** for product information, ensuring that **category and product titles are accurately displayed** across the application. The change primarily impacts **sidebar utility functions** and any UI components that rely on them for presenting correct product metadata.

This commit performs a significant **refactoring** by renaming the core content collection from `products` to `directory` and its individual items from `product` attributes to `entry` across the codebase. This change improves the semantic clarity of the content structure, which now better reflects its diverse nature beyond just "products." The update impacts the **content configuration** in `src/content.config.ts`, all **content files** under `src/content/directory/`, and numerous **UI components**, **page-level logic**, **schema definitions**, and **utility functions** that consume or process this content. A new utility file, `src/util/directory.ts`, was also introduced to manage these new directory entries, ensuring consistent data handling and a more accurate representation of the project's content architecture.

This commit introduces an **automated system for displaying product availability badges** in the documentation sidebar, driven by a new `wid` property in content frontmatter. It **implements a new `productAvailabilityBadge` function** within `src/util/sidebar.ts` to dynamically generate these badges, integrating it into the sidebar rendering logic. This **new capability** updates the `src/schemas/base.ts` to include the optional `wid` property and modifies numerous **documentation pages** to leverage this new system by adding `wid` to their frontmatter and removing explicit badge configurations. The change streamlines the process of indicating product availability, ensuring consistency across affected **documentation content** and reducing manual maintenance.

This commit introduces a **new `GranularControlApplicationsList` component** and its corresponding `granular-control-applications` content collection, enabling the display of granular application controls within the documentation. It also **enhances the `ProductAvailabilityText` component** with improved error handling and a new `parentheses` prop, while refactoring its underlying `product-availability` content collection for better data structure. These **enhancements and new capabilities** provide a **more robust and flexible system for rendering content-driven application and product availability information**. The work improves **UI presentation** and **data parsing resilience** across the **content management system**, leveraging better TypeScript and Zod integration, and ensuring compliance with Astro loader custom parser return values.

This commit **fixes Windows compatibility** for the `middlecache` system by adjusting how file paths are resolved. The `middlecacheLoader` in `src/util/custom-loaders.ts` now explicitly uses `path.join` to construct **relative file loader paths**, ensuring they resolve correctly on Windows. Additionally, the `product-availability` **content collection** has been **refactored** and moved to a more standard location, with its import path updated in `src/content.config.ts`. The **CI workflow** is also updated to exclude this new content collection path from code checks. This change ensures **absolute middlecache paths** function reliably across all supported operating systems.

This commit **updates the documentation style guide** by adding an explanatory note to `src/content/docs/style-guide/formatting/dates-and-times.mdx`. It clarifies **why the `semgrep` linter flags the use of dates** in documentation, providing context for this specific check. The added text emphasizes the project's preference for reflecting universal truths rather than transitory, time-bound information in its documentation. This **documentation improvement** helps contributors understand and adhere to best practices for writing more enduring and less frequently outdated content, thereby enhancing the overall quality and longevity of the project's documentation.

This commit **introduces a new capability** to integrate and display **product availability data** sourced from `middlecache`. It establishes a pattern for **remote-to-local data retrieval** by adding a `middlecacheLoader` utility and defining a new `product-availability` **content collection** with a custom schema. A new `ProductAvailabilityText` **Astro component** is created to render this data, initially demonstrated by integrating it into the `Cloudflare Trace` documentation. This **feature addition** lays the groundwork for dynamic content integration using custom loaders and content collections across the project.

This commit introduces a **maintenance update** to the **CI/CD pipeline** by modifying the **Semgrep static analysis workflow**. It **removes the `--error` flag** from the `semgrep` command within the `.github/workflows/semgrep.yml` configuration. This change ensures that **Semgrep findings will no longer cause the GitHub Actions build to fail**, preventing non-critical security or code quality issues from blocking pull requests. The primary impact is a **relaxation of Semgrep's enforcement level**, allowing builds to pass even when findings are present.

This commit primarily provides a **bug fix** for **Semgrep style guide rules** by refining date detection regexes in `.semgrep/style-guide-dates-in-docs.yaml`. This update specifically prevents **false positives** when scanning for month names and IPv6 addresses, improving the accuracy of code analysis. Concurrently, a **chore** is performed on the **GitHub Actions workflow** in `.github/workflows/semgrep.yml` to correctly process multiline commit messages by converting newlines to spaces, ensuring robust CI/CD operations.

This commit **adjusts the Semgrep static analysis workflow** within **GitHub Actions** to prevent job failures caused by Semgrep warnings. Previously, the workflow would exit with an error code, leading to a failed status in the GitHub UI and potentially obscuring annotations. Now, the `.github/workflows/semgrep.yml` configuration ensures the **Semgrep job always exits successfully**, even when warnings are present. This **maintenance chore** improves the developer experience by keeping **Semgrep annotations visible** and avoiding unnecessary top-level error indications in the UI.

This commit **fixes a bug** within the **Semgrep GitHub Actions workflow** that previously caused failures when commit messages contained double quotes. It updates the `semgrep.yml` configuration to replace double quotes with single quotes specifically within the commit message environment variable, ensuring proper execution of Semgrep scans. Additionally, the workflow and job names have been **refactored** for improved clarity and maintainability, enhancing the reliability of the CI/CD pipeline for static analysis.

This commit significantly **enhances the Semgrep GitHub Actions workflow** by improving its feedback mechanisms and robustness. It now displays **style guide warnings as GitHub annotations**, allows users to **skip checks using a `[skip style guide check]` commit message tag**, and ensures **accurate propagation of exit codes** from the `semgrep scan` tool. These **feature enhancements** primarily affect the **`.github/workflows/semgrep.yml`** configuration and the **`tools/semgrep-repo-rules`** script, alongside a **refactoring of the `.semgrep/style-guide-dates-in-docs.yaml`** rules for better clarity. This work provides clearer, more actionable feedback in CI, making it easier to manage code quality and style guide adherence within the project.

This commit introduces a **new capability** by implementing a **custom Semgrep rule** (`dates-in-docs.yaml`) designed to detect fixed time references and "coming soon" phrases within documentation files. This rule is integrated into the **CI/CD pipeline** via `.github/workflows/semgrep.yml` to automatically scan pull requests, ensuring early detection of potentially outdated content. Additionally, a local script (`tools/semgrep-repo-rules`) is provided, allowing developers to run these checks against both committed and uncommitted changes in their working directory. This enhancement aims to **improve documentation consistency and prevent outdated information**, while also offering flexibility to skip scans using `[skip semgrep]` in commit messages.