# Coding Standards & Best Practices for Python Code

This document outlines the core coding standards, best practices, and quality control procedures for the codebase.

## Python Version Compatibility

    - The project supports Python 3.10+ (`requires-python = ">=3.10,<3.15"`). Never use features introduced after Python 3.10 without a compatibility fallback.
    - Common pitfalls:
      - `datetime.UTC` was added in Python 3.11. Use `datetime.timezone.utc` instead.
      - `StrEnum` was added in Python 3.11. Always import it from `pipelex.types` which handles retrocompatibility.
      - `type` statement (PEP 695) was added in Python 3.12. Use `TypeAlias` from `typing` instead.
      - `ExceptionGroup` / `except*` was added in Python 3.11. Avoid unless using the `exceptiongroup` backport.

## Variables, loops and indexes

    - Variable names should have a minimum length of 3 characters. No exceptions: name your `for` loop indexes like `index_foobar`, your exceptions `exc` or more specific like `validation_error` when there are several layers of exceptions, and use `for key, value in ...` for key/value pairs.
    - When looping on the keys of a dict, use `for key in the_dict` rather than `for key in the_dict.keys()` otherwise you won't pass linting.
    - Avoid inline for loops, unless it's ultra-simple and holds on oneline.
    - If you have a variable that will get its value differently through different code paths, declare it first with a type, e.g. `pipe_code: str` but DO NOT give it a default value like `pipe_code: str = ""` unless it's really justified. We want the variable to be unbound until all paths are covered, and the linters will help us avoid bugs this way.

## Enums and tests

    - When defining enums related to string values, always inherit from `StrEnum`
    - When you need the enum value as a string, don't use `str(enum_var)` or `enum_var.value`, just use `enum_var` itself, that is the point of using StrEnum!
    - Never test equality to an enum value: use match/case, even to single out 1 case out of 10 cases. To avoid heavy match/case code in awkward places, add @property methods to the enum class such as `is_foobar()`. This is to avoid bugs: when new enum values are added we want the linter to complain. Use the `|` operator to group cases
    - As our match/case constructs over enums are always exhaustive, NEVER add a default `case _: ...`. Otherwise, you won't pass linting.
    - `StrEnum` must be imported from `pipelex.types` (handles python retrocompatibility):
    ```python
    from pipelex.types import StrEnum
    ```

## Optionals

- Don't write things like `a = b if b else c`, write `a = b or c` instead.

## Imports

### **Imports at the top of the file**

    - Avoid as much as possible import statements outside of a module's top-level scope.
    - Do not import libraries in functions or classes unless in very specific cases, to be discussed with the user, as they would required a `# noqa: ...` comment to pass linting
    - Do not bother with ordering the imports or removing unused imports, our Ruff linter will handle it for us.
    - `if TYPE_CHECKING:` blocks must always be the **last** block in the imports section, placed after all regular imports.

### **Removing unused imports**

    - To remove unused imports, run `make fix-unused-imports` or `make fui` (shorthand). This is faster and cheaper than rewriting with LLM.

### **No re-exports in `__init__.py`**

    - Do NOT fill `__init__.py` files with re-exports.
    - Always use direct full-path imports everywhere. For example:

- **Logging and Pretty Printing**:

    - Both `log()` and `pretty_print()` can be imported from `pipelex` directly:
    ```python
    from pipelex import log, pretty_print

    log.info("Hello, world!")
    ```
    - Both have a title arg which is handy when logging/printing objects:

    ```python
    log.verbose("Hello, world!", title="Your first Pipelex log")
    pretty_print(output_object, title="Your first Pipelex output")
    ```
    - Both handle formatting json using Rich, pretty_print makes it prettier.

- **StrEnum and Self type**:

    - Both `StrEnum` and `Self` must be imported from `pipelex.types` (handles python retrocompatibility):
    ```python
    from pipelex.types import Self, StrEnum
    ```

## Typing

### **Always Use Type Hints**

    - Every function parameter must be typed
    - Every function return must be typed
    - Use type hints for all variables where type is not obvious
    - Use dict, list, tuple types with lowercase first letter: dict[], list[], tuple[]
    - Use type hints for all fields
    - Use Field(default_factory=...) for mutable defaults
    - Use `# pyright: ignore[specificError]` or `# type: ignore` only as a last resort. In particular, if you are sure about the type, you often solve issues by using cast() or creating a new typed variable.

### **BaseModel / Pydantic Standards**

    - Use `BaseModel` and respect Pydantic v2 standards
    - Use the modern `ConfigDict` when needed, e.g. `model_config = ConfigDict(extra="forbid", strict=True)`
    - Keep models focused and single-purpose
    - For list fields with non-string items in BaseModels, use `empty_list_factory_of()` to avoid linter complaints:
      ```python
      from pydantic import BaseModel, Field
      from pipelex.tools.typing.pydantic_utils import empty_list_factory_of
      
      class MyModel(BaseModel):
          names: list[str] = Field(default_factory=list)  # OK for strings
          numbers: list[int] = Field(default_factory=empty_list_factory_of(int), description="A list of numbers")
          items: list[MyItem] = Field(default_factory=empty_list_factory_of(MyItem), description="A list of items")
      ```

## Factory Pattern

    - Use factory pattern for object creation when dealing with multiple implementations
    - Our factory methods are named `make_from_...` and such

## Protocols and Interfaces

    - When a getter (e.g. `get_report_delegate()`, `get_library_manager()`) returns a `Protocol` type, callers must only rely on methods declared on that `Protocol`. If you need to call a method that lives on a concrete implementation, **extend the `Protocol`** — do not work around it with `isinstance(...)` checks, `cast(...)`, or inline imports of the concrete class.
    - When extending the `Protocol`, also update every implementation (including no-op / null implementations) so structural typing is satisfied. For implementations where the method has nothing to do, give it a `pass` body — that is the whole point of having a `Protocol` rather than a concrete base class.
    - Smell to watch for: an inline `from ... import ConcreteImpl  # noqa: PLC0415` followed by `if isinstance(delegate, ConcreteImpl): delegate.some_method(...)`. This is almost always a missing method on the `Protocol`. Promote it.
    - Use `@override` (from `typing_extensions`) on every implementation of a `Protocol` method, so that signature drift between the `Protocol` and its implementations is caught by the type checker.

## Error Handling

    - Always catch exceptions at the place where you can add useful context to it.
    - Use try/except blocks with specific exceptions
    - Convert third-party exceptions to our custom ones except in pydantic validators where you can raise a ValueError or a TypeError
    - **NEVER write `except Exception:` (or bare `except:`).** Catching the generic `Exception` is FORBIDDEN except in two cases, each of which REQUIRES a one-line comment directly under the `except` naming which case applies: **(1)** the absolute root of a CLI command or an API endpoint handler; **(2)** the wrapped call invokes genuinely unbounded code whose exception surface cannot be enumerated — a user-registered callback or function, dynamic plugin dispatch, or a third-party SDK with no documented exception types (this is only for genuinely open-ended surfaces; "I didn't look the exceptions up" does NOT qualify). There are NO other exceptions to this rule. The following are NOT valid justifications and the agent must reject them when tempted:
        - "I want to log and re-raise" → just let it propagate; logging happens at the root.
        - "I want to swallow it just in case" → don't. Unknown failures must surface.
        - "I want to convert it to our error type" → only if you know which specific exception(s) the call can raise; catch those explicitly. Catching a project base class (`PipelexError`) or a package base (`CogtError`, `ToolError`) is acceptable when the operation legitimately fails in many domain-specific ways — that is still specific. Catching `Exception` is not.
        - "The third-party lib isn't well typed" → look up the actual exceptions it raises and catch those by name.
        - "It's safer this way" → it isn't. It hides bugs. Crashing loudly on an unexpected exception is the desired behavior.
        If you find yourself writing `except Exception` outside the two cases above, STOP and ask the user before proceeding.
    - **Do NOT add try/except speculatively.** Only add a `try`/`except` when (a) a specific exception type is documented or known to be raised by the call, AND (b) you have a concrete recovery path or a meaningful context to attach. Otherwise, let the error bubble up — that is the correct behavior, not a gap to fill.
    - **Use `try`/`finally` (without `except`) for required cleanup.** When a piece of code MUST run on the way out — releasing a resource, closing a tracer, tearing down a library, clearing per-context state on a manager — put it in a `finally` block (or a `with`/context manager). This is the correct way to guarantee cleanup on both the happy path and the error path: it does NOT suppress the exception, it just ensures cleanup happens before the exception propagates. Do not invent an `except Exception` just to attach cleanup; use `finally` instead. Conversely, do not move cleanup into the success path "to keep things simple" — if it must run on errors too, it belongs in `finally`.
    - When reviewing existing code, treat any `except Exception` you encounter as a bug to fix (replace it with the specific exception(s) actually raised, or remove the try/except entirely), not as a precedent to follow.
    - Always add `from exc` to the exception raise statements
    - Always write the error message as a variable before raising it, for cleaner error traces
   
   ```python
   try:
       self.models_manager.setup()
   except RoutingProfileLibraryNotFoundError as exc:
       msg = "The routing library could not be found, please call `pipelex init config` to create it"
       raise PipelexSetupError(msg) from exc
   ```

## Documentation

1. **Docstring Format**
   ```python
   def process_image(image_path: str, size: tuple[int, int]) -> bytes:
       """Process and resize an image.
       
       Args:
           image_path: Path to the source image
           size: Tuple of (width, height) for resizing
           
       Returns:
           Processed image as bytes
       """
       pass
   ```

2. **Class Documentation**
   ```python
   class ImageProcessor:
       """Handles image processing operations.
       
       Provides methods for resizing, converting, and optimizing images.
       """
   ```
