# Logging Monitoring - Technical Overview

This document provides a detailed technical overview of the Logging Monitoring module.

## 1. Introduction and Purpose

The Logging Monitoring module serves as the centralized logging facility for the Codomyrmex project. Its core purpose is to provide a consistent, configurable, and easy-to-use interface for all other modules to record diagnostic information, operational events, warnings, and errors. It abstracts the underlying Python `logging` mechanism, offering a simplified setup and usage pattern tailored for the project.

Key responsibilities include:
- Singleton-like configuration: Ensuring logging is set up once for the entire application.
- Flexibility: Allowing configuration of log levels, formats, and output handlers (console, file) via environment variables.
- Standardization: Providing a standard way (`get_logger(__name__)`) for modules to obtain logger instances.
- Ease of Integration: Minimizing boilerplate code for logging in other modules.

## 2. Architecture

The module's architecture is centered around `logger_config.py`, which leverages Python's built-in `logging` module and the `python-dotenv` library.

- **Key Components/Sub-modules**:
  - `logger_config.py`:
    - `setup_logging()`: This is the primary configuration function. It performs the following:
        1. Uses `python-dotenv` to load environment variables from a `.env` file (expected in the project root).
        2. Reads `CODOMYRMEX_LOG_LEVEL`, `CODOMYRMEX_LOG_FILE`, and `CODOMYRMEX_LOG_FORMAT` environment variables, applying defaults if they are not set.
        3. Validates the log level and format strings.
        4. Configures the Python `logging` system using `logging.basicConfig()`. This typically configures the root logger.
        5. Sets up handlers: a `StreamHandler` for console output (to `sys.stdout`) and optionally a `FileHandler` if `CODOMYRMEX_LOG_FILE` is specified.
        6. Applies the chosen log format to these handlers.
        7. Includes a flag (`_logging_configured`) to ensure it only configures logging once (idempotency).
    - `get_logger(name: str) -> logging.Logger`:
A simple factory function that calls `logging.getLogger(name)`. This returns a logger instance that inherits the configuration applied by `setup_logging()`.
  - `__init__.py`:
    - Exports `setup_logging` and `get_logger` for easy public access.
    - Contains a module-level docstring explaining usage.

- **Data Flow**:
  1. Application starts and calls `setup_logging()` (typically from the main script).
  2. `setup_logging()` reads environment variables and configures the global Python logging settings.
  3. Any module in the application calls `get_logger(__name__)` to obtain its specific logger instance.
  4. When a logger method (e.g., `logger.info()`, `logger.error()`) is called, the log record is processed by the configured handlers (console, file) and formatted accordingly.

- **Core Algorithms/Logic**:
  - Relies on the standard Python `logging` module's hierarchical logger structure and handler/formatter mechanism.
  - Logic for parsing environment variables and applying defaults.
  - Error handling for file operations (e.g., if a log file cannot be opened).

- **External Dependencies**:
  - `python-dotenv`: Used by `setup_logging()` to load configurations from `.env` files. This dependency is expected to be managed at the project level (root `pyproject.toml`).
  - Standard Python modules: `logging`, `os`, `sys`.

```mermaid
graph TD
    A[Application Main Script] -- Calls --> S(codomyrmex.logging_monitoring.setup_logging);
    ENV[.env File] -- Loaded by --> S;
    S -- Configures --> PLS[Python Logging System (Root Logger, Handlers, Formatters)];
    
    M1[Module A] -- Calls --> GL(codomyrmex.logging_monitoring.get_logger);
    GL -- Returns Logger Instance --> M1;
    M1 -- Logs Message --> PLS;

    M2[Module B] -- Calls --> GL;
    GL -- Returns Logger Instance --> M2;
    M2 -- Logs Message --> PLS;

    PLS -- Writes to --> CONSOLE[Console Output];
    PLS -- Optionally Writes to --> LOGFILE[Log File];
```

## 3. Design Decisions and Rationale

- **Centralized Configuration (`setup_logging()`):** Ensures consistency and avoids conflicting logging setups from different parts of the application. Python's `logging` module behaves globally, so a single point of configuration is crucial.
- **Environment Variable Driven Configuration:** Provides flexibility for different environments (development, testing, production) without code changes. Aligns with common practices for application configuration (12-factor app principles).
- **Use of `.env` files (via `python-dotenv`):** Simplifies local development by allowing developers to keep configurations out of version control. This is facilitated by the `environment_setup` module's guidance.
- **`get_logger(__name__)` pattern:** Standard Python logging practice, which helps in tracing log messages back to their source module.
- **Defaulting to `sys.stdout` for console:** Standard practice for application logs, allowing easy redirection if needed at the OS level.
- **Idempotent `setup_logging()`:** Prevents accidental re-configuration and potential issues.
- **`logging.basicConfig(force=True)`:** Used in `setup_logging` to ensure that our configuration takes precedence even if some other library or part of the code tried to call `basicConfig` earlier without handlers. This makes the setup more robust in complex projects.

## 4. Data Models

- The module does not define complex internal data structures for its public API. It primarily deals with:
  - Configuration strings from environment variables.
  - Standard `logging.Logger`, `logging.Handler`, and `logging.Formatter` objects from the Python `logging` module.

## 5. Configuration

Configuration is primarily through the following environment variables, typically set in a `.env` file at the project root:

- `CODOMYRMEX_LOG_LEVEL`: (Default: `INFO`) Sets the minimum severity level for messages to be logged. Accepted values: `DEBUG`, `INFO`, `WARNING`, `ERROR`, `CRITICAL`.
- `CODOMYRMEX_LOG_FILE`: (Default: None - console only) Specifies the path to a log file. If set, logs will be written to this file in append mode.
- `CODOMYRMEX_LOG_OUTPUT_TYPE`: (Default: `TEXT`) Specifies the output format of the logs. Can be `"TEXT"` or `"JSON"`.
- `CODOMYRMEX_LOG_FORMAT`: (Default: `"%(asctime)s - %(name)s - %(levelname)s - %(message)s"`)
  A Python logging format string, used when `CODOMYRMEX_LOG_OUTPUT_TYPE` is `"TEXT"`. Can also be set to the special value `"DETAILED"` to use a more verbose predefined format (`"%(asctime)s - %(name)s - %(levelname)s - %(module)s:%(funcName)s:%(lineno)d - %(message)s"`). This setting is ignored if `CODOMYRMEX_LOG_OUTPUT_TYPE` is `"JSON"`.

## 6. Scalability and Performance

- **Performance**: Python's `logging` module is generally efficient. For very high-performance scenarios, the overhead of logging, especially to files or with complex formatting, can become a factor. Disabling or raising the log level (e.g., to `ERROR` or `CRITICAL`) in production can mitigate this.
    - The current implementation does not use asynchronous logging, which could be a future enhancement if performance under very high log volume becomes critical.
- **Scalability**: The module itself is stateless after configuration. Scalability concerns are more related to the log management infrastructure (e.g., log file rotation, centralized log aggregation systems like ELK stack or Splunk) if the application generates vast amounts of logs, which are outside the scope of this specific module.

## 7. Security Aspects

- **Log Content**: Developers should be mindful not to log sensitive information (passwords, API keys, personal data) unless absolutely necessary and properly secured (e.g., if logs are encrypted or access-controlled). This module does not add any specific filtering for sensitive data.
  When using JSON logging (via the custom `JsonFormatter` in `logger_config.py`), be aware that all fields passed in the `extra` dictionary to logger calls (e.g. `logger.info("message", extra={"key": "value"})`) will be included as top-level fields in the JSON output. Review logged data to ensure no sensitive information is unintentionally exposed in these `extra` fields.
- **File Permissions**: If logging to a file (`CODOMYRMEX_LOG_FILE`), the application must have appropriate write permissions for the specified path. The module itself does not manage file permissions or log rotation.
- **Log Injection**: While log messages are typically strings, if user-supplied data is logged directly without sanitization, there could be a risk of log injection (e.g., forging log entries with newline characters). This is a general application concern rather than specific to this module, but users should be aware.

## 8. Future Development / Roadmap

- **Advanced Structured Logging**: Further enhancements to JSON logging, such as more configurable fields or integration with specific structured logging libraries if needed.
- **Asynchronous Logging**: For high-throughput applications, an option to perform logging I/O operations asynchronously to minimize impact on application performance.
- **Log Rotation Configuration**: Basic built-in support for log rotation if `CODOMYRMEX_LOG_FILE` is used, though often this is handled by external tools (like `logrotate` on Linux).
- **Configuration from a file**: Allow loading configuration from a dedicated logging config file (e.g., YAML or JSON) in addition to environment variables.
- **More granular control over third-party library logging**: Potentially a mechanism to easily set log levels for specific external libraries via configuration. 
## Navigation Links

- **Parent**: [Project Overview](../README.md)
- **Module Index**: [All Agents](../../AGENTS.md)
- **Documentation**: [Reference Guides](../../README.md)
- **Home**: [Root README](../../README.md)
