Add configurable LiteLLM logging control 🙉

config.yaml.example

@@ -62,3 +62,21 @@ chat_model_n_ctx: 32768
 
 # Number of layers to offload to GPU. If -1, all layers are offloaded.
 chat_model_n_gpu_layers: -1
+
+# External logger configuration
+external_loggers:
+  litellm: false      # Enable/disable LiteLLM logging (includes LiteLLM Proxy, Router, and core)
+  sqlalchemy: false   # Enable/disable SQLAlchemy logging
+  uvicorn.error: false # Enable/disable Uvicorn error logging
+  aiosqlite: false    # Enable/disable aiosqlite logging
+
+# Note: External logger configuration can be overridden by:
+# 1. Environment variables:
+#    CODEGATE_ENABLE_LITELLM=true     # Controls all LiteLLM loggers
+#    CODEGATE_ENABLE_SQLALCHEMY=true
+#    CODEGATE_ENABLE_UVICORN_ERROR=true
+#    CODEGATE_ENABLE_AIOSQLITE=true
+# 2. CLI arguments:
+#    --enable-litellm    # Enables LiteLLM logging
+#    --enable-sqlalchemy # Enables SQLAlchemy logging
+#    etc.

docs/cli.md

@@ -142,6 +142,12 @@ codegate generate-certs [OPTIONS]
   - Case-insensitive
   - Overrides configuration file and environment variables
 
+- `--enable-litellm`: Enable LiteLLM logging
+  - Optional flag
+  - Default: false
+  - Enables logging for LiteLLM Proxy, Router, and core components
+  - Overrides configuration file and environment variables
+
 ## Error handling
 
 The CLI provides user-friendly error messages for:
@@ -217,4 +223,4 @@ Generate certificates with default settings:
 codegate generate-certs
 ```
 
-<!-- markdownlint-configure-file { "no-duplicate-heading": { "siblings_only": true } } -->
+<!-- markdownlint-configure-file { "no-duplicate-heading": { "siblings_only": true } } -->

docs/configuration.md

@@ -36,6 +36,13 @@ Values from higher-priority sources take precedence over lower-priority values.
   - Server certificate: `"server.crt"`
   - Server private key: `"server.key"`
 
+- External logger configuration
+  - external_loggers:
+   - litellm: `false`
+   - sqlalchemy: `false`
+   - uvicorn.error: `false`
+   - aiosqlite: `false`
+
 ## Configuration methods
 
 ### Configuration file
@@ -85,6 +92,10 @@ Environment variables are automatically loaded with these mappings:
 - `CODEGATE_CA_KEY`: CA key file name
 - `CODEGATE_SERVER_CERT`: server certificate file name
 - `CODEGATE_SERVER_KEY`: server key file name
+- `CODEGATE_ENABLE_LITELLM`: enable LiteLLM logging
+- `CODEGATE_ENABLE_SQLALCHEMY`: enable SQLAlchemy logging
+- `CODEGATE_ENABLE_UVICORN_ERROR`: enable Uvicorn error logging
+- `CODEGATE_ENABLE_AIOSQLITE`: enable aiosqlite logging
 
 ```python
 config = Config.from_env()
@@ -200,6 +211,20 @@ Available log formats (case-insensitive):
 - `JSON`
 - `TEXT`
 
+### External logger configuration
+
+External loggers can be configured in several ways:
+
+1. Configuration file:
+
+   ```yaml
+   external_loggers:
+     litellm: false
+     sqlalchemy: false
+     uvicorn.error: false
+     aiosqlite: false
+   ```
+
 ### Prompts configuration
 
 Prompts can be configured in several ways:

docs/logging.md

@@ -10,6 +10,39 @@ Logs are automatically routed based on their level:
 - **stdout**: INFO and DEBUG messages
 - **stderr**: ERROR, CRITICAL, and WARNING messages
 
+## External Logger Configuration
+
+CodeGate provides control over external loggers through configuration:
+
+### LiteLLM Logging
+
+LiteLLM logging can be controlled through:
+
+1. CLI flag:
+   ```bash
+   codegate serve --enable-litellm
+   ```
+
+2. Environment variable:
+   ```bash
+   export CODEGATE_ENABLE_LITELLM=true
+   ```
+
+3. Configuration file:
+   ```yaml
+   external_loggers:
+     litellm: true      # Enable/disable LiteLLM logging (includes LiteLLM Proxy, Router, and core)
+     sqlalchemy: false  # Enable/disable SQLAlchemy logging
+     uvicorn.error: false # Enable/disable Uvicorn error logging
+     aiosqlite: false   # Enable/disable aiosqlite logging
+   ```
+
+The configuration follows the priority order:
+1. CLI arguments (highest priority)
+2. Environment variables
+3. Config file
+4. Default values (lowest priority)
+
 ## Log formats
 
 ### JSON format
@@ -45,6 +78,7 @@ YYYY-MM-DDThh:mm:ss.mmmZ - LEVEL - NAME - MESSAGE
 - **Exception support**: full exception and stack trace integration
 - **Dual output**: separate handlers for error and non-error logs
 - **Configurable levels**: support for ERROR, WARNING, INFO, and DEBUG levels
+- **External logger control**: fine-grained control over third-party logging
 
 ## Usage examples
 
@@ -89,21 +123,24 @@ The logging system can be configured through:
 1. CLI arguments:
 
    ```bash
-   codegate serve --log-level DEBUG --log-format TEXT
+   codegate serve --log-level DEBUG --log-format TEXT --enable-litellm # to enable LiteLLM debug
    ```
 
 2. Environment variables:
 
    ```bash
-   export APP_LOG_LEVEL=DEBUG
+   export CODEGATE_APP_LOG_LEVEL=DEBUG
    export CODEGATE_LOG_FORMAT=TEXT
+   export CODEGATE_ENABLE_LITELLM=true
    ```
 
 3. Configuration file:
 
    ```yaml
    log_level: DEBUG
    log_format: TEXT
+   external_loggers:
+     litellm: true
    ```
 
 ## Best practices
@@ -129,3 +166,7 @@ The logging system can be configured through:
    better log aggregation and analysis.
 
 4. Enable `DEBUG` level logging during development for maximum visibility.
+
+5. Configure external loggers based on your needs:
+   - Enable LiteLLM logging when debugging LLM-related issues
+   - Keep external loggers disabled in production unless needed for troubleshooting

src/codegate/cli.py

@@ -4,7 +4,7 @@
 import signal
 import sys
 from pathlib import Path
-from typing import Dict, Optional
+from typing import Optional
 
 import click
 import structlog
@@ -246,6 +246,12 @@ def show_prompts(prompts: Optional[Path]) -> None:
     default=None,
     help="Path to the vector SQLite database file (default: ./sqlite_data/vectordb.db)",
 )
+@click.option(
+    "--enable-litellm",
+    is_flag=True,
+    default=False,
+    help="Enable LiteLLM logging (includes LiteLLM Proxy, Router, and core)",
+)
 def serve(
     port: Optional[int],
     proxy_port: Optional[int],
@@ -267,11 +273,12 @@ def serve(
     ca_key: Optional[str],
     server_cert: Optional[str],
     server_key: Optional[str],
+    enable_litellm: bool,
 ) -> None:
     """Start the codegate server."""
     try:
-        # Create provider URLs dict from CLI options
-        cli_provider_urls: Dict[str, str] = {}
+        # Create provider URLs dictionary from CLI arguments
+        cli_provider_urls = {}
         if vllm_url:
             cli_provider_urls["vllm"] = vllm_url
         if openai_url:
@@ -281,6 +288,9 @@ def serve(
         if ollama_url:
             cli_provider_urls["ollama"] = ollama_url
 
+        # Create external loggers dictionary from CLI arguments
+        cli_external_loggers = {"litellm": enable_litellm}
+
         # Load configuration with priority resolution
         cfg = Config.load(
             config_path=config,
@@ -290,7 +300,8 @@ def serve(
             cli_host=host,
             cli_log_level=log_level,
             cli_log_format=log_format,
-            cli_provider_urls=cli_provider_urls,
+            cli_provider_urls=cli_provider_urls if cli_provider_urls else None,
+            cli_external_loggers=cli_external_loggers,
             model_base_path=model_base_path,
             embedding_model=embedding_model,
             certs_dir=certs_dir,
@@ -302,8 +313,8 @@ def serve(
             vec_db_path=vec_db_path,
         )
 
-        # Set up logging first
-        setup_logging(cfg.log_level, cfg.log_format)
+        # Initialize logging
+        setup_logging(cfg.log_level, cfg.log_format, cfg.external_loggers)
         logger = structlog.get_logger("codegate").bind(origin="cli")
 
         init_db_sync(cfg.db_path)
@@ -505,7 +516,7 @@ def generate_certs(
         cli_log_level=log_level,
         cli_log_format=log_format,
     )
-    setup_logging(cfg.log_level, cfg.log_format)
+    setup_logging(cfg.log_level, cfg.log_format, cfg.external_loggers)
     logger = structlog.get_logger("codegate").bind(origin="cli")
 
     ca = CertificateAuthority.get_instance()

src/codegate/codegate_logging.py

@@ -48,6 +48,47 @@ def _missing_(cls, value: str) -> Optional["LogFormat"]:
             )
 
 
+# Define all LiteLLM logger names
+LITELLM_LOGGERS = ["LiteLLM Proxy", "LiteLLM Router", "LiteLLM"]
+
+
+def configure_litellm_logging(enabled: bool = False, level: LogLevel = LogLevel.INFO) -> None:
+    """Configure LiteLLM logging.
+
+    Args:
+        enabled: Whether to enable LiteLLM logging
+        level: Log level to use if enabled
+    """
+    # Configure the main litellm logger
+    logger = logging.getLogger("litellm")
+    logger.disabled = not enabled
+    if not enabled:
+        logger.setLevel(logging.CRITICAL + 1)  # Effectively disables all logging
+    else:
+        logger.setLevel(getattr(logging, level.value))
+        logger.propagate = False
+        # Clear any existing handlers
+        logger.handlers.clear()
+        # Add a handler to ensure logs are properly routed
+        handler = logging.StreamHandler()
+        handler.setLevel(getattr(logging, level.value))
+        logger.addHandler(handler)
+
+    # Also configure the specific LiteLLM loggers
+    for logger_name in LITELLM_LOGGERS:
+        logger = logging.getLogger(logger_name)
+        logger.disabled = not enabled
+        if not enabled:
+            logger.setLevel(logging.CRITICAL + 1)
+        else:
+            logger.setLevel(getattr(logging, level.value))
+            logger.propagate = False
+            logger.handlers.clear()
+            handler = logging.StreamHandler()
+            handler.setLevel(getattr(logging, level.value))
+            logger.addHandler(handler)
+
+
 def add_origin(logger, log_method, event_dict):
     # Add 'origin' if it's bound to the logger but not explicitly in the event dict
     if "origin" not in event_dict and hasattr(logger, "_context"):
@@ -58,13 +99,17 @@ def add_origin(logger, log_method, event_dict):
 
 
 def setup_logging(
-    log_level: Optional[LogLevel] = None, log_format: Optional[LogFormat] = None
+    log_level: Optional[LogLevel] = None,
+    log_format: Optional[LogFormat] = None,
+    external_loggers: Optional[Dict[str, bool]] = None,
 ) -> logging.Logger:
     """Configure the logging system.
 
     Args:
         log_level: The logging level to use. Defaults to INFO if not specified.
         log_format: The log format to use. Defaults to JSON if not specified.
+        external_loggers: Dictionary of external logger names and whether they should be enabled.
+                        e.g. {"litellm": False, "sqlalchemy": False, "uvicorn.error": False}
 
     This configures two handlers:
     - stderr_handler: For ERROR, CRITICAL, and WARNING messages
@@ -74,6 +119,16 @@ def setup_logging(
         log_level = LogLevel.INFO
     if log_format is None:
         log_format = LogFormat.JSON
+    if external_loggers is None:
+        external_loggers = {
+            "litellm": False,
+            "sqlalchemy": False,
+            "uvicorn.error": False,
+            "aiosqlite": False,
+        }
+
+    # Configure LiteLLM logging based on external_loggers setting
+    configure_litellm_logging(enabled=external_loggers.get("litellm", False), level=log_level)
 
     # The configuration was taken from structlog documentation
     # https://www.structlog.org/en/stable/standard-library.html