- Now can specify template directly: --hf-chat-template "model:template"
- Updated check_hf_chat_template to return tuple (should_use, template_name)
- Updated _load_huggingface_tokenizer to accept template_name parameter
- Updated README with new syntax and template examples

- Added 'auto' as a valid value for --hf-chat-template
- When --hf-chat-template auto is used, it auto-detects and applies HF template to all models
- Updated README with new syntax

- Changed --hf-chat-template from boolean to action=append
- Added check_hf_chat_template() function for model-specific checking
- Updated _finalize_chat_template_detection to use new function
- Updated README with new syntax

- Added --hf-chat-template CLI flag to use transformers apply_chat_template
- Added _load_huggingface_tokenizer() to load HF tokenizer for GGUF models
- Added _format_messages_hf() method for HF chat template formatting
- Updated generate_chat and generate_chat_stream to use HF tokenizer when available
- Updated format_messages to check for HF tokenizer first
- Added documentation in README.md

- Added --reply-filters CLI flag to make content filtering optional
- Supports comma-separated values: --reply-filters malformed,tool_calls
- Supports model-specific filters: --reply-filters text:malformed --reply-filters image:tool_calls
- Supports specific model names: --reply-filters text:llama-3.1:malformed
- Added check_reply_filter() and check_single_filter() helper functions
- Updated stream_chat_response and generate_chat_response to use new filtering
- Updated ToolCallParser._filter_malformed_content for conditional filtering
- Added documentation in README.md

- Give more time for Vulkan memory to be freed after unloading image models

- Add garbage collection and torch.cuda.empty_cache() after unloading image models
- Add a small delay to allow VRAM to be freed before loading new model
- This should help prevent OOM errors when switching between image and text models

- Remove stripping in strip_tool_calls_from_content function
- Whitespace (spaces, newlines) are valid content and should be preserved

- When reloading a default model that was loaded from a URL,
  check for cached file path and use it instead of the URL

- Replace NaN and Inf values with valid values before saving
- Clip image values to valid range [0, 1] to prevent black images

- When 'default' model is requested but not loaded (was unloaded for image model),
  the code now tries to reload the default model
- Cleanup image models first to free VRAM, then reload the text model

- Add --image-cpu-offload CLI flag for explicit sequential CPU offload
- Enable sequential CPU offload only on 3rd OOM retry or when --image-cpu-offload is set

- Skip URLs when listing the default model in list_models()
- This prevents download URLs from appearing in available models list

- Enable sequential CPU offload if --offload-strategy or --offload-dir is specified
- Add retry logic: on OOM, retry with attention_slicing, then with sequential_offload
- Clear CUDA cache between retry attempts

- Add --image-precision with choices: bf16, f32, f16, f8
- bf16 recommended for modern GPUs (RTX 30/40 series) to avoid NaN issues
- Enable VAE tiling for diffusers when --vae-tiling is specified

- Changed torch_dtype from float16 (when CUDA available) to float32
- This prevents NaN/Infinity values in image output that cause black/corrupted images
- FP16 can cause numerical overflow on some models like SDXL

- Add 'steps' parameter to ImageGenerationRequest (overrides quality-based default)
- Add 'guidance_scale' parameter to ImageGenerationRequest (overrides CLI --image-cfg-scale)
- Use request values in diffusers pipeline call

- Import time module inside try block with alias to avoid UnboundLocalError
- This prevents Python's exception handling from affecting variable scope

- Remove duplicate 'image' entry in list_models()
- Remove vision: alias (user doesn't want it)
- Skip URLs in loaded models listing (they're download sources)
- Add full traceback to diffusers error for debugging

- Recursively scan huggingface cache directory (hub/, xet/, etc.)
- Also fix remove-model to search recursively in huggingface cache

- Add get_all_cache_dirs() to find GGUF, HuggingFace, and Diffusers caches
- Update --list-cached-models to show all cache locations
- Update --remove-all-models to clean all cache directories
- Update --remove-model to search across all caches
- Add better error handling for diffusers image extraction

- When --file-path is set and --url is 'auto', use the Host header
  from the request (what the client used to connect) instead of
  the client's IP address
- This ensures the returned URL points to the correct server

- Remove debug print for empty filtered chunks
- Fix strip_tool_calls_from_content to preserve whitespace-only chunks ('\n\n', ' ')
- These whitespace characters are essential for proper message composition

- Changed context arguments to use action='append' allowing multiple values
- Added get_ctx_by_index() helper function for index-based context retrieval
- Updated text, audio, and image model loading to use indexed context values
- Users can now specify different context sizes per model

- Cleanup image models before loading text models to prevent OOM errors
- Applied to both text model loading paths in get_model_for_request

- Cleanup any existing models (text, audio, etc.) from VRAM before loading
  image models to prevent out of memory errors when switching between model types
- Applied to both diffusers and stable-diffusion-cpp loading paths

- Add fallback to use configured --image-model when unknown model name is sent
- Cache dynamically loaded StableDiffusion models for reuse across requests
- Always check cache (not just in loadall mode) so ondemand mode reuses models

- Detect GGUF models and skip diffusers, use stable-diffusion-cpp instead
- Add HuggingFace model ID resolution for GGUF files
- Add support for VAE, LLM, T5XXL paths from CLI args
- Add clip_on_cpu support for VRAM savings
- Use all available CPU cores instead of hardcoded 4 threads

When loading image models dynamically (in ondemand mode), the code now:
1. Checks if model URL is cached
2. If not cached, downloads the model before loading

This fixes the 'Could not resolve sd.cpp model path' error when using
image models without --loadall or --loadswap flags.

- Set llama-cpp-python verbose flag to match debug mode
- Remove n_gpu_layers from stable_diffusion_cpp (not supported)

Fix: verbose=True when debug flag set, fix stable_diffusion_cpp n_gpu_layers bug, remove capability pre-check for image generation

User will experiment with Vulkan environment variables from launching script.
Keep only CUDA_VISIBLE_DEVICES setting for now.

For stable-diffusion-cpp-python:
- GGML_VK_VISIBLE_DEVICES=
- GGML_VULKAN_DEVICE=

For llama-cpp-python (additional):
- VK_ICD_FILENAMES=/dev/null
- VK_DRIVER_FILES=/dev/null
- VK_LOADER_DRIVERS_DISABLE=*
- VK_LOADER_LAYERS_DISABLE=~all~

All variables are restored on cleanup for subsequent Vulkan models.

- Use VK_ICD_FILENAMES=/dev/null to disable Vulkan ICD and force CUDA
- This is the correct variable for llama.cpp to disable Vulkan
- Restore VK_ICD_FILENAMES on cleanup for subsequent Vulkan models

- Set GGML_DISABLE_VULKAN=1 and GGML_VULKAN_DEVICE='' before loading model
- These must be set before llama_cpp import since it reads them at init
- Restore Vulkan settings on cleanup so subsequent Vulkan models work
- Addresses issue where GGUF models ran on CPU instead of CUDA with --backend nvidia

- Make cleanup() method handle StableDiffusion and other non-ModelManager objects
- Add try-except in on-demand swap to handle cleanup failures gracefully
- Check if cleanup method exists before calling

- Add model_backend_types dict to track backend for each model
- Update set_default_model to accept backend_type parameter
- Modify get_model_for_request to swap models on-demand when in ondemand mode
- Unload current model from VRAM and load new model when request arrives for different model
- Respect --backend flag when loading models on-demand
- Only activates when no --loadall or --loadswap flag is specified