Created codai/models/cache/__init__.py with:
- get_model_cache_dir()
- get_all_cache_dirs()
- get_cached_model_path()
- is_huggingface_model_id()
- download_huggingface_model()
- download_model()
- list_cached_models()
- remove_cached_model()
- remove_all_cached_models()

This extracts the cache-related functionality into a separate module.

- Rename codai/litellm_backend.py to codai/openai/litellm.py
- Create codai/openai/__init__.py
- Update imports in coderai and codai/__init__.py

- Add litellm to requirements.txt
- Add --parser CLI arg (auto/litellm, default auto)
- Create codai/litellm_backend.py module with:
  - LiteLLMBackend class for standardized responses
  - Rate limit headers (x-ratelimit-remaining-tokens, x-ratelimit-limit-tokens)
  - Qwen tool-call resilience (parse <tool> and <tool_call> tags)
  - Error handling with litellm exception mapping
- Update chat completions endpoint to use litellm when --parser litellm
- Update codai/__init__.py to export litellm components

- Added litellm>=1.40.0 to requirements.txt
- Added --parser argument (auto/litellm, default auto)

Note: Full litellm integration requires significant refactoring of the
chat completion endpoints to use litellm.completion() for standardized
responses, adding rate limit headers, and error handling.

QwenParser:
- Add repetition guard to handle looping models
- Improve flexible tag matching for tool/tool_call/function_call
- Add JSON recovery for unclosed JSON
- Add circuit breaker after first valid call
- Support <call=name> in coder style fallback

API:
- Add repeat_penalty parameter to ChatCompletionRequest
- Add repeat_penalty parameter to CompletionRequest

- Added pre-cleaning for thinking/special tokens
- Unified tag matching for both <tool> and <tool_call>
- Added markdown code block stripping inside tags
- Added lazy JSON parsing fallback
- Added _parse_coder_style() and _relaxed_val() helper methods

- Added _clean_json_string() method to BaseParser for cleaning JSON strings
- Updated QwenParser.parse() with 3-step parsing strategy:
  1. Qwen format: <tool=func_name>...</tool>
  2. JSON format with flexible tag matching
  3. Fallback coder style with parameter tags
- Fixed syntax issues in the module

- Handle both dict and pydantic model formats for tools
- Add try/except around tool conversion and extraction
- More robust error handling to prevent 500 errors

- Move model_parser.py into codai/ directory
- Add __init__.py to make it a proper Python module
- Create ModelParserAdapter class to wrap ModelParserDispatcher
- Replace ToolCallParser() with ModelParserAdapter() in 4 locations
- Update import to use 'from codai import ModelParserDispatcher'

This enables model-specific tool call parsing for Qwen, DeepSeek,
Llama, Mistral, Claude, Command R, Gemma, Grok, and Phi models.

- Add Qwen-specific tool call parsing in ToolCallParser
- Support for Instruct-style: <tool_call>{JSON}</tool_call>
- Support for Coder-style: <tool_call><function=name><parameter=k>v</parameter></function></tool_call>
- Add model_name attribute to ToolCallParser for model-specific parsing
- Update ModelManager.load_model to set model name on tool parser
- Fix duplicate method definitions in ToolCallParser class

- Pass response_format to llama.cpp create_chat_completion
- Supports {'type': 'json_object'} for JSON output mode
- Applied to both streaming and non-streaming responses

When chat_template is 'default' (embedded GGUF template), allow llama.cpp
to handle tool messages via create_chat_completion instead of forcing
manual formatting. This allows the GGUF's native template to be used
even with tools.

When chat_template is 'default', it means llama.cpp detected an embedded
template in the GGUF model. Don't fall back to manual formatting - instead
let llama.cpp's create_chat_completion use its internal template handling.

- Add provider object with provider_name and provider_id
- Add system_fingerprint (null)
- Add logprobs in choices (null)
- Add native_finish_reason in choices
- Add usage.prompt_tokens_details with cached_tokens and audio_tokens
- Add usage.completion_tokens_details with reasoning_tokens and audio_tokens
- Apply to both streaming and non-streaming responses

- Directly set chat_template to known template names (qwen3, qwen, llama3, etc.)
  instead of trying to load non-existent HuggingFace tokenizers
- Add use_manual condition to use manual formatting when chat_template is set
  but hf_tokenizer is None (applies to both generate_chat and generate_chat_stream)
- This ensures GGUF models loaded from URLs with known templates use proper
  <|im_start|> formatting instead of failing on create_chat_completion

When HF tokenizer loading fails, try known template names based on model name:
- Qwen models: try qwen3, qwen templates
- Llama models: try llama3, llama templates
- Phi models: try phi template
- Mistral models: try mistral template

This helps when the tokenizer can't be loaded but we know the model family.

The model_backend_types attribute was not being initialized properly due to
incorrect indentation, causing 'MultiModelManager' object has no attribute
'model_backend_types' error when trying to load models on-demand.

- Add uppercase quantization suffixes (_Q4_K_M, etc.) to handle cached GGUF filenames
- Add progressive fallback to try shorter model names when tokenizer loading fails
- Example: Qwen3.5-27B-Uncensored-HauhauCS-Aggressive -> try Qwen3.5-27B-Uncensored -> Qwen3.5-27B -> Qwen3.5 -> Qwen
- Add warning when all tokenizer loading attempts fail (will use manual formatting instead)

- Added _aggressive_vram_cleanup method to properly clear VRAM
- Moves model to CPU before deletion
- Deletes pipeline, vae, text_encoder, tokenizer explicitly
- Multiple rounds of gc.collect()
- Uses torch.cuda.synchronize() before clearing cache
- Increased delay to 5 seconds after cleanup

- Now can specify template directly: --hf-chat-template "model:template"
- Updated check_hf_chat_template to return tuple (should_use, template_name)
- Updated _load_huggingface_tokenizer to accept template_name parameter
- Updated README with new syntax and template examples

- Added 'auto' as a valid value for --hf-chat-template
- When --hf-chat-template auto is used, it auto-detects and applies HF template to all models
- Updated README with new syntax

- Changed --hf-chat-template from boolean to action=append
- Added check_hf_chat_template() function for model-specific checking
- Updated _finalize_chat_template_detection to use new function
- Updated README with new syntax

- Added --hf-chat-template CLI flag to use transformers apply_chat_template
- Added _load_huggingface_tokenizer() to load HF tokenizer for GGUF models
- Added _format_messages_hf() method for HF chat template formatting
- Updated generate_chat and generate_chat_stream to use HF tokenizer when available
- Updated format_messages to check for HF tokenizer first
- Added documentation in README.md