- 17 Mar, 2026 4 commits
- 16 Mar, 2026 36 commits
-
-
Your Name authored
-
Your Name authored
- Added --force-reasoning with choices: 'stop', 'inject', 'both' (default) - Add model-family detection for reasoning stop tokens - Get appropriate stop tokens for Qwen, DeepSeek, Llama3, Mistral, Gemma, Hermes/Yi - Add system prompt injection for forcing reasoning on non-native models - Add extract_reasoning_content() function to parsers for extracting thinking tags
-
Your Name authored
- Added --force-reasoning argument to enable reasoning mode for models that support it (Qwen3, DeepSeek R1, etc.) - Modified chat_completions endpoint to check both API parameter enable_thinking and CLI flag force_reasoning - When either is true, injects agentic template to enable thinking
-
Your Name authored
- Add enable_thinking parameter to ChatCompletionRequest - When enable_thinking=True, inject agentic system prompt to force thinking/reasoning - Uses AgenticTemplateManager to inject thought tags for supported models
-
Your Name authored
-
Your Name authored
- Add --force-reasoning CLI flag to force thinking mode for models like qwen3 coder - Add check_force_reasoning() function to determine if reasoning should be forced - Modify QwenParser to extract thinking/reasoning content instead of stripping it - Add reasoning field to response message in non-streaming chat completions - Prepend reasoning content to generated text in streaming responses - Update OpenAIFormatter to include reasoning in response when available
-
Your Name authored
-
Your Name authored
-
Your Name authored
-
Your Name authored
- Simplify OpenAIFormatter by using litellm's ModelResponse and ChatCompletionChunk directly - Add fallback support for when litellm is not available or fails - Maintain compatibility with existing API - Remove redundant format_litellm_full and format_litellm_chunk methods
-
Your Name authored
The issue was caused by importing StreamingResponse and JSONResponse inside the chat_completions function. In Python, when you have an import statement anywhere inside a function, it creates a local variable for that name throughout the entire function scope. This caused the code in the original implementation path to fail because Python saw StreamingResponse as an unassigned local variable. Fix: Move StreamingResponse and JSONResponse imports to module level and remove redundant imports from inside the function.
-
Your Name authored
-
Your Name authored
The litellm library doesn't export Delta, Choices, etc. directly. Rewrote the formatter to build response dictionaries directly.
-
Your Name authored
-
Your Name authored
- Remove the --parser argument and litellm backend handling code - Add OpenAIFormatter class in codai/models/parsers.py for final response sanitization - Integrate formatter into both streaming and non-streaming response paths - Use litellm's ModelResponse and ChatCompletionChunk for proper OpenAI format
-
Your Name authored
-
Your Name authored
- Changed model name format from openai/... to coderai/... - This ensures the model is correctly identified as coderai/TeichAI/Qwen3-8B-...
-
Your Name authored
- Use litellm.openai instead of 'openai' string for custom_handler - This ensures proper registration of the coderai provider with litellm
-
Your Name authored
- Changed model name format from openai/... to coderai/... - Added litellm.custom_provider_map to map coderai to openai handler - This allows litellm to use its internal HTTP handler for custom providers - Example: TeichAI/Qwen3-8B-... now becomes coderai/TeichAI/Qwen3-8B-...
-
Your Name authored
- Instead of defaulting to 'huggingface' for org/model paths, now preserves the original org name as the provider - Example: TeichAI/Qwen3-8B-... now becomes openai/TeichAI/Qwen3-8B-... instead of openai/huggingface/TeichAI/Qwen3-8B-...
-
Your Name authored
- Add logic to set api_base to server's own URL for non-Ollama models - Extract host/port from request headers (X-Forwarded-For, Host header) - Determine protocol (http/https) based on global_args - Include debug output showing the determined api_base - This ensures litellm can properly connect to local server when using litellm backend with local models
-
Your Name authored
-
Your Name authored
-
Your Name authored
-
Your Name authored
-
Your Name authored
-
Your Name authored
-
Your Name authored
-
Your Name authored
-
Your Name authored
- When using HuggingFace inference endpoints, set api_key to None to avoid auth errors
-
Your Name authored
- When model starts with 'ollama:', construct api_base from request host and port - api_base is now passed to LiteLLMBackend for local connections
-
Your Name authored
- Don't check environment for OPENAI_API_KEY - Use fake key directly in LiteLLMBackend if no key passed
-
Your Name authored
- If no API key is provided in request, use a fake key to allow litellm to proceed - Check both request body and Authorization header for API key
-
Your Name authored
- Add tool_parser parameter to litellm backend calls in coderai endpoint - ModelParserAdapter now passed to both streaming and non-streaming calls - Enables model-specific tool call parsing for external models via litellm
-
Your Name authored
-
Your Name authored
-