• Your Name's avatar
    Fix known template fallback and use_manual condition for GGUF models · fb8ec881
    Your Name authored
    - Directly set chat_template to known template names (qwen3, qwen, llama3, etc.)
      instead of trying to load non-existent HuggingFace tokenizers
    - Add use_manual condition to use manual formatting when chat_template is set
      but hf_tokenizer is None (applies to both generate_chat and generate_chat_stream)
    - This ensures GGUF models loaded from URLs with known templates use proper
      <|im_start|> formatting instead of failing on create_chat_completion
    fb8ec881
coderai 331 KB