-
Your Name authored
- Add model_backend_types dict to track backend for each model - Update set_default_model to accept backend_type parameter - Modify get_model_for_request to swap models on-demand when in ondemand mode - Unload current model from VRAM and load new model when request arrives for different model - Respect --backend flag when loading models on-demand - Only activates when no --loadall or --loadswap flag is specified
362b8452
| Name |
Last commit
|
Last update |
|---|---|---|
| .vscode | ||
| .gitignore | ||
| LICENSE.md | ||
| README.md | ||
| aaa | ||
| build.sh | ||
| coder | ||
| coderai | ||
| requirements-nvidia.txt | ||
| requirements-vulkan.txt | ||
| requirements.txt | ||
| requirements.txt~ |