ComfyUI_Simple_Qwen3

Correct installation of llama-cpp-python:

Qwen3 support hasn't been added to the standard library, llama-cpp-python, which is downloaded via pip install llama-cpp-python - this didn't work. The standard version llama-cpp-python hasn't been updated for a long time. llama-cpp-python 0.3.16 last commit on Aug 15, 2025 and it doesn't support qwen3.

Check the version number of llama-cpp-python from JamePeng you're using: - Version 0.3.17 or latest supports qwen3-VL. - Version 0.3.30 or latest supports qwen3.5. - Version 0.3.35 or latest supports gemma4.

Variant 2 - Build from source code (I recommend this variant)

<details>

<summary>Installing software before compilation</summary>

1. Check that you have CUDA Toolkit installed. For example: C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v13.0 - Try installing: https://developer.nvidia.com/cuda-downloads - Check that the PATH in Environment Variables includes the CUDA Toolkit bin folder (For example: C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v13.0\bin). - After installing CUDA Toolkit, restart your computer.

2. Check that the NVIDIA Driver and CUDA Toolkit versions match (the driver can and most often should be newer than the CUDA Toolkit version): Run command in CMD nvidia-smi.

3. Check that you have Visual C++ Redistributable installed. - Try installing: https://learn.microsoft.com/en-us/cpp/windows/latest-supported-vc-redist?view=msvc-170 - Install both versions (x86 and x64).

4. Check that you have Visual Studio 2022 installed. - Install Visual Studio 2022. - Install the following packages (they will not be installed by default): ☑ Desktop development with C++ (in Workloads tab).

☑ MSVC v143 - VS 2022 C++ x64/x86 build tools (in Individual components tab).

☑ Windows 10/11 SDK (in Individual components tab).

☑ CMake tools for Visual Studio (in Individual components tab).

- The environment variable for MSVC is not added to the PATH by default. Run this command every time in your terminal before compiling: call "C:\Program Files\Microsoft Visual Studio\2022\Community\VC\Auxiliary\Build\vcvars64.bat"

5. If you use python_embeded for Comfy-UI, may need to add missing libs folders: python_embeded\include, python_embeded\libs (Not Lib\site-packages), python_embeded\DLLs: - From here https://github.com/astral-sh/python-build-standalone/releases download Python appropriate version (for example cpython-3.13.11+20251217-x86_64-pc-windows-msvc-install_only.tar.gz) - unzip and copy the necessary folders to python_embeded. </details>

<details>

<summary>Build llama-cpp-python from source code</summary>

1. Clone the repositories using Git: - https://github.com/JamePeng/llama-cpp-python - https://github.com/ggml-org/llama.cpp

git clone https://github.com/JamePeng/llama-cpp-python.git
git clone https://github.com/ggml-org/llama.cpp.git

3. Automatically set the paths to MSVC (Windows only):

call "C:\Program Files\Microsoft Visual Studio\2022\Community\VC\Auxiliary\Build\vcvars64.bat"

<details>

<summary>4. Optional: For fast build with Ninja</summary>

Using Ninja enables parallel compilation across CPU cores, significantly reducing build time (but may increase CPU temperature). Verify Ninja is installed with Visual Studio 2022:

ninja --version
1.12.1

set CMAKE_GENERATOR=Ninja
set MAX_JOBS=16

</details>

5. Go to the llama-cpp-python folder

cd *path_to_src*\llama-cpp-python

*path_to_comfyui*\python -m pip install json_repair,colorama

set CMAKE_ARGS=-DGGML_CUDA=on -DCMAKE_CUDA_ARCHITECTURES=120 -DGGML_CUDA_FA=ON -DGGML_CUDA_FA_ALL_QUANTS=ON -DCMAKE_BUILD_TYPE=Release
*path_to_comfyui*\python_embeded\python -m pip install . 

✅ The command above is for embedded Python (typical for ComfyUI). Adjust the Python path if you're using a system or virtual environment.

Replace 120 with your сompute сapability number:

RTX 50-series (Blackwell) → 120
RTX 40-series → 89
RTX 30-series → 86
RTX 20-series → 75

⏱️ Build time: Without Ninja, compilation may take 30–60 minutes depending on your hardware.

⏱️ Build time: With Ninja, compilation may take 1–2 minutes depending on your hardware.

💡 Tip: In subprocess mode, you can launch it immediately. In other modes, you need to restart Comfu-ui.

</details>

<details>

<summary>Simple bat file for fast rebuild</summary>

cd llama-cpp-python\vendor\llama.cpp\
git pull
cd ..\..\
git pull --rebase
call "C:\Program Files\Microsoft Visual Studio\2022\Community\VC\Auxiliary\Build\vcvars64.bat"
set CMAKE_GENERATOR=Ninja
set MAX_JOBS=16
set CMAKE_ARGS=-DGGML_CUDA=on -DCMAKE_CUDA_ARCHITECTURES=120 -DGGML_CUDA_FA=ON -DGGML_CUDA_FA_ALL_QUANTS=ON -DCMAKE_BUILD_TYPE=Release
H:\ComfyUI128\python_embeded\python.exe -m pip install . --no-cache-dir --no-build-isolation
pause

✅ The command above is for embedded Python (typical for ComfyUI). Adjust the Python path if you're using a system or virtual environment.

Replace 120 with your сompute сapability number:

RTX 50-series (Blackwell) → 120
RTX 40-series → 89
RTX 30-series → 86
RTX 20-series → 75

💡 Tip: In subprocess mode, you can launch it immediately. In other modes, you need to restart Comfu-ui.

</details>

Installation of ComfyUI_Simple_Qwen3-VL-gguf:

1.Installation to custom_nodes - Use ComfyUI Manager and find ComfyUI_Simple_Qwen3-VL-gguf - OR copy this project to the folder path_to_comfyui\ComfyUI\custom_nodes

  cd path_to_comfyui\ComfyUI\custom_nodes
  git clone https://github.com/KLL535/ComfyUI_Simple_Qwen3-VL-gguf

Use Cases: 3 Ways to Configure Your Model

Models (for example):

<img width="2048" height="448" alt="03522-929995336568847" src="https://github.com/user-attachments/assets/0dc6c148-c049-4fc4-9363-eedb04db2785" />

<details>

<summary>Qwen3.8-27B</summary>

• https://huggingface.co/Blackfrost-AI/Qwen3.8-27B-ABLITERATED-GGUF

For example (for 16 Gb VRAM): Qwen3.8-27B-ABLITERATED-Q3_K_S.gguf + mmproj-BF16.gguf

💡 WARNING: Parameters not specified in this list have default values. See Model Configs sections.

{
    "model_path": "I:\\LLM\\qwen\\Qwen3.8-27B\\Qwen3.8-27B-ABLITERATED-Q3_K_S.gguf",
    "mmproj_path": "I:\\LLM\\qwen\\Qwen3.8-27B\\mmproj-BF16.gguf",
    "n_batch": 4096,
    "top_p": 0.8,
    "top_k": 20,
    "repeat_penalty": 1.05,
    "chat_handler": "qwen35",
    "image_min_tokens": 512,
    "image_max_tokens": 1536
}

</details>

<details>

<summary>Ernie Image Prompt Enhancer</summary>

Highly specialized LLM for Ernie Image.

• https://huggingface.co/Green-Sky/Ernie-Image-Prompt-Enhancer-Ministral-3B-GGUF

• https://huggingface.co/unsloth/Ministral-3-3B-Reasoning-2512-GGUF

For example: Ernie-Image-Prompt-Enhancer-Ministral-3.8B-Q4_K_M.gguf + mmproj-BF16.gguf

💡 TIP: A special system prompt is required; in the templates it is called Ernie Prompt Enhancer.

💡 TIP: The user prompt should look like this: {"prompt": "{prompt}", "width": {width}, "height": {height}}

💡 TIP: mmproj should be left empty "" if image input are not needed.

💡 TIP: The result will only be in Chinese.

{
    "model_path": "H:\\LLM3\\ernie\\Ernie-Image-Prompt-Enhancer-Ministral-3.8B-Q4_K_M.gguf",
    "mmproj_path": "H:\\LLM3\\ernie\\mmproj-BF16.gguf",
    "n_ctx": 4096,
    "temperature": 0.8,
    "top_p": 0.8,
    "top_k": 64,
    "repeat_penalty": 1.05,
    "chat_handler": "llava15",
    "raw_mode": true,
    "prompt_template": "[SYSTEM_PROMPT]{system}[/SYSTEM_PROMPT][INST]{user}{images}[/INST]",
    "stop": "[\"</s>\",\"[INST]\",\"[/INST]\"]",
    "image_min_tokens": 1024,
    "image_max_tokens": 1024
}

</details>

<details>

<summary>gemma-4-12B</summary>

• https://huggingface.co/lmstudio-community/gemma-4-12B-it-QAT-GGUF

{
    "model_path": "I:\\LLM\\gemma\\gemma-4-12B-it-QAT-GGUF\\gemma-4-12B-it-qat-uncensored-heretic-UDmerge-Q4_K_XXL.gguf",
    "mmproj_path": "I:\\LLM\\gemma\\gemma-4-12B-it-QAT-GGUF\\mmproj-gemma-4-12B-it-QAT-BF16.gguf",
    "n_ctx": 12288,
    "max_tokens": 10240,
    "temperature": 0.5,
    "top_p": 0.9,
    "top_k": 20,
    "repeat_penalty": 1.05,
    "chat_handler": "gemma4",
    "enable_thinking": true
}

For Audio/Video:

{
    "model_path": "I:\\LLM\\gemma\\gemma-4-12B-it-QAT-GGUF\\gemma-4-12B-it-qat-uncensored-heretic-UDmerge-Q4_K_XXL.gguf",
    "mmproj_path": "I:\\LLM\\gemma\\gemma-4-12B-it-QAT-GGUF\\mmproj-gemma-4-12B-it-QAT-BF16.gguf",
    "n_ctx": 80384,
    "n_batch": 512,
    "max_tokens": 10240,
    "temperature": 0.5,
    "top_p": 0.9,
    "top_k": 20,
    "repeat_penalty": 1.05,
    "chat_handler": "gemma4",
    "enable_thinking": true,
    "max_frames": 120,
    "audio_sample_rate": 16000
}

</details>

<details>

<summary>HY-MT2 (translate)</summary>

• https://huggingface.co/tencent/Hy-MT2-1.8B-GGUF

For example: Hy-MT2-1.8B-Q4_K_M.gguf

💡 TIP: Here I made a prompt template in which the target_language is set through the system_prompt_override input. Just supply the text with the target language there, for example Russian. And the text that needs to be translated should be submitted to the user_prompt input.

💡 WARNING: The model is highly specialized and understands only strictly defined tasks.

{
    "model_path": "H:\\LLM3\\Hy-MT2-1.8B-Q4_K_M.gguf",
    "n_ctx": 4096,
    "n_batch": 4096,
    "use_mmap": true,
    "top_p": 0.6,
    "top_k": 20,
    "repeat_penalty": 1.05,
    "system_prompt_default": "Russian",
    "raw_mode": true,
    "prompt_template": "<｜hy_begin▁of▁sentence｜>Translate the following segment into {system}, without additional explanation.<｜hy_place▁holder▁no▁3｜><｜hy_User｜>{user}<｜hy_Assistant｜>",
    "stop": "[\"<｜hy_place▁holder▁no▁2｜>\"]"
}

• https://huggingface.co/tencent/Hy-MT2-7B-GGUF

For example: Hy-MT2-7B-Q4_K_M.gguf

{
    "model_path": "H:\\LLM3\\Hy-MT2-7B-Q4_K_M.gguf",
    "n_ctx": 4096,
    "n_batch": 4096,
    "use_mmap": true,
    "top_p": 0.6,
    "top_k": 20,
    "repeat_penalty": 1.05,
    "system_prompt_default": "Russian",
    "raw_mode": true,
    "prompt_template": "<|startoftext|>Translate the following segment into {system}, without additional explanation.<|extra_4|>{user}<|extra_0|>",
    "stop": "[\"<|eos|>\"]"
}

• https://huggingface.co/mradermacher/Hy-MT2-30B-A3B-GGUF

For example: Hy-MT2-30B-A3B.Q4_K_M.gguf

💡 TIP: "n_cpu_moe": 12 to 16G VRAM

{
    "model_path": "H:\\LLM3\\Hy-MT2-30B-A3B-Q4_K_M.gguf",
    "n_ctx": 4096,
    "n_batch": 4096,
    "temperature": 0.3,
    "top_p": 0.6,
    "top_k": 20,
    "repeat_penalty": 1.05,
    "n_cpu_moe": 12,
    "system_prompt_default": "Russian",
    "raw_mode": true,
    "prompt_template": "<|start_header_id|>user<|end_header_id|>\\n\\nTranslate the following segment into {system}, without additional explanation.\\n\\n{user}<|eot_id|><|start_header_id|>assistant<|end_header_id|>\\n\\n",
    "stop": "[\"<|eot_id|>\",\"<|start_header_id|>\"]"
}

</details>

<details>

<summary>Qwen3.6-35B-A3B</summary>

• https://lmstudio.ai/models/qwen/qwen3.6-35b-a3b

For example: Qwen3.6-35B-A3B-Q4_K_M.gguf + mmproj-Qwen3.6-35B-A3B-BF16.gguf

💡 Tip: Q4_K_M is already quite an old quantization. Search for models on huggingface and choose models with better quantization, such as UD_IQ from unsloth. They will be smarter and lighter.

• https://huggingface.co/mudler/Qwen3.6-35B-A3B-APEX-GGUF

For example: Qwen3.6-35B-A3B-APEX-I-Quality.gguf + mmproj.gguf

• https://huggingface.co/unsloth/Qwen3.6-35B-A3B-GGUF

For example: Qwen3.6-35B-A3B-UD-IQ4_XS.gguf + mmproj-BF16.gguf

💡 Tip: If there is a BF16 version for mmproj, choose it, it is better than F16.

Examples:

This model not fit in 16 Gb VRAM. Settings for n_cpu_moe offloading:

💡 Tip: use_mmap = false - Provides better speed, but the model may take longer to load, it needs to be tested.

💡 Tip: split_mode = 0 - Provides better speed on a single GPU, eliminating performance drops after launch.

{
    "model_path": "H:\\LLM\\lmstudio-community\\Qwen3.6-35B-A3B-GGUF\\Qwen3.6-35B-A3B-Q4_K_M.gguf",
    "mmproj_path": "H:\\LLM\\lmstudio-community\\Qwen3.6-35B-A3B-GGUF\\mmproj-Qwen3.6-35B-A3B-BF16.gguf",
    "use_mmap": true,
    "max_tokens": 4096,
    "temperature": 0.8,
    "top_p": 0.95,
    "top_k": 40,
    "n_cpu_moe": 20,
    "chat_handler": "qwen35",
    "enable_thinking": true,
    "image_min_tokens": 1024,
    "image_max_tokens": 2048
}

{
    "model_path": "H:\\LLM2\\qwen\\Qwen3.6-35B-A3B-UD\\Qwen3.6-35B-A3B-UD-IQ4_XS.gguf",
    "mmproj_path": "H:\\LLM2\\qwen\\Qwen3.6-35B-A3B-UD\\mmproj-BF16.gguf",
    "use_mmap": true,
    "max_tokens": 4096,
    "temperature": 0.8,
    "top_p": 0.95,
    "top_k": 40,
    "n_cpu_moe": 16,
    "chat_handler": "qwen35",
    "enable_thinking": true,
    "image_min_tokens": 1024,
    "image_max_tokens": 2048
}

{
    "model_path": "H:\\LLM2\\qwen\\Qwen3.6-35B-A3B-APEX\\Qwen3.6-35B-A3B-APEX-I-Quality.gguf",
    "mmproj_path": "H:\\LLM2\\qwen\\Qwen3.6-35B-A3B-APEX\\mmproj.gguf",
    "use_mmap": true,
    "max_tokens": 4096,
    "temperature": 0.8,
    "top_p": 0.95,
    "top_k": 40,
    "n_cpu_moe": 20,
    "chat_handler": "qwen35",
    "enable_thinking": true,
    "image_min_tokens": 1024,
    "image_max_tokens": 2048
}

</details>

<details>

<summary>Qwen3.6-27B</summary>

• https://huggingface.co/unsloth/Qwen3.6-27B-GGUF

For example: Qwen3.6-27B-UD-IQ3_XXS.gguf + mmproj-BF16.gguf

Fit in 16 Gb VRAM:

{
    "model_path": "H:\\LLM\\lmstudio-community\\Qwen3.6-27B-GGUF\\Qwen3.6-27B-UD-IQ3_XXS.gguf",
    "mmproj_path": "H:\\LLM\\lmstudio-community\\Qwen3.6-27B-GGUF\\mmproj-BF16.gguf",
    "top_p": 0.95,
    "top_k": 40,
    "chat_handler": "qwen35",
    "enable_thinking": true,
    "image_min_tokens": 1024,
    "image_max_tokens": 2048
}

</details>

<details>

<summary>Nemotron-3-Nano-Omni-30B</summary>

• https://huggingface.co/unsloth/NVIDIA-Nemotron-3-Nano-Omni-30B-A3B-Reasoning-GGUF

For example: NVIDIA-Nemotron-3-Nano-Omni-30B-A3B-Reasoning-UD-IQ4_NL.gguf + mmproj-BF16.gguf

Not fit in 16 Gb VRAM -> Use n_cpu_moe = 24:

{
    "model_path": "H:\\LLM2\\nemotron\\NVIDIA-Nemotron-3-Nano-Omni-30B\\NVIDIA-Nemotron-3-Nano-Omni-30B-A3B-Reasoning-UD-IQ4_NL.gguf",
    "mmproj_path": "H:\\LLM2\\nemotron\\NVIDIA-Nemotron-3-Nano-Omni-30B\\mmproj-BF16.gguf",
    "n_batch": 8192,
    "max_tokens": 4096,
    "temperature": 0.6,
    "top_p": 0.95,
    "top_k": 40,
    "n_cpu_moe": 24,
    "chat_handler": "qwen35",
    "enable_thinking": true
}

💡 WARNING: Chat handler qwen35 is not compatible with this model. The model will work, but the quality may deteriorate.

There is an alternative solution: override the chat template. If you only need to process text and/or images, you can use this template overrides: Thinking version (add these lines):

    "chat_handler": "llava15",
    "raw_mode": true,
    "prompt_template": "<|im_start|>system\\n{system}<|im_end|>\\n<|im_start|>user\\n{images}{user}<|im_end|>\\n<|im_start|>assistant\\n<think>\\n",
    "stop": "<|endoftext|>"

Non-thinking version (add these lines):

    "chat_handler": "llava15",
    "raw_mode": true,
    "prompt_template": "<|im_start|>system\\n{system}<|im_end|>\\n<|im_start|>user\\n{images}{user}<|im_end|>\\n<|im_start|>assistant\\n",
    "stop": "<|endoftext|>"

</details>

<details>

<summary>Gemma4-26B-A4B</summary>

• https://huggingface.co/noctrex/gemma-4-26B-A4B-it-uncensored-heretic-MXFP4_MOE-GGUF

For example: Huihui-gemma-4-26B-A4B-it-abliterated-MXFP4_MOE.gguf + mmproj-BF16.gguf

Not fit in 16 Gb VRAM -> set n_cpu_moe.

{
    "model_path": "H:\\LLM2\\gemma\\Huihui-gemma-4-26B-A4B-it-abliterated-MXFP4_MOE\\Huihui-gemma-4-26B-A4B-it-abliterated-MXFP4_MOE.gguf",
    "mmproj_path": "H:\\LLM2\\gemma\\Huihui-gemma-4-26B-A4B-it-abliterated-MXFP4_MOE\\mmproj-BF16.gguf",
    "n_ctx": 4096,
    "n_batch": 512,
    "max_tokens": 4096,
    "top_p": 0.95,
    "top_k": 40,
    "n_cpu_moe": 10,
    "chat_handler": "gemma4"
}

</details>

<details>

<summary>Gemma4-E4B</summary>

• https://huggingface.co/unsloth/gemma-4-E2B-it-GGUF
• https://huggingface.co/unsloth/gemma-4-E4B-it-GGUF
• https://huggingface.co/HauhauCS/Gemma-4-E4B-Uncensored-HauhauCS-Aggressive

For example: gemma-4-E4B-it-IQ4_XS.gguf + mmproj-BF16.gguf

Option appeared enable_thinking": false, but he doesn't turn off thinking :).

{
    "model_path": "H:\\LLM2\\gemma4\\gemma-4-E4B-it-IQ4_XS.gguf",
    "mmproj_path": "H:\\LLM2\\gemma4\\mmproj-BF16.gguf",
    "n_ubatch": 2048,
    "temperature": 1.0,
    "top_p": 0.95,
    "min_p": 0.01,
    "top_k": 64,
    "repeat_penalty": 1.0,
    "chat_handler": "gemma4"
}

You can write custom prompt template and then thinking will turn off.

{
    "model_path": "H:\\LLM2\\gemma4\\gemma-4-E4B-it-IQ4_XS.gguf",
    "mmproj_path": "H:\\LLM2\\gemma4\\mmproj-BF16.gguf",
    "n_ubatch": 2048,
    "temperature": 1.0,
    "top_p": 0.95,
    "min_p": 0.01,
    "top_k": 64,
    "repeat_penalty": 1.0,
    "chat_handler": "gemma4",
    "raw_mode": true,
    "prompt_template": "<|turn>system\n{system}<turn|>\n<|turn>user\n{images}\n{user}<turn|>\n<|turn>model\n",
    "stop": "[\"<turn|>\",\"<eos>\",\"<|end_of_turn|>\"]"
}

</details>

<details>

<summary>Sulphur prompt enhancer</summary>

An interesting uncensored fine-tuned model for LTX 2.3.

• https://huggingface.co/SulphurAI/Sulphur-2-base/tree/main/prompt_enhancer

💡 Warning: A highly specialized model for enhance prompts for LTX 2.3.

💡 Warning: The module is poorly described, so the following settings are set by eye. More optimal settings may exist.

💡 Tip: system_preset_to_user_prompt: true means that the system prompt will be passed to the user prompt (before user prompt).

💡 Tip: user_prompt_after_content: false means that the image will be transmitted at the end.

system_prompt: none or LTX I2V or LTX T2V or enhance this for video generation

{
    "model_path": "H:\\LLM2\\sulphur\\sulphur_prompt_enhancer_model-q8_0.gguf",
    "mmproj_path": "H:\\LLM2\\sulphur\\mmproj-BF16.gguf",
    "n_batch": 4096,
    "use_mmap": true,
    "temperature": 0.8,
    "top_p": 0.9,
    "top_k": 40,
    "chat_handler": "qwen35",
    "system_preset_to_user_prompt": true,
    "user_prompt_after_content": false,
    "force_mmproj": false,
    "image_min_tokens": 1024,
    "image_max_tokens": 2048
}

</details>

<details>

<summary>Cydonia-24B</summary>

An interesting fine-tuned model based on mistral.

• https://huggingface.co/mradermacher/Cydonia-24B-v4.3-absolute-heresy-GGUF

There is no visual encoder (mmproj) here, but you can take it from the base model (Mistral-Small), for example from here:

• https://huggingface.co/ggml-org/Mistral-Small-3.1-24B-Instruct-2503-GGUF/tree/main

💡 Warning: This is diffefent mmproj projector! If the projector didn't freeze during fine-tune, it may have degraded (the vector space "floated"). In this case, there is a 95% chance that the projector is not damaged.

For example: Cydonia-24B-v4.3-absolute-heresy.IQ4_XS.gguf + mmproj-Mistral-Small-3.1-24B-Instruct-2503-f16.gguf

💡 Warning: I couldn't find a compatible chat handler, so I'm using a custom one.

{
    "model_path": "H:\\LLM2\\Cydonia_24b\\Cydonia-24B-v4.3-absolute-heresy.IQ4_XS.gguf",
    "mmproj_path": "H:\\LLM2\\Cydonia_24b\\mmproj-Mistral-Small-3.1-24B-Instruct-2503-f16.gguf",
    "top_p": 0.9,
    "min_p": 0.02,
    "top_k": 40,
    "chat_handler": "llava15",
    "raw_mode": true,
    "prompt_template": "[SYSTEM_PROMPT]{system}[/SYSTEM_PROMPT][INST]{images}{user}[/INST]",
    "stop": "[\"</s>\",\"[INST]\",\"[SYSTEM_PROMPT]\"]"
}

</details>

<details>

<summary>Qwen3.5-9B</summary>

• https://huggingface.co/unsloth/Qwen3.5-0.8B-GGUF
• https://huggingface.co/unsloth/Qwen3.5-2B-GGUF
• https://huggingface.co/unsloth/Qwen3.5-4B-GGUF
• https://huggingface.co/unsloth/Qwen3.5-9B-GGUF

For example: Qwen3.5-9B-Q4_K_M.gguf + mmproj-BF16.gguf

And a new option appeared enable_thinking": true, - If you want the model to think (this may give a better result), write true, but this will take more time and require more context, plus the think section will have to be cut off later.

Other parameters should be selected based on recommendations, based on the task, or empirically, as you prefer.

```json { "model_path": "I:\\LLM\\qwen\\qwen35-9b\\Qwen3.5-9B-Uncensored-HauhauCS-Aggressive-Q4_K_M.gguf", "mmproj_path": "I:\\LLM\\qwen\\qwen35-9b\\mmproj-Qwen3.5-9B-Uncensored-HauhauCS-Aggressive-BF16.gguf", "temperature": 0.5, "top_p": 0.8, "top_k": 20, "repeat_penalty": 1.0, "chat_handl

Method 1: The New Advanced Configurator (Recommended)

The 🌐 LLM Config (Advanced) node provides a clean, organized interface for all 70+ parameters. - Parameters are grouped into collapsible sections (Model, Memory, Sampling, Hardware, etc.), so you only see what you need. - Windows users can use the "Browse" buttons to select GGUF files from anywhere on the disk. - Outputs a ready-to-use JSON configuration string to the main node.

Method 3: Manual Text Config (Best for Power Users & Stacking)

<img width="1331" height="696" alt="Image" src="https://github.com/user-attachments/assets/320192ed-d0c2-46bb-bc44-7f24d8348f3a" />

You can bypass the UI widgets entirely and pass configuration directly as a text string. - Flexible Formatting: You don't need perfect JSON. If the json_repair library is installed, it will automatically fix missing commas or quotes. - Stacking & Overwriting: Configurations are stackable. Each additional config_override input overwrites the specified fields and leaves the rest unchanged. - Use Case: This is the only way to pass brand-new, experimental parameters to the backend script before they are officially added to the Advanced Configurator's UI widgets.

💡 Pro Tip: You can combine all three methods! Set a base configuration using a Preset, tweak a few settings using the Advanced Configurator, and inject a final, specific override (like a custom stop sequence) via the config_override text input. The system resolves them in that exact order of priority.

Model Configs:

Possible model configurations that can be passed to the config_override input.

<details>

<summary>Configurator Parameters</summary>

📁 Model & Paths

🗄️ Memory & Context

🎲 Sampling & Generation <div class="rdm-tbl-wrap"><table class="rdm-tbl"><thead><tr><th>Field</th><th>Type</th><th>Default</th><th>Description</th></tr></thead><tbody><tr><td>max_tokens</td><td>int</td><td>2048</td><td>Maximum tokens to generate. Thinking models usually need more (4096+). Smaller = faster but may truncate response</td></tr><tr><td>temperature</td><td>float</td><td>0.7</td><td>Sampling temperature. Lower (0.1) = deterministic/focused, Higher (1.5+) = creative/random. 0.7 is balanced</td></tr><tr><td>top_p</td><td>float</td><td>0.92</td><td>Nucleus sampling cutoff. Model considers tokens whose cumulative probability reaches top_p. Lower = more focused</td></tr><tr><td>min_p</td><td>float</td><td>0.05</td><td>Minimum probability threshold. Tokens with prob &lt; min_p × top_token_prob are filtered out. Great for reducing garbage</td></tr><tr><td>top_k</td><td>int</td><td>0</td><td>Limit to top-K most likely tokens. 0 = disabled. Good for strict output control</td></tr><tr><td>repeat_penalty</td><td>float</td><td>1.1</td><td>Penalty for repeating tokens. Values &gt;1 discourage repetition loops. 1.1 is mild, 1.5+ is aggressive</td></tr><tr><td>presence_penalty</td><td>float</td><td>0.0</td><td>Penalty based on token presence. Positive values encourage new topics, negative favor repetition</td></tr><tr><td>frequency_penalty</td><td>float</td><td>0.0</td><td>Penalty based on token frequency. Positive values reduce repetition of common words</td></tr><tr><td>enable_thinking</td><td>bool</td><td>False</td><td>Enable thinking/reasoning process for Gemma, Qwen, MiniCPM, GLM models. Requires more output tokens</td></tr><tr><td>remove_thinking</td><td>bool</td><td>False</td><td>Cleans model output by removing <think>...</think> or &lt;</td><td>channel&gt;...&lt;channel</td><td>&gt; sections.</td></tr><tr><td>force_reasoning</td><td>bool</td><td>False</td><td>For Qwen3: force reasoning mode even on simple queries. Makes model always &quot;think&quot; before answering</td></tr><tr><td>words_to_ban</td><td>string</td><td>&quot;&quot;</td><td>Comma-separated list of banned words. Applies logit_bias of -100 to their tokens. Example: woman,Woman,man,Man</td></tr></tbody></table></div>

⚙️ Hardware & Acceleration

💬 Chat, Prompts & Variables

💬 Prompt Template

🖼️ Multimodal & Media

⚡ Speculative Decoding

Speculative decoding accelerates text generation by using a draft model (or statistical n-gram) to predict multiple tokens ahead, which are then verified by the target model in a single batch pass. This can significantly speed up inference when the draft predictions are accurate.

🔢 Embeddings

🛠️ Debug, System & Advanced

Notes & Nuances

1. Browse Button Limitation The Browse Model and Browse MMProj buttons currently work only on Windows (using native file dialog via ctypes). Linux/macOS users must manually type paths. If there's demand, I can implement GTK/Qt dialogs for other platforms.

2. Override Input Behavior The config_override input strictly overwrites fields passed through it. This means: - Values shown in widgets may differ from actual output if override is used - Override has highest priority (applied last) - Use override for dynamic/runtime changes, widgets for static defaults

3. Parameter Naming Consistency All parameters use canonical names (n_ctx, n_gpu_layers, max_tokens). Old names (ctx, gpu_layers, output_max_tokens) are automatically converted via old_names_patch() for backward compatibility.

4. Widget Reordering Parameters are rendered in a fixed order matching the Python node's INPUT_TYPES(). However, the underlying architecture supports reordering via **kwargs, so future versions may allow custom layouts without breaking saved workflows.

5. Preset Storage Presets are saved to ComfyUI/user/SimpleQwenVL_configs/system_prompts_user.json. The file is created automatically on first use.

6. Multi-GPU Caveats cuda_device parameter may not work correctly in direct_clean and keep_vram modes, as ComfyUI itself may have already initialized CUDA with different settings.

7. Vision ID Logic add_vision_id with auto mode calculates: True if (num_images != 1 or num_videos > 0) else False. This matches Qwen3/Qwen3.5 requirements for multi-image scenarios.

8. Memory Pool Sizing pool_size default (4194304 = 4MB) works for most models. If you encounter ggml_new_object: not enough space, increase to 8MB (8388608) or 16MB (16777216).

</details>

Configuration Files & Presets

<details> <summary>Rules & File Hierarchy</summary>

The system uses a stackable configuration approach. Files are loaded in the following order of priority:

1. ComfyUI/user/SimpleQwenVL_configs/system_prompts_user.json (Recommended) This is the primary user settings file. It is created automatically on first use. The new Advanced Configurator reads from and writes to this file directly via its Save/Rename/Delete buttons. Edit this file or manage it via the UI.

2. system_prompts_user.json (Legacy Node Folder) Located in the node's root directory. Supported for backward compatibility with older setups. If both this file and the user/ directory file exist, the user/ directory file takes precedence. Manual editing is discouraged in favor of the UI manager.

3. system_prompts.json (Base Project Settings) Located in the node's root directory. Contains default, project-level presets maintained by the developer. Do not edit this file, as your changes will be overwritten during node updates.

</details>

Method 2: Model Preset Dropdown (Best for Workflow Reusability)

Once you have tuned your settings (either via the Advanced Configurator or manually), you can save them as a named preset. - Use the Save, Rename, and Delete buttons in the Advanced Configurator to manage your library. - Presets are saved to system_prompts_user.json in user folder. - In the main node (or configurator), simply select your saved preset from the model_preset dropdown list. This instantly loads all associated parameters, making it easy to switch between different models without rewiring your workflow.