CLIPLoaderGGUF Node Documentation

Overview

The CLIPLoaderGGUF node is a component of the ComfyUI-GGUF package. This node provides functionality to load CLIP (Contrastive Language–Image Pre-training) model files stored in the GGUF format. This format is particularly suited for running efficiently on low-end GPUs by allowing for GGUF quantized models, which significantly reduce VRAM usage. The CLIPLoaderGGUF node is part of a suite of nodes designed to work with GGUF quantization support.

Node Functionality

What This Node Does

The CLIPLoaderGGUF node is designed to load text encoders used in CLIP models, specifically those that are formatted in GGUF or traditional formats like safetensors/bin. It acts as a bridge to incorporate these models into a ComfyUI workflow, replacing default components with quantized alternatives for improved performance on less capable hardware.

Inputs

The CLIPLoaderGGUF node accepts the following inputs:

• clip_name: The name of the CLIP model file to load. The node provides a selection from all available CLIP model files located in directories designated for CLIP and GGUF files.
• type: The type of CLIP model being loaded. Options include:
  • stable_diffusion
  • stable_cascade
  • sd3
  • stable_audio
  • mochi
  • ltxv
  • pixart
  • wan

These inputs allow the node to locate the correct model file and load it according to its specific requirements.

Outputs

The node produces the following output:

• CLIP: The loaded CLIP model that can be utilized within a ComfyUI workflow. This output is formatted to be compatible with other nodes and processes in the workflow, enabling seamless integration.

Usage in ComfyUI Workflows

The CLIPLoaderGGUF node is utilized in workflows that require advanced text-and-image processing capabilities provided by CLIP models. It's especially useful in scenarios where VRAM efficiency is critical, as the GGUF format allows the models to run with reduced resource consumption. When creating a workflow in ComfyUI, users should replace the standard CLIP loading node with CLIPLoaderGGUF to take advantage of GGUF quantized models, ensuring the selected file and type match their project requirements.

Workflow Integration

1. Replace Standard Nodes: In an existing ComfyUI workflow, substitute the standard CLIP loading node with the CLIPLoaderGGUF node.
2. Select Model: Choose the appropriate CLIP model file and type from the node's input options. Ensure the correct file is placed within the ComfyUI/models/clip or equivalent directory.
3. Use in Workflows: Connect the CLIPLoaderGGUF output to other nodes that require a CLIP model, such as those performing text-to-image transformations or other related tasks.

Special Features and Considerations

• Quantization: The node supports loading models with GGUF quantization, making it ideal for systems with limited VRAM.
• Compatibility: It handles both GGUF and traditional model files seamlessly, allowing users to switch between models without workflow disruptions.
• Multi-Model Support: With different configurations under the ComfyUI umbrella, users can choose various model types, providing flexibility in designing workflows tailored to specific tasks.

In summary, the CLIPLoaderGGUF node is a powerful tool for integrating efficient CLIP models into ComfyUI workflows, facilitating advanced data processing on low-resource hardware.