ComfyUI-LTXVideo

Overview

ComfyUI-LTXVideo is a repository containing a collection of custom nodes designed to enhance the functionality of ComfyUI when working with the LTXV model. This repository provides advanced tools for video processing, including updates and workflows that showcase improvements in video quality, resolution, sequence conditioning, and prompt handling.

Installation

Preferred Installation via ComfyUI-Manager

1. Use the ComfyUI-Manager.
2. Search for ComfyUI-LTXVideo in the list of available nodes.
3. Follow the installation instructions provided.

Manual Installation

1. Ensure ComfyUI is installed on your system.

2. Clone the ComfyUI-LTXVideo repository into the custom-nodes directory within your ComfyUI installation:

   git clone https://github.com/Lightricks/ComfyUI-LTXVideo custom-nodes/ComfyUI-LTXVideo
   

3. Navigate to the newly cloned directory and install the required packages:

   cd custom-nodes/ComfyUI-LTXVideo
   pip install -r requirements.txt
   

4. For portable ComfyUI installations, install the required packages using the embedded Python executable:

   .\python_embeded\python.exe -m pip install -r .\ComfyUI\custom_nodes\ComfyUI-LTXVideo\requirements.txt
   

Models

1. Download the LTX Video model ltx-video-2b-v0.9.1.safetensors from Hugging Face.
2. Place the model file into the models/checkpoints directory.
3. Install a T5 text encoder, for example google_t5-v1_1-xxl_encoderonly, using the ComfyUI Model Manager.

Purpose

This repository is designed to provide ComfyUI users with enhanced capabilities for video processing using the LTXV model. It integrates advanced video workflow nodes that enable higher resolution outputs, improved sequence conditioning, and better artifact handling. Additionally, prompt generation for video synthesis is optimized to improve overall model performance.

Node Offerings

The following nodes are included within this repository:

• ModifyLTXModel
• AddLatentGuide
• LTXForwardModelSamplingPred
• LTXReverseModelSamplingPred
• LTXRFForwardODESampler
• LTXRFReverseODESampler
• LTXAttentionBank
• LTXPrepareAttnInjections
• LTXAttentioOverride
• LTXPerturbedAttention
• LTXAttnOverride
• LTXFlowEditCFGGuider
• LTXFlowEditSampler
• LTXFetaEnhance

Special Features

• Frame Conditioning: Supports interpolation between specified frames to ensure smoother transitions.
• Sequence Conditioning: Allows for motion interpolation from a given sequence, thereby extending videos seamlessly from different points (beginning, middle, or end).
• Prompt Enhancer Node: Optimizes text prompts to align with the capabilities of the LTXV model for improved video synthesis outcomes.

Utility in ComfyUI Workflows

The ComfyUI-LTXVideo custom nodes are designed to enhance the video processing capabilities of users, allowing them to create high-quality video outputs with greater control over sequence and frame conditioning. The seamless integration of prompt enhancement ensures that users can generate context-aware prompts that leverage the full potential of the model. Overall, these nodes aid in crafting sophisticated video workflows within ComfyUI.

Example Workflows

• Image-to-Video: A workflow showcasing conversion from image inputs to video outputs.
• Text-to-Video: Demonstrates the transformation of text prompts into video sequences.
• Frame Interpolation: Visualizes smooth transitions by interpolating between key frames.
• Sequence Conditioning: Specifies how to extend a video sequence either from the first or the last position.

To execute the example workflows, additional custom nodes such as ComfyUI-VideoHelperSuite are required. Install missing nodes using the "Install Missing Custom Nodes" feature in ComfyUI Manager.

Explore the available workflows to fully realize the potential of these custom nodes in your video processing tasks.