Documentation for HyVideoPromptMixSampler Node

Overview

The HyVideoPromptMixSampler node is a part of the HunyuanVideoWrapper module for ComfyUI. This node is designed to blend video generation results from two different sets of text prompts. It leverages prompt embeddings to generate a sequence of frames, allowing for creative mixtures between two sets of textual descriptions. This node is particularly useful in workflows that require merging concepts or styles defined by separate prompts to create a coherent but transitional video output.

Functionality

This node performs the following main functions:

• Generates a sequence of video frames based on input text embeddings.
• Mixes or blends two sets of embeddings across time, achieving a gradual transition between two different textual concepts.
• Allows customization for blending characteristics and sharpness.
• Integrates with ComfyUI workflows to enhance video-generative capabilities.

Inputs

The HyVideoPromptMixSampler node accepts the following inputs:

Required Inputs

• Model (HYVIDEOMODEL): The video generation model that the node will use for producing video frames.
• HyVideo Embeds (HYVIDEMBEDS): The first set of text embeddings used to guide the video generation process.
• HyVideo Embeds 2 (HYVIDEMBEDS): The second set of text embeddings, offering an alternative guide.
• Width (INT): Specifies the width of the output frames.
• Height (INT): Specifies the height of the output frames.
• Num Frames (INT): Number of frames to be generated in the video sequence.
• Steps (INT): The number of inference steps to perform, influencing the video detail and fluidity.
• Embedded Guidance Scale (FLOAT): Influences how strongly the prompts guide the generative process.
• Flow Shift (FLOAT): Adjusts the flow dynamics of the frame transitions.
• Force Offload (BOOLEAN): Determines if resources are offloaded to optimize memory usage.
• Seed (INT): The seed for random number generation, useful for reproducibility.
• Alpha (FLOAT): Controls the sharpness and intensity of the blending between the two sets of embeddings.

Optional Inputs

• Interpolation Curve (FLOAT): An optional input defining the strength of the blending curve over time, providing finer control of how concepts transition.
• Feta Args (FETAARGS): Additional arguments to modify feature enhancement settings.

Output

Produced Output

• Samples (LATENT): Returns the generated sequence of latent video frames as the output, ready to be decoded into a visible video format.

Usage in ComfyUI Workflows

The HyVideoPromptMixSampler node can be integrated into ComfyUI workflows to achieve creative video synthesis tasks. It is particularly suited for:

• Creating artistic video clips that smoothly transition between two different themes or concepts as defined by the input prompts.
• Blending styles or narrative elements in video production where hybrid visualization is desired.
• Experimenting with dynamic visual storytelling by adjusting the blending parameters and observing the effects on the generated video content.

Special Features and Considerations

• Blending Sharpness with Alpha: The alpha parameter allows precise control over how sharply or smoothly the transition between the two sets of prompts occurs, providing creative flexibility.
• Temporal Interpolation: Through the interpolation curve, users can dictate how heavily the influence of each set of embeddings is applied over time, offering advanced customization.
• Experimental Flag: This node is marked as experimental, indicating that it might undergo changes or improvements and could benefit from user feedback.

In summary, the HyVideoPromptMixSampler node offers powerful capabilities for blending video content guided by dual prompts, offering fine control over visual transitions and artistic video generation in ComfyUI workflows.