ComfyUI

What is ComfyUI?

ComfyUI is an open-source, node-based interface and inference engine for generative AI. Users can connect various steps such as model loading, prompt encoding, sampling, ControlNet, LoRA, scaling, image processing, and output to create a visual workflow, thereby enabling precise control over each parameter and data flow.

It first became popular thanks to the Stable Diffusion workflow, and today its official scope has been expanded to include images, videos, audio, 3D content, as well as external model APIs. This workflow can be run on local hardware, or it can be utilized through the official Comfy Cloud service or by exporting it as an API call.

Node-based AI drawing workflow

Each node is responsible for a specific step, and the connections between nodes represent the transfer of images, latent spaces, conditions, models, masks, or parameters. Compared to interfaces that rely on just a set of form parameters, ComfyUI is better suited for handling complex workflows, reusing elements, and generating content in bulk.

Node diagrams can also become quite large. It is recommended to group nodes, assign names to them, add descriptions, and consolidate adjustable parameters in the input area, so as to prevent the team from facing difficulties in maintaining them.

Model and sampling control

Users can select Checkpoint, VAE, text encoder, sampler, scheduler, number of steps, CFG, seed, and resolution separately; they can also combine multiple models within the same workflow. Different models have varying directory structures, precision levels, memory requirements, and license conditions.

Text-to-image and image-to-image

The basic workflow supports text-to-image generation, image redrawing, partial redrawing, image enlargement, style conversion, and high-resolution restoration. Nodes allow for precise control over noise reduction levels, masking, dimensions, and multi-stage sampling.

ControlNet, LoRA, and conditional control

ComfyUI allows posture, depth, edges, line art, reference images, and LoRA styles to be used as conditions in the workflow. Multiple control branches can influence the generation process based on different weights and time periods.

The licenses for models and LoRA may restrict commercial use, use in relation to specific individuals, or redistribution. It is necessary to read the original license when downloading them, rather than relying solely on the instructions provided by the workflow creator.

Video, audio, and 3D creation

The official documentation and the ecosystem map cover areas such as video generation from text, video generation from images, frame interpolation, lip synchronization, audio generation, as well as 3D preview techniques like Gaussian Splat and point clouds. Video processing typically requires more video memory, more processing time, and more disk space.

The compatibility between third-party nodes and models changes rapidly; before running public workflows, it is necessary to verify the required version, model files, resolution, and video memory.

Workflow template

The official website and the app offer community workflows and templates; users can start with proven processes and then replace the models, prompts, and input materials. Templates help to shorten the time required to get started, but after they are imported, issues such as the absence of models or custom nodes often arise.

App Mode simplifies the interface

App Mode allows complex node diagrams to be transformed into a simplified user interface, showing only images, prompts, options, and action buttons. It is suitable for making internal workflows accessible to designers, clients, or non-technical colleagues.

Before releasing the app, it is necessary to restrict high-risk parameters, external calls, and file paths, and provide alerts for error conditions and usage levels.

ComfyUI Desktop version

Desktop offers a more user-friendly installation and update experience for local use, and it includes components such as Manager by default. Users still need to have compatible hardware, drivers, model files, and sufficient disk space.

The desktop version does not charge based on the number of inference executions when performing computations on the local device, but the user is responsible for the costs related to electricity, GPUs, and storage; calling online models such as Partner Nodes may result in the consumption of Credits or incur third-party fees.

Installation on Windows, macOS, and Linux

The official documentation provides installation instructions for Windows, macOS, and Linux. Windows users can choose between the desktop version or the portable version.

macOS can take advantage of the Apple chip environment; Linux and advanced users usually use a manual Python environment.

Before installation, check the requirements for Python, PyTorch, GPU drivers, and video memory. Do not mix the dependencies of multiple AI tools with the system’s Python; using a separate environment makes it easier to revert to an earlier state.

VRAM management

ComfyUI loads and unloads models according to the requirements of the workflow, and offers various GPU memory strategies. The models that can be executed actually depend on factors such as parameter size, precision, resolution, batch size, attention mechanisms, and system memory.

The low VRAM mode can lower the entry barrier, but it usually increases the data exchange between the CPU and GPU, which may lead to a significant slowdown in performance.

ComfyUI Manager

Manager has become the key element upon which official organization maintenance relies; it is used to search for, install, enable, disable, and update custom nodes, and it alerts users to any missing nodes when importing workflows. The new interface is primarily used for installing registered nodes from the Registry.

The older version of Manager also provided snapshots, which made it possible to save the state of the environment before making updates and to restore that state in case of dependency conflicts. The nightly version is more similar to the development branch; its stability and security scanning capabilities are not as good as those of the officially released version.

Custom node

Communities can use Python and front-end components to expand nodes; the developers claim that there are over 60,000 such nodes in this ecosystem. Custom nodes can provide support for models, image processing, API connections, as well as special user interfaces.

Installing a node is equivalent to running third-party code on the local machine. It is necessary to check the repository, its maintenance status, dependencies, and the installation scripts; nodes of unknown origin should not be placed in environments that contain sensitive files or API keys.

Dependency conflicts and environment snapshots

Different nodes may require incompatible versions of Python packages. Before making any updates, it is advisable to save snapshots of the Manager, as well as lists of workflows and models. In case of problems, it is better to revert to combinations that have been proven to work, rather than attempting to update all components at once.

Workflow JSON

ComfyUI workflows can be saved in JSON format, which describes the nodes, connections, parameters, and interface layout. Outputs such as images can also include workflow metadata, facilitating reloading and reproduction.

Before sharing JSON or images, it is necessary to check whether they contain prompts, file names, internal paths, account information, or API keys. Reproduction also requires the use of the same model, nodes, and version.

Local API

A local ComfyUI server provides interfaces for workflow queues, history, files, and WebSocket status. Developers can export the workflows that have been debugged in the UI in API format, and then modify prompts, seeds, images, and other node inputs through the program.

By default, services should not be exposed directly to the public internet. For remote access, reverse proxies, authentication, HTTPS, network restrictions, and file isolation are required.

Comfy Cloud

Comfy Cloud is the officially hosted version; it uses an NVIDIA RTX 6000 Pro GPU with 96GB of video memory, and comes pre-installed with over 900 models as well as common custom nodes. Users do not need to download large models or manage drivers – they can simply load templates and start using them right away.

The creation and editing of workflows do not consume GPU Credits; only the actual execution of the workflows and the use of Partner Nodes incur costs. Once they are generated, images, videos, and audio can be downloaded.

Comparison of Comfy Cloud plans and prices

On the pricing page, the annual payment rates for the main plans are shown as: Standard at $16 per month, Creator at $28 per month, and Pro at $80 per month; the corresponding annual amounts are $192, $336, and $960 respectively. Credits purchased on a monthly basis are not carried over to subsequent months.

The additional Top-up Credits purchased are valid for a maximum of one year.

How are Credits consumed?

Credits are used for the active GPU runtime and Partner Nodes. Editing workflows, waiting time, and idle time are not charged.

A 5-second video example requires approximately 11 Credits, but the actual amount used varies depending on the model, resolution, number of frames, number of steps, and the Partner Node.

Partner Nodes

Partner Nodes allow users to invoke collaborative models such as Nano Banana, Kling, Luma, Runway, etc., within the node diagram. They share the same Credit balance, and the cost per invocation is determined by the model and its parameters.

Partner Nodes can also be used within ComfyUI locally, but they still involve calls to cloud APIs. Some providers offer BYOK options, with the associated costs charged directly by those providers.

Import custom models and LoRA

Creator, Pro, and Team versions allow users to import their own models and LoRA files from CivitAI or Hugging Face. The Cloud version only supports compatible formats and nodes; importing such files does not grant any commercial license for the models.

Cloud API

The Cloud API accepts workflow JSON that is compatible with local APIs; it allows for modifying node inputs programmatically, running tasks in batches, tracking progress via WebSocket, and providing real-time previews. It is suitable for deploying prototype workflows as applications or within production pipelines.

The current concurrent API usage for Standard, Creator, and Pro is 1, 3, and 5 respectively; the queue can hold up to 100 workflows. For larger scales, restrictions need to be agreed upon through Team or Enterprise plans.

MCP connects to AI agents

The official documentation provides an MCP interface that allows compatible AI agents to call Comfy workflows in order to generate images, videos, audio, and 3D content. Before carrying out tasks that are resource-intensive, such agents should display the estimated amount of resources required, and manual approval is necessary for batch generation as well as for publishing the resulting content externally.

Open-source licenses and GitHub

The official GitHub repository for ComfyUI is currently located in the Comfy-Org organization; the old address will redirect to this new location. The core project is licensed under GPL-3.0, and its source code, issues, released versions, and contribution history are all available publicly.

The ComfyUI code is open source, but this does not mean that the models downloaded, the custom nodes, Partner Nodes, or the generated content are all subject to the same license. For commercial projects, it is necessary to examine each dependency separately.

ComfyUI usage tutorial

Complete a basic task.

1. In ComfyUI, specify the trigger conditions, input data, and the final action.
2. Connect to the required application using a test account and grant only the minimum necessary permissions;
3. Configure the node-based AI drawing workflow as well as model and sampling controls, mapping each field individually;
4. Add branch handling, failure management, retry options, and manual approval;
5. Run with a small amount of test data and check the output at each step;
6. Enable the official process only after confirming that there are no duplicate writes or abnormal charges.

Create reusable professional workflows

1. Choose business processes with a high frequency of repetition and clear rules;
2. Draw the paths for triggering, decision-making, execution, and rollback;
3. Separate the node-based AI drawing workflow, model and sampling control, as well as text-to-image and image-to-image generation, into reusable modules;
4. Set log settings, budgets, timeout values, and permission limits;
5. Assign a responsible person to randomly check the results and handle the failed queues;
6. Review application permissions, business rules, and actual costs on a monthly basis;

Which users is it suitable for?

• Creators who need precise control over the AI drawing process;
• Technical art for creating images, videos, audio, and 3D content;
• Teams that aim for reuse, batch processing, and automated generation;
• Users who run open-source models on local GPUs;
• Developers who integrate the generated workflows into their products through APIs;
• Enterprises that require high VRAM in the cloud as well as hosted inference services.

Product advantages

• All models, parameters, and steps are visualized and adjustable;
• It is open-source, can be run locally, and features a large ecosystem of nodes;
• Workflows can be saved, shared, reproduced, and converted into APIs;
• Supports images, videos, audio, and 3D content generation;
• The official Desktop, Cloud, Manager, API, and MCP components form a complete system;
• App Mode allows ordinary users to handle complex processes.

Restrictions and Precautions

• The learning curve is steep, and complex node diagrams as well as dependency conflicts need to be managed;
• Third-party nodes pose a risk of code execution, and model permissions also need to be reviewed one by one;
• Local execution is limited by video memory, drivers, and disk space, while Cloud billing is based on Credits and the duration of the task.
• Public services must have authentication; keys and internal paths should be removed before sharing workflows.
• The content generated must also comply with portrait, copyright, and platform policies.

Frequently Asked Questions

Is ComfyUI free?

The local open-source version is free; Comfy Cloud offers 5 free GPU sessions, after which pricing is based on packages and Credits.

Does ComfyUI require a graphics card?

Local generation usually requires compatibility with GPUs, and the specific requirements depend on the model and workflow; Comfy Cloud cannot be used in the absence of suitable hardware.

Can ComfyUI generate videos?

Yes, the official core components, templates, and custom nodes support a variety of video models and processing workflows.

Does ComfyUI provide an API?

It provides local server APIs and official Cloud APIs, allowing the execution of exported workflow JSON files with progress tracking via WebSocket.

Is ComfyUI open source?

It is open source; the official core repository is licensed under GPL-3.0. The licenses for the models and third-party components need to be verified separately.