ChatTTS

What is ChatTTS?

ChatTTS is a generative text-to-speech model developed by 2Noise, designed specifically for chat assistants, multi-person conversations, and informal content, rather than for traditional read-aloud purposes.

The project supports both Chinese and English, and it is possible to generate speech locally using Python, the command line, or the example WebUI. English is still labeled as experimental in the project documentation; therefore, it cannot be assumed to have the same level of stability as Chinese.

First, distinguish between official projects and websites with the same name.

The website with the same name links to the official repository, but its pages contain inconsistencies regarding the number of training hours, it refers to models that have already been released as future projects, and it claims to offer multilingual SDKs – statements that have not been confirmed by the official project. For actual deployment and licensing purposes, it is necessary to rely on 2Noise’s current repository and model cards.

Model and training data status

The public model is not the complete version of the 100,000-hour training model, nor is it a commercial speech engine that has been optimized for end-users. The training scale and sound quality advertised on the website cannot replace actual testing of the public weights.

Main functions

Dialogic speech synthesis

• Enter text in Chinese, English, or a mixture of both, and the model will generate speech that matches the rhythm of everyday conversation.
• It is suitable for generating continuous audio for LLM assistants, dialogue prototypes, research demonstrations, and multi-turn interactions.
• The output can be saved in WAV format at a sampling rate of 24,000 Hz; examples of commands in the repository also generate MP3 files.
• Its role is to convert text into speech; it is not responsible for understanding business issues, retrieving knowledge, or generating dialogue text.

Multiple speakers and random voices

• The model supports multiple speakers, and developers can sample random speaker embeddings to obtain different voices.
• After saving the speaker embeddings obtained from sampling, it is possible to reuse speakers with similar vocal timbres in subsequent inference.
• Random speakers are not created by uploading samples for identity replication; currently, the available repositories do not list zero-sample voice cloning as a completed capability.
• The voice cloning feature promoted on the website with the same name leads to other products, and it cannot be considered an official capability of the ChatTTS model.

Laughter, pauses, and informal control

• Sentence-level prompts allow for adjusting the intensity of oral speech, laughter, and pauses, enabling control over the level of spoken language, the tendency to laugh, and the length of pauses.
• At the word level, the main controls available are laugh, uv_break, and lbreak; they allow laughter or various pauses to be inserted at specified locations.
• Temperature, top_P, and top_K are used to adjust the randomness of sampling; changes in these parameters affect the timbre, rhythm, and stability.
• More emotion control is still part of the roadmap; the current publicly available models are not able to generate content consistently based on any given emotion tags.

Batch inference and local tools

• The Python interface allows multiple segments of text to be submitted at once, with an audio array being returned as a result; it is suitable for generating test materials in bulk offline.
• The warehouse provides a command-line interface that allows you to save one or more pieces of text as local audio files.
• The example WebUI facilitates testing local models in a browser, but it is an development example only – it is not a production platform that includes accounts, queues, and SLAs.
• The official repository also provides Colab examples; running them in the cloud is affected by session duration, GPU quotas, and external storage rules.

Input, parameters, and output

Local installation and usage guide

1. Prepare a Python 3.11 environment; if a GPU is required, make sure that the graphics driver, CUDA, PyTorch, and the available video memory are compatible with each other.
2. Clone the code from 2Noise’s official repository, or install the ChatTTS package available on PyPI; do not obtain unknown executable files from websites with similar names.
3. Install project dependencies in an isolated virtual environment; before running it for the first time, check the version of those dependencies as well as any known security issues.
4. Initialize the ChatTTS.Chat object and call its load method to load the model; the load_models function mentioned in older tutorials may be outdated.
5. First, run infer using short sentences to verify that the model download, inference device, and the process of saving audio at 24000Hz are all functioning properly.
6. When it is necessary to fix a role, the speaker embedding is sampled and saved; then the same embedding is used to test the consistency of multiple segments of dialogue.
7. Use sentence-level or word-level control markers to adjust speech, laughter, and pauses; change only a small number of parameters at a time while retaining the comparison samples.
8. Batch tasks include text cleaning, length limitation, retry on failure, and manual audio review, in order to prevent abnormal inputs from causing memory exhaustion or low-quality audio.
9. Before making it available to the public, synthesized speech should be marked, and permissions, rights related to the individuals involved, and any content-related risks must be checked; commercial projects are not allowed to use the weights associated with the current non-commercial model.

Hardware and performance

The official performance figures are intended only for estimating performance in the development environment; they do not guarantee the throughput in a production setting. Formal testing should be carried out by taking into account factors such as the length of the text, the number of concurrent tasks, the number of speakers, and the hardware used.

Capacity boundaries and known limitations

• Autoregressive models can lead to speaker drift, unstable audio quality, abnormal pauses, or significant variations in the outputs for the same sentence.
• The project suggests taking multiple samples to obtain the best results, which means it is not suitable for real-time, critical broadcasts that require no verification.
• English is still provided with experimental support; complex proper nouns, abbreviations, numbers, mixed languages, and long sentences may require preprocessing.
• Fixing the speaker embedding can only improve the consistency of the sound quality; it cannot verify information such as identity, age, gender, or emotional state with accuracy.
• The currently available models do not offer complete zero-sample sound cloning, arbitrary emotion control, or guaranteed formal streaming generation.
• Synthesized speech does not guarantee the absence of high-frequency noise or artifacts, nor is it suitable for broadcast-quality production; further noise reduction can also alter the tone of the voice.

Roadmap feature

Price and usage costs

2Noise does not offer any official paid cloud packages for ChatTTS; the code and research models can be downloaded freely. The term “free” here means that no subscription fee is required for using this project, but it does not imply that there are no costs associated with hardware, cloud GPUs, storage, electricity, or maintenance.

The website with the same name does not have any public pages related to ChatTTS subscriptions or billing, and it cannot serve as a source for obtaining commercial licensing from 2Noise. Any paid versions, hosted APIs, or clients must have their developers, model licenses, and data policies verified separately.

APIs, SDKs, and deployment methods

• When exposing ChatTTS as a web API on your own, it is necessary to handle queues, timeouts, memory isolation, input filtering, authentication, and abuse monitoring.
• The OpenAI API notebooks in the official repository are deployment or compatibility examples, and they do not represent the public interfaces provided by 2Noise.
• The community WebUI, API encapsulation, and images are not official products; their versions, licenses, and security qualities vary.

Open-source licenses and commercial restrictions

• \"Open-source code\" does not mean that the model weights can be used for commercial purposes; both licenses must be followed separately.
• The model card specifies that it is intended for educational and research purposes only; the generated audio must not be used in commercial advertisements, paid products, customer service, or profit-driven content.
• AGPLv3+ imposes specific requirements on network services and modified versions; organizations that plan to deploy it publicly should have compliance experts review their actual architecture.
• If the business requires commercial voice services, it should choose a model whose license explicitly permits commercial use, or obtain written authorization from the rights holder.

Privacy, security, and risks of misuse

When deployed locally, the text entered and the generated audio can remain on one’s own device or server, which represents a significant privacy advantage over third-party cloud services. The project does not provide any privacy policies regarding user accounts, online generation, or cloud storage, as 2Noise does not operate an open-generation platform.

• Model files usually need to be downloaded from external model hosting platforms, and the deployment environment still generates network requests, cache data, and records of dependency installations.
• The web page with the same name uses an embedded third-party service for handling interactions; sensitive text should not be submitted without first checking that service’s policies.
• Third-party App Store applications, community APIs, and mirrors handle their own data; therefore, the criteria used for 2Noise’s local projects cannot be applied to them.
• Generating voices that resemble those of real people can be used for impersonation, fraud, and deception; therefore, authorization for the use of such voices and scripts is required, and synthetic content must be clearly labeled.
• The project incorporates high-frequency noise and plans to release a detection model, but this does not guarantee that every audio segment can be detected, that watermarks will be applied, or that the content cannot be misused.
• External APIs should limit the length of requests, the number of concurrent requests, and the presence of hazardous content; they must also record necessary audit information, while avoiding the storage of unnecessary personal data.

Suitable for users and typical use cases

• Speech researchers: Evaluating conversational TTS, multiple speakers, and prosodic control.
• Python developers: Create local, non-commercial prototypes or research tools.
• Universities and laboratories: Conduct voice experiments in Chinese and English within the scope of the permits granted.
• LLM application researchers: Generate conversational responses for dialogue assistant prototypes and test interactions.
• Audio technology learners: Study sampling parameters, speaker embedding, and audio storage processes.
• Not suitable for commercial products, advertisements, paid content, and unauthorized voice imitations.

Advantages and limitations

Summary

ChatTTS is suitable for researching conversational speech, multiple speakers, and fine-grained prosodic control; the official repository provides a complete getting-started guide ranging from Python inference to a local WebUI.

When using it, it is necessary to keep the official 2Noise project, the website with the same name operated by NEXGOE LLC, the community space, and third-party applications separate from one another. At present, the weighting scheme is only applicable for educational and research purposes as well as for non-commercial use; special caution is required for commercial projects, the promotion of sound cloning, and the online creation of sensitive content.