Convai

What is Convai?

Convai is a real-time AI character platform developed by Convai Technologies Inc.; it helps in creating NPCs or virtual beings for games, XR applications, virtual worlds, and training programs – entities that are capable of communicating, perceiving their environment, and taking actions.

It combines character design, knowledge, language models, speech, vision, animation, and runtime interfaces; developers can create characters using no-code tools and then integrate those characters into Unity, Unreal Engine, or web-based applications.

Main functions

Role-based thinking and knowledge base

• The character creator allows you to set the name, background story, goals, personality, speaking style, and behavioral boundaries, in order to define the character’s identity and the way it responds.
• The knowledge base allows for the uploading of project-related materials, enabling characters to provide responses based on a specific world view, product, course, or task, rather than relying solely on the knowledge contained in general models.
• Narrative design and action configuration allow for the definition of events, objectives, and actions that can be invoked, so that the dialogue can drive the progress of tasks or the game flow.

Real-time voice and multilingual conversations

• It receives the player’s voice or text input during operation, and returns text, synthesized speech, as well as character actions associated with the response.
• The platform offers over 60 languages and more than 500 voice options; the specific languages, voices, and the number of third-party voice credits available depend on the current plan.
• The real-time experience is influenced by the network, speech recognition, model response, and rendering process; it is therefore necessary to test the latency in the target regions and on the target devices before releasing the product.

Vision and environmental perception

• Visual capabilities allow the system to receive camera feeds, first-person perspective views, or information about the game environment, enabling the character to describe objects and respond in line with the surroundings.
• Developers can pass scenario metadata, objects, and status to the character, so that the responses are related to the current location, visible objects, or events.
• Visual recognition may miss objects or misjudge situations; when it comes to safety, educational assessment, and real-world operations, one cannot rely solely on automated results.

Actions, animations, and lip syncing

• Characters can generate gestures, facial expressions, lip movements, and server-side animations in response to dialogue, and they can also carry out complex sequences of actions defined by the developers.
• Action interfaces are suitable for tasks such as opening doors, picking up objects, moving, giving directions, or playing animations, but the available actions must be safely exposed to the characters through the project code.
• Avatar Studio allows for the configuration of appearance, clothing, scenes, lighting, cameras, and animations within a web page, thereby reducing the workload related to art design and scripting during the prototype stage.

Memory, evaluation, and role versions

• The paid plan offers features for long-term memory and role versions, allowing characters to retain information across sessions and to manage different versions after modifications.
• The Professional plan and higher offer a session evaluation feature, which is suitable for assessing the quality of interaction, role consistency, and task completion.
• Memory increases the risk of personal information and erroneous data being retained for long periods; therefore, projects should limit the content that can be recorded and include mechanisms for deletion, correction, and restarting.

Code-free experience and motion capture

• Playground is used to create, adjust, and test characters; it is suitable for verifying character traits, knowledge, and dialogue effectiveness before writing code.
• Convai Sim is designed for large-scale simulations, enabling characters to navigate within a scene and interact with objects.
• XR Animation Capture allows for the recording of human movements using Meta Quest devices, with the resulting animations being used for characters in Unity.

Input and output

Usage process

1. Register an account and create a character, filling in the background story, personality, goals, language, and speaking style.
2. Upload knowledge materials that are permitted for use, and set the topics that characters can answer on, as well as the prohibited content and narrative rules.
3. In the Playground, test the character using multiple rounds of text and voice input to check facts, tone, latency, and abnormal inputs.
4. Choose the Unity SDK, Unreal Engine plugin, Web plugin, or API, create a project key, and integrate it following the documentation.
5. Map scene objects, actions, animations, and visual information to characters, and restrict the available operations and their permissions.
6. Release only after conducting tests for concurrency, quotas, privacy, content security, and failure recovery in the target device and network environment.

Platforms and integration

Price packages

Each interaction is counted as one user input and one role response, and this applies to both voice and text inputs. Unused monthly interactions are not carried over, and the concurrency limit is determined based on the total number of roles and users under the same API key.

The annual price for Business plans is currently marked as a \"limited-time offer\"; this price does not include any applicable taxes. The region, account type, promotions, and payment cycle can affect the final amount, so it is necessary to refer to the actual billing page or sales contract for accurate details.

Key differences in the plans

API, SDK, and open-source status

• The platform provides API references, Unity SDKs, Unreal Engine plugins, Web plugins, module frameworks, and sample projects, which can be used to customize the integration at runtime.
• The current Unreal Engine SDK and its older plugin repositories are licensed under the Apache 2.0 license, while some Unity WebGL, Web SDK, and example repositories do not specify a license.
• The fact that a code repository is publicly accessible does not mean that it can be copied or used for commercial purposes without restriction; repositories lacking a license should be treated under default copyright protections.
• Convai’s hosting platform, models, accounts, and cloud services are not an entirely open-source product; the fact that the SDKs or plugins are open source does not mean that the platform itself is open source.

Content rights, privacy, and security

• The terms classify role settings, knowledge, scenarios, voice, text, and visual information as customer content, and state that the customer retains ownership of both the customer content and the AI-generated content.
• The Creator package grants Convai a worldwide, royalty-free license to store and use customer content as well as AI-generated content for the development and improvement of its services, unless otherwise agreed in writing.
• The Enterprise package does not collect, store, or use user-generated data without explicit consent; the specific procedures for handling such data are determined by the enterprise agreement and settings.
• The privacy policy states that information related to account communications, devices, IP addresses, usage patterns, location estimates, and interactions with various roles is collected, and this information may be shared with service providers involved in cloud, hosting, and analysis services.
• Upon account closure, the policy states that the account data will be deleted within 30 days; if the account remains inactive, its data will be removed after two years, while other data can continue to be retained for business and compliance purposes.
• Data may be transmitted to the United States; when deploying roles for end-users, it is necessary to provide notifications, obtain the required consent, and address the rights of those end-users.
• The platform offers content security measures, as well as secure, compliant, and locally deployable options for enterprise solutions; however, network transmission and generative models cannot guarantee error-free operation or absolute security.

Payments, Refunds, and Usage Restrictions

• Paid subscriptions are charged on a billing cycle basis, and payments are generally non-refundable unless otherwise agreed upon by both parties; taxes must be paid separately.
• The price increase usually takes effect from the next billing cycle; the cancellation date, downgrade procedures, and remaining credit amount should be checked on the account settlement page.
• Creating multiple accounts to claim the free quota repeatedly is prohibited; abnormal use may result in standard charging or service termination.
• Character responses, visual recognition, and action decisions may be inaccurate; therefore, in areas such as medical care, law, finance, security training, or the control of real devices, human supervision and deterministic rules are necessary.
• Concurrency is aggregated based on API keys, and multiple roles or a large number of end users share the same capacity limit; capacity testing should be conducted using peak traffic levels before going live.
• Third-party voices, models, avatars, engines, and cloud rendering may have additional fees, licensing terms, and data processing requirements.

Frequently Asked Questions

What is Convai mainly used for?

It is used to create AI NPCs and virtual avatars that can engage in voice or text-based conversations, perceive their surroundings, execute actions, and display animations; it is suitable for use in games, XR applications, training programs, and virtual worlds.

What is the data limit included in the free plan?

The Free plan includes 100 interactions per month, 60 minutes of cloud rendering time, and 1 concurrent role; any unused interactions are not carried over.

Does Convai support Unity and Unreal Engine?

Supported. Developers can use the Unity SDK, Unreal Engine plugins, as well as web plugins and APIs to integrate characters into various operating environments.

Is Convai an open-source product?

It is not a fully open-source product. Some Unreal plugins are licensed under the Apache 2.0 license, but the hosting platform and repositories without licenses cannot be considered open source as a result of this.

Can the generated content be used for commercial purposes?

The terms state that the customer owns the customer-generated content as well as the AI-generated content; however, the user must ensure that the input materials, third-party voices, avatars, animations, and the final output do not violate anyone’s rights, and must also comply with the specific terms of the plan and any third-party licenses.

Will Convai use role data to improve its services?

The terms of the Creator package permit storing and using customer content as well as AI-generated content for the development and improvement of services, unless otherwise agreed in writing; data processing under the Enterprise package is determined by explicit consent, settings, and enterprise agreements.