DeepMotion

What is DeepMotion?

DeepMotion is an AI-based human motion capture and 3D animation platform developed by DeepMotion, Inc. Its main products include Animate 3D, which converts videos into animations, and SayMotion, which transforms text into animations. It enables users to create character animations that can be imported into game engines, 3D software, and film production processes, using either ordinary camera footage or natural language.

The platform focuses on handling the movement of human figures and humanoid characters; it is not a code-generation tool, nor is it a modeling software that creates complete 3D models directly from images. The quality of the movements remains affected by factors such as the angle of shooting, occlusions, clothing, lighting, the proportions of the characters, and subsequent reorientation.

Comparison of core products

Main features of Animate 3D

Single-camera markerless motion capture

Users upload videos of ordinary people, and the system identifies the movements of their joints to create bone animations; no motion capture suits, sensors, or specialized studios are required. It is suitable for converting videos taken with smartphones, archival material, as well as videos of dances, sports, and performances into editable motion sequences.

Whole-body, hand, and face tracking

Animate 3D can track movements of the entire body, and it is possible to enable hand and face tracking as needed. Animating the whole body costs 1 point per second, while hand and face tracking each cost an additional 0.5 points; therefore, enabling all of these features at the same time increases the consumption.

Multi-person motion capture

In the same video, it is possible to select multiple characters and assign roles to each of them; the maximum number of characters allowed varies depending on the package, ranging from 2 to 8. When characters overlap, cover each other, or are outside the frame, the stability of tracking is reduced. For tasks involving multiple characters, it is necessary to ensure that their positions and the background are clear.

Physical and motion correction

The platform offers foot locking, physical filtering, motion smoothing, slow-motion processing, root bone settings, and speed adjustment. The Rotoscope Pose Editor enables frame-by-frame adjustment of joints; users can compare different parameters by rerunning the process, but excessive smoothing may result in the loss of details from the original motion.

Custom roles and redirection

Users can choose the default role, create a profile picture, or upload custom humanoid characters in FBX or GLB format; the system then redirects the captured movements to the target skeleton. If the skeleton, proportions, and naming of the character do not meet the requirements, further adjustments will still be necessary using Blender, Maya, Unity, or Unreal Engine.

Export in multiple formats

Common output formats include FBX, BVH, GLB, MP4, JPG, PNG, and GIF; Professional and Studio also offer DMPE. FBX and BVH are suitable for animation workflows, while GLB is useful for web pages and real-time applications. Videos and images are primarily used for previewing and communication.

Main functions of SayMotion

Text generation of 3D animations

Users describe in text the characters’ actions, speed, emotions, direction of their limbs, and style of performance; SayMotion then generates multiple 3D version of those actions. It is useful for quickly finding performances that are not available in existing motion libraries, and it can also be used to create initial drafts for scripts, game prototypes, and virtual characters.

Action insertion and extension

Inpainting allows new segments with textual descriptions to be inserted at the beginning, middle, or end of a movement, and it can be used to extend existing animations or make partial modifications to them. Users can also combine different movements, and control the transitions through adjustments to length, foot locking, physical filtering, and smoothing.

Hints for assistance and story structuring

Prompt Enhancer helps to supplement descriptions of actions, while Story Mode organizes longer sequences of actions through sequential instructions. The generation model may still misinterpret complex spatial relationships, interactions between two characters, or precise timing; key scenes need to be created step by step, selected, and manually adjusted.

Video shooting requirements

• Use real people or videos with clear, human-like movements; cartoons and abstract characters are not suitable as subjects for capture.
• The camera should be placed at a distance of about 2 to 6 meters from the person, with its position remaining fixed; normal movements should be directed straight at the camera.
• Ensure that the entire body, half of the body, or the face, which needs to be tracked, remains visible at all times, preventing props, furniture, and other characters from blocking the joints.
• For ground movements, a three-quarters angle can be used to reduce overlap between different body parts.
• Use neutral and even lighting, avoiding overly bright or dim light, motion blur, and strong backlighting.
• Create a clear contrast between the characters and the background; the clothing should not be loose enough as to cover key joints such as the knees and elbows.
• Select videos that meet the requirements regarding length, file size, resolution, and frame rate; the current values are those indicated on the pricing page.

Animate 3D Tutorial

1. Register and verify your DeepMotion account, then select Animate 3D and your current plan in the portal.
2. Prepare a video that meets the shooting guidelines by first removing irrelevant sections and ensuring that the characters, joints, and the ground are clearly visible.
3. Create an animation task, upload a video, and select single or multiple people, as well as the area to be animated – the whole body, hands, face, or the target character.
4. Set the foot locking, physical filtering, root skeleton, smoothing, and output parameters, then submit it for processing.
5. In the previewer, check for foot slippage, overlapping, joint jumping, and character mismatches by comparing with the original video.
6. If necessary, use the Rotoscope Pose Editor to make corrections frame by frame, or use rerun to adjust the settings and generate the content again.
7. Download FBX, BVH, GLB files or preview the media, then import them into the target 3D software or game engine to carry out redirection and cleanup.

SayMotion usage process

1. Select a character and enter the action, direction, speed, emotion, and duration; avoid requesting too many conflicting details at once.
2. Select the number of variants and generate them; compare rhythm, posture, steps, and spatial direction.
3. Use cutting, merging, or Inpainting to add new actions and build longer performances segment by segment.
4. Enable smoothing, physical filtering, and foot locking to check whether the transitions are smooth.
5. Export in the desired format, then use animation software or engines to reposition the characters, adjust the camera angles, and make final tweaks.

Suitable for users and scenarios

• Independent game developers: A process for quickly converting actor videos into prototype animations and importing them into Unity, Unreal Engine, or Roblox.
• 3D Animator: Creates basic skeletal animations for Blender, Maya, 3ds Max, or other tools.
• Film, television, and virtual production teams: Extract movements from live footage or archived videos, thereby reducing the need for manual creation of key frames.
• Creators in the fields of dance, sports, and education: they convert live demonstrations into digital avatar movements or visual content.
• VR, AR, and virtual avatar product teams: Integrate human body tracking into applications via APIs or real-time SDKs.
• Content creators: They use text to create short movements, and then combine these with video footage to develop more elaborate character performances.

Price of Animate 3D

The figure below shows the monthly price equivalent to an annual payment as displayed on this page; the monthly payment amount may vary. Prices, taxes, settlement methods, and any promotions are subject to those indicated on the actual settlement page and in the account dashboard.

The points are reset on a monthly basis and are not carried over; typically, 1 point corresponds to 1 second of full-body animation or 1 3D pose. The help center still mentions a free allowance of 60 seconds per month, which does not match the current figure of 25 points indicated on the pricing page – the actual amount available after logging in should be taken as the reference.

SayMotion price

Animate 3D and SayMotion have separate subscription and quota systems; it is not permissible to use their points or benefits interchangeably. The commercial licenses as well as the API options for each subscription plan should also be verified separately before purchase.

Free Points Program

Users of Animate 3D can earn points by creating English labels for newly uploaded actions and adjusting the joints in the pose editor; these labels and adjustments are used to improve Animate 3D and SayMotion, and users should understand that this is a contribution mechanism intended for improving the models.

• The quality of tags is evaluated based on the type of action, style, and the completeness of the description; currently, the evaluation models are primarily in English.
• Posture correction is scored based on the degree to which bone length and joint angles match those of a human mechanical model.
• Users may recover some or all of the points spent on tasks, but not all animations are available for free automatically.
• The method of acquisition does not change the license; the commercial rights remain determined by the package used at the time of creation.
• The points earned are reset at the end of each monthly subscription period and are not carried over to the following month.

Commercial licenses and refunds

The freemium version is intended for personal, non-commercial use only; the paid plan grants a permanent, non-exclusive, fully paid, royalty-free, and worldwide commercial license for the products created during the subscription period. The animations can be incorporated into larger works such as games, software, applications, videos, or films for distribution.

• Animation files cannot be sold, relicensed, or made available for download as standalone digital assets.
• It is not allowed to sell almost identical actions as asset packs simply by changing the character.
• End-users should not be able to extract animation assets from the final product without performing reverse engineering.
• All purchases are final transactions with no refunds; users can cancel their subscription, and the plan remains valid until the end of the current billing cycle.
• The upgrade will incur a proportional difference fee, and the appropriate amount for the new plan will be added immediately.

API, SDK, and GitHub

DeepMotion offers Animate 3D and SayMotion REST APIs for batch uploading, creating, querying status, receiving Webhooks, managing roles, and downloading animations. The Animate 3D API uses a Client ID and Client Secret to obtain access tokens; access to this API as well as pricing details are subject to application.

1. Register for a portal account and submit an access request in the API section; wait to receive the client credentials and package.
2. Store the keys securely on the server and obtain tokens through the authentication process; do not save the keys in the frontend or in public repositories.
3. Upload a video or submit a text task, and save the identifier of the returned task.
4. The completion status is obtained through polling or Webhooks, after which the resulting files such as FBX and BVH can be downloaded.
5. Add retry, timeout, quota, content validation, and file cleanup policies in the production environment.

The vendor provides Python and JavaScript/TypeScript SDK packages, along with two API documentation sets and example repositories. No standard open-source license metadata was found in those repositories, and the core cloud platform, models, and Real-Time SDKs are not made available in source code form; therefore, the DeepMotion product itself cannot be considered open source.

Platforms and integration

Privacy and data security

DeepMotion handles account and contact information, devices and logs, Cookies, usage metadata, as well as uploaded videos and 3D models. By default, the content uploaded is not made available to other Animate 3D users, but it is still processed in the cloud and may interact with the service provider’s data infrastructure.

• Infrastructure and service providers include Google Cloud, SendGrid, Okta, and Citrix Podio, among others.
• The information may be processed in the United States or in other countries where the service provider’s facilities are located, which involves cross-border data transfer.
• After the account is terminated, the personal profile can be retained for a commercially reasonable period of time through backups, archiving, or auditing; no specific number of days is specified.
• Content that is voluntarily made public may not be completely removed; it is necessary to verify the rights related to portraits, performances, and the permissions for using such materials before uploading them.
• The tags and action corrections of the free points program are intended to improve the product, and they are not suitable for including descriptions that are subject to confidentiality agreements.
• Enterprises and sensitive projects should further confirm aspects such as data storage, deletion, exclusion from training datasets, sub-processors, and security contracts.

Advantages

• Basic human motion capture can be achieved using ordinary videos, thereby reducing the requirements in terms of hardware and venue.
• It covers both video capture and text generation, and is capable of merging and editing these two types of content.
• It supports the entire body, hands, face, multiple people, custom characters, and various animation output formats.
• It offers practical post-processing features such as frame-by-frame pose correction, foot locking, physical filtering, and motion smoothing.
• Web pages, REST APIs, and real-time SDKs cover individual creation, batch processing, and in-app integration.

Restrictions and Precautions

• Obstructions, moving cameras, low contrast, loose clothing, and complex ground movements can cause joint tremors, slipping, or overlapping movements.
• The generated actions still require the animator to check the skeleton, redirection, contact points, rhythm, and character proportions.
• Text generation does not guarantee an accurate reproduction of complex choreographies, multi-person interactions, or professional sports techniques.
• The free plan does not include a commercial license, and the paid license also prohibits the independent resale and redistribution of animation assets.
• The points are reset every month, and tracking of the hands and face consumes additional points.
• The old pricing information in some help documents is inconsistent with the current prices; it is necessary to check the backend dashboard and the payment page before making a purchase.
• The core models and cloud platform are not open source; the availability of API repositories does not mean that it is possible to host the services oneself.

Frequently Asked Questions

Does DeepMotion require motion capture suits?

It’s not necessary. Animate 3D extracts movements from ordinary single-camera videos of human bodies, but high-quality footage, joints that are not obscured, and a stable camera still have a significant impact on the results.

Can the free version be used for commercial purposes?

No. Freemium is intended for personal, non-commercial use; commercial projects require a paid plan in order to create animations.

How many animations can be created with 1 point?

Typically, 1 point corresponds to 1 second of full-body animation or 1 3D pose. Hand and face tracking require an additional 0.5 points per second each.

Will the points be carried over?

No. The points for monthly and annual subscription plans are reset at the start of each subscription month, and any unused points are not carried over to the next period.

What input and output formats are supported?

Animate 3D can accept MP4, MOV, and AVI videos; the common export formats are FBX, BVH, GLB, MP4, JPG, PNG, and GIF. The specific formats, maximum duration, and file size limits vary depending on the package chosen.

Can the generated actions be packaged as assets for sale?

Animations cannot be resold or relicensed as standalone digital assets. They can be integrated into complete works such as games, applications, videos, and movies, and the resulting products can be released in accordance with the relevant terms.

Is DeepMotion open source?

The core products are not open-source software that has been verified. The vendor provides API documentation, examples, and information on how to integrate the SDK, but it does not offer an open-source license for its core cloud platform, models, or real-time SDKs.

Will I get a refund if I cancel the subscription?

The terms state that the purchase is final and non-refundable. No further charges will be applied in case of cancellation, and the paid plan remains valid until the end of the current billing cycle.

Summary

DeepMotion is suitable for quickly creating character animations using ordinary videos or text, which can then be integrated into game development, film production, and 3D creation processes. When choosing a package, it is necessary to take into account factors such as points, video length, number of users, export specifications, and commercial licensing; moreover, time should be allocated for tasks related to animation refinement and integration into the project.