Social Media

【Tutorial】Use DeepMotion Mocap to Drive MMD Model

Related AI


In this video

DeepMotion Mocap Animation: A Comprehensive Guide for MMD Models


DeepMotion is an innovative tool that allows you to create realistic motion capture (mocap) animations for MikuMikuDance (MMD) models. This technology transforms video input into a detailed animation. This article will guide you through the process, including importing your animation into Blender, modifying shape key values, and tweaking recorded values.

Creating Animation Using DeepMotion

To start, navigate to the DeepMotion website. The platform allows you to create an animation from any video. Upload your chosen video manually or by dragging and dropping it into the browser.

DeepMotion offers several customization features. You can activate physics filter, face tracking, and hand tracking for a more detailed output. If your video contains only the upper body, remember to select the 'Upper Body Only' option. Set the motion smoothing value to 0.1 for optimal results.

After setting your preferences, click the 'Create' and 'Start Job' buttons. The platform will then generate your animation. Upon completion, you can download it as an FBX file.

Importing Animation into Blender

After downloading your animation, the next step is to import it into Blender. This software allows you to fine-tune your animation, ensuring it aligns perfectly with your MMD models.

You will need to use a Python script to adjust the shape key values and rename the old shape key names to MMD formats. Keep in mind that different MMD models may have varying shape key names, so you might need to manually modify these.

Tweaking Shape Key Values

The Python script you will use in Blender has two main functions: adjusting shape key values and renaming them. For example, you might need to find which MMD shape key corresponds to the 'Jaw Open'. By adjusting the value, you can see which part the shape key controls.

To fine-tune the animation, you can create a new mapping by copying the MMD shape key. You may also need to adjust recorded values. This process can involve taking the difference between two values and multiplying them by a coefficient. For instance, the MMD mouth shape key might correspond to two shape keys in the DeepMotion model, including 'Jaw Open' and 'Mouth Close'.

Exporting and Importing into Unity

After adjusting the animation in Blender, export it as an FBX file. The face animation will be used for the model's facial expressions, while a separate T-pose animation will be used to control the model's body movements. Import these files into the Unity editor.

In Unity, set your MMD character to humanoid and create a timeline game object. You can add an animation track to your MMD character in the timeline. You can then add the body animation to it.

For the face animation, add an override track and drag your face animation to it. To ensure proper function, you may need to adjust the name of the model's face mesh to match the name recorded in the animation clip.

Final Comparison and Conclusion

After these steps, you should be able to see your MMD model moving with the new mocap animation. If done correctly, the body and face animation will work in harmony, resulting in a realistic and engaging performance.

Using DeepMotion and following these steps, you can create high-quality mocap animations for MMD models. Whether for gaming, film, or other creative projects, this guide will help you harness the power of mocap technology to bring your characters to life. As always, remember to stay creative.



We Use Cookies to Enhance Your Experience

Our website uses cookies to provide you with a personalized experience and to improve our website. By clicking 'Accept', you consent to our use of cookies. To learn more about how we use cookies and your options, please see our Cookie Policy page.