Yesterday’s post here https://josephstankowicz.wordpress.com/2013/06/07/skinned-animation-content-creation/ details the process of creating a skinned mesh animation, today I will cover the runtime side of things, and maximizing performance of skinned mesh animations.
To compute the render position of a vertex of a skinned mesh, the transformations of bones that influence the vertex need to have their hierarchy’s concatenated, and the final positions blended based on the weighting of the bone on that vertex, as well as the original vertex position. A simpler explanation of this is, the position of each bone needs to be figured out every frame, and each corner of each triangle rendered on the character takes bone positions relevant to itself and places itself relative to those bones. In most engines, every time the animation is updated, the position of each bone is updated within its hierarchy. Then the final vertex position is figured out by taking these final bone positions, multiplying them against the weighting values, and concatenating that with the base vertex position. Within this entire process, there are multiple points to address performance and memory concerns with skinned mesh animations.
We will start with bone count. The more bones in the skeleton used by a skinned mesh, the more data to be updated when the animation changes. The positive tradeoff here is more visual fidelity, more power for the animator to achieve the desired look of the animation. The downsides are memory and performance concerns. Every bone added to a skeleton is generally going to add another curve be stored in the animation file. This is more data to be stored in memory, and reducing bone counts can get you some runtime memory back. Every bone added to the skeleton is also another bone to be updated every time the animation updates its position, taking up more time for the CPU. While this might not be too expensive for an individual character, when you have many characters on screen, the effect of adding or removing a single bone can have a small but noticeable effect on runtime performance.
The next piece of the performance puzzle for animating skinned meshes is the quantity of keyframes. Keyframes are useful during content creation because they allow the animator finer control of exactly where things are on a given point in the animation. The downside is this keyframe data is stored in the animation, and includes the position, rotation, and scale, as well as the tangent information. This will cost you runtime and on disk memory. This information for an individual bone on an individual keyframe is not too expensive, but it adds up quickly as the number of keyframes in an animation rises, and the quantity of bones. Another note on this is animation length. While it does not have a direct bearing on animation memory cost, generally animations follow a pattern of so many key frames per second, so the longer the animation, the more keyframes, and the more memory the animation needs. Memory can be reclaimed from animations by removing keyframes manually, by removing bones from the skeleton, and by adjusting the asset import settings for the animation. In the animation tab of the importer for the animation files, there is an animation compression drop down field. Turning on animation compression can affect the visual fidelity of the animation, so the rotation, position, and scale error fields will need to be tweaked to find a balance between memory usage of the animation and visual quality. The source animation might also need to be updated to become more compression friendly. If the animation moves the character a long distance, compression can cause errors. If there are parts of the character that should stay still, you might see them visually float about, this is most common with feet on a character standing still in an idle animation.
For the more advanced readers, you will probably be wondering how the transformation information for an animation is actually stored and used. Obviously a transformation matrix of four floats by four floats, is a lot of information to be stored per bone on the skeleton. This gives two places to reclaim memory: using a smaller data type than float, and reducing the total count of data stored. You have no way to change what Unity does for animations on the back end, but if you ever end up working on engine and rendering code, this becomes relevant. A smaller data type to consider for something like this would be a sixteen bit fixed point data type, instead of a thirty two bit floating point data type. This does reduce the range of accuracy for the animation, so it will need to be a setting that can be enabled and disabled per animation. The second place memory gains can be had is through using quaternions to represent bone positioning instead of transformation matrices. A quaternion can represent a point and a rotation using just four values, instead of the sixteen values of a transformation matrix. Quaternions also do not suffer from gimbal lock problems, and are much easier to composite, making them more functional way to represent a rotation than using a vector3 of euler angles. Unity does use quaternions internally to represent the rotation of transforms, you can access this through Transform.rotation.
How can you identify if animations are a CPU and framerate bottleneck for your game? The easiest way is to use the Unity profiler, if you have access to Unity Pro. If your framerate issues are caused by animation, you will see animation and skinning related functions taking up a lot of time. If you do not have access to Unity Pro, then you will need to use other techniques. I went into detail in this blog post https://josephstankowicz.wordpress.com/2013/05/29/framerate-is-how-many-times-your-game-can/. A quick test would be to have a very short animation setup with minimal key frames for all of your characters, such as a t-pose, and run your game with all calls to play animations switched to that animation. Another option would be to swap out your characters for testing characters for far fewer bones. If there is a noticeable improvement in framerate, then animation and skinning was probably where you were spending a lot of frame time.
If you decide you want to reduce the CPU load of your animations, you have a few options. You can reduce bone counts, which will involve a lot of work for the animators. You can reduce the number of character on screen, especially unique characters where Unity cannot cache bone information. While this might not work with all game designs, there are still a lot of games where you can find a way to limit the number of characters spawned in a level. If you have the time, and do not want to reduce the visual quality of your animations on the higher end devices, but wish to support lower end devices, then a level of detail system for your character models, skeletons, and animations will let you run your game with the lower bone count characters on the lower end devices where CPU performance is more of a concern. The worst case you might decide that you cannot make any changes to your animations and skeletons, and accept the higher CPU cost while looking for performance savings elsewhere.
What about identifying memory costs of animations? Build yourself a debug menu, wrapped around the #if ENABLE_PROFILER check. Then you can iterate over all loaded animations, and call the Profiler.GetRuntimeMemorySize while the game is running on target, and you can view how much memory each asset is taking. The Unity script reference for this is here http://docs.unity3d.com/Documentation/ScriptReference/Profiler.GetRuntimeMemorySize.html. Using this, you can identify what the final memory cost of your animations are after they are compiled, and loaded on device. You can also use this information to test changes to see how they affect memory usage of your animations.
Once you have a way to measure memory costs of your animations, you might decide you want to reduce these values to get more runtime memory for other content. There are a few ways you can address this. The first is to investigate the animation compression and keyframe reduction settings. These do affect the visual quality of the animations, so you will want to do thorough testing to make sure the tradeoff is acceptable. The next option is to have your animator manually remove keyframes to reduce the size of these animations. Sometimes they might have extra keys, especially on values like scale that might never change. If you’re still looking for ways to reduce animation memory, and have engineering time available, a selective loading system can give you a lot of memory flexibility. For many games, there are going to be many animations you only need loaded during a specific moment. For a fighting game, you only need victory pose animations at the end of the fight. For a cinematic third person game, you only need cutscene animations loaded when the cutscene is playing. In cases like this, loading in with the bare minimum set of animations, and then loading and unloading animations as needed will allow you to have a large variety of animations available for a character without blowing out your memory budgets. Other cases to look for are animations relying on other characters, if your hero has special kill animations for different enemy types, you do not need special kill animations for enemies not in the loaded level. You can even extend this to an animation level of detail system, if you have multiple idle animations for your hero for visual variety, you can load only the first one on lower end devices. The options for improving CPU performance through bone reduction also helps with memory. Less bones means less curves in the animation and less keyframes, which reduces the memory usage of all animations for that character.
If you do decide to make tradeoffs to support lower end devices, you can use this as an advanced feature list for higher end devices. If it is a recently released higher end device in the category, such as a new iPad model you have as the only device that runs the highest bone count characters on, this can actually work in your favor for featuring with different app stores. Apple loves it when developers have “Only on the latest iPad” feature in games to help sell new devices.