Shulou(Shulou.com)06/01 Report--

What this article shares to you is about how to analyze the principle of Unity3D skeletal animation. The editor thinks it is very practical, so I share it with you. I hope you can get something after reading this article. Let's take a look at it.

Recently, I have studied the principle of the bone animation of the model in the game, and made a study note, which is convenient for everyone to study and discuss.

First, bone animation, joint animation, Keyframe animation

In the actual game, these three basic animations are most used.

In Keyframe animation, the model is a fixed pose in each Keyframe, which is equivalent to a "snapshot". A smooth animation performance can be obtained by interpolating and smoothing in different keyframes. One of the advantages of Keyframe animation is that it only needs to do interpolation calculation, compared with other animation, the amount of calculation is very small, but the disadvantage is also obvious. Based on the fixed "snapshot" interpolation calculation, the performance is greatly limited. At the same time, if the interpolation is not smooth enough, it is easy to appear spikes and other phenomena.

Joint animation is a kind of animation that appeared in the early days. In this kind of animation, the whole model is not a Mesh, but is divided into several Mesh, organized by the parent-child relationship, so that the Mesh of the parent node will drive the Mesh of the child node to transform, so that the layers of transformation relations can get the position of each child Mesh in different keyframes. Compared with Keyframe animation, joint animation depends on the animation data of each Keyframe, and the position of each Mesh can be calculated in real time, which is no longer limited to the fixed position, but because of the scattered Mesh, cracks are easy to appear at the joint of different Mesh.

Skeletal animation is a further type of animation, the principle of which is extremely simple, but it has great advantages in solving problems. The model is divided into two parts: bone Bone and skin Mesh. The basic principle can be described as follows: the skeleton of the model can be divided into basic multi-layer parent-son bones. Driven by animation key frame data, the position of each parent-son bone is calculated, and the vertices of the skin mesh are dynamically calculated by vertex mixing based on bone control. In bone animation, it usually contains bone hierarchy data, mesh Mesh data, mesh skin data Skin Info and animation Keyframe data of bones.

Second, the principle analysis of SkinnedMesh animation.

The key to understanding Skinned Mesh is the calculation process of skinning. the skin here is not the mapping texture that we commonly use in shader, but the mesh of the model, so skinning is actually the process of calculating the position transformation of mesh relative to the bone. If our bones do not change, then our mesh does not change, and the mesh at this time is equivalent to a static mesh. So the understanding of skinned refers to mesh: d with skinned data skin info. In the actual example, skin info mainly includes which bones affect the vertices on the current mesh and the influence weights of each bone. To borrow the expression in article 1, the whole transformation process of mesh vertices can be expressed in the form of matrix:

Vworld = Vmesh * BoneOffsetMatrix1 * CombineMatrix1 * W1 + Vmesh * BoneOffsetMatrix2 * CombineMatrix2 * W2 +... + Vmesh * BoneOffsetMatrixn * CombineMatrixn * Wn

Among them, BoneOffsetMatrix and CombineMatrix will explain the solving process in detail in the following explanation.

1. Bones in skeletal animation

In our common model modeling, art usually sets the model to a "big" font with horizontal hands and separate legs. Why do you want to use this way? This has something to do with the bones of our model. Usually there is a limit on the number of model bones (usually 30, which will be explained below). The model cannot be one bone, but if it is one bone, then the performance of the model will appear relatively single. if you want to show an animation similar to the human body, there will usually be multiple bones. For each bone, how to establish its organizational relationship is basically set up when modeling. Usually, the art will choose the pelvis of the model as the root bone of the model, so based on the root bone, the father-son relationship of each bone relative to the root bone can be derived. Through the space where the bone is located, we will choose the midpoint between the two feet as the origin, and then we will find that the root bone does not coincide with the origin, and then the art will build a Scene_Root as an additional bone, its position is the world origin, and the real root bone Bip01 will be the only child bone of Scene_Root.

Please ignore my soul painting: d

Based on the root bone, we can derive the parent-child transformation matrix of each bone relative to the root bone, which is usually converted into a rotation matrix without translation and zooming. of course, for example, one of your model animation is to lengthen the body of the model (such as the King of the Sea Thief: d), then you can add translation and zooming to the transformation matrix. At this time, there is a basic intuitive feeling for the whole model, the specific animation of the model depends on the bones of the model, it affects the whole body, and the movement of the whole model drives the external mesh to move together, which can show the bone animation of the whole model. In this way, we have a basic understanding of bone animation.

To borrow the words of reference 1, the essence of bones is actually a coordinate space. When we animate bones, the transformation of bones in key frames is mainly rotation matrix, so the transformation of bones is the rotation transformation of bone space. To put it simply, a bone animation brings transformation, which first acts on the root bone, affects the coordinate space of the root bone, and then recursively affects the sub-bones of the root bone. Based on this, we can understand the role of joints, joints are the origin of the bone's own space. The position of the joint is described by the position of the current bone in the bone space of its parent node. The rotation around the joint is the rotation of the bone space itself, so the transformation is transmitted recursively through the joints. In the expression of C++, you can define a basic bone class, which mainly contains the position in its own world coordinates, the position in the parent node, its first child node, and the pointer to its sibling bone. The code is based on article 1:

Class Bone {Bone* masked pFirstChild; Bone* masked pSibling; float massix, massiy, massiz; / / pos in parents' space float m_wx, m_wy, m_wz / / pos in world space / / public: Bone (float x, float y, float z): m_pSibling (NULL), m_pFirstChild (NULL), m_pFather (NULL), NULL x (x), m_pFather (y), massiz (z) {} / / void SetFirstChild (Bone* pChild) {m_pFirstChild = pChild; massipFirstChild-> m_pFather = this;} / / void SetSibling (Bone* pSibling) {m_pSibling = pSibling Masked pSibling-> m_pFather = masked pFathering;}}

In this way, when the skeleton of the parent node changes, the bone of the child node will change accordingly, which can be called UpdateBoneMatrix. This operation can be expressed by a method ComputeWorldPos, which can be implemented recursively in Bone. The code is based on article 1:

Class Bone {void ComputeWorldPos (float fatherX, float fatherY, float fatherZ) {m_wx = fatherX + missux; m_wy = fatherY + mroomy; m_wz = fatherZ + maliz; / / the parameter if passed by the sibling node with the parent node (m_pSibling! = NULL) m_pSibling-> ComputeWorldPos (fatherX, fatherY, fatherZ) if (massipFirstChildChild null) m_pFirstChild-> ComputeWorldPos (m_wx, m_wy, m_wz)}}

In this way, when the skeleton of the parent node changes, the skeleton of the child node will change accordingly, so as to get the latest position, orientation and other information, and the bone will change, which will drive the external mesh to change, so the whole model will show the chu to move. Based on this, you can understand why bones are the core of bone animation.

2. Skinning in skeletal animation

After talking about the bones, we can have a general understanding of the transformation of the bones of the whole model in the animation. at that time, the model is only internal, and the external performance is the change of the skin of the model. so the second part of bone animation is the calculation of skin. The skin here is the Mesh mentioned earlier.

First of all, what needs to be clear is the space in which the Mesh is located. When modeling, the Mesh of the model is in the same space as the bone, and each vertex in the Mesh is located based on the origin of the Mesh. But when the model is in motion performance, it does the corresponding action according to the transformation of the bone, and the corresponding vertices on the Mesh need to be transformed, so the vertices of the Mesh need to be converted to the coordinate space where the corresponding bones are located, and the corresponding position transformation, so the corresponding need to add skin information, that is, skin info, mainly which bones affect the current vertices, the weight of the influence, and so on. To borrow the expression of article 1, you can use C++ to represent a vertex class, the code is based on article 1:

# define MAX_BONE_VERTEX 4class Vertex {float BONE_VERTEX; / / local pos in mesh space float m_wx, m_wy, massiwz; / / pos in world space / / skin info int mboneNum; Bone* m _ bones [Max _ BONE_VERTEX]; float m _ boneWeights [Max _ BONE_VERTEX];}

Of course, this is just a simple statement, specifically, there will be a standard design in the engine. So how do our vertices calculate their position when following the bones? We need to introduce the concepts of BoneOffsetMatrix and Transform Matrix.

Previously, we have mentioned that vertices need to be attached to the bone for position calculation, but when modeling, the position of the vertex is based on the Mesh origin, usually, the origin of the Mesh is in the same coordinate space as the root bone of the model bone, then BoneOffsetMatrix is used to transform the vertices in the Mesh from the Mesh space to the bone space.

When modeling, for each bone, we can get its corresponding Transform Matrix (used to calculate the space of the parent node layer by layer), in which the Transform Matrix of the root bone is based on the transformation of world space, so for each child bone below, to calculate its Transform Matrix, we need to do a matrix joint multiplication operation. The final matrix multiplication result matrix is Combined Transform Matrix. Based on this matrix, the vertices can be converted from the bone space to the world space. On the other hand, the inverse matrix of this matrix (generally only considering the operation that can be inverted) is transformed from the world space to the bone space. Because the definition of Mesh is based on the Mesh origin, the Mesh origin is in the world space, so the inverse matrix is the required Offset Matrix, also known as Inverse Matrix. This inverse matrix is generally obtained in the initial position, which can be obtained by taking the inverse.

In the actual calculation, each bone may correspond to multiple vertices, if each vertex saves its corresponding bone transformation matrix, then a large number of vertices will report a lot of wrong transformation matrix. So we only need to save the current bone in the initial position, corresponding to the transformation matrix from the world space to its bone space, then each vertex corresponding to each transformation operation only needs to be operated with offset Matrix.

For Transform Matrix and offset Matrix above, rotation, translation, and scaling are included. In fact, offset Matrix depends on the initial position of the bone, which generally only includes translation (there is no animation at this time, so there is no rotation and scaling), and in animation, it is generally based on scaling (so most animation keyframes are represented by quaternions). It is included in the matrix for compatibility considerations.

Here is based on translation, do a basic skin calculation process, the code is mainly based on article 1:

Class BoneOffset {public: float m_offx, m_offy, massively offz; / / only consider translating} class Bone {public: BoneOffset* massiboneOffset; / / void ComputeBoneOffset () {m_boneOffset.m_offx-= massiwx; m_boneOffset.m_offy-= massively; m_boneOffset.m_offz-= m_wz If (m_pSibling! = NULL) mroompSibling-> ComputeBoneOffset (); if (m_pFirstChild! = NULL) mroompFirstChild-> ComputeBoneOffset () }} / / calculation of vertex classes class Vertex {public: void ComputeWorldPosByBone (Bone* pBone, float& outX, float& outy, float& outz) {/ / convert from mesh space to bone space outx = massif x + pBone- > massiboneOffset.malioffx; outy = massively + pBone- > massiboneOffset.massively; outz = massiz + pBone- > m_boneOffset.m_offz / / convert from bone space to world space outx + = pBone- > mleavwx; outy + = pBone- > mleavwy; outz + = pBone- > mleavwz;} / / void BlendVertex () {float m_wx = 0; float m_wy = 0; float m_wz = 0; for (int iS0; I < m_boneNum) in GPU Float tx +) {float tx, ty,tz; ComputeWorldPosByBone (m_bones [I], tx, ty,tz); tx * = m _ boneWeights [I]; ty * = m _ boneWeights [I]; tz * = m _ boneWeights [I]; m_wx + = tx; m_wy + = ty; m_wz + = tz }}}

A careful look at the above code, you can understand the overall skin transformation process, of course, here only use the translation transformation of the matrix transformation, if you consider the rotation and scaling, then go back to the original calculation formula. At this point, there is a detailed explanation for the bone transformation and skin transformation in the basic bone animation. Let's talk about how bone transformations are handled in Unity.

Third, Unity3D skeleton animation processing

The previous description of the bone transformation in the bone animation, the calculation of skin, are carried out in CPU. In the actual game engine, these are dealt with separately, the more general processing is to put the animation data driver of the bone in CPU, calculate the transformation matrix of the bone, and then transfer it to GPU for skinning calculation. In DX10, the general shader given the size of the register in 128the size of a transformation matrix for 4x4, if you remove the last line (0j0d0Power1) can be represented by 3 float, then at most, um, 42, if you consider performance optimization, do not fully occupy the size of the register, then it will generally be limited to the size of 30 bones. After the transformation matrix of these bones is calculated in CPU, it can be encapsulated as skin info and transferred to GPU.

In the calculation of GPU, the vertices on these mesh will be taken out for corresponding position calculation, and the latest position will be obtained based on the bone conversion matrix and bone weight, thus a vertex calculation and description will be carried out. One of the reasons why we deal with the two parts of bone animation separately is that the processing power of CPU is not as fast as that of GPU. Generally speaking, the number of bones in a model is small, but the number of vertices on mesh is large. Taking advantage of the parallel processing ability of GPU, we can share the computing pressure of CPU.

After DX11 or DX12 (I don't remember clearly), the calculation result of the bone transformation matrix is no longer stored in the register, but in a buffer. The buffer size is set based on the number of bones at the first calculation, and then each time the bone animation data drives to get a new transformation matrix, the corresponding buffer transformation matrix is changed in turn. In this way, the number of roots of the bone is no longer limited by the size of the register. However, in actual optimization, the number of bones in the model will be optimized as much as possible. after all, the more the number, especially the more bones that affect vertices, the greater the amount of computation. The normal mind is to optimize the number of bones rather than expand the size of the buffer: d

In article 2, the performance of the skinning calculation of GPU is optimized greatly, and the main thinking is the same: the bone transformation is carried out in CPU, and the result of the transformation is transferred to GPU, thus the skinning calculation is carried out. The basic thinking is consistent with the transformation thinking mentioned earlier, and the basic optimization focus is to use a buffer to cache the transformation matrix to optimize performance. Here I'll focus on the code in the shader part, where the code handling in the cpu part is basically the same as the previous code idea:

If you use CPU's computational bone transformation, then GPU's shader:

Uniform float4x4 _ Matrices [24]; / / the maximum number of bones set is 24struct appdata {float4 vertex:POSITION; float2 uv:TEXCOORD0; / /, which stores the transformation matrix of the bone, with x _ big y as the index and weight of the first bone, and z _ other as the second index and weight float4 tangent:TANGENT;}. V2f vert (appdata v) {v2fo; / / skin calculation position, notice that it is actually the expression of matrix change and weight float4 pos = mul (_ Matrices [v.tangent.x], v.vertex) * v.tangent.y + mul (_ Matrices [v.tangent.z], v.vertex) * v.tangent.w / / General mvp calculation o.vertex = mul (UNITY_MATRIX_MVP, pos) O.uv = TRANSFORM_TEX (v.uv, _ MainTex); return o;} / / how to calculate index and weight, where a skin vertex is affected by two bones Vector4 [] tangents = new Vector4 [mesh.vertexCount]; for (int ibones; I < mesh.vertexCount;++i) {BoneWeight boneWeight = mesh.boneWeights [I]; tangents [I] .x = boneWeight.boneIndex0; tangents [I] .y = boneWeight.weight0; tangents [I] .z = boneWeight.boneIndex1 Tangents [I] .w = boneWeight.weight1;} newMesh.tangents = tangents

The optimization strategy is to store the transformation matrix in a mapping way, see the code:

Inline float4 indexToUV (int index) {int row = (int) (index / _ MatricesTexSize.x); int col = (index-row * _ MatricesTexsize.x; return float4 (col/_MatricesTexSize.x, row/_MatricesTexSize.y, 0,0);} / / calculate the current transformation matrix inline float4x4 getMatrix (int frameStartIndex, float boneIndex) {int matStartIndex = frameStartIndex + boneIndex*3; float4 row0 = tex2Dlod (_ MatricesTex, indexToUV (matStartIndx)) Float4 row1 = tex2Dlod (_ MatricesTex, indexToUV (matStartIndx + 1)); float4 row2 = tex2Dlod (_ MatricesTex, indexToUV (matStartIndx + 2)); float4 row3 = float4 (0meme 0be0); float4x4 mat = float4x4 (row0, row1, row2, row3); return mat;} v2f vert (appdata v) {v2f o; float time = _ Time.y / / calculate the corresponding index int framIndex = (int) ((_ Time.y + v.uv2.x) * _ AnimFPS)% (_ AnimLength * _ AnimFPS); int frameStartIndex = frameIndex * _ MatricesTexFrameTexls; / / remove the corresponding transformation matrix float4 mat0 = getMatrix (frameStartIndex, v.tangent.x); float4 mat1 = getMatrix (frameStartIndex, v.tangent.z) Float4 pos = mul (mat0, v.vertex) * v.tangentwony + mul (mat1, v.vertex) * v.tangent.w; o.vertex = mul (UNITY_MATRIX_MVP, pos); o.uv = TRANSFOR_TEX (v.uv, _ MainTex); return o

The above is the study of how to analyze the principle of Unity3D skeletal animation. The editor believes that there are some knowledge points that we may see or use in our daily work. I hope you can learn more from this article. For more details, please follow the industry information channel.

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.