How to create a smooth vertex color transition

[Razorwing]

This tutorial shows how to combine two meshes in a way that allows for a smooth, vertex shaded transition between them. Transitions like these give more definition to your models and help them look more real. I've attached a max scene and example nif.

The tutorial model would have looked like this without a shade effect. With the shade effect it looks like this:

[title]Step one.[/title]
[1st]T[/1st]his step is identical to the first step of the texture shade tutorial, i.e. all I did was make a sphere and a plane primitives and then cut off the bottom half of the sphere.

[title]Step two.[/title]
[1st]I[/1st]n editable poly mode I selected the border of the sphere and capped it. The Cap button is on the Edit Borders rollout of the Edit Poly modifier when you've selected borders on the Edit Poly modifier. I then detached the polygon to a new mesh, and on that mesh I flipped the normals so that the polygon is facing upward. Finally I sized it to 150% of it's original size.

[title]Step three.[/title]
[1st]T[/1st]he polygon we created is going to become what I personally call a cavitymaker. We're going to use it to cut a hole in the ground plane. To do that, we move the mesh to a Z position of 0,2 so that it winds up above the ground plane. Add an edit poly modifier, select the border, and then hold down the ctrl key on your keyboard while you left-click-and-drag downward on the Z plane. Instant extrusion. Now cap the border, and we have turned the previously 2d surface into a 3d object looking not unlike a compact disc.

[title]Step four.[/title]
[1st]S[/1st]elect the disc and then go to the "Create" panel, select "Compound Objects" in the drop-down list, and click the "Boolean" button. In the rollout, select Subtraction (B-A) as the Operation. Then depress the button called "Pick Operand B", and then click on the ground plane. Now we have a ground plane with a huge gaping hole in it. Note - When you've done this the ground plane will take on the name and color of the extruded poly mesh. In the tutorial scene I reset that to the ground plane's name and color, just to make it look clearer.

[title]Step five.[/title]
[1st]A[/1st]ttach the two remaining meshes to one another and turn the combined mesh into an editable poly. Then select the outer border of the sphere, and then by ctrl+leftclick also select the inner border of the ground plane. On the Edit Borders rollout, click the button Bridge, which creates polygons between borders.
[indent][/indent]In a visual sense you might think you're back to where you started - but not quite. At the start we had a solid plane on which a sphere with no bottom was sitting, which means Oblivion's lighting system gets confused. We can pretend the lighting system is thinking, "Hmm... should the surface go straight forward like the plane, or should it be a sphere?" By combining the meshes in a way that makes the edges line up Oblivion's lighting system can make sense of the model - it should be shaded flatly up until the sphere shape. But more importantly we have created a buffer zone in which vertex color can be displayed like we want it to.

[title]Step six.[/title]
[1st]N[/1st]ow I added a VertexColor modifier and painted the vertices around the sphere's baseline with black. In this case I used 100% opacity, but usually I prefer 75% when I want it to look dark. One interesting thing I can mention about vertex painting is - take a moment to look at your scene. Notice something different compared to what it would look like ingame? You are most likely working with very bright light turned on so you can see the model clearly. It will hardly ever be as bright ingame as you need to have it in 3D Studio Max to see what you're doing. What this means is that you pretty much have to shade your model darker than you might think at first, because otherwise the shading might look undetectable ingame when the ambient light might flood everything with a dark grey.

[title]Step seven.[/title]
[1st]Y[/1st]our first instinct would probably be to add a border texture on the polys we created with the Bridge function, but to get the right effect in my opinion you want the whole plane to have the same texture, otherwise the border texture will obfuscate the shade effect. In step seven I added textures and simple mapping to the model just to highlight the visual effect of the vertex shading. I can also add to the subject that if this had been a real model I was going to use I would have raised the baseline of the sphere on the Z plane slightly to get a subtle softness to it that emanates as far as the vertexpainting does.
[indent][/indent]Don't forget to add NiVertexColorProperty to the meshes in Nifskope, vertex mode VERT_MODE_SRC_AMB_DIF and lighting mode LIGHT_MODE_EMI_AMB_DIF. It makes the models more stable and may or may not make the vertex painting more noticeable. I think it does the latter but I never really checked since I know I'm supposed to add the property regardless.