First, let's think a bit about how Unity's terrain system renders the textures you paint on it:
You have a control texture that gets projected over your terrain, like a typical terrain heightmap. The four channels (RGBA) of that texture each correspond to a separate entry in the terrain paint palette. In the control texture, Red is the first texture you define in the terrain tool, Green is the second, Blue is the third, and Alpha is the fourth. When you paint some dirt, you're actually painting red or green or blue or alpha onto this control texture.
(So you can imagine -- the control texture resolution has a huge effect on how granular your different terrain texture changes can be.)
Then the shader takes over. It applies your dirt texture based on how much red it sees at a particular coordinate. Then, it overlays your second texture (some sand?) based on how much green is in the control texture, etc. and so on. For convenience, I use the fourth texture slot for whatever I want to clump -- in this case, the fourth texture is my grass texture.
In this grass texture file, I make a sort of "heightmap" and put it in the grass texture's alpha channel. 0 (black) = no grass, 1.0 (white) = yes grass. This alpha channel is the mask that we will use to modulate the blend between grass and dirt. (A better implementation, as done in Source Engine with $blendmodulatetexture, would be to store multiple blend masks in separate channels of one texture. But I didn't need that so I didn't do it.)
In the shader code below, Line 38 is where all the magic happens. I tell the terrain shader to look at how much "alpha" is in the control texture. When there's a little, it won't overlay anything. But when there's a lot of alpha, it will modulate itself with the strength of the mask in the grass texture's alpha channel, and then delete some of the first three layers -- that's so we can get really clean but smoothed transitions when we replace those deleted pixels with fresh grass clumps.
(NOTE: this shader, when it's named "Nature/Terrain/Diffuse," will override the default terrain shader.)
One really important note: I could've used an "if( )" statement to detect whether the alpha value of the control texture was above a certain threshold or not -- and if so, then override that pixel with the grass clump no matter what. That would've been really simple and fairly tunable.
However, you usually want to avoid using "if( )" statements in your shader code because graphics cards are massively parallel computers that rely on executing the same code constantly over and over -- if you throw branching in there, your GPU cores start freaking out. In some cases (SM 2.0) your graphics card will just execute both branches anyway, and then just throw away the data from the other branch. (I suspect, though, that the quantity of my shader instructions might be out-weighing the benefit of not-branching, at this point...)
In general, I could probably improve upon this implementation a lot, but I've already tuned the numbers to suit my particular needs. As far as I'm concerned, this effect is done and I'm going to move on with the rest of the game. I'd suggest a similar attitude for your own projects -- this is just a small piece of your game, don't obsess too much over one piece of one shader. Paste my code into your project, tweak the numbers, and move on.
