So, just to give the cliffs notes to the conversation today, we pow-wowed on a game plan for terrain to get a better idea of where it needs to be, and how to get it there.
The unanimous consensus was that the detail textures need to be a full color diffuse texture which looks as-the-image-intends up close, where the detail textures are utilized.
So from there, we worked out the following concept for how terrain rendering of materials could proceed:
BaseTexture: Basically what 'DiffuseTexture' does now, but trades the confusing nomenclature. This texture basically paints onto the terrain-wide diffuse texture map that acts as your long-distance detailing(how mountains look in the distance, yada yada)
Detail Diffuse Texture: Full color, up-close texture. Behaves similarly to the detail textures now, where they will blend in based on distance, but doesn't do the weird light/darkening gray image business. You pass in a full color texture and it'll render it as expected.
Detail Diffuse Texture, Alpha Channel: While most cases of the detail texture brightening/darkening are annoying because of how the math is kinda weird, being able to take contribution from the base texture's color to help create visual variation and cut back on repetition is still useful, so we had the idea that the alpha channel could basically act as a 'Blend Mask'. Would basically be a lerp of the blend values of your diffuse textures. At 0(black), it would use the color of the BaseTexture, at 0.5, it would be a 50/50 blend of color, and at 1 it would fully use the Detail Diffuse's color. This would allow you to, say, have a grass texture with flowers, where the grass can pick up some color variation from the BaseTexture, but the flowers would keep their detail-texture-based color and be unmodified. Would also have a override in the mat editor to just assign a flat constant to the channel for simplicity if no special blend masks are required.
Detail Normal Texture: Normal map texture. Natch
Detail Normal Texture, Alpha Channel: Your height information. This and the normal will get used in Lukas' much nicer blending logic as detailed in his blog thread, and also can be utilized in parallax mapping for added depth fidelity.
Detail Composite Texture: This is for the PBR side of things. It'll contain the roughness, AO and metalness info for the detail texture, so that it renders the same as any non-terrain materials.
I also figured that with this setup, it would be quite easy to have the frontend accept picking regular materials, to which we would-in the backend out of view- pull the appropriate textures and fill in the data(allowing a clean, uniform hook to secondary material information, like particle effects to deploy, footstep sounds, etc).
So the front end would see a uniform interface as an option for picking materials, but the backend keeps a separate, streamlined terrain material to keep things efficient and targeted. This has the bonus of letting us get the texture packing to get the draw calls down as long as physically possible(potentially in 1!).
So yeah, I think this is actually a pretty sweet setup, that keeps the positives of the current terrain system, while shifting to a more intuitive and standardized setup with the pros of other, modern terrain systems.
If you guys have any ideas of suggestions to build on this stuff, by all means, throw it in the hat so we can get everything dialed in, but I feel this is a really good starting point.
I'm also gunna put together several new, PBR-ready terrain materials a) for better testing and b) a better example than the poor, old textures we currently have. They've served us well, but they're oooooold at this point, and need some replacing.