[Duion]

I think regular players will not notice the shadow update every x seconds, it took me 200 hours playtime in Skyrim to notice it and I see a lot more than regular players.
But the flickering in Torque is barely visible, if you make the shadows very smooth, only some diagonal shapes can get very bad.

[Steve_Yorkshire]

Looking good, Nils!

[JeffR]

Sorta reminds me of a bit brighter, less depressing STALKER

The STALKER series are a piece of art and a big inspiration, so that's a compliment

I just started replaying CoP, amusingly enough. Hearing that makes me excited

Also, nice to see it in motion with the video. Pretty much the only complaint I could level at it is the lod/foliage pop-in, but that's not exactly an uncommon issue, even in AAA-scope games.

Beyond that it all looks rather solid. Keep up the excellent work!

Not entirely @ Duion, but there were some changes that reduced the flickering effect.
The resource on this page should already be an improvement: http://pastebin.com/LfqdCC0B

!!!

I decided to give this a whirl real fast:

Stock:


With the PCF filter:


That absolutely CRUSHES the ugly noise from the current method. You said Tom authored that? I need to ask about rolling that into stock.

I will note that I did tweak the shadowmap settings from what's stock in the default levels. They're, to be blunt, kinda insane. The shadows have 4 splits, reaching out to 400meters, meaning each 1024 map has to cover 100 meters of distance. It's no surprise that we see aggressive aliasing and peterpanning with that setup. I think I'll make a PR to dial back the shadows to a more 'normal' configuration like what I've got in my above test there that's closer to the distances we usually see shadows in play most games. That does a lot to deal with the aliasing and peter-panning, as said.

I'll have to take Andrewmac's vsm attempt and toss it in there too and see what happens. That worked pretty nicely in a broad sense, but had some blocky artifacting. I think getting PCF on there would smooth all that out and improve the shadows as a whole.

Anywho, rambling aside, thanks a bunch for bringing that bit to my attention

[Nils]

Pretty much the only complaint I could level at it is the lod/foliage pop-in, but that's not exactly an uncommon issue, even in AAA-scope games.

Without multithreading this is as far as it goes running this scene at 30fps. The creation of groundcover cells for example, requires a lot of the CPU, and DM is pretty much CPU bound in it's current state. When everything (except for PhysX and such) is sharing the same thread, generating the cells takes to much time, so you'll see these blocks of foliage pop-in way too late; which is a way too short distance from the camera. Otherwise the groundcover cell would gradually build-up; which it does on parts of the map where there's not so much going on. For statics and forest it's more or less the same; settings will need to be kept this low when playing (and capturing) for now. This results in fast LOD switching and a lot of popping going on; but surely unacceptable when it would be a final release in 2016.

I just started replaying CoP

Awesome

You said Tom authored that?

@ JeffR, yes correct!

[andrewmac]

I'll have to take Andrewmac's vsm attempt and toss it in there too and see what happens. That worked pretty nicely in a broad sense, but had some blocky artifacting. I think getting PCF on there would smooth all that out and improve the shadows as a whole.

PCF and variance methods (VSM, ESM, EVSM) are not really compatible. PCF uses depth and VSMs store depth moments which you then use Chebychev's Inequality on to determine the probability of it being in shadow or not. Because of the "shadowed or not shadowed" nature of it you get hard falloff, or blocky artifacts as you put it. VSMs, however, have the advantage of being able to be prefiltered with a blur pass. You can't do with a regular depth shadowmap because when you blur a regular shadowmap you blend depth values into areas where there wasn't depth values and thus create non-existant surfaces. So yeah, if you want VSMs to be usable you just need to spend a day on improving the blur pass. Which should be very simple, its just a blur pass.

Regarding the linked code, its PCF 3x3, which is pretty good but can be better without being more expensive (which is a rare occurrence in graphics programming). If you spend a little time getting shadow samplers working in T3D you can do optimized PCF 5x5 which gives the same quality as 5x5 but with as many samples as 3x3. It's what Torque 6 and Unity are currently using. Here's an article on it if you're interested: http://www.ludicon.com/castano/blog/art ... ry-part-1/ It's the best quality to performance ratio I found of any of the shadowmap techniques.

[JeffR]

I'll have to take Andrewmac's vsm attempt and toss it in there too and see what happens. That worked pretty nicely in a broad sense, but had some blocky artifacting. I think getting PCF on there would smooth all that out and improve the shadows as a whole.

PCF and variance methods (VSM, ESM, EVSM) are not really compatible. PCF uses depth and VSMs store depth moments which you then use Chebychev's Inequality on to determine the probability of it being in shadow or not. Because of the "shadowed or not shadowed" nature of it you get hard falloff, or blocky artifacts as you put it. VSMs, however, have the advantage of being able to be prefiltered with a blur pass. You can't do with a regular depth shadowmap because when you blur a regular shadowmap you blend depth values into areas where there wasn't depth values and thus create non-existant surfaces. So yeah, if you want VSMs to be usable you just need to spend a day on improving the blur pass. Which should be very simple, its just a blur pass.

Regarding the linked code, its PCF 3x3, which is pretty good but can be better without being more expensive (which is a rare occurrence in graphics programming). If you spend a little time getting shadow samplers working in T3D you can do optimized PCF 5x5 which gives the same quality as 5x5 but with as many samples as 3x3. It's what Torque 6 and Unity are currently using. Here's an article on it if you're interested: http://www.ludicon.com/castano/blog/art ... ry-part-1/ It's the best quality to performance ratio I found of any of the shadowmap techniques.

Dang, I was afraid they wouldn't be compatible. Ah well.

And yeah, Az and Timmy poked and prodded on some of the sampler stuff and got hardware PCF functional, which is nice. Have to get that rolled in if everything checks out on it.

I can still remember back in the day when PCF just started being a thing and everyone was all 'oh man it looks so nice but ahrgrgaghgrgrhgh my frames!' haha.

@ Nils
Speaking of shadows, what distance do you have for the visible shadows? The stock settings for the stock maps put the distance out to 400 which, honestly, is pretty freaking far. Most AAA games I'd been checking for comparison seem to max the shadow distance out to around 200m. Dialing back the shadow distance yeilds a better utilization of the shadowmaps' resolution, and helps cut back on the jaggies. If you're pushing a really high distance, I'd suggest dialing it back to 150-200 with maybe 50 of fade distance and see how that does to help combat the remaining shadow edge issues.

[Nils]

@ JeffR - Beside distance and softness has logweight a lot of influence on the quality. To keep it nice-looking at a closer range but still have some shadow at a far range (DM has a big map), tweaking the logweight could be important. The shadow distance should be adapted to how immersive you want to go. The sudden popping-in of shadows can be really annoying when moving around the map sprinting or driving...

shadowType = "PSSM";
texSize = "1024";
shadowDistance = "360";
shadowSoftness = "0.8";
numSplits = "4";
logWeight = "0.974";
fadeStartDistance = "0";
lastSplitTerrainOnly = "1";

(logweight value is too sensitive in T3D imo)

Reducing the texSize does improve performance enormously, but surely doesn't help in the battle against the flickering.

[JeffR]

Yeah, you definitely don't want shadow pop-in for sure. That said, I'd been trying out several AAA games, and the shadows don't seem to usually be rendered further out than roughly 200m.

With the fadeStartDistance in there, you can control when the shadows begin to fade out to avoid any visual popping, too.

I'd say try cutting the shadow distance to about 200, and have the fadeStartDistance at about 150(so it gives an entire 50m for it to smoothly fade out, which should make it subtle enough that unless you're specifically looking for it, you won't notice).

Cutting back on the distance should give 2 boons: better performance because you're hitting fewer objects to be rendered in the shadowmap, and tighten up the space that each slice is applied to, which should help with resolution and edge jitter.

[Nils]

Thanks, it sounds very plausible @ JeffR. Of course depending on the title I can imagine setting the shadows to a value like that. There are certainly recent AAA titles with insane draw distances but I wouldn't dare to compare with a T3D powered game

DM has the option to set the shadow "distance" in the options and the default is linked to the overall quality. Increasing the minimum object size to draw shadows is a quick method to gain performance -> $pref::PSSM::smallestVisiblePixelSize = "128";.



It's possible that the "medium" settings may come close to the values you suggest. Though in the screenshot you can already see that 360 meters (current default) isn't really that far way in DM. Further are the splits also drawn closer, which does increase the visibility of the flickering because the pixel size of the shadow maps are also bigger. It will take some time to tweak all the settings to be perfect for each "quality" defaults

Another option is to have a close look at what draws shadows and what not. Often not all objects or materials need to draw shadows at all.

[JeffR]

Yeah, that's definitely another optimization. Another is having a duplicate material on a lower LOD that just has the 'casts shadows' flipped off and the like.

One thing that came up we'll be looking at implementing is 'Shadow Proxies', ie, the option to define a submesh or submeshes as being used specifically to render shadows. So artists could author a model that has the same basic silhouette as the visual mesh, but far lower poly density(as unless you're right up on it, the shadows'll generally only need to be "good enough") which saves a lot of horsepower in the shadow pass.

As for shadow draw distance, I need to do more profiling with more titles, but the 200-ish seems to be pretty standard. That said, there are fewer 'lets make the GPU melt' PC-focused games these days, so there's a lot of compromising with consoles, but that number seems fairly consistent. I need to pull up games like Witcher 3 or some others for a good feel on how far out they draw shadows. Maybe the recent Far Crys would be another good sample-set...

Did a quick google search on MGS5 for an image with PC, and it seems like they selectively drop out lower profile meshes from shadowing(like the rocks) before buildings, but even about where you hit the main block of buildings in the town, it doesn't look like they're rendering shadows anymore:



Hmmmm.....