Jump to content

Torque 3D 3.9 Released


JeffR

Recommended Posts

Hey everyone!


It's my pleasure to be able to announce that version 3.9 is now released! This has been quite a big release, with a whopping 142 merged PR's totaling to 515 commits, which is more than DOUBLE what went into 3.8. A herculean effort on everyone's part for sure!


Here's a list of all the fine people that helped make this release possible:

  • Azaezel
  • rextimmy
  • John3
  • Bradan
  • pacomont
  • dakk
  • blackwc
  • jamesu
  • ghost
  • irei1as
  • JeffProgrammer
  • OTHGMars
  • Lopuska
  • Alex-doc
  • TheDushan
  • RoundedIcon
  • camporter
  • rasteron
  • Bloodknight
  • Ragora
  • ZOD2015

 

And still more of you that helped file issues and test. A huge, huge thanks to all of you!


Several big things to note with this release is the full and proper roll-over to deferred rendering, the initial implementation of the entity/component stuffs, switching over to epoxy for OpenGL handling and DirectX 11 integration.


Yeah, yeah, where can I get it?

As always, you can nab it from the wiki's download page, found here.


So, next is 4.0, right

A capital question, question text! Indeed, the next major release will be 4.0. If you've been chumming around on the forums or in chat, you may have gotten hints that 4.0 is slated to be a pretty big deal, and I'm more than happy to intimate why.


The big aspect of 4.0 is that it's going to be the build where we're OK with breaking backwards compatibility in order to get stuff updated. For the released up until now, we've put a fair bit of effort into avoiding huge breaking changes because, well, that sucks to have to rework your project when that kinda stuff hits. However, it also means that large improvements to the understructure, rendering, art pipeline, etc can't be implemented as those will, naturally, impact and probably break existing setups.

So, we've been holding back on quite a number of big things for 4.0, and now that 3.9 is out, we can charge full-steam ahead to get all this stuff in there are really bring T3D back to where it should be.


So, here's a main features list that we're looking at bringing into Torque for 4.0:

  • Physically-Based Rendering.

    This is, unsurprisingly a pretty big one. Luckily, it's also most of the way done! The only main piece that needs finishing is reflection probes, and the core features are in. From there it would just be a matter of dialing in the supporting art pipeline and tweaking the math to make it look as good as possible.
  • MacOS support.

    This was originally slated for 3.9, if you look at the 3.8 release blog, but a lack of machines to test on, and the age-addled old platform code made it impractical to really tackle initially. The good news is, near the end of 3.9's development, with SDL and epoxy to take a LOT of the workload off, huge strides were made and it's actually almost completed. There's still some issues with rendering on some machines and catching errors and warnings that crop up, but it's looking very practical to have MacOS as an official platform for 4.0! You can see Timmy's screenshot in the 3.9 RC thread showing off T3D running on MacOS with his and JeffH's work now.
  • SDL as the main platform layer.

    Torque3D's supported SDL for quite a while now, and it's naturally the main platform layer for Linux(and MacOS). However, it's been a little bit of a red-headed stepchild to the Win32 platform code. For 4.0, that's changing and SDL will be adopted as the core platform layer with whatever glue needed sitting atop it. This will drastically simplify the platform code and make it easier to maintain. It also offers a good bit of future proofing should any other platforms come into the equation down the line. SDL's stuff is very nearly at parity with the Windows side, it's pretty much just a matter of jumping over the last bits and doing cleanup/bugfixes to make it rock-solid.
  • New Project Manager

    The old project manager was useful, but was hacked together with QT and php, so maintaining it was a little sketchy and fell by the wayside for the more standard CMake gui. However, there's lots of management things CMake can't do, which is why the Project Manager will be making a comeback, it'll just be a graphical frontend sitting upon the more standard cmake project generation. This new PM will let you manage your updates, and hopefully in the future hook into online repositories for easier grabbing and integration of updates as well as fetching content packages without hassle.
  • New Shadergen

    As great a job as the current workhorse does of generating shaders for all the many materials you need to make a game, updating and expanding it has proven to be a pretty annoying thing to deal with due to how it's structured. Excellent peice of tech, but a chore to maintain, just like ye olde German tanks in WW2. As such, we'll be looking into restructuring shadergen to not only make it easier to update and expand upon in the backend-sense, but also easier to build materials/shaders for the end user as well, up to, and probably including a node-based approach for engineering your fancy shaders for your fancy materials.
  • Graphics API refinements

    DirectX 9 has proven to be a monstrous workhorse of the graphical APIs, reliably serving well beyond what anyone though it would. However, while it has proven to be a rock-solid API, the time has come to retire the poor girl and send her off to the glue facto-eeeer, the farm. Yes. As such, DirectX 11 will be taking over as the main Dx rendering API, and will continue to see refinements to bring it up to speed now that Dx9 isn't hampering it or OpenGL. Timmy's been poking at this and has noted improved performance in Dx11 by quite a lot, and some gains in OpenGL as well.
  • Threads. Threads for days

    One major blocker to rendering performance that even dropping Dx9 won't help is that we curretly don't thread or do much to optimize out the render calls themselves. That'll change in 4.0. We've been brainstorming the optimal approach for a few months now and have a pretty solid plan of attack that will see the render calls threaded out, and also proper batching of rendered geometry where appropriate. So fewer, faster drawcalls for the same workload. We'll also be looking at threads for handling resource loading and spawning of objects to cut out all those hitches when the map starts or stuff is created.
  • Hardware Skinning

    Another thing that's been pretty much complete outside a few oddball behaviors we'll be getting in there is Hardware Skinning. When it's behaving, the current implementation already notes a huge improvement in performance for animated meshes, so getting it polished up and in should be a huge, immediate performance boon.
  • Physics API

    T3D's had a physics abstraction layer for a long time now, and it's been pretty useful in letting the end user decide if they wanted PhysX or Bullet. However, Torque's stock physics has still served a useful niche in being network-reliable, or lightweight for basic physics mechanics, and some people find it to be a pretty workable thing. As such, the Torque physics will be converted over into a Physics plugin, so all physics and collisions will go through the Physics API and standardize all the behavior for that across the engine, cleaning up quite a lot and making it easier to maintain.
  • Entities, Components, and Assets

    This is the big one from non-rendering side. 3.9 already has the initial implementation of the entity/component stuff, as well as the Assets/Modules systems, but they're not yet fully utilized. That'll change in 4.0. If you've been checking out my work blog with the recent updates, I've been covering work I've been doing on the improved asset pipeline, as well as continued work with the Entity/Component stuff. For 4.0, this will become the standard, and all existing gameplay classes(Vehicle, Player, Item, Shapebase) will be replaced with GameObjects built of entities and components to do the same work. The idea is to give users a similar base to what is in Torque now in terms of starting objects, but remove all the bulky, hardcoded functionality. It should be a standard point, not an anchor, after all. Tying to that, we also will have:
  • Assimp support

    This was actually mostly done back in the day, just some issues with animation stuffs and a few other minor problems. I'd poked at this with some R&D time a while back, so it shouldn't take much tweaking to get it polished up and hooked into the assets system, which will allow quite a few new 3d model formats, including FBX.
  • A new base template

    This is also basically complete. This will be a much more streamlined starting template intended to easily drop in modules, assets and the like into to build up your game project, as opposed to having to spend time stripping out the stuff you don't need. Numerous ancillary improvements also help load times, easier to read and comprehend code and the like. This will replace the empty and full templates, cutting down on the effort needed to maintain(no need to duplicate changes).
  • Vive support

    This comes from our own Mango's efforts. It's pretty much done, but didn't quite squeak in for 3.9. It will be going into 4.0 for sure.

 

We're also looking at possible C# support, based on Lukas's fine work.

So, the main takeaway from this is that the major goals for 4.0 are:


A cleaned up codebase.

Improved rendering and performance.

Easier to use and expand - Source code should be for rendering, optimization and fancy stuff, it shouldn't be REQUIRED for the prototyping phase.


As always, if you note any issues, please make an issue on the github page, and if you have anything you'd like to contribute or suggestions of things you'd like to see, by all means hit us up on the forums, we'd love to hear feedback!


Here's the release notes for 3.9


Have a good one, and happy developing!

Link to comment
Share on other sites

  • MacOS support.

    This was originally slated for 3.9, if you look at the 3.8 release blog, but a lack of machines to test on, and the age-addled old platform code made it impractical to really tackle initially. The good news is, near the end of 3.9's development, with SDL and epoxy to take a LOT of the workload off, huge strides were made and it's actually almost completed. There's still some issues with rendering on some machines and catching errors and warnings that crop up, but it's looking very practical to have MacOS as an official platform for 4.0! You can see Timmy's screenshot in the 3.9 RC thread showing off T3D running on MacOS with his and JeffH's work now.
  • SDL as the main platform layer.

    Torque3D's supported SDL for quite a while now, and it's naturally the main platform layer for Linux(and MacOS). However, it's been a little bit of a red-headed stepchild to the Win32 platform code. For 4.0, that's changing and SDL will be adopted as the core platform layer with whatever glue needed sitting atop it. This will drastically simplify the platform code and make it easier to maintain. It also offers a good bit of future proofing should any other platforms come into the equation down the line. SDL's stuff is very nearly at parity with the Windows side, it's pretty much just a matter of jumping over the last bits and doing cleanup/bugfixes to make it rock-solid.
  • Graphics API refinements

    DirectX 9 has proven to be a monstrous workhorse of the graphical APIs, reliably serving well beyond what anyone though it would. However, while it has proven to be a rock-solid API, the time has come to retire the poor girl and send her off to the glue facto-eeeer, the farm. Yes. As such, DirectX 11 will be taking over as the main Dx rendering API, and will continue to see refinements to bring it up to speed now that Dx9 isn't hampering it or OpenGL. Timmy's been poking at this and has noted improved performance in Dx11 by quite a lot, and some gains in OpenGL as well.
  • Hardware Skinning

    Another thing that's been pretty much complete outside a few oddball behaviors we'll be getting in there is Hardware Skinning. When it's behaving, the current implementation already notes a huge improvement in performance for animated meshes, so getting it polished up and in should be a huge, immediate performance boon.
  • A new base template

    This is also basically complete. This will be a much more streamlined starting template intended to easily drop in modules, assets and the like into to build up your game project, as opposed to having to spend time stripping out the stuff you don't need. Numerous ancillary improvements also help load times, easier to read and comprehend code and the like. This will replace the empty and full templates, cutting down on the effort needed to maintain(no need to duplicate changes).
  • Vive support

    This comes from our own Mango's efforts. It's pretty much done, but didn't quite squeak in for 3.9. It will be going into 4.0 for sure.

 

It seems to me like these features will take drastically less time to develop than the other listed features. They also don't break backwards compatibility. In my opinion these are useful features that I think justify a 3.10 release rather than holding them back until the rest of the features are finished.

Link to comment
Share on other sites

  • MacOS support.

    This was originally slated for 3.9, if you look at the 3.8 release blog, but a lack of machines to test on, and the age-addled old platform code made it impractical to really tackle initially. The good news is, near the end of 3.9's development, with SDL and epoxy to take a LOT of the workload off, huge strides were made and it's actually almost completed. There's still some issues with rendering on some machines and catching errors and warnings that crop up, but it's looking very practical to have MacOS as an official platform for 4.0! You can see Timmy's screenshot in the 3.9 RC thread showing off T3D running on MacOS with his and JeffH's work now.
  • SDL as the main platform layer.

    Torque3D's supported SDL for quite a while now, and it's naturally the main platform layer for Linux(and MacOS). However, it's been a little bit of a red-headed stepchild to the Win32 platform code. For 4.0, that's changing and SDL will be adopted as the core platform layer with whatever glue needed sitting atop it. This will drastically simplify the platform code and make it easier to maintain. It also offers a good bit of future proofing should any other platforms come into the equation down the line. SDL's stuff is very nearly at parity with the Windows side, it's pretty much just a matter of jumping over the last bits and doing cleanup/bugfixes to make it rock-solid.
  • Graphics API refinements

    DirectX 9 has proven to be a monstrous workhorse of the graphical APIs, reliably serving well beyond what anyone though it would. However, while it has proven to be a rock-solid API, the time has come to retire the poor girl and send her off to the glue facto-eeeer, the farm. Yes. As such, DirectX 11 will be taking over as the main Dx rendering API, and will continue to see refinements to bring it up to speed now that Dx9 isn't hampering it or OpenGL. Timmy's been poking at this and has noted improved performance in Dx11 by quite a lot, and some gains in OpenGL as well.
  • Hardware Skinning

    Another thing that's been pretty much complete outside a few oddball behaviors we'll be getting in there is Hardware Skinning. When it's behaving, the current implementation already notes a huge improvement in performance for animated meshes, so getting it polished up and in should be a huge, immediate performance boon.
  • A new base template

    This is also basically complete. This will be a much more streamlined starting template intended to easily drop in modules, assets and the like into to build up your game project, as opposed to having to spend time stripping out the stuff you don't need. Numerous ancillary improvements also help load times, easier to read and comprehend code and the like. This will replace the empty and full templates, cutting down on the effort needed to maintain(no need to duplicate changes).
  • Vive support

    This comes from our own Mango's efforts. It's pretty much done, but didn't quite squeak in for 3.9. It will be going into 4.0 for sure.

 

It seems to me like these features will take drastically less time to develop than the other listed features. They also don't break backwards compatibility. In my opinion these are useful features that I think justify a 3.10 release rather than holding them back until the rest of the features are finished.

 

Yeah, it's wholely possible that we can do a non-breaking update a la a 3.10. It's something I'll keep an eye towards.


I would say though that the new base template is definitely breaking because it's a complete restructure compared to the current ones. If someone wanted to update their project, it would be a bit of an investment, and not just a drop-in-and-done thing. Not an EGREGIOUS amount of effort, but definitely not an 'update a few scripts and ye good' thing.


The main thing of it is how fast stuff like that can cascade. If we drop out Dx9 and strip out the stuff related to that, there's some script changes there, if we did that before the new template, no biggie, if it happens after the new template, then that's a bit more complex to parallelize. Same with something like PBR. If it gets wrapped up and implemented in the next couple months, that impacts a ton of things, including existing art assets, meaning that doing a 3.10 release is a lot more complicated, etc.


As a separated list, it's no big thing, but depending on the order stuff gets merged in, it can get weird, that's why I figured I'd lay the plan out in a safer 'assume breakage' target. If we can swing a lot of that stuff without that being a concern though, then a 3.10 update is a definite possibility.

Link to comment
Share on other sites

Can we change the versioning scheme? I'd like to offer this:


Major version: Dr. Seuss book titles, in alphabetical order ascending.

Minor version: Genus/species names of rodents, in alphabetical order by genus first then species ascending.

Build/update version: Pastry names from around the world, in alphabetical order ascending.


I feel this would make it almost instinctive to understand for even the bloodiest of raw newcomers. As you can see, it follows the natural order of the Universe and therefore is the most natural way to organize such things.

Link to comment
Share on other sites

Hey this is great news! Thanks so much for all of this effort, to all involved. Open source power at its finest - revealing the true strength of Torque!


I am so ready to convert all my resources and tools over to module form! Is that a realistic goal for this release, or is the module system not complete in this build? I've been twiddling my thumbs for a bit dealing with other company matters while this whole 3.9->4.0 update blows over...but if the module system is ready for my tinkering I could start pumping out some tools/resources!


Speaking of the new modules etc., is there a 'guide' or explanation someplace on how to use all this new shiny stuff? Like templates to follow, tutorials on new module creation and so on?

Link to comment
Share on other sites

We're also looking at possible C# support, based on Lukas's fine work.

Ooh, we are?

Interesting :P Although, I'd like to point out that my work on the C# stuff is split into: "supporting any desired scripting language" and "generate C# libs for scripting Torque".


So it might pave the way for other languages too.

Link to comment
Share on other sites

@LukasPJ: That was kinda the main point behind the mass-convert https://github.com/GarageGames/Torque3D/pull/887. To give a more straightforward set of hooks to play off of, wasn't it?

Yeah it was, but there is still a few changes here and there that needs to be made. And I've also created a parser that outputs an XML file with information about script-exports, which I use for generating a C#-bridge (but could be used to generate other bridges as well).

Link to comment
Share on other sites

  • 2 weeks later...
Hi Guys,


After porting some projects over to v3.9 I notice that there seems to be an issue with real time shadows. They just don't seem to render correctly. Is there a known issue in this regard ?

If your seeing hitches, probably https://github.com/GarageGames/Torque3D/pull/1442 ( http://forums.torque3d.org/viewtopic.php?f=9&t=387 for the status-thread with pics)*


*edit: should note due to a bug with translucent objects, had to shut down the capacity to specify a cycle for the freqency lights update dynamic marked materials at. immobile, unanimated objects can still be left updating their shadows at a low rate.

Link to comment
Share on other sites

  • 3 weeks later...

We're also looking at possible C# support, based on Lukas's fine work.

 

I think this is a terrible idea.


The main strength of TS is that it's a scripting language aimed at making games, and even more, it's aimed at making games using Torque. Shoe-horning another general purpose language just because it's "flavour of the month" is going to be problematic, and a real turn-off. It'll have to be hacked all to hell to fit Torque and rather than elegant ways of working with scripting within the engine. There will be workarounds, hacks, and things that never work quite the way that they need to.


TS isn't broken. There's no need to fix that.

Link to comment
Share on other sites

We're also looking at possible C# support, based on Lukas's fine work.

 

I think this is a terrible idea.


The main strength of TS is that it's a scripting language aimed at making games, and even more, it's aimed at making games using Torque. Shoe-horning another general purpose language just because it's "flavour of the month" is going to be problematic, and a real turn-off. It'll have to be hacked all to hell to fit Torque and rather than elegant ways of working with scripting within the engine. There will be workarounds, hacks, and things that never work quite the way that they need to.


TS isn't broken. There's no need to fix that.

 

C# is faster, more modern and has a lot of libraries available.. that's some of the benefits of c#.. other languages too.. ideally, being able to strip out TS could make the codebase simpler too, but that's a whole other discussion..


Anyways, as I mentioned the work I've been doing was really to enable c# to be plugged in to the engine and not build into the engine. So it's not as invasive as you think.

Link to comment
Share on other sites

×
×
  • Create New...