MangoFusion Posted October 29, 2015 Share Posted October 29, 2015 Hey guys,Anyone hanging around IRC may know I've been working on a rewrite of the TorqueScript VM (codenamed TorqueScript 2) over the past few months, based on the T2D codebase. The rewrite involves changing the bytecode & the interpreter runtime in order to improve runtime performance, as well as paving the way for more useful features.Why rewrite the VM? The main reason really is its fairly slow compared to modern interpreters (e.g. lua). Another reason is that the way TS manipulates variables makes supporting constructs such as arrays, lua tables, and closures a complete pain.Some of you may be wondering "why is MangoFusion bothering rewriting TS when we should be using [popular language X]?". The main reason in my case is I have a few projects which rely on TS in the pipeline and it would be more trouble than its worth sticking my feet into someone elses mysterious mess. Also I think embedding another language tends to create too many compromises I'm not too happy with, and they often fall short of supporting everything useful for game development (e.g. serializable coroutines).At the end of the day, with the exception of a few neat runtime features, TS2 will mostly just be a drop-in replacement for the existing TS vm. I don't have any plans to add in anything too complicated like a JIT or a hand-optimized interpreter since both things would incur a bad maintenance cost as well as make everything harder to compile in.Also I should stress TS2 is merely the codename, so I will likely designate a new name for it before pushing it out there.--Now enough about what TS2 will be. What is it now? Well it can currently parse, compile, and execute bytecode for much of the core torquescript language. The only major language feature I haven't implemented yet is array support, but that should be rather trivial.I'm currently working on fixing up the console interop so calling methods on objects from c++-land works correctly, which will also allow stuff to boot correctly (up until now I've just been testing it with main.cs basically).Future plans are to decouple the script code from the rest of the T2D codebase and spin it off into an embeddable project. Because quite frankly, i'm probably nuts.In any case I will be pushing my TS2-related changes to a branch this weekend, in the hope someone might find it useful. Quote Link to comment Share on other sites More sharing options...
andrewmac Posted October 29, 2015 Share Posted October 29, 2015 I'll be merging it into Torque 6 as soon as it's available. It should be an easy transition since Torque 6 isn't script heavy at the moment and hopefully it'll provide good test results for you. Looking forward to it! Quote Link to comment Share on other sites More sharing options...
Nils Posted October 30, 2015 Share Posted October 30, 2015 (edited) This sounds good @MangoFusion! :D :D :D Edited October 30, 2015 by Nils Quote Link to comment Share on other sites More sharing options...
chriscalef Posted October 30, 2015 Share Posted October 30, 2015 Yay, thank you Mango! Just to clarify though, this is in T2D? Is it easy to drop into T3D? Quote Link to comment Share on other sites More sharing options...
Timmy Posted October 30, 2015 Share Posted October 30, 2015 Awesome work mango! Quote Link to comment Share on other sites More sharing options...
rlranft Posted October 30, 2015 Share Posted October 30, 2015 Nice. Another reason to stick with TorqueScript is that there is a ton of example script that is already present and functioning - it doesn't have to be ported and tested in MyCoolScript or whatever. And it makes it easy, therefore, to test the new interpreter - if it breaks it's not compatible.... Quote Link to comment Share on other sites More sharing options...
MangoFusion Posted October 31, 2015 Author Share Posted October 31, 2015 @chriscalefit's actually based on t3d's scripting implementation (e.g. it has foreach), so it should be reasonably easy to port back across. I'd caution about using it in an actual project until its actually been proven stable of course. ;)@rlranftThere's a few differences with how arrays work (which had to be changed to allow for true array support). Besides that everything else is the same. Quote Link to comment Share on other sites More sharing options...
rlranft Posted October 31, 2015 Share Posted October 31, 2015 What did you do to arrays? Make them more like Lua tables? In normal use, how many cases do you figure would break? I mean, right now there are some unorthodox ways to access array members but they're not in general use - it wouldn't hurt if those went away... lol. Quote Link to comment Share on other sites More sharing options...
chriscalef Posted October 31, 2015 Share Posted October 31, 2015 @Mango: sounds good, I can wait but I'm excited that you're working on it! While we're on the subject of Torque Script though, you wouldn't happen to have done anything re: the floating point precision, would you? I just lost half a day on a bug in my code that turned out to be because of the six digit limit on float values passed from the engine to the console, and while I understand it's for efficiency, I sure would appreciate that value being set somewhere under the control of the user so the tradeoff of efficiency versus precision could be more transparent. My particular case involved use of lat/long coordinates, where -123.076 is simply nowhere near enough precision. I resolved it for my own case by moving that part of the logic into the engine, where it probably should have been anyway for performance, but all the same, that's a pretty painful data truncation to have just built in to the engine with no easy way to modify it. IMHO.Thanks again for your efforts! Quote Link to comment Share on other sites More sharing options...
Steve_Yorkshire Posted November 2, 2015 Share Posted November 2, 2015 Also I should stress TS2 is merely the codename, so I will likely designate a new name for it before pushing it out there.KawaiiDesuSempai Script! ;) Quote Link to comment Share on other sites More sharing options...
JeffR Posted November 2, 2015 Share Posted November 2, 2015 KorkScript, duh. Quote Link to comment Share on other sites More sharing options...
MangoFusion Posted November 7, 2015 Author Share Posted November 7, 2015 Yo guys, just an update.While I was going to push stuff out last weekend, I bumped into a little problem with the console interop which put a spanner in the works (i.e. i have to refactor stuff earlier than I thought). My current initial release goal is sometime tomorrow.@rlranftArrays are just types with accessor implemented for []. ATM I just have a simple array of console values type. You could implement a lua table fairly easily.@chriscalefAll numbers are F64s, and they aren't converted to strings unless neccesary (e.g. for concat operators and string accessors). So there should be less of a problem. Quote Link to comment Share on other sites More sharing options...
chriscalef Posted November 7, 2015 Share Posted November 7, 2015 Ah, that's good to hear, @MangoFusion! Keep them as floats the whole time, what a thought! :) Quote Link to comment Share on other sites More sharing options...
MangoFusion Posted November 10, 2015 Author Share Posted November 10, 2015 Just another update...I'm taking the time to address another long-standing the torquescript interop has... namely with strings. Usually Con::getData, Con::getVariable and such return "const char*" pointers, sometimes to temporary strings and sometimes to dynamic strings. The problem however is there are no guarantees placed on if this pointer remains valid, especially if you call another console-related function. This leads to all sorts of weird hacks in various places in the engine in order to account for parameter values being clobbered.Basically instead of this... const char* myStringValue = Con::getVariable("$whatever"); // Do something with myStringValue, assuming it wont ever change (it *might*) You do something like this... ConsoleStringValuePtr value = Con::getVariable("$whatever"); const char* myStringValue = result.c_str(); // myStringValue remains constant as long as value still exists on stack In some ways this is similar to the String type in T3D (or your classic std::string), though I am aiming to wrap in support for static unallocated strings, as well as a sensible allocator for small strings into this too. Quote Link to comment Share on other sites More sharing options...
MangoFusion Posted November 16, 2015 Author Share Posted November 16, 2015 Another update...I've finished refactoring everything to use the new interfaces, just dealing with a few regressions of behaviour.Hopefully should push something initial to one of my branches some time this week. 8-) Quote Link to comment Share on other sites More sharing options...
JeffR Posted November 16, 2015 Share Posted November 16, 2015 <3 Quote Link to comment Share on other sites More sharing options...
Johxz Posted November 16, 2015 Share Posted November 16, 2015 Nice ;) Quote Link to comment Share on other sites More sharing options...
MangoFusion Posted November 20, 2015 Author Share Posted November 20, 2015 Ok guys, time for an initial push!There are still a few things to implement and change (e.g. no dso support, exec doesn't work properly), but at this stage I'm reasonably happy to release something in an initial alpha state.The code as it stands is now located in a branch at https://github.com/jamesu/Torque2D/tree/twistscriptWhile currently it wont actually load T2D due to a problem loading the modules, it will execute basic scripts (such as test.cs). Also the code is currently only fixed for OSX, so don't expect it to build for windows yet. Quote Link to comment Share on other sites More sharing options...
MangoFusion Posted November 24, 2015 Author Share Posted November 24, 2015 Just an update on this...After fixing exec and a few opcodes, it can now execute a fair bit into the T2D initialization process (albeit slightly modified to account for the changed array usage). There are however a few issues still preventing it from fully loading, mainly to do with object slots.I'm currently aiming to get it working so far as to booting everything up before the end of the month - then other stuff such as DSO support can be reimplemented. Quote Link to comment Share on other sites More sharing options...
JeffR Posted November 24, 2015 Share Posted November 24, 2015 Sounding pretty good :D Quote Link to comment Share on other sites More sharing options...
MangoFusion Posted November 29, 2015 Author Share Posted November 29, 2015 Time for another update!The "twistscript" branch can now run T2D and most of its demos with no problems (with minor modifications to the script to fix bugs and issues). The few major problems with remain are mainly related to input, as well as eval not currently working.Code-wise I've now migrated most stuff - with the exception of user fields - over to use the ConsoleValue* system, so for the first time in forever its now possible to execute script with no string processing whatsoever - well, provided you implement the correct handlers for types.As mentioned I'm planning on incorporating other stuff into this once I have all the basic stuff working properly so stay tuned! Quote Link to comment Share on other sites More sharing options...
Recommended Posts
Join the conversation
You can post now and register later. If you have an account, sign in now to post with your account.