Jump to content

TorqueScript 2


MangoFusion

Recommended Posts

Hey guys,


Anyone hanging around IRC may know I've been working on a rewrite of the TorqueScript VM (codenamed TorqueScript 2) over the past few months, based on the T2D codebase. The rewrite involves changing the bytecode & the interpreter runtime in order to improve runtime performance, as well as paving the way for more useful features.


Why rewrite the VM? The main reason really is its fairly slow compared to modern interpreters (e.g. lua). Another reason is that the way TS manipulates variables makes supporting constructs such as arrays, lua tables, and closures a complete pain.


Some of you may be wondering "why is MangoFusion bothering rewriting TS when we should be using [popular language X]?". The main reason in my case is I have a few projects which rely on TS in the pipeline and it would be more trouble than its worth sticking my feet into someone elses mysterious mess. Also I think embedding another language tends to create too many compromises I'm not too happy with, and they often fall short of supporting everything useful for game development (e.g. serializable coroutines).


At the end of the day, with the exception of a few neat runtime features, TS2 will mostly just be a drop-in replacement for the existing TS vm. I don't have any plans to add in anything too complicated like a JIT or a hand-optimized interpreter since both things would incur a bad maintenance cost as well as make everything harder to compile in.


Also I should stress TS2 is merely the codename, so I will likely designate a new name for it before pushing it out there.


--


Now enough about what TS2 will be. What is it now? Well it can currently parse, compile, and execute bytecode for much of the core torquescript language. The only major language feature I haven't implemented yet is array support, but that should be rather trivial.


I'm currently working on fixing up the console interop so calling methods on objects from c++-land works correctly, which will also allow stuff to boot correctly (up until now I've just been testing it with main.cs basically).


Future plans are to decouple the script code from the rest of the T2D codebase and spin it off into an embeddable project. Because quite frankly, i'm probably nuts.


In any case I will be pushing my TS2-related changes to a branch this weekend, in the hope someone might find it useful.

Link to comment
Share on other sites

Nice. Another reason to stick with TorqueScript is that there is a ton of example script that is already present and functioning - it doesn't have to be ported and tested in MyCoolScript or whatever. And it makes it easy, therefore, to test the new interpreter - if it breaks it's not compatible....

Link to comment
Share on other sites

@chriscalef


it's actually based on t3d's scripting implementation (e.g. it has foreach), so it should be reasonably easy to port back across. I'd caution about using it in an actual project until its actually been proven stable of course. ;)


@rlranft


There's a few differences with how arrays work (which had to be changed to allow for true array support). Besides that everything else is the same.

Link to comment
Share on other sites

What did you do to arrays? Make them more like Lua tables? In normal use, how many cases do you figure would break? I mean, right now there are some unorthodox ways to access array members but they're not in general use - it wouldn't hurt if those went away... lol.

Link to comment
Share on other sites

@Mango: sounds good, I can wait but I'm excited that you're working on it!


While we're on the subject of Torque Script though, you wouldn't happen to have done anything re: the floating point precision, would you? I just lost half a day on a bug in my code that turned out to be because of the six digit limit on float values passed from the engine to the console, and while I understand it's for efficiency, I sure would appreciate that value being set somewhere under the control of the user so the tradeoff of efficiency versus precision could be more transparent.


My particular case involved use of lat/long coordinates, where -123.076 is simply nowhere near enough precision. I resolved it for my own case by moving that part of the logic into the engine, where it probably should have been anyway for performance, but all the same, that's a pretty painful data truncation to have just built in to the engine with no easy way to modify it. IMHO.


Thanks again for your efforts!

Link to comment
Share on other sites

Yo guys, just an update.


While I was going to push stuff out last weekend, I bumped into a little problem with the console interop which put a spanner in the works (i.e. i have to refactor stuff earlier than I thought). My current initial release goal is sometime tomorrow.


@rlranft


Arrays are just types with accessor implemented for []. ATM I just have a simple array of console values type. You could implement a lua table fairly easily.


@chriscalef


All numbers are F64s, and they aren't converted to strings unless neccesary (e.g. for concat operators and string accessors). So there should be less of a problem.

Link to comment
Share on other sites

Just another update...


I'm taking the time to address another long-standing the torquescript interop has... namely with strings. Usually Con::getData, Con::getVariable and such return "const char*" pointers, sometimes to temporary strings and sometimes to dynamic strings. The problem however is there are no guarantees placed on if this pointer remains valid, especially if you call another console-related function. This leads to all sorts of weird hacks in various places in the engine in order to account for parameter values being clobbered.


Basically instead of this...

 

const char* myStringValue = Con::getVariable("$whatever");
// Do something with myStringValue, assuming it wont ever change (it *might*)

 

You do something like this...

 

ConsoleStringValuePtr value = Con::getVariable("$whatever");
const char* myStringValue = result.c_str();
// myStringValue remains constant as long as value still exists on stack

 

In some ways this is similar to the String type in T3D (or your classic std::string), though I am aiming to wrap in support for static unallocated strings, as well as a sensible allocator for small strings into this too.

Link to comment
Share on other sites

Ok guys, time for an initial push!


There are still a few things to implement and change (e.g. no dso support, exec doesn't work properly), but at this stage I'm reasonably happy to release something in an initial alpha state.


The code as it stands is now located in a branch at https://github.com/jamesu/Torque2D/tree/twistscript


While currently it wont actually load T2D due to a problem loading the modules, it will execute basic scripts (such as test.cs). Also the code is currently only fixed for OSX, so don't expect it to build for windows yet.

Link to comment
Share on other sites

Just an update on this...


After fixing exec and a few opcodes, it can now execute a fair bit into the T2D initialization process (albeit slightly modified to account for the changed array usage). There are however a few issues still preventing it from fully loading, mainly to do with object slots.


I'm currently aiming to get it working so far as to booting everything up before the end of the month - then other stuff such as DSO support can be reimplemented.

Link to comment
Share on other sites

Time for another update!


The "twistscript" branch can now run T2D and most of its demos with no problems (with minor modifications to the script to fix bugs and issues). The few major problems with remain are mainly related to input, as well as eval not currently working.


Code-wise I've now migrated most stuff - with the exception of user fields - over to use the ConsoleValue* system, so for the first time in forever its now possible to execute script with no string processing whatsoever - well, provided you implement the correct handlers for types.


As mentioned I'm planning on incorporating other stuff into this once I have all the basic stuff working properly so stay tuned!

Link to comment
Share on other sites

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.

Guest
Reply to this topic...

×   Pasted as rich text.   Paste as plain text instead

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.

×
×
  • Create New...