Page 1 of 1

Torque3D is a major part of my master's thesis

Posted: Thu May 18, 2017 2:01 pm
by LukasPJ
Hi all,

I just wanted to share some information about my master's thesis you might find interesting.

In my master's thesis (in which I am cooperating with a fellow student at Aarhus University), I take a look at machine learning and AI through the eyes of the game-developer with the overall question "is machine learning ready to be applied by a small indie studio?" looking at not only the possibility of using machine learning, but also the feasibility.

Unfortunately, Twitch keeps deleting my videos.. So I have an old crappy YouTube video, which shows the basics of the game here: https://www.youtube.com/watch?v=83mbhlgD49Y
It is a top-down shooter, all players are AI (although ideally we would have human players as well, but alas it was not a priority) and they are limited in vision. Each tick they are allowed to choose an action based on an information vector. We had a hackathon where people submitted AI's for a tournament, after the hackathon we used these AI's for training and evaluation of our own -- machine learning based -- AI.

We have had limited success so far, maxing out at 16% wins against the best hackathon AI, and 30% against the worst, but we see a lot of interesting trends and we have a few more tweaks we are going to make.
For those who are following the field, we are using Dueling-Double-Q-Recurrent-Neural-Networks with experience-replay and a simple weight evolutionary algorithm as well as stochastic gradient descent with an ADAM optimizer. Pretty much following in the foot-steps of DeepMind and other major projects out there.

Our game source (albeit ugly) is available here: https://github.com/lukaspj/Speciality/tree/speciality , it utilizes almost stock T3D, with a C# layer on top of it.
The machine learning AI code is available here: https://github.com/lukaspj/Shooter-Learner , and is a lot prettier by comparison.

I have a draft of the thesis available at: https://1drv.ms/b/s!AtrTgygwv8wijupGwI9hoYWJrfbpfw
It's due June 15th, if you are interested and read the draft, then any comments to the content is much appreciated, but it's still very much in a draft-stage, and the TODO's are in a mix of danish and english.


If you have any questions about machine learning or other related topics, then feel free to ask them. I will be happy to answer, especially since it will help me to prepare for the defense of the thesis.

I hope it was remotely interesting! I will be happy to provide more details to anyone who wants them. I just you might think of it as an interesting story how Torque3D has helped me implement my thesis project.

Re: Torque3D is a major part of my master's thesis

Posted: Thu May 18, 2017 3:40 pm
by JeffR
Unfortunately, Twitch keeps deleting my videos


Classy. Very classy, twitch.

But yeah, I'll definitely be reading through the draft. Machine learning and neural nets are very interesting and deep topics :D