How many decal roads have YOU put into a map?

  • 1
  • 2
16 posts Page 1 of 2
chriscalef
Posts: 332
Joined: Mon Feb 09, 2015 7:48 pm
by chriscalef » Sun May 31, 2015 11:29 pm
Well this might seem a little random, but I just wrote something rather cool, I do believe, and thought I'd throw it out here for grins and giggles.

I'm working on my monolithic OpenSimEarth open world sandbox game project, and one thing that occurred to me recently is that I would really like to be able to import OpenStreetMap vector data.

The way to do this that I already had available to me was to incorporate OSM's vector data into my FlightGear terrain building process, and then get it into my world via my WorldServer terrain importer. This brings the roads in as textures on the terrain, but it has many flaws. First, it takes forever to process all that complexity using the flightgear (terragear) tools, and second, the end result is far from satisfactory, being very pixelated and unrealistic in appearance.

However, recently I had the thought: what if I imported the OSM data directly into Torque as an XML file, and then created decalRoads or meshRoads everywhere I should have a road? This would give me much better looking roads, with much more flexibility and better performance.

In order to do this efficiently, I made it into a two step process: First, on the engine side, I wrote a function called readOSM(const char *filename) which reads an XML file exported by OSM, and stores the required data in an sqlite database. Then, on the script side I wrote a function called makeStreets() which accesses this database and creates an appropriate decalRoad out of all the nodes, after converting all of the latitude/longitude coords into XYZ values for my local T3D mission of course.

There are still many problems, this being the first pass and all (primary among them large road segments standing straight up vertically, any place the road crosses a seam between two terrain tiles). However, I was pleasantly surprised by the scale I was able to achieve in just a few hours of messing around. At first I had a huge problem with slow sqlite connection speed, but then I realized I had to use "BEGIN:... COMMIT;" to make all the SQL happen at once, and after that things became most acceptable in terms of performance.

These pictures demonstrate an import of 10M of OSM data including approximately 4500 Ways, 26000 Nodes, and 49000 WayNodes. Not all of these Ways are turned into DecalRoads because some of them represent buildings or other map decorations, but all roads and paths have been drawn with one of several different available widths and one of the three different possible road textures provided in a stock T3D build.

Forward progress will of course include new road and path textures, as well as of course rivers and streams, and perhaps meshRoads in steep terrain. All in all, I'm not too unhappy with my first attempt, however.

Image

Image
Looking west from somewhere around the U of O.

Image
Looking south from above Skinner Butte.
Nils
Posts: 160
Joined: Thu Feb 05, 2015 3:32 am
 
by Nils » Mon Jun 01, 2015 4:23 pm
That's awesome @
User avatar
chriscalef


You have unleashed the power of T3D! UE4 doesn't even have decal roads ;-)

I whished we had that for the 8km2 map, while laying decal roads on top of a satellite image.
This method would have spared us so much time :shock:
JeffR
Steering Committee
Steering Committee
Posts: 750
Joined: Tue Feb 03, 2015 9:49 pm
 
by JeffR » Mon Jun 01, 2015 5:40 pm
That's a pretty interesting approach!

I think that, doing something on that scale, having a new class to do roads via that data would be better. You could batch roads together to reduce drawcalls by a sizeable amount that way.

So a 'OSMRoad' class that takes in the data reference, then then manages the entire road network in as few render calls as possible, as it manages everything. That'd be pretty neat.
chriscalef
Posts: 332
Joined: Mon Feb 09, 2015 7:48 pm
by chriscalef » Mon Jun 01, 2015 7:19 pm
Hm, that would be interesting. As is, they actually do render pretty efficiently, but I'm definitely at the outside limit at this point for my hardware. I think I have somewhere around 3000 active decalRoads in this example, which does run but I'm down to about 4-5 fps.

For my next trick, though, I'm going to start searching for nodes by distance from player, and then instantiate the roads only as they come into range. This won't work very well for flying vehicles, but should be fine for first person work.

Then the really fun part - it is my intent to use the Mission Editor to let people decorate the world, and then on save, put the newly placed objects into my sqlite DB instead of the .mis file. That way I can also search out and instantiate mission objects by player location instead of loading them all at once at the start of the mission.
chriscalef
Posts: 332
Joined: Mon Feb 09, 2015 7:48 pm
by chriscalef » Mon Jun 01, 2015 7:42 pm
So @
User avatar
JeffR
, is there a quick guide or tutorial anywhere that would lay out exactly how you combine a lot of objects into one draw call? Often wondered about that but it's definitely not my forte.
JeffR
Steering Committee
Steering Committee
Posts: 750
Joined: Tue Feb 03, 2015 9:49 pm
 
by JeffR » Thu Jun 04, 2015 3:46 pm
A quick guide? Can't say there is off the top of my head.

The basic premise would be to have a single object with a struct to hold a road's nodes, textures, etc. And then just have lists of the roads it contains.

Then you'd find all roads that share a material and batch them together into as few buffers as possible to massively trim down your draw calls.

I have an example of how to do that with an update/modification to the convex shape object I was working on. I thought I uploaded it to git, but it looks like I didn't do the right version. I'll get that uploaded and link you it so you can see how I batched by material.

I dunno if that'd be the optimal way to do it for this, but it may serve as a good starting point.
Duion
Posts: 813
Joined: Sun Feb 08, 2015 1:51 am
 
by Duion » Thu Jun 04, 2015 5:32 pm
Looks like a quite insane project, imagine you have 4.5 fps with only the roads, what will happen if you add buildings?
I would try to cut the satellite image into small chunks and then model the roads over it in a modeling app, so you have chunks of roads you can then place into Torque3D.
chriscalef
Posts: 332
Joined: Mon Feb 09, 2015 7:48 pm
by chriscalef » Thu Jun 04, 2015 7:02 pm
@
User avatar
JeffR
Thanks for the suggestion, and yeah if you could find that example it would help a lot, for all this time using Torque I've somehow managed to stay mostly away from the actual buffers/draw calls end of the action, could use a good sample. I have a feeling I'm going to be needing a lot more of that soon.

@
User avatar
Duion
Yeah this was sort of a testing benchmark, I would hopefully never stack it with this many roads in a real scenario (unless the hardware allows and user prefers...)

Now that I have all the nodes in my database, though, I can start devising different strategies to sort and exclude roads & features based on distance/visibility/direction from the camera. Even for flying vehicles, unless you're at very high elevation you really can't see distant streets, because of all the buildings and trees.

Which would be the next step to add here. With the forests, either I'm doing something wrong or the trees don't care about roads, but I may need to make them care about that. And then for buildings, there is a certain amount of vector data available from OSM describing particular real world buildings, but at least for Eugene this data is pretty slim relative to the total area. There are some places in the world where the FlightGear community has gotten some impressive results using this strategy, however.

Overall my thought for OpenSimEarth has been to obtain a basic set of houses/retail/industrial buildings to randomly distribute in the blank areas, based on zoning data, and then borrow as much from FG as possible in terms of the osm2city logic.

Then of course the key to it all would be hooking up the world editor to save to my DB instead of the mission file, and then releasing it as a public sandbox/playground.
rlranft
Posts: 298
Joined: Thu Feb 05, 2015 3:11 pm
 
by rlranft » Fri Jun 05, 2015 1:48 am
How did you get around the "too many decal road nodes" crash?
chriscalef
Posts: 332
Joined: Mon Feb 09, 2015 7:48 pm
by chriscalef » Fri Jun 05, 2015 5:01 am
Not familiar with that crash, I didn't do anything special to get around it. Could it be something relating to too many nodes on a single road? I have a lot of roads, but maybe not so many per road.
  • 1
  • 2
16 posts Page 1 of 2

Who is online

Users browsing this forum: No registered users and 1 guest