Jump to content

How many decal roads have YOU put into a map?


chriscalef

Recommended Posts

Well this might seem a little random, but I just wrote something rather cool, I do believe, and thought I'd throw it out here for grins and giggles.


I'm working on my monolithic OpenSimEarth open world sandbox game project, and one thing that occurred to me recently is that I would really like to be able to import OpenStreetMap vector data.


The way to do this that I already had available to me was to incorporate OSM's vector data into my FlightGear terrain building process, and then get it into my world via my WorldServer terrain importer. This brings the roads in as textures on the terrain, but it has many flaws. First, it takes forever to process all that complexity using the flightgear (terragear) tools, and second, the end result is far from satisfactory, being very pixelated and unrealistic in appearance.


However, recently I had the thought: what if I imported the OSM data directly into Torque as an XML file, and then created decalRoads or meshRoads everywhere I should have a road? This would give me much better looking roads, with much more flexibility and better performance.


In order to do this efficiently, I made it into a two step process: First, on the engine side, I wrote a function called readOSM(const char *filename) which reads an XML file exported by OSM, and stores the required data in an sqlite database. Then, on the script side I wrote a function called makeStreets() which accesses this database and creates an appropriate decalRoad out of all the nodes, after converting all of the latitude/longitude coords into XYZ values for my local T3D mission of course.


There are still many problems, this being the first pass and all (primary among them large road segments standing straight up vertically, any place the road crosses a seam between two terrain tiles). However, I was pleasantly surprised by the scale I was able to achieve in just a few hours of messing around. At first I had a huge problem with slow sqlite connection speed, but then I realized I had to use "BEGIN:... COMMIT;" to make all the SQL happen at once, and after that things became most acceptable in terms of performance.


These pictures demonstrate an import of 10M of OSM data including approximately 4500 Ways, 26000 Nodes, and 49000 WayNodes. Not all of these Ways are turned into DecalRoads because some of them represent buildings or other map decorations, but all roads and paths have been drawn with one of several different available widths and one of the three different possible road textures provided in a stock T3D build.


Forward progress will of course include new road and path textures, as well as of course rivers and streams, and perhaps meshRoads in steep terrain. All in all, I'm not too unhappy with my first attempt, however.


http://indiemotionsoftware.com/images/terrainPager/osm_all_eugene.png


http://indiemotionsoftware.com/images/terrainPager/osm_all_eugene_2.png

Looking west from somewhere around the U of O.


http://indiemotionsoftware.com/images/terrainPager/osm_all_eugene_3.png

Looking south from above Skinner Butte.

Link to comment
Share on other sites

That's awesome @chriscalef


You have unleashed the power of T3D! UE4 doesn't even have decal roads ;-)


I whished we had that for the 8km2 map, while laying decal roads on top of a satellite image.

This method would have spared us so much time :shock:

Link to comment
Share on other sites

That's a pretty interesting approach!


I think that, doing something on that scale, having a new class to do roads via that data would be better. You could batch roads together to reduce drawcalls by a sizeable amount that way.


So a 'OSMRoad' class that takes in the data reference, then then manages the entire road network in as few render calls as possible, as it manages everything. That'd be pretty neat.

Link to comment
Share on other sites

Hm, that would be interesting. As is, they actually do render pretty efficiently, but I'm definitely at the outside limit at this point for my hardware. I think I have somewhere around 3000 active decalRoads in this example, which does run but I'm down to about 4-5 fps.


For my next trick, though, I'm going to start searching for nodes by distance from player, and then instantiate the roads only as they come into range. This won't work very well for flying vehicles, but should be fine for first person work.


Then the really fun part - it is my intent to use the Mission Editor to let people decorate the world, and then on save, put the newly placed objects into my sqlite DB instead of the .mis file. That way I can also search out and instantiate mission objects by player location instead of loading them all at once at the start of the mission.

Link to comment
Share on other sites

A quick guide? Can't say there is off the top of my head.


The basic premise would be to have a single object with a struct to hold a road's nodes, textures, etc. And then just have lists of the roads it contains.


Then you'd find all roads that share a material and batch them together into as few buffers as possible to massively trim down your draw calls.


I have an example of how to do that with an update/modification to the convex shape object I was working on. I thought I uploaded it to git, but it looks like I didn't do the right version. I'll get that uploaded and link you it so you can see how I batched by material.


I dunno if that'd be the optimal way to do it for this, but it may serve as a good starting point.

Link to comment
Share on other sites

Looks like a quite insane project, imagine you have 4.5 fps with only the roads, what will happen if you add buildings?

I would try to cut the satellite image into small chunks and then model the roads over it in a modeling app, so you have chunks of roads you can then place into Torque3D.

Link to comment
Share on other sites

@JeffR Thanks for the suggestion, and yeah if you could find that example it would help a lot, for all this time using Torque I've somehow managed to stay mostly away from the actual buffers/draw calls end of the action, could use a good sample. I have a feeling I'm going to be needing a lot more of that soon.


@Duion Yeah this was sort of a testing benchmark, I would hopefully never stack it with this many roads in a real scenario (unless the hardware allows and user prefers...)


Now that I have all the nodes in my database, though, I can start devising different strategies to sort and exclude roads & features based on distance/visibility/direction from the camera. Even for flying vehicles, unless you're at very high elevation you really can't see distant streets, because of all the buildings and trees.


Which would be the next step to add here. With the forests, either I'm doing something wrong or the trees don't care about roads, but I may need to make them care about that. And then for buildings, there is a certain amount of vector data available from OSM describing particular real world buildings, but at least for Eugene this data is pretty slim relative to the total area. There are some places in the world where the FlightGear community has gotten some impressive results using this strategy, however.


Overall my thought for OpenSimEarth has been to obtain a basic set of houses/retail/industrial buildings to randomly distribute in the blank areas, based on zoning data, and then borrow as much from FG as possible in terms of the osm2city logic.


Then of course the key to it all would be hooking up the world editor to save to my DB instead of the mission file, and then releasing it as a public sandbox/playground.

Link to comment
Share on other sites

  • 1 month later...

Hey @chriscalef is there a good place I should go to find heightmap data for real-world locations? I've been thinking I'd really like to recreate some real terrains... I'm not too concerned about doing the whole streaming terrain thing - if I could just find a way to get a black/white heightmap image that'd be awesome.

Link to comment
Share on other sites

Well, if you were in the US or interested in US locations, the answer is emphatically yes:


http://nationalmap.gov/viewer.html


For Australia or Europe, or other parts of the world, I'm sure there are sources but I'm not sure where they are... however a little looking around reveals that you can find SRTM data at (I think) 30 meter resolution here:


http://eros.usgs.gov/


There might be better data somewhere - for the US data in my first link you can get it down to 10 meter resolution, I'm not sure what is the best available in the rest of the world. You could also ask around on forum.flightgear.org for sources, a lot of those guys are based outside of the US.


You also might potentially want to take a look at my original "T3DTerrainMaster" project on github,


http://github.com/ChrisCalef/T3DTerrainMaster


just for ideas on how to import the height data, although I'm sure you can figure that out easily enough on your own.


Good luck, let me know how it goes!

Link to comment
Share on other sites

Another tip: if you don't already have it, grab 3DEM from here:


http://www.visualizationsoftware.com/3dem


It's an extremely useful tool for converting from SRTM and several other height data formats into .raw or image files. With it you can combine multiple height data files into one and then save them all in a format more useful to your final application.

Link to comment
Share on other sites

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.

Guest
Reply to this topic...

×   Pasted as rich text.   Paste as plain text instead

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.

×
×
  • Create New...