Microsoft Flight Simulator – WORLD “Azure AI, Offline and Online Mode, Details, Bing Maps [Video]
Hi everyone I am Lionel Fuentes lead engine programmer here at Asobo studios.
We are developing the next generation of Microsoft Flight Simulator.
As you can see behind me people are hard at work testing the current work in progress version that we have. When we started working Microsoft Flight simulator that was at the same time scary, and super exciting because there are so many things to do.
I will talk about one of the biggest things we had to do and this is the key feature for flight simulator and that is, the world. I will speak about the world through four main points.
I will speak about the data sources, the augmentation from those data sources that we could compute on Azure, more on this later. A little bit about the impact on bandwidth, and then some procedural generation techniques that we developed. So the world, our planet is really beautiful. It is full of natural wonders from Isles Rock, to the Himalayan Mountains, to also man made features like the Statue of Liberty, Mont St Michelle, the Eiffel tower.
So we wanted to get all of those features in the simulator for you to enjoy.
So when we started doing this we realized how much the world is big.
The world is really big.
To the point we even had to change internal way represent a 3D point in space in our engine like from a single precision floating point to the multiple floating point. Because we couldn’t handle that big of a world.
And the biggest impact of this was on the amount of data because you need a lot of data to represent the world. Luckily enough Bing maps was here with 2 petabytes of aerial imagery, elevation data, and 3D scans for static number of cities.
So all of this data was there sitting ready for us to stream in the simulator this allowed us to get all the airports in the world, which we estimate to more than 40,000, to get all the cities across the world which we estimate to more than 2 million, to get all the roads in the world, to get all the mountains in the world, basically the entire world is there available for you to fly.
Not only do you have the entire world, but it also matches the real world.
The big consequence for this is that now VFR flying is available for you everywhere.
So for those of you that don’t know what VFR means, it means visual flight rules and it’s a way of flying your aircraft by relying on visual queues in the environment like where you see a given city, or given river.
So for example when uh in real life we do some piloting and when in the DR400 we can see the Garonne River, and the city of Martigues we can deduce where we are going.
The amazing thing is now we can re-create this very experience right inside the simulator.
So I encourage you once you get the chance to try it to fly over your house.
So we got all those data but then we needed really more data than that.
The reason we needed those data is because we still didn’t know where to place trees.
We still didn’t know what shape buildings should have, and uh what color it’s roof should be, what the type of roof over there so in order to generate those additional data we relayed on Azure cloud computing. Azure allowed us to run AI algorithms, and computer vision algorithms, in order to detect all the trees in the world which we estimate to more than 1.5 trillion. It allows us to improve the buildings generation system by detecting the color of the roofs or the roof type. It also allows us to add missing buildings information where we don’t have these information in the data sources we use.
It also allows us to improve on the quality of Bing maps aerial imagery like sometimes some area can be captured with clouds on top of them and then we can detect those and replace them with texture synthesis that happens at the run time. So to put some images on what I’m talking about this is an Island near New Guinea. If we only had Bing maps data there, the island would be kind of flat and you wouldn’t have the trees and the buildings.
So thanks to detection we could run on Azure, we know where to place trees, we know where to place buildings, we know what the buildings should look like and this allows us to create a much more believable and precise/accurate representation of the earth. But what about the impact of my internet connection because at this point you may be wondering we are streaming all of those data, like Bing maps data, Azure augmentation data, we represent the entire world so what happens if I don’t have a high bandwidth available?
What happens if even if I have no internet connection at all?
We asked ourselves this very question and I would like to answer it through three main points. The first one is that we do adaptive streaming. What this means is that we measure real time bandwidth you’re using and adapt our bandwidth requirements according to this measure. The net result of this is that the better your bandwidth, the better your overall experience. Next we developed a full offline mode.
This is for the extreme case where you have zero available bandwidth we synthesize the ground texture.
We also still have all the buildings data and all the water data and for some regions also the trees data the results of this that we still have a world where VFR (Visual flight rules) flying is possible. It gives us a reasonably accurate representation of the world.
And the third point is that we can pre-cache arbitrary regions of the world.
What this means here is that you can select regions of the world that you want to uh pre-cache and you can access them in the offline setting while still having the same quality you would enjoy in an online setting. To put some images again on those words I will show the three different quality levels that you can experience. First one here, offline mode.
This is the Seattle area in offline, and as you can see all the buildings are there all the water information is there and the city is recognizable. Second here is online Paris.
So the majority of buildings here are being automatically generated and you also have the uh water information and its more accurate version of the world, and here again VFR (visual flight rules) flying is possible. Third here is where we have the best quality.
This is the city of New York you have may have recognized it.
This is when we photogrammetry data in Bing maps for over 400 cities.
But now, how about the fine details.
Because it is impossible to accurately capture all the very fine details of the world.
Like you cannot know exactly where each blade of grass is, or which precise details you have on a given building so, in order to add this level of precision to Microsoft Flight Simulator we relayed on procedural generation. We are adding ground details for dirt, grass, and asphalt.
We generate grass blades of actual 3D geometric grass blades we display millions of them on screen. We also add water shading up to the point that we even have the water effected by the wind which creates waves on the water. We also add night lights both along the roads and on the buildings. We also add details to the buildings and all of this helps create a much more believable world. As you can see here this taken in Bretagne France and as you can see you have the procedural grass being generated, you have those little houses that are generated, you also have those trees that are being generated, procedural.
You also have the water shading for the sea and it gives a much more realistic representation of the area. So this is it for the world of Microsoft Flight Simulator.
We hope you really enjoy it as much as we do.
Thank you very much.