Location: Main

This update goes under the skin

Drag[en]ginePosted Sun 30th Mar 2014 at 20:10:36 by Dragonlord Authorized by Dragonlord

This update contains not that much text which is good for you. Most work happended under the engine hood which is not that interesting to the outside world. This time around I had more ZPOC related experiments going on which produced a couple of bug and enhancement tickets to be added (bad for me as this means more work but good for you as these are problems fixed before the first release). Just read the descriptions in the videos ZPOC2 and ZPOC3 . Now on to to what is really important.

Skin system enhancement

First things first I wrenched around in the shader system adding shader inclusion support. This helps me to make more optimized shaders with more abilities that are more stable and easier to maintain. The power of the skin system bases a lot on this system there. Nothing for the game developer to get in contact with but nevertheless a very important piece of software, without which all this magic (which in some parts could even beat AAA engines) would not exist.

Besides this I've added also a video texture property. This has been requested since the current system requires artists to rely on the coders to provide a dynamic skin with a video attached to it. In some cases though you just want to play a video or animation on a texture (for example a monitor) without requiring to deal with dynamic skins. This is now possible.

Furthermore I've modified the behavior of renderables. Up to now you had to choose between static property content (value, color, image, video) and a renderable. This imposed not required limitations. Now the two are separated. Every property can have now an optional renderable name assigned. The static content (value, color, image, video) is used until a dynamic skin is assigned to a component holding a renderable with the matching name. A renderable is now a true extension to a skin property. If dynamic content is present it overrules the static content. If not the static content is used. I've tested this system in the ZPOC using "zoid paints".

There have been also other smaller changes not worth nothing here. Now on to the big one.

Sub Surface Scattering and Translucency

SSS is a buzzword but getting it working is a problem. Papers about this topic are mixed and the results are mixed as well. I don't want to talk long about SSS since I'm sure many know what it is and what it helps so see the new absorption and absorption.range texture property for a description. As always I've spent time to figure out how to add simple and easy to use texture properties to handle this case. So I've reduced it to absorption and absorption.range to provide both SSS and translucency.

I've taken the liberity to use the algorithm from my SSAO implementation since I've noticed an interesting behavior it exposes. This behavior I used for the SSS implementation. This implementation allows me to do SSS and translucency with only one single shader pass with no multi-pass blurring or texture space blurring hacks as typically employed. Furthermore this implementation is physically inspired which means it's not just blur-without-a-reason and not just a pseudo-SSS as some recent AAA engines like to use. This especially means the SSS and translucency implementation used in the Drag[en]gine runs in average 0.25ms. This makes SSS and translucency rather cheap and usable everywhere it's of use. So I'll let you off the hook for now with some images.
| |

And yes, the Stanford test models (dragon, armadillo, buddha) are included in the editor so you can easily and quickly test your materials with them.

Outlook

More tickets to work down. Next tickets are about the game editor. All in all working off more tickets.

Insight into my compressed room

Drag[en]ginePosted Sun 16th Feb 2014 at 1:13:58 by Dragonlord Authorized by Dragonlord

Windows in games is a huge problem. They are always done by using a flat texture on facade meshes. This results in flat looking facades which break the credibility of the world. The Drag[en]gine provides an interesting solution to this problem to spice up your game world without adding extra geometry or making things complicated. But first things first...

Preloading CPU Mip-Map and Compression

Besides bug fixing tickets one topic has been texture preloading. Loading textures is one of the most time consuming processes and different drivers can yield vastly different results. The most simple solution is to upload a texture base level and let the driver compress and mip-map the texture while uploading. This happens synchronously and can take from a second up to a lot of seconds with large textures. Also the quality of drivers compressing texture varies a lot. One of the tasks had been to solve this speed and quality problem. The mip map calculation and texture compressing has been moved now into CPU land. While loading textures asynchronously they are now mip mapped and compressed on the CPU before being uploading synchronously to the GPU. The performance is totally something else. Previously loading the game world took up to half a minute on certain drivers. With the new loading routine the loading of the game world takes only a few seconds now. Furthermore texture compression quality is now stable and fast no matter what GPU and driver you are using.

Meanwhile in Zi...

In between I took also a couple of days off to tackle a proof of concept side project. I use this to test certain engine features I can not test with the main game project. In this case I took a stab on that Shield Liger model I had around. I create the project files from scratch and whipped up a first working prototype in a couple of days. While working well this showed me where I still have to apply changes before the first engine version is released and I found bugs I fixed I didn't come across in the main game project. Last but not least this gives a little off-topic video to look at: YouTube Video.

Wanna see my room?

No, I'm not going to end up like those wannabe game-devs filling up their news posts with totally irrelevant videos of their dog, bed room and weather outside just to rake in PR although there is nothing worthwhile to talk about to begin with. I'm talking about rooms behind flat walls.

As mentioned in the intro this update deals with the problem of flat windows. Some years ago I came across a nice article on the Internet of a clever guy who had a clever idea. For some strange reason I've not seen a single AAA game-dev picking up that idea although it's cooler than a polar bear in a freezer. In short he replaced the flat texture with a cube map. Taking that basic idea I wrapped up a working solution which is as simple to use as anything else in the Drag[en]gine.

For this purpose new texture properties had been added:


If you want the details have a look at the linked wiki pages. In short this allows to specify an environment map type cube map to simulate a fake room behind the flat window mesh the texture is applied to. The texture coordinates of the unwrapped window and environmentroom.size define the size and orientation of the room. The cube texture coordinates are calculated in a way the fake room appears under the right angle while looking at it from different angles. This gives the impression there is a room behind a window instead of a flat texture.

The video below gives an impression on how this looks like comparing the conventional flat window texture with the environment room texture on the same flat wall: ModDB Video

Fake rooms combine with other texture properties. They sort of replace the color texture property with a simulation of a rectangular room. Otherwise normal, reflectivity and roughness still work the same allowing to have window glass reflections while looking into the room. All this without touching shaders at all nor trying to get all tangled up with shader node systems.

Outlook

The next tickets to work off are a bit more technical so the next news post most likely will take a bit longer since technical stuff doesn't show much with videos or images. The close goal is still the first engine release. Working off tickets is still coming along nicely.

Take it to the web

Drag[en]ginePosted Sun 19th Jan 2014 at 20:15:47 by Dragonlord Authorized by Dragonlord

As mentioned in the last news post I decided to pull forward in the backlog a topic which has been brought up recently a couple of times: web engine. The idea of getting a high performance game engine into the web is not new but getting the job done right is still a problem. There exist currently only two possible solutions. Either you use a plugin to go directly to the engine or you use some javascript recompilation.

Drag[en]gine Browser Plug-in

   Using javascript recompilation though is slow and inflexible since you run all through javascript with WebGL. Implementations are not standardized and the browsers influence a lot the actual performance. WebGL too is a nice thing but not on the level you need it for high performance game engines. This leaves only a plugin as a viable solution until browser javascript can fully access system C-libraries natively without recompilation.

   Using plugins usually ends up with limited engine access. This is not what I wanted for the Drag[en]gine. Thus I've implemented a plugin system which does not only place the engine in the website but gives you control over it.

   The Drag[en]gine plugin comes as a bundle of a plugin library and various JavaScripts. The plugin currently supports NPAPI for largest support but ActiveX will follow. Additional support is simple since due to the logic residing in the JavaScripts. The plugin itself provides direct access to an engine instance you can fully control from the startup to shutdown.

The JavaScripts provide default launcher functionality. They do the following for you:
Load installed Drag[en]gine , modules and installed games
Sets up the engine using found launcher configurations on your computer
Checks all game requirements are fulfilled
Provides support to launch an installed game
Cleaning up the engine after exiting a game
Logging directly into JavaScript
   The provided launcher functionality works similar to the console and GUI launcher installed by the Drag[en]gine. You can thus use the plugin to simply play games through your browser window or (and this is more interesting) to play games straight from the web. You can simply fill the engine with resources from the web and start it up with a Web-Game profile. This way you can make a game with web-content but using the installed Drag[en]gine for maximum performance and compatibility. The same platform independence holds true for the Drag[en]gine between the Web and PC like between PCs.

Creating a game for PC makes it automatically work on the Web and vice-versa with NO extra work, NO recompilation, NO repacking and NO second versions to manage.

   Here a little commented video (1680x982) to show how this might look like. The HTML/CSS you can completely modify since the JavaScripts provide functionality leaving the design up to you: ModDB Video.

Outlook

   Now is time for some more remaining ticket reduction work. Let's see what the next news post holds in store for you.

Lighting Wishes

Drag[en]ginePosted Mon 23rd Dec 2013 at 14:32:54 by Dragonlord Authorized by Dragonlord

In the last news post AniCator asked for ambient occlusion for the cloudy day system. First things first the Drag[en]gine knows already texture defined ambient occlusion using the ambient.occlusion texture property. You can generate these using Blender3D for example. Since he asked though if dynamic ambient occlusion would be possible I took a little side-step to plug this into the engine.

Dynamic Ambient Occlusion

Dynamic ambient occlusion is something game developers like to use since some time. You can plug those in as a post-processing effect and get done with it. Unfortunately most implementations found so far in games are crappy. I do not wanted to have crappy SSAO in the game. Better no SSAO than crappy one. So I dug around recent implementations evaluating their usefullness (and/or crapiness) with my own modification idea in mind. As it turns out somebody else had a similar idea just recently which is usually a good sign if two minds come up with the same idea. The implementation sails under the name of Scalable Ambient Occlusion or SAO for short. The basic implementation is nice and is screen resolution and depth resistent with constant cost but suffers from some math which likes to produce the dark halos as mentioned in the article above. Altering the math a bit though with a more "sane" mathematical background helps to reduce the problem. Removing black halos though tends to introduce white halos but these should be less problematic.

Cheap implementations plug SSAO in as a post-processing effect messing with all lighting. This is utterly wrong and is another source of bad results. In the Drag[en]gine the ambient occlusion (both texture and dynamic) affects only the ambient lighting. The direct lighting is not "directly" affected but more about this later.

So here first some images of the SAO in action. The shadow areas receive subtle occlusion effects. Over-done SSAO looks crap so the goal has been to have physically plausible ambient oclcusion instead.


And here a test video showing first the pure ambient occlusion texture and then some sample location s with SSAO enbled and disabled. At the end you can also see how the effect does not affect geometry in plain sun light but only in shadows.
Video ModDB

AO Self-Shadowing

With a little extra line of code in the shader the ambient occlusion can be used also to provide a form of limited self-shadow casting. The Drag[en]gine limits this to texture ambient occlusion only since this AO is of higher quality and of smaller scale. This shows the result.


And here a video with a turning light source. The self-shadowed version is a subtle enhancement of the shape perception with the advantage of having no additional cost (no cone-stepping required). As such though the method has it's limits.
Video ModDB

Outlook

There's room for improvements with this entire SSAO stuff but right now it is more important to work off more of the remaining tickets for the first release.

The Large News Bomb

Drag[en]ginePosted Mon 23rd Dec 2013 at 14:26:26 by Dragonlord Authorized by Dragonlord

This year everything went somehow different than planed in the first place, for the worse or the better. Besides hardware issues some tasks had been so large and intertwined that making a news post earlier had been tricky. Besides that I also didn't feel like making a news post in the last couple of month. Whatever the reasons out of the work done these tasks had been the most time consuming and important ones.

The main focus had been on reworking the shader system to lift it up to the next level. This had been unfortunately required to get all the bells and whistles done I've planed for this project. This included skin and lighting shaders. This took quite some time since the shader system in the Drag[en]gine is deeply integrated for versatility and performance reasons. Also decal rendering masks have been improved to allow more natural decals (see puddle image).

Another focus had been on optimizing rendering and lighting on various ends to increase performance where possible. This took also longer than expected.

An unfortunate focus had been on upgrading my entire PC leaving me with nearly 2 month of no chance to work on the project. Never before an upgrade went so TARFU. At last the HD7970 is now humming alongside the QuadCore as it should except I'm only running at 75% possible performance but the upcoming driver upgrade should fix this (hopefully).

The last huge task involved redesigning the reflection system. I've evaluated and test-implemented various reflection approaches from simple env-map blending to local reflections to find a good system which actually works with Physically Based Rendering. The system in place now works with a combination of global and local reflections with distance roughness, optional projection boxes and optional mask boxes. This all works across all geometry types in the engine without any additional work by the developer.

Eventually a first version of a weather system has been added to the game to test the reflection system.

This is only the most important part of all the work done. The rest is included as the typical bullet list at the end of the news post to not scare away people with a huge text wall. To counter the text here a few images and some test videos. To not spam only the first of the images in both profiles are linked. Find the rest in the respective gallery going to the left ([u]15 images in total[/u]).


Here a small video of the reflection system in action. I think images work better but sometimes people like more something that moves. So here all put together (video-grab messed up the colors).
Video ModDB

And here something for the lovers of details. The new weather system also includes cloudy weather code. You all know how lighting conditions change quickly if clouds pass by the sun on a cloudy day. This code uses the Drag[en]gine Sky System to simulate this effect. Since in a detective game locations (sets) are important I would like to add some effects to them like this one here. Watch the entire video to see it. Other weather conditions will be added like foggy or soft/heavy rain to make revisiting locations more interesting.
Video ModDB

Outlook

Although during this year work has been slower than anticipated the number of task tickets left for the first release got less. The end is not far away if nothing else breaks right before the end. With these interwined tasks out of the door news posts will be more frequent than they have been up to now.

The Big What Has Happened In All The Time Short List!


Here the very short list of what happened during all this time. You can stop reading here if you don't want to know all the lengthy details.

  • Reworking skin shaders affecting entire render pipeline. Generic and customized generated shaders using extended über-shaders. Texture properties applicable on all geometry types with the same result (components, decals, particles, terrains, prop fields, asf). All variations simultaneously usable at all times (multi-purpose shaders). Higher stability, better optimization potential and easier to maintain and upgrade.
  • Support for code generated shaders in addition to modified über-shaders
  • Reworking light shaders with a similar system to skin shaders.
  • Splitting of shadow rendering into static and dynamic shadow maps with automatic switching of objects between static and dynamic
  • Shader based debug shapes rendering for better visual feedback in editors
  • Shader parameter support using UBOs for optimized parameter handling in skin and light shaders, faster parameter switching and higher reuse potential
  • Sun shadow optimizations culling shadow shafts with occlusion maps (shadow casters in shadows)
  • Improved decal rendering with per-texture property masks: color.transparency.multiplier, ambient.occlusion.transparency.multiplier, normal.transparency.multiplier, roughness.transparency.multiplier, reflectivity.transparency.multiplier
  • Added transparency.multiplier texture property
  • Game similar editable sky in editors with pre-set skies to test with a single click
  • Improved particle emissivity rendering replacing transparency based hack
  • Improved tone-mapping and bloom fixing smearing problems due to ATI/nVidia driver quirks
  • Global reflections using 4 closest env-maps blended into 1 for weaker systems or 8 closest env-maps tapped in shader for stronger systems delivering better results
  • Distance dependent reflection roughness
  • Local reflections (SSR) using screen-space or view-space algorithm with long ray support
  • Fake bounced-reflection for local reflections avoiding black reflected metals
  • Optimizing texture format usage for less memory using special OpenGL formats
  • Added reflected texture property
  • Env-map probes with render groups for reflection control and probe priority
  • Box projected environment maps with mask shapes
  • Equi-rectangular env-maps for hardware with cube map problems
  • Rendering layer masks for fine control over visibility
  • JPEG image module
  • Light volume box clamping optimization using static shadow map
  • Weather profiles with cloudy day effect (see video)
  • Indexed rendering and vertex cache optimizer
  • TBO based instanced and improved cluster calculation for prop fields
  • Imposter billboards for prop fields in the distance
  • "shape list" property type for game objects