VFM02: Unity 5 Realtime Render Test

Within literally a single minute of opening Unity 5, I had created a new project, dropped in my (very work-in-progress) Maya scene, and bam: rendering exactly as I wanted by default… in realtime. I spent only an additional 30 minutes loading up and tweaking scripts to calibrate Unity’s camera, getting a more polished look (anti-aliasing, vignetting, chromatic aberration, slight bloom, depth of field, screenspace ambient occlusion, etc). The prototype image is nothing breathtaking (yet!) — this test is entirely about process, efficiency, and effectiveness.


For virtual filmmaking, this blows Maya out of the water for rendering, even though it isn’t as perfect as a Maya render can ultimately get. The trade off is minimal. It took me a half an hour to get this set up and only a split second to screenshot it at 2K+ res. Getting additional shots as I develop this set is going to be a simple process of opening up Unity again with changes made in Maya automatically updated in the scene, kicking off and launching a build, and taking screenshots through cycling around the same anchored cameras. We’re talking minutes, instead of hours or days. Imagine, rendering a 60 second animation will take exactly 60 seconds… instead of two days… and it will look like filmic CG. We are now there.

Beyond getting instant renders, the ability to iterate and review has launched into warp speed here, freeing up more time to try even more things and get the shot just right, ultimately through less work. And if I’d want to change things much later along in the project, going back, making that change, and getting a new capture will be trivial compared to Maya, especially for animated shots. Setting up new shots, like below, took seconds.


I spent all day yesterday in Maya trying (and failing) to achieve a render as good as this: learning and tweaking render settings, quality levels, and testing lighting with lengthy (40 minutes to 2.5 hours) single still image test renders between 2K and 4K. With every tweak, I’d have to wait minutes for IPR to re-render even a 10% scaled preview so I could (barely) see how my changes were taking effect in as practical a workflow as it could provide.

Compared to that, the benefits of using a game engine to render in realtime are enormous, especially for an indie or small team studio. Only now that game engines can render graphics this well is that even professionally viable. Literally, Unity 5 was released yesterday. Machinema is not a new concept, but machinema now possible at this ‘next-gen’ level of quality certainly is new indeed (and it’s only going to get better from here).

I wouldn’t expect a multi-million dollar funded film and VFX house to jump over to this (though they too might find this as appealing), but for one guy wanting to tell a cinematic story on a no/micro budget, this presents a huge advantage. Additionally, I can create assets for both a film and a future related video game simultaneously, preparing them all in-engine as I go. That is also enormous.


There are a few issues I’m aware of right off the bat:

1) Resolution — My now four-year-old 27″ iMac has a maximum resolution of 2560×1440. When I bake a build and run it, that’s the best I can get for the realtime render from a direct screenshot or video capture. This is perfectly sufficient for the current (outgoing) standard of 2K (1080p), but I’m going to want to get 4K renders at least, if possible. There are two solutions to this off the top of my head: 1) get the new 5K iMac. Then screenshots can actually be more than 4K when running a build at fullscreen. 2) DVI-to-HDMI my existing iMac to a 4K UHDTV as a second monitor and run the build over there. Of course, both of these solutions are a tad ridiculous and certainly expensive.

UPDATE 2015.03.05: 3) The Unity Asset Store has plug-ins that will do screenshots at any resolution– even video capture. There’s a $20 one called Screenshot Creator, but upon loading it into Unity 5, I got a bunch of errors and it was unusable. There’s a free one, Instant Screenshot, which works, but it doesn’t render with camera effects. I could get an 8K render and scale it down to 4K in Photoshop to artificially create anti-aliasing at least, but I lose all other desired effects, especially depth of field. That’s probably the most viable solution so far. It still beats the heck out of waiting hours for a Maya render at 4K. You have to wonder why a game engine doesn’t have the ability to take screenshots at any res as a built-in feature, especially these days– but I guess that’s why the Asset Store is great if not often frustrating as various assets often break or don’t play nice with others.

2) Performance Framerate — Running this build on my four-year-old iMac, I got 17fps at first. All the camera effects dragged this down. I may be able to have a ‘naked’ camera (no effects) and replicate those effects in post via After Effects. Now, if the objective is to just get still image plate renders and bring them into Photoshop and After Effects, this doesn’t even matter. But, I’m increasingly considering going all-CG with filmmaking, at least for some of the types of projects I want to do, which means I’ll need to animate. Doing everything in Unity allows me significant animation advantages, I’d think, since I can review playback and iterate in realtime. So, one solution is I could get a much cheaper Windows gaming PC with super-beefy specs and use that to capture engine video over at least 30fps to 60fps performance runtime (the capture would only get probably 30fps at best, but for film I only need 24 and will drop the video down to 24fps in After Effects anyway). Alternatively, maybe the 5K iMac can run this stuff sufficiently, which means getting it would kill two birds with one stone. But performance framerate will be exceptionally important not only to get a clean capture, but to also ensure voice actor performances and animations all sync up as intended on realtime playback.

(Later, I did some tests and discovered the screenspace ambient occlusion on the camera tanked the framerate most. Camera Motion Blur even lead to crashes consistently. Without them, I could get anywhere from 36 – 155fps (???) on my old machine and can simply bake ambient occlusion instead. With no camera effects, I get perfect 60fps+.)

3) Limitations of Realtime Assets — The gap on this is closing, especially now, but there will be certain limitations on realtime assets, when in Maya you can have as many polys and as high-res texture maps as your system can handle. I’d say it’s very likely a hybridization between using game engines and Maya to render elements of varying complexity is the way to go, compositing them together in post. For example, I can use a game engine to render environments, but use Maya to render characters. Or I can do multiple passes in a game engine by simply turning on and off layers of game objects or making them invisible, like in Maya. But ultimately— this issue may actually be an advantage, as it will force me to develop assets through games practices that prep two types of product for the same IP simultaneously: films and video games.

4) Less than Perfect Rendering — The contemporary game engines of today do a damn good job, but of course not as good as something meticulously set up and rendered multiple times in passes through Maya with Vray. Still, this is maybe something I’d just have to accept as a trade-off for working this way in exchange for the enormous time saving advantages. To work around this, I can always enhance capture in Photoshop and/or After Effects manually, which is likely still saving time. And I have to ask myself, do I really have the time or any ability to create a renderfarm to do this in Maya practically anyway? That’s probably a stretch, especially for something more than just a short film. Finally, what if photo-realism isn’t the goal for a CG-film project, but instead a stylized hyperrealism? Then this will matter much less. Look how amazing this recent short CG film is in its totally stylized look fit for a game engine, made by three people: Le Gouffre — inspiring! They even talk about how they enhanced every single shot with a ton of layers in After Effects to fully maximize their film’s look, so there really is no requirement that the plate render be perfect either way.


Using this layout, I have a 1920×800 (2K CinemaScope) Game Viewport,
so I can grab render captures directly out of the Editor without
even having to kick off and launch a build at all.

Overall, not too bad. Will be thinking a lot about this in the future. I will probably be upgrading my Unity 4 Pro licenses to Unity 5 Pro very soon. At this point, I’ll be returning to modeling up the VFM02 set again now, knowing I can get a render of it at a desirable quality without having to painfully mess with Maya’s rendering and lights any further. This changes my virtual filmmaking formula, which is why I love prototyping– that’s exactly what VFM02 is all about.

UPDATE: 2015.03.04.23:11

Did a camera flythrough animation and video capture test. With all camera effects on, I’ll need stronger hardware to get this at a perfectly smooth playback in realtime, as it ended up dropping frames occasionally. Otherwise, I might be able to get it at a better framerate with effects off (UPDATE: yep, at perfect 60fps), and then simulate them in post via After Effects.


For comparison, this same shot took Maya two days to render when I did it for VFM02’s initial graybox edit back in January. Using Unity 5 and lossless screen capture, I got this in a high-definition 2KSCOPE .mov within 30 minutes.

IMDb / LinkedIn

← Blog Home

Maya Mentalray Final Gathering Test Render

Using the first model I made for VFM02’s virtual set, here I experimented with Maya Mentalray Final Gathering. I used tips from the first lesson in yesterday’s Gnomon lighting tutorial.


Click here for the full 4KSCOPE render.

This was far more straightforward to set up than the more complex full scene with many light sources. I’ll be going back to that full scene again in Maya to wrestle it to this level. Later I’ll be experimenting with lighting it in Unity 5 for possibly a much, much faster set-up and realtime render capture of complex scenes.

IMDb / LinkedIn

← Blog Home

Next-Gen Game Engines for Realtime CG Filmmaking


Unity 5 was released today at Unity’s GDC 2015 Press Conference. Like Monday’s announcement with Unreal Engine 4, Unity offers a free full-featured version for individuals or small businesses under an annual revenue limit. The Pro version is either $75 a month royalty-free or a flat fee of $1500 (upgrade $750) royalty-free (versus UE4’s 5% royalty over $3000 revenue per quarter, which could end up being millions). Valve also announced today their Source 2 engine is on its way, free to all as well. This is the GDC of free next-gen game engines, that’s for sure.

While all of this was going on at GDC, I made significant progress today in Maya with Gnomon lighting tutorials. With simple test scenes, this worked out extremely well. However, with the more complex virtual set scene featuring many light sources for VFM02, I’m frustrated with ray result quality issues and render times, as they directly correlate to quality. It even makes the process of adjusting lighting to set up and try various things very lengthy and tedious. Especially, when I’m used to game engines doing this on the fly already (though usually not at perfect filmic quality). In prototyping methods to make high-concept indie movies on micro-budgets, I must always be thinking of techniques to keep this process as relatively fast as possible.

A single 4K render at Production quality earlier in the day took 2.5 hours and Final Gathering left unacceptable artifacts everywhere. Increasing rays to fix this would inflate render times exponentially. This last render is filled with artifacts and grain, glows are blown out more than their preview, and the lighting in IPR previews and the editor looked little like the final render. I also tried a Photography simulation Lens Shader, which is calibrated too dark here after some changes. After this 40 minute 2K render, I laughed at how bad it turned out (especially compared to how well the stand-alone set piece test render turned out).


Now, I’m sure there are dials to twist in just the right way that I’ll figure out over time. Currently, though the full set is taking so long to properly light and render at sufficient quality, that the entire time I was thinking– why am I not just doing this in a game engine in real time? One obvious reason is how I want to render at 4K, but…

Below is Unity 5’s realtime graphics demo from today, which proves we might be at an intersection between games and film now where indie filmmaking can find enormous value in machinema, finally at a professional-grade quality. For filmmaking, a hybrid between machinema captures, Maya renders, and After Effects enhancement compositing is a likely route. I’m increasingly considering going all-CG, including characters, and this made a strong case for that direction. They even claimed this was running on a typical contemporary gamer PC, nothing ridiculous.

While you watch this, keep reminding yourself that this is apparently rendered and post-processed all in realtime with a free game engine, because it’s going to be easy to forget:

Especially since I’m already well versed in Unity, trying to light the full VFM02 scene in Unity and get a capture from realtime rendering will be an appealing test. But I’ll do that later— I still think the best route is to figure out lighting rendering in Maya since the quality can be so high with it at such high resolution. But working in realtime may be so much of an advantage for rapid iteration, playback, and refinement that it could end up pulling me into Unity 5 for virtual filmmaking more than I might expect.

UPDATE: 2015.03.04.10:30

Well before turning in last night into today’s early AM, I went back into Maya, deleted all lights except one I had set up as the Sun, and added just two fill lights. I realized Incandescence and Glow actually can substantially affect lighting in the scene as actual light sources and not just appear as fake post-process effects. So, I could remove the dozens of lights I had set for all of the florescent bulbs. I woke up to a completed 4K render– but forgot to see how long it took! I at least know the test renders were much faster than before.


This is at least not a wreck nor did it take nearly as long, like the previous renders. While the light on the graybox elements reminds me of stuff I used to get a decade and a half ago on Bryce 3D, the actual modeled and textured set piece element (the only one so far) looks very well rendered. It makes me think that this lighting works and once the set is fully modeled, it won’t look so bad— the worst part right now is how the Sun key light scatters artifacts all over the flat surfaces of the graybox, but you can’t see any of that on the modeled set piece. Shadows seem slightly too flat or the scene is too lit now or something– this is still off. The glow effect on the florescent bulb material is also stronger on the 4K render than it is on the 1K test render, so that’s fun– the size of the florescent tube objects also affects this, so maybe I should bring down the ones along the walls. The Moon and Sky-sphere are stubbed in there– ideally I can get them to work so I don’t have to replace them in post. In fact, it’s my goal that I won’t have to do much of anything for it in post.

Luckily, I think this is back to a usable point where I can continue to tweak this, continue to go through the Gnomon Maya mentalray lighting tutorials, and resume modeling the set. Lesson learned about quantity of lights in a scene and glows actually being light sources too (not just fake ones).

And I am still curious to see what this looks like in Unity 5 and Unreal Engine 4.

IMDb / LinkedIn

← Blog Home