SkateReel - Technical Sound Design Demo Reel
SkateReel Behind-The-Scenes
Phase 1: Prototype
Synchronized Actions
Game engines aren’t necessarily designed for showcasing linear media. However, Unreal Engine’s Sequencer is optimized for a timeline-based workflow tailored towards building cinematics for games. By triggering events along this timeline in the same way that events would occur when you fire a weapon or take a footstep in a typical game, I was able to synchronize actions in a game without necessarily having a full player character, animations, VFX, and mechanics.
Prototyping this idea was key to the development process, as is the case with most game development work. By building a basic sequencer that starts when the game begins, and plays a sound when one of the test events is encountered—all while maintaining sync with a video—I was able to confirm the idea as viable before diving into the bulk of the work.
Phase 2: Field Recording
Record Something
I am not ashamed to admit that one of the many sources of inspiration for this project was a desire to get out of the house and record something. Throughout the project, I recorded, edited, and exported roughly 200 clips from:
Riding over a variety of surface materials
Performing tricks in motion and at a stand-still
General board handling and clothing rustles
Grinding on different materials (okay I may have faked this one, I’m not the best skateboarder out there)
Body falls and impacts
Ambiences
Cinematic moments, like lead-ins and slow-mo sections
The Gear
Microphones: Sennheiser MKH418s and a DIY, home-made M/S microphone using Primo electret capsules
Recorder: Sound Devices MixPre 3II (192khz, 32bit)
Other: Arturia MiniBrute Synthesizer (for the bass dives featured in the slow-mo sections)
Sonic Montage
Here’s a brief glimpse into some of the raw source recordings!
Phase 3: alignment
Frame-By-Frame
While the sequencer in Unreal Engine is excellent for playback in game, alignment on a frame-by-frame basis would be unnecessarily difficult if done entirely in engine. For this step of the process, I used the audio workstation Reaper. I’ve written many custom scripts in Lua to extend the functionality of Reaper and accommodate my preferred workflow, so doing this step was relatively swift and efficient.
Randomization
In addition to setting up alignment in Reaper, I was able to use some basic tools to simulate the randomization that would occur in the game engine. This helped to validate whether certain sounds would mesh well when randomized, without requiring extensive testing and rebuilding in the game engine.
Keeping Track
After wrapping up all of the linear work for SkateReel, I rendered all of the individual audio files out to my UE4 project using the UCS Renaming Tool I built for Reaper. This tool helps to standardize the naming conventions for files in compliance with the Universal Category System, and made keeping track of the many assets in this project quick and easy.
Phase 4: Integration
Dynamic Grind Timings
While static alignment in an audio workstation is relatively straightforward, once in the game engine, event timings need to be dynamic. For example, when doing a grind on a railing, some grinds are just half a second long while others are well over a second. To accommodate for this, I built a custom data asset (more on those in the next section) that includes three different sections for each of the grind surface types, metal and concrete:
Grind Start Sounds
Grind Body Sounds
Grind End Sounds
The grind start and end sounds are straight-forward as their names imply, and simply play sounds that book-end the grind. However, the grind body sounds are comprised of sample-accurate loops. When a grind starts, the “Grind Data Asset” randomly selects a sample from the “Grind Body Sounds” category, then begins seamlessly looping it until the sequencer fires a “Grind End Sound”. Together, with small fade outs to blend the start and end sounds into the loop, you end up with a cohesive grind audio system that is fully extensible to any length grind rail in the game without audible clicks or pops between samples.
Future-Proofing with Data Assets
Most mixing in Unreal Engine (in its current state) is handled through ‘classes’. For example, the sounds for skateboarding on various surface materials fall under the ‘Surface Class’, allowing for separate overarching mix decisions than other audio assets. Originally, I intended to do a ‘passive sound mix’ on the Surface Class that was modified by the Grind and Board Class. That way, when you do a grind or ollie, the sound of the surface you're riding on fades in and out accordingly. Unfortunately, the static fade in/fade out times meant it was best to implement this sort of functionality at the blueprints level for finer control. Data assets are perfect for this sort of situation, among many others, allowing you to set predetermined timing defaults on a case-by-case basis. The end result is a more future-proof system, in which we not only have distinct fades for asphalt/brick/cobblestone/sidewalk surfaces, but can easily introduce new surface types and respective fade lengths over the course of future development.
When you right-click on a “Play Surface Sound” node in the sequencer, you can pass information to the Surface Data Asset like the “Surface Type” enumeration for the material you’re currently riding on. The Sequence Director will then take care of seamlessly selecting and playing the appropriate sounds, as is the case with the grind system demonstrated below.
Phase 5: Reflection
The One-Person Post-Mortem
When asked about their favorite phase of the game development cycle, most individuals don’t typically answer with “the post-mortem”. For the uninitiated, the post-mortem is usually comprised of a meeting where the developers on a team sit down and seriously address any pitfalls that may have arisen throughout a project’s lifecycle. While confronting some of these limitations can often be stressful, and it is undoubtedly difficult to admit faults or lapses in performance, the lessons learned are often critical for improving in future works. Taking a step back and looking at the progression and development of this reel, there are a few matters that are quite glaring. And while I could rather easily re-open the project and make changes so that the outward-facing product is as perfect as can be, the self-imposed limitation of ‘shipping’ this reel and exposing my reflection process seems more important to me.
Sound Design
Okay so perhaps titling this section “The One-Person Post-Mortem” might actually be a bit ironic. Credit where credit is due, when reaching out for feedback, a couple of individuals mentioned that the delay effect on the board flipping sound is a bit excessive. In retrospect, I agree. Here’s a demonstration of the dry vs wet sound. When isolated like this, without the rest of the sound from the game going on, the version with effects sounds more engaging. This particular demo calls for realistic sounds more than designed sounds (except the slow motion and cinematic moments). While a bit of slap-back delay is expected, especially when skateboarding on concrete, the effects were simply a bit heavy handed on this front.
Integration
I ran into a pretty major road block in the development of this reel, namely with the Sequence Director blueprint that handles the logic behind all of the event playback (a pretty important aspect of this project, as you can imagine). After weeks of work, the sequence director suddenly crashed whenever I opened the blueprint. I tried everything to fix it, from duplicating the sequencer to deleting batches of nodes at once. To this day, I am still unaware of the roots of the issue. I’ve since filed a formal bug report through Unreal Engine’s dedicated site; even after switching to the newest experimental engine versions, the issue persists. The only fix for it was to pull from an automated cloud backup from a few days prior, then rebuild roughly 6-8 hours worth of blueprinting from memory alone. That process was rather painful, and if I could go back and change one thing, I’d implement basic source/version control, even for something of this scale. I’ve managed private GitHub repositories in previous projects, and think that would be a pretty safe route to pursue going forward. While full Perforce integration on a solo portfolio piece might be excessive, something simple would have saved a ton of headache here.
UPDATE 11/22/2020 - The bug report I made concerning the issue above has been addressed and resolved as of UE4.26 Preview 7! I have since been able to reload the original Sequence Director just as it was originally left off before the crash. If anyone from the Unreal Engine development team gets around to reading this, I’d like to extend a sincere thank you for the fix and all of the work that goes on behind-the-scenes to address niche bugs like this one.
Thank you for checking out the behind-the-scenes of SkateReel! You can download the ‘playable’ demo here. If you have any questions or would just like to get in touch, feel free to reach out through the contact form above or shoot me a message at:
aaron.cendan@gmail.com