Sound design for the switchboard scene
/ by Jan
Hi there, this is Jan Morgenstern writing from a currently particularly inhospitable part of northern Germany. I’m responsible for all things going “bonk” in Elephants Dream, and as some of you expressed their interest in occasional digressions into sound design and compositional topics in this blog, I’ve written a little blurb on SFX design for a scene near the beginning of the movie.
The Setting
In this scene, the action takes place between giant telephone switchboards from the pre-digital era (such as these). Between the switchboards, long wires are frantically buzzing through the air, constantly connecting and disconnecting from the boards. The setting itself is part of an infinite machine, so there are no walls, just endless rows of weird machinery in the distance.
You can catch a glimpse of an early version of the switchboard in our teaser, and this post by Toni explains the inner workings of the wires. Anyway, let’s move on to the sonic aspects.
The Paranoia Simulator
As the wire action between the switchboards will be one of the defining elements of this scene, it was clear that it’d need a distinctive sound that would later take the most prominent part in the soundscape (apart from the dialogue, which has top priority at all times). Bassam had the idea of having the wires “leak” short snippets of phone conversations that surround the characters at all times, constantly moving around them or even chasing them, and thus creating a diffuse, surreal, and threatening atmosphere that envelopes our heroes.
In order to find appropriate source material for this effect, I rummaged through the web in search of some public domain voice recordings. Half an
hour later, I had recordings of some presidential speeches, old proceedings from US courts, and talks from the last Blender conference on my HD, and glued them together in order to create a single, large audio file of about 1 hour of playing time. I might end up using different source material in the finished movie, as we’ve been considering using recordings of our actors (maybe reading dutch christmas poems? ;)
As I played around with the voices, it quickly turned out to be rather tedious to create the desired illusion of movement manually, since the levels, panorama positions, and pitches of the voices had to be manipulated in unison in order to create a halfway convincing effect. Besides, as I only had an early animatic version of the scene as a reference, changes in the visuals would probably have made it necessary to throw large parts away and re-create them from scratch later. So I decided to delegate the dirty work to an automated signal chain I created in a software called Reaktor, which is a modular DSP environment. Reaktor can be thought of as a kind of virtual electronics breadboard in that it allows its user to build signal processing structures for almost every conceivable kind of purpose by interconnecting modules – ranging from simple arithmetic operators through various types of oscillators, modulators and filters to I/O modules.
At the core of the signal chain (the affectionally-named Paranoia Simulator) is a sample player that gets initialized to start playing my source file at a random point in time. In order to introduce some additional chaos, the initial playback pitch and level are slightly varied in a random fashion. The audio is then passed through a bandpass filter, once again with a randomized center frequency, to simulate a lo-fi transmission.
This signal is then fed to a substructure that simulates a sound source’s movement along a horizontal plane, along with the resulting level attenuation, time shifts, and doppler effects. In the original design, it also placed the source within the stereo panorama according to different microphone setups, but as I needed 5-channel surround output this time, I left this part out and fed the resulting (mono) signal to a surround panorama module instead, which in turn feeds the simulator’s audio outputs.
The movement itself is created with a simple Lissajous generator, which
generates 2-dimensional motion patterns based on sine functions. Once
again, the parameters are constantly varied randomly in order to have the voices move around the listener in a most unpredictable way. This generator can easily be replaced with a user control in order to provide manual control of the movement. This way, I can quickly add some layers of deterministic movement against a randomized backdrop if some of the wires are doing anything relevant in the foreground.
Enough yakking, let’s hear this doohickey! :)
This is the output from a single processing strand, with no further
processing:
surrounded_by_bassam.mp3 (Stereo, 812kB)
surrounded_by_bassam.ac3 (5.1 Surround, 1.3MB, playable with VLC)
For the final effect, I increased the voice count to a larger number
(10 in this case) and added a bit of reverb:
voices.mp3 (Stereo, 984kB)
voices.ac3 (5.1 Surround, 1.6MB)
Jack in (and out of) the box
There are still some things happening in the visuals that don’t have an acoustic counterpart yet. For example, plugging jacks into and out of a switchboard makes a distinct clicking sound. I recorded plugging an audio jack into the rear of an old outboard unit, loaded the resulting sounds into a sampler, assigned them to different keys on the keyboard, and triggered them randomly:
jacks.mp3 (Stereo, 420kB)
For the surround mix, I simply added a second stereo layer of clicking sounds at a lower level (to avoid drawing the viewer’s attention away from the screen) to the rear speakers.
Bells’n’whistles
The soundscape is already getting a little crowded at this point, so I’ve tried to keep additional sonic elements more or less in the background; also, since there’s already a lot of movement, I preferred the remaining sounds to stay at static positions in order to provide some anchoring. As the characters roam freely around a large industrial environment, there’ll be omnipresent sounds of machine parts everywhere. Since the purpose of the machine is not clear at this point, the environment sounds should be somewhat diffuse in character and should not point to a single, obvious origin, so I’ve mostly resorted to a rather eclectic mix of mechanical, hydraulic, electric, purely synthetic, and organic sounds.
Putting it all together
Now I had separate stems of the 3 elements I described on my HD, so I could go ahead and mix them in order to get the final SFX track for this scene. To enhance the feeling of vast space, I’ve added some reverb to all elements – as the viewer’s focus rests on the wires and jacks in the foreground, those got less reverb and more direct sound, and vice versa for the background machine parts. In a last step, I’ve slightly attenuated the frequencies between 500Hz to 2.5kHz in order to make sure the dialogue won’t get disturbed by other elements (this frequency range is most critical to speech comprehensibility). This is a somewhat preliminary measure, as I suspect there’ll be plenty of things that’ll need some additional tweaking in the final mix.
This is the semi-finished SFX track for the scene:
soundscape_final.mp3 (Stereo, 1.4MB)
soundscape_final.ac3 (5.1 Surround, 2.1MB)
That’s it! Of course, I appreciate any feedback (like “I made it to the third paragraph before falling asleep and/or into a coma!”). I’d also love to hear from people with surround rigs hooked up to their computers out there (if any) whether the AC3 files work for them.
« Character Design | Actors Cas Jansen and Tygo Gernandt sign up »
Only thing I can critique is the jack plugin in and out, it sounds like typing in a keyboard. It doesn’t sound nice.
Perhaps it could be a more powerfull feeling, like the one you hear when you plug some cheap speakers to a sound card and the speakers are on and the volume at max. It is a more powerfull sound and one people expect to hear when it plugs. It does that glitch when connecting and the same sound but looking more amplified and with a big bass along with a minor feedback issue.
I’m no sound expert, so I might not express myself decently. If you want me to rephrase it I can try.
Either than that sound really good.
Cheerz,
— Rui —
Correcting an issue with my last post:
“and the same sound but looking more amplified” = “and does the same sound you recorded but looking more amplified”
ac3 works, and it sounds pretty spooky. And those jack in/out sound sounds realy like keyboard typing.
I think that the quantity of the jack sound could be increased dramatically, if it won’t crowd the whole mix too much (maybe make some of them on the “border of hearing”), and the telephone voices could get more wacky than the last (a bit).
Now it sounds like a small room, and it should sound like an endless machine :P
Your rig is stupefying, very clever, I look forward to hear more, and read more about Your work here.
Yeonil
Very cool idea. I think its great.
In my minds eye though I sort of picture the following (might not be appliciable to the scene since I don’t know it)
I picture moving through this scene and I feel like The voices should be more closely tied to a loud click. And that they should fade away as I pass them by… etc…
It would be way cool if you first played a Jacksound Then that triggered a random section of a voice.
Listening to it I feel that connection of a distinct Conversation associated with a plug is missing.
(Not trying to complain, just giving feedback) :)
Personally, I think thats sound really great, we should not forget that some parts still to be defined, so flexibity is important in that fase; for the ‘jacks’ sound, it seems really ok to me, I think that what is confusing people is that you eard a single sample already with reveb, and it may sounds strange but it should mix great in between the general reveb that the scene will have.
It will be hard to get every sound in place in a short that will be all the time, surrounded by mechanical noises. I’m also curious about if the clip will get some parts with music; I personally believe that if the kind of sounds we ear in this page are important to recreate the ambient of the scene, some mix of music can undoubtly transport the result to greater level
It’s really nice to follow this process so close. And to see professionals at work :)
But I was wondering if there aren’t any good Open Source alternatives to Reaktor that can be used to get the same results?
Because I thought this Orange project was also about trying to use as much Open Source software as possible?
regards!
oslo
Not knowing anything whatsoever about sound design (and very little about Blender), I have to ask…
Is it possible to write a blender script that will send out the position (relative to the camera) of the aurally significant objects, then use that actual location data to drive the sound design? From the Blender Conference presentations I’ve seen on the web, it seems like that may be possible.
By tying a random piece of conversation to each (aurally relevant) wire through script, would you be able to increase the realism while allowing flexibility for the animators?
It looks very similar to buzz if anyone is looking for another app to use:
http://www.buzzmachines.com/
I’d say use what you use though. Hard enough to get things done
without learning new software.
Very cool! I would really like to hear/learn more about this kind of stuff!
I think the suggestions made by Knowsnuttin and sirdude are great. The Jack In\out sound does sound like a keyboard, but I think that if you synchronize it with the plugs, it’ll connect pretty naturally in peoples minds as they watch it.
There’s a big difference between hearing a sound just by itself, and seeing an object do something with a sound playing at the same time.
Reaktor looks like an incredibly usefull tool, I can’t wait until the Open-source Audio/Music scene is this advanced! (3D is getting there… ;)
yeonil: As for adding stuff, keep in mind that in the final mix, there’ll be dialogue, foley and music elements in addition to the SFX. Sounds mixed in very quietly at this stage, especially if they’re not in a frequency range that easily stands out from the rest, are likely to be drown out completely in the final mix.
oslo: I think the OSS application that comes closest to Reaktor in terms of functionality would be csound, although it’s a different kind of beast – csound is a programming language optimized for signal processing, while Reaktor is more akin to a modular synthesizer (although it’s far more powerful than classic modular synths).
knowsnuttin: This would be possible in theory, but I suspect it’d take about several times longer than just adding sounds for the objects in the foreground manually in postpro. And it probably wouldn’t even replace doing it, as the fact that it’s physically correct and in sync doesn’t neccessarily mean that it also sounds good ;)
Addressing a couple ealier comments:
Various: About the jack noises – being extremely familiar with audio equipment and whatnot, I thought they were jacks and plugs immediately, but I can see how people unfamiliar with the noise could mistake it for typing.
Yeonil: As for the “small room” comparison – I agree to some extent, but I would say large room. It sounds like the switchboard floor of a phone company, which is sizeable.
Jan’s followup: I could respond better if I knew what feel the scene had – it currently feels like someone is “sneaking” through the wires and machinery, music and dialogue and other fx aside. If that’s the goal, bravo! If it’s supposed to feel bigger, perhaps more voice layers and greater dynamic range could help.
knowsnuttin: this would be a phenomenal plugin if a good amount of effort, testing, and refinement went into it. I agree with Jan – often what is real is not aesthetically pleasing.
My own additions:
– Currently it is just mummery and conversation. I think to add to a more “crowded” feel (again, if this is the goal), occasional laughter/crying/raised voices might help (basically what I meant earlier by increasing the dynamic range). I know more will be added in the final mix, and the article mentioned that some stuff might be added from the actors, and I think this would be a great help.
– I can’t say much more until I see the visual that goes with the sound. My initial reaction was that the steam pressure release noises didn’t exactly fit with a phone wire visual, but I guess with the endless machinery concept it will be much more than just phone lines.
-Fx
Just a clarification on my eariler comment. I did not say
I thought the jacks needed to be synced with the image, (would be nice but probably not needed)
What I did say though was that the jacknoise should be before a
random audio clip is played. (and or before a given audo track is raised in volume)
I’m not a pro at the audio software but if I were doing it in buzz, what I’d do is setup a small “machine”
that first played the jack noise and then did all the complicated stuff to play a snippet of audio and then another jack noise at the end.
Then duplicate this machine a bunch of times and tweak the input files and some variables to get random segments.
(I haven’t done it though so could potentially be a lot of work and not needed)
Reaktor looks alot like a freeware program called Psycle (http://psycle.pastnotecut.org/songbase.php)
As for the plug/jack sound I think it sounds fine. It has that distinctive double click and a broader frequency range then keyclick. I think that if you add some louder ones that correspond with visuals of nearby jacks it should be very effective.
20to30: useless you say … what did you want to see mentioned here? I think you might be on the wrong Blog to say that ….
I find this post natural, it shows how part of the sound is being done for the Animation, thus enlightening anyone reading this blog with some of the sound work and techniques being used.
I think this is what is expected from this Blog, a blog focusing on how things are being done.
You didn’t expect a E-Learn Blog where someone would teach you how to use Blender from day one, did you ?
OFF-TOPIC now:
A new contestant to GiMP is rising, it is not Open Source which kills its direct hit on the GiMP, but it works under Linux and is looking really good.
Check Pixel at http://www.kanzelsberger.com/pixel/ , looking really good and might become the real contestant of Photoshop for Linux (and other platforms).
I will keep an eye on it. Although currently is still buggy.
Cheerz,
— Rui —
Agree, ‘Jack in (and out of) the box’ need more dirty sound.
I think the jacks sounds as much as typing as it sounds like my dog eating. Maybe somebody should clean his keyboard? (yuck).
Thanks for your comments, all! Though I’m really not sure what 20to30 is on about there…
By the way guys, I wouldn’t be so quick to pass judgement on what the jack sounds should be like without seeing the animation. Though we’ve been posting images from the teaser to give an idea of this scene scene, the way the final thing is unfolding is different. We’ve been working together with Jan on this and it’s turning out very nicely for what we need. This is meant to be a background atmo track with feelings of paranoia, it’s not supposed to be actually corresponding to on-screen action.
Cheers,
Matt
Hey Jan congratulations!. Really like the sound efects between 42s and 46s of “soundscape_final” Sounds like a “train station walk/talk” noise. ;)
The jack sound is very good, but seems a bit in the High Frequency range to be impactful IMHO. Maybe boost the lows or add another sound right under the jack click. Where the jack would be a “THINK” sound maybe add a “THUNK” sound where the end of the jack is going into the hole and pushing out the air. Also maybe a few random buzz noises that occur when a jack is plugged into an electric guitar with the amp turned up. All mixed very well that might make for an even better jack sound.
JABayne
Is Ton mixed in there somewhere? (No pun intended. ;) ) I could swear that I heard him. :D
Maybe I missed it, how are you planning on mixing the audio with the rendered video? Will you use a outside composting program like Adobe Premiere, or does/will blender support mixing audio into rendered AVI files?
Thanks, great stuff on this blog.
Knowing Jan recorded REAL JACKS being plugged and unplugged, it’s funny to hear so many argue that the jacks don’t sound realistic! But they are easily recognized by those who know the sound (e.g. FxMan51). Anyway, these jacks are not of this world, so they don’t have to be “realistic”. :-D
I think the point is: it’s not as simple as it seems. Look at all the work that went into only about a minute of audio.
Great work, Jan, and thanks for insight into your creative process! Also, I think the music for the teaser is beautiful. Another reason I can’t wait to see “Elephant’s Dream”!
Hehe indeed awesome sounds effects. I’ve got reaktor as well, it came free with some hardware of my pc but still I couldn’t find how to use it properly or at least make something usefull :) But playing with those premade effects it sounds indeed like factories, strange music and such.
Anyways, keep going, I’m soo exited to see the finished movie. :D
Hey, thanks again for the responses.
DwarvenFury: Good hearing! :)
chrishillman: Perhaps one of our video whizzes can chime in on this; but as we’ll need to render the material to multiple output formats suitable for different distribution media (DVD, HD-DVD and web distribution, 35mm prints, etc.), I suspect we’d start from the lowest denominators (raw frames(?) and uncompressed 48kHz, 24bit audio) and feed them into different encoders.
JD-Multi: Yes, some of Reaktor’s factory ensembles are pretty mind-boggling, although I’m having a hard time thinking of real-world applications – it’s been a long time since I last thought “gee, this song could really use the sounds of a cellular automaton playing ‘game of life'” ;)
I think what chrishillman was asking is what program are you guys using?
Virtualdub? (j/k) (For future reference, do not use that program for genereal-purpose Video Editing under any circumstances. It claims to be a video-processing utility, not an editing program and lives up very well to the processing part.)
I made a new story!
http://www.ninjabuddy.org/Storys/Shadow%20of%20Dreams.htm