What is Audio Post Production?
Bruce C. Nazarian, MPSE, Revised by Scott G.G. Haller, MPSE
Audio Post Production is the process of creating the soundtrack for moving
images. Ever since the once silent movies developed a prerecorded track,
filmmakers have been looking to control and improve the quality of the sound
of their visions. As soon as moviemakers realized there was a way to control
and enhance the sound of their pictures, Audio Post was born and has been
a fact of life ever since. In television, audio was originally "live," like
the visual program it was part of. As TV evolved and grew to include "videotaped" and "filmed" programming,
its need for audio post increased. Nowadays, it would be difficult to find
any feature film or television show (or video game) that hasn't been through
What is involved in Audio Post?
Audio post usually consists of several processes. Each different project may need some or all of these processes in order to be complete. The processes are:
- Production Dialogue Editing
- ADR (Automated Dialogue Replacement or Looping)
- Sound Effects Design and Editing
- Foley Mixing and Editing
- Music Composition and Editing
- Mixing (also called Re-Recording)
What does all that mean in English ?
It's really pretty simple, once you know the breakdown:
- Production Dialogue Editing - In order for the production audio recorded on the set or on location to be properly mixed, a Dialogue Editor needs to prepare it. This means locating the takes used by the Picture Editor from the recorded production audio, checking sync (so the audio works with the picture properly), and eliminating extraneous noise so the Dialogue Mixer has clean dialogue to use during the mix.
- ADR [Automated Dialogue Replacement] - In cases where the production audio is too noisy or otherwise unusable (bad line reading, airplane fly-by, etc.), or where the filmmakers want to add voice over narration or simply add dialogue that was never recorded, the line will be programmed or “cued” for “looping” or ADR. This process takes place on the ADR Stage, a specialized recording studio where the actor can record while watching the edited picture, matching the sync of the original line or fitting the new lines with the actions.
- After a loop lines have been recorded, the ADR Editor will check the sync carefully, modifying the take if necessary to precisely match it to the picture, and prepare it for the Mixing Stage.
- Sound Effects Design and Editing - Ever wonder how they made the sound of Darth Vader's helmet breath, or the roar of Jurassic dinosaurs, or those great explosions that seem to get bigger every year? Sound Effects Editors and Sound Designers are how. They are the craftspeople who add the computer beeps, gunshots, laser blasts, massive explosions; and more subtle sounds like background ambiences such as air, rivers, birds, and city traffic. Sound Designers use a variety of technologies from bleeding edge to tried & true to create unique sound effects that have never been heard before, or to artistically create specific "mood" sounds to complement the filmmakers’ vision of the visuals. Sound Effects Editors put those sounds in sync with the picture as well as selecting from libraries of hundreds of thousands of prerecorded sounds; and organize them so the FX Mixers can “PreDubb” those sounds efficiently.
- Foley - Taking its name from Jack Foley, the Hollywood sound editor regarded as the "father" of these effects, Foley effects are sounds that are created by recording (usually) everyday movement while watching the edited picture. Different from the environmental backgrounds (“BGs”) and hard effects (FX), Foley effects are sounds like footsteps, object handling, the rustling of clothing, etc. The people involved in this process are the Foley Walkers or Artists who perform those sounds and the Foley Mixer who records them. After the Foley Effects are “shot,” the Foley Editor will use his/her craft to polish those sounds to ensure that they are exactly in sync with the final picture.
- Music Composition - Music for motion pictures falls into two general categories: Score and Source. The Composer is the individual hired to prepare the dramatic underscore. Source music is what we hear coming from an on screen or off screen device like stereos, televisions, ice cream trucks, and so on. Source music may be original or licensed from a number of libraries that specialize in the creation of "generic" music. Songs (music with vocals) may occupy either function, depending on the dramatic intent of the director. For "Pulp Fiction" for example, Director Quentin Tarantino hired a Music Supervisor (Karyn Rachtman) to "score" the picture using period music of the 1970's almost exclusively. Most contemporary films use a combination of score and source music.
- Music Editing - The Music Editor assists the Composer in the preparation of the dramatic underscore. Frequently working also with the Music Supervisor, the Music Editor will take timings for the Composer during a spotting session in order to notate the specific locations in the film where underscore or source music will punctuate the narrative. Once the underscore is recorded and the source music gathered, the Music Editor would be the person who edits or supervises the final synchronization of all music elements prior to the mix.
- Mixing (also called Dubbing) - The Mixers have the responsibility of balancing the various elements, i.e., the Dialogue & ADR, Music, Sound Effects, and Foley Effects, in the final mix. The Dialogue Mixer, (also called the Lead Mixer or Gaffing Mixer) commands the mixing stage; his/her partners in the mix traditionally were the Effects Mixer and the Music Mixer. As of now, the Lead Mixer commonly does the Music mixing as well, reducing the traditional mixing team by a third. On huge pictures with tight deadlines, it is possible that several teams of mixers are working simultaneously on numerous stages in order to complete the mix by the release date.
Although motion pictures had been edited and mixed using 35mm magnetic film for many decades, the 90s saw the development and acceptance of digital sound editing and recording systems. Digital technology became the standard because of the efficiency and quality it can bring to the editorial and mixing process.
Where does post-production begin?
If you haven't shot your film yet, it begins before you shoot - by selecting the finest production dialogue mixer you can afford. The little bit extra paid to a great production mixer can save you tenfold later in post production.
What does the production sound mixer do?
The production mix team are the individuals charged with recording your live dialogue in sync with the camera team. The Production Sound Mixer is your most important ally at this stage in the movie's production. Although you will be anxious to complete as many setups as possible during each shooting day, a little extra time given to the sound mixer to allow him or her to capture location ambience (called Room Tone) will pay off handsome dividends later during the dialogue editing. The production mixer will have a Boom Operator, who handles the boom mikes; and usually a cable person, who will be in charge of wrangling the audio cables needed to mike the set appropriately. For years they recorded with a Nagra reel to reel tape recorder, but digital recordings with an ever increasing variety of formats is currently the standard.
We are shooting our film on location... what now?
For decades, each day after the completion of the shoot, the production sound reels were sent to an audio post house for transfer into "dailies." If the film was being edited film-style, using 35mm mag audio and film dupes (as opposed to digitally, using an Avid or Final Cut or other computer editing system), the production’s chosen takes will be transferred to 35mm mag film. This sprocket-based medium allowed the film editor or assistant to sync that day's select film takes with the audio track that corresponded to it and run the sound and picture together “double system” on a flatbed or Moviola as they cut the picture together with splicers, tape, and glue.
Now productions are largely being edited digitally using computer-based editing systems, and the procedures are a bit different. Frequently, a video post house will be engaged during shooting to Telecine the selected film takes. (At least, if the movie is even being shot on film.) In addition, they will transfer the production audio from DAT or some other format and generally synchronize the dailies onto some form of videotape for later digitizing into the picture editing system. Synching dailies at the video house eliminates the need for the assistant film editor to do it, and allows the assistant to load the editing system instead. An important task to accomplish during the digitizing is for the assistant to correctly log in the dailies time code that is recorded on the production source recordings. This will allow the EDL (Edit Decision List) that is created later on to accurately reflect the original time code that was shot with that scene, and allow the audio post house to electronically automate the re-loading of the production dailies, should they need to be replaced.
And since the only thing constant is change, the current digital evolution is to load the production audio directly into the picture editing system, increasingly simply copying them in as computer files of off drives or DVDs provided by the production mixer, instead of having to do a real-time recording, and have the picture assistants once again synchronize the dailies themselves. The advantage of having the production sound remain in the digital realm during the picture editing is that the files can simply be transferred (again as computer files) into the sound editing systems and the dialogue cutting can begin right away without having to retransfer and re-synch everything.
And this goes on all during the filming?
Yes. Dailies transfers will continue until there are no more dailies coming in, and shooting has wrapped.
We are done shooting... now what happens?
Now the real fun begins. The editor has been synching dailies all during shooting, choosing which scenes should begin to form the final cut. During the next several weeks, the process of editing will continue as the decisions are narrowed down to final choices. It is at this time that the final form of the film begins to take shape. Although the film editor may have been assembling the "editor's cut" during the shooting period, the first formal edit period is generally referred to as the director's cut, and it is when the first full assembly of the film is refined.
Do I need Audio Post during editing?
Well, yes. During the editing you may still need reprints of selected takes or outtakes. The audio post facility will duplicate these for you. But the real job is starting to come into view: the locked cut.
What is the locked cut?
In short, the final version of the finished film. Although it may receive a small edit here or there in the next few weeks, the film is essentially "locked" into this form.
What happens once the cut is locked?
Audio Post begins now in earnest. Once the cut has been locked, the film can be spotted for the placement of sound effects and music. The Supervising Sound Editor, the Director and possibly the Film Editor and Composer will gather at one or more spotting sessions to determine the film's audio post needs. "Spotting for music" is the process of viewing the locked cut and deciding where the music score will be, and where the source music will be needed. "Spotting for sound" is the process of determining:
- if and where any dialogue problems may exist, so that ADR can be cued to be recorded
- where sound effects are needed and what kind
- what Foley effects will be needed in the film, and where
- If Sound design (the creation of special sound effects), will also be needed.
What actually happens after 'spotting'?
The real job of audio post has now begun. In the next weeks or months, the sound editors will locate and synchronize all of the sound effects needed in the film. If necessary, they will create Field Recordings of new sound effects needed for the film. The Foley supervisor will cue all of the Foley effects that will be needed; they will be recorded by the Foley Mixer and the Foley Walkers; the ADR supervisor will cue all of the Automated Dialogue Replacement lines that need to be recorded during the ADR sessions, and the Music Editor will begin providing for the needs of the Composer and/or music supervisor. The Dialogue editor(s) will begin preparing the production audio for final mixing, and the ADR editors can commence editing in the ADR lines, once they have been recorded.
Typically, the next few weeks or months are occupied with sound editing of all types. The Director will be checking on the various aspects of the sound job as time progresses, to be sure that his vision is being realized. Usually, there is provision for one or more "effects reviews" where the effects are listen to and approved. The same goes for Foley, Dialogue, ADR, Sound Design and Music. When everything is completed and approved, the next step is Mixing (also called 'dubbing' or 're-recording').
What happens during the mix?
During the mix, the edited production dialogue and ADR, sound effects, Foley and Musical elements that will comprise the soundtrack are assembled in their edited form, and balanced by a number of mixers to become the final soundtrack. In New York, single-mixer sessions are more commonplace than in Hollywood, where two-mixer and three-mixer teams are the norm.
The mixers traditionally divide the chores between themselves: the Lead Mixer usually handles dialogue and ADR, and may also handle music in a two-person team. In that case, the Effects mixer will handle sound effects and Foley. In three-person teams, they usually split Dialogue, Effects and Music; sometimes the music mixer handles Foley, sometimes the effects mixer covers it.
To keep the mix from becoming overwhelming, each mixer is actually creating a small set of individual sub-mixes, called STEMS. These mix stems (dialogue, effects, Foley, music, adds, extras, etc.) are easier to manipulate and update during the mix.
When mixing is done, what then?
After the mix is completed and approved, films generally require a last step called Printmastering, that combines the various stems into a final composite soundtrack. When this is completed, an optical or digital sound track can be created for a feature film release print.
It is also usual at this time to run an 'M & E' (which stands for Music and Effects) track. This is essentially the film's soundtrack with the English language dialogue removed. This allows foreign language versions of the project to be dubbed easily, while preserving the original music, sound effects and Foley. During the M & E, effects or Foley that are married to the production dialogue tracks are removed along with the dialogue. To "fully-fill" an M & E for a quality foreign release, those effects and Foley must be replaced.
Television movies usually do not require print masters, unless they have been created using SURROUND SOUND techniques. In most cases, the final stems are combined during a process called LAYBACK, at which time the soundtrack is united with a final edited master videotape for ultimate delivery.
What about optical soundtracks?
Optical soundtracks (we mentioned them earlier). Almost all of the release formats, including the digital ones have provision for some kind of optical soundtrack, even if only as a backup. The optical soundtrack refers to the two-channel soundtrack that is carried on the optical track of the film release print.
How do I get an optical soundtrack?
Once your surround sound format has been selected (see the paragraph below for more), you need to order an optical soundtrack negative for the film. In the case of LCRS mixes, a traditional two-channel; Printmaster track is created, and this is sent to an optical sound house for the creation of the optical negative. The optical sound house will record the soundtrack onto 35mm film using a special camera, and some will also develop their own soundtrack masters. Once the optical negative is shot and developed, it can be incorporated into your answer printing process, and a composite answer print containing your complete soundtrack can be printed or "shot" at your film lab. This usually happens during the first or second trial answer print phase.
What about: THX - Dolby - Ultra*Stereo - DTS - SDDS?
This is a BIG question. This one point alone causes much confusion amongst filmmakers. Please take a moment and read this paragraph carefully. If you need more information after that, please contact either Gnome Productions or Magnolia Studios and we will help you out.
First, about THX.
THX [tm] is not something that you DO to your soundtrack, it is just a set of sound reproduction or mixing conditions that optimize the sound of your film's soundtrack in exhibition. Simply put, the THX standards that many dubbing stages and movie theaters adhere to are a way of being certain that "what you mix is what you get", so to speak. You may choose to mix in a stage that is THX certified, and you may not. If you do, your soundtrack should sound reasonably the same in THX theaters all around the world. It is this standardization that THX brings to the filmmaking community.
You may want to visit the THX Web Site for further information. They can be found at http://www.thx.com/thx/thxmain.html.
To make sense out of the rest of the names, we need to know about Film (and Television) Surround Sound Film sound tracks (and some television ones) go beyond just Left-Right Stereo; there is a Center Channel for the dialogue, and at least one "Surround Sound" channel. The Surround channel is used to project the sound out into the theater, to "surround" the audience. This is to enhance the illusion of being "in the picture". This four-channel format is called LCRS (for the Left, Center, Right and Surround channels that the soundtrack contains). Although the technical means behind this process is beyond the scope of this discussion, suffice it to say that it works well enough to have become a standard format for release prints for many years.
You've probably already figured out that you cannot reproduce a four-channel soundtrack from a medium that only plays back two tracks. You are very right. In order to reproduce the LCRS soundtrack from a traditional film optical soundtrack (more on opticals later) you need a way to encode the channels.... the Matrix
The Surround Sound Matrix Encoder (or, how to put FOUR into the space where TWO should go!)
The solution is to use an encoding device that can fold the four channels of audio down into the two channels available on the film's optical soundtrack. When the audio tracks have been processed this way, they are labeled Lt/Rt [Left Total/Right Total] in order to distinguish them from ordinary Left/Right Stereo soundtracks. The Surround Sound Matrix Encoder is a necessary piece of hardware that the audio post house must have available during your film's mix, in order to create the surround soundtrack.
The Licensing of Surround Sound formats
Now we're really getting into the heart of the matter. Dolby Labs, Ultra*Stereo Labs, DTS (Digital Theater Systems) and Sony [SDDS] all have technologies available for the encoding of film surround soundtracks into film release prints. Although these processes vary somewhat as to their method, they essentially accomplish similar things. Additionally, some of these vendors offer Digital Encoding formats (Dolby Digital, DTS and SDDS currently, and Ultra*Stereo soon to come).
The Differences in Surround Sound formats
In the most basic form, Theatrical Surround Sound consists of LCRS: Left, Center, Right, and mono Surround. A soundtrack can be encoded into this format by using a Dolby or Ultra*Stereo encoding matrix during the film's Printmastering session. DTS also has a process called DTS Stereo that can create a typical LCRS film soundtrack (check with DTS directly for more on their specific processes...).
Surround Sound formats beyond L-C-R-S
Some of the surround sound encoding processes can create different, more complex soundtrack formats; Dolby SR/D and DTS, for example, can create six-track soundtracks for release, and Sony's SDDS is an eight-track format. In the case of six tracks, you get Left, Center, Right, Left Surround, Right Surround and a Sub-woofer channel (for enhanced low-frequency response). The split surrounds (as they are called) make it possible to move sounds around in the surround speakers, or to use stereo background sounds for even more impressive film soundtracks (Jurassic Park comes to mind, here). And if you heard Jurassic Park in a good THX theater with a DTS Digital soundtrack, you know what the sub-woofers are there for! That T-Rex really gave the sub woofers a run for their money, as well as Jeff Goldblum... Six-track sound reproduction has been with us for a while, since 70mm film releases have had the ability to deliver a six-track soundtrack that was magnetically encoded on the release print. This, unfortunately, was very expensive to produce, and problematic to control quality.
Sony's SDDS (Sony Dynamic Digital Sound) uses an eight-track delivery configuration that adds two speakers in between the Left/Center and Center/Right positions in the front speaker wall. Known variously as InterLeft, InterRight or LeftCenter and RightCenter, these channels allow for additional separation of music, effects and dialogue in the front speaker wall, while preserving the split surround format.
The Differences in Digital Sound delivery methods
The three digital systems (Dolby, DTS and SDDS) use proprietary methods to deliver the digital audio to the theater; two of these methods (Dolby, SDDS) encode the digital soundtrack onto the release print. DTS uses a different method, that of encoding a "timing stripe" onto the release print, and synchronizing a digital audio playback from an accompanying CD-ROM that carries the encoded soundtrack. In either case, the digital audio is reproduced in the theater with the same fidelity it was recorded at during the encoding process. This system neatly bypasses the traditional limitations of optical soundtracks: noise, bandwidth limitations, and headroom (transient peak) limits. Soundtracks sound cleaner, clearer and louder as a result. Please don't take this as a condemnation of optical soundtracks. A well mixed movie can (and they still do) sound great with a well produced optical soundtrack.
To summarize this difficult topic:
- THX specifies a set of standards
that affect how sound is recorded and reproduced in a movie theater.
You get the benefits of the THX standard whenever you mix in a THX certified mixing stage.
There is NO additional fee required.
You may display the THX logo in your film's credits if you sign a simple one-page form.
- Dolby Surround is a 4-channel optical surround format;
this format is encoded in the optical soundtrack
You must license this format from Dolby Labs; There IS a license fee for this service
- Ultra*Stereo is a 4-channel optical surround format;
this format is encoded in the optical soundtrack
You must license this format from Ultra*Stereo Labs; There IS a license fee for this service
- DTS is a 6-channel digitally-encoded surround
format; this format is encoded on an external CD-ROM, but the timing and
other information in encoded on the film release print;
You must license this format from Digital Theater Systems (DTS); There IS a license fee for this service
- Dolby Digital is a 6-channel digitally-encoded
surround format; it is encoded on the film release print;
You must license this format from Dolby Labs; There IS a license fee for this service
- SDDS is an 8-channel digitally-encoded surround format; it is encoded on the film release print;
You must license this format from Sony Corporation - SDDS division; There IS a license fee for this service
I have got a video project - What's this DVD, AC-3?
Relax - take a breath and we'll walk you through this... It's actually pretty simple;
Surround sound program on video materials are now released in a number of analog AND digital forms...
- Straight Left-Right Stereo program is still utilized a lot for Television, and Industrial formats...
- VHS Home video releases can be encoded in Dolby Surround (L,C,R,S), just like feature films;
- Laserdisc releases have also been using digitally encoded L,C,R,S surround formats, just like VHS
- NEW DIGITAL VIDEO RELEASE FORMATS have allowed for new DIGITAL SOUND FORMATS
- AC-3 - is a digitally-encoded surround sound format that is capable of reproducing six tracks of sound
- AC-3 actually refers to Dolby's Audio Compression 3 format used to compress the data
- DVD releases are also utilizing AC-3 digital sound format as well as traditional Surround Sound
All of these formats can easily be handled or prepared by a knowledgeable sound house. Please contact us if you have specific questions that you would like answered... no obligation, of course...
My mix sounded great on the mixing stage - but my print isn't in sync!
Well, we didn't say this would be EASY, just that we could help take some of the mystery out of it for you... You should IMMEDIATELY contact your post sound house and tell them what you've experienced. The Sound Supervisor on your show should be willing to take some time and help you sort this out. In the meantime, here's a few things that you can check on:
Some likely possibilities:
(1) If the Final Mix Printmaster has been transferred or copied, be sure the copy was done correctly. We have had experiences where a perfectly fine Printmaster was thrown out of sync because a copy was made first, and the optical shot from the copy;
(2) If the soundtrack DRIFTS from being in sync to gradually being more and more OUT of sync during the reel, suspect this possibility: If the Printmaster is on Multitrack tape, the SMPTE code on the tape could cause the optical soundtrack to drift in speed; If you mixed to VIDEO TAPE, a slight difference between 29.97 frame code and 30.00 frame code could throw you out of sync by many frames over 1000 film feet. If the soundtrack was shot on 35mm mag, a mistake in running the film chain at video speed could cause the mag film to be "offspeed", just like the Multitrack tape example above;
(3) If the Mag Printmaster was in sync when you reviewed the final mix, check to be sure the film lab didn't accidentally "misprint" the soundtrack by moving the optical negative a perf or two, or a frame or two when they married it to the picture. This can easily happen IF THE HEAD POP or TAIL POP is not EXACTLY CORRECT on your final Printmaster.
(4) If you printmastered in 2000-foot film reels, and FOR ANY REASON these reels were then separated and rejoined later, this poses a prime opportunity for sync to slip. If the beginning of a 2000 foot reel is in sync, and the last 1000 feet is suddenly (and consistently) out of sync until the end of the reel, suspect this phenomenon immediately.
(5) If one or two shots suddenly are out of sync but were IN sync when you mixed, ask yourself this: did you mix from an Avid or Lightworks (or other electronic edit system) output? If so, it's possible the film negative was not cut to the exact same shot length as the electronic output; Have you verified the length of all optical effects? If you have inserted optical effects, they may not have been counted exactly right, and you may have gained (or lost) a perf or frame or two in the effect; either way, your soundtrack will lose sync right then and there, and STAY out of sync for the rest of the reel (unless another optical effect error magically puts it back in sync again!)
(6) Finally, when all else fails, it is remotely possible that the optical negative might be offspeed. A quick call to the optical sound house will help them verify this for you.
My foreign distributor says I need an "Emenee" to make a sale?
Actually, it's an "M and E" or "M&E". This element comprises the "MUSIC and EFFECTS" elements of your original soundtrack, with ALL of the English language dialogue and Walla removed to allow for foreign language dubbing. In most contemporary post sound packages, an "M&E" is allowed for in the original bid. This process requires preparation during the original sound editing, as well as some additional Foley coverage that might NOT be needed for a straight domestic release. If you NEED an M&E, be sure that you tell your post sound house that UP FRONT. It WILL add some dollars to your post bid, but you WILL want it, if you are to have any possibility of a foreign release or sale at all. Preparing this element NOW will buy you plenty of "peace of mind" later on. The M&E can be on Mag, on DA-88, on DAT, or on almost any format that can be synchronized. It DOES NOT need to be converted to an Optical soundtrack form at this time... only later, when a new foreign Printmaster is created after the foreign language has been added to it.
Do I need to know about the academy rolloff ?
Well, although it is a holdover from film sound's infancy, we need to be aware of it, since it does have some relevance in certain circumstances. The academy rolloff is a specific frequency response curve that is used in dubbing stages to simulate the effect that the old-time optical soundtrack would have on the frequency of the final soundtrack. With advances in technology in today's film industry, its use is diminishing, although it has been used on mono theatrical trailers to this day.
How do I get more info about Surround Sound Licensing?
It would be best to consult the various vendors themselves...
THX can be reached at http://www.thx.com/thx/thxmain.html, or in San Rafael, CA. 415-662-1800
Dolby Labs can be reached at http://www.dolby.com, or locally in L.A 213-845-1880
Digital Theater Systems can be reached at http://www.dtstech.com, or locally in L.A. 818-706-3525
Sony Corporation maintains a web page at http://www.sony.com
Ultra*Stereo Labs can be reached directly by telephone at: 818-609-7405