Video production requires a lot of storage space. It is not unusual to finish a days shoot with over 1TB of footage. With every increase in camera resolution, comes an increased requirement for data storage. When you’re possibly spending thousands of dollars to ensure a shoot day goes smoothly, you don’t want anything to happen that could undo that investment after the fact. In this post I’ll break down some best practices to manage your data throughout the whole production process.
IN THE FIELD
The biggest threat to data integrity during a shoot is human error. Poor communication and fatigue are both factors that can contribute to human errors and data loss. While these factors can be hard to control on a long, physically challenging shoot day, there are certain protocols you can follow.
- Always format your cameras media (SD cards, CF cards, etc) prior to shooting anything. This essentially gives you a clean slate to record to, reducing the chance of read/write errors.
- Never delete clips in-camera. Deleting footage can increase the risk of card read errors or corrupted data.
- If your camera allows it, record to two media cards simultaneously.
- If any card gives you an error, immediately remove it from the camera and set it aside. Continued use of a rogue card could exacerbate the problem, or prevent data recovery attempts later on.
- When a card is full, remove it from the camera and wrap or mark it with red camera tape or gaffers tape. This will notify everyone to NOT erase anything on that card.
- Shoot on multiple smaller cards, rather than a single large one. In the event of an error, this will mean that the problem only affects a part of the days footage.
- Copy and backup your footage as soon as possible. It is best to have a laptop on location solely for this purpose.
Transfers and Backups
Footage should be transferred via a computer and backed up as soon as is practically possible. This might mean stopping shooting at regular intervals to allow for data transfers. You might also want to delegate this role to one person as their sole responsibility. Always copy the footage to at least two hard drives. LaCie “Rugged” hard drives are a popular choice for this role. As their name suggest, they are built to withstand rough handling on set or while traveling. You can also consider using a specialty data transfer program such as ShotPut Pro or Hedge. These programs will verify your backup data to make sure it matches the original files – reducing the chance of unnoticed copying errors. If you have time, copy footage to hard drives sequentially, rather than simultaneously. This takes longer but prevents the possibility of a single read error being duplicated on both copies.
Even after copying and backing up footage, original media cards should be left as long as possible before they are formatted again. This allows extra time to check the footage for any read/write errors corruptions. Once you are certain it is safe to erase a media card, you can wrap it or mark it with green tape. On multi-day shoots this is particularly important as the cards will certainly need to be re-used. It is essential that everybody knows what data is backed up and safe to erase.
IN TRANSIT
If you’ve done everything right on location, you should end up with at least two hard drives containing duplicated files. However one of the most high risk periods for losing footage is when traveling from location back to the studio, particularly if it involves air travel. The most important protocol here is to separate the drives as much as possible while traveling. This means that any unfortunate events, such as an airline losing your bag, will only affect one copy. Here are some tips to reduce the likelihood of losing multiple hard drives in transit.
- Put separate people in charge or separate hard drives.
- Where possible, keep hard drives in separate vehicles.
- Do not leave hard drives unattended in vehicles. They can be highly vulnerable to theft or overheating.
- Place one drive in carry-on baggage and the other in check-in baggage.
- If you need to post hard drives, take extra precautions. Make additional back-ups and only have one copy in postal-transit at any given time. Only send the next copy after you’re sure the previous one has arrived safely.
“The most important protocol here is to separate the drives as much as possible while traveling”
In the Studio
You’d be forgiven for letting out a sigh of relief once you get all of your footage safely back to your office or studio. But the job is not over yet. Those hard drives you used in the field likely don’t have the speed required for prolonged video editing. You’re going to want to transfer all of your footage once again to a faster solution. This could mean external desktop drives, a RAID array, or a NAS Server. Either way, you’re going need two copies of the data locally. Additionally, you’ll want another copy of the footage which is stored off site. This is a good time to put those field drives to use – storing those in separate locations with all the original files will offer additional redundancy.
The overall idea of data storage is to always have two version of the footage available locally, and an extra one off-site. The local versions mean you have an immediate back-up if one of the drives fails, while the off-site version offers protection in the event of theft or fire which could render all of the local drives useless.
Cloud storage is of course an option for off-site storage. However, the fast speed requirements and high storage needs of video data mean you could spend days – or even weeks – getting everything backed up. If a hard drive does fail, the last thing you want to do is wait days for files to download so you can restore your data.
Redundancy is key
Data management is all about redundancy. By being prepared for the worst, you’ll be well equipped to deal with all sorts of unforeseen circumstances. Hopefully you never need to fall back on your back-up hard drives, though if you do, a proper data management system will make your problems easy to solve.
