User:Ulan/Offload process
Offload and Conversion Process
The offload and conversion process makes the liaison between the Filming Unit and the Editorial. The new files recorded on stage are delivered to be offloaded on the main storage drive, and then converted into a format compatible with the Editorial work flow.
Receiving the Delivery
After a few takes on stage, the Unit on stage will deliver the new recordings stored on Sony SxS Cards. They are placed into safe boxes following a colour code. Each drive that contains unique version of the files are placed into a Red Box, and each drive containing a copy of files already backed up are placed into a Yellow Box. Following those rules, the recorded takes delivered to offload are placed in the Red Box.
Preparing the Cards and the Workstation
To avoid the risk of loosing the original files, the SxS cards must be set on READ-ONLY mode. To do so, slide the black switch of each card in order to see a red dot on the orange surface.
Alongside, before any connection between the cards and the workstation, it is important that the workstation’s device list contains no errors. The application called CF Rescan must be ran to avoid any corrupt transfer.
Mounting the SxS Cards
The SxS cards can be plugged into the card reader. The label tape can be temporary removed if the card does not fit in the reader. Once inserted, CF Rescan must be ran again before to access the content of the cards.
In the following example, three SxS Cards are mounted as G:\ , H:\, J:\, and one SxS card is recognized as a Removable Device and mounted as I:\.
Understanding the folder semantic
The project's storage drive is organized in a precise way, following semantic rules.
The first rule is that each day of recording is in a folder name by its Date Code, stage and DayRoll, following the format :
YY = Year as two digits, MM = Month as two digits, DD = Day as two digits,
TX = Stage/Unit name as one letter and one digit, XX = Day roll up to three digits.
Example Situation :
Date Code: 151123 * Year = 15 = 2015 * Month= 11 = November * Day = 23 = 23rd Stage : B2 * Stage= B * Unit= C DayRoll : Day15
| Value | |
|---|---|
| YY | Year as two digits |
| MM | Month as two digits |
| DD | Day as two digits |
| TX | Satge/Unit as one letter, one digit |
| XX | D |
For a day of Video and MoCap capture, each of the Day folders should contain (at least):
* \JB\151123_B2_Day15\_ajaAlbert\ (Daily Backup of the Albert Drive) * \JB\151123_B2_Day15\_MXF\ (Original Clips) * \JB\151123_B2_Day15\_SxS\ (Converted Videos) * \_VideoContinuity\ (Metadata)
To keep the SxS and MXF video folder in order, every delivery is stored in a folder named by the current Camera Roll. It is a number that increase by one each delivery. Inside the Camera Roll folder, the clip a stored by their Camera : CAM_W, CAM_X ...
CR = Camera roll as incremented number, # = Camera name as one letter.
Following the same rules the .CSV metadata files are placed into a Camera Roll folder in \_videoDirectory\.
Copying SxS into the Main Storage
Following the directories semantic for this project, the content of the SxS cards must be copied into the folder CAM_# of the _MxF directory. For the unit B2 on Day35, the 145th camera roll on the camera X, the path is :
See the example below.
Checking video integrity
Finally, to detect any corrupt video clip in the new \_SxS\ folder, the software XDCAM Browser will inspect them and show a thumbnail if the video contains no error. To do so, open the tool XDCAM Browser located on the desktop, and use the top left explorer frame to navigate to the copied clips. Following the example of the last [#Copying SxS into the Main Storage|outline chapter], for the unit B2 on Day35, the 145th camera roll on the camera X, the path is:
Correcting Errors
Spot the mistakes
Most of the errors committed on set are written in the Camera Sheet. The most commons are listed below. You can access the Camera Sheet following the path:
XX = Day roll up to three digits.
Misslate
A misslated take happens when the name of the take on the .CSV file does not correspond to the name of the take – written on the slate. The easiest way to correct this mistake is to edit the .CSV file located in :
| Name | Value |
|---|---|
| YY | Year as two digits |
| MM | Month as two digits |
| DD | Day as two digits |
| TX | Satge/Unit as one letter, one digit |
| XX | Camera roll as an incremented number |
| mm | Minute as two digits (According to the GMT Time zone. ) |
| CR | Camera roll as an incremented number |
YY = Year as two digits, MM = Month as two digits, DD = Day as two digits,
TX = Stage/Unit name as one letter and one digit, XX = Day roll up to three digits,
CR = Camera roll as incremented number,
HH = Hour as two digits, mm = Minute as two digits. (According to the GMT Time zone. )As an example: The unit B2 on Day35, for the 145th camera roll the take 150410_73F_01 has been misslated as 150410_73E_02. The correction needs to be done in the .CSV related to this take:
The new file must be saved in the same folder with the extension “_misslate.csv” as the example above. Now the error is corrected, the new file will be used in the conversion in the next chapter.
Double-Roll
A Double-Roll designate two takes that should have been one. This happens when the camera is restarted during the filming. To correct this error, we must rename the takes as different parts P1, P2, etc.... However, the new named takes will need to be delivered beside in a Redelivery. We will see the full manipulation later on the [#Redelivery|outline Redelivery Chapter].
To separate the Double Rolled takes and make the first delivery cleaner, we can open the related .CSV file and move the double-rolled informations in a new document. Then the first conversion won't deal with the double-roll.
As an example: The unit B2 on Day35, for the 145th camera roll, The take 150410_73E_01 double rolled with the take 150410_73E_02. To avoid delivering the wrong files, we can isolate the takes from the .CSV :
To the new .CSV :
You can now process the normal delivery, and use the new CSV file when following the [#Redelivery|outline Redelivery Chapter]
Conversion process
There are currently 3 video convert tools which are SxS to MXF, ALE Creator, and Pico to MXF.
The SxS to MXF converter and the ALE Creator are for the SxS Cards. SxS to MXF creates the MXF, compatible with AVID. The ALE Creator help with populating meta data for those clips.
SxS to MXF Conversion tool
Above is an example of the tool that is used to create the MXFs. This will dispatch items across onto the farm - which is currently an Afanasy 1.7 farm.
Static Settings
Project: JB
The project is statically assigned as JB to this project.
User Settings
The rest are settings that you change from card offload to card offload.
- The Cam Roll, is an offload number that we increment by 1 each time we offload cards
(started at 1), and is at 118 in the example above.
- The Cameras points to the cards offload folder which is named as the reel for main unit. (118th Cam Roll pointing to 118th Cameras). It follow the path :
YY = Year as two digits, MM = Month as two digits, DD = Day as two digits, TX = Stage/Unit name as one letter and one digit, XX = Day roll up to three digits, CR = Camera roll as incremented number.
- The Continuity points to the CSV file that gets offloaded during the offload procedure and placed into the corresponding folder by production. Its file name is usually :
YY = Year as two digits, MM = Month as two digits, DD = Day as two digits, HH = Hour as two digits, mm = Minute as two digits. (According to the GMT Time zone. )
- The Output Directory is where the farmed files are processed to. (In the screenshot, we proceed the files in a redelivery of the Camera Roll 118, this is a particular situation defined in the [#Redelivery|outline Redelivery Chapter] below.)
- The GO! Button will start the process off.
ALE Creator
[[Image:]]
Above is an example of the tool that is used to create the ALE. This will dispatch items across onto the farm.
Static Settings
Project: JB
The project is statically assigned as JB to this project.
User Settings
The rest are settings that you change from card offload to card offload.
- The Reel Number, is an offload number that we increment by 1 each time we offload cards - started at 1, and is at 118 in the example above. We also call this number Camera Roll.
- The Cameras points to the cards offload folder which is named as the Reel Number for main unit. (118th Cam Roll pointing to 118th Cameras). It follow the path :
YY = Year as two digits, MM = Month as two digits, DD = Day as two digits, TX = Stage/Unit name as one letter and one digit, XX = Day roll up to three digits, CR = Camera roll as incremented number.
- The Continuity points to the CSV file that gets offloaded during the offload procedure and placed into the corresponding folder by production. Its file name is usually :
Y = Year as two digits, M = Month as two digits, D = Day as two digits, H = Hour as two digits, mm = Minute as two digits. (Done to the time zone of GMT. )
- The Output Directory is where the farmed files are processed to. (In the screenshot, we proceed the files in a redelivery of the Camera Roll 118, this is a particular situation defined in the [#Redelivery|outline Redelivery Chapter] below.)
YY = Year as two digits, MM = Month as two digits, DD = Day as two digits, TX = Stage/Unit name as one letter and one digit, XX = Day roll up to three digits, CR = Camera roll as incremented number.
- The Date, to change each morning.
- The GO! button will start the process off.
Pico to MXF Converter
[Screenshot]
Above is an example of the tool that is used to create the MXFs. This will dispatch items across onto the farm - which is currently an Afanasy 1.7 farm.
Static Settings
Project: JB
The project is statically assigned as JB to this project.
User Settings
Afanasy Farm Render
To accelerate the conversion, the three last script will process the files to a computers farm using the software Afanasy 1.7. This tool will dispatch the load between numerous computers and gather the results in the new converted files. It is important to keep an eye on the conversion process as some errors can happen.
In the Jobs tab, the tool lists all the tasks in process.
While waiting for the tasks to be processed, you can check the activity by clicking on “Listen” in the Right-Click menu (Expect for BMX tasks). If a task goes wrong, you can also delete it in the same menu. If all the tasks need to be cancelled, press Ctrl+A to select them all then Del to delete them.
Delivering the converted files
To officially announce the delivery, an email has to be sent to the Producer, the Editorial, the Production Coordinator and the IT. From your attributed email address address, use a template as following :
| To: | IT; Editorial |
| Cc: | Prod. Coordinator; Producer; |
| Subject : | JB| Reels ### |
| Message | Location: M:\ViconDB\WBS\JB\YYMMDD_TX_DayXX\_MXF\CR
Notes:
… (additional infos, correction and notes from the camera sheet) |
Redelivery
Correcting the errors
Depending on the feedback from the Editorial, there might be some undetected errors that are problematic for the edition. Every time a delivery contains a wrong information, the correction is applied through a redelivery. The main rule when fixing a problem is to let the delivered file untouched. All correction has to be made in a copied file, in a new folder. To find and correct a problem, you can use :
- The Camera Sheet ( “ T:\Prod\jb\camera_sheets\refCam_lead_summary\dayXX “ )
- The _videoContinuity folder ( \JB\YYMMDD_TX_DayXX\_videoContinuity\)
- The .ALE files of the failed delivery (\JB\YYMMDD_TX_DayXX\_MXF\CR\CAM_#)
- The tool XDCAM Browser to find and compare Time Codes (Always open the SxS files)
- The Chapter about commons errors [#Spot the mistakes|outline Page 5].
Packing the Redelivery
The files or folders concerned by the correction must be annotated with the “_Redelivery” extension. To redeliver MXF or ALE Files, they must follow the path (With the same CR as the 1st delivery):
YY = Year as two digits, MM = Month as two digits, DD = Day as two digits, TX = Stage/Unit name as one letter and one digit, XX = Day roll up to three digits, CR = Camera roll as incremented number.To redeliver a .CSV file, it must follow the path :
YY = Year as two digits, MM = Month as two digits, DD = Day as two digits, TX = Stage/Unit name as one letter and one digit, XX = Day roll up to three digits, CR = Camera roll as incremented number. == Sending == To officially announce the redelivery, a response email must to be sent to the Producer, the Editorial, the Production Coordinator and the IT. Simply click “Reply All” on your original mail (1st delivery) and explain the correction. Reuse the same template as the first time, count and list all the MXF and ALE files of the redelivery.
DO NOT change the subject of the mail (Re: JB | Reels #) as we want the mail to be listed in the same discussion than the first delivery.
Double-Roll
To correct a Double-Roll you need to isolate the wrong data in the .CSV file to process the takes separately. Reed the [#Double-Roll|outline Page 6] about Double Roll to know how to isolate the data.
Once the .CSV file is created and placed into a folder with the extension _Redelivery, write the correct name of the take, plus the extension _P1, _P2 …
Then, we want to change the start, duration and end fields to only include one second (roughly in the middle) of each file. This manipulation allows the Editorial to keep track of each part of the double rolled take.
In the scheme below, the lighter blue parts are the time codes we will write in the new .CSV :
Case Example
First of all, the error should be specified in the Camera Sheet as following:
[[Image:]]
We can see that the take 7J_01 has been filmed in to clips for the Cam W and X.
Manually, we need to find the clips concerned following :
- The two clips in XDCAM Browser, which should be one.
- The CSV which contains the right time code about the take 7J_01
By comparing the Time code and Duration of each clip, we can spot the P1 and P2 of the 7J_01. On the example above, The first clip from the CamW has a duration (01:24) shorter than the whole 7J_01's duration in the .CSV (3:20). And the last frame (Red cursor) has a time code that is somewhere inside the take 7J_01 (26:50 < 28:16 < 30:10). We probably found the first part of the double roll. Logically the next clip is the part 2, maybe followed by part 3 (in case of triple-roll).
To create our new .CSV, we now need to get 1 exact second placed somewhere important in the clip (When the team is filming) and write its start and cut. We don't need the duration field, as the tools can guess it themselves. The new .CSV can now be user for the MXF conversion and ALE creation, setting the Output with the _Redelivery extension !
[[Image:]]