Stage 05 · Resilience

Rehearse the Missing Feed

Resilience means knowing how the show changes when a dependency fails, who leads the response and how everyone confirms a safe return.

A fallback plan is not the same thing as a backup device. It is a sequence of decisions that lets the crew detect a problem, protect the programme, communicate a change and restore the intended route. In esports, a missing observer view or an interrupted remote contribution can affect both the audience and the live match workflow. A useful plan recognises that people need a clear next action while the cause is still being investigated.

Describe the failures that matter to this show

Begin with the production brief and signal map. Consider loss of a game source, frozen video, missing commentary, audio distortion, graphics failure, replay unavailable, a remote guest dropping, a control-room network issue or a platform ingest interruption. Include operational failures too: a key operator becomes unavailable, a critical cue is missed, or a match delay changes the running order. Prioritise scenarios by audience impact and likelihood, not by how dramatic they sound.

For each scenario, write the observable symptom rather than guessing at a cause. “The programme return is frozen” is a useful report; “the encoder is broken” may be speculation. Define who can verify the symptom, who owns diagnosis and who is authorised to change the programme. Separate rapid protective action from root-cause investigation. If the audience is seeing an unintended picture, the crew may need to take a safe slate or alternate source before anyone has identified the failed component.

Design fallbacks that can really be used

Map a primary and alternative route for important programme elements. A spare observer view helps only if it is powered, configured, available under the event's game rules and visible to the operator. A second internet path helps only if it is independent enough to survive the failure and the output can use it. A backup encoder is not an end-to-end recovery plan if it depends on the same failed input or network switch as the primary.

Review shared dependencies: electrical supply, local network, venue connectivity, remote contribution service, credentials, operator access and the physical location of equipment. Avoid describing two paths as independent until the dependencies have been checked. NIST contingency-planning guidance offers a useful general framework for identifying priorities, responsibilities and recovery steps; the production should adapt such principles to its own size and risk rather than copy a generic plan.

Choose a safe audience state for each failure. Options might include holding the current programme, taking a prepared slate, moving to a verified alternate source or pausing while the tournament authority resolves a competition question. The correct response depends on editorial policy, competition rules, platform requirements and the fault itself. Decide these boundaries before air. A producer should not have to invent the show's incident policy while a feed is already missing.

Write the recovery sequence in plain language

For each priority scenario, identify five items: detection signal, accountable lead, immediate protective action, communication path and restoration check. Add a clear escalation route and a threshold for updating stakeholders or audience-facing talent. State what information the crew should preserve for diagnosis, such as time, source label, visible symptom and relevant error message. Keep the sequence short enough to use during a live incident.

Incident elementOperational questionExample of useful evidence
DetectHow will the team know the path is unhealthy?Named monitor, return view or operator check
ProtectWhat safe programme state can be taken now?Verified alternate, slate or agreed hold
LeadWho can authorise the change?Named production role and escalation route
RestoreWhat proves the primary route is safe again?Picture, sound, sync and output confirmation

Drill without creating a second incident

Rehearse controlled failures during a test window. Tell the drill lead what can be interrupted, protect real competition and audience services, and make it clear when the exercise begins and ends. Test one meaningful failure at a time before combining scenarios. Record how long detection and recovery took, but also note whether the crew understood who had authority and whether the chosen fallback preserved a coherent show.

Include the people who depend on the change. A technical switch may affect talent timing, graphics, replay and tournament operations. Practise the brief message that keeps those groups aligned: what happened, what is live now, what they should do, and when another update is expected. A recovery plan that works in the machine room but leaves the director or producer guessing is incomplete.

Where a plan uses vendor support, confirm contact routes and service expectations ahead of the event. Organisations such as the EBU, SMPTE, AES and AMWA publish professional media engineering material; vendor documentation and platform status channels can also inform operational preparation. None removes the need to verify the particular venue and system. Check current guidance, record the version or event profile used, and test the configuration with the actual kit and people.

Restore deliberately and learn from the event

Do not return to the primary route solely because a device appears healthy. Confirm the source is stable, its picture and audio are correct, the programme output is receiving it and the relevant people agree that the change is safe. Tell the crew when the fallback is no longer active. Record temporary patches so they do not remain hidden after the event.

After a real fault or drill, review the timeline without assigning blame. Ask what first indicated the issue, what information was missing, which decision helped and whether the fallback created new risk. Capture changes as clear actions with owners and due dates: update a signal diagram, change a channel label, add a test step, adjust staffing or rehearse a different branch. Close the action only when the improvement has been checked.

Resilience is a production habit, not an extra layer reserved for large tournaments. A small team can improve its readiness by naming its most important show dependencies, establishing one safe fallback for each, and practising the first minute of response. Consistency matters: clear ownership and accurate status reduce unnecessary improvisation. When an expected feed disappears, the crew should already know how to keep the audience experience coherent while the technical team works toward a verified recovery.

Next: Questions From the Gallery →