Line Timing Semi-Automation with Microservices

Technologies Used

Node.js, Angular.js, Evertz Quartz Protocol, SNMP

Goal

CNN is a global news network, so this comes with the need to anchor shows from remote locations. Instead of switching multiple cameras on-site, CNN feeds each camera back to the control room. Because of delays inherent in video transmission over long distances, these camera feeds must be brought in sync with each other. Otherwise, lip flap will quickly become obvious.

Quality Control operators use VALID (Video/Audio Lineup and ID) color bars/tone and a VALID reader to time the lines before production begins. They were using a PHP-based web application to do this timing. The app worked, but operators still needed to make some video routes manually and it wasn’t easily expanded. Worse, there were two single points of failure: the app was hosted on a single (aging, circa 2003!) server and router control depended on a single serial to IP interface.

I was tasked with re-writing the PHP app to be more deployable and to fit CNN’s internal web app design specification. Along the way, I was able to make the architecture more dynamic and general purpose.

Approach

There are two back-end requirements for the timing application: video router control and processing amplifier (proc amp) control. In the old application, router control was via a soon-to-be-obsolete serial interface and proc amp control was via SNMP, though the OIDs and IP addresses were hard-coded.

Old application architecture

Old application architecture

To control the video router, I built a microservice around the Evertz Quartz protocol, a simple TCP protocol. The service dynamically reads source and destination port information from the router and presents a REST interface with full control as well as a WebSocket interface for receiving crosspoint changes.

To control proc amps, I built microservice that reads IP, port and model information from Evertz Magnum interface. Based on this information, control objects are created with the correct OIDs and behavior. This service is more dynamic than the previous SNMP implementation in that it dynamically receives configuration info for all proc amps, not just a select few that are hard coded.

New application architecture

New application architecture

Both microservices authenticate with either Active Directory or an internal auth token. Because they are dependent on existing production configuration data, all future configuration changes will be automatically reflected in the microservices.

Finally, an Angular.js app was written to interact with these microservices. The level of automation is high enough that all actions can now be done via the web interface, removing the requirement for direct router control.

qc_line_timing_interfaceStatus

All parts of the system were tested in dev and production. The Quartz microservice was deployed using Chef to a pair of Virtual Machines behind a Virtual IP address (VIP). The ProcAmp microservice was deployed to two physical hosts, also behind a VIP. They needed to be on bare-metal in order to have access to a network that was not accessible to any of CNN’s VM clusters. Finally, the front-end was deployed to AWS S3. The app is now fully in production.

About The Author

Kyle Anderson
I'm a media and IT professional and JavaScript developer who worked most recently as an Associate Broadcast IT Engineer (Tier II) for CNN in Atlanta. One of my life-long goals is to help bridge data divides - missing connections between software systems and data stores - promoting inter-system communication and automation. Many of the projects described here reflect this goal in some way or another.