Download PDFOpen PDF in browser

Low Resource, Post-processed Lecture Recording from 4K Video Streams

21 pagesPublished: July 18, 2022


Many universities are using lecture recording technology to expand the reach of their teaching programs, and to continue instruction when face to face lectures are not possi- ble. Increasingly, high-resolution 4K cameras are used, since they allow for easy reading of board/screen context. Unfortunately, while 4K cameras are now quite affordable, the back- end computing infrastructure to process and distribute a multitude of recorded 4K streams can be costly. Furthermore, the bandwidth requirements for a 4K stream are exorbitant - running to over 2GB for a 45-60 minute lecture. These factors mitigate against the use of such technology in a low-resource environment, and motivated our investigation into meth- ods to reduce resource requirements for both the institution and students. We describe the design and implementation of a low resource 4K lecture recording solution, which addresses these problems through a computationally efficient video processing pipeline. The pipeline consists of a front-end, which segments presenter motion and writing/board surfaces from the stream and a back-end, which serves as a virtual cinematographer (VC), combining this contextual information to draw attention to the lecturer and relevant content. The bandwidth saving is realized by defining a smaller fixed-size, context-sensitive ‘cropping window’ and generating a new video from the crop regions. The front-end utilises compu- tationally cheap temporal frame differencing at its core: this does not require expensive GPU hardware and also limits the memory required for processing. The VC receives a small set of motion/content bounding boxes and applies established framing heuristics to determine which region to extract from the full 4K frame. Performance results coupled to a user survey show that the system is fit for purpose: it is able to produce good presenter framing/context, over a range of challenging lecture venue layouts and lighting conditions within a time that is acceptable for lecture video processing.

Keyphrases: 4K video post-processing, Automated Lecture Recording Systems, Presenter Tracking, video segmentation, Virtual Cameraman

In: Aurona Gerber (editor). Proceedings of 43rd Conference of the South African Institute of Computer Scientists and Information Technologists, vol 85, pages 15--35

BibTeX entry
  author    = {Charles Fitzhenry and Tanweer Khatieb and Patrick Marais and Stephen Marquard},
  title     = {Low Resource, Post-processed Lecture Recording from 4K Video Streams},
  booktitle = {Proceedings of 43rd Conference of the South African Institute of Computer Scientists and Information Technologists},
  editor    = {Aurona Gerber},
  series    = {EPiC Series in Computing},
  volume    = {85},
  pages     = {15--35},
  year      = {2022},
  publisher = {EasyChair},
  bibsource = {EasyChair,},
  issn      = {2398-7340},
  url       = {},
  doi       = {10.29007/vfb6}}
Download PDFOpen PDF in browser