The objective of this project is to develop image processing algorithms based mainly on vision science models that address different challenges in moviemaking, from shooting to exhibition.
Given that in terms of sensing capabilities cameras are in most regards better than human photoreceptors, the superiority of human vision over camera systems lies in the better processing which is carried out in the retina, thalamus and visual cortex. Therefore, rather than working on the hardware, improving lenses and sensors, we resort instead to existing knowledge on visual neuroscience and models on visual perception to develop software methods mimicking neural processes in the human visual system, and apply these methods to images captured with a regular camera. We take the same approach when addressing projection/exhibition, developing vision-based image processing algorithms that try to overcome the limitations in contrast and color reproduction that current display systems have. For shooting and post-production we also work on problems such as noise reduction, HDR generation and color stabilization, using in these cases image processing methods based on PDEs.
From a technological standpoint, our project will impact how movies are made (in less time, with less equipment, with smaller crews, with more artistic freedom) but also which movies are made (since good-visual-quality productions will become more affordable.) We also anticipate a considerable technological impact in the realm of consumer video. From a scientific standpoint, this project will imply finding solutions for several challenging open problems in image processing and computer vision, but it also has a strong potential to bring methodological advances to other domains like experimental psychology and visual neuroscience.
Our research is funded by the following public institutions: European Research Council, Ministerio de Economía y Competitividad, Institució Catalana de Recerca i Estudis Avançats.
We have been granted the ERC Proof of Concept Grant 2016 VIPERCON for bringing to market a patented tone mapping algorithm developed in the context of the IP4EC ERC Starting Grant.