Recolorable Posterization of Volumetric Radiance Fields

Abstract

Volumetric radiance fields have recently gained significant attention as a promising representation for photorealistic scene reconstruction. However, non-photorealistic rendering of such representation has been barely explored. In this study, we investigate artistic posterization of volumetric radiance fields. We extend the recent palette-based image editing framework, which naturally introduces intuitive color manipulation of the posterized results, into the radiance field. Our major challenge is how to apply the stylization effects coherently across different views. Based on the observation that computing a palette frame-by-frame can produce flickering, we propose to pre-compute a single palette from the volumetric radiance field covering its whole visible colors. We present a method based on volumetric visibility to sample visible colors from the radiance field while avoiding occluded and noisy regions. We demonstrate our workflow via application to pre-trained volumetric radiance fields with various stylization effects. We also show that our approach can produce more coherent and robust stylization effects compared to baseline methods computing palette on image.

Kenji Tojo

Ph.D Student (2nd year)

Nobuyuki Umetani

Associate Professor

My research interests include interactive smart engineering design tool using physics simulation and machine learning.