Textual documents, such as manuscripts and historical newspapers, make up an important part of our cultural heritage. Massive digitization projects have been conducted across the globe for a better preservation of, and for providing easier access to such, often vulnerable, documents. These digital counterparts also allow to unlock the rich information contained inside and across them thanks to various types of computational models for document image understanding. In this article, we will shed a light on the document image processing pipeline, from scan to information extraction. As it turns out, human perceptual-driven algorithms are among the most powerful approaches for generic document image understanding, required to deal with a myriad of layouts. In this context, we will in particular explain Gestalt visioning and the linked concept of text homogeneity that allows for enhanced layout analysis and even damage recognition, especially relevant in a cultural heritage setting. We conclude with a recent promising development, namely joint visual and language processing, that will take document image understanding to the next level in the future.