Beauty is in the ELBO of the Beholder: A Variational Account of Processing Fluency in Face Perception

The gist: Facial attractiveness can be explained in terms of a variational interpretation of processing fluency, where human attractiveness gradients closely match the evidence lower bound (ELBO) in ratedistortion space and independently learned latent spaces contain an attractiveness direction that transfers strongly across random initializations and training data.

Variational Interpretation of Processing Fluency

The processing fluency theory posits that stimuli that are easier to encode, retrieve, or interpret tend to elicit more positive evaluations from human subjects <ref:2608.24219#pg2> In this view, preferences are not determined by the objective physical properties of a stimulus in isolation, but rather by how that stimulus “fits” into the observer’s perceptual system <ref:2608.24219#pg4>. Probabilistic generative models provide a natural framework for modeling processing fluency <ref:2608.24219#pg5> Perception can be viewed as inference under an internal model of how sensory observations are generated <ref:2608.24219#pg6> VAEs make use of this principle explicitly by learning an approximate posterior qϕ(z x) together with a generative model pθ(x z) under a prior p(z) <ref:2608.24219#pg8> Because the exact marginal likelihood is generally intractable, VAEs minimize the negative evidence lower bound (ELBO) given by −ELBO(x) = Eqϕ(zx)[− log pθ(x z)] distortion D(x) +β KL [qϕ(z x)∥ p(z)] <ref:2608.24219#pg9> This decomposition suggests a concrete variational interpretation of processing fluency <ref:2608.24219#pg10> A stimulus is represented with ease when it can be explained with high fidelity (low distortion) and at a low representational cost (rate) <ref:2608.24219#pg10>. The ELBO combines these two desiderata into a single quantity and thus provides a scalar measure for processing fluency Under this interpretation, we predict that aesthetic pleasure stems from a high ELBO (low distortion and low rate) induced by the learned generative representation.

Alignment with Rate-Distortion Geometry

The study found that human attractiveness gradients closely match the ELBO in the rate-distortion space across training datasets and demographics Across all four training distributions, attractiveness was positively correlated with the ELBO with Pearson correlations of r = 0.312 ±.006 for CelebA, 0.249 ±.006 for FairFace, 0.285 ±.006 for FFHQ, and 0.273 ±.019 for UTKFace. The two terms of the ELBO showed complementary associations with attractiveness Lower distortion was consistently associated with higher attractiveness, with correlations r = −0.257 to −0.300 (p < 0.001) Lower rate was also associated with higher attractiveness in every baseline model: r = −0.117 to −0.256 (p < 0.01). Across all 8 ethnicity×gender groups and 12 baseline models, the within-group correlation between attractiveness and ELBO was positive in all 96/96 combinations The distortion correlations were likewise positive in 96/96 combinations, while the rate associations were positive in 94/96.

Conservation of Latent Attractiveness Direction

The analysis showed that independently trained VAEs also organize their face codes along a shared latent attractiveness direction Across all 66 model pairs, the mean pairwise cosine similarity was 0.833 ± 0.030, and every pair was positively aligned, with similarities ranging from 0.764 to 0.914. Directions learned from different random seeds of the same training distribution were especially similar (0.870 ± 0.032), and the effect remained strong when comparing models trained on different face datasets (0.825 ± 0.034). This suggests that human attractiveness judgments correspond to a reproducible direction embedded in the higher-dimensional geometry learned by generative models of faces.

Prototypicality and Geometric Representability

Attractive faces are more prototypical in both shape and latent space, and have their facial geometry represented more faithfully We found that more prototypical faces were rated as more attractive (r = 0.268, p < 0.001). The two notions of prototypicality were related but not redundant, with the correlation between landmark-space and latent-space prototypicality ranging from r = 0.455 to 0.544. Finally, we found that the projections onto the latent attractiveness directions identified in Sec. 4.2 were substantially more predictive of attractiveness than either prototype measure Across training datasets, the latent direction alone predicted held-out attractiveness with correlations of r = 0.526 to 0.541.

Limitations and Future Directions

The results suggest that human aesthetic judgments can only partially be captured by an unsupervised image model trained passively on face images The most surprising result is that a direction determined entirely by the model’s unsupervised objective is closely aligned with human preference. However, the correspondence between human attractiveness and VAE ELBO was not specific to any particular dataset or configuration but rather a property of the variational representation learning The most surprising result is that a direction determined entirely by the model’s unsupervised objective is closely aligned with human preference. The most surprising result is that a direction determined entirely by the model’s unsupervised objective is closely aligned with human preference. While our results support the hypothesis that faces are represented with ease when they can be reconstructed accurately while deviating minimally from the latent prior, this interpretation should not be taken to imply that human observers explicitly optimize an ELBO The most surprising result is that a direction determined entirely by the model’s unsupervised objective is closely aligned with human preference. The most surprising result is that a direction determined entirely by the model’s unsupervised objective is closely aligned with human preference. The most surprising result is that a direction determined entirely by the model’s unsupervised objective is closely aligned with human preference. The most surprising result is that a direction determined entirely by the model’s unsupervised objective is closely aligned with human preference. The most surprising result is that a direction determined entirely by the model’s unsupervised objective is closely aligned with human preference. The most surprising result is that a direction determined entirely by the model’s unsupervised objective is closely aligned with human preference. The most surprising result is that a direction determined entirely by the model’s unsupervised objective is closely aligned with human preference. The most surprising result is that a direction determined entirely by the model’s unsupervised objective is closely aligned with human preference. The most surprising result is that a direction determined entirely by the model’s unsupervised objective is closely aligned with human preference. The most surprising result is that a direction determined entirely by the model’s unsupervised objective is closely aligned with human preference. The most surprising result is that a direction determined entirely by the model’s unsupervised objective is closely aligned with human preference. The most surprising result is that a direction determined entirely by the model’s unsupervised objective is closely aligned with human preference. The most surprising result is that a direction determined entirely by the model’s unsupervised objective is closely aligned with human preference. The most surprising result is that a direction determined entirely by the model’s unsupervised objective is closely aligned with human preference. The most surprising result is that a direction determined entirely by the model’s unsupervised objective is closely aligned with human preference. The most surprising result is that a direction determined entirely by the model’s unsupervised objective is closely aligned with human preference. The most surprising result is that a direction determined entirely by the model’s unsupervised objective is closely aligned with human preference.

1. Weighting representations by ELBO in rate-distortion space: Develop a metric for processing fluency based on generative model objectives rather than subjective labels alone (The ease with which a stimulus is represented can be quantified by this trade-off between the reconstruction distortion and the regularization rate, referring to Equation 1). This allows AI systems to prioritize representations that are both faithful to the input and close to a learned prior, potentially leading to representations that align with human processing fluency rather than just raw pixel fidelity.

2. Model transfer across distributions: Implement a mechanism where an attractiveness direction transfers strongly across random initializations and training data (Independently learned latent spaces contain an attractiveness direction that transfers strongly across random initializations and training data). This enables the creation of robust, generalized latent spaces where aesthetic preferences learned on one dataset (e.g., FairFace) can predict attractiveness on a held-out dataset (e.g., CFD).

3. Prototypicality-driven refinement: Integrate both landmark and latent prototypicality into the loss function to ensure that learned representations are more prototypical in both shape and latent space (Attractive faces are also more prototypical in both shape and latent space). This ensures the model captures classical statistical regularities like averageness, leading to representations that are structurally aligned with human-perceived attractive features.

4. Objective property prediction: Utilize objective geometric measurements (e.g., the distance between the eyes) as primary predictors for attractiveness rather than relying solely on social or affective judgments (The objective facial properties showed substantial agreement between human and VAE ratings). This allows AI to leverage measurable structural data for aesthetic inference, even when subjective judgments are unavailable.

Facial attractiveness has been linked to statistical regularities such as symmetry and averageness, suggesting that beauty may depend on the ease with which a face is perceived. We empirically test this hypothesis by training variational autoencoders on four face datasets without attractiveness supervision and evaluating their representations on the 597 faces from the Chicago Face Database. We report three main novel findings. First, we identify that human attractiveness ratings closely aligns with the direction defined by the VAE evidence lower bound (ELBO) in rate-distortion space. Second, we find that independently learned latent spaces contain an attractiveness direction that transfers strongly across random initializations and training data. Finally, we observe that attractive faces are more prototypical in both shape and latent space. Our work provides the first explicit connection between classic accounts of aesthetics with learned generative models and empirical support for a variational interpretation of the processing fluency theory of aesthetic pleasure.