Bridging the Perception Gap in Image Super-Resolution Evaluation

Shaolin Su1, Josep M. Rocafort1,2, Danna Xue1,2,
David Serrano-Lozano1,2, Lei Sun3, Javier Vazquez-Corral1,2
1Computer Vision Center    2Universitat Autònoma de Barcelona    3INSAIT, Sofia University "St. Kliment Ohridski"
IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) 2026
Paper Code
"While SR models have evolved rapidly,
the evaluation criteria used to assess them have remained largely unchanged."

Abstract

As super-resolution (SR) techniques advance, we observe a growing distrust of evaluation metrics in recent SR research. An inconsistency often emerges between certain evaluation criteria and human perceptual preference. Although current SR research employs varying metrics to evaluate SR performance, it remains underexplored how robust and reliable these metrics actually are. To bridge this gap, we conduct a comprehensive analysis of widely used image quality metrics, examining their consistency with human perception when evaluating state-of-the-art SR models. We show that some metrics exhibit only limited—or even negative—correlation with human preferences. We further identify several intrinsic challenges in SR evaluation that compromise the effectiveness of both full-reference (FR) and no-reference (NR) image quality assessment (IQA) frameworks. To address these issues, we propose a simple yet effective Relative Quality Index (RQI) framework, which assesses the relative quality discrepancy between image pairs. Our framework enables easy integration and notable improvements for existing IQA metrics in SR evaluation. Moreover, it can be utilized as a valuable training guide for SR models, enabling the generation of images with more realistic details while maintaining structural fidelity.

1. How Long-Standing Metrics can Actually Assess Modern SR Models?

We conduct a comprehensive user study to collect user scores on outputs from different SR models, to analyze how existing image metrics correlate with human perception. We focus on the classic X4 SR task and gather human opinions on seven representative SR models (RealESRGAN, BSRGAN, SwinIR, HAT, StableSR, SeeSR, and PASD) across five SR benchmarks (DIV2K-wild, RealSR, DRealSR, Set5, and Set14). We then select varying widely adopted metrics (show by column) to examine their cosistency with human preferences (where the last column is our proposed metric).

Consistency evaluations of quality metrics with human perception on widely used SR benchmarks.

Observations

  1. Traditional distortion-oriented FR metrics (well-known SSIM and PSNR), show poor or even negative correlations with human perceptual judgments. The result is consistent with the perception-distortion trade-off.
  2. Similar behaviors can be observed from the perception-oriented FR metrics DISTS and LPIPS. While they exhibit moderate consistency on DIV2K and Set5&Set14, the performance drops significantly on RealSR and DRealSR, probably due to the relatively low quality of GT images in RealSR and the large image resolutions in DRealSR.
  3. Somewhat counterintuitively, NR-IQA metrics---including both traditional models NIQE, PI and deep learning-based models Clip-IQA, MANIQA---exhibit higher consistency with human perception. However, they may also suffer from the risk of lacking reference to make moderate evaluations.
  4. Both AFINE and DeQA-Score achieve better consistency with human preferences, indicating that more advanced IQA metrics can provide more accurate evaluations.

Let's dig deeper into the evaluation challenges

More detailed analyse of how current metrics may fail in different aspects for SR evaluation.
Existing IQA frameworks may fail with varying challenges:
  1. Distortion-based FR metrics (SSIM) tend to favor averaged and smoother regions over perceptually diverse textures (1st column);
  2. NR metrics (Clip-IQA) fail to capture fidelity changes in fine details such as characters, due to the absence of proper references (2nd column);
  3. Perceptual FR-IQA metrics (LPIPS) fail when the GT image quality is inferior and the SR output perceptually surpasses it (3rd column);
  4. SOTA SR model outputs often exhibit subtle perceptual differences, while many existing metrics are primarily designed to capture large, easily noticeable degradations (4th column).

2. A General and Lightweight Framework to Tackle the Challenges

We propose a general training framework, Relative Quality Index (RQI), that boldly serve arbitrary images (including distorted ones) as reference, to train IQA models and evaluate the relative quality discrepancy between image pairs.

The proposed RQI training scheme (b) differs from the traditional FR-IQA training scheme (a) in three key aspects: RQI is asymmetric (swapping inputs produces the opposite result); it calculates relative discrepancies between any two degraded images from the same sequence; and it constructs denser pairwise comparisons to facilitate fine-grained perceptual predictions.

This simple design enables RQI to 1. accurately assess fidelity based on references, 2. remain reliable even when ground truths are of suboptimal quality, and 3. perform fine-grained discrimination among high-quality SR results, thus effectively alleviates the afore mentioned challenges.

3. Results

Effectiveness of RQI when serving as a metric

We show different cases where RQI can handle while existing metrics fail. SSIM favours blurry RealESRGAN over textured SeeSR (top left); Clip-IQA fails to evaluate character-level consistency between BSRGAN and PASD (top right); LPIPS is biased by vague GT details when comparing HAT and SeeSR (bottom left); AFINE struggles with subtle quality differences in the bottom-right case.

Effectiveness of RQI when serving as an auxillary training loss for SR Models

RQI-driven optimisation not only improves perceptual realism but also enforces structural consistency, leading to higher-fidelity super-resolved results — with the most pronounced gains on diffusion-based models.

Visual comparison of SR models trained with and without RQI as an auxiliary loss. RQI-guided training produces sharper, more realistic details (wolf fur, penguin feathers) while preserving structural fidelity (dome geometry, regularised grille bars).

Citation

If you find this work useful for your research, please cite:

@inproceedings{su2026rqi,
  title     = {Bridging the Perception Gap in Image Super-Resolution Evaluation},
  author    = {Su, Shaolin and Rocafort, Josep M. and Xue, Danna and
               Serrano-Lozano, David and Sun, Lei and Vazquez-Corral, Javier},
  booktitle = {Proceedings of the IEEE/CVF Conference on Computer Vision
               and Pattern Recognition (CVPR)},
  year      = {2026}
}

Acknowledgments

This work was supported by grants PID2021-128178OB-I00 and PID2024-162555OB-I00 funded by MCIN/AEI/10.13039/501100011033 and ERDF "A way of making Europe", the Generalitat de Catalunya CERCA Program, the grant Càtedra ENIA UAB-Cruïlla (TSI-100929-2023-2), and the 2025 Leonardo Grant for Scientific Research and Cultural Creation from the BBVA Foundation. Shaolin Su was supported by the HORIZON MSCA Postdoctoral Fellowship (project number 101152858). David Serrano-Lozano was supported by the FPI grant from the Spanish Ministry of Science and Innovation (PRE2022-101525). Lei Sun was partially funded by the Ministry of Education and Science of Bulgaria's support for INSAIT and by the European Union's Horizon Europe programme (grant agreement 101168521).