Rank-Aware Hyperbolic Alignmentfor Vision-Language Dataset Distillation

ECCV 2026

Jongoh Jeong1, Sun-Kyung Lee2, Kuk-Jin Yoon1

1KAIST 2ETRI

Paper Slides Poster Code BibTeX
TL;DRRAHA distills compact image-text datasets by combining rank-aware semantic range matching with hyperbolic image-text alignment, preserving dominant shared relations while compressing weak residual correlations that would otherwise waste synthetic capacity.
Diagram comparing Euclidean hyperspherical image-text embeddings with Lorentz hyperboloid image-text embeddings.
Abstract

Compress paired data without flattening its hierarchy.

Vision-language dataset distillation compresses large image-text datasets into compact synthetic sets so vision-language models can be trained efficiently while retaining training utility and downstream performance. RAHA adds a geometry-aware objective: lift representations to hyperbolic space, expose the rank-k shared semantic range, then regulate the residual subspace instead of aligning every direction uniformly.

+9.6 pp

Flickr30k mean retrieval at 1000 pairs over Baseline: 38.0 vs. 28.4.

+6.5 pp

Flickr8k mean retrieval at 1000 pairs over Baseline. The 200-pair gain is already +5.8 pp.

+4.4 pp

COCO mean retrieval at 1000 pairs over Baseline, with +4.0 pp Flickr8k transfer at 500 pairs.

Vision-language dataset distillation setup showing a large real image-text dataset compressed into a compact learned synthetic set.
Setup: RAHA learns a small synthetic image-text dataset from a large real paired dataset. The compact output is reused to train vision-language models while preserving retrieval, transfer, and classification utility.
Motivation

Rank structure meets hyperbolic hierarchy.

Vision-language dataset distillation should compress more than representative samples. Text captions are inherently hierarchical: a generic caption such as dog should supervise many visual descendants, while fine captions such as yellow dog or brown dog must stay specific enough to preserve details. RAHA keeps this generic-specific alignment intact instead of flattening every caption into one Euclidean relation.

This leads to three desiderata for VLDD: preserve the bidirectional relevance that trains retrieval models, retain the generic-to-specific hierarchy carried by text, and allocate limited synthetic capacity to the low-rank shared signal shown by LoRS rather than matching every feature direction uniformly. RAHA turns these desiderata into hyperbolic lifting, range-residual decomposition, and dominant cross-relevance matching.

Motivation diagram showing LoRS and MERU feeding into RAHA.
Conceptual connection to RAHA:
Rank-aware relation structure and hyperbolic semantic geometry are combined for image-text dataset distillation.

Research Questions:

What must compression preserve?

A distilled image-text set is useful only if a model trained on it learns the same bidirectional ranking behavior as from real data. This makes cross-modal relevance the core information capacity to preserve, beyond visual appearance alone.

How should text hierarchy be retained?

The relevance signal is not flat because captions move from generic categories to fine attributes and relations. This calls for geometry that keeps generic text broad while keeping fine-grained image-text pairs distinct.

Where should capacity be spent?

LoRS shows that useful image-text relations are concentrated in low-rank structure. RAHA therefore separates range and residual subspaces, matching dominant shared semantics while regulating weaker residual variation.

Method

Lift hyperbolically, Decompose, then Match dominant cross-relevance.

The pipeline stays compact: synthetic pairs are optimized through hyperbolic contrastive alignment plus range/residual relevance distillation.

1

Synthetic pairs

Learn pixels and continuous text embeddings.

2

Lorentz lift

Use geodesic image-text contrastive alignment.

3

Tangent SVD

Estimate the real batch coupling rank.

4

Range match

Preserve dominant shared semantic order.

5

Residual control

Keep weak interactions from dominating.

Step-by-step RAHA walkthrough showing real and synthetic batches, hyperbolic lift, SVD decomposition, relevance matching, and synthetic set update.
ℒtotal = ℒhITC + 𝜆rangeℒrange + 𝜆residualℒresidual

ℒhITC aligns synthetic image-text pairs after the Lorentz hyperbolic lift.

ℒrange matches the dominant low-rank cross-relevance structure from real pairs.

ℒresidual controls weaker residual correlations so the compact set does not spend capacity on noise.

𝜆range, 𝜆residual balance semantic range preservation against residual refinement.

Analysis

Range matching carries the retrieval signal, and residual compression keeps it usable.

Component-level analysis shows RAHA is strongest when hyperbolic contrastive alignment is paired with dominant range supervision, controlled residual refinement, and hyperbolic rather than Euclidean relevance matching.

Fig. 2 component ablation chart.
Fig. 2: Component attribution under the same protocol. Range matching provides the main relevance-transfer gain, while residual control improves the full model as a refinement when anchored by the range.
Fig. A6 panels (a) and (b).
Fig. A6 panels (c) and (d).
Fig. A6: Sensitivity and ablation analysis for matching terms, geometry, and residual control.
Retrieval

Retrieval across synthetic budgets.

The chart and table show mean retrieval for 100, 200, 500, and 1000 pairs. The paper's asterisked batch-size variant is folded into Baseline, and each dataset panel uses the nearest y-axis ceiling above the strongest method.

Retrieval mean recall across 100, 200, 500, and 1000 synthetic pairs.
Mean recall across Flickr8k, Flickr30k, and COCO. Axes are tightened per dataset. Higher is better.
DatasetPairsBaseline meanRAHA meanΔ
Flickr8k10020.420.4+0.0
20019.525.3+5.8
50025.930.7+4.8
100030.637.1+6.5
Flickr30k10022.820.7-2.1
20022.025.7+3.7
50028.932.9+4.0
100028.438.0+9.6
COCO1007.07.2+0.2
2008.310.2+1.9
50011.213.7+2.5
100014.218.6+4.4
Transfer

Distilled relevance survives encoder architecture change.

Cross-architecture transfer is reported on Flickr8k by replacing image backbones and text encoders while keeping the distilled set fixed. RAHA separates most clearly at 200 and 500 pairs.

PairsBaseline
source		RAHA
source		Baseline
transfer		RAHA
transfer		Δ
10020.420.47.36.9-0.4
20018.825.37.28.7+1.5
50025.930.78.712.7+4.0
Cross-architecture transfer chart for Baseline and RAHA at 100, 200, and 500 pairs.
Transfer improves as synthetic image-text capacity grows.
Robustness Against Noise

Perturbation type reveals stability under compression.

Robustness is measured on Flickr8k as degradation δ after applying noise. The chart reports only the Table 5 mean columns for image-side and text-side perturbations across synthetic-pair budgets. Lower is better.

Minimal robustness chart comparing Baseline and RAHA degradation delta for perturbation types a, b, and c.
Table 5 mean-column view. Image-side and text-side mean δ are plotted separately for 100, 200, and 500 synthetic pairs. For the 100-pair per-noise entries, RAHA is more robust than Baseline on (a) JPEG/bit quantization and (b) AWGN for both modalities, and on (c) PGD-10 it lowers image-side δ while tying text-side δ at 0.0.
(a)

JPEG compression on images and bit quantization on text. Included in each modality mean from Table 5.

(b)

AWGN for both image and text. Included in the image-side and text-side mean δ columns.

(c)

PGD-10 for both image and text. Included in the mean-column robustness summary, with lower δ indicating less degradation.

Synthesized pairs

Less residual debris, more readable pairs.

The paper's qualitative comparison shows initialization, Baseline, and RAHA outputs. RAHA keeps stronger image-text agreement while reducing high-frequency artifacts.

Fig. 1 qualitative synthesized image-text pairs comparing initialization, Baseline, and RAHA.
Fig. 1: Qualitative examples of synthesized image-text pairs. Each group compares initialization, Baseline, and RAHA, showing that RAHA produces more semantically aligned images and text with fewer high-frequency artifacts.
Prompted classification

Prompted classification with corresponding qualitative samples.

At 100 pairs, RAHA improves top-1 prompted classification on all four benchmarks reported in the appendix. RAHA-distilled 2×2 samples are shown beside the accuracy table, with full per-dataset comparisons below.

CIFAR-100
RAHA-distilled samples on CIFAR-100.
CUB-200-2011
RAHA-distilled samples on CUB-200-2011.
Stanford Cars
RAHA-distilled samples on Stanford Cars.
ImageNet-1K
RAHA-distilled samples on ImageNet-1K.
DatasetBaselineRAHAΔ
CIFAR-10014.5515.60+1.05
CUB-20012.5314.43+1.90
Cars5.637.80+2.17
ImageNet-1K7.627.88+0.26
CIFAR-100
CIFAR-100 qualitative distilled samples from the appendix.
Distilled data comparison on CIFAR-100. Left: initial samples. Middle: Baseline-distilled. Right: RAHA-distilled.
CUB-200-2011
CUB qualitative distilled samples from the appendix.
Distilled data comparison on CUB-200-2011. Left: initial samples. Middle: Baseline-distilled. Right: RAHA-distilled.
Stanford Cars
Stanford Cars qualitative distilled samples from the appendix.
Distilled data comparison on Stanford Cars. Left: initial samples. Middle: Baseline-distilled. Right: RAHA-distilled.
ImageNet-1K
ImageNet qualitative distilled samples from the appendix.
Distilled data comparison on ImageNet-1K. Left: initial samples. Middle: Baseline-distilled. Right: RAHA-distilled.

Citation

@inproceedings{jeong2026raha,
  title     = {Rank-Aware Hyperbolic Alignment for Vision-Language Dataset Distillation},
  author    = {Jeong, Jongoh and Lee, Sun-Kyung and Yoon, Kuk-Jin},
  booktitle = {Proceedings of the European Conference on Computer Vision (ECCV)},
  year      = {2026}
}