SkinFlow: Efficient Information Transmission for Open Dermatological Diagnosis via Dynamic Visual Encoding and Staged RL

TL;DR

This work reframes dermatological diagnosis as an information transmission problem, arguing that maximizing recoverable visual information is key to precision beyond mere parameter scaling. It introduces SkinFlow, which combines a Dynamic Visual Encoding module to effectively unfold diagnostic visual manifolds with a two-stage reinforcement learning pipeline that first compresses describable features into text and then decodes implicit textures into diagnoses. A clinically grounded evaluation framework emphasizes safety and hierarchical relevance over rigid label matching, and empirical results show a 7B SkinFlow model surpassing much larger models on Fitzpatrick17k, with strong performance on open-world datasets. The study highlights the practical impact of optimizing geometric capacity and information flow for reliable, open-vocabulary dermatological diagnosis and suggests broader applicability to other visually intensive medical domains.

Abstract

General-purpose Large Vision-Language Models (LVLMs), despite their massive scale, often falter in dermatology due to "diffuse attention" - the inability to disentangle subtle pathological lesions from background noise. In this paper, we challenge the assumption that parameter scaling is the only path to medical precision. We introduce SkinFlow, a framework that treats diagnosis as an optimization of visual information transmission efficiency. Our approach utilizes a Virtual-Width Dynamic Vision Encoder (DVE) to "unfold" complex pathological manifolds without physical parameter expansion, coupled with a two-stage Reinforcement Learning strategy. This strategy sequentially aligns explicit medical descriptions (Stage I) and reconstructs implicit diagnostic textures (Stage II) within a constrained semantic space. Furthermore, we propose a clinically grounded evaluation protocol that prioritizes diagnostic safety and hierarchical relevance over rigid label matching. Empirical results are compelling: our 7B model establishes a new state-of-the-art on the Fitzpatrick17k benchmark, achieving a +12.06% gain in Top-1 accuracy and a +28.57% boost in Top-6 accuracy over the massive general-purpose models (e.g., Qwen3VL-235B and GPT-5.2). These findings demonstrate that optimizing geometric capacity and information flow yields superior diagnostic reasoning compared to raw parameter scaling.

Figures (7)

• Figure 1: Two-stage reinforcement learning framework for dermatological diagnosis. In Stage 1, the model performs medical caption generation. The LLM scores each attribute field of the generated description, and these field-wise scores are integrated into a caption reward to refine medical feature learning. In Stage 2, the model predicts disease categories based on learned representations. The LLM evaluates each prediction, and a customized reward function converts these evaluations into a final diagnostic reward that optimizes classification accuracy and ranking consistency.
• Figure 2: Illustration of frequency disjoint basis construction. The process transforms the weight matrix $\mathbf{W}_{ori} \in \mathbb{R}^{d\times d}$ from the spatial domain to the Fourier domain via the Discrete Fourier Transform (DFT). The frequency spectrum is then partitioned into disjoint groups based on frequency index (e.g., $\mathbf{P}^1$ corresponds to the central low-frequency components in blue, while $P^2$ corresponds to the peripheral high-frequency components in orange). To generate a specific spatial basis $\mathbf{B}_1$, we retain only the learnable parameters belonging to Group 1 ($\mathbf{P}^1$) and mask all other frequency indices to zero. Finally, an inverse DFT (iDFT) reconstructs the spatial basis matrix. This design ensures that each basis $\mathbf{B}_k$ specializes in a distinct frequency band, minimizing spectral redundancy.
• Figure 3: Visualizing manifold unfolding and virtual capacity. We evaluate the geometric representation capability on four classic non-linearly separable datasets: Spirals, XOR, Circles, and Moons. (Left column of each sub-figure) Constrained by Cover's Theorem, a standard static layer (width $d=2$) is topologically restricted to a single separating hyperplane, resulting in severe underfitting (accuracy $\approx 50\%$). (Middle column of each sub-figure) Without increasing the physical width ($d=2$), FDLinear ($K=12$) dynamically constructs high-order decision boundaries, successfully disentangling complex manifolds (e.g., the intertwined spirals). This empirically validates the "Virtual Width Expansion" hypothesis. (Right column of each sub-figure) The vector fields visualize the orientation of the generated weight matrix $W(\bar{x})$ across the input space. The rotating and radiating patterns (labeled "Field Adaptation") demonstrate that FDLinear acts as a sample adaptive layer, modulating its projection direction based on local geometric curvature.
• Figure 4: Effectiveness of Stage 1 Caption Training. (a) Training reward curves and (b) validation reward curves. The blue line represents the model trained directly on the general-purpose baseline, while the red line denotes the model further trained based on the caption-enhanced Stage I model.
• Figure 5: Visual attention attribution analysis. We visualize the cross-attention maps of the final diagnostic token relative to the input image across different models. The histograms (right) show the distribution of attention weights. (1) Attention concentration: While baselines (Qwen2.5/3-VL) exhibit diffuse attention, often distracted by healthy skin or background noise, Our method demonstrates precise lesion localization, sharply focusing on pathological features (e.g., papules, ulcers). (2) Effect of FDLinear and stage 1 training: Comparing "w/o stage1 & DVE" with "Ours", the introduction of the Dynamic Vision Encoder (DVE) and stage 1 training significantly improve the model to adaptively highlights important diagnostic details. (3) Confidence shift: The histograms reveal that our method (yellow) assigns higher attention weights ($>0.06$) to key regions than the baselines, indicating a shift from uncertain global scanning to confident diagnostic reasoning.