Given an image with a specified garment region and a fabric swatch, our method generates visually realistic re-textured garments that preserve garment geometry, illumination, and image layout, while faithfully reproducing fabric texture, pattern density, and woven structure.
Trying different fabrics on existing garments is a widely applicable problem in digital fashion and computer graphics. A comprehensive transformation involves both material reflectance and geometric deformation from fabric drape. In this work, we focus on the visual aspects of this challenge and simplify fabric try-on to a re-texturing task that replaces garment materials while preserving the original geometry and illumination.
Prior approaches perform garment re-texturing via 3D or UV-space reconstruction and rendering, making them sensitive to reconstruction accuracy and rendering fidelity. Recent diffusion-based material transfer methods either lack fine-grained geometric and material control or suffer from domain gaps due to training on synthetic rendered data.
We propose a fabric try-on framework that leverages the generative priors of modern image editing models. Motivated by the in-context generation capability of Multimodal Diffusion Transformers, we reformulate garment re-texturing as a two-stage process consisting of fabric removal and fabric application via an intermediate material-normalized image.
We further introduce a real-image data curation pipeline and a context-aware tile augmentation strategy, enabling coherent and photorealistic fabric try-on from a single image. Extensive experiments show that our method achieves high-quality, controllable fabric transfer while preserving garment geometry and illumination, without requiring costly reconstruction or rendering pipelines.
We propose FabricTryOn, a two-stage garment re-texturing framework that reformulates fabric transfer as material normalization followed by controlled fabric application.
1. Two-stage re-texturing framework: We decompose garment re-texturing into fabric removal and fabric application. The first stage removes the original garment material to produce a material-normalized intermediate image, while the second stage applies the target fabric swatch to synthesize the final re-textured garment. Both stages are independently trained on the same image editing backbone.
2. Real-image data curation: To construct large-scale paired supervision without manual annotation, we develop an automated data curation pipeline consisting of garment identification, fabric removal and extraction, and fabric alignment. This produces high-quality garment–fabric training pairs from real-world fashion images.
3. Tile augmentation: To improve robustness to non-tileable fabric inputs, we introduce a context-aware tile augmentation pipeline that converts arbitrary fabric crops into seamless larger tileable textures while preserving local woven structure and pattern consistency.