English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Transformers have achieved great success in pluralistic image inpaintingrecently. However, we find existing transformer based solutions regard eachpixel as a token, thus suffer from information loss issue from two aspects: 1)They downsample the input image into much lower resolutions for efficiencyconsideration, incurring information loss and extra misalignment for theboundaries of masked regions. 2) They quantize $256^3$ RGB pixels to a smallnumber (such as 512) of quantized pixels. The indices of quantized pixels areused as tokens for the inputs and prediction targets of transformer. Althoughan extra CNN network is used to upsample and refine the low-resolution results,it is difficult to retrieve the lost information back.To keep input informationas much as possible, we propose a new transformer based framework "PUT".Specifically, to avoid input downsampling while maintaining the computationefficiency, we design a patch-based auto-encoder P-VQVAE, where the encoderconverts the masked image into non-overlapped patch tokens and the decoderrecovers the masked regions from inpainted tokens while keeping the unmaskedregions unchanged. To eliminate the information loss caused by quantization, anUn-Quantized Transformer (UQ-Transformer) is applied, which directly takes thefeatures from P-VQVAE encoder as input without quantization and regards thequantized tokens only as prediction targets. Extensive experiments show thatPUT greatly outperforms state-of-the-art methods on image fidelity, especiallyfor large masked regions and complex large-scale datasets. Code is available athttps://github.com/liuqk3/PUT

Reduce Information Loss in Transformers for Pluralistic Image Inpainting