ESRGAN Slim

Version :Last Updated :NPM installs per week :CDN hits per week :

These models are the fastest available ESRGAN models, and are intended to run in a browser.

The model weights were trained using the image-super-resolution Python repo and subsequently converted to Tensorflow.js models.

Paper

The Super-Resolution Generative Adversarial Network (SRGAN) is a seminal work that is capable of generating realistic textures during single image super-resolution. However, the hallucinated details are often accompanied with unpleasant artifacts. To further enhance the visual quality, we thoroughly study three key components of SRGAN - network architecture, adversarial loss and perceptual loss, and improve each of them to derive an Enhanced SRGAN (ESRGAN). In particular, we introduce the Residual-in-Residual Dense Block (RRDB) without batch normalization as the basic network building unit. Moreover, we borrow the idea from relativistic GAN to let the discriminator predict relative realness instead of the absolute value. Finally, we improve the perceptual loss by using the features before activation, which could provide stronger supervision for brightness consistency and texture recovery. Benefiting from these improvements, the proposed ESRGAN achieves consistently better visual quality with more realistic and natural textures than SRGAN and won the first place in the PIRM2018-SR Challenge.

— ESRGAN: Enhanced Super-Resolution Generative Adversarial Networks

Samples

Here are some examples of upscaled images using these models.

View non-upscaled versions

View images all at same size

Original	2x Upscaled	3x Upscaled	4x Upscaled

Demo

Installation

npm install @upscalerjs/esrgan-slim

Usage

Browser

Using a transpiler

If using a transpiler (such as tsc, webpack, or vite) import the model with:

import Upscaler from 'upscaler';
import x2 from '@upscalerjs/esrgan-slim/2x';

const upscaler = new Upscaler({
  model: x2,
})

Using a script tag

If importing Tensorflow.js using script tags, import the specific model and UpscalerJS with:

<script src="https://cdn.jsdelivr.net/npm/@upscalerjs/esrgan-slim@latest/dist/umd/2x.min.js"></script> <!-- loads the 2x model -->
<script src="https://cdn.jsdelivr.net/npm/upscaler@latest/dist/browser/umd/upscaler.min.js"></script>

<script type="text/javascript">
  const upscaler = new Upscaler({
    model: ESRGANSlim2x,
  })
</script>

The model will be made available on the global window object. See Available Models for information on referencing by name.

You can also import all models in this package via the index.min.js import:

<script src="https://cdn.jsdelivr.net/npm/@upscalerjs/esrgan-slim@latest/dist/umd/index.min.js"></script>

If so, all model configurations will be available on the global object ESRGANSlim.

Node

Require the model with:

const Upscaler = require('upscaler/node'); // if using @tensorflow/tfjs-node-gpu, change this to upscaler/node-gpu
const x2 = require('@upscalerjs/esrgan-slim/2x');

const upscaler = new Upscaler({
  model: x2,
})

The model will work for both node and node-gpu flavors of Tensorflow.js.

Available Models

ESRGAN Slim ships with four models corresponding to the desired scale of the upscaled image:

• 2x: @upscalerjs/esrgan-slim/2x
• 3x: @upscalerjs/esrgan-slim/3x
• 4x: @upscalerjs/esrgan-slim/4x
• 8x: @upscalerjs/esrgan-slim/8x (note: the 8x model runs only in Node)

All models are also exported via the root export:

import Upscaler from 'upscaler';
import models from '@upscalerjs/esrgan-slim';

const upscaler = new Upscaler({
  model: models.x2,
  // model: models.x3,
  // model: models.x4,
  // model: models.x8,
})

If referencing the models via script tags, refer to the models by their global names:

• 2x: ESRGANSlim2x
• 3x: ESRGANSlim3x
• 4x: ESRGANSlim4x
• 8x: ESRGANSlim8x

Specific model files can be loaded by specifying the scale:

<script src="https://cdn.jsdelivr.net/npm/@upscalerjs/esrgan-slim@latest/dist/umd/2x.min.js"></script>
<script src="https://cdn.jsdelivr.net/npm/@upscalerjs/esrgan-slim@latest/dist/umd/3x.min.js"></script>
<script src="https://cdn.jsdelivr.net/npm/@upscalerjs/esrgan-slim@latest/dist/umd/4x.min.js"></script>
<script src="https://cdn.jsdelivr.net/npm/@upscalerjs/esrgan-slim@latest/dist/umd/8x.min.js"></script>

Performance + Speed Measurements

Architecture

This model is trained via a Python implementation of the ESRGAN architecture. The Python repo has instructions on training from scratch.

Training Details

The model is trained on 4 scales.

The model is trained on the Div2k dataset.

It was trained for 500 epochs, with the following hyperparameters:

• architecture: rdn
• C: 1
• D: 2
• G: 4
• G0: 64

The batch size was 12, and the batches per epoch was 20. The learning rate was set to 0.0004. The HR patch size was set to 128 or 129 depending on the scale (ensuring it is divisible by the scale) with the LR patch size being the resultant scale HR_patch_size / scale.

License

MIT License © Kevin Scott

The original ESRGAN repository is licensed under an Apache License 2.0