Skip to main content
Unenhanced file for esrgan-thickEnhanced file for esrgan-thick

ESRGAN Thick

Version :Last Updated :NPM installs per week :CDN hits per week :

These models are the best ESRGAN upscaling models, at the cost of slower inference. They are best suited to a Node.js environment with a GPU, although can be used in the browser with significant latency.

The model weights were trained using the image-super-resolution Python repo and subsequently converted to Tensorflow.js models.

Paper​

The Super-Resolution Generative Adversarial Network (SRGAN) is a seminal work that is capable of generating realistic textures during single image super-resolution. However, the hallucinated details are often accompanied with unpleasant artifacts. To further enhance the visual quality, we thoroughly study three key components of SRGAN - network architecture, adversarial loss and perceptual loss, and improve each of them to derive an Enhanced SRGAN (ESRGAN). In particular, we introduce the Residual-in-Residual Dense Block (RRDB) without batch normalization as the basic network building unit. Moreover, we borrow the idea from relativistic GAN to let the discriminator predict relative realness instead of the absolute value. Finally, we improve the perceptual loss by using the features before activation, which could provide stronger supervision for brightness consistency and texture recovery. Benefiting from these improvements, the proposed ESRGAN achieves consistently better visual quality with more realistic and natural textures than SRGAN and won the first place in the PIRM2018-SR Challenge.

— ESRGAN: Enhanced Super-Resolution Generative Adversarial Networks

Samples​

Here are some examples of upscaled images using these models.

Original2x Upscaled3x Upscaled4x Upscaled8x Upscaled
Original imageUpscaled image using esrgan-thick/2xUpscaled image using esrgan-thick/3xUpscaled image using esrgan-thick/4xUpscaled image using esrgan-thick/8x

Demo​

Installation​

npm install @upscalerjs/esrgan-thick

Usage​

Browser​

Using a transpiler​

If using a transpiler (such as tsc, webpack, or vite) import the model with:

import Upscaler from 'upscaler';
import x2 from '@upscalerjs/esrgan-thick/2x';

const upscaler = new Upscaler({
  model: x2,
})

Using a script tag​

If importing Tensorflow.js using script tags, import the specific model and UpscalerJS with:

<script src="https://cdn.jsdelivr.net/npm/@upscalerjs/esrgan-thick@latest/dist/umd/2x.min.js"></script> <!-- loads the 2x model -->
<script src="https://cdn.jsdelivr.net/npm/upscaler@latest/dist/browser/umd/upscaler.min.js"></script>

<script type="text/javascript">
  const upscaler = new Upscaler({
    model: ESRGANThick2x,
  })
</script>

The model will be made available on the global window object. See Available Models for information on referencing by name.

You can also import all models in this package via the index.min.js import:

<script src="https://cdn.jsdelivr.net/npm/@upscalerjs/esrgan-thick@latest/dist/umd/index.min.js"></script>

If so, all model configurations will be available on the global object ESRGANThick.

Node​

Require the model with:

const Upscaler = require('upscaler/node'); // if using @tensorflow/tfjs-node-gpu, change this to upscaler/node-gpu
const x2 = require('@upscalerjs/esrgan-thick/2x');

const upscaler = new Upscaler({
  model: x2,
})

The model will work for both node and node-gpu flavors of Tensorflow.js.

Available Models​

ESRGAN Thick ships with four models corresponding to the desired scale of the upscaled image:

  • 2x: @upscalerjs/esrgan-thick/2x
  • 3x: @upscalerjs/esrgan-thick/3x
  • 4x: @upscalerjs/esrgan-thick/4x
  • 8x: @upscalerjs/esrgan-thick/8x

All models are also exported via the root export:

import Upscaler from 'upscaler';
import models from '@upscalerjs/esrgan-thick';

const upscaler = new Upscaler({
  model: models.x2,
  // model: models.x3,
  // model: models.x4,
  // model: models.x8,
})

If referencing the models via script tags, refer to the models by their global names:

  • 2x: ESRGANThick2x
  • 3x: ESRGANThick3x
  • 4x: ESRGANThick4x
  • 8x: ESRGANThick8x

Specific model files can be loaded by specifying the scale:

<script src="https://cdn.jsdelivr.net/npm/@upscalerjs/esrgan-thick@latest/dist/umd/2x.min.js"></script>
<script src="https://cdn.jsdelivr.net/npm/@upscalerjs/esrgan-thick@latest/dist/umd/3x.min.js"></script>
<script src="https://cdn.jsdelivr.net/npm/@upscalerjs/esrgan-thick@latest/dist/umd/4x.min.js"></script>
<script src="https://cdn.jsdelivr.net/npm/@upscalerjs/esrgan-thick@latest/dist/umd/8x.min.js"></script>

Performance + Speed Measurements​

  • Note: Speed measurements are not recorded for the 8x model. Speed measurements are performed against mobile devices in order to measure against fixed hardware, and the 8x model does not work reliably in a browser.

Architecture​

This model is trained via a Python implementation of the ESRGAN architecture. The Python repo has instructions on training from scratch.

Training Details​

The model is trained on 4 scales.

The model is trained on the Div2k dataset.

It was trained for 500 epochs, with the following hyperparameters:

  • architecture: rrdn
  • C: 4
  • D: 3
  • G: 32
  • G0: 64
  • T: 10

The batch size was 12, and the batches per epoch was 20. The learning rate was set to 0.0004. The HR patch size was set to 128 or 129 depending on the scale (ensuring it is divisible by the scale) with the LR patch size being the resultant scale HR_patch_size / scale.

License​

MIT License © Kevin Scott

The original ESRGAN repository is licensed under an Apache License 2.0