MAXIM Denoising

Version :Last Updated :NPM installs per week :CDN hits per week :

MAXIM Denoising is a Tensorflow.js model for denoising images.

The models were converted from weights provided by the original MAXIM paper and repository. More information on the conversion process can be found in this repository.

Paper

The MAXIM paper (MAXIM: Multi-Axis MLP for Image Processing) was published in 2022, and was nominated as one of the best papers at CVPR 2022. The MAXIM architecture is capable, via training, of supporting a variety of image enhancement tasks. It's also an efficient architecture, making it particularly appealing for JavaScript applications.

Abstract:

Recent progress on Transformers and multi-layer perceptron (MLP) models provide new network architectural designs for computer vision tasks. Although these models proved to be effective in many vision tasks such as image recognition, there remain challenges in adapting them for low-level vision. The inflexibility to support high-resolution images and limitations of local attention are perhaps the main bottlenecks. In this work, we present a multi-axis MLP based architecture called MAXIM, that can serve as an efficient and flexible general-purpose vision backbone for image processing tasks. MAXIM uses a UNet-shaped hierarchical structure and supports long-range interactions enabled by spatially-gated MLPs. Specifically, MAXIM contains two MLP-based building blocks: a multi-axis gated MLP that allows for efficient and scalable spatial mixing of local and global visual cues, and a cross-gating block, an alternative to cross-attention, which accounts for cross-feature conditioning. Both these modules are exclusively based on MLPs, but also benefit from being both global and `fully-convolutional', two properties that are desirable for image processing. Our extensive experimental results show that the proposed MAXIM model achieves state-of-the-art performance on more than ten benchmarks across a range of image processing tasks, including denoising, deblurring, deraining, dehazing, and enhancement while requiring fewer or comparable numbers of parameters and FLOPs than competitive models.

— MAXIM: Multi-Axis MLP for Image Processing

Sample Enhanced Image

Original

Enhanced

Demo

Installation

npm install @upscalerjs/maxim-denoising

Usage

Browser

Using a transpiler

If using a transpiler (such as tsc, webpack, or vite) import the model with:

import Upscaler from 'upscaler';
import model from '@upscalerjs/maxim-denoising'

const upscaler = new Upscaler({
  model,
})

Using a script tag

If importing Tensorflow.js using script tags, import the model and UpscalerJS with:

<script src="https://cdn.jsdelivr.net/npm/@upscalerjs/maxim-denoising@latest/dist/umd/index.min.js"></script>
<script src="https://cdn.jsdelivr.net/npm/upscaler@latest/dist/browser/umd/upscaler.min.js"></script>

<script type="text/javascript">
  const upscaler = new Upscaler({
    model: UpscalerJSMaximDenoising,
  })
</script>

Node

Require the model with:

const Upscaler = require('upscaler/node'); // if using @tensorflow/tfjs-node-gpu, change this to upscaler/node-gpu
const model = require('@upscalerjs/maxim-denoising');

const upscaler = new Upscaler({
  model,
})

The model will work for both node and node-gpu flavors of Tensorflow.js.

Model Details

This model is unquantized and has a dynamic input size. Image sizes must be a multiple of 64. UpscalerJS will automatically take care of padding and trimming images.

If providing a patch size, ensure that the patch size is a multiple of 64. If not, UpscalerJS will emit a warning and the patch size will be increased to the nearest multiple of 64 for compatibility.

Additional Models

There are two additional fixed input size models that are derived from a Tensorflow implementation of MAXIM:

• @upscalerjs/maxim-denoising/64 - quantized uint8, input size of 64
• @upscalerjs/maxim-denoising/256 - quantized uint8, input size of 256

These models run considerably faster but exhibit inferior inference capabilities.

To test these models, clone the UpscalerJS repo, pull the models with dvc pull and inspect the models/maxim-experiments folder.

Dataset

All weights were trained on the SIDD.

License

MIT License © Kevin Scott

The original MAXIM repository is licensed under an Apache License 2.0