# SIHR: a MATLAB/GNU Octave toolbox for single image highlight removal
[![DOI](https://zenodo.org/badge/156116891.svg)](https://zenodo.org/badge/latestdoi/156116891)
[![license](https://img.shields.io/github/license/vitorsr/SIHR)](https://github.com/vitorsr/SIHR/blob/master/LICENSE)
## Summary
An ongoing effort of developing new and implementing established single image highlight removal (SIHR) methods on MATLAB/GNU Octave.
Highlight, specularity, or specular reflection removal (see <sup>1</sup> for a proper Web of Science expression, see [1] for a reference work entry, see [2], [3] for survey on this problem) concerns the following decomposition.
[![Dichromatic Reflection Model](paper/figures/drm.jpg)](https://link.springer.com/referenceworkentry/10.1007%2F978-0-387-31439-6_532)
I welcome and encourage contributions to this project upon review. Please check [`CONTRIBUTING.md`](CONTRIBUTING.md) for more details.
*Disclaimer 1: this repository is intended for research purposes only.*
*Disclaimer 2: ~~some~~ most of these methods are based on chromaticity analysis, they fail **miserably** for grayscale images.*
<sup>1</sup> `((remov* NEAR/1 (highlight* OR specular*)) OR (separat* NEAR/1 (reflect* OR specular*)))`
### *Raison d'être*
I started out this repository by implementing, translating and collecting code snippets from the *rare* available<sup>2,3,4,5</sup> codes. Oftentimes papers are cryptical, codes are in C/C++ (requires compilation and major source code modification for general testing), or are just unavailable. See, e.g. this CSDN post<sup>6</sup> that has no valid links at all.
In this context, this repository aims to be a continous algorithmic aid for ongoing research and development of SIHR methods.
<sup>2</sup> Tan and Ikeuchi. [Online]. Available: <http://tanrobby.github.io/code/highlight.zip>
<sup>3</sup> Shen *et al.* [Online]. Available: <http://ivlab.org/publications/PR2008_code.zip>
<sup>4</sup> ~~Yang *et al.* [Online]. Available: <http://www6.cityu.edu.hk/stfprofile/qiyang.htm>~~
<sup>5</sup> Shen and Zheng. [Online]. Available: <http://ivlab.org/publications/AO2013_code.zip>
<sup>6</sup> ~~https://blog.csdn.net/nvidiacuda/article/details/8078167~~
## Usage
Calling this toolbox's functions is **very** straightforward:
```MATLAB
I_d = AuthorYEAR(I); % I is a double-valued input image of dimension
% M×N×3 containing linear RGB values and
% I_d is the calculated diffuse reflection
% using AuthorYEAR method.
% The specular component is simply
% I_s = I - I_d;
```
### Methods
The following methods are available.
|Year| Method | Function |
|:--:|----------------------------|----------------|
|2005| Tan and Ikeuchi [4] | `Tan2005` |
|2006| Yoon *et al.* [5] | `Yoon2006` |
|2008| Shen *et al.* [6] | `Shen2008` |
|2009| Shen and Cai [7] | `Shen2009` |
|2010| Yang *et al.* [8] | `Yang2010` |
|2013| Shen and Zheng [9] | `Shen2013` |
|2016| Akashi and Okatani [10] | `Akashi2016` |
The following improvement is available.
|Year| Method | Function |
|:--:|----------------------------|----------------|
|2019| Yamamoto and Nakazawa [11] | `Yamamoto2019` |
### Environment
The environment this repository is being developed is:
* [MATLAB](https://www.mathworks.com/products/matlab.html)
* [Image Processing Toolbox](https://www.mathworks.com/products/image.html)
* [GNU Octave](https://www.gnu.org/software/octave/)
* [Image package](https://octave.sourceforge.io/image/index.html)
* `pkg install -forge image`
#### Tested environments
Octave 4.2 Ubuntu 18.04
Octave 5.1 (latest) Windows 10 1903
MATLAB 9.1 (R2016b) Windows 10 1903
MATLAB 9.6 (R2019a) Windows 10 1903 Ubuntu 16.04 (MATLAB Online)
### Instalation
1. [`git clone`](https://github.com/vitorsr/SIHR.git) or [download](https://github.com/vitorsr/SIHR/archive/master.zip) the repository.
1. Start Octave or MATLAB.
1. `cd('path/to/SIHR')` i.e. change current folder to `SIHR` root (where `SIHR.m` is located).
1. `run SIHR.m` for session path setup.
1. `help SIHR` or `doc SIHR` provides a summary of the methods available.
## Performance
This section aims to clarify how well (or not) the methods reproduced in this project were at reproducing results in literature.
Note: Akashi and Okatani's [10] method has highly fluctuating results because of random initialization.
### Dataset
In technical literature, there exist two ground truth datasets commonly used right now. One by Shen and Zheng [9] which is distributed alongside their code, and one by Grosse *et al.* [12] in a dedicated page<sup>7</sup>.
Other test images are included alongside the code for Shen *et al.* [6] and Yang *et al.* [8].
Follow the instructions in [`images`](https://github.com/vitorsr/SIHR/tree/master/images) in order to download a local copy of these images from the respective authors' pages.
<sup>7</sup> Grosse *et al.* [Online]. Available: <http://www.cs.toronto.edu/~rgrosse/intrinsic/>
### Quality
Quantitative results reported are usually regarding the quality of the recovered diffuse component with respect to the ground truth available in the Shen and Zheng [9] test image set.
Reproduced results below are available in the [`utils/my_quality.m`](https://github.com/vitorsr/SIHR/blob/master/utils/my_quality.m) script.
#### Highest (self and peer-reported | reproduced) PSNR results (in dB)
|Year| Method | *animals* | *cups* | *fruit* | *masks* | Reproduced | *animals* | *cups* | *fruit* | *masks* |
|:--:|--------------------|:---------:|:---------:|:---------:|:---------:|--------------|:---------:|:------:|:-------:|:-------:|
|2005| Tan and Ikeuchi | 30.2 | 30.1 | 29.6 | 25.6 | `Tan2005` | 30.4 | 31.6 | 30.4 | 25.8 |
|2006| Yoon *et al.* | - | - | - | - | `Yoon2006` | 32.9 | 33.3 | 36.6 | 34.1 |
|2008| Shen *et al.* | 34.6 | 37.7 | 37.6 | 31.7 | `Shen2008` | 34.2 | 37.5 | 38.0 | 32.1 |
|2009| Shen and Cai | 34.8 | 37.6 | 36.9 | 34.0 | `Shen2009` | 34.9 | 37.6 | 36.7 | 34.0 |
|2010| Yang *et al.* | *37.2* | 38.0 | 37.6 | 32.2 | `Yang2010` | 36.5 | 37.5 | 36.2 | 33.5 |
|2013| Shen and Zheng | **37.3** | **39.3** | *38.9* | 34.1 | `Shen2013` | 37.5 | 38.3 | 38.2 | 32.7 |
|2015| Liu *et al.* | 33.4 | 37.6 | 35.1 | **34.5** | - | - | - | - | - |
|2016| Akashi and Okatani | 26.8 | 35.7 | 30.8 | 32.3 | `Akashi2016` | 32.7 | 35.9 | 34.8 | 34.0 |
|2016| Suo *et al.* | - | - | **40.4** | 34.2 | - | - | - | - | - |
|2017| Ren *et al.* | - | 38.0 | 37.7 | **34.5** | - | - | - | - | - |
|2018| Guo *et al.* | 35.7 | *39.1* | 36.4 | *34.4* | - | - | - | - | - |
#### Highest (self and peer-reported | reproduced) SSIM results
|Year| Method | *animals* | *cups* | *fruit* | *masks* | Reproduced | *animals* | *cups* | *fruit* | *masks* |
|:--:|--------------------|:---------:|:---------:|:---------:|:---------:|--------------|:---------:|:------:|:-------:|:-------:|
|2005| Tan and Ikeuchi | 0.929 | 0.767 | 0.912 | 0.789 | `Tan2005` | 0.928 | 0.895 | 0.907 | 0.821 |
|2006| Y