Home Hardie et al. [1] He and Kondi [2] Farsiu et al. [3] Sroubek and Flusser [4]

Accurate image registration for MAP image super-resolution

Below is shown a complete list of the experimental results for our implemenetation. The results are also compared with the Hardie et al. [1], He and Kondi [2], Farsiu et al. [3] and Sroubek and Flusser [4] methods. The first row contains the ground truth images (obtained from the USC-SIPI database [5]), while the rest rows contain the reconstructed High-Resolution images applying different types of registration methods (SIFT, SIFT & MI, SURF, SURF & MI, Harris and Harris & MI).

Ground truth images

clock_256

Clock

boat_256

Boat

eyechart_256

Eyechart

lena_numbers_256

Numbered-Lena

  

car_256

Car
Reconstructed High-Resolution images for He and Kondi [2] wtih respect to the registration method
Reconstructed HR images for the 256 × 256 Clock sequence (obtained from the USC-SIPI database [5]) using the method of He and Kondi [2] applying different types of registration methods.

SIFT

SIFT & MI

SURF

SURF & MI

Harris

Harris & MI
Reconstructed HR images for the 256 × 256 Boat sequence (obtained from the USC-SIPI database [5]) using the method of He and Kondi [2] applying different types of registration methods.

SIFT

SIFT & MI

SURF

SURF & MI

Harris

Harris & MI
Reconstructed HR images for the 256 × 256 Eyechart sequence (obtained from the USC-SIPI database [5]) using the method of He and Kondi [2] applying different types of registration methods.

SIFT

SIFT & MI

SURF

SURF & MI

Harris

Harris & MI
Reconstructed HR images for the 256 × 256 Numbered-Lena sequence (obtained from the USC-SIPI database [5]) using the method of He and Kondi [2] applying different types of registration methods.

SIFT

SIFT & MI

SURF

SURF & MI

Harris

Harris & MI

SIFT

SIFT & MI

SURF

SURF & MI

Harris

Harris & MI
Reconstructed HR images for the 256 × 190 Car sequence using the method of He and Kondi [2] applying different types of registration methods.

Numerical results for He and Kondi [2]

The numerical results are summarized in Table I showing the PSNR, and Table II showing the ISNR for 35 dB, 30 dB and 25 dB degradation noise, where the mean values, the standard deviations and the median values of the PSNR and ISNR for the reconstructed HR images above, are presented. These values are obtained through 15 random realizations of the experiment using different transformation parameters and noise realizations.
Table I

PSNR statistics (in dB) for the He and Kondi [2] Super-Resolution method.

PSNR Registration Meethod Clock Boat Eyechart Numberd-Lena Car
mean std median mean std median mean std median mean std median mean std median
35 dB Gaussian noise SIFT 24.93 0.47 24.91 22.81 0.68 22.76 21.38 0.16 21.92 21.59 0.39 21.53 24.40 0.76 24.14
SIFT & MI 25.80 0.10 25.61 23.53 0.71 23.10 24.72 0.09 24.89 22.47 0.33 22.43 25.05 0.40 25.12
SURF 24.88 0.76 25.19 22.11 0.67 22.77 21.63 0.89 22.01 21.98 0.37 21.88 23.74 0.78 23.63
SURF & MI 25.84 0.11 25.54 23.00 0.48 23.10 23.80 0.25 24.08 22.97 0.37 22.90 23.81 0.74 23.45
Harris 23.57 0.18 23.43 22.29 0.82 22.59 22.55 0.38 22.77 22.10 0.26 22.18 22.77 0.13 22.77
Harris & MI 25.13 0.22 25.07 24.45 0.59 24.58 24.14 0.19 24.23 22.24 0.13 22.40 26.00 0.26 26.53
30 dB Gaussian noise SIFT 22.96 0.61 23.21 22.32 0.73 22.14 21.03 0.85 20.59 21.64 0.15 21.69 24.10 0.77 23.97
SIFT & MI 24.42 0.21 24.80 22.80 0.61 22.98 22.98 0.19 22.22 21.69 0.28 21.75 23.74 0.19 23.46
SURF 23.32 0.28 23.89 21.81 0.73 21.96 21.05 0.57 21.02 21.94 0.37 22.05 23.18 0.66 23.15
SURF & MI 23.07 0.84 23.75 22.32 0.51 22.54 22.69 0.13 22.40 21.96 0.14 21.91 23.52 0.23 23.06
Harris 21.11 0.26 21.09 21.89 0.85 21.94 21.63 0.12 21.48 21.06 0.03 21.03 22.67 0.29 22.76
Harris & MI 23.99 0.34 23.38 24.15 0.74 24.19 23.51 0.54 23.98 21.54 0.74 21.86 25.08 0.72 25.05
25 dB Gaussian noise SIFT 22.48 0.26 22.37 22.14 0.73 21.99 20.51 0.81 20.35 20.70 0.42 20.76 23.60 0.93 23.23
SIFT & MI 22.54 0.14 22.03 22.69 0.16 22.61 22.12 0.19 22.01 21.61 0.57 21.57 23.82 0.17 23.74
SURF 22.34 0.21 22.35 21.37 0.87 21.62 21.66 0.04 21.62 20.87 0.35 21.80 23.52 0.50 23.38
SURF & MI 22.76 0.04 22.79 22.55 0.33 22.36 22.39 0.79 22.52 21.97 0.41 21.96 23.74 0.21 23.65
Harris 21.34 0.64 21.21 21.15 0.68 20.82 21.38 0.17 21.46 20.76 0.19 20.59 22.42 0.19 22.86
Harris & MI 22.73 0.59 22.65 22.95 0.31 22.83 22.57 0.14 22.06 21.37 0.30 21.48 24.55 0.53 24.85

 

In Table I it may be seen that the combination of feature-based initialization of the registration parameters followed by fine tuning by the maximization of the mutual information criterion provides consistently higher accuracy. The PSNR values in bold indicate the best quality reconstructed images with respect to the registration method (along columns).
Table II

ISNR statistics (in dB) for the He and Kondi [2] Super-Resolution method. Baseline is the SURF-based registration method.

ISNR Registration Meethod Clock Boat Eyechart Numberd-Lena Car
mean std median mean std median mean std median mean std median mean std median
35 dB Gaussian noise SIFT 1.47 0.17 1.38 1.99 0.48 1.43 1.75 0.43 1.65 1.48 0.32 1.31 1.56 0.72 1.84
SIFT & MI 1.40 0.14 1.45 1.53 0.61 1.37 2.09 0.17 2.32 1.62 0.41 1.50 1.62 0.72 1.45
SURF - - - - - - - - - - - - - - -
SURF & MI 1.39 0.37 1.47 1.84 0.53 1.63 2.46 0.75 2.25 1.50 0.37 1.45 1.49 0.77 1.79
Harris 1.97 0.28 1.77 1.74 0.26 1.67 2.02 0.86 1.87 1.74 0.86 1.57 1.71 0.47 1.64
Harris & MI 1.71 0.17 1.77 1.39 0.82 1.21 3.80 0.11 3.39 1.76 0.19 1.96 2.57 0.10 2.58
30 dB Gaussian noise SIFT 1.61 0.64 1.48 1.55 0.18 1.35 2.77 0.27 2.54 1.96 0.56 1.54 1.67 0.77 1.87
SIFT & MI 1.97 0.21 1.61 1.79 0.60 1.62 3.32 0.14 2.80 1.21 0.30 1.17 1.45 0.18 1.17
SURF - - - - - - - - - - - - - - -
SURF & MI 1.92 0.33 1.55 1.43 0.29 1.19 3.03 0.87 2.55 1.94 0.48 1.35 1.79 0.19 1.49
Harris 1.75 0.11 1.73 1.75 0.17 1.86 1.62 0.31 1.80 1.86 0.53 1.99 1.76 0.18 1.82
Harris & MI 1.73 0.31 1.79 1.35 0.35 1.22 3.85 0.46 3.74 1.11 0.74 1.77 1.79 0.18 2.15
25 dB Gaussian noise SIFT 1.77 0.43 1.92 1.39 0.61 1.69 3.18 0.07 3.25 1.69 0.47 1.63 1.35 0.28 1.13
SIFT & MI 1.78 0.21 1.86 1.17 0.75 1.13 1.65 0.55 1.56 1.44 0.27 1.32 1.66 0.76 1.36
SURF - - - - - - - - - - - - - - -
SURF & MI 1.75 0.18 1.78 2.94 0.80 1.70 1.60 0.42 1.53 1.76 0.01 1.76 1.94 0.42 1.83
Harris 1.69 0.24 1.70 1.94 0.17 2.02 2.04 0.67 2.25 1.69 0.65 1.53 1.40 0.18 1.21
Harris & MI 1.61 0.16 1.52 1.54 0.40 1.72 2.88 0.89 3.07 1.84 0.58 1.76 2.10 0.88 2.00

 

The results in Table II show the ISNR statistics for the compared SR methods. Registration using SURF was taken to be the reference method for computing the ISNR value. The ISNR values in bold indicate the best performance with respect to the registration method (along columns).

REFERENCES

[1] M. Vrigkas, C. Nikou and L. P. Kondi, “Robust maximum a posteriori image super-resolution,” Journal of Electronic Imaging, vol. 23, no. 4, pp. 043016, July 2014.
[2] M. Vrigkas, C. Nikou and L. P. Kondi, “Accurate image registration for MAP image super-resolution,”Signal Processing: Image Communication, vol. 28, no. 5, pp. 494-508, May 2013
[3] M. Vrigkas, C. Nikou and L. P. Kondi, “A fully robust framework for MAP image super-resolution,” in Proc. IEEE International Conference on Image Processing, pp. 2225-2228, Orlando, FL, September 30-October 3 2012.
[4] M. Vrigkas, C. Nikou and L. P. Kondi, “On the improvement of image registration for high accuracy super-resolution,” in Proc. IEEE International Conference on Acoustics, Speech and Signal Processing, pp. 981-984, Prague, Czech Republic, May 22-27 2011.
[5] R. C. Hardie, K. J. Barnard, and E. E. Armstrong, “Joint MAP image registration and high-resolution image estimation using a sequence of undersampled images,” IEEE Transactions on Image Processing, vol. 6, no. 12, pp. 1621–1633, December 1997.
[6] H. He and L. P. Kondi, “Resolution enhancement of video sequences with simultaneous estimation of the regularization parameter,” SPIE Journal of Electronic Imaging, vol. 13, no. 3, pp. 586–596, 2004.
[7] S. Farsiu, M. Elad, and P. Milanfar, “Multi-frame demosaicing and super-resolution of color images,” IEEE Trans. on Image Processing, vol. 15, pp. 141–159, 2006.
[8] F. Sroubek and J. Flusser, “Resolution enhancement via probabilistic deconvolution of multiple degraded images,” Pattern Recognition Letters, vol. 27, no. 4, pp. 287–293, 2006.
[9] The USC-SIPI image database,” 1977, http://sipi.usc.edu/database/

Contact

University of Ioannina
Department of Computer Science & Engineering
P.O. BOX 1186
GR 45110 Ioannina, Greece

Telephone: +30 265 100 8915
Office: B6
E-mail: m v r i g k a s /at\ c s e \dot\ u o i /dot/ g r