The following solutions helps to find duplicate image file for each product and to remove it.
Create file delete.php in Magento base directory, put the following code into the file.

<?php
include('app/Mage.php');
Mage::app()->setCurrentStore(Mage_Core_Model_App::ADMIN_STORE_ID);
error_reporting(E_ALL | E_STRICT);
ini_set('display_errors', 1);
ob_implicit_flush(1);

$mediaApi  = Mage::getModel("catalog/product_attribute_media_api");
$_products = Mage::getModel('catalog/product')->getCollection();
$i         = 0;
$total     = count($_products);
$count     = 0;
foreach ($_products as $_prod) {
    $_product   = Mage::getModel('catalog/product')->load($_prod->getId());
    $_md5Values = array();

    //protected base image
    $baseImage = $_product->getImage();
    if ($baseImage != 'no_selection') {
        $filePath = Mage::getBaseDir('media') . '/catalog/product' . $baseImage;
        if (file_exists($filePath)) {
            $_md5Values[] = md5(file_get_contents($filePath));
        }
    }

    $i++;
    echo "processing product $i of $total " . PHP_EOL;

    // Loop through product images
    $_images = $_product->getMediaGalleryImages();
    if ($_images) {
        foreach ($_images as $_image) {
            //protected base image
            if ($_image->getFile() == $baseImage) {
                continue;
            }

            $filePath = Mage::getBaseDir('media') . '/catalog/product' . $_image->getFile();
            if (!file_exists($filePath)) {
                continue;
            }
            $md5 = md5(file_get_contents($filePath));
            if (in_array($md5, $_md5Values)) {
                $mediaApi->remove($_product->getId(), $_image->getFile());
                echo 'removed duplicate image from ' . $_product->getSku() . PHP_EOL;
                $count++;
            } else {
                $_md5Values[] = $md5;
            }
        }
    }
}

To launch the script just launch in terminal

>php delete.php

Or visit

http://YourWebsiteURL/delete.php

from your browser.

Thanks to Aadil for a solution.