Intuition Behind Histogram of Oriented Gradients (HOG)

Histogram of Oriented Gradients (HOG) is a feature descriptor used in computer vision and image processing for the purpose of object detection. The intuition behind HOG is that the local object appearance and shape within an image can be characterized by the distribution of intensity gradients or edge directions.

Gradients and Edge Detection:

The first step in HOG is to compute the gradient of the image. The gradient at each pixel in the image is a vector that points in the direction of the greatest rate of change in intensity.
The gradient captures the edge information of the image, highlighting areas with significant intensity change, which typically correspond to object boundaries.

Orientation Binning

The image is divided into small connected regions called cells (e.g., 8x8 pixels).
For each cell, a histogram of gradient directions (or orientations) is computed. This histogram represents the distribution of gradient directions within the cell.
The orientations are binned into a fixed number of bins (e.g., 9 bins covering 0 to 180 degrees).

Local Contrast Normalization:

To make the feature descriptor more robust to changes in illumination and contrast, the histograms of cells are normalized.
Normalization is performed over larger spatial regions called blocks (e.g., 2x2 cells). The histograms of the cells within the block are concatenated and then normalized.
This step helps in reducing the effect of variations in lighting and improves the descriptor's invariance to contrast changes.

Feature Vector:

The final HOG descriptor for the image is obtained by concatenating the normalized histograms of all the blocks in the image.
This results in a high-dimensional feature vector that captures the local gradient structure and can be used for further tasks such as object detection or classification.

import matplotlib.pyplot as plt
import cv2

# Load the black and white image
image_path = '5.png'  # Replace with your image path
image = cv2.imread(image_path, cv2.IMREAD_GRAYSCALE)

# Resize the image to 32x16
resized_image = cv2.resize(image, (50, 48))

# Display the resized image
plt.imshow(resized_image, cmap='gray')
plt.title('Resized Image (32x16)')
plt.axis('off')  # Hide axis
plt.show()

# Print the pixel values of the resized image
print("Pixel values of the resized image:")
for i in range(resized_image.shape[0]):
    for j in range(resized_image.shape[1]):
        print(resized_image[i, j], end=" ")
    print()

Pixel values of the resized image:
255 255 255 255 255 255 255 255 255 255 255 255 255 255 255 255 255 248 237 239 250 255 255 255 255 255 255 255 255 255 255 255 255 255 255 255 255 255 255 255 255 255 255 255 255 255 255 255 255 255 
94 94 94 94 94 94 94 94 94 94 94 94 94 94 94 94 94 94 94 94 94 94 94 94 94 94 94 94 94 94 94 94 94 94 94 94 94 94 94 94 94 94 94 94 94 94 94 94 94 94 
0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 
0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 
0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 
0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 
0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 
0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 
0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 
0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 
0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 
0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 2 4 4 4 7 7 7 7 7 7 7 7 7 3 2 0 0 0 
0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 81 124 124 124 236 246 246 246 246 246 246 246 246 100 62 16 0 0 
0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 17 87 152 174 200 215 215 215 252 255 255 255 255 255 255 255 255 207 194 52 0 0 
0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 25 128 224 255 255 255 255 255 255 255 255 255 255 255 255 255 255 255 255 69 0 0 
0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 8 42 76 212 244 255 255 255 255 255 255 255 255 255 255 239 212 207 172 171 171 46 0 0 
0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 25 115 143 255 242 237 226 220 220 220 204 202 187 185 185 168 139 132 94 93 93 25 0 0 
0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 54 252 255 255 207 191 149 128 128 128 69 60 4 0 0 0 0 0 0 0 0 0 0 0 
0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 4 11 21 21 2 0 0 48 220 220 212 171 158 124 106 106 106 57 50 4 0 0 0 0 0 0 0 0 0 0 0 
0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 26 67 123 124 12 0 0 13 63 51 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 
0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 19 38 65 105 150 149 15 0 0 11 50 41 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 
0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 97 191 217 255 255 247 25 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 
0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 40 56 153 247 250 248 151 139 14 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 
0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 47 65 162 255 255 247 136 124 12 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 
0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 169 234 245 255 231 187 26 14 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 
0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 184 255 255 255 244 222 120 111 79 59 38 29 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 
0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 184 255 255 255 255 251 195 189 134 100 64 50 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 
0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 127 176 216 255 255 253 232 230 209 197 183 161 80 63 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 
0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 92 128 193 255 255 255 255 255 255 255 255 228 129 102 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 
0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 33 45 68 91 125 173 173 175 247 255 255 245 210 176 41 10 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 
0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 38 94 94 97 193 218 238 242 255 227 116 28 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 
0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 2 57 119 191 204 254 255 255 63 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 
0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 43 103 175 192 254 252 239 59 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 
0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 55 128 155 254 242 191 47 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 
0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 8 64 137 162 254 242 191 47 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 
0 0 0 0 0 0 0 0 0 0 0 1 3 3 4 2 2 1 0 0 0 1 3 62 122 192 205 254 242 189 46 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 
0 0 0 0 0 0 0 0 0 0 0 27 123 131 183 95 62 30 0 0 3 57 67 229 250 252 253 255 230 130 32 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 
0 0 0 0 0 0 0 0 0 0 0 28 128 136 191 99 64 31 0 0 3 59 70 236 255 255 255 255 230 128 31 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 
0 0 0 0 0 0 0 0 0 0 75 139 238 239 246 234 229 225 221 221 221 229 230 252 255 255 243 200 163 17 4 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 
0 0 0 0 0 0 0 0 0 0 141 217 255 255 255 255 255 255 255 255 255 255 255 255 221 177 155 74 59 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 
0 0 0 0 0 0 0 0 0 0 174 255 255 255 255 255 255 255 255 255 255 255 255 255 200 128 101 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 
0 0 0 0 0 0 0 0 0 0 126 200 255 255 255 204 184 202 219 204 179 116 111 111 88 56 44 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 
0 0 0 0 0 0 0 0 0 0 87 156 255 255 255 163 128 160 191 165 120 8 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 
0 0 0 0 0 0 0 0 0 0 37 64 106 106 106 67 53 66 79 68 50 4 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 
0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 
0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 
0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 
0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0

For a black and white image it's moderately easy to predict the digit

But its difficult to predict an animal just based on the pixel values (in this case R+G+B/3) because the surrounding ,posture and other factors might also affect this

Thus the two rabbits may be predicted as different species

For such classifications we use HOC

Why HOG Works ?

Localized Information: HOG captures localized information about the gradient directions, which is crucial for detecting edges and shapes in an image.
Invariant to Illumination: The normalization step makes HOG robust to changes in illumination and contrast, which are common in real-world images.
Rich Descriptors: By capturing the distribution of gradient orientations, HOG provides rich descriptors that are effective for capturing the essential structure of objects.

Now let's take an image

# Load the black and white image
image_path = 'dog.jpeg'
image = cv2.imread(image_path)
plt.imshow(image)
plt.axis('off')  # Hide axis
plt.show()

import cv2
import numpy as np
import matplotlib.pyplot as plt
from skimage.feature import hog

Step 1: Image Preprocessing

The first step is to preprocess the image to a consistent size and convert it to grayscale. Grayscale conversion reduces the complexity by converting the image to a single channel.


image = cv2.imread('dog.jpeg')
# Convert to grayscale
gray_image = cv2.cvtColor(image, cv2.COLOR_BGR2GRAY)

# Display the grayscale image
plt.figure(figsize=(10, 10))
plt.subplot(221)
plt.title('Grayscale Image')
plt.imshow(gray_image, cmap='gray')

<matplotlib.image.AxesImage at 0x7b07975f85b0>

Step 2: Calculate Gradient and Orientation

Gradients represent the change in intensity of the image. They are computed using derivatives in both the x and y directions (Gx and Gy). The magnitude and orientation (angle) of the gradient are then calculated as follows:

Magnitude= (Gx^2 + Gy^2)^1/2

Orientation=arctan (Gy/Gx)

# Calculate the gradients
gx = cv2.Sobel(gray_image, cv2.CV_64F, 1, 0, ksize=3)
gy = cv2.Sobel(gray_image, cv2.CV_64F, 0, 1, ksize=3)

# Calculate gradient magnitude and direction
magnitude, angle = cv2.cartToPolar(gx, gy, angleInDegrees=True)

# Display the gradient magnitude
plt.subplot(222)
plt.title('Gradient Magnitude')
plt.imshow(magnitude, cmap='gray')

# Display the gradient direction
plt.subplot(223)
plt.title('Gradient Direction')
plt.imshow(angle, cmap='gray')

<matplotlib.image.AxesImage at 0x7b079792d060>

Step 3: Build Histogram

Intuition:

The image is divided into small spatial regions (cells), and for each cell, a histogram of gradient orientations is created. The idea is to capture the distribution of edge directions within each cell.

Mathematical Explanation:

Each pixel within a cell votes for an orientation bin based on its gradient magnitude and orientation. The histogram bins typically cover 0 to 180 or 0 to 360 degrees, depending on the implementation.

# Calculate HOG features and HOG image
hog_features, hog_image = hog(gray_image, orientations=9, pixels_per_cell=(8, 8),
                              cells_per_block=(2, 2), block_norm='L2-Hys',
                              visualize=True, transform_sqrt=True)

# Display the HOG image
plt.subplot(224)
plt.title('HOG Image')
plt.imshow(hog_image, cmap='gray')

plt.tight_layout()
plt.show()

This is what from scratch implementation of Building the histogram would look like

# Calculate gradients
grad_x = cv2.Sobel(image, cv2.CV_64F, 1, 0, ksize=3)
grad_y = cv2.Sobel(image, cv2.CV_64F, 0, 1, ksize=3)

# Calculate magnitude and orientation
magnitude = cv2.magnitude(grad_x, grad_y)
orientation = cv2.phase(grad_x, grad_y, angleInDegrees=True)

plt.subplot(1, 2, 1)
plt.imshow(magnitude, cmap='gray')
plt.title('Gradient Magnitude')

plt.subplot(1, 2, 2)
plt.imshow(orientation, cmap='gray')
plt.title('Gradient Orientation')
plt.show()

# Parameters for HOG
cell_size = 8
num_bins = 9
angle_unit = 180 / num_bins

def calculate_histogram(magnitude, orientation, cell_size, num_bins):
    height, width = magnitude.shape
    histogram = np.zeros((height // cell_size, width // cell_size, num_bins))
    for i in range(0, height // cell_size):
        for j in range(0, width // cell_size):
            cell_magnitude = magnitude[i * cell_size:(i + 1) * cell_size, j * cell_size:(j + 1) * cell_size]
            cell_orientation = orientation[i * cell_size:(i + 1) * cell_size, j * cell_size:(j + 1) * cell_size]
            cell_histogram = np.zeros(num_bins)
            for p in range(cell_size):
                for q in range(cell_size):
                    bin_idx = int(cell_orientation[p, q] // angle_unit)
                    # Ensure bin_idx is within the valid range
                    bin_idx = min(bin_idx, num_bins - 1)
                    cell_histogram[bin_idx] += cell_magnitude[p, q]
            histogram[i, j, :] = cell_histogram
    return histogram

histogram = calculate_histogram(magnitude, orientation, cell_size, num_bins)

# Display histogram for a specific cell
cell_hist = histogram[0, 0, :]
plt.bar(range(num_bins), cell_hist)
plt.title('Histogram of Gradients for First Cell')
plt.xlabel('Orientation Bins')
plt.ylabel('Magnitude')
plt.show()

# Checking the histogram values for debugging
print(f"Histogram of first cell: {cell_hist}")
print(f"Sum of histogram values (should be > 0 if non-empty): {np.sum(cell_hist)}")

Applications

HOG descriptors are widely used in various computer vision tasks, including:

Object Detection: HOG was popularized by its success in pedestrian detection, where it outperformed many existing methods.
Image Classification: HOG features are used as input to machine learning algorithms for classifying images into different categories.
Face Recognition: HOG can be used to extract features for recognizing faces or other objects in images.