Histogram equalisation is a quick way to enhance contrast and bring out detail in an otherwise flat image by both stretching and flattening an image's histogram. The operation is available in both vImage and Metal Performance Shaders (MPS). The technique for the latter is a little bit tricky, so this post demonstrates how to execute an equalisation using MPS with a little companion project.
Histogram Equalisation in Practice
Given this rather flat image of some clouds with the majority of pixels are bunched towards lighter tones:
Equalising the histogram evens out the tonal distribution across the entire available range and flattens the curve slightly.
Implementing with Metal Performance Shaders
Our first step is some basic setup: we need a device, which is the interface to the GPU, and a command queue and command buffer:
let device = MTLCreateSystemDefaultDevice()!
let commandQueue = device.newCommandQueue()
let commandBuffer = commandQueue.commandBuffer()
If our source image of the clouds, inputImage, is provided as a CIImage, our next step is the create a Metal texture that represents it. We can do this by creating a Core Image context that uses the Metal device and calling its render function to draw to a texture created by the device:
let textureDescriptor = MTLTextureDescriptor.texture2DDescriptorWithPixelFormat(
.RGBA8Unorm,
width: Int(inputImage.extent.width),
height: Int(inputImage.extent.height),
mipmapped: false)
let colorSpace = CGColorSpaceCreateDeviceRGB()!
let inputTexture = device.newTextureWithDescriptor(textureDescriptor)
let destinationTexture = device.newTextureWithDescriptor(textureDescriptor)
let ciContext = CIContext(MTLDevice: device)
ciContext.render(
inputImage,
toMTLTexture: inputTexture,
commandBuffer: commandBuffer,
bounds: inputImage.extent,
colorSpace: colorSpace)
Now to start the equalisation process. We need to calculate the histogram of inputImage. MPSImageHistogram requires an empty MPSImageHistogramInfo instance:
var histogramInfo = MPSImageHistogramInfo(
numberOfHistogramEntries: 256,
histogramForAlpha: true,
minPixelValue: vector_float4(0,0,0,0),
maxPixelValue: vector_float4(1,1,1,1))
let calculation = MPSImageHistogram(
device: device,
histogramInfo: &histogramInfo)
The histogram info object is wrapped in a Metal buffer and the MPSImageHistogram can be encoded to the command buffer:
let histogramInfoBuffer = device.newBufferWithBytes(
&histogramInfo,
length: sizeofValue(histogramInfo),
options: MTLResourceOptions.CPUCacheModeDefaultCache)
calculation.encodeToCommandBuffer(
commandBuffer,
sourceTexture: inputTexture,
histogram: histogramInfoBuffer,
histogramOffset: 0)
Now that the MPSImageHistogramInfo is populated with the image's histogram, it can be equalised. The histogram info object used to instantiate an MPSImageHistogramEqualization which has its transform function encoded to the command buffer before being encoded itself:
let equalization = MPSImageHistogramEqualization(
device: device,
histogramInfo: &histogramInfo)
equalization.encodeTransformToCommandBuffer(
commandBuffer,
sourceTexture: inputTexture,
histogram: histogramInfoBuffer,
histogramOffset: 0)
equalization.encodeToCommandBuffer(
commandBuffer,
sourceTexture: inputTexture,
destinationTexture: destinationTexture)
The final step is to commit the command buffer for execution on the GPU:
commandBuffer.commit()
The final target texture is destinationTexture which can be converted to a CIImage for display with the init(MTLTexture:options) initialiser:
let ciImage = CIImage(
MTLTexture: destinationTexture,
options: [kCIImageColorSpace: colorSpace])
Easy!
The Accelerate v/Image Alternative
You may decide to plump for vImage which does it in one step:
vImageEqualization_ARGB8888(
&imageBuffer,
&outBuffer,
UInt32(kvImageNoFlags))
...and the demo project includes an example of that.
Equalisation versus Contrast Stretch
Sadly, Metal Performance Shaders doesn't include contrast stretch. Contrast stretch is not dissimilar to equalisation, but doesn't attempt to flatten the curve shape, it simply stretches it to the full value range. vImage, however, does include contrast stretching. Here's a vImage equalisation (pretty much identical to its MPS equivalent):
...and here's vImage contrast stretching:
Although Core Image doesn't have these histogram operations, it can go some way to mimicking them with the tone curve filter generated by auto adjustment filters:
guardlet filter = inputImage
.autoAdjustmentFiltersWithOptions(nil)
.filter({ $0.name == "CIToneCurve"})
.firstelse
{
return
}
filter.setValue(inputImage, forKey: kCIInputImageKey)
Which yields:
Core Image for Swift
Version 1.3 of my book, Core Image for Swift, is due out very shortly which explores histogram operations with vImage in more detail.Core Image for Swift is available from both Apple's iBooks Store or, as a PDF, from Gumroad. IMHO, the iBooks version is better, especially as it contains video assets which the PDF version doesn't.
