Research

My research is mainly focused on developing a metric called Structural Texture Similarity Metric. This metric aims to compare two texture patches in accordance to human perception.

“Texture” is a term that can relate to various excitements from different senses in humans. The most common notion is the tactile sensation, e.g., it describes the difference between smooth vs. rough. One can also talk about texture in music, when it refers to the overall sound, and how multiple voices (or instruments) interact in a composition. As for the visual texture, we are yet to find a definition the majority would agree upon, but the most widely accepted description is that it is an image, consisting of repeating parts, whose placement and characteristics (shape, size, color, orientation etc.) can be randomized, but that appears spatially homogeneous. Trying to meaningfully quantify something that humans have troubles explaining with words is an ever challenging problem and a highly motivating topic for research.

Image compression, as we usually understand it, means representing the original image with fewer bits, with or without distortions, but remaining (somewhat) faithful to the actual signal content, i.e., the pixel values. However, we may be interested in looking at the higher-level story: for example, if you see a zebra in a prairie on one image, how differently do you perceive some other zebra in some other prairie, on another image? This question of semantic perception of images is important in many applications. To be able to meaningfully assess the similarity of images on a semantic level, we need to develop similarity metrics that would be targeted towards comparing regions, rather than pixel-by-pixel values. Textures are a particularly challenging task, since they can have very different pixel representations, yet still be perceived by human observers as “same textures” (e.g., two different samples of zebra fur). The metric that would compare the textures on this level is useful for image annotation (comparing to the existing, pre-annotated set of images), compression (e.g., the grass on the football field), also for image retrieval, quality control (e.g., fabrics, marbles), for medical applications (automatic segmentation and labeling of medical images) and so on.

Subjective Tests

Similarity clusters, Windows: [clusterTest_Win.zip]

Similarity clusters, Mac: [clusterTest_MacOS.zip]

(Apologies to the Linux users, I don't have Linux versions available.)