Context

Image semantic matching is a sub-field of computer vision which aims to extract and identify semantic correlation between parts of images. In layman terms, image semantic matching algorithms can determine parts of different images that represent the same feature, even with some visual variations. For instance, imagine two images of dogs with different breed, coat colour and orientation. A robust semantic matching algorithm will be able to match the locations of the same features (i.e. ears, eyes, tail, etc.) of the two dogs. Semantic matching algorithms have practical applications in autonomous vehicles and robotics, enabling better perception and understanding of surroundings. Semantic matching also facilitates for more advanced image edit and manipulation. 

Aim

The research project aims to develop a robust semantic matching algorithm, where the model could could have an increase in matching accuracy compared with previous state-of-the-art (SOTA) models. Lower computational power to achieve robustness is also preferable. 

Method

In this research, previous SOTA models will be recreated locally and tested with popular semantic correspondence datasets. The advantages and issues of those SOTA models will then be explored and considered. Based on the findings, issues in current SOTA models will be analyzed. Novel methods will then be suggested to solve such issues while retaining the advantages of previous SOTA models. The novel methods will be trained and tested with the datasets mentioned previously. The results will be compared against that from previous SOTA models to see whether there are any improvements in the accuracy on matching corresponding feature points between various images. If there are no significant improvements on the novel method, other methods will then be suggested and the process repeats.