Utilizing sketches to detect objects within images, AI has the potential to significantly enhance tumor detection and facilitate the search for elusive bird species.

 

Leading researchers from the University of Surrey have developed a ground breaking sketch-based object detection tool that could revolutionize cancer detection and other fields. The tool, which will be showcased at the Computer Vision, Pattern, and Recognition Conference (CVPR), enables machine learning systems to detect specific objects in images based on user sketches while discounting more general options.

Professor Yi-Zhe Song, who heads the research at the University of Surrey's Institute for People-Centred AI, highlighted the uniqueness of the tool, stating that traditional descriptive words cannot capture the individualism and specificity that sketches convey. The AI system learns from the sketches provided by the user to precisely locate the desired object while disregarding others. This capability has promising applications in medical diagnostics, where it could assist in identifying aggressive tumors, as well as in wildlife conservation efforts by detecting rare animals.

An example presented by the researchers involves using the tool to search for a single zebra within an image crowded with zebras, based solely on a sketch of a zebra eating. The AI system takes into account visual cues such as pose and structure but relies on the exact instructions given by the amateur artist to guide its search.

Professor Song emphasized the significance of this sketch-based approach, enabling AI to leverage human creativity in computer vision. Instead of AI dictating decisions, humans can now interact with AI from a different perspective, instructing it to behave exactly as directed while retaining the necessary level of human intervention.

The University of Surrey, known for its research-intensive environment, has a strong presence at the CVPR 2023, with an exceptional number of accepted papers, including one nominated for the Best Paper Award. The university's Institute for People-Centred AI combines decades of technical expertise in machine learning with multidisciplinary research to address the technical, ethical, and governance aspects necessary for AI to truly prioritize human-centered applications.

The research conducted at the University of Surrey aligns with its mission to produce world-leading research that positively impacts society and contributes to sustainable development. This commitment has led to the university's high ranking in the Times Higher Education (THE) University Impact Rankings 2022, where it placed 55th globally, reflecting its performance in advancing the United Nations' Sustainable Development Goals (SDGs).