The application of AI that utilizes sketches to identify objects within images has the potential to enhance tumor detection and aid in the search for rare bird species.

 

Researchers from the University of Surrey have developed a groundbreaking AI tool that uses sketches to teach machine learning algorithms to detect specific objects in images while discounting others. This innovation has the potential to revolutionize cancer detection and aid in wildlife conservation efforts.

The unique sketch-based object detection tool developed by the University of Surrey will be showcased at the prestigious Computer Vision, Pattern, and Recognition Conference (CVPR). With this tool, users can sketch an object, and the AI system will use the sketch as a reference to search for a matching object within an image, while disregarding more general options.

Professor Yi-Zhe Song, leading the research at the University of Surrey's Institute for People-Centred AI, explained the significance of the tool:

"An artist's sketch contains visual cues that cannot be adequately conveyed through words alone, exemplifying the saying 'a picture paints a thousand words.' While current AI systems rely on simple descriptive words to generate images, they fail to capture the uniqueness of the user's vision or the precise match they seek."

He further highlighted the potential applications of the sketch-based tool, particularly in medicine and wildlife conservation:

"Our sketch-based tool allows AI to detect the exact object specified by the user while disregarding others. This capability can be incredibly valuable in medicine, enabling the identification of more aggressive tumors, and in wildlife conservation, facilitating the detection of rare animals."

An example shared by the researchers involves using the tool to search for a single zebra among a picture filled with zebras. By providing a sketch of a zebra eating, the AI tool considers visual cues such as pose and structure to accurately locate the desired object, following the instructions given by the amateur artist.

Professor Song emphasized the significance of AI's ability to detect objects based on individual sketches, enabling a new level of human-AI interaction:

"The integration of human-generated sketches in AI-based object detection represents a significant advancement in leveraging human creativity within the field of Computer Vision. It allows humans to engage with AI from a fresh perspective, no longer letting AI dictate the decisions but instructing it to behave exactly as intended, with the necessary human intervention."

This research will be presented at the CVPR 2023, a renowned conference for showcasing world-leading AI research. The University of Surrey has an impressive number of papers accepted for the event, surpassing other educational institutions, with over 18 papers accepted and one nominated for the Best Paper Award.

The University of Surrey is committed to research excellence and innovative teaching practices that have a positive impact on society. The Institute for People-Centred AI combines decades of technical expertise in machine learning with interdisciplinary research to address the technical, ethical, and governance aspects of AI, ensuring that AI advancements truly prioritize people's needs. The university's dedication to impactful research has contributed to its ranking as the 55th best university in the world according to the Times Higher Education University Impact Rankings 2022, which evaluates universities based on their performance in relation to the United Nations' Sustainable Development Goals (SDGs).