Enhancing Remote Sensing Image Retrieval: A Hierarchical Approach Integrating Visual and Semantic Similarities

Abstract

The heightened revisiting frequency and expanded observation capabilities of satellites lead to the daily generation of a substantial volume of remote sensing images. Retrieving relevant data accurately from this extensive archive holds significant importance. Deep learning content-based image retrieval (CBIR) uses a feature extraction network pretrained on image classification tasks to derive image-level features. Subsequently, a similarity measure is applied on these features to identify the archive images most closely resembling the query image. While image-level labels facilitate CBIR in retrieving images from the same category as the query image, they do not empower CBIR to differentiate between implicit subcategories. For instance, although CBIR can discern between broader categories such as “residential” and “forest,” it lacks the necessary semantic statistical information to distinguish more nuanced distinctions such as “high-density residential” from “medium-density residential.” To enhance image retrieval for greater similarity, we propose a hierarchical image retrieval (HIR) approach that combines visual similarity with semantic statistics. In the first stage, visually similar images are identified using CBIR, while the second stage refines the selection based on semantic similarity derived from land-cover classification. The experimental results indicate that HIR achieves 20% higher retrieval accuracy for “residential” subcategories and over 1% increase in retrieval accuracy across all classes.