The increasing frequency of natural disasters necessitates efficient building damage detection for effective disaster response. This study addresses limitations in deep learning models, particularly their inability to classify minor as well as major damage classes due to inadequate detection of structural features like edges and corners in satellite images. To overcome these challenges, we propose the utilization of a fusion-based data augmentation technique that combines edge detection, contrast enhancement, and unsharp masking to enhance structural feature detection. We further evaluate the generalizability of this approach using domain adaptation techniques, including supervised fine-tuning and unsupervised Deep CORAL to address domain shifts between source (xBD) and target (Ida-BD) datasets. Experimental results demonstrate that the proposed augmentation improves damage classification accuracy by 5–7% in minor and major damage classes and enhances localization accuracy by 2.5%. Additionally, the integration of domain adaptation techniques validates the robustness in handling out-of-domain datasets. By improving structural feature detection and mitigating domain discrepancies, the proposed methodology enhances performance and adaptability of deep learning models for disaster response. This study demonstrates the potential of fusion-based augmentation and domain adaptation to enable reliable and efficient building damage detection in diverse disaster scenarios.