Multimodal Learning for Operational Risk Detection in Delivery Route Planning

We propose a privacy-aware multimodal framework for detecting operational risks in last-mile delivery routes by integrating spatial, temporal, and semantic information. Our method fuses geospatial embeddings derived from latitude–longitude coordinates or representative zip-code centroids, ensuring GDPR compliance when precise locations are withheld, with temporal features from delivery attempt timestamps and semantic embeddings extracted by a locally hosted large language model (LLM). To capture localized delivery failure patterns, we incorporate sequence-to-pattern mining at the zip-code level, enabling the model to recognize high-risk operational contexts. This unified architecture is designed to maintain predictive robustness even when partial location data is missing, ensuring routes remain analyzable despite privacy constraints. Empirical evaluation on real-world logistics datasets comprising approximately 2,000 routes, each visiting between 2 and 26 customers, shows that multimodal embeddings significantly enhance precision in detecting early failures compared to random baselines during validation. However, test-set results reveal generalization challenges and overfitting risks, suggesting that further work is needed on data augmentation, embedding fine-tuning, and regularization strategies. By enabling early, data-driven risk detection, our approach has the potential to support more reliable and sustainable last-mile delivery operations.