A decision support methodology for dynamic, stochastic, and integrated combinatorial decision making: A case study in first-mile logistics in supply chains

Real-world operations involve making decisions amidst stochasticity, sequential dynamics, combinatorial choices, interconnectedness with other decisions, all within limited resource constraints. While literature and industry usually address some of these aspects, and mostly do so individually, real-world operations can potentially be improved by integrated decision support schemes that consider these dimensions simultaneously, an area that remains underexplored. This research aims to answer the question: How decision-making problems characterized by stochastic, dynamic, and combinatorial aspects be efficiently addressed, considering the interconnected nature of decisions and practical constraints? This work focuses on the challenges posed by first-mile logistics operations as an archetypical example of a problem where several interconnected decisions such as procurement, routing, and inventory management must be made under uncertainty from several sources, considering sequential dynamics and including combinatorial (sub)-problems, making it highly challenging to produce practical decision support tools. By targeting specific challenges in the first mile, the study aims to conceptualize and solve the integrated first-mile logistics as a sequential decision process (SDP). SDPs provide a unifying framework, integrating key concepts and techniques from various disciplines to address sequential decision-making under uncertainty. To address the research question, we propose a decision support methodology to solve first-mile logistics as a sequential decision process. This methodology includes three modules aligned with the components of an SDP: decision points, states, decisions, reward function, exogenous information, and transition functions. The decision module encompasses computational tools such as algorithms, mathematical models, or rules to generate decisions at specific decision points. The prediction module models sources of uncertainty to create samples and anticipate potential values, assessing the impact of decisions on future outcomes. Lastly, the simulation module represents system rules, dynamics, and evolution as decisions are implemented, enabling the fine-tuning of strategies and evaluation of performance based on solution quality and computational time. These modules are designed to respond to the critical challenges of dynamic, stochastic, and integrated combinatorial decision-making, instantiated for the first-mile logistics operation. The decision support methodology aims to: (i) solve the first-mile logistics decision-making problem by jointly addressing purchasing, routing, and inventory management decisions within efficient computational times, (ii) anticipate changes in uncertain information, such as supply and purchase prices at suppliers and customer demand, as well as their potential impact on decisions in future periods, and (iii) provide solution approaches that can be extended to address additional aspects of first mile decision making, focusing on additional sources of uncertainty and dynamism in decision making. This approach leverages the strengths of exact and approximate optimization techniques to search decision spaces, coupled with approximate dynamic programming and reinforcement learning, to anticipate potential values of uncertainty sources and assess the impact of current decisions on future events. The integration of these methods across the three modules facilitates the development, fine-tuning, and evaluation of practical decision tools that can adapt to the dynamic and uncertain nature of first-mile logistics. This document outlines the proposed decision support methodology and details the steps taken to address the research question of the doctoral studies. The core of the document comprises five chapters, each corresponding to an article produced during the research project. Three articles have been published (two in peer-reviewed journals and one as a conference paper), the fourth has been resubmitted after major revisions, and the fifth is nearing submission. Each article tackles subtasks aimed at addressing the research question. Each chapter is self-contained, including its own theoretical framework, methodology, results, conclusions, and future directions. The document concludes with a summary of contributions, conclusions, and proposed future work. The research results highlight the significance and advantages of an approach that integrates interconnected decisions while anticipating sources of uncertainty and their effects on sequential decision-making. Computational studies validated the contribution of each element, interconnected decisions, anticipation of uncertainty, and their impact on sequential choices by measuring their influence on overall methodology performance. These studies showed that while each component individually offers advantages over other strategies, combining all elements provides the best results for the objective function. The proposed strategies were validated against benchmark solution methods from existing literature and custom-built strategies tailored to each specific problem, confirming the effectiveness of each solution component. The flexibility of these strategies was also tested under different problem configurations, identifying key factors for successful performance, such as cost structures, the number of suppliers in the problem, and route characteristics. Each anticipatory optimization strategy used the three modules of the decision support methodology, emphasizing the importance of each module in enhancing overall performance. The research emphasized the importance of carefully calibrating the number of scenarios when sampling sources of uncertainty and developing strategies that yield high-quality decisions with shorter computational times. This approach supports the seamless integration of these strategies into the simulation module, effectively capturing the problem's dynamics and the evolution of decision-making. The main contribution of this research lies in the successful conceptualization of integrated first-mile logistics problems within the framework of sequential decision processes (SDPs), demonstrating its effectiveness for modeling decision-making components, incorporating integrated decisions, and considering uncertainty and subsequent dynamic decisions. The core methodological contribution comprises two main approaches. First, a framework for multi-period stochastic dynamic routing problems is introduced, effectively leveraging the interaction between stochastic programming, adaptive learning, and heuristics, enabling the method to generalize across various dynamic decision problems. Stochastic programming supports scenario-based decision-making while considering future decisions. Adaptive learning allows for the construction of parameters that capture decision behavior in dynamic environments, reducing computational burden and enabling integrated decision-making within the stochastic program. Heuristics efficiently explore the decision space, speeding up the search process. This framework is detailed in Chapter 4, with its generalization to other multi-period stochastic dynamic routing problems discussed in Chapter 5. Finally, a novel approach is developed to solve stochastic dynamic combinatorial problems by mitigating overly optimistic future cost estimates using perfect information models. This approach incorporates reinforcement learning, approximate dynamic programming, and approximate optimization, exemplified through a routing and purchasing problem. A Rollout algorithm is used to estimate the impact of current decisions on future costs, with scenarios solved using a heuristic approach. To counter optimistic estimations, a penalty is trained to adjust the future cost, achieving a better balance with immediate costs. This penalty is state-dependent at each decision point, allowing for a more precise capture of decision-making dynamics. A detailed explanation of this solution approach is provided in Chapter 6. The success of the proposed anticipation methodologies lies in their ability to make high quality decisions in short computational times with the integration of uncertainty sampling and parameter tuning in a simulated decision making environment. While the advantages are obvious, notable limitations were also identified. A key limitation is the assumption that sources of uncertainty can be modeled and remain constant throughout the stochastic process, which may not be suitable for highly dynamic or information-limited processes. In addition, the approach depends on costly simulations that presume problem parameters remain unchanged during and after training. As the problem size increases, scalability issues may arise that affect decision making times and limit the feasibility of lengthy simulations, thus calling into question the effectiveness in larger and more complex scenarios. Future research could focus on dynamically adapting to changing sources of uncertainty and developing anticipative methods that do not rely on simulations, ensuring a balance between decision accuracy and scalability.

• Autores: Cuellar Usaquén, Daniel Hernando
• Tipo de recurso: Doctoral thesis
• Fecha de publicación: 2024
• Institución: Universidad de los Andes
• Repositorio: Séneca: repositorio Uniandes
• Idioma: eng
• OAI Identifier: oai:repositorio.uniandes.edu.co:1992/75402
• Acceso en línea: https://hdl.handle.net/1992/75402
• Palabra clave: Agri-food supply chains; Approximate dynamic programming; First mile logistics; Exact and approximate optimization; Ingeniería
• Rights: openAccess
• License: Attribution-NonCommercial-NoDerivatives 4.0 International