Internship M2 in Engineering (H-F) CESI

Title : Bike sharing system rebalancing by reinforcement learning algorithms

Keywords: bike sharing system, bike repositionning, reinforcement learning

Description

This internship project focuses on a specific component of a broader initiative to improve the dynamic rebalancing of bike-sharing systems [1,2,3]. The problem is addressed in two stages. Based on data at the station and travel needs at a given moment t, the number of bicycles available and needed will be predicted at time t+1. Points of origin and destination can be grouped together to improve the performance of spatio-temporal calculations of flow gradients from the micro scale at the station to the city scale [3,4].

This approach will thus make it possible to predict more quickly the number of bicycles used on the network and at stations in order to obtain a quasi-dynamic description of the system [6,7]. In a second stage, using these new estimated input data, real-time rebalancing is deployed. A reinforcement learning algorithm is then used and trained to propose and refine the dynamic redistribution strategy for bicycles [8,9]. The advantage of this approach lies in its ability to adapt to contextual disturbances and to resolve issues on a large scale. However, this performance comes at a cost and is detrimental to ensuring the most optimised solution is achieved.

This internship will focus on the first stage of the project, which concerns the prediction and modeling of bicycle availability and demand dynamics. The objective will be to design and evaluate predictive models capable of capturing both spatial and temporal dependencies in the bikeshare system. The intern will explore and compare different machine learning approaches, such as time series forecasting, graph neural networks, or spatio-temporal convolutional architectures, to estimate short-term variations in bicycle flows at the station and network levels by using clustering, for example [10].

The performance of the models will be evaluated against real operational data, and the results will serve as input for the reinforcement learning framework used in the second phase of the project .

Depending on the progress and interests of the intern, additional exploration may include studying the integration of uncertainty quantification in predictions or the use of online learning methods to adapt models in real time as new data become available.

The internship will provide the opportunity to gain hands-on experience in data science, spatio-temporal modeling, and urban mobility systems, while contributing to an innovative research topic with potential real-world applications.

Objectives :

O1. Develop predictive models to estimate short-term bicycle availability and demand at both the station and network levels using spatio-temporal data.

O2. Analyze and preprocess heterogeneous datasets, including trip records, station metadata, weather conditions, and temporal factors, to create robust inputs for modeling.

O3. Implement and compare different machine learning approaches (e.g., time series forecasting, graph neural networks, spatio-temporal models) to capture flow dynamics in the bikeshare system.

O4. Evaluate the performance and scalability of predictive algorithms under realistic conditions, using metrics relevant to operational decision-making in mobility systems.

O5. Provide data-driven inputs for the reinforcement learning module, enabling the development of adaptive and real-time rebalancing strategies in the second phase of the project.

O6. Integrate uncertainty quantification to assess the confidence of predictions and their impact on rebalancing decisions.

O7. Explore online or incremental learning techniques to enable continuous model adaptation as new data streams become available.

Expected scientific/ technical production

The internship is expected to lead to both methodological and applied outcomes, including:

O1. A cleaned and structured dataset integrating multimodal information (trip data, station metadata, weather, temporal and spatial context) suitable for spatio-temporal modeling.

O2 .A set of predictive models (baseline statistical models and advanced machine learning architectures) for short-term demand and availability forecasting in bikeshare systems.

O3. A comparative performance analysis report, detailing the accuracy, robustness, and computational efficiency of the different modeling approaches.

O4. A prototype or simulation tool demonstrating the integration of prediction outputs into a reinforcement learning environment for dynamic rebalancing.

O5.Technical documentation and reproducible code, following open science practices, to facilitate future extensions and integration into the larger project framework.

O6. Preparation of a scientific report or publication draft, presenting the methodology, results, and implications for large-scale mobility optimization.

Lab presentation

CESI LINEACT (UR 7527), Laboratory for Digital Innovation for Businesses and Learning to Support the Competitiveness of Territories, anticipates and accompanies the technological mutations of sectors and services related to industry and construction. The historical proximity of CESI with companies is a determining element for our research activities. It has led us to focus our efforts on applied research close to companies and in partnership with them. A human-centered approach coupled with the use of technologies, as well as territorial networking and links with training, have enabled the construction of cross-cutting research; it puts humans, their needs and their uses, at the center of its issues and addresses the technological angle through these contributions.

Its research is organized according to two interdisciplinary scientific teams and several application areas.

• Team 1 'Learning and Innovating' mainly concerns Cognitive Sciences, Social Sciences and Management Sciences, Training Techniques and those of Innovation. The main scientific objectives are the understanding of the effects of the environment, and more particularly of situations instrumented by technical objects (platforms, prototyping workshops, immersive systems...) on learning, creativity and innovation processes.

• Team 2 'Engineering and Digital Tools' mainly concerns Digital Sciences and Engineering. The main scientific objectives focus on modeling, simulation, optimization and data analysis of cyber physical systems. Research work also focuses on decision support tools and on the study of human-system interactions in particular through digital twins coupled with virtual or augmented environments.

These two teams develop and cross their research in application areas such as

• Industry 5.0,

• Construction 4.0 and Sustainable City,

• Digital Services.

Areas supported by research platforms, mainly that in Rouen dedicated to Factory 5.0 and those in Nanterre dedicated to Factory 5.0 and Construction 4.0.

Link to the research axes of the research team involved

CESI Lineact Research Thematic: Management of Multimodal Transport Systems (GSTM)

Bibliography :

[1] Z. Jiang, C. Lei, and Y. Ouyang, "Optimal investment and management of…

CESI est une école d'ingénieurs qui fait de la promotion sociale par l'excellence un modèle de réussite. Rejoignez un environnement stimulant où l'esprit d'équipe, la diversité des projets et l'autonomie ne font qu'un. Découvrez une école qui a su développer un modèle unique et se donne les moyens au quotidien de relever les grands défis de l'époque. Nos 25 campus, 28 000 étudiants, 8000 entreprises partenaires et 106 000 alumni témoignent de l'impact de CESI au niveau national.

CESI accompagne ses étudiants en utilisant des méthodes innovantes de pédagogie active. L'établissement forme avec rigueur les futurs ingénieurs, techniciens et managers, dans les secteurs suivants : l'Industrie & l'Innovation, le BTP, l'Informatique et le Numérique et le Développement Durable. Parallèlement, CESI concrétise son engagement dans la Recherche à travers des activités menées au sein de son Laboratoire d'Innovation Numérique, CESI LINEACT.

Les partenariats établis avec 130 universités à travers le globe, attestent de l'engagement international de CESI. Ces liens privilégiés offrent aux élèves ingénieurs une mobilité sortante et entrante à l'échelle internationale, façonnée notamment par des stages obligatoires faisant partie intégrante de leur cursus.