Dynamic control of traffic lights

Research output: Chapter in Book/Report/Conference proceedingChapterAcademicpeer-review

Abstract

Traffic lights are put in place to dynamically change priority between traffic participants. Commonly, the duration of green intervals and the grouping, and ordering in which traffic flows are served are pre-fixed. In this chapter, the problem of minimizing vehicle delay at isolated intersections is formulated as a Markov Decision Process (MDP). Solving the MDP is hampered by a large multi-dimensional state space that contains information on the traffic lights and on the queue lengths. For a single intersection, an approximate solution is provided that is based on policy iteration (PI) and decomposition of the state space. The approach starts with a Markov chain analysis of a pre-timed control policy, called Fixed Cycle (FC). The computation of relative states values for FC can be done fast, since, under FC, the multi-dimensional state space can be decomposed into sub-spaces per traffic flow. The policy obtained by executing a single iteration of Policy Iteration (PI) using relative values is called RV 1. RV1 is compared for two intersections by simulation with FC, a few dynamic (vehicle actuated) policies, and an optimal MDP policy (if tractable). RV1, approximately solves the MDP, and compared to FC, it shows less delay of vehicles, shorter queues, and is robust to changes in traffic volumes. The approach shows very short computation times, which allows the application to networks of intersections, and the inclusion of estimated arrival times of vehicles approaching the intersection.
Original languageEnglish
Title of host publicationMarkov Decision Processes in Practice
PublisherSpringer New York LLC
Pages371-386
ISBN (Electronic)9783319477664
ISBN (Print)9783319477640
DOIs
Publication statusPublished - 2017

Publication series

NameInternational Series in Operations Research and Management Science
Volume248
ISSN (Print)0884-8289

Fingerprint

Dynamic Control
Telecommunication traffic
Markov Decision Process
Intersection
Traffic
Cycle
Policy Iteration
State Space
Traffic Flow
Time of Arrival
Vehicle Dynamics
Queue Length
Control Policy
Grouping
Markov processes
Queue
Markov chain
Approximate Solution
Inclusion
Subspace

Cite this

Haijema, R., Hendrix, E. M. T., & van der Wal, J. (2017). Dynamic control of traffic lights. In Markov Decision Processes in Practice (pp. 371-386). (International Series in Operations Research and Management Science; Vol. 248). Springer New York LLC. https://doi.org/10.1007/978-3-319-47766-4_13
Haijema, Rene ; Hendrix, Eligius M.T. ; van der Wal, J. / Dynamic control of traffic lights. Markov Decision Processes in Practice. Springer New York LLC, 2017. pp. 371-386 (International Series in Operations Research and Management Science).
@inbook{370f61a270494c7bbbf7e5f95f6715cd,
title = "Dynamic control of traffic lights",
abstract = "Traffic lights are put in place to dynamically change priority between traffic participants. Commonly, the duration of green intervals and the grouping, and ordering in which traffic flows are served are pre-fixed. In this chapter, the problem of minimizing vehicle delay at isolated intersections is formulated as a Markov Decision Process (MDP). Solving the MDP is hampered by a large multi-dimensional state space that contains information on the traffic lights and on the queue lengths. For a single intersection, an approximate solution is provided that is based on policy iteration (PI) and decomposition of the state space. The approach starts with a Markov chain analysis of a pre-timed control policy, called Fixed Cycle (FC). The computation of relative states values for FC can be done fast, since, under FC, the multi-dimensional state space can be decomposed into sub-spaces per traffic flow. The policy obtained by executing a single iteration of Policy Iteration (PI) using relative values is called RV 1. RV1 is compared for two intersections by simulation with FC, a few dynamic (vehicle actuated) policies, and an optimal MDP policy (if tractable). RV1, approximately solves the MDP, and compared to FC, it shows less delay of vehicles, shorter queues, and is robust to changes in traffic volumes. The approach shows very short computation times, which allows the application to networks of intersections, and the inclusion of estimated arrival times of vehicles approaching the intersection.",
author = "Rene Haijema and Hendrix, {Eligius M.T.} and {van der Wal}, J.",
year = "2017",
doi = "10.1007/978-3-319-47766-4_13",
language = "English",
isbn = "9783319477640",
series = "International Series in Operations Research and Management Science",
publisher = "Springer New York LLC",
pages = "371--386",
booktitle = "Markov Decision Processes in Practice",

}

Haijema, R, Hendrix, EMT & van der Wal, J 2017, Dynamic control of traffic lights. in Markov Decision Processes in Practice. International Series in Operations Research and Management Science, vol. 248, Springer New York LLC, pp. 371-386. https://doi.org/10.1007/978-3-319-47766-4_13

Dynamic control of traffic lights. / Haijema, Rene; Hendrix, Eligius M.T.; van der Wal, J.

Markov Decision Processes in Practice. Springer New York LLC, 2017. p. 371-386 (International Series in Operations Research and Management Science; Vol. 248).

Research output: Chapter in Book/Report/Conference proceedingChapterAcademicpeer-review

TY - CHAP

T1 - Dynamic control of traffic lights

AU - Haijema, Rene

AU - Hendrix, Eligius M.T.

AU - van der Wal, J.

PY - 2017

Y1 - 2017

N2 - Traffic lights are put in place to dynamically change priority between traffic participants. Commonly, the duration of green intervals and the grouping, and ordering in which traffic flows are served are pre-fixed. In this chapter, the problem of minimizing vehicle delay at isolated intersections is formulated as a Markov Decision Process (MDP). Solving the MDP is hampered by a large multi-dimensional state space that contains information on the traffic lights and on the queue lengths. For a single intersection, an approximate solution is provided that is based on policy iteration (PI) and decomposition of the state space. The approach starts with a Markov chain analysis of a pre-timed control policy, called Fixed Cycle (FC). The computation of relative states values for FC can be done fast, since, under FC, the multi-dimensional state space can be decomposed into sub-spaces per traffic flow. The policy obtained by executing a single iteration of Policy Iteration (PI) using relative values is called RV 1. RV1 is compared for two intersections by simulation with FC, a few dynamic (vehicle actuated) policies, and an optimal MDP policy (if tractable). RV1, approximately solves the MDP, and compared to FC, it shows less delay of vehicles, shorter queues, and is robust to changes in traffic volumes. The approach shows very short computation times, which allows the application to networks of intersections, and the inclusion of estimated arrival times of vehicles approaching the intersection.

AB - Traffic lights are put in place to dynamically change priority between traffic participants. Commonly, the duration of green intervals and the grouping, and ordering in which traffic flows are served are pre-fixed. In this chapter, the problem of minimizing vehicle delay at isolated intersections is formulated as a Markov Decision Process (MDP). Solving the MDP is hampered by a large multi-dimensional state space that contains information on the traffic lights and on the queue lengths. For a single intersection, an approximate solution is provided that is based on policy iteration (PI) and decomposition of the state space. The approach starts with a Markov chain analysis of a pre-timed control policy, called Fixed Cycle (FC). The computation of relative states values for FC can be done fast, since, under FC, the multi-dimensional state space can be decomposed into sub-spaces per traffic flow. The policy obtained by executing a single iteration of Policy Iteration (PI) using relative values is called RV 1. RV1 is compared for two intersections by simulation with FC, a few dynamic (vehicle actuated) policies, and an optimal MDP policy (if tractable). RV1, approximately solves the MDP, and compared to FC, it shows less delay of vehicles, shorter queues, and is robust to changes in traffic volumes. The approach shows very short computation times, which allows the application to networks of intersections, and the inclusion of estimated arrival times of vehicles approaching the intersection.

U2 - 10.1007/978-3-319-47766-4_13

DO - 10.1007/978-3-319-47766-4_13

M3 - Chapter

SN - 9783319477640

T3 - International Series in Operations Research and Management Science

SP - 371

EP - 386

BT - Markov Decision Processes in Practice

PB - Springer New York LLC

ER -

Haijema R, Hendrix EMT, van der Wal J. Dynamic control of traffic lights. In Markov Decision Processes in Practice. Springer New York LLC. 2017. p. 371-386. (International Series in Operations Research and Management Science). https://doi.org/10.1007/978-3-319-47766-4_13