Energy Storage Arbitrage in Day-Ahead Electricity Market Using Deep Reinforcement Learning

Large scale integration of renewable and distributed energy resources increases the need for flexibility on all levels of the energy value chain. Energy storage systems are considered as a major source of flexibility. They can help with maintaining a secure and reliable grid operation. The problem is that these technologies are capital intensive and therefore, there is a need for new algorithms that enable arbitrage while ensuring financial feasibility. To this end, in this research, we develop a constrained deep Q-learning based bidding algorithm to determine the optimal bidding strategy in the day-ahead electricity market. The proposed algorithm ensures compliance to energy storage system constraints. It takes imperfect, yet reasonably accurate, 24-hour-ahead price forecast data as an input and returns the optimal bidding strategy as output. The numerical results and the sensitivity analysis show that the proposed algorithm effectively contains the impact of price forecast uncertainty to guarantee financial feasibility.