The Brazilian electricity market is characterized by having around 65% of its installed capacity coming from hydropower plants, with multiple agents coexisting in the same hydro cascades. Currently, it also contains certain peculiarities that distinguish it from other markets, such as the Energy Reallocation Mechanism (MRE), a centerpiece of the Brazilian market’s design. This paper proposes replacing the MRE with a bid- based short-term market called the virtual reservoir model. To simulate the behavior of the hydros in this new market, an agent-based model is implemented using the reinforcement Q-learning algorithm, simulated annealing and linear programming. In the simulations, we use real data – encompassing more than 98% of the total hydro installed capacity and three years of market data – from the Brazilian power system. The results indicate that the management of (virtual) reservoirs can be the responsibility of each hydro: these can save water according to their own risk perceptions, while maintaining current efficiency and security levels. The results also suggest that the final monthly short-term market prices can substantially decrease in comparison with the current prices.