Renewable energy sources such as wind and solar have a high degree of unpredictability and time variation. As a result, balancing supply and demand in real time is becoming ever more challenging and the power grids need greater flexibility on many levels. The proposed approach addresses this challenge by harnessing the flexibility in demand of many types of loads. We develop a distributed control algorithms for automated demand dispatch, which can be used by grid operators as ancillary service to regulate demand-supply balance. The proposed approach uses local control solutions that a) take into account local measurements, constraints, and preferences, and b) lead to a controllable input-output model for the aggregate dynamics. 

The local control problem can be defined by a family of Markov decision processes, parameterized by a weighting factor that appears in the one-step reward function. This talk introduces a methodology for solving an entire family of MDPs. In our application to demand control, the focus will be on a family of average-cost optimal control models in which the one-step reward function is defined by Kullback-Leibler divergence with respect to nominal dynamics. The proposed ODE methodology can be seen as a generalization of the linearly solvable MDP framework of Todorov to the case with exogenous disturbances, such as weather or customer behavior.