There is a multiple-objective Reinforcement Learning branch in the community.
The idear is to 1:
assign a family of agents to each objective. The solutions obtained by the agents in one family are compared with the solutions obtained by the agents from the rest of the families. A negotiation mechanism is used to find compromise solutions satisfying all the objectives.
Also there a paper that might be interest to you:
Multi-objective optimization by reinforcement learning for power system dispatch and voltage stability.
I did not find a public url for it though.