Effective online decision-making in complex multi-agent systems