First of all, I have no idea how you managed to get into that situation.
More importantly, there already is a check for AFK players in mini-games, including derby. But if all the derby players really manage to get stuck at the same time, there should be some kind of time limit which ends the round or kicks the inactive players.