First it happened on the second try, it did the first lap of Seers then switched to Gnome Stronghold for some reason. The next time I did not notice when it happened but at least after some successful laps of Seers course.
Surely implementing a GUI is a good idea, would get rid of the problem without much troubleshooting at all.