I reccommend skipping spiritual mages, rangers and warriors or blocking them permanently if you have the possibility.
No it won't ruin it. And there's an easy way to gather fast slayer points, every 10th task you go get one from Duradel. If you go to Chaeldar or Neive for your other tasks this won't reset your slayer task count.