There's a packet within the RS client which checks if the client has been modified (injected in our case) and that seems to identify the fact that the player is using a bot client.
Then, as maxi said, the second strategy involves profiling using their fancy analyzers etc, I really don't know anything about that, but it seems the most likely answer.
EDIT:
and the way mirror works, is by copying the original runescape JVM, injecting it, then sending back the data, so the original jvm is not modified at all. Feel free to correct me if i'm wrong!