Data on bans

[ceesaw]

Has anybody attempted to gather data on instances of bots being banned? It'd be interesting to take a look at how different factors may explain variation in ban rates. Putting together a rigorous picture of what might cause bans would allow for better conversation about 'anti-ban' measures. 

 

Edited May 23, 2018 by ceesaw

[Gwansky]

Go ahead and do it You can be banned at a later date/hour and you would never know what got you banned. 

[Apaec]

I don't think you'll be able to get a big enough sample size. Nonetheless, I think the time of developing scripts/bots to go 'undetected' has more or less passed - here's my theory copied from a thread a couple days ago:

Quote

As for botting, the real issue is that detecting bots is far easier than pretending to be human (this is why you can't talk to siri like you can another human). On jagex's side, all they need are some beefy statistical analysis tools. On the botting side, you would need complicated state of the art neural network/machine learning/AI engine systems to get even close to replicating a human. Furthermore, Jagex has professional developers and a large amount of money - scripters tend to be hobbyists and work alone, and botting sites have comparatively little investment. 

If you look at it from this perspective, you can see that Jagex really do have all the eggs in their basket - it just seemed to take them until 2010 to act on this. This is why I bot with the assumption that my account is already flagged - avoiding bans is now keeping the sessions short and efficient to avoid your 'flag' being prioritised.

That being said, there might still be knowledge to gain from a good bit of data. Unfortunately I can't contribute anything as I rarely bot and haven't been banned in a few years, but I wish you the best of luck with this

Apa

[Tommm39]

2 hours ago, Apaec said:

I don't think you'll be able to get a big enough sample size.

Precisely this. Even if someone was able to amass a large amount of data about banning patterns and apply some kind of data mining algorithm it would be of little use. The sample size that would be required for results with a high predictive power would be very large. Also there are far too many variables to be considered. 

Besides even if you were able to put together some kind of algorithm to determine what factors are more likely to get you banned and you avoid them, there's still no guarantee. 

[Jackthedon]

Another point is that....anyone who actually did go ahead and do this who was in any way successful is never going to share said information.

Imagine for instance I discovered a way to bot undetected and then I announced it. I'd ruin the entire idea, from saturation to the point that they'd step up their antiban and also that people would be gold farming much easier causing gp to crash.

The only way this will ever work is if YOU do ALL the work and keep it all to yourself.

[yfoo]

21 hours ago, Apaec said:

I don't think you'll be able to get a big enough sample size. Nonetheless, I think the time of developing scripts/bots to go 'undetected' has more or less passed - here's my theory copied from a thread a couple days ago:

As for botting, the real issue is that detecting bots is far easier than pretending to be human (this is why you can't talk to siri like you can another human). On jagex's side, all they need are some beefy statistical analysis tools. On the botting side, you would need complicated state of the art neural network/machine learning/AI engine systems to get even close to replicating a human. Furthermore, Jagex has professional developers and a large amount of money - scripters tend to be hobbyists and work alone, and botting sites have comparatively little investment. 

If you look at it from this perspective, you can see that Jagex really do have all the eggs in their basket - it just seemed to take them until 2010 to act on this. This is why I bot with the assumption that my account is already flagged - avoiding bans is now keeping the sessions short and efficient to avoid your 'flag' being prioritised.

That being said, there might still be knowledge to gain from a good bit of data. Unfortunately I can't contribute anything as I rarely bot and haven't been banned in a few years, but I wish you the best of luck with this

Apa

I mostly agree with your observations on the capabilities/resources of Jagex vs that of hobbiest scriptors and the notion that undetected botting is much harder today than ever, However I would like add some of my own insights onto your points on needing to use advanced AI techniques (NN, ML, etc.) to remain undetected. With undetected meaning not flagged by Jagex's Botwatch, not a manual review by a mod. 

I always suspected that Jagex uses a neural network to classify a player as a bot or legitament. NNs are good technique for pattern recognization, and one common technique is using NNs for image recognization. However NNs today do have their flaws. For example, there are cases where NNs fail to recognize a slightly altered image with changes imperceptible to humans. Furthermore there are examples of white noise being incorrectly classified as something else. Basically I just summarized the abstract of this article: https://arxiv.org/abs/1412.1897

So how do these details relate to botting. After this point all I can offer is conjecture. One key aspect that goes into creating a NN is training data, and what better place to obtain training data than to run public scripts (both premium/free) offered on various botting sites. Secondly, as proper bans are handed out the NN is given reinforcement learning (jagex validates the NN classiifcation as correct and the NN algorithm's weights are adjusted to better detect some general pattern). It is likely for this reason public goldfarming scripts are banned quickly, premium or not. They are profiled and given reinforment from every single ban. It is likely this reason why private scripts are considered less "detectable", Jagex's anticheat procures a different set of data that is unmatched to a known script. This may anaglous to how you can fool a image recognizer's NN by changing the image, you likely can fool Jagex's botwatch with a slightly different script. Im pretty sure Apa writes private scripts, he might give you a reasonable quote if you ask him nicely. 

Finally Jagex really shot themselves in the foot by allowing 3rd party clients. 3rd party clients use the same techniques (reflection/injection) as botting clients to obtain relevant game data. The existance of 3rd party clients gives plausable deniabiliy as to whether someone is using BestLite/WorstBuddy or a botting client, its too late now to change this, you saw how r/2007scape reacted when runelite was going to be shutdown. It is alot easier if prevent botting can also be done by detecting the existance of client tampering and banning any account that logs on. Alot less bots can be ran at anytime if everyone was forced to mirror mode. Trusting the user's client allows attack vectors to more easily found and exploited, the existance of Osbot is evidence of this. 

TLDR: private scripts prevent bans. 

Edited May 24, 2018 by PayPalMeRSGP

[ceesaw]

2 hours ago, PayPalMeRSGP said:

I mostly agree with your observations on the capabilities/resources of Jagex vs that of hobbiest scriptors and the notion that undetected botting is much harder today than ever, However I would like add some of my own insights onto your points on needing to use advanced AI techniques (NN, ML, etc.) to remain undetected. With undetected meaning not flagged by Jagex's Botwatch, not a manual review by a mod. 

I always suspected that Jagex uses a neural network to classify a player as a bot or legitament. NNs are good technique for pattern recognization, and one common technique is using NNs for image recognization. However NNs today do have their flaws. For example, there are cases where NNs fail to recognize a slightly altered image with changes imperceptible to humans. Furthermore there are examples of white noise being incorrectly classified as something else. Basically I just summarized the abstract of this article: https://arxiv.org/abs/1412.1897

So how do these details relate to botting. After this point all I can offer is conjecture. One key aspect that goes into creating a NN is training data, and what better place to obtain training data than to run public scripts (both premium/free) offered on various botting sites. Secondly, as proper bans are handed out the NN is given reinforcement learning (jagex validates the NN classiifcation as correct and the NN algorithm's weights are adjusted to better detect some general pattern). It is likely for this reason public goldfarming scripts are banned quickly, premium or not. They are profiled and given reinforment from every single ban. It is likely this reason why private scripts are considered less "detectable", Jagex's anticheat procures a different set of data that is unmatched to a known script. This may anaglous to how you can fool a image recognizer's NN by changing the image, you likely can fool Jagex's botwatch with a slightly different script. Im pretty sure Apa writes private scripts, he might give you a reasonable quote if you ask him nicely. 

Finally Jagex really shot themselves in the foot by allowing 3rd party clients. 3rd party clients use the same techniques (reflection/injection) as botting clients to obtain relevant game data. The existance of 3rd party clients gives plausable deniabiliy as to whether someone is using BestLite/WorstBuddy or a botting client, its too late now to change this, you saw how r/2007scape reacted when runelite was going to be shutdown. It is alot easier if prevent botting can also be done by detecting the existance of client tampering and banning any account that logs on. Alot less bots can be ran at anytime if everyone was forced to mirror mode. Trusting the user's client allows attack vectors to more easily found and exploited, the existance of Osbot is evidence of this. 

TLDR: private scripts prevent bans. 

Firstly, thank you for the reply, it makes for a very interesting read. It would seem that the emergence of AI techniques stands to benefit Jagex more-so than the 'hobbyist scripter'. As such, it is unsurprising that many regard bottling to be harder than it has ever been. Looking forward botting will likely continue to become harder until it ceases to meaningfully exist (your industry is terminally ill).

I don't come from a technology background, however, it would seem that this is an issue of the marginal benefit provided by the introduction of new technology to each sides arsenal. Due to their scope Jagex have always outpaced the hobbyist scripture in their ability to realise 'cutting edge' technologies. Where the discrepancy between the impact of old and new technologies on each sides output (good bot detection vs good scripts) is small Jagex realises a small advantage over the hobbyist and botting is relatively 'safe. 

The introduction of more advanced techniques stands to increase the marginal benefit provided by realising cutting edge technology over older tech. As you note, AI techniques (albeit with a few hiccups) have significant current ability and future potential to enhance botwatch systems. Looking forwards, this discrepancy will likely grow until Jagex's advantage is so massive that botting will cease to exist. 

@Apaec Provides an interest quote which fits with my picture above. Its important to view this dynamically, Jagex have always had the advantage, however the size of this advantage is what is significant. The growth of this advantage will kill botting(your industry is terminally ill). I know little of the underlying tech, so perhaps you could shed some light on when technology will advance to a stage where this occurs. 

"On the botting side, you would need complicated state of the art neural network/machine learning/AI engine systems to get even close to replicating a human. Furthermore, Jagex has professional developers and a large amount of money - scripters tend to be hobbyists and work alone, and botting sites have comparatively little investment. "

 

2 hours ago, PayPalMeRSGP said:

 

Edited May 24, 2018 by ceesaw

[Zor]

My biggest surprise is that osbot or other botting clients haven't had something in the client that collects tons of data on bans and stuff from the bots