Road to an automated 100m (+Writing my own scripts)

[Colonel_Panic]

Introduction

Hi all. I'm new here, so I figured I'd give a bit of background about myself.

 

I'm a software engineer by trade (graduated with a CS degree several years ago and work full-time as a software engineer) and used to play a ton of Runescape.  In the past, I've tried to get into script development, but always got sidetracked and gave up without producing anything.

This time, I'm hoping to see it through by setting some short and long term goals to keep me motivated. 

I have experience in Java, although it's been a few years since I've used it; however, the OSBot devs did a great job with the API so I don't think it'll be too long till I'm making progress.

 

Short Term Goals

• Develop script that can do tutorial island  (DONE)
  • This should give me a lot of practice as it's doing a large variety of tasks
  • In order to not jump into this entirely blind, I've been looking at some open source tutorial island scripts as well (shout out to @lisabe96's RQ Tutorial Island Completer)
  • This may end up being trashed if the ban rate is too high, but either way the scripting experience will be valuable.
• Write script to mine clay  (DONE - With caveat.  See Week 1 updates for more info)
• Build infrastructure to facilitate combining scripts well, and combine tutorial island + clay miner (DONE)
  • This will aid in the future as I hope to have bots complete prerequisites, and then automatically jump to gold farming once those are finished
• Build script to automatically mule
• F2P Fishing Script (V1 Done, still a WIP to improve it to a gold-farm ready script)
• 10 mill automatically farmed and muled
• MySQL Server for managing bots and facilitating communication between them (see update for 9/25 for more info)

 

Long Term Goals

• Automatic account replacement as they're banned
• Alerts (via email, twitter, or text message) when something seems off
• Profit in the black (should be a relatively quick one to complete as I won't be paying too much on servers at the start)
• Communication between bots (so I don't end up with to many on the same world, or to alert others should something go wrong (e.g. JMod in a certain world)
• 100m profit

Progress

9/18:

Spoiler

Spent a lot of time analyzing lisabe96's RQ Tutorial Island Completer (as well as some other Tut island scripts, but lisabe96's was the best I've found).  

I started by simply watching the script run and writing down any bugs or weird issues that looked off.  

Once I had a list, I set out to fix all the bugs and get rid of some of the "non-human" actions (e.g. clicking on an NPC once we've already started talking to them, or trying to spam click objects that only need to be clicked once).

Once this was done and I had several bots complete the island with no problems, I introduced my own form of anti-ban that introduces randomness to the actions so they're more human like.  This may do nothing, but it was fun to experiment with nonetheless as I was able to tweak it until it looked passable.  

I'm currently working on adjusting this now so that once it finishes tutorial island, it will move on to a new task.  For now, this task will be mining clay as it should be an easy introductory script for me to write from scratch.  I'm hoping to have this up and running within a day so I can fire up a few suicide bots and get an idea on how long they'll last before being banned (hopes aren't high at this point).

UPDATE 2 (9/18)
Clay miner is "sorta" done.

It's working as well as it can, but there is a big problem that I didn't foresee (see bottom)

Features:

• Toggles run energy if I have above a certain threshold (random so it's not always toggling at same value)
• Hops world if there are too many other players competing at clay
• Uses correct pick axes for level
• Banks -> Mines -> repeats

 

HOWEVER. The big problem is that there are just too many F2P clay mining bots.  I had no idea how many there actually were, and I'm not able to get any clay as my mining level is too low.  Looking around they all seem to be using rune pick axes, so I'm thinking I'll need to have it power mine somewhere else to get some levels before actually trying to get clay.  If anyone has any input here, I'd appreciate it.

 

9/19:

Spoiler

Fishing script v1 is done. As is the process of cleaning up the faulty architecture around script states that I mentioned before.

It still will need work before I can really gold farm on it as currently it won't automatically move on to lobsters as it won't have the required lobster pot.  I figure I'll either have it automatically run to the GE and sell off all loot and buy a lobster pot, or I'll take this an excuse to automate some of the muling process to have a mule give them lobster pots once they reach fishing level 40.

I also am currently fishing in Al Kharid and I want to find some creative solution to deal with the scorpion that roams around there; currently I simply run away if it attacks me.  

9/22:

Spoiler

My Fisher now is completely automated. I haven't been able to truly test it 100% as I haven't had an account run for longer than 24 hours to get rid of the F2P trade restrictions (to test mule features fully).

 

However, I can create an account, type the credentials into osbot and leave it to:

-finish tutorial island

-sell items to afford to buy required fishing gear

-fish until 40 fishing

-fish lobsters until I tell it to mule

 

I'll hopefully get some long runtimes and progress this weekend and take some pictures.

 

This is a big stepping stone as it not only is essentially "gold farm ready", but it really reassured me that I should be good to go developing any more complex scripts in the future.

At this point I'm going to need to do some more logistical planning to figure out if I want to just suicide bot f2p at a larger scale (more dev ops/ban management related work), or write something for a P2P method and spend less time fighting mass bans. I'd love some input if there's anyone out there still reading this 

 

Also, I finally set a money goal. For now it's a meager 10 mill fully automated as a short term and the long term is 100m. Just wanted to keep my goals very realistic at this stage as I'm sure I'll run into more problems with bans and system management as I scale up.


Night update:

https://imgur.com/a/O4Ckl

 

Have 3 characters fishing lobsters and just fired up 2 more. Will leave these going until I encounter a problem.  I'll hopefully post follow up pictures tomorrow with longer progress stats ;)

9/23

Spoiler

https://imgur.com/a/QjcOZ

All bots still running. One of the fresh accounts got 40 fishing properly, sold gear to get a lobster pot,  and made it to karamja to start fishing.  The other one was stuck at the GE this morning, so that's definitely something I need to look into.

I'm also getting a few log-outs after an EventExecuter timed out.  It logs back in and continues fine, but still that's another thing I need to look into.

The current plan as of now is to spend some time cleaning and optimizing the code.  I have a feeling some of the weird bugs I'm seeing are due to timing issues and inconsistencies.  I also want to look more into conditional sleeps as those seem to be the way to go for some of the stuff I'm doing.

 

Once that's done, I plan to scale up on a VPS to around 10 bots.  Hopefully this will be done before tomorrow so I can get some run time before bans start coming on Monday.  

9/24

Spoiler

Update:

Ended out the weekend with just shy of 35k lobsters from 5 accounts.

I tested out auto muling just now and was very pleased to have 4/5 bots mule properly.  One got stuck in the bank and died to an NPC and I'm 90% sure that was due to me maxing out the CPU on the VPS I'm using, so code wise I think I'm still okay.

 

I spent a few hours today working on building my own proxy server so I can supply my own proxies.  I've decided that I'm going to go with a dedicated server and up the number of bots I'm running to around 30.

I'm still curious to see how things change during the week when bans come more frequently.  I'm guessing I'll need to add the ability to get 7 qp on the accounts for them to be able to offload before the bans come down; but I'm going to experiment a bit this week to see if any last more than 24 hours.  If they do, then I won't worry about questing for now. 

 

I'll experiment a bit more this week and then once I'm fully ready I'll pull the trigger on a dedicated server and try to push the limit on what I can do.  

9/25

Spoiler

Decided to pull back on my plans to expand to 30 bots this week as muling is going to be a big pain in the ass if I do (because I currently have to PM each bot to activate their muling logic). Instead I'm going to integrate communication with a MySQL server.  This should allow me to:
- Quickly monitor status of each bot
- View which accounts have been banned and which are in the pipeline
- View what accounts were doing when they were banned
- Better manage IP and Mules between bots
- Communicate between bots to facilitate muling.

Once this is done The only big thing I'll have to do is write a manager to automatically fire up farmers and mules. Write a script for mules (should take almost no time).
And finally figure out automatic account creation.

9/28

Spoiler

Haven't made a ton of progress these past few days as I've been busy with work.  I did make a list of MVP (Minimal Viable Product, aka the minimum I need to feel comfortable expanding) features.

Required before expanding: (D: Done, P: In Progress)
[D] Script to farm
[ ] Script to mule
[P] Automatic account replacement when bans come in
[P] Way of keeping track of accounts so I can quickly tell which have been banned/locked and which are ready to be farmed
   - [D] SQL DB to maintain data
   - [P] Java end to update database and replace accounts as they're banned
   - [ ] Intelligent way for java end to be notified when DB changes (so I can use the DB to notify bots it's time to mule)
[ ] Script to get quest points
[D] New account source (The long term goal is still to make these all myself as I already have the tutorial island script done.  However, after dealing with locks whenever I bot them through tutorial island on a datacenter IP, I've decided to simply buy tutorial island finished level 3 accounts for this first expansion)

The little work I did do was around the SQL database.  I have the schema defined and the database built, and I'm about 70% of the way through integrating it into my Java "Account manager" which is responsible for starting bots, replacing them and updating their status in the database when they're banned or locked, and notifying a bot when it's time to mule.

 

Edited September 28, 2017 by Colonel_Panic

[Runnwith]

PogChamp

This is going to be good. I'm excited to see your progress. Good luck and I believe in you.

[SetYaName]

Goodluck man !

[GPSwap]

Good luck!

[Eagle Scripts]

Goodluck with your project

[Fearsy]

Best of luck

[Apaec]

Nice read, good luck!

Also, if you haven't already, be sure to check out Explv's tutorial island script source, found here https://github.com/Explv/Tutorial-Island/tree/master/src

Apa

[Colonel_Panic]

28 minutes ago, Apaec said:

Nice read, good luck!

Also, if you haven't already, be sure to check out Explv's tutorial island script source, found here https://github.com/Explv/Tutorial-Island/tree/master/src

Apa

Thanks Apa! I don't know how I missed that.

I have a working prototype currently, but I'll definitely go back and look through Explv's code to see what I might be able to improve on.

[Code Hero]

Just now, Colonel_Panic said:

• Develop script that can do tutorial island  (DONE)
• Write script to mine clay 

It's funny that you finished a very complex script, yet the "hello world" clay miner is still under development. 

Good luck with your project! 

[uta]

Goodluck

[Colonel_Panic]

48 minutes ago, Code Hero said:

It's funny that you finished a very complex script, yet the "hello world" clay miner is still under development. 

Good luck with your project! 

Haha that's because I started with a roughly working copy thanks to lisabe96.  There were definitely some bugs and some unrealistic actions it took that I had to fix, but the core code was there already.  It's always easier to add on and fix an existing code base than to start from scratch ;)

They clay miner is 100% my code and I'm trying to do it only using the API documentation as a resource (vs looking back at the tutorial island stuff or other open source scripts).

Anyways, the clay miner should be done tonight.  Just working out some kinks that I missed on version 1 that could lead to it getting stuck.

Edited September 19, 2017 by Colonel_Panic

[Colonel_Panic]

Update: Clay miner is "sorta" done.

It's working as well as it can, but there is a big problem that I didn't foresee (see bottom)

Features:

• Toggles run energy if I have above a certain threshold (random so it's not always toggling at same value)
• Hops world if there are too many other players competing at clay
• Uses correct pick axes for level
• Banks -> Mines -> repeats

 

HOWEVER. The big problem is that there are just too many F2P clay mining bots.  I had no idea how many there actually were, and I'm not able to get any clay as my mining level is too low.  Looking around they all seem to be using rune pick axes, so I'm thinking I'll need to have it power mine somewhere else to get some levels before actually trying to get clay.  If anyone has any input here, I'd appreciate it.

I might just toss this script and find something else in F2P to bot.  That'll set back my time frame for working on mule scripts, but it seems better than pushing forward with Clay when it's already saturated with bots.  

Edited September 19, 2017 by Colonel_Panic

[Colonel_Panic]

So I've moved on to working on a new script to fish in f2p to hopefully avoid some of the issues with competition that I saw with mining.  

Ideally it will be done very quickly, but there's some architecture flaws in my tutorial island script that are really annoying the dev side of me.  They aren't causing any real problems, especially since I don't plan on releasing any of these early scripts, so my eyes will be the only ones seeing them.  But still, odds are I'm going to get tired of looking at them and I'll need to remake some stuff to get it to an acceptable standard.  

For those of you script writers who care:  The original RQ Tutorial island solver used a state interface class, and each new script/task would introduce a new child of that class.  The actual state data was kept in a string.  I hated using and comparing strings for state data, so I switched to enums, but that essentially breaks the whole parent-child relationship (and purpose of having a state interface in the first place) as child classes can't add new enum values to their parents.  



Also, I've been brainstorming some simple ways to communicate between bots.  Might not be useful, but it would at least be a cool feature with some potential use cases.  I'm thinking something like keeping bots spread out across worlds and notify each other if certain alert conditions are met (e.g. JMod seen in certain world, so other bots are notified to avoid that world for a set time).  Probably won't work on this for a bit, so I added it to the long term goals for now.

Edited September 20, 2017 by Colonel_Panic

[Colonel_Panic]

Update:

Fishing script v1 is done. As is the process of cleaning up the faulty architecture around script states that I mentioned before.

It still will need work before I can really gold farm on it as currently it won't automatically move on to lobsters as it won't have the required lobster pot.  I figure I'll either have it automatically run to the GE and sell off all loot and buy a lobster pot, or I'll take this an excuse to automate some of the muling process to have a mule give them lobster pots once they reach fishing level 40.

I also am currently fishing in Al Kharid and I want to find some creative solution to deal with the scorpion that roams around there; currently I simply run away if it attacks me.  

[Colonel_Panic]

Big progress!

 

My Fisher now is completely automated. I haven't been able to truly test it 100% as I haven't had an account run for longer than 24 hours to get rid of the F2P trade restrictions (to test mule features fully).

 

However, I can create an account, type the credentials into osbot and leave it to:

-finish tutorial island

-sell items to afford to buy required fishing gear

-fish until 40 fishing

-fish lobsters until I tell it to mule

 

I'll hopefully get some long runtimes and progress this weekend and take some pictures.

 

This is a big stepping stone as it not only is essentially "gold farm ready", but it really reassured me that I should be good to go developing any more complex scripts in the future.

 

 

At this point I'm going to need to do some more logistical planning to figure out if I want to just suicide bot f2p at a larger scale (more dev ops/ban management related work), or write something for a P2P method and spend less time fighting mass bans. I'd love some input if there's anyone out there still reading this

 

 

Also, I finally set a money goal. For now it's a meager 10 mill fully automated as a short term and the long term is 100m. Just wanted to keep my goals very realistic at this stage as I'm sure I'll run into more problems with bans and system management as I scale up.

 

Edited September 22, 2017 by Colonel_Panic