Road to an automated 100m (+Writing my own scripts)

[Colonel_Panic]

First end to end communication successful

 

Here's what the whole sequence currently looks like:

1. Script tries to establish a connection with my custom server
2. Server accepts request and spawns a new thread to communicate with the script over a socket
3. Script asks for a new account over the socket
4. Custom server communicates with MqSQL Database, gets the next available account, and returns it to the script over the socket

I'm not sure what all I'll need/want to call over the socket, so I've built it in a pretty open fashion so adding new functions to the server and calling them from the script shouldn't require any real changes to the socket reader.

Edited September 30, 2017 by Colonel_Panic

[THS]

I can only revel in agony

[flexike]

Your progress is really dope, I wish you good luck! I'm currently a CS major, and gonna get my degree in a year if everything is going alright, and this topic is just motivating. I got into scripting like 2 days ago, but unfortunately I've only had basic java knowledge, so it's a work in progress.

Once again, best of luck and thank you for the motivation!

[thatguycalledrob]

Looking seriously good! It seems you're ahead of me on the moneymaking side of things.
goodluck!

[TL57]

Keep it up dude...

[ry4n]

On 9/19/2017 at 6:58 PM, Colonel_Panic said:

Introduction

Hi all. I'm new here, so I figured I'd give a bit of background about myself.

 

I'm a software engineer by trade (graduated with a CS degree several years ago and work full-time as a software engineer) and used to play a ton of Runescape.  In the past, I've tried to get into script development, but always got sidetracked and gave up without producing anything.

This time, I'm hoping to see it through by setting some short and long term goals to keep me motivated. 

I have experience in Java, although it's been a few years since I've used it; however, the OSBot devs did a great job with the API so I don't think it'll be too long till I'm making progress.

 

Short Term Goals

• Develop script that can do tutorial island  (DONE)
  • This should give me a lot of practice as it's doing a large variety of tasks
  • In order to not jump into this entirely blind, I've been looking at some open source tutorial island scripts as well (shout out to @lisabe96's RQ Tutorial Island Completer)
  • This may end up being trashed if the ban rate is too high, but either way the scripting experience will be valuable.
• Write script to mine clay  (DONE - With caveat.  See Week 1 updates for more info)
• Build infrastructure to facilitate combining scripts well, and combine tutorial island + clay miner (DONE)
  • This will aid in the future as I hope to have bots complete prerequisites, and then automatically jump to gold farming once those are finished
• Build script to automatically mule
• F2P Fishing Script (V1 Done, still a WIP to improve it to a gold-farm ready script)
• 10 mill automatically farmed and muled
• MySQL Server for managing bots and facilitating communication between them (see update for 9/25 for more info)

 

Long Term Goals

• Automatic account replacement as they're banned
• Alerts (via email, twitter, or text message) when something seems off
• Profit in the black (should be a relatively quick one to complete as I won't be paying too much on servers at the start)
• Communication between bots (so I don't end up with to many on the same world, or to alert others should something go wrong (e.g. JMod in a certain world)
• 100m profit

Progress

9/18:

  Reveal hidden contents

Spent a lot of time analyzing lisabe96's RQ Tutorial Island Completer (as well as some other Tut island scripts, but lisabe96's was the best I've found).  

I started by simply watching the script run and writing down any bugs or weird issues that looked off.  

Once I had a list, I set out to fix all the bugs and get rid of some of the "non-human" actions (e.g. clicking on an NPC once we've already started talking to them, or trying to spam click objects that only need to be clicked once).

Once this was done and I had several bots complete the island with no problems, I introduced my own form of anti-ban that introduces randomness to the actions so they're more human like.  This may do nothing, but it was fun to experiment with nonetheless as I was able to tweak it until it looked passable.  

I'm currently working on adjusting this now so that once it finishes tutorial island, it will move on to a new task.  For now, this task will be mining clay as it should be an easy introductory script for me to write from scratch.  I'm hoping to have this up and running within a day so I can fire up a few suicide bots and get an idea on how long they'll last before being banned (hopes aren't high at this point).

UPDATE 2 (9/18)
Clay miner is "sorta" done.

It's working as well as it can, but there is a big problem that I didn't foresee (see bottom)

Features:

• Toggles run energy if I have above a certain threshold (random so it's not always toggling at same value)
• Hops world if there are too many other players competing at clay
• Uses correct pick axes for level
• Banks -> Mines -> repeats

 

HOWEVER. The big problem is that there are just too many F2P clay mining bots.  I had no idea how many there actually were, and I'm not able to get any clay as my mining level is too low.  Looking around they all seem to be using rune pick axes, so I'm thinking I'll need to have it power mine somewhere else to get some levels before actually trying to get clay.  If anyone has any input here, I'd appreciate it.

 

9/19:

  Reveal hidden contents

Fishing script v1 is done. As is the process of cleaning up the faulty architecture around script states that I mentioned before.

It still will need work before I can really gold farm on it as currently it won't automatically move on to lobsters as it won't have the required lobster pot.  I figure I'll either have it automatically run to the GE and sell off all loot and buy a lobster pot, or I'll take this an excuse to automate some of the muling process to have a mule give them lobster pots once they reach fishing level 40.

I also am currently fishing in Al Kharid and I want to find some creative solution to deal with the scorpion that roams around there; currently I simply run away if it attacks me.  

9/22:

  Reveal hidden contents

My Fisher now is completely automated. I haven't been able to truly test it 100% as I haven't had an account run for longer than 24 hours to get rid of the F2P trade restrictions (to test mule features fully).

 

However, I can create an account, type the credentials into osbot and leave it to:

-finish tutorial island

-sell items to afford to buy required fishing gear

-fish until 40 fishing

-fish lobsters until I tell it to mule

 

I'll hopefully get some long runtimes and progress this weekend and take some pictures.

 

This is a big stepping stone as it not only is essentially "gold farm ready", but it really reassured me that I should be good to go developing any more complex scripts in the future.

At this point I'm going to need to do some more logistical planning to figure out if I want to just suicide bot f2p at a larger scale (more dev ops/ban management related work), or write something for a P2P method and spend less time fighting mass bans. I'd love some input if there's anyone out there still reading this 

 

Also, I finally set a money goal. For now it's a meager 10 mill fully automated as a short term and the long term is 100m. Just wanted to keep my goals very realistic at this stage as I'm sure I'll run into more problems with bans and system management as I scale up.


Night update:

https://imgur.com/a/O4Ckl

 

Have 3 characters fishing lobsters and just fired up 2 more. Will leave these going until I encounter a problem.  I'll hopefully post follow up pictures tomorrow with longer progress stats

9/23

  Reveal hidden contents

https://imgur.com/a/QjcOZ

All bots still running. One of the fresh accounts got 40 fishing properly, sold gear to get a lobster pot,  and made it to karamja to start fishing.  The other one was stuck at the GE this morning, so that's definitely something I need to look into.

I'm also getting a few log-outs after an EventExecuter timed out.  It logs back in and continues fine, but still that's another thing I need to look into.

The current plan as of now is to spend some time cleaning and optimizing the code.  I have a feeling some of the weird bugs I'm seeing are due to timing issues and inconsistencies.  I also want to look more into conditional sleeps as those seem to be the way to go for some of the stuff I'm doing.

 

Once that's done, I plan to scale up on a VPS to around 10 bots.  Hopefully this will be done before tomorrow so I can get some run time before bans start coming on Monday.  

9/24

  Reveal hidden contents

Update:

Ended out the weekend with just shy of 35k lobsters from 5 accounts.

I tested out auto muling just now and was very pleased to have 4/5 bots mule properly.  One got stuck in the bank and died to an NPC and I'm 90% sure that was due to me maxing out the CPU on the VPS I'm using, so code wise I think I'm still okay.

 

I spent a few hours today working on building my own proxy server so I can supply my own proxies.  I've decided that I'm going to go with a dedicated server and up the number of bots I'm running to around 30.

I'm still curious to see how things change during the week when bans come more frequently.  I'm guessing I'll need to add the ability to get 7 qp on the accounts for them to be able to offload before the bans come down; but I'm going to experiment a bit this week to see if any last more than 24 hours.  If they do, then I won't worry about questing for now. 

 

I'll experiment a bit more this week and then once I'm fully ready I'll pull the trigger on a dedicated server and try to push the limit on what I can do.  

9/25

  Reveal hidden contents

Decided to pull back on my plans to expand to 30 bots this week as muling is going to be a big pain in the ass if I do (because I currently have to PM each bot to activate their muling logic). Instead I'm going to integrate communication with a MySQL server.  This should allow me to:
- Quickly monitor status of each bot
- View which accounts have been banned and which are in the pipeline
- View what accounts were doing when they were banned
- Better manage IP and Mules between bots
- Communicate between bots to facilitate muling.

Once this is done The only big thing I'll have to do is write a manager to automatically fire up farmers and mules. Write a script for mules (should take almost no time).
And finally figure out automatic account creation.

9/28

  Hide contents

Haven't made a ton of progress these past few days as I've been busy with work.  I did make a list of MVP (Minimal Viable Product, aka the minimum I need to feel comfortable expanding) features.

Required before expanding: ( Done, P: In Progress)
[D] Script to farm
[ ] Script to mule
[P] Automatic account replacement when bans come in
[P] Way of keeping track of accounts so I can quickly tell which have been banned/locked and which are ready to be farmed
   - [D] SQL DB to maintain data
   - [P] Java end to update database and replace accounts as they're banned
   - [ ] Intelligent way for java end to be notified when DB changes (so I can use the DB to notify bots it's time to mule)
[ ] Script to get quest points
[D] New account source (The long term goal is still to make these all myself as I already have the tutorial island script done.  However, after dealing with locks whenever I bot them through tutorial island on a datacenter IP, I've decided to simply buy tutorial island finished level 3 accounts for this first expansion)

The little work I did do was around the SQL database.  I have the schema defined and the database built, and I'm about 70% of the way through integrating it into my Java "Account manager" which is responsible for starting bots, replacing them and updating their status in the database when they're banned or locked, and notifying a bot when it's time to mule.

 

I’m currently in the middle of a very similar project. Instead of Java and MySQL I’ll be using a Python & MongoDB backend for the account management / proxy management. I haven’t had too much experience with Java however so do you have any pointers on that side of things? 

[tarxan]

congrats on your progress, i plan to start writing my own scripts but no coding experience.. Should i learn Python first?

[Hernandez2000]

On 8/1/2019 at 6:46 PM, tarxan said:

congrats on your progress, i plan to start writing my own scripts but no coding experience.. Should i learn Python first?

I think hes been away from the forum.

But osbot uses java so you should prob start with java. When they talked about python they were talking about back end (server) stuff which is less important for just making scripts and stuff. But I wouldnt worry too much about which language you start with first because a lot of programmers learn multiple because a lot of the core programming skills and knowledge transfer from languages pretty well. But if you are just learning programming stuff to make bot scripts I def vote java, focusing on one will help a lot.

But take my advice with a grain of salt I'm a pleb tier web developer who originally tried to learn C# for unity and then java for android apps

[justselling1acc]

This is very cool. I hope you achieve your goals! Are there anymore updates coming soon? I would like to see your progress.