OSTracker - User data tracking for OSBot scripts

[Tom]

About

OSTracker is a Client-Server implementation designed to track important data from people using your scripts. It has been designed in a way that allows multiple clients (scripts & users) to upload and store data to one server which runs standalone from OSBot and utilises MySQL 8. OSTracker is allowed on the SDN.

Tracked Data (per user)

• Experience gained for all skills
• Items Received, Lost, or Spent (e.g. arrows and runes)
• Time ran
• Error logs (if setup)

Future Data

• Script configurations

Requirements

For the server, you will need the following

• Java JRE 8+ (Download)
• MySQL Server 8 (Download)
• Ability to build the jar using maven
• A server to run OSTracker on
• OSTracker-Server code (Download)

For the client, you will need the following

• A script
• The OSTracker client code within your script
• OSTracker-Client code (Download)

Installation

MySQL Server

The MySQL 8 Server is the place all the user data is stored, and is required by OSTracker to function. MySQL Server should ideally be installed on the same server that the OSTracker server will run on, however that is not a requirement and it will work remotely with the correct setup on your end.

If you are using a Linux based operating system, I would recommend looking up specific instructions for setting up MySQL Server. If you are using Windows, you can see the instructions below.

Spoiler

1. Download the MySQL Community Server installer using the link displayed in the requirements above.
2. Run the installer
3. Add the latest version available to the right hand list using the arrows in the middle of the installer

4. Once on the Installation page, press execute to begin the installation of MySQL Server 8, it may take a little while to complete, but once it has, press next.
5. On the product configuration page, click next to configure MySQL Server 8
6. Continue with the default option (Standalone MySQL Server / Classic MySQL Replication)


7. On the connectivity page, select the config type that best suites your environment. You can then configure any port settings to the port you wish to use.
Please note that if you are running the OSTracker server on a different server to the client, you will need to port forward the MySQL port, and enable remote SQL for all addresses (see instructions online)

8. Continue with default values until you arrive at the 'Accounts and Roles' page. On this page, you will be required to set a password for the root MySQL user. Set this password to anything you desire, but make sure you remember it as it will be required during the setup of the OSTracker Server.

9. Continue to the end of the installation with the default values, to ensure the MySQL server always runs on your windows machine.
10. Click execute on the Apply Configuration page, then click Finish.
11. Done
 

OSTracker Server

To run the OSTracker server, you will need a Windows/Linux-based server to run it on. Depending on your users, having a server is ideal as it will allow you to track data 24/7.

Implementing the Server is quite easy using the instructions below.
 

Spoiler

1. Download the source for OSTracker-Server (Download)
2. Open the project in the IDE of your choice, I personally use IntelliJ. Make sure you open it as a Maven project
3. You will need to modify Main.java to use the correct connection details for your MySQL Server. If you setup MySQL Server using the instructions above, this is where you will need to remember the password you created for root. e.g.
 

public static void main(String[] args){
    System.out.println("Server started...");
    MySQLServer.getInstance().setMySQLCredentials("127.0.0.1:3306", "root", "123");
    MySQLServer.getInstance().setMySQLUserCredentials("myusername", "mypassword");
    server = new Server(PORT);

}

4. It is highly recommended that you use the inbuilt functionality to create a MySQL user specifically for OSTracker, to do this, you will need to uncomment line 18 of Main.java, and provide the login credentials you wish to use. Please note that once you set the credentials, you will need to connect at least once with the root user to setup the new user.
5. Once the settings match your configuration, you will need to build the jar to run on your server. You need to do this with maven using the 'install' argument. If you are not familiar with how to build a maven project, I recommend doing a quick search as it is quite straight forward in most IDE's.
6. Upload the compiled .jar file (make sure its the with-dependencies version) to the server of your choice, and run the following command to start the server 'java -jar OSTracker-Server.jar'

OSTracker Client


The client implementation goes directly into your script, specifically into the onStart and onExit(), but you will need the entire OSTracker Client project source within your script.

Instructions:

Spoiler

1. Download the source for OSTracker-Server (Download)
2. Implement the project code into your script, it just needs to be accessible from the script main class.
3. Insert the relevant code from this snippet, you will need to create a Tracker variable, and you will need to copy the code from onStart and onExit for the Client to function correctly.
 

private Tracker tracker;

@Override
public void onStart(){
    try {
        tracker = new Tracker(getBot(), getName()).establishConnection("127.0.0.1", 1337) // Mandatory
        .setupMysql() // Mandatory
        .setUpdateInterval(0.3) // Defaults to an hour
        .start(); // Mandatory
    } catch (InvalidSetupException e) {
        if(tracker != null){
            tracker.stop();
        }
        e.printStackTrace();
    }
}

@Override
public void onExit(){
    if(tracker != null) {
        tracker.stop();
    }
}

 

4. Make sure the configuration settings match the OSTracker Server IP address, and if necessary, the port is open.
5. Done, easy as that.

Accessing the data

There is no easy way to explain this, but you will at least need some experience with SQL to reliably view your data in the most accurate and effective way. Once you have connected to your MySQL Server, whether its through terminal or a third party software such as MySQL Workbench, you gain access to powerful queries that will allow you to view the tracked data in all sorts of ways.

I have included some example queries below.

Viewing all items received by all players in the last 24 hours

SELECT username, itemname, status, sum(amount) AS cnt from `osbot-client`.`users`
    inner join `osbot-client`.`scriptitems` on `osbot-client`.`users`.id = `osbot-client`.`scriptitems`.user
    inner join `osbot-client`.`items` on `osbot-client`.`items`.id = `osbot-client`.`scriptitems`.item
    inner join `osbot-client`.`itemstatus` on `osbot-client`.`scriptitems`.itemStatus = `osbot-client`.`itemstatus`.id
where status = "Received" and date > date_sub(date, INTERVAL 1 DAY)
group by status, itemName
having cnt > 1;

Viewing Total Time Ran for all users in milliseconds

SELECT username, sum(duration)
from `osbot-client`.`runtimes`
         inner join `osbot-client`.`users` on `osbot-client`.`users`.id = `osbot-client`.`runtimes`.user
group by username;

Viewing Total Experience Gained for each user for a specific skill

SELECT username, skillName, exp
from `osbot-client`.`experiencegained`
         inner join `osbot-client`.`users` on `osbot-client`.`users`.id = `osbot-client`.`experiencegained`.user
         inner join `osbot-client`.`skills` on `osbot-client`.`skills`.id = `osbot-client`.`experiencegained`.skill
WHERE skillName = "STRENGTH"
group by username;

 

Advanced

Error Tracking

It's possible to track and upload things such as Stacktraces which allows scripters to get ahead of issues quickly before users even report it, see the example below.
In your onLoop, you could have something like the following

try {
   // Logic Loop
} catch (Exception e) {
    if (tracker != null) {
        StringWriter sw = new StringWriter();
        PrintWriter pw = new PrintWriter(sw);
        e.printStackTrace(pw);
        tracker.getSessionTracker().getSession().addLog(sw.toString().replaceAll("\r", "").replaceAll("\n", ""));
        tracker.getSessionTracker().getSession().setVersion(getVersion());
    }
}

Edited December 16, 2019 by Tom

[Czar]

Epic, unexpected release, good job mate.

[Chris]

kekw

[Tom]

1.01

Added logging support (advanced implementation, not automatic)
Grouped items together in each session, so rather than having 20 entries of 1 bone, it will be 1 entry of 20 bones (if it was found in the same session)

[Juggles]

Wow this is impressive Tom. Very much above your knowledge 

[BeepBooPvm]

I might have a go and create a container and an Ansible role to set this up, would that be ok @Tom ?

[Tom]

4 minutes ago, gay_retard said:

I might have a go and create a container and an Ansible role to set this up, would that be ok @Tom ?

Lots of different ways to do things, I dont see why not

[Ragnar Lothbrok]

Would you be alright with me building a web app that uses's this for the community - I plan on releasing the source code so people can run it independently 

[Tom]

On 4/2/2020 at 8:31 AM, Ragnar Lothbrok said:

Would you be alright with me building a web app that uses's this for the community - I plan on releasing the source code so people can run it independently 

Late response but go ahead

[danny mole]

amaing ideas!

[4head4head4head]

i thought sockets were blocked by osbot's security policy?

[Tom]

23 hours ago, 4head4head4head said:

i thought sockets were blocked by osbot's security policy?

Socket servers are I believe, however regular sockets are not