We Can Do Better: Node System

[Bobrocket]

Educating OSBot, one rant at a time

EDIT: This tutorial is meant for people who already have some grasp on how to write a script. If you are completely new to scripting, this tutorial is not for you.

The node method of making scripts is definitely by far one of the best methods out there, but it has so many flaws that make for bad programming practices. Imagine you're making a pickpocket script, you might have the following nodes:

EatNode

WalkToBankNode

WalkFromBankNode

BankingNode

PickpocketNode

Now, given how most node-based scripts work, it simply does a for loop on each node and runs the first node in the list that can be executed. So if we enter our nodes in like this

PickpocketNode, BankingNode, WalkToBankNode, WalkFromBankNode, EatNode (EatNode being last, PickpocketNode being first)

we have actually just have a big issue: eating is the lowest priority. The idea of nodes is to keep the logic for one thing self-contained within another, but if we enter in our EatNode last, we will need to check to make sure our health is high enough in PickpocketNode (to ensure we don't, you know, die). This means you either ship along some global statics (bad!!!!), a script settings object (good!!!) or just forget about it.

Now, what if I could tell you that we can prioritise our nodes based on what happened last? Now we can say have our eat node a very high priority after we've walked to the bank (maybe we're in DMM and have 1 food left but low hp) and very high priority after we've pickpocketed something (because we may have just taken damage), but fairly low priority otherwise. We can constantly give our pickpocket node a high priority, and run our walk from bank node immediately after we've banked.

This does a few things for us:

• We don't have static priority - this is great because we as humans don't have static priority for things either!
• We no longer rely on the order we put nodes in to our list, we only care about when they should be ran

Now, the node system I propose isn't perfect, but it's a damn sight better and provides us a lot more legroom for upgrading in the future. Also, this will get you a lot more comfortable with some of Java's more advanced features, namely annotations which make every high level programmer cum immediately.

 

The Goal: Make a flexible node system that has dynamic priority

The Result: By the end of this, you will have a working node system with two example nodes (ImmediateNode and DefaultNode) which will show you how flexible the system is.

Lazy Kids: Leave now. All code here has been screenshotted so you can't copy/paste it. Learn something or GTFO

 

Step 1: How the Fuck Will We Do This?

We need to decide how we're going to store things, and how things will be written in code.

For this project, we need a manager of some sort that handles the sorting of nodes, we need something to handle priority, and we need to determine how we will denote prioritisation.

For this, we will create a NodeManager class, a Priority enum, and we will cover prioritisation in step 2.

What the fuck is this? This enumerated type is less of an enumerated type and more of a... well.. class. However, this "class" only has a set number of values! This means that we can only specify LOWEST, LOW, DEFAULT, HIGH, HIGHEST, IMMEDIATELY. If you want to be an absolute madman, you can add additional priorities in this file.

Well, you lost me. I understand NONE of this. This is actually really simple! This class stores references to a list of NodeObjects (come to that later), our last executed node, as well as a "default" or empty node (which we will return when we can't execute anything else to avoid nasty nulls), and finally a Comparator. A comparator simply compares two objects. In this case, we want to compare the priority of two different nodes.

We will also create an interface called Node.

This looks nothing like my Node class! That's because this is an interface. In programming, an interface is meant to represent the barebones object (in simpler terms, an interface is a blueprint for an object). Read the big documentation comment at the top of it if you want to sneak in a Script instance.

 

Step 2: Decide How to Manage Prioritisation

Now, we need to create a way to determine the priority of our nodes. There are two real ways to do this:

• A second parameter in our NodeManager#addNode(Node node) method, which would have every dynamic priority attached. This could actually get very messy, so I'm not even going to explain it better.
• We can attach annotations to each Node class we write, which keeps the logic contained and doesn't clog up our methods with useless garbage.

Step 2.1: What's an Annotation?

In Java, we have these nice little things called "annotations", and they make programming in Java a whole lot nicer. Annotations are effectively little nuggets of code that annotate our methods, variables and classes. They're great because they bridge the gap between human-readable code & complicated data structures. We've actually come across annotations before, at least if you know what a ScriptManifest is.

We're going to use annotations here and we're going to love it.

Step 3: Write Annotations

We're going to need two annotations here: a bog standard PrioritisedNode annotation, and a Condition annotation (I'm calling mine "After" in this implementation)

Hey, I kinda get this! It's very simple, but is also very, very powerful.

Holy fuck, this is similar! That's because it is.

Step 4: Write a Node!

We're going to write an "ImmediateNode". This node will have an IMMEDIATE priority, but after it is executed it will have a LOW priority (so that something else can execute!).

Oh fuck, you lost me. Now, this is a normal node (written just like you'd normally write a node), except we've plugged in our annotations that we just made. The first annotation (PrioritisedNode), says that by default we have an IMMEDIATE priority. The second annotation (After) says that after we execute ImmediateNode (this node), its priority is set to LOW.

Step 5: Write another Node!

Oh, fuck! We can use this to write other nodes, too! We're now going to write a "DefaultNode". This node will always have the DEFAULT priority.

Bitch, did you just gender your code? Damn straight I did. Notice how we omit the (priority = <something>) part in our annotation here? This is because we're using the default value we set earlier! We also set no After annotation, which means it will never have a different priority.

Step 6: What do we do now???

Well now that we've complicated things, we're going to need to update our NodeManager class. We need to add a few more things:

• A NodeObject class, this will allow us to track these annotations and make them in to something a little more computable
• A getNextNode method, which will get the next possible node to execute.

We'll start with the getNextNode method (within NodeManager).

Why do we sort? Why do we loop? WHY? We call Collections#sort(List, Comparator) to sort the list based on our previous condition - the comparator we made earlier! This doesn't return anything, but instead modifies the list we pass through. We iterate because we also need to find nodes that we CAN execute, otherwise we may just be executing garbage we can't do. We also set our lastNode variable provided we find a node, so that we can properly calculate our priorities next run.

Step 7: Objectify the Nodes

We're going to create something called an inner class - this is a class that is special to another class, kinda like you are to your parents. We're going to insert an extra class statement at the very bottom of NodeManager (but not outside the last bracket!) - this keeps things cleaner, especially because we don't want to access this class outside the manager.

?????????????????? This is a bit more complicated to explain, so we're going to ignore the constructor. Instead, we'll focus on getPriority(NodeManager) - if the last node doesn't exist (it's null), we return our default priority. Otherwise, we return the priority given by our After annotation, but if that doesn't exist we return our default priority.

Step 8: ???

Step 9: Profit!

We have now created our node system! We can use it like so:

And, hopefully, after all this hard work, we'll get this output in our logger box:

Notice how, although we added DefaultNode first, it ran ImmediateNode first? And furthermore, even though ImmediateNode has a priority of IMMEDIATE, it doesn't get ran that second time? That's because of the After condition we put in that!

Conclusion

Nodes are great, but they aren't perfect. So I made them perfect. Use this in all kinds of scripts, and claim you wrote the code yourself. Be proud of yourself, you just actually read a tutorial in its entirety.

 

Exercises (ie things I was too lazy to type up)

• Java does not allow two annotations of the same type to exist on a single object. This means one node can only have one priority change - how can we make one node have many priority changes? (Hint: make a third annotation whose only value is an array of After annotations, and iterate through them in NodeManager)
• Can you expand the code to do more things than just changing priority after one node executes?

Edited May 28, 2017 by Bobrocket

[k9thebeast]

Cool i'll put this on my list to read but I've been addicted to tasks for a while. Coming from my horrible getState() designs. This will probably be my next fetish

edit: not that I've read yet but it always helps to put a github up or some pastebins just so noobs can copy and paste code and start editing it to fit their needs

Edited May 28, 2017 by k9thebeast

[Bobrocket]

1 minute ago, k9thebeast said:

Cool i'll put this on my list to read but I've been addicted to tasks for a while. Coming from my horrible getState() designs. This will probably be my next fetish

edit: not that I've read yet but it always helps to put a github up or some pastebins just so noobs can copy and paste code and start editing it to fit their needs

People won't learn if all they do is copy and paste. After a while I will publicise it on my Github, but I want people to actually read it and learn something. It'll take you maybe 5 minutes to copy all the code from the screenshots anyway.

[Swizzbeat]

This is just an overcomplicated PriorityQueue.

[Bobrocket]

7 hours ago, Swizzbeat said:

This is just an overcomplicated PriorityQueue.

Actually, no.

I'm going to assume you don't know what a Queue in programming is just so other people can understand why you're wrong (although I can see why you would be confused).

A queue is a "First In First Out" collection. This means that if you enter a queue first, you leave it first. A queue, unlike a list, has special methods to allow this functionality: #peek() (will check the first element in the queue, but not remove it) and #poll() (will check the first element in the queue and remove it).

A queue is effectively the same as a queue in real life, but in code. When programming in an OO fashion, what you want to do is try and represent the problem in code as close as you can to the real life problem, hence why queue, stack etc all exist.

Now, our system here is not a queue, because no elements ever leave it. We don't instantiate new elements, we only keep the old, original nodes we created. Now, you could argue that we can just use the queue's #peek() functionality to get the first element and never remove anything, right? Although this is perfectly correct, it isn't exactly best practice.

Now, PriorityQueue would offer a performance increase, but not by much. A PriorityQueue is based on the heap data structure, because it does something really clever: it only "sorts" the first element. This is because it's not computationally viable to sort on every insertion, but we know that no matter what based on our definition of a queue, that the first element in the queue needs to go out first. The only difference to a standard queue is that a PriorityQueue isn't necessarily First In First Out, because it sorts the items and kiiinda violates the whole idea of a queue anyway (it's actually really a heap like I said). PriorityQueue's are great because they can actually "restore" the heap after removing the first element in about O(log(n)) which is pretty good, considering a lot of sorting algorithms are O(nlog(n)) or more.

Anyway, on to the actual issue at hand, a PriorityQueue (in Java) will only sort once #poll() is called. Furtherfore (I may be wrong here but I believe this is how it works), sorting the top element only occurs during insertion. Again, this is because it's not computationally viable to sort the entire thing on insertion. So, to correctly implement this as a PriorityQueue, we would likely need to recreate a queue every time we call #getNextNode() in our NodeManager class. Now, you could argue that this will actually run in O(log(n)) time vs O(nlog(n)) time (since Collections#sort() uses QuickSort), the extra object creation isn't really ideal and may actually slow the script down more.

Thanks for bringing this up though so that I could clear it up. If anything I said is wrong please let me know, I just woke up.

[Swizzbeat]

7 hours ago, Bobrocket said:

Actually, no.

I'm going to assume you don't know what a Queue in programming is just so other people can understand why you're wrong (although I can see why you would be confused).

A queue is a "First In First Out" collection. This means that if you enter a queue first, you leave it first. A queue, unlike a list, has special methods to allow this functionality: #peek() (will check the first element in the queue, but not remove it) and #poll() (will check the first element in the queue and remove it).

A queue is effectively the same as a queue in real life, but in code. When programming in an OO fashion, what you want to do is try and represent the problem in code as close as you can to the real life problem, hence why queue, stack etc all exist.

Now, our system here is not a queue, because no elements ever leave it. We don't instantiate new elements, we only keep the old, original nodes we created. Now, you could argue that we can just use the queue's #peek() functionality to get the first element and never remove anything, right? Although this is perfectly correct, it isn't exactly best practice.

Now, PriorityQueue would offer a performance increase, but not by much. A PriorityQueue is based on the heap data structure, because it does something really clever: it only "sorts" the first element. This is because it's not computationally viable to sort on every insertion, but we know that no matter what based on our definition of a queue, that the first element in the queue needs to go out first. The only difference to a standard queue is that a PriorityQueue isn't necessarily First In First Out, because it sorts the items and kiiinda violates the whole idea of a queue anyway (it's actually really a heap like I said). PriorityQueue's are great because they can actually "restore" the heap after removing the first element in about O(log(n)) which is pretty good, considering a lot of sorting algorithms are O(nlog(n)) or more.

Anyway, on to the actual issue at hand, a PriorityQueue (in Java) will only sort once #poll() is called. Furtherfore (I may be wrong here but I believe this is how it works), sorting the top element only occurs during insertion. Again, this is because it's not computationally viable to sort the entire thing on insertion. So, to correctly implement this as a PriorityQueue, we would likely need to recreate a queue every time we call #getNextNode() in our NodeManager class. Now, you could argue that this will actually run in O(log(n)) time vs O(nlog(n)) time (since Collections#sort() uses QuickSort), the extra object creation isn't really ideal and may actually slow the script down more.

Thanks for bringing this up though so that I could clear it up. If anything I said is wrong please let me know, I just woke up.

Did you really just give me a lecture on what a queue is and what FIFO means lmao.

Iirc (don't do much in Java as it's a shit language) a PriorityQueue is sorted on every enqueue. All that needs to happen each loop is a new PriorityQueue created, elements added, #peek/poll.

You really can't argue about script optimizations as they're negligible compared to the bot client internal bottlenecks. We're not designing an HTTP server where every little heap optimization matters as to not overload the GC and increase latency.

You still really didn't give any argument against why your solution isn't just a glorified PriorityQueue so agree to disagree.

[Bobrocket]

3 hours ago, Swizzbeat said:

Did you really just give me a lecture on what a queue is and what FIFO means lmao.

Iirc (don't do much in Java as it's a shit language) a PriorityQueue is sorted on every enqueue. All that needs to happen each loop is a new PriorityQueue created, elements added, #peek/poll.

You really can't argue about script optimizations as they're negligible compared to the bot client internal bottlenecks. We're not designing an HTTP server where every little heap optimization matters as to not overload the GC and increase latency.

You still really didn't give any argument against why your solution isn't just a glorified PriorityQueue so agree to disagree.

I wanted to explain not for you but for others who are reading this and don't understand, so that maybe they can learn something else - note where I say "just so other people......".

You're totally right about what you said there about how PriorityQueue's are sorted, and I even said the exact same. The main reason it isn't a glorified PriorityQueue is because we never poll, the list remains constant sized. Plus, if we look at our code, in the #getNextNode() method we iterate from the top of the sorted list (highest priority) to the bottom, finding ones that #canExecute(). To do similar things with a Queue, we'd need to poll. This means queue recreation on every call to #getNextNode() (like we've both already said, numerous times) but I'm going to avoid performance issues since you raise a good point. I was never trying to say one method is more/less efficient than the other, just trying to give insight. I don't particularly care if a call in my script takes 3ms or 4ms, call it antiban :^ )

At the end of the day, it doesn't matter if it's an overglorified PriorityQueue or not, because it's a tutorial and offers a cleaner, more flexible alternative to the node system most people currently use. More importantly, it should help encourage people to look in to "cleaner code alternatives" such as enumerated types and reflection where possible. Reflection can do a lot of neat things and it's a shame people don't use it more in scripts (even though there's not really too much usage in it)

 

[Deceiver]

is there a better way to add nodes than what you have shown? seems super messy doing that especially if you have like 10+ tasks/nodes in your script. 

 

ot; nice guide me learn

off topic; me deceiver hi u

[Bobrocket]

6 hours ago, raijin said:

is there a better way to add nodes than what you have shown? seems super messy doing that especially if you have like 10+ tasks/nodes in your script. 

 

ot; nice guide me learn

off topic; me deceiver hi u

NodeManager#addNode(Node Class);

This is exactly the same as a normal node system, just with the added annotations. It's not messy at all, especially considering the fact that your other two solutions are to abstract a #getPriority() method in your Node class (which will lead to switch cases, other nasty code etc) or in NodeManager#addNode() you take a second parameter of all the priority conditions. Both of these are much messier, harder to edit and as a result less ideal.

Honestly, actually try the code and you'll see that it's not messy like you think.

[Jammer]

On 2017-05-29 at 1:05 AM, Bobrocket said:

we have actually just have a big issue: eating is the lowest priority. The idea of nodes is to keep the logic for one thing self-contained within another, but if we enter in our EatNode last, we will need to check to make sure our health is high enough in PickpocketNode (to ensure we don't, you know, die). This means you either ship along some global statics (bad!!!!), a script settings object (good!!!) or just forget about it.

Reviving an old thread. Why is making an object prefered over global statics? Does it have to do with cpu usage or is it just more convenient?

[dreameo]

16 minutes ago, Jammer said:

Reviving an old thread. Why is making an object prefered over global statics? Does it have to do with cpu usage or is it just more convenient?

I think in this context, having things that are global would mean some nodes rely on others (dependencies) which then defeats the purpose of self contained nodes. 

[Explv]

27 minutes ago, Jammer said:

Reviving an old thread. Why is making an object prefered over global statics? Does it have to do with cpu usage or is it just more convenient?

Because OOP.

Regarding the original post though, would not recommend.

Edited November 24, 2017 by Explv