Working OCR - Thoughts?

[Alek]

I've always wanted a way to collect data visually and although this approach is impractical for many applications, there are a few advantages in having optical character recognition. Below is a short demonstration video in scanning the screen for the character 'E'.

 

https://www.youtube.com/watch?v=itYcXz5603w

 

This is my first time ever trying to make an OCR type of system and I am pretty clueless to what algorithms already exist. Searching a canvas for a single character alone consumes a ton of resources and ensuring that my search is optimized is very important. 

 

When developing the OCR search, I initially came up with something I called the ink blot. The ink blot would search pixel by pixel and try to create a shape based on the color of pixels which surround it. This shape would then be compared against the character which I was searching for and determined if they had identical points. The problem was clear, after about 500 pixels (I was searching over 250k pixels), creating shapes for each pixel point was too resource intensive.

 

The second search method which I created is something I'll refer to as the typewriter. The character which I'm looking for is essentially stamped at each position and checked to see if the colors at that stamp match the points of my character. This means the character 'E' was checked quickly over a few thousands times at varying positions on the screen. 

 

        ocr.setBot(getBot());
        if (ocr.getBot() != null) {
            OC character = ocr.findChar('E');
            if (character != null) {
                System.out.println("Found character at: " + character.getX() + "," + character.getY());
            } else {
                System.out.println("The character was not found!");
            }
        } else {
            System.out.println("The OCR bot is null");
        }

This was a fun project which took me a little under 3 hours to complete, so searching for full words and other features from a fully developed OCR system is not apparent.

 

With this said, would any script developers or botters find use from something like this? 

[Isolate]

Use: to tell when trying to use the auto account login if 1. you've been banned, 2. game updates, stop spamming login

[Alek]

Use: to tell when trying to use the auto account login if 1. you've been banned, 2. game updates, stop spamming login

 

Funny that you mention this because Maxi wanted me to work on OSRS update handling found here. There are a few codes which already exist that should be able to tell this.

[Soldtodie]

Use: to tell when trying to use the auto account login if 1. you've been banned, 2. game updates, stop spamming login

 

Just get 5 pixel and check if they are the same color = Finish.

[Mysteryy]

I've always wanted a way to collect data visually and although this approach is impractical for many applications, there are a few advantages in having optical character recognition. Below is a short demonstration video in scanning the screen for the character 'E'.

 

https://www.youtube.com/watch?v=itYcXz5603w

 

This is my first time ever trying to make an OCR type of system and I am pretty clueless to what algorithms already exist. Searching a canvas for a single character alone consumes a ton of resources and ensuring that my search is optimized is very important. 

 

When developing the OCR search, I initially came up with something I called the ink blot. The ink blot would search pixel by pixel and try to create a shape based on the color of pixels which surround it. This shape would then be compared against the character which I was searching for and determined if they had identical points. The problem was clear, after about 500 pixels (I was searching over 250k pixels), creating shapes for each pixel point was too resource intensive.

 

The second search method which I created is something I'll refer to as the typewriter. The character which I'm looking for is essentially stamped at each position and checked to see if the colors at that stamp match the points of my character. This means the character 'E' was checked quickly over a few thousands times at varying positions on the screen. 

 

        ocr.setBot(getBot());
        if (ocr.getBot() != null) {
            OC character = ocr.findChar('E');
            if (character != null) {
                System.out.println("Found character at: " + character.getX() + "," + character.getY());
            } else {
                System.out.println("The character was not found!");
            }
        } else {
            System.out.println("The OCR bot is null");
        }

This was a fun project which took me a little under 3 hours to complete, so searching for full words and other features from a fully developed OCR system is not apparent.

 

With this said, would any script developers or botters find use from something like this? 

 

 

In what way did you ultimately decide to compare the pixles? I am somewhat familiar with letter recognition using linear algebra. You can compare two vectors and finding the difference determine what character you are currently examining, or not examining. I have never really thought to do this with a bot client, but you kind of sparked an interest, maybe I will do it for fun once I have some time. :p

Just get 5 pixel and check if they are the same color = Finish.

 

 

This would be the worst character recognition system to date.

[Soldtodie]

 

This would be the worst character recognition system to date.

 

But for checking if you are banned it works perfect.

[Czar]

do you not response code m9s

 

https://rswiki.moparisthebest.com/index.php?title=317_Protocol#Response_Codes

 

just 1 byte

 

for system update theres a tiny packet which i'm sure isnt difficult to identify

 

Ontopic: GJ on ocr man, i remember talking about it XD

Edited March 1, 2015 by Czar

[Alek]

do you not response code m9s

 

https://rswiki.moparisthebest.com/index.php?title=317_Protocol#Response_Codes

 

just 1 byte

 

for system update theres a tiny packet which i'm sure isnt difficult to identify

 

This was not meant to replace response codes. 

[Czar]

I know , it was directed at the other posters here, i forgot to mention that 

Edited March 1, 2015 by Czar

[Joseph]

But for checking if you are banned it works perfect.

but there's already a method that tells us if the player has been banned or if there's an update

and more. Not exactly sure what method it is.

It's a method within either the bot or client class it returns an int and depending on the int is what is happening

Edited March 1, 2015 by josedpay

[pker123]

I have no idea how to sccript but it looks so cool good job!!!

[Eliot]

I suppose this is neat, but almost entirely useless. Instead, lets add industry standards to this bot like web walking.

[Cruel]

I suppose this is neat, but almost entirely useless. Instead, lets add industry standards to this bot like web walking.

 

 

^ this

[darkxor]

Yeah, sometimes simple OCR is very good, if bot have'nt some API for example, and need identify text or object on screen. But, algorithm chosen depends on task needed to solve. In botting, usually can do pixel-based search of specific color key pixel(s), then test possible candidate areas more precisely. It gives high performance.

[Apaec]

Oh nice, ocr is great