Poker-AI.org

Poker AI and Botting Discussion Forum
It is currently Mon Nov 13, 2023 12:49 pm

All times are UTC




Post new topic Reply to topic  [ 48 posts ]  Go to page Previous  1, 2, 3  Next
Author Message
 Post subject: Re: screen scraping
PostPosted: Mon Aug 06, 2018 5:42 pm 
Offline
Senior Member
User avatar

Joined: Sun Mar 10, 2013 10:31 am
Posts: 139
The main threat in recognition is the filters. Good filter are 95% of sucsess recognition.
Here is source (10x) and 3 filters.
1st just do black/white from orign
2nd gausian blur (it is very fast but my advice to write it yourself (few strings of code))
3rd simple treshold

Impotant thig that I doubled sise of image. It been done to gausian blur works well. And separating of digits works well and fast too. In my expirianse any doubling or tbling small images is wery fast, it is faster then time I can measure :)


Attachments:
filter2_10x.png
filter2_10x.png [ 82.3 KiB | Viewed 21863 times ]
filter1_10x.png
filter1_10x.png [ 47.16 KiB | Viewed 21863 times ]
orign_10x.png
orign_10x.png [ 17.97 KiB | Viewed 21863 times ]


Last edited by nefton on Mon Aug 06, 2018 6:01 pm, edited 1 time in total.
Top
 Profile  
 
 Post subject: Re: screen scraping
PostPosted: Mon Aug 06, 2018 5:44 pm 
Offline
Senior Member
User avatar

Joined: Sun Mar 10, 2013 10:31 am
Posts: 139
cant add more than 3 attachment so filter 3 and the code here
Code:
#include <iostream>
#include "opencv2/opencv.hpp"
#include "common_ocr_functions.h"

int main() {

   cv::Mat source = cv::imread("D:/PROJECTS/opencv_sample/example.png");

   //cv::imshow("source", source);
   //cv::waitKey();

   //ShowLarge10x("source", source, true);

   cv::Mat filter1(source.rows * 2, source.cols * 2, CV_8UC1);

   for (int row = 0; row < source.rows; row++) {
      for (int col = 0; col < source.cols; col++) {
         cv::Vec3b point = source.at<cv::Vec3b>(row, col);
         uint8_t b = point[0];
         uint8_t g = point[1];
         uint8_t r = point[2];
         uint8_t h, s, v;
         RGB2HSV(r, g, b, &h, &s, &v);
         filter1.at<uint8_t>(row * 2 + 0, col * 2 + 0) = v;
         filter1.at<uint8_t>(row * 2 + 1, col * 2 + 0) = v;
         filter1.at<uint8_t>(row * 2 + 0, col * 2 + 1) = v;
         filter1.at<uint8_t>(row * 2 + 1, col * 2 + 1) = v;
      }
   }

   //ShowLarge10x("filter1", filter1, true);

   cv::Mat filter2(source.rows * 2, source.cols * 2, CV_8UC1);

   cv::GaussianBlur(filter1, filter2, cv::Size(4, 4), 0.0);

   //ShowLarge10x("filter2", filter2, true);

   cv::Mat filter3(source.rows * 2, source.cols * 2, CV_8UC1);

   for (int row = 0; row < filter2.rows; row++) {
      for (int col = 0; col < filter2.cols; col++) {
         uint8_t v = filter2.at<uint8_t>(row, col);
         filter3.at<uint8_t>(row, col) = 0;
         if (v > 100) filter3.at<uint8_t>(row, col) = v;
      }
   }

   ShowLarge10x("filter3", filter3, true);

   std::cout << "\n\n";
   std::system("pause");
   return 0;
}


Attachments:
filter3_10x.png
filter3_10x.png [ 29.04 KiB | Viewed 21863 times ]
Top
 Profile  
 
 Post subject: Re: screen scraping
PostPosted: Mon Aug 06, 2018 5:52 pm 
Offline
Senior Member
User avatar

Joined: Sun Mar 10, 2013 10:31 am
Posts: 139
As we see in your sample of stack digits and point can be easy separated by vertical lines.
It is very good because it is very fast!
Think anybody can do this.
Next steps may be different.
Most easy to release is just cut numbers and point and resize it to some sise like 20x30
And then compare with template by calculating average difference by point


Top
 Profile  
 
 Post subject: Re: screen scraping
PostPosted: Mon Aug 06, 2018 6:16 pm 
Offline
Senior Member
User avatar

Joined: Sun Mar 10, 2013 10:31 am
Posts: 139
It is very good that by filters we can separate digits by vertical lines. Else it is wery dificult way. (Hope PS engeneers dont read this forum :) )


Top
 Profile  
 
 Post subject: Re: screen scraping
PostPosted: Tue Aug 07, 2018 11:41 am 
Offline
Junior Member

Joined: Thu Apr 05, 2018 2:57 pm
Posts: 23
@nefton

Thanks! I am going to study all your posts before i comment again. Its a lot to process for me. I am not familiar with manipulating images and scraping at all (i never heard of filters for instance :shock: ).

I did finish the easy part. My hole cards and the community cards are recognized 100% correctly now. So now its time to get the stack sizes recognized.. :mrgreen:


Top
 Profile  
 
 Post subject: Re: screen scraping
PostPosted: Tue Aug 07, 2018 12:12 pm 
Offline
Junior Member

Joined: Thu Apr 05, 2018 2:57 pm
Posts: 23
Hi,

Am i correct that your code just transforms the stack image for better processing later?

In my java program, i have transformed the stack image to a true black and white image. Below are 3 results.

1. original size
2. x4
3. x10

Can i use these? There seem to be black horizontal lines to seperate the digits. Or do i need to use the filter you suggested?

Second question. I don't understand why we need to scale the image up. I mean, the black horizontal lines will be there in the original x1 image as well.


Attachments:
blackAndWhite-x10.png
blackAndWhite-x10.png [ 292.93 KiB | Viewed 21851 times ]
blackAndWhite-x4.png
blackAndWhite-x4.png [ 46.97 KiB | Viewed 21851 times ]
blackAndWhite-x1.png
blackAndWhite-x1.png [ 1.55 KiB | Viewed 21851 times ]
Top
 Profile  
 
 Post subject: Re: screen scraping
PostPosted: Tue Aug 07, 2018 3:02 pm 
Offline
Junior Member

Joined: Thu Apr 05, 2018 2:57 pm
Posts: 23
ive studied a lot of digits. And all digits have the same height (14 pix). Which is nice..but the width seems to be different.

All except '1' and '4' the width is either 7 or 8. I have encountered at least 3 different digits of 7. But they all were 8x14.

My idea is to just make masks (one for each number,,,and one for each number which occurs at a different size)..

so because 7 always is 8x14 i wil make one mask. But 9 is either 7x14 and 8x14. So I will make 2 masks for 9. Then check
which mask has the highest match with actual screenshot. That mask is the winner. Or is this approach too slow? And maybe not accurate?
Cause i dont know how many other versions of the digits there are cause lack of sample size?


Attachments:
7-3.png
7-3.png [ 194 Bytes | Viewed 21848 times ]
7-2.png
7-2.png [ 194 Bytes | Viewed 21848 times ]
7.png
7.png [ 194 Bytes | Viewed 21848 times ]
Top
 Profile  
 
 Post subject: Re: screen scraping
PostPosted: Tue Aug 07, 2018 7:27 pm 
Offline
Junior Member

Joined: Thu Apr 05, 2018 2:57 pm
Posts: 23
it seems to work.

way i do it:

1. create black and white masks for all the digits. in my case digits have always height of 14. The width varies from 2, 5,7,8,9.


2. extract all possible digit regions from the stack (based on vertical black colums seperating digits)

for each digit region:
- if region size is 2, the digit is a '.'
- if region size is 5 the digit is always a 1. (other digits are always wider)
- if region size is 7: try to match vs 7 width masks -> pick best match
- if region size is 8: try to mach vs 8 width masks -> pick best match
- if region size is 9: try to match vs 9 width masks -> pick best match


Top
 Profile  
 
 Post subject: Re: screen scraping
PostPosted: Wed Aug 08, 2018 6:23 am 
Offline
Senior Member
User avatar

Joined: Sun Mar 10, 2013 10:31 am
Posts: 139
1. You show original b/w image "true" as you say. I call it binary.
But wen you show x4 and x10 there are smooth eges. it is not correct. Think you just scale it in paint or somthing else.
My adwice to write your oun simple function to correct scale x10

2. Your solution will work. And it will be wery fast dont wory about it. Even more. I`ll say it will be the fastest solution.
But it will work on "ege of working". Add some noise and it will fail.
When you compare your digit with template you will get a simple integer number of wrong compare points.
And there will be wery thick line betwin "8", "0", "6" in different variations.
You can solwe it by adding more then 1 "6" template for example. Let say You will have 4 "6" templates.
I do it wery long ago too ))
But in my opinion it is wrong and bay way.
It is not hard ant not long to get no binary but greyscale image. And doubling size solwe many problems with different "points". And then if you have large beautiful greyscale image of your digit you need to have only one beautiful template for yeach digit. And compare function will give you exelent results. Not "on ege of working".

3. For now you take screenshoot of your poker client. But soon you will decide that your bot is wery easy to diskover by poker client. So you will think about hardware solutions. And ones it appears you should be redy to work with noise, with not correct colors and ets.


Top
 Profile  
 
 Post subject: Re: screen scraping
PostPosted: Wed Aug 08, 2018 11:45 am 
Offline
Junior Member

Joined: Thu Apr 05, 2018 2:57 pm
Posts: 23
@tefton

Thanks! I run the bot on a linux machine. This machine is running virtualbox (windows 7) and here the poker client resides (and only the poker client). I take a screenshot in linux (also capturing the desktop of the virtual box). So i think
that is a pretty save solution. Is it not?

You are right about the scaling image i uploaded. Those were not correct. As of now, i don't scale my images. Just use the original. See how it goes. Up to now i have 100% accuracy for hole&board cards, and stack size. But i need to test it with more screenshots of course.

I don't understand what you mean by worrying about noise. What noise is this?


Top
 Profile  
 
 Post subject: Re: screen scraping
PostPosted: Wed Aug 08, 2018 11:59 am 
Offline
Senior Member
User avatar

Joined: Sun Mar 10, 2013 10:31 am
Posts: 139
Jannus wrote:
I don't understand what you mean by worrying about noise. What noise is this?


You have only several screenshots. In your binarization algoritm one "7" different from another "7".
What is this difference? It is some "new" point that appear near "7". You cannot predict this. And if one point appears right side of "7" and another left side of "6" - there will no be empty line betwen them, and you cannot separate it.
So that why I said that your algoritm works on an "edge of working".
But if it works already on 100 stacks for example in 100% so let it work further :)


Top
 Profile  
 
 Post subject: Re: screen scraping
PostPosted: Wed Aug 08, 2018 12:36 pm 
Offline
Junior Member

Joined: Thu Apr 05, 2018 2:57 pm
Posts: 23
got it. Thanks! Yeah got now 30 different stacks classified with 0 errors. Got no samples anymore :lol: So first i am going to play some manual poker and gather some more screenshots :mrgreen:


Top
 Profile  
 
 Post subject: Re: screen scraping
PostPosted: Wed Aug 08, 2018 7:11 pm 
Offline
Junior Member

Joined: Thu Apr 05, 2018 2:57 pm
Posts: 23
Image

Still got a 100% recognition 8-) . I only had to add a single mask for digit '1' (there was also a '1' with a width of 4 instead of 5). Tomorow i will test a lot more stacks, cause today most of my work has been with this live result scraper viewer (see Screenshot). This way i can easily check if all stacks are recognized correctly during live play. Instead of gathering screenshot after screenshot and check afterwards.

Done for today. Peace.


Top
 Profile  
 
 Post subject: Re: screen scraping
PostPosted: Sat Aug 11, 2018 10:18 am 
Offline
Junior Member

Joined: Thu Apr 05, 2018 2:57 pm
Posts: 23
im writing a function that checks if an image is located in a bigger image (with a search region passed as a parameter)

is this ok? (it works)

Code:
// this function returns x,y coordinates of the location of target inside a region of src
   public Point findImageRegion(BufferedImage src,BufferedImage target,int startx,int starty, int endx,int endy) {
      Point p = new Point(0,0);
      
      for(int x=startx;x<endx;x++) {
         
         // target cant be found because there is not enough room in the region anymore
         if ( (x +target.getWidth()) > endx  ) {
         
            break;
         }
      
         for(int y=starty;y<endy;y++) {
            
            // target cant be found anymore, because there is not enough room in the region
            if ( ( y + target.getHeight()) > endy) {

               break;
            }
            
            if ( this.isImageHere(src, target, x, y)) {
               return new Point(x,y);
            }
            
         }
         
      }   
      return p;
   }



Code:
   // returns true if target is in src starting from topleftx,y
   private boolean isImageHere(BufferedImage src,BufferedImage target,int topleftx,int toplefty) {
      
      int errors = 0;
      for(int x=0;x<target.getWidth();x++) {
         
         
         for(int y=0;y<target.getHeight();y++) {
            
            if ( src.getRGB(topleftx+x,toplefty+y) != target.getRGB(x, y)) {
               errors++;
            }
            
            if ( errors > PIXEL_ERROR_THRESHOLD) {
               return false;
            }
            
         }
      }
      

      return true;
   }


So this does work. Im just curious about speed. I have little experience in that department....


Top
 Profile  
 
 Post subject: Re: screen scraping
PostPosted: Sun Aug 12, 2018 8:47 am 
Offline
Site Admin
User avatar

Joined: Sun Feb 24, 2013 9:39 pm
Posts: 642
Quote:
Im just curious about speed.


- What's stopping you timing it and comparing the time to your budget? If you need it to be faster:
- - take a look at https://en.wikipedia.org/wiki/String-se ... _algorithm and its references. I know of someone who used KMP for your problem
- - Seems to me you could speed things up a lot by initially comparing a subset of the "needle" that is known to be statistically improbable in the "haystack".
- - Image recognition is a well studied problem. DNNs are overkill for your problem but I reckon there must be older techniques suitable for you out there. https://www.quora.com/How-do-image-reco ... ithms-work


Top
 Profile  
 
 Post subject: Re: screen scraping
PostPosted: Mon Aug 13, 2018 8:32 am 
Offline
Senior Member
User avatar

Joined: Sun Mar 10, 2013 10:31 am
Posts: 139
Your code is very ... slow. Also it is wery... strange.
To chek width of image... Why?
there how your code can be done in 1 function
Code:
//will search src1 in src2
for (int start_row = 0; start_row < src2.rows - src1.rows; start_row ++){
   for (int start_col = 0; start_col < src2.cols - src1.cols; start_col ++){
      int treshold = 0;
      for (int row = 0; row < src1.rows; row++){
         for (int col = 0; col < src1.cols; col++){
            if (scr1.at(row, col) != scr2.at(start_row + row, start_col + col)) treshold++;
            if (treshold >= 20) break;
         }
         if (treshold >= 20) break;
      }
      if (treshold < 20) YouFoundItCongards(start_row, start_col);
   }   
}

But as you see it have 4!!!! recrutive cycles.
Treshold is good idea, but imagine that your searc image have few black lines at the start. So it will try to search every blak point of an image.
The main idea how I solve this is TwoPhaseSearch();
In 1st phase we search few KEY points. In simplest way they can be definded manualy. Or some grid of src1 image.
And when we find them - then we search whole image there.
As you see you can also write 3PhaseSearh() or even more )))) But 2Phase was enouth yet.


Top
 Profile  
 
 Post subject: Re: screen scraping
PostPosted: Tue Aug 14, 2018 8:57 pm 
Offline
Senior Member

Joined: Fri Nov 25, 2016 10:42 pm
Posts: 122
Jannus wrote:
thanks for your post. incredibly helpful :)

But what i was wondering..is it possible to completely remove the OCR part. For example the stack size scraping. Just take screenshots of all the
10 digits. Store these as single image files (one for each digit). Then look for those images in the right places of the screenshot. Then reconstruct the stacksize based on the digits that are there. If this works (and is fast enough) you will have 100% accuracy and no need for OCR what so ever. I am going to try this approach first..if it doesnt work I will try Tesseract or OpenCV.

Will let you know how it went


On some sites you will see only 10 digits, but on most of sites today you have many variants of how one digit looks like depending on the position. It is likely the effect of font smoothing / scaling. Openholdem is very good at it, by applying different filters, and big advantage is that you can find already done TM files from other guys who already done it.


Top
 Profile  
 
 Post subject: Re: screen scraping
PostPosted: Tue Aug 14, 2018 9:59 pm 
Offline
Veteran Member

Joined: Wed Mar 20, 2013 1:43 am
Posts: 267
mlatinjo wrote:
Jannus wrote:
thanks for your post. incredibly helpful :)

But what i was wondering..is it possible to completely remove the OCR part. For example the stack size scraping. Just take screenshots of all the
10 digits. Store these as single image files (one for each digit). Then look for those images in the right places of the screenshot. Then reconstruct the stacksize based on the digits that are there. If this works (and is fast enough) you will have 100% accuracy and no need for OCR what so ever. I am going to try this approach first..if it doesnt work I will try Tesseract or OpenCV.

Will let you know how it went


On some sites you will see only 10 digits, but on most of sites today you have many variants of how one digit looks like depending on the position. It is likely the effect of font smoothing / scaling. Openholdem is very good at it, by applying different filters, and big advantage is that you can find already done TM files from other guys who already done it.


I completely agree. You can sometimes make your life easier though, like you can change the table background, or sometime smoothing will be disabled when you make the table very small. There are things like that, but I also assume it makes you easier to detect.


Top
 Profile  
 
 Post subject: Re: screen scraping
PostPosted: Wed Aug 15, 2018 8:31 pm 
Offline
Junior Member

Joined: Thu Apr 05, 2018 2:57 pm
Posts: 23
nefton wrote:
Your code is very ... slow. Also it is wery... strange.
To chek width of image... Why?

I don't understand the question completely. But are you referring to these lines?

Code:
if ( (x +target.getWidth()) > endx  ) {
         
            break;
 }


These are because the search image cant be found anymore, because we already looked through the source image up to a point that the search image does not fit anymore in the remaing part of the source image. But im guessing this is not what you meant.


Last edited by Jannus on Wed Aug 15, 2018 8:42 pm, edited 2 times in total.

Top
 Profile  
 
 Post subject: Re: screen scraping
PostPosted: Wed Aug 15, 2018 8:41 pm 
Offline
Junior Member

Joined: Thu Apr 05, 2018 2:57 pm
Posts: 23
nefton wrote:
In 1st phase we search few KEY points.


so just store some (say 4) RGB values for 4 x,y coordinates point. Then look for those 4 points, When you found it, compare the other x,y of the image to see if you get more matches then then the 4 you already had, and above threshold -> disco?

What is a good threshold? I now use 15% error of the search image. (so 0.15 * src1-width * src1-height)


Top
 Profile  
 
Display posts from previous:  Sort by  
Post new topic Reply to topic  [ 48 posts ]  Go to page Previous  1, 2, 3  Next

All times are UTC


Who is online

Users browsing this forum: No registered users and 1 guest


You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot post attachments in this forum

Search for:
Powered by phpBB® Forum Software © phpBB Group