Poker-AI.org
http://poker-ai.org/phpbb/

screen scraping
http://poker-ai.org/phpbb/viewtopic.php?f=26&t=3133
Page 3 of 3

Author:  nefton [ Wed Aug 15, 2018 10:13 pm ]
Post subject:  Re: screen scraping

Jannus wrote:
(so 0.15 * src1-width * src1-height)

No! You measure differens binary yet. Yes or No. It is wrong way. If your image greyscale fore example, each point 0-255 value;
So average differense is Summ(ABS(Point(image) - Point(template))) / template.rows / template.cols;
For RGB image Differense is average for 3 chanels (but you have not use it, I use it once and want rewrite this stupid code )
And then you can compare your average difference with any number. 0.15 or somthing else.
0.15 - is wery BAD! If your image compare with template average difference = 0.15 - you use wery bad filters! 0.05 - is good value.
For example you have "6" image. With your not good filters it compare with "6" = 0.15; with "0" = 0.18; with "8" = 0.18;
It situation = age of working; Shoud be with GOOD filters : with "6" = 0.05; with "0" = 0.30; with "8" = 0.30.
It is GOOD recognition;

Author:  Jannus [ Tue Aug 28, 2018 9:23 am ]
Post subject:  Re: screen scraping

Hi!

Sorry i coulnt post sooner, but I had to help a friend with a construction job. Today is the first time i am working on the bot again.

I understand that greyscale is better. But i already have put a lot of work using binary. And i still have 100% accuracy. So i am just going on
using binary. If i run into problems, i will switch to grayscale.

The 15% (which was a random number) i dont use when recognizing numbers. I still use the best matching mask for that. But when i am looking for..lets say..a fold button...i look for a pattern and accept a certain error rate. As of now, i have again 100% accuracy recognizing buttons, etc. Speed (as you pointed out) is not the best. lol. But i am trying to get this scraping part done as soon as possible so I can test my AI bit..because if AI fails..everything fails.

Cheers!

Author:  sadnok [ Tue Jul 16, 2019 8:05 pm ]
Post subject:  Re: screen scraping

nefton wrote:
cant add more than 3 attachment so filter 3 and the code here
Code:
#include <iostream>
#include "opencv2/opencv.hpp"
#include "common_ocr_functions.h"

int main() {

   cv::Mat source = cv::imread("D:/PROJECTS/opencv_sample/example.png");

   //cv::imshow("source", source);
   //cv::waitKey();

   //ShowLarge10x("source", source, true);

   cv::Mat filter1(source.rows * 2, source.cols * 2, CV_8UC1);

   for (int row = 0; row < source.rows; row++) {
      for (int col = 0; col < source.cols; col++) {
         cv::Vec3b point = source.at<cv::Vec3b>(row, col);
         uint8_t b = point[0];
         uint8_t g = point[1];
         uint8_t r = point[2];
         uint8_t h, s, v;
         RGB2HSV(r, g, b, &h, &s, &v);
         filter1.at<uint8_t>(row * 2 + 0, col * 2 + 0) = v;
         filter1.at<uint8_t>(row * 2 + 1, col * 2 + 0) = v;
         filter1.at<uint8_t>(row * 2 + 0, col * 2 + 1) = v;
         filter1.at<uint8_t>(row * 2 + 1, col * 2 + 1) = v;
      }
   }

   //ShowLarge10x("filter1", filter1, true);

   cv::Mat filter2(source.rows * 2, source.cols * 2, CV_8UC1);

   cv::GaussianBlur(filter1, filter2, cv::Size(4, 4), 0.0);

   //ShowLarge10x("filter2", filter2, true);

   cv::Mat filter3(source.rows * 2, source.cols * 2, CV_8UC1);

   for (int row = 0; row < filter2.rows; row++) {
      for (int col = 0; col < filter2.cols; col++) {
         uint8_t v = filter2.at<uint8_t>(row, col);
         filter3.at<uint8_t>(row, col) = 0;
         if (v > 100) filter3.at<uint8_t>(row, col) = v;
      }
   }

   ShowLarge10x("filter3", filter3, true);

   std::cout << "\n\n";
   std::system("pause");
   return 0;
}



Can someone help me with this code?

pretty new into all of this and already been trying for a few hours to get this code working as the function rgb2hsv is missing here, i have found some standard rgb2hsv codes already out there but they are not using uint8 but unsigned char instead.

Not exactly sure why seems like we prefer uint8 over unsigned char if we are planning to save small numbers rather than characters

Author:  HontoNiBaka [ Wed Jul 17, 2019 2:14 am ]
Post subject:  Re: screen scraping

Afaik uint8_t is just a typedef for unsigned char, so the types are exactly the same.

Author:  nefton [ Wed Jul 17, 2019 5:31 pm ]
Post subject:  Re: screen scraping

Hmm, seems there is some problems with my code.
There are some more code :)
Code:
void RGB2HSV(uint8_t r, uint8_t g, uint8_t b, uint8_t* h, uint8_t* s, uint8_t* v){
   unsigned __int8 rgbMin, rgbMax;
   rgbMin = r < g ? (r < b ? r : b) : (g < b ? g : b);
   rgbMax = r > g ? (r > b ? r : b) : (g > b ? g : b);
   *v = rgbMax;
   if (*v == 0) { *h = 0; *s = 0; return; }
   *s = 255 * long(rgbMax - rgbMin) / (*v);
   if (*s == 0) { *h = 0; return; }
   if (rgbMax == r) *h = 0 + 43 * (g - b) / (rgbMax - rgbMin);
   else if (rgbMax == g) *h = 85 + 43 * (b - r) / (rgbMax - rgbMin);
   else *h = 171 + 43 * (r - g) / (rgbMax - rgbMin);
   return;
}

void ShowLargeCallBackFunc(int event, int x, int y, int flags, void* userdata)
{

   if (flags == cv::EVENT_FLAG_RBUTTON){
      cv::Mat* scr = (cv::Mat*)userdata;
      int type = scr->type();
      int channels = scr->channels();
      if (channels == 1){
         std::cout << "Pos(" << x / 10 << ", " << y / 10 << ")  Val( " << (int)scr->at<unsigned char>(y, x) << " )" << std::endl;
      }
      if (channels == 4){
         int r = scr->at<cv::Vec4b>(y, x)[2];
         int g = scr->at<cv::Vec4b>(y, x)[1];
         int b = scr->at<cv::Vec4b>(y, x)[0];
         unsigned __int8 h, s, v;
         RGB2HSV(r, g, b, &h, &s, &v);
         std::cout << "Pos( " << x / 10 << ", " << y / 10 << " )  RGB( " << r << ", " << g << ", " << b << " )";
         std::cout << " HSV( " << (int)h << ", " << (int)s << ", " << (int)v << " )" << std::endl;
      }
      if (channels == 3){
         int r = scr->at<cv::Vec3b>(y, x)[2];
         int g = scr->at<cv::Vec3b>(y, x)[1];
         int b = scr->at<cv::Vec3b>(y, x)[0];
         unsigned __int8 h, s, v;
         RGB2HSV(r, g, b, &h, &s, &v);
         std::cout << "Pos( " << x / 10 << ", " << y / 10 << " )  RGB( " << r << ", " << g << ", " << b << " )";
         std::cout << " HSV( " << (int)h << ", " << (int)s << ", " << (int)v << " )" << std::endl;
      }
   }
   return;
}



void ShowLarge10x(cv::String winname, cv::Mat scr, bool wait_to_cklick){
   int type = scr.type();
   int channels = scr.channels();
   cv::Mat Img(scr.rows * 10, scr.cols * 10, type);
   if (channels == 1){
      for (int row = 0; row < Img.rows; row++){
         for (int col = 0; col < Img.cols; col++){
            Img.at<unsigned char>(row, col) = scr.at<unsigned char>(row / 10, col / 10);
         }
      }
   }
   if (channels == 3){
      for (int row = 0; row < Img.rows; row++){
         for (int col = 0; col < Img.cols; col++){
            Img.at<cv::Vec3b>(row, col) = scr.at<cv::Vec3b>(row / 10, col / 10);
         }
      }
   }
   if (channels == 4){
      for (int row = 0; row < Img.rows; row++){
         for (int col = 0; col < Img.cols; col++){
            Img.at<cv::Vec4b>(row, col) = scr.at<cv::Vec4b>(row / 10, col / 10);
         }
      }
   }
   cv::imshow(winname, Img);
   cv::setMouseCallback(winname, ShowLargeCallBackFunc, &Img);
   if (wait_to_cklick) {
      cv::waitKey(0);
      cv::destroyWindow(winname);
   }
   return;
}

Show large shows image (1b, 3b, or 4b per pixel).
Also if you cklick right mouse button on any pixel it shows x,y RGB and HSV of that pixel.
Wery helpful, everybody must have this!

Author:  nefton [ Wed Jul 17, 2019 5:36 pm ]
Post subject:  Re: screen scraping

Some more code
Wery fast Blur code:
Code:
void Blur3px1b(cv::Mat* img_1b) {
   for (int row = 1; row < img_1b->rows - 1; row++) {
      for (int col = 1; col < img_1b->cols - 1; col++) {
         uint8_t b0 = img_1b->at<uint8_t>(row + 0, col + 0);
         uint8_t b1 = img_1b->at<uint8_t>(row - 1, col - 1);
         uint8_t b2 = img_1b->at<uint8_t>(row - 1, col + 0);
         uint8_t b3 = img_1b->at<uint8_t>(row - 1, col + 1);
         uint8_t b4 = img_1b->at<uint8_t>(row + 0, col - 1);
         uint8_t b5 = img_1b->at<uint8_t>(row + 0, col + 1);
         uint8_t b6 = img_1b->at<uint8_t>(row + 1, col - 1);
         uint8_t b7 = img_1b->at<uint8_t>(row + 1, col + 0);
         uint8_t b8 = img_1b->at<uint8_t>(row + 1, col + 1);
         int result = b0 / 2 + b1 / 16 + b2 / 16 + b3 / 16 + b4 / 16 + b5 / 16 + b6 / 16 + b7 / 16 + b8 / 16;
         if (result > 255) result = 255;
         img_1b->at<uint8_t>(row, col) = result;
      }
   }
   return;
}


Best resize function ever:
Code:
void ResizeSuper(cv::Mat* source_1b, cv::Mat* dest_1b) {

   dest_1b->setTo(0);

   //define size of points
   double source_x_point_size = 1.0 / source_1b->cols;
   double source_y_point_size = 1.0 / source_1b->rows;
   double dest_x_point_size = 1.0 / dest_1b->cols;
   double dest_y_point_size = 1.0 / dest_1b->rows;

   //foreach source row
   for (int source_row = 0; source_row < source_1b->rows; source_row++) {
      //Надо определить все ряды наначения (1 или много) куда входит этот ряд хотябы частично
      //Определяем начало и конец этого исходного ряда
      double source_start_y = source_y_point_size * source_row;
      double source_end_y = source_y_point_size * (source_row + 1);
      //Определяем первый ряд назначения который хотя б частично перекрывает наш ряд исходный
      int dest_start_row = source_start_y / dest_y_point_size;
      //Определяем последний ряд назначения который хотя б частично перекрывает наш ряд исходный
      int dest_end_row = source_end_y / dest_y_point_size;
      //std::cout << "Source_row: " << source_row << " \tDest: " << dest_start_row << " - " << dest_end_row << "\n";
      //Для всех рядов назначения определяем степень вхождения в них исходного ряда
      for (int dest_row = dest_start_row; dest_row <= dest_end_row; dest_row++) {
         //Определяем начало и конец ряда назначения
         double dest_start_y = dest_y_point_size * dest_row;
         double dest_end_y = dest_y_point_size * (dest_row + 1);
         //Определяем начало перекрывающегося промежутка (share_space_y_min)
         double share_space_y_min = dest_start_y;
         if (source_start_y > dest_start_y) share_space_y_min = source_start_y;
         double share_space_y_max = dest_end_y;
         if (source_end_y < dest_end_y) share_space_y_max = source_end_y;
         //Определяем коэффициент фхождения в них исходного ряда
         double dest_y_coeff = (share_space_y_max - share_space_y_min) / dest_y_point_size;
         //std::cout << "\tDest row: " << dest_row << " \tKoeff_y: " << dest_y_coeff << "\n";

         //======================================================================================
         //Для каждой исходной колонки
         for (int source_col = 0; source_col < source_1b->cols; source_col++) {
            //Надо определить все колонки наначения (1 или много) куда входит эта колонка хотябы частично
            //Определяем начало и конец этой исходной колонки
            double source_start_x = source_x_point_size * source_col;
            double source_end_x = source_x_point_size * (source_col + 1);
            //Определяем первую колонку назначения которая хотя б частично перекрывает нашу исходную колонку
            int dest_start_col = source_start_x / dest_x_point_size;
            //Определяем последнюю колонку назначения которая хотя б частично перекрывает нашу исходную колонку
            int dest_end_col = source_end_x / dest_x_point_size;
            //std::cout << "Source_col: " << source_col << " \tDest: " << dest_start_col << " - " << dest_end_col << "\n";
            //Для всех колонок назначения определяем степень вхождения в них исходной колонки
            for (int dest_col = dest_start_col; dest_col <= dest_end_col; dest_col++) {
               //Определяем начало и конец колонки назначения
               double dest_start_x = dest_x_point_size * dest_col;
               double dest_end_x = dest_x_point_size * (dest_col + 1);
               //Определяем начало перекрывающегося промежутка (share_space_x_min)
               double share_space_x_min = dest_start_x;
               if (source_start_x > dest_start_x) share_space_x_min = source_start_x;
               double share_space_x_max = dest_end_x;
               if (source_end_x < dest_end_x) share_space_x_max = source_end_x;
               //Определяем коэффициент фхождения в них исходной колонки
               double dest_x_coeff = (share_space_x_max - share_space_x_min) / dest_x_point_size;
               //std::cout << "\tDest col: " << dest_col << " \tKoeff_x: " << dest_x_coeff << "\n";

               //Прибавляем нашу исходную точку к соответствующей точке назначения с коэффициентами площади
               if (dest_y_coeff * dest_x_coeff > 0.001) {
                  dest_1b->at<uint8_t>(dest_row, dest_col) += dest_y_coeff * dest_x_coeff * source_1b->at<uint8_t>(source_row, source_col);
               }
            }
         }
      }
   }

   return;
}

Author:  nefton [ Wed Jul 17, 2019 6:02 pm ]
Post subject:  Re: screen scraping

Some other resize:

Code:
void ResizeFast(cv::Mat* source_1b, cv::Mat* dest_1b) {
   double scale_x = (double)source_1b->cols / dest_1b->cols;
   double scale_y = (double)source_1b->rows / dest_1b->rows;
      for (int row = 0; row < dest_1b->rows; row++) {
      int source_row = lround(scale_y * row);
      for (int col = 0; col < dest_1b->cols; col++) {
         int source_col = lround(scale_x * col);
         dest_1b->at<uint8_t>(row, col) = source_1b->at<uint8_t>(source_row, source_col);
      }
   }
   return;
}


cv::Mat FilterTwiceLinear(cv::Mat* source_1b) {

   cv::Mat out2x_1b(source_1b->rows * 2, source_1b->cols * 2, CV_8UC1);
   out2x_1b.setTo(0);
   
   for (int row = 1; row < source_1b->rows - 1; row++) {
      for (int col = 1; col < source_1b->cols - 1; col++) {
         uint8_t v0 = source_1b->at<uint8_t>(row + 0, col + 0);
         uint8_t v1 = source_1b->at<uint8_t>(row - 1, col - 1);
         uint8_t v2 = source_1b->at<uint8_t>(row - 1, col + 0);
         uint8_t v3 = source_1b->at<uint8_t>(row - 1, col + 1);
         uint8_t v4 = source_1b->at<uint8_t>(row + 0, col + 1);
         uint8_t v5 = source_1b->at<uint8_t>(row + 1, col + 1);
         uint8_t v6 = source_1b->at<uint8_t>(row + 1, col + 0);
         uint8_t v7 = source_1b->at<uint8_t>(row + 1, col - 1);
         uint8_t v8 = source_1b->at<uint8_t>(row + 0, col - 1);

         double v_lu = 0.446*v0 + 0.150*v1 + 0.202*v2 + 0.202*v8;
         double v_ru = 0.446*v0 + 0.150*v3 + 0.202*v2 + 0.202*v4;
         double v_rd = 0.446*v0 + 0.150*v5 + 0.202*v4 + 0.202*v6;
         double v_ld = 0.446*v0 + 0.150*v7 + 0.202*v8 + 0.202*v6;

         out2x_1b.at<uint8_t>(row * 2 + 0, col * 2 + 0) = std::lround(v_lu);
         out2x_1b.at<uint8_t>(row * 2 + 0, col * 2 + 1) = std::lround(v_ru);
         out2x_1b.at<uint8_t>(row * 2 + 1, col * 2 + 0) = std::lround(v_ld);
         out2x_1b.at<uint8_t>(row * 2 + 1, col * 2 + 1) = std::lround(v_rd);
      }
   }

   return out2x_1b;
}

cv::Mat FilterTwiceSquare(cv::Mat* source_1b) {

   cv::Mat out2x_1b(source_1b->rows * 2, source_1b->cols * 2, CV_8UC1);
   out2x_1b.setTo(0);

   for (int row = 1; row < source_1b->rows - 1; row++) {
      for (int col = 1; col < source_1b->cols - 1; col++) {
         uint8_t v0 = source_1b->at<uint8_t>(row + 0, col + 0);
         uint8_t v1 = source_1b->at<uint8_t>(row - 1, col - 1);
         uint8_t v2 = source_1b->at<uint8_t>(row - 1, col + 0);
         uint8_t v3 = source_1b->at<uint8_t>(row - 1, col + 1);
         uint8_t v4 = source_1b->at<uint8_t>(row + 0, col + 1);
         uint8_t v5 = source_1b->at<uint8_t>(row + 1, col + 1);
         uint8_t v6 = source_1b->at<uint8_t>(row + 1, col + 0);
         uint8_t v7 = source_1b->at<uint8_t>(row + 1, col - 1);
         uint8_t v8 = source_1b->at<uint8_t>(row + 0, col - 1);

         double v_lu = 0.655*v0 + 0.075*v1 + 0.135*v2 + 0.135*v8;
         double v_ru = 0.655*v0 + 0.075*v3 + 0.135*v2 + 0.135*v4;
         double v_rd = 0.655*v0 + 0.075*v5 + 0.135*v4 + 0.135*v6;
         double v_ld = 0.655*v0 + 0.075*v7 + 0.135*v8 + 0.135*v6;

         out2x_1b.at<uint8_t>(row * 2 + 0, col * 2 + 0) = std::lround(v_lu);
         out2x_1b.at<uint8_t>(row * 2 + 0, col * 2 + 1) = std::lround(v_ru);
         out2x_1b.at<uint8_t>(row * 2 + 1, col * 2 + 0) = std::lround(v_ld);
         out2x_1b.at<uint8_t>(row * 2 + 1, col * 2 + 1) = std::lround(v_rd);
      }
   }

   return out2x_1b;
}

cv::Mat FilterTrippleSquare(cv::Mat* source_1b) {

   cv::Mat out2x_1b(source_1b->rows * 3, source_1b->cols * 3, CV_8UC1);
   out2x_1b.setTo(0);

   for (int row = 1; row < source_1b->rows - 1; row++) {
      for (int col = 1; col < source_1b->cols - 1; col++) {
         uint8_t s0 = source_1b->at<uint8_t>(row + 0, col + 0);
         uint8_t s1 = source_1b->at<uint8_t>(row - 1, col - 1);
         uint8_t s2 = source_1b->at<uint8_t>(row - 1, col + 0);
         uint8_t s3 = source_1b->at<uint8_t>(row - 1, col + 1);
         uint8_t s4 = source_1b->at<uint8_t>(row + 0, col + 1);
         uint8_t s5 = source_1b->at<uint8_t>(row + 1, col + 1);
         uint8_t s6 = source_1b->at<uint8_t>(row + 1, col + 0);
         uint8_t s7 = source_1b->at<uint8_t>(row + 1, col - 1);
         uint8_t s8 = source_1b->at<uint8_t>(row + 0, col - 1);

         double v1 = 0.495*s0 + 0.124*s1 + 0.190*s2 + 0.190*s8;
         double v3 = 0.495*s0 + 0.124*s3 + 0.190*s2 + 0.190*s4;
         double v5 = 0.495*s0 + 0.124*s5 + 0.190*s4 + 0.190*s6;
         double v7 = 0.495*s0 + 0.124*s7 + 0.190*s6 + 0.190*s8;

         //double v2 = 0.624*s0 + 0.156*s2 + 0.062*s4 + 0.062*s8 + 0.048*s1 + 0.048*s3;
         //double v4 = 0.624*s0 + 0.156*s4 + 0.062*s2 + 0.062*s6 + 0.048*s3 + 0.048*s5;
         //double v6 = 0.624*s0 + 0.156*s6 + 0.062*s4 + 0.062*s8 + 0.048*s5 + 0.048*s7;
         //double v8 = 0.624*s0 + 0.156*s8 + 0.062*s2 + 0.062*s6 + 0.048*s1 + 0.048*s7;
         //double v0 = 0.125*(v1 + v2 + v3 + v4 + v5 + v6 + v7 + v8);
         double v2 = (v1 + v3) / 2;
         double v4 = (v3 + v5) / 2;
         double v6 = (v5 + v7) / 2;
         double v8 = (v7 + v1) / 2;
         double v0 = (v1 + v3 + v5 + v7) / 4;

         out2x_1b.at<uint8_t>(row * 3 + 1, col * 3 + 1) = std::lround(v0);
         out2x_1b.at<uint8_t>(row * 3 + 0, col * 3 + 0) = std::lround(v1);
         out2x_1b.at<uint8_t>(row * 3 + 0, col * 3 + 1) = std::lround(v2);
         out2x_1b.at<uint8_t>(row * 3 + 0, col * 3 + 2) = std::lround(v3);
         out2x_1b.at<uint8_t>(row * 3 + 1, col * 3 + 2) = std::lround(v4);
         out2x_1b.at<uint8_t>(row * 3 + 2, col * 3 + 2) = std::lround(v5);
         out2x_1b.at<uint8_t>(row * 3 + 2, col * 3 + 1) = std::lround(v6);
         out2x_1b.at<uint8_t>(row * 3 + 2, col * 3 + 0) = std::lround(v7);
         out2x_1b.at<uint8_t>(row * 3 + 1, col * 3 + 0) = std::lround(v8);
      }
   }

   return out2x_1b;
}

Author:  kramods [ Thu Jul 25, 2019 1:02 pm ]
Post subject:  Re: screen scraping

nefton wrote:
Some other resize:

}[/code]


Thanks for sharing.

Page 3 of 3 All times are UTC
Powered by phpBB® Forum Software © phpBB Group
http://www.phpbb.com/