units less then a pixel , Screen cupture searching partial image of screen within larger portion of screen

Question

the whole thing is , that with CopyFromScreen() Method i was able to compare an image (previously captured) to an image that is the same as the captured one cause it actually captured from that particular screen .

or more simply ,for example, an icon , cropped from desktop and saved as an image along with a textfile referencing its location(point) and its size (size(h,w))

so when we have captured.bar via File.WriteAllBytes(byte[]made it .bar) to a file + cuptured.txt with reference to the location and size we have a copy saved to hdd and its location and size in txt. (i didn't have to post all of the code but it works fine so if someone needs it.. i share feel free)

the way i achieve this is using a method i made DoCopyFScr() - to cupture portion of screen which is using the Bitmap2ByteArr() - saving the bitmap as ByteArr then later when needed using byte[]- saved, to compare against its captured location on screen.

    public void DoCopyFScr(Point SourceP, Size Mysz, string initDir, string Fname, bool SaveIt, bool DoCompare)
    {

        SetForegroundWindow(A window handle suplied here);
        pause(200);
        Point pZero = new Point(0, 0);

        using (Bitmap bitmap = new Bitmap(Mysz.Width, Mysz.Height))
        {
            using (Graphics g = Graphics.FromImage(bitmap))
            {
                g.CopyFromScreen(SourceP, pZero, Msyz);
            }
            Image img = (Image)bitmap;
            Clipboard.SetImage(img);
            ItsCopiedFromScreen = true;
            MyBtmpToByteArr b2arr = new MyBtmpToByteArr();// inststance of My bitmap to byte[]

            AAdToAppLog("start Converting Bitmap To ByteArr");// logging programs activity in a listview via reflection
            MyCuptredBtmpToolBox.CuptByteArr = b2arr.Bitmap2ByteArr(bitmap);
            AAdToAppLog("Done Convertion ArrSize " + b2arr.Bitmap2ByteArr(bitmap).Length + ", Resting CopyDestionation SelctdIndx");
                                Combo_CopyFromScrn.SelectedIndex = 0;
            if (SaveIt)
            {
                string btmpsRefrenceSave = MyCuptredBtmpToolBox.SourceX_Cuptured.ToString() + "," + MyCuptredBtmpToolBox.SourceY_Cuptured.ToString() + "," + MyCuptredBtmpToolBox.RectWidth_Cuptured.ToString() + "," + MyCuptredBtmpToolBox.RectHeight_Cuptured.ToString() + "," + TBX_FileSaveName.Text;
                if (ItsLoadedFromFile)
                {
                    File.WriteAllBytes(initDir + TBX_FileSaveName.Text, b2arr.Bitmap2ByteArr(bitmap));// See next block Method Bitmap2ByteArr method
                    File.WriteAllText(initDir + TBX_FileSaveName.Text.Replace("bar", "txt"), btmpsRefrenceSave);
                }
                else
                {
                    File.WriteAllBytes(initDir + TBX_FileSaveName.Text + ExtBar, b2arr.Bitmap2ByteArr(bitmap));
                    File.WriteAllText(initDir + TBX_FileSaveName.Text.Replace("bar", "txt") + ExtTXT, btmpsRefrenceSave);
                }
                //MyCuptredBtmpToolBox.CupturedStaticBmp.Save(MyCuptredBtmpToolBox.FnameToSave.Replace(".bar", ".") + ExtJpg, ImageFormat.Jpeg);
                bitmap.Save(initDir + TBX_FileSaveName.Text + ExtJpg, ImageFormat.Jpeg);
            }
            if (DoCompare)
            {
                AAdToAppLog("starting Compare " + Fname +" With Btmp2Arr From Screen");
                R_ComparwByteArrClass tstCmp = new R_ComparwByteArrClass();
                if (itsAutomated)
                {
                    CompareByt2Btmp = R_ComparwByteArrClass.ByteArrCompare(DoAutoLoadCuptByteArr(initDir, Fname + ExtBar), b2arr.Bitmap2ByteArr(bitmap));
                }
                else
                {
                    CompareByt2Btmp = R_ComparwByteArrClass.ByteArrCompare(MyLoadedBtmpToolBox.LoadedByteArr, b2arr.Bitmap2ByteArr(bitmap));
                }
                bring(MyApp);
                AAdToAppLog(CompareByt2Btmp.ToString());

            }
        }

    }

the bitmap to ByteArr class with method to convert the cuptured btmp to byte[]

  public class MyBtmpToByteArr
    {


        public byte[] Bitmap2ByteArr(Bitmap btmpToConvertintoByteArr)
        {


            //Convert image to a byte array
            System.Drawing.ImageConverter MyImgCvrtr = new System.Drawing.ImageConverter();
            byte[] btImage = new byte[1];
            btImage = (byte[])MyImgCvrtr.ConvertTo(btmpToConvertintoByteArr, btImage.GetType());


            return btImage;
        }
    }

but if the location is changed i can't make it identify even though i gave the new location of the image on screen , when i did test though , using paint and with turning grid on i positioned two copies of the same cropped image and i did success but on a real scenario with a game screen where there is a shared background for few icons , i try to compare them to each other and do not succeed even though i gave the location , is there a unit that is smaller than a pixel maybe the graphics of the game uses a kind of unit that's a fraction of pixel ?? or maybe other unit so mybe the given location Point1 is X100 Y100 for first icon , and the other instead of being in 200,200 is actually at 200.33, 200.55 or so ... and that is why i dont get a dead lock position/location ? p.s i think it's using a diretx if not it's on openGL allmost sure on directx if it matters for what units is used to locate the images does a unit smaller than pixels exists at all... and commonly used ? if not how would you implement a founction that will loop on from say x100 to x150 and y100 to y150 so i could search for the image

Answer 1

made The second part of my Question MySelf

StpX ,StpY limits the loop max iterations

the original Point which is the location of the capture is initPoint CountDown is doing the math how many rounds left till end, and in the inner loop theres a "hand break" that triggers when search is successfull

        Point SearchFromPoint = Point.Empty;
        Point initPoint = new Point(150, 100);
        Size imgSz = new Size(320, 240);
        int StpX = initPoint.X + 1, StpY = initPoint.Y + 1;

        int CountDownX, CountDownY;
        for (int StrtX = initPoint.X - extraX; StrtX < StpX; StrtX++)
        {
            SearchFromPoint.X = StrtX;
            for (int StrtY = initPoint.Y - extraY; StrtY < StpY; StrtY++)
            {
                CountDownX = (initPoint.X - StrtX); CountDownY=(initPoint.Y - StrtY);
                 SearchFromPoint.Y = StrtY;
                AAdToAppLog("Search Report For: " + imgName + " Still trying Search On " + CountDownX + ":" + CountDownY +" Point(" + SearchFromPoint.X + ", " + SearchFromPoint.Y + ")");
                DoCopyFScr(SearchFromPoint, tmpSz, initDir, Fname, false, true);
                    if (CompareByt2Btmp.ToString() == R_ComparwByteArrClass.CompHashResult.HashCompare_Ok.ToString())
                    {
                        same = true;
                        AAdToAppLog("Search Report For: " + imgName + "Search Completed Successfully On Try " + CountDownX + ":" + CountDownY);

                        break;
                    }
            }

        }

and this is what i think... concludes the search method as opposed to the fixed known location scenario.

if there's any thing ou want to comment on this feal free to post .

Answer 2

What you're likely seeing is differing rendering artifacts from either scaling, aliasing, or subpixel hinting.

WPF based applications for example are DPI independent, so they use 'display points' instead of pixels, and you can have a fraction of a display point which can account for the kind of behavior you're seeing.

A 'pixel' is a very physical thing (the point on your monitor) but not everything strictly has a 1:1 mapping to real world pixels.

So instead you're going to need a more clever mechanism for comparing similar images - this related stackoverflow post may be a good place to start.