How to put the mouse position in a part of an image found in other image?

Question

I'm using the AForge library to find a (part of an) image inside other image using this code example.

(The images below are just for give an example)

I'm using this desktop screenshot in 1920x1080 px.:

And I search for and locate this piece of the image above (55x557 px.):

But I resize both images to it's 25% (to gain comparission speed), so when I compare the images the desktop screenshot is 480x270 px. and the cutted image is 13x14 px.

Using the AForge library it returns me the relative coordinates of the found (cutted) image inside the resized desktop screennshot, the coordinates are x=86, y=200.

Now I need to set the mouse position in my dektop, in the center of the VMWare icon (more exactlly in the center of the found cutted image), and here is where I'm confussed, what is the arithmetic operation to set the mouse position there?

I'll remember:

Resolutions:

My Desktop: 1920x1080

Image 1 : 1920x1080

Image to find in Image1 : 55x57

Resized Image1: 480x270

Resized Image to find in Image1 : 13x14

Relative coordinates of the found resized Image to find in Image1 :

x=86, y=200

Answer 1

When you reduce the image you do this:

intReducePct = 25
ReducedImg1 = ResizeImage(desktopBMP, intReducePct)

' save a restore factor
intFactor = (100 / intReducePct)      ' == 4

' I dont know what the AFOrge search returns, a Point probably
foundPt = Aforge.FindImgInImg(...)

' convert fountPt based on reduction factor (you reduced by 1/4th,
'    so scale up by 4x, basically)
Dim actualPoint As New Point(foundPt.X * intFactor, foundPt.Y * intFactor) 

AForge returned x=86, y=200; and 86*4 = 344; 200*4=800 but that is the Top/Left (?) in the original image and you apparently want the center of the found image, so also adjust for the original bitmap:

' convert fountPt based on reduction factor + bmpFind size:
Dim actualPoint As New Point((foundPt.X * intFactor) + (bmpFind.Width \ 2),
                             (foundPt.Y * intFactor) + (bmpFind.Height \ 2))

bmpFind would be the original image before reduction. Option Strict will insist on some CTypes, but this should be the gist of it.

Answer 2

If you really found the spot and if you really are working with the full desktop reduced to 1/4th and not a window, then you can simply multiply back to the original scale and move the mouse like this:

newPos= new Point(foundX * 4, foundY * 4);
Cursor.Position = newPos;

If your FoundPosition is not the Middle but the TopLeft you would adapt the newPos like this:

newPos= new Point(foundX * 4 + originalWidth / 2, foundY * 4 + originalHeight / 2);

If you are in a window you must also calculate the relative positiom to screen coordinates with the PointToScreen() function before setting the mouse position.

Answer 3

I want to share this generic usage method that I've wrote to simplify things:

''' <summary>
''' Finds a part of an image inside other image and returns the top-left corner coordinates and it's similarity percent.
''' </summary>
''' <param name="BaseImage">
''' Indicates the base image.
''' </param>
''' <param name="ImageToFind">
''' Indicates the image to find in the base image.
''' </param>
''' <param name="Similarity">
''' Indicates the similarity percentage to compare the images.
''' A value of '100' means identical image. 
''' Note: High percentage values with big images could take several minutes to finish.
''' </param>
''' <returns>AForge.Imaging.TemplateMatch().</returns>
Private Function FindImage(ByVal BaseImage As Bitmap,
                           ByVal ImageToFind As Bitmap,
                           ByVal Similarity As Double) As AForge.Imaging.TemplateMatch()

    Dim SingleSimilarity As Single

    ' Translate the readable similarity percent value to Single value.
    Select Case Similarity

        Case Is < 0.1R, Is > 100.0R ' Value is out of range.
            Throw New Exception(String.Format("Similarity value of '{0}' is out of range, range is from '0.1' to '100.0'",
                                              CStr(Similarity)))

        Case Is = 100.0R ' Identical image comparission.
            SingleSimilarity = 1.0F

        Case Else ' Image comparission with specific similarity.
            SingleSimilarity = Convert.ToSingle(Similarity) / 100.0F

    End Select

    ' Set the similarity threshold to find all matching images with specified similarity.
    Dim tm As New AForge.Imaging.ExhaustiveTemplateMatching(SingleSimilarity)

    ' Return all the found matching images, 
    ' it contains the top-left corner coordinates of each one 
    ' and matchings are sortered by it's similarity percent.
    Return tm.ProcessImage(BaseImage, ImageToFind)

End Function

An Usage Example:

Private Sub Test() Handles MyBase.Shown

    ' A Desktop Screenshot, in 1920x1080 px. resolution.
    Dim DesktopScreenshoot As New Bitmap("C:\Desktop.png")

    ' A cutted piece of the screenshot, in 50x50 px. resolution.
    Dim PartOfDesktopToFind As New Bitmap("C:\PartOfDesktop.png")

    ' Find the part of the image in the desktop, with the specified similarity.
    For Each matching As AForge.Imaging.TemplateMatch In
        FindImage(BaseImage:=DesktopScreenshoot, ImageToFind:=PartOfDesktopToFind, Similarity:=80.5R) ' 80,5% Similarity.

        Dim sb As New System.Text.StringBuilder

        sb.AppendFormat("Top-Left Corner Coordinates: {0}", matching.Rectangle.Location.ToString())
        sb.AppendLine()
        sb.AppendFormat("Similarity Image Percentage: {0}%", (matching.Similarity * 100.0F).ToString("00.00"))

        MessageBox.Show(sb.ToString)

    Next matching

End Sub