Hearthstone Arena Helper

June 4, 2015 in Projects »C#Plugin

Arena Helper is a plugin for Hearthstone Deck Tracker that helps drafting Hearthstone arena decks. Hearthstone is a popular free-to-play card game by Blizzard Entertainment. The game has a game mode called Arena. In the Arena mode, the player has to create a deck consisting of 30 cards. The cards are picked one at a time, so there are 30 choices the player has to make. Every choice consists of selecting one card out of three randomly selected cards. Not every card that is presented, is equally good as the others. New players, but even experienced players, don’t always know which card has the most value. This plugin helps the player to make a choice, by showing what other expert players think of the value of the presented cards. The plugin tries to visually detect the arena heroes and card choices. Detected cards are displayed alongside the value of the card, that is specified in ADWCTA’s Arena Tier List. The created deck can be saved to Hearthstone Deck Tracker.

The plugin uses perceptual hashing to detect the Hearthstone arena heroes and cards. The plugin uses OpenCV, or rather Emgu CV, to calculate the perceptual hash. It was written using C#, because Hearthstone Deck Tracker uses it. This article explains how the plugin works and provides code examples to implement your own image recognition algorithms based on perceptual hashing.

Arena Helper For Hearthstone

Arena Helper For Hearthstone

For the latest news, releases and downloads, visit the Arena Helper project page on GitHub. The rest of this article explains the technical details of the plugin.

How It Works

The plugin takes screenshots of the Hearthstone window and uses this visual information to detect game states, cards and heroes. The plugin doesn’t use memory reading functions and it doesn’t inject data or code into the Hearthstone process.

Lets say we have a hashing function that takes an image and returns a 64-bit number that represents the image. Two different images will result in two different hash numbers. If we have two images that are almost the same, the two hashes will also be almost the same. So we can use the hashing function to compare two images and find out how similar they are. If we know the exact position of the visual elements we want to detect, and we know what the visual hashes of these elements should be, we can try to detect them. This requires us to precompute all of the perceptual hashes that we want to detect.

The image below shows how Arena Helper tries to detect the Hearthstone game state and the heroes. The red rectangles indicate the location of the visual elements we want to detect. These locations are defined in the code of the plugin and the visual hashes of the elements are precomputed and stored in a database. The image in the top rectangle shows us if we are in the arena state. The bottom rectangles represent the position and size of the heroes.

arena-helper-perceptual-hash

The visual hashes of the three heroes are calculated and the hashes are matched against the hashes stored in the database. If the hashes are close enough, we have detected the heroes. Detecting cards is a similar process.

Resolution Independent

We define the exact location and size of the visual elements in the code of Arena Helper. This information is only correct for a specific resolution. If the resolution is changed, we need to convert our stored locations to be compatible with the new resolution.

For example, we can define a rectangle that indicates the location and size of the first hero portrait in the image above. The code examples below will be using C#.

1

Rectangle portraitcroprect = new Rectangle(143, 321, 173, 107);

This rectangle is correct for the reference resolution of 1280 x 960. If the game runs in a more standard resolution like 1920 x 1080, we need to convert the location and size of the rectangle. It depends on the game how the visual elements change when the resolution changes, but for Hearthstone, the following conversion functions work most of the time. If the resolution is extremely wide, the conversion function will not work.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31

// Reference resolution: 1280 x 960 (4:3)
 
// hsrect: The current location and size of the game window
// x: X-position to convert
// y: Y-position to convert
// width: The width of the reference resolution
// height: The height of the reference resolution
private Point GetHSPos(Rectangle hsrect, int x, int y, int width, int height)
{
    // Get normalized position
    double nx = x / (double)width;
    double ny = y / (double)height;
 
    // Convert to actual position
    double ratio = ((double)width / (double)height) / ((double)hsrect.Width / hsrect.Height);
    int px = (int)((hsrect.Width * ratio * nx) + (hsrect.Width * (1 - ratio) / 2));
    int py = (int)(ny * hsrect.Height);
    return new Point(px, py);
}
 
// hsrect: The current location and size of the game window
// x: Width to convert
// y: Height to convert
// width: The width of the reference resolution
// height: The height of the reference resolution
private Point GetHSSize(Rectangle hsrect, int x, int y, int width, int height)
{
    double scalefactor = (double)hsrect.Height / height;
 
    return new Point((int)(scalefactor * x), (int)(scalefactor * y));
}

To convert the portrait rectangle, we can use the functions as follows.

1
2
3
4
5
6

// Get the Hearthstone window rectangle
var hsrect = Helper.GetHearthstoneRect(false);
 
// Get the position and size
Point pos = GetHSPos(hsrect, portraitcroprect.X, portraitcroprect.Y, 1280, 960);
Point size = GetHSSize(hsrect, portraitcroprect.Width, portraitcroprect.Height, 1280, 960);

Now that we have the converted rectangle in the form of pos and size, we can copy the rectangle from a screenshot of the Hearthstone window and calculate the perceptual hash.

Image Recognition With Perceptual Hashing

The perceptual hashing function uses the techniques from Looks Like It. The implementation was done in C#, using the Emgu CV library that is a wrapper for the OpenCV library. The code below is compatible with Emgu version: Emgu.CV-3.0.0-rc1.

The GetImageHash function takes a bitmap and returns a 64 bit perceptual hash.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50

private ulong GetImageHash(Bitmap bitmap)
{
    // Copy the image and convert to grayscale
    Bitmap sourcebm = new Bitmap(bitmap);
    Image<Gray, float> sourceimage = new Image<Gray, float>(sourcebm);
 
    // Apply a convolution filter
    CvInvoke.Blur(sourceimage, sourceimage, new Size(4, 4), new Point(-1, -1));
 
    // Resize to 64x64 pixels
    Image<Gray, float> resimage = new Image<Gray, float>(new Size(64, 64));
    CvInvoke.Resize(sourceimage, resimage, new Size(64, 64));
 
    // DCT
    IntPtr compleximage = CvInvoke.cvCreateImage(resimage.Size, Emgu.CV.CvEnum.IplDepth.IplDepth32F, 1);
    CvInvoke.Dct(resimage, resimage, Emgu.CV.CvEnum.DctType.Forward);
 
    Image<Gray, float> dctimage = Image<Gray, float>.FromIplImagePtr(resimage);
 
    // Calculate the mean
    double mean = 0;
    for (int x = 0; x < 8; x++)
    {
        for (int y = 0; y < 8; y++)
        {
            mean += dctimage[y, x].Intensity;
        }
    }
    mean -= dctimage[0, 0].Intensity;
    mean /= 63;
 
    // Calculate the hash
    ulong hash = 0;
    ulong index = 1;
    for (int x = 0; x < 8; x++)
    {
        for (int y = 0; y < 8; y++)
        {
            Gray color = dctimage[y, x];
            if (color.Intensity > mean)
            {
                hash |= index;
            }
 
            index <<= 1;
        }
    }
 
    return hash;
}

To calculate the distance between two hashes we create the GetHashDistance function. This function calculates how many bits are different when looking at two hashes.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16

private int GetHashDistance(ulong hash1, ulong hash2)
{
    ulong index = 1;
    int distance = 0;
    for (int i = 0; i < 64; i++)
    {
        if ((hash1 & index) != (hash2 & index))
        {
            distance++;
        }
 
        index <<= 1;
    }
 
    return distance;
}

Now that we have the GetImageHash and GetHashDistance functions, we have all the tools we need to recognize images, visual elements, that are embedded in a screenshot.

Hearthstone Deck Tracker Overlay

Arena Helper integrates with the overlay system of Hearthstone Deck Tracker. It displays the values of the cards under the actual cards in the Hearthstone window, just like how Hearthstone Deck Tracker does it. The image below shows the Arena Helper overlay.

Arena Helper Overlay

Plugins For Arena Helper

Arena Helper includes a plugin system. The system allows site owners to integrate with Arena Helper and enables retrieving custom values for detected cards. It can also be used to record the arena drafting process. More information about the plugin system can be found at: How To Write Plugins For Arena Helper

Download Arena Helper For Hearthstone

For the latest news and releases, visit the Arena Helper on GitHub project page. The latest release of the plugin is always available for download at the Arena Helper releases page.

Related Articles