C# Image Analysis (“auto-gaming”) – with source

By | 2010.08.16

For the past couple of weeks I have been playing Age of Conan with Sara.

I am Weathor, Aquilonian Priest of Mitra. MWAHAHA. :)

As always I’m looking for something to tinker with. Just playing the game is not that fun. I’ve been searching for addons for AoC, but I can’t find anything useful. Basically they don’t support any mentionable scripting on the client or server. So I thought it would be fun to do an app that integrates this. Totally useless, but fun. (Yeah, I wanted to play with image analysis…)

I figured the app needed to do three main things:

  • Make screenshot of a specific application that is running, for example Age of Conan.
    It is important that it takes the app and not the whole screen as we need to know where things are on the screen.
  • Analysis of the screenshots being taken.
    I’ve added a module interface so its easy to make new modules. You get a lot of functionality for free.
  • Send keystrokes/commands to the game.
    I’m using PostMessage Win32 API, but it seems to require the application to be active (front), maybe a protection built into the game?

After a few hours of hacking I have a working application. I’m using the new .Net 4.0 Parallel.For/ForEach feature to split the processing into mutliple threads. It actually works quite fast too, even with high resolution. Up to 10fps on 1680×1050 – the main time hog is taking the screenshot. The analysis and copying is very fast. See screenshot bellow. Note that the speed drops a bit when I set it to stretch (shrink) image to fit on the second screen. There is a lot of room for optimizing though…

The application is basically a framework for capturing screen, analyzing it and reacting to the result. The analysis module architecture makes it easy to get started for people who want to extend it. Each module lets the framework what area of the screen it is interested in and then this area is served to it.

The analysis modules are executed in parallel and/or in sequence depending on what the modules want. It also has a priority queue so it is possible to pre-process the image in one module and then do analysis in another.

As you can see from the screenshot I have 3 working modules: Health, Stamina and Mana. The modules each look at a specific line on the screen where they count the number of correctly colored pixels. Just a straight forward way of reading a percentage indicator.

image 

A simple image analysis module looks something like this:

public int Health = 0;

public override unsafe void NewImage(int imageHeight, int imageWidth)

{

    _bounds = new Rectangle(10, imageHeight - 60, 115, 1);

    Program.SharedStorage.Set("Health", Health.ToString());

    Health = 0;

}

public override unsafe AnalyzerPixelReturnData ProcessPixel(int row, int col, PixelData pixel, byte* srcByteArray, byte* dstByteArray, int imageHeight, int imageWidth)

{

    if (pixel.Red > 170

        && pixel.Green < 20

        && pixel.Blue < 20)

    {

        Health++;

    }

    return new AnalyzerPixelReturnData() { IsModified = true, NewPixelData = red };

}

Image analysis modules can choose between doing their own iteration or having it done for them. In this sample it is being done for the module. ProcessPixel() is called with row number, column number and pixel color. Only rows/cols within _bounds object are served to it.

Applications supports that the target window both moves and is resized, but if it is resized then the analysis modules probably needs to recalculate where they are looking for what.

Source code:

FacebookLinkedInTwitterOrkutDiggShare/Bookmark