Testing Framework is Complete

The testing framework now tests map information functions and as well as events/callbacks. These were the last two groups of test cases that I planned to add to the initial testing framework, bringing it to completion. That said, most functions in BWAPI use information obtained from Starcraft's memory, and like any RTS game, Starcraft has a very, very large state space, so of course the testing framework is no where near exhaustive. However, for now its safe to say that the framework tests unit commands, unit, player, and map information functions, and verifies that events/callbacks are generated when and only when they should be. However, if any bugs pop up in the future, I can add test cases to the framework to ensure that the bugs stay fixed.

I've also run the testing framework on the BWAPI client and it runs without error, so it looks like the client is nearing completion as well.

BWAPI Beta 3.0 has been released

BWAPI Beta 3.0 has been released. This release fixes a number of bugs, improves the performance and functionality of the client-server architecture, adds several new functions, merges several Orders together, and more. Here is the full list of changes.

Unit Command Test Cases Complete

After pulling an all-nighter, I'm happy to say that I've completed test cases for every unit command and they all pass on the server and client without a single failed assert. Perhaps the hardest test cases of the 43 unit commands were cancelTrain(slot), cancelMorph, and cancelConstruction, as different attributes of the unit had to be latency-compensated for different lengths of time, making them rather tricky puzzle pieces.

An off-by-one mistake in the latency compensation code would over-compensate or under-compensate the unit or player attributes (i.e. giving the player the appearance of having more or less resources than he actually has), leading to a failed assert that would only pop up for one or two frames at a time and then vanish in a game that's being run at 180 frames per second. Nonetheless, I have determined the shape of these puzzle pieces and now the latency compensation code is a perfect match for even these three unit commands, allowing for a seamless transition from latency-compensated values to actual values read from Starcraft memory.

As explained in my last post, my next main goal is to make test cases for events and callbacks, and more importantly start making the internal adjustments in GameImpl.cpp that will be necessary to make those new test cases run successfully on the client.

Test Framework Update

As planned, I've added test cases for the following unit commands:

• siege
• unsiege
• cloak
• decloak
• burrow
• unburrow
• lift
• land
• load
• unload
• unloadAll
• unloadAll(Position)

I've also added test cases for the following unit commands:

• setRallyPosition
• setRallyUnit
• rightClick(Position)
• patrol
• holdPosition
• stop

All test cases run without error as a DLL in Broodwar memory and as a separate client process. Now only the following unit commands are left to be tested:

• attackMove
• attackUnit
  
• rightClick(Unit)
• follow
• repair
• returnCargo
• cancelConstruction
• haltConstruction
• cancelTrain
• cancelTrain(Slot)
• cancelAddon

Once I add test cases for these last 11 functions, I plan to add a test map that specifically deals with unit destruction and AI module call backs. For example, when a unit dies it should generate an onUnitDestroy callback. I also plan to re-write the way BWAPI generates events and callbacks to facilitate the process of making them behave the same way on the client as on the server. To do this, I'll make BWAPI just generate Events in the appropriate places, rather than both Events and perform the callbacks immediately. Then, right before AIModule::onFrame() is called I'll translate the events into callbacks and perform them. This will make the AIModule callback behavior virtually identical to the Client Events since they'll both be called at the same point of execution in GameImpl::update() for the most part. I'll also need to make some modifications to the way units get destroyed or become invisible so that their last frame of valid unit information is still available at the time of the callback or Event. For example, right now when an enemy unit dies, the client doesn't have access to the last frame of valid unit information because it is erased in GameImpl::onUnitDestroy(). I plan to make this function just set a flag in the UnitImpl class to indicate that the unit is dying, and only add UnitDestroy events. The UnitData struct for the recently destroyed unit would then be cleared after the AIModule::onFrame() and Client update functions have been called.

Testing Framework Runs Successfully on the Client!

Since my last blog post almost a week ago I've re-written the latency code so that it is all nicely written in a single template class which is then used both in the BWAPI dll (server) and on the client-side of the client-server connection. I've also recently fixed a few other bugs in the client.

As a result, I have now successfully ran the testing framework through all the test cases on all three major test maps (TerranTest, ProtossTest, and ZergTest) without error. This is a major milestone toward completing the client-server architecture.

My long-term goals now are to make the testing framework comprehensive as well as make sure all test cases continue to succeed both server-side as a normal AI module DLL and client-side. My next immediate goal will be to add test cases for siege/unsiege, cloak/decloak, burrow/unburrow, lift/land, and load/unload/unloadAll/unloadAllPosition.

Client-Server Optimization Progress

The server code is now optimized for getting unit and map information into shared memory for the client. To get a ballpark estimate of how much of a speed improvement was attained, I ran both the most recent version revision of BWAPI and the most recent release of BWAPI - 2.8 - on the same replay and measured the frames per second each version could achieve when playing with /speed 0 and the replay mode on Fastest x16. Before optimization, the client and server could run at 53 FPS. After optimization, the speed increased to 87 FPS. While these numbers are specific to my computer, other computers will likely get a similar 50-70% speed increase.

Also, the test framework now tests every special ability, using related unit information commands like isStimmed, isStasised, and isLockedDown to verify that the abilities were executed successfully.

Client-Server Optimization

I've decided to take a break from the testing framework to work on the client server architecture.

The client-server architecture is a new alternative way of making AIs with BWAPI. Traditionally, people would write their AI in a DLL which would get loaded by BWAPI at the start of a match. In contrast, the client-server architecture lets users write their AI as a separate executable program which connects to their local BWAPI server (loaded in Broodwar memory), and communicates with it via shared memory.

This new architecture is almost fully functional, however up to this point it has been extremely slow due to the method that BWAPI uses to get information from Broodwar into the shared memory bank. The inefficient/simplistic implementation currently just calls relevant BWAPI::Unit member functions, and copies the results into shared memory. Since each unit's state is comprised of about 100 different pieces of information, this results in about 100 BWAPI function calls per unit. And each of these functions call other functions in order to generate the appropriate error codes when needed and determine if the user/AI is allowed to access the given unit (hidden units are not accessible unless Flag::CompleteMapInformation is enabled).

To optimize the process of getting information into shared memory, BWAPI will compute the UnitData struct directly from Broodwar memory, bypassing the BWAPI function calls. Thus the last few days I've been moving implementation details of each individual Unit member function to a single function, called Unit::update(), so that it computes the entire UnitData struct directly from Broodwar memory without using any unnecessary or redundant function calls. In addition, to prevent duplicate code I've been simplifying the Unit member functions so they just compute their answers from the UnitData struct, just like the client-side implementation of the Unit object does. This should also go a long way to ensuring that BWAPI behaves consistently whether you're writing you AI as a DLL or as a separate client process.

In short, I'm making a rather large internal change to how the BWAPI::Unit class works in order to optimize the client-server architecture, however the end-user functionality should go unchanged. Also, the test framework module is a DLL, so it can be run both as a DLL in Starcraft memory the traditional way or be loaded into a separate program such as BWAPI's AIModuleLoader and run in a separate client process, so the test framework will be able to test both implementations to make sure they behave correctly and indistinguishably once everything is complete.