Can you trust the Windows App Certification Kit (WACK)?
Yesterday I finished a Windows 8 store app after a couple of weeks work. Everything looked good on a test slate, my dev machine and my colleague’s machine. The app had also passed the WACK tests less than a week ago.
When creating the app packages for the store I rerun the WACK tests and was puzzled by the result. It failed in three of the test sections! One of them was the Direct3D feature level support. I restarted my computer multiple times and rerun the tests, but the test result stayed the same. This lead to hours of tedious testing where I cut down the app into pieces to find out what was causing the problem.
I found nothing and wrote my colleague a mail where I asked him to look into the problem today. I listed a couple of possible reasons and the last one was “my computer might have a bad day”
I got this reply now “WACK tests passed all tests here, your computer must have had a bad day ;)”.
Lesson learned, let someone with a happy computer run the WACK tests if you out of the blue get a failed test result.