Development Diary # 10: Implementing Feedback from the Beta

Sorry it’s been a while since our last update! It’s crunch time for Cally’s Caves 2, which means we are spending a ton of time polishing, bug fixing, and getting ready to submit to Apple. In our last post we detailed how we hoped our Beta test would go, and at this point we have wrapped up the testing phase.  It didn’t last the 8 weeks we planned on, it actually only took about 2 weeks to receive feedback from all of our testers.  We thought it would be a good time to discuss the feedback we received, and how we are using it to make our game a lot more user-friendly and fun.

Screenshot 2014-05-03 15.56.53

The Testers

We ended up having 15 people beta test Cally’s Caves 2, which doesn’t seem like a lot, but for an indie developer like us, it was a gold mine of useful information.  I put a post up on my personal Facebook and Twitter pages, and the testers all volunteered from there.  Of all the testers, only 2 people tested the game on iPad, 5 people used iPhone 5 or 5s, and the rest were on the older iPhone 4 or 4s models.  We were lucky to have a wide range of devices testing the game, and that’s mostly just due to luck with who signed up.  We also (as detailed in our pre-beta dev diary), sent out an introduction package and survey with the beta files, which most of the people filled out.  It also helped to have my phone near me, as testers would often text me the bugs, which I would then add to my to-do list.  We did have people report quite a few bugs, which was amazing because we may have never found them if we were testing the game ourselves.

battle tank

Resolution Bugs

The first major category of bug finds was resolution glitches.  Cally’s Caves 2 is designed for 4 different resolutions, to account for all the post-retina-display Apple products the app will be available on.  When you are making a game where you have to draw everything in every frame on screen, it’s easy to mess things up when you are doing it for 4 different resolutions.  There are also slightly different aspect ratios between some of the resolutions, which makes everything just a little bit tricker.  One of the problems was that the upgrade store menu items were being obscured by an iAd.  Another issue was with some of the text placement on the UI buttons, which wasn’t centered.  On the iPhone 4/4s resolution, the player’s field of view is smaller, so we received good feedback about “leaps of faith” where a player had to jump to a platform they couldn’t see.  All of these are minor details, but how those details are handled can be the difference between a polished app, and an amateur one.

Uh oh..

Game Breaking Crashes

There were quite a few crashes for our testers, as we expected.  The crash pictured above happened every once in a while when a slime enemy is defeated.  The slime would then spit out money, experience orbs, and health pickups endlessly, allowing the player to level up several times before the app hard crashed.  The bug was caused by a misplaced bracket in the code, and was the most catastrophic of all the bugs found.  We also had an issue with our iAds, which only show up in the pause menu.  I forgot to put in a line to disable the iAds if a player went to the map to warp to another room.  This resulted in the iAd displaying over the main game screen, obscuring the UI and weapon swap object.  Since a player has to tap the weapon swap object to change weapons, this was pretty disastrous.  The last game-crashing bug we found was with a variable that wasn’t being initialized properly in gameplay.  This was a very quick fix, but still led to some crashes for our testers.  Overall, these 3 game-crashing bugs were not very frequent, and we found a workaround for the iAds one.  We consider ourselves lucky there weren’t more bugs!



Tutorials and ease-of-use were two areas where we expected problems with the testers.  This was mostly due to the fact that the beta did not include any tutorials, as we had yet to implement them.  We attempted to communicate as much as we could in the beta documentation, but people still needed to figure out certain things.  Some questions included; “Why, and how do the weapons level up?” “What does it mean when Cally Levels up?” and “how do I get up to this high area I can’t reach?”  These systems are pretty straightforward and basic in Cally’s Caves 2, but we learned that they still needed decent explanations.  The beauty of the beta not including tutorials, however, was that it pointed us towards what needs to be explained the most.  We’ve now implemented a series of tutorials across the length of the game, and rotating tips for the pause screens to provide some extra information.  Our beta allowed us to make sure the most common questions we were asked were addressed clearly in the game.

Overall Feedback

We found all these specific bugs and issues in the beta test, but we were also excited to receive overall feedback.  We wanted to know if the game seemed fair to the player, or too difficult.  I was curious how people liked my music, and 0HK0 wanted to know how people were receiving the art.  We were fortunate to receive positive feedback on most counts, although we took it with a grain of salt, since most of the testers were friends.  The more specific feedback helped us make the game more cohesive.  For example, in one tester’s feedback they mentioned that the bear cub enemy in the game is too cute, and doesn’t really fit in with the rest of the enemies in the game.  So we decided to redo the art for the bear cub, to make it more menacing.  Others pointed out that one of the backgrounds didn’t really fit in the later levels of the game, where the player is supposed to be deep in the cave systems.  These general pieces of feedback may seem inconsequential, but to us they were extremely useful to help us make our game the best it can possibly be.

Now that we have wrapped up the beta test, we’ve been implementing changes based on the feedback we received.  We hope that by having a trial run and seeing how players react, we can make Cally’s Caves 2 appeal to a wider audience, and make the game generally feel more polished.  We are now in the final stage of development, in which we are getting the game ready for submission to Apple, doing translations, and preparing our trailer!  We will make sure to do a dev diary on how we made our trailer when it is done.

We are currently 5 1/2 months into the development of Cally’s Caves 2.

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s