Putting something you crafted into someone else’s hands is a harrowing experience.  When that person’s gaze locks onto the object you created, a shift happens.  What was once your private project is no longer your own, and a singular experience becomes shared one.  It can be amazing or terrifying, but either way you will walk away from that shared experience with an idea of how you could have approached your creation differently.  The beauty of game development is you can choose to incorporate the feedback you gain from sharing your creation with others before your product is finalized.  This is why we have dedicated a two-month period to beta testing Cally’s Caves 2.  In today’s Dev Diary we will discuss how we have approached the 8-week beta test period of our game’s development, and what we hope to gain from putting Cally’s Caves 2 in the hands of playtesters.

Content Complete

The first step towards beta testing is completing the alpha phase of development.  In our case, the alpha and beta periods of Cally’s Caves 2 are delineated by the addition of new content.  While we were coding in new systems and adding stuff that drastically changed the overall game, we were in alpha.  Once we finished adding in new stuff and were satisfied that the game matched our initial design, we were ready for beta.  We wrapped up the alpha phase on March 30th, and are now focused on pre-beta testing and bug fixing.  At this point we may be content complete, but that doesn’t mean everything in the game is totally finished.  Some of the sprites currently in the game are still placeholder art, and the game’s map/warp system is also still based on our initial rough sketches (although fully coded and functional).  The main storyboards for the game are also incomplete and using placeholder art, however we are happy enough with the game to put it tester’s hands.  The key will be explaining what is incomplete to the testers, so they don’t waste time worrying about what we already know.  We’d rather get the game into tester’s hands early with placeholder art than later with everything totally polished.  We will simply keep putting in the final art as we complete it, and update the beta version.  We want to make sure every piece of feedback is incorporated into the game, and that means getting people playing it as soon as possible.

Making a Tester Package

We prepared a document to give to all the testers of the game with two goals in mind.  Firstly, we wanted to introduce people to the game (since some of our testers are people we have not met IRL), and give them a brief introduction and explanation of the placeholder art.  Secondly, we wanted to deliver a survey to the testers, to ask them about specific concerns we had about the game.

The introduction section is the one chance we have to explain the game to our testers. Fortunately, all of our testers have played the first Cally’s Caves, so we decided not to give players an introduction to every single system in the game. Instead, we decided to communicate what is going to be added before we ship the final product. This will help keep our testers focused on the areas we need feedback on (instead of reporting stuff we already know will be replaced by the final version).  We also strongly suggest that the testers be brutally honest with their feedback.  A bunch of comments that say “I liked the game” isn’t really going to help us make our game better, so getting testers that won’t be afraid to be honest with their criticism is absolutely crucial.

The survey we gave players begins with some very specific questions.  When you are developing a game it is nearly impossible to view it objectively, so the survey questions are key to getting the most out of a beta test. The specific questions are geared towards issues that we are already aware of, but are having a hard time making a final decision on.  We hope that with a large enough sample, we can find guidance in the answers that players submit.  The specific questions are followed by a series of open-ended questions and a blank feedback area.  We feel that it’s important to get as much feedback as possible, while also empowering the beta tester and demonstrating that their input can have a huge impact on the game.  We also begin with the specific questions to help train the tester to think critically about the game (not every beta tester is a seasoned gaming veteran), so by the time they reach the open-ended feedback section they will be accustomed to critiquing the game.

Incorporating the Feedback

 After our beta testers have finished their survey/feedback and have sent it back to us, we are ready to make changes where appropriate.  Not every objection or suggestion will result in sweeping changes to the game, but we will discuss each and every piece of feedback thoroughly.  It would be ignorant to dismiss any piece of feedback as incorrect, no matter how minor the issue.  The changes we decide to make as a result of the feedback will be put into the game’s to-do-list (named “Cally’s Caves 2 Code Issues,” which is our daily reference document.  It has been our guide since the very beginning of development, and has had code issues on it ranging from “add guns” to “adjust font size on pause menu on iPhone 4s resolution.”  Every day when we start working on the game, we refer to the code issues list and tackle as many of them as we can.  We hope this workflow will result in a productive beta test period, as we will get our testers feedback and then put the relevant changes directly in our daily to-do list.

We were incredibly nervous when we put Cally’s Caves 2 into someone’s (who wasn’t a part of the development team) hands for the first time.  But seeing them play the game also immediately gave us ten ideas on how to improve Cally’s Caves 2.  This – and our experience with the first Cally’s Caves – demonstrated to us exactly how important having outsiders playtest the game is.  You can never truly tell how someone is going to play your game until you put it in their hands and watch what happens.

We are currently 4 1/2 months into the Development of Cally’s Caves 2, and our Beta Test will run for the next 8 weeks.  We will write a follow-up dev diary on the specific pieces of feedback we got at a later date.

If you are interested in beta testing Cally’s Caves 2, email me at jordanfwc@gmail.com and we will sort it out!