This was a class project for Usability Testing.
Our main goal was to evaluate how experienced gamers use the tutorial system in Assassin’s Creed IV: Black Flag and deliver design recommendations. Unfortunately we cannot provide the results of our study due to Microsoft security reasons, please send me an email if you have any questions.
Our team began by focusing on who the “experienced gamer” is by creating personas to focus our communication. We then played AC4 to get a strong understanding of what exactly the new features are and how we should go about evaluating the game’s tutorial. With this knowledge, we focused on three high level objectives and formed measurable research questions for each objective. After testing our participants, we quantified our observations and presented our findings in order of severity to progressing in gameplay.
|Assassin’s Creed IV: Black Flag||Our team will be conducting a usability test on the tutorial system for Assassin’s Creed IV: Black Flag (AC4). We need to evaluate the tutorial system and characterize its effectiveness for experienced users.|
|Overall Objectives||For experienced gamers new to AC4:
|Research Questions||Identify helpful tutorial information:
Identify superfluous tutorial information:
Identify confusing tutorial information:
Identify absent tutorial information:
|Participant Characteristics||Experienced Gamer – 6 to 8 participants
|Methodology||SummaryThis usability study is mainly exploratory in that we will look to generate hypotheses regarding experienced gamers using the AC4 tutorial system. In doing so, we will gather assessment data about the effectiveness of the AC4 tutorial system for experienced gamers. We will collect quantitative data about game function execution, categorization of tutorial prompts, and confidence in learned game functions. We will also collect qualitative data on participants’ experiences using the tutorial system.Observation studyIn our observation study design, all participants will playtest through AC4 from the beginning. We will not be able to counterbalance participants considering the sequential learning involved with initial game functions.
We will conduct 6-8 individual 1.5 hour usability study sessions. The first 15 minutes of each session will be used to explain the session to the participant and confirm basic background information with the participant. During the middle 1 hour, participants will play to progress through AC4. The last 15 minutes will be reserved to administer our post-test and debrief our participant.
Pre-test arrangements (3 minutes)
Have the participant:
Introduction to the session (7 minutes)
Background interview (3 minutes)
Discuss the participant’s:
Task (1 hour)
Participants will start at the beginning of AC4 with the task of progressing through the game story.
Post-test debriefing (10 minutes)
|Task List||Participants will start at the beginning of AC4 with the task of progressing through the game story for 1 hour.While AC4 contains features and game functions that occur past 1 hour of gameplay, the main tutorial system is presented within the 1 hour. Also, an experienced gamer new to AC4 would have to start at the beginning when playing the game outside the context of usability testing.Areas of interest-place tutorial prompts and screenshots.|
|Test Environment||We will utilize Microsoft Studios research testing facilities to conduct the sessions. Participants will play AC4 on an Xbox 360. The video information on screen will be captured using OVO recording software. We will also bring a recorder to gather a set of backup recordings.|
|Test Moderators Role||Our role is to carry out our test methodology in an unbiased manner. As the participants progress with the test, we will gather our data through observational notes, participant talk aloud, surveys, and interviews.After our test session, we will analyze and tabulate the data to answer our research questions. We will then present the data to our classmates for feedback; then create a succinct written report.|
|Data to be collected and evaluation measures||Our measures will answer our research questions. We will collect both performance and preference measures.Performance:Collection Method – Remote observation
|Report contents and presentation||We will deliver a presentation and written report that: