Official True Fan Quiz

Star Wars True Fan Quiz Analysis: Can a quiz be designed that accurately classifies an individual as a ‘Star Wars Fan’?

Introduction

Since its first debut in 1977, Star Wars (SW), with its rich story of mystique and science fiction, has undisputedly left a lasting mark on popular culture. From merchandise and apparel, to memes and quoted sayings like “I am your father”, many consider the epic space opera to have evolved from a film on the screen to a pervasive genre of the modern era. Within this culture and ever-enlarging story, there have emerged over the decades several Star Wars enthusiasts and collections of fanfare that together comprise a subculture of modern society; we call them: ‘Star Wars Fans’.

While the term, ‘Star Wars Fan’ is highly variable and frequently up for debate, several quizzes and tests have emerged over the years purporting that those individuals who pass them must possess sufficient knowledge of Star Wars to be Star Wars Fans. Admittedly, all these quizzes are subject to significant limitations as the very term ‘fan’ is highly subjective and might be considered more as a perspective an individual perceives of one’s self, as opposed to a well-defined label within society.

Recognizing these limitations however, designers of TrueSWFans.com worked to create a series of timed quizzes to test their accuracy in effectively identifying individuals that the general public would agree are ‘Star Wars Fans’.

Methods and Materials

Twenty quizzes containing twenty unique multiple choice Star Wars questions each were written and tested by voluntary subjects. The quizzes were timed, and respondents were allowed 5 minutes total to complete each quiz. Respondents were allowed to take several different quiz forms if they desired, but only one submission for each quiz form was allowed. All twenty quiz forms were written in English, with five of the forms translated into Spanish to allow native Spanish-speakers to also participate.

As part of the quiz form assembly, four hundred unique questions from all eleven canon Star Wars films: Episodes I through IX, Rogue One, and Solo – as well as a few questions from Star Wars: The Mandalorian and Obi-Wan Kenobi were written and stratified based on predicted level of difficulty. An emphasis was placed on writing easy and moderate questions that were considered as fundamental to testing core facts of the Story of Star Wars so as to more accurately evaluate an individual’s knowledge of the principal aspects of Star Wars, rather than obscure, more advanced subjects that would not necessarily be reflective of an individual’s interest in the franchise.   

Questions were then organized into groups of twenty such that a fixed ratio of easy, moderate, and difficult questions was contained within each quiz form. In particular, the difficulty-level stratification process entailed placing the questions within a rating scale comprised of 6 levels of difficulty, with the majority of questions falling within the easy to moderate range, and a minority of questions falling into a higher level of predicted difficulty. Fixed ratios for each question form were also set for certain question types, such as identifying a particular scene or character from a Star Wars film, etc.

Prior to beginning a quiz, each respondent was asked to submit responses to the following questions for more accurately classifying their level of interest in the Star Wars Franchise:

1. Do you consider yourself a fan of Star Wars? (Y/N)
2. How many times have you seen the complete original Star Wars Trilogy? (Never, Once, 2 times, 3 times, or 4+ times)
3. How many items of Star Wars paraphernalia do you own? (None, 1-2 items, 3-4 items, or 5+ items)

Question 1 was chosen on the underlying principle that the term ‘fan’ is subjective and thus, understanding how an individual perceives their own interest in Star Wars, may serve to indicate an average score that most fans of Star Wars should be able to attain when taking the quizzes. Questions 2 and 3 were chosen as it was expected that those individuals who exhibited higher numerical responses would also be more likely to consider themselves as Star Wars Fans, and were predicted to score higher.

The quizzes were taken electronically, and quiz scores were then quantified and compared based on how an individual answered the three questions previously stated. Scores were then collected and respondents were allowed (but not required) to take multiple quiz forms, but were not allowed to repeat the same quiz form. For those individuals who did take multiple quiz forms, their scores from each form were collectively averaged to a single score. Scored were then grouped (based on their answers to the three survey questions- self-labeling as a Star Wars fan, number of times having viewed the original trilogy, and number of items of Star Wars paraphernalia owned). Averages for each group across all twenty quizzes were then calculated and 95 percent confidence intervals were generated.

Results

A total of 196 respondents took the available twenty quiz forms (5 of which were also translated into Spanish) a total of 259 times. 37 respondents self-identified as non-fans (NF) of Star Wars, while 159 respondents self-identified as fans (F) of Star Wars.

Study Population Demographics:

Figures 1 demonstrates the relationship between labeling oneself a Star Wars Fan and number of times having watched the Original Trilogy (OT) of Star Wars. For those that individuals that had never seen the Original Trilogy, only 8% considered themselves to be Star Wars Fans. Only 29% considered themselves fans if they had seen the Original Trilogy once, and 75% considered themselves fans if they had seen it twice. For those respondents who had seen the OT three or more times, greater than 96% collectively considered themselves to be Star Wars Fans.

Figures 2 demonstrates the relationship between labeling oneself a Star Wars Fan and number of items of Star Wars Paraphernalia owned. 44% of respondents still considered themselves Star Wars fans although they owned no items of Star Wars paraphernalia. On the other hand, collectively, greater than 95% of respondents who owned at least one item of Star Wars paraphernalia considered themselves fans.

Average scores based on Star Wars Fan self-labeling:

When comparing average scores of those respondents who considered themselves fans vs those who did not, the average score was 49.1% in the Non-fan (NF) group (SD 14, CI 4.6) and 76.8% in the Fan (F) group (SD 18.0, CI 2.9) (see Figure 3).

Average scores by Original Star Wars Trilogy viewing number:

Average scores of respondents according to the number of times they had seen the Original Trilogy (OT) of Star Wars were compared (see Figure 4). For those who had never seen the OT, the average score was 42.7% (SD 12.2, CI 7.7). For those who reported having seen the OT once, the average score was 61.6% (SD 12.9, CI 5.8). Individuals who had seen the Original Trilogy twice on average answered 52.2% of questions correctly (SD 12.2, CI 5.3), whereas respondents who had seen the OT three times had an average score of 64.0% (SD 19.6, CI 7.5). For those individuals that reporting they had watched the Original Trilogy of Star Wars four or more times, the average score was 79.9% (SD 15.6, CI 2.9). From the 95% confidence intervals generated, the average score for respondents who had viewed the OT four or more times was statistically significant as compared to the groups that had watched the OT three or less times.

Average scores by number of items of SW paraphernalia owned:

Average scores of respondents according to the number of items of SW paraphernalia they owned were also compared (see Figure 5). Respondents who reported not owning any SW items answered an average of 56.0% questions correctly (SD 16.6, CI 4.2). The average score for respondents who reported owning 1-2 items was 70.4% (SD 18.5, CI 5.8), while individuals who owned 3-4 items answered on average 73.0% correctly (SD 16.3, CI 6.8). Respondents who reporting owning five or more items of SW paraphernalia had an average score of 82.7% (SD 15.4, CI 3.7). From the 95% confidence intervals generated, the average score for respondents who did not own any items of SW paraphernalia was statistically significant as compared to the groups that owned one or more items.

Individual quiz analysis

An analysis of individual quiz forms was also conducted to evaluate if a particular quiz form was notably easier or harder than the majority. Given the data previously mentioned demonstrated statistically significant lower scores for the group that did not consider themselves SW Fans, it was determined that exclusion of this group when analyzing average scores by quiz form would increase accuracy when determining which forms were relatively harder or easier. The lowest average quiz form score was 65.4% (SD 13.4, CI 6.9), and the highest average quiz form score was 87.5% (SD 22.9, CI 12.3), with a total difference of 22.1%, or the equivalent of approximately 4 quiz questions, as each question was worth 5% of the total quiz score (100% / 20 questions= 5% per question).

To further evaluate relative quiz form difficulty, an analysis of each quiz form by the number of viewings of the Original Trilogy was performed. Given that the data above demonstrated that the 4+ OT viewings group scores were higher and statistically significant when compared to the other group OT viewings of the quiz (see Figure 4), it was determined that this particular group’s average scores would likely be the most representative of quiz form difficulty. The lowest average quiz form score by 4+ OT viewings was 68.0% (SD 12.9, CI 9.1), and was from the same quiz form that had the lowest average score for those who self-labeled as SW Fans. The highest average quiz form score was 90.0% (SD 5.6, CI 3.7), which form had the third highest average score in the group that self-labeled as SW Fans. The total difference between quiz forms was 22%, or the equivalent of approximately 4 quiz questions.

Conclusions

When comparing average scores between those respondents who self-labeled as fans vs non-fans, fans scored higher than non-fans by more than 25%, and the difference appears to have been statistically significant. This finding followed the expected pattern that those who considered themselves SW Fans would also possess increased knowledge regarding facts of the storyline as compared to those who did not consider themselves fans.

When comparing scores of respondents based on the number of times they had viewed the complete Original Trilogy, the averages were directly proportional to the number of viewings, as seen in Figure 4. Interestingly, the group that had seen the OT four or more times appears to have demonstrated a statistically significant average score higher than all other groups, based on 95% confidence intervals. This may be accounted for by the fact that the four or more group included several potential quantities of viewings as compared to the other groups that only included a precise, single quantity of viewings. Regardless, approximately 55% of questions within each quiz were based on material not contained within the OT. Thus, the finding that number of viewings was directly proportional to average quiz scores suggests that viewings of Original Trilogy carries an important correlation to level of overall Star Wars knowledge and may serve as an appropriate marker for labeling an individual as a ‘Star Wars Fan’, particularly if the individual has seen the films four or more times.

Like viewings of the Original Trilogy, the number of items of Star Wars paraphernalia owned also demonstrated a directly proportional relationship to average quiz scores: as the number of items owned increased, so did the average quiz score, as depicted in Figure 5. Interestingly, when compared against all other groups, the average score for the group that did not own any items of paraphernalia appears to have been statistically significant based on 95% confidence intervals. This finding indicates that, while owning at least one or more items of paraphernalia does not necessarily preclude an individual from being considered a fan, not owning any items of paraphernalia would suggest that the individual is likely not a fan.

Post-Study Adjustments

Setting a minimum ‘Star Wars Fan’ passing score

Determining a plausible passing score for a ‘Star Wars Fan’, was primarily decided by viewing the group that self-labeled as Star Wars Fans. The average score for this group was 76.8%, with the 95% confidence interval being 72.0% to 77.7%. Given that actual quiz scores are only possible within 5% increments, using the lower limit of the 95% confidence interval suggested that a 70% cut-off would be appropriate for identifying an individual as a ‘Star Wars Fan’.

Individual quiz form adjustments

Given the findings that the difference between highest and lowest-scoring quiz forms by two different methods was approximately 22% as seen the in the results section, a pairing method was used to substitute questions between forms so as to bring all quiz form averages to within 5% (less than 1 quiz question) of each other.

As the 4+ OT group’s average scores were thought to be the most representative of actual quiz form difficulty for the previously mentioned reasons, the twenty quiz forms were ranked in ascending order by average score for this group. The lowest-scoring quiz form was then paired with the highest-scoring quiz form and questions were interchanged between quiz forms until their new averages were approximately equal to one another (this method was possible as the data included average % correct for each individual question). The second-lowest scoring form and the second-highest scoring forms were then paired, and so forth with questions interchanged till all scores were approximated. After employing this technique, it was found that all quiz form averages were within 3.17% of each other, or less than one quiz question.