Proseminar Master’s Thesis
1. Introduction to the Master’s Thesis
The master’s thesis I decided to analyse is Gamication as a Means of Increasing User Commitment in a Mobile Application for Behavior Change by Lukas Skotsch. It was written in 2018 at the Institute for Interactive Systems and Data Science of the Faculty of Computer Science and Biomedical Engineering of TU Graz.
The work includes a theoretical part, a project for an app, user testing and final reflections.
I chose it because one of my own possible research questions is whether gamification can activate behavioural change in a citizen science project about light pollution. While Skotsch’s thesis wants to draw attention to health, mine aims to raise awareness on light pollution. Nevertheless, I believe that this thesis can help me understand how gamification can be used for education and habit improvement.
2. Evaluation of the Academic Quality of the Master’s Thesis
Quality of Design
The text is set in Palatino, a serif font which I find quite pleasant to read. The line-width is appropriate, although I would have slightly increased the line-height. The wide margins allow the page to “breathe”. The hierarchy is outlined by different font-sizes and by a decimal numbering system in titles and is therefore clearly understandable. Font-sizes could be more consistent in the tables. The image quality could be higher, some are small screenshots.
The overall feeling of the design is serious and essential.
Degree of Innovation
The research questions at the center of the thesis are the following:
1. Can gamification help users stay motivated to reach their goals by keeping them motivated to use a behavior change application?
2. Is there a measurable correlation between personality and preferences for gamification elements?
The work researched gamification and persuasive system design theories which were contemporary at the time (2018). Adobe Acrobat’s AI assistant listed these contemporary methods for 2018:
- A/B comparison: The author developed and tested a gamified and a non-gamified version of the app.
- Attention to a control condition: Retaining the same motivational constructs and app functions while removing game elements was a reasonable design decision.
- Mixed methods: Combining usability scores, rating scales, interviews, and open answers reflected established HCI and UX evaluation practice.
- Use of standardized questionnaires in research: SUS was well-established for usability, and BFI-10 SUS evaluates the usability of technology, while BFI-10 measures human personality traits.
In the abstract Skotsch states: “Due to the lack of standardized assessment frameworks for gamification, a study design for gauging the effect of the gamification on the users commitment to the application was developed. The evaluation was done with an A/B experiment of both qualitative and quantitative nature.” This sentence demonstrates the new approach of this thesis in the first research question.
In chapter 2.5.4 “Unique Feature of this Work” (p. 45) the author writes: “While some of the studies investigated correlation between player types and preference of game elements, none of them took a step outwards from player personalities and looked at a correlation of game elements”: This references the originality of the second research question.
After comparing the gamified and the non gamified app and interviewing users, the author concluded that the gamified version was preferred. While the research questions appear to be original, I find the first answer to be quite predictable and the second one did not bring any relevant results.
Independence
In chapter 2.5.3 “Analysis of Results” (p. 45) the author claims to have analysed 25 studies about gamification in web apps. He also writes: “However, […] comparing and compiling a list of results is barely possible, as no standardized evaluation method exists.” In my opinion, identifying the absence of standardised evaluation methods creating his own collection of methods to answer the questions of the thesis demonstrates independence.
Organization / Structure
This is the content of the thesis:
- Introduction and Research Question
- Background and Related Work- theory
- Design and Requirements
- Implementation Details – developing details
- Application Showcase
- Evaluation
- Lessons Learned
- Suggestions for Future Work
- Summary and Outlook
I find it to be very well-structured: first the questions are outlined, then there is theoretical research of gamification, then the applications (gamified and non gamified) are presented, the user testing is performed and finally the questions are answered with a reflection.
I find every section to be quite exhaustive and connected to the following one. It was easy for me to grasp the concepts and follow through. Since this thesis was written for a computer science degree, the focus lies more on developing rather than design. As a designer, I notice some gaps when it comes to design and iteration.
Clarity of Communication
The concepts were easily understandable for me. I only had issues with the development section, but as this is a computer science thesis and I have a design background it seems reasonable. The author included a short summary at the end of every chapter and a glossary at the end, but it only explains acronyms and I think it could be clearer.
The research questions are clearly outlined at the beginning and theory about gamification, behaviour change as well as user research methods are very well developed and explained.
Scope of the Work
The research about gamification and testing are exhaustive and meaningful, but the UI design could have been refined and more iterations could have been carried out. In fact, only two versions of the app are mentioned, with testing phases happening after each design phase.
In chapter 8.2 “Further Evaluation and Research” the author states: “Given the fact that the main element of the included gamification is based on a weekly cycle, a study spanning two or more weeks of continued usage would be of interest, but was not conducted due to time and resource limits.” I believe that a longer testing period would have given important information about the app’s impact on behavioural change.
Spelling / Care / Accuracy
Apart from some typos that could have been removed with a spelling check, the vocabulary demonstrates a professional proficiency of the language.
Within the text, sources are cited using parenthetical author–date references [e.g. (Ajzen, 1991)]. When clicking on them, you can jump to the complete citation in the bibliography.
After researching how to cite within the text, I found out that there are two ways to do that: author-date references, which can be narrative or parenthetical, and a numbered system.
Here is an example that Gemini suggested to me to explain these methods:
Narrative author-date reference
“According to Brooke et al. (1996), the System Usability Scale is a quick and efficient tool.”
Parenthetical author-date reference
“The System Usability Scale is considered a highly efficient industry standard for measuring software tools (Brooke et al., 1996).”
Numbered system
“The System Usability Scale is a highly efficient industry standard [1].”
Tables and images are also presented consistently and are referenced in the text.
Bibliography
I asked Adobe Acrobat’s AI assistant to analyse the literature by date this is what came out:
- Recent research: Many of the empirical gamification, health-app, and technology references date from 2011–2018. The related-work review includes studies up to 2017, and the bibliography cites 2018 documentation for frameworks and services such as Ionic, Flutter, React Native, Rails, and health or gamification apps. This means the author did not rely only on older literature.
- Foundational older sources: The thesis also uses earlier work where it establishes a theory or concept still relevant to the topic: the Health Belief Model (1974), social cognitive theory and self-efficacy (1977 onward), Theory of Planned Behavior (1985/1991), ubiquitous computing (1991), the SUS usability scale (1996), and classic game-design frameworks from 2004 and 2009.
- Very old historical material: A small number of sources are much older, such as Dewey’s education work originally published in 1916. These are used to provide historical context for learning through play, not as evidence for current app effectiveness.
I find using modern sources for gamification principles and older ones for psychological theory quite reasonable, as the first is constantly evolving and the latter is less subject to change over the decades.
I then asked Adobe Acrobat’s AI assistant to analyse the literature by type and this is the result:
“The bibliography is mainly a research-oriented mix of academic literature, conference papers, books, and web documentation. The reference list is interdisciplinary: it combines computer science and UX research with psychology, health behavior theory, game design, and practical mobile-development documentation. Most scholarly sources support the conceptual background and evaluation of gamification; web sources mainly support descriptions of software frameworks, apps, and technical tools rather than the thesis’s core claims.”
While I think that the bibliography is a good combination of different media, I think research about UX and UI principles is missing when it comes to the content.
The bibliography is alphabetized by author and includes titles, editors, publishers, page ranges, DOIs, URLs, and access dates for websites. The citations do not follow a traditional style like APA or Chicago, but a layout generated by Biber (LaTeX), a citation management subsystem. This is declared at the beginning of the document. The interactive pdf allows readers to click on links and go back to the pages where the citation is mentioned in the text.
While I find the parenthetical citing in the text effective, I believe that the author could have used a standardised citation method like APA or Chicago.
In addition, there is a list of figures, which I am also going to incorporate into my thesis.
3. Evaluation of the Practical Project / Work Produced
The author developed an app that wants to help its users to live a healthier life, featuring missions and tasks. The ultimate goal is to beat their inner pigdog (Schweinehund). A gamified app and a non-gamified one was created, then an A/B test was conducted to establish preference for the gamified version.
The app which was developed is scarcely documented in the thesis. There are only some screenshots in a section called “Application showcase”, but no link to interact with it. The author claims to also have created and tested a non-gamified version of the app, but it is not visible.
The project features a weekly competition with points for each completed task and a reward system with badges. I see the project’s connection with the first research question (Can gamification help users stay motivated to reach their goals by keeping them motivated to use a behavior change application?), but I find the second research question (Is there a measurable correlation between personality and preferences for gamification elements?) not to be incorporated into the project. Nevertheless, a personality test was also conducted but no correlation with preference for gamification features was found in the users’ behaviour.
I find the documentation of the app incomplete as not all of the four sections (Home, Ziele, Motivieren, Profil) are present in the screenshots. The documented sections and all of the gamification elements are well-explained though.
I think the UI design of this thesis is not sufficiently developed. It is missing competitor research on UI frameworks, the illustrations are primitive and the layout is not always consistent. On the other side, common wording, icon design and placement is used. I reckon that the author could have improved his project by checking it with the 10 Usability Heuristics for User Interface Design by the Nielsen Norman Group, and by following an iterative design method.
I personally would not have used the pigdog (Schweinehund) expression, as I am against speciesism and picturing any animal in a negative way. I also think that when presenting a product we should focus on positive goals and consequently positive wording instead of blaming someone or something for our bad habits.
Overall, I think that from a design point of view this thesis does not have the quality for a master’s thesis, but I understand that the author’s background differs from mine and that in 2018 app design had different standards.
4. Final Conclusion
To summarise my review, these are the thesis’ strengths:
- good content structure with summaries after every chapter
- hyperlinks for index sections and bibliography
- clarity of communication and glossary
- innovative research questions
- extensive theoretical and user research
- complete bibliography and image sources
On the other hand, these are its weaknesses:
- predictable research answers
- interactive project is not accessible
- UI design
- few iterations
- short testing period
- no standardised citation method
In conclusion, the theoretical and user research is exhaustive and the structure is clean, but it is lacking design principles and methods due to the author’s computer science background.
5. Relevance to my own final thesis
These are the recommendations I found for myself after analysing this master’s thesis:
- research UI principles and competitors
- pick a design method and follow it
- carry out many iterations
- execute a first check of the project using the 10 Usability Heuristics for User Interface Design by the Nielsen Norman Group
- deeply focus on user testing
- use a mixture of sources (books, papers, websites)
- pick an innovative research question that does not have a predictable answer
- pick a standardised citation method
- communicate something with the layout of the thesis