Short title: Applied UX Design for Educators
Authors: Dora Obodo (Increible graduate student at Vanderbilt),
Deborah McCabe (MICA UX Design Peer), &
Kendra H. Oliver (Me)
Overview:
User Experience (UX) design applies a user-centric design process to develop new technologies that improve the student experience. Today, educators and university administrators are interested in leveraging technology to engage current and new audiences better. However, case studies for applying UX in course design are limited and need to be more present in the chemistry education field. Here, we present a case study detailing the UX-based approach used in the design evolution of a drug discovery course for external audiences. We found that our external users had extremely low completion rates using a traditional learning management system to develop the courses. We followed the UX double diamond design process to identify design challenges and address them strategically from a user-centered perspective. We constructed the personas of our users, identified key features, and developed low, mid, and functional prototypes for the new user-based course with iterative usability tests. This case study demonstrates UX design's impact on online chemistry content development by universities and institutions that want to engage external users.
INTRODUCTION
Chemistry is multifaceted, and the pharmaceutical industry (Pharma) creates many innovations that touch all parts of our everyday lives1,2. This requires a multidisciplinary perspective and drug discovery is dependent on disciplines. Integration between pharmacology, medicinal, analytical, and biological chemistry are need to champion research and development of drugs for common and rare diseases, establish and formulate said drugs, and ultimately bring them to market 3–5. As an ever-growing field in which ideas are changing, Pharma offers many opportunities to young scientists entering the field and veteran scientists looking to advance in their career trajectories 6–8. These chemists are looking for state-of-the-art training in chemistry and the pharmaceutical industry and turn to continuing education opportunities 9,10.
As a result, leaders at the forefront of academic and pharmaceutical innovation are increasingly investing in training for pharmaceutical chemists and professionals 9. Digital and online resources like massive open online courses (MOOCs) have been used to deliver didactic content. Platforms such as Coursera (Stanford), MIT OpenCourseWare and EdX (from Harvard and MIT) offer several courses related to drug discovery and design, which often aim to teach skills and develop core competencies required to build a pharmaceutical career 11–13. As compared to degree granting programs, these systems often prioritize accessibility and inclusivity for non-traditional and non-academic students 14–16. However, designing effective digital chemical education courses remains challenging compounded by the fact that chemistry is often perceived as hard to learn or difficult to engage virtually17,18. Thus, higher education chemistry and science departments often struggle with how best to deliver this content to non-traditional students 15,19,20.
A better understanding of designing online programs centered on user experience (UX) may allow universities to better engage non-traditional pharmaceutical professionals 21. Traditionally, digital teaching occurs through learning management systems (LMS) such as Blackboard, Brightspace, or Canva. These platforms centralize technologies and teaching resources online for learners 22–24. LMS offer many benefits for teaching chemistry. For example, teachers can use LSM to organize and share recorded lectures, materials and assessments. Depending on the instructional design approach, these materials can be used by student to revisit the material (self-directed learning) or prepare before coming to class (flipped classroom) 25. Typically, LSM support in-person training however, the COVID19 pandemic brought to light the limitations of these systems26–28. This transition also highlighted the distinction between instructional design which aims to transfer knowledge or skills to learners or students, whereas UX design wants to create a usable (typically digital) experience 29–32. The success of digital learning environments to engage learners must consider the intended audience's distinct pain points and behaviors. Behavior-based measures of platform success, such as course completion rates, may provide a better indication of educational platform performance as compared to more traditional pedagogical measures. Educators typically don’t consider UX-based design strategies which could be alterative complementary measures to support accessibility and inclusivity when designing online resources for non-traditional learners.
Designing courses based on UX-design principles offer several benefits 21,33. In general, trends in digital learning show low student retention, often with 30% - 50% rate of completion in LMS-style courses and with rates approaching 40% for MOOCs 34–39. This number may be attributed to poor and often unintuitive course design 35,40,41. These courses may also require logging into various platforms, which hinders the user learning experience. A UX-design centered approach requires the educator to consider the digital experience for the learner. Studies show there to be a correlation between student engagement, as described by components of the learning environments (e.g., interactions between students and the learning systems, technologies, resources, navigation) and eventual learning outcomes 42–44. Studies also show that continuing education students often value flexibility, simplicity, and a collaborative learning environment. These are ideas that must be prioritized when designing chemistry content for professionals.
Here, we present a case study detailing the UX-based approach used in the design evolution of a drug discovery course which was initially hosted on a traditional LMS. We found that for our users the tradition LMS course had low completion rates. We followed the UX double diamond design process to identify design challenges and address them strategically from a user-centered perspective. We constructed personas of our users, identified key features, and developed low, mid, and functional prototypes for the new user-based course with iterative usability tests. This case study demonstrates the impact UX could have on online chemistry content development by universities and institutions that want to engage external users.
MATERIALS & METHODS
VUSM Basic Sciences Drug Discovery LMS Online Program
The program was initiated in 2018, leveraging an external iteration of an LMS to increase current students' drug discovery research literacy and showcase the therapeutic development expertise within the Vanderbilt School of Medicine Basic Sciences. Specifically, the interactive curriculum of the program was designed for external participants interested in using online resources to enhance their understanding of drug discovery. The program was structured as a series of two-week mini courses that covered various topics crucial to drug discovery and development. The program was closed in March of 2023. The course structure and offerings during first launch in summer and fall 2021 can be seen in Supplemental Figure 1. A sample course syllabus and course schedule for one module in the course can be seen in Supplemental Figure 2.
Participant recruitment and audience
Once the course was launched, we recruited students by advertising via LinkedIn, Vanderbilt School of Basics Sciences email listservs, and a subscription email Newsletter offered on the course website homepage. From there, students could register, enter, and complete the course. Students who enrolled in one or more courses were automatically subscribed to the course newsletter. Demographics of subscribed students from newsletter can be viewed in Supplemental Figure 3.
IRB Approval
This project’s data collection includes compiling web search results, user interviews, and observation of twenty participants’ use of our LMS courses and three iterations of a prototype. We used semi-structured interviews with the participants. This is a low-risk human research project whose ethics has been cleared by the Maryland Institute College of Arts and Vanderbilt University Research Ethics Committee. The ethics approval numbers are VU220210 and MICA IRB Protocol 117 for Vanderbilt University and MICA, respectively.
User Experience (UX) Design
UX is an emerging and evolving field that offers a process for developing products informed by the audience's (user's) needs. UX design is the process of designing (digital or physical) products that are useful, easy to use, and pleasing to engage. This process is guided by how a product's design can enhance the user experience and identify the value provided. For a comprehensive overview of UX Design for educators we suggest Earnshaw et al 2018. Case studies of applying UX design for educators are limited and are nonexistent in the chemistry education field.
Double Diamond UX Design Process and Design Methods
The British Design Council popularized the double diamond process in 2005. The two diamonds represent exploring an issue more widely or deeply (divergent thinking) and then taking focused action (convergent thinking, Supplemental Figure 4). The first phase is the "discover" phase which involves taking steps to understand an issue rather than merely assuming a solution. Within this phase, we conducted two methods; a competitive analysis and a quantitative assessment of student completion rates based on the first implementation of the LMS course and interviews. The next step in the process is to “define” the problem, during which we refine the insight gathered from the discovery phase to determine a course of action. The method used for this phase was generating personas. The third phase is to “develop” a new prototype based on the feedback we have collected and continue testing said prototype through users. The final stage is “deliver,” which involves building out the prototyped solutions at a small scale, rejecting those that will not work, and improving the ones that will. This was called our functional prototype.
Competitive analysis.
To start, we performed a competitive analysis of direct and indirect competitors to the LMS-style program (Table 1). A direct competitor offers the same course or targets the same student audience. An indirect competitor targets a different student type or offers a different course product to the same target audience. To do this, we conducted a google search for LMS-styled drug design and development programs with direct or indirect infrastructures and narrowed down the comparisons. After reading through course descriptions, we compared them with our program based on their potential appeal to students, objectives, types of content offered, and other factors. We also evaluated key features for each type of competitor (Supplemental Table 2).
Quantitative evaluation.
We performed quantitative analysis on the LMS course, extracting usage metrics about user registration and completion rates to build user pathways with rates (Supplementary Figure 5).
We interviewed students currently or previously enrolled in the LMS-style course to understand users' needs after taking the LMS-style program. We wanted to understand user motivations, experiences, and key outcomes, as this would allow us to later classify user groups. We interviewed nine people: four advanced science undergraduate students just starting their careers and five working professionals interested in getting into or switching tracks within the drug discovery industry. The questions focused on the users’ prior experiences and present emotions about online education. We also asked a series of questions about their motivations, uses, and engagement with the online courses, including the educational, social, and cultural factors that may have influenced their experience. The interview was recorded and transcribed. Readers can find a complete interview guide in Supplementary Document 1.
Generating personas. From interviews with participants of the course, we defined and contextualized the qualitative information by creating user types to test whether they fit our target user groups. User groups are primarily defined based on occupation and qualifications. From here, we developed personas, which are used to characterize behaviors, expectations, and pain points of our defined user groups (Figure 2).
SWOT Analysis, and value proposition.
Based on the interviews, we summarized the next steps for the course using a Strengths, Weaknesses, Opportunities, and Threats (SWOT) analysis to determine the strengths and weaknesses of the course based on the user's qualitative and quantitative data (Supplemental Figure 6). A strengths, weaknesses, opportunities, and threats (SWOT) analysis has become a key tool used by businesses for strategic planning. It is often used by organizations to evaluate their position and to analyze their internal and external environments during times of indecision 45–47. Strengths refer to the internal elements of an organization that facilitate reaching its goals, while weaknesses are those internal elements that interfere with organizational success. Opportunities—external aspects that help an organization reach its goals—are not only positive environmental aspects but also opportunities to address gaps and initiate new activities. Threats, on the other hand, are aspects of the organization’s external environment that are barriers or potential barriers to reach its goals 48–51. Value can be created through multiple elements, such as price, quality and location 52.
Prototyping.
Prototyping describes the development of early, tangible models of ideas that can be tested with the user to refine the design. We went through 2 stages of course content and structure prototyping in which the first involved creating an app for the course while the second included adding features to the app. All prototyping was designed in Figma, a free web-based design tool (Figure 4).
Usability Testing.
We aimed to evaluate our prototype by testing it with participants who completed the LMS version of the courses to try to walk through various platform features. The researchers watched, listened, and took notes as the users completed these tasks. We also asked about the users' expectations and preferences for the platform and asked them to compare it to their previous LMS experience. The goal was to identify any usability problems, collect qualitative and quantitative data and determine the participant’s satisfaction with the new prototype. Readers can find a complete usability guide in Supplementary Document 2.
Functional Prototype.
We finally developed a temporary website design as a functional model for the new course platform. We designed this final website using Wix, a free website creation tool. The final web-based version of the course was entirely hosted on the Vanderbilt School of Medicine Basic Sciences website. We recruited students to participate in the new online courses through email newsletters attached to the course and LinkedIn posts from a leading faculty member teaching the new course. From these media platforms, we extracted usage metrics about user registration and completion rates.
Thematic evaluation of usability tests. For both the low-fidelity and high-fidelity usability tests, the user’s comments were evaluated based on summarizing themes developed by the research team and were grouped as positive or negative sentiments. Any comments about individual features and aspects of the interface were added to the “design” theme. “Ease of use” referred to comments about the enjoyability, efficiency, and effectiveness of the user's ability to perform tasks. The “content” theme evaluated users’ expectations on content that they thoughts was present or missing from the experience. “Course structure” summarized comments about the information architecture and structure of the courses. “Ease of navigation” described comments on the design and visual aspects that made the interface intuitive to maneuver. Finally, “accessibility” related to comments on if the interface allowed for a stimulating learning environment.
RESULTS
DISCOVER
While the idea of an online learning ecosystem for drug development is not new, we aimed to make it specific for an external audience of working pharmaceutical professionals. To understand how our offering compares with the current online learning ecosystems, we performed a competitive analysis with three direct and indirect competitors in this space (Table 1). Our three direct competitors were all certificate or full Online Master’s programs that used LMS to administer their content. All required a background in biology or chemistry and offered the opportunity to learn the drug development process. In contrast to our system, direct competitors offered synchronous components in addition to their asynchronous online content, and some used various additive technologies in conjunction with the learning management system to improve the student experience. However, the number of systems students navigate could potentially complicate its use and decrease compliance. Similarly, it appeared that competitors sometimes developed new learning systems to supplement the course delivery platform. Our three indirect competitors offered programs marketed to external and professional users but outside of our content area. Most were hosted on MOOCs that were easy to use, easy to navigate, and were offered asynchronously. In contrast to our course, most systems were price-based, often with higher quality products having higher prices, longer durations, and lower levels of interactivity.
The LMS version of our course officially launched in Summer of 2021 and consisted of nine self-contained modules covering key ideas and processes in the drug development pipeline, with three key pathways: core modules, drug metabolism and pharmacokinetics, and medicinal chemistry (Supplementary Figure 1). The course was hosted on Brightspace, an LMS which supported multimedia and an array of learning resources such as quizzes, activities, and discussion boards and allowed for external non-institutional access. Each module featured one introductory section and three-to-five content sections with at least one short video, a discussion board, and several activities and quizzes (Supplementary Figure 2). The introductory section introduced students to the faculty member teaching the class, the proposed learning objectives, instructions for completing the class and how to get support for any additional questions or concerns. Finally, students earned a completion certificate for the course if they completed all module quizzes with a 100% score. Modules could be completed asynchronously with deadlines for the quizzes and activities. If participants did not complete the quizzes by the course closure date (two weeks after the start of a module), they did not achieve the course completion certificate. This structure allowed participants to learn at their own pace and to self-select sections to partake in based on their interests, which was the main draw of the course.
As students registered through the course website, we gauged interest by analyzing traffic to the website. Following course launch, the top webpage gained 1,894 page views. Of these views, 1,466 were unique, indicating individual sessions (i.e people) who explored the main program website, while a quarter indicated possible greater interest by returning to the page more than once. Traffic to the site came primarily from direct access (23.55%), followed by the emails, and finally from LinkedIn. 24% from the emails and newsletters, and 11.99% from LinkedIn (Supplementary Figure 5). The total number of registrants (495 registrants) seems to correlate with the quarter of people who showed more interest in visiting the site.
Finally, 120 registrants enrolled in one or more modules of the course (59% registered for one course, 34% registered for two-four courses, and 8% registered for over six courses). We could infer the demographics of those enrolled as they also subscribed to the course newsletter using a subscription sign-up form during enrollment (Supplemental Figure 3). There were 104 subscribers, who were mostly Vanderbilt University affiliated male professionals working in Nashville, TN. MailChimp predicted that 29% were female while 55.1% were male. It also predicted that most subscribers (51.4%) were between 25-34 years old. Top demographics showed that of the subscribers, 45.2% were current members of the Vanderbilt community, while 45.1% were external community members. Finally, in the subscription form, subscribers indicated that they were mostly interested in the basic core content (39.4%), drug metabolism, and pharmacokinetics. The next most popular topic was professional development (21.2%), with very little interest in special topics (8.6%) or entrepreneurship (6.7%).
An overall 0.05% of registrants completed all courses. Students incidentally dropped out of the course as they progressed through the modules with the highest number of learners joining the first module. Overall completion rates for all individual courses were 0.1% or lower. Additionally, engagement with the material was very low, with most learners only logging into the system once. This suggested a significant issue in our content delivery or the platform that resulted in participants not completing or engaging with the material.
DEFINE
Because our user completion rates were so low, we decided to collect qualitative data by interviewing participants to better understand their experience of the course. We defined two hypothetical types of users based on participants which helped us sample participants who could represent the bulk of user experiences. The first user was an advanced undergraduate student or recent graduate who was interested in exploring potential careers in drug discovery (Type B). The second user type was an advanced professional working in the field interested in building a network with other professionals (Type A). We interviewed four and five users from both groups, respectively. We found that Type A users typically had very little time to dedicate to content and therefore required the platform and content to be easily accessible, flexible, and user-friendly. Their habits were often to log on to the website on weeknights and weekends (e.g., to counterbalance time spent with kids and daily life). As such, they also have varied levels of interaction with the course, often only selecting content they found most relevant. On the other hand, Type B users were often willing to explore, and had more time to engage with the course content, and thus required for the course to be clear, well thought out, and fulfilling. However, both user types found the free video-based content to be positive and enjoyable and found the content to be valuable to their career development. This information was used to develop personas (Figure 2).
We summarized the results of our quantitative and qualitative data within an analysis which identified key areas of strength, weakness, opportunity, and threat (SWOT) of our program, which helps define a well thought out plan for the next iteration of course design (Supplemental Figure 6). The strengths of the program were our institutional reputation and excellent content created and led by experts in the field. Our weaknesses were in two major areas: content depth and the course platform. The depth and amount of content was often a challenge as it required a more advanced background in chemistry. The amount of education activities also seemed overwhelming for our target audience. As a result, some courses need clearer expectations and requirements for completion. Users identified that the threats to completing the course included how difficult it was to access content as the course has a difficult login process and ideally that the videos would be accessible outside of the course. Finally, opportunities to resolve these conflicts would be to create an online delivery platform that only accentuated our video-based content which is an opportunity to clarify and simplify the course requirements. Two additional opportunities were the addition of a badging and certificate program and opportunities to develop an online communities and networks for participants.
Based on our user interviews, we identified three key features for the UX-based redesign of the online courses (Figure 3). These three features included having the registration and course on same site, having clearer expectation for learners to receive credit for the course, and increased accessibility to videos. Thus, the best design to meet these needs would be to build an app-based platform for the course, a more accessible mechanism which also offers content delivery on several devices. The platform would host more videos with less focus on activities to hold up time. We will also introduce micro-credentials, or badges that recognize learning achievements, providing more streamlined course requirements and expectations, and giving an opportunity to network outside of the course.
DEVELOP
For the new courses, we developed three mobile app prototypes (low-fidelity, mid-fidelity, and high- fidelity), with each fidelity levels of the prototypes denoting their level of detail and closeness to a final product. We performed usability testing on our mid-fidelity and high-fidelity prototypes using Figma, a flexible software tool used by design practitioners for application interface design and prototyping53,54. The low-level prototype offered a simple user interface based on a series of continuous pages starting from an introductory homepage, with directed links to course pages where one could find all videos, quizzes, and additional links to resources for more information and all flowing back to the home page (Supplemental Figure 7). The mid-level prototype offered a more complex setup, with a homepage that offered more functionality and personalization for users to customize their experience (e.g., creating a list of videos to watch) (Supplemental Figure 8). The high-fidelity prototype included much more functionality, a search, information about the instructors, and more, making it easier for our network to hear lectures, earn badges and join the online community (Supplemental Figure 9).
To perform usability testing, we recruited users to test the platform and performed interviews during their experience. Our testers (advanced undergraduates, experts in biomedical sciences, and Vanderbilt faculty and staff), included people both familiar (n=2) and unfamiliar with the LMS version of the course. We evaluated testers' perceptions of the prototypes in terms of interface layout, features, ease of use and navigation, course structure and setup, and digestibility.
The testers had an overall positive experience of the mid-fidelity prototype, especially its ease of use and navigation (Figure 4). One tester said, “I really like it; it reminds me of other online video learning platforms”. This suggests that the platform is approachable and easy to navigate. However, there was some confusion about specific features, such as the layout of the home page and how to find content. We also identified the need for a results page after taking quizzes, which impacted how users thought about content presentation and packaging. Finally, although testers were excited by the ease of use on mobile and tablet devices, many wanted a web-based platform as an additional option, as it was a more familiar interface, which raised course setup concerns. During our second usability testing with the high-fidelity prototype, we gained deeper insights into the expectations and behaviors of users. overall, users found a positive experience in most of the thematic areas defined. The high-fidelity prototype received improved responses in the feature and layout design, and content available/missing code, indicating that the prototype was of a more acceptable format that met testers’ intuitions. As the user experience improved, users began to prioritize the course content, allowing us to identify areas to improve instructionally. We identified a few significant problematic themes about the initial landing page that the user engages with (home pages vs. profile), the quiz questions format, the similarity between the visual elements of the videos, and user expectations about starting a course. The testers began commenting more on text and aesthetic personal preferences as well.
DELIVER
Finally, in the functional product of the new course, called Ph.ideo, we created a website using Wix which was also available in app form. The website has a home page providing intuitive access to different user experiences based on the outcomes and interactions a user desires with the program including earning a badge, earning a certificate, or in-person workshops. The website also includes an about page with relevant overview information, an FAQ page, a link to all the course modules, and external links to the full online site. At the bottom of the page are a subscription form for a newsletter and images with links to all videos. The functional model focuses on ease of use, visual appeal, and concise and clear text content (Supplemental Video 1).
Finally, we launched the courses in May 2022 and collected quantitative data about course participation and completion. To pilot Ph.ideo, we offered a free one-month period for past participants. We then emailed 230 people of our previous newsletter subscribers to try out the new course. From there, out of 76 registrants for the course, we found a 45.45% completion rate, a major improvement from initial completion rates of 0.05% on the LMS platform. In addition, students were also interested in earning badges, which were specific to each module. 73% of registrants for the course earned badges, meaning they completed one or more modules without necessarily completing the entire course. This suggests that badges are a more intuitive approach to an asynchronous do-it-your-way style online MOOC where students are not beholden to completing the entire course, just sections they feel are relevant to them (Table 2).
DISCUSSION
Through UX Design approaches, including interviews, persona building, and prototyping, we transitioned an online program from a learning management system to a simple video-based web application that was more suited for external participant needs. Employing UX Design evidence-based tools increased retention and completion rates for an audience of external users and students looking to learn or improve their foundational concepts in medicinal chemistry and pharmacology for the pharmaceutical industry. As the material for this subject matter is already challenging, we aimed to lower the activation energy needed to navigate the online learning systems and processes. We found that reducing these barriers led to increased engagement with the drug discovery content.
We were interested in developing non-credit online courses and resources for working professionals to advance their careers. We also included early career scientists (potentially masters, doctoral, and postdoctoral students) interested in learning introductory drug discovery and development concepts. In our discovery process, we found through a comparative analysis that we might better meet our intended audience expectations if it had more qualities typically associated with MOOCs (i.e., ease of use, accessibility, asynchronous). Yet, our product must also leverage the most appreciated features within a traditional LMS system (content management, interactions, community). Through user interviews, we were able to identify two critical weaknesses in our LMS course, which included the content accessibility and the usability of the course platform. While future efforts could increase the accessibility of the course material, here we focused on developing a digital course platform that could better fit user behavior. With various iterations of prototyping, we delivered a functional online course prototype that improved the completion rate from 0.05% to 45.45% by redesigning the system based on user needs.
Our process leads us to focus on three key features. First, we wanted registration and course content to be located on the same site or, at a minimum, not require participants to migrate between two systems. Password and user account exploitation is considered one of the critical issues in network security management 55–58. Previously two systems were used to register the student and to host content. Unfortunately, integrating these two systems created many negative experiences for users. We combined our registration and content hosting through a web-based application. Second, we reduced the participant's tasks and clarified expectations. Others have found that reducing requirements to those essential to learning objectives decreases stress and anxiety while not significantly impacting other learning or performance measures in traditional learning environments 59. When conducting the redesign, consideration of the assessment of learning outcomes lead to explicit assessment mechanisms that reduced redundancy and requirement clarity. Starting with assessment mechanisms and then building objectives was also seen in other MOOCs to instruct learners to work toward learning outcomes 60. Various learning outcomes in MOOCs have been studied, such as knowledge, skills, academic achievement, engagement, motivation, satisfaction, and perceptions of learning 61–66. Assessment tasks throughout the course differ in difficulty and complexity, which could align with different levels of learner motivation. Particularly with an external audience, learners' motivations, such as course complement, professional needs, satisfying curiosity, and connecting with people, their learning needs might differ as well 67. Examples of different learner groups are active learners who aim to complete and pass the course and lurkers who do not involve themselves in learning or complete any assessment tasks 68. Measurement with the same standard within a course might not satisfy all learners' learning needs. One way could be to apply the assessments in each learner group but differentiated in difficulty and complexity. In a word, differentiated learning assessments might align with learners' motivation levels and lead all of them to fulfill learning goals. Finally, we moved all video content to Vimeo, which increased the accessibility of the video outside of course content. Rapid globalization, along with the growing trend of openness and sharing, has created a standard for the accessibility of information through digital platforms 69. The potential of technology to enhance learning has been well established 70,71. Particularly for external audiences hosting a video on platforms like YouTube, Vimeo, and DailyMotion increases their broader accessibility. Therefore, it is reasonable to use these platforms to foster new ways of informal teaching and learning 72. These video platforms offer a tremendous opportunity at an affordable cost to learn long sequences (e.g., a 20-part video series on learning to play the dulcimer) as well as short knowledge insights (e.g., how to clean a clogged drain) and ongoing routine activities (e.g., exercise classes). The 2021 Pew study of adult Americans found that 81% are YouTube users 73, of whom 86% found YouTube videos useful for informal learning 74. Additionally, resources exist to help educators with low-cost, high-impact video production 75.
This study has a few limitations, including measuring knowledge transfer and factoring in the digital literacy of our target populations. Future work must more thoroughly evaluate whether learning outcomes were equivalent in these two systems. Learning outcomes describe the measurable skills, abilities, knowledge, or values that students should be able to demonstrate as a result of completing a course. They are student-centered rather than teacher-centered in describing what the students will do, not what the instructor will teach. We created learning objectives for the courses, along with activities that evaluated these objectives. Based on user feedback, we cut all redundant learning outcomes measuring activities to the bare minimum. Reduced requirements decreased the time and energy necessary for the participant. Since progression through the course required completing objective-measuring activities, we used completion rates to indicate learning outcomes. Additional studies would be required to test if these changes lead to reduced knowledge transfer more generally. Additionally, in future studies, we should assess the digital literacy of our audience or compare the digital literacy between personas. Digital literacy refers to an individual's ability to find, evaluate, and communicate information through typing and other media on various digital platforms. A better understanding of digital literacy in our population could explain why specific design decisions lead to higher engagement and completion rates. Additional studies would be required to assess digital literacy specifically.
Our results emphasize the need for applying UX Design principles, particularly for external learners. Informal education efforts that seek to engage a broader, non-academic audience have alternative motivating factors for learners that impact their engagement. Our findings echo previous work contending that universities should utilize UX Design when creating broad-based learning systems online. However, our work provides an example of applying this approach and demonstrated benefits. Reimagining the design of education content for an external audience requires interactivity, motivational incentives, and presentation considerations. In a sense, UX Design blends communication with pedagogical practice to improve the experience for learners. Others can use our process to model improvements for other digital resources within the sciences and beyond.
REFERENCES
1. Grund J. The societal value of pharmaceuticals. Balancing industrial and healthcare policy. Pharmacoeconomics [Internet]. 1996 Jul;10(1):14–22. Available from: http://dx.doi.org/10.2165/00019053-199610010-00003 PMID: 10160467
2. Kumar V, Sundarraj RP. The Societal Value of Pharmaceutical Innovation. In: Kumar V, Sundarraj RP, editors. Global Innovation and Economic Value [Internet]. New Delhi: Springer India; 2018. p. 147–187. Available from: https://doi.org/10.1007/978-81-322-3760-0_4
3. Rotella DP. The Critical Role of Organic Chemistry in Drug Discovery. ACS Chem Neurosci [Internet]. 2016 Oct 19;7(10):1315–1316. Available from: http://dx.doi.org/10.1021/acschemneuro.6b00280 PMID: 27623164
4. Lolli M, Narramore S, Fishwick CWG, Pors K. Refining the chemical toolbox to be fit for educational and practical purpose for drug discovery in the 21st Century. Drug Discov Today [Internet]. 2015 Aug;20(8):1018–1026. Available from: http://dx.doi.org/10.1016/j.drudis.2015.04.010 PMID: 25959506
5. Nussbaumer P. Medicinal Chemists of the 21(st) Century--Who Are We and Where to Go? ChemMedChem [Internet]. 2015 Jul;10(7):1133–1139. Available from: http://dx.doi.org/10.1002/cmdc.201500133 PMID: 25924977
6. Parvin MM, Kabir MMN. Factors affecting employee job satisfaction of pharmaceutical sector. Australian Journal of Business and Management Research [Internet]. New South Wales Research Centre Australia (NSWRCA); 2012 Apr 5;01(09):113–123. Available from: https://www.academia.edu/download/51866700/ajbmr-19-23i1n9a13a.pdf
7. Parkhurst M. The role of the health care sector in expanding economic opportunity [Internet]. issuelab.org; 2007 [cited 2023 Feb 14]. Available from: https://www.issuelab.org/resources/1707/1707.pdf
8. Dong M. The Pharmaceutical Industry and the Separation Scientist: Perspectives, Trends, and Career Opportunities. LCGC North Am [Internet]. chromatographyonline.com; 2022; Available from: https://www.chromatographyonline.com/view/the-pharmaceutical-industry-and-the-separation-scientist-perspectives-trends-and-career-opportunities
9. Research and Development in the Pharmaceutical Industry [Internet]. Congressional Budget Office. 2021 [cited 2022 Nov 28]. Available from: https://www.cbo.gov/publication/57126
10. Stanley AG, Jackson D, Barnett DB. The teaching of drug development to medical students: collaboration between the pharmaceutical industry and medical school. Br J Clin Pharmacol [Internet]. 2005 Apr;59(4):464–474. Available from: http://dx.doi.org/10.1111/j.1365-2125.2005.02374.x PMCID: PMC1884803
11. Tham JCK. Audience, User, Producer: MOOCs as Activity Systems. Handbook of Research on Writing and Composing in the Age of MOOCs [Internet]. IGI Global; 2017 [cited 2023 Feb 14]. p. 254–277. Available from: https://www.igi-global.com/chapter/audience-user-producer/172591
12. Wan HT, Hsu KY. An innovative approach for pharmacists’ continue education: Massive open online courses, a lesson learnt. Ind J Pharm Educ [Internet]. EManuscript Services; 2016 Jan 1;50(1):103–108. Available from: http://dx.doi.org/10.5530/ijper.50.1.13
13. Liyanagunawardena TR, Williams SA. Massive open online courses on health and medicine: review. J Med Internet Res [Internet]. 2014 Aug 14;16(8):e191. Available from: http://dx.doi.org/10.2196/jmir.3439 PMCID: PMC4155756
14. Hew KF, Cheung WS. Students’ and instructors’ use of massive open online courses (MOOCs): Motivations and challenges. Educational Research Review [Internet]. 2014 Jun 1;12:45–58. Available from: https://www.sciencedirect.com/science/article/pii/S1747938X14000128
15. Terras MM, Ramsay J. Massive open online courses (MOOCs): Insights and challenges from a psychological perspective. Br J Educ Technol [Internet]. Wiley; 2015 May;46(3):472–487. Available from: https://onlinelibrary.wiley.com/doi/10.1111/bjet.12274
16. Kop R. The challenges to connectivist learning on open online networks: Learning experiences during a massive open online course. Int Rev Res Open Distrib Learn [Internet]. Athabasca University Press; 2011 Mar 25;12(3):19. Available from: https://id.erudit.org/iderudit/1067613ar
17. Royal Society of Chemistry. Public attitudes to chemistry [Internet]. Available from: https://www.rsc.org/globalassets/04-campaigning-outreach/campaigning/public-attitudes-to-chemistry/public-attitudes-to-chemistry-research-report.pdf
18. Babinčáková M, Bernard P. Online Experimentation during COVID-19 Secondary School Closures: Teaching Methods and Student Perceptions. J Chem Educ [Internet]. 2020 Sep 8;97(9):3295–3300. Available from: http://dx.doi.org/10.1021/acs.jchemed.0c00748 PMCID: PMC7482317
19. Tawfik AA, Reeves TD, Stich AE, Gill A, Hong C, McDade J, Pillutla VS, Zhou X, Giabbanelli PJ. The nature and level of learner–learner interaction in a chemistry massive open online course (MOOC). Journal of Computing in Higher Education [Internet]. 2017 Dec 1;29(3):411–431. Available from: https://doi.org/10.1007/s12528-017-9135-3
20. Noble AR. Developing General Chemistry II Online: Online Approaches to Chemical Education [Internet]. American Chemical Society; 2017. p. 71–80. Available from: https://doi.org/10.1021/bk-2017-1261.ch006
21. Minichiello A, Hood JR, Harkness DS. Bringing User Experience Design to Bear on STEM Education: A Narrative Literature Review. Journal for STEM Education Research [Internet]. 2018 Dec 1;1(1):7–33. Available from: https://doi.org/10.1007/s41979-018-0005-3
22. Daniels P. Course Management Systems and Implications for Practice. 2009;7(2):97–108. Available from: https://www.proquest.com/openview/b1d565f7a7a3aac46b485441df276d9b/1?pq-origsite=gscholar&cbl=60410
23. McDaniel R, Fanfarelli JR, Lindgren R. Creative Content Management: Importance, Novelty, and Affect as Design Heuristics for Learning Management Systems. IEEE Trans Prof Commun [Internet]. 2017 Jun;60(2):183–200. Available from: http://dx.doi.org/10.1109/TPC.2017.2656578
24. Picciano AG. The evolution of big data and learning analytics in American higher education. Online Learn [Internet]. The Online Learning Consortium; 2012 Jun 18 [cited 2023 Feb 14];16(3). Available from: https://olj.onlinelearningconsortium.org/index.php/olj/article/view/267
25. Saade R, Huang Q. Prototyping a learning management system for higher education. Proceedings of the 2008 InSITE Conference [Internet]. Informing Science Institute; 2008 [cited 2023 Feb 14]. Available from: https://www.informingscience.org/Publications/3213
26. Pal D, Vanijja V. Perceived usability evaluation of Microsoft Teams as an online learning platform during COVID-19 using system usability scale and technology acceptance model in India. Child Youth Serv Rev [Internet]. 2020 Dec;119:105535. Available from: http://dx.doi.org/10.1016/j.childyouth.2020.105535 PMCID: PMC7527281
27. Alzahrani L, Seth KP. Factors influencing students’ satisfaction with continuous use of learning management systems during the COVID-19 pandemic: An empirical study. Educ Inf Technol (Dordr) [Internet]. 2021 Apr 6;26(6):6787–6805. Available from: http://dx.doi.org/10.1007/s10639-021-10492-5 PMCID: PMC8023780
28. Chen T, Peng L, Jing B, Wu C, Yang J, Cong G. The Impact of the COVID-19 Pandemic on User Experience with Online Education Platforms in China. Sustain Sci Pract Policy [Internet]. Multidisciplinary Digital Publishing Institute; 2020 Sep 7 [cited 2023 Feb 14];12(18):7329. Available from: https://www.mdpi.com/2071-1050/12/18/7329
29. Boling E, Gray CM. Instructional Design and User Experience Design: Values and Perspectives Examined Through Artifact Analysis. In: Hokanson B, Exter M, Grincewicz A, Schmidt M, Tawfik AA, editors. Intersections Across Disciplines: Interdisciplinarity and learning [Internet]. Cham: Springer International Publishing; 2021. p. 93–107. Available from: https://doi.org/10.1007/978-3-030-53875-0_8
30. Parrish PE. Aesthetic principles for instructional design. Educ Technol Res Dev [Internet]. Springer Science and Business Media LLC; 2009 Aug;57(4):511–528. Available from: https://idp.springer.com/authorize/casa?redirect_uri=https://link.springer.com/article/10.1007/s11423-007-9060-7&casa_token=LZBmyRDHkQYAAAAA:OG8xBDQYhd4ZfF8gXkiyR5tQkHVhCjVkXCMsKXlPOZ5gctvk1DrpX1Zr8nq0_yMdeWSQbiWfKskFqADT_g
31. Gray CM, Parsons P, Toombs AL, Rasche N, Vorvoreanu M. Designing an aesthetic learner experience: UX, instructional design, and design pedagogy. Int J Des Learn [Internet]. IUScholarWorks; 2019 Dec 9 [cited 2023 Mar 1];11(1):41–58. Available from: http://files.eric.ed.gov/fulltext/EJ1258797.pdf
32. Gagne RM, Wager WW, Golas KC, Keller JM, Russell JD. Principles of instructional design, 5th edition. Perform Improv Advis [Internet]. Wiley; 2005 Feb;44(2):44–46. Available from: https://onlinelibrary.wiley.com/doi/10.1002/pfi.4140440211
33. Kahraman ZEH. Using user-centered design approach in course design. Procedia - Social and Behavioral Sciences [Internet]. 2010 Jan 1;2(2):2071–2076. Available from: https://www.sciencedirect.com/science/article/pii/S187704281000323X
34. Hawkins A, Graham CR, Sudweeks RR, Barbour MK. Academic performance, course completion rates, and student perception of the quality and frequency of interaction in a virtual high school. Distance Education [Internet]. Routledge; 2013 May 1;34(1):64–83. Available from: https://doi.org/10.1080/01587919.2013.770430
35. Machado M, Tao E. Blackboard vs. moodle: Comparing user experience of learning management systems. 2007 37th Annual Frontiers In Education Conference - Global Engineering: Knowledge Without Borders, Opportunities Without Passports [Internet]. 2007. p. S4J–7–S4J–12. Available from: http://dx.doi.org/10.1109/FIE.2007.4417910
36. Nash Supervisor RD. Course completion rates among distance learners: Identifying possible methods to improve retention [Internet]. [cited 2023 Feb 15]. Available from: https://citeseerx.ist.psu.edu/document?repid=rep1&type=pdf&doi=c20f87408bd04280d816f8062f716cf9611634f8
37. MOOC completion rates [Internet]. [cited 2023 Feb 15]. Available from: http://www.katyjordan.com/MOOCproject.html
38. Craig Petersen Director H. NATIONAL SURVEY OF STUDENT ENGAGEMENT (NSSE) [Internet]. [cited 2023 Feb 15]. Available from: https://www.usu.edu/aaa/pdf/survey/National%20Survey%20of%20Student%20Engagement%20-%202008.pdf
39. Moore JC, Fetzner MJ. The road to retention: A closer look at institutions that achieve high course completion rates. Online Learn [Internet]. The Online Learning Consortium; 2019 Feb 8;13(3). Available from: https://eric.ed.gov/?id=EJ862352
40. Aljeraiwi AA, Sawaftah WA. The quality of blended learning based on the use of blackboard in teaching physics at king Saud university: Students’ perceptions. J Educ Psychol Sci [Internet]. Deanship of Scientific Research; 2018 Jun 1 [cited 2023 Feb 14];19(02):616–646. Available from: https://journal.uob.edu.bh/handle/123456789/3200
41. Bouchrika I. 51 LMS Statistics: 2023 Data, Trends & Predictions [Internet]. Research.com. 2020 [cited 2023 Feb 15]. Available from: https://research.com/education/lms-statistics
42. Oliver B, Tucker B, Gupta R, Yeo S. eVALUate: an evaluation instrument for measuring students’ perceptions of their engagement and learning outcomes. Assessment & Evaluation in Higher Education [Internet]. Routledge; 2008 Dec 1;33(6):619–630. Available from: https://doi.org/10.1080/02602930701773034
43. Boulton CA, Kent C, Williams HTP. Virtual learning environment engagement and learning outcomes at a “bricks-and-mortar” university. Comput Educ [Internet]. 2018 Nov 1;126:129–142. Available from: https://www.sciencedirect.com/science/article/pii/S036013151830174X
44. Bae Y, Han S. Academic Engagement and Learning Outcomes of the Student Experience in the Research University: Construct Validation of the Instrument. Educational Sciences: Theory and Practice [Internet]. Educational Consultancy, Ltd (EDAM). Kisikli Mh. Alemdag Cd. Yan Yol Sk., SBK Is Merkezi No:5 Kat:1, Uskudar-Istanbul, 34692 Turkey. Tel: +90-216-481-30-23; Fax: +90-216-481-31-36; e-mail: editor@edam.com.tr; Web site: http://www.estp.com.tr/; 2019 Jul [cited 2023 Mar 1];19(3):49–64. Available from: http://files.eric.ed.gov/fulltext/EJ1232932.pdf
45. Rozmi ANA. The Perception of ICT Adoption in Small Medium Enterprise: A SWOT Analysis. IJIBS [Internet]. ijibs.utm.my; 2018 [cited 2023 Mar 2];9(1). Available from: https://ijibs.utm.my/index.php/ijibs/article/view/71
46. Benzaghta MA, Elwalda A, Mousa MM, Erkan I, Rahman M. SWOT analysis applications: An integrative literature review. Journal of Global Business Insights [Internet]. digitalcommons.usf.edu; 2021 [cited 2023 Mar 2];6(1):55–73. Available from: https://digitalcommons.usf.edu/globe/vol6/iss1/5/
47. Wu Y. The marketing strategies of IKEA in China using tools of PESTEL, five forces model and SWOT analysis. Proceedings of the International Academic Conference on Frontiers in Social Sciences and Management Innovation (IAFSM 2019) [Internet]. Paris, France: Atlantis Press; 2020 [cited 2023 Mar 2]. Available from: https://www.atlantis-press.com/article/125934515
48. Aldehayyat JS, Anchor JR. Strategic planning tools and techniques in Jordan: awareness and use. Strateg Change [Internet]. Wiley; 2008;17(7-8):281–293. Available from: https://onlinelibrary.wiley.com/doi/10.1002/jsc.833
49. Fleisher CS, Bensoussan BE. Strategic and competitive analysis: methods and techniques for analyzing business competition [Internet]. academia.edu; 2003 [cited 2023 Mar 2]. Available from: https://www.academia.edu/download/30415465/http___www.mindshifts.com.au_downloads_strategic_competitive.pdf
50. Lee KL, Lin SC. A fuzzy quantified SWOT procedure for environmental evaluation of an international distribution center. Inf Sci [Internet]. Elsevier; 2008 Jan 15;178(2):531–549. Available from: https://www.sciencedirect.com/science/article/pii/S0020025507004161
51. Shrestha RK, Alavalapati JRR, Kalmbacher RS. Exploring the potential for silvopasture adoption in south-central Florida: an application of SWOT–AHP method. Agric Syst [Internet]. Elsevier; 2004 Sep 1;81(3):185–199. Available from: https://www.sciencedirect.com/science/article/pii/S0308521X03001963
52. Hassan A. The value proposition concept in marketing: How customers perceive the value delivered by firms– A study of customer perspectives on supermarkets in Southampton in the United Kingdom. Int J Mark Stud [Internet]. Canadian Center of Science and Education; 2012 May 29 [cited 2023 Mar 2];4(3). Available from: http://www.ccsenet.org/journal/index.php/ijms/article/view/17696
53. Sharma V, Tiwari AK. A Study on User Interface and User Experience Designs and its Tools. World Journal of Research and Review (WJRR) [Internet]. wjrr.org; 2021;12(6). Available from: https://www.wjrr.org/download_data/WJRR1206016.pdf
54. Wang J, Xu Z, Wang X, Lu J. A Comparative Research on Usability and User Experience of User Interface Design Software. International Journal of Advanced Computer Science and Applications; West Yorkshire [Internet]. West Yorkshire, United Kingdom, West Yorkshire: Science and Information (SAI) Organization Limited; 2022;13(8). Available from: http://proxy.library.vanderbilt.edu/login?url=https://www.proquest.com/scholarly-journals/comparative-research-on-usability-user-experience/docview/2718752307/se-2
55. Zviran M, Haga WJ. A Comparison of Password Techniques for Multilevel Authentication Mechanisms. Comput J [Internet]. Oxford Academic; 1993 Jan 1 [cited 2023 Mar 8];36(3):227–237. Available from: https://academic.oup.com/comjnl/article-abstract/36/3/227/311547
56. Zviran M, Haga WJ. Password Security: An Empirical Study. Journal of Management Information Systems [Internet]. Routledge; 1999 Mar 1;15(4):161–185. Available from: https://doi.org/10.1080/07421222.1999.11518226
57. Zviran M, Haga WJ. Cognitive passwords: The key to easy access control. Comput Secur [Internet]. Elsevier; 1990 Dec 1;9(8):723–736. Available from: https://www.sciencedirect.com/science/article/pii/016740489090115A
58. Gong L, Lomas MA, Needham RM, Saltzer JH. Protecting poorly chosen secrets from guessing attacks. IEEE J Sel Areas Commun [Internet]. 1993 Jun;11(5):648–656. Available from: http://dx.doi.org/10.1109/49.223865
59. Hicks JL, Dale RA, Hendricson WD, Lauer WR. Effects of reducing senior clinical requirements. J Dent Educ [Internet]. Wiley; 1985 Mar;49(3):169–175. Available from: https://pubmed.ncbi.nlm.nih.gov/3855908/ PMID: 3855908
60. Wei X, Saab N, Admiraal W. Assessment of cognitive, behavioral, and affective learning outcomes in massive open online courses: A systematic literature review. Comput Educ [Internet]. 2021 Apr 1;163:104097. Available from: https://www.sciencedirect.com/science/article/pii/S0360131520302955
61. Barak M, Watted A, Haick H. Motivation to learn in massive open online courses: Examining aspects of language and social engagement. Comput Educ [Internet]. Elsevier; 2016 Mar 1;94:49–60. Available from: https://www.sciencedirect.com/science/article/pii/S0360131515300828
62. Deng R, Benckendorff P, Gannaway D. Linking learner factors, teaching context, and engagement patterns with MOOC learning outcomes. J Comput Assist Learn [Internet]. Wiley; 2020 Oct;36(5):688–708. Available from: https://onlinelibrary.wiley.com/doi/10.1111/jcal.12437
63. Jung E, Kim D, Yoon M, Park S, Oakley B. The influence of instructional design on learner control, sense of achievement, and perceived effectiveness in a supersize MOOC course. Comput Educ [Internet]. Elsevier; 2019 Jan 1;128:377–388. Available from: https://www.sciencedirect.com/science/article/pii/S0360131518302720
64. Jung I, Lee J. The effects of learner factors on MOOC learning outcomes and their pathways. Innovations in Education and Teaching International [Internet]. Routledge; 2020 Sep 2;57(5):565–576. Available from: https://doi.org/10.1080/14703297.2019.1628800
65. Li Q, Baker R. The different relationships between engagement and outcomes across participant subgroups in Massive Open Online Courses. Comput Educ [Internet]. Elsevier; 2018 Dec 1;127:41–65. Available from: https://www.sciencedirect.com/science/article/pii/S0360131518302094
66. Shapiro HB, Lee CH, Wyman Roth NE, Li K, Çetinkaya-Rundel M, Canelas DA. Understanding the massive open online course (MOOC) student experience: An examination of attitudes, motivations, and barriers. Comput Educ [Internet]. Elsevier; 2017 Jul 1;110:35–50. Available from: https://www.sciencedirect.com/science/article/pii/S0360131517300519
67. Zheng S, Rosson MB, Shih PC, Carroll JM. Understanding Student Motivation, Behaviors and Perceptions in MOOCs. Proceedings of the 18th ACM Conference on Computer Supported Cooperative Work & Social Computing [Internet]. New York, NY, USA: Association for Computing Machinery; 2015 [cited 2023 Mar 8]. p. 1882–1895. Available from: https://doi.org/10.1145/2675133.2675217
68. Alario-Hoyos C, Perez-Sanagustin M, Delgado-Kloos C, Parada G HA, Munoz-Organero M. Delving into participants’ profiles and use of social tools in MOOCs. IEEE Trans Learn Technol [Internet]. Institute of Electrical and Electronics Engineers (IEEE); 2014 Jul;7(3):260–266. Available from: https://ieeexplore.ieee.org/abstract/document/6766737/
69. Cohen A, Soffer T, Henderson M. Students’ use of technology and their perceptions of its usefulness in higher education: International comparison. J Comput Assist Learn [Internet]. Wiley; 2022 Oct;38(5):1321–1331. Available from: https://onlinelibrary.wiley.com/doi/10.1111/jcal.12678
70. Soffer T, Kahan T, Livne E. E-assessment of online academic courses via students’ activities and perceptions. Studies in Educational Evaluation [Internet]. Elsevier; 2017 Sep 1;54:83–93. Available from: https://www.sciencedirect.com/science/article/pii/S0191491X16300190
71. Soffer T, Cohen A. Implementation of Tel Aviv University MOOCs in academic curriculum: A pilot study. International Review of Research in Open and Distributed Learning [Internet]. Érudit; 2015;16(1):80–97. Available from: https://www.erudit.org/en/journals/irrodl/1900-v1-n1-irrodl04978/1065929ar/abstract/
72. Marone V, Rodriguez RC. “What’s So Awesome with YouTube”: Learning Music with Social Media Celebrities. ONLINE J COMMUN MEDIA TECHNOL [Internet]. Bastas; 2019 Oct 11 [cited 2023 Mar 8];9(4):e201928. Available from: https://www.ojcmt.net/download/whats-so-awesome-with-youtube-learning-music-with-social-media-celebrities-5955.pdf
73. Auxier B, Anderson M. Social Media Use in 2021 [Internet]. pewresearch.org; 2021 [cited 2023 Mar 8]. Available from: https://www.pewresearch.org/internet/wp-content/uploads/sites/9/2021/04/PI_2021.04.07_Social-Media-Use_FINAL.pdf
74. Smith A, Toor S, Van Kessel P. Many turn to YouTube for children’s content, news, how-to lessons. Pew Research Center [Internet]. internet.psych.wisc.edu; 2018;7. Available from: https://internet.psych.wisc.edu/wp-content/uploads/532-Master/532-UnitPages/Unit-02/Smith_Pew_2018.pdf
75. Castillo S, Calvitti K, Shoup J, Rice M, Lubbock H, Oliver KH. Production Processes for Creating Educational Videos. CBE Life Sci Educ [Internet]. Am Soc Cell Biol; 2021 Jun;20(2):es7. Available from: http://dx.doi.org/10.1187/cbe.20-06-0120 PMCID: PMC8734383
76. Earnshaw, Y., Tawfik, A. A., & Schmidt, M. (2018). User Experience Design. In R. E. West (Eds.), Foundations of Learning and Instructional Design Technology. BYU Open Textbook Network. https://open.byu.edu/lidtfoundations/user_experience_design