Abstract

Indeed is the largest job site in the world with over 250 million unique visitors every month, Users at this quantity come with differences that rise the question of their experience and Indeed effectiveness to deliver its services to the population. Attention is being made to identifying issues of usability during the account creation of the Indeed website platform of new users without a college degree. The study consists of live testing of users who were at least 18 years of age, without a college degree, actively looking for a job and preferably not having experience with Indeed’s website. Retrospective Think Aloud (RTA) was the primary method encouraged with test participants to allow an opened perspective of website interaction experience through the overall account creation and resume uploading experience.
Keywords: Abstract, RTA

Executive Summary

The Fantastic Four Enterprise (FFE) project team conducted a remote usability test between September 2020 to October 2020. The purpose of this test was to identify usability issues and opportunities for improvement in the account creation, resume uploading and editing experience on Indeed.com for job seekers without a college degree, that were a minimum age of 18, actively looking for a job online and did not have an active Indeed account. The study consisted of 6 participants with live testing sessions lasting from 15 minutes to 60 minutes in length of time.
The findings of our study provided insights to the navigation as one of the major components of concern for Indeed for its users all over the world with varying demographics and levels of education. Overall, the navigation was found to be easy to navigate concluded from 5 out of 6 participants. Along the process of navigating the sight a measurement of task completion was collected finding 5 out of 6 participants were able to complete them successfully.

Research Purpose

The purpose of this study is to identify usability issues and opportunities for improvement in the account creation, resume uploading, and editing experience on Indeed for job seekers without a college degree. Through this study we are seeking to:

• Identify obstacles users may encounter when creating an account on Indeed.com
• Identify obstacles users may encounter when uploading their resume on Indeed.com
• Identify the usability of Indeed.com for job seekers when they edit their resume after uploading
• Understand how the users are interacting with the Indeed’s website, how they feel during the process if there are certain expectations for specific features while performing the three tasks
• Identify technical issues and errors in the website if problems occur
• Evaluate the user's behavior using the website how they are trying to proceed with their tasks, observe their problem-solving approach
• Identify their feelings frustrations, observe for their pain points as well as things they find helpful.
• Discuss their opinions about the parts they like and dislike

Research Questions

In addition to identifying usability issues in the account creation and resume upload process, we also seek to answer these questions:

• Are users able to find the "Create an Account" button?
• Is the navigation process of uploading a resume clear to the user?
• Are there parsing errors when the user uploads their resume?
• Do parsing errors impede the user's desire to complete their resume?
• Do users find their process easy and smooth?
• Are the users able to navigate themselves easily to find their resume to edit when they come to the site for the second time?
• How quickly they can reach the information they need
• How quickly they can perform each task
• Are they able to accurately predict which section to go to find the information
• How relevant their indeed resume comparing to their original uploaded version

Context/Background

Indeed is the #1 job site in the world with over 250 million unique visitors every month. Indeed strives to put job seekers first, giving them free access to search for jobs, post resumes, and research companies.

Indeed’s mission is to help people get jobs. One of the ways Indeed helps people get jobs is by connecting them to new opportunities using their Indeed resume. By having a resume in their Indeed account, job seekers can apply directly to certain jobs, or they can allow their resumes to be discovered by employers on Indeed

Our team wanted to understand the job search process specifically for job seekers without a college degree. We classified this group as: Participants graduated from high school continuing their study in college or two years association degree program

Participants graduated from high school who are currently not enrolled in any degree program Participants that do not have plans to enroll in a college degree program and prefer to continue their career presently

Our observation focus on the usability study is to understand the user’s behavioral patterns. We try to empathize with their feelings and viewpoints to find the answer to our questions.

Methodology

We team used remote testing resources, recording the live interaction of participants with minimal instruction electing qualitative feedback collection. Participants were recruited through each member’s personal network including friends, family, workplace, and close connections that fit our screening criteria. Those who were elected from our screening process were then scheduled for a test session.

We conducted a Qualitative Usability test, that was moderated remotely. Participants performed a series of tasks by screen sharing via Zoom that was promoted by the moderator and observer. Each session was recorded. In the multiple group meetings after the testing sessions, we watched the recordings discussed our findings. We used Miro collaborative board to create an affinity diagram. We iterated our findings created customer journey maps for each user. We used spreadsheets to calculate and analyze our findings. In the spreadsheets, we sectioned our findings, transacted information from our collaborative board to the spreadsheet to find the frequencies of each iterated statement in each sheet for the 6 users. In another word, the percentages of our findings were calculated based on the frequency of each statement.

Session

The test sessions lasted between 25 minutes and 45 minutes. Each session included a moderator to facilitate and an observer to observe and record notes. The pre-test arrangements were completed before the usability testing. During the first 10 minutes, the moderator briefly went over the consent form, Usability Study Questionnaire, introductions, and purpose of the study.

After introductions, the moderator prompted users to complete a series of tasks and asked probing questions necessary to collect more information from the participant. After testing, the moderator will use the last 10 minutes to debrief with the participant using post-test questions.

Participants

For our testing session, all participants were recruited from our immediate networks (family, friends, neighbors, workplace, et cetera) that that fit our screening criteria. Recruitment was done via in-person, phone, electronic mail, and social media. Our screening criteria included:

• Job seekers without a college degree
• Must be at least 18 years old
• Does not have an Indeed.com account or an "actively" user account on the website
• Currently searching for a job, either "actively" or "casually"
• Uses the internet to seek employment opportunities

We recruited participants from the following demographics:

• Participants graduated from high school continuing their study in college or having completed an associate degree program.
• Participants graduated from high school who are currently not enrolled in any degree program
• Participants that do not have plans to enroll in a college degree program and prefer to continue their career presently

Participant Profiles

USER T - High School Graduate/GED; Age Group 18-29; Actively searching for a job

Participant Profiles

Test participants attempted completion of the following tasks in phases:

Phase One: Pre-Test Arrangements

• Fill out the Usability Screener Questionnaire • Complete the Consent form Phase Two: Introduction to the Testing Session:
• Introduction of moderator and observer
• Explain purpose of study
• Encourage participants to express any thoughts or feelings that may come during the testing process.

Phase Three: Tasks

• Log into Video Conferencing invitation via Zoom Video Communications
• Share their screen on the Video Conferencing
• Open an internet browser
• Visit Indeed.com via URL http://www.indeed.com
• Create a new account on Indeed.com
• Upload their resume (if they have one pre-made)
• Logout of their newly created Indeed.com account
• Log into their newly created Indeed.com account
• Edit their Indeed.com profile-resume

Phase Four: Post – Test Questions

• How do you feel about the website and your process?
• How was the overall experience for you?
• Can you try to think about the clarity of features how would you describe your experience in this aspect?
• What did you like/ dislike about the usage and your experience?
• What (if any) was the biggest challenge you encountered while creating an account?
• What (if any) was the biggest challenge you encountered while uploading or editing your resume?
• What do you think about the accuracy of the uploaded resume comparing to your original version?
• How do you feel about tracking your location on the website?
• How do you feel about predicting certain functions to navigate yourself easily?
• Are there any additional features you wish were included?
• What do you feel about the time frame for completing your tasks?
Phase Five: Closing
• Thank participants for their participation.
• If there are any additional questions, please electronic mail.

Summary of Key Takeaways

To find the results, our group utilized affinity diagramming to organize the qualitative data we received from testing. Through this method, we were able to determine key themes that came up during testing. In addition to an affinity diagram, we also created a user map journey for each of the six participants. The user map journey provided us with a better understanding of each users' interaction with Indeed.com and the opportunity to identify specific issues that were present in the customer experience. From this, we found:


• Overall Website Navigation
• Five out of six (83%) participants were able to navigate and understand Indeed.com.
• "Platform is easy to work and navigate" - User S


• Completion of Tasks
• Five out of six participants (83%) were able to complete the major tasks which included, creating an Indeed account, uploading their resume, and editing their resume after initially logging out.
• "The whole process was very easy" - User E


• Challenge with Completion of Tasks
• One out of the six (16%) participants found the process to completion of tasks challenging and needed additional prompting.
• Three of the six users (50%) indicated that they were expecting the "Submit" button at the end of the editing page, but they found it on the top.

Task Findings:

Creating an Account While all participants were able to eventually create an account, 50% of participants did take additional time to find the "Sign in" button. This was determined by the observation of the moderator and observer who noticed that the cursor initially hovered on the center of the page before eventually finding the function in the top right corner. Uploading Resume 83% of users were able successfully upload their resume without additional prompts from the moderator. Users from this percentile agreed that the process to upload their resume was simple and straightforward

• "It’s user friendly, tells you what to do next" - User T
Parsing Overview 67% of users were able to observe parsing errors after uploading their resume. Users noticed that their uploaded resume was divided differently than the original. Users that had a significant number of experiences on their resume had to edit more than users with less experiences on their resume.
• “They broke up my resume into many parts" - User S
The page divided one of the user's resume she needed to edit more than the majority of users. Same user was asked the question "what does headline mean? what am I supposed to put there?” User S Logout of Account All users were able to logout of their account. However, 33% of users expressed confusion in finding the button that would allow them to save their resume before logging out. These users initially scrolled to the bottom of the screen.
• “I was looking for the save and continue button” User R
• “I wasn't sure where submit button was, it rapidly changed the screen that I was not sure what happened to my resume” User E
Login to Account 83% of users were able to login without any prompting from moderators. This may be due to their familiarity with previously logging in. Editing Resume 100% of users had to make edits to their uploaded resume. 50% of users mentioned uncertainty regarding the lack of clarity with specific functions in the Indeed resume edit feature. Users in this percentage mentioned confusion regarding the definition of "Headline" and the "Skills" listed. 33% of users had portions from their resume that were not uploaded to Indeed.com.
• " What does headline mean? What am I supposed to put there?" User S
Logout 83% of users were able to logout of the account without any prompting from the moderator. One of the six users (16%) that was unable to logout without being prompted indicated the icon to logout was difficult to understand. One of the six users shared his confusion after logging out and finding that his resume remained on the screen.

Specific Findings, Implications and Recommendations

The findings and recommendations section of the report is the most detailed of all and may be represented in various ways. At UCS, we tend to use a table with three sections: observations Interpretations recommendations.

We find that the interpretation section allows the reader to understand how we arrived at our recommendation while giving him/her the freedom to agree or disagree with our perception of the problem. Regardless of the method used, it is important to give the reader unbiased data presented in a concise and easily understood format. If you are not familiar with the product you are testing, it may not be feasible to rank the problems and recommendations. Talk about these issues with the primary stakeholders to see how they want to proceed. Should include:

• information about what is and is not working
• the severity of the problems
• specific recommendations for improvement
• screen shots of application
• descriptive statistics of data collected for effectiveness and efficiency
• Satisfaction survey results

Conclusion

Most of the participants found Indeed.com to be clean and user-friendly, and easy to use. 83% of participants shared that they were able to successfully upload their resume. However, 67% of participants observed changes to their resume when parsed (with 2 participants having sections of their resume completely omitted by Indeed.com's AI).

Having a simplified resume parsing AI on a job search website is key to all of the participants. By working closer with users through implementing FFE's recommendations, this shall ensure a more user-centered web design and user experience.