top of page

Adobe Learning Manager

Enhance Learning: Improve content discovery for better engagement by leveraging AI.

001 (1).png
EMPLOYER

Adobe Experience Cloud

LOCATION

Bangalore, India

INDUSTRY

EdTech

WEBSITE
PROJECT YEAR

2023

PROJECT BRIEF

Adobe learning manager is a learning management system (LMS) by Adobe. It is mostly used by businesses to train and educate their employees or customers.

Adobe wanted ALM to help learners discover relevant learning resources by recommending them better content. In turn, it was intended to improve learners' habit of learning and increase engagement on the platform. 

TL;DR

The project starts with some guerilla research in order to understand the ecosystem of Adobe learning manager followed up by primary & secondary research. Based on the inshights, it was decided to design an MVP for the new recommendation system by improving the existing one.

The MVP: We ask learners about their interests and populate the homepage with relevant learning resources.

After release we see a substantial adoption of the intervention hence, we decide to improve the recommendation experience on mobile leveraging AI.

The enhancement: The learners can create a custom smart track for themselves by choosing a product and answering few questions. The smart track is basically a structured sequence of courses or section/sub-section of courses curated based on the learner's level of knowledge.

MY CONTRIBUTIONS
UX Research
User Persona
Journey Mapping
Scenarios Mapping
User Interviews
Academic Literature Review
User Testing
Interactive Prototyping

PROBLEM STATEMENT

Improve content discovery to improve learner's habit of learning to drive engagement

Introduction - Adobe Learning Manager

Adobe Learning Manager (ALM) is a highly customizable learning management system (LMS) that has been in use for over a decade and has established itself as a leading LMS in the industry. The system has been designed to provide organizations with a robust platform to manage and deliver online learning content to their employees & customers. ALM offers a range of features, including:

  • Course Creation and Management: ALM provides a comprehensive set of tools for creating and managing online courses. This includes the ability to create courses from scratch or import existing course content in a variety of formats, including SCORM, AICC, and xAPI.

  • Learner Tracking and Reporting: ALM provides a comprehensive tracking and reporting system that enables organizations to monitor learner progress and evaluate the effectiveness of their training programs. This includes detailed analytics and reporting capabilities that enable organizations to assess learner engagement, completion rates, and knowledge retention.

  • Assessment and Certification Management: ALM provides a comprehensive assessment and certification management system that enables organizations to design, manage, and deliver assessments and certifications. This includes a range of question types, including multiple choice, true/false, and essay questions.

  • Collaboration and Communication Tools: ALM provides a range of collaboration and communication tools that enable learners to connect and interact with their peers and instructors. This includes discussion forums, messaging, and social learning tools.

The current state of ALM is robust and provides a wide range of features and capabilities to organizations. However, there is a need to enhance the platform to meet the changing demands of the modern workforce. In particular, there is a need to leverage AIML to create a personalized learning experience for learners that enables them to generate their own learning paths based on their career goals and aspirations.

Decoding Adobe Learning Manager

Upon receiving the problem statement, my initial step involved a comprehensive exploration of the platform and its ecosystem. This endeavour aimed to figure out the scope and extent of possibilities that can be explored to address the problem. The following activities were conducted in this phase.

Understanding the platform

  • UX Audit - Aim of conducting this audit was to get familiar with the platform and identify possible pain points and usability issues.

  • User Journey Mapping - I wanted to understand how learners are interacting with the platform and their pain points in this journey.

Understanding the users

  • User InterviewsThe purpose of conducting user interviews at this stage was to get an idea of how learners perceive the platform and what are their motivations and goals wrt ALM.

Understanding the business

  • Business goals - The problem must be addressed in alignment with the company's business goals and hence I wanted to be pretty clear of what the roadmap of the product is like and how the company aims to achieve it's goals.

image.png

UX Audit

Identified & presented 17 major issues out of 43 identified to the leadership which included both experiential and usability problems.

  1. Learners are often bombarded with a high volume of emails within Adobe Learning Manager, causing information overload and hampering their focus.

  2. The lack of a unified experience between the mobile and desktop versions of Adobe Learning Manager results in inconsistencies and disjointed user interactions. 

  3. The absence of proper onboarding and guided navigation in Adobe Learning Manager creates a steep learning curve for learners, leading to confusion and frustration. 

Typical Learner's Journey

After conducting the audit, I moved on the figure out a typical learner's journey on ALM. The major goal here was to find their pain points at each step.

Table2.png

Top insights: User Interviews

A number of user interview sessions were conducted in which a total of 10 participants took part. The user interview questions were mostly qualitative and segmented around different aspects of the platform.

Following insights were derived from this activity.

  1. Learners have different content format preferences: video or written materials.

  2. Discovering relevant content on the platform is not intuitive and time-consuming.

  3. Technical issues like inconsistent internet connectivity and lack of mobile friendliness hinder the learning experience.

  4. Collaboration and feedback motivate learners to complete courses.

  5. Personalized course recommendations and platform customization are valued by learners.

Frame 2.png

In these user interview sessions, certain responses were repeatedly observed around different aspects of the platform. Some of those are mentioned below. 

Personalized Recommendations

"It would be great if the system could suggest courses that are tailored to my interests."


"I would love to see course recommendations based on my previous learning history."

Email Communication

"I'm getting too many emails from the system, it's overwhelming."


"I wish I could choose what notifications I receive via email."

Course Engagement

"I love it when courses have real-world case studies, it helps me understand how to apply what I'm learning."


"I feel more motivated to complete a course when I know I can collaborate with others and get feedback."

Mobile App Experience

"I can't find the assignment on the app, it's not very user-friendly."


"The app keeps crashing when I try to access the course materials."

Course Completion Rates

"I stopped doing the course because I didn't feel motivated anymore."


"I think I would have completed the course if I had someone to guide me or keep me accountable."

Technical Barriers

"I keep getting disconnected from the course because of my slow internet speed."


"I can't access the course materials on my phone because the website isn't mobile-friendly."

Feedback & Suggestions

"I wish the course had more interactive elements like quizzes and discussions."


"The course structure is too complicated, it's hard to navigate."

Course Content

"I don't really see the point of learning about theory when I don't know how to apply it in real life."


"It would be great if the course content was more practical and relevant to my job."

Learning preferences

"I really like video-based content because it keeps me engaged."


"I prefer written materials because I can go at my own pace and take notes."

Adobe's Business Goals

It is extremely necessary to understand company's roadmap and business goals for the project at early stage so the problem can be addressed in alignment with it. Following is a summary of the business roadmap Adobe had for the product in the concerned quarter.

  1. Explore directions towards building learning as a habit.

  2. Redesign journeys that bring users inside ALM, and enable better discovery of value and meaningful interaction.

  3. Redesign the email communication journeys to be more effective and well timed, and to bring people back in to engage with ALM.

  4. Enhanced Recommendations for Employee Use cases

  5. Learner Spotlight on Learner Interface. Built in Visualisations with Widgets that offer OOB analytics.

EMPOWER THE LEARNERS

In this quarter we need to focus more on how we can make employees more efficient and skilled in the direction they want to.

-Amit Dayal (Director of D-ALP)

Learner's Persona

Aron Stone

32, Marketing Associate
Works from 
10:00 AM TO 05
:00 PM in an MNC

Aron works at a marketing agency and is looking to upskill to take on more responsibilities at work. He has some prior experience with Adobe Creative Suite, but is relatively new to e-learning platforms like Adobe Learning Manager.

Goals

What progress do learners want to make?
What are they trying to avoid?

  • Complete required training in the given time.

  • Upskill self to be a more valuable employee.

  • Transition into a different role internally.

  • Understand aspects of a company like workflows and ethics

Challenges

What blocks the employees from achieving or maintaining their goals?

  • Lack of direction of learning (structured learning)

  • Lack of a learning habit

  • Lack of understanding of which skills relate to which roles

  • Feeling bored from some learning activities

  • Unclear ROI on taking up a course

Motivations

What motivates the employees?
What are they trying to avoid?

  • Longevity of career in a company.

  • Eligibility of higher salary/promotions.

  • Better suiting opportunities/projects.

  • Expediting promotions.

  • Internal drive to be more knowledgeable.

  • Healthy competition among peers - seeing others progress in learning.

Progress Signals

What signals indicate progress or regression towards employees’ goals?

  • Showcase of their work reflecting newly acquired skills

  • Acknowledgement from their managers & team members

  • Their effort into learning being accounted for during appraisals

  • Learners are able to apply to new opportunities based on the skills they learnt.

Tech Life Remote Life.png

LEARNERS' TOP 3 NEEDS 

Learning might not be the on the learners’ top priority list. They don’t have time and hence don’t want spend much time to search and filter through ocean of content. 

Learners take decision in a very tiny time window. If the learners aren’t seeing relevant content they’ll quit the platform.

Learners should be able to see the tangible outcome the time they are investing. Detailed & relevant feedback is extremely important so that they can identify their strengths & weaknesses.

Previous research on LMS

I started with past research which has already been done in context of a Learning Management system (LMS). I looked at a number of previous research papers to figure out pain points and problems generally found in an LMS. Following are the insights.

  1. Integration with other tools and systems is important to ensure a seamless learning experience.

  2. Both Authors and students identified the need for more efficient and effective communication tools within the LMS, such as discussion forums and messaging systems.

  3. Authors and students highlighted navigation as a major usability concern in the LMS, struggling to find and access content, assignments, and assessments.

  4. Personalization is becoming increasingly important in LMS design. 

  5. The LMS should be designed to provide learners with timely and meaningful feedback on their progress and performance. 

Following are insights from independent research papers.

  • User experience is an essential aspect of LMS design. The paper suggests that LMS should be designed to be engaging, motivating, and user-friendly to maximize learner engagement and satisfaction.

  • Personalization is becoming increasingly important in LMS design. LMS should be designed to cater to the individual learning styles and needs of each learner.

  • The use of analytics and data-driven insights can help to improve LMS design and learning outcomes. LMS should be designed to collect data on learner performance and behavior and use that data to provide personalized feedback and recommendations.

  • Collaboration and social learning should be integrated into LMS design. The paper suggests that LMS should provide tools and features that enable learners to interact with each other and with instructors to facilitate knowledge sharing and community building.

  • The paper also suggests that LMS should be designed to support mobile learning, as more learners are accessing learning content on mobile devices.

"Learning Management Systems: A Review of Literature and Future Directions" by Ayoub et al. (2021)
  • Usability is a critical factor in the adoption and success of eLearning management systems. A system that is user-friendly and intuitive can help to reduce frustration and improve learner engagement.

  • E-learning management systems should provide a variety of tools and features to support different types of learning activities, such as video lectures, interactive quizzes, and social learning activities.

  • Integration with other tools and systems is important to ensure a seamless learning experience. For example, integration with video conferencing tools and other collaboration tools can facilitate virtual classroom sessions and group projects.

  • The study identified several areas for improvement in ALM, including the need for better navigation, more intuitive workflows, and more customization options for course content.

  • The study also suggests that the use of learner-centered design principles can help to improve the usability and effectiveness of e-learning management systems.

"Evaluating e-Learning Management Systems" by Aragón et al. (2018)
  • LMS should be designed to be user-centered and focused on meeting the needs and preferences of learners. This can be achieved through personalized learning paths, adaptive learning technologies, and user-
    centered design principles.

  • Effective LMS design should incorporate gamification elements to motivate learners and increase engagement. This can be achieved through features such as badges, leaderboards, and progress bars.

  • Social learning and collaboration should be integrated into the LMS design to facilitate knowledge sharing and community building. This can be achieved through features such as discussion forums, social networking 
    tools, and collaborative learning activities.

  • The LMS should be designed to provide learners with timely and meaningful feedback on their progress and performance. This can be achieved through features such as automated grading, self-assessment tools, and progress tracking dashboards.

  • The study emphasizes the importance of evaluating the effectiveness of the LMS and its impact on learner outcomes. This can be achieved through the use of learning analytics and user feedback mechanisms.

 "Design Guidelines for Developing an Effective Learning Management System" by Lin et al. (2018)
  • Learner characteristics, such as prior knowledge and motivation, can impact the effectiveness of LMS. For example, learners with higher prior knowledge may benefit more from self-directed learning activities, while learners with lower motivation may require more guidance and support from instructors.

  • System use, including the frequency and quality of interactions with the LMS, can impact the effectiveness of the system. Learners who use the system more frequently and engage in more meaningful interactions with the system tend to achieve better learning outcomes.

  • The study found that ALM was effective in improving learners' knowledge and skills in the subject matter. However, the effectiveness of ALM was influenced by learner characteristics and their level of engagement with the system.

  • The study suggests that instructors play a critical role in facilitating effective use of the LMS. Instructors should provide guidance and support to learners to ensure they are engaging with the system effectively.

  • The study also emphasizes the importance of collecting data and using learning analytics to monitor learner progress and improve the effectiveness of the LMS.

 "Learning Management System Effectiveness: An Investigation of the Interaction Between System Use and Learner Characteristics" by Liang et al. (2020)
  • Both authors and students identified navigation as a key issue with LMS usability. They found it difficult to locate and access content, assignments, and assessments within the system.

  • Authors identified the need for greater flexibility and customization in the LMS, such as the ability to modify course layouts and organize content in a more intuitive manner.

  • Students emphasized the need for clear and concise instructions, as well as more engaging multimedia content, to support their learning.

  • Both faculty and students identified the need for more efficient and effective communication tools within the LMS, such as discussion forums and messaging systems.

  • The study emphasizes the importance of involving both Authors and students in the design and development of LMS to ensure their needs and preferences are taken into account.

 "Improving Learning Management System (LMS) Usability: Recommendations from Faculty and Students" by Reinsch et al. (2019)

Competitor Analysis

Moving ahead I looked upon the goods and bads of ALM's top three rivals. - Docebo, Cornerstone and Degreed LMS. I figured out the top feature for each LMS because of which we loose customers to them. This can be prominent areas of improvements that can be taken up actively.

  • Docebo has a more intuitive user interface that is easier to navigate, making it simpler for learners and administrators to use the platform.

  • Cornerstone offers more advanced customization options and integrations, making it a better fit for organizations with complex learning and development needs.

  • Degreed LMS offers more advanced analytics and reporting. Their platform provides detailed analytics and reporting capabilities, including real-time data on learner progress, engagement, and completion rates, as well as insights into the effectiveness of learning content and activities.

Let's have a deeper look into the strengths and weaknesses of all our three competitors to understand them better.

Docebo.png

ALM, Course description page

Docebo, Course description page

Docebo

Strengths

  • User-friendly interface: Docebo’s platform is known for its intuitive and easy-to-use interface, making it a popular choice for learners of all levels.

  • Advanced reporting and analytics: Docebo’s reporting, and analytics tools provide detailed insights into learner behavior and course effectiveness, helping organizations make data-driven decisions.

  • Customization options: Docebo offers a wide range of customization options, allowing organizations to tailor the platform to their specific needs and branding.

  • Gamification features: Docebo’s platform includes gamification features like badges, points, and leaderboards, which can increase learner engagement and motivation.

  • Social learning capabilities: Docebo includes features like discussion forums and social sharing, enabling learners to collaborate and learn from each other.

Weaknesses

  • User-friendly interface: Docebo’s platform is known for its intuitive and easy-to-use interface, making it a popular choice for learners of all levels.

  • Advanced reporting and analytics: Docebo’s reporting, and analytics tools provide detailed insights into learner behavior and course effectiveness, helping organizations make data-driven decisions.

  • Customization options: Docebo offers a wide range of customization options, allowing organizations to tailor the platform to their specific needs and branding.

  • Gamification features: Docebo’s platform includes gamification features like badges, points, and leaderboards, which can increase learner engagement and motivation.

  • Social learning capabilities: Docebo includes features like discussion forums and social sharing, enabling learners to collaborate and learn from each other.

Cornerstone.png

ALM, Homepage

Cornerstone, Homepage

Cornerstone

Strengths

  • User-friendly interface: Cornerstone OnDemand's interface is generally intuitive and easy to use, making it simple for users to navigate the platform and find the content they need.

  • Personalization options: Cornerstone OnDemand offers some personalization options, such as the ability to customize the homepage and create individualized learning paths.

  • Robust reporting and analytics: Cornerstone OnDemand offers detailed reporting and analytics features, which can provide valuable insights into user behaviour and learning outcomes.

  • Social learning features: Cornerstone OnDemand includes social learning features, such as the ability for users to collaborate with others & share knowledge and resources. 

  • Mobile accessibility: Cornerstone OnDemand is accessible from mobile devices, which can improve the user experience by allowing users to access the platform on-the-go and complete learning activities from anywhere.

Weaknesses

  • Complexity: While Cornerstone OnDemand's interface is generally user-friendly, the platform can be complex and 
    overwhelming for some users, especially those who are not familiar with learning management systems. 

  • Limited customization options: While Cornerstone OnDemand offers some personalization options, there are limitations in terms of how much customization can be done. 

  • Technical issues: Cornerstone OnDemand experiences frequent technical issues & downtime, which can impact the user experience by making it difficult for users to access the platform or complete learning activities.

  • Lack of gamification features: Cornerstone OnDemand does not offer robust gamification features, which can impact the 
    user experience by making learning a less engaging and 
    motivating process.

  • Limited search functionality: While Cornerstone OnDemand does include a search function, it may not always return 
    relevant results or be as powerful as users expect. 

Group 16906.png

ALM, Reports

Cornerstone, Reports

Degreed LMS

Strengths

  • User-friendly interface: Degreed LMS has a clean and modern interface that is easy to navigate, making it user-friendly for learners.

  • Personalized recommendations: The platform's AI-powered recommendation engine provides personalized learning content and activities based on the user's interests, skills, and career goals, which can improve the user experience by providing relevant and engaging content.

  • Customizable learning paths: Degreed LMS allows organizations to create custom learning paths that are tailored to their specific needs, which can improve the user experience by providing a more relevant and targeted learning experience.

  • Intuitive search functionality: Degreed LMS has a robust search functionality that allows users to easily find and discover relevant learning content, which can improve the user experience by reducing the time and effort required to find the right content.

  • Responsive design: The platform is designed to be responsive and adaptable to different devices and screen sizes, which can improve the user experience by ensuring that the platform is accessible and usable on any device.

Weaknesses

  • Inconsistent content quality: Because Degreed LMS sources content from a variety of providers, there can be inconsistency in the quality of the learning materials, which can impact the user experience by making it difficult to find high-quality content that meets their needs. This can also lead to frustration and disengagement for users who have to sift through lower-quality content to find what they need.

  • Limited customization options: While Degreed LMS allows organizations to create custom learning paths, some users may find the customization options to be limited compared to other platforms, which can limit their ability to tailor the platform to their specific needs.

  • Limited social learning features: Degreed LMS does not have as strong of a focus on social learning as some of its competitors, which may be a disadvantage for users who value collaborative learning experiences.

  • Limited feedback mechanisms: Some users may find the feedback mechanisms in Degreed LMS to be limited, which can make it difficult to track progress and receive feedback on their learning activities.

  • Limited customization options for the user interface: While the platform has a clean and modern interface, some users find the customization options for the interface to be limited, which can impact the experience for users with different preferences and needs.

Areas of improvement on learner’s side

Considering all the data we've gathered till now, a card sorting activity was conducted in a workshop and following five areas of improvement were marked as high priority. The final problem statement could cater to one of these areas.

Tech Life Life Management.png

Personalized Recommendations (40% participants aligned)

We went ahead with 'Personalized Recommendations' because it aligned with the product roadmap for the year, and it was a repeated ask from the customers of ALM as well. The idea was to create an MVP for recommendations and build on top of that as per feedback.

Mobile Experience Enhancement
(30% participants aligned)

Social
Interaction

(15% participants aligned)

Notifications & Reminders
(10% participants aligned)

Feedback Mechanism
(5% participants aligned))

Learning Scenarios

Now let's look at some learning scenarios where the learners try to discover courses. Majorly, on ALM course discovery happens on the home page or the catalog page. Following are the scenarios on both the pages.

In a scenario lets say Aron (Our persona) wants to become a proficient user of a product offered by a certain company that uses ALM as their training platform. Aron will log on to ALM and land on home page and further he'll navigate to the catalog page to discover relevant courses for himself. Following are his gaols and challenges.

Aron lands on the homepage.

Goals

  • Discover Relevant Learning Content: Aron wants to find learning resources that align with her interests and career goals.

  • Access Personalized Recommendations: Aron expects to see tailored recommendations based on her previous learning activities and preferences.

  • Stay Updated on Progress: Aron desires to track her learning progress, including completed courses, ongoing activities, and achievements.

Challenges

  • Inadequate Actionable Insights: The homepage has very limited information on Aron’s progress and the actions needed to be taken, esp. critical ones.

  • Limited Personalization: If the homepage doesn’t offer highly relevant recommendations, and hence he has to spend more time searching for relevant courses.

Screenshot_20230508-160653.png

Aron navigates to the catalog page

Screenshot_20230508-160653.png

Goals

  • Discover Relevant Learning Content: Aron wants to find learning resources that align with her interests and career goals.

  • Efficient Course Selection: Aron aims to quickly identify and select the most relevant courses from the catalog that align with her desired learning outcomes and fit into her personalized learning path.

  • Relevant and Up-to-Date Content: Aron expects the catalog to provide up-to-date courses and learning resources that reflect the latest industry trends and technologies.

  • ROI and Investment required: Aron wants to quickly figure out how the course benefits her, what will he learn, how much time she has to invest and what are the pre-requisites for enrolling onto a learning material.

Challenges

  • Overwhelming Course Options: Emily may feel overwhelmed by the large number of courses available in the catalog, making it difficult for her to determine which ones are most relevant to her learning goals.

  • Lack of Personalization: Emily may struggle prioritise a learning material over other.

  • Inadequate information for decision making: Without a personalized learning path, Emily may find it challenging to plan her learning journey efficiently and allocate time to specific courses or topics.

  • Motivation and Engagement: Emily may struggle to stay motivated and engaged throughout her discovery journey, especially without a clear path that aligns with her interests and long-term aspirations.

  • Content Relevance: It might be difficult for Emily to decide which learning material might be relevant to her. She might overlook some of the relevant courses which can help her upskill.

Existing state of recommendations on ALM

There are certain places in ALM where the platform recommends learning materials to the learners. However, it has been obeserved that the engagement around these recommendations are almost nil. Hence, we decided to improve how we are recommending courses to learners in a quest to increase learning consumption on the platform. The existing recommendation areas are shown below.

Group 16909.png

Courses are being recommended as 'Related courses' on course description page. These recommendations have relevance as it is related to the course currently being viewed. Hence we will not be accommodating any change here for the MVP.

Courses are being actively recommended on the homepage based on 'Areas of interest' and 'Peer activity'. However, these recommendations are quite irrelevant because 'Areas of interest' per say is defined by the Manager primarily and then by the learner. However, it has been observed that less that 1% of the learners go to the concerned section to setup their areas of interest. Peer activity mostly recommends courses which are mandatory for employees in an organization and not necessarily contribute to self-motivated learning in skill development.

For the MVP, we intend to improve this area at it takes a substantial and influential real estate on the platform.

How do we improve recommendations?

The best way to improve recommendations is to get to know what the learners want to learn about. Hence, the first step towards building up this enhancement is to get to know learners' interests deliberately.

After conducting a brainstorming session, the team concluded that we will be asking the following questions as the first step towards this enhancement.

What products/topics they want to learn about?

What role they want to learn about?

What is their expertise level for the product/role?

So the next time when a learner logs in after this release, we will be asking them the above questions via a non-dismissible modal popup.

Imapact

The end user experience will be significantly improved by providing personalized course recommendations based on their interests, including specific products, roles, and levels. By incorporating explicit interests and insights gained from customer behaviour, we can ensure that the recommended courses are highly relevant to the learner's goals and career path.

 

This personalized approach will help learners to quickly find and access relevant content, ultimately saving them time and improving their overall learning experience.

How will it work?

Once we capture this information, we will replace the existing rows of recommendations on the homepage with relevant recommendations. These recommendations will show the courses from the selected products and roles along with the existing recommendation rows ('Areas of interest' & 'Peer Activity') so that there is no regression of existing features & functions.