Schedule a Demo

TrainCaster LMS Online Training Blog

5 min read

LMS System Selection & Evaluation Toolkit Part 2

Mar 25, 2019 12:10:43 PM

LMS Selection ToolkitThis is the second in a two-part series.

Last post we talked about defining the LMS selection process, and gather background information so you are well prepared to evaluate potential LMS applications. In this post we focus on creating a requirements list, with specific tasks and scenarios to aid in the evaluation of each LMS.


Selecting an LMS is based on needs: the better you understand your needs and requirements, the easier it will be to identify an LMS that meets your needs. Defining the features you need can be a time consuming task, but at a bare minimum, ask yourself the following questions:

  • How many users does the LMS need to support?
  • Do I need a hosted or installed solution
  • Is SCORM/AICC support required?
  • How easy is the user interface?
  • Is customer support readily available and responsive to my needs?

It is also useful to create some task scenarios that you need to accomplish using the LMS. Write up a few – something like, ‘I need to assign several courses to trainees in the accounting department. The courses have to be completed in a specific order and re-training has to be done every 5 years.’ Or, ‘I’m the supervisor in the window installation department. How will I know if all of my team has completed the necessary training for installation? How can I find out who is ready to do a particular product installation?’ Use these scenarios to help identify key features that you need in an LMS. You will use them again in STEP 4.

Before writing the scenarios, come up with a list of general goals LMS users will have. Ask yourself: What does every user need to be able to do using this application? For example, everyone needs to take courses - assigned and from the course catalog; every user needs to know which courses are required and when training is due; every user needs to be able to see their training history. There are also general tasks that specific types of users need to accomplish: create a new course, modify an exisiting course, report on training completions/status.

Once you've come up with the list of general goals, you can write task scenarios. These scenarios provide context so LMS evaluators engage with the interface and perform tasks as they would in the real-world.

Poorly written scenarios tell users which features to use when interacting with the LMS, rather than allowing the user to choose how to complete the task. By recording the way individuals choose to complete the task, you can compare LMS's to see how easy the system will be before you commit.

Here are 3 tips for writing scenarios:

(1) Make the Task Realistic

User goal: Take and resume an assigned course.
Poor task: Take the New Hire Training course.
Better task: Take and complete a course that is due within 30 days.

(2) Make the Task Actionable

User goal: Report on training compliance in a department.
Poor task: You want to see training compliance. Go to Reports and run the training compliance report.
Better task: Use the LMS reporting tools to report on training compliance in your department. Get a specific list of compliant/non-compliant employees..

(3) Avoid Clues and Describing the Steps

User goal: Look up quiz results to improve quiz questions..
Poor task: You want to see quiz results for New Hire Training. Go to the LMS, sign in, and run the Quiz Results Report.
Better task: In an effort to improve quiz questions, look up the results of the New Hire Quiz. Determine if the questions are effective or need to be improved.


Additional Resources - Useful information about website/application usability.

LMS Scenario Form
Use this form to record your task scenarios. Includes examples and brief process description.

Not sure if you need a SCORM compliant LMS? We’ve got the guide for you! Download our SCORM 101 eBook to see if that is a feature you need.



Depending on the size of your team and the number of potential vendors you’ve identified you have a couple of choices as to how to proceed with the actual evaluation of potential LMS vendors.

If you’ve identified 3 or fewer vendor’s:

Have each member of the team conduct an individual study of each LMS, answering basic questions about the overall experience, usability, & features included.

If you’ve identified more than 3 potential vendors:

If you’ve identified more than 3 potential vendors, assign a single vendor to each team member. The team member completes the individual study as detailed above. Come together as a team to review the preliminary results of the evaluation, and to further prune the list of potential LMS’s down to 2 or 3 vendors to study in detail.

Use the tasks/scenarios you created in STEP 3 during this evaluation process – how easy is it to accomplish the real-world tasks you’ve defined? This will give you better insight into how well the LMS meets the needs of your organization.

Hours & Budgeting

At the bare minimum allocate at least one hour to analyze each potential vendor, including testing, tallying features, using task scenarios, and making recommendations.

If you don’t have the resources or budget required, limit the number of vendors you are evaluating – not the depth of your evaluation.


Additional Resources

LMS Evaluation Checklist
Download the LMS Evaluation Checklist. Easy to check off LMS features, usability, overall impression, etc.



Talk with each of the finalist vendors. Get detailed information about pricing, additional fees, training, and customer support. Ask for references and check them. You will have this LMS for at least the next 3-5 years, so due diligence here is important. Compare best offer price quotes.

Final selection should be based on your evaluation results, pricing, and ‘bang for your buck’. Although it’s easy to choose the cheapest solution, remember to factor in hidden costs like purchase of support and training packages, purchase of additional software or off-the-shelf courses, fees for additional user licenses or disk space, to name a few.

Best of luck in your LMS search!

BL Snodgrass
Written by BL Snodgrass

UX expert, elearning veteran.