Sign-offs for e-Learning Courses
When creating e-learning courses I ask stakeholders to sign-off on the final product. This is only after the review process is complete. Very often the stakeholders are also the subject matter experts for the course. Either way, I find that a sign-off provides confidence for me and my team that the course meets the approval and needs of the stakeholder, the content is accurate, and that we have a commitment from the stakeholders/SMEs for future reviews.
Here is what I include in a typical sign-off form. Granted, this may change depending on the project and any addendum requested by the stakeholder/SME. I keep it as simple and succinct as possible. The following points are included in the agreement.
- That the design and content for the course provides accurate information and complies with the organization’s current policies and procedures.
- To conduct future reviews on a specific scheduled basis to verify its accuracy.
- To inform the course designer(s) of any future content, policy and procedure changes that affect the on-line learning program.
- The course will not be released on the learning management system (LMS) or Intranet without the completion of this form/agreement.
Please keep in mind, I am referring to a sign-off for courses created for internal use. A sign-off for an external client would be approached with greater detail.
Are there additional things you include in a sign-off? Please feel free to share in the comments section.
If You are Reviewing It Online, Why Do You Want Me to Print the Course
All too often people reviewing a web-based training (WBT), including subject matter experts (SMEs), request the course printed for them. If it is the absolutely only way they will review it, then I do accommodate them. Keep in mind this is after I have exhausted all other attempts of getting them to do a proper online review.
Here are reasons not to print courses for a review:
- It is important for anyone reviewing a course to not just look at content, but to review the entire learning experience including the delivery medium.
- If they themselves are not willing to participate online how can they expect, or request, our audience to participate.
- Online courses are very often non-linear. Thus, do not fit in a printed, linear format.
- Courses are interactive. They may contain anything from simple rollovers to complex games or simulations. Interactivity does not translate to a printed page.
- Once printed it is occasionally handed around for others to review without the designer’s knowledge. This can result in not being able to identify the origins of edits, if needed. It can also result in draft content mistakenly being distributed to the end user. This can all be prevented by setting appropriate access in an LMS.
- Depending on the authoring tools used, it can be time consuming to print a course. For example, a course that contains many interactive Flash elements will require many screenshots to be taken. Time is better spent on on design and development.
- It is more environmentally friendly to review online. As a fellow e-learning designer said to me recently, “I killed many trees with “WBT to be printed out” for SMEs, higher ups, etc.”
The reality is people reviewing courses are going to push for a printed version and sometimes the only way to get them to review it will be to comply. However, I am not going to comply without at least explaining the importance of an online review. In the end, even if I send them a printed version, or screenshots, I always supply easy access to the online course along with several reminders of how important it is to also review it online.
Hey You Rapid e-Learning Peeps, Slooow Down and Take a Little Drive on the ISD Side of Town
When I read e-Learning Magazine’s article by Bob Little, Rapid e-Learning Polarizes Opinion, I was very irked by it. Especially when I read the following excerpt.
“While purists sneer that e-learning produced via rapid tools may lack quality in terms of adhering to instructional design principles and may just be brain dumps by subject matters experts, if such e-learning materials improve workers’ performance, who can criticize their place in the learning and development armoury?”
I will say this, if they are not adhering to instructional design principles, then they are far less likely to improve workers’ performance. I have never been a fan of the term rapid e-learning. I believe there are some great rapid development tools, but these still require sound instructional design, which takes time and effort, starting with a needs analysis.
As stated, I am not fond of the term rapid e-Learning, but I do not wish to be negative or come across as bashing efforts made under only good intentions. Here is the reality, not all organizations have the luxary of employing an instructional designer or perhaps enough instructional designers. However, they do wish to offer their staff online training. So, the subject matter experts gets a hold of the new fangled software that says “create e-learning in only a matter of hours” or some such thing. They then do their very best, but because they did not identify the learner’s needs and create an effective course design it falls short and does not result in learning or impact behavior.
So, here is what I suggest.
First off, be willing to dedicate more time to the analysis and design phases of your project. Learn as much as you can about instructional design, more specifically e-learning design. Here are a few ways to get started:
- Follow e-learning blogs (the eLearningLearning blog community and eLearningPulse are chock full of great blogs)
- Attend conferences (there are plenty of great e-learning conferences out there)
- Join e-learning and instructional design societies and groups (the eLearning Guild and ASTD both have plenty of books, articles, research, webinars, conferences and more)
- Network with e-learning designers and ask for advice and reviews of your work (LinkedIn has plenty of e-learning and ID groups and Twitter is perfect for connecting with people in the e-learning world)
The more time you invest in instructional design, the more effective your courses will be and your audience will appreciate it too. And remember, sloooow doooown and spend some time in instructional design!
Big Question - Working with SMEs
This month’s Big Question at Learning Circuits is on working with subject matter experts (SME). First off, thank you Tony at Learning Circuits for using my suggestion as this month’s question. In my opinion, working successfully with SMEs is the most challenging part of instructional design. I have written quite a bit about this subject on my blog. Here is a link to my posts with SME tags and my tips on working with SMEs (pasted below from a prior post along with an addendum).
Originally posted November 16, 2008
When working with subject matter experts (SME), you must keep them involved and engaged in every step of the e-learning process. Here are some tips to accomplish this goal.
Lay the foundation
Introducing e-learning to the SME lays the foundation of a successful collaboration. E-learning is still new to many people and many misconceptions exist. Someone not exposed to quality e-learning may think it will be a page turner, which is not good e-learning. Explain what e-learning is, when it is appropriate, and its capabilities. Suggest they look at courses already available. This will provide some reference to what they may expect in a course.
Introduce their role in the process
Inform SMEs of your expectations of them, allowing them to schedule their time and prepare for the tasks ahead. They will not just hand over some form of subject matter and that’s that. They will help identify training needs, learning objectives, etc. Also explain other expectations you have of them as it relates to the different phases of the course’s creation (proofing prototypes, drafts, clarifying subject matter, etc.).
Ask the right questions (Analysis)
Never assume the SME knows the training need. A needs analysis is very important. Reality is that needs analyses are often informally conducted. If a needs analysis has not been conducted by you or anyone else then it will be your job to identify the training need. And your SME will be your first resource.
Share the plan (Design)
Unless your SME has worked with an instructional designer before, a training design plan will be new to them. So, prepare them on how to interpret it and make sure you explain the difference between terminal objectives and enabling objectives. Once you and your SME have identified the appropriate objectives and agree to the design, get their approval. Remember, write your design plan to a non-training audience. You do not want to confuse them with a lot of training jargon.
Put them to work (Development)
Keep your SME updated during the development stage. Development takes a significant amount of time and they may wonder, “Whatever happened to that e-learning designer?” Do not let SMEs forget about the project while you are busy creating it. Let them know of your progress. If possible upload what is available so far and call it a prototype. And get their feedback.
Provide detailed instructions for reviewing drafts. I also provide a checklist. They should not think they are to look at grammar or content only. Be sure they also look at the flow of the course, accuracy of questions, usability, and identify any technical problems. Also request SMEs to provide names of anyone else familiar with the subject well enough to provide useful feedback.
Market the course (Implementation)
You may release the course on your learning management system, but there is more to implementation. It must be marketed. Participation requires strong communication efforts and buy-in from supervisers. Have your SME help communicate the importance of the course. SMEs can help promote the course via e-mails, intranet announcements, employee newsletters, presentations, etc.
Is it effective? (Evaluation)
SMEs are subject matter experts because they apply the knowledge the course will teach, or work closely in some form with those applying it. Thus, they have an inside view of seeing the knowledge or skills applied and have a relationship with those applying it. This will be helpful in connecting you to the people that will provide evaluation data. SMEs can also support the process by selling the importance of working with you in evaluating the course.
In the end, effective collaboration with your SMEs will compliment your project. The key to this collaboration is keeping your SMEs informed, invested, and involved throughout the process. And always give them appropriate credit and a big thank you.
Addendum - September 1, 2009
I will add that recently I started a new position with the bank that acquired my prior bank/employer. One thing I learned during this transition is that when joining a new team of instructional designers it is very important to get a good grasp of how they approach working with SMEs. Although the team works well with SMEs some aspects of my approach created shifts in a process in which the SMEs had become very accustomed. However, using my new team’s approach while introducing aspects of my own worked well, but I made it a point to explain to the SMEs why I was approaching certain tasks differently than they experienced in prior projects. SMEs can become very accustomed to specific ways of working with their instructional designers. So prep your SMEs well, especially if they already have pre-conceptions of the SME-instructional designer relationship. This is not to say one way is wrong or right, but it is what works well for each person and for the needs of the project.
you have to fight for the right TO BE ENGAGING
I just read a comment on a blog where a someone was very frustrated by bland, unengaging page turners. It got me thinking. How do you get a an organization out of the rut of making page turners and to start creating more engaging and effective courses? Here are my first thoughts:
- Put on your instructional designer hat and do everything you can to educate all involved (SMEs, clients, managers, and audience too) on what effective e-learning is and how all involved can benefit from it.
- Show all involved what effective e-learning looks like, actual examples. Here is just one place where you can find examples -http://minutebio.com/blog/free-e-learning/ (this my Free e-Learning collection ).
- Find case studies, articles, evaluations, etc. that support your case.
- Create a prototype to demonstrate the level of interactivity and engagement your organization can produce in a course. Get your co-workers involved so they will be vested in the “new approach.” This will earn you supporters and people who can rally against the archaic page turners the organization still wants to produce.
- When you launch your prototype/course and your audience provides positive feedback. Be quick to send that feedback to the powers to be along with any evaluation you have done. They will have a hard time arguing against more interactive courses then.
- Continue to evaluate your courses even after you have been given the go ahead and resources to create more interactive courses. If you can demonstrate positive results for all 4 levels of evaluation, especially “results,” they will have little argument for ever implementing a page turner again.
What else can be done to address the organization stuck in page turner mode? Please feel free to make suggestions. Thanks.
Building a Camel
“You put together a committee to build a horse and get a camel.”
That is what I have been going through with my SMEs. I had the right subject matter experts (SMEs), all with the expertise needed, and not too many to be productive or reach a reasonable consensus. Somewhere along the line they took it upon themselves to invite more into the mix and now reaching a concensus is becoming difficult.
The bigger problem is they all want to throw more content into the course. The SMEs want everyone to be subject matter experts too. While the learners need the skills to do to do THEIR job effectively, not to master all the skills they are not expected to ever apply.
Today the SMEs surprised me. They wanted to push the training up 3 weeks due to an earlier than expected system release. This is the first time I have ever experienced an early system release. Usually these are delayed, maybe on time, but never early. Anyhow, what is an e-learning guy to do?
It was an opportunity to make sure we built a horse!
Here is what I did:
- Shaved 1 week from the amount of time participants have to take it prior to the system release. This means I have 2 weeks less development time and participants have 1 less week to take the course. It’s a compromise between the learners/participants and me, and the SMEs still get to keep there training prior to the system launch date. I hate giving participants less time, but it is a necessary evil.
- Revisit the storyboards and determine what can be eliminated from the course that is not essential to participants being able to use the new system. This will accomplish the following:
- Shave a lot of development time off. Especially if it includes eliminating unnecessary simulations. I know for a fact there are numerous topics, demos and sims that are what I call “nice to knows.” It is great to provide background info or more about the system’s inner workings, but for the sake of time on our part and the participants part too, they can use the system effectively without this some of this info. And believe me, my audience will appreciate a succinct course, while still being able to learn exactly what they need in order to use this system.
- Redesigning some of the interactive sims (try me sims) into demos (show me sims) instead. The very simple tasks will transfer and be retained just as well with out the practice, in my opinion. I will still invest the time needed to develop the interactive “try me” sims for the more complicated tasks.
Yes, it is a pain to have less time to develop this course, but at least it can be a more succinct, focused and hopefully a more effective course.
What I Ask During a Course Review
I just released the first draft of a new WBT course and as usual I have a slew of people reviewing the course. This includes Subject Matter Experts (SMEs) among others. In the past I have provided a general list of what aspects of the course should be reviewed (e. g. grammar, accuracy of content, navigation, technology, etc.). This time around I compiled a far more detailed list of concerns reviewers should be attentive to during their review. It is meant more as guide to what they should be looking for, but can also be used as a questionnaire.
Here is what I included:
Grammar
- Check spelling, grammar, and consistency of language.
Objectives/Learning needs
- Does the course answer your questions/concerns about the subject?
- Do you feel prepared to begin applying the new knowledge/skills learned?
- Does the course meet the objectives presented at the beginning of the course?
- Do you feel you now have a better understanding of the subject at hand?
Navigation
- Were there any links or buttons that did not work?
- Were all navigational elements marked appropriately?
- Were you able to navigate through the course with ease?
Graphics
- Do you find the graphics helpful?
- Do the graphics appear properly?
- Was text in the graphics clear and visible?
Animation
- Does the animation appear properly?
- Was text in the animation clear and visible?
- Do you find the animation helpful?
Simulations/Interactivity
- Are the soft skill simulations reflective of realistic scenarios?
- Do the simulations, interactive exercises and/or pop-ups function properly?
- Are the software simulations/demonstrations realistic and appear to reflect the actual “live” system?
Assessment
- Do the questions measure your understanding of the content presented?
- Are there questions that address content not presented in the course?
- Are the questions/answers accurate and pose no potential exceptions that could make an answer incorrect?
- Is the feedback provided helpful?
- Does the assessment provide correct scoring results?
Misc. technology
- Does the audio function properly?
- Do the videos function properly and appear professional?
I am sure as time goes on questions will be added and some will be eliminated. What would you include, eliminate or change on this list? Any input would be great.
Has Anyone Seen My SMEs?
When working with subject matter experts (SME), you must keep them involved and engaged in every step of the e-learning process. Here are some tips to accomplish this goal.
Lay the foundation
Introducing e-learning to the SME lays the foundation of a successful collaboration. E-learning is still new to many people and many misconceptions exist. Someone not exposed to quality e-learning may think it will be a page turner, which is not good e-learning. Explain what e-learning is, when it is appropriate, and its capabilities. Suggest they look at courses already available. This will provide some reference to what they may expect in a course.
Introduce their role in the process
Inform SMEs of your expectations of them, allowing them to schedule their time and prepare for the tasks ahead. They will not just hand over some form of subject matter and that’s that. They will help identify training needs, learning objectives, etc. Also explain other expectations you have of them as it relates to the different phases of the course’s creation (proofing prototypes, drafts, clarifying subject matter, etc.).
Ask the right questions (Analysis)
Never assume the SME knows the training need. A needs analysis is very important. Reality is that needs analyses are often informally conducted. If a needs analysis has not been conducted by you or anyone else then it will be your job to identify the training need. And your SME will be your first resource.
Share the plan (Design)
Unless your SME has worked with an instructional designer before, a training design plan will be new to them. So, prepare them on how to interpret it and make sure you explain the difference between terminal objectives and enabling objectives. Once you and your SME have identified the appropriate objectives and agree to the design, get their approval. Remember, write your design plan to a non-training audience. You do not want to confuse them with a lot of training jargon.
Put them to work (Development)
Keep your SME updated during the development stage. Development takes a significant amount of time and they may wonder, “Whatever happened to that e-learning designer?” Do not let SMEs forget about the project while you are busy creating it. Let them know of your progress. If possible upload what is available so far and call it a prototype. And get their feedback.
Provide detailed instructions for reviewing drafts. I also provide a checklist. They should not think they are to look at grammar or content only. Be sure they also look at the flow of the course, accuracy of questions, usability, and identify any technical problems. Also request SMEs to provide names of anyone else familiar with the subject well enough to provide useful feedback.
Market the course (Implementation)
You may release the course on your learning management system, but there is more to implementation. It must be marketed. Participation requires strong communication efforts and buy-in from supervisers. Have your SME help communicate the importance of the course. SMEs can help promote the course via e-mails, intranet announcements, employee newsletters, presentations, etc.
Is it effective? (Evaluation)
SMEs are subject matter experts because they apply the knowledge the course will teach, or work closely in some form with those applying it. Thus, they have an inside view of seeing the knowledge or skills applied and have a relationship with those applying it. This will be helpful in connecting you to the people that will provide evaluation data. SMEs can also support the process by selling the importance of working with you in evaluating the course.
In the end, effective collaboration with your SMEs will compliment your project. The key to this collaboration is keeping your SMEs informed, invested, and involved throughout the process. And always give them appropriate credit and a big thank you.
For a more in depth exploration of this topic, go to the article I published in 2006 on e-LearningGuru, Working With Your SMEs.
Adding Pizzazz to e-Learning Videos
I am currently working on adding short videos into a web based training course. These are approximately 2 minute videos of the subject matter experts (SMEs) who are extrapolating on the content at hand. The SMEs did a wonderful job, but…a little extra “pizzazz” would increase attention.
So here’s what I did…
I imported each video into Flash and added layers above the video. In these layers I added text that emphasized key information in their presentation. Where graphics, charts, and screenshots helped, I add those too. I was very careful not to distract the learner away from the presenter, but rather highlight their message. It is also important not to obscure the presenter. In my case, I left some room above and to the sides when taking the video. This allowed me the space to have the text and images tween in and out of the canvas.
In one particular case, the SME was discussing a scenario that involved a series of numbers and calculations. By showing, and labeling, the calculations next to the SME, along with numerous animated arrows, it allows the learner much more visualization of the scenario.
The course is still in the works, but I have shown the videos to a small group and they have been received very well. I have shown these to the SMEs too, who liked them. FYI: Always show videos to SMEs early on. It is better to find out they are not happy with the results and re-shoot the video, than invest a lot of time into it something you may not use.
