This summer the Learning and Teaching Systems and Services Consultation Committee conducted a survey of the Ryerson community and asked “What Do You Need from Ryerson’s Online Learning and Teaching Systems?” 1293 students responded to Ryerson’s survey.  87% of these student respondents are currently using Blackboard.

337 responses were received for the faculty and staff survey but only 66% of these respondents are currently using Blackboard.

Only 66% of instructors surveyed actually use Blackboard — approximately 20% fewer than the 87% of students surveyed.  Common feedback from instructors not using Blackboard was that they “did not use Blackboard this year because it is too confining and cumbersome.” Only 68% of instructors surveyed believe that Blackboard has actually improved the learning experience.  70% of students surveyed believe that Blackboard has changed their learning experience in a positive way. Only 6% of the overall student respondents believe that Blackboard has negatively impacted their learning experience and approximately 24% do not feel that Blackboard has changed their learning experience in any way.

The majority of instructors and students using Blackboard are doing so primarily as part of courses delivered face to face in the classroom, in comparison to using it as part of a mix of classroom (hybrid) or distance courses.

Students cite frustration that their experience with Blackboard varies drastically from course to course or from one instructor to another. Many respondents want a consistent or standardized use of Blackboard in their courses.

The most common qualitative feedback in the surveys was around the poor usability of Blackboard, and the need to improve the user interface to be more intuitive and user friendly.  Feedback indicated that Blackboard is frustrating to use.

Blackboard Usage by Students

Students commonly cite the ability to view assignment grades online as one of the most important features of Blackboard.   Student responses are generally more concerned about access to view their grades, whereas instructors were more concerned with feedback mechanisms for improvement rather than simply providing grade outcomes.

Links to external readings and supplementary information such as video, podcasts and websites was also rated with high importance by students.  Posting course material such as lecture notes, recorded lectures, lab manuals, handouts, practice questions, sample quizzes, and exams from past years were frequently cited.

Blackboard’s wiki and blog features had the largest amount of ‘do not use’ responses.  Most students cited that collaboration tools were the least useful Blackboard features listed in the survey list.  Chat and virtual meetings, wikis, and blogs were the only tools cited by students as being less than useful

Regarding how their use of Blackboard has changed over time, many student responses were similar in the belief that Blackboard has not kept pace over time with how students want and expect to use technology tools in learning. One student remarks that “Blackboard was very good when it started out, but technology has matured and Blackboard needs to adapt as well.”  In other examples, students expressed that their Blackboard use has decreased over time because Blackboard does not run well on smartphones, or because better tools already exist outside of Blackboard that students prefer to use.

Blackboard Usage by Instructors

The top instructor uses of Blackboard are for posting announcements and course outlines.  Almost 85% of Ryerson instructors on Blackboard utilize these functions.  Students feel that the most important use of Blackboard is for posting course outlines and lecture notes.  Overall, the survey results indicated instructors are primarily using Blackboard in the ways that students believe to be most useful to their learning success and that there is a good deal of the alignment between instructors’ actual use of Blackboard and students’ perceived usefulness of Blackboard features.

The way in which instructors are using Blackboard has not changed over time, and a majority of instructors do not feel they need more functionality than what they are already using in Blackboard.  Most instructors’ first use of Blackboard was for posting announcements and course outlines and those are still their top primary use.  There is a slight increase in instructor use of posting student assignment (8%) and the gradebook to mark and track the assignments (4%).

LMS Features

Six of the top most important LMS features as determined by the instructor responses are related to assessment.  Some, but not all, of the features listed on the survey are available in the current implementation of Ryerson Blackboard.  The most frequently cited ‘very important’ feature, by over 66% of instructors, is the ability to automatically contact students based on certain performance indicators.  The other popular assessment features are related to automation and integration such as exporting marks to RAMSS (57.8%) or importing test response system results (47.8%), and the integration of markups and comments within a document with the gradebook (49.3%). The survey results suggest that staff and faculty view emerging learning analytics and assessment functionality (47.4%) as key components in an LMS, more so than content, delivery and collaboration tools.  The 6th on this list is the ability for instructors to release content adaptively based on student’s performance (45.3%).

Students ranked mobile access as a top five ‘very important’ feature of an LMS, while a majority of instructors viewed mobile as ‘not important.’  The same occurred for the ability to view recorded lectures, which students perceive to be almost as important as mobile access, but which instructors ranked as a top-three ‘not important’ feature.   This is consistent with many of the qualitative responses received from instructors where the perception seemed to be that online lectures discouraged students from attending classes in person.

Blackboard Training

A generalization can be derived from some of the recurring themes that appear in the instructor responses to the question on training.  ‘Clunky’ is a word that appears frequently to describe Blackboard.  Some instructors believe they do not have time to learn a new LMS, while others think that a change is justifiable only if the new system is truly more user friendly than the current one, and that the learning curve on the new system is low.  Many staff and faculty are happy with the current training and support options provided for Blackboard.

The majority of students reported being self-taught on Blackboard.  Others learned via other students, professors, online help, or at another institution.

Cloud Hosting

Over half of the students and instructor responses cited that they expected to be either satisfied or very satisfied with a cloud-hosted LMS solution. Most of the negative feedback related to cloud hosting expressed concerns over privacy and security. There was also a stated preference for internal or at least domestic (i.e. Canadian) data storage, where respondents perceived privacy laws to be better enforced.

Other Instructional Tools

Some commonly cited LMS features that respondents believe are not currently available in Blackboard but that they would like to use as part of an LMS are listed in the figure below:

Both instructors and students are interacting with video and YouTube to meet learning objectives.  For an LMS to be successful at Ryerson, it should seamlessly interface with YouTube, for example by allowing embedded video or easily accessible links.  In addition, integration with Turnitin for assignment submission is also an important feature.  The high usage of DropBox suggests that instructors and students needed some kind of secure data storage for transferring and sharing information. (With the recently available Google Drive, there may be changes to the usage of external Ryerson solution.) Other tools used by students and instructors are listed below:

Conclusions

From the survey, we learned that majority of instructors and students at Ryerson use the LMS for its basic features, however there is a clear desire to use more advanced tools such as social networked learning and learning analytics, provided these features work well and are integrated seamlessly with an easy to use LMS.

Credits

This post is a summary of a full report written by Brad Stewart, MBA as his Major Research Paper in the Ryerson MBA program.  Brad assisted the committee with analyzing the survey results.

The Learning and Teaching Systems and Services Consultation committee is preparing a Request for Proposal (RFP) for learning and teaching systems and services. Depending on the responses we receive we may upgrade Blackboard or replace it, and may also add other services needed for learning and teaching online. The committee would like to invite you to submit stories (use cases) about how you would like the system to work. Our goal is to incorporate a variety of use cases into the RFP so that we get more information back from vendors about how flexible and usable their software really is than we would get by simply asking them to complete a checklist of features.

One way to write a use case is to imagine one way you would like Blackboard (or other systems you use) to facilitate the work of instructors and students. It could be something Blackboard doesn’t do at all or that it should do differently. The key is to focus on a specific story or scenario that describes how the system would facilitate learning and teaching.

Some Notes on Writing a Use Case

Don’t worry about getting all the details of the story right. If you teach, start by describing a pedagogical scenario, including the objectives and activities or simply outline a process and list the players/actors in the story. For example, the objectives could be to develop student critical thinking and the activities may include peer-review. The players could be instructors, teaching assistants, students, and any others. Actors can also be components of a system like a bulletin board, gradebook, or calendar. If you are a student, you may want to start by describing a learning activity such as the need to submit a e-portfolio assignment, loaded with video, design and graphics. You may then want the instructor to review your project and insert video or other relevant feedback.

Don’t worry about how the system should work when there is a problem. Just focus on how people should work with the system in order to get the job done. Tell a story that ends in success. If we decide to add your use case to the RFP we’ll follow up with you to get more details if they are needed.

Here’s a relatively simple example of a use case followed by questions we are working on including in the RFP:

A professor wishes to add significant events to the online course calendar. The events include assignment release and due dates, extra test preparation seminar dates at times that are not in the official course schedule, test dates, and events that advise students when they should begin working on various stages of an assignment.

In some cases a course shell may include multiple course sections – each with different dates for each event.

In other cases individual students may wish to create events in their own course calendar.

1. Describe in detail the options, and any required workarounds, to accomplish all of these scenarios in your system.
2. Describe the options and any required workarounds to share this information with the student’s Ryerson Google calendar. Make sure you include as one option how a Google secondary calendar for the course could be populated and made visible to the student.
3. Ryerson already populates student’s primary Google calendars with course schedule information. If Ryerson wanted to ensure that the information provided by the LMS would not duplicate this information in a secondary Google calendar but did want the information in the LMS course calendar, how would that be accomplished?

How to Send in a Use Case

Please email your use cases to lms@ryerson.ca by Tuesday November 12 and please accept our thanks in advance!

– Learning and Teaching Systems and Services Consultation Committee