November 15, 2007
Course Analytics, Part 3 - Assessment
In the previous parts of this series (prequel), I have looked at Visitor Behaviour, When Are Our Students Online?. In this part, I shall briefly review user activity around assessment. The 10 week course is assessed by a short, multiple choice computer marked assessment 5 weeks in, and a more substantial written end-of-course assessment in the final week.
Measuring activity based on the number of pageviews, we see spikes in activity - as indicated by pageviews - just prior to the two assessment deadlines (the assessment dates were midnight on Tuesday June 5th and midday on Tuesday July 10th).
The Top Content Summary paints a similar picture:
The 'course assessment' page links to both the CAM and the ECA. Here's how students navigated to - and from - that page.
The CMA, deadline Tuesday June 5th
If we define reports that display pageview activity for just the pages relating to the CMA, we see a peak in activity around the time of the CMA submission date - Tuesday June 5th - students are completing the assessment at the 'scheduled' time.
What is quite surprising is the short average time spent on page; it would be useful to be able to plot the actual distribution of time spent on this page to see whether there is a blip at longer durations.
It would also be useful to know how many people printed off this page.
A summary statistic identifies the pages from which the users directly reached the CMA page:
We see a high proportion coming from the Course Assessment page, as we might expect, but also from the front page and the course calendar, as well as the precursor teaching material (Lesson 4, Conclusion).
We can check the entrance to the CMA page from the Navigation Summary report, which also describes where users went after leaving the CMA page.
The high percentage of exits - people who left the site from this page - is worth noting. A quarter of students completed the assessment (or printed it off, maybe, and then left the site.
By setting the CMA page as a goal, it is possible to use Google Analytics to further break down the path users took to reach the CMA page:
The {entrance} denotes the point at which users entered the site. We see quite a high proportion of users entering the site and going directly to the CMA page (that is, not going via any teaching material pages).
The ECA, deadline Tuesday July 10th
The same pattern of behaviour is observed for the end-of-course assesssment: there is a peak in activity (as measured by pageviews) in the few days leading up to the ECA cut-off date, around the time the ECA is scheduled in the course calendar:
Looking at the daily breakdown of visits to the ECA page in the immediate run up to the ECA (the cut-off date was midday on Tuesday July 10th), we see a peak of activity on the Sunday and Monday immediatley before the Tuesday cut-off. There also appears to be a significant amount of ECA related activity on the previous Sunday (July 1st).
Once again, the average time on page result does not appear to be that useful, although it may be possible to correlate or otherwise combine it with other, visitor data sets?
The list of originating pages mirrors that found for the CMA page:
The Navigation Summary breaks this down further, and again shows a high proportion of exits:
By defining the ECA page as a Goal page, we can tunnel into the navigation path leading to the ECA page in a little more detail:
Once again, we see a short path between site entry and reaching the ECA page - students are coming to the site to specifically engage with the assessment material.
In the next installment, I shall look at search behaviour with the course site...
Blogged with Flock
Tags: courseanalytics, assessment, elearning
Posted by ajh59 at November 15, 2007 02:54 PM