I love Datacamp. They have a platform that has been extremely successful at democratising data science knowledge. This is no mean feat in an emerging field and is a laudable goal as well. Earlier in the year I had the chance to develop a programme for DataCamp around neural networks, specifically using Keras in Python. It was immensely fun and gave me an insight into the process of design and development that the company undertakes. This review of that process serves a couple of purposes. Firstly, it allows me to showcase my design thinking around learning experience with some materials which aren’t the sole property of the organisation for which I work and secondly it may benefit that platform or others in thinking through how they do the design and development of their online learning platforms moving forward. You can find the Github repo with the materials here. I was asked to produce a track proposalfor them, then to choose one of those courses from the track and break it out in more detail, and then to do the same with a lesson within that course. This seems like a great way to get a flavour of the design and development decision making process.
Designing as an Andragogy Expert, not a Subject Specialist
Designing a DataCamp course was a fun challenge for two reasons. Firstly, I didn’t know Keras, and secondly, designing for an online platform which already has a range of material limitations is always interesting. Nevertheless, I do know adult education, and this has been an opportunity to demonstrate the value of employing someone with strong andragogic knowledge over a subject specialist for the initial design work. This isn’t to say that a subject specialist wouldn’t have helped at the development stage and having them to consult with at the design stage would have benefitted me as you can see from my design notes. Nevertheless, competent, experienced design work is valuable too, and tends to be sorely lacking in both face to face and online learning provision design as we increasingly look to subject specialists and assume that because they know their subject they will know how best to teach it. This is something which universities have realised in the last decade and worked to redress, and it is something that I hope the professional education sector will come to understand too.
My Gestalt Design Process
How I design a course is different from how DataCamp wanted me to work, so after contorting myself for a few hours, I chose to ignore their process for a bit and work my own way before translating it back into their preferred format. In an ideal world, their system shouldn’t ask that of the designer, but it is hard to work within everyone’s ideal and still get the scale and responsiveness the platform needs to thrive. Therefore, I was expecting some restrictions like this. However, as no explanation was given for why those restrictions were in place, I chose to move outside their process. I may have been wrong, but at the time I read this as a lack of thought on the andragogic development process rather than a conscious decision taken. I fully recognise that I can only review this from my position as a user-developer, so this critique should be taken from this perspective only.
I began by writing aims. These are broad, statements of intent which characterise the change in status in the person undertaking the learning experience. For example:
• The Learner can identify and employ common deep learning techniques in a variety of contexts.
Then I operationalised these aims into Objectives. Taking the aim above, related objectives could be:
• The learner is able to explain the value and application of deep learning in a professional context.
• The learner demonstrates familiarity with the basic functions of Keras as a tool for deep learning model development.
• The learner has experienced making a feed-forward neural network which has demonstrable predictive power.
Out of these I develop Learning Outcomes. These are measurable things which will demonstrate that learning is likely to have taken place and are tied to specific tasks. Considering the first Objective, learning outcomes could include:
• Other learners agree that the description of how the learner would find value in applying a deep recurrent neural network in the context of satellite orbit planning is prescient.
I use this model along with many other practitioners, because one of the things I’m looking for when I assess the quality of provision is to see whether the learner and any facilitators can see the point of what they are doing at each moment. With the aims/objectives/learning outcome model this gives the chance for that information to be referred back to at any time. Whenever necessary, you should be able to say ‘this task asks x of me which is what the learning outcome y is looking for, and this is being looked for because one of the objectives of this is z and that will make me a better r which is one of the central aims of the course’.
Still, it was time to get my head out of the sky, and go back to the DataCamp design framework. That meant I needed to write a tonne of learning outcomes. On DataCamp, everything is a learning outcome. Thus, some of them were achievable and measurable, others were not depending upon what was asked for and their context. Inevitably this means that there were dependencies within them (that is, one may be a subset of another), but this was not made explicit. I relented to my own judgement here and at least labelled mine to indicate a hierarchy. It allowed me to keep track of what was what a little better and should someone else be looking to reverse engineer my design choices, it allowed them better navigation. You can see these contrasted in the task, track and course proposal documents. It wasn’t just a matter of copy/pasting them though, I also needed to drop my expectations of what was measurable and work within the chapter/course/track framework.
Developer Flow
A momentary aside if I may: While I had my issues with the DataCamp framework, In terms of the content itself and the way it was required to be presented, I loved having version control in Github and writing in markdown – this is going to bridge the gap between data science practice and resource creation for those usually building the courses, and so I admire the ‘bringing the mountain to Mohammed’ mentality that this flow has demonstrated.
Content Development
So, I had a pile of learning outcomes, and I needed to develop the framework around the course. This is where DataCamp’s Task/Lesson/Chapter/Course/Track nomenclature comes i. More can be read about their guidelines here, but to quote from them:
• A course should consist of 4 chapters.
• A chapter should consist of 3-4 lessons.
• Chapter 1 should have 3 lessons.
• A course should consist of 44 - 60 exercises.
Before I was lacking structural clarity before with the learning outcomes, but not here! I knew exactly what was wanted. This has pros and cons. It was good insofar as the framework time-binds the provision. Once you have planned for 44-60 exercises, you are done. Once you have 3-4 lessons you are done with that chapter. This likewise makes it easy to review and means that participants of multiple courses across different languages or packages can develop expectations for a new piece of provision and have those expectations met. On the other hand, I found this framework restrictive when I was unable to have much flexibility on the weighting of tasks. When I am apportioning the material structurally, one of my concerns is the narrative experience of the learner. This narrative is vital in ensuring that the learner and facilitators always understand why a task is being completed. This is even more relevant in an online context when the learner can simply choose to stop engaging at any point if they feel so inclined. This is hard to develop fully when you have to deliver x lessons with y tasks across z chapters and there is no opportunity to deviate for any reason. I did my best to frame the narrative within each chapter, using the first to indicate how each built its skill base on the last, but this created awkward prerequisites which makes it harder for people to step in and out of the delivery at the chapter they most wish to explore and which could have a negative impact on the uptake.
Assessing the Learning
Finally, I had to consider the assessment. How would I know that someone had learned? The frustrating thing for me was that the platform measures task completion rather than skill development and doesn’t afford the designer the chance to make tasks which are as context grounded as they could be. For instance, having a task completion time limit would have enabled course 2, chapter 4, lesson 3 to feel more contextually grounded. Likewise, this strict activity division means that some individual tasks are harder to justify to the learner than they might otherwise be if they were more joined up with other parts of the workflow. A good example of this was asking students to prepare sample data for use, which taken alone could be seen to be unrelated to the learning outcomes. Without the rapport that one can build in a face-to-face learning environment, getting past these ‘humps’ is harder and could be a dropping point for some participants.
Another issue with the task completion mindset is that it fails to encourage help-seeking behaviour which has been identified as key to adult learning. Even the most basic cross-validation work means that learners are siloed. This may be acceptable in a world of Stackoverflow, but denies the possibility of growth through social interaction or the ability to assess anything other than quantitative data. My final issue with the assessment is that it denies freedom of expression. Ths is important in establishing if someone has learned about a task: It is one thing for my son to build a lego model brick by brick under my instruction to match my example and quite another to have him see my three-headed-duck-car creation, think for a bit and build a mountaintop temple. In educational theory terms, it disallows the assessment of learning towards the top of the hierarchy of Bloom’s Taxonomy, and these are the skills that will be most valuable in the data science sector where technology changes quickly and adaptability through material synthesis is a key driver of success. I’m sure some will complain that coding is a quantitative exercise and I shouldn’t try to impose my wishy-washy social science-based theory onto their task completion and that is true to a point, I accept that this sort of instruction is good for quantitative consideration. However even embracing that and writing off any consideration of assessment beyond task completion, I found myself looking for ways to defensively write the tasks so that users would be inclined to complete operations in Keras rather than other ways they knew such as data pre-processing in Scikitlearn and Word2Vec. Overall, I believe that It is a design decision which may be giving up too much assessment equity for conformity and as such calls for more consideration.
Ultimately you can figure out for yourself how well I navigated all the challenges of developing within the DataCamp framework. I found the lack of distinction at the ideation stage challenging, and the narrative building within the learning experience design and assessment trying. On the other hand, I loved the developer-user flow, loved the course/chapter/lesson/exercise specification and given a little more flexibility in their delivery I would be excited to develop learning using them again. You can see more design notes from me in the repo which is here and if you would like me to work on your learning projects, you can contact me to speak further.