S3-Post 02

Reflection on Practice

The MSc Applied eLearning has been a wonderful learning experience over two years, from September 2016 up until July 2018. It provided me with a learning framework to acquire new skills, knowledge and perspectives. The course allowed me to revisit areas of education, such as learning theories that I first encountered during pre-service teacher education and later built on during a MEd degree in Trinity College Dublin. Later, an understanding and appreciation of instructional design models provided me with the conceptual framework required to design, develop and evaluate eLearning resources throughout the two years. The timetabled tutorials and workshops guided me on my research pathway which ultimately led to my research proposal at the end of Year 1, and my Applied eLearning Project, comprising eLearning resource and journal paper, at the end of Year 2.

Learning Theories

During the lectures for this first module of Year 1, and in researching my paper, I greatly enhanced my knowledge and understanding of learning theories. I researched how the various learning theories can be applied to eLearning. Applications that utilise elements of gamification are often behaviourist in orientation (Zichermann & Cunningham, 2011). Behaviourism is the theory is that learning occurs as a result of appropriate behaviours being repeated and then rewarded. Examples of such apps that are prevalent in the education sector where I work (post-primary) are Socrative where students can race spaceships across the screen by answering questions correctly and Kahoot where students must quickly and accurately answer multiple-choice questions that appear on the teacher’s screen. They usually answer the questions on mobile devices. However, if a particular quiz is designed to encourage group or whole class discussion then it is possible that any learning that occurs can be explained in the context of social constructivism (Vygotsky, 2008).

The paper I submitted for the Learning Theories module in Year 1 was called The van Hiele Model and Learning Theories: Implications for Teaching and Learning Geometry. The research I conducted during the process of creating this paper greatly enhanced my understanding of how geometry is learned and how the van Hiele Model was adopted in the post-primary Mathematics syllabi in the US, Russia and Ireland. The model is set in the context of two of the major learning theories, cognitivism and constructivism. The van Hiele levels are similar to Piaget’s stages (Piaget, 1953). I already have a constructivist approach to the use of an application called GeoGebra to support the teaching and learning of geometry in my school. I intend to incorporate the van Hiele theory into future digital content that I create with this software application. Here is the link to this paper from my ePortfolio:

Instructional Design and eAuthoring

As part of the assessment of the second module of Year 1, I designed, developed and evaluated an eLearning resource called ‘Composting’ as part of a four-person group. I volunteered to carry out the development work for the eLearning resource, using Adobe Captivate 9. The three other members of my group took the lead on creating a storyboard, and with regard to subject matter expertise in the areas of recycling and composting. The eLearning resource created was an interactive multimedia tutorial, and the Captivate output chosen was HTML5 code. This HTML5 code output resulted in the eLearning resource being ‘responsive’. This meant that my group’s web based eLearning project worked across a range of devices including PC, Mac, iPad and iPhone. Here is the link to the eLearning resource developed for this module:

I enjoyed conducting literature reviews throughout the two years of the MSc course. This process began with Learning Theories during which I discovered the Van Hiele theory, which has relevance to the teaching, and learning of post-primary geometry (Usiskin, 1982). During the Instructional Design & eAuthoring module, I wrote five annotated bibliographies. I greatly enjoyed this activity, and it was a joy to discover two seminal papers at this juncture. These papers described e-Learning 2.0 (Downes, 2005) and a new learning theory, connectivism (Siemens, 2005). Here is the link from my ePortfolio:

Technology Enhanced Learning, Teaching, and Assessment (TELTA)

The TELTA module was fully online and this made it very interesting and challenging. Each week for eight weeks there was a project to be completed, sometimes individual and other times group-based. The group-based projects required collaboration and this was achieved using, for example, Blackboard and Coggle’s collaborative features. (Coggle is a mind-mapping application).

For Topic 1:  Exploring the Current eLearning Landscape of the TELTA Module, I wrote a response to the key reading What’s the use of a VLE. On reflection, I considered this to be an important piece of learning, as it enlightened me in relation to current thinking about, and use of, the VLE (or LMS). Given that much of the TELTA module involved using the collaborative technology integrated into Blackboard (DIT’s institutional LMS, ‘Webcourses’), it was interesting to discover that in a survey of DIT lecturers, 93% of them used the LMS for file sharing while only 5% used it to collaborate via webinars (O’Rourke, Rooney, and Boylan (2015).

Over the past few years, I have been trying to move my school towards adopting a single institutional LMS. During that time, a variety of LMSs including Edmodo and Schoology have been explored and trialled. For one of the individual projects in this module, I developed a screencast demonstrating how to set up new users on Moodle. I did this because I was interested in encouraging my work colleagues to migrate to Moodle as the school’s LMS of choice. I used Adobe Captivate 9 to create the screencast, and the output option chosen was a single video file. I then published this video on Vimeo, and I also uploaded it to my ePortfolio.

It was interesting to use Slack to collaborate with the other students and the lecturers during this module. I had used Slack during the previous module (Instructional Design), after it was recommended to me by one of my group members. I used Coggle as part of a group project to create a mindmap that looked at motivation as an issue for online learners.

At the end of the TELTA module, I had to submit an annotated mindmap as a capstone task, indicating where, why and how technologies could be integrated into my teaching, learning and assessment practices. The purpose of this was to enhance, modify and transform the teaching, learning and assessment experiences of the student. I designed an eight-week module called Synthetic Geometry for Transition Year Students, and technology was heavily embedded into this proposed module. I created an annotated mindmap using Mindomo to demonstrate how the devices, apps, learning theories, research, syllabus and scheme of work could be integrated for the teaching, learning and assessment of Transition Year Geometry. Here is the link to the assessments for this module:

Research Proposal

I had spent a lot of time working on my research proposal during Year 1 of the MSc and this work paid off. I was very happy with the proposal that I submitted on 29/06/17. Moreover, the feedback I received in the ‘Research Proposal Marksheet’ was very positive. The deadline date of 26/09/17 was easily met. One of the lessons learnt was that preparing a research proposal takes a considerable amount of time. The other lesson learnt was that this time and effort is worth it because the resulting document is a very good roadmap for the actual research project.

Independent Research related to my Workplace

I am the ICT Co-ordinator in my school, and this is classified as an Assistant Principal post of responsibility. At the end of Year 1, I decided to carry out some research relating to a new Digital Learning Framework for Post Primary Schools.

Butler et al (2013) recommended that the UNESCO ICT Competency Framework for Teachers be used to guide schools in the implementation and review of the Digital Strategy for Schools: 2015 – 2020 (Department of Education and Skills, 2015). For example, under the heading technology literacy, objective TL.4.h. requires teachers to “Describe the function and purpose of tutorial and drill and practice software and how it supports students’ acquisition of knowledge of school subjects.” (UNESCO, 2011, p. 23).

This UNESCO document was customised for Irish post-primary schools and the document Digital Learning Framework for Post Primary Schools was made available to all schools for the 2017/18 school year. The framework “…provides a common reference with descriptors of digital competence for teachers and school leaders promoting innovative pedagogical approaches which embed the use of digital technologies.” (Department of Education and Skills, 2017, p.1). “The Digital Learning Framework for Post Primary Schools supports and complements the SSE process in relation to embedding digital technologies into teaching and learning.” (Department of Education and Skills, 2017, p.1).

The framework is directly aligned to the domains and standards of the document Looking at our School 2016 – A Quality Framework for Post Primary Schools. This document “articulates effective and highly effective practice for the use of digital technologies in the same two key dimensions Teaching and Learning and Leadership and Leadership and Management (Department of Education and Science, 2017, pp. 1-2).

Applied Research Project (Year 2)

I designed, developed and evaluated five domain model artefacts for a possible future adaptive learning system during Year 2. This provided me with some great insights into the relationships between concepts, topics and learning outcomes. The evaluation process, especially the interviews with key informants, was invaluable in informing me about the strengths and weaknesses of my artefacts. The evaluation provided evidence that the screencast I developed to explicate these artefacts was very effective.

To properly evaluate eLearning, it is important to consider what the literature can tell us. To this end, ten main categories of literature can be identified. These include descriptive case studies, comparisons with traditional learning (often using matched pairs design), tools and instruments for evaluating eLearning (e.g. student perception questionnaires or data analytics from LMS usage), Return on Investment (ROI) reports and benchmarking models. They also include product evaluation, performance evaluation (or student assessment), handbooks (often using a management model of evaluation whose primary aim is to provide feedback to influence e-learning implementation and future development), meta-studies (based on US literature) and studies on the contribution of evaluation to metadata (Attwell, 2006).

I was happy to be able to write a journal paper that had a coherent thread that connected the research questions to the research design, and right through to the results and conclusions.

References:

Attwell, G. (2006). Evaluating E-learning: A Guide to the Evaluation of E-learning. Evaluate Europe Handbook Series2, 1610-0875.

Butler, D., et al. (2013). A Consultative Paper Building Towards a Learning Society: A National Digital Strategy for Schools. Retrieved from http://www.education.ie/en/Schools-Colleges/Information/Information-Communications-Technology-ICT-in-Schools/Digital-Strategyfor-Schools/Building-Towards-a-Learning-Society-A-National-Digital-Strategy-for-Schools-Consultative-Paper.pdf

Department of Education and Skills. (2015). Digital Strategy for Schools: 2015 – 2020. Retrieved from https://www.education.ie/en/Publications/Policy-Reports/Digital-Strategy-for-Schools-2015-2020.pdf

Department of Education and Skills. (2016). Looking At Our School 2016: A Quality Framework for Post-Primary Schools. Retrieved from https://www.education.ie/en/Publications/Inspection-Reports-Publications/Evaluation-Reports-Guidelines/Looking-at-Our-School-2016-A-Quality-Framework-for-Post-Primary-Schools.pdf

Department of Education and Skills. (2017). Digital Learning Framework for Post-Primary Schools. Retrieved from http://www.pdsttechnologyineducation.ie/en/Planning/Digital-Learning-Framework-and-Planning-Resources-Post-Primary/Digital-Learning-Framework-for-Post-Primary-Schools.pdf

Downes, S. (2005). E-learning 2.0. In eLearn Magazine. New York: ACM.  Available http://elearnmag.acm.org/archive.cfm?aid=1104968

O’Rourke, K.C., Rooney, P. and Boylan, F.  (2015). What’s the Use of a VLE? Irish Journal of Academic Practice, 4(1),11.

Piaget, J. (1953). The Origin of Intelligence in the Child. London: Routledge and Kegan Paul.

Siemens, G. (2005). Connectivism: A Learning Theory for the Digital Age. International Journal of Instructional Technology and Distance Learning, 2(1), 3-9.

UNESCO (2011). UNESCO ICT Competency Framework for Teachers Version 2.0. Retrieved from http://unesdoc.unesco.org/images/0021/002134/213475E.pdf

Usiskin, Z (1982). Van Hiele Levels and Achievement in Secondary School Geometry. Retrieved from http://ucsmp.uchicago.edu/resources/van_hiele_levels.pdf

Vygotsky, L. S. (1978). Mind in society: The development of higher psychological processes. Cambridge, MA: Harvard University Press.

Zichermann, G., & Cunningham, C. (2011). Gamification by design: Implementing game mechanics in web and mobile apps. ” O’Reilly Media, Inc.”.

 

S3-Post 01

Year 2 Project Reflection

Limitations of the Applied Project

It was important to be mindful from the outset of the time limitations imposed by the overall research project, and to adhere and adjust throughout, to the Research Proposal Timetable (Table 1) and LTTC Guidelines / Deadlines (Table 2). Although the overall time available to expedite the entire project (10 months) was short, and meeting the various milestones and deadline dates demanding, it was possible to achieve all of these without unduly compromising on integrity or quality.

Possible Methodological Limitations
As detailed in my research proposal, I suggested that I would invite teachers from the Irish Mathematics Teachers’ Association (IMTA) to participate in my survey. I had hoped “to get upwards of fifty responses” using this method.

Prior to implementing the IMTA survey, I decided to carry out a ‘test survey’ using Mathematics teachers from the post-primary school where I worked. I received 10 responses between 27/02/18 to 03/03/18. Having reflected on the ‘test questionnaire’ and responses, I decided to add six extra questions for the IMTA survey. It took from 05/03/18 to 06/04/18 to get 26 responses from IMTA participants.

Since the number of participants was lower than the anticipated 50 responses, I decided to combine the two surveys so that I had 36 responses to the common 30 questions in both questionnaires, as well as 26 responses to the extra six questions. It was also necessary to ‘manually’ merge the two sets of data that had each been imported from Google Forms to Microsoft Excel. I would like to think that the ‘additional’ 10 responses, and the improved survey, mitigated for the use of dual source (IMTA and school) participants and two slightly different questionnaires.

Lessons learnt from the various project milestones and phases

Figure 1: Gantt Chart (September – December 2017)

Milestone 1: Finalising your proposal and schedule (Deadline: 26/09/17)
I had spent a lot of time working on my research proposal during Year 1 of the MSc and this work paid off. I was very happy with the proposal that I submitted on 29/06/17. Moreover, the feedback I received in the ‘Research Proposal Marksheet’ was very positive. The deadline date of 26/09/17 was easily met. One of the lessons learnt was that preparing a research proposal takes a considerable amount of time. The other lesson learnt was that this time and effort is worth it because the resulting document is a very good roadmap for the actual research project (Table 2).

Milestone 2: Ethics Submission/Approval (Deadline: 31/10/17)
I began work on my Ethics Submission on 04/11/17 and persevered with it for 11 consecutive days. I made an online submission to Dublin Institute of Technology’s Research Ethics and Integrity Committee (REIC) on 14/11/17. The lesson learned was that I had to carry out work on my project that I had planned for later in the year at that juncture. For example, fairly detailed information was required by REIC in relation to the survey and interview questions. I discovered that the work that I had to carry out at this juncture for my ethics submission really paid off later in the year. Approval was granted for this research to be undertaken by email on 06/02/18 (Table 2).

Milestone 3: Literature Review and Interim Report (Deadline: 17/11/17)

A very comprehensive literature review in relation to domain model and adaptive learning had already been carried out for the Research Proposal. Consequently, this particular literature review, submitted on 10/12/17, dealt with conceptual understanding of a function.   There were two noteworthy pieces of literature that I read. The first was the description of “function concept” as “one of the most important topics in high school mathematics” (Dubinsky & Wilson, 2013). The second related to the results of an experimental study suggesting that conceptual and procedural knowledge in mathematics should be taught iteratively (Rittle-Johnson, Siegler, & Alibali, 2001).  The interim report was submitted on 19/11/17 (Table 2).

Milestone 4: WIP Presentation 1 (28/11/17)

The formative feedback that I received the day after my presentation was very positive. One of the things that I have learnt as a student on this course is the value of timely formative assessment in relation to my personal learning and motivation. I learnt that I included too much technical detail, and I didn’t tell the overall ‘story’ of my project as well as I could (Table 2).

Phase 1 – Design and Develop Digital Artefacts (01/09/17 to 31/12/17)
At the time of the Research Proposal, I envisaged using a single domain model authoring tool (GAT) from the GRAPPLE project (GRAPPLE, 2011). Instead, I delayed the start of the development work on my domain model from early September to early October 2017, while waiting for work to be completed on the successor to GAT, the GAM Authoring Tool (GAM AT).   Ultimately, I ended up using three different authoring tools: GAM AT, Mindomo, and Rhumbl Maps (Craenen, 2017; Mindomo, 2018; Rhumbl, 2018). I feel I learnt more about different design and development approaches to domain model authoring by using different tools. Moreover, the different design paradigms associated with these tools led to different domain model designs that provided me with a greater insight into domain modelling. Microsoft Access and Excel were the two software applications that were of great assistance in the learning outcome decomposition process. Generally, I found myself working iteratively with these five tools/applications. The five domain model artefacts (three Mindomo, one GAM AT, one Rhumbl Maps) were developed between 07/10/17 and 15/02/18 (Tables 1, 3).

Phase 2 – Design online survey, interview format, topics/questions (01/09/17 to 31/12/17)
The early versions of the survey questionnaires and interview themes were created during the ethics submission preparation period between 04/11/17 and 14/11/17. I learnt that it was valuable to have draft versions of these created early in the research project. This made it easier to improve their quality later in the project. The final versions of the questionnaires were ready by 27/02/18 (Mk 2 – School) and 05/03/18 (Mk 3 – IMTA) (Tables 1, 3).

Figure 2: Gantt Chart (January – June 2018)

Milestone 5: Implementing and evaluating the project with your target user group (16/03/18)
This involved completing Phases 3, 4, 5 (see below) (Table 2).

Phase 3 – Implement online survey of IMTA members (01/01/18 to 31/01/18)
The project seemed to be behind by about one month due to a number of factors. One was the delay in availability of GAM AT until early October 2017. A second factor was the decision to use two more authoring tools (Mindomo and Rhumbl Maps). The third was the unexpected publishing of a new draft syllabus for Junior Cycle Mathematics in November 2017 (NCCA, 2017). These factors conspired to delay the end of the domain model authoring process, and the beginning of the evaluation process, until 15/02/18 (Tables 1, 3).

Before the online survey and interviews could take place, one further artefact needed to be created. This was a screencast explicating the learning outcome decomposition process, and the creation of the domain model artefacts, for the all participants in the research study. I wasn’t aware that I was required to create the screencasts, as well as the domain model artefacts, until a supervision meeting. The lesson learnt was that the screencast turned out to be both useful and necessary in relation to educating the participants about learning outcome decomposition, domain models, and adaptive learning so that they could answer the survey and interview questions. The screencasts were developed using Adobe Captivate 9 between 23/01/18 and 04/03/18 (Tables 1, 3).

A total of 134 emails were exchanged between the IMTA and me between 19/11/17 and 05/04/18 with the two surveys completed between 27/02/18 and 06/04/18 (Table 3).

Phase 4 – Implement interviews with key informants (01/01/18 to 31/01/18)
My sample was key informants and there were no major problems with sourcing these. Most of the interviews took place between 06/03/18 and 21/03/18. Nine key informants agreed to be interviewed, six of who were on the list in my research proposal. I decided to use two extra Mathematics Educations Professionals (MEPs), sourced from Project Maths Development Group (PMDG) and National Council for Curriculum and Assessment (NCCA), instead of two Mathematics teachers. The two MEPs had a lot of experience with designing and implementing new Mathematics syllabuses for Ireland. The main lesson learnt was that it takes a lot of time to organise these interviews. I used ‘snowballing’ for one key informant. I used a third party to send the initial email to another key informant, rather than ‘cold-calling’ (Tables 1, 3).

Phase 5 – Data Analysis (01/01/18 to 31/01/18)
I carried out all of the transcribing of the interviews myself. This took an enormous amount of time. It took in the region of 12 to 15 hours to transcribe one hour of interview audio. The transcribing process took all of the month of April, including working full time at it for two weeks during the Easter holidays in April 2018. The Google Forms data analysis was greatly assisted by Microsoft Excel. It still took a considerable amount of time, especially having to merge two surveys and triangulate with the interview data. The actual analysis of the data by theme and by question mainly took place during May 2018 (Tables 1, 3).

Milestone 6: WIP Presentation 2 (15/05/18)
The formative feedback that I received the day after my presentation was quite positive. However, it appeared that I could have provided more information about the findings and responses of the participants, and provided less detail on the technical details of the project (Table 2).

Milestone 7: Applied e-learning Project Submission (24/05/18)
This was submitted to the Supervisor and First Reader well in advance of the deadline date 4th June 2018 (Table 2).

Milestone 8: Journal Paper Submission (09/07/18)
This was emailed to the Supervisor and First Reader a week after the deadline date of 2nd July 2018 having been granted an extension of one week (Table 2).

Milestone 9: e-Portfolio Submission (09/07/18)
The URL was submitted to the Supervisor and First Reader a week after the deadline date of 2nd July 2018 having been granted an extension of one week (Table 2).

Tables

References

Craenen, M. (2017). GAM Authoring Tool: An authoring tool for GALE. (Master’s thesis), Eindhoven University of Technology, Eindhoven, Netherland. Retrieved from https://pure.tue.nl/ws/files/91106245/Craenen_Marc_Thesis_CSE.pdf

Dubinsky, E., & Wilson, R. T. (2013). High school students’ understanding of the function concept. The Journal of Mathematical Behavior, 32(1), 83-101. https://doi.org/10.1016/j.jmathb.2012.12.001

GRAPPLE. (2011). Welcome to the GRAPPLE project Website. Retrieved from http://grapple.win.tue.nl/home.html

Mindomo. (2018). Collaborative mind mapping, concept mapping, outlining. Retrieved from https://www.mindomo.com/

NCCA. (2017). Draft Specification for Junior Cycle Mathematics. Dublin: NCCA Retrieved from https://www.ncca.ie/media/3163/jcmathematics_draft_specification.pdf

Rhumbl. (2018). Interactive data visualization for exploration, engagement and analytics. Retrieved from https://rhumbl.com/

Rittle-Johnson, B., Siegler, R. S., & Alibali, M. W. (2001). Developing conceptual understanding and procedural skill in mathematics: An iterative process. Journal of Educational Psychology, 93(2), 346-362. https://doi.org/10.1037/0022-0663.93.2.346