Student Outcomes
Chapter: Building a Cohesive ssessment strategy

Choosing, customizing and designing resources

From here there are three possible ways to proceed:

Choose from an existing array of resources

This method definitely has its merits. Firstly, there are enough number of organizations across the world who specialize in creating tools to track progress on student outcomes, therefore you can leverage the best of the expertise in design that is out there. Secondly, designing your own instruments is time consuming and may not have the validity and/or reliability of the existing resources which have been built over time. Let’s try a simple exercise to understand how choosing from an existing array of resources works.

Here is Pooja’s classroom vision:

My students will be hardworking and passionate about their work. They will relate their daily learning to their daily lives. They will be curious and will question the right and wrong behind arguments. They will be confident in the work they undertake be it academics or extra-curricular and have faith that with continuous efforts they will excel. They will be caring towards each other and respect their country, culture and people. They will be sensitive towards their surroundings and will make a conscious attempt to maintain and protect it. They will have an opinion of their own but will respect the opinions of others as well. They will be team players and will realize the strength in unity.

 

Here are a set of resources as examples of what Puja could use to track progress in her class. Think about:

  • What assessment strategy aligns with her approach in class and vision?
  • Do these resources perfectly align with Pooja’s vision for her students?
  • What are possible gaps between her vision and these resources?
  • What did you take away from the process, what was challenging, what was a delight and what would you advise others to keep in mind when they are trying to map alignment between vision and measuring outcomes?

 

The challenge you might have encountered is that many of these external resources are aligned to a different purposes.  The criteria measured by each resource has been defined differently as intended in the vision, therefore making it difficult to just plug and play a resource. What you encountered is not just the messiness of the landscape, but also experience firsthand the challenges encountered by different partners when they embark on the journey of finding tools aligned to their contexts and vision. What is most important is this exercise is a strong push on the vision in itself – what does the vision mean, how do the outcomes flow from it, how are each of the attributes defined on their own and in relation to the vision.

 

How might we evaluate resources when we choose them

criteria-for-choosing-resources

 

Customize existing resources to suit your needs

 

Let’s try our hand at customizing the same set of resources that we used in our previous reflective exercise:

Here is Pooja’s classroom vision again for your reference:

My students will be hardworking and passionate about their work. They will relate their daily learning to their daily lives. They will be curious and will question the right and wrong behind arguments. They will be confident in the work they undertake be it academics or extra-curricular and have faith that with continuous efforts they will excel. They will be caring towards each other and respect their country, culture and people. They will be sensitive towards their surroundings and will make a conscious attempt to maintain and protect it. They will have an opinion of their own but will respect the opinions of others as well. They will be team players and will realize the strength in unity.

Here are a set of resources as examples of what Pooja could use to track progress in her class. What are some of the changes, pressures and customizations would she as a teacher want to consider?

 

Some of the factors for customization are:

  • Translation into different languages
  • Adjusting for grade levels
  • Description of the criteria being measured
  • Reducing the number of survey items

What are the some of the other ways in which you would customize these resources?

 

While all of the above axes hold true for consideration when you pick resources to customize, you also need to remember that

a) Cultural relevance and differences is key when you pick resources to customize

For example, when Maria Kurisoo from Noored Kooli came across the Tolerance Scale that measures the level of tolerance that children/youth have for differences described as “a measure of comfort and acceptance of ethic and cultural diversity in people who are different than oneself”, her first reaction to it was,

There is no way I could use this tool in Estonia! In Estonia, most of the population is registered as atheists and for those who do practice some faith, religion is an extremely private affair.  So having a survey that mentions religion and God is an absolutely no-no.

b) There is an inherent conflict with construct validity when we customize literature backed tools to our contexts (be it translations or modifying the key questions of an instrument)

 

Designing your own resources

If you are looking to design your own resources and frame your assessment strategy here are a few things to keep in mind from our participants at the round table. For example:

Anna Choi from the OECD said at the round table,

Self-reported student responses are prone to many biases.  However these still provide some relevant information relatively easily and cheaply compared to other methods and with triangulation method using teachers and parental assessments, we can get more information on these elements.  Forced choices, situational judgment tests, performance test measures and games are other ways to measure skills and outcomes.  Some of these skills are difficult to conceptualize and difficult to meaningfully compare in different cultural contexts.  When developing these measurement tools, it is important to consider the different contexts that students face, whether it be cultural, religions, ethnicity, and societal norms. 

Iva Boneva from Centre for Inclusive Education said,

Questionnaires and focus groups with students, parents and teachers. Also – how often students attend school compared to before (or – % of the dropouts), do they have a favourite subject, do they have a life plan (here the best way is to conduct a focus group with students to be able to see what comes from the student and what – from the family), do children take interest in extracurricular activities, do children talk with their parents about what they want to do in the future, do they plan/talk about continuing studies, others. It is important to ask students, parents and teachers in the beginning, middle and at the end. Another source of information/verification sometimes are the people from the community – to ask them what are the role models/successful people for their community – this is quite telling about what forms children’s aspirations

If you are looking at assessments with the lens of a school or an organization it would be great to learn from Lauren Elgin from Summit Public School: 

At Summit Public Schools (SPS) we have a model in which courses are project based, with content learned through playlists on a what is called the Personalized Learning Platform (PLP). After working through a playlist of resources, the student requests approval to take a content assessment to determine whether they have mastered the content. We are able to track the number of attempts that a student makes prior to passing a content assessment. Many students may make multiple attempts prior to passing a content assessment. One thing we continue to work on is supporting students in strategy shifting, this is a natural place to practice and measure — if you don’t pass the first time what are you going to do differently to prepare to retake the assessment. What this allows us to do is develop interventions for individual students to support their approach to working through challenging content and persisting. We are able to see that with clear practices to model and support use of strategy shifting reduced the number of attempts a student made prior to passing a content assessment. While certainly there are many variables at play including the level of rigor of certain focus areas over others leading to a student perhaps passing the content more quickly, this is a clear measure that I used in many mentoring conversations with my students which I ultimately saw have a significant impact on my mentee’s behavior.

Below are also two checklists you could use to create your own surveys and rubrics:

6-things-to-remember-surveys

 

6-things-to-remember-rubrics

 

 

 


Continue Reading...

Please share your thoughts.

Notify of
avatar