Tutorial 7: Resources
Tutorial 6: Beginning the First Cycle
Tutorial 7: Collecting Data
A. Creating a Data Collection Plan
Research methods are a specific set of procedures or tools for understanding more about how your action influenced the actions of others. You will need to decide what kind of artifacts you are going to collect. To make your decision, revisit your cycle research question. The first part should describe the action you plan to take and the second part the outcome you expect to effect. Now look at your outcome and ask yourself what sort of data would help you see the outcome from the eyes of others? How will you get past your own frames of thinking that shape how you "see" the outcome? In action research, it is good to collect more than one form of data and see if, when together, they help you understand what happened from multiple perspectives. In collaborative action research, you may be collecting data in a team.
Here is a list to help you think about the artifacts you might collect:
Perspectives of others: What do they think or say?
Surveys or questionnaires (students, parents, co-workers)
Focus groups/ informal discussions/student reflections,
Case Studies or subsamples of participants
Observations of Performances of others: What do they do?
Students' blogs or journals
Assessments of skills (tests, quizzes, homework, report cards, grades)
Rubrics (student created) and student portfolios or projects (see the Buck Institute for examples)
Checklists of different actions
Number of interruptions, disruptions to learning
Logs of number of meetings
Minutes of the meetings
Photos/videos or other forms of visual data
audio recordings (for analysis of ways of talking)
Attendance or online participation, or completing rates
diaries or journals, field notes
Request for assistance or information
Sociograms (who gets help from whom)
Number/type of requests for help
Number/type of questions asked
It is important to make your stance clear. You are not trying to be an objective bystander beyond the politics of the situation. Instead, you are trying to effect change and you are likely to have strong desires to have the change result in specified outcomes. Your evidence is not meant to "prove to others" that your actions caused the outcomes. Rather it is for you to understand, at a deeper level, how your actions contributed to the outcome. Your honesty is to yourself first and to others second. You want to know what about a given practice is working. If you lie to yourself by overestimating the outcomes, you will fail to learn as much as you might. You are trying to figure out what actions lead to what reactions. Your care in looking at the outcomes from many different perspectives will be vital to your learning.
With this understanding, you are free to collect that type of evidence that will help you understand what is happening. You can use traditional social sciences methods and you can creatively figure out what evidence will help you understand the change.
Here are some examples:
B. Learning How to Collect Data
Action Researchers collect many different forms of evidence, photographs, logs, journals, structured observations. Some of the data collection strategies overlap with evaluation research. The Online Evaluation Resource Library OERL has created a set of innovative tutorials to help researchers new to data collection learn some of these research strategies. While these efforts are meant to train evaluation researchers, they can be very useful for action researchers who decided to use one or more of these methods. Each of the OERL modules contains (1) step-by-step strategies and tools of evaluators (2) a scenario which helps you see how to apply the tools to an evaluation problem and 3) a case study in which there are questions to be answered (and then compared to peers and experts). You do not have to leave your names and if some of the methodological concepts are new to you, don't panic. Take the learning you need from the tutorials.
Here is the list--As you can see, many of these will be of use to action researchers.
These tutorials will help you learn to write a simple and brief survey or develop a protocol for an interview that matches your research question and helps you avoid simple mistakes (for example combining more than one question in a single item).
* Designing an Evaluation:
.......Methodological Approach and Sampling
* Developing Written Questionnaires:
.......Determining if Questionnaires Should be Used
* Developing Interviews:
.......Preparing an Interview Protocol
* Data Collection:
.......Procedures, Schedule, and Monitoring
* Instrument Triangulation and Adaptation
* Developing Observation Instruments
THE LISTENING RESOURCE-- A blog
The Listening Resource blogs by Susan Elliot are very useful for developing strategies in qualitative research.
If you are creating surveys, make sure that you have permission and consent to collect this information. If your work involves distributing surveys at the end of training sessions, or student activities that involve reflection, then you might not have to take any special provisions. However, if you are collecting information from protected groups that are not a part of their educational process, it is imperative that you have the proper oversight (from your principal, district and/or university) and that you obtain participant consent, and if children are involved, parental consent.
See the discussion of ethics in action research.
C. Building a Valid and Reliable Data Collection Plan
You will need to think about how to assure that the data you collect is a valid measure of your action and that your method of analysis was reliable or a reasonably accurate representation of the data. This chapter by Richard Sagor in Guiding School Improvement with Action Research will help you think through some of the issues around the validity of evidence to measure what you are exploring or the truthfulness of the selected data. The reliability of the measures address issues of accuracy -- did you summary the data to report the date to others in a way that adequately represents what happened.
D. Building Your Knowledge about Research Methods
Whenever possible, it is better to use an instrument that has been validated by others. Searching online can help you find data instruments, but here are some lists to get you started.
National Institute for Learning Outcomes Assessment (NILOA) has as their mission to "discover and disseminate ways that academic programs and institutions can productively use assessment data internally to inform and strengthen undergraduate education, and externally to communicate with policy makers, families, and other stakeholders." This is another great source for figuring out what data to collect and what you learn is likely to also apply to other learners with some adjustments.
The High School Survey of Student Engagement (HSSSE) might be of use as many teachers are concerned about student engagement.
Here is a list of resources for finding assessment tools (mostly in science, technology, and engineering) have been copied from Purdue University's Assessment Center
The following resources provide further information on other databases, additional assessments, research journals.
ATIS: Assessment Tools in Informal Science - Another database of assessment tools. Certain assessments in the Assessment Center link to ATIS as they can be found on their site
AWE: Assessing Women and Men in Engineering - Provides assessment tools for K-16 formal and informal education outreach activities.
Burros Institute of Mental Instruments - A searchable database of evaluations of tests.
The Buck Institute for Education has many great resources for project-based learning and ways to assess the outcomes with rubrics.
ETS Test Collection Database - A searchable database of tests.
MCAS: Massachusetts Comprehensive Assessment System - A database of assessments and questionnaires that test knowledge on technology, science, and engineering.
NAEP: National Assessment of Educational Progress - Largest nationally representative assessment of math, reading, science, writing, the arts, civics, economics, geography, and US History.
PARE: Practical Assessment Research and Evaluation - An online journal that provides access to refereed articles that can have a positive impact on assessment, research, evaluation, and teaching practice.
Tech Tally: Approaches to Assessing Technical Literacy - This book will be of special interest to individuals and groups promoting technological literacy in the United States, education and government policymakers in federal and state agencies, as well as the education research community.
STEM Education Instruments - The ITEST Learning Resource Center has compiled information on various instruments to help researchers, evaluators, and practitioners, identify and locate instruments used to assess learning and other related outcomes in STEM learning environments.
E. Sharing your Action Research Report- Tech Tools
Blogs and Websites are much easier to construct than with earlier software. Using basic word processing skills you can share your action research with a larger audience by posting your work on a website. Many universities support their alumni by posting their work both during and after their program.
While this list is not exhaustive, it will give you some ideas to get you started:
To Create a Website with word processing skills (no programming skill needed) try using one of these tools:
Wix -- Wix is easy to use and it has a free version, and if you want a more professional look, it has a reasonable rate. The downside is the apps that make it more interactive all have a cost associated with them.
Google Sites -- Google sites, both old and new is a free tool. Google is currently redesigning Sites (now called classic sites) however, new sites is under construction and is slowly developing even basic options. Currently, there is limited ability to control the style, size, and color of text.
Weebly -- Many find this the best choice. It shares many of the same features as Wix. Wix appears slightly easier but the ease will be determined by which is more similar to programs you have used in the past.