Research Tools
We are aware of the problems involved in administering too many assessments and measures at one time. As such, we selectively administer each semester.
Open-Ended Questionnaires (Students):
Dear Incoming Student Letters: Students are asked to write, anonymously if they wish, a letter to future students of the course which begins with the prompt, ‘If you want to be successful in this class, this is my advice,’ Students may define ‘success’ in any way they choose. Administered to every class at the end of each term. We have also done a random sample of the most frequent responses to the "Dear Incoming Student..." prompt.
“Why I have persisted in this class…”: This prompt is administered frequently, after the second of three exams in the course (at which point approximately 50% of those initially enrolled have usually dropped the course). The question asks students for their own views of why they persisted in the class and did not drop.
“What is a Professor’s job?”: An open-ended prompt in which students are asked to describe what they believe to be a college professor’s most important teaching responsibilities. Administered frequently, at the beginning of the term.
Survey of Expected and Achieved Exam Grades: After students turn in their exams, we ask each of them to submit a piece of paper with their name and their expected grade. We compare these against the actually earned grade, and ask students to explain discrepancies. This survey is administered occasionally.
De-Briefing Interview: These semi-structured interviews are administered on a voluntary basis to students who have persisted in our courses. Topics focus on common themes found in campus mythology about our courses and students’ understanding of course/learning objectives. Interviews are conducted at the end of the Spring term by former students who are at or beyond the graduate academic level. Due to dependence on graduate student availability, these interviews are conducted infrequently.
Background Surveys, Understanding of the Syllabus, Available Resources and College Catalog: Over the years we have asked a variety of open-ended and closed questions to our classes pertaining to students’ backgrounds, their understanding and/or familiarity with the course syllabus and the various kinds of support in the service of learning how to learn, as well as their familiarity with the college catalog and student handbook. Initially administered approximately half-way through the course as a pilot questions, many of these questions have been integrated into our student questionnaires.
Student Questionnaires (Current and Former): A qualitative and quantitative, semi-structured questionnaire which we administer to students who have been successful in one or more of our courses. Questions include basic background and biographical information, some family history, as well as academic experiences. The questionnaire for former students (those who have been successful in our courses and have completed at least one semester / quarter of upper division work at their transfer institution) includes additional questions pertaining to their transfer experience (including the level of academic demand they have encountered).
Standard Surveys (Students):
Health Survey: This survey was administered to students (on a voluntary basis) by our College in 2007 to screen for symptoms of depression. We have recently begun to administer this survey to students who persisted in our classes (as a sub-population of the overall Columbia College population). We administer the survey after the second of three exams in the course (at which point approximately 50% of those initially enrolled have usually dropped the course).
GRIT Test: This multiple choice scale provides a measure of persistence. It is administered occasionally. Source: Angela L. Duckworth, C. Peterson, M.D. Matthews, and D. R. Kelley, “Grit: Perseverance and Passion for Long-Term Goals.” Journal of Personality and Social Psychology, 92 (6), 2007, 1087-1101. Occasionally administered, during the first two weeks of class.
Identity Style Inventory (ISI): A likert scale measure designed to assess responses the task of identity development. Source: Michael D. Berzonsky, “Identity Style and Coping Strategies.” Journal of Personality, 60, l992, 771-778. Occasionally administered, during the first two weeks of class.
Student Motivations for Attending University (SMAU) Scale: A likert scale measure designed to assess motivation for attending college (i.e. economic, humanitarian, personal growth, etc.). Source: James E. Côté and Charles G. Levine, “Attitude Versus Aptitude: Is Intelligence or Motivation More Important for Positive Higher-Educational Outcomes?.” Journal of Adolescent Research, 15 (1), 2000, 58-80.
Resilient Quotient (RQ) Test: A likert scale measure designed to assess resilience. Source: Karen Reivich and Andrew Shatté, The Resilience Factor. New York: Broadway Books, 2002.
Myers-Briggs Type Indicator: A “self-report questionnaire designed to make Jung’s theory of psychological types understandable and useful in everyday life” (Isabel Briggs Myers, Introduction to Type). Administered to selected individuals who have made exceptional progress, seemed to have unusual talent, and with whom we have had extensive contact. Source: Isabel Briggs Myers, Mary H. McCaulley, Naomi L. Quenk, Allen Hammer, MBTI Manual: A Guide to the Development and Use of the Myers-Briggs Type Indicator Palo Alto, CA: Consulting Psychologists Press, 1998.
Photo Journal
We have recently initiated a visual dimension to our research. In order to capture body language and other behavioral dimensions of the classroom, a photo profile of the classroom is recorded on the first day of the semester, and throughout. Consistent with our commitment to transparency, students are aware that we conduct research on our classrooms and are given the option of not signing a photograph consent form. Our research assistant and former student, Nalatie Alpers, is the photographer on this project. Having experienced the HD x HS environment (as a former student) and having an understanding of its guiding philosophy, Nalatie is skillful at knowing what behaviors (+/–) to look for.
Reliability and Validity (Students)
Performance requirements (as stipulated in the course syllabus) and students’ grades along with external ratings (i.e. upper-division performance requirements and grades and prospective indicators such as advancing to graduate school, awards, internships, etc.). These data are acquired from former students (in any of our courses) on a voluntary basis over time.
Pedagogical Survey (Colleagues)
Response Summary: This table documents responses to ads we placed between February 2006 and January 2007 in The Nation, The New York Review of Books, and The Chronicle of Higher Education looking for other educators attempting to challenge more than accommodate problems in higher education. After we received a response, we emailed the respondent a voluntary questionnaire.