Jbartkie@    anelm.edu
anselm webpage

 Does Participation in Research Methods Influence Reflective Judgment Performance?

 Intervention programs outlined within the literature suggest that some techniques may be used to increase the use of reflective judgment. Unfortunately, the extent to which these programs influence the development of reflective judgment has not been studied. The current study attempted to observe the experimental psychology programís influence on reflective judgment, as it hold many of the same objectives as those outlined in literature.

A significant difference was found between the junior and senior reflective judgment scores obtained in this study. This lends some support for the use of intervention programs to increase studentís use of reflective judgments skills. However, a cautionary note must be made. Despite the significant difference found between the junior and senior groups, the scores did not indicate the senior classí utilization of a higher stage of reflective judgment. Members of the senior classí reflective judgment scores reflected agreement with some statements that reflected reflective judgment assumptions of stage five (i.e. knowledge is contextual) six (i.e. knowledge is constructed by comparing evidence and opinion on different sides of an issue or across contexts) (Kitchener & King, 1981). While the juniorsí reflective judgment scores indicated an agreement with a different rang of statements reflecting stage four (i.e. knowledge is uncertain because of situational variables) and five (Kitchener & King, 1990). This reflects a range of reflective judgment use. As indicated in the skill theory of learning, individuals possess a range of ability in all skills, including cognitive ability. These abilities may be shaped through the process of modeling and experimentation with knowledge (Fischer, 1980). The observed scores indicate that senior and junior participants are reliant on different conceptions of knowledge. However, seniors agreed with more statements that reflected more advanced conceptions of knowledge, as juniors agreed with more statements that implied less advanced conceptions of knowledge. Hence, the senior participants may be in the process of formulating conceptions of knowledge that go beyond stage five. Where as, the junior participantsí scores may be a reflection of their new experimentation with the stage five assumptions of knowledge.

The population studied scored a full stage higher than the stage four found in other studies using undergraduate students (Kitchener & King, 1990; Kitchener & King 1981). This may indicate that the population used in this study was more homogeneous than used in previous studies. For example, participants were of from the New England region, upper-middle middle class, Catholic liberal arts college, and psychology majors. In addition, the participants in this study are required, by the institution, to take a specific core course curriculum. This includes a two year sequence in humanities, a two semester minimum of a foreign language, two semesters of a science, Theories on the Philosophy of Nature and Man, Ethics, a philosophy elective, Biblical Theology, and two additional theologies elective. This is in conjunction with the twelve psychology courses required to earn a Bachelorís of Arts in Psychology degree. Taking this into consideration, the students studied were subjected to a stringent curriculum and the product of attrition from both the institution and psychology program. Thus, the experimental program offered by the psychology department may not be the only factor regarding why the psychology students performed well on the reflective judgment tasks. However, it may be the source of minimal improvements in reflective judgment scores, as most students, by their junior year, have completed most of their core class requirements. Still, the previous research studied students from a state university population (Kitchener & King, 1990). It may be inferred that students from the institution studied come from a different demographic than those of a state university. Also, the previous research was published ten years prior to this current study. Indicating, two different cohorts are being compared. Thus, the comparison may not be valid.

One additional explanation for the different mean reflective judgment scores found between the previously studied populations and the current population may lie within the instruments used. The dependent measure was specifically developed for this study. Previous studies used the RJI and RCI as their dependent measures. The RJI and RCI were constructed by the same researchers, Kitchener, King and Wood. Thus, it may be reasoned that they are measuring the same concept. The dependent measure used for this study was also modeled after a section of the RCI. However, it is not clear that the dependent measure used in this study and those used in previous studies are comparable. Further research is needed to establish concurrent validity for this measure.

An attempt to show internal consistency within the dependent measure was made. As previously stated, the reflective judgment scores from sections two and three were positively correlated. Yet, it is assumed that the reflective judgment scores obtained in sect-2 did not reach between group significance because the participants were unable to discriminate between the reflective judgment stages implied in each statement. For example, participants expressed agreement with two contradictory statements. This is evident because the average number of statements agreed with by juniors and seniors were seven out of eleven. Once the average stage level agreed with was calculated, the agreement with higher stages were canceled out by the agreement with the lower stages, resulting in an over-all low reflective judgment score. This may have occurred due to perceived demand effects on the part of the participants or it may indicate that participants use a range of reflective judgments techniques depending on the information being considered. Further research is needed to determine why participants responded in this fashion and the validity of its use in determining reflective judgment ability.

Similar subject biases were found in the completion of sect-1. A majority of respondents from both the junior and senior groups answered the four-point Likert Scale as if it was a forced answer question. This was done by either using the scales extremities (very similar or very dissimilar) or using the middle ranges of the scale only in responding to the questions. Either method used to respond to the statements, indicated agreement with blatantly contrasting statements. Thus, even though this section of the survey was modeled after the RCI, it was a poor indicator of reflective judgment.

However, it can be argued that the completion of sect-1, did give the participants an opportunity to ponder the ill-structured problem of who should deliver humanitarian aid to the Middle East and created demand characteristics within the participants as to the experimenter Ďs response expectance. As a result, creating perceived demand characteristics within the sample population (Bransford & Johnson, 1972). Also, the senior participants are members of the same class as the researcher. Thus, they may have set an emotional tone that motivated the participants to completing the survey in as they perceived the researcher expected. Also, all participants have been educated in the psychological field.. This may have made the participants more sensitive to the nature of the study and given the ability to respond as they perceived exterminator expectancy (Rosenthal & Rosnow, 1975). It is important to note that the senior class may have been more susceptible to this phenomenon than the junior class, as the senior class is concluding their experimental methods curriculum. Still, sensitivity to the instrument is a factor in all studies, but it may have been amplified due the peer relationship that existed between the participants and experimenter. .This may have contributed to the significant between group significance found.

In addition to looking at how information was processed through the reflective judgment model, this study also attempted in sect-4 to identify were the participants acquired their information. As previously reported a significant difference was found between the junior and seniorís use of the internet in obtaining information. The root of this difference is difficult to determine. It is suspected that the junior participants may have had more encouraged more in high school to use the internet as a research tool.

However, more interesting are the implications created by the lack of difference between the two groups. According to the data collected the juniors and seniors used the same methods of obtaining information. However, the reflective judgments scores did show a significant difference between the two groups. This indicates that they possess the same information but are processing it in two different manors.

In conjunction with the findings presented in this study, one must consider their life implications in the educational field. It seems despite the significant between group differences found in this study, there are alternative explanations to why an increase of reflective judgment was found. These explanations and concerns are not specific to this study alone. They may be brought up in all studies pertaining to reflective judgment. For example the RJI, may also be creating response sets in participants. The questions in the interview are direct and make the experimenterís interests self-evident. In the interview participants are asked, "Can you ever know your position is right?" and "How is it possible that people have such different opinions on the same topic?" (Wood, 2001). One may infer, that the proposition of these questions indicates the researcherís interest in thinking skills. That information alone may influence the participantís responses.

Also, the research reviewed in the literature was conducted in a college stetting. Unfortunately, the institutionís statistics regarding attrition were not provided or considered in the studiesí discussion. This makes it difficult to assess if reflective judgment increases due to the college experience, or if the characteristics of the student body become more homogenous within each class. These are important concerns regarding the formation of reflective judgment. Yet, this issue is not addressed within the literature, making it questionable whether the reflective judgment model is a valid model of cognitive development.