of Graduating Seniors at TAMU:
Quality of Learning Environment
and Estimated Progress, 1994
The Office of Measurement and Research Services (MARS) has instituted a system of surveys of Texas A&M students as part of Texas A&M University's commitment to outcome assessment. Surveys are one component of the outcome assessment plan. The surveys provide faculty and administrators with information about students' behaviors, attitudes, and perceptions about the quality of their learning environment.
This report contains the following sections:
Background: An important issue behind the surveys is the question concerning student-faculty relationships, and the students' perceptions of faculty's mentoring functions. Measurement and Research Services was asked by the administration of Texas A&M University to survey students on this issue. Several questions were developed that dealt with the topic of faculty/student relations, including students' perceptions of standards of performance and fairness. A review of the literature revealed other issues of importance to college students, such as advising, interactions with the administration, help with studies, a sense of belongingness, and help for solving personal problems. As part of the outcomes assessment, a basic survey consisting of a set of seventeen items was developed and administered annually to a sample of the general undergraduate population. This is the sixth annual data collection. The results of previous years are available from Measurement and Research Services--see the reference list at the end of the report.
Graduating Seniors: The basic survey was modified for use for graduating seniors. A group of ten new items were developed and used in the place of the seven basic items, additional items asked about the post graduation plans and job offers, and about courses that they wished they had or had not taken. Furthermore, in order to provide comparative data with other universities, the committee recommended the inclusion of an additional twenty-three items from the College Student Experiences Questionnaire (CSEQ)by C. Robert Pace *. These twenty-three items assess students' perceptions of their own progress in different areas. Altogether, the final form of the survey instrument for graduating seniors consists of forty-one items.
Content: The final survey instrument for the graduating seniors, as described above, consisted of forty-one items. The first ten items were intended to assess their attitudes and opinions regarding the quality of their program of study, the courses they took, the advising they received and the value of their education. These items were in the form of a statement to which they were to indicate agreement or lack of agreement on a five point Likert scale. The response options were: (1) strongly disagree, (2) disagree, (3) undecided, (4) agree, or (5) strongly agree. Other questions had variable responses appropriate to the question relating to job offers and career plans. Two open-ended questions were directed toward asking whether there were courses that they had taken but had not wanted to take or, conversely, had not taken but wished that they had taken.
The remaining twenty-three items required students to estimate their progress towards specific cognitive, social and affective goals of college education. Students were asked to look back over their time in college and estimate their progress or gains in each area represented by the survey items. The specific question was: In thinking over your experiences in college up to now, please indicate to what extent you feel you have made progress in each of the following aspects.. The response options were: (1) very much, (2) quite a bit, (3) some, (4) very little. These items were taken with permission from the College Student Experiences Questionnaire (CSEQ), 3rd edition, by C. Robert Pace.
In addition to being an established instrument, the CSEQ manual provides national norms for doctoral-granting universities. Norms for the second edition of the CSEQ were published in 1987 and updated for the third edition in 1990. The institutions making up the norm group for the third edition are listed at the bottom of Page 14. Results of the CSEQ items are treated separately in Part II of this report.
Response Scales: Items one through ten required the students to give their sentiments regarding each of the items. Sentiments is used in the psychometric sense to refer to personal reactions, interests, values, likes, and dislikes. In the case of sentiments, there is no right or wrong response. Furthermore, the stronger responses are found at both ends of the scale (strongly disagree, strongly agree) and the weakest response or zero point is in the middle of the scale (undecided).
The CSEQ items required the students to make judgments of magnitude on a four-point scale. It was assumed that the ordinal nature of the scale was apparent to the students, i.e., very much was more than quite a bit, which was more than some, which was more than very little. The items were preceded with new instructions on how to respond. With all the items, the interviewers were trained to make sure students understood the task demand.
In the Spring semester of 1994 a stratified random sample of 1100 was selected from the population of seniors who had applied for graduation in May. The sample was stratified by ethnicity and gender. Although the sample was not stratified by college, the sample size was large enough to ensure that most colleges were adequately represented in the survey.
Students in the sample were contacted by telephone in the evenings. They were first told the purpose of the survey and then were asked if they were willing to participate. The interviewers were undergraduate students hired for this purpose and trained in the actual survey. At least three attempts were made to contact each student in the sample except for those whose phone numbers were incorrectly listed in the Student Information Management Systems (SIMS). Approximately 8% of the students in the sample did not have their correct phone numbers listed in SIMS. The actual contact rate was 63.4%; the number of usable responses was 54%. The actual numbers related to the survey are shown in Table 1 (contacts) The demographic comparisons between the sample and the population are shown in Table 2 (gender) , in Table 3 (ethnicity), and in Table 4(colleges).
Category Freq. % __________________________________
Misses 205 27.5
Wrong Numbers 80 10.7
Refusals 167 22.4
Responses 293 39.3
__________________________________
Total 745 100
Return to Sample.Sample Contacted Population Gender Freq. % Freq. % ______________________________________________________
Female 240 44.4 1301 44.1
Male 300 55.6 1649 55.9
______________________________________________________
Total 540 100.0 2950 100.0
Return to Sample.Sample Contacted Population Ethnicity Freq. % Freq. % ______________________________________________________
African American 13 2.4 72 2.4
Hispanic American 49 9.0 239 8.1
Anglo-American 450 83.3 2497 84.6
Other 28 5.2 142 4.8
______________________________________________________
Total 540 100.0 2950 100.0
Return to Sample.Sample Contacted Sample Population College Freq. % Freq. % Freq. % ___________________________________________________________________
Agriculture 94 17.4 176 16.0 440 14.9
Architecture 21 3.9 43 3.9 120 4.1
Business
Administration 86 15.9 198 18.0 524 17.8
Education 21 3.9 50 4.5 235 8.0
Engineering 123 22.8 253 23.0 637 21.6
Geosciences 15 2.8 21 1.9 52 1.8
Liberal Arts 90 25.8 243 22.1 656 22.2
Sciences 32 5.9 57 5.2 144 4.9
Veterinary
Medicine 27 5.0 60 5.4 142 4.8
___________________________________________________________________
Total 540 100.0 1101 100.0 2950 100.0
In order to summarize the data, a single score on each item was constructed by summing the percent response in the categories strongly agree and agree.
Table 5 presents the actual survey items and the responses for the total sample. The percentages in the last column are the percentage of responses for the two extreme categories for each item, as described above. Some highlights of the information presented in Table 5 are as follows.
In general, the sampled seniors give high ratings to their educational experience at TAMU. This rating was evident from responses toItem 1, and Items 7 through 10 (Item 1 : Good career preparation, 89%; Item 7 : Feel good about the education, 98%; Item 8 : High prestige of A&M degree, 97%; Item 9 : Would advise friends to attend, 92%; and Item 10 : Realistic goal setting, 90%), which dealt with educational experience at TAMU in general.
been a good career preparation. 0.34 2.73 9.19 62.1 26.6 88.7
Item 2. My advisor helped me plan
a successful program of
study. 5.12 17.4 17.1 38.9 21.5 60.4
Item 3. If I had known at the begin-
ning what I know now about my major
field of study, I would have
chosen a different major. 31.4 45.4 8.19 10.9 4.10 76.8
Item 4. The campus activities outside
of class have been important
to my education. 1.71 13.3 10.2 41.6 33.1 74.7
Item 5. My course were so demanding of
my time that I was not able to
participate in many aspects of
college life. 14.0 55.6 9.56 17.1 3.75 69.6
Item 6. I believe my academic perfor-
mance at A&M would have been better
if I had not had to support
myself financially. 7.51 37.2 18.2 20.2 17.0 32.1
Item 8. I feel a degree from A&M
has high prestige. 0.34 0.34 2.4 37.7 59.2 96.9
Item 10. My experience at A&M has
helped me to set realistic goals
for myself. 0.34 2.06 7.90 57.0 32.6 89.7 _____________________________________________________________________________________ * Item score is the sum of the responses of the Agree and Strongly agree categories.
The item response information for males and females is presented in Table 6. It can be seen that males' and females' perceptions of their educational experience and quality at TAMU were very similar on all of the items. None of the differences were greater than 3% which is within the margin of error.
The item responses by ethnic groups are presented in Table 7. and again in Figure 1. in graphic forms. For each item, the percentages and the graphic dot plots showed the overall level of values, the range of the values, and any clumping of values. Examining the dot plots can help to identify cases of extreme differences among the ethnic groups. The items with the greatest spread are item 4 (The campus activities outside of class have been importantto my education), and item 9 (I would advise my friends and relatives to attend A&M). The item with the tightest clumping is item 6 (I believe my academic performance at A&M would have been better if I had not had to support myself financially.)
The interpretation of Figure 1. requires one caution: the sample sizes for the ethnic groups varied greatly (cf. Table 3 (ethnicity)), with some groups having very small numbers (e.g., thirteen African-American students, 28 Others ). This was due to the fact that the samples were proportional in size to each group's representation in the total population. Small sample size can produce unstable estimates. This might be one factor contributing to the large variation of some item responses among the ethnic groups. This should be kept in mind while interpreting Figure 1..
Af.A. H.A. O. An.A. _______________________________________
Item 1 100.0 96.0 100.0 87.3 Item 2 62.5 56.0 37.5 61.5 Item 3 87.5 76.0 62.5 77.0 Item 4 37.5 80.0 75.0 75.4 Item 5 87.5 56.0 62.5 70.6 Item 6 25.0 32.0 25.0 32.5 Item 7 87.5 100.0 100.0 98.0 Item 8 100.0 100.0 87.5 96.8 Item 9 50.0 100.0 87.5 92.8 Item 10 62.5 84.0 1
