The Outcomes Assessment Toolbox: Glossary of Assessment Terms

The glossary assembled below contains baseline terminology that may be useful for education abroad professionals. The glossary is by no means comprehensive, but supplies some of the most common terms used in assessment. The source of each definition is indicated in parentheses after the definition.

Sincere thanks to Janna Behrens, Amherst College, for her work!


Key to sources of definitions

(1) Association of American Colleges and Universities, VALUE Rubric Development Projectwww.aacu.org/value/rubrics/

(2) The Forum on Education Abroad, Glossaryhttps://forumea.org/resources/glossary/

(3) Leskes, Andreas. Beyond Confusion: An Assessment Glossary. Peer Review, Winter/Spring 2002: www.aacu.org/peerreview/pr-sp02/pr-sp02reality.cfm

(4) Truman University, Glossary of Common Assessment Termshttp://assessment.truman.edu/glossary/

(5) Stark, Philip B., Glossary of Statistical Terms: http://statistics.berkeley.edu/~stark/SticiGui/Text/gloss.htm

(6) Loyola Marymount University, Glossary of Assessment Termshttp://academics.lmu.edu/spee/officeofassessment/assessmentresources/glossaryofassessmentterms/

(7) American Speech-Language-Hearing Association, Self Study Guidelineshttp://www.asha.org/academic/accreditation/accredmanual/section4/


Ambiguity [is i]nformation that may be interpreted in more than one way. (1)

Analysis [is t]he process of recognizing and using features of a text to build a more advanced understanding of the meaning of a text.(1)

Assessment [is t]he process of measuring effectiveness, usually through the articulation of goals, the development of associated measures and the identification of observable outcomes, used to inform whether the initial goals were achieved. In the context of education abroad, the following types of assessment are common: 1) assessment of a student’s academic work in a particular course ending in the determination of the student’s grade for the course; 2) assessment or evaluation of a particular program (see Program Review), and 3) assessment of the outcomes achieved by education abroad programs (assessment typically measures a program’s growth or progress along particular parameters). Although the terms assessment and evaluation have often been used as synonyms, assessment measures progress by looking at defined variables while evaluation is an interpretation or judgment relating to quality. (2)

Assessment for accountability [is] assessment of some unit (could be a department, program or entire institution) to satisfy stakeholders external to the unit itself. Results are often compared across units. Always summative. Example: to retain state approval, the achievement of a 90 percent pass rate or better on teacher certification tests by graduates of a school of education. (3)

Assessment for improvement [is] assessment that feeds directly, and often immediately, back into revising the course, program or institution to improve student learning results. Can be formative or summative (see “formative assessment” for an example). (3)

Assessment of individuals uses the individual student, and his/her learning, as the level of analysis. Can be quantitative or qualitative, formative or summative, standards-based or value added, and used for improvement. Would need to be aggregated if used for accountability purposes. Examples: improvement in student knowledge of a subject during a single course; improved ability of a student to build cogent arguments over the course of an undergraduate career. (3)

Assessment of institutions uses the institution as the level of analysis. Can be quantitative or qualitative, formative or summative, standards-based or value added, and used for improvement or for accountability. Ideally institution-wide goals and objectives would serve as a basis for the assessment. Example: how well students across the institution can work in multi-cultural teams as sophomores and seniors. (3)

Assessment of programs uses the department or program as the level of analysis. Can be quantitative or qualitative, formative or summative, standards-based or value added, and used for improvement or for accountability. Ideally program goals and objectives would serve as a basis for the assessment. Example: how sophisticated a close reading of texts senior English majors can accomplish (if used to determine value added, would be compared to the ability of newly declared majors).(3)

Assessment plan [is a] document that outlines the student learning outcomes and program objectives, the direct and indirect assessment methods used to demonstrate the attainment of each outcome/objective, a brief explanation of the assessment methods, an indication of which outcome(s)/objectives is/are addressed by each method, the intervals at which evidence is collected and reviewed, and the individual(s) responsible for the collection/review of evidence. (3)

Assumption [r]efers to the existence of phenomena rather than the assignment of value to them. An individual employs assumptions to give a pattern to the world, and believes these are aspects of and not simply his or her perception of it (e.g., an individual can change or improve.) Cultural assumptions provide a person with a sense of reality and frame what is true for the individual. (2)

Backload (–ed, –ing) [is the a]mount of effort required after the data collection. (3)

Behavioral observations [measures] the frequency, duration, topology, etc. of student actions, usually in a natural setting with non-interactive methods, for example, formal or informal observations of a classroom. Observations are most often made by an individual and can be augmented by audio or videotape. (3)

Benchmark [is a] criterion-referenced objective performance datum that is used for comparative purposes. A program can use its own data as a baseline benchmark against which to compare future performance. It can also use data from another program as a benchmark. In the latter case, the other program often is chosen because it is exemplary and its data are used as a target to strive for, rather than as a baseline. (4)

Bias [is a] measurement procedure or estimator is said to be biased if, on the average, it gives an answer that differs from the truth. The bias is the average (expected) difference between the measurement and the truth. For example, if you get on the scale with clothes on, that biases the measurement to be larger than your true weight (this would be a positive bias). The design of an experiment or of a survey can also lead to bias. Bias can be deliberate, but it is not necessarily so. See also nonresponse bias. (5)

Blind experiment [is when, i]n a blind experiment, the subjects do not know whether they are in the treatment group or the control group. In order to have a blind experiment with human subjects, it is usually necessary to administer a placebo to the control group. (5)

Categorical variable [is a] variable whose value ranges over categories, such as {red, green, blue}, {male, female}, {Arizona, California, Montana, New York}, {short, tall}, {Asian, African-American, Caucasian, Hispanic, Native American, Polynesian}, {straight, curly}, etc. Some categorical variables are ordinal. The distinction between categorical variables and qualitative variables is a bit blurry. C.f. quantitative variable. (5)

Causation, causal relation [is when t]wo variables are causally related if changes in the value of one cause the other to change. For example, if one heats a rigid container filled with a gas, that causes the pressure of the gas in the container to increase. Two variables can be associated without having any causal relation, and even if two variables have a causal relation, their correlation can be small or zero. (5)

Chance variation or error [is when] random variable can be decomposed into a sum of its expected value and chance variation around its expected value. The expected value of the chance variation is zero; the standard error of the chance variation is the same as the standard error of the random variable—the size of a “typical” difference between the random variable and its expected value. See also sampling error. (5)

Collective Portfolios [is when f]aculty assemble samples of student work from various classes and use the “collective” to assess specific program learning outcomes. Portfolios can be assessed by using scoring rubrics; expectations should be clarified before portfolios are examined. (3)

Commercial, norm-referenced, standardized exams [is a g]roup administered, mostly or entirely multiple-choice, “objective” tests in one or more curricular areas. Scores are based on comparison with a reference or norm group. Typically must be purchased from a private vendor. (3)

Competency [is t]he cluster of skills, abilities, habits, character traits, and knowledge a person must have in order to perform effectively within a certain environment. Competencies are demonstrated through behaviors and can be developed though training and individual effort. (2)

Conclusions [is a] synthesis of key findings drawn from research/evidence. (1)

Content analysis is a procedure that categorizes the content of written documents. The analysis begins with identifying the unit of observation, such as a word, phrase, or concept, and then creating meaningful categories to which each item can be assigned. For example, a student’s statement that “I learned that I could be comfortable with someone from another culture” could be assigned to the category of “Positive Statements about Diversity.” The number of incidents that this type of response occurred can then be quantified and compared with neutral or negative responses addressing the same category. (3)

Content development [are t]he ways in which the text explores and represents its topic in relation to its audience and purpose. (1)

Context [are t]he historical, ethical. political, cultural, environmental, or circumstantial settings or conditions that influence and complicate the consideration of any issues, ideas, artifacts, and events. (1)

Contextual factors [are c]onstraints (such as limits on cost), resources, attitudes (such as biases) and desired additional knowledge which affect how the problem can be best solved in the real world or simulated setting. (1)

Control for a variable [is t]o control for a variable [but trying] to separate its effect from the treatment effect, so it will not confound with the treatment. There are many methods that try to control for variables. Some are based on matching individuals between treatment and control; others use assumptions about the nature of the effects of the variables to try to model the effect mathematically, for example, using regression. (5)

Control group [are t]he subjects in a controlled experiment who do not receive the treatment. (5)

Controlled experiment [is a]n experiment that uses the method of comparison to evaluate the effect of a treatment by comparing treated subjects with a control group, who do not receive the treatment. (5)

Controlled, randomized experiment [is a] controlled experiment in which the assignment of subjects to the treatment group or control group is done at random, for example, by tossing a coin. (5)

Correlation [is a] measure of linear association between two (ordered) lists. Two variables can be strongly correlated without having any causal relationship, and two variables can have a causal relationship and yet be uncorrelated. (5)

Course-embedded assessment refers to techniques that can be utilized within the context of a classroom (one class period, several or over the duration of the course) to assess students’ learning, as individuals and in groups. When used in conjunction with other assessment tools, course-embedded assessment can provide valuable information at specific points of a program. For example, faculty members teaching multiple sections of an introductory course might include a common pre-test to determine student knowledge, skills and dispositions in a particular field at program admission. There are literally hundreds of classroom assessment techniques, limited only by the instructor’s imagination. (3)

Critical value is the value of the test statistic beyond which we would reject the null hypothesis [in an hypothesis test]. The critical value is set so that the probability that the test statistic is beyond the critical value is at most equal to the significance level if the null hypothesis be true. (5)

Critique [i]nvolves analysis and synthesis of a full range of perspectives (1)

Cross-sectional study [is a] cross-sectional study compares different individuals to each other at the same time it looks at a cross-section of a population. The differences between those individuals can confound with the effect being explored. For example, in trying to determine the effect of age on sexual promiscuity, a cross-sectional study would be likely to confound the effect of age with the effect of the mores the subjects were taught as children: the older individuals were probably raised with a very different attitude towards promiscuity than the younger subjects. Thus it would be imprudent to attribute differences in promiscuity to the aging process. C.f. longitudinal study. (5)

Cultural rules and biases [are b]oundaries within which an individual operates in order to feel a sense of belonging to a society or group, based on the values shared by that society or group. (1)

Culture [is t]he set of distinctive spiritual, material, intellectual, and emotional features of a society or a social group. Culture encompasses art and literature, lifestyles, ways of living together, value systems, traditions, behaviors, and beliefs. (2)

Curricular map [is a] curricular map is a means to visualize the range of interdependent relationships that foster student learning through multiple opportunities. They can be used to verify collective expectations across actual educational practices. (3)

Direct assessment of learning gathers evidence, based on student performance, which demonstrates the learning itself. Can be value added, related to standards, qualitative or quantitative, embedded or not, using local or external criteria. Examples: most classroom testing for grades is direct assessment (in this instance within the confines of a course), as is the evaluation of a research paper in terms of the discriminating use of sources. The latter example could assess learning accomplished within a single course or, if part of a senior requirement, could also assess cumulative learning. (3)

Direct measures evaluate student work products in light of learning outcomes for the program. Examples of direct measures include exams and rubrics for capstone projects, portfolios, papers, and performances. (6)

Domain [are a f]ield of study or activity and a sphere of knowledge and influence. (1)

Embedded assessment [is] a means of gathering information about student learning that is built into and a natural part of the teaching-learning process. Often uses for assessment purposes classroom assignments that are evaluated to assign students a grade. Can assess individual student performance or aggregate the information to provide information about the course or program; can be formative or summative, quantitative or qualitative. Example: as part of a course, expecting each senior to complete a research paper that is graded for content and style, but is also assessed for advanced ability to locate and evaluate Web-based information (as part of a college-wide outcome to demonstrate information literacy). (3)

Empathy [is t]he recognition and understanding of the states of mind of others, including beliefs, desires, and particularly emotions, without injecting one’s own. The concept is often characterized as the ability to “put oneself into another’s shoes” or to identify with the feelings of the other person on that person’s terms. Empathy relies on the ability to set aside temporarily one’s own perception of the world and assume an alternative perspective. (2)

Estimator is a rule for “guessing” the value of a population parameter based on a random sample from the population. An estimator is a random variable, because its value depends on which particular sample is obtained, which is random. (5)

Evidence [is s]ource material that is used to extend, in purposeful ways, writers’ ideas in a text. (1)

Exemplar [is a] model or pattern to be copied or imitated. (1)

Experiment [is when, as opposed to an observational study] the experimenter decides who receives the treatment. (5)

External assessment [is the] use of criteria (rubric) or an instrument developed by an individual or organization external to the one being assessed. Usually summative, quantitative, and often high-stakes (see below). Example: GRE exams. (3)

Formative assessment [is] the gathering of information about student learning-during the progression of a course or program and usually repeatedly-to improve the learning of those students. Example: reading the first lab reports of a class to assess whether some or all students in the group need a lesson on how to make them succinct and informative. (3)

Frontload (–ed, –ing) [is the a]mount of effort required in the early stage of assessment method development or data collection. (3)

High stakes use of assessment [is] the decision to use the results of assessment to set a hurdle that needs to be cleared for completing a program of study, receiving certification, or moving to the next level. Most often the assessment so used is externally developed, based on set standards, carried out in a secure testing situation, and administered at a single point in time. Examples: at the secondary school level, statewide exams required for graduation; in postgraduate education, the bar exam. (3)

Histogram [is a] histogram is a kind of plot that summarizes how data are distributed. (5)

Hypothesis testing is formalized as making a decision between rejecting or not rejecting a null hypothesis, on the basis of a set of observations.Two types of errors can result from any decision rule (test): rejecting the null hypothesis when it is true (a Type I error), and failing to reject the null hypothesis when it is false (a Type II error). For any hypothesis, it is possible to develop many different decision rules (tests).  (5)

Implications [is h]ow inquiry results apply to a larger context or the real world. (1)

Indirect assessment of learning gathers reflection about the learning or secondary evidence of its existence. Example: a student survey about whether a course or program helped develop a greater sensitivity to issues of diversity. (3)

Indirect measures  evaluate student perceptions of their learning and the educational environment that supports learning. Examples of indirect measures include surveys, focus groups, and interviews. (6)

Institutional portfolios provide a means of assessing the impact of the entire educational experience on student learning. They can be used to drive internal improvement and external accountability. Like student portfolios, they allow for internal improvement and external accountability, but on the level of the whole institution. (3)

Intercultural experience [is t]he experience of an interaction with an individual or groups of people whose culture is different from your own. (1)

Intercultural/cultural differences [are t]he differences in rules, behaviors, communication and biases, based on cultural values that are different from one’s own culture. (1)

Interpretation [is d]etermining or construing the meaning of a text or part of a text in a particular way based on textual and contextual information. (1)

Interpretive Strategies [are p]urposeful approaches from different perspectives, which include, for example, asking clarifying questions, building knowledge of the context in which a text was written, visualizing and considering counterfactuals (asking questions that challenge the assumptions or claims of the text, e.g., What might our country be like if the Civil War had not happened? How would Hamlet be different if Hamlet had simply killed the King?). (1)

Learning goals [are g]oals are general statements about knowledge, skills, attitudes and values expected in graduates of the program. Goals are written to align with the holistic vision of the mission. Typically, multiple goals are drawn from the mission statement. (6)

Learning outcomes [is] (1)) [t]he knowledge, skills, and abilities an individual student possesses and can demonstrate upon completion of a learning experience or sequence of learning experiences (for example, in courses, degrees, education abroad programs). In an education abroad context, learning outcomes may include language acquisition, cross-cultural competence, discipline-specific knowledge, and research skills. (2)) Advance statements about what students ought to understand or be able to do as a result of a learning experience. (2)

Limitations [is c]ritique of the process or evidence. (1)

Linear association [are t]wo variables are linearly associated if a change in one is associated with a proportional change in the other, with the same constant of proportionality throughout the range of measurement. (5)

Literal meaning [is the i]nterpretation of information exactly as stated. For example, “she was green with envy” would be interpreted to mean that her skin was green. (1)

Local assessment [are] means and methods that are developed by an institution’s faculty based on their teaching approaches, students, and learning goals. Can fall into any of the definitions here except “external assessment,” for which is it an antonym. Example: one college’s use of nursing students’ writing about the “universal precautions” at multiple points in their undergraduate program as an assessment of the development of writing competence. (3)

Longitudinal Data collected on the same individuals over time for use in a longitudinal study. A study that investigates development, learning, or other types of change in individuals over time. (4)

Longitudinal study A study in which individuals are followed over time, and compared with themselves at different times, to determine, for example, the effect of aging on some measured variable. Longitudinal studies provide much more persuasive evidence about the effect of aging than do cross-sectional studies. (5)

Mean The sum of a list of numbers, divided by the number of numbers. Also called “average.” (5)

Median “Middle value” of a list. The smallest number such that at least half the numbers in the list are no greater than it. If the list has an odd number of entries, the median is the middle entry in the list after sorting the list into increasing order. If the list has an even number of entries, the median is the smaller of the two middle numbers after sorting. The median can be estimated from a histogram by finding the smallest number such that the area under the histogram to the left of that number is (5)0%. (5)

Metaphor Information that is (intended to be) interpreted in a non-literal way. For example, “she was green with envy” is intended to convey an intensity of emotion, not a skin color. (1)

Mode For lists, the mode is a most common (frequent) value. A list can have more than one mode. For histograms, a mode is a relative maximum (“bump”). (5)

Multiple perspectives Consideration of how text-based meanings might differ depending on point of view. (1)

Multivariate data A set of measurements of two or more variables per individual. (5)

Nonresponse In surveys, it is rare that everyone who is “invited” to participate (everyone whose phone number is called, everyone who is mailed a questionnaire, everyone an interviewer tries to stop on the street…) in fact responds. The difference between the “invited” sample sought, and that obtained, is the nonresponse. (5)

Nonresponse bias in a survey, those who respond may differ from those who do not, in ways that are related to the effect one is trying to measure. For example, a telephone survey of how many hours people work is likely to miss people who are working late, and are therefore not at home to answer the phone. When that happens, the survey may suffer from nonresponse bias. Nonresponse bias makes the result of a survey differ systematically from the truth. (5)

Nonresponse rate The fraction of nonresponders in a survey: the number of nonresponders divided by the number of people invited to participate (the number sent questionnaires, the number of interview attempts, etc.) If the nonresponse rate is appreciable, the survey suffer from large nonresponse bias. (5)

Norm [are w]ays of behaving to which the majority of participants in a society adhere. They are socially monitored and are often unwritten and unstated. Norms are most evident when they are not followed and the individual or group is sanctioned in some way for this deviation. This often occurs when an individual finds him or herself in a foreign country, dealing with a new culture where the norms are different. (2)

Normal curve is the familiar “bell curve (5)

Null hypothesis [is when, i]n hypothesis testing, the hypothesis we wish to falsify on the basis of the data. The null hypothesis is typically that something is not present, that there is no effect, or that there is no difference between treatment and control. (5)

Objectives [refer] to the specific knowledge, skills, or attitudes that students are expected to achieve through their college experience; expected or intended student outcomes. (4)

Outcome map  [are] the specific educational practices in your program that address a single learning outcome. It can help you determine how and where the student learning outcome is being addressed in the curriculum. (6)

Outlier is an observation that is many SD’s from the mean. It is sometimes tempting to discard outliers, but this is imprudent unless the cause of the outlier can be identified, and the outlier is determined to be spurious. Otherwise, discarding outliers can cause one to underestimate the true variability of the measurement process. (5)

Parameter [is a] numerical property of a population, such as its mean. (5)

Performance assessment [is a] method for assessing how well students use their knowledge and skills in order to do something. Music students performing a new piece of music before a panel of judges are undergoing performance assessment; students who are expected to demonstrate an understanding of basic grammar, spelling, and organizational skills while writing a paper are undergoing performance assessment; business students asked to write a proposal to solve a problem presented in a case study are undergoing performance assessment. (3)

Population [is a] collection of units being studied. Units can be people, places, objects, epochs, drugs, procedures, or many other things. Much of statistics is concerned with estimating numerical properties (parameters) of an entire population from a random sample of units from the population. (5)

Portfolios [are c]ollections of multiple student work samples usually compiled over time and rated using rubrics. The design of a portfolio is dependent upon how the scoring results are going to be used.  (3)

Probability [is t]he probability of an event is a number between zero and 100%. (5)

Program review [is t]he comprehensive evaluation of a program based on a critical examination of its component parts. Evaluators may be from inside the organization (internal review) or from outside of the organization (external review). A review (with of without an on-site component) may be of an individual program or a set of programs offered by an institution or program provider organization, of an education abroad office (campus-based or organizationally-based) that offers the programs, or both. (2)

Qualitative assessment collects data that does not lend itself to quantitative methods but rather to interpretive criteria (see the first example under “standards”). (3)

Qualitative variable is one whose values are adjectives, such as colors, genders, nationalities, etc. (5)

Quantitative assessment collects data that can be analyzed using quantitative methods (see “assessment for accountability” for an example). (3)

Quantitative Variable [is a] variable that takes numerical values for which arithmetic makes sense, for example, counts, temperatures, weights, amounts of money, etc. (5)

Quartiles [exist in threes]. The first or lower quartile (LQ) of a list is a number (not necessarily a number in the list) such that at least (1)/(4) of the numbers in the list are no larger than it, and at least (3)/(4) of the numbers in the list are no smaller than it. The second quartile is the median. The third or upper quartile (UQ) is a number such that at least (3)/(4) of the entries in the list are no larger than it, and at least (1)/(4) of the numbers in the list are no smaller than it. (5)

Quota Sample is a sample picked to match the population with respect to some summary characteristics. It is not a random sample. (5)

Random error [is possible in a]ll measurements [because they] are subject to error, which can often be broken down into two components: a bias or systematic error, which affects all measurements the same way; and a random error, which is in general different each time a measurement is made, and behaves like a number drawn with replacement from a box of numbered tickets whose average is zero. (5)

Random experiment [is a]n experiment or trial whose outcome is not perfectly predictable, but for which the long-run relative frequency of outcomes of different types in repeated trials is predictable. (5)

Random sample is a sample whose members are chosen at random from a given population in such a way that the chance of obtaining any particular sample can be computed. (5)

Random variable is an assignment of numbers to possible outcomes of a random experiment. (5)

Range is the largest value [of a set of numbers] in the set minus the smallest value in the set. Note that as a statistical term, the range is a single number, not a range of numbers. (5)

Reliability describes how well a particular assessment method provides consistent results, regardless of who uses the method or when it is used. (6)

Rubrics [are a] set of categories that define and describe the important components of the work being completed, critiqued or assessed. Each category contains a graduation of levels of completion or competence with a score assigned to each level and a clear description of what criteria need to be met to attain the score at each level. (3)

Sample [is a] sample is a collection of units from a population. (5)

Sampling error [is when, i]n estimating from a random sample, the difference between the estimator and the parameter can be written as the sum of two components: bias and sampling error. (5)

Selection bias [is a] systematic tendency for a sampling procedure to include and/or exclude units of a certain type. (5)

Self-selection occurs when individuals decide for themselves whether they are in the control group or the treatment group. Self-selection is quite common in studies of human behavior. (5)

Self-study is a procedure whereby an education program describes, evaluates, and subsequently improves the quality of its efforts. Self assessment involves the identification of a program’s strengths and limitations and the delineation of the steps necessary to correct those limitations. Self study requires a commitment to change rather than to maintaining the status quo. (7)

Significance, Significance level, Statistical significance of an hypothesis test [are] the chance that the test erroneously rejects the null hypothesis when the null hypothesis is true. (5)

Sources [are t]exts (written, oral, behavioral, visual, or other) that writers draw on as they work for a variety of purposes — to extend, argue with, develop, define, or shape their ideas, for example. (1)

Standard deviation of a set of numbers is the (square-root of the mean of the squares) of the set of deviations between each element of the set and the mean of the set. (5)

Standardized test is used to measure the performance of a group against that of a larger group. Standardized tests are often used in large-scale assessment projects, where the overall results of the group are more important than specific data on each individual client. Standardized tests are not authentic. They are most useful for reporting summative information, and are least useful for classroom diagnosis and formative purposes. (3)

Standards [set] a level of accomplishment all students are expected to meet or exceed. Standards do not necessarily imply high quality learning; sometimes the level is a lowest common denominator. Nor do they imply complete standardization in a program; a common minimum level could be achieved by multiple pathways and demonstrated in various ways. Examples: carrying on a conversation about daily activities in a foreign language using correct grammar and comprehensible pronunciation; achieving a certain score on a standardized test. (3)

Summative assessment [is] the gathering of information at the conclusion of a course, program, or undergraduate career to improve learning or to meet accountability demands. When used for improvement, impacts the next cohort of students taking the course or program. Examples: examining student final exams in a course to see if certain specific areas of the curriculum were understood less well than others; analyzing senior projects for the ability to integrate across disciplines. (3)

Supporting material [are] explanations, examples, illustrations, statistics, analogies, quotations from relevant authorities, and other kinds of information or analysis that supports the principal ideas of the presentation. Supporting material is generally credible when it is relevant and derived from reliable and appropriate sources. Supporting material is highly credible when it is also vivid and varied across the types listed above (e.g., a mix of examples, statistics, and references to authorities). Supporting material may also serve the purpose of establishing the speakers credibility. For example, in presenting a creative work such as a dramatic reading of Shakespeare, supporting evidence may not advance the ideas of Shakespeare, but rather serve to establish the speaker as a credible Shakespearean actor. (1)

Transcript analysis are examined to see if students followed expected enrollment patterns or to examine specific research questions, such as to explore differences between transfer and freshmen enrolled students. (3)

Triangulate (triangulation) [is t]he use of a combination of assessment methods in a study. An example of triangulation would be an assessment that incorporated surveys, interviews, and observations. (3)

Uncontrolled experiment [is a]n experiment in which there is no control group; i.e., in which the method of comparison is not used: the experimenter decides who gets the treatment, but the outcome of the treated group is not compared with the outcome of a control group that does not receive treatment. (5)

Validity refers to the degree to which a study accurately reflects or assesses the specific concept that the researcher is attempting to measure. Validity has three components: (1) relevance – the option measures your educational objective as directly as possible, (2) accuracy – the option measures your educational objective as precisely as possible, and (3) utility – the option provides formative and summative results with clear implications for educational program evaluation and improvement. (3)

Value added [is] the increase in learning that occurs during a course, program, or undergraduate education. Can either focus on the individual student (how much better a student can write, for example, at the end than at the beginning) or on a cohort of students (whether senior papers demonstrate more sophisticated writing skills-in the aggregate-than freshmen papers). Requires a baseline measurement for comparison. (3)

Variance, population variance is the square of the standard deviation of the list, that is, the average of the squares of the deviations of the numbers in the list from their mean. (5)

Worldview is the cognitive and affective lens through which people construe their experiences and make sense of the world around them. (1)