Sunday, November 30, 2014

Teacher Competence and Student Assessment

 When teachers are charged with 3020-a, the DOE Attorney fights to keep the Respondents' students' grades out of the hearing as "irrelevant". As a fighter for the defense of teacher tenure and employment, I believe that student grades are relevant to determining whether a teacher is competent or not. And the decision to use grades is always case-specific, especially since I believe each 3020-a proceeding is unique and deserves scrutiny.

Teachers who are not charged with grade changing should demand production of their students' grades as relevant to a disciplinary hearing, because the student grades show that the assessments were done (tests, portfolios, projects) and the teacher is the one to provide objective comments on the output of each student.

If a Math teacher's students go from a Level 2 to a Level 3 in math during the time they are in Respondent's classroom, this is key evidence that shows the Respondent knows what he/she is doing in his/her content area. If students do well in a science class, Respondent, a science teacher, must be able to use these scores.....If, on the other hand, students are special needs kids, discipline problems, or "I could care less" kids, then an argument can and should be made to the arbitrator on exactly this point.

Looking at the issue of whether student scores/assessment should be used in evaluating teachers, there is alot of links (see below) which mandate this connection.

Secondly, there are no prohibitions anywhere, and most certainly there is no statement anywhere that student scores are "not relevant".

Here are some supporting links:

Student achievement = 40%

http://www.nysut.org/~/media/Files/NYSUT/Resources/2013/April/TED/TED_Handbook.pdf
[see p. 36, 37]




[See paragraph #1]

Betsy Combier



Standards for Teacher Competence in Educational Assessment of Students

Developed By The
American Federation Of Teachers
National Council On Measurement In Education
National Education Association

This is not copyrighted material. Reproduction and dissemination are encouraged. 1990
LINK

The professional education associations began working in 1987 to develop standards
 for teacher competence in student assessment out of concern that the potential educational benefits of student assessments be fully realized. The Committee[1] appointed to this project completed its work in 1990 following reviews of earlier drafts by members of the measurement, teaching, and teacher preparation and certification communities. Parallel committees of affected associations are encouraged to develop similar statements of qualifications for school administrators, counselors, testing directors, supervisors, and other educators in the near future. These statements are intended to guide the preservice and inservice preparation of educators, the accreditation of preparation programs, and the future certification of all educators. 

A standard is defined here as a principle generally accepted by the professional associations responsible for this document. Assessment is defined as the process of obtaining information that is used to make educational decisions about students, to give feedback to the student about his or her progress, strengths, and weaknesses, to judge instructional effectiveness and curricular adequacy, and to inform policy. The various assessment techniques include, but are not limited to, formal and informal observation, qualitative analysis of pupil performance and products, paper-and-pencil tests, oral questioning, and analysis of student records. The assessment competencies included here are the knowledge and skills critical to a teacher's role as educator. It is understood that there are many competencies beyond assessment competencies which teachers must possess.
 

By establishing standards for teacher competence in student assessment, the associations subscribe to the
 view that student assessment is an essential part of teaching and that good teaching cannot exist without good student assessment. Training to develop the competencies covered in the standards should be an integral part of preservice preparation. Further, such assessment training should be widely available to practicing teachers through staff development programs at the district and building levels. 

The standards are intended for use as:
 

·         a guide for teacher educators as they design and approve programs for teacher preparation

·         a self-assessment guide for teachers in identifying their needs for professional development in student assessment

·         a guide for workshop instructors as they design professional development experiences for in-service teachers

·         an impetus for educational measurement specialists and teacher trainers to conceptualize student assessment and teacher training in student assessment more broadly than has been the case in the past.

The standards should be incorporated into future teacher training and certification programs. Teachers who have not had the preparation these standards imply should have the opportunity and support to develop these competencies before the standards enter into the evaluation of these teachers. 

The Approach Used To Develop The Standards
 

The members of the associations that supported this work are professional educators involved in teaching,
 teacher education, and student assessment. Members of these associations are concerned about the inadequacy with which teachers are prepared for assessing the educational progress of their students, and thus sought to address this concern effectively. A committee named by the associations first met in September 1987 and affirmed its commitment to defining standards for teacher preparation in student assessment. The committee then undertook a review of the research literature to identify needs in student assessment, current levels of teacher training in student assessment, areas of teacher activities requiring competence in using assessments, and current levels of teacher competence in student assessment. 

The members of the committee used their collective experience and expertise to formulate and then revise statements of important assessment competencies. Drafts of these competencies went through several revisions by the Committee before the standards were released for public review. Comments by reviewers from each of the associations were then used to prepare a final statement.
 

The Scope of a Teacher's Professional Role and Responsibilities for Student Assessment
 

There are seven standards in this document. In recognizing the critical need to revitalize classroom assessment, some standards focus on classroom-based competencies. Because of teachers' growing roles in education and policy decisions beyond the classroom, other standards address assessment competencies underlying teacher participation in decisions related to assessment at the school, district, state, and national levels.
 

The scope of a teacher's professional role and responsibilities for student assessment may be described in terms of the following activities. These activities imply that teachers need competence in student assessment and sufficient time and resources to complete them in a professional manner.
 

·         Activities Occurring Prior to Instruction

·         (a) Understanding students' cultural backgrounds, interests, skills, and abilities as they apply across a range of learning domains and/or subject areas;

·         (b) understanding students' motivations and their interests in specific class content;

·         (c) clarifying and articulating the performance outcomes expected of pupils; and

·         (d) planning instruction for individuals or groups of students.

·         Activities Occurring During Instruction

·         (a) Monitoring pupil progress toward instructional goals;

·         (b) identifying gains and difficulties pupils are experiencing in learning and performing;

·         (c) adjusting instruction;

·         (d) giving contingent, specific, and credible praise and feedback;

·         (e) motivating students to learn; and

·         (f) judging the extent of pupil attainment of instructional outcomes.

·         Activities Occurring After The Appropriate Instructional Segment (e.g. lesson, class, semester, grade)

·         (a) Describing the extent to which each pupil has attained both short- and long-term instructional goals;

·         (b) communicating strengths and weaknesses based on assessment results to students, and parents or guardians;

·         (c) recording and reporting assessment results for school-level analysis, evaluation, and decision-making;

·         (d) analyzing assessment information gathered before and during instruction to understand each students' progress to date and to inform future instructional planning;

·         (e) evaluating the effectiveness of instruction; and

·         (f) evaluating the effectiveness of the curriculum and materials in use.

·         Activities Associated With a Teacher's Involvement in School Building and School District Decision-Making

·         (a) Serving on a school or district committee examining the school's and district's strengths and weaknesses in the development of its students;

·         (b) working on the development or selection of assessment methods for school building or school district use;

·         (c) evaluating school district curriculum; and

·         (d) other related activities.

·         Activities Associated With a Teacher's Involvement in a Wider Community of Educators

·         (a) Serving on a state committee asked to develop learning goals and associated assessment methods;

·         (b) participating in reviews of the appropriateness of district, state, or national student goals and associated assessment methods; and

·         (c) interpreting the results of state and national student assessment programs.

Each standard that follows is an expectation for assessment knowledge or skill that a teacher should possess in order to perform well in the five areas just described. As a set, the standards call on teachers to demonstrate skill at selecting, developing, applying, using, communicating, and evaluating student assessment information and student assessment practices. A brief rationale and illustrative behaviors follow each standard. 

The standards represent a conceptual framework or scaffolding from which specific skills can be derived. Work to make these standards operational will be needed even after they have been published. It is also expected that experience in the application of these standards should lead to their improvement and further development.
 

Standards For Teacher Competence In Educational Assessment Of Students

1. Teachers should be skilled in choosing assessment methods appropriate for instructional decisions. 

Skills in choosing appropriate, useful, administratively convenient, technically adequate, and fair assessment methods are prerequisite to good use of information to support instructional decisions. Teachers need to be well-acquainted with the kinds of information provided by a broad range of assessment alternatives and their strengths and weaknesses. In particular, they should be familiar with criteria for evaluating and selecting assessment methods in light of instructional plans.
 

Teachers who meet this standard will have the conceptual and application skills that follow. They will be able to use the concepts of assessment error and validity when developing or selecting their approaches to classroom assessment of students. They will understand how valid assessment data can support instructional activities such as providing appropriate feedback to students, diagnosing group and individual learning needs, planning for individualized educational programs, motivating students, and evaluating instructional procedures. They will understand how invalid information can affect instructional decisions about students. They will also be able to use and evaluate assessment options available to them, considering among other things, the cultural, social, economic, and language backgrounds of students. They will be aware that different assessment approaches can be incompatible with certain instructional goals and may impact quite differently on their teaching.
 

Teachers will know, for each assessment approach they use, its appropriateness for making decisions about their pupils. Moreover, teachers will know of where to find information about and/or reviews of various assessment methods. Assessment options are diverse and include text- and curriculum-embedded questions and tests, standardized criterion-referenced and norm-referenced tests, oral questioning, spontaneous and structured performance assessments, portfolios, exhibitions, demonstrations, rating scales, writing samples, paper-and-pencil tests, seatwork and homework, peer- and self-assessments, student records, observations, questionnaires, interviews, projects, products, and others' opinions.
 

2. Teachers should be skilled in developing assessment methods appropriate for instructional decisions. 

While teachers often use published or other external assessment tools, the bulk of the assessment information they use for decision-making comes from approaches they create and implement. Indeed, the assessment demands of the classroom go well beyond readily available instruments.
 

Teachers who meet this standard will have the conceptual and application skills that follow. Teachers will be skilled in planning the collection of information that facilitates the decisions they will make. They will know and follow appropriate principles for developing and using assessment methods in their teaching, avoiding common pitfalls in student assessment. Such techniques may include several of the options listed at the end of the first standard. The teacher will select the techniques which are appropriate to the intent of the teacher's instruction.
 

Teachers meeting this standard will also be skilled in using student data to analyze the quality of each assessment technique they use. Since most teachers do not have access to assessment specialists, they must be prepared to do these analyses themselves.
 

3. The teacher should be skilled in administering, scoring and interpreting the results of both externally-produced and teacher-produced assessment methods. 

It is not enough that teachers are able to select and develop good assessment methods; they must also be able to apply them properly. Teachers should be skilled in administering, scoring, and interpreting results from diverse assessment methods.
 

Teachers who meet this standard will have the conceptual and application skills that follow. They will be skilled in interpreting informal and formal teacher-produced assessment results, including pupils' performances in class and on homework assignments. Teachers will be able to use guides for scoring essay questions and projects, stencils for scoring response-choice questions, and scales for rating performance assessments. They will be able to use these in ways that produce consistent results.
 

Teachers will be able to administer standardized achievement tests and be able to interpret the commonly reported scores: percentile ranks, percentile band scores, standard scores, and grade equivalents. They will have a conceptual understanding of the summary indexes commonly reported with assessment results: measures of central tendency, dispersion, relationships, reliability, and errors of measurement.
 

Teachers will be able to apply these concepts of score and summary indices in ways that enhance their use of the assessments that they develop. They will be able to analyze assessment results to identify pupils' strengths and errors. If they get inconsistent results, they will seek other explanations for the discrepancy or other data to attempt to resolve the uncertainty before arriving at a decision. They will be able to use assessment methods in ways that encourage students' educational development and that do not inappropriately increase students' anxiety levels.
 

4. Teachers should be skilled in using assessment results when making decisions about individual students, planning teaching, developing curriculum, and school improvement. 

Assessment results are used to make educational decisions at several levels: in the classroom about students, in the community about a school and a school district, and in society, generally, about the purposes and outcomes of the educational enterprise. Teachers play a vital role when participating in decision-making at each of these levels and must be able to use assessment results effectively.
 

Teachers who meet this standard will have the conceptual and application skills that follow. They will be able to use accumulated assessment information to organize a sound instructional plan for facilitating students' educational development. When using assessment results to plan and/or evaluate instruction and curriculum, teachers will interpret the results correctly and avoid common misinterpretations, such as basing decisions on scores that lack curriculum validity. They will be informed about the results of local, regional, state, and national assessments and about their appropriate use for pupil, classroom, school, district, state, and national educational improvement.
 

5. Teachers should be skilled in developing valid pupil grading procedures which use pupil assessments. 

Grading students is an important part of professional practice for teachers. Grading is defined as indicating both a student's level of performance and a teacher's valuing of that performance. The principles for using assessments to obtain valid grades are known and teachers should employ them.
 

Teachers who meet this standard will have the conceptual and application skills that follow. They will be able to devise, implement, and explain a procedure for developing grades composed of marks from various assignments, projects, inclass activities, quizzes, tests, and/or other assessments that they may use. Teachers will understand and be able to articulate why the grades they assign are rational, justified, and fair, acknowledging that such grades reflect their preferences and judgments. Teachers will be able to recognize and to avoid faulty grading procedures such as using grades as punishment. They will be able to evaluate and to modify their grading procedures in order to improve the validity of the interpretations made from them about students' attainments.
 

6. Teachers should be skilled in communicating assessment results to students, parents, other lay audiences, and other educators. 

Teachers must routinely report assessment results to students and to parents or guardians. In addition, they are frequently asked to report or to discuss assessment results with other educators and with diverse lay audiences. If the results are not communicated effectively, they may be misused or not used. To communicate effectively with others on matters of student assessment, teachers must be able to use assessment terminology appropriately and must be able to articulate the meaning, limitations, and implications of assessment results. Furthermore, teachers will sometimes be in a position that will require them to defend their own assessment procedures and their interpretations of them. At other times, teachers may need to help the public to interpret assessment results appropriately.
 

Teachers who meet this standard will have the conceptual and application skills that follow. Teachers will understand and be able to give appropriate explanations of how the interpretation of student assessments must be moderated by the student's socio-economic, cultural, language, and other background factors. Teachers will be able to explain that assessment results do not imply that such background factors limit a student's ultimate educational development. They will be able to communicate to students and to their parents or guardians how they may assess the student's educational progress. Teachers will understand and be able to explain the importance of taking measurement errors into account when using assessments to make decisions about individual students. Teachers will be able to explain the limitations of different informal and formal assessment methods. They will be able to explain printed reports of the results of pupil assessments at the classroom, school district, state, and national levels.
 

7. Teachers should be skilled in recognizing unethical, illegal, and otherwise inappropriate assessment methods and uses of assessment information. 

Fairness, the rights of all concerned, and professional ethical behavior must undergird all student assessment activities, from the initial planning for and gathering of information to the interpretation, use, and communication of the results. Teachers must be well-versed in their own ethical and legal responsibilities in assessment. In addition, they should also attempt to have the inappropriate assessment practices of others discontinued whenever they are encountered. Teachers should also participate with the wider educational community in defining the limits of appropriate professional behavior in assessment.
 

Teachers who meet this standard will have the conceptual and application skills that follow. They will know those laws and case decisions which affect their classroom, school district, and state assessment practices. Teachers will be aware that various assessment procedures can be misused or overused resulting in harmful consequences such as embarrassing students, violating a student's right to confidentiality, and inappropriately using students' standardized achievement test scores to measure teaching effectiveness.
 

Invitation to Users
 

The associations invite comments from users that may be used for improvement of this document. Comments may be sent to:

Teacher Standards in Student Assessment
American Federation of Teachers
555 New Jersey Avenue, NW
Washington, DC 20001

Teacher Standards in Student Assessment
National Council on Measurement in Education
1230 Seventeenth Street, NW
Washington, DC 20036
 
Teacher Standards in Student Assessment
Instruction and Professional Development
National Education Association
1201 Sixteenth Street, NW
Washington, DC 20036





[1] The Committee that developed this statement was appointed by the collaborating professional associations: James R. Sanders (Western Michigan University) chaired the Committee and represented NCME along with John R. Hills (Florida State University) and Anthony J. Nitko (University of Pittsburgh). Jack C. Merwin (University of Minnesota) represented the American Association of Colleges for Teacher Education, Carolyn Trice represented the American Federation of Teachers, and Marcella Dianda and Jeffrey Schneider represented the National Education Association.

From ERIC:
Teacher Competence and Students' Conflict Handling Strategies
Malm, Birgitte; Lofgren, Horst
Research in Education, v76 p62-73 Nov 2006
In a series of studies, teacher competence was studied from a student perspective. Students from different grades in secondary schools were interviewed about their views on what characterises a good teacher. Based on interviews and open-ended questionnaires, it was possible to construct a questionnaire with fixed alternatives that could be applied to a greater number of students. This questionnaire has since been tried, revised and subsequently used as a measure of teacher competence from a student perspective. A reasonable assumption is that teacher competence is related to students' attitudes to school work and learning as well as to students' self-confidence and self-conceptions. Related to student development, studies have been conducted to find out whether there are some basic strategies for handling conflict situations among teenagers. In this article a causal model is tested in which teacher competence, students' school attitudes and self-confidence are related to students' conflict handling strategies. The results show some substantial relationships between factors in the school environment and ways of handling conflicts. There were also some interesting differences between boys and girls. The main focus of the article is on a measurement model concerning teacher competence and a causal model in which conflict handling strategies are related to teacher competence, students' school attitudes and self-confidence.
 
 

Wednesday, November 26, 2014

New York State Education Department's Test Security Unit


From Betsy Combier:

And are they investigating principals who scrub and cheat?
Just askin'

New York Unit Focuses on Cheating by Teachers

Thirty-two Teachers and Administrators Settled Cases of Alleged Misconduct with New York State in the Past Two Years.

by Leslie Brody
 
Richard M. Brzeski had his teaching license suspended for two years by New York after he acknowledged helping fifth-graders on a state math test. His district in Rockville Centre also fired him.
James L. Basham, a social studies teacher, had his license suspended for a year after he admitted helping students on the Regents exam in U.S. history.
And Osman A. Abugana, a Brooklyn teacher, was fined $3,000 by the state after admitting that he changed five students’ scores on the Regents physics exam from failing to passing. The city tried to fire him but a hearing officer suspended him without pay for a semester instead.
They are among 32 teachers and administrators who settled cases of alleged misconduct with the New York State Education Department’s Test Security Unit in the past two years, according to files obtained under the Freedom of Information Law.
Among those settled cases, the unit found a range of alleged transgressions, including tipping students off to wrong answers, giving cheat sheets of math formulas, correcting students’ responses and completing essays for a disabled child. New York had such a big backlog of alleged violations, and some inquiries took so long, that at least one case involved tests in 2010.
Mr. Abugana’s lawyer said his client declined to comment. Lawyers for Mr. Brzeski and Mr. Basham declined to comment.
The Test Security Unit was launched in 2012 after a state-appointed investigator, Henry M. Greenberg, found New York education authorities failed to devote enough time, attention and expertise to rooting out fraud. This focus on test integrity arose in the wake of high-profile cheating scandals in Atlanta and Washington, D.C., among other places.
Teachers unions say the increasing use of student test results to rate some teachers has magnified the temptation to cheat.
“The state’s overreliance on testing and data has created intense pressure around standardized testing and unfortunately it appears that a few teachers have succumbed to that pressure,” said Carl Korn, spokesman for New York State United Teachers. “We believe the number has always been minuscule. We would like it to be zero but that’s not realistic.”
The union counts 206,000 classroom teachers statewide, and the state administers more than three million exams a year. Tina Sciocchetti, executive director of the six-member Test Security Unit, said that while the number of breaches appears small, the stakes are high. Students have to pass Regents exams to graduate, for example, and officials say any abuses taint the system.
In one case in upstate New York, Watervliet teacher Noel Santiago made copies of a math test and reviewed the material with his eighth-graders before the three-day exam period was complete, documents from his tenure hearing say.
According to a hearing officer’s decision upholding Mr. Santiago’s dismissal, a 13-year-old warned Mr. Santiago that having copies of the test was wrong and that when the student tried to alert his guidance counselor, Mr. Santiago threatened to write him up on a disciplinary charge.
Mr. Santiago didn’t respond to requests for comment, but in his tenure hearing he denied threatening the student with retaliation for attempting to disclose his use of the test booklet. According to the hearing officer’s ruling, Mr. Santiago also said there was no evidence his behavior compromised the test.
State officials say these cases are complex and the state takes into account sanctions already imposed by local school districts. Ms. Sciocchetti said that in the case of Mr. Abugana, the New York City hearing ended in a suspension that cost him at least $40,000 in lost pay.
The teacher said in his hearing that he didn’t know what he did was wrong because the state used to let scorers hunt for overlooked points to help students on the cusp of passing Regents exams.
When the Test Security Unit opened, it cleared a backlog of hundreds of test-fraud allegations and received 916 new ones in its first two years. Of the new cases, 359 remained in various stages of investigation and local disciplinary proceedings as of September. Another 206 were closed after investigators found no proof of violations.
In 19 of the 32 cases settled by the unit, educators agreed to penalties but didn’t admit wrongdoing, so their names were redacted. Sometimes districts imposed sanctions beyond the state’s steps.
The settlements brought fines totaling $191,994. Most came from six teachers and three administrators involved in Glen Cove cheating scandals that made headlines last year.
State officials imposed corrective action plans on dozens of schools. They say they have boosted test security in the past two years by visiting test sites, adding training for proctors, mandating that witnesses of misconduct report it, and prohibiting cellphones in exam rooms.
Write to Leslie Brody at leslie.brody@wsj.com
Established in March 2012, the New York State Education Department’s Test Security Unit is responsible for ensuring the security and integrity of New York State assessments. The TSU works to deter and remedy testing misconduct by educators and administrators who are involved in the administration and scoring of New York State assessments. TSU’s legal and investigative personnel review and investigate allegations of cheating submitted to the Department from sources that include school districts, educators, parents, and the public. The TSU carefully determines whether testing misconduct occurred, and if so, what corrective actions are warranted, including potential disciplinary proceedings pursuant to Part 83 and/or Education Law §3020-a. The TSU serves an important training and educational function as well, developing model testing policies and practices, and educating district personnel about them.
The TSU’s responsibilities include:
  • Ensuring security and integrity of New York State assessments;
  • Developing model New York State test security policies and procedures;
  • Intake of complaints about educator cheating via Incident Report Form found on the TSU website and from other sources;
  • Reviewing alleged testing irregularities involving educators;
  • With Integrity Officers, conducting comprehensive investigations into complaints;
  • Pursuing discipline and corrective action where testing misconduct is verified;
  • Providing training materials to New York State educators; and
  • Reporting to the public about TSU activities and the results of its investigations.