North Dakota

Consolidated State Application

Accountability Workbook
for State Grants under Title IX, Part C, Section 9302 of the Elementary and
Secondary Education Act (Public Law 107-110)

SUBMITTED: JANUARY 31, 2003

AMENDED: August 4, 2006

The North Dakota Accountability Workbook herein provides amendments, submitted to the U.S.
Department of Education on May 10, 2006 and subsequently approved on [TBD], that relate to
the following section of the Workbook:

• Amend Section 4.1 to allow the State to use only the grade span assessments (grades
4, 8, and 11) during 2005-06 for the purposes of determining adequate yearly progress
for all public schools and districts. During 2005-06 the State will not use the most
advantageous result of averages over one, two, and three years when determining
adequate yearly progress. The State will base adequate yearly progress decisions for
2005-06 for every public school and district upon the three-year uniform average of data
from assessments in grades 4, 8, and 11.
• Amend Section 5.3 to remove all references to a proxy achievement rate determination
for students with disabilities.
• Amend Section 5.3 to allow the State to use an alternate assessment for students with
persistent disabilities for the purposes of determining adequate yearly progress and
restricting any such determinations of proficient and advanced scores to not greater than
2 percent of students assessed on the North Dakota State Assessment.

U. S. Department of Education
Office of Elementary and Secondary Education
Washington, D.C. 20202
 NORTH DAKOTA CONSOLIDATED STATE APPLICATION ACCOUNTABILITY WORKBOOK

Instructions for Completing Consolidated State Application

Accountability Workbook

By January 31, 2003, States must complete and submit to the Department this
Consolidated State Application Accountability Workbook. We understand that some of
the critical elements for the key principles may still be under consideration and may not
yet be final State policy by the January 31 due date. States that do not have final
approval for some of these elements or that have not finalized a decision on these
elements by January 31 should, when completing the Workbook, indicate the status of
each element which is not yet official State policy and provide the anticipated date by
which the proposed policy will become effective. In each of these cases, States must
include a timeline of steps to complete to ensure that such elements are in place by
May 1, 2003, and implemented during the 2002-2003 school year. By no later than May
1, 2003, States must submit to the Department final information for all sections of the
Consolidated State Application Accountability Workbook.

Transmittal Instructions
To expedite the receipt of this Consolidated State Application Accountability Workbook,
please send your submission via the Internet as a .doc file, pdf file, rtf or .txt file or
provide the URL for the site where your submission is posted on the Internet. Send
electronic submissions to [email protected].

A State that submits only a paper submission should mail the submission by express
courier to:

Celia Sims
U.S. Department of Education
400 Maryland Ave., SW
Room 3W300
Washington, D.C. 20202-6400
(202) 401-0113

ND Accountability Application Workbook 2 May 10, 2006

 NORTH DAKOTA CONSOLIDATED STATE APPLICATION ACCOUNTABILITY WORKBOOK

PART I: Summary of Required Elements for State Accountability

Systems

Instructions

The following chart is an overview of States' implementation of the critical elements

required for approval of their State accountability systems. States must provide detailed
implementation information for each of these elements in Part II of this Consolidated
State Application Accountability Workbook.

For each of the elements listed in the following chart, States should indicate the current
implementation status in their State using the following legend:

F: State has a final policy, approved by all the required entities in the State (e.g.,
State Board of Education, State Legislature), for implementing this element in its
accountability system.

P: State has a proposed policy for implementing this element in its accountability
system, but must still receive approval by required entities in the State (e.g.,
State Board of Education, State Legislature).

W: State is still working on formulating a policy to implement this element in its

accountability system.

ND Accountability Application Workbook 3 May 10, 2006

 NORTH DAKOTA CONSOLIDATED STATE APPLICATION ACCOUNTABILITY WORKBOOK

Summary of Implementation Status for Required Elements of

State Accountability Systems

Status State Accountability System Element

Principle 1: All Schools

F 1.1 Accountability system includes all schools and districts in the state.

F 1.2 Accountability system holds all schools to the same criteria.

F 1.3 Accountability system incorporates the academic achievement standards.

F 1.4 Accountability system provides information in a timely manner.

F 1.5 Accountability system includes report cards.

F 1.6 Accountability system includes rewards and sanctions.

Principle 2: All Students

F 2.1 The accountability system includes all students

F 2.2 The accountability system has a consistent definition of full academic year.

2.3 The accountability system properly includes mobile students.

F

Principle 3: Method of AYP Determinations

F 3.1 Accountability system expects all student subgroups, public schools, and LEAs to reach
proficiency by 2013-14.

3.2 Accountability system has a method for determining whether student subgroups, public
F schools, and LEAs made adequate yearly progress.

F 3.2a Accountability system establishes a starting point.

F 3.2b Accountability system establishes statewide annual measurable objectives.

F 3.2c Accountability system establishes intermediate goals.

Principle 4: Annual Decisions

F 4.1 The accountability system determines annually the progress of schools and districts.

STATUS Legend:
F – Final state policy
P – Proposed policy, awaiting State approval
W – Working to formulate policy

ND Accountability Application Workbook 4 May 10, 2006

 NORTH DAKOTA CONSOLIDATED STATE APPLICATION ACCOUNTABILITY WORKBOOK

Principle 5: Subgroup Accountability

F 5.1 The accountability system includes all the required student subgroups.

5.2 The accountability system holds schools and LEAs accountable for the progress of student
F subgroups.

F 5.3 The accountability system includes students with disabilities.

F 5.4 The accountability system includes limited English proficient students.

F 5.5 The State has determined the minimum number of students sufficient to yield statistically
reliable information for each purpose for which disaggregated data are used.

5.6 The State has strategies to protect the privacy of individual students in reporting
F achievement results and in determining whether schools and LEAs are making adequate
yearly progress on the basis of disaggregated subgroups.

Principle 6: Based on Academic Assessments

F 6.1 Accountability system is based primarily on academic assessments.

Principle 7: Additional Indicators

F 7.1 Accountability system includes graduation rate for high schools.

7.2 Accountability system includes an additional academic indicator for elementary and middle
F schools.

F 7.3 Additional indicators are valid and reliable.

Principle 8: Separate Decisions for Reading/Language Arts and Mathematics

F 8.1 Accountability system holds students, schools and districts separately accountable for
reading/language arts and mathematics.

Principle 9: System Validity and Reliability

F 9.1 Accountability system produces reliable decisions.

F 9.2 Accountability system produces valid decisions.

F 9.3 State has a plan for addressing changes in assessment and student population.

Principle 10: Participation Rate

F 10.1 Accountability system has a means for calculating the rate of participation in the statewide
assessment.

F 10.2 Accountability system has a means for applying the 95% assessment criteria to student
subgroups and small schools.
STATUS Legend:
F – Final policy
P – Proposed Policy, awaiting State approval
W– Working to formulate policy

ND Accountability Application Workbook 5 May 10, 2006

 NORTH DAKOTA CONSOLIDATED STATE APPLICATION ACCOUNTABILITY WORKBOOK

PART II: State Response and Activities for Meeting State

Accountability System Requirements

Instructions

In Part II of this Workbook, States are to provide detailed information for each of the
critical elements required for State accountability systems. States should answer the
questions asked about each of the critical elements in the State's accountability system.
States that do not have final approval for any of these elements or that have not
finalized a decision on these elements by January 31, 2003, should, when completing
this section of the Workbook, indicate the status of each element that is not yet official
State policy and provide the anticipated date by which the proposed policy will become
effective. In each of these cases, States must include a timeline of steps to complete to
ensure that such elements are in place by May 1, 2003, and implemented during the
2002-2003 school year. By no later than May 1, 2003, States must submit to the
Department final information for all sections of the Consolidated State Application
Accountability Workbook.

ND Accountability Application Workbook 6 May 10, 2006

 NORTH DAKOTA CONSOLIDATED STATE APPLICATION ACCOUNTABILITY WORKBOOK

PRINCIPLE 1. A single statewide Accountability System applied to all public

schools and LEAs.

EXAMPLES FOR EXAMPLES OF

CRITICAL ELEMENT MEETING STATUTORY NOT MEETING
REQUIREMENTS REQUIREMENTS

1.1 How does the State Every public school and LEA is A public school or LEA is not
Accountability System required to make adequate required to make adequate
include every public school yearly progress and is included in yearly progress and is not
and LEA in the State? the State Accountability System. included in the State
Accountability System.
State has a definition of “public
school” and “LEA” for AYP State policy systematically
accountability purposes. excludes certain public schools
• The State Accountability and/or LEAs.
System produces AYP
decisions for all public
schools, including public
schools with variant grade
configurations (e.g., K-12),
public schools that serve
special populations (e.g.,
alternative public schools,
juvenile institutions, state
public schools for the blind)
and public charter schools.
It also holds accountable
public schools with no
grades assessed (e.g., K-
2).

STATE RESPONSE AND STATE ACTIVITIES FOR MEETING REQUIREMENTS

The State of North Dakota stipulates that every public school and LEA is held accountable to the
provisions of adequate yearly progress and is included in the State Accountability System. The North
Dakota Assessment System assesses all students within a single, unified, statewide assessment that
measures students’ performance in terms of the State’s challenging content and achievement standards
and that all schools and all LEAs are measured for adequate yearly progress within a single, unified
accountability system.

North Dakota, through an agreement with the U.S. Department of Education, has established an
assessment waiver plan to bring the State into full compliance with ESEA, Section 1111(b)(1)
requirements. This waiver plan, approved through August 2003, is enclosed as Appendix A: North
Dakota State Assessment Waiver Agreement Plan and can be accessed at the following web site:
http://www.dpi.state.nd.us/testing/assess/plan.pdf. To date, the State has met all objectives identified
within the waiver agreement plan. The State stipulates that it will meet all requirements identified within
the Waiver Agreement Plan.

ND Accountability Application Workbook 7 May 10, 2006

 NORTH DAKOTA CONSOLIDATED STATE APPLICATION ACCOUNTABILITY WORKBOOK

The evidence of a single, unified, statewide assessment and accountability system is demonstrated by
the grounding authority for State content standards and assessments in North Dakota State Law and in
the North Dakota Department of Public Instruction’s adherence to the requirements of Federal Law.

(a) Authority within State Law for State Content Standards.

North Dakota state law (NDCC 15.1-02-04.3) places responsibility for the development of State academic
content standards with the State Superintendent (refer to Appendix B: North Dakota Century Code
citations or reference the North Dakota Century Code at the following web site,
http://www.state.nd.us/lr/cencode/T151C02.pdf ). The North Dakota Department of Public Instruction has
developed and adopted academic content standards in mathematics (reference Appendix D: North
Dakota Mathematics Content Standards or reference these standards at the following web site,
http://www.dpi.state.nd.us/standard/content/math.pdf) and English language arts (refer to Appendix F:
North Dakota English Language Arts Content Standards or reference these standards at the following
web site, http://www.dpi.state.nd.us/standard/content/english.pdf). These State content standards have
been developed at grades 4, 8, and 12 in accordance with the North Dakota Standards and Assessment
Development Protocols (refer to Appendix C: North Dakota Standards and Assessment Development
Protocols or reference the following web site, http://www.dpi.state.nd.us/standard/content/toc.pdf). North
Dakota mathematics and English language arts academic content standards meet the requirements of
section 1111(b)(1).

The State Superintendent oversees and approves all standards development. A State-level advisory
committee consisting of LEA and SEA representatives, titled the Standards, Assessment, Learning and
Teaching (SALT) Team, advises the Department of Public Instruction on the process and quality of
standards development committee work. North Dakota’s standards development protocols currently are
being revised to incorporate improvements into the development process and to accommodate the
development of grade-level content standards in grades 3, 5, 6, and 7.

North Dakota will continue to use adopted content standards as the basis for statewide assessments at
grades 4, 8, and 12 in accordance with No Child Left Behind Act (NCLBA), section 1111(b)(1). In addition,
North Dakota will expand its statewide assessments into grades 3, 5, 6, and 7, in accordance with
NCLBA section 1111(b)(1) by 2005-06, based on state-defined, grade-level content standards in
reading/English language arts and mathematics. These grade-level content standards will be developed
and adopted in accordance with North Dakota’s standards development protocols.

North Dakota has submitted its plan for the development of grade-specific content standards to meet the
requirements of NCLBA. This submission was a part of the State’s Consolidated Application for ESEA
funding, dated June 2002. Refer to Appendix H: North Dakota State ESEA Consolidated Application,
pages 3-6, or refer to the following web site: http://www.dpi.state.nd.us/grants/DOEapp.pdf). The North
Dakota State Consolidated Application has since been approved by the U.S. Department of Education.

(b) Authority within State Law for State Assessments.

North Dakota state law (NDCC 15.1-21-08) places responsibility with the State Superintendent for the
administration of State assessments to all public schools statewide that are aligned to the State’s content
standards in reading and mathematics (refer to Appendix B: North Dakota Century Code citations or
reference the North Dakota Century Code at the following web site,
http://www.state.nd.us/lr/cencode/T151C21.pdf). State law requires that the assessments be
administered to at least one grade level selected within each of the following grade spans: grades three
through five; grades six through nine; and grades ten through twelve. The North Dakota Department of
Public Instruction has developed and administers assessments at grades 4, 8, and 12 to correspond with
the State’s content standards.

ND Accountability Application Workbook 8 May 10, 2006

 NORTH DAKOTA CONSOLIDATED STATE APPLICATION ACCOUNTABILITY WORKBOOK

In April 2003, the North Dakota Legislative Assembly enacted legislation regarding the administration of
state assessments in high school. Enrolled Senate Bill 2065, which becomes effective in August 2003,
requires that the administration of the state assessments in high school occur during the fall of the
eleventh grade effective during at least the 2005-06 school year. The assessment development and
administration schedule presented throughout this application assumes the administration of the high
school state assessments at the twelfth grade until the 2004-05 school year when the assessments will
be administered at the eleventh grade. Eleventh grade assessments will be aligned to the proper grade
level content standards. The content standards development process will provide for the proper alignment
of all content standards.

State law requires that the State assessments compile aggregated results and disaggregated results. The
State assessments must compile student achievement data that allows for a comparison of individual
students, classrooms within a given school and school district, schools within the district, and school
districts within the state. The test scores must also allow for comparisons based on students’ gender,
ethnicity, economic status, service status (i.e., migrant, disability, limited English proficient) and
assessment status (i.e., enrollment period within a school and LEA), unless doing so enables the
identification of any student. (Refer to NDCC 15.1-21-08 within Appendix B: North Dakota Century
Code citations or reference the North Dakota Century Code at the following web site,
http://www.state.nd.us/lr/cencode/T151C21.pdf.

State law requires the State Superintendent to present to the legislative council the test scores publicly for
the first time at a meeting of a legislative committee designated by the legislative council. At the meeting,
the superintendent and representatives of the testing service that created the tests shall provide detailed
testimony regarding the testing instrument, the methodology used to test and assess the students, and
the significance of the test scores. (Refer to NDCC 15.1-21-09 within Appendix B: North Dakota
Century Code citations or reference the North Dakota Century Code at the following web site,
http://www.state.nd.us/lr/cencode/T151C21.pdf ).

State law requires the State Superintendent to require that the entity developing a test to be administered
under section 15.1-21-08 not include questions that might be deemed personal to a student or to the
student’s family and that the entity developing the test not include questions requiring responses that
might be deemed personal to a student or to the student’s family. Before a test is finalized for use in North
Dakota, the State Superintendent must require that the test be reviewed by a standards alignment
committee appointed by the State Superintendent to ensure that the test meets the requirement of
privacy. (Refer to NDCC 15.1-21-11 within Appendix B: North Dakota Century Code citations or
reference the North Dakota Century Code at the following web site,
http://www.state.nd.us/lr/cencode/T151C21.pdf ).

State law requires school districts to allow any individual over the age of twenty to view any test
administered under sections 15.1-21-08 as soon as the test is in the possession of the school district.
(Refer to NDCC 15.1-21-14 within Appendix B: North Dakota Century Code citations or reference the
North Dakota Century Code at the following web site, http://www.state.nd.us/lr/cencode/T151C21.pdf ).

(c) Fulfilling the Requirements of the ESEA Waiver Plan

North Dakota, through an agreement with the U.S. Department of Education, has established an
assessment waiver plan to bring the State into full compliance with ESEA, Section 1111(b)(1)
requirements. . This waiver plan, approved through August 2003, is enclosed as Appendix A: North
Dakota State Assessment Waiver Agreement Plan and can be accessed at the following web site:
http://www.dpi.state.nd.us/testing/assess/plan.pdf. During the 2001-02 school year, North Dakota
administered its state assessment and is on schedule to meet fully all provisions set forth within the
waiver plan.

State assessments have been developed and adopted thus far in mathematics and reading at grades 4,
8, and 12 in accordance with North Dakota’s approved assessment waiver agreement and the North

ND Accountability Application Workbook 9 May 10, 2006

 NORTH DAKOTA CONSOLIDATED STATE APPLICATION ACCOUNTABILITY WORKBOOK

Dakota Standards and Assessment Development Protocols

(http://www.dpi.state.nd.us/standard/content/toc.pdf). North Dakota will proceed to develop state
assessments in mathematics and reading at additional grades (grades 3, 5, 6, and 7) by 2005-2006 in
accordance with State protocols and section 1111(b)(1) requirements. North Dakota will proceed to
develop state assessments in science at grades 4, 8, and 12 by 2007-2008 in accordance with State
protocols and section 1111(b)(1) requirements. Additionally, North Dakota will expand its science
assessment, voluntarily, at grades 3, 5, 6, and 7 by 2007-2008, based on the availability of ESEA Title VI
funds, in accordance with State protocols and section 1111(b)(1) standards.

North Dakota has submitted its plan to expand the development of grade specific assessments to meet
the requirements of NCLBA. This submission was an element of the State’s Consolidated Application for
ESEA funding, dated June 2002. Refer to Appendix H: North Dakota State ESEA Consolidated
Application, pages 7-10, or refer to the following web site:
http://www.dpi.state.nd.us/grants/DOEapp.pdf). The North Dakota State Consolidated Application has
since been approved by the U.S. Department of Education.

The North Dakota Assessment System provides for a single, unified, statewide assessment that
measures the performance of all students in all schools and all LEAs in terms of the State’s challenging
content and achievement standards.

(d) Fulfilling the requirements of ESEA Consolidated Application. Agreement to administer a

statewide accountability system based on adequate yearly progress.

State law grants to the State Superintendent of Public Instruction authority to apply for, abide by the
requirements of, and administer any federal funded program on behalf of the State of North Dakota. In
June 2002, the State Superintendent of Public Instruction signed the North Dakota Consolidated
Application for programs administered under the Elementary and Secondary Education Act. This
application included a signed certificate of assurances that obligated the State to administer a single,
unified assessment and accountability system based on adequate yearly progress. With the signature of
the State Superintendent, the State of North Dakota entered into an agreement with the U.S. Department
of Education to abide with all provisions of Section 1111 of the ESEA, including all elements of
accountability based on adequate yearly progress.

(e) Accountability System applies to all public schools within North Dakota, including schools
with variant grade configurations, schools serving special populations, and schools that
with no grades assessed.

The State of North Dakota stipulates that all public schools, regardless of grade configuration or service
population, will participate in the state accountability system. State law defines any public school to
include any educational institution supported through State funding. The state accountability system will
include all public schools identified as K-12, all alternative public schools, the North Dakota School for the
Deaf and the North Dakota State Youth Correctional Center.

Most schools within North Dakota minimally cover grade spans of K-6, 6-8, or 9-12. However, a review of
School Year 2001-02 statewide student enrollments reveals 10 individual schools with student
populations that do not fit within the typical grade span observed statewide. The following data indicate
the respective number and type of school grade spans that do not correspond to the general assessment
grade spans. Refer to Appendix I: Schools Falling Outside Assessment System Grade Spans for a
list of schools identified with a type of organization that does not allow for any assessments within the
State Assessment System.

ND Accountability Application Workbook 10 May 10, 2006

 NORTH DAKOTA CONSOLIDATED STATE APPLICATION ACCOUNTABILITY WORKBOOK

Type of School Organization (grade span)

Kindergarten K-1 K-2 K-3 6-7 9-10

Number
of 3 2 2 1 1 1
Schools

Students who attend any of the schools above will eventually graduate to a higher grade level in another
designated school. As such, there is a clearly identified school that will receive each student from their
school-of-origin listed above. Where schools-of-origin exist with grade spans that do not allow for the
administration of the State Assessment, as are the cases above, student achievement reports from the
receiving school will be forwarded to the school-of-origin by the State. No reports will be issued that might
identify an individual student. Each school in which no assessments occur will be link directly to the
supporting district. As students are promoted to school plants where assessments occur, students will
participate in the assessment and accountability system. Every school, regardless of classification, resides
within a district that participates in the State accountability system. Listed below are the linkages for schools
that do not assess students currently because of their classification. Some schools will begin assessing
students by at least 2005-06 when the State begins assessing all grades 3-8 and high school.

Non-Assessed Schools Linked to District Accountability

1. Naughton School linked to Naughton Public School District;

2. Agassiz Middle School linked to Fargo Public School District;
3. Eagle Kindergarten Center linked to Fargo Public School District;
4. Davenport Elementary School linked to Kindred Public School District;
5. Early Childhood Center linked to West Fargo Public School District;
6. Griggs County Central Elementary School linked to Griggs County Central District;
7. Stevenson School linked to Bowline Butte Public School District;
8. Zimmerman Elementary School linked to Wahpeton Public School District;
9. Central Campus School linked to Minot Public School District;
10. New Kindergarten School linked to New 8 Public School District District.

(f) Rules for Performance, Participation, and Graduation For Alternative High Schools

Rules regarding performance and participation rate.

General rule regarding performance and participation rate: Given the inherently high transfer rate
between traditional and alternative high schools, and the cumbersome nature of tracking such transfers,
the student performance and participation measures for alternative high schools will be rolled up to the
traditional high school, school district of residence, or the State.

Beginning of the year definition. To identify the status of students within the Accountability System, the
State will employ a “beginning of the school year” definition. For the 2001-02 school year, the beginning
of the school year is defined as 150 school days prior to the first day of the spring testing window. In
subsequent years, the number of days will be defined as the number of school days preceding the first
day of the testing window, as determined by the State. This will accommodate both the fall and spring
testing windows.

ND Accountability Application Workbook 11 May 10, 2006

 NORTH DAKOTA CONSOLIDATED STATE APPLICATION ACCOUNTABILITY WORKBOOK

Specific rules for performance and participation.

1. If a student is enrolled in an alternative high school and is a resident of the school district
in which the alternative high school is located, the student’s performance and
participation are attributed to the resident school district.

2. If a student attends an alternative high school but is enrolled in the school district’s
traditional high school, then the student’s performance and participation are attributed to
the traditional high school.

3. If a student is a resident of a school district other than the one in which the alternative
high school is located, and the student’s resident school district claims the student for
pupil membership, the student’s performance and participation are attributed to the
student’s school district of residence.

4. If a student transfers from one school district to another (whether the receiving school
district is the location of the alternative high school or not) since the beginning of the
school year, then the student’s performance and participation are attributed to the State.

General rule regarding graduation rate.

General rule for graduation. Given the inherently high transfer rate between traditional and alternative
high schools, and the cumbersome nature of tracking such transfers, the student graduation measure for
alternative high schools will be rolled up to the traditional high school, school district of residence, or the
State.

Cohort definition. To identify the status of students within the Accountability System, the State will employ
a cohort definition. A cohort begins from entry as identified by the school’s definition (grade 9 for a grade
9-12 school, or grade 10 for grade 10-12 school) and extends until age 21 or until graduation, whichever
occurs first. The formula to determine graduation rate is stated in the Accountability Workbook, page 50.

Specific rules for graduation.

1. If a student is enrolled in an alternative high school and is a resident of the school district
in which the alternative high school is located, the student’s graduation is attributed to the
resident school district.

2. If student attends an alternative high school but is enrolled in the school district’s
traditional high school, then the student’s graduation is attributed to the traditional high
school.

3. If the student is a resident of a school district other than the one in which the alternative
high school is located, and the student’s resident school district claims the student for
pupil membership, the student’s graduation is attributed to the student’s school district of
residence.

4. If a student transfers from one school district to another since the beginning of the cohort,
then the student’s graduation is attributed to the State.

(g) Rules for Performance and Participation For Atypical Education Settings

General rules for performance and participation. The following general rules apply when determining the
educational entity to which a student’s performance and participation will be attributed.

ND Accountability Application Workbook 12 May 10, 2006

 NORTH DAKOTA CONSOLIDATED STATE APPLICATION ACCOUNTABILITY WORKBOOK

1. If the student physically attends the public school, performance and participation are attributed to
that school, the school district, and the State.

2. If the public or private school or facility serves the student on a contract basis, the student’s
school district of residence is responsible; student performance and participation are attributed to
the school district of residence.

3. If the student is served in a state facility, student performance and participation are attributed to
the State.

Beginning of the year definition. To identify the status of students within the Accountability System, the
State will employ a “beginning of the school year” definition. For the 2001-02 school year, the beginning
of the school year is defined as 150 school days prior to the first day of the spring testing window. In
subsequent years, the number of days will be defined as the number of school days preceding the first
day of the testing window, as determined by the State. This will accommodate both the fall and spring
testing windows.

Specific rules for performance and participation. Students may attend school in other than the public
school in their school district of residence for either a brief or extended period of time due to (1) choice;
(2) developmental or health concerns; or (3) behavior/discipline issues or adjudication.

(1) Specific rules for performance and participation related to choice.

If a student attends a school and school district other than his or her school or school district of
residence and the serving school district claims pupil membership for the student, performance and
participation are attributed to the serving school, school district, and State. This applies to:
a. Job Corps students
b. Air Force Base students
c. Open enrolled students

(2) Specific rules for performance and participation related to developmental and health concerns.

a. If a student is served under contract to a public or private facility or to another public or

private school or school district, performance and participation are attributed to the school
district of residence and State. This applies to:
i. Anne Carlsen Center
ii. Developmental Center
iii. Adolescent Unit of Jamestown State Hospital
iv. Students attending psychiatric treatment or mental health facilities
v. Some students receiving special education services

b. If a student is placed in a treatment facility out of North Dakota and the North Dakota school
district of residence claims pupil membership, performance and participation are attributed to
the school district and State.

c. If a student is served at the North Dakota School for the Deaf (NDSD), performance and
participation are attributed to NDSD and the State.

(3) Specific rules for performance and participation related to behavior/discipline or adjudication issues.

a. If a student is served at the Youth Correctional Center or State Penitentiary, performance and
participation are attributed to the State.

b. If a student is incarcerated and is claimed by the school district of residence for pupil
membership, performance and participation are attributed to the school district. However, if a

ND Accountability Application Workbook 13 May 10, 2006

 NORTH DAKOTA CONSOLIDATED STATE APPLICATION ACCOUNTABILITY WORKBOOK

student is incarcerated and is not claimed by the school district of residence for pupil
membership, performance and participation are attributed to the State.

c. If a student is served at the Adolescent Unit at the North Dakota State Hospital, performance
and participation are attributed to the State.

d. If a student is served at Dakota Boys Ranch (Minot and Fargo), performance and
participation are attributed to the State.

e. If a student is served at Home on the Range (Beach), performance and participation are
attributed to the Beach school district.

f. If the student who is less than 16 years of age is truant and the school district of residence
claims pupil membership for the student, participation for that student is attributed to that
school district of residence and the State. However, if the student who is less than 16 years
of age is truant and the school district of residence does not claim pupil membership for the
student, participation for that student is attributed to the State.

g. If a student who is less than 16 years of age is suspended or expelled from school, as
evidenced through appropriate documentation, and who is claimed by the school district of
residence for purposes of pupil membership, performance and participation are attributed to
the school district of residence and the State according to the beginning of year rule.
However, if a student who is less than 16 years of age is suspended or expelled from school
and who is not claimed by the school district of residence for purposes of pupil membership,
participation is attributed to the State.

h. If a student is placed with foster parents who reside in a North Dakota school district,
performance and participation are attributed to the serving school district and State.

(h) Definition of “public school” for AYP determination.

For the purposes of determining AYP, a public school within North Dakota is identified by the grade levels it
serves and is approved to operate based upon its meeting criteria established in State law (NDCC 15.1-06-
06). Schools report their approval status annually, as identified on the State’s MIS 02 report for school
approval. The Department of Public Instruction will reference this grade level approval status for the
purposes of classifying and reporting public schools.

ND Accountability Application Workbook 14 May 10, 2006

 NORTH DAKOTA CONSOLIDATED STATE APPLICATION ACCOUNTABILITY WORKBOOK

EXAMPLES FOR EXAMPLES OF

CRITICAL ELEMENT MEETING STATUTORY NOT MEETING
REQUIREMENTS REQUIREMENTS

1.2 How are all public schools All public schools and LEAs are Some public schools and LEAs
and LEAs held to the same systematically judged on the are systematically judged on the
criteria when making an AYP basis of the same criteria when basis of alternate criteria when
determination? making an AYP determination. making an AYP determination.

If applicable, the AYP definition is

integrated into the State
Accountability System.

STATE RESPONSE AND STATE ACTIVITIES FOR MEETING REQUIREMENTS

The State of North Dakota stipulates that all public schools and LEAs will be judged systematically on the
basis of the same criteria when making AYP determination. The State will adopt the definition of AYP as
set forth within ESEA section 1111.

All schools and LEAs will be measured for AYP in terms of their demonstrated achievement of each of the
following criteria:
• A school’s or LEA’s aggregate proficiency in both mathematics and reading, determined
independently;
• A school’s or LEA’s proficiency, determined on the disaggregated achievement results for each
subgroup (ethnicity, disability, limited English proficient, and economic disadvantaged),
determined independently;
• A school’s or LEA’s aggregate participation rate that equals or exceeds 95%;
• A school’s or LEA’s disaggregated participation rate that equals or exceeds 95% within each
subgroup, determined independently;
• A secondary school’s or LEA’s achievement of the required graduation rate;
• An elementary or middle school’s or LEA’s achievement of the required attendance rate;
• A school’s or LEA’s achievement of Safe Harbor in the aggregate or disaggregated for each
subgroup, determined independently.
• The rules of statistical reliability apply to all independent measures of AYP. Any application of
statistical reliability for safe harbor is contingent on the study of the effects of the binomial
distribution on safe harbor and an agreement between the U.S. Department of Education and the
North Dakota Department of Public Instruction. Refer to section 9.1 for an overview of this study.

Each criteria stated above will apply to all public schools and LEAs, without exception.

The incorporation of federal guidance regarding the election of a local school district to determine
adequate yearly progress for a Title I targeted assistance school based solely on the achievement of
students who are served by Title I or who are eligible to receive Title I service.

On February 4, 2004, the U.S. Department of Education issued guidance regarding the determination of
adequate yearly progress under Section 1116(b)(1)(D) in schools with a targeted assistance Title I
program. This provision is a restricted privilege accorded to Title I targeted assistance schools. The
following provisions apply.

a. Minimum group size and subgroup accountability. The authority to invoke the provisions of Section
1116(b)(1)(D) of the Act may be exercised only if the number of students served or eligible to be served

ND Accountability Application Workbook 15 May 10, 2006

 NORTH DAKOTA CONSOLIDATED STATE APPLICATION ACCOUNTABILITY WORKBOOK

by Title I services meets the minimum group size of ten students as provided within Sections 5.5 and 5.6
of the North Dakota Accountability Workbook. If, among the targeted assistance students, there are
sufficient numbers of students in various subgroups to meet the minimum group size, subgroup
accountability will be required. As in other schools, students who are members of subgroups that fall
below the minimum group size at the school level will be included in the appropriate subgroups for district
and State adequate yearly progress calculations.

b. Separate decisions for reading and mathematics. In the case of a targeted assistance school where
Title I services are offered in only one subject, the school will still be held accountable for both
reading/language arts and mathematics. An LEA will have the flexibility to calculate adequate yearly
progress based on either of the following conditions:
• The reading and mathematics assessment scores/participation rates of only students served or
who are eligible for Title I services; or
• The assessment scores/participation rates of only students served or eligible for Title I services in
the subject in which Title I services are provided and of all students in the school for the subject in
which Title I services are not provided.

c. Other academic indicators. The LEA may examine the data for the entire school or only those students
receiving or eligible to receive Title I services to determine if adequate yearly progress was made for the
other indicator. When adequate yearly progress is being calculated based upon only the students
receiving or eligible for Title I services, those same students would be used to calculate progress on the
other indicator if the LEA wants to take advantage of the safe harbor provisions. If data for the other
indicators (i.e., graduation and attendance) cannot be collected on those students receiving or eligible for
school regardless of student participation in Title I services, the State will calculate adequate yearly
progress based on all students in the school regardless of student participation in Title I services.

d. Including results at the district and state level. When calculating adequate yearly progress for a LEA,
the State will review the progress of all students attending schools without Title I programs and with
schoolwide Title I programs; in schools operating targeted assistance programs, the State may review the
progress of only the students who are served or eligible for Title I services or may review the progress of
all students in the school. The calculation of adequate yearly progress decisions for LEAs will be
conducted consistent with the USDE guidance of February 4, 2004. The progress of all students,
regardless of eligibility for Title I services, will be reviewed in adequate yearly progress calculations for the
State. All students must participate in the assessment at the school level, even if the school only uses
results from students receiving or eligible for Title I services for accountability purposes.

e. Adequate yearly progress for schools without tested grades. When Title I services are provided in
grades that are not assessed, an LEA has several options for determining adequate yearly progress:
• Back-mapping the performance of students in the grade assessed to the grades in which services
are provided; or
• Forward-mapping by examining the achievement of students who received services once those
students are in the grades where assessments are given; or
• Examining the achievement of the students who would have been eligible for Title I services in
the grades assessed.

The Department stipulates that it will actively inform schools of their opportunity to invoke the privilege of
this provision, draft procedures regarding the administration of this privilege, require schools to submit a
written request indicating their desire to exercise this privilege, provide technical assistance and guidance
to each school regarding the administration of this privilege, monitor for the correct application of this
privilege according to the guidelines set forth within the February 4, 2004 guidance, and maintain records
concerning the use of this privilege. Furthermore, the Department commits itself to develop clear
procedures and guidance to schools that encourages and simplifies the determination of adequate yearly
progress according to this privilege.

ND Accountability Application Workbook 16 May 10, 2006

 NORTH DAKOTA CONSOLIDATED STATE APPLICATION ACCOUNTABILITY WORKBOOK

EXAMPLES FOR EXAMPLES OF

CRITICAL ELEMENT MEETING STATUTORY NOT MEETING
REQUIREMENTS REQUIREMENTS

1.3 Does the State have, at a State has defined three levels of Standards do not meet the
minimum, a definition of student achievement: basic, legislated requirements.
basic, proficient and proficient and advanced. 1
advanced student
achievement levels in Student achievement levels of
reading/language arts and proficient and advanced
mathematics? determine how well students are
mastering the materials in the
State’s academic content
standards; and the basic level of
achievement provides complete
information about the progress of
lower-achieving students toward
mastering the proficient and
advanced levels.

STATE RESPONSE AND STATE ACTIVITIES FOR MEETING REQUIREMENTS

The State of North Dakota has established achievement standards in reading and mathematics, based on
four distinct levels of student achievement: novice, partially proficient, proficient, and advanced.

The State of North Dakota has developed and adopted academic achievement standards in mathematics
(refer to Appendix E: North Dakota Mathematics Achievement Standards, at the following web site,
http://www.dpi.state.nd.us/standard/perform/index.shtm) and English language arts (refer to Appendix G:
North Dakota English Language Arts Achievement Standards, at the following web site,
http://www.dpi.state.nd.us/standard/perform/index.shtm). These State achievement standards have been
developed at grades 4, 8, and 12 in accordance with North Dakota’s content and achievement standards
protocols (http://www.dpi.state.nd.us/standard/content/toc.pdf). North Dakota mathematics and English
language arts academic achievement standards meet the requirements of section 1111(b)(1). North
Dakota’s achievement standards in science will be completed by the winter 2003, in accordance with
State standards development protocols and section 1111(b)(1) requirements.

North Dakota will continue to use adopted achievement standards as the basis for statewide
assessments at grades 4, 8, and 12 in accordance with section 1111(b)(1). In addition, North Dakota will
expand its statewide assessments into grades 3, 5, 6, and 7 in mathematics and reading, in accordance
with section 1111(b)(1) by 2005-06, based on State-defined, grade-level achievement standards.
Additionally, North Dakota will expand its statewide assessments, voluntarily, based on the availability of
ESEA Title VI funding, into grades 3, 5, 6, and 7 in science by 2007-2008, based on State-defined,
achievement standards. All achievement standards at grades 3, 5, 6, and 7 will be aligned with North
Dakota’s corresponding grade-level content standards. These achievement standards will be developed
and adopted in accordance with North Dakota’s standards development protocols.

1
System of State achievement standards will be reviewed by the Standards and Assessments Peer
Review. The Accountability Peer Review will determine that achievement levels are used in determining
AYP.

ND Accountability Application Workbook 17 May 10, 2006

 NORTH DAKOTA CONSOLIDATED STATE APPLICATION ACCOUNTABILITY WORKBOOK

North Dakota proposes to develop narrative achievement standards at grades 3, 5, 6, and 7

contemporaneously with the development of grade-level content standards. The content expectation
committees will also draft the narrative achievement standards. These narrative achievement standards
will act as the primary calibration tool for the cut-point standards setting performed to align the State
assessment scale scores to State achievement standards.

The State of North Dakota will only recognize and reference student achievement proficiency ratings
generated by the North Dakota State Assessment and its Alternate Assessment. No other student
achievement assessment tools or means will be recognized. No local assessments will be recognized as
an alternative to the North Dakota State Assessment.

The only definitions of proficiency levels recognized by the State AYP accountability system are those
proficiency levels set for the North Dakota State Assessment through the standards-setting process.
Refer to Appendix J: North Dakota State Assessment, Bookmark Standards Setting Technical
Report, 2002 for the established definitions of the North Dakota proficiency levels. The State cut scores
for the North Dakota State Assessment will constitute the defining scales for identifying schools and
districts for AYP. Refer to Appendix K: State Superintendent’s Approval Notification of North Dakota
State Assessment Cut Scores for the State’s announced policy regarding the establishment of
proficiency level cut scores.

ND Accountability Application Workbook 18 May 10, 2006

 NORTH DAKOTA CONSOLIDATED STATE APPLICATION ACCOUNTABILITY WORKBOOK

EXAMPLES FOR EXAMPLES OF

CRITICAL ELEMENT MEETING STATUTORY NOT MEETING
REQUIREMENTS REQUIREMENTS

1.4 How does the State provide State provides decisions about Timeline does not provide
accountability and adequate adequate yearly progress in time sufficient time for LEAs to fulfill
yearly progress decisions for LEAs to implement the their responsibilities before the
and information in a timely required provisions before the beginning of the next academic
manner? beginning of the next academic year.
year.

State allows enough time to

notify parents about public school
choice or supplemental
educational service options, time
for parents to make an informed
decision, and time to implement
public school choice and
supplemental educational
services.

STATE RESPONSE AND STATE ACTIVITIES FOR MEETING REQUIREMENTS

During the 2001-02 school year and pursuant to the State’s Assessment Waiver Plan approved by the
U.S. Department of Education, the State received its baseline assessment results from its assessment
vendor during the fall, 2002. The Department of Public Instruction used these baseline assessment
results to conduct the 2001-02 AYP review of each school and LEA in February 2003.

The State released its initial AYP reports in February 2003. The State provides technical assistance on
programmatic issues related to AYP reports to LEAs and schools. In the spring of each school year, the
Department of Public Instruction conducts a workshop for all schools identified as not achieving AYP. At
this workshop, schools are provided with a timeline of required activities and information on implementing
all required AYP provisions. Schools are informed of their responsibilities on parent notification, school
choice, supplemental services, and other corrective actions sanctions, and are given guidance on writing
a school improvement plan. The schools prepare and implement these requirements before the beginning
of the next academic school year. Additionally, schools receive ongoing guidance throughout the school
year including informative memos on required procedures, example forms and ideas for implementation.
The Title I website for program improvement also contains the information distributed at the spring
workshop to help schools as they implement required provisions before and during the school year. Refer
to Appendix L: Program Improvement Activities at the following website:
www.dpi.state.nd.us/title1/progress/index.shtm.

For the 2002-03, and 2003-04 school years, all final assessment scores will be made available to the
State from the State’s assessment contractor by June of each respective year. It is anticipated that the
State will be in a position to conduct its AYP determination and report dissemination by late July of each
respective year. Schools will receive their AYP status reports during the summer of each respective year.
This notification will arrive in time for schools and LEAs to notify, in turn, parents regarding their right to
seek a supplemental service, travel service, or school choice option under program improvement with
ESEA section 1116.

ND Accountability Application Workbook 19 May 10, 2006

 NORTH DAKOTA CONSOLIDATED STATE APPLICATION ACCOUNTABILITY WORKBOOK

Beginning during the 2004-05 school year and for every school year thereafter, the State will conduct fall
assessments that will ensure the State’s ability to conduct ongoing achievement cut-score analyses and
AYP determinations well in advance of the end of a given school year. The advancement of a fall
assessment schedule is designed to improve the quality of cut-score analyses, the generation and
dissemination of reports, the timely notification of schools and LEAs, the more conducive turn-around
time for school- and LEA-reporting to parents, and the more relaxed deliberation of parents in determining
their parental rights options identified within ESEA, section 1116.

ND Accountability Application Workbook 20 May 10, 2006

 NORTH DAKOTA CONSOLIDATED STATE APPLICATION ACCOUNTABILITY WORKBOOK

EXAMPLES FOR EXAMPLES OF

CRITICAL ELEMENT MEETING STATUTORY NOT MEETING
REQUIREMENTS REQUIREMENTS

1.5 Does the State The State Report Card includes The State Report Card does not
Accountability System all the required data elements include all the required data
produce an annual State [see Appendix A for the list of elements.
Report Card? required data elements].
The State Report Card is not
The State Report Card is available to the public.
available to the public at the
beginning of the academic year.

The State Report Card is

accessible in languages of major
populations in the State, to the
extent possible.

Assessment results and other

academic indicators (including
graduation rates) are reported by
student subgroups

STATE RESPONSE AND STATE ACTIVITIES FOR MEETING REQUIREMENTS

The State stipulates that it will produce and disseminate a State Report Card and Profile for the state as a
whole, for each LEA, and for each public school to meet all accountability requirements specified within
ESEA section 1111. The State Report Card and Profile will publish all aggregate student achievement
data, all disaggregate student achievement data by subgroup, graduation rates, attendance rates,
participation rates, and AYP status for the State, each LEA, and each school respectively.

State law requires the dissemination of individual student assessment reports to parents and schools in
an understandable format. State law also requires the presentation of State assessment results to the
Legislative Council summarizing overall student achievement. Further, State law requires that aggregated
and disaggregated student achievement results be published for the review of the public. Refer to
Appendix B: North Dakota Century Code, Assessment Statutes for a summary of State’s reporting
requirements. Refer to Appendix M: Testimony Before the Education Committee by the Department
of Public Instruction, October 10, 2002 for a summary outline of the testimony delivered to the
Legislative Council’s Interim Education Committee on October 10, 2002.

The State’s assessment contractor scores, prints, packages and ships all student achievement reports to
the respective schools. Teachers are instructed to review the results of each student’s assessment with
each student and subsequently with each student’s parents. Teachers are instructed to review a student’s
performance at the subject level, the standards level, and at the benchmark level. Further, teachers are
instructed to clarify the meaning of the State content standards and achievement standards. The back-
side of all reports offers a summary of these standards and identifies a web site for a more detailed
presentation of the State’s standards. Refer to Appendix N: North Dakota State Assessment, Student
Achievement Reports for illustrations of the various achievement reports.

ND Accountability Application Workbook 21 May 10, 2006

 NORTH DAKOTA CONSOLIDATED STATE APPLICATION ACCOUNTABILITY WORKBOOK

The Department of Public Instruction receives all student achievement data for each school and district
from the State’s assessment contractor through a comprehensive data transfer. The Department compiles
the data, identifies and corrects any inconsistencies, generates disaggregated reports according to
defined subgroup populations, and prints summative reports for each school, each district, and the State.
The results of these reports are forwarded to each school and district. These results are also listed on the
State Report Card and School Profile of the Department’s web site. Refer to Appendix O: North Dakota
State Report Card and Profile for an illustration of the content. The 2001-02 State Report Card and
Profile and its web site are under development and will be completed in early February 2002.

The State will produce all district- and school-level reports for the districts and schools. These reports will
include both aggregated and disaggregated student achievement data. Refer to Appendix O: North
Dakota School Report Card and Profile for an illustration of the content of these student achievement
profiles. Districts may use these reports as the foundation for their locally produced report cards and
profiles. These State-generated reports will offer quality assurances regarding the generation of any
district achievement data.

All public information will be disseminated through the public media, as described below.

(1) The Department of Public Instruction will present an annual report to the North Dakota Legislative
Council as required by law (refer to Appendix B: North Dakota Century Code, Assessment Statutes
for an overview of State statutes on public disclosure of State Assessment results, and Appendix M:
Testimony Before the Education Committee by the Department of Public Instruction, October 10,
2002 for the 2002 presentation to the Legislative Council).

(2) The Department of Public Instruction will publish press releases for use by radio/television, the print
media, and other publication media. The content for these press releases will reflect the school profile
and report card. Refer to Appendix O: North Dakota School Report Card and Profile for an illustration
of this content. Refer to Appendix P: State Superintendent’s Release of State Assessment Results
for the November 2002 press release on the 2001-02 school year achievement data, submitted by the
State Superintendent.

(3) The Department of Public Instruction will publish the school report card and profile electronically
through the Department’s website, Refer to Appendix O: North Dakota School Report Card and
Profile or refer to the following web site, http://www.dpi.state.nd.us/dpi/reports/profile/0102/50128.htm
This public information process supplements the Department’s communication to parents regarding
standards and assessment.

The State will publish all aggregated and disaggregated student achievement data by school, district, and
the State on the Department of Public Instruction web site. This publication will allow school districts to
access information on their district and other districts for use in general school improvement activities. Refer
to Appendix T: North Dakota Sample School Report Card and Profile or refer to the following web site,
http://www.dpi.state.nd.us/dpi/reports/profile/0102/50128.htm.

Parents will have access to the information through their students’ individual achievement reports, the
Department of Public Instruction website, the dissemination of their district’s local school report card and
profile, and other forms of public documents. The Department of Public Instruction will analyze data and
review policies on a regular basis in order to assure that data are used to advance school improvement
plans.

ND Accountability Application Workbook 22 May 10, 2006

 NORTH DAKOTA CONSOLIDATED STATE APPLICATION ACCOUNTABILITY WORKBOOK

EXAMPLES FOR EXAMPLES OF

CRITICAL ELEMENT MEETING STATUTORY NOT MEETING
REQUIREMENTS REQUIREMENTS

1.6 How does the State State uses one or more types of State does not implement
Accountability System rewards and sanctions, where rewards or sanctions for public
include rewards and the criteria are: schools and LEAs based on
sanctions for public schools adequate yearly progress.
and LEAs? 2 • Set by the State;

• Based on adequate yearly

progress decisions; and,

• Applied uniformly across

public schools and LEAs.

STATE RESPONSE AND STATE ACTIVITIES FOR MEETING REQUIREMENTS

Sanctions and Rewards Based on Adequate Yearly Progress.

The state of North Dakota has established an accountability system that is based on the state’s definition
of adequate yearly progress and is applied uniformly across all public schools and districts in the state.
All schools and districts are held to the same standards. All schools and districts will receive written
notification of whether they are satisfactory in making adequate yearly progress. However, the state does
not hold schools and districts not receiving Title I funds to the requirements of Section 1116 of the No
Child Left Behind Act.

The state of North Dakota is in the process of revising the previous system of rewards and sanctions to
align with the No Child Left Behind Act. The Department of Public Instruction has always had a system of
rewards and sanctions in place. The previous and new system for rewards and sanctions for public
schools and LEAs in North Dakota is based solely on a district’s or school’s adequate yearly progress
status.

The state’s previous system of rewards was based on a state assessment that measured student
progress using national percentiles. Schools that scored above the 65 percentile for 3 consecutive years
were identified as Title I distinguished schools. These districts and schools were recognized and served
as models under the statewide school support system.

As of 2001-2002, the state assessment measures student progress against our North Dakota state
standards in reading and mathematics. North Dakota teachers, under the direction of our state
assessment contractor, went through a standard setting process and identified cut scores for proficiency
on the state assessment. Schools that meet or exceed the standard are declared satisfactory in making
adequate yearly progress.

2
The state must provide rewards and sanctions for all public schools and LEAs for making adequate
yearly progress, except that the State is not required to hold schools and LEAs not receiving Title I funds
to the requirements of section 1116 of NCLB [§200.12(b)(40)].

ND Accountability Application Workbook 23 May 10, 2006

 NORTH DAKOTA CONSOLIDATED STATE APPLICATION ACCOUNTABILITY WORKBOOK

System of Sanctions

The state’s previous system of sanctions remains intact. Schools that were in program improvement
status in the old law remained in the same category after the No Child Left Behind Act was enacted.
North Dakota currently has twenty-three schools identified for program improvement. Twenty-one of the
schools are currently in the fourth year of program improvement. Two schools are in their third year of
program improvement. All twenty-three schools have submitted a program improvement plan that is
currently being peer reviewed against established rubrics that assess the quality of the plans. All twenty-
three schools have notified parents and community members of their program improvement identification
and the appropriate parent options available to them. Annual workshops for schools identified for
program improvement were held during April 2002 and April 2003. School personnel were apprised of
the new regulations in the No Child Left Behind Act.

The school choice and supplemental service provisions are currently being implemented for the twenty-
three schools in program improvement status. The state of North Dakota created a supplemental service
application and went through a request for proposal process in August 2002. This process resulted in two
supplemental service providers being approved to offer supplemental services. In December 2002, the
Department of Public Instruction went through a second request for proposals process. A second round
of supplemental service providers was announced in March 2003.

Current North Dakota law allows for open enrollment so the choice provision can be implemented in
districts with more than one school per grade span. State law authorizes a process for the SEA to take at
least one of the actions against LEAs in corrective action, listed in the NCLB legislation. In April 2003 the
North Dakota Legislative Assembly enacted law to authorize open enrollment across school district lines.
This provision effectively authorizes the Department to implement at least one of four alternative
governance options, or another option that leads to “major restructuring” to improve student achievement
for schools in year seven of the program improvement timeline.

System of Rewards.

The State of North Dakota is developing a system of rewards that includes distinguished schools
designations or financial rewards. These strategies will recognize schools that have significantly closed
the achievement gap, exceeded adequate yearly progress, or have made the greatest gains in student
performance. The Department of Public Instruction is working with various advisory groups, including the
Committee of Practitioners, Title I School Support Team, the Standards Assessment and Learning Team,
and the State’s Assessment and Accountability Technical Advisory Committee to develop criteria on what
constitutes a significant gain for recognition awards and financial rewards.

All districts and schools in the state that significantly exceed the adequate yearly progress expectations
for any given year will be recognized as a distinguished school. Distinguished schools and districts will
receive certificates of distinction and receive public recognition of this distinguished status.

Title I districts and schools that have significantly closed the achievement gap or have made the greatest
gains in student performance in achieving status or improvement will be eligible for financial rewards.
Financial rewards will vary and most likely be limited depending on the number of eligible schools. The
Department of Public Instruction will conduct an annual review of the established distribution method in
order to optimize the financial impact to schools. This annual review will be conducted based on the
recommendations of the Committee of Practitioners, School Support Team, SALT Team, and the State
Technical Advisory Committee. The State will reward those districts and schools that have achieved the
upper 10% to 25% of ranked achievement gains, allowing for statistical significance. The State
Superintendent will review, amend, and approve the annual distribution formula to meet the available
funds and established criteria.

ND Accountability Application Workbook 24 May 10, 2006

 NORTH DAKOTA CONSOLIDATED STATE APPLICATION ACCOUNTABILITY WORKBOOK

PRINCIPLE 2. All students are included in the State Accountability System.

EXAMPLES FOR EXAMPLES OF

CRITICAL ELEMENT MEETING STATUTORY NOT MEETING
REQUIREMENTS REQUIREMENTS

2.1 How does the State All students in the State are Public school students exist in
Accountability System included in the State the State for whom the State
include all students in the Accountability System. Accountability System makes no
State? provision.
The definitions of “public school”
and “LEA” account for all
students enrolled in the public
school district, regardless of
program or type of public school.

STATE RESPONSE AND STATE ACTIVITIES FOR MEETING REQUIREMENTS

The North Dakota Assessment System assesses all students, regardless of status, within a single,
unified, statewide assessment that measures students’ performance in terms of the State’s challenging
content and achievement standards.

North Dakota state law (NDCC 15.1-21-08) places responsibility with the State Superintendent for the
administration of State assessments to all public school students that are aligned to the State’s content
standards in reading and mathematics (refer to Appendix B: North Dakota Century Code citations or
reference the North Dakota Century Code at the following web site,
http://www.state.nd.us/lr/cencode/T151C21.pdf ). State law requires that the assessments be
administered to at least one grade level selected within each of the following grade spans: grades three
through five; grades six through nine; and grades ten through twelve. The North Dakota Department of
Public Instruction has developed and administers assessments at grades 4, 8, and 12 to correspond with
the State’s content standards.

State law provides for the assessment of all students within the designated grade levels. Therefore, all
students are to be included within the State assessment and accountability system. No exceptions or
systematic exemptions to the State assessment and accountability system are allowed.

ND Accountability Application Workbook 25 May 10, 2006

 NORTH DAKOTA CONSOLIDATED STATE APPLICATION ACCOUNTABILITY WORKBOOK

EXAMPLES FOR EXAMPLES OF

CRITICAL ELEMENT MEETING STATUTORY NOT MEETING
REQUIREMENTS REQUIREMENTS

2.2 How does the State define The State has a definition of “full LEAs have varying definitions of
“full academic year” for academic year” for determining “full academic year.”
identifying students in AYP which students are to be included
decisions? in decisions about AYP. The State’s definition excludes
students who must transfer from
The definition of full academic one district to another as they
year is consistent and applied advance to the next grade.
statewide.
The definition of full academic
year is not applied consistently.

STATE RESPONSE AND STATE ACTIVITIES FOR MEETING REQUIREMENTS

The State of North Dakota requires all students enrolled in public schools within North Dakota to
participate in the State Assessment system. Refer to Appendix B: North Dakota Century Code citations
or reference the North Dakota Century Code at the following web site,
http://www.state.nd.us/lr/cencode/T151C21.pdf ). All students, regardless of their enrollment status,
participate in the State Assessment. This total inclusion policy includes those students who may have
enrolled in a district or school after the beginning of a school year.

For the purpose of identifying students whose achievement results are to be included within a school’s or
LEA’s AYP determination, a student must be in school for the full academic year. A ‘full academic year
means a student has been enrolled at a school or within a LEA for a period equal to or exceeding 173
instructional days, as provided by State statute and recorded within the school’s pupil membership file on
the State’s Online Reporting System.

Any student who may have been enrolled in a school or district after the beginning of a school year is
identified on their assessment demographic sheet. Students or school personnel mark a special code on
the assessment demographic sheet that identifies their late enrollment status. This code identifies the
student and to remove them from the school’s student roll for AYP identification purposes. Refer to page
29 for codes “R” and “S” of Appendix Q: North Dakota State Assessment, Test Coordinator’s Manual
2002-03 for the enrollment code identification fields. A student who has not been enrolled in a school for
the entire year but has been enrolled in the district for the entire year will not be included into AYP
consideration for the school but will be included into AYP consideration for the district.

All students must be accounted for regarding their enrollment status. This is a required entry on the
demographic sheet of all students. Student participation rates will be compared to a school’s and district’s
Average Daily Membership student count used to reimburse schools and districts for their State
foundation aid. Therefore, the State references reimbursement census data to confirm student
participation rates. Refer to page 29 for codes “R” and “S” of Appendix Q: North Dakota State
Assessment, Test Coordinator’s Manual 2002-03 for the enrollment code identification fields. Student
participation rates will be compared to the schools and districts Average Daily Membership student count
used to reimburse school’s and district’s for their State foundation aid. Therefore, the State references
reimbursement census data to confirm student participation rates.

ND Accountability Application Workbook 26 May 10, 2006

 NORTH DAKOTA CONSOLIDATED STATE APPLICATION ACCOUNTABILITY WORKBOOK

The State is developing a statewide student data analysis and reporting system to aid the State in
monitoring the enrollment patterns and participation rates of students. The TetraData application will allow
the State to link district enrollment files with the State’s assessment participation files in order to assure
that all enrolled students are accounted for in the State Assessment system files. Refer to Appendix R:
TetraData Data Analysis and Reporting System Summary for an overview of the TetraData system’s
purpose and design.

The State currently is reviewing its ESEA and accreditation monitoring policies. The State is pursuing an
amendment to its monitoring requirements that would mandate districts to produce evidence regarding
the enrollment dates of all students. Monitors would check for any students who had enrolled after the
beginning of a school year and cross-check their participation status in the State Assessment data file.
Any failures to include such students would be identified as a compliance violation of the school’s and
district’s ESEA compliance agreement. Refer to Appendix S: Consolidated Application Certification
and Assurances for the State Assessment requirement for receipt of federal ESEA funding. A school or
district may be sanctioned for any compliance violation of their ESEA assurances agreement.

It is the expressed policy of the State of North Dakota to include all students within the North Dakota
State Assessment.

ND Accountability Application Workbook 27 May 10, 2006

 NORTH DAKOTA CONSOLIDATED STATE APPLICATION ACCOUNTABILITY WORKBOOK

EXAMPLES FOR EXAMPLES OF

CRITICAL ELEMENT MEETING STATUTORY NOT MEETING
REQUIREMENTS REQUIREMENTS

2.3 How does the State State holds public schools State definition requires students
Accountability System accountable for students who to attend the same public school
determine which students were enrolled at the same public for more than a full academic
have attended the same school for a full academic year. year to be included in public
public school and/or LEA for school accountability.
a full academic year? State holds LEAs accountable for
students who transfer during the State definition requires students
full academic year from one to attend school in the same
public school within the district to district for more than a full
another public school within the academic year to be included in
district. district accountability.

State holds public schools

accountable for students who
have not attended the same
public school for a full academic
year.

STATE RESPONSE AND STATE ACTIVITIES FOR MEETING REQUIREMENTS

All students participating in the State assessment must be accounted for regarding their enrollment
status. This is a required entry on the student demographic sheet of all students. Student participation
rates will be compared to the school’s and district’s Average Daily Membership student count used to
reimburse schools and districts for their State foundation aid. Therefore, the State references
reimbursement census data to confirm student participation rates.

The State requires all schools to account for all students regarding their enrollment status within the
school and district and their inclusion within the State Assessment. The enrollment status of each student
is a required entry on the demographic sheet of all students. Refer to page 29 for codes “R” and “S” of
Appendix Q: North Dakota State Assessment, Test Coordinator’s Manual 2002-03 for the enrollment
code identification fields. Student participation rates will be compared to the school’s and district’s
Average Daily Membership student count used to reimburse school’s and district’s for their State
foundation aid. Therefore, the State references reimbursement census data to confirm student
participation rates.

The State is developing a statewide student data analysis and reporting system to aid the State in
monitoring the enrollment patterns and participation rates of students. The TetraData application will allow
the State to link district enrollment files with the State’s assessment participation files in order to assure
that all enrolled students are accounted for in the State Assessment system files. Refer to Appendix R:
TetraData Data Analysis and Reporting System Summary for an overview of the TetraData system’s
purpose and design.

The State currently is reviewing its ESEA and accreditation monitoring policies. The State is pursuing an
amendment to its monitoring requirements that would mandate districts to produce evidence regarding
the enrollment dates of all students. Monitors would check for any students who had enrolled after the
beginning of a school year and cross-check their participation status in the State Assessment data file.

ND Accountability Application Workbook 28 May 10, 2006

 NORTH DAKOTA CONSOLIDATED STATE APPLICATION ACCOUNTABILITY WORKBOOK

Any failures to include such students would be identified as a compliance violation of the school’s and
district’s ESEA compliance agreement. Refer to Appendix S: Consolidated Application Certification
and Assurances for the State assessment requirement for receipt of federal ESEA funding. A school or
district may be sanctioned for any compliance violation of their ESEA assurances agreement.

It is the expressed policy of the State of North Dakota to include all students within the North Dakota
State Assessment.

ND Accountability Application Workbook 29 May 10, 2006

 NORTH DAKOTA CONSOLIDATED STATE APPLICATION ACCOUNTABILITY WORKBOOK

PRINCIPLE 3. State definition of AYP is based on expectations for growth in

student achievement that is continuous and substantial, such that all students
are proficient in reading/language arts and mathematics no later than 2013-2014.

EXAMPLES FOR EXAMPLES OF

CRITICAL ELEMENT MEETING REQUIREMENTS NOT MEETING
REQUIREMENTS

3.1 How does the State’s The State has a timeline for State definition does not require
definition of adequate yearly ensuring that all students will all students to achieve
progress require all students meet or exceed the State’s proficiency by 2013-2014.
to be proficient in proficient level of academic
reading/language arts and achievement in reading/language State extends the timeline past
mathematics by the 2013- arts 3 and mathematics, not later the 2013-2014 academic year.
2014 academic year? than 2013-2014.

STATE RESPONSE AND STATE ACTIVITIES FOR MEETING REQUIREMENTS

The State of North Dakota stipulates that its State Accountability System provides for an established
timeline that ensures that all students will be proficient in reading and mathematics by the 2013-14
academic year, as specified within ESEA section 1111.

The State of North Dakota has developed and adopted academic achievement standards in mathematics
(refer to Appendix E: North Dakota Mathematics Achievement Standards at the following web site,
http://www.dpi.state.nd.us/standard/perform/index.shtm) and English language arts (refer to Appendix G:
North Dakota English Language Arts Achievement Standards at the following web site,
(http://www.dpi.state.nd.us/standard/perform/index.shtm). These State achievement standards have been
developed at grades 4, 8, and 12 in accordance with North Dakota’s content and achievement standards
protocols (http://www.dpi.state.nd.us/standard/content/toc.pdf). North Dakota mathematics and English
language arts academic achievement standards meet the requirements of section 1111(b)(1). North
Dakota’s achievement standards in science will be completed by the winter 2003, in accordance with
State standards development protocols and section 1111(b)(1) requirements.

It is the policy of the State that all students achieve proficiency as defined within the State’s challenging
achievement standards by the 2013-14 academic year.

For the purposes of determining AYP, proficiency means the aggregation of all student achievement
within the “proficient” and “advanced” performance levels of the State’s achievement standards. Schools
and LEAs must evidence a steady improvement of student achievement from the below-proficient
performance level (the aggregate of the novice and partially-proficient performance level) to the proficient
performance level.

3
If the state has separate assessments to cover its language arts standards (e.g., reading and writing),
the State must create a method to include scores from all the relevant assessments.

ND Accountability Application Workbook 30 May 10, 2006

 NORTH DAKOTA CONSOLIDATED STATE APPLICATION ACCOUNTABILITY WORKBOOK

EXAMPLES FOR EXAMPLES OF

CRITICAL ELEMENT MEETING REQUIREMENTS NOT MEETING
REQUIREMENTS

3.2 How does the State For a public school and LEA to State uses different method for
Accountability System make adequate yearly progress, calculating how public schools
determine whether each each student subgroup must and LEAs make AYP.
student subgroup, public meet or exceed the State annual
school and LEA makes measurable objectives, each
AYP? student subgroup must have at
least a 95% participation rate in
the statewide assessments, and
the school must meet the State’s
requirement for other academic
indicators.

However, if in any particular year

the student subgroup does not
meet those annual measurable
objectives, the public school or
LEA may be considered to have
made AYP, if the percentage of
students in that group who did
not meet or exceed the proficient
level of academic achievement
on the State assessments for that
year decreased by 10% of that
percentage from the preceding
public school year; that group
made progress on one or more of
the State’s academic indicators;
and that group had at least 95%
participation rate on the
statewide assessment.

STATE RESPONSE AND STATE ACTIVITIES FOR MEETING REQUIREMENTS

The State of North Dakota stipulates that the State will determine AYP for each public school and LEA as
provided for within ESEA section 1111, including emphasis on the school identification method
referenced to proficiency ratings, safe harbor provisions, statistical reliability assurances, graduation rates
for secondary schools, attendance rates for elementary and middle schools, and a minimum assessment
participation rate of 95%.

ND Accountability Application Workbook 31 May 10, 2006

 NORTH DAKOTA CONSOLIDATED STATE APPLICATION ACCOUNTABILITY WORKBOOK

The State of North Dakota will only recognize and reference student achievement proficiency ratings
generated by the North Dakota State Assessment and its Alternate Assessment. No other student
achievement assessment tools or means will be recognized. No local assessments will be recognized as
an alternative to the North Dakota State Assessment.

The only definitions of achievement levels recognized by the State AYP accountability system are those
proficiency levels set for the North Dakota State Assessment through the standards-setting process.
Refer to Appendix J: North Dakota State Assessment, Bookmark Standards Setting Technical
Report, 2002 for the established definitions of the North Dakota achievement levels. The State cut scores
for the North Dakota State Assessment will constitute the defining scales for identifying schools and
districts for AYP. Refer to Appendix K: State Superintendent’s Approval Notification of North Dakota
State Assessment Cut Scores for the State’s announced policy regarding the establishment of
achievement level cut scores.

All student achievement data collected during the administration of the State Assessment will be used to
aggregate overall student achievement and to disaggregate student achievement results into each of the
required student subgroups to determine AYP. Refer to Appendix X: North Dakota Assessment
System 2001-02 Baseline Impact Data to review the State’s impact data.

AYP will be determined using 2001-02 school year data as the baseline. The starting points are
calculated pursuant to the prescribed ESEA section 1111 requirements. The same starting point and
annual, measurable objectives apply to all student subgroups resulting in 100% proficiency of all students
by 2013-2014.

In calculating AYP for student aggregated and subgroup populations, the State will employ a binomial
distribution statistical model to ensure high levels of reliability. Ninety-five percent of students,
considering statistical reliability, in each applicable student sub-group must be tested in order for the
school to make AYP.

In calculating AYP for any student subgroup that did not meet the AYP goal but did decrease the
percentage of students in the applicable student sub-group by 10% or more, the school or district will then
be judged to have made AYP if the respective subgroup also meets the state’s other criteria when using
the safe harbor provision (graduation rate for high school and attendance rates for elementary and
secondary). Goals must be met for all applicable student subgroups in accordance with 34 CFR
200.19(d)(2)(i). Following a study of the effects of statistical reliability on safe harbor and with the
concurrence of the State and the U.S. Department of Education, the State will employ the binomial
distribution statistical method within the calculation of safe-harbor status, including safe harbor for
subgroups.

All students’ scores will be used as an aggregate to determine the AYP of schools as a whole. All
schools’ scores will be used as an aggregate to determine the AYP of LEAs. All LEAs scores will be used
as an aggregate to determine the AYP of the State

Please refer to Appendix T: State AYP Computation Rules for the working rules used to determine
AYP.

ND Accountability Application Workbook 32 May 10, 2006

 NORTH DAKOTA CONSOLIDATED STATE APPLICATION ACCOUNTABILITY WORKBOOK

EXAMPLES FOR EXAMPLES OF

CRITICAL ELEMENT MEETING REQUIREMENTS NOT MEETING
REQUIREMENTS

3.2a What is the State’s starting Using data from the 2001-2002 The State Accountability System
point for calculating school year, the State uses a different method for
Adequate Yearly established separate starting calculating the starting point (or
Progress? points in reading/language arts baseline data).
and mathematics for measuring
the percentage of students
meeting or exceeding the State’s
proficient level of academic
achievement.

Each starting point is based, at a

minimum, on the higher of the
following percentages of students
at the proficient level: (1) the
percentage in the State of
proficient students in the lowest-
achieving student subgroup; or,
(2) the percentage of proficient
students in a public school at the
20th percentile of the State’s total
enrollment among all schools
ranked by the percentage of
students at the proficient level.

A State may use these

procedures to establish separate
starting points by grade span;
however, the starting point must
be the same for all like schools
(e.g., one same starting point for
all elementary schools, one same
starting point for all middle
schools…).

ND Accountability Application Workbook 33 May 10, 2006

 NORTH DAKOTA CONSOLIDATED STATE APPLICATION ACCOUNTABILITY WORKBOOK

STATE RESPONSE AND STATE ACTIVITIES FOR MEETING REQUIREMENTS

Based on the administration of the rules identified within section 3.2 above, the State has established
starting points for calculating Adequate Yearly Progress.

Using baseline data from the 2001-02 school year, the State has established starting points of proficiency
separately in reading and math for each grade level. Refer to Appendix X: North Dakota Assessment
System 2001-02 Baseline Impact Data to review the State’s impact data. The same starting point for
reading and math will apply to the aggregate student population within each subject and to each student
subgroup for each of the three grade levels. Each starting point is based, at a minimum, on the higher of
the following percentages of students at the proficient level: (1) the percentage in the State of proficient
students in the lowest-achieving student subgroup; or, (2) the percentage of proficient students in a public
school at the 20th percentile of the State’s total enrollment among all schools ranked by the percentage of
students at the proficient level.

The State has established the following starting points for AYP.

Subject Grade Level

Four Eight Twelve
Reading/language 65.1% 61.4% 42.9%
arts
Mathematics 45.7% 33.3% 24.1%

ND Accountability Application Workbook 34 May 10, 2006

 NORTH DAKOTA CONSOLIDATED STATE APPLICATION ACCOUNTABILITY WORKBOOK

EXAMPLES FOR EXAMPLES OF

CRITICAL ELEMENT MEETING REQUIREMENTS NOT MEETING
REQUIREMENTS

3.2b What are the State’s annual State has annual measurable The State Accountability System
measurable objectives that are consistent uses another method for
objectives for determining with a state’s intermediate goals calculating annual measurable
adequate yearly progress? and that identify for each year a objectives.
minimum percentage of students
who must meet or exceed the The State Accountability System
proficient level of academic does not include annual
achievement on the State’s measurable objectives.
academic assessments.

The State’s annual measurable

objectives ensure that all
students meet or exceed the
State’s proficient level of
academic achievement within the
timeline.

The State’s annual measurable

objectives are the same
throughout the State for each
public school, each LEA, and
each subgroup of students.

STATE RESPONSE AND STATE ACTIVITIES FOR MEETING REQUIREMENTS

Based on the administration of the rules identified within section 3.2 above, the State has established
measurable objectives for determining Adequate Yearly Progress.

Using baseline data from the 2001-02 school year, the State has established measurable objectives for
proficiency separately in reading and math for each year from 2001-02 to 2013-14. Refer to Appendix X:
North Dakota Assessment System 2001-02 Baseline Impact Data to review the State’s impact data.
The same measurable objectives for reading and math will apply to the aggregate student population
within each subject and to each student subgroup for each of the three grade levels.

The measurable objectives are determined using the baseline percentage of proficient students statewide
from the 2001-02 school year and prorating the expected annual growth required to achieve 100% by
2013-14. The following chart identifies the measurable objectives established for Adequate Yearly
Progress.

ND Accountability Application Workbook 35 May 10, 2006

 NORTH DAKOTA CONSOLIDATED STATE APPLICATION ACCOUNTABILITY WORKBOOK

Subject/ School Years

Grades 01- 02- 03- 04- 05- 06- 07- 08- 09- 10- 11- 12- 13-
02 03 04 05 06 07 08 09 10 11 12 13 14
Reading
4 65.1 68.0 70.9 73.8 76.7 79.6 82.6 85.5 88.4 91.3 94.2 97.1 100.0
8 61.4 64.6 67.8 71.1 74.3 77.5 80.7 83.9 87.1 90.4 93.6 96.8 100.0
12 42.9 47.7 52.4 57.2 61.9 66.7 71.5 76.2 81.0 85.7 90.5 95.2 100.0

Math
4 45.7 50.2 54.8 59.3 63.8 68.3 72.9 77.4 81.9 86.4 91.0 95.5 100.0
8 33.3 38.9 44.4 50.0 55.5 61.1 66.7 72.2 77.8 83.3 88.9 94.4 100.0
12 24.1 30.4 36.8 43.1 49.4 55.7 62.1 68.4 74.7 81.0 87.4 93.7 100.0
Graduation 89.9 TBD
Attendance 93.0

* 2004-05 will initiate a new method for determining graduation rates based on the disaggregated
tracking of 9th grade cohorts through to graduation. In 2005 the State will recalculate the target
graduation rate using the 20% ranking rule for graduation rates.

ND Accountability Application Workbook 36 May 10, 2006

 NORTH DAKOTA CONSOLIDATED STATE APPLICATION ACCOUNTABILITY WORKBOOK

EXAMPLES FOR EXAMPLES OF

CRITICAL ELEMENT MEETING REQUIREMENTS NOT MEETING
REQUIREMENTS

3.2c What are the State’s State has established The State uses another method
intermediate goals for intermediate goals that increase for calculating intermediate goals.
determining adequate in equal increments over the
yearly progress? period covered by the State The State does not include
timeline. intermediate goals in its definition
of adequate yearly progress.
• The first incremental
increase takes effect not
later than the 2004-2005
academic year.

• Each following incremental

increase occurs within
three years.

STATE RESPONSE AND STATE ACTIVITIES FOR MEETING REQUIREMENTS

The State of North Dakota stipulates that the State Superintendent has established State intermediate
goals for determining adequate yearly progress that meet the provisions of ESEA section 1111. The
intermediate goals are based on the respective measurable objectives established from the 2001-02
baseline data, set forth within Principle 3.2b above. Refer to Appendix X: North Dakota Assessment
System 2001-02 Baseline Impact Data to review the State’s impact data.

The intermediate goals will be based on the respective measurable objectives for reading, mathematics,
graduation, and attendance determined independently and defined for the following years:

Step 1: 2001-02 through 2003-04 and set at the 2001-02 baseline AYP cut-point.
Step 2: 2004-05 through to 2006-07 and set at the 2004-05 measurable objective;
Step 3: 2007-08 through to 2009-10 and set at the 2008-09 measurable objective;
Step 4: 2010-11 through to 2012-13 and set at the 2010-11 measurable objective; and
Step 5: 2013-14 and set at the 2013-14 measurable objective.

The intermediate goals will be the effective AYP cut-point for all years within each respective step. The
intermediate goal will constitute the AYP cut-point upon which all school and district program
improvement identification will be made. The following chart identifies the respective intermediate goals
for each respective subject and grade level.

ND Accountability Application Workbook 37 May 10, 2006

 NORTH DAKOTA CONSOLIDATED STATE APPLICATION ACCOUNTABILITY WORKBOOK

Subject/ School Years

Grades 01-02 02-03 03-04 04-05 05-06 06-07 07-08 08-09 09-10 10-11 11-12 12-13 13-14
Reading
4 65.1 73.8 82.6 91.3 100.0
8 61.4 71.1 80.7 90.4 100.0
12 42.9 57.2 71.5 85.7 100.0

Math
4 45.7 59.3 72.9 86.4 100.0
8 33.3 50.0 66.7 83.3 100.0
12 24.1 43.1 62.1 81.0 100.0
Graduation 89.9 *
Attendance 93.0

* 2004-05 will initiate a new method for determining graduation rates based on the disaggregated
tracking of 9th grade cohorts through to graduation. In 2005 the State will recalculate the target
graduation rate using the 20% ranking rule for graduation rates.

ND Accountability Application Workbook 38 May 10, 2006

 NORTH DAKOTA CONSOLIDATED STATE APPLICATION ACCOUNTABILITY WORKBOOK

PRINCIPLE 4. State makes annual decisions about the achievement of all public
schools and LEAs.

EXAMPLES FOR EXAMPLES OF

CRITICAL ELEMENT MEETING REQUIREMENTS NOT MEETING
REQUIREMENTS

4.1 How does the State AYP decisions for each public AYP decisions for public schools
Accountability System school and LEA are made and LEAs are not made annually.
make an annual annually. 4
determination of whether
each public school and LEA
in the State made AYP?

STATE RESPONSE AND STATE ACTIVITIES FOR MEETING REQUIREMENTS

The State of North Dakota stipulates that it will conduct annual reviews of school and district achievement
data for the purposes of determining whether each public school and LEA had made AYP as provided
within ESEA section 1111.

North Dakota state law (NDCC 15.1-21-08) places responsibility with the State Superintendent for the
annual administration of State assessments that are aligned to the State’s content standards in reading
and mathematics (refer to Appendix B: North Dakota Century Code citations or reference the North
Dakota Century Code at the following web site, http://www.state.nd.us/lr/cencode/T151C21.pdf ). State
law requires that the assessments be administered annually to at least one grade level selected within
each of the following grade spans: grades three through five; grades six through nine; and grades ten
through twelve. The North Dakota Department of Public Instruction has developed and administers
assessments at grades 4, 8, and 12 to correspond with the State’s content standards. The State will use
only the grade span assessments (grades 4, 8, and 11) during 2005-06 for the purposes of determining
adequate yearly progress for all public schools and districts. During 2005-06 the State will not use the
most advantageous result of averages over one, two, and three years when determining adequate yearly
progress. The State will base adequate yearly progress decisions for 2005-06 for every public school and
district upon the three-year uniform average of data from assessments in grades 4, 8, and 11

State law further requires that the State assessments compile aggregated results and disaggregated
results. The annual State assessments must compile student achievement data that allows for a
comparison of individual students, classrooms within a given school and school district, schools within the
district, and school districts within the state. The test scores must also allow for comparisons based on
students’ gender, ethnicity, economic status, service status (i.e., migrant, LEP, disability), and
assessment status (i.e., enrollment status and participation status), unless doing so enables the
identification of any student. (Refer to NDCC 15.1-21-08 within Appendix B: North Dakota Century
Code citations or reference the North Dakota Century Code at the following web site,
http://www.state.nd.us/lr/cencode/T151C21.pdf ).

The State will make its annual AYP review and determinations based solely on student achievement data
generated by the annual State Assessment and on official graduation and attendance data reported to
and monitored by the State. The State will issue annual AYP status reports to all LEAs and schools
identifying each LEA’s and school’s overall performance in terms of AYP performance goals.

4
Decisions may be based upon several years of data and data may be averaged across grades within a
public school [§1111(b)(2)(J)].

ND Accountability Application Workbook 39 May 10, 2006

 NORTH DAKOTA CONSOLIDATED STATE APPLICATION ACCOUNTABILITY WORKBOOK

The State will produce all district- and school-level reports for districts and schools regarding their
respective student achievement levels. These reports will include both aggregated and disaggregated
student achievement data. Refer to Appendix O: North Dakota School Report Card and Profile for an
illustration of the content of these student achievement profiles. Districts may use these profile reports as
the foundation for their locally produced profiles. The State-generated reports will offer quality assurances
regarding the generation of any district achievement data.

The State requires all districts to disseminate student achievement report cards and profiles to their
communities. This mandate is required as a condition of their receipt of federal funds. Refer to Appendix S:
Consolidated Application Certification and Assurances for the State assessment requirement for
receipt of federal ESEA funding. To assure compliance with this provision for the development and
dissemination of performance profiles per Title I funding, the Department will require timely and
comprehensive reports as a condition of receiving uninterrupted Title I funds. Further, evidence of these
profiles will be one of the criteria in the Department’s Title I monitoring program.

The State currently is reviewing its ESEA and accreditation monitoring policies. The State is pursuing an
amendment to its monitoring requirements that would mandate districts to produce evidence regarding
the dissemination of achievement profiles to their communities. Monitors would check for the production
and dissemination of such achievement profiles. Any failures to disseminate such profiles would be
identified as a compliance violation of the school’s and district’s ESEA compliance agreement. Refer to
Appendix S: Consolidated Application Certification and Assurances for the State Assessment
requirement for receipt of federal ESEA funding. A school or district may be sanctioned for any
compliance violation of their ESEA assurances agreement.

All AYP review and determination activity will be conducted annually and completed by July of each
respective year.

ND Accountability Application Workbook 40 May 10, 2006

 NORTH DAKOTA CONSOLIDATED STATE APPLICATION ACCOUNTABILITY WORKBOOK

PRINCIPLE 5. All public schools and LEAs are held accountable for the
achievement of individual subgroups.

EXAMPLES FOR EXAMPLES OF

CRITICAL ELEMENT MEETING REQUIREMENTS NOT MEETING REQUIREMENTS

5.1 How does the definition of Identifies subgroups for defining State does not disaggregate data
adequate yearly progress adequate yearly progress: by each required student
include all the required economically disadvantaged, subgroup.
student subgroups? major racial and ethnic groups,
students with disabilities, and
students with limited English
proficiency.

Provides definition and data

source of subgroups for adequate
yearly progress.

STATE RESPONSE AND STATE ACTIVITIES FOR MEETING REQUIREMENTS

The State of North Dakota stipulates that the State’s definition of adequate yearly progress includes all
required subgroups as provided within ESEA section 1111.

State law requires that the State assessments compile aggregated results and disaggregated results. The
State assessments must compile student achievement data that allows for a comparison of individual
students, classrooms within a given school and school district, schools within the district, and school
districts within the state. The test scores must also allow for comparisons based on students’ gender,
ethnicity, economic status, service status (i.e., migrant, LEP, and disability), and assessment status (i.e.,
enrollment and participation status), unless doing so enables the identification of any student. (Refer to
NDCC 15.1-21-08 within Appendix B: North Dakota Century Code citations or reference the North
Dakota Century Code at the following web site, http://www.state.nd.us/lr/cencode/T151C21.pdf ).

The results generated by the North Dakota State Assessment are reported in mathematics and reading
for grades 4, 8, and 12. Results are reported at the individual student, school, district, and State level.
Results are disaggregated by gender, ethnicity, disability, limited English proficiency status, migrant
status, and economic disadvantaged status. AYP determination includes consideration for ethnicity,
disability, limited English proficiency, and economic status. The following tables summarize the level of
the disaggregated reports.

ND Accountability Application Workbook 41 May 10, 2006

 NORTH DAKOTA CONSOLIDATED STATE APPLICATION ACCOUNTABILITY WORKBOOK

Disaggregated Levels for ND State Assessment in

Mathematics and Reading
Grades 4, 8, and 12
(** refers to AYP subgroups)
Reporting Individual School District State
Level Student
Gender
N/A * * *
Ethnicity **
N/A * * *
Disability **
N/A * * *
Limited English
Proficient ** N/A * * *

Migrant
N/A * * *
Economically
Disadvantaged ** N/A * * *

The State and its assessment contractor assume the full responsibility for generating aggregate and
disaggregated student achievement reports. Local districts do not generate these reports.

Student demographic information is gathered at the time of the assessment administration on the
individual student’s assessment demographic sheet. On this sheet the student or a school official
completes basic information about the student, including their name and other essential information. The
assessment requires completion of certain demographic and special codes that are included on the
demographic sheet and detailed for testing coordinators within the Test Coordinator’s Manual. Refer to
pages 28-31 of Appendix Q: North Dakota State Assessment, Test Coordinator’s Manual 2002-03
for a listing of the various demographic and special categories used to describe a student. These codes
are then used during the process of classifying student achievement by subgroup populations.

A central concern of any demographic collection process is the introduction of erroneous information on
the part of an individual. This is especially troublesome within an assessment system where information
can be inadvertently omitted. The State of North Dakota has established a plan to centralize student
identification information with the use of a data analysis and reporting application contracted through
TetraData Corporation. Refer to Appendix R: TetraData Data Analysis and Reporting System
Summary for an overview of the application. This application will allow the State to routinely link student
identification information statewide with the database supplied by CTB/McGraw-Hill in order to identify
and reconcile incorrect information. The use of this data linkage application will enhance the accuracy,
reliability, and speed of collecting the demographic information used to classify school, district, and State
subgroup achievement reports.

Disaggregated reports will approximate the presentation format identified with the State Report Card and
Profile. Refer to Appendix O: North Dakota State Report Card and Profile for an example of the
presentation format used to report disaggregated subgroup achievement data.

The State alone may authorize the publication of any reports regarding the State Assessment for
accountability purposes based on State Assessment data. The State’s contractor (CTB/McGraw-Hill)
produces all reports for the State Assessment. The State recognizes no other assessment reports
produced by other outside sources, including districts and schools, as authoritative regarding the State
Assessment.

ND Accountability Application Workbook 42 May 10, 2006

 NORTH DAKOTA CONSOLIDATED STATE APPLICATION ACCOUNTABILITY WORKBOOK

EXAMPLES FOR EXAMPLES OF

CRITICAL ELEMENT MEETING REQUIREMENTS NOT MEETING REQUIREMENTS

5.2 How are public schools Public schools and LEAs are held State does not include student
and LEAs held accountable for student subgroup subgroups in its State
accountable for the achievement: economically Accountability System.
progress of student disadvantaged, major ethnic and
subgroups in the racial groups, students with
determination of adequate disabilities, and limited English
yearly progress? proficient students.

STATE RESPONSE AND STATE ACTIVITIES FOR MEETING REQUIREMENTS

The State of North Dakota stipulates that it will conduct an AYP review and determination for each school
and LEA based on the progress of student subgroups as provided within ESEA section 1111.

As identified in Principle 5.1 above, the State provides a method to identify, record, and report student
achievement for all subgroups. The State will disaggregate and hold schools and LEAs accountable for the
performance of each of the following student subgroups:

• All Students
• Asian/Pacific
• Black
• Hispanic
• Native American
• White
• Economic disadvantaged
• Limited English Proficient
• Students with Disabilities

The State will determine whether each subgroup within each school and LEA achieved the annual
measurable objective, or met the “Safe Harbor” provision, and met the 95% participation rate criteria. For
a school or LEA to make AYP, every group for which a school or LEA is accountable must make AYP.
Any subgroup that makes AYP based on the safe harbor provision must also make AYP based on the
appropriate secondary indicator (i.e., graduation rate or attendance rate). The rules for statistical reliability
will apply in reviewing and determining subgroup accountability.

ND Accountability Application Workbook 43 May 10, 2006

 NORTH DAKOTA CONSOLIDATED STATE APPLICATION ACCOUNTABILITY WORKBOOK

EXAMPLES FOR EXAMPLES OF

CRITICAL ELEMENT MEETING REQUIREMENTS NOT MEETING REQUIREMENTS

5.3 How are students with All students with disabilities The State Accountability System
disabilities included in the participate in statewide or State policy excludes students
State’s definition of assessments: general with disabilities from participating
adequate yearly progress? assessments with or without in the statewide assessments.
accommodations or an alternate
assessment based on grade level State cannot demonstrate that
standards for the grade in which alternate assessments measure
students are enrolled. grade-level standards for the
grade in which students are
State demonstrates that students enrolled.
with disabilities are fully included
in the State Accountability
System.

STATE RESPONSE AND STATE ACTIVITIES FOR MEETING REQUIREMENTS

The State of North Dakota stipulates that all students with disabilities will be included in the State’s
definition of adequate yearly progress as provided within ESEA section 1111.

State law requires that the State assessments compile aggregated results and disaggregated results. The
State assessments must compile student achievement data that allows for a comparison of classrooms
within a given school and school district, schools within the district, and school districts within the state.
The test scores must also allow for comparisons based on students’ gender, ethnicity, economic status,
service status (i.e., migrant, LEP, and disability), and assessment status (i.e., enrollment and participation
status), unless doing so enables the identification of any student. (Refer to NDCC 15.1-21-08 within
Appendix B: North Dakota Century Code citations or reference the North Dakota Century Code at the
following web site, http://www.state.nd.us/lr/cencode/T151C21.pdf ).

It is the policy of the Department of Public Instruction to include all students with disabilities in the North
Dakota accountability system. See enclosed Appendix U: Individualized Education Program Planning
Process, or access this document at http://www.dpi.state.nd.us/speced/guide/iep/index.shtm. The State’s
individualized education program (IEP) form (page 4), required for every student eligible under the
Individuals with Disabilities Education Act (IDEA), includes a section requiring the description of the
student’s participation in district-wide and statewide assessments. The IEP team must indicate whether
the student will participate without accommodations, with accommodations (which must be stated), or in
the Alternate Assessment. This element of the IEP is addressed by the school district as it conducts a
self-assessment in preparation for the Office of Special Education monitoring. If violations are found,
corrective actions are determined and evidence of completion is required.

The State Assessment Program Test Coordinator’s Manual, 2002-2003, (Appendix Q) provides very
limited opportunity for a school to exclude a student from participation in the State Assessment by
invalidating an assessment. Any school that proposes to invalidate a student’s test must provide written
documentation to the Department of Public Instruction stating the reason for test invalidation. The
authorizing administrator must sign the form. If a school systematically fails to include students in the
State Assessment, sanctions will be imposed. Any non-participating student or any invalidated
assessment will be included into the calculation to determine the participation rate of the school, district
and State.

ND Accountability Application Workbook 44 May 10, 2006

 NORTH DAKOTA CONSOLIDATED STATE APPLICATION ACCOUNTABILITY WORKBOOK

All students who participate in the North Dakota Alternate Assessment will have levels of performance
included within the State accountability system. The State Assessment Program Test Coordinator’s
Manual, 2002-2003, provides guidance for use of the Alternate Assessment (Appendix Q), and in the
use of accommodations (pages 33 – 35).

As provided under pending federal regulations and allowances offered by the U.S. Department of
Education, the State may use alternate achievement standards to calculate AYP for schools and districts
for students with the most significant cognitive disabilities who take an alternate assessment. This
allowance is offered until the newly proposed regulations related to alternate assessments become
finalized. The State attests that the State’s alternate assessments and achievement standards are
aligned with North Dakota’s academic content standards and reflect qualified, professional judgment of
the highest learning standards possible for these students.

Additionally, the State acknowledges that the percentage of students identified as proficient on the
alternate assessment at the district and State levels may not exceed 1.0 percent of all students in the
grades assessed. The State stipulates that it will monitor the participation and achievement rates of
students in the alternate assessments within each district and across the State; no district or the State will
be permitted to exceed this limit. The State will calculate and monitor the overall enrollment of students,
the participation rates of students within the State Assessment, and the participation and achievement
rates of students within the Alternate Assessment.

The incorporation of an alternate assessment provision identified in federal guidance, dated May 10,
2005, affecting students with persistent cognitive disabilities, beginning with the 2004-05 school year.

On May 10, 2005, U.S. Secretary of Education Margaret Spellings announced the details of a new policy
designed to assist students with persistent cognitive disabilities to more appropriately participate in their
state assessment and accountability system. Under this policy a limited number of students with
persistent cognitive disabilities, approximating two percent, will be allowed to have the results from
alternate assessments classified as proficient when specifically based on alternate achievement
standards. This provision is a policy separate from and in addition to the current regulation that allows up
to one percent of all students being tested, i.e., those students with a significant cognitive disability, to
have the results from alternate assessments classified as proficient based on alternate achievement
standards. This proposed provision would apply to the 2004-05 school year calculations affecting
adequate yearly progress decisions. This provision would apply statewide only to those schools, districts,
or the State that would not have made adequate yearly progress solely based on the achievement of
students with disabilities.

The Department of Public Instruction submits that the State is eligible to exercise the provision of applying
a proxy-adjusted proficiency rate for students with disabilities, as outlined in the May 10, 2005 guidance.

Under this provision the State will calculate a proxy (i.e., a statewide proficiency rate constant) that will be
applied to empirical proficiency rates to determine an adjusted proficiency rate of students with
disabilities. This adjusted percent proficient is what will be used to reexamine if a school, district, or the
State made adequate yearly progress for the 2004-05 school year. This adjusted proficiency rate will be
derived through the following process.

1. The State will calculate what 2.0 percent of the total number of students assessed within the
State equals solely within the students with disabilities subgroup, by dividing 2.0 by the
percentage of students who have disabilities statewide. This number, which will be a constant for
every school, will be the basis for flexibility in school adequate yearly progress determinations.

ND Accountability Application Workbook 45 May 10, 2006

 NORTH DAKOTA CONSOLIDATED STATE APPLICATION ACCOUNTABILITY WORKBOOK

2. The State will identify all schools that did not make adequate yearly progress solely on the basis
of the students with disabilities subgroup and the proficiency rate of those students in each
school, district, and the State.

3. The State will calculate the adjusted percent proficient for the students with disabilities subgroup
for each school, district, and the State. This adjustment is equal to the sum of the actual percent
of proficient scores of this subgroup plus the proxy percent calculated in Step 1.

4. The State will compare this adjusted percent proficient to the state’s annual measurable objective
for each school identified in Step 2. This comparison will be conducted without the use of the
state’s binomial distribution reliability model.

a. If the adjusted proficiency rate for the students with disabilities subgroup for each school,
district, or the State meets or exceeds the state’s measurable objective, the school,
district, or the State may be considered to have made adequate yearly progress for the
2004-05 school year.

b. If the adjusted proficiency rate for the students with disabilities subgroup for each school,
district, or the State does not meet or exceed the state’s annual measurable objective,
then the school, district, or the State will not have made adequate yearly progress.
5. This process will be applied to reading and mathematics separately and will be applied to
schools, districts, and the State.

6. The actual percent proficient will be reported to parents and the public; the State reserves the
right to also report the adjusted percent proficient.

During 2005-06, the State will provide for the use of an alternate assessment for students with persistent
disabilities for the purposes of determining adequate yearly progress and restricting any such
determinations of proficient and advanced scores to not greater than two percent of students assessed on
the North Dakota state alternate assessment.

The Department of Public Instruction stipulates that the North Dakota assessment and accountability
system has met all requirements as specified within Title I and Individuals with Disabilities Education Act
regulations. The state’s assessment system met full compliance through the 2003 peer review process
and will undergo peer review again in November 2005 May 2006. The Department of Public Instruction
stipulates that it will abide by the established core principles identified within the May 10, 2005 guidance.

1. Statewide assessment participation rates for students with disabilities, for purposes of measuring
adequate yearly progress, will remain at or above 95%. During the 2003-04 school year, the
statewide students with disabilities participation rate equaled 98.05% in reading and 97.83% in
mathematics. (Reference the following web address for the state report:
http://www.dpi.state.nd.us/dpi/reports/profile/0304/AYPState/aypstate2004.pdf.);

2. Alternate assessments in reading/language arts and mathematics are available for students with
disabilities who are unable to participate in the regular assessment even with accommodations
(as required by the Individuals with Disabilities Education Act) and the State will report results to
the Secretary and the public based on these alternate assessments in reading/language arts and
mathematics;

3. Appropriate accommodations are available for the state assessment for students with disabilities;

4. The State stipulates that all students with disabilities will continue to be included in the state’s
assessment system. The Department of Public Instruction will cross-check that all identified
special education students will be included in the statewide assessment file, as captured on the
state’s Online Reporting System. This system also monitors special education enrollment for
payment purposes.

ND Accountability Application Workbook 46 May 10, 2006

 NORTH DAKOTA CONSOLIDATED STATE APPLICATION ACCOUNTABILITY WORKBOOK

The Department of Public Instruction has previously submitted support data and narrative within the
state’s Consolidated Performance Report, which is available to the U.S. Department of Education, to
establish the state’s eligibility status. Following a review by the U.S. Department of Education on the
state’s eligibility status to receive this proposed provision, the Department of Public Instruction will submit
additional evidence as identified within the “Sound State Education Policies” within the May 10, 2005
guidance.

ND Accountability Application Workbook 47 May 10, 2006

 NORTH DAKOTA CONSOLIDATED STATE APPLICATION ACCOUNTABILITY WORKBOOK

EXAMPLES FOR EXAMPLES OF

CRITICAL ELEMENT MEETING REQUIREMENTS NOT MEETING REQUIREMENTS

5.4 How are students with All LEP student participate in LEP students are not fully
limited English proficiency statewide assessments: general included in the State
included in the State’s assessments with or without Accountability System.
definition of adequate accommodations or a native
yearly progress? language version of the general
assessment based on grade level
standards.

State demonstrates that LEP

students are fully included in the
State Accountability System.

STATE RESPONSE AND STATE ACTIVITIES FOR MEETING REQUIREMENTS

The State of North Dakota stipulates that all limited English proficient students will be included in the
State’s definition of adequate yearly progress as provided within ESEA section 1111.

State law requires that the State assessments compile aggregated results and disaggregated results. The
State assessments must compile student achievement data that allows for a comparison of classrooms
within a given school and school district, schools within the district, and school districts within the state.
The test scores must also allow for comparisons based on students’ gender, ethnicity, economic status,
service status (i.e., migrant, LEP, and disability) and assessment status (i.e., enrollment and participation
status), unless doing so enables the identification of any student. (Refer to NDCC 15.1-21-08 within
Appendix B: North Dakota Century Code citations or reference the North Dakota Century Code at the
following web site, http://www.state.nd.us/lr/cencode/T151C21.pdf ).

It is the policy of the Department of Public Instruction to include all LEP students in the State Assessment
program. The State Consolidated Application (Part I-H), which was approved by the U.S. Department of
Education (see Appendix H), indicates the State’s commitment to include all students in the State
Assessment.

The School Report Card and Profile, as illustrated in Appendix O, reports LEP student achievement
against the State standards, compared with other students.

Accountability for LEP student achievement is predicated on the ability of schools and LEAs to assess all
students suspected of having limited English proficiency, to identify those meeting the federal definition of
LEP, and to record all LEP students who participate in the State Assessment. The Department of Public
Instruction reconciles all discrepancies in LEP student numbers reported via the Survey of the State’s
Limited English Proficient Students and Available Educational Programs and Services compared with
State Assessment statistics and the TetraData data analysis and reporting system.

The State provides to LEP student the right to accommodations in the classroom and in the State
Assessment. Accommodations are listed in the Test Coordinator’s Manual for the statewide achievement
testing program on pages 33 - 35, located in Appendix Q, and at the following website:
http://www.dpi.state.nd.us/testing/assess/testmanl.pdf. The North Dakota State Task Force on Limited
English Proficiency, convened by the Department of Public Instruction in 2000, developed guidance for
LEP students and state content standards. During the 2002-2003 school year, the Task Force will further

ND Accountability Application Workbook 48 May 10, 2006

 NORTH DAKOTA CONSOLIDATED STATE APPLICATION ACCOUNTABILITY WORKBOOK

refine the guidance for limited English proficient students and statewide achievement testing. This will be
provided to schools and districts, along with specific accommodations for the levels of English language
proficiency. See Appendix H: North Dakota State ESEA Consolidated Application for further
discussion of accommodations.

The incorporation of federal guidance regarding the assessment of new limited English proficient (LEP)
students and the extended classification of LEP students for two years beyond program status for the
purposes of determining adequate yearly progress.

On February 20, 2004, the U.S. Department of Education released guidance regarding the assessment of
new LEP students and the inclusion of new LEP students within the State’s accountability system. This
guidance acknowledges that teachers need sufficient time with their students to provide high-quality
instruction before it is appropriate to hold a particular school accountable. The following provisions are
limited by federal regulation and restricted only to those LEP students who are newly arrived to the United
States within the allowance of the current instructional year. Any LEP student identified outside this
restricted definition of “new” will be covered by provisions of the current State Accountability Workbook.
The following provisions apply.

a. Definition of first year of enrollment. A “new” LEP student is understood as any student who has arrived
in the United States and whose direct instruction is initiated and conducted within the course of the
current school year. Under the State’s accountability plan for schools and districts, within the limits of
federal regulation, only those students who have been under the direct instruction of a school since the
beginning of the school year will be included in the determination of adequate yearly progress under the
assessment provisions outlined in subsection b below. The designation of entry into the United States is
recorded within the LEP record of the pupil membership file on the State’s Online Reporting System.

b. Assessing new limited English proficient students. For purposes of participation in the State
assessment system, LEP students, during their first year of enrollment in U.S. schools, must take an
English proficiency assessment, as provided within Section 1111(b)(7) of the Act, and may participate in
the reading/language arts assessment, as provided within Section 1111(b)(3) of the Act. The assessment
results of first-year LEP students who take the reading/language arts assessment will not be included in
determining adequate yearly progress for achievement,. New LEP students who take the English
language proficiency assessment and/or the reading/language arts assessment, will be counted as
participants toward meeting the 95% assessment participation required for adequate yearly progress
determinations for reading.

New LEP students will take the State’s mathematics assessment, as provided within Section 1111(b)(3)
of the Act, with accommodations as necessary; however, the State will not include these new LEP
students’ assessment results from the mathematics assessments during their first year in U.S. schools.
New LEP students who take the mathematics assessment will be counted as participants toward meeting
the 95% assessment participation required for adequate yearly progress determinations for mathematics.

c. Two-year extension of LEP status for accountability reporting. The State will include in the LEP
subgroup any student who had previously been considered an LEP student during the past one or two
years, to calculate adequate yearly progress for schools, districts, and the State. The determination of
when a student has attained English language proficiency and is no longer an LEP student will follow the
State’s method and definition as provided within the State’s Consolidated Application. New LEP students
will not be included into the base calculation of determining a minimum LEP subgroup size.

The Department of Public Instruction will conduct an independent review of all new LEP student reports,
as captured within the State’s pupil membership file, and validate all data points used to substantiate the
claim of a new LEP student’s status.

ND Accountability Application Workbook 49 May 10, 2006

 NORTH DAKOTA CONSOLIDATED STATE APPLICATION ACCOUNTABILITY WORKBOOK

EXAMPLES FOR EXAMPLES OF

CRITICAL ELEMENT MEETING REQUIREMENTS NOT MEETING REQUIREMENTS

5.5 What is the State's State defines the number of State does not define the required
definition of the minimum students required in a subgroup number of students in a subgroup
number of students in a for reporting and accountability for reporting and accountability
subgroup required for purposes, and applies this purposes.
reporting purposes? For definition consistently across the
accountability purposes? State. 5 Definition is not applied
consistently across the State.
Definition of subgroup will result in
data that are statistically reliable. Definition does not result in data
that are statistically reliable.

STATE RESPONSE AND STATE ACTIVITIES FOR MEETING REQUIREMENTS

The State of North Dakota has established a definition for the minimum number of students in a subgroup
for both reporting and accountability purposes. The definition is consistent with the minimum number
identified within Principle 9. The State has established a test of statistical significance for the method of
determining a minimum number within a given population and referenced to the established measurable
objective, safe harbor, participation rate, graduation rate, or attendance rate. Refer to Principle 9.1 for a
detailed overview of this method of statistical significance.

5
The minimum number is not required to be the same for reporting and accountability.

ND Accountability Application Workbook 50 May 10, 2006

 NORTH DAKOTA CONSOLIDATED STATE APPLICATION ACCOUNTABILITY WORKBOOK

EXAMPLES FOR EXAMPLES OF

CRITICAL ELEMENT MEETING REQUIREMENTS NOT MEETING REQUIREMENTS

5.6 How does the State Definition does not reveal Definition reveals personally
Accountability System personally identifiable identifiable information.
protect the privacy of information. 6
students when reporting
results and when
determining AYP?

STATE RESPONSE AND STATE ACTIVITIES FOR MEETING REQUIREMENTS

The Department of Public Instruction employs a four level procedure, described below, to eliminate the
possibility of compromising student identification through an inadvertent publication of student
achievement results. These procedures are designed to eliminate any violation of FERPA law regarding
student privacy.

(1) Minimal N Value Rule. The Department employs an N<10 value, where any population value N less
than 10 will prohibit the reporting of students within an identified population. Any population value N of 10
or greater will allow the reporting of students within an identified subgroup.

(2) Single-populated Level Rule. The Department employs a rule where if all students within a school or
subgroup report at a certain performance level and no other performance levels record any students, then
the Department will record a limited percentage of students, presented as an inequality, to serve as a
representative finding. As such, if all students were to reside within a given level, for example “partially
proficient”, then reporting on that level will identify any and all students. This would be a violation. To
remedy this situation, a representative inequality (e.g. <5% or >95%) will be recorded.

(3) Total Population Below Proficient Rule. The Department employs a rule to allow for the proper
identification of a school or district where all students’ achievement scores fall below proficient (i.e., the
combination of partially proficient and novice). It is in the interest of the public and students that any
school or district with 100% below-proficient achievement scores be identified for not making Adequate
Yearly Progress. To eliminate the possibility of identifying any student, the reports for schools and
districts with 100% below-proficient achievement scores will record an inequality to serve as a
representative finding (e.g., <5% or >95%). This representative finding would eliminate any possible
student identification and also allow for the proper identification of the school or district. In the absence of
this rule, extremely low performing schools would be exempt from not making Adequate Yearly Progress,
thereby violating the principle of validity.

(4) Distinguished Students Rule. The Department employs a rule to allow for the proper identification of a
school or district where all students’ achievement scores rest above proficient (i.e., the combination of
proficient and advanced). It is in the interest of the public and students that any school or district with
100% above-proficient achievement scores be identified as making Adequate Yearly Progress. To
eliminate the possibility of identifying any student, the reports for schools and districts with 100% above-
proficient achievement scores will record an inequality to serve as a representative finding (e.g., <5% or

6
The Family Education Rights and Privacy Act (FERPA) prohibits an LEA that receives Federal funds
from releasing, without the prior written consent of a student’s parents, any personally identifiable
information contained in a student’s education record.

ND Accountability Application Workbook 51 May 10, 2006

 NORTH DAKOTA CONSOLIDATED STATE APPLICATION ACCOUNTABILITY WORKBOOK

>95%). This representative finding would eliminate any possible student identification and also allow for
the proper identification of the school or district. In the absence of this rule, high performing schools would
not be recognized for making Adequate Yearly Progress.

ND Accountability Application Workbook 52 May 10, 2006

 NORTH DAKOTA CONSOLIDATED STATE APPLICATION ACCOUNTABILITY WORKBOOK

PRINCIPLE 6. State definition of AYP is based primarily on the State’s academic

assessments.

EXAMPLES FOR EXAMPLES OF

CRITICAL ELEMENT MEETING REQUIREMENTS NOT MEETING
REQUIREMENTS

6.1 How is the State’s Formula for AYP shows that Formula for AYP shows that
definition of adequate decisions are based primarily on decisions are based primarily on
yearly progress based assessments. 7 non-academic indicators or
primarily on academic indicators other than the State
assessments? Plan clearly identifies which assessments.
assessments are included in
accountability.

STATE RESPONSE AND STATE ACTIVITIES FOR MEETING REQUIREMENTS

The State of North Dakota stipulates that the State’s definition of adequate yearly progress is based
primarily on academic assessments as provided within ESEA section 1111.

North Dakota state law (NDCC 15.1-21-08) places responsibility with the State Superintendent for the
administration of State assessments that are aligned to the State’s content standards in reading and
mathematics (refer to Appendix B: North Dakota Century Code citations or reference the North
Dakota Century Code at the following web site, http://www.state.nd.us/lr/cencode/T151C21.pdf ). State
law requires that the assessments be administered to at least one grade level selected within each of the
following grade spans: grades three through five; grades six through nine; and grades ten through twelve.
The North Dakota Department of Public Instruction has developed and administers assessments at
grades 4, 8, and 12 to correspond with the State’s content standards.

The State AYP plan meets the requirements of the ESEA, including emphasis on the school identification
method referenced to student achievement proficiency rating, safe harbor provisions, statistical reliability
assurances, graduation rates for secondary schools, attendance rates for elementary schools, and a
minimum assessment participation rate of 95%. The primary means for the identification of schools and
LEAs is, nevertheless, student achievement data.

The State of North Dakota will only recognize and reference student achievement proficiency ratings
generated by the North Dakota State Assessment and its Alternate Assessment. No other student
achievement assessment tools or means will be recognized. No local assessments will be recognized as
an alternative to the North Dakota State Assessment.

The only definitions of achievement levels recognized by the State AYP accountability system are those
proficiency levels set for the North Dakota State Assessment through the standards-setting process.
Refer to Appendix J: North Dakota State Assessment, Bookmark Standards Setting Technical
Report, 2002 for the established definitions of the North Dakota achievement levels. The State cut scores
for the North Dakota State Assessment will constitute the defining scales for identifying schools and
districts for AYP. Refer to Appendix K: State Superintendent’s Approval Notification of North Dakota
State Assessment Cut Scores for the State’s announced policy regarding the establishment of
performance level cut scores.

7
State Assessment System will be reviewed by the Standards and Assessments Peer Review Team.

ND Accountability Application Workbook 53 May 10, 2006

 NORTH DAKOTA CONSOLIDATED STATE APPLICATION ACCOUNTABILITY WORKBOOK

PRINCIPLE 7. State definition of AYP includes graduation rates for public High schools and an
additional indicator selected by the State for public Middle and public Elementary schools (such
as attendance rates).

EXAMPLES FOR EXAMPLES OF

CRITICAL ELEMENT MEETING REQUIREMENTS NOT MEETING REQUIREMENTS

7.1 What is the State definition State definition of graduation rate: State definition of public high
for the public high school school graduation rate does not
graduation rate? • Calculates the percentage meet these criteria.
of students, measured
from the beginning of the
school year, who graduate
from public high school
with a regular diploma (not
including a GED or any
other diploma not fully
aligned with the state’s
academic standards) in
the standard number of
years; or,

• Uses another more

accurate definition that
has been approved by the
Secretary; and

• Must avoid counting a

dropout as a transfer.

Graduation rate is included (in the

aggregate) for AYP, and
disaggregated (as necessary) for
use when applying the exception
clause 8 to make AYP.

8
See USC 6311(b)(2)(I)(i), and 34 C.F.R. 200.20(b)

ND Accountability Application Workbook 54 May 10, 2006

 NORTH DAKOTA CONSOLIDATED STATE APPLICATION ACCOUNTABILITY WORKBOOK

STATE RESPONSE AND STATE ACTIVITIES FOR MEETING REQUIREMENTS

The State of North Dakota stipulates that it has established the graduation rate of each high school as a
component for determining adequate yearly progress, as provided within ESEA section 1111.

The graduation rate defined within ESEA section 1111 requires the State to report graduates, retentions,
and dropouts, within cohorts, in the aggregate and disaggregated by subgroups. The State has initiated
measures to collect and report this information to the specification of the Act; however, the State’s full
capacity to do so will not become effective until 2005. In the interim, until State data to perform the
required calculations becomes available, the State will define and use an alternative measure, based on
schools’ reported dropout and graduation data within cohorts where graduation occurs in a standard
number of years. The interim measure, effective for the graduating classes of 2003 and 2004, will be
defined by the following equation:

Number of Graduates
_______________________________________________________
(divided by)

Number of Graduates + Dropouts Yr1 + Dropouts Yr2 + Dropouts Yr3 + Dropouts Yr4

The State stipulates that, as required under final Title I regulations, this definition will avoid counting a
dropout as a transfer and will not include students who receive a non-standard diploma (e.g., attendance
certificate, GED). Students that transfer in or out of the school after the State Assessment administration
will not be included in the denominator or numerator.

The State has established the target graduation rate based on the same 20% ranking rule used for
determining achievement targets. Any district with a graduation rate lower than this target point will be
identified for not making Adequate Yearly Progress. This target point will remain as the State reference
for graduation throughout the duration of the 2001-2005 school years. Based on this interim definition, the
State has established a graduation target point of 89.9% based on North Dakota 2001-02 graduation
baseline impact data. This 89.9% target rate will be applied for the first time to 2002-03 graduation rates.
Refer to Appendix Y: North Dakota 2001-02 Graduation Impact Data for a summary of the impact
data.

In 2005 when the State transfers from its current definition of graduation to that used within NCLBA, the
State will recalculate the target graduation rate using the 20% ranking rule for graduation rates. This
target point will remain as the State definition for graduation throughout the duration of the 2005-2014
school years. Therefore, it is anticipated that the State’s interim graduation target point of 89.9% will be
revised with the scheduled 2005 recalculation.

The State will begin reporting graduation rates using the NCLBA definition in 2005, using collected cohort
State data from 2001 – 2005. The rate will be calculated based on the following equation:

# Graduates (with regular diploma) who completed high school in four years

(divided by)

[# Graduates (same as above) + # of 9th grade dropouts/retentions + # 10th grade dropouts/retentions

+ # 11th grade dropouts/retentions + # 12th grade dropouts/retentions + # students who complete 12th
grade without a regular diploma]

ND Accountability Application Workbook 55 May 10, 2006

 NORTH DAKOTA CONSOLIDATED STATE APPLICATION ACCOUNTABILITY WORKBOOK

The data for each class will be tracked forward from 9th grade. Dropouts are defined as students who
leave school prior to graduation for reasons other than transfer to another school. Students who are
retained in grade, and thus leave their original class, will not count toward the number of graduates, but
will be included in the denominator as members of the original class.

The State stipulates that any school or district that has met the requirements of safe harbor for any
specified subgroup must also demonstrate that it has met the requirements for graduation rate for that
same specified subgroup as required under 34 CFR 200.19(d)(2)(i). The State anticipates having a
student data warehouse in place by 2005 to accommodate the monitoring and reporting of disaggregated
graduation rates. Prior to its full implementation, the State will require schools or districts that have met
safe harbor within a specified subgroup to also evidence the achievement of the graduation rate for that
specified subgroup. The State will independently review all school and district information to validate the
authenticity of these data. Following 2005, the State anticipates an ability to automate this activity with the
statewide student data warehouse.

The restricted extension of a graduation cohort beyond four years, as defined within an individualized
education program, for students served within provisions of the Individuals with Disabilities Education Act
or as provided within a LEP service program for specified LEP students;

The graduation rate defined within ESEA section 1111 requires the State to report graduates, retentions,
and dropouts, within cohorts, in the aggregate and disaggregated by subgroups. The Department
reaffirms its commitment to proceed with the determination of adequate yearly progress based on these
elements.

The Department of Public Instruction is also mindful of its responsibility to administer the provisions of the
Individuals with Disabilities Education Act. Inherent throughout the Act is the paramount importance of
schools to provide appropriate instruction to each student with a disability according to the design of that
student’s unique individualized education program. By definition, a student’s individualized education
program sets the course of study for that student, including curriculum, instructional strategy, assessment,
service supports, and educational schedule, including anticipated graduation. Within North Dakota law,
services to students with disabilities may extend beyond the traditional twelve years up to the age of 21
inclusive. In such circumstances, a student with a disability may properly graduate, according to the
dictates of the student’s individualized education program, at age 21, several years beyond a traditional
graduation that, for a typical student, may be completed within four years of entering high school.

To ensure that schools properly exercise their instructional duties according to a student’s individualized
education program, the Department of Public Instruction monitors school’s compliance with the provisions
of the IDEA, including the proper development and administration of a student’s individualized education
program. The Department monitors graduation rates of all students with disabilities, including those
students whose graduation rates extend to age 21. The State provides oversight on all services provided
to students with disabilities, including the proper conclusion of their services and the bestowal of
graduation at a time prescribed within the student’s individualized education program.

Given the high educational standards and service schedules set forth within a student’s individualized
education program and the civil rights granted to students with disabilities to receive the full benefit of
these standards and service schedules, it is incumbent on the State to offer every support to schools to
provide the full benefit of instruction to all students with disabilities, regardless of the duration of their
education. It is likewise incumbent on the State to eliminate any barriers that might impede or otherwise
deter schools from properly administering their duties to all students, regardless of disability status. This
concern includes the bestowal of a standard graduation on students with disabilities, whose individualized
education programs require a high school instruction period that extends beyond the traditional four
years. Any policy that places pressures on schools to divert their full attention on the needs of students
with disabilities must be reviewed and amended accordingly.

ND Accountability Application Workbook 56 May 10, 2006

 NORTH DAKOTA CONSOLIDATED STATE APPLICATION ACCOUNTABILITY WORKBOOK

It is to this aim, to eliminate any impediment to the proper delivery of high school instruction to some
students with disabilities, that the Department of Public Instruction allows for the extension of the
standard graduation date beyond the traditional four years for students with disabilities whose individual
education programs identify such a need. This extension meets the spirit of the Individuals with
Disabilities Education Act and to advance a considered policy that encourages the properly paced
delivery of instruction for all students with disabilities.

Based on the above stated principles, but uniquely referenced to limited English proficient students, the
Department of Public Instruction additionally extends the standard graduation provision for those LEP
students identified by a school’s LEP service team as requiring additional time to achieve the full benefit
of their high school education. Any extended standard graduation schedule must be documented in a
service plan for that student, signed by the members of the LEP team, and submitted to the Department
as partial confirmation of the request.

The Department stipulates that this privilege of an extended standard graduation date must be
documented within each student’s individualized education program or LEP service program and that all
programs will be monitored for proper compliance, according to monitoring protocols. The Department will
require districts to submit evidence of any extension of a standard graduation in order to confirm its
proper use. The Department will maintain records regarding incidence rates for any such claims;
however, no student identifiable reports will be issued. The State stipulates, furthermore, that any
awarded extension of a standard graduation schedule will culminate in the award of a standard diploma to
students with individualized education programs and to LEP students with a LEP service program.

Graduation Indicator, Transition Extension

To determine the adequate yearly progress graduation indicator for 2004-05, the State will calculate
graduation rates according to the current definition, which has been applicable for determinations for
2001-04, within Section 7.1 of the State Accountability Plan. The State will delay the implementation of
the anticipated graduation rate formula, originally set for implementation for 2004-05, until the 2005-06
school year. At that point determinations will be referenced on graduation data collected on June 30 of
the previous year and will allow for the disaggregated reporting of subgroups as stipulated in the State
Accountability Plan.

ND Accountability Application Workbook 57 May 10, 2006

 NORTH DAKOTA CONSOLIDATED STATE APPLICATION ACCOUNTABILITY WORKBOOK

EXAMPLES FOR EXAMPLES OF

CRITICAL ELEMENT MEETING REQUIREMENTS NOT MEETING REQUIREMENTS

7.2 What is the State’s State defines the additional State has not defined an
additional academic academic indicators, e.g., additional academic indicator for
indicator for public additional State or locally elementary and middle schools.
elementary schools for the administered assessments not
definition of AYP? For included in the State assessment
public middle schools for system, grade-to-grade retention
the definition of AYP? rates or attendance rates. 9

An additional academic indicator

is included (in the aggregate) for
AYP, and disaggregated (as
necessary) for use when applying
the exception clause to make
AYP.

STATE RESPONSE AND STATE ACTIVITIES FOR MEETING REQUIREMENTS

The State of North Dakota has adopted the attendance rate for elementary and middle schools as the
additional academic indicator for determining AYP.

The State has established an attendance target point based on North Dakota 2001-02 attendance
baseline impact data. Refer to Appendix Z: North Dakota 2001-02 Attendance Impact Data for a
summary of the impact data. The State has set the target attendance rate at the second standard
deviation below the norm of ranked district attendance rates. Any district with an attendance rate lower
than this target point will be identified for not making Adequate Yearly Progress. This target point will
remain as the State definition for attendance throughout the duration of the 2001-2014 school years.
Based on the State’s attendance rate definition, the State attendance target point has been set at 93%.

Attendance rate is defined as the aggregate days of attendance in a school or school district divided by
the aggregate days of enrollment. The attendance rate is included in the aggregate for AYP. Attendance
data are collected through the State’s ADM (average daily membership) reporting system.

Attendance Indicator, Transition Extension

To determine the adequate yearly progress attendance indicator for 2004-05, the State will calculate
attendance rates according to the current definition within Section 7.2 of the State Accountability Plan. To
allow for a successful transition to previous year reporting, attendance rate determinations for 2004-05
will reference data collected on June 30, 2004. This will result in 2004-05 attendance determinations that
are identical to the 2003-04 attendance determinations; however, no school or district will be held to a
double-jeopardy effect. Schools previously identified in 2003-04 as not making adequate yearly progress
for attendance will be held harmless with any subsequent 2004-05 identification. All reports will indicate
that the attendance data references previous year data; no “did not make adequate yearly progress”
designations will be reported. Effective for the 2005-06 school year, determinations will be referenced on
attendance data collected on June 30 of the previous year, data that would not have been previously
reported.

9
NCLB only lists these indicators as examples.

ND Accountability Application Workbook 58 May 10, 2006

 NORTH DAKOTA CONSOLIDATED STATE APPLICATION ACCOUNTABILITY WORKBOOK

EXAMPLES FOR EXAMPLES OF

CRITICAL ELEMENT MEETING REQUIREMENTS NOT MEETING REQUIREMENTS

7.3 Are the State’s academic State has defined academic State has an academic indicator
indicators valid and indicators that are valid and that is not valid and reliable.
reliable? reliable.
State has an academic indicator
State has defined academic that is not consistent with
indicators that are consistent with nationally recognized standards.
nationally recognized standards, if
any. State has an academic indicator
that is not consistent within grade
levels.

STATE RESPONSE AND STATE ACTIVITIES FOR MEETING REQUIREMENTS

Attendance rates are widely recognized as a valid indicator of student success. Attention to student
achievement in addition to attendance and graduation rates offers a balanced presentation of key student
performance indicators.

The State’s ADM reporting system provides a reliable means of identifying students and monitoring
student attendance rates.

The State of North Dakota has established a definition for the minimum number of students in a subgroup
for both reporting and accountability purposes. The definition is consistent with the minimum number
identified within Principle 9. The State has established a test of statistical significance for the method of
determining a minimum number within a given population and referenced to the established measurable
objective, participation rate, graduation rate, or attendance rate. Refer to Principle 9.1 for a detailed
overview of this method of statistical significance.

Safe Harbor for Academic Indicators

If a high school whose calculated graduation rate is below the state-defined measurable objective, as
determined by the method prescribed in Section 7.1, reports sufficient improvement in the graduation rate
to meet the terms of safe harbor, then the high school will be identified as having made adequate yearly
progress for the graduation rate. Safe harbor for the graduation rate is defined as a ten percent reduction
in the not-graduated rate.

If an elementary or middle school whose calculated attendance rate is below the state-defined
measurable objective, as determined by the method prescribed in Section 7.2, reports sufficient
improvement in the attendance rate to meet the terms of safe harbor, then the elementary or middle
school will be identified as having made adequate yearly progress for the attendance rate. Safe harbor for
the attendance rate is defined as a ten percent reduction in the absence rate.

No statistical significance test will be conducted during the calculation of safe harbor. The safe harbor
provision for graduation and attendance will be applied in the determination of adequate yearly progress
for school districts and the State.

ND Accountability Application Workbook 59 May 10, 2006

 NORTH DAKOTA CONSOLIDATED STATE APPLICATION ACCOUNTABILITY WORKBOOK

PRINCIPLE 8. AYP is based on reading/language arts and mathematics

achievement objectives.

EXAMPLES FOR EXAMPLES OF

CRITICAL ELEMENT MEETING REQUIREMENTS NOT MEETING
REQUIREMENTS

8.1 Does the state measure State AYP determination for State AYP determination for
achievement in student subgroups, public student subgroups, public
reading/language arts and schools and LEAs separately schools and LEAs averages or
mathematics separately for measures reading/language arts combines achievement across
determining AYP? and mathematics. 10 reading/language arts and
mathematics.
AYP is a separate calculation for
reading/language arts and
mathematics for each group,
public school, and LEA.

STATE RESPONSE AND STATE ACTIVITIES FOR MEETING REQUIREMENTS

The State of North Dakota stipulates that the State will measure achievement in reading and mathematics
separately for determining AYP.

North Dakota state law (NDCC 15.1-21-08) places responsibility with the State Superintendent for the
administration of State assessments that are aligned to the State’s content standards in reading and
mathematics (refer to Appendix B: North Dakota Century Code citations or reference the North
Dakota Century Code at the following web site, http://www.state.nd.us/lr/cencode/T151C21.pdf ). State
law requires that the assessments be administered to at least one grade level selected within each of the
following grade spans: grades three through five; grades six through nine; and grades ten through twelve.
The North Dakota Department of Public Instruction has developed and administers assessments at
grades 4, 8, and 12 to correspond with the State’s content standards.

State law requires that the State assessments compile aggregated results and disaggregated results. The
State assessments must compile student achievement data that allows for a comparison of classrooms
within a given school and school district, schools within the district, and school districts within the state.
The test scores must also allow for comparisons based on students’ gender, ethnicity, economic status,
service status (i.e., migrant, LEP, disability), and assessment status (i.e., enrollment and participation
status), unless doing so enables the identification of any student. (Refer to NDCC 15.1-21-08 within
Appendix B: North Dakota Century Code citations or reference the North Dakota Century Code at the
following web site, http://www.state.nd.us/lr/cencode/T151C21.pdf ).

The State will produce all district- and school-level reports for districts and schools regarding their
respective student achievement levels in both reading and mathematics separately. These profile reports
will include both aggregated and disaggregated student achievement data. Refer to Appendix O: North
Dakota Sample School Report Card and Profile for an illustration of the content of these student
achievement profiles. Districts may use these profile reports as the foundation for their locally produced

10
If the state has more than one assessment to cover its language arts standards, the State must create
a method for including scores from all the relevant assessments.

ND Accountability Application Workbook 60 May 10, 2006

 NORTH DAKOTA CONSOLIDATED STATE APPLICATION ACCOUNTABILITY WORKBOOK

profiles. These State-generated reports will offer quality assurances regarding the generation of any district
achievement data.

The State requires all districts to disseminate student achievement profiles to their communities. This
mandate is required as an element of their receipt of federal funds. Refer to Appendix S: Consolidated
Application Certification and Assurances for the State assessment requirement for receipt of federal
ESEA funding. To assure compliance with this provision for the development and dissemination of
performance profiles per Title I funding, the Department will require timely and comprehensive reports as a
condition of receiving uninterrupted Title I funds. Further, evidence of these profiles will be one of the
criteria in the Department’s Title I monitoring program.

The State currently is reviewing its ESEA and accreditation monitoring policies. The State is pursuing an
amendment to its monitoring requirements that would mandate districts to produce evidence regarding
the dissemination of achievement profiles to their communities. Monitors would check for the production
and dissemination of any such achievement profiles. Any failures to disseminate these profiles would be
identified as a compliance violation of the school’s and district’s ESEA compliance agreement. Refer to
Appendix S: Consolidated Application Certification and Assurances for the State Assessment
requirement for receipt of federal ESEA funding. A school or district may be sanctioned for any
compliance violation of their ESEA assurances agreement.

ND Accountability Application Workbook 61 May 10, 2006

 NORTH DAKOTA CONSOLIDATED STATE APPLICATION ACCOUNTABILITY WORKBOOK

PRINCIPLE 9. State Accountability System is statistically valid and reliable.

EXAMPLES FOR EXAMPLES OF

CRITICAL ELEMENT MEETING REQUIREMENTS NOT MEETING
REQUIREMENTS

9.1 How do AYP State has defined a method for State does not have an
determinations meet the determining an acceptable level of acceptable method for
State’s standard for reliability (decision consistency) determining reliability (decision
acceptable reliability? for AYP decisions. consistency) of accountability
decisions, e.g., it reports only
State provides evidence that reliability coefficients for its
decision consistency is (1) within assessments.
the range deemed acceptable to
the State, and (2) meets State has parameters for
professional standards and acceptable reliability; however,
practice. the actual reliability (decision
consistency) falls outside those
State publicly reports the estimate parameters.
of decision consistency, and
incorporates it appropriately into State’s evidence regarding
accountability decisions. accountability reliability (decision
consistency) is not updated.
State updates analysis and
reporting of decision consistency
at appropriate intervals.

STATE RESPONSE AND STATE ACTIVITIES FOR MEETING REQUIREMENTS

The State of North Dakota stipulates that all AYP determinations meet the State’s standard for acceptable
validity and reliability. The State has adopted, with the technical assistance of Richard Hill of the National
Center for the Improvement of Educational Assessments, a test for statistical significance that establishes
a balance between systemic validity and reliability.

Background

Each state must create an accountability system in response to the requirements of the No Child Left
Behind Act (NCLBA). Among the requirements is the determination of whether schools and subgroups
within the school either have achieved a particular percentage of students at the proficient level or higher
(i.e., met the “status” requirement) or have improved their percentage of students achieving at the
proficient level or higher over the prior year’s level (i.e., met the “improvement” or “safe harbor”
requirement). If a school or a subgroup fails one or both those tests, it is identified as not making
Adequate Yearly Progress (AYP). Results for subgroups are not required to be included “in a case in
which the number of students in a category is insufficient to yield statistically reliable information.” States
are left to determine what that proper number might be.

One issue to be addressed is how low reliability can go before it is “insufficient.” If the stakes for
identification are lower, then a fairly low level of reliability might be acceptable. If the stakes for
identification are higher, however, then one would want to be fairly certain that a school had been
correctly classified before applying the proscribed consequences to the school. In NCLBA, annual
judgments are made about whether a school has made AYP. If a school is identified as not making AYP

ND Accountability Application Workbook 62 May 10, 2006

 NORTH DAKOTA CONSOLIDATED STATE APPLICATION ACCOUNTABILITY WORKBOOK

two years in a row, a series of proscribed consequences is set in motion. So, unquestionably, one would
want the decision about whether a school had not made AYP two years in a row to be highly reliable.
Given the effects of identification for not making adequate yearly progress, a reasonable argument can be
constructed for wanting a reliable decision to be made every year for every school.

Selecting a Fixed N

There exists an approach that requires a school or subgroup to have a particular number of assessed
students (for example, 30) in order to be considered for AYP identification, regardless of the performance
of the school or subgroup. This appears to be an approach that will not work well for either measuring
status or safe harbor. If a certain fixed number is chosen, schools will not be directly accountable for
subgroups with fewer than that number (i.e., those subgroups will be included in the school’s total score,
but the performance of that subgroup by itself will not be considered). No matter how small a number is
chosen, this will exclude many subgroups, leading to an incomplete look at the performance of the
school. Thus, one could argue that a number like 30 is far too large a number—a requirement that
subgroups meet this minimum N will eliminate the vast majority of subgroups in North Dakota.

On the other hand, the results for schools and subgroups are supposed to be “statistically reliable.” That
would mean, at a minimum, that if a subgroup causes a school to not make AYP, another sample of
students in that subgroup drawn for that school would be likely to have the same result. While reasonably
modest numbers of students often (but not always) can be used to reliably determine whether a subgroup
has met the status requirement, it takes large numbers (hundreds of students) to reliability detect whether
a school has made sufficient improvement.

So, on the one hand, a state should pick a fairly small N for purposes of validity (say, certainly something
no larger than 10), but needs a very high N (say, 300 or more) for purposes of reliability. A value that
provides reasonable validity is wholly inadequate for reliability purposes; conversely, a value that provides
reasonable reliability is wholly inadequate for validity purposes. A figure between those two is largely
inadequate for both purposes. This is the reason that the selection of any given fixed value for minimum
N remains problematic. Until one looks carefully at the issue, one presumes that a modest fixed N will be
a reasonable compromise between reliability and validity. A careful look tells us that choosing any value
is wholly inadequate for at least one of the two concerns, if not both. In short, there isn’t a reasonable
answer to this dilemma. One is not faced with a reasonable balancing of concerns over reliability and
validity when arbitrary N values are considered; any purported answer will be clearly wrong for at least
one of the two.

Given that one cannot have validity without reliability, it would be justifiable for a state to select a
minimum N of 300. Granted, an N of this size will eliminate virtually every subgroup in a state, essentially
eliminating this aspect of NCLBA. But such an N would at least ensure that decisions would be
sufficiently reliable.

Selecting an Alternative Method: the North Dakota Model

An alternative to selecting a fixed N is to run a test of statistical significance. That way, schools and
subgroups that are far from the standard do not need to have a large N for a reliable decision to be made.
For example, suppose the standard for a state is 50 percent proficient. If no students in a subgroup are
proficient, a reliable decision (i.e., one that has less than a 1 percent probability of misclassifying the
subgroup) that the subgroup does not meet the status test can be made if there are just seven students in
the subgroup. That is, if 50 percent of the students in a subgroup are proficient, there is less than 1
chance out of 100 that no students within a sample of seven would be proficient. Thus, in cases where
results are extremely low, the inadequate performance of the subgroup can be reliably detected even with
small Ns. On the other hand, if 499 out of 1000 students were proficient, one would not be certain that
another sample of students from that same subgroup wouldn’t have at least 50 percent proficient. So,
this system will select a group that is far away from the standard even if the group is small, but will not
select a group that is very, very close to the standard even if the group is quite large. Not only is this a
better application of statistics than the fixed N approach, it also is more fair and valid. Certainly, one

ND Accountability Application Workbook 63 May 10, 2006

 NORTH DAKOTA CONSOLIDATED STATE APPLICATION ACCOUNTABILITY WORKBOOK

would want to identify and target resources to very low-achieving subgroups before doing the same to
subgroups that are very close to the state’s standard.

However, even this system cannot solve the problem of measuring subgroup improvement. Measuring
improvement over one year is difficult to do because the amount of improvement desired is small, relative
to status (10 percentage points or less) and the measurement is made between two samples of students
rather than one. A subgroup’s status might be 50 percentage points away from the state’s standard; as
such, detecting differences that large can be done with samples as small as seven (as shown above).
But a subgroup’s required annual improvement can be no larger than 10 percent, and is often
considerably smaller than that. Given that measurement of improvement is made by comparing one
sample to another, each with its own sampling error, reliable judgments require, at a minimum, scores of
students, and more typically require hundreds of students.

Take this specific example. Suppose a subgroup has 50 percent of its students passing one year. To
make AYP through the safe harbor provision, the subgroup must improve to 55 percent passing the
following year. If the subgroup really does improve its performance by 5 percentage points, how many
students will it take, each year, to have at least a 95 percent probability that the subgroup’s performance
will increase over the previous year, much less go up the required amount?

A school with 50 percent of its students failing is supposed to reduce that percentage by 5 in one year,
and a z-score of 2.33 cuts off the upper 1 percent of the area under a normal curve. So, to reject the null
hypothesis at the .01 level one-tailed, the standard error of the difference can be no bigger than 5/2.33, or
2.15.

Now, suppose we hypothesize that a school has N students in each of two years, and its proportion of
students passing goes from 50 percent passing the first year to 55 percent the second year. The
equation we need to solve is as follows:

2.15 = P1 * Q1 / N + P 2 * Q 2 / N , or

2.15 = 50 * 50 / N + 55 * 45 / N

Solving for N produces a result of 1076.25. Rounding up means that an N of 1,077 students per year is
required to have a 99 percent probability that a school’s observed scores will increase from one year to
the next if its true percentage of proficient students increases from 50 to 55.

The results above assume that the two samples are independent (as would be the case, for example, if
testing were done at just one grade and the same grade was tested two consecutive years). If the results
of the two years are not independent but are positively correlated, the required N drops. This would be
the case if, for example, we followed the progress of a group of students from one year to the next. In
that case, the standard error of the difference scores is computed as follows:

σ P1− P 2 = σ P21 + σ P2 2 − 2rσ P1σ P 2

Now, suppose we continued our example from above (determine the standard error of difference scores
when a school’s true score changes from 50 percent passing to 55 percent), but followed the same cohort
of students from one year to the next. Suppose further that all the students tested in one year are tested
the next, and suppose the correlation between scores from one year to the next is .7 (a typical intra-
school student-level correlation of scores across one year). We still need a standard error of the
difference of 2.15, but now the equation is:

2.15 = P1 * Q1 / N + P 2 * Q 2 / N − 2 * .7 P1 * Q1 * P 2 * Q 2 / N

ND Accountability Application Workbook 64 May 10, 2006

 NORTH DAKOTA CONSOLIDATED STATE APPLICATION ACCOUNTABILITY WORKBOOK

Solving for N produces a result of 245. So, even if the same students are tracked from one year to the
next, it takes a very large number to be 99 percent certain that the observed results from one year to the
next will increase if the percent proficient goes from 50 to 55.

Specifics of North Dakota’s Approach

Schools or subgroups will be identified as not making AYP if their status score is insufficiently high, and
failing that, if their improvement is insufficient. This section will describe in more detail how each of those
judgments will be made.

Status

North Dakota will establish a required statewide status score equal to the percentage of students
proficient or higher in the 20th percentile school in the state, as required by NCLB. Call that value π0.
Once that “starting point” has been established, each subgroup will pass the status test if the null
hypothesis that the proportion of students for that school is equal to π0 cannot be rejected at the .01 level.

Exact probabilities vs. normal approximation. The exact probability that the null hypothesis can be
rejected, given X students proficient out of N tested and a population proportion of π0, is:

X0
P( X ≤ X 0 | π 0 , N ) = ∑ CiN π i (1 − π ) N −i
i =0

For example, if N = 3 and π0 = .5, the probability that X = 0 is .125 and the probability that X = 1 is .375.
The probability that X ≤ 1 is .5.

To further illustrate, suppose the starting point for North Dakota is 40 percent proficient, and suppose a
certain subgroup of 10 students has 2 proficient students. The observed percentage of students passing
in the subgroup is 20, which is less than the required value of 40. But would one reject the null
hypothesis that the true population percentage for that subgroup is 40? The test for the subgroup would
proceed as follows:

The probability of having 0 students proficient out of 10 if π0 = .40 is .0001.

The probability of having 1 student proficient out of 10 if π0 = .40 is .0016.
The probability of having 2 student proficient out of 10 if π0 = .40 is .0106.

Therefore, the probability of 2 or fewer students proficient out of 10 if π0 = .40 is .0123. Since this value is
greater than .01, this subgroup would not be identified as not having met the AYP status standard. If, on
the other hand, only 1 student had been proficient, the subgroup would be identified as not having met
the AYP status standard, and therefore would have to meet the improvement standard to avoid having
the school identified as failing to make AYP.

Computing these exact probabilities is computationally intense. Before today’s super-fast computers, the
amount of computation required was so extreme that often these exact probabilities were estimated
through normal approximation. With that method, one first computes the standard error of the mean as
π 0 (1 − π 0 ) / N , computes a z-score, and then determines the probability of a z-score that extreme or
more in a table of normal probabilities.

Taking our second example of 1 student proficient out of 10 with π0 = .40, we would compute the
standard error as .1549. In that case, the observed proportion of proficient students (.10) would yield a z-
score of (.10 - .40) / .1549, or –1.94. The probability of observing a z-score of that value or lower is .026.
Thus, in this case, the normal approximation is not a very good approximation of the exact (correct)
probability. We would have not rejected the null hypothesis in this case, but as was shown above, we

ND Accountability Application Workbook 65 May 10, 2006

 NORTH DAKOTA CONSOLIDATED STATE APPLICATION ACCOUNTABILITY WORKBOOK

should have. A general rule of thumb is that the normal approximation works well if pN > 5. In this case,
it equals 1, so the normal approximation does not approximate well.

It would be reasonable to compute exact probabilities only for the most extreme cases and use the
normal approximation for the remainder of the calculations. In fact, until recently, that was fairly common
practice. However, since computers can make the complex calculations for the exact probability quickly
and that using one method for all calculations leads to easier programming than using multiple methods,
North Dakota proposes to make the exact calculations for all schools and subgroups. Given that there
will be schools and many subgroups in North Dakota for which pN < 5 (and therefore many cases in
which the exact calculations would need to be done anyway), this is by far the most practical approach for
assessing status. However, the calculations for safe harbor are much more complex, and therefore the
normal approximation will be used for those tests.

For the purposes of calculating AYP, the State will reference a school’s and LEA’s current year’s
achievement results in addition to two previous years’ achievement results. Additionally, the State will
reference the combined effect across all grades tested within the school and LEA. The addition of a total
of three years data and the combined effect of all grades will increase the N value and ensure greater
reliability.

Choosing an alpha level. North Dakota has elected to use an alpha =0.01 level to conduct these tests of
statistical significance. This level of confidence will be applied to each subgroup tested for achievement,
participation rate, graduation rate, and attendance rate within each school, each district, and the State.

The selection of an alpha=0.01 assures a reasonably high level of confidence given the multiple tests
conducted within the process of determining AYP. However, these tests applied on the various
subgroups are not independent. Reading and math are well correlated, and some of the subgroups are
so highly inter-correlated as to be assessing virtually the same students because of their cross-over into
other subgroups. Thus, for most schools, the probability of an error across all the tests done is likely to
be below .05, which is the standard often used in educational research.

To balance all factors and mitigate the combined effect of different tests, the State has adopted an
alpha=0.01 as the means to ensure highly reliable identifications.

Safe Harbor: A Provisional Proposal and Study

Provisionally Adopting the ESEA Safe Harbor Definition. The State proposes to conduct a study of the
effects of the binomial distribution on the determination of safe harbor. Until this study of the binomial
distribution’s application on safe harbor is completed, the State of North Dakota will adopt the definition of
safe harbor established within ESEA section 1111. In the event that the study of the binomial distribution
improvement model produces findings that support the use of the binomial distribution, the State reserves
the right to propose the replacement of the ESEA definition of safe harbor with another that applies the
binomial distribution. The State Superintendent will communicate with the U.S. Department of Education
regarding any proposed changes affecting the State’s AYP safe harbor definition.

Conducting a Study of Statistical Reliability on Safe Harbor. The approach described in the sections
above will work well for assessing status. In contrast to selecting a fixed N, where many subgroups would
pass AYP regardless of performance, only the very smallest subgroups will receive this automatic pass in
North Dakota. Subgroups of even modest size will need to have at least some reasonable portion of their
students proficient in order to pass the status test. On the other hand, by selecting an alpha-level of 0.01,
North Dakota assures that those subgroups identified as not having met AYP would be very likely to have
a value lower than the state-required amount even if another sample of students were drawn. This
approach provides an excellent balance between validity (accountability for all subgroups) and reliability
(assuring that those subgroups identified have not been so identified simply on the basis of random
fluctuation).

ND Accountability Application Workbook 66 May 10, 2006

 NORTH DAKOTA CONSOLIDATED STATE APPLICATION ACCOUNTABILITY WORKBOOK

Assuring this same appropriate balance for measuring safe harbor will not be as easy. The amount of
improvement required each year is small relative to the standard error for most groups. As a result, it is
possible to reliably detect the required amount of annual improvement only for very large groups, as was
shown in an earlier section. Measuring improvement reliably will require a considered study. Because the
statistical test for safe harbor is conducted on each element of performance independently and because
there is no cumulative effect, it is proper to maintain an alpha=0.01 within the safe harbor test for
performance.

Conducting a Study of the Safe Harbor Model. The proper application of the binomial distribution to
measure the annual improvement of schools and districts for safe harbor requires a careful analysis in
order to ensure the vital interests of validity and reliability are honored. A statistical reliability model for
safe harbor remains largely theoretical; it has not been tested or validated. The application of the binomial
distribution within the status test can be demonstrated as effective, practical, and balanced; as such, the
application of the binomial distribution within the status measurement has been used and validated
successfully. In the absence of any demonstrable data or validation and in order to sufficiently analyze
the effect of the binomial distribution on the number and types of identifications and protections it allows,
the Department of Public Instruction will conduct a study of the binomial distribution within safe harbor
prior to its adoption and implementation. The State will proceed to use the binomial distribution for
determining status. The Department of Public Instruction will access the technical expertise of the
National Center for the Improvement of Educational Assessments and the North Dakota Technical
Advisory Committee to conduct this study. This study, which is expected to take several months, will
provide evidence on the effects of adopting the binomial distribution for determining safe harbor within
AYP. The State Superintendent will review any recommendations and communicate these findings to the
U.S. Department of Education.

Supporting Research Study

The ESEA places a high standard upon States to administer an accountability system that requires, on
the one hand, a consistent pressure to identify low performing schools, and on the other hand, a counter
force that limits any identifications of schools or districts where insufficient confidence exists. States must
provide for a meaningful and just accountability system. Any fair accountability system must adhere to
foundational principles, rest upon objective and universally applied rules, evidence balance between
validity and reliability, evoke a sense of confidence among schools and the public, and demonstrate
administrative practicality.

The North Dakota accountability model is as a viable system that supports these principles. After careful
consideration of the various options for implementing a valid and reliable accountability system, the
Department of Public Instruction asserts that the binomial distribution model offers a comprehensive and
balanced approach to accomplish these aims. Its principles are sound. The State’s supporting impact
data demonstrate that this statistical model constitutes a viable accountability system.

The Department of Public Instruction will conduct an ongoing study of the effects of the binomial
distribution model in ensuring the administration of a valid and reliable accountability system. This study,
to be conducted based on 2001-2004 data for final release by September 2004, will assess the impact of
the statistical model’s vitality, including the selected alpha=0.01 level, in balancing the dynamic factors of
identification and confidence. This study will assess the impact of the binomial distribution on the
identification patterns for achievement, safe harbor, participation rates, graduation rates, and attendance
rates for subgroups, schools, and districts. This study will assess impact data related to Type I errors and
Type II errors. Additionally, this study will analyze the level of confidence placed in the accountability
system by schools, parents, policymakers, and the public. The Department of Public Instruction will
submit any findings to the U.S. Department of Education for further review. It is in the best interest of the
State and the advancement of quality accountability systems that the North Dakota Accountability System
be peer reviewed for the purposes of improvement and possible wider dissemination.

ND Accountability Application Workbook 67 May 10, 2006

 NORTH DAKOTA CONSOLIDATED STATE APPLICATION ACCOUNTABILITY WORKBOOK

EXAMPLES FOR EXAMPLES OF

CRITICAL ELEMENT MEETING REQUIREMENTS NOT MEETING
REQUIREMENTS

9.2 What is the State's process State has established a process State does not have a system for
for making valid AYP for public schools and LEAs to handling appeals of accountability
determinations? appeal an accountability decision. decisions.

STATE RESPONSE AND STATE ACTIVITIES FOR MEETING REQUIREMENTS

The State of North Dakota has established its accountability system upon assessments that are
documented as valid and reliable measures of student achievement. Validity denotes the
appropriateness, meaningfulness, and usefulness of any inferences made from an assessment tool. As
such, validity addresses whether an assessment truly assesses what it purports to assess and whether it
will lead any user to an appropriate understanding and application of results. The State’s Assessment
System imbeds the elements of content validity (alignment to State content standards), item design
validity, related assessment validity, and consequential validity.

(a) Content validity (alignment to State content standards).

The activities conducted by the State to assure that all test items are aligned to the State’s content
standards. This review of content coverage, conducted by North Dakota teachers, offers assurance that
the State Assessment indeed does assess student achievement in terms of the State standards in
breadth. North Dakota teachers affirm that the State Assessment does assess the breadth of the
standards and that each standard is covered sufficiently to generate meaningful results. Each standard is
identified and is supported by a sufficient number of items to offer enough data to reach a valid indication
of a student’s performance.

(b) Test design validity.

The construction of individual test items and the test as a whole are critical elements of validity.
Additionally, the effects of any test item or the test as a whole on subgroups of students similarly
becomes an element of validity. The Department of Public Instruction has contracted with CTB/McGraw-
Hill to develop and administer an augmented, multiple measures assessment at each respective grade
level. These assessments meet high technical specifications to assure validity, reliability, and
comparability, thereby offering confidence in the application of any information gained through the use of
the assessments.

Refer to Appendix V: North Dakota State Assessment, Test Specifications for a summary review of
the technical specifications incorporated within the State Assessment. This summary identifies a variety of
factors that impact test validity and the appropriate use of acquired information. Refer to pages 1-11 of
Appendix W: North Dakota State Assessment, 2002 Preliminary Technical Report, for actual impact
data supporting the overall validity of the North Dakota State Assessment.

(c) Related assessment validity.

An inherent consideration confirming the validity of any assessment is how well it correlates with other
assessment tools of comparable quality. To quantify comparability among differing assessment tools
requires an ability to directly link individual student achievement among different assessment tools. To do

ND Accountability Application Workbook 68 May 10, 2006

 NORTH DAKOTA CONSOLIDATED STATE APPLICATION ACCOUNTABILITY WORKBOOK

so requires a data analysis and reporting tool capable of managing such linkages among different
databases.

The State of North Dakota has never possessed the ability to track the performance of individual student
or system performance levels in a meaningful manner based on quality disaggregated data analysis. The
State has never owned, developed, or accessed a single, statewide student data system. This absence of
a statewide data system has resulted in an inability to access accurate, meaningful information regarding
student demographics, student achievement levels, school performance, teacher quality indicators,
systemic improvements, or statewide systems monitoring. In the area of assessment, this absence of a
statewide data system has resulted in an inability to sufficiently study correlations of student achievement
among assessment tools or instructional methods of varying quality.

To eliminate these deficiencies and to advance meaningful school improvement measures, the
Department of Public Instruction has contracted with TetraData Corporation, in October 2002, to develop
and administer a statewide data analysis and reporting system. This data analysis and reporting system
will allow for the linkage of various databases in order to track individual student, staff, and institutional
achievement levels, including the correlation of student achievement across different assessment tools.

Included in this capability, is the capacity of the State to compile, compare, and validate student
achievement on all grade-level State assessments and to compare these with other assessment tools or
classroom grading. Official student files will be linked to State Assessment files that will, in turn, be linked
to other assessment tools and classroom grading in order to conduct correlation studies. This will offer
the State an auditing capability that will approach 100% accuracy, thereby ensuring a high degree of
confidence in any correlation study. Refer to Appendix R: TetraData Data Analysis and Reporting
System Summary for an overview of the project. The system will be functional statewide by March 2003.

With the development of this statewide data analysis and reporting system, the State will be able to
monitor and confirm the contextual validity of its State Assessment.

(d) Consequential validity.

The fundamental purpose for the administration of any assessment is to learn how well individual
students and populations of students perform against a standard. The intended consequence of such
learning is to apply this knowledge to the improvement of instruction for each student individually and for
all students collectively and by subgroup. Consequential validity means that the State Assessment is
designed in such a manner as to accomplish this aim with end users. Does the assessment lend itself to
reaching correct conclusions from the data?

North Dakota has never conducted follow up studies to record the application of its assessments to
enhance instruction. This analysis has never been attempted because of the difficulty in accurately
measuring the effort of schools to integrate assessment data into school improvement or, more
importantly, measuring the effect of such improvement efforts on students’ achievement. Beginning with
the 2001-02 baseline data generated through the first administration of the standards-based North Dakota
State Assessment, the State of North Dakota will begin a process of confirming the contextual validity of
its State Assessment.

The Department of Public Instruction has contracted with TetraData Corporation, in October 2002, to
develop and administer a statewide data analysis and reporting system. This data analysis and reporting
system will allow for the linkage of various databases in order to track individual student, staff, and
institutional achievement levels, including the correlation of student achievement across different
assessment tools. Additionally, the Department of Public Instruction will contract with an independent,
outside contractor to conduct a study of how well schools use the data from the State Assessment to
improve standards-based instruction.

This study will use survey tools with teachers and administrators to assess the degree that data from the
state assessment are used for overall school improvement, especially instruction. This study will also

ND Accountability Application Workbook 69 May 10, 2006

 NORTH DAKOTA CONSOLIDATED STATE APPLICATION ACCOUNTABILITY WORKBOOK

survey the efforts of school personnel to reform instructional practices. Finally, this study will use the data
linkage and analysis functions within the TetraData application to measure actual student achievement.
Because student cohorts can be linked to teachers and schools that engage in reform activities,
meaningful measurements can be derived on the effects of these efforts. Specific attention can be made
to track the broad effects of using State Assessment data to improve instructional areas identified as
deficient in the data.

The State seeks to implement a valid assessment and accountability system. Evidence of such an effort
will be marked by the State’s ability to monitor the alignment of its assessment to State content
standards, to assure high technical specification in the development of its State Assessment, to correlate
the State Assessment with other outside assessments and classroom grading, and to assure the
meaningful application of the assessment for school reform.

e) AYP Identification Method.

Principle 9.1 identifies the State’s method of identifying schools and LEAs for program improvement. This
Principle carefully balances the need to protect the interests of schools and LEAs from misidentification
with the public interest of knowing the overall performance of their schools. Refer to Principle 9.1 for a
thorough analysis of this issue.

f) Automatic Appeals.

Any school or district that has been identified through the AYP determination process will automatically
receive an appeal to clarify and correct information within the determination process and to present
extenuating information that may have bearing on the validity or reliability of the foundational information
or the determination process itself. In the case of AYP decisions regarding schools, the school district
must consider the appeal, with the assistance of the State, and render a final decision within 30 days,
after the submission date of the appeal. In the event of a district identification, if a district appeals a
decision regarding AYP, the Department of Public Instruction must make a final determination within 30
days of the date of the appeal. The State Superintendent will determine all appeals regarding AYP
identification.

The Department of Public Instruction will provide ongoing technical assistance to districts and schools
regarding the AYP determination process, all program improvement and corrective action activities,
including appeals regarding AYP decisions.

ND Accountability Application Workbook 70 May 10, 2006

 NORTH DAKOTA CONSOLIDATED STATE APPLICATION ACCOUNTABILITY WORKBOOK

EXAMPLES FOR EXAMPLES OF

CRITICAL ELEMENT MEETING REQUIREMENTS NOT MEETING
REQUIREMENTS

9.3 How has the State planned State has a plan to maintain State’s transition plan interrupts
for incorporating into its continuity in AYP decisions annual determination of AYP.
definition of AYP necessary for validity through
anticipated changes in planned assessment changes, State does not have a plan for
assessments? and other changes necessary to handling changes: e.g., to its
comply fully with NCLB. 11 assessment system, or the
addition of new public schools.
State has a plan for including new
public schools in the State
Accountability System.

State has a plan for periodically

reviewing its State Accountability
System, so that unforeseen
changes can be quickly
addressed.

STATE RESPONSE AND STATE ACTIVITIES FOR MEETING REQUIREMENTS

The State of North Dakota has developed a long-term plan to advance assessment system
improvements, assessment system expansion, and enhancements to the State’s accountability system.

I. Assessment System Improvements

(a) Assessment development procedural improvements.

The State Superintendent instituted a state-level advisory committee consisting of LEA and SEA
representatives, titled the Standards, Assessment, Learning and Teaching (SALT) Team, and authorized
this committee to advise the Department of Public Instruction on standards and assessment development
committee work. North Dakota’s assessment development protocols currently are being revised by the
Department of Public Instruction with the advise of the SALT Team to incorporate improvements into the
assessment development process and to accommodate the expansion of current assessments (grades 4,
8, and 12) into grades 3, 5, 6, and 7 and in science in grades 3-8 and 12. Refer to Appendix Y: North
Dakota Standards and Assessment Development Protocols regarding the procedures to be followed
for the development and improvement of state assessments. Completion of the revised State Assessment
Protocols is expected by May 2003.

Department of Public Instruction has adopted certain improvements in the development of assessment
RFPs. These improvements are identified within the white paper, Model Contractor Standards and State
Responsibilities for State Testing Programs, Education Leaders Council, 2002. Additionally, the

11
Several events may occur which necessitate such a plan. For example, (1) the State may need to
include additional assessments in grades 3-8 by 2005-2006; (2) the State may revise content and/or
academic achievement standards; (3) the State may need to recalculate the starting point with the
addition of new assessments; or (4) the State may need to incorporate the graduation rate or other
indicators into its State Accountability System. These events may require new calculations of validity and
reliability.

ND Accountability Application Workbook 71 May 10, 2006

 NORTH DAKOTA CONSOLIDATED STATE APPLICATION ACCOUNTABILITY WORKBOOK

Department is considering for adoption several innovations identified by other States and developed
within the white paper, State Innovation Priorities for Testing, Education Leaders Council. The
Department is expected to submit a comprehensive list of assessment procedure improvements and
potential innovations to the State Superintendent by May 2003.

The State Superintendent, by State law, is responsible for the oversight of all assessment development
and administration duties (refer to Appendix B: North Dakota Century Code, Assessment Statutes
regarding the delineation of State Assessment oversight responsibilities). The State Superintendent has
commissioned the SALT Team as the primary advisory committee. The State will contract with an outside
consultant to conduct an independent, systematic review of the State Assessment system and to issue
recommendations to the State Superintendent on the improvement of the system. The Department
contracts with CTB/McGraw-Hill to conduct the development and improvement of the State’s Assessment.

(b) Ongoing assessment refinement.

The Department of Public Instruction is developing a long-term plan for the ongoing replacement of test
items with additional selective- and constructive-response test items. This replacement plan will be written
into the next generation of RFP documents that are scheduled for release in May 2003. The Department
has identified, as a high priority, (1) the administration of an independent audit of the current State
Assessment’s breadth and depth of standards coverage, (2) the expansion of high-quality constructive-
response test items, and (3) the advancement of discussions with other States to collaborate in the
development of high quality test items and other assessment strategies.

(1) Test item rigor analysis. The Department of Public Instruction will conduct a thorough analysis of the
current North Dakota State Assessment regarding its rigor of higher order thinking skills and
understanding. The Department will contract with an independent, outside facilitator to conduct this
analysis. The RFP has not yet been drafted. The depth and breadth analysis will be conducted in early
2003 as a baseline evaluation of the current State Assessment in anticipation of its enhancement with
future replacement items and the future development of other grade-level assessments. It is anticipated
that the project will convene educators from across the State, including classroom teachers,
administrators, content specialists, and university professors, to conduct an audit of the current State
Assessment in terms of an agreed upon evaluation criteria. This effort would evaluate the State
Assessment against five levels of increasing difficulty: (1) identity and recall; (2) use of concepts; (3)
explanation and reasoning; (4) evaluation and extension; and (5) integration and performance.

(2) Item replacement policy. It is the long-term commitment of the Department of Public Instruction to
employ an item-replacement model that steadily increases the number and quality of constructive-response
test items, including greater use of extended constructive-response items. Future RFPs for the North
Dakota State Assessment will include a schedule for the improved quality of constructive-response test
items. The Department has adopted a policy requiring future RFPs to incorporate the recommendations of
the Education Leaders Council, Model Contractor Standards & State Responsibilities for State Testing
Programs, 2002 (refer to page 19 within Appendix Y: North Dakota Standards and Assessment
Development Protocols or reference the following web site,
http://www.dpi.state.nd.us/standard/content/toc.pdf).

(3) State consortium efforts. Following the selection of the State’s next assessment vendor following an
RFP selection process, the Department of Public Instruction will explore with the State’s next approved
vendor the prospects of initiating a series of discussions with other States who contract with the approved
vendor to establish a consortium of States committed to assessment improvement. This consortium of
States would share the costs and advance the development of high quality test items, specifically
constructive-response and extended-response test items. By convening States that share a common
vendor, there are greater opportunities to achieve successes by unifying efforts, maximizing gains, and
minimizing copyright impediments. The State will begin discussions with interested States beginning in
July 2003.

ND Accountability Application Workbook 72 May 10, 2006

 NORTH DAKOTA CONSOLIDATED STATE APPLICATION ACCOUNTABILITY WORKBOOK

(c) Assessment innovations.

The Department is drafting a series of recommendations regarding the possible adoption of several
innovations identified by other States and developed within the white paper, State Innovation Priorities for
Testing, Education Leaders Council. The Department is expected to submit a comprehensive list of
assessment procedural improvements and potential innovations to the State Superintendent by July
2003.

In addition to this anticipated list of improvements and innovations, the Department is considering the
integration of two established products into the current State Assessment: (1) test item task banks
developed by previous development work, and (2) a web-based scoring application for extended-
response test items.

II. Expansion of the State Assessment System

North Dakota, through an agreement with the U.S. Department of Education, has established an
assessment waiver plan to bring the State into full compliance with ESEA, Section 1111(b)(1)
requirements. . This waiver plan, approved through August 2003, is enclosed as Appendix A: North
Dakota State Assessment Waiver Agreement Plan and can be accessed at the following web site:
http://www.dpi.state.nd.us/testing/assess/plan.pdf. During the 2001-02 school year, North Dakota
administered its state assessment and is on schedule to meet fully all provisions set forth within the
waiver plan.

State assessments have been developed and adopted thus far in mathematics and reading at grades 4,
8, and 12 in accordance with North Dakota’s approved assessment waiver agreement and the North
Dakota Standards and Assessment Development Protocols
(http://www.dpi.state.nd.us/standard/content/toc.pdf). North Dakota will proceed to develop state
assessments in mathematics and reading at additional grades (grades 3, 5, 6, and 7) by 2005-2006 in
accordance with State protocols and section 1111(b)(1) requirements. North Dakota will proceed to
develop state assessments in science at grades 4, 8, and 12 by 2007-2008 in accordance with State
protocols and section 1111(b)(1) requirements. Additionally, North Dakota will expand its science
assessment, voluntarily, at grades 3, 5, 6, and 7 by 2007-2008 in accordance with State protocols and
section 1111(b)(1) standards.

North Dakota has submitted its plan to expand the development of grade specific assessments to meet
the requirements of NCLBA. This submission was an element of the State’s Consolidated Application for
ESEA funding, dated June 2002. Refer to Appendix H: North Dakota State ESEA Consolidated
Application, pages 7-10, or refer to the following web site:
http://www.dpi.state.nd.us/grants/DOEapp.pdf). The North Dakota State Consolidated Application has
since been approved by the U.S. Department of Education.

The North Dakota Assessment System provides for a single, unified, statewide assessment that
measures the performance of all students in terms of the State’s challenging content and achievement
standards.

III. Enhancement to the State’s Accountability System

The Department of Public Instruction will develop a state-level advisory committee that will advise the
State Superintendent on the development and review of all State AYP policies and submit
recommendations to the State Superintendent. The State Superintendent will review and approve the
disposition of all recommendations. The Department of Public Instruction anticipates the development of
this advisory committee by July 2003.

ND Accountability Application Workbook 73 May 10, 2006

 NORTH DAKOTA CONSOLIDATED STATE APPLICATION ACCOUNTABILITY WORKBOOK

PRINCIPLE 10. In order for a public school or LEA to make AYP, the State
ensures that it assessed at least 95% of the students enrolled in each subgroup.

EXAMPLES FOR EXAMPLES OF

CRITICAL ELEMENT MEETING REQUIREMENTS NOT MEETING
REQUIREMENTS

10.1 What is the State's method State has a procedure to The state does not have a
for calculating participation determine the number of absent procedure for determining the
rates in the State or untested students (by rate of students participating in
assessments for use in subgroup and aggregate). statewide assessments.
AYP determinations?
State has a procedure to Public schools and LEAs are not
determine the denominator (total held accountable for testing at
enrollment) for the 95% least 95% of their students.
calculation (by subgroup and
aggregate).

Public schools and LEAs are held

accountable for reaching the 95%
assessed goal.

STATE RESPONSE AND STATE ACTIVITIES FOR MEETING REQUIREMENTS

The State of North Dakota requires all students enrolled in public schools within North Dakota to
participate in the State Assessment system. Refer to Appendix B: North Dakota Century Code
citations or reference the North Dakota Century Code at the following web site,
http://www.state.nd.us/lr/cencode/T151C21.pdf ). All students, regardless of their enrollment status,
participate in the State Assessment. This total inclusion policy includes those students who may have
enrolled in a district or school after the beginning of a school year. Any student who may have been
enrolled in a school or district after the beginning of a school year is identified on their assessment
demographic sheet. Students or school personnel mark a special code on the assessment demographic
sheet that identifies their late enrollment status. This code is used to identify the student and to remove
them from the school’s student roll for AYP identification purposes. Refer to page 29 for codes “R” and
“S” of Appendix Q: North Dakota State Assessment, Test Coordinator’s Manual 2002-03 for the
enrollment code identification fields. A student who has not been enrolled in a school for the entire year
but has been enrolled in the district for the entire year will not be included into AYP consideration for the
school but will be included into AYP consideration for the district.

All students must be accounted for regarding their enrollment status. This is a required entry on the
demographic sheet of all students. Student participation rates will be compared to the school’s and
district’s Average Daily Membership student count used to reimburse school’s and district’s for their State
foundation aid. Therefore, the State references reimbursement census data to confirm student
participation rates. Student participation rates may be identified within the aggregate and disaggregated
by subgroup.

Refer to page 29 for codes “R” and “S” of Appendix Q: North Dakota State Assessment, Test
Coordinator’s Manual 2002-03 for the enrollment code identification fields. Student participation rates
will be compared to the school’s and district’s Average Daily Membership student count used to

ND Accountability Application Workbook 74 May 10, 2006

 NORTH DAKOTA CONSOLIDATED STATE APPLICATION ACCOUNTABILITY WORKBOOK

reimburse school’s and district’s for their State foundation aid. Therefore, the State references
reimbursement census data to confirm student participation rates.

Participation rates on the North Dakota State Assessment are calculated as follows:

# of students with test results

# of students enrolled at the time of test administration

Students participating in the North Dakota Alternate Assessment are included in the numerator and
denominator.

The State is developing a statewide student data analysis and reporting system to aid the Sate in
monitoring the enrollment patterns and participation rates of students. The TetraData application will allow
the State to link district enrollment files with the State’s assessment participation files in order to assure
that all enrolled students are accounted for in the State Assessment system files. Refer to Appendix R:
TetraData Data Analysis and Reporting System Summary for an overview of the TetraData system’s
purpose and design.

The State currently is reviewing its ESEA and accreditation monitoring policies. The State is pursuing an
amendment to its monitoring requirements that would mandate districts to produce evidence regarding
the enrollment dates of all students. Monitors would check for any students who had enrolled after the
beginning of a school year and cross-check their participation status in the State Assessment data file.
Any failures to include such students would be identified as a compliance violation of the school’s and
district’s ESEA compliance agreement. Refer to Appendix S: Consolidated Application Certification
and Assurances for the State assessment requirement for receipt of federal ESEA funding. A school or
district may be sanctioned for any compliance violation of their ESEA assurances agreement.

It is the expressed policy of the State of North Dakota to include all students within the North Dakota
State Assessment.

ND Accountability Application Workbook 75 May 10, 2006

 NORTH DAKOTA CONSOLIDATED STATE APPLICATION ACCOUNTABILITY WORKBOOK

EXAMPLES FOR EXAMPLES OF

CRITICAL ELEMENT MEETING REQUIREMENTS NOT MEETING
REQUIREMENTS

10.2 What is the State's policy State has a policy that State does not have a procedure
for determining when the implements the regulation for making this determination.
95% assessed regarding the use of 95%
requirement should be allowance when the group is
applied? statistically significant according
to State rules.

STATE RESPONSE AND STATE ACTIVITIES FOR MEETING REQUIREMENTS

The State of North Dakota has established a definition for the minimum number of students in a subgroup
for both reporting and accountability purposes. The definition is consistent with the minimum number
identified within Principle 9. The State has established a test of statistical significance for the method of
determining a minimum number within a given population and referenced to the established measurable
objective, participation rate, graduation rate, or attendance rate. Refer to Principle 9.1 for a detailed
overview of this method of statistical significance.

The State will apply the binomial distribution on all school and subgroup participation data to determine
the appropriate statistical participation rate. The binomial distribution is calculated with the alpha=0.01,
the target set at 95%, and the sample size consisting of all students enrolled at the time of the
assessment.

The incorporation of federal guidance regarding the full participation of all students within the State
assessment and the restricted omission of certain students with significant medical conditions.

On March 26, 2004, the U.S. Department of Education released guidance regarding the calculation of
adequate yearly progress based on the participation of at least 95% of students in a State’s annual
assessment of student achievement, measured by a total school population and by subgroups. Full
participation within the Act ensures a valid and reliable means of determining accountability for all
schools, districts, and the State. The following provisions apply.

a. Multi-year averaging. In calculating adequate yearly progress for participation rates, the State will
access participation data from the current year, a two-year average, and a three-year average. If any of
these accessed data points meet or exceed 95%, the specified school, district, or State will be deemed to
meet the requirements of adequate yearly progress.

b. Significant medical emergencies. In calculating adequate yearly progress for participation rates, the
State will omit, within restricted conditions, any student who has been prohibited from participating in the
State assessment during the entire testing window, including make-up dates, due to a significant medical
emergency. The Department of Public Instruction will consider only those requests for the omission of a
student with a significant medical emergency that have been documented by the affected school. The
Department reserves the right to monitor the circumstances surrounding any requests for omission.
Students must be enrolled within the affected school during the testing window, as evidenced on the pupil
membership files retained within the State Online Reporting System. The State will document the receipt
of any omission requests, remove the student from the calculation of adequate yearly progress, and
record and report aggregate incidence rates of omissions statewide. The State will not issue student
omission reports that might lead to the identification of an individual student.

ND Accountability Application Workbook 76 May 10, 2006

 NORTH DAKOTA CONSOLIDATED STATE APPLICATION ACCOUNTABILITY WORKBOOK

Appendix A
Required Data Elements for State Report Card

1111(h)(1)(C)

1. Information, in the aggregate, on student achievement at each proficiency level on the State academic
assessments (disaggregated by race, ethnicity, gender, disability status, migrant status, English
proficiency, and status as economically disadvantaged, except that such disaggregation shall not be
required in a case in which the number of students in a category is insufficient to yield statistically reliable
information or the results would reveal personally identifiable information about an individual student.

2. Information that provides a comparison between the actual achievement levels of each student
subgroup and the State’s annual measurable objectives for each such group of students on each of the
academic assessments.

3. The percentage of students not tested (disaggregated by the student subgroups), except that such
disaggregation shall not be required in a case in which the number of students in a category is insufficient
to yield statistically reliable information or the results would reveal personally identifiable information
about an individual student.

4. The most recent 2-year trend in student achievement in each subject area, and for each grade level,
for the required assessments.

5. Aggregate information on any other indicators used by the State to determine the adequate yearly
progress of students in achieving State academic achievement standards disaggregated by student
subgroups.

6. Graduation rates for secondary school students disaggregated by student subgroups.

7. Information on the performance of local educational agencies in the State regarding making adequate
yearly progress, including the number and names of each school identified for school improvement under
section 1116.

8. The professional qualifications of teachers in the State, the percentage of such teachers teaching with
emergency or provisional credentials, and the percentage of classes in the State not taught by highly
qualified teachers, in the aggregate and disaggregated by high-poverty compared to low-poverty schools
which (for this purpose) means schools in the top quartile of poverty and the bottom quartile of poverty in
the State.

The State of North Dakota stipulates that it will include all data elements provided within ESEA section
1111 within the State Report Card and Profile. Refer to Appendix O: North Dakota State Report Card
and Profile. The State will include attendance rate for elementary and middle schools.

ND Accountability Application Workbook 77 May 10, 2006

 NORTH DAKOTA CONSOLIDATED STATE APPLICATION ACCOUNTABILITY WORKBOOK

Table of Appendices

A. North Dakota State Assessment Waiver Agreement Plan

B. North Dakota Century Code

C. North Dakota Standards and Assessment Development Protocols

D. North Dakota Mathematics Content Standards

E. North Dakota Mathematics Achievement Standards

F. North Dakota English Language Arts Content Standards

G. North Dakota Language Arts Achievement Standards

H. North Dakota State ESEA Consolidated Application

I. Schools Falling Outside Assessment System Grade Span

J. North Dakota State Assessment, Bookmark Standards Setting Technical

Report, 2002

K. State Superintendent’s Approval Notification of North Dakota State

Assessment Cut Scores

L. Program Improvement Activities

M. Testimony Before the Education Committee by the Department of Public

Instruction, October 10, 2002

N. North Dakota State Assessment, Student Achievement Reports

O. North Dakota State Report Card and Profile

P. State Superintendent’s Release of State Assessment Results

Q. North Dakota State Assessment, Test Coordinator’s Manual, 2002-03

R. TetraData Data Analysis and Reporting System Summary

S. Consolidated Application Certification and Assurances

T. State Adequate Yearly Progress Computation Rules

ND Accountability Application Workbook 78 May 10, 2006

 NORTH DAKOTA CONSOLIDATED STATE APPLICATION ACCOUNTABILITY WORKBOOK

U. Individualized Education Program Planning Process

V. North Dakota State Assessment, Test Specifications

W. North Dakota State Assessment, 2002 Preliminary Technical Report

X. North Dakota Assessment System 2001-02 Baseline Impact Data.

Y. North Dakota Graduation 2001-02 Impact Data.

Z. North Dakota Attendance 2001-02 Impact Data.

AA. North Dakota Participation 2001-02 Impact Data.

BB. North Dakota Composite AYP Identification Impact Data

ND Accountability Application Workbook 79 May 10, 2006