NOTES: Although the ROSIE can be used for the purpose of internal assessment, it can also be conducted as an external assessment by a ROSIE trained Pennsylvania Key (PA) Program Quality Assessor (hereinafter referred to as “Assessor”).
When using the ROSIE for an internal assessment to meet the expectations for one of the Keystone STARS Standards, please consult the Keystone STARS Performance Standards and the Keystone STARS Program Manual for additional information.
How to conduct an objective, authentic classroom assessment
The Keystone Stars Standards expect programs to conduct internal assessments annually and, once every three years, engage their internal assessment team in discussion of their internal assessment results with an Assessor from the PA Key.
- The internal assessment should be an observation; therefore, someone who has knowledge of the selected Program Observation Instrument (POI) and its expectations should plan to observe for the recommended amount of time (for the specific POI), in the classroom. For the ROSIE, it is acceptable for the teacher to complete the observation because it is best conducted when children are not present in the classroom.
- The observer should be able to commit to observing and taking notes and should not have any classroom responsibilities during the observation.
- The internal assessment should be viewed as an integral part of a program’s commitment to continuous quality improvement rather than just another piece of paperwork to be completed to meet an expectation.
- Information from an objective internal assessment can yield much information about the program’s practices, providing a foundation for the program to celebrate their successes and build upon existing practices.
ROSIE assessment at-a-glance
- Two hours to complete
- No children present, when possible
- No teacher interview
- No program documents needed for scoring
- Assessment based on observation only
Required materials
- Rating Observation Scale for Inspiring Environments (ROSIE) ISBN: 978-0-87659-321-9
Supplemental materials
- Inspiring Spaces for Young Children (highly recommended) ISBN: 978-0-87659-317-2
Purchase the materials
- There are multiple sources where the required and supplemental materials can be purchased. An online search by resource name or ISBN number may result in several options from which to
- Be sure to allow adequate time for ordering and delivery of the materials.
- Although only one ROSIE scale book is needed to conduct the assessment (the score sheet may be copied), you may want to consider ordering a copy for each classroom that will be using the ROSIE.
Conduct the internal assessment (start to finish)
1. Learn about the ROSIE
- The more you learn about the quality practices described in the ROSIE, the more accurate your own internal assessment will
- It is recommended that the teaching team from each classroom being assessed, and the internal assessment team becomes familiar with the expectations of the ROSIE, especially the “Seven Principles of ”
- Register for the online self-paced Foundations of the Rating Observation Scale for Inspiring Environments (ROSIE) on the Professional Development (PD) Registry (available sometime in 2024/2025).
- Request a Facilitated Program Observation Instrument Orientation (FPO). This one-hour session with an Assessor provides an opportunity to learn some of the basic information about the ROSIE and ask specific questions. The FPO is structured to meet each individual’s or program’s specific needs and each person attending receives one hour of PQAS (Pennsylvania Quality Assurance System) credit. Please note the FPO is not a substitute for the online self-paced PA Foundations of the Rating Observation Scale for Inspiring Environments (ROSIE) course, but rather provides a basic overview.
- Access additional information on the ROSIE.
- Read the ROSIE book cover to cover, including the definitions found on page 46 and the scoring process found on pages 5, 8-9, and 47.
- Reflect on the expectations for each Principle of Design indicator.
2. Prepare to conduct the internal assessment
- If time has elapsed since your purchase and review of the ROSIE, re-familiarize yourself with the scale expectations.
- Gather all materials needed: something to capture your notes whether a laptop or paper/pencil, ROSIE scale book, copy of score sheet from back of ROSIE scale book (page 47).
- Set aside at least two hours of time for the assessment.
- It is not mandatory that the assessment is completed in a single time block; however, it should be completed within one week.
3. When to conduct the internal assessment
- It is best to conduct the assessment when no children are present.
4. Conduct the internal assessment
- Approach the observation as an objective observer who looks at the classroom from an aesthetics perspective. Make scoring decisions based on the current observation of the classroom, not on changes that are planned for the future or what might have been done in the past.
- Read each indicator carefully, paying close attention to the expectations for each indicator. Some indicators include more than one criterion to fully meet the indicator’s expectation (for example, a designated science area to explore natural materials, and use science equipment AND an organized system to test theories and record their findings).
- Take notes that relate to each indicator to support your scoring decisions. Taking notes is extremely helpful for various reasons. First, note taking will help you score accurately. If you are reviewing the assessment in order to check your scores, looking back at notes will help you. Second, note taking will help you/your teachers create goals based on observed data that was written down. Finally, your notes will assist the Assessor in understanding your program’s practices and will support deeper internal assessment consultation conversations. Remember, providing numerical scores is not enough to support your program’s growth and development.
5. Determine your ROSIE level (scoring)
- Score the assessment using the scoring information found on pages 5, 8-9, and 47.
- Score each principle independently of all other principles.
- Read across the three levels (left to right) of the indicator to determine the level that best reflects your classroom environment.
- Assign one point for Sprouting, two points for Budding, and three points for Blooming. Remember, do not rely on memory, rather use your notes which reflect the most current classroom practices.
- Once you determine your level of growth, circle the one number that completely reflects your level on the scoring guide and then proceed to the next indicator.
- After scoring each item, add up the scores from each principle of design column and enter the number at the bottom of the column.
- Add the total score for each Design Principle column to determine your overall level of growth.
- Sprouting level (1-73 points), Budding level (74-124 points), and Blooming level (125-147points).
6. Review findings
- Conducting a ROSIE assessment is only the first step of the process toward growth. Review your observation with those people responsible for implementing changes.
- Review your assessment findings with your Internal Assessment Team, Quality Coach and Assessor.
- They can celebrate the Principles of Excellence (where your program excels) with you.
- Together you can discuss areas where your team feels growth could occur (Principles for Improvement).
- Identify goals to achieve growth and include them in your Continuous Quality Improvement Plan (CQI).
Next steps
- Incorporate your Principles for Improvement into your CQI plan.
- Assure everyone has contributed ideas and ensure everyone knows the next steps to take toward improvement.
- When planning for CQI goals, look toward implementing the next level’s indicators.
- Start small. Focus on one indicator at a time.
- Be patient. Do not be discouraged if your initial score falls within Sprouting or Budding. Growth takes time.
- Change what you can and accept those things that are outside of your program’s ability to improve.
- Share your plan for growth (Principles for Improvement) with everyone involved on your team.
- Plan to revisit your goals and program’s administrative practices regularly to assess and update as goals are achieved. The CQI process is cyclical as you continuously reflect on your practices and your drive for growth.
Created in 2007 by the Pennsylvania Office of Child Development and Early Learning (OCDEL), The Pennsylvania Key implements the work and supports the policies developed and managed by OCDEL. Learn More. >