Grand Awards Judging Process
Judging involves four processes: registration/orientation, review of projects, finalists' interviews and selection of award winners. All Grand Awards judges must register at the registration area in the Vista Point area off concourse 1 of the San Jose McEnery Convention Center (SJMCC) between 1:00 p.m. and 5:30 p.m. on Tuesday, May 11. Beginning at 1:00 p.m., there will be training sessions and the opportunity to review projects in Exhibit Halls 1 - 3. At 5:45 p.m. there will be dinner and general introductions for the judges in their assigned meeting areas. There will be a formal orientation for ALL judges in the South Hall starting at 7:00 p.m. At the conclusion of this orientation, Grand Awards judges should return to their assigned meeting rooms to receive their judging interview schedules. At this time, all judges are welcome to preview finalists' projects in preparation for judging day on Wednesday. Grand Awards judges are encouraged to review and pre-judge the projects to which they have been assigned in preparation for their finalists' interviews on Wednesday. The exhibit hall will close promptly at 11:30 p.m. on Tuesday.
As a special treat for all judges there will be a panel discussion with several Nobel Laureates and Draper Prize recipients on Tuesday afternoon between 4:30 p.m. and 5:30 p.m. Topics of this discussion will cover a wide range of subjects including the award recipients themselves, their scientific/engineering achievements and any other subject of interest that may be posed. It is hoped that this discussion will serve as a catalyst for the discussions the judges will have on Wednesday with the finalists.
Wednesday is the day that all judging interviews are conducted and all official scoring and determination of winners takes place. The Grand Awards judging day is divided into an open period in the morning and three judging sessions that conclude at 6:00 p.m. The morning starts with breakfast, a Category meeting and roll call to determine final attendance. Each group of Grand Awards judges will receive final instructions for conducting the finalists' interviews and receive their scoring scan cards before the first period of judging begins at 9:30 a.m. Lunch will be available to the Grand Awards judges in their respective meeting areas at noon. The second judging period begins at 1:15 p.m. and at the conclusion of this period all scheduled interviews will have been completed. There is a short break until 4:15 p.m. after which there is a third judging session to provide a final opportunity to all judges for discussion with the finalists. Special Awards judges can utilize the Finalists' Open Period Grid chart to determine when they can interview the Finalists in the first and second judging periods. The third period is available to ALL judges to make their final award selections. At the conclusion of this session, awards must be determined and the certifications of those awards turned in to Society for Science & the Public at the Judging Control Center in a room in Exhibit Hall 1 of the San Jose McEnery Convention Center (RSCC).
Once all of the judging work has been completed, all judges are invited to attend a Judges' Celebration.
Ideally, the ratio of judges to finalists in Grand Awards judging would be approximately 0.8:1. This optimum ratio is a goal in the recruitment of judges, but in general the number of judges will either be greater than or less than the number of finalists. To account for these differences, the number of projects in each category, the number of Grand Awards judges present, the number of potential interview periods, and the maximum number of finalist interviews allowed are considered in arranging a Grand Awards judge’s interview schedule. The goal is to provide each finalist with up to a maximum of ten interviews. Usually, this translates to a judge being asked to interview between eight and fourteen finalists. The maximum number of interviews a judge can be assigned is fourteen (out of seventeen total interview periods).
Over 10,000 interviews occur on Wednesday morning and afternoon. An optical scan card system is used to assist in compilation of the scoring data.
On Tuesday evening, each Grand Awards judge receives an interview schedule and a blue Intel® ISEF Judge’s Record. Notes and scores relative to each project must be kept on the record as an aid to further discussion, and it becomes the judge's own record of how each particular project was scored. On Wednesday morning, each judge receives an envelope of scan cards for the finalists' projects on the schedule. An additional copy of the interviewing schedule is attached. The scan cards are preprinted with the appropriate information concerning interview time, the Judge’s ID number, and the project number (booth location) of the finalist's project. Judges should review these cards and check for accuracy against the schedule. Any discrepancy should be brought to the attention of the category co-chair immediately.
Once the score has been determined (0-100), it should be written in the appropriate blocks on the scan card and the number indicated by filling in the proper ovals corresponding to the digits of the score. ONLY A BLACK SANFORD SHARPIE PEN SHOULD BE USED TO MARK ANY JUDGING SCAN CARD (they will be provided).
Judges should turn in scan cards at the end of each interview, if possible, or at least at the next open period. All of the scan cards should be turned in at the designated areas of the HUB in the exhibit hall. HUB personnel will check the cards for completeness and can create replacement cards when necessary. It is important that scan cards are not held until the end of judging because it will seriously delay completion of the final scoring process.
Statistical Information Derived from Interviews
Each Intel ISEF Grand Awards Category is divided into subcategories. Each judge within a subcategory evaluates a number of projects within that subcategory. In addition, the judge most likely evaluates some projects outside the subcategory. Because a student project can receive as many as 10 judging interviews and subsequent scores, a statistical analysis is conducted to standardize the results. Prior to this statistical analysis, a printout of each set of judge's scores is distributed to each of the category co-chairs for further distribution to each of the judges in that co-chair's category. Each judge should compare the set of scores maintained on the blue record sheet to those on the printout. Once all scores are confirmed, the statistical analysis is conducted to aid the judges and co-chairs in their final discussion to select the Grand Awards winners.
A Score Analysis Report for each Intel ISEF Grand Awards Category includes high to low lists of finalists' projects for each project giving 1) a score average, 2) a average rank value, and 3) an averaged Z score.
Following the completion of all scheduled interviews at the end of the second judging session, the statistical data is generated and distributed to the Intel ISEF Grand Awards co-chairs and the Grand Awards judges.
Judges should remember that this statistical data is provided only as an aid in determining the winners. It does not have to be the only factor - the final decisions still rest exclusively with the Grand Awards judges and their co-chairs.
Final Awards Selection and Certification
Society for Science & the Public determines the number and distribution of Intel ISEF Grand Awards within each category. The certification sheet necessary to record awards is included in the Judging Record Folder that is provided to the co-chairs of each Grand Awards category at the beginning of judging on Tuesday. To complete the process of certification, the Grand Awards Category co-chair, after discussion with all that category’s judges utilizing the Score Analysis Report as a guide, enters the project number and the name of each winner on the certification sheet. The Judging Category co-chair then presents the completed certification to the Society for Science & the Public Judging Control Center where it is entered into a certification database. Following the entry of the information, two copies of the final certification are printed. The final copies, contents of which are confidential, are signed and certified by the Grand Awards Category co-chair. One copy is retained by Society for Science & the Public, and the other copy is given to the Category co-chair. All responsibilities of the judges for a Grand Awards Category are completed with final certification of results.
Society for Science & the Public 1719 N Street N.W. , Washington, DC 20036, 202-785-2255.