Basic Educational Data System (BEDS) Institutional Master File

Abstract

Series Title:

Institutional Master File (IMF)
Basic Educational Data System

Creating Agency:

Information Center on Education
New York State Education Department

Technical Specifications:

The IMF contains 14 rectangular flat files of numeric and character data. Each file represents a single school year.

Summary:

The IMF contains data on each school district, public and private school, and Board of Cooperative Educational Services (BOCES) in New York State. Each record identifies the institution's name, location, and chief officer. Each record also contains information on enrollment statistics, the number and type of professional and paraprofessional staff, and limited information on facilities, such as the number and type of classrooms and buildings. Additionally, School and BOCES Data Records identify special educational programs available at the institution. The individual files are arranged by year, from 1967-1981, and contain four record types: School Data Record (public and private), District Data Record, and BOCES Data Record.

File Size:

File size varies according to reporting year. The average file contains 7200 records. The size of each record varies according to record type and reporting year. District Data Records contain 840-900 characters. The size of the School Data Record ranges from 1322-1620 characters. The BOCES Data Records contain 500-900 characters. Individual record sizes are noted on each file layout.

Distribution Specifications:

All or selected years of the IMF can be downloaded from our web site or our reference staff can send you copies for a fee. 

To request copies of this data contact us at:

New York State Archives
Cultural Education Center
Albany, NY 12230
Phone: (518) 474-8995
Fax: (518) 473-9985
E-Mail: archref@nysed.gov

Please note:

The data collection forms in Appendix D cannot be downloaded. They are available only in paper copies and must be ordered from the State Archives. If you cannot access the files, please contact archinfo@nysed.gov

How to Use this Manual

This Users' Guide is designed to facilitate access to the information contained in the Basic Educational Data System's (BEDS) Institutional Master File. The Guide is divided into five sections to simplify use. Section I describes the primary functions of BEDS and its two main components: The Institutional Master File (IMF) and the Personnel Master File (PMF). Section II provides an outline of the technical specifications and informational content of the Institutional Master File. Sections III and IV describe the State Archives' role in the preservation and availability of archival records of New York State Government. These sections also provide information on the Archives' access and reference services.

The main text of the User's Guide is supplemented by Appendices A-D which contain the annual file layouts/codebooks for the IMF as well as sample forms used to collect the IMF data. Appendices A-C contain the physical file layouts and coding information for each record type included in the IMF. The sections are separated according to record type and then arranged in chronological order: Appendix A describes district records, Appendix B describes school records (public and non-public), and Appendix C describes BOCES records.

When appropriate, the user is referred to actual questions on the IMF Data Forms, located in Appendix D of the manual. Appendix D does not contain a complete set of data forms. Rather, it contains a selection of forms which illustrate the most common questions and formats used to collect IMF data. Additional forms can be obtained by contacting the State Education Department's (SED) Information Center on Education (ICE).

Basic Educational Data System

The Basic Educational Data System (BEDS) was designed to collect, store, and disseminate information about New York State's elementary and secondary schools. The data in BEDS has been collected annually since 1967 and provides information on public and private schools, and school districts. In 1970, BEDS also began to collect data from the Boards of Cooperative Educational Services (BOCES).

BEDS supports two master files, the Institutional Master File (IMF) and the Personnel Master File (PMF). The PMF contains demographic information on public school and BOCES professional staff in teaching and non-teaching assignments. It also identifies the staff member's professional background, specific assignments, and student registration figures per assignment. The PMF also identifies the type of student taught by the professional (below average, average, honors, etc.). The IMF contains data on each school district, public and private school, and BOCES in New York State. Each record provides information on enrollment statistics, the number and type of professional and paraprofessional staff and limited information on facilities such as the number and type of classrooms and buildings. The School and BOCES Data Records identify special educational programs available at each institution. Public and private school facilities, educational resources, and the number and type of staff. This Guide will provide users with the technical information, documentation, and other descriptive information necessary to use the IMF.

In 1965, the State Education Department's (SED) Bureau of Statistical Services was charged with developing an information system on public schools that would reduce the data collection burden on schools, provide SED program units with informational support, and provide centralized research data on schools in New York State. During the early 1960s, data collection by SED was decentralized. The Bureau of Statistical Services, established in 1904, collected and published basic statistics on the number of teachers and students in New York's public schools and the amount of funds dedicated to public education. Also during the 1960s, various SED units collected detailed information on schools, teachers, and students to support individual programs and functions. There was no centralized policy for data collection and there was no central repository for such information. Therefore, SED was unable to efficiently respond to requests for information from school administrators, the State Legislature, educational researchers, and others.

The design and implementation of the BEDS began in 1965 and was completed in 1967. During that same year, the Information Center on Education was formed from the Bureau of Statistical Services to administer the system. ICE designs and operates data systems in all areas of education and coordinates all external data collection by SED. It is composed of two units, the Bureau of Statistical Services and the Bureau of Educational Data Systems.

ICE transferred the 1967-1981 IMF files to the State Archives in 1991. The data files for subsequent years are scheduled to be transferred to the Archives ten years after their date of creation.

Institutional Master File

The IMF data sets are rectangular flat files containing numeric and character data. The size of each file varies according to the reporting year. The 1967-1981 files are stored on two 2400 ft. magnetic tapes at 6250 bpi. The unit of analysis for all records within each file is the institution (the individual public or private school, BOCES, or District.)

The IMF contains four distinct record types: the School Data Record (including public and private schools), District Data Record, and BOCES Data Record. Each record begins with a header section identifying the institution type, its name and location, and the name of its chief officer. Each record also includes data on the facility's number and type of staff, enrollment statistics, current programs, and educational resources.

The key identifying elements in the header section, such as record type, school code, name and location, are consistent from year to year. However, the specific questions asked about programs and facilities vary slightly on each survey, causing significant changes in the way the data is coded. All changes have been identified by SED and are reflected in the annual codebooks in Appendices A - C of this Guide.

A. PUBLICATIONS

ICE generates several annual reports based on IMF data. These reports and associated publications are listed below.

  1. High School Distribution - The report lists the number and percent distribution of graduates going on to post-secondary education, employment, military service, and other activities. The reports are arranged according to individual schools and include district, county, region, and state summaries. The report is published as Distribution of High School Graduates and College-Going Rate.
  2. High School Dropout Rate (public and non-public) - The report identifies the number of dropouts and the dropout rate from the previous school year. It is arranged according to school and includes county, regional, and state summaries.
  3. Schools by Grade Organization (public and non-public)- The report produces a list of schools with each of the different possible combinations of grades such as grades 1 and 2, grades 1,2 and 3, etc. Each category includes a summary page which counts the number of schools for each county within a specified region. The report also includes upstate and statewide totals.
  4. Grade by Grade Enrollment Figures for Non-Public Schools by County and Affiliation - The report is used to produce the non-public school enrollment publication. It is organized according to county and diocese or group affiliation.
  5. Public and Non-Public Enrollment Totals by District - The report produces enrollment statistics for each district. It provides separate statistics for public and non-public schools. The report also includes the total enrollment figures for each district.
  6. Listing of Schools with a Grade 12 (public and non-public) - Originally, this report was generated only by special request. However, SED received numerous requests for the list and instituted it as a standard annual report.
  7. Grade by Grade Enrollment Figures for Non-Public Schools by County and Region - This run produces two reports. The first summarizes data for all non public schools, with the exception of those regulated by the Department of Mental Hygiene, the Department of Correctional Services, and the Division for Youth. The second report provides summary data for the schools not included in the first report.
  8. Directory of School Administrators (public and non-public) - The report produces the listings used to compile the publication, Directory of School Administrators.
  9. School District Code Manual - The report produces the listings used to compile the publication, Code Manual of Public School Districts.
  10. School District Enrollment Projections - The report projects the enrollment by district for the next 3 years based upon the enrollment data contained on the two most current years of the IMF.
  11. Annual Educational Summary: Statistical and Financial Summary of Education in New York State - The information in this publication is compiled from IMF reports and ICE's ST-3 financial reporting system. The publication presents annual statistics on school district enrollment and finances. It also illustrates financial trends for previous years. The majority of the information is presented in summary tables and figures. However, individual school district data are shown for revenues, expenditures, property value and attendance.

B. DATA COLLECTION

Every October, on "Information Day," school administrators compile data for the IMF using three forms (see Appendix D of this guide for sample forms):

  1. School Data Form - completed by principals of public and non-public schools.
  2. School District Data Form - filled out by the district's superintendent of schools.
  3. BOCES - completed by the BOCES district superintendent.

After the forms are returned to SED, small batches of data are processed, edited and errors are identified. Asterisks are substituted for invalid data. However, it is possible that questions with a high percentage blank responses have been left blank or filled with spaces by SED staff. Such occurrences are identified in Appendices A - C of this guide. The batches are then added to a skeletal master file which already contains identifying and geographic data on the school, district, or BOCES.

C. ARRANGEMENT AND DESCRIPTION

The records are sorted and linked by the 12 digit school code developed by SED. This code identifies the following information:

1-2 County Location
3-4 City/Town Location
5-6 School District Number
7-8 District Type
9-12 School Code Within District

The 1967-1981 IMF files are stored, by year, on 2 magnetic tapes. The records in each file are arranged within that year by school code. The records have not been separated physically according to record type, rather they are stored as a single annual file. However, ICE created annual file layouts which provide the data specifications for each record type. The individual records can be accessed via the first variable, "Record Type" which is defined as follows:

1 = Public School
2 = Non-Public School
3 = District
4 = BOCES

D. CODE BOOK SAMPLE

Each codebook represents a different record type in the IMF: School, District, and BOCES. The codebooks have been arranged according to record type in order to reflect this difference and to facilitate longitudinal comparisons between common data sets. Each section begins with an introduction which describes standard variables and their coding structure. The introduction also identifies significant variables that are not standard to the IMF, but do occur in more than one file. This section also identifies variables with a high number of undefined values. Where possible, explanations have been given for the undefined variables.

It should be noted that BEDS did not compile BOCES statistics until 1970. Also, in 1968, the public and non-public school code books were combined into a single document. Institution-specific questions have been identified in the "REMARKS" section of the code book.

The following example is taken from the 1967 Public School Code Book. The letters in parentheses refer to the explanations which follow the sample.

FILE LAYOUT

Project: IMF (A)
File: 1967 SCHOOL DATA RECORDS - PUBLIC (B)
Program: ICE (C) 
Record Size: 1322 CHARACTERS (D)
Number of Records: 4529(E)

field description code equiv. card col. #char remarks
RECORD TYPE (F) (G) (H) (I) 1=PUBLIC 2=NONPUBLIC (J)
SCHOOL CODE   2-13 12 FOR AN EXPLANATION OF SCHOOL CODE SEE APPENDIX A SEC. I, NO. 2
DATA FLAG   14 1 1=YES 0=NO

(A) Indicates that this is a file layout for the Institutional Master File.
(B) Identifies the specific data set defined in the file layout.
(C) Specifies the name of the State Education Department that created the data set.
(E) Identifies the total number of records contained in the data set. Note: This does not indicate the total number of records for the entire 1967 IMF, rather it identifies the total number of 1967 Public School Records.
(F) Describes the variable being examined.
(G) Identifies the code values for the variables. This has often been left blank by ICE staff. This column is most frequently used to indicate when the breakdown of a particular data set should be repeated for the next data set.
(H) Specifies the variable's physical position in the data set.
(I) Calculates the number of characters in the variable.
(J) Identifies valid codes for the variables and other comments associated with the variable. Also indicates whether a particular variable has a significant amount of undefined values (unless already identified in the "POTENTIAL PROBLEM AREAS" section of this manual). NOTE: a ^ in this section indicates that the section is filled with spaces.

Each code book contains several variables which are used to identify the specific institution being analyzed. Four of these data elements are accessed through codes developed by SED. The coded information is defined in the "Code Manual for Public School Districts in New York State." This manual is published periodically by SED's Information Center on Education. The manuals that are relevant to the 1967-1981 IMF files were published in 1968, 1970, 1971, 1976, 1978, and 1980. Copies of these manuals are available in the State Archives.

The following variable codes can be defined using the code manuals:

  1. School Code
  2. Associated Supervisory District Code
  3. BOCES Code The Code Manual began to include valid BOCES codes in 1976. For earlier codes, contact the State Education Department Information Center on Education.

Access and Terms of Availability

The State Archives is the final repository for many State government records. One of the Archives' primary responsibilities is to identify, preserve, and make available for research use the archival records of New York State government. Archival records are those records that have enduring legal, administrative, historical and educational, or other research value. Archival records no longer actively used by their creating agencies are transferred to the State Archives where their preservation and future accessibility are ensured.

Reference and Access Services

The State Archives reference staff will assist the researcher with locating the requested information and printed code books and will provide other necessary advice. The State Archives presently provides the following services to allow access to electronic records:

Data will be provided to the public on the requested media at the cost of media replacement. The data is not to be redistributed. Processing and technical queries should be directed to the State Archives Research Services Unit, (518) 474-8955.

Appendices A-C. File Layouts/Codebooks

Appendices A-C contain the physical file layouts and coding information for each record type included in the IMF. The sections are separated according to record type and then arranged in chronological order: Appendix A describes district records, Appendix B describes school records (public and non-public), and Appendix C describes intermediate district (Board of Cooperative Educational Services--BOCES) records.

You may use the table below to download individual sections of Appendices A-C in Word format. The sections range in size from 12 to 68 kb.

Download corresponding data from the Institutional Master File

Appendix A
District
Appendix B
School
Appendix C
BOCES
Introduction Introduction Introduction
1967 1967 (non-public)
1967 (public)
 
1968 1968  
1969 1969  
1970 1970 1970
1971 1971 1971
1972 1972 1972
1973 1973 1973
1974 1974 1974
1975 1975 1975
1976 1976 1976
1977 1977 1977
1978 1978 1978
1979 1979 1979
1980 1980 1980
1981 1981 1981

 

Appendix D. Sample Data Forms

Appendix D contains examples of the data forms that were completed by individual district, school, and BOCES officials. The information collected from the forms was used to create the IMF. This section does not contain a complete set of forms for the 1967-1981 IMF, but rather samples of the forms used to collect the data in the IMF. However, the forms which reflect the most common questions and formats used to collect IMF data. The forms are arranged in the following order:

Appendix D  
1970 District
Public School
Non-Public School
BOCES
1973 District
Public School
1976 District
Public School
Non-Public School
BOCES
1978 District
Public School
Non-Public School
1979 District
Public School
Non-Public School
BOCES

Please note: The data collection forms are available only in paper copies.

Institutional Master File Data

You can download copies of the annual IMF by clicking on the links below.
Each file was compressed using WinZip.

1967-1968
1968-1969
1969-1970
1970-1971
1971-1972
1972-1973
1973-1974
1974-1975
1975-1976
1976-1977
1977-1978
1978-1979
1979-1980
1980-1981
1981-1982