This Users' Guide is designed to facilitate access to the information contained in the Basic Educational Data System's (BEDS) Institutional Master File. The Guide is divided into five sections to simplify use. Section I describes the primary functions of BEDS and its two main components: The Institutional Master File (IMF) and the Personnel Master File (PMF). Section II provides an outline of the technical specifications and informational content of the Institutional Master File. Sections III and IV describe the State Archives' role in the preservation and availability of archival records of New York State Government. These sections also provide information on the Archives' access and reference services.
The main text of the User's Guide is supplemented by Appendices A-D which contain the annual file layouts/codebooks for the IMF as well as sample forms used to collect the IMF data. Appendices A-C contain the physical file layouts and coding information for each record type included in the IMF. The sections are separated according to record type and then arranged in chronological order: Appendix A describes district records, Appendix B describes school records (public and non-public), and Appendix C describes BOCES records.
When appropriate, the user is referred to actual questions on the IMF Data Forms, located in Appendix D of the manual. Appendix D does not contain a complete set of data forms. Rather, it contains a selection of forms which illustrate the most common questions and formats used to collect IMF data. Additional forms can be obtained by contacting the State Education Department's (SED) Information Center on Education (ICE).
The Basic Educational Data System (BEDS) was designed to collect, store, and disseminate information about New York State's elementary and secondary schools. The data in BEDS has been collected annually since 1967 and provides information on public and private schools, and school districts. In 1970, BEDS also began to collect data from the Boards of Cooperative Educational Services (BOCES).
BEDS supports two master files, the Institutional Master File (IMF) and the Personnel Master File (PMF). The PMF contains demographic information on public school and BOCES professional staff in teaching and non-teaching assignments. It also identifies the staff member's professional background, specific assignments, and student registration figures per assignment. The PMF also identifies the type of student taught by the professional (below average, average, honors, etc.). The IMF contains data on each school district, public and private school, and BOCES in New York State. Each record provides information on enrollment statistics, the number and type of professional and paraprofessional staff and limited information on facilities such as the number and type of classrooms and buildings. The School and BOCES Data Records identify special educational programs available at each institution. Public and private school facilities, educational resources, and the number and type of staff. This Guide will provide users with the technical information, documentation, and other descriptive information necessary to use the IMF.
In 1965, the State Education Department's (SED) Bureau of Statistical Services was charged with developing an information system on public schools that would reduce the data collection burden on schools, provide SED program units with informational support, and provide centralized research data on schools in New York State. During the early 1960s, data collection by SED was decentralized. The Bureau of Statistical Services, established in 1904, collected and published basic statistics on the number of teachers and students in New York's public schools and the amount of funds dedicated to public education. Also during the 1960s, various SED units collected detailed information on schools, teachers, and students to support individual programs and functions. There was no centralized policy for data collection and there was no central repository for such information. Therefore, SED was unable to efficiently respond to requests for information from school administrators, the State Legislature, educational researchers, and others.
The design and implementation of the BEDS began in 1965 and was completed in 1967. During that same year, the Information Center on Education was formed from the Bureau of Statistical Services to administer the system. ICE designs and operates data systems in all areas of education and coordinates all external data collection by SED. It is composed of two units, the Bureau of Statistical Services and the Bureau of Educational Data Systems.
ICE transferred the 1967-1981 IMF files to the State Archives in 1991. The data files for subsequent years are scheduled to be transferred to the Archives ten years after their date of creation.
The IMF data sets are rectangular flat files containing numeric and character data. The size of each file varies according to the reporting year. The 1967-1981 files are stored on two 2400 ft. magnetic tapes at 6250 bpi. The unit of analysis for all records within each file is the institution (the individual public or private school, BOCES, or District.)
The IMF contains four distinct record types: the School Data Record (including public and private schools), District Data Record, and BOCES Data Record. Each record begins with a header section identifying the institution type, its name and location, and the name of its chief officer. Each record also includes data on the facility's number and type of staff, enrollment statistics, current programs, and educational resources.
The key identifying elements in the header section, such as record type, school code, name and location, are consistent from year to year. However, the specific questions asked about programs and facilities vary slightly on each survey, causing significant changes in the way the data is coded. All changes have been identified by SED and are reflected in the annual codebooks in Appendices A - C of this Guide.
ICE generates several annual reports based on IMF data. These reports and associated publications are listed below.
Every October, on "Information Day," school administrators compile data for the IMF using three forms (see Appendix D of this guide for sample forms):
After the forms are returned to SED, small batches of data are processed, edited and errors are identified. Asterisks are substituted for invalid data. However, it is possible that questions with a high percentage blank responses have been left blank or filled with spaces by SED staff. Such occurrences are identified in Appendices A - C of this guide. The batches are then added to a skeletal master file which already contains identifying and geographic data on the school, district, or BOCES.
The records are sorted and linked by the 12 digit school code developed by SED. This code identifies the following information:
1-2 County Location
3-4 City/Town Location
5-6 School District Number
7-8 District Type
9-12 School Code Within District
The 1967-1981 IMF files are stored, by year, on 2 magnetic tapes. The records in each file are arranged within that year by school code. The records have not been separated physically according to record type, rather they are stored as a single annual file. However, ICE created annual file layouts which provide the data specifications for each record type. The individual records can be accessed via the first variable, "Record Type" which is defined as follows:
1 = Public School
2 = Non-Public School
3 = District
4 = BOCES
Each codebook represents a different record type in the IMF: School, District, and BOCES. The codebooks have been arranged according to record type in order to reflect this difference and to facilitate longitudinal comparisons between common data sets. Each section begins with an introduction which describes standard variables and their coding structure. The introduction also identifies significant variables that are not standard to the IMF, but do occur in more than one file. This section also identifies variables with a high number of undefined values. Where possible, explanations have been given for the undefined variables.
It should be noted that BEDS did not compile BOCES statistics until 1970. Also, in 1968, the public and non-public school code books were combined into a single document. Institution-specific questions have been identified in the "REMARKS" section of the code book.
The following example is taken from the 1967 Public School Code Book. The letters in parentheses refer to the explanations which follow the sample.
FILE LAYOUT
Project: IMF (A)
File: 1967 SCHOOL DATA RECORDS - PUBLIC (B)
Program: ICE (C)
Record Size: 1322 CHARACTERS (D)
Number of Records: 4529(E)
field description | code equiv. | card col. | #char | remarks |
---|---|---|---|---|
RECORD TYPE (F) | (G) | 1 (H) | 1 (I) | 1=PUBLIC 2=NONPUBLIC (J) |
SCHOOL CODE | 2-13 | 12 | FOR AN EXPLANATION OF SCHOOL CODE SEE APPENDIX A SEC. I, NO. 2 | |
DATA FLAG | 14 | 1 | 1=YES 0=NO |
(A) Indicates that this is a file layout for the Institutional Master File.
(B) Identifies the specific data set defined in the file layout.
(C) Specifies the name of the State Education Department that created the data set.
(E) Identifies the total number of records contained in the data set. Note: This does not indicate the total number of records for the entire 1967 IMF, rather it identifies the total number of 1967 Public School Records.
(F) Describes the variable being examined.
(G) Identifies the code values for the variables. This has often been left blank by ICE staff. This column is most frequently used to indicate when the breakdown of a particular data set should be repeated for the next data set.
(H) Specifies the variable's physical position in the data set.
(I) Calculates the number of characters in the variable.
(J) Identifies valid codes for the variables and other comments associated with the variable. Also indicates whether a particular variable has a significant amount of undefined values (unless already identified in the "POTENTIAL PROBLEM AREAS" section of this manual). NOTE: a ^ in this section indicates that the section is filled with spaces.
Each code book contains several variables which are used to identify the specific institution being analyzed. Four of these data elements are accessed through codes developed by SED. The coded information is defined in the "Code Manual for Public School Districts in New York State." This manual is published periodically by SED's Information Center on Education. The manuals that are relevant to the 1967-1981 IMF files were published in 1968, 1970, 1971, 1976, 1978, and 1980. Copies of these manuals are available in the State Archives.
The following variable codes can be defined using the code manuals:
The State Archives is the final repository for many State government records. One of the Archives' primary responsibilities is to identify, preserve, and make available for research use the archival records of New York State government. Archival records are those records that have enduring legal, administrative, historical and educational, or other research value. Archival records no longer actively used by their creating agencies are transferred to the State Archives where their preservation and future accessibility are ensured.
The State Archives reference staff will assist the researcher with locating the requested information and printed code books and will provide other necessary advice. The State Archives presently provides the following services to allow access to electronic records:
Data will be provided to the public on the requested media at the cost of media replacement. The data is not to be redistributed. Processing and technical queries should be directed to the State Archives Research Services Unit, (518) 474-8955.