Test Development Project by Tests and Measures Testing Corporation Report

Introduction and Purpose of the Test Development

The project revolves around the construction of a content-valid test for the subject of reliability that will be employed by Tests and Measures Testing Corporation. That is for their future courses on applied tests as well as measures. The prepared content-valid test will be attached in the Appendix A section of this report. Moreover, the primary purpose of this project will quantify the level of understanding of the term and definition of reliability. The content-valid test will also assess the various forms of reliability that are regularly employed in projecting the reliability of various tests.

Is your time best spent reading someone else’s essay? Get a 100% original essay FROM A CERTIFIED WRITER!

Save 25%

Background on Reliability

Reliability is measured to be among the most imperative rudiments of test development (Swanson, 2014). It is a terminology used to characterize the extent to which a given evaluation or test can produce consistent and non-biased results (Kaplan & Saccuzzo, 2018). In different terminologies, reliability can also characterize the consistency as well as repeatability of a given measure, evaluation or test (Mohajan, 2017). Also, for a test or measure to be considered to be reliable, it should manifest similar results recurrently across different samples in a research population. Nevertheless, such a repeated occurrence in the consistency of the data evaluated from a test should occur under similar conditions across the samples of a population.

Item Development Process

Step 1: Planning. In this stage, the prerequisites for creating a content-valid test will be identified based on the purpose and goal of the project. The essentialities to be planned in this case will include test instructions, scoring methods, data interpretation, length of the test and test format.

Step 2: Definition of the Content. The developer of the content-valid test provided a comprehensive explanation of the meaning of reliability. The developer also defined the content-valid tests' parameters and issued projections of different forms of reliability including internal consistency reliability, inter-rater reliability, parallel forms reliability as well as test-retest reliability.

Step 3: Test Specifications. The developer of the content-valid test created the procedure that was used using the content matrix, which was a technique that featured the items that would be sampled on the content-valid test. The items that were included in the test included knowledge, understanding, and application (Check Appendix B to see the content matrix).

Step 4: Development of Items. The creator of the content-valid test developed 10 multiple choice questions that summarized the procedure covered by the content matrix. All the questions created were assessed for their accurateness, relevance, and grammatical correctness.

Step 5: Test Designing and Assembling. The content-valid test designer used a multiple choice questions technique (one primary response and three distractors), with all the questions arranged appropriately based on the items of the test. The answer key was also created and later evaluated for its precision where the minimum acceptable score was set at 0.8 or 80% (check Appendix C to see the test answer key).

Proposed Method of Administration

The sample population taking part in the project were offered the 10 item test to fill using the questionnaire format. The sample participants then read all the questions issued to them keenly and selected a single response from the four available possible responses. All selected responses were shaded using a colored crayon that was offered to the respondents by the content-valid test developer. The time to be allocated to every respondent to responded to the 10 multiple choice questions is 25 minutes.

Recommendations

Ultimately, the designer of the content-valid test recommends that to promote the accuracy of the results collected from the test respondents, additionally, 10 questions should be included. Nevertheless, the questions should not pass a maximum of 20 to avoid creating a fatigue effect on the respondents. Also, the number of possible choices to the questions should be reduced to three instead of four to reduce the time taken by the respondents to complete the content-valid test. Also, the reduction will also ensure that the respondents of the test questions have attained sufficient time to answer and proofread their responses before submitting their filled 10-question questionnaire to the test developer.

Conclusion

In conclusion, the primary use of this report is to provide a description of the procedure used by the designer of the content-valid test to be employed in the future by the Test and Measures Testing Corporation. The developed content-valid test will be used by the company for applied tests as well as measures. Also, the content-valid test will assess the sample population's knowledge of the meaning of the term reliability. The test will have 10-multiple choice queries focused on the concept of reliability. Also, the questions featured were created using the content matrix methodology and also checked using the item writing guidelines. The five-step procedure that should be used to administer the content-valid test was also featured with the steps being planning, content definition, test specifications, items development and test designing and assembling. Ultimately, appropriate recommendations for improving the content-valid test were offered by the developer and they were focused on enhancing the effectiveness of the test when being used for data collection.

References

Kaplan, R. M., & Saccuzzo, D. P. (2018). Psychological testing: Principles, applications, & issues (9th ed.). Boston MA: Wadsworth, Cengage Learning.

Mohajan, H. (2017). Two Criteria for Good Measurements in Research: Validity and Reliability. MPRA, 17(3), 58-82.

Swanson, E. (2014). Validity, Reliability, and the Questionable Role of Psychometrics in Plastic Surgery. PRS Global Open: International Open Access Journal of the American Society of Plastic Surgeons, 2(6), 161. doi:10.1097/GOX.0000000000000103

Appendices

Appendix A: Content Valid Test

Question Possible choices

1. Reliability is measured to be ... in test development

• Among the most imperative
• Least essential
• Averagely important
• None of the above

2. Reliability is the extent to which an evaluation ...

• Produces consistent results
• Non-biased results
• Produces consistent and non-biased results
• None of the above

3. Choose the terms that do not represent reliability of data

• Repeatability
• Consistency
• Precision
• Biased

4. The purpose of a content-valid test is to ...

• Identify reliability
• Measure the knowledge of reliability
• Identify the different forms of reliability
• All the above

5. The degree of consistency or agreement in a repeated administration of the diagnostic test is termed as ...

• Rater reliability
• Intra-rater reliability
• Intra reliability
• Nonreliability

6. Test-Retest Reliability is the ... Evaluation of a test's consistency measured at different occasions over a defined period

• Retesting the same test over and over again indefinitely
• Retesting the reliability of information already validated
• None of the above

7. The measure of a reliability that is attained after administering varying versions of assessment methods or tool is known as ...

• Assessment reliability
• None reliability
• Parallel forms reliability
• Forms reliability

8. Which are the main forms of reliability? Internal consistency reliability, inter-rater reliability, test-retest reliability and parallel forms reliability

• Inter-rater reliability, non-rater reliability, test-retest reliability and parallel forms reliability
• All-reliability, test-retest reliability, consistency, and parallel forms reliability
• Internal consistency reliability, inter-rater reliability, and test-retest reliability

9. Do you think the reliability of data is essential for a project?

• Yes
• No
• Not applicable
• None of the Above

10. What is internal consistency reliability?

• Consistency of all forms of data in a project
• Method of gauging how well a survey measures the intended measure
• All the above
• Both 1 and 2

Appendix B: Content Matrix

Appendix C: Test Answer

Question Answer

1. A

2. C

3. D

4. D

5. B

6. A

7 C

8. A

9. A

10. B