Develop a training procedure to improve the rater-reliability of a modified 5S audit checklist

Abstract

5S is a lean tool, which is a visual housekeeping technique that can be implemented in a shop floor to eliminate waste and improve the process. The performance of the 5S tool in a workplace is evaluated using a 5S audit checklist and the observers perform the evaluation. The scores of the completed checklist reflect the performance of 5S tool in the workplace. A study by Natarajan (2011) has shown that the current 5S audit checklist commonly being used in aerospace had low rater-reliability scores in a particular scenario. Therefore the objective of this thesis was to improve the reliability for the 5S audit checklist. First, the content validity of the checklist was tested with the help of five industry people and the checklist was modified. Then, a training procedure was developed on how to rate a workplace using the modified audit checklist. Then a study was performed to test the reliability of the modified checklist. For the study two groups of students were selected as raters. The first group, the control group, did not receive any training. The control group was asked to rate a virtual reality model of a factory workplace using the modified checklist. The second group, the experimental group, received the specially developed training on how to rate a workplace and was asked to rate the same virtual reality model. This study was performed again with a time gap of 3 weeks and the experimental group dint receive any training while doing it for the second time. The obtained results were analyzed using SPSS software and the control group results showed that the modified checklist had impact on improving the rater-reliability scores of the modified checklist but the experimental group results showed that training the raters has helped in improving the rater-reliability of the modified checklist. This suggests that training the raters before audits could produce good results because training will create a common frame of reference within the raters.