Hole drilling is one of the most widely used operations in order to place bolts or rivets. Drilling holes in composites leads to drilling-induced defects and damages such as delamination, burr, microcracking, swelling, splintering, and fiber pullout. A reliable and effective simulation toolkit for composite drilling process is essential to capture the drilling induced damage and perform optimization of drilling parameters to achieve desired hole quality. Under this study, a combined experimental and numerical approach is developed to extract the composite response and damage distribution caused by the drilling operation for model exploration and validation. With an advanced reaction force evaluation system and the use of non-destructive inspection techniques, specimens of different configurations are drilled and post-drilling damage quantification are evaluated for validation of the toolkit. An enhanced user-defined material model is developed for the ABAQUS/explicit to simulate the time-dependent material removal process during drilling and associated damage distribution. Accounting for the effect of the shear dominant cutting during the drilling, enhanced damage models are implemented based on the use of a fiber-orientation dependent shear failure criterion coupled with a failure mechanism driven energy dissipation for the post-peak material characterization. The applicability and high-fidelity of the developed toolkit are demonstrated by comparing both the global response and local failure predictions with the testing results of quasi-isotropic coupons with a backing plate of two different configurations.