This paper reports the seismic vulnerability of vernacular Newari buildings based on the damage observations during four major earthquakes (1934, 1988, 2011 and 2015) that struck Bhaktapur city. Some specific features that contributed to collapse prevention in traditional masonry buildings are also highlighted in this paper. In this paper, we have outlined the insights of each earthquake using forensic interpretations and the recommendations by various researchers after 1934 and 1988 earthquakes are compared in terms of their implementation. With the help of damage database recorded during 1934, 1988 and 2015 earthquakes, we have created damage probability matrices and empirical fragility functions for traditional masonry structures. The fragility functions and damage probability matrices derived in this study highlight that most of the vernacular Newari buildings are likely to be collapsed in the case of strong to major earthquakes.