That's not why they are leaving though. There could be the same number of anti-semitic cranks in the Labour Party but if the leadership was politically aligned with these politicians, they (as in the politicians who have resigned) would still be there. Why is Jon Lansman supported by the left but Berger not? They are both Jewish. The chair of Berger's CLP is Jewish, but he was against her. Is it because he, as a Jewish man, hates Jews?
It can be acknowledged that there is a problem with a number of cranks (small in the context of an organisation with over 500k members) that hasn't been dealt with properly, whilst at the same time acknowledging that said problem has been amplified and used by those with a different political agenda for political purposes. No amount of pearl clutching will change that simple fact.