I think even according to R' Slifkin the 'traditional view' at least has significant though not universal support from Askhenazic Rishonim. R' Slifkin just argues that those Rishonim had never actually seen an olive, and if they had they would've described a smaller kazayit.

I had a joke about Chanukah in roga's comments a few years ago that went something like this:

Maccabee One: We can't light the menorah, we don't have olives to make pure oil.
Maccabee Two: Why can't we just use giant mutant olives?
Maccabee One: You dolt, those won't be invented for a thousand years!