Sian Astley said those worried about immigration are not ‘nasty people or rampant racists’ and hit back against an accusation her party had an ‘agenda of hatred and division’