So initially I thought it was to do with the Layouts. If I removed the extra Layouts the issue went away.
But it turns out that it is the preload. Taking preload of the stacks images stops the refresh.
I am not sure why this would be the case. But for some reason the Preload and Layouts together produces this issue. Alone the problem goes away.