It looks like there's no anti-aliasing for the alpha channel which makes jittery edges:
The solution I now use, is exporting video with a green background and use the keyer in fcp to get rid off the backgrounds:
Will there be anti-aliasing for png sequence exports with transparency?