Could you just put the url portion in an Other > Windows > Run an Application action?
https://player.twitch.tv/?channel=shroud&enableExtensions=true&muted=true&parent=twitch.tv&player=popout&volume=0%27,%27popout_chat%27,%27width=800,height=400%27
Edit - That is, remove the 'javascript:void(window.open())' portion from what you're already doing.