Mindset + Skillset + Toolkit = Success
So I had created a Planning Poker game for Sprint Story estimation in the past. It's available here.
The problem was that Peer.JS uses WebRTC and in the absence of a decent TURN server (which I don't have) doesn't work well with NAT and people behind firewalls.
Solution: Use websockets. So I got Socket.IO, messed around with it and changed the old code to use it. This solves the issues people were having with the firewalls and unable to join games.
To use it go here. Type in a name. Click new game. To invite others copy and paste the share URL near the top.