Can I keep only the crowd noise?

Yes. When you extract the stems, you'll receive both the separated music track and the isolated crowd noise track. You can keep just the crowd noise if that's what you need.