If you've been looking to dabble in some AI and serverless, Phil Haack shared his process to create a SentimentBot for GitHub issues with Azure Functions.
Perhaps the combination of machine learning and human judgement could make the problem more tractable.
I decided to play around with Azure Functions because they have specific support for GitHub Webhooks. GitHub Webhooks and Azure Functions go together like Bitters and Bourbon.
If you want to skip the code and just test it out, head to this issue.