The idea of recruiting applicants to process applications is an interesting and powerful one. Beyond patents, competitive licensing in general looks like a good domain for deploying this model.

It doesn't work everywhere, but has already proven itself in knowledge (wikipedia), and looks likely work in politics and law too.

The best people to search for prior art are your competitors, so they should be notified at the submission stage, with an open interface for adding prior art links.

Applicants could still do all sorts of work: Add prior art on other applications, process elements of other applications.
How to enforce honesty: Well, there will still be patent staff, and now there will be reputation, a very simple complaints mechanism, and reliability through agreement. And of course there is still legal penalty for making false statements etc.

If people have their work contested and judged in error, then they can have their own applications slowed down, be banned from future contributions etc. Make the balance of rewards so that the best thing to do is put up as much good evidence for their case as they can, not make mistakes. That is basically how peer review works in academia, and in jurisprudence. And, I guess too, in things like planning applications - where local owners are asked to comment on development applications. Making this into a wiki process might be an effective... although fraught, way to arrive at an agreed statement. We already iterate over contracts (agreements) why not iterate over disagreements? That is effectively what diplomacy is designed to facilitate, and what civilisation rewards.

As another idea: what about using it as open-source design optimisation. If you have an idea, push it up, like a patent, but with an Open license. Then people can edit the 'patent' design, optimising it.

Take cookery: I put up a recipe, then people comment on amounts, temperatures, times, substitute ingredients. That is effectively how Michelin * restaurants work - optimising.

Blog comments would be better, to, if they were iterated over :-)