And no, the privacy rules don’t just stop the police from “misusing” their powers on the innocent.

Of course. But the reason for the rules is to forestall abuse of innocents. We draw a broad line because the end never justifies the means. But in the absence of a case where innocents can be abused, there can be no reasonable justification for handcuffing the police. All of your arguments amount to "I didn't do anything, so I don't want them to be able to check whether I did." But the problem only arises if it's possible that you didn't do anything and they come after you anyway -- it's the likelihood of false positives that raises the civil liberties issue, and that's what you haven't made a case for.