The Intelligence community used to be intelligent. Spies and agents knew, back in the day, that security depended on precise information and specific threats. It requires identifying plausible, actual dangers, and targeting those groups and individuals most likely involved in violent or otherwise threatening activity. They would have laughed at the whole concept of metadata collection. You do not improve the likelihood of finding a needle in a haystack by increasing the number of haystacks you have to sift through. When you treat entire segments of the society as suspect, you decrease your capability of accurately identifying real threats. The practice of “casting a wide net” by the modern Intelligence community has now manifested itself in the #MuslimBan, whereby 200 million people are now regarded as potential terrorists. This is not based on anything like a rational, tested, and proven theory for effective prevention of terrorism; it is based on bigotry and paranoia; and these are fatal flaws for anyone engaged in serious threat assessment.
In my opinion, the technology for metadata collection never had anything to do with security. Like every other technological development in the US, it is far more likely that it has been developed for private sector use under the pretext of security. It is designed for marketing purposes, for fine-tuning companies’ consumer demographic profiles. Mapping social networks, habits, areas of interest, and so on; provides companies with vital information about customers, and I believe that is the reason for this technology, and it will be transferred to the private sector (if it is not already the case), for this purpose.
But creating the rationale for metadata collection has tremendous policy ramifications. Not only does it spread unwarranted and irrational suspicion and hatred, it also undermines real, effective intelligence gathering, and this only increases the likelihood of terrorist plots going undetected.