Palantir: A Private Sector Solution to a Public Sector Problem?

Palantir Technologies, a Silicon Valley firm of self-identified geeks, has recently been sought out by government agencies such as the CIA, DHS, and FBI to enhance their abilities to find needles in very big haystacks.  Palantir has created a user friendly technology that empowers government analysts with the ability to integrate unstructured open source information with data from various agency databases to analyze them for outstanding correlations and connections in an attempt to mitigate the burden of rummaging around through the immense amount of information available to them  

Currently, information sharing and coordination between government agencies is limited, as is integration of open source data such as message traffic.  This is due to both the sheer amount of data and security clearance complications.  Software programs such as the products Palantir has developed allow analysts to map hostile activity and identify critical vulnerabilities by integrating and mining information from various data streams to find the subtle correlations that could be indicators of larger patterns and threats.  

In particular, the software created by Palantir marks each bit of data with a security level; therefore, analysts have access to all information at or below their clearance level even if it is embedded in a file with more highly classified material.  Such an innovation in knowledge sharing could be a key component in the future of intelligence gathering, especially given the difficulty of information sharing and coordination across various agencies and security clearance levels.  Further development and deployment of software of this type could bring interoperability of intelligence collection agencies to an unprecedented level allowing analysts to pick up on correlations that would have previously gone unnoticed or taken more critical time to discover.  
 

http://online.wsj.com/article/SB125200842406984303.html
http://www.nytimes.com/2009/05/12/science/12cyber.html?_r=1&scp=1&sq=palantir&st=cse
http://www.palantirtech.com/

Amy Qualliotine

Intern, CSIS Technology and Public Policy