Analytics For Data Deconfliction: Fill & Download for Free

This answer was written in late 2012, and is no longer accurate. For a more up-to-date answer, see the answer written by my esteemed co-worker Anirvan Mukherjee below.Original answer:Short answer: To first order, not at all important.Longer answer: Palantir is almost entirely an analyst driven tool: your data is a huge rock that needs to be moved, and Palantir is the lever. In almost every video on Palantir's analysis blog (http://www.palantir.com/category/blogs/analysis-blog/) what you're seeing is somebody going through the steps of looking at a piece of data that has been presented to them, making a conscious decision about what aspect of that data to explore further, and using Palantir to let them explore the data easily and quickly. The computer doesn't decide which piece of data to present, or how much data to present, or even how to present the data; Palantir simply makes it very easy for human analysts to make these decisions.This raises an interesting question though: one of the reasons Palantir doesn’t make extensive use of ML in its core workflows or products is because it is designed to help with exactly the problems that are basically intractable to ML: dirty data sets, adaptable adversaries, highly complex markets…lots of people have tried to build computer algorithms to magically detect terrorist activity or bank fraud, and they have all turned out to be inadequate to the task, or worse, pure flim-flam (see, e.g., http://nyti.ms/eWZ4WT).Machine-learning algorithms are trained on a data set, and tend to perform very well on a certain subset of problems, and very poorly on most others. This makes it very difficult to build data analysis and exploration tools* that are both make heavy use of machine learning and are also data agnostic. The classic example of this is automated entity extraction from documents: Over-tuned entity extractors miss a lot of possible entities. Under-tuned ones have an unacceptable level of noise. Tuned just right, an entity extractor will work fantastically...on exactly the set of documents it was tuned on, and usually perform very badly on a different type of document (say, the Enron e-mails versus the USPTO database.) However, if you make it easy for human users to find, tag, and associate data on their own, you obviate the need for an entity extractor in most cases. This is the power of Palantir: instead of relying on computers to learn, it relies on humans to learn, and lets computers do the messy but conceptually simple parts.On the other hand, once your data is cleaned, normalized, deconflicted, deduplicated, and attached to a powerful API capable of looking at the data via many different modalities…then you can start thinking about what pieces of your problem are tractable to machine learning, and which ones make sense to automate. Palantir doesn’t do much ML…yet. But Palantir specializes in pretty much all the things you want to do before you start implementing your ML on real world data, outside of a clean laboratory environment. For example, one notable exception to the above is the use of graph analytical algorithms for building out some types of extremely dense graphs in the cyber security domain; when huge amounts of data are present, deciding how far down the rabbit hole to go can be tricky. Palantir Gotham has some tools that are designed to make "guesses" about how related various entites are, in order to present the end user with a reasonably compact graph as a starting point for further exploration. This kind of analysis is used in combatting cyber fraud, where the amount of data to be examined is immense, and the number of possible leads to follow is huge. The computer "curates" the starting graph, making it easier to get a handle on the data, before the analyst begins the job of sifting through it. This would be impossible and/or useless if data from several different sources wasn’t already integrated, normalized, and deduplicated. But the analyst still makes the final decision about what constitutes an "interesting" connection or piece of information.*Also, Palantir isn't strictly, or even primarily, a "graph exploration tool". A graph is one modality for visualzing relationships between pieces of data. Palantir Gotham makes use of graph representations, but also maps, timelines, timewheels, text, word clouds, and any other visualization you can code in Java and plug in to the API, and allows you to pivot between them easily. Palantir Metropolis on the other hand makes heavy use of scatter and time charts for representing financial and numerical data.