Colson Services: Fill & Download for Free

Generally speaking, I think companies overly specialize their data science roles (though there are some conditions when specialization is appropriate). In most cases full-stack data science generalists will add far more value.The lure of specialization is process efficiencies due to the Division of Labor. However, this is only effective when the design can be done up-front. That is, the learning has already been done and you know exactly what you want to build already (down to the millimeter of specification) and all that remains is to execute on the plan. In such cases specialists will execute efficiently.But data science products and services can rarely be designed up-front. Data science is about learning and discovery - not execution. Algorithmic products and services are novel capabilities with inherent uncertainty. Coefficients, models, model types, hyper parameters, all the elements you’ll need must be learned through experimentation, trial and error, and iteration. With data science, you learn as you go, not before you go. And, this learning-while-doing style of working requires fluidity between the functions (data engineering, ml engineering, inference, analysis, etc). If you staff each function with a different specialists then the coordination costs and “wait-times” go way up - its just too many people to corral. Instead, go with a full-stack data science generalist - one that can perform all of those functions. She will iterate fluidly between those functions as she learns. Coordination costs and wait-time fall towards zero. She learns faster, innovates more, and produces step-change value for the company - not process efficiencies. She’ll also be better motivated as she has the autonomy, mastery, and purpose.More on this in my HBR Article: Why Data Science Teams Need Generalists, Not Specialists.

One area of recommendation engines that is still relatively nascent is the use of human computation as a resource in the system. Machines are better at some tasks (e.g. quickly performing complex calculation on large amounts of structured data); humans are better at others (e.g. interpreting meaning in images and language). There is a certain class of problem that calls for precisely the combination of these capabilities.One example of this is Stitch Fix, an online personal shopping service for apparel (disclosure: I work there). The service would not be effective with either humans or machines alone. The contributions of the various resources are complimentary and additive (possibly synergistic). The machines provide fast calculations on structured data which serves to rank and qualify the recommendations. This enables the recommendations to be reduced down to manageable and ordered chunks. In turn, this enables expert-humans to further process the results, leveraging additional data accessible only to them. For example, unstructured data such as raw text and images typically require humans to extract the meaning, yet they materially improve the recommendations. Humans can also contribute their own proprietary knowledge such as trends they have observed but which are not yet manifest in the data.You can find more info on Stitch Fix from a short talk here:Human computation is an emerging field. There is a great conference taking place this week: HCOMP 2013

From a business model perspective, Stitch Fix is retail. That is, it buys inventory at wholesale cost and sell it at retail prices. However, the experience for the customer is very different from traditional retail. Rather than searching from store to store to find relevant merchandise, the customer simply provides her preferences (for size, style, fit, price, ...etc) in a style profile. The service then uses a combination of machine-learning and expert-human judgment in its algorithms in order to recommend five personalized items for her. The items are then shipped directly to the customer where she can try them on in the privacy of her own home, with her own wardrobe, and with feedback from friends or family. She pays for only what she decides to keep and there is free shipping both ways. The only upfront cost is a $20 styling fee which applies to her purchases if she keeps anything. There is also a 25% discount if the she buys all 5 items. Customers can request a shipment whenever they wish or can opt into a subscription cadence (monthly, quarterly, ...etc).For more on how its technology works: Stitch Fix Technology.[Disclosure: I work there!]