(Originally posted on the HyperStudio blog November 25, 2013)
Think of the word “algorithm” and you might picture a data scientist crunching numbers in front of a terminal, analyzing functions and equations that you can’t begin to understand. If they’re building models of weather systems, you might be right (I can’t help you there). But recommendation systems are another story. The algorithms can be complex, but the output tends to be very simple: a list of some news articles, movies, or other content you might like.
Recommendation science is not rocket science, but it has always been the realm of the engineer. There’s no doubt that a good engineer can hone and chisel a recommendation algorithm to near-perfection (whatever that means), but first you have to choose the type of stone. Each recommendation system has certain methods and assumptions baked into it, and determining what kinds of inputs belong can be more of an art than a science.
Recommendation systems are generally divided into two types: collaborative filtering, and content-based filtering. Content-based filtering focuses on the product itself, like a traditional library classification system. Netflix provides one example: after culling their records down to several thousand movies and TV shows, they hire freelance film buffs to tag content with delightfully contrived categories like “Mind-Bending Romantic Foreign Movies” or “Understated Detective TV Shows” (though they also match you to similar users). While these are more fun than generic, automated tags (and Netflix deserves credit for using humans), these categories are still boxes; they place cultural products into certain discourses and implicitly exclude others. Tagging systems are inherently stale and lacking in dynamism, and folksonomies aren’t always feasible and come with their own problems.
One attempt to sidestep classification is via collaborative filtering, which looks at the user, their past behavior, and similar users or social networks for clues into what the user might like. Consider Amazon, who uses this model extensively; given the massive scale of products on offer, many from third parties, it is more manageable to leverage machine-learning algorithms that watch what you buy and browse, rather than attempting to infer the properties of thousands of new products a day. A user-history based approach maximizes efficiency but at the sake of variety, assuming that you want to keep seeing more of the same.
If it’s looking to your social networks, it feeds you what’s already most popular, stifling individual preference and shepherding audiences into identical routes. If you were to sit around a table with “users like you” and start a conversation, would you rather have seen everything they’ve seen, or something a little different? If you’re all on the same pages, how would anyone bring anything new to a discussion? What about all the possible treasures out there that haven’t been discovered yet? Collaborative filtering may work when shopping for consumer products, but it creates a filter bubble for art and culture.
The challenges and limitations behind each of these models is very different, and they point to a need to focus on the objects and users of recommendation systems, rather than the algorithms. As taste plays more of a factor in recommendation, this becomes even more crucial. If you click away from a product on Amazon, maybe it’s because you didn’t like the price, the quality, or the reviews. Regardless, the company assumes something about you. It gets even more complicated with art or music. Art has ever-changing material, cultural, and discursive properties; which ones are most important to a given viewer? Do they like being challenged and broadening their horizons, or prefer staying in their comfort zone within a certain style or mood? If so, is it worth trying to change their mind?
This brings up another variable that goes underserved when the focus is on the algorithm: what is the metric for a “successful” recommendation system, and how does that change from company to institution? Industry tends to give people more of what they want, with the end goal of a click or purchase. Cultural institutions like ours can break convention here, and at HyperStudio we hope to challenge users by making unique connections and new introductions, so long as users are ultimately delighted and informed. Unencumbered by industry demands like growth and scale, we can maintain a very different metric for success, and it could be unique to each project or user.
We also hope to open up further discussion about these systems and their limitations. The extreme secrecy behind companies’ proprietary algorithms and the domain’s traditionally engineering bent make the recommendation system something of a black box. Given the extent to which automatic recommendations affect what we read or hear about, it’s important to understand what can go into them. HyperStudio and other open-source initiatives can play a role in making them more transparent: creation and discussion of recommendation algorithms could lead to insight about the decisions computers are making for us and their assumptions about what we want. Perhaps in our effort to improve them, we’ll discover some ways in which proprietary algorithms are failing us.
Regardless, I should hope that people have a different relationship to art than to a product or piece of information, and cultural institutions should treat their audiences differently from companies. It’s important to devise recommendation systems that avoid reducing cultural objects to the level of products, and museum audiences to consumers. At the heart of the collaborative and content-based systems is the notion that more personalization leads to higher quality, and that existing networks, discussions and canons are there to be reinforced. These are meaningful and important signals, but they need not be the only ones. While quality is always important, categories should be fuzzy, as should networks of people; the most important signals are often the nodes that link them, and we hope to surface these new connections. When it comes to art and culture, looking past the current limitations of discovery will be vital for generating new ideas and conversations.