Collaborative Filtering and Recommender Systems: Human or Machine?

The Internet is quickly expanding from being a medium of computers and static information, to being a medium of video, audio, and multimedia broadcast. Independent producers of many types of communications content suddenly have the potential to reach huge audiences, directly. As a society, we currently have an unprecedented opportunity to make positive changes to our methods of communication.

Exciting, but it raises some questions: With countless people from all over the world attempting to distribute content on the Internet, how will everybody wade through it all to find what they like? In traditional media such as newspapers, television, and the music industry, content passes through several layers of filtering before reaching the public. Journalists decide which stories they want to cover, large organizations such as governments decide what information they want to relsease, news gathering institutions decide what information they want to distribute, news organizations decide what information they want to publish or broadcast, etc. Unfortunately, each layer of filtering carries an agenda and bias. How do we design or engage in filtering processes that best meet our individual needs and those of our communities?

Collaborative Filtering refers to the filtering that takes place via the interaction of many people sharing tastes and preferences. The longest standing method of Collaborative Filtering is word-of-mouth recommendations. People naturally tell those around them about the cultural they like and the information they are interested in. Cultural gems and information often becomes widely popular purely by spreading through this word-of-mouth process. Unfortunately, word-of-mouth filtering currently loses much of its influence on large scales, where it is overshadowed by well funded marketing campaigns. How can we allow natural word-of-mouth processes to have more impact on our media?

One current approach is Automated Collaborative Filtering (ACF), which allows people to search for content that is popular with people who tend to share their tastes. Preferences can be collected actively or passively. In the former, preferences are collected by allowing users rate content themselves. In the latter they are collected by monitoring activities of the user, such as the songs they listen to frequently. Many web sites and media players are already utilizing collaborative filtering technologies.

These tools are changing the way people discover culture and information, but it is important to remember that communications technologies are just tools, and cannot do anything exciting by themselves. Rather than replacing human to human communication with technologies, let us seek to replace anonymous, dehumanizing, and bias/agenda prone technologies with technologies that are grounded in human relationships.

The scope of word-of-mouth filtering could be expanded with the help of tools that make it convenient for people to recommend, record recommendations, and mine recommendation information for statistical purposes. Perhaps we could call this Recommendation Software.

Example: You see a short news clip that greatly interests you, and wish to recommend it to some friends. Pop up your hot list, click off the names of those who you think would also be interested. The next time they are searching for news information online, they see the news clip you recommended, along with the ratings/comments of other friends who have seen it, and statistical information about its popularity, whether anybody has logged counter-information or claims the information is false... etc. People already use email for this purpose, but it can easily lead to in-box clutter when too many people are forwarding or recommending the same thing. This phenomenon keeps many of us from using email as a means of recommending or forwarding information. Recommendation Software would allow us to efficiently use the Internet for this purpose. It would also have a tendency to self-popularize, as it would give people incentive to encourage their friends to sign up.

Possible features and benefits:

• Keep track of content items (books/movies/bands/etc.) that you want to check out.
• Create affinity groups: contacts who share an interest in certain categories of content items.
• Be alerted to content items that seem to be catching the attention of your contacts.
• See if any contacts have already encountered a given content item, and how much they liked it.
• Block or weight the recommendations of specific contacts, possibly by category (so a given contact's music recommendations carry weight, but their news recommendations don't).
• Features to help bust myths, misinformation, and hoaxes, such as the ability to register/be alerted to false information, ability to send hoax alerts back through the channels through which you received it.
• Tools that allow parties interested in certain subjects to find other parties also interested in the same. (Perhaps a box to check if you know of any groups related to the subject matter of a content item, and a looking-for-related groups feature)
• Plugging into media players, file sharing programs, Etc. Best if it worked across platforms/programs.
• Tip-the-artist features.

The effect of current filtering processes on our information and culture is both subtle and profound. They affect how we communicate as much as what we communicate about. By expanding the scope of word-of-mouth recommendation, we are moving towards the realization of a large-scale dialogue that better reflects the public mind. A dialogue of the commons, so to speak.

Unfortunately, there may simply be no good business models for Recommendation Software, as it would be most effective if freely available and cross-platform. An open source programming project would be the best approach. Although I am not a programmer, I would be interested in promoting or helping to fund such a project. Please email me if you know of any such development projects in process.

Informative ACF links at the University of Berkeley
Doug Oard's Information Filtering Page