Abstract

This paper describes how to find or filter high-quality ideas submitted by members collaborating together in online communities. Typical means of organizing community submissions, such as aggregating community or crowd votes, suffer from the cold-start problem, the rich-get-richer problem, and the sparsity problem. To circumvent those, our approach learns a ranking model that combines 1) community feedback, 2) idea uniqueness, and 3) text features—e.g., readability, coherence, semantics, etc. This model can then rank order submissions by expected quality, supporting community members in finding content that can inspire them and improve collaboration among members. As illustrative example, we demonstrate the model on OpenIDEO—a collaborative community where high-quality submissions are rewarded by winning design challenges. We find that the proposed ranking model finds winning ideas more effectively than existing ranking techniques (comment sorting), as measured using both Discounted Cumulative Gain and human perceptions of idea quality. We also identify the elements of winning ideas that were highly predictive of subsequent success: 1) engagement with community feedback, 2) submission length, and 3) a submission’s uniqueness. Ultimately, our approach enables community members and managers to more effectively manage creative stimuli created by large collaborative communities.

BibTeX Citation

@inproceedings{ahmed:cscw_2016_winning,
    address = {Portland, USA},
    author = {Faez Ahmed, and Mark Fuge},
    booktitle = {20th ACM Conference on Computer-Supported Cooperative Work & Social Computing},
    title = {Capturing Winning Ideas in Online Design Communities},
    month = {February},
    year = {2017},
    organization = {ACM}
}