Learning to Search More Deeply

Timeline

2016

Description

Weiyao Wang (Math) and Jennifer Du , along with NCCU Physics majors Jarrett Weathersby and Samuel Watson, spent ten weeks learning about how search engines often provide results which are not representative in terms of race and/or gender. Working closely with entrepreneur Winston Henderson, their goal was to understand how to frame this problem via statistical and machine-learning methodology, as well as to explore potential solutions.

Project Results

In order to understand Google's algorithm, the team web-scraped search results and used machine-learning to understand the importance of each feature. They then performed sentiment analysis to quantify public opinions from Twitter and used community-based crawling and seeding to collect information relevant to minority groups.

Download the Executive Summary (PDF)

Faculty Sponsor

• Sayan Mukherjee, Professor, Statistical Science

• Michael Lindon, Ph.D candidate, Statistics

• Jennifer Du, Duke University Computer Science


• Weiyao Wang, Duke University Computer Science, Mathematics, and Political Science


• Jarrett Weathersby, North Carolina Central University Physics


• Samuel Watson, North Carolina Central University Physics

• Sociology


• Anthropology


• Economics


• All quantitative STEM