X-axis: Information Sources
Y-axis: Percentage of Respondents,
Categories: City website, phone call to city official, internet search, GenAI, Social Media, Ask a friend
Survey collection
Our final data collection method was an online survey designed to gather surface-level insights into residents' experiences and sentiments regarding retrieving information about Ann Arbor through GenAI. Conducted via Google Forms, we collected 656 anonymous responses, with 88.7% of participants being Ann Arbor residents.
Moderated field study
Conducted 70 moderated field studies with Ann Arbor residents located on campus, downtown, up north near the Huron river, and many other public spaces in the city. Participants were also recruited from survey respondents who indicated an interest in being interviewed, taking place over Zoom. These moderated field study sessions, taking place between May 13 and May 15, 2024, offered insights into first impressions, usability challenges, and general emotions related to GenAI.
The study began with questions about participants' knowledge of GenAI and what local concerns they may have, preparing them for subsequent tasks. Participants then engaged with UM-GPT to ask general questions about Ann Arbor, ranging in complexity - some of which were hypothetical queries containing false information. Finally participants were asked to reflect on this experience and describe their satisfaction with the GenAI interface and its responses.