Academic search engines: constraints, bugs and recommendations
Academic search engines (i.e., digital libraries and indexers) play an increasingly important role in systematic reviews however these engines do not seem to effectively support such reviews, e.g., researchers confront usability issues with the engines when conducting their searches. To investigate whether the usability issues are bugs (i.e., faults in the search engines) or constraints, and to provide recommendations to search-engine providers and researchers on how to tackle these issues. Using snowball-sampling from tertiary studies, we identify a set of 621 secondary studies in software engineering. By physically re-attempting the searches for all of these 621 studies, we effectively conduct regression testing for 42 search engines. We identify 13 bugs for eight engines, and also identify other constraints. We provide recommendations for tackling these issues. There is still a considerable gap between the search-needs of researchers and the usability of academic search engines. It is not clear whether search-engine developers are aware of this gap. Also, the evaluation, by academics, of academic search engines has not kept pace with the development, by search-engine providers, of those search engines. Thus, the gap between evaluation and development makes it harder to properly understand the gap between the search-needs of researchers and search-features of the search engines.
Fri 18 NovDisplayed time zone: Beijing, Chongqing, Hong Kong, Urumqi change
14:00 - 15:30 | Best Practices for TestingA-TEST at SRC GLR Chair(s): Beatriz Marín Universitat Politècnica de València | ||
14:00 30mTalk | Guidelines for GUI testing maintenance: a linter for test smell detection A-TEST Tommaso Fulcini Politecnico di Torino, Giacomo Garaccione Politecnico di Torino, Riccardo Coppola Politecnico di Torino, Luca Ardito Politecnico di Torino, Marco Torchiano Politecnico di Torino | ||
14:30 30mTalk | Academic search engines: constraints, bugs and recommendations A-TEST | ||
15:00 30mTalk | Closing A-TEST 2022 A-TEST Ákos Kiss University of Szeged, Hungary, Beatriz Marín Universitat Politècnica de València, Niels Doorn Open Universiteit and NHL Stenden University of Applied Sciences |