ASReview

ASReview markets itself as an AI-powered active learning tool to streamline systematic review screening. However, the reality reveals significant shortcomings that undermine its practical utility.

• The machine learning models are fragile and domain-dependent, often requiring extensive tuning and user expertise to avoid poor performance.
• It frequently suffers from data sparsity and cold-start problems, where insufficient initial training data leads to unreliable prioritization.
• The promise of drastically reducing screening workload is often overstated, with real-world time savings being marginal for many topics.

• ASReview's user interface is minimalistic but non-intuitive, demanding steep learning curves for new users.
• It operates largely as a standalone tool, lacking seamless integration with popular reference managers, systematic review platforms, or collaboration tools.
• Export and import functionalities are limited, complicating workflow continuity and reproducibility.

• The AI decision-making process is largely a black box, offering little explainability on why studies are prioritized or excluded.
• There are minimal options for user intervention or manual override of AI decisions without disrupting the learning process.
• This opacity raises concerns about bias, errors, and accountability in critical review stages.

• ASReview focuses mainly on title and abstract screening, neglecting later review stages such as data extraction or risk of bias assessment.
• It is less effective for reviews with highly heterogeneous studies, non-English literature, or niche topics with sparse data.
• The tool does not yet support multi-user collaboration natively, restricting its use in team settings.

• Being a research-driven open-source project, ASReview suffers from infrequent updates and variable documentation quality.
• User support channels are limited, placing the burden on individual teams to troubleshoot and customize.

ASReview offers an intriguing glimpse into AI-assisted review but remains an immature, niche tool with significant limitations in usability, transparency, and real-world effectiveness. Its deployment should be cautious and supplementary, not foundational.

Recommendation: Use ASReview only as an experimental adjunct to established review processes, not as a replacement for rigorous human screening and judgment.