ORES-Inspect

Try ORES-Inspect on Toolforge: https://ores-inspect.toolforge.org/

Read our summary paper, presented at Wiki Workshop 2024: https://arxiv.org/abs/2406.08453

Repository containing code for auditing the ORES edit quality models, including the ORES-Inspect tool.

Read more about this project and leave feedback at Research:ORES Inspect: A technology probe for machine learning audits on enwiki.

Repository contents:

• audit_web_client - All code for the front-end and back-end of ORES-Inspect. See the README in that directory for development details, including how to run ORES-Inspect locally and how to contribute.
• notebook - Jupyter Notebooks analyzing static Wikipedia data dumps.
• scripts - Scripts for downloading dumps from dumps.wikimedia.org
• scripts-*-SHW - Code for predicting reverts.
• sql - SQL for accessing the ToolsDB OIDB. Should be moved to audit_web_client.
• src - Python code for processing downloaded dumps.
• figures - Matplotlib or other figures generated by analysis code.

Timeline

An initial version of this project started in January 2020. A revised version of this project, and development of ORES-Inspect, started in January 2021. ORES-Inspect was presented at the Wiki Workshop in June 2024.

Citation

To cite this work, cite our Wiki Workshop 2024 paper:

Zachary Levonian, Lauren Hagen, Lu Li, Jada Lilleboe, Solvejg Wastvedt, Aaron Halfaker, Loren Terveen. 2023. ORES-Inspect: A technology probe for machine learning audits on enwiki. In Wiki Workshop 2024, Online. DOI:https://doi.org/10.48550/arXiv.2406.08453