The future of standardised assessment: Validity and trust in algorithms for assessment and scoring [Scholarly Article - European Journal of Education, January 2023]

Title: 

The future of standardised assessment: Validity and trust in algorithms for assessment and scoring

Author:

Cesare Aloisi

Published:

European Journal of Education, 17 January 2023

https://onlinelibrary.wiley.com/doi/full/10.1111/ejed.12542

Abstract:

This article considers the challenges of using artificial intelligence (AI) and machine learning (ML) to assist high-stakes standardised assessment. It focuses on the detrimental effect that even state-of-the-art AI and ML systems could have on the validity of national exams of secondary education, and how lower validity would negatively affect trust in the system. To reach this conclusion, three unresolved issues in AI (unreliability, low explainability and bias) are addressed, to show how each of them would compromise the interpretations and uses of exam results (i.e., exam validity). Furthermore, the article relates validity to trust, and specifically to the ABI+ model of trust. Evidence gathered as part of exam validation supports each of the four trust-enabling components of the ABI+ model (ability, benevolence, integrity and predictability). It is argued, therefore, that the three AI barriers to exam validity limit the extent to which an AI-assisted exam system could be trusted. The article suggests that addressing the issues of AI unreliability, low explainability and bias should be sufficient to put AI-assisted exams on par with traditional ones, but might not go as far as fully reassure the public. To achieve this, it is argued that changes to the quality assurance mechanisms of the exam system will be required. This may involve, for example, integrating principled AI frameworks in assessment policy and regulation.

 

University Researchers Investigate Machine Learning Compute Trends [InfoQ, 2022]

Title:

University Researchers Investigate Machine Learning Compute Trends

 

Author:

Anthony Alford, Director, Development at Genesys Cloud Services

 

Published:

InfoQ, 8 March 2022

https://www.infoq.com/news/2022/03/ml-compute-trends/

 

From the article:

A team of researchers from University of Aberdeen, MIT, and several other institutions have released a dataset of historical compute demands for machine learning (ML) models. The dataset contains the compute required for training 123 important models, and an analysis shows that since the year 2010 the trend has significantly increased.  

 

The analysis was presented in a paper published on arXiv.

Top 5 Data Science and Analytics Trends in 2020? Businesses are waking up to the wonders that they can achieve with Data Science, ML and Artificial Intelligence

Title:
Top 5 Data Science and Analytics Trends in 2020? Businesses are waking up to the wonders that they can achieve with Data Science, ML and Artificial Intelligence

Author:
Kamalika Some

Published:
Analytics Insight, 16 July 2020
https://www.analyticsinsight.net/top-5-data-science-analytics-trends-2020/

From the article:
Artificial intelligence (AI) and machine learning (ML) are two technologies that have witnessed a massive growth trend over the years. In a quest to look for a quick, cost-efficient and innovative way to gain advantages from data science, enterprises are relying more on the use of rapidly growing big data available at their disposal. Data and analytics combined with artificial intelligence (AI) technologies will be paramount to predict, prepare and respond in a proactive and accelerated manner to ensure business continuity during this global crisis and after-forward.