English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Tools annual

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Tools monthly

Global: Zendy Plus monthly

Presents the access of premium content as premium feature

Premium Content

Global: Zendy Plus annual

Global: Zendy Free Plan

Deep learning models have proven to be highly successful. Yet, theirover-parameterization gives rise to model multiplicity, a phenomenon in whichmultiple models achieve similar performance but exhibit distinct underlyingbehaviours. This multiplicity presents a significant challenge and necessitatesadditional specifications in model selection to prevent unexpected failuresduring deployment. While prior studies have examined these concerns, they focuson individual metrics in isolation, making it difficult to obtain acomprehensive view of multiplicity in trustworthy machine learning. Our workstands out by offering a one-stop empirical benchmark of multiplicity acrossvarious dimensions of model design and its impact on a diverse set oftrustworthy metrics. In this work, we establish a consistent language forstudying model multiplicity by translating several trustworthy metrics intoaccuracy under appropriate interventions. We also develop a framework, which wecall multiplicity sheets, to benchmark multiplicity in various scenarios. Wedemonstrate the advantages of our setup through a case study in imageclassification and provide actionable insights into the impact and trends ofdifferent hyperparameters on model multiplicity. Finally, we show thatmultiplicity persists in deep learning models even after enforcing additionalspecifications during model selection, highlighting the severity ofover-parameterization. The concerns of under-specification thus remain, and weseek to promote a more comprehensive discussion of multiplicity in trustworthymachine learning.

An Empirical Investigation into Benchmarking Model Multiplicity for Trustworthy Machine Learning: A Case Study on Image Classification