Evaluatology: The Science and Engineering of Evaluation

评估是人类存在的重要方面，在各个领域中都扮演着至关重要的角色。然而，评估往往以经验和临时的方式进行，缺乏普遍概念、术语、理论和方法论的共识。这种缺乏协议的情况有着重大的影响。本文旨在正式介绍评估学这一学科，该学科涵盖了评估的科学和工程。我们提出了一个通用的评估框架，包括可以应用于各种学科的概念、术语、理论和方法论。 我们的研究表明，评估的本质在于进行实验，有意地将明确定义的评估条件应用于不同的对象，并通过测量和/或测试来推断不同对象的影响。基于评估的本质，我们提出了五个公理，聚焦于评估结果的关键方面，作为基础评估理论。这些公理是我们建立通用评估理论和方法论的基石。在评估单个对象时，创建具有不同等价级别的评估条件非常重要。通过将这些条件应用于不同的对象，我们可以建立参考评估模型。这些模型允许我们一次改变一个独立变量，同时保持所有其他变量作为控制。在评估复杂情况时，关键在于建立一系列保持传递性的评估模型。在评估科学的基础上，我们提出了基准的正式定义，作为一种简化和抽样的评估条件，保证不同等价级别。这个概念是跨学科通用基准工程方法的基石，我们称之为基准学。

Introducing the discipline of evaluatology to propose a universal framework for evaluation across various disciplines.

The essence of evaluation lies in conducting experiments that intentionally apply a well-defined evaluation condition to diverse subjects and infer the impact of different subjects by measuring and/or testing. Five axioms serve as the foundational evaluation theory. Establishing reference evaluation models and a series of evaluation models that maintain transitivity are crucial in evaluating a single subject and complex scenarios, respectively. A benchmark-based engineering approach to evaluation across various disciplines, referred to as benchmarkology, is proposed.

The paper proposes a formal discipline of evaluatology and a universal framework for evaluation. The five axioms serve as the foundational evaluation theory. The concept of a benchmark as a simplified and sampled evaluation condition that guarantees different levels of equivalency is introduced. Benchmarkology is proposed as a benchmark-based engineering approach to evaluation across various disciplines. The paper provides insights into the importance of establishing reference evaluation models and maintaining transitivity in evaluation. No experiments, datasets, or open-source code are mentioned in the paper.

No related works are mentioned in the paper.