Search result: GB/T 45087-2024
| Standard ID | Contents [version] | USD | STEP2 | [PDF] delivered in | Standard Title (Description) | Status | PDF |
| GB/T 45087-2024 | English | 919 |
Add to Cart
|
7 days [Need to translate]
|
Artificial intelligence - Performance testing methods for server systems
| Valid |
GB/T 45087-2024
|
| Standard ID | GB/T 45087-2024 (GB/T45087-2024) | | Description (Translated English) | Artificial intelligence - Performance testing methods for server systems | | Sector / Industry | National Standard (Recommended) | | Classification of Chinese Standard | L61 | | Classification of International Standard | 35.160 | | Word Count Estimation | 46,493 | | Date of Issue | 2024-11-28 | | Date of Implementation | 2024-11-28 | | Issuing agency(ies) | State Administration for Market Regulation, National Standardization Administration |
GB/T 45087-2024.Artificial Intelligence Server System Performance Test Method
ICS 35.160
CCSL61
National Standard of the People's Republic of China
Artificial Intelligence Server System Performance Testing Method
Released on 2024-11-28
Implementation on 2024-11-28
State Administration for Market Regulation
The National Standardization Administration issued
Table of Contents
Preface III
Introduction IV
1 Range 1
2 Normative references 1
3 Terms and Definitions 1
4 Abbreviations 3
5 Test Mode 4
5.1 Closed Mode 4
5.2 Open Mode 4
6 Training performance test 4
6.1 Testing Process 4
6.2 Training and Testing Requirements 5
6.3 Training and Testing Results 6
6.4 Test Scenario 7
6.5 Test scenario configuration requirements 11
6.6 Indicators and test methods 12
6.7 Training test system requirements 16
7 Reasoning performance test 17
7.1 Testing Process 17
7.2 Reasoning Test Requirements 17
7.3 Reasoning Test Results 18
7.4 Test scenario 18
7.5 Scenario Configuration Requirements 24
7.6 Indicators and test methods 24
7.7 Inference test system requirements 29
Appendix A (Informative) Artificial Intelligence Server System Performance Test Tool Example 31
Appendix B (Normative) AUTOML Training and Testing Requirements 32
B.1 Training Requirements 32
B.2 Training result log requirements 32
Appendix C (Normative) Test Code Disclosure Rules 33
C.1 General 33
C.2 Training and testing code disclosure rules 33
C.3 Reasoning test code disclosure rules 33
Appendix D (Informative) Test scenario type description 35
D.1 Image Recognition 35
D.2 Object Detection 35
D.3 Semantic Segmentation 35
D.4 Recommendation 35
D.5 Natural Language Processing 35
D.6 Speech Recognition 35
D.7 Optical Character Recognition 36
D.8 Face Recognition 36
D.9 Multimodality 36
Appendix E (Informative) Energy efficiency and efficiency indicators and test methods 37
E.1 Training 37
E.2 Reasoning 38
Reference 40
Preface
This document is in accordance with the provisions of GB/T 1.1-2020 "Guidelines for standardization work Part 1.Structure and drafting rules for standardization documents"
Drafting.
This document was proposed and coordinated by the National Technical Committee for Information Technology Standardization (SAC/TC28).
This document was drafted by. China Electronics Technology Standardization Institute, Huawei Technologies Co., Ltd., Inspur Electronic Information Industry Co., Ltd.
Intel (China) Co., Ltd., Pingtou Ge (Shanghai) Semiconductor Technology Co., Ltd., iFlytek Co., Ltd., H3C Information Technology Co., Ltd.
Technology Co., Ltd., Advanced Micro Devices (China) Co., Ltd., Beijing University of Aeronautics and Astronautics, Cambrian Technologies Co., Ltd., Nanjing
NARI Ruiteng Technology Co., Ltd., China Southern Power Grid Co., Ltd. Ultra-high Voltage Transmission Company, Sinopec Yingke Information Technology Co., Ltd.
Company, Guangdong Research Institute of China Telecom Co., Ltd., Shanghai Enflame Technology Co., Ltd., Institute of Software, Chinese Academy of Sciences, Beijing
Ren Technology Development Co., Ltd., Shanghai Qianshi Technology Co., Ltd., Shanghai Supercomputing Center, Shanghai Wenyu Information Technology Co., Ltd., Midea Group
Tuan (Shanghai) Co., Ltd., Guoke Chushi (Chongqing) Software Co., Ltd., Shanghai Artificial Intelligence Research Institute Co., Ltd., Sichuan Huakun Zhenyu Intelligent
Technology Co., Ltd., Shenzhen Kunyun Information Technology Co., Ltd., China Railway Construction Corporation Limited, China Railway Fifth Survey and Design Institute Group Co., Ltd.
Co., Ltd., Southwest University of Science and Technology.
The main drafters of this document are. Dong Jian, Xu Yang, Zhang Qi, Wang Waner, Cao Xiaoqi, Huang Jianbin, Liang Zhaoming, Bao Wei, Wu Shaohua, Wang Haining,
Lin Xiaodong, Ma Shanshan, Gao Hui, Zhang Yibo, Tao Yumei, Yang Yuze, Zheng Huiping, Liu Rubing, Li Lanbo, Ji Tuo, Luan Zhongzhi, Cheng Guipeng, Huang Xiancui,
Jun Mu, Chao Shi, Heng Ye, Ning Wang, Dongqing Liu, Xianxu Li, Chunyu Shi, Jingqing Mei, Lingzhong Meng, Ruiquan Ding, Qiulin Cheng, Geng Wu, Huazhen Yu, Dandan Zhang,
Zhong Kaitao, Ren Pei, Fu Xinjie, Hu Yanling, Song Haitao, Bai Shiyu, Liu Dong, Luan Lihong, Li Dong, Zheng Zhong, Yu Wenxin.
introduction
The AI server system includes AI servers, clusters, and high-performance computing facilities, and is a platform for various deep learning models.
(including large-scale pre-trained models) is the core carrier of training and reasoning, and is the core tool for various industries to use artificial intelligence technology to improve production efficiency.
The AI server system is designed to handle AI computing tasks and is very similar to general-purpose servers in terms of architecture, computing methods, and usage.
The server systems are quite different, and their test processes, loads, and indicators are all unique.
The benchmarking method is proposed, and requirements are put forward for the functionality and fairness of the benchmarking tools.
The issuing organization of this document calls attention to the fact that when declaring compliance with this document, it may involve 7.4.2, 7.7.1 and the systemic nature of artificial intelligence servers.
The use of patents related to the test method.
The issuing organization of this document takes no position on the authenticity, validity and scope of this patent.
The patent holder has promised to the issuing agency of this document that he is willing to cooperate with any applicant under reasonable and non-discriminatory terms and conditions.
The patent holder's statement has been filed with the issuing agency of this document, and relevant information can be obtained through
Get the contact information below.
Patent holder. China Electronics Technology Standardization Institute
Address. No. 1, Andingmen East Street, Dongcheng District, Beijing
Please note that in addition to the above patents, some of the contents of this document may still involve patents. The issuing agency of this document does not assume the responsibility for identifying patents.
responsibility.
Artificial Intelligence Server System Performance Testing Method
1 Scope
This document defines the server system performance test mode and describes the training performance and reasoning performance test of the artificial intelligence server system.
method.
This document is applicable to the performance testing and evaluation of artificial intelligence server systems.
2 Normative references
The contents of the following documents constitute the essential clauses of this document through normative references in this document.
For referenced documents without a date, only the version corresponding to that date applies to this document; for referenced documents without a date, the latest version (including all amendments) applies to
This document.
GB/T 41867-2022 Information Technology Artificial Intelligence Terminology
3 Terms and definitions
The terms and definitions defined in GB/T 41867-2022 and the following apply to this document.
3.1
System under test
A system that processes the test jobs given by the tester and returns results that meet the requirements.
Note. The system under test consists of artificial intelligence server system hardware, operator implementation library, machine learning framework software, model compilation components and other necessary hardware and software.
3.2
tested party
An organization or individual that provides the system under test and test information and assists in the implementation of the test.
3.3
reference modelreferencemodel
A standardized model for defining system test requirements.
[Source. ISO /IEC 14776-414.2009, 3.1.87, modified]
3.4
Timing
Get and return the current timestamp of the system under test.
Note. It is assumed that the time of each node in the system under test (3.1) is consistent.
3.5
A server in an information system that can provide high-performance computing and processing capabilities for artificial intelligence applications.
Note 1.AI servers contain computing modules designed specifically for AI computing, providing dedicated accelerated computing capabilities for AI applications.
Note 2.Based on general-purpose servers, servers equipped with AI acceleration cards that provide dedicated computing acceleration capabilities for AI applications are called “AI” servers.
Compatible with the server".
Note 3.A server designed specifically for AI accelerated computing and providing dedicated AI computing capabilities is called an “AI all-in-one server”.
[Source. GB/T 41867-2022, 3.1.3, modified]
......
|