GB/T 38676-2020 PDF in English
GB/T 38676-2020 (GB/T38676-2020, GBT 38676-2020, GBT38676-2020)
Standard ID | Contents [version] | USD | STEP2 | [PDF] delivered in | Name of Chinese Standard | Status |
GB/T 38676-2020 | English | 145 |
Add to Cart
|
0-9 seconds. Auto-delivery.
|
Information technology -- Big data -- Functional testing requirements for storage and processing systems
| Valid |
Standards related to (historical): GB/T 38676-2020
PDF Preview
GB/T 38676-2020: PDF in English (GBT 38676-2020) GB/T 38676-2020
GB
NATIONAL STANDARD OF THE
PEOPLE’S REPUBLIC OF CHINA
ICS 35.240
L 67
Information technology - Big data - Functional testing
requirements for storage and processing systems
ISSUED ON: APRIL 28, 2020
IMPLEMENTED ON: NOVEMBER 01, 2020
Issued by: State Administration for Market Regulation;
Standardization Administration of the PRC.
Table of Contents
Foreword ... 3
1 Scope ... 4
2 Normative references ... 4
3 Terms and definitions ... 4
4 Abbreviations ... 4
5 Overview ... 5
6 Functional testing requirements for big data storage subsystems ... 5
6.1 Testing requirements for basic functions ... 5
6.2 Functional testing requirements for distributed file storage ... 6
6.3 Functional testing requirements for distributed structured data storage ... 7
6.4 Functional testing requirements for distributed columnar data storage ... 8
6.5 Functional testing requirements for distributed graph data storage ... 8
7 Functional testing requirements for big data processing subsystems ... 9
7.1 Testing requirements for basic functions ... 9
7.2 Functional testing requirements for batch processing framework ... 10
7.3 Functional testing requirements for stream processing framework ... 11
7.4 Functional testing requirements for graph computing framework ... 12
7.5 Functional testing requirements for memory computing framework ... 12
7.6 Functional testing requirements for batch-stream fusion computing framework 13
Information technology - Big data - Functional testing
requirements for storage and processing systems
1 Scope
This Standard specifies the testing requirements for basic functions, distributed
file storage, distributed structured data storage, distributed columnar data
storage, distributed graph data storage, batch processing framework, stream
processing framework, graph computing framework, memory computing
framework, and batch-stream fusion computing framework of big data storage
and processing systems.
This Standard applies to the testing of big data storage and processing systems.
2 Normative references
The following documents are indispensable for the application of this document.
For the dated references, only the editions with the dates indicated are
applicable to this document. For the undated references, the latest edition
(including all the amendments) are applicable to this document.
GB/T 37722-2019 Information technology - Technical requirements for big
data storage and processing systems
3 Terms and definitions
The terms and definitions defined in GB/T 37722-2019 are applicable to this
document.
4 Abbreviations
The following abbreviations apply to this document.
API: Application Programming Interface
CPU: Central Processing Unit
DAG: Directed Acyclic Graph
GPU: Graphics Processing Unit
structured data, semi-structured data, and unstructured data that exist in
big data storage subsystems; test whether the operations are performed
normally;
i) It shall be tested whether big data storage subsystems can read data from
the message queue and write the calculation results to the database in real
time;
j) It shall be tested whether big data storage subsystems can collect data to
the real-time retrieval platform, and perform real-time query based on the
primary index key.
Note: The testing requirements in this clause correspond to the requirements in 6.1 of
GB/T 37722-2019.
6.2 Functional testing requirements for distributed file storage
The functional testing requirements for distributed file storage are as follows:
a) It shall be tested whether big data storage subsystems can perform file
upload, download, read-write, copy, move, deletion, access control and
other operations;
b) It shall be tested whether big data storage subsystems can back up
multiple copies of files and recover the original data through the copies;
c) It shall be tested that when the big data storage subsystem node/software
fails, such as power failure, data node failure, etc., it will not affect the
normal operation of the system and business;
d) It shall write to the copy file, and then check the block check file, to verify
that the block check file of the node where the copy file is located is the
same;
e) It shall be tested that before deleting a node, a single data node first exits
the service cluster; cluster data is redistributed; data is not damaged; and
business is not interrupted;
f) It shall be verified that after a node is added, the node joins the cluster; the
system data is redistributed; and the business is not interrupted;
g) It shall be tested that according to the configured compression and
encryption algorithms, the data is compressed, encrypted and decrypted,
the original data will be not damaged or lost;
h) It shall be tested whether big data storage subsystems can perform unified
retrieval, cataloging, adding and deleting operations on file data;
6.4 Functional testing requirements for distributed columnar data storage
The functional testing requirements for distributed columnar data storage are
as follows:
a) It shall create a table, write data, to test data can be stored in big data
storage subsystems in the form of key value.
b) It shall set user permissions for tables, column families, and columns
respectively, including read, write, and create, etc., to test whether the
corresponding users can create, read, and write tables, column families,
and columns.
c) It shall encrypt the specified column, create a table, and write data. Test
whether the attributes of the table are column encrypted and whether the
data is stored in non-plain text.
d) It shall be tested whether big data storage subsystems can back up and
restore database objects including tables, indexes, functions, triggers, etc.;
test the progress and history of data backup and restoration tasks.
e) It shall import data in batches, create secondary indexes when importing;
to test whether big data storage subsystems can query the imported data
through indexes.
f) It shall construct multiple tables and import data, to test whether the
contents of the tables merged by big data storage subsystems according
to the association rules/relationships are consistent with expectations.
Note: The testing requirements in this clause correspond to the requirements in 6.4 of
GB/T 37722-2019.
6.5 Functional testing requirements for distributed graph data storage
The functional testing requirements for distributed graph data storage are as
follows:
a) It shall define the graph data model, upload data files and graph rule
mapping files, to test whether the queried graph data is consistent with the
defined data model;
b) It shall write/import graph data to test whether it is possible to query,
traverse, and analyze graph data;
c) It shall be tested whether big data storage subsystems can, through the
graph database development interface, perform operations such as
metadata management, graph data management;
f) It shall test the dynamic resource scheduling strategy of task, check the
changes in the resource occupation during the task operation;
g) It shall create a first-level tenant, to test whether it is possible to create
multiple levels of sub-tenants under it, configure the upper limit of resource
usage for it; the sum of resources of each level of sub-tenants does not
exceed the parent tenant;
h) It shall create a first-level tenant A and tenant B, respectively allocate a
certain amount of resources, specify tenant A to submit tasks; the task
occupying resources far exceeds its configured resource capacity; to test
whether the task can be successfully submitted, and whether the
resources of tenant B can be reduced. Then specify tenant B to submit
tasks; the task occupies resources for its full allocation; to test whether the
task can be submitted successfully, and whether the resources of tenant
A can be reduced;
i) It shall submit the tasks of various distributed computing frameworks at the
same time, to test whether the tasks can be successfully executed;
j) It shall construct task A and task B, with the output of A being the input of
B, submit task A and task B, to test whether A and B can be automatically
scheduled according to the dependency;
k) It shall submit the task, to view the changes in its resource occupation;
after the task is completed, the task's resource occupation shall be
checked again; to test whether big data processing subsystems can
dynamically allocate computing resources according to job requirements,
and automatically manage and recycle resources;
l) It shall submit multiple tasks; these tasks are topologically sorted in a non-
loop directed graph; to test whether the tasks can be automatically
scheduled according to the topological results;
m) It shall submit a complex task, to test whether the amount of CPU and
memory scheduled by big data processing subsystems for different
subtasks is different.
Note: The testing requirements in this clause correspond to the requirements in 7.1 of
GB/T 37722-2019.
7.2 Functional testing requirements for batch processing framework
The functional testing requirements for batch processing framework are as
follows:
a) It shall create tables for the data by creating structured and unstructured
e) It shall create a message processing task, construct abnormalities such
as nodes and processes in the process of message processing, to test
whether the messages on abnormal nodes will be redistributed to other
normal nodes and processes for processing.
Note: The testing requirements in this clause correspond to the requirements in 7.3 of
GB/T 37722-2019.
7.4 Functional testing requirements for graph computing framework
The functional testing requirements for graph computing framework are as
follows:
a) It shall read the graph data through the API to find the attributes of a certain
edge/point, to test whether the results are consistent with expectations;
b) It shall be tested whether big data processing subsystems provide
tools/command lines/graphical interfaces for full import and incremental
import of data to define the import;
c) It shall be tested whether big data processing subsystems support
analysis and query of real-time graph data;
d) It shall define the graph data model, upload data files and graph rule
mapping files (tags and attributes on nodes/edges), to test whether the file
upload is successful;
e) It shall be tested whether big data processing subsystems support the
built-in common graph indicator calculation function, such as counting the
number of points and edges, etc.;
f) It shall use tools to pressurize, to test whether big data processing
subsystems can automatically allocate the added graph data calculation
and query services to different nodes;
g) It shall initiate the graph database query request by simulating multiple
clients, to test whether the corresponding data result can be queried.
Note: The testing requirements in this clause correspond to the requirements in 7.4 of
GB/T 37722-2019.
7.5 Functional testing requirements for memory computing framework
The functional testing requirements for memory computing framework are as
follows:
a) It shall in a distributed computing environment, test the function of
...... Source: Above contents are excerpted from the PDF -- translated/reviewed by: www.chinesestandard.net / Wayne Zheng et al.
|