Research and application of artificial intelligence service platform for the power field

1 Introduction

Artificial Intelligence (AI) is currently one of the most subversive technologies, with strong processing power in computational intelligence, perceptual intelligence, and cognitive intelligence.As new technologies are most likely leading toward the fourth industrial revolution, AI is the core support technology of intelligent energy, with strong optimization and learning abilities that deal with highdimensional, time-changing, and nonlinear problems, which can effectively solve the challenges faced by energy systems.

With the continuous deepening and development of integrated business applications in the energy field, the conventional analytical methods are unable to fully meet the needs of business development, e.g., grid complex network, equipment wide points, and different operating characteristics.In addition, the conventional operation and maintenance methods face difficulty in accurately evaluating the state of the equipment and targeted investment, renovation, and operation.Large-grid wide-area interconnection forms a complex network, the fault identification, optimization control, and other problems of which are difficult to effectively solve by using existing methods.

In recent years, power systems have attracted considerable interest in the direction of AI research and development, mainly with respect to business systems focusing on the promotion of a certain AI technology.In terms of load prediction, some studied focused on the deep learning algorithm to realize grid-load prediction [2-4], and put forward the nonintrusive load-monitoring model based on the sequence-to-sequence method and attention mechanism to realize the nonintrusive load decomposition application [5].In terms of fault prediction and processing, Wang [6] and He et al.[7] applied the deep learning algorithm to transformer fault diagnosis.Xu et al.[8] proposed a method to self-learn the fault data and realize fault-type identification of transmission line based on the deep learning framework.Shan et al.[9] proposed a technique of scheduling a fault-management robot based on deep learning.From the aspect of cruise image processing, some studies had adopted the deep learning algorithm to realize the classification of transmission line inspection of aerial photography data [10-13].Moreover, Cheng et al.[14] used the convolution neural network algorithm to train the sample library rendered by the 3D MAX software to realize the classification of images with an insulator.In terms of transient stability assessment, Hu et al.[15] and Zhu et al.[16] proposed transient stability assessment methods for power systems based on a deep confidence network.By using the deep learning framework, a method of transient stabilization of a power system based on stacked auto-encoders (SAE) was proposed in [17].In [18], based on the short-term disturbed track after fault removal, a convolution neural network was used in a deep learning model to extract features and construct the nonlinear mapping relationship between the short-term disturbed track and stable category.In terms of safety attack and defense, Wu et al.[19] applied AI technology to maintain the steady-state safety of power grid systems.Wang et al.[20] proposed a defense mechanism based on interval state estimation to detect cyberattacks on smart grids.Karimipour et al.[21] proposed an unsupervised anomaly detection method based on statistical correlation between measurements to distinguish between actual failures and interferences and intelligent network attacks.In the last two years, the power sector has also built frameworks or platforms based on AI algorithms, e.g., Tang et al.[22] built a three-dimensional framework of a large-scale power grid for digital simulation, and provided the knowledge model of power-grid simulation analysis and its application methods, i.e., the power-grid simulation knowledge-mining method and the AI model with the ability of diversified power-grid transfer learning.Sogabe et al.[23] embedded two deep-enhanced learning algorithms for discrete and continuous motion space design into a physical model, thus forming an algorithm library.Ferrag and Maglaras [24] devised a smart-grid energy framework based on deep learning and blockchain technology.Chi et al.[25] presented a heterogeneous multicore SoC hardware architecture containing a distributed processing unit (DPU).At present, the application mode of the fragmented effect of AI leads to poor technology portability, duplication of construction leads to waste in investment, and the lack of unified evaluation makes it difficult to guarantee the application effect.In addition, a large number of related research and development projects are being conducted that may be contradictory, resulting in data labeling, lack of standardization and generality, difference in interface standards, difficulty in measuring deep-learning training results, and other drawbacks.In view of these problems, it is necessary to build a standardized integrated and innovative application platform based on the common problems of intelligent power systems.

The AI platform for the power field from the viewpoint of functional positioning provides a model training environment and a production service environment; supports natural language processing and computer vision services; provides algorithms, models, and services for power business applications; and creates an ecological chain of “platform applications and services.”

This paper provides an analysis of the deep learning framework, architecture design, key technology and application scenarios, and lays the foundation for the development and application of AI platform for the power field.

2 Deep Learning

Deep learning is a new branch of machine learning that has made breakthroughs in speech recognition, target detection, text processing, and more recently in AI [26].Deep learning can be understood as a multi-hidden neural network, which realizes the approximation of any complex function by learning a deep nonlinear network structure and characterizing the distributed representation of data [27].Deep learning focuses on the deep structure and characteristics of an automatic learning network, usually with 5, 10, or even hundreds of layers.A deeper hierarchy gives the network a stronger ability to automatically extract data features from a large number of samples.

Suppose we have system s, with n layers (S1, S2,...Sn), with input I and output O, it is represented as

2.1 Deep Belief Network

Let us suppose a bipartite graph with no link between the nodes in each layer.In this graph, one layer is visible, i.e., the input data layer (V), and the other is hidden (H).If all nodes are random binary variable nodes (only 0 or 1 can be taken as values), and assuming that the total probability distribution P (V, H) satisfies the Boltzmann distribution, we call this system as the restricted Boltzmann machine (RBM).

The deep belief network (DBN) is the first proposed deep-learning-network model consisting of several restricted RBMs.

2.2 SAE

The structure of an SAE is similar to that of a DBN and is stacked with multiple autoencoders (AEs).A basic AE can be thought of as a three-layer neural network with the same number of neurons as in the input layer.

An AE consists of an encoder and a decoder.Let us assume that the input of AE is

where dx is the dimension of the input.The encoder projects x from input layer to the hidden layer by mapping function f:

where dh is the dimension of the variable vector in the hidden layer, and function f(x) is expressed as

2.3 Convolutional Neural Networks

A convolutional neural network (CNN) is different from SAE and DBN in that it is a supervised, deep-learningnetwork model comprising an input layer, a convolution layer, a subsampling layer, a fully connected layer, and an output layer.The convolution and pooled layers are usually taken from several and are set alternatively.

A convolution is expressed as

where asterisk represents convolution.A two-dimensional convolution is expressed as follows:

In a CNN, a two-dimensional convolution is defined as

The mapping of forward propagation is formulated as

and a convolution operation is given as

2.4 Recurrent Neural Networks

Recurrent neural networks (RNN) are a type of neural network used to process sequential data; this is reflected in the calculation of the network’s memory of the previous information and used in the calculation of the current output.The network is designed to have connections between neuron nodes of hidden layers, and the input of a hidden layer includes the input from the input layer and the output obtained from a previous task.

Output layer O and hidden layer S of an RNN are calculated as

2.5 Deep Reinforcement Learning

Deep reinforcement learning (DRL) is a combination of deep learning and enhanced learning.An end-to-end learning algorithm for perceived action, originally published as DeepMind, RL is a trial and error learning algorithm, which is more in line with human learning habits.

The Bellman equation is the core formula of RL:

Further, the Bellman optimality is equated as

3 Architecture Design

The AI platform for the power field adopts a flexible hierarchical structure, and each layer is connected through a standard interface.The platform architecture design includes application architecture, technical architecture, data architecture, and deployment architecture.

3.1 Application Architecture

The application architecture of the AI platform for a power field is divided into five layers from bottom to top, as shown in Fig.1: data lake storage, heterogeneous resource scheduling service, model development training warehouse, model reasoning, and platform management.

(1) Data lake storage integrates SQL, Non-SQL, and MPP databases to store multisource heterogeneous data, including power system scheduling data, customer service data, detection image, energy data, and GIS data.

Fig.1 Application architecture of electric AI platform

(2) Heterogeneous resource scheduling services include resource pool management, job management, task management, and scheduling services.

(3) Model development training includes training dataset warehouse, distributed training, visual tagging, and workflow.

(4) Model warehouse includes a CNN algorithm, deep neural network (DNN) algorithm, and model evaluation.

(5) Model reasoning includes an MXNET computing engine, a TensorFlow computing engine, and a Caffe computing engine.

(6) The platform management includes deployment operations, data management, and mirror management.

3.2 Technical Architecture

The technical architecture of an AI platform for the power industries consists of five layers, i.e., data-acquisition, data-storage, data-processing, resource-scheduling, and service-performance layers, and two engines, namely, the deep-learning-algorithm and machine-learning-algorithm engines, as shown in Fig.2.

(1) Data acquisition layer.The platform adopts multisource heterogeneous data, and data is collected using Flume, Kafka, Kettle, Sqoop, and more such tools.

(2) Data storage layer.To save the heterogeneous data collected, the platform uses various databases, e.g., the file storage uses NFS, the NoSQL database uses HBase/Neo4j, the relational database uses MySQL, the MPP database uses GBase, and the memory database uses Redis.

(3) Data processing layer.This layer processes data by using MR, Spark, Impala, data labeling, and other technologies.

(4) Resource scheduling layer.This layer uses a resource orchestration engine, a resource scheduling engine, Docker, Kubernetes, CUDA-Driver, and other technologies.

(5) Deep learning algorithm engine.This used the TensorFlow, Caffe, MXNet, PyTorch, etc.

(6) Machine learning algorithm engine.This engine uses Sklearn, Mllib, Python, R, and other technologies.

(7) Service performance layer.The layer uses intact and Vue, Spring boot, Istio, MyBatis, Traefik, etc.

3.3 Data Architecture

The sensor collects multisource data and places it in the data-lake unified storage pool for management.Through extraction, cleaning, transformation, and mining, the source data finally form various BI reports and unified knowledge map.Some data are labeled as data needed for model training, output models, and various AI services.The data architecture of an AI platform for the power field is shown in Fig.3.

(1) Data from the storage layer are obtained through data acquisition, data integration, and other processes and placed into the data lake for unified storage.

(2) The data-labeling service in the compute layer calls the unified storage data in the data lake.GPU clusters are used for fixed and bulk labeling, and the results of data labels are stored into a data lake.

(3) Deep-learning services in the compute layer invoke data lakes to store data uniformly.GPU clusters are used to provide training and external services, and deep-learning results are stored in data lakes.

3.4 Deployment Architecture

The deployment architecture of the AI platform for power fields is divided into three layers, as shown in Fig.4: internal and external network isolation, headquarters and directly affiliated companies, provincial and municipal companies.

Fig.2 Technical architecture of the electric AI platform

Fig.3 Data architecture of the electric AI platform

Fig.4 Deployment architecture of the electric AI platform

The deployment architecture has the following capabilities.

(1) Basic support ability of AI in extranet involves deployment of information; providing voice recognition, face recognition, and other services; and synchronizing with the basic information of headquarters.

(2) Headquarters AI Foundation Support Capabilities are deployed in the information intranet through a unified operation and controlled from the headquarters of the National Network, complete with information extranet and direct units, including information interaction of provincial and municipal companies.

(3) The basic support capacity of AI in terms of direct units involves uploading a mature model to the headquarters and sharing its usage.

(4) The basic support capacity of AI for provincial and municipal companies involves the synchronization of the model shared with the headquarters, uploading of mature models to the headquarters, and support of personalized model building.

4 Key Technologies

4.1 Algorithm Model of Multihardware Compression and Conversion Technology

The algorithm model of the multihardware compression and conversion technology include the compression and acceleration of the original deep network model, and direct application of the million-level DNN model to the embedded device about mobile.The disadvantages of high storage and high power consumption promote the application of DNNs in applications with limited resources and real-time online processing.

(1) Parameter pruning

Parameter pruning removes redundant, less informative weights from existing trained deep network models, thereby reducing the parameters of the network model.Parameter pruning can speed up the calculation of the model, compress its storage space, and prevent it from overfitting.Parameter pruning can be divided into unstructured and structured pruning according to whether all the nodes must be deleted or filtered one at a time.Unstructured pruning considers each element for filtering and deletes the parameters, the number of elements of which is 0 in the filter.In contrast, structured pruning removes the entire filter of the convolutional layer without introducing additional datatype storage, thereby directly compressing the network and speeding up the calculation of the entire network.

(2) Parameter sharing

In parameter sharing, a mapping type is designed to allow sharing of the same data among multiple parameters; the methods of parameter sharing differ in several aspects.At present, the more widely used parametersharing expressions include parameter quantization, hash function, and structured linear mapping.Among them, the core concept of parameter quantization compression and accelerated deep network model is to replace the original 32-bit floating-point parameters with lower bits.Hash functions and structured linear mappings are used to achieve parameter sharing with the fully connected layer, greatly reducing the memory overhead of the model.

(3) Low-rank decomposition

The core concept of low-rank decomposition is to estimate and decompose the original convolution kernel in the depth model by using matrix or tensor decomposition technology.Convolution calculation shows the highest computational complexity in the entire CNN.The internal redundancy of the model is effectively reduced by decomposing the 4D convolution kernel tensor.In addition, the matrix parameters of the 2D fully connected layer can be processed using lowrank decomposition technology.However, owing to the different decomposition methods of the convolutional and fully connected layers, this project reviews and analyzes the application of low-rank-decomposition technology in DNNs according to these two layers.

(4) Design of compact convolutional filters

The design of compact convolution kernels mainly reduces the storage and computational complexity of models with the designing of special structured convolution kernels or compact convolution calculation units.Based on this concept, the original large filter size is directly decomposed into two 1 × 1 convolution filters, which greatly accelerate the network calculation and achieve high target-recognition performance.

(5) Knowledge distillation

Knowledge distillation mainly utilizes the knowledge of large networks and transfers this knowledge to the compact distillation model.The idea is to use the knowledge learned by a large model neural network (teacher network) a priori, and train a smaller neural network with better performance by transferring prior knowledge to a smallscale neural network.Therefore, the scale of the small network parameters after distillation would be much smaller than that of the original large network, and the purpose of compressing the network is be achieved.There is an intuitive way of transferring knowledge of a Big model (teacher network) by using the class probability vectors generated by the teacher network as soft targets for training the small model (student network).

4.2 Hybrid Heterogeneous Resource Distributed Scheduling Technology

The emergence of heterogeneous computing has solved the thermal and energy bottlenecks encountered in conventional methods to increase computing power by increasing the CPU clock frequency and number of cores.From an implementation perspective, heterogeneous computing formulates a series of software and hardware standards to allow different types of computing devices to share computing processes and results.At the same time, the calculation process is continuously optimized and accelerated to increase the computing efficiency.

(1) Distributed elastic scheduling technology based on container-based heterogeneous multisource algorithm

Container technology is a virtualization technology that has emerged in the last few years.It performs dynamic load balancing and elastic scaling based on Docker containers.When the application system faces high-concurrency and large-volume burst requests, the system monitors the CPU by using the application container.The usage of resources rapidly increases memory.The corresponding strategy involves setting up of the alarm threshold to trigger during the elastic scaling mechanism when the capacity is increased in steps; after the peak value is reached, the capacity is reduced by a fixed step size.When the application container is started or deleted, the corresponding key value of the elastic scaling module changes in Etcd.After Confd detects this change, it dynamically generates a Haproxy configuration file and reloads it.Next, it offloads front-end user requests to the back-end application container by providing specific services and implementing load balancing of the application system.

(2) Distributed-task-scheduling algorithm and system

Distributed task scheduling involves a framework that coordinates multiple nodes processing the same task to avoid data being repeatedly processed.The nodes of the cluster share the processing of large batches of tasks to improve the processing efficiency of batch tasks.The taskscheduling system completes task scheduling by quickly and efficiently distributing tasks, which will be executed with appropriate computing resources.In this process, the task-scheduling system ensures resource node load balancing, high-quality task-scheduling QoS, real-time updating of the task and resource statuses, monitoring of these statuses, task-scheduling error handling, and so on.The task-scheduling system plays an irreplaceable role in the entire task-scheduling process.

4.3 Automated Model Training and Evaluation Technology

Automated training technology (AutoML) is an endto-end process of machine learning, and is proposed to automate all human tasks and reduce the threshold of machine learning.In typical machine-learning tasks, AutoML automatically learns by using the steps of modeling, optimization, and evaluation so that machine learning models can be applied without human intervention.The main problems of AutoML include the selection of the correct features, model family, and model parameters.

(1) Feature engineering

In machine-learning tasks, the quality of features largely determines the pros and cons of model performance.The automation problem of feature engineering involves the automatic construction of features based on data so that subsequent learning models show good performance.To achieve this goal, we primarily focused on the method of feature enhancement, that is, we performed some postprocessing on these features through methods, such as dimensionality reduction, feature generation, and feature coding, to improve learning performance.

(2) Model selection

The model selection consists of two functions, namely selection of classifiers and setting up of the corresponding hyper parameters.In this AutoML setup, the task automatically selects the classifier and sets its hyper parameters to achieve a good learning performance.This requires a deep network architecture suitable for learning problems.

(3) Optimization algorithm selection

The optimization algorithm selection if focused on achieving good efficiency.However, as learning tools become more complex, optimization is not only a major consumer of computing budgets but can also affect learning performance.Therefore, the goal of algorithm selection is to automatically find an optimization algorithm to achieve a balance between efficiency and performance.

(4) Evaluation technology

After generating candidate configurations, the evaluator must measure their respective performances.In this case, the evaluator does not need to consider the configured search space.As such, the easiest way is to learn the model parameters and evaluate the performance.

4.4 Software and Hardware Acceleration Technology for the Algorithm Model

When the designer tries to achieve the best performance from the algorithm but the software method is no longer available, the designer could try to accelerate it through hardware/software repartition.Hardware acceleration refers to the usage of hardware modules instead of software algorithms to take full advantage of the fast characteristics inherent in hardware.From a software perspective, interfacing with a hardware acceleration module is the same as calling a function, the only difference being that this function exists in the hardware and is transparent to the calling function.Depending on the algorithm, the execution time can be up to 100 times faster.The use of hardware provides a much faster performance of various operations, such as calculation of complex mathematical functions, moving data from one place to another, and performing the same manipulation multiple times.

5 Application Scenarios

5.1 Scheduling Voice Visualization Scenarios

In the field of scheduling, a scheduling voice visualization scenario was constructed to provide services such as filling in of scheduling logs, managing scheduling services, and tracing fault problems.Scheduling voice visualization scenarios can free employees from heavy basic work, help improve work efficiency, and enhance safety and quality inspection of key tasks.

(1) Filling in the scheduling log

This involves automatically translating the dispatcher’s calling content into text in real time, and copy pasting the relevant content into the dispatch log system after the call, eliminating the need for recording while the call is in progress.

(2) Scheduling business management

Here, the process of random inspection of recordings is converted into text records for inspection, and cases of noncompliance are confirmed through these recordings.In addition, we provide screening according to the department and seat to improve the pertinence of inspection.

(3) Backtracking of faults

Here, the dispatch recordings are converted into text; searched through keywords; and filtered according to time, unit, department, and seat.Finally, the corresponding recording is quickly located for investigation.

5.2 Drone-based Line Inspection

The fields of operation and inspection involve the inspection of transmission lines using drones and the provision of services, such as visually assisted autonomous line inspection, intelligent management of line inspection images, and intelligent recommendation of line inspection schemes.These help improve the reliability and efficiency of UAV-based line inspection.

(1) Visually assisted autonomous line inspection

Here, visual sensors are used to realize obstacle avoidance and autonomous patrol inspection based on wire tracking between poles.In addition, according to the target-recognition results of the airborne edge computing platform, the imaging parameters and drone attitude are automatically adjusted.

(2) Intelligent management of line-inspection images

The algorithm automatically eliminates duplicate and low-quality images, matches the line-of-sight images of the same scene at different times, and builds a multidimensional structured line-of-sight image library based on information, such as space time and tasks.

(3) Intelligent recommendation of line-inspection plan

According to historical line-inspection data and manual line-inspection experience, segmented quantitative evaluation of the line reliability level and automatic line inspection schemes are recommended.In addition, reliable line-inspection coverage is achieved at a smaller cost.Moreover, for emergency inspections in special situations, optimal inspection plan is recommended in a timely manner.

5.3 Metering and energy-analysis scenarios

The proposed framework sets up measurement and energy-analysis scenarios in the marketing field and provide services, such as electrical-energy-measurement equipment evaluation, energy analysis, and user portraits; this will help ensure the quality of the measurement equipment, promote the safe and reliable operation of user loads, and improve the user’s power consumption experience.

(1) Evaluation of electric-energy-measurement equipment

To deeply explore the components of electricalenergy-measurement equipment and the impact of these components on the quality of the equipment, a component-comparison analysis robot was designed and developed.In the full performance detection and spot check of the electricity metering equipment, the component comparison and analysis can be used to compare the AI of the components, such as smart energy meters, power consumption information collection terminals, and measurement boxes, and avoid abnormal components.It can automatically identify the structural design and analyze the advantages and disadvantages of the comprehensive indexes to provide a reference for the design of electrical-energymeasurement equipment.

(2) Energy analysis

The AI algorithm-based user load identification and curve decomposition technology was adopted to construct a spatiotemporal database of typical pollution/emission reduction equipment waveforms for the entire industrial chain load, to achieve full information monitoring of typical industrial load equipment.Through the development of ubiquitous Internet-of-Things technology-based load intelligent identification micro-application functional modules for environmental management companies and selection of pilot provinces, in conjunction with the provincial environmental protection department in industrial parks, high-energy-consuming and highly polluting industrial enterprises could be selected for trial.

(3) User portrait

Based on massive data of electricity-consumption behavior, the model can identify the behavior characteristics of different customer groups, construct a predictive analysis model, analyze customer’s energy-consumption characteristics, effectively predict customer electricity consumption risks, comprehensively analyze customer payment habits and preferences, and identify customer complaints hotspots.As such, accurate portraits of electricity customers could be identified to provide targeted value-added services to customers, optimize the electricity business environment, and improve customers' proactive service capabilities.

6 Conclusions

To break the data barrier, improve the portability of technology, and avoid investment waste caused by repeated construction, this paper proposes an AI platform framework for the field of power.The AI platform integrates three typical scenario applications, namely, scheduling voice visualization, unmanned patrol, and energy analysis.

The platform can meet the needs of current power system model training, natural language processing, and computer vision services.However, the platform faces technical difficulty in its integration with existing systems, such as EMS management, DTS, SCADA, and WAMS.As such, for the construction of this platform, we must determine how to unify the data interface, ensure integrity and confidentiality in the process of multisource heterogeneous data transmission, and manage the data stored in the platform.

At present, there are still some deficiencies in the platform, and these must be further strengthened in the follow-up research and development.

(1) The platform’s data labeling function requires considerable amount of manual labeling.Therefore, the next step in this study is to correct the data labeling algorithm to reduce the workload to a certain extent.

(2) The spdin between platform models must be increased.Current models implemented by different deep learning engines cannot be run over other platforms, e.g., the Caffe model cannot be directly replaced with the TensorFlow model; therefore, the expansion of the model must be restricted to a certain extent.

(3) The model evaluation capabilities must be increased; at present, there is no effective evaluation mechanism after model training.

Acknowledgements

This work was supported by National Key Research and Development Project(2017YFE0112600) and Science and Technology Project of China Electric Power Research Institute(Research on the Key Technologies and Typical Application Scenarios of the Artificial Intelligence Basic Framework for Integrated Energy).