Research on big data applications in Global Energy Interconnection

Dongxia Zhang1, Robert Caiming Qiu2, 3

1. China Electric Power Research Institute, Beijing 100192, P.R. China

2. Shanghai Jiao Tong University, Shanghai 200240, P.R. China

3. Tennesse Technological University, Cookeville, Tennesse 38501, USA

Abstract:Construction of Global Energy Interconnection (GEI) is regarded as an effective way to utilize clean energy and it has been a hot research topic in recent years. As one of the enabling technologies for GEI, big data is accompanied with the sharing, fusion and comprehensive application of energy related data all over the world. The paper analyzes the technology innovation direction of GEI and the advantages of big data technologies in supporting GEI development, and then gives some typical application scenarios to illustrate the application value of big data. Finally, the architecture for applying random matrix theory in GEI is presented.

Keywords:Global Energy Interconnection, Big data, Clean energy, Random matrix theory.

1 Introduction

As a systematic method to solve energy shortage,environmental pollution and climate change, Global Energy Interconnection (GEI) has attracted public attentions. GEI can be briefly defined as “ultra-high voltage power grid +ubiquitous smart grid + clean energy” and it aims to serve as a secure, reliable, efficient, and interactive platform for optimized energy development and utilization. GEI can transfer renewables from “one pole and one belt”(arctic, equatorial) energy bases to load centers by intercontinentally connecting power networks in different time zones and seasons by ultra-high voltage transmission systems. It can also take advantages of large scale energy systems interconnection, including less load peak valley gap, less conservation, and higher reliability [1].

Big data will play an important role in enabling GEI.On one hand, the GEI involves hundreds of millions of measurements, monitoring and controlling devices/systems,which generate large amounts of data during energy production, transmission, transaction, and consumption.On the other hand, GEI also shows a strong demand for data mining. Due to high penetration of renewables and wide involvement of customers, GEI faces challenges of uncertainty and complexity, therefore, physical modelbased methods cannot accommodate it. However, as a data-driven method, big data analytics is often effective.Through big data analysis, all stages of energy production,distribution, transformation and consumption can be predicted scientifically; decentralized and centralized coordination of energy management can be achieved;potential risks in every stage can be detected [2-6].

Random matrix theory (RTM) is regarded as a universal data analysis method in paper [7], which is proposed to apply in the analysis of power big data for the first time,and its validation is proved in [8].

This paper presents the direction of technological innovation of GEI and its demand for big data, and identifies the main application scenarios for big data analytics. It also analyzes its application value, and finally summarizes the analytical framework of RMT applied in GEI.

2 Technological innovation needs in development and utilization of global clean energy

In order to promote the development and utilization of global clean energy, we need to achieve technological innovation in 5 areas, including clean energy generation,clean energy on-site development and utilization, longdistance transmission and distribution of large-scale clean energy, system flexibility improvement, both decentralized autonomy and integrated coordination of clean energy management.

2.1 Clean energy generation

Clean energy such as wind energy, solar energy,oceanic energy has the characteristics of intermittency and randomness, and its energy density is far lower than the traditional energy sources such as coal, oil, and natural gas. In order to develop clean energy as much as possible,it is imperative to assess new energy resources in order to decide the location of new energy generation stations. Wind power, and solar power generations are closely related to climate and weather. It is important to improve wind power generation forecast accuracy based on weather data and other historical data so as to reduce unit reservation and ensure power system security. The development and application of clean energy cannot be independent from the support of relevant policies and systems, and the analysis on the effects of related policies and market mechanisms on renewable energy development can provide references for governments and regulators to adjust the policies, as well as optimize the market mechanisms, and for energy organizations to take actions.

2.2 Distributed renewable energy development and utilization

GEI has not only large scale renewable energy bases massively integrated into power grid, but also distributed renewable energy resources integrated into all-level power distribution networks and even customer sides.The grid integration of distributed energy resources may cause problems such as low power quality (harmonics,frequency and voltage fluctuations), bidirectional power flow, and protection issues. In order to promote the development of distributed energy resources, innovations in both technologies and business models are needed.For example, with the energy routers and other power electronic equipment, energy internet and micro grid can be constructed, in which distributed energy resources can be effectively managed with less negative impact to utility power grids. Under the appropriate incentive mechanism, distributed energy resource owners can participate in energy trade and demand response.

2.3 Centralized development of clean energy and long-distance transmission

One of the benefits of GEI is to develop Arctic area wind power, Equatorial PV and renewable power in other large-scale energy bases [9]. Such massive renewable energy generations will be transferred to load centers via long distance transmission lines, therefore, flexible DC technology, DC power grid technology, and ultrahigh voltage transmission technology are required. Since GEI is a large scale interconnected energy system and its complexity and uncertainty bring about high risks in power system security and reliability, wide area real-time monitoring can help operators in decision making for emergency control.

2.4 Improve system flexibility

Enough flexibility is needed to address the challenges resulting from the intermittency of renewable energy resources. More flexible resources have to be developed and utilized: flexible load/demand response, energy storage, energy conversion (for example, excess electricity produced from clean energy is stored as hydrogen gas or even transformed to methane) and V2G. Here, energy storage not only means electricity storage but also cooling/heating storage. How to maximize the usage of flexible resources is now still an open question and it will be more important and difficult in the context of GEI.

3 Requirements of big data technologies

3.1 Multiple source data fusion

Data fusion means integrating data from multiple data sources to produce more consistent, accurate and useful information. The expectation is that fused data is more informative and synthetic than the original inputs.The management and control of GEI should combine decentralized & autonomous mode with high level comprehensive & coordinated mode. For both control modes, multi-source data fusion is necessary. Specifically,for distribution network management, based on the integrated data from distribution automation systems,GIS, and AMI across provincial or district areas, such applications as outage management, power quality management, energy conservation service, demand response potential analysis. can be developed; for power system security analysis and control strategy decision making, data from WAMS and SCADA are required across regional or even continental/global area [10,11].

3.2 Advanced data processing technology

Power system related data has the characteristics of 3“V” of big data: volume, variety, and value. Besides,power system has its unique characteristics: power generation, delivery and consumption are instantaneous,and some applications must rely on real-time information processing. For GEI, these characteristics are more pronounced. Therefore, advanced big data processing technologies are required for GEI, including but not limited to distributed databases, distributed computing,memory computing. Of course, advanced knowledge of representation and visualization are also necessary technologies.

3.3 Data driven analysis methods

GEI is more complicated in its technical components and more active in interacting with the outside world than smart grid. GEI faces severe uncertainties and outside factors, such as weather, market, policy, customer preference, and energy consumption behavior. These uncertainties and factors have more evident effect that cannot be ignored. Traditional analysis methods are primarily based on physical models, but they have difficulties in describing uncertainty and complex relationships. As the data driven method, big data analytics,which is independent from physical models, will play a more and more important role in GEI. At present, data driven methods primarily include decision tree algorithms,support vector machine (SVM), artificial neural network(ANN), deep learning, reinforcement learning, etc.

4 Typical application scenarios

The application of big data in GEI covers many technical fields. Considering the article length, it only gives four typical application scenarios as examples.

4.1 Energy production and consumption forecast

With the high penetration of solar and wind power,scheduling and operation of power system face the challenge of increasing uncertainty. Therefore, accurate forecasts of renewable energy production and energy demands at different levels are both important. Although there have existed many methods for wind and PV generation output forecasting and electricity load prediction, most of them are based on small datasets, without utilizing big volumes of data such as those provided by smart meters.

In the case of GEI, to forecast energy production and consumption are more complicated and more factors have to be considered. For example, charging and discharging of EVs and demand response, which are related with user preference, and V2G incentive mechanism, have effects on energy production and consumption. In addition, under different energy pricing mechanisms, the effects of multienergy conversion/storage on energy production and consumption have to be considered.

Big data analysis is more effective when it comes to energy production and consumption forecasting based on big databases. The deployment of smart meters and the establishment of numerical weather forecasts and GPS systems make it possible to give more detailed and accurate predictions by means of big data analytics.

4.2 Analysis and evaluation of policy and market mechanism

The policies for promoting new energy development and market mechanism for encouraging customers to participate in demand response vary from country to country, affected by energy endowment, energy consumption structure, as well as social and economic development situation. Using big data analysis methods, based on relative historical and present data, the effectiveness of the policies and market mechanisms can be validated, which can provide reference for policy making or policy adjusting.

4.3 Energy consumption behavior analysis

The identification and prediction of energy flexibility on the demand side lay foundation for implementing demand response. To understand prosumers’ preference and energy production/consumption behavior is the basis for analyzing their will and potential for participating in demand response and energy trade. At the same time, the latest smart meters developments allow us to monitor in real-time the power consumption level of the customers’appliances. Besides, smart meters enable Nonintrusive Load Monitoring (NILM). By means of NILM, the appliances being used in their individual consumption can be inferred. For prosumers’ behaviors analysis, data analytics algorithms such as clustering, deep learning,need to be adopted.

4.4 Operation mode arrangement

It is a difficult issue to arrange operation modes of GEI,spatially involving scattered local power/energy balance,regional/global comprehensive coordination scheduling,and temporally dealing with 5 min, 10 min, 30 min, one hour, day-ahead scheduling. How to estimate the flexibility of GEI at various levels and effectively manage the flexible sources are essential for energy balance in GEI.Data analytics can play an important role in this aspect,from energy management in micro-grid based on deep reinforcement learning to all kinds of flexibility estimation on all kinds of flexible sources such as energy conversion/storage and flexible loads.

4.5 Security risk analysis

Power system disturbance identification, stability assessment, and emergency control are fundamental to ensure the security of the supply. With the deployment of Wide Area Measurement Systems (WAMS) control centers being flooded with massive volumes of data,therefore, transforming data into knowledge, preferably automatically, is an actual challenge for system operators.

GEI is large in size and complex in structure.Meanwhile, it is penetrated by more and more new elements, such as wind, solar power, flexible loads and electric vehicles. All these elements result in strong interaction, multiple coupling and high randomness in GEI. On this occasion, model-based methods, establishing physical models with assumptions and simplifications as essential preconditions are questionable. Data driven method, such as deep learning and reinforcement learning,is model free and it is looked upon as effective supplements to traditional methods.

5 Big data architecture based on RTM in GEI

A matrix with random variables as elements is called a random matrix. RMT concerns with the probability distribution of the eigenvalues, eigenvectors, and singular values of large-dimensional matrices. Single Ring Law and M-P law, two most famous theorems in RTM, show that the empirical spectrum can convergence to certain area when the order of the matrix tends to infinity. It is the characteristic of large dimension random matrix that make RTM suitable for big data analytics. The authors of the paper have tried using RTM in big data analytics of smart grid and demonstrated the effectiveness in power system stability analysis, assistant decision making for emergency control strategy, and anomaly detection, etc [7,8, 12-16].

Using RTM in data analytics has the following advantages [12-16]:

(1) In constructing random matrix, historical data and on-line data can be put together in one matrix; data fusion from different sources can be realized easily.

(2) By building augmented random matrix, state variables & influence factor parameters can be integrated into one matrix.

(3) By combining RMT with other methods, such as entropy theory, time series, deep reinforcement learning,more effective technical solutions can be obtained.

To take advantage of RTM, the paper aims at presenting a big data architecture based on RTM for GEI.

5.1 The single-ring theorem and M-P Law

(1) The single-ring theorem

Assuming X={xi j} a non-Hermitian random matrix of orderN×T, the elements in the matrix are independent and identically distributed and the expectation E (xij)=0 and variance=1 are satisfied. For the case of multiple non-Hermitian matrices, letL be the number of matrices,be the matrix of singular values equivalent to non-Hermitian matrices. WhenN, T→∞andc=N/ T ∈ (0 ,1], the empirical spectral distribution ofeigenvalues satisfies the single-ring theorem, and the probability density function is:

According to the single-ring theorem, the eigenvalues of high-dimensional non-Hermitian matrixwill be distributed between the outer ring radius 1 and the inner ring radius

(2) M-P Law

It is assumed that the N×Torder non-Hermitian matrices X satisfy the elements in the matrix as independent and identically distributed, expectationµ= 0, and variance σ< ∞. When N, T→ ∞and c=N/ T ∈ (0 ,1], the empirical spectral distribution of the covariance matrix SNdoes not randomly convergeto the density functionf(λs N),andthe formula is as follows:

where,λsNis the eigenvalue of the covariance matrix SN,

(3) Linear eigenvalue statistics

The linear eigenvalue statistics (LESs) indicate the statistical characteristics of large random matrices. The linear eigenvalue statistics of a random matrix X are defined as [7]

where,λi(i =1,2,… ,n) are the eigenvalues of random matrix X,ϕ(⋅) is the test function.

The mean spectral radius (MSR) , a special case of LESs, is defined as the mean distribution radius of eigenvalues, formulated by:

where,λiis the matrix eigenvalue, N the number of eigenvalues. Geometrically, the distancebetween the point where the eigenvalue is located and the origin, that is,the eigenvalue radius.

5.2 Procedure of big data analytics based on RTM

Fig. 1 illustrates the flowchart of big data analytics for GEI based on RTM. Based on scenario analysis, it is decided what data are needed. For real time analysis,split window of random matrix is used for eigenvalue calculation, if correlation analysis the augmented matrix is necessary. For detailed information, please see references[7,8, 12-16].

Fig. 1 Flowchart of big data analytics for GEI based on RTM

6 Conclusions

(1) GEI is a very large scale complicated system.High penetration of renewable energy generation, EVs and flexible loads make GEI operate in more uncertain conditions, and outside factors, climate and weather,policy and market mechanism, users’ energy consumption behaviors, have evident effect on GEI. Traditional analysis method, based on physical model, are not suitable for describing uncertainties and non-physical systems. Big data analytics, however, as a data driven method, is expected to offer more effective technical solutions.

(2) Big data analytics, as one of enabling technologies,will create very important applications in GEI. Application scenarios include but not limited to renewable energy production/consumption prediction, customer energy consumption behavior/demand response potential analysis,stability analysis and assistant decision-making for emergency control strategy, nearly covering every technical area. The paper only talks about some typical application scenarios.

(3) RTM focuses on probability distribution of eigenvalues of large dimension random matrix. Authors of the paper have demonstrated that RTM is effective in big data analytics of smart grid. The paper suggests that RMT is more suitable for usage in big data analytics of GEI.Also, the architecture of big data analytics based on RMT for GEI is presented.


This work was supported by National High-technology Research and Development Program of China (863 Program) (2015AA050203); the State Grid Science and Technology Project (5442DZ170019-P).


[1] Liu Z (2015)Global Energy Internet. Beijing: China Electric Power Press

[2] Wang J (2015) Big Data: the innovation driven force for Power Utilities. State Grid Corporation of China, 2015(12): 58-61

[3] Wang J, Ji Z, Shi M et al (2015) Scenario analysisand application research on big data in smart power distribution and consumption systems. Proceedings of the CSEE, 35(8): 1829-1831

[4] Zhao Y (2015) Big Data Structure-Big Data Techniques and Algorithm Analysis. Beijing: Electronic Industry Press, 2015

[5] Zhang D, Miao X, Liu L et al (2015) Research ondevelopment strategy for smart grid big data. Proceedings ofthe CSEE, 35(1):2-6

[6] Liang X (2018) Application and research of global grid database design based on geographic information. Global Energy Interconnection, 1(1): 87-95

[7] He X, Ai Q, Qiu R et al (2015) A big data architecture design for smart grids based on random matrix theory[C]. 2015 IEEE Transactions on Smart Grid

[8] Xu X, He X, AI Q et al (2015) A correlation analysis method for power systems based on random matrix theory[C]. 2015 IEEE Transactions on Smart Grid

[9] Tian S, Luan W, Zhang D et al (2015) Technical forms and key technologies on energy internet.Proceedings of the CSEE,35(14): 3482-3486

[10] EPRI (2016) Big data challenges for the grid: EPRI survey resultsand analysis Data analytics and applications demonstrationnewsletter[R/OL]. [2016-01-20]

[11] EPRI (2014) The whys, whats, and hows of managing data as anasset

[12] Song Y, Li X, Xu Y (2015) Coordinated operatingmode of demand and generation dispatching in smart grid. Electric Power Information and Communication Technology, 13(7): 8-14

[13] Liu W Zhang D, Wang X et al (2016) Power system transient stability analysis based on random matrix theory. Proceedings of CSEE, 36 (18):4854-4867

[14] Liu W, Zhang D, Ding Y et al (2017) Power grid vulnerability identification methods based on random matrix theory and entropy theory. Proceedings of CSEE, 37(20): 5893-5901

[15] Liu W, Zhang D, Wang X et al (2018) A decision making strategy for generating unit tripping under emergency circumstances based on deep reinforcement learning.Proceedings of CSEE, 38(1): 109-119

[16] Wu Q, Zhang D, Liu D et al (2016) A method for power system steady stability situation assessment based on random matrix theory. Proceedings of CSEE, 36(20):5414-5420

Received:20 November 2017/ Accepted: 12 December 2017/ Published:25 August 2018

Dongxia Zhang

Robert Caiming Qiu


Dongxia Zhang

receivedher bachelor and master degree at Taiyuan University of Technology, Taiyuan, in 1985 and 1992 respectively, and Ph.D. degree at Tsinghua University, Beijing, in 1999. She is senior engineer in China Electric Power Research Institute. Her research interests include power system analysis and planning, smart grid technology, big data application in power systems.

Robert Caiming Qiu

receivedhis bachelor degree at Xidian Unversity , Xi’an, in 1987,master degree at University of Electronic Science and Technology of China, Chengdu,in 1990 and Ph.D. degree at New York University in 1995. He is IEEE fellow.He is currently professor in Tennessee Technological University (TTU), USA and Distinguished Professor in Shanghai Jiao Tong University,Shanghai, China.His research interests include wireless sensor technology, big data theory and applications.

(Editor Ya Gao)