Feeding the big data and artificial intelligence ‘information-appetite’

The promise of big data and artificial intelligence is everywhere. writes Jon Wells, VP customer solutions, Networked Energy Services. One almost gets the impression that there is no problem that cannot be solved with these new technologies. The answer to everything is ‘big data and artificial intelligence’.

Big data and artificial intelligence is the answer

Open a web browser and you see advertising tuned to your latest online shopping. Turn on the TV and you see advertisements about how our leading IT providers are using big data and artificial intelligence to address social, economic and environmental challenges. Two extremes of direct application of big data and artificial intelligence.

The tools used to derive timely, actionable insight to both the biggest and the most mundane challenges have certainly hit the mainstream. Using these tools has direct application to the smart grid. They can be used to increase reliability, improve operational efficiency, reduce energy loss, increase fair energy supply by reducing fraud and theft, identify illegal use of energy, enable other green energy initiatives such as distributed generation, energy storage, and electric vehicles, and focus restoration by sociological and business priorities.

The piece which is often left out of all the buzz is where is all this data coming from and how it gets to the big data and artificial intelligence platforms. We know it ends up in data lakes and data marts, but where is this data created, how does it get to the systems that can create the value from it, and how do we know that it is secure as it makes this journey? And, then, how is this managed in a smart grid?

The smart grid is the answer

Initiatives like the Clean Energy Package in Europe and the proposed Green New Deal in the US are driving the Energy Transition and putting the focus onto the smart grid to achieve the improvements above. Similarly to big data and artificial intelligence, whenever the question concerns energy efficiency, the answer seems to be ‘the Smart Grid’.

A smart grid is generally split into three segments: the high-voltage, medium-voltage and the low-voltage. The high- and medium voltage pieces are highly visible – they are major engineering projects and come with sophisticated communications, security and management capabilities in-built.

Getting information to feed the big data and artificial intelligence platforms is no great challenge here because the infrastructure is already there.

The low-voltage grid is more challenging – the equipment is highly distributed, often antiquated, unmonitored and unmanaged, and mostly ‘passive’ from an IT perspective.

It has little or no mechanism to share information back to these big data and artificial intelligence platforms that are waiting for it. As such, this represents a suboptimal use of major investments by DSOs.

This is unfortunate because it is in the low-voltage grid that the energy transition, driven by the Clean Energy Package and other green energy and conservations initiatives, is going have the largest impact over the next decades:

• Increased distributed generation

and storage – using residential-scale equipment to generate solar, wind and hydro energy, store locally and feed-back into the local low-voltage grid • Community energy and micro-grid – balancing the supply of energy within a community to minimise the demand on external centrally generated energy.

Both of these require a low-voltage grid that is highly optimised, and which can be dynamically switched through modes of operation to maintain that optimisation as demand and generation changes.

Thus the problem becomes how to create information about the performance of the low-voltage grid, and then communicate that, securely, to the ever-hungry maws of the big data and artificial intelligence platforms.

Internet of Things is the answer

Connection of everything in the low-voltage grid to ‘the Internet of Things’ could be the answer.

Of course, ‘everything’ is really limited to those things with enough IT capability to connect and share information, where the coverage provides the service and where it is technically and economically viable to use the service at the volumes required. That is fine in the high- and medium- voltage grids but still has challenges in the low-voltage grid, where many millions of consumers and their equipment need to be connected and managed.

Energy suppliers need to consider the costs of deploying IT-enabled equipment deep into the low-voltage grid, the costs of physically installing SIMs and associated SIM management, and the costs of monthly subscription for connecting to millions of end-points to collect many gigabytes (or even terabytes) of data each day.

Energy suppliers also need to consider the technology capabilities – there are several applicable network technologies, which can be used (NB-IoT and LTE-M being the most common).

These are wireless technologies, but it is also possible to connect through powerline communications to back-end systems which are Internet-enabled. This approach does not involve a subscription fee, but is dependent on distances, quality and noise-levels of the power cable, and, so, like wireless communications, needs to be considered carefully.

Smart meters are the answer

So, the ability to connect to all low-voltage devices is a potential challenge – let’s look at the devices themselves and see if they are the answer.

The all-pervasive IT-enabled equipment in the low-voltage grid are smart meters. These come in various shapes and sizes, ranging from the barely-smart through to the truly smart, and are generally deployed at the edges of the low-voltage grid. Barely-smart meters are typically able to communicate low-volumes of ‘basic’ consumption information relatively infrequently, and simply exist to provide automated billing.

At the other extreme, the truly-smart can be configured dynamically to report back on a wide range of voltage and power quality metrics, on a regular basis.

Of course, the truly-smart meters tend to attract a premium price tag that needs to be considered, when the DSO is also assessing their medium- and long-term investment strategy and business case. The reality is that, all too often, the DSO is under pressure to follow a policy of cost reduction, and this drives some to the barely-smart version of the smart meter. Unfortunately, these cannot actively participate in feeding the demands of big data and artificial intelligence, and so represent a lost opportunity to leverage the investments made in these platforms.

In any case, the smart meters generally only provide information about the customer service points and (sometimes) the substation transformer. This still leaves a big gap of coverage – effectively, the power cabling and associated distribution devices.

However, some of the truly-smart meters are addressing this space to provide an end-to-end view of low-voltage grid performance.

Don’t look for the silver-bullets – practical solutions are needed

Putting aside the buzz around big data and artificial intelligence, Smart Grid and Smart Meters, there are practical solutions to presenting the volumes and types of information that are required to form timely insight for energy and operational efficiency and sociologically balanced green and fair energy programmes.

Where will information come from?

The low-voltage grid data needs to be created somewhere. Dedicated monitoring systems can be deployed, but they are often too expensive to be deployed as a ‘blanket’ – rather they are deployed in specific known problem areas.

The most prevalent source of information across the low-voltage grid remains smart meters. The truly-smart meters allow large volumes of voltage and power information to be reported back to the DSO with enough frequency that they can spot trends, detect outages and short-term inefficiencies, gain insight and take action.

DSOs should look at their smart meter procurement policy and be confident that smart meters will justify their big data and artificial intelligence investments and generate timely and actionable insight.

Where barely-smart meters are being deployed, DSOs will find themselves without detailed information of the low-voltage grid, be unable to feed big data and artificial intelligence platforms and be unable to adapt to the changing demands in the low-voltage grid.

Communication of volumes of data

The volume of data quickly scales up when one considers the millions of end-points that will have a smart meter; potentially to many gigabytes and even terabytes of data per day.

The volumes and the subscription cost will challenge the standard wireless ‘Internet of Things’ connectivity model. Communications of at least some of this payload over PLC will significantly reduce the cost and data volumes using wireless and will allow the best of both technologies to be leveraged by the DSO.

A hybrid model of PLC and wireless will ensure both volumes and subscription cost remain manageable, and the data can be carried to the ever-hungry maws of the big data and artificial intelligence platforms.

PLC has received bad press over the last few years, creating an impression that it is old technology. In fact, there are truly smart meters based on PLC that employ

the highest quality protocols to achieve high information rates, even in the most challenging network environments.

Robust

Some truly-smart meters extend these options by providing connectivity to physical networks, which terminate at the home, the multi-dwelling unit or in the street. In these cases, the standard communications provided by the smart meter is augmented, and either used to carry more information more frequently, or to provide a back-up in the event of one of the communications mechanisms failing.

The latter resolves the problems of having ‘holes’ in the big data. Some DSOs can even leverage fibre infrastructure provided by government programmes or their own investments and diversifications.

Secure

A lot of privileged information about the consumers and about the DSO is transported.

The data lakes and data marts are highly secure, but the source of the data in the low-voltage grid and the communication through the low-voltage grid also needs to be as secure.

The built-in security features of the smart meters, the wireless and PLC communications needs to be carefully assessed so that the information shared with the big data and artificial intelligence platforms isn’t, accidentally, shared with the cyber-criminal fraternity. Again, typically, this is where the barely-smart meters are lacking, and so justify an extra careful assessment before selection.

Not just electrical energy

Truly-smart meters tend to have additional communications capabilities in-built to allow connection within the consumer’s residence.

This can be used to either connect to other WAN communications, such as the local ISP or community fibre infrastructure, in-home devices or to gather information from other utility meters such as gas and water. All three utilities – electricity, gas and water – are scarce resources, and can be exposed into big data and artificial intelligence platforms via the truly smart meters.

Not just end-points

Finally, the flow of energy within the lowvoltage grid is as important to understand as the energy provided by and delivered to its end-points. The latest truly-smart metering solutions use their own on-board analytics to derive more information about how energy flows within the low-voltage grid, allowing far more fine-grain business insight to be generated and the guesswork to be taken out of what is happening between the substation and the consumer. SEI

Open a web browser and you see advertising tuned to your latest online shopping. Turn on the TV and you see advertisements about how our leading IT providers are using big data and artificial intelligence to address social, economic and environmental challenges. Two extremes of direct application of big data and artificial intelligence.

The tools used to derive timely, actionable insight to both the biggest and the most mundane challenges have certainly hit the mainstream. Using these tools has direct application to the smart grid. They can be used to increase reliability, improve operational efficiency, reduce energy loss, increase fair energy supply by reducing fraud and theft, identify illegal use of energy, enable other green energy initiatives such as distributed generation, energy storage, and electric vehicles, and focus restoration by sociological and business priorities.

The piece which is often left out of all the buzz is where is all this data coming from and how it gets to the big data and artificial intelligence platforms. We know it ends up in data lakes and data marts, but where is this data created, how does it get to the systems that can create the value from it, and how do we know that it is secure as it makes this journey? And, then, how is this managed in a smart grid?

The smart grid is the answer

Initiatives like the Clean Energy Package in Europe and the proposed Green New Deal in the US are driving the Energy Transition and putting the focus onto the smart grid to achieve the improvements above. Similarly to big data and artificial intelligence, whenever the question concerns energy efficiency, the answer seems to be ‘the Smart Grid’.

A smart grid is generally split into three segments: the high-voltage, medium-voltage and the low-voltage. The high- and medium voltage pieces are highly visible – they are major engineering projects and come with sophisticated communications, security and management capabilities in-built.

Getting information to feed the big data and artificial intelligence platforms is no great challenge here because the infrastructure is already there.

The low-voltage grid is more challenging – the equipment is highly distributed, often antiquated, unmonitored and unmanaged, and mostly ‘passive’ from an IT perspective.

It has little or no mechanism to share information back to these big data and artificial intelligence platforms that are waiting for it. As such, this represents a suboptimal use of major investments by DSOs.

This is unfortunate because it is in the low-voltage grid that the energy transition, driven by the Clean Energy Package and other green energy and conservations initiatives, is going have the largest impact over the next decades:

• Increased distributed generation

and storage – using residential-scale equipment to generate solar, wind and hydro energy, store locally and feed-back into the local low-voltage grid • Community energy and micro-grid – balancing the supply of energy within a community to minimise the demand on external centrally generated energy.

Both of these require a low-voltage grid that is highly optimised, and which can be dynamically switched through modes of operation to maintain that optimisation as demand and generation changes.

Thus the problem becomes how to create information about the performance of the low-voltage grid, and then communicate that, securely, to the ever-hungry maws of the big data and artificial intelligence platforms.

Internet of Things is the answer

Connection of everything in the low-voltage grid to ‘the Internet of Things’ could be the answer.

Of course, ‘everything’ is really limited to those things with enough IT capability to connect and share information, where the coverage provides the service and where it is technically and economically viable to use the service at the volumes required. That is fine in the high- and medium- voltage grids but still has challenges in the low-voltage grid, where many millions of consumers and their equipment need to be connected and managed.

Energy suppliers need to consider the costs of deploying IT-enabled equipment deep into the low-voltage grid, the costs of physically installing SIMs and associated SIM management, and the costs of monthly subscription for connecting to millions of end-points to collect many gigabytes (or even terabytes) of data each day.

Energy suppliers also need to consider the technology capabilities – there are several applicable network technologies, which can be used (NB-IoT and LTE-M being the most common).

These are wireless technologies, but it is also possible to connect through powerline communications to back-end systems which are Internet-enabled. This approach does not involve a subscription fee, but is dependent on distances, quality and noise-levels of the power cable, and, so, like wireless communications, needs to be considered carefully.

Smart meters are the answer

So, the ability to connect to all low-voltage devices is a potential challenge – let’s look at the devices themselves and see if they are the answer.

The all-pervasive IT-enabled equipment in the low-voltage grid are smart meters. These come in various shapes and sizes, ranging from the barely-smart through to the truly smart, and are generally deployed at the edges of the low-voltage grid. Barely-smart meters are typically able to communicate low-volumes of ‘basic’ consumption information relatively infrequently, and simply exist to provide automated billing.

At the other extreme, the truly-smart can be configured dynamically to report back on a wide range of voltage and power quality metrics, on a regular basis.

Of course, the truly-smart meters tend to attract a premium price tag that needs to be considered, when the DSO is also assessing their medium- and long-term investment strategy and business case. The reality is that, all too often, the DSO is under pressure to follow a policy of cost reduction, and this drives some to the barely-smart version of the smart meter. Unfortunately, these cannot actively participate in feeding the demands of big data and artificial intelligence, and so represent a lost opportunity to leverage the investments made in these platforms.

In any case, the smart meters generally only provide information about the customer service points and (sometimes) the substation transformer. This still leaves a big gap of coverage – effectively, the power cabling and associated distribution devices.

However, some of the truly-smart meters are addressing this space to provide an end-to-end view of low-voltage grid performance.

Don’t look for the silver-bullets – practical solutions are needed

Putting aside the buzz around big data and artificial intelligence, Smart Grid and Smart Meters, there are practical solutions to presenting the volumes and types of information that are required to form timely insight for energy and operational efficiency and sociologically balanced green and fair energy programmes.

Where will information come from?

The low-voltage grid data needs to be created somewhere. Dedicated monitoring systems can be deployed, but they are often too expensive to be deployed as a ‘blanket’ – rather they are deployed in specific known problem areas.

The most prevalent source of information across the low-voltage grid remains smart meters. The truly-smart meters allow large volumes of voltage and power information to be reported back to the DSO with enough frequency that they can spot trends, detect outages and short-term inefficiencies, gain insight and take action.

DSOs should look at their smart meter procurement policy and be confident that smart meters will justify their big data and artificial intelligence investments and generate timely and actionable insight.

Where barely-smart meters are being deployed, DSOs will find themselves without detailed information of the low-voltage grid, be unable to feed big data and artificial intelligence platforms and be unable to adapt to the changing demands in the low-voltage grid.

Communication of volumes of data

The volume of data quickly scales up when one considers the millions of end-points that will have a smart meter; potentially to many gigabytes and even terabytes of data per day.

The volumes and the subscription cost will challenge the standard wireless ‘Internet of Things’ connectivity model. Communications of at least some of this payload over PLC will significantly reduce the cost and data volumes using wireless and will allow the best of both technologies to be leveraged by the DSO.

A hybrid model of PLC and wireless will ensure both volumes and subscription cost remain manageable, and the data can be carried to the ever-hungry maws of the big data and artificial intelligence platforms.

PLC has received bad press over the last few years, creating an impression that it is old technology. In fact, there are truly smart meters based on PLC that employ

the highest quality protocols to achieve high information rates, even in the most challenging network environments.

Robust

Some truly-smart meters extend these options by providing connectivity to physical networks, which terminate at the home, the multi-dwelling unit or in the street. In these cases, the standard communications provided by the smart meter is augmented, and either used to carry more information more frequently, or to provide a back-up in the event of one of the communications mechanisms failing.

The latter resolves the problems of having ‘holes’ in the big data. Some DSOs can even leverage fibre infrastructure provided by government programmes or their own investments and diversifications.

Secure

A lot of privileged information about the consumers and about the DSO is transported.

The data lakes and data marts are highly secure, but the source of the data in the low-voltage grid and the communication through the low-voltage grid also needs to be as secure.

The built-in security features of the smart meters, the wireless and PLC communications needs to be carefully assessed so that the information shared with the big data and artificial intelligence platforms isn’t, accidentally, shared with the cyber-criminal fraternity. Again, typically, this is where the barely-smart meters are lacking, and so justify an extra careful assessment before selection.

Not just electrical energy

Truly-smart meters tend to have additional communications capabilities in-built to allow connection within the consumer’s residence.

This can be used to either connect to other WAN communications, such as the local ISP or community fibre infrastructure, in-home devices or to gather information from other utility meters such as gas and water. All three utilities – electricity, gas and water – are scarce resources, and can be exposed into big data and artificial intelligence platforms via the truly smart meters.

Not just end-points

Finally, the flow of energy within the lowvoltage grid is as important to understand as the energy provided by and delivered to its end-points. The latest truly-smart metering solutions use their own on-board analytics to derive more information about how energy flows within the low-voltage grid, allowing far more fine-grain business insight to be generated and the guesswork to be taken out of what is happening between the substation and the consumer. SEI

Source: https://www.smart-energy.com/industry-sectors/data_analytics/feeding-the-big-data-and-artificial-intelligence-information-appetite/