Discover Siemens IWLAN
Industrial Ethernet Book Issue 101 / 13
Request Further Info   Print this Page   Send to a Friend  

Building IoT-Ready Manufacturing Networks

Building IoT-ready networks to enable data logging and acquisition, remote monitoring and cloud-based solutions requires careful planning and execution. Converged architectures that incorporate best practices and technology are delivering resiliency, wireless access, defense-in-depth security and cloud connectivity.

THIS ARTICLE IS PART TWO of a presentation that presents an in-depth look at building IoT ready manufacturing networks. Click here to read part 1.

Real-time determinism/performance

Industrial automation control system (IACS) networks differ significantly from their IT counterparts in their need to support real-time communications, which means communicating messages with minimal latency (time delay between message sent and message received) and jitter (the variance of the latency), significantly lower than typical enterprise applications.

Real-time communications help the IACS become more deterministic. Although the network plays a role in the deterministic nature of a system, a number of other factors, such as end-device latency and response time, are also involved. But the network has an important role, not just by sending packets quickly and consistently, but in the services it offers and supports, such as quality-of-service (QoS) and precision time. IACS networks have different real-time communications requirements based on the type of application.

Network Address Translation (NAT)

NAT is a networking technology that enables Control System Engineers (OT) to build IACS applications reusing IP (IPv4) addresses, while allowing those IACS applications to integrate into the larger plant-wide architecture. Plant-wide architectures require unique IP addressing. NAT can be configured to translate only specific IP addresses from inside the IACS application to the outside plant-wide architecture. Doing so provides the added benefit of effectively hiding the inside IP addressing schema of the IACS application.

Whether you are an end user, OEM or system integrator, Internet Protocol (IP) addresses within your Industrial Automation and Control System (IACS) application may need to be reused. Network Address Translation (NAT) enables the reuse of IP addressing without introducing a duplicate IP address error into your IACS application architecture.

Technology and business aspects drive the decision to use NAT. From a business perspective, OEMs use NAT to enable the replication of skids and machines, including IP addressing. This helps to reduce development and commissioning costs.

From a technology perspective, end users use NAT when the IP address space within the plant-wide network infrastructure is limited and not every device needs to communicate outside the skid or machine-level network.

NAT IACS Use Cases

Single Skid/Machine Aggregated by One NAT Switch, Single VLAN

A common use case, shown above, is the coordination of control functions of an OEM skid or machine by a line controller. In this use case, a single Layer 2 virtual LAN (VLAN 2) exists; however the skid or machine IACS devices have a different IP address range (inside) than the line controller (outside). The machine industrial Ethernet switch (IES) translates the inside IP address (192.168.1.x) of the machine controller to an outside IP address (10.10.10.x) on VLAN 2.

This scalable use case enables the integration of multiple skids or machines with duplicated IP addressing into the same line controller VLAN. Each skid or machine IES would have to translate the duplicated inside IP addresses to unique outside IP addresses to avoid a duplicate IP error within the VLAN.

For this use case, a NAT-capable Layer 2 IES is required for each skid or machine. A Layer 3 switch is not required since a single VLAN is used.


Single skid/machine aggregated by one NAT switch, single VLAN.

Single Skid/Machine Aggregated by One NAT Switch, Multiple VLANs

A variation of the previous use case uses multiple VLANs-VLAN 10 for skid or machine 1, VLAN 20 for skid or machine 2 and VLAN 30 for the line controller. As in the previous use case, the IP addresses are duplicated for the IACS devices within each skid or machine. The machine 1 IES translates the inside IP address (192.168.1.x) of the machine controller to an outside IP address (10.10.10.x) on VLAN 10. The IES switch also translates the outside IP address of the default gateway (Layer 3 switch) to an inside IP address.

The machine 2 IES translates the inside IP address (192.168.1.x) of the machine controller to an outside IP address (10.10.20.x) on VLAN 20. Likewise, the machine 2 IES switch also translates the outside IP address of the default gateway to an inside IP address.

Each machine controller has a unique outside IP address and default gateway IP address on its own respective VLAN. The Layer 3 switch routes the outside IP address of each machine controller either to the line controller (vertical interlocking) on VLAN 30, or to the other machine VLAN (horizontal interlocking).

This scalable use case enables the integration of multiple skids or machines with duplicated IP addressing into the same line controller VLAN. Each skid or machine IES would have to translate the duplicated inside IP addresses to unique outside IP addresses to avoid a duplicate IP error within the VLAN.

For this use case, a NAT-capable Layer 2 IES is required for each skid or machine. A Layer 3 switch is required for routing between VLANs.

Multiple Skids/Machines Aggregated by One NAT Switch, Multiple VLANs

A variation of the previous two use cases uses a single NAT-capable IES to translate IP addresses from multiple skids or machines. In this use case, the NAT IES supports multiple instances of NAT, on a per-VLAN basis. As in the previous use cases, the IP addresses are duplicated for the IACS devices within each skid or machine.


Multiple skids/machines aggregated by one NAT switch, multiple VLANs.

Each machine IES aggregates the IACS devices onto its VLAN. The single NAT IES translates the inside IP addresses (192.168.1.x) within each VLAN to its outside IP addresses-VLAN 10 (10.10.10.x) and VLAN 20 (10.10.20.x)-using a separate instance of the NAT table for each VLAN. Each machine controller has a unique outside IP address on its own respective VLAN. The single NAT IES also translates the IP addresses of the default gateway, which is a Layer 3 switch.

The Layer 3 switch routes the outside IP addresses of each machine controller either to the line controller (vertical interlocking) on VLAN 40, or to the other machine VLANs (horizontal interlocking).

This scalable use case enables the integration of multiple skids or machines with duplicated IP addressing into the same line controller VLAN. Each skid or machine has unique outside IP addresses within their respective VLANs to avoid a duplicate IP error. For this use case, a single NAT-capable Layer 2 IES can be used to aggregate multiple machines, while a non-NAT IES is used within each machine. A Layer 3 switch is required to enable routing between the VLANs.

Wireless access

Plant-wide architectures increasingly use IEEE 802.11 wireless networks for critical Industrial Automation and Control System (IACS) applications that require reliable data transmission with low levels of latency and jitter. Wireless Local Area Networks (WLANs) differ significantly from traditional wired LANs in their use of shared radio frequencies, susceptibility to interference and coverage impairments. Deploying a plant-wide WLAN requires thoughtful planning and design as well as periodic monitoring to meet expectations for bandwidth, throughput, reliability and security.

WLAN IACS equipment use cases

Wireless IACS equipment can be characterized by the type of mobility and operational requirements when relocating within the plant-wide architecture. Wireless IACS equipment may stay within a single Cell/Area Zone and remain associated to a single access point (AP) until powered down or disconnected. Wireless equipment may roam across the Industrial Zone and associate to multiple APs while remaining operational.

Fixed position devices in the WLAN architecture have a permanent operational location, also known as "static." Fixed position wireless is an alternative to a wired connection for hard-to-reach and remote locations where cabling is too expensive or impossible to install. Usage areas include process control, machine condition monitoring, fixed environmental monitoring and energy industries. In the manufacturing environment, a common use case is a stand-alone OEM machine or skid that needs to be integrated into a plant-wide architecture over a wireless link.

Nomadic equipment stays in place while operating, then moves to a new location in the shutdown state. After relocation, a new wireless connection needs to be established. Common examples are process skids, storage tanks, reactors and portable manufacturing equipment.

Mobile (non-roaming) equipment changes position during an operation, remaining in the same coverage area within a Cell/Area Zone. Common examples are rotary platforms and turntables, Automated Storage and Retrieval Systems (ASRS), assembly systems with tracks and overhead cranes. These applications may require rapid changes in position and orientation of the wireless client relative to the AP.

Mobile (roaming) equipment changes position during an operation by roaming across multiple coverage areas within the Industrial Zone. Common examples are automatic guided vehicles (AGVs), large ASRS, overhead cranes and train cars.

Autonomous and unified WLANs

Two different wireless network architectures can be utilized to deploy equipment to equipment communication over wireless: Autonomous WLAN and Unified WLAN. With two differing architectures, WLAN allows users to make an informed architecture selection that meets both business and technical needs for scalability within the plant-wide architecture.


Autonomous WLAN Architecture.

The benefits of the Autonomous WLAN architecture include:

  • A lower initial hardware cost
  • Simplified design and deployment
  • More granular control of Quality of Service (QoS) for prioritization of critical IACS application network traffic

The benefits of the Unified WLAN architecture include:

  • Plant-wide scalability
  • Support for plant-wide mobility
  • Advanced optimization and recovery mechanisms
  • Enhanced security

Autonomous WLAN architectures do not utilize the centralized management structure found in the Unified WLAN. Each Access Point (AP) functions as its own stand-alone device, as an AP or Workgroup Bridge (WGB), without the need for a WLC. The Autonomous WLAN architecture is therefore less costly to implement, thus may be more suitable for smaller IACS applications, such as an OEM machine or skid. Autonomous WLAN APs utilized in the WLAN architecture may be later repurposed to the Unified Access architecture with additional hardware under the following conditions:

  • If deployment needs change or large scale plant-wide growth requires an architectural transitioning
  • If the OEM machine/skid is integrated into a plant-wide architecture

The Unified WLAN architecture has the ability to address large-scale plant-wide 802.11 wireless needs. The Unified Access architecture allows for centralized management and control of the wireless access points distributed throughout the plant.

By utilizing a Wireless LAN Controller (WLC) and Lightweight Access Points (LWAP), a centralized management model is created, thus introducing security and self-healing mechanisms to the wireless architecture. The Unified WLAN architecture also introduces foundational services, including intrusion prevention and wireless guest access, for better control over devices seeking to connect to the WLAN.


Unified WLAN Architecture.

Holistic Defense-in-depth Security

By default, a converged IACS network is generally open. Openness facilitates both technology coexistence and IACS device interoperability, which helps to enable the choice of best-in-class IACS products. This openness also requires that configuration and architecture secure and harden IACS networks.

The degree of hardening depends upon the required security stance. Business practices, corporate standards, security policies, application requirements, industry security standards, regulatory compliance, risk management policies and overall tolerance to risk are key factors in determining the appropriate security stance.

No single product, technology or methodology can fully secure IACS applications. Protecting IACS assets requires a defense-in-depth security approach, which addresses internal and external security threats. This approach uses multiple layers of defense (administrative, technical, and physical) at separate IACS levels that address different types of threats.

The Industrial Network Security Framework, which uses a defense-in-depth approach, is aligned to industrial security standards such as IEC-62443 (formerly ISA-99) Industrial Automation and Control Systems (IACS) Security and NIST 800-82 Industrial Control System (ICS) Security.

Designing and implementing a comprehensive IACS network security framework should serve as a natural extension to the IACS. Network security should not be implemented as an afterthought. The industrial network security framework should be pervasive and core to the IACS. However, for existing IACS deployments, the same defense-in-depth layers can be applied incrementally to help improve the security stance of the IACS.

Converged Plantwide defense-in-depth layers include:

  • Control System Engineers-IACS device hardening (for example, physical and electronic), infrastructure device hardening (for example, port security), network segmentation (trust zoning), industrial firewalls (with inspection) at the IACS application edge, IACS application authentication, authorization and accounting (AAA)

  • Control System Engineers in collaboration with IT Network Engineers-computer hardening (OS patching, application white listing), network device hardening (for example, access control, resiliency), wireless LAN access policies

  • IT Security Architects in collaboration with Control Systems Engineers-Identity Services (wired and wireless), Active Directory (AD), Remote Access Servers, plant firewalls, Industrial Demilitarized Zone (IDMZ) design best practices

Many organizations and standards bodies recommend segmenting business system networks from plant-wide networks by using an Industrial Demilitarized Zone (IDMZ). The IDMZ exists as a separate network located at a level between the Industrial and Enterprise Zones, commonly referred to as Level 3.5.

Sometimes referred to as a perimeter network, the IDMZ is a buffer that enforces data security policies between a trusted network (Industrial Zone) to an untrusted network (Enterprise Zone). The IDMZ is an additional layer of defense-in-depth to securely share IACS data and network services between the Industrial and Enterprise Zones.

The demilitarized zone concept is commonplace in traditional IT networks, but is still in early adoption for IACS applications. An IDMZ environment consists of numerous infrastructure devices, including firewalls, VPN servers, IACS application mirrors and reverse proxy servers, in addition to network infrastructure devices such as switches, routers and virtualized services.

Plant-wide deployment of Industrial Firewalls (IFW), which is part of a holistic defense-in-depth industrial security stance, helps to harden the IACS network infrastructure and creates smaller zones of trust. Industrial firewalls have the ability to restrict and inspect traffic flow throughout the plant-wide IACS network.

It is common for OT personnel to apply industrial firewalls to protect their legacy IACS applications - equipment, machines or skids. It is becoming more common for Original Equipment Manufacturers (OEMs) to include an industrial firewall as part of their offering.

To support this convergence of OT and IT, modern industrial firewalls support the capability of being deployed and managed using several different methodologies that are either locally or centrally managed. Locally managed is common for OT plant personnel and OEM applications. Centrally managed is common for IT.

The management and security of the evolving coexistence of technologies within the plant require an authentication, authorization and accounting (AAA) approach. Identity services is required to support centrally managed secure wired or wireless computer access to the IACS networks by plant personnel and contractors.

Secure Remote Connectivity

Secure Remote Access is a comprehensive and versatile remote-access solution that supports the widest range of connectivity options, endpoints, and platforms to meet your organization′s changing and diverse remote access needs.

The Secure Remote Access solution gives IT administrators a single point of control to assign granular access based on both user and device. It provides both full and controlled client-based network access to web-based applications and network resources for a highly secure, flexible remote access deployment.

Arun Siddeswaran, Manager, Solution Engineering, Cisco Systems, Gregory Wilcox, Global Business Development Manager for Rockwell Automation and Paul Didier, Global Solutions Architect for Cisco Systems.


Source: Industrial Ethernet Book Issue 101 / 13
Request Further Info    Print this Page    Send to a Friend  

Back

Sponsors:
Discover Siemens IWLAN
China International Machine Tools Expo 2017
Sensors Midwest 2017

Get Social with us:


© 2010-2017 Published by IEB Media GbR · Last Update: 16.08.2017 · 24 User online · Legal Disclaimer · Contact Us