Data Integrity in the ArcGIS Utility Network
Written by: Zach Lawlor
...On California Climate Change Disclosure Rules
Written by Leslie Wong and Hussein Sayani
Introduction
Welcome back, fellow utility network aficionados! If you're just tuning in, catch up on our epic journey through the ArcGIS Utility Network (click here to read Blog #1)—it's like binge-watching your favorite show but with more maps and fewer cliffhangers (hopefully). In this article, we're tackling a topic that we have all probably been on the wrong side of: data integrity. Data integrity can be defined as an assurance that information is complete, accurate, and consistent throughout the data’s entire lifecycle.
If you’re like me, you tend to learn or understand a topic better when examples are presented. You can compare the topic of data integrity to the foundation of a building. Just as a building needs a solid foundation to stand tall and secure, data needs to be accurate and reliable for decisions based on it to be sound.
Importance of Data Integrity
Now that data integrity has been defined, you may be asking yourself, “Why should I change what I’m doing?” Well, do professional race car drivers navigate the track without guardrails? The answer is no. They still use guardrails because even though they are professionals, they cannot prevent everything. The same goes for GIS managers navigating data integrity.
Data integrity is crucial to building trust throughout an organization, as GIS systems are only as dependable as the data they contain. Getting people to buy into these systems and use them throughout their normal workflows depends on data accuracy, and deploying tools to help improve and support data integrity is crucial to reaching this goal. Some persuasive examplesinclude:
1. Complete Asset IDs: With many Utility Network deployments, there is also a component that integrates other systems. When an integration occurs, it is often important to have unique IDs for every asset to build the link between systems. Without these links, users will not be able to fully deploy and use these integrations, impacting operations and management.
2. Inconsistent attribute values: One segment of a water main may be labeled as “active,” while segments on either side are labeled “abandoned.” The lifecycle designation for the water main in the middle leads to inaccurate asset reporting when deciding the current assets under active management, which can have a downstream effecton financial planning.
How Does the Utility Network Handle Data Integrity?
With the background set and a foundational understanding of data integrity outlined, it is time to get to what everyone is here for. At a high level, the Utility Network framework provides built-in data integrity processes and workflows to ensure data correctness as a baseline. Still, these can be further configured to best suit the needs of each organization implementing the Utility Network. To dive a bit deeper, these built-in processes and workflows are automated, while some may require manual intervention depending on the type of process being implemented and run.
You may be sitting on the edge of your seat waiting for the details to be revealed on how exactly the Utility Network manages data integrity so let’s get to the chase. Below are some of the critical functionalities included within the Utility Network, along with explanations of how they support data integrity across all features within the network.
BACK TO HOMEPAGE
Share on Social Media!
Leslie Wong
Senior Associate
lwong@langan.com
Hussein Sayani
Senior Project Manager
hsayani@langan.com
Zach Lawlor, GISP Senior GIS Analyst
In the above image, a combination of network rules and network topology ensures that data integrity within the Utility Network is maintained by not allowing certain asset types to connect. For this example, a residential service meter cannot connect to a distribution water main.
1. Network Rules – To make the Utility Network function, many rules are needed to ensure that network components are properly connected. Expanding on this, rules restrict the types of features that can be connected to each other, like water hydrants to service laterals and overhead transformers to poles. These rules serve as feature restrictions and prevent editors from inputting incorrect features that do not make sense within the physical network.
2. Network Topology – Network topology is crucial for the successful deployment of a Utility Network. It manages information about features and supports the connectivity between those features. To do this within the Utility Network framework, features must follow the restrictions while also conforming to the applicable network rules that we just defined. With the network topology enabled, users can efficiently utilize the robust tracing functionality within the Utility Network, allowing real-time network analysis to occur. Using the network rules as the backbone, network topology helps reinforce data integrity within the Utility Network by ensuring that all features connect to other permitted features.
3. Attribute Rules – Not native to the Utility Network, attribute rules enhance the editing experience and support data integrity by automatically calculating values based on expressions or constraining acceptable values for an attribute. Three different types of attribute rules can be utilized to improve data integrity across a Utility Network: calculation, constraint, and validation.
Calculation Rules These rules are used to automatically populate attribute fields on a feature. When setting these rules up, they can either be configured as immediate, meaning they run immediately following an edit operation, or as a batch, meaning that they are evaluated all at once at a user-specified time. No matter which way they are configured, the rules serve as an enormous backstop for users and managers to maintain data integrity. An example of this type of attribute rule is if an editor specifies a pipe diameter as well as a material type in their respective fields, an immediate calculation rule can be set up on a labeling field to automatically generate the pipe’s label every time these two fields get modified, allowing for standardized labeling across the network.
Constraint Rules Unlike calculation rules, constraint rules do not populate attributes within features. Instead, constraint rules specify a feature’s permissible attribute configurations and general relationships. In other words, constraint rules are used to meet specific conditions on a feature. They are evaluated at once during an edit operation for specific triggering events, such as inserts, updates, or deletes. When a constraint rule is violated, a user-defined error message is returned. Implementing this type of rule can be immensely helpful in maintaining data integrity. To give an example of this rule, think of when users are populating an asset ID field. In that case, a constraint rule can be set to only allow string values that are all uppercase, promoting consistency across datasets within the Utility Network.
Validation Rules Unlike calculation and constraint rules, there is no option for validation rules to be run immediately upon an edit operation. This type of rule can only be run in bulk at a user-defined time. They are very similar to constraint rules because they make sure that specific conditions are met on a feature. When these rules are evaluated by a user, instead of an error message appearing, error features are created in feature classes stored in the database, allowing users or managers to perform quality assurance checks on features and review areas of concern as an iterative process. To illustrate how a validation rule might be implemented, think of connected pipes and valves within a water network. A user may configure a validation rule to confirm that the size of the valve is greater than or equal to a specific diameter. By doing this, no valves can be improperly defined to be less than the diameter of the connected pipe, reducing the number of possible errors in the network.
Conclusion
Data integrity confirms that a Utility Network accurately represents the real world and serves as a true "Digital Twin." Network Rules, Network Topology, and Attribute Rules are built-in mechanisms that help support data integrity. Without these safeguards, human error can lead to inconsistencies, confusion, increased costs, and operational inefficiencies. By effectively utilizing these tools, organizations can maintain the reliability and accuracy of their Utility Networks.
To learn more about the ArcGIS Utility Network or see a demo, reach out to Langan’s Digital Solutions Team; stay tuned for more information about how the ArcGIS Utility Network can benefit your organization.