5 signs you need a data engineer
When John Robba started working with Primus Builders on a data analysis strategy, it quickly became apparent that more was needed than just some new software and technology tools.
“The number one complaint I heard was nothing talks to anything else,” said Robba, who’s now the company’s CTO. “They had these islands of information over here and other islands over there. So, I came up with a laundry list of things that needed to be completed, put a committee together, and strategized the importance of each step and how we were going to do it. But there was just a bunch of work to do.”
As Robba dug into the issues, he realized that part of that work was hiring data engineers or people who specialize in managing and understanding data. Based on a recent study from FMI that shows 96% of data collected in the engineering and construction industry goes unused, it seems many construction firms need to take similar actions.
But how do you know if you’re ready to hire a data engineer? Here are five signs that Robba says construction firms can watch for as they progress on their data analysis journey:
- People don’t know what the data figure is supposed to be or who to get it from. (Don’t have a data catalog). Robba said all companies need somewhere data is kept, but often that information is siloed within departments. When companies can’t find the data they need, they must sleuth out where that data lives and who to ask—supervisors, lead managers, field managers, etc. But typically, no one is assigned to these tasks, so the data remains unknown. That’s where data engineers come in. It’s their job to spend the painstaking hours to find out where the data point lives and surface it for the enterprise. Data engineers are uniquely qualified to understand the complexity of finding such information and pulling it all together. “If you have one metric that you’re looking for and you have to ask multiple people, you probably need a resource internally,” Robba said. “If you have to take hours asking three people, how much money did that cost me to get that answer?”
- People are often arguing whether data is correct. When no one is in charge of data or has a handle on how it’s collected, discrepancies can often crop up. Data engineers can determine the correct number by showing reports and data directly from the source. More than simply settling arguments, this ability is crucial because it creates a single source of truth for companies that everyone trusts and relies upon. That data can then be used to make real-time decisions with actionable insights into specific jobs—and predictive analysis for future jobs. “The data engineer is the person who comes and tells everyone that there’s only one source of truth, and that’s the database,” Robba said. “But first, they have to ensure everyone is gathering the data correctly at the source. That’s the real value of a data engineer.”
- People are getting inconsistent numbers across different tools (data is siloed). Workers often use different technology tools and software on the jobsite. But those different approaches lead to inconsistent data, depending on which app or tool is being used. When that happens, companies no longer have a single trusted source of truth. Data engineers can streamline the data collection process and ensure that the different apps, tools and software are properly connected and “talking” to each other. They can also analyze the data collection process and see if each tool is working correctly—and whether the end-users are collecting it properly. “The problem could be as simple as rounding. And then over the course of a couple of years, that number turns into a big deficit,” Robba said. “The data engineer would be able to get into those granular details, compare data sets ABCD, and go, ‘Wow, this is the problem. And here’s what we do to fix it.'”
- It takes too much time to generate reports and gather data. Time suck is a common complaint within companies when it comes to being able to use and analyze data. This issue is often a workflow problem that needs data engineers to sort out and streamline. But typically, end users don’t even realize they’re being inefficient or bottlenecking the process. Data engineers can help firms create automated workflows that take the end-users out of the equation and speed up data entry. They also have the expertise to ferret out where inefficiencies and bottlenecks are occurring. Reports that once took hours to create can then be done with the touch of a button. “Even if you get the number you need, it may not be the most efficient way to get it. So, it’s taking hours to execute that data search versus minutes.” Robba said. “A data engineer could look at that and go, ‘Well, this is a silly calculation.’ Or instead of one big thing, we can make three small things that are more individualized and efficient.”
- There isn’t an incremental approach to data analysis. Jobsites are complex places with a myriad of workflows and data points. So, of course, data collection is an equally complex process that often requires an incremental approach. But companies are often moving so fast they don’t have the time to even begin this process, let alone understand it. Data engineers have the insight and knowledge to help companies understand how to gather data better and move toward the end goal of analysis. Data engineers take the time to speak with various stakeholders and bridge workflows with company goals in mind. With that kind of allocated resource, what used to take managers hours of Excel analysis can be done in just a few minutes with a glance at a data-driven dashboard. “A building gets built essentially in increments, right? You know what the end goal is though—we’re going to build this. Cool. But how are we going to do that?” Robba said. “With data analysis, it’s almost like you don’t know what you don’t know. And a data engineer knows things that people like project managers wouldn’t even begin to understand, so that helps to create that incremental approach.”
No matter where you are on your data analysis journey, it’s important to take your time finding the right personnel to move your company forward. Though they can be hard to find, a growing number of engineers are interested in working with construction companies—and vice versa.
“You need to invest in your future,” Robba said. “Because if you’re not doing it, your competition definitely is.”