Unstructured Big Data Is a Big Manufacturing Challenge

Big Data includes tons of unstructured content, from PDFs to video files. Manufacturers now have to pay more attention to the unstructured content in order to get their arms about Big Data. While much of the Big Data, such as database files, is structured, one of the biggest challenges is managing the unstructured content.

Greg Milliken, VP of Marketing at M-Files, notes that the amount of unstructured data manufacturers have to manage is growing exponentially and is becoming a major challenge. "When we talk about unstructured files of content, we refer to documents such as spreadsheets, Word files, video, or CAD. Usually we manage these files in folders or shared drives in Windows," he told us. "The structured data resides in ERP or accounting, where it's organized in tables. Yet unstructured files contain a huge amount of data that is necessary to run a manufacturing business."

Engineers generate content such as CAD files that have their own structure and processes. This data needs to be organized so users can find parts and materials. "The problem is that a lot of files are difficult to organize, and the team does not always work within an organized structure," said Milliken. "Typically, unstructured data is organized by project, by date, or by customer. When you look at file organization, it's subjective across the business."

MORE FROM DESIGN NEWS: Learning to Use Big Data

The subjective approach to organizing data can make it difficult to locate it once it moves to production. "The challenge is to capture the data in such a way that it helps workers find what they need when they need it. It's about how the data fits into the workflow," said Milliken. "You need to organize the data based on use."

One of the challenges in managing data is the flood of different types of information that needs to be managed. "It's not just office documents. Now its social media -- not the popular social media, but the industrial social media. Then you throw in email," said Milliken. "All of that increases information dramatically. Every business has a history of revision, and you may want the old version, so that data becomes important. All of that unstructured content has become a big-data problem."

Many manufacturers have to be able to prove the quality of their processes, whether it's to verify requirements in the pharmaceutical field or prove to a customer that set processes are being followed. "One of our focuses is compliant-driven data. In manufacturing, you may have to follow quality standards, either because it's mandatory or because a customer requests it," said Milliken. "You have to show that you followed quality processes correctly, and you have to document it and prove it."

MORE FROM DESIGN NEWS: Moving Big Data to Equipment Design

As the manufacturer manages data, the company needs to be able to handle both structured and unstructured data. "The management of unstructured content cannot be uncoupled from the structured content," said Milliken. "We believe the fundamental secret sauce of organizing content is by meta data. Organizing information depends on meta data."

He said he doesn't see data management as run-of-the-mill organization challenge for manufacturers. He believes the ability to effectively manage data is rising to the level of competitive requirement. "Your quality as a manufacturer is more and more about your ability to manage information."

Rob Spiegel has covered automation and control for 15 years, 12 of them for Design News. Other topics he has covered include supply chain technology, alternative energy, and cyber security. For 10 years he was owner and publisher of the food magazine, Chile Pepper.

Atlantic Design & Manufacturing, the largest advanced design and manufacturing trade show serving the Northeastern US, will take place in New York, June 9-11, 2015. It's your chance to meet qualified suppliers, get hands-on with the latest technologies, and expand your network. Learn more here.