Most people who have spent any time at all in a manufacturing career know that it can be exceptionally rewarding. From an initial request for bid, to lending some design assistance, to taking out some of the cost, to manufacturing and shipping the first order, each step along the way contributes to a successful program. If all goes well, it establishes or reinforces a long-term business relationship between the supplier and the customer.
Of course challenges abound. Procuring raw materials, finding skilled labor (or training unskilled labor), working out each step of the production process, and adding yet another order to the production schedule—all of these and more are part of the daily grind to get things done.
Difficulties don’t end there. Once the product is in the field, it might not perform as intended. Certainly the majority of products perform as they should, but field failures do happen, and liability is a factor. High-profile incidents include contaminated food, tainted pharmaceuticals, and defective manufactured components, especially in the automotive sector. Since 1966 nearly 400 million automobiles sold in the U.S. have been subject to recalls.
When product failures are outside of the public view—when they don’t involve consumer products—it’s fair to say that they generate less attention, but they can be devastating and costly nonetheless. Imagine discovering a pipe defect only after it’s put into service in an offshore oil operation and you can imagine a very expensive remediation.
Finding the root cause of a food safety issue or a product failure is possible only when every product or package is marked properly and processing records are complete. Whether it’s a salmonella outbreak from bad produce or a defective component on a pickup truck, traceability enables investigators to categorize incidents, find trends or patterns, and work backward to find the source of the problem. Product markings and comprehensive records also help in tracking down the origin of a product failure even if it’s not a widespread problem but just a single event.
For pharmaceuticals, the Food & Drug Administration requires that production records must be kept for one year beyond the expiration date of the batch or, for some over-the-counter drugs, three years after distribution of the batch. For manufactured goods, it can be an entirely different timeframe.
“For oil and gas pipe, and some tubes like automotive brake lines, manufacturers should retain the records for decades,” said Jay Deneen, founder of Deneen Engineering. But until recently, decades of storage just wasn’t practical.
An electrical engineer with a long career in developing and integrating control systems for tube and pipe producers, Deneen is knowledgeable in the processes for making and testing tube and pipe products, and he understands the challenges with long-term data storage.
He spent the first six years of his career doing engineering work for a steel pipe producer and subsequently, as a controls engineering manager, developed the hardware and control systems to automate processes for tube and pipe production for Thermatool Mill Systems. He founded Deneen Engineering in 1999 to continue this line of work on his own. Research in blockchain and cryptocurrency at the university level put him in the right place at the right time to introduce a robust recordkeeping system to the industry.
Disasterproof Traceability
Everyone knows what a mill test report is, but this isn’t the only record used for tube or pipe applications. For many components made from tube or pipe, a series of manufacturing steps is carried out by various companies that contribute to the final product or assembly. For example, the chain of events for making a pipeline starts with a steel mill and includes the processes of pipe production (welding or piercing), ultrasonic testing, heat treating, hydrotesting, threading, and assembly. Each company that contributes to such a project is required to keep its own records for traceability and provides copies of the record when it ships the order to its customer.
When making pipe to an American Petroleum Institute (API) standard, compliance with the standard includes establishing and maintaining a complete data record. However, Deneen points out that API doesn’t indicate how this record is to be established and, just as importantly, maintained. During an audit the API might request the hydrotesting data for a specific lot of pipe and the test record might be antiquated—perhaps stored in a database in an obsolete system. While this doesn’t violate the rules, it does leave the company vulnerable, Deneen said.
If one of the reports is damaged or destroyed in some sort of an accidental event—say, a flood or a fire—re-establishing the entire record might be impossible. Of course, records should be duplicated (or triplicated) and stored in separate locations in more than one format, but when this level of diligence isn’t carried out, records can be lost forever.
“To have a truly robust system, records must be archived and immutable, storage must be redundant, and recovery must be possible in perpetuity, irrespective of the hardware and software,” Deneen said.
Just as digital technology has revolutionized nearly everything in our lives, it can be used to establish a single permanent record for a product’s traceability. It can create a record that meets the criteria Deneen cited: Permanently stored in several locations and hackproof. The specific technology is a blockchain.
A Decentralized, Bulletproof Database
The term blockchain doesn’t really seem to indicate much of anything, but it’s an ideal term for the technology. The essence of blockchain is that it’s a database. Each information container is a block; when one block is filled, another block is added and connected to the previous one. A blockchain can grow without a specific, imposed limit.
The blockchain concept has several key points that make it essentially bulletproof.
- Decentralized. A blockchain is a database with many identical copies stored on validator nodes. Every time one blockchain is updated, each copy is updated. If a node’s server crashes, the data isn’t lost. Many copies of the database still exist.
- Collective. Nobody owns a public blockchain. In a case of building a pipeline, everyone involved from making the steel to inspecting the assembly contributes to the database. Each entity contributes records or reports to it; no entity has sole control over it.
- Immutable. Information can be entered and blocks can be added, but that’s it. The information in a blockchain cannot be changed. It’s akin to etching something in stone; after it’s written, it can’t be edited. A record can be appended with revisions, but its previous revisions cannot be altered. If a hacker were to successfully change one copy of the blockchain, it would be evident. That block’s predecessors and decedents wouldn’t add up and the single hacked database wouldn’t match the others. Blockchains are compared, or validated, after each block addition to ensure that all of them are identical. The system is one of majority rule. The majority of identical blockchains is considered to be the accurate record; if a nonidentical blockchain is discovered, it is discarded.It’s theoretically possible that a hacker might change more than half the node’s blockchain records successfully, but the time and expense needed to carry this out are considered to be insurmountable.How so? Every blockchain is copied to more than just a few locations, and each copy has the entire record. “Every blockchain is validated by thousands of nodes,” Deneen said. “If one node crashes or one is compromised, thousands of copies of that blockchain still exist.”
- Transparent yet Private. There are two keys used for processing block transactions, one public and one private.The public key, similar to an anonymous email address, is associated with the transaction and the data records of the transaction, All of the information is visible to anyone with the public key, but the key is not associated with an owner.The private key is required to authorize transactions. Since the private key is encrypted, it is almost impossible to associate it to a public key, so the data originator is anonymous.
- Peer-to-Peer. There are no intermediaries of the data transaction. It’s overseen only by a preprogrammed math algorithm. Sending a data record from point A to points B, C, and D doesn’t require a web of multiple databases and corporation intermediaries; point A places the data on the blockchain, and points B, C, and D can recover it when they choose to, and as many times as they would like, by entering the public address on the internet.
Many people associate blockchain with cryptocurrencies, and indeed this is a primary use of the blockchain technology. Deneen illustrated the strengths of the system by describing how money is transferred by banks and by blockchain.
“To transfer funds using a bank, the sender’s bank initiates an ACH transfer,” Deneen said, referring to an automated clearinghouse. “The funds go from the sender’s bank account to a clearinghouse and on to the recipient’s bank account. It involves five entities: the sender, the sender’s bank, the clearinghouse, the recipient’s bank, and the recipient. Three of the entities are middlemen.
“To transfer funds using cryptocurrency, the sender and receiver have cryptocurrency accounts, which aren’t associated with banks,” he said. “Blockchain technology removes the middlemen, making it a peer-to-peer transfer with only a math algorithm as the judge and transfer agent,” Deneen continued. “The transfer and associated data are recorded through mining on the blockchain, which essentially is just a ledger entry.”
It’s a simple system that runs on complex algorithms to perform the transfer and keep it secure.
Deneen explains that the process by which food products go from a ranch or a farm to a processor and then to a grocery store would be an ideal use of blockchain technology.
“Food quality is a big issue,” he said. “For produce items, the necessary record follows the produce every step of the way—the pick date, the truck temperature, the facility that did the washing and packaging, the truck temperature again, and store that shipment was delivered to.” In the case of an outbreak of an illness tied to a specific food item, investigators can go through the relevant records in just a few minutes with blockchain, Deneen said.
The pipe industry has one more aspect to it, he added. Unlike a food item, which has a specified shelf life, pipelines are in service 24/7 for decades. In the drilling industry, pipe is used for a time, then removed from service, refurbished, retested, and resold for future use. Blockchain simplifies cradle-to-grave or even cradle-to-cradle traceability. It’s just a matter of having access to the blockchain address and updating it with the latest information on refurbishment, retesting, and resale.
Does each entity need to use the same system? No.
“Blockchain is local-database-agnostic,” Deneen said. “As long as everyone has an oracle—a communication protocol that connects data to a blockchain—they can update the blockchain.”
A Frictionless Audit
Few activities are more nerve-wracking than an audit. Whether it concerns paying taxes, regulatory compliance, or adherence to a manufacturing standard, nobody wants an auditor to come knocking. Aside from an uncertain outcome, an audit can chew up a substantial amount of time—time that could be much better spent running the business.
The use of blockchain can’t alleviate the uncertainty of process conformance, but it sure does remove the burden of hunting down records and providing them to the auditors.
“All the auditing team needs is the public address of the blockchain,” Deneen said.
