Artificial neural networks (ANN) are mathematical models applied through computational procedures inspired by biological neural networks. They consist of three basic elements:
- Inputs – perceived information like images, sounds, temperatures etc.
- Neural network – grid-like interconnected units in order to process the input and provide a suitable output.
- Outputs – processed information like what the image displays, what is the meaning of the sound, whether something is hot or cold and so on.
Often qualified as an emerging technology, this field sums up decades of experience and development. Nowadays, neural networks are present in several promising applications like self-driving cars, voice recognition, image classification/search and industrial improvement. This article gives an overview of how ANNs are applied in the industrial improvement domain.
A Brief History
Modeling the data traffic across the brain has no clear beginning. Aristotle, Freud, Helmholtz and Mach are some names to mention in the attempt to understand brain functions. However, it was only on 1942 that McCulloch & Pitts proposed the first formal mathematical model for neural networks. From then on, this field of study gained a lot of attention. As most success stories, neural networks experienced ups and downs throughout history. Because of the public skepticism from some renowned mathematicians, neural networks’ popularity greatly decreased during the 70’s and only a few continued its research, notably outside the US (e.g. Amari, Fukushima and Kohonen). Nonetheless, this topic eventually gained a lot of momentum because of a simple, yet brilliant, method called backpropagation, made popular by Rumelhart & McClelland in 1986. Before this achievement, it would take an impractical amount of time for ANNs to learn anything important, even on modern CPUs. However, thanks to backpropagation, ANNs can now perform complicated tasks such as the classification of handwritten digits, in a matter of seconds on regular personal computers.
ANNs and Chemical Plants
One of the first reported works regarding the usage of neural nets in this field is the article by Hoskins & Himmelblau (1988) [1] titled “ARTIFICIAL NEURAL NETWORK MODELS OF KNOWLEDGE REPRESENTATION IN CHEMICAL ENGINEERING”. In this groundbreaking work, the authors not only present the desirable capabilities of ANN to the field of engineering, but also present one of its most important usages up to date: fault detection and diagnosis. To this day, the number of scientific articles on chemical engineering applications of Neural Networks is constantly growing.
The use of machine learning in chemical plants and chemical engineering can be broken down into several components. We will mention the following important applications in this article: modeling, automation & control, fault detection and diagnosis, and product formulation.
Modeling
What happens if you put an unskilled cook to prepare a really expensive dish? In the best-case scenario, he will have some luck and get to work as if he was born to do this, the dish will be delicious and everyone will be happy. However, it is very likely that the dish won’t taste good because of over or undercooking, because he used the wrong ingredients or because of incorrect seasoning. The outcome is a big waste of money, unhappy customers and even a kitchen that’s on fire. Well, I guess most of my colleagues won’t be happy with this statement but a chemical plant, no matter how complex, is like a fancy and really expensive kitchen. If you don’t cook the dish right you will waste a huge amount of money, the product will be awful, the customers will be unsatisfied and you may even set fire to the whole plant.
In order to prevent such catastrophic scenarios, you must prepare yourself for the activity you are about to perform. No matter what it takes, you must know what to do to keep operations running smooth and the product coming out right. Nowadays, the best way to do this is to sit on a computer and run many simulations to figure out what you need to do before you do it. However, performing simulations is a bit complex. Models are the basic components of simulators and, since modern plants are increasingly complicated, we are becoming more and more limited in our ability to forecast all that happens on a plant using the laws of nature (phenomenological modeling). On the other hand, industrial businesses demand quick and reliable methods for efficient processes.
Remember our unskilled cook? What happens if we let him watch experienced cooks making the exact same dish over and over again? After some time, he will become an expert in that dish and probably will learn the key components to preparing a delicious dish. He may not exactly become an experienced cook, he may not understand why he needs to do what he is doing, but he will know exactly how to handle the ingredients and seasonings in order to produce that tasty food. This anecdote is a good picture of what artificial intelligence is doing to the industry right now. Machine learning professionals are feeding huge amounts of operational data to neural network models until the model learns the behavior of the plant. After the learning, process operators, managers and owners of such plants use these methods to forecast the plant behavior and make important decisions. They can even implement these models to automate the plant operation.
Automation and Control
One major focus of automation & control is stability. A good example of this is the Segway. The Segway has sensors to detect inclination and movement in real time, feeds this information to a controller circuit, which then actuates the wheel motors to maintain the cart stable in the upright position while it moves or stands still. The same principle applies to a host of industrial equipment. They have a huge amount of strategically placed sensors measuring temperature, pressure, velocity etc. These sensors pass information to a controller circuit, which then actuates the heaters, coolers, valves, motors and many other equipment in order to maintain the product specification stable in the desired set point. When you make a complex arrangement of equipment in order to manufacture a specific product, the amount of data generated is huge and designing a proper controlling system becomes a challenge.
We begin by stating that, in certain circumstances, even measuring the desired variable is sometimes infeasible. One example I came across recently was the measurement of sulfur content in diesel production plants. As some may know, diesel is a product of oil distillation that may contain sulfur, a harmful substance to the environment and the human health. A process called hidrodessulfuration (HDS) removes the sulfur from diesel before selling. However, due to technical reasons, direct measurement of sulfur content (the product quality variable) in real time was not possible, and was carried out only once a day. Should anything go wrong with that plant, they would only know about it the day after. It turns out that the plant already displayed temperature, flow and pressure measurements in real time on 33 different points across the equipment. What one of the field engineers did was to take two years of operational data and train a neural network to guess the product quality based on real time measurement of the other variables. The model was successful and they implemented this model as a software sensor (soft sensor) on the supervisory system. They are now able to monitor the product quality in real time.
Soft sensors or virtual sensors are starting to have widespread adoption in industrial applications. They’re software designed to perform indirect measurements of quantities that are either impractical or expensive to measure via direct measurement. Kadlec [7] and collaborators present a long list of applications of soft sensors in their work published in 2009. The development of such sensors mostly rely on data driven methods like MLP (multilayer perceptron), SOM (self-organizing maps) and PCA (principal components analysis).
The discipline of A&C employs many different machine-learning methods. Some methods worthy of mentioning are process identification and neural networks model predictive control (NNMPC). In short, NNMPC comprises training a neural network to predict the behavior of a plant (process identification) and then implementing it in the controller unit to keep the desired values on their set point automatically. Describing how this technology works is too technical and we will limit ourselves to say that the oil & gas industry is extensively using it nowadays.
Fault Detection (FD) and Diagnosis
Correctly identifying faults in industrial processes in real time is a crucial task. Failing to do so may lead to severe problems ranging from millions of dollars in capital losses to the loss of lives due to accidents and explosions. As mentioned before, Hoskins & Himmelblau presented the first application of machine learning to fault diagnosis on chemical plants in 1988. This topic is still of great importance in recent days. Since industrial processes have greatly increased in complexity, it becomes harder and harder for a human operator to take into account all of the process variables and be aware of all process failures. There are two main approaches to fault detection: model-based and data-driven. Machine learning relies on the second approach, which is constantly growing in research and application. Different from model-based approaches, in which the quantitative model is known a priori, data-driven FD methods are only dependent on the measured process variables.
The Leonard Kramer problem is an illustrative abstract example of fault detection. It is not based on any real life problem. Yet, the authors designed it to be visualized on a 2D plot. A “square” around zero defines the normal operating region, as diagonal “rectangles” define the fault region.
Leonard Kramer problem
Leonard & Kramer [2] first proposed this example on a work published in 1990. The example presented is a simplified version of many real process diagnosis problems. Two operating conditions named X1 and X2 are given. To make it more intuitive, let us consider that those variables represent two quality variables of a manufactured product. The process is operating well when X1 and X2 are zero. However, changes may occur in the process, leading to fault conditions. This means that the process can then operate under three conditions, namely: Normal, type 1 fault (F1) and type 2 fault (F2). Notice that in the picture the normal operating conditions, represented by blue crosses, are not exactly zero, but dispersed around zero. This happens because this work considers noises and errors when measuring both variables, which is a closer representation of a real process. Therefore, the task of the data scientist is to train a model that identifies the state of the process (Normal, F1 or F2) using the pattern presented on the chart. One major challenge of this task is the confusion of states close to the boundaries around the normal region, which often prevents models from reaching 100% accuracy.
Tennessee Eastman Process Challenge (TEPC)
Downs and Vogel [3] first proposed this problem in 1993; however, Bathelt, Ricker & Jelali [4] updated it in 2015. The latter authors claim:
”Although being a rather old process model, the Tennessee Eastman model of Downs and Vogel [1993] remains an important tool throughout all disciplines of the system theory for the purpose of comparison studies or validation of algorithms (e.g. fault diagnosis in Yin et al. [2012], system identification in Bathelt and Jelali [2014]). Its strength is the fact that it was modeled based on a real process. This led to a non-linear model of a rather complex multicomponent system.”
Tennessee Eastman process simulates a complex arrangement of industrial equipment to manufacture two anonymous products named G and H, using four anonymous raw materials named A, C, D and E, containing impurities named B and F. In comparison with the Leonard-Kramer problem, TEPC has 41 measurable variables (as opposed to the previous x1 and x2), 21 types of faults (as opposed to the previous F1 and F2) and a function to calculate the capital losses. TEPC is a method for generating complex patterns of data in order to train and validate various sorts of data driven methods.
Product Formulation
Companies launch loads of different products to the market every day. From soap to rocket propellant, product formulation is one of those fields that commonly rely on a very experienced professional, with a refined sense, often leading to secret recipes passed on from generation to generation. The main problem with product formulation is that it often takes a lot of time, capital and trial & error to achieve the expected quality. However, would it be possible to train artificial intelligence machines to perform this intellectually exhaustive work? It is not only possible, but it is also already being performed on a wide range of applications. Below we cite some reported works.
Biodiesel production
Most developed countries have environmental policies that promote the development and application of renewable energy. One of the promising forms of renewable energy is biodiesel. Biodiesel is a fuel oil with similar combustion properties of diesel. A work by Yuste & Dorado[5], published in 2006, optimized the production of biodiesel from waste olive oil with the aid of artificial intelligence. Conducting experiments to come up with the ideal formula of biodiesel is time consuming, as it takes a lot of trial and error. The authors used machine learning to perform the experiments virtually, coming up with a very good data driven model of biodiesel production. By utilizing this model, they were able to determine the best way to manufacture biodiesel faster than they would with experimental methods.
Flame Retardant
Juntao Xia [6] and collaborators produced a similar work in 2000. They used neural networks to develop a flame retardant polymer, an essential product to prevent fire from spreading. The main goal of this work was to find a formulation with a good limiting oxygen index (LOI) and with a low halogen content, which is greatly pollutant. They too used artificial intelligence. To test their artificial intelligence model, they conducted laboratory experiments and confirmed that the neural network guessed the optimal LOI index with an error margin of only 0.6%. This is one good example of artificial intelligence working to develop products that help mankind.
Recent Developments
We could go further with the amount of amazing products formulated with machine learning assistance. Recent developments show that machine are learning how to perform much more complex tasks like hydrogen fuel cells assessment, protein folding prediction and even quantum states analysis. One recent development onproduct formulation is the usage of deep learning techniques in order to come up with better (stronger, harder, lighter etc.) materials through the understanding of their molecular structure using complex crystallographic charts.
Remarks
Although we talked about some of the applications of machine learning that the industry and academia are already employing, there is still so much room for innovation. New machine learning methods appear every day and relatively old methods do not appear to dominate any specific field. Experienced operators are still performing too many time-consuming procedures that artificial intelligence can possibly carry out in a more efficient manner. Therefore, the integration between data science and industry may lead to a host of breakthrough innovations.
References:
[1] http://www.sciencedirect.com/science/article/pii/0098135488870157
[3] http://www.sciencedirect.com/science/article/pii/009813549380018I
[4] http://depts.washington.edu/control/LARRY/TE/download.html
[5] http://pubs.acs.org/doi/abs/10.1021/ef050226t
[6] http://jfs.sagepub.com/content/19/1/3.short
[7] http://www.sciencedirect.com/science/article/pii/S0098135409000076
