Energy and Operations Post #4: Ways to measure/estimate power use of equipment
In general, there are three ways to go about measuring/estimating power usage of a machine:
- Measuring it directly with a wattmeter.
- Measuring current only with an ammeter (and voltage with a voltmeter).
- Look at the label on the machine.
I have used all three methods during my internship, and there are advantages and disadvantages to each.
In terms of accuracy, the best option is to attach a wattmeter that can log data at frequent intervals (ideally 1 second or less), handle power factors, and leave it recording for a production cycle (usually a week).
For 120V single phase machines (i.e. office/desk equipment) I’ve used the Watts Up-Pro (say it out loud and you’ll get the joke) wattmeter. These meters are pretty inexpensive ($40-$150) and they’re easy to use.
The Watt-up-pro watt meter is very nice for your every day stuff around the house or office.
For three-phase power systems, it’s not so cheap. Easy-to-use hand-held wattmeters that can log data tend to be expensive. At Raytheon, we bought AEMC 8335 Power Quality Analyzers – one kit with three probes (I recommend the MN193BK 5/100A clamps) will run you around $5,000.
I used the AEMC 8335 Power Analyzer for my data gathering. I'll post about how to use it another day.
In my research, I found some cheaper options, though the cheapest will still cost in the thousands of dollars and may not have all the support you may need. There is also a bit of a learning curve on them, though they’ve gotten easier to use in recent years. I think use of the 8335 will require a separate blog post, in fact.
In addition to the cost of the meters themselves, there are also operational costs to use them. In order to affix them inside or on top of the disconnect, the machine should be shut down for safety, which doesn’t make the floor operators very happy. Also, for a lot of equipment, the bulk of the meter and the probes (the things hanging off the meter that attach to the wires) doesn’t fit inside the disconnect. After multiple “installations”, we got it down to about 20 minutes to put the meter on and 5 minutes to take it off, but downtime is downtime.
The next best option to a wattmeter is to have an electrician use an ammeter (~$100 for the one pictured below) to measure the current when the machine is running and when it is idle.
This Fluke ammeter is the one I've seen electricians use at Raytheon.
It is important to get both values, because energy use over a given time period is pretty much driven by how often the machine is “on”, but sometimes machines can still use significant power when it is “idle”. The current is what varies the most during machine operation, whereas the voltage (should be) constant. Therefore, if you know that the power system supplies 480V to the equipment, then you can generally assume that’s the voltage all the time – it does fluctuate, but for energy calculations you can assume it’s constant and you don’t have to use a voltmeter and waste time measuring it. Just remember to measure power on all three wires if it is three phase, because not all loads are balanced.
The advantage to this method is that it is fast, doesn’t require machine downtime, and the meter itself is relatively cheap.
One drawback to this method is that you can’t really measure the power factor easily, and as I wrote about in my last blog post, the power factor can be significant in terms of how much “real power” your machine is using, which is what you get charged for by the power company. That being said, when I use this method, I use a rule of thumb. If what you are measuring is a “resistive” load (heating elements in convection ovens) then power factor is probably pretty close to 1. If you are measuring an “inductive” load (motors, pumps and everything) then I use a power factor of 0.7.
The easiest, and least accurate way to determine how much energy your equipment uses is to look at the power supply label on the machine itself (pictured below).
The maximum power draw by the equipment on the label is often much more than it typically uses.
Energy and Operations Post #2: An idiot’s guide to the industrial power supply system
This post is intended to explain in plain terms what the power distribution system is like in a manufacturing plant. I’ve put together a glossary of terms/basic explanation so you can translate electrician-ese. Where relevant I’ve referenced a link to something more technical.
At the IADC, and as far as I understand many industrial plants around the country, there are a variety of power supplies to different types of equipment – it isn’t uniform as it tends to be in the home. As Mike Norelli described:
Electricity is delivered to the IADC with a voltage of 15 kV at two points and then distributed through itsown internal electricity grid to ten substations spread out in the facility. These substations do not alignwith departments or manufacturing value streams, rather, the substations align with geographic areas of the plant. While it is preferred to have substations align with organizational departments, especially from a reporting and accountability standpoint, it is often not realistic because equipment and departments relocate over time. From these substations, the electricity is stepped down to various voltages (typically 480V, 220V and 120V) and then delivered to equipment on the manufacturing floor or wall outlets in the offices.
The transformers in a substation are far less fun.
You will find similar circuit panels on the manufacturing floor.
Look for these bus ducts overhead - they typically run the length of a large room, or potentially the building.
This disconnect box will be attached to the piece of equipment. Inside it are the wires to deliver power to the equipment.
Not your mother's power plug.
Energy and Operations Post #1: Why should operations (and you) care about energy waste?
In my introductory post about energy and operations, more a reflection on my current internship at Raytheon in Andover, MA, I spoke about my experience in broad terms and all the things I’ve learned in the four months I’ve been at that plant. But, I realized that I hadn’t done a good job motivating anyone to care about energy or my project in the first place. In this first post in the informal weekly series which will hopefully coalesce into a decent sized portion of my thesis, I will be motivating the problem. I’ll be largely drawing on previous LGO theses, and the second chapter in Mike Norelli’s in particular.
Basically, corporations should care about energy if for no other reason than it costs money to use – duh. Electricity bills from where I work, a 1.1 million square foot manufacturing plant in Andover, MA, run about $9 million a year. Of course, there are other forms of energy than electricity – plants also use a lot of natural gas for heating, coal or a variety of other sources. But electricity, at least in Andover, is the most expensive form of energy per watt-hour, and that’s what I’ve been focusing on in my internship.
In fact, I should point out now that electricity in Andover, MA is A LOT more expensive than in other places around the country. According to the Department of Energy, retail electricity in 2010 costs about $0.14 per kilowatt-hour for industrial customers in MA ( and $0.19 for residential customers). That puts MA just behind Connecticut ($0.15) for the most expensive industrial electricity costs in the lower 48 states. For comparison, the average national retail price is half as expensive, at $0.07 , and the cheapest electricity for industrial customers was in Utah, coming in at $0.04/kWh. In China, it is approximately $0.11/kWh, which is actually surprisingly high. These estimates are all blended rate estimates, which is a weighted average of demand costs and consumption costs.
However, it’s one thing to say that electricity costs a lot, it’s another to say that a lot of it is wasted, and yet another to say that something can be done about it. But to start, we need to see where all that electricity is going in the typical manufacturing plant.
From a 1998 survey (yeah, it’s old) done by the DOE the typical electronics manufacturer in the U.S. uses 65% of its electricity on “direct uses”, which basically means the “plug load” of floor equipment (heating, cooling and machine drives) and offices (computers). The remainder is used on “indirect uses” like HVAC (, lighting and other facility support (kitchens). I’m not quite sure how much I can reveal about my specific company yet, but the numbers I’ve gathered are actually quite similar at the IADC.
Energy use of a typical electronics manufacturer in 1998
| End Use | |
|---|---|
| TOTAL FUEL CONSUMPTION | 100% |
| Direct Uses-Total Process | 65% |
| Process Heating | 18% |
| Process Cooling and Refrigeration | 4% |
| Machine Drive | 40% |
| Electro-Chemical Processes | 2% |
| Other Process Use | 2% |
| Direct Uses-Total Nonprocess | 29% |
| Facility HVAC (f) | 15% |
| Facility Lighting | 13% |
| Other Facility Support | 4% |
| End Use Not Reported | 5% |
There is a plethora of information out there about how to best reduce some of the costs for indirect uses, and even some of the direct uses. Two great resources are the Manufacturing Institute’s energy efficiency toolkit and the Environmental Protection Agency’s Lean and Energy Toolkit. Personally, I have been focusing on the direct uses – the 65%. Within that 65% at the IADC, however, is a variety of machinery and technology ranging from thirty-plus year old ovens to the latest in component placement equipment for circuit card manufacturing. I’ll drill down a little bit more into some of my findings from my data gathering over the past couple months to put some of the numbers in perspective. But as a teaser, what I have found is that the energy use of equipment greatly depends on the schedule of operations, which lends itself to some (hopefully) interesting analysis and optimization possibilities.
Also, the flavor of these posts will be on industrial and operational energy use. However, a great resource I’ve used to learn some of the basics and get some easy and (generally) very practical tips and data about reducing energy use in the home is the Mr. Electricity site. (One word of warning – he lives in Austin, TX, so when he advocates using only a space heater to heat the rooms you are in, I would take it under advisement…)
Out of darkness: my internship experience
I’ve gone dark the past few months – I have no real excuse, but my radio silence happened to roughly correspond with my internship start date, so we’ll go with that explanation.
My internship project is to figure out how to best reduce energy waste in the manufacturing process. I am following on to the work done by Mike Norelli, an on-cycle LGO ’10 who wrapped up in December. Since January, I’ve been located at Raytheon in Andover, MA, at the Integrated Air Defense Center (IADC). This plant is best known for manufacturing the majority of Patriot Missile System (the erstwhile “Scudbusters” from Operation Desert Storm), but they also make components for a variety of other advanced radar systems.
The Patriot Missile System
The IADC has about 4,400 (largely unionized) employees, and has a footprint of 1.2 million square feet. The plant includes a mix of offices, production areas, kitchens, and server rooms. Moreover, they make everything from circuit cards to giant radar systems installed on Navy vessels. From Mike Norelli’s thesis:
The IADC had an annual electricity consumption of approximately 57,574 MWhs in 2009, which is the equivalent amount of electrical energy used by 5,126 average American homes. The IADC’s peak power during this was 11,410 kW, occurring in mid August. Since the IADC is such a large energy user, it negotiates its rates directly with its electricity provider. Its approximate annual electricity bill is $9 million.
Following on to Mike’s project was great because he already had built a network of people who were familiar with the project and I could essentially hit the ground running. Unfortunately, that doesn’t mean I didn’t struggle with defining my project for about 2 months. Although I will graduate with a degree in Electrical Engineering, I really didn’t know the first thing about electricity. I never got that light bulb to light up in Physics lab in high school. I had to have somebody explain to me the difference between a Kilowatt (power) and Kilowatt-hour (energy) - the former is a rate, the latter is cumulative: like speed versus total distance traveled in your car. I also had no idea how much energy an oven used versus a laptop versus a soldering iron (or even how to spell “soldering”). And then there’s real power versus apparent power, power factors, inductive loads versus resistive loads, 3 phase versus single phase…
I still don't really get it
The funny thing is, I found a similar level of ignorance throughout the facility, including those in management who have been tasked to somehow reduce the use of energy in their departments. A big part of my project is simply mapping out, as best as we can, how much energy each piece of equipment in a particular department uses under the theory that you can’t control what you can’t measure.
As I’m writing this post, I’m realizing that I have learned so much about energy in my 4 months there – I really owe Raytheon a major debt because I think what I ultimately give the company will be far less than what I’ve taken from them in terms of my own education. I think I’ll start a series of Sunday posts about what I’ve learned so far about energy use in a manufacturing facility – maybe my perspective of ignorance will help teach others who are coming from the same perspective, that’s kind of why I started this blog in the first place. I’m still no expert, but you have to start somewhere.
I’ve also learned a lot about how to work, collaborate and lead in a production environment. My previous jobs were in a start-up and in a research lab – neither of which could be classified as production. I’ve also generally worked sitting at a desk with a computer, whereas about half of my time now is spent on the floor with people. So there have been some interesting work-related personal challenges for me at Raytheon:
- Leadership: The majority of the workforce at IADC, and particularly those I have been working with, have all been much older than me, generally in their late 40′s and 50′s. I was wondering if I could lead effectively in this situation. One piece of advice given to me is if somebody is old enough to be your parent, then they expect you to treat them like that. Another way of putting it is: don’t be a brat. Good parents want to support their kids, and maybe because of that I have gotten great support from pretty much everybody at the facility in that age group. In fact, I would say I have gotten better support from them than from some of my peers closer to my age!
- Culture: If working in a startup was chaos, and working in a research lab was pretty smooth and controlled, I would say that working in a production environment, to be a bit pithy, is controlled chaos. As an example, at one point, I concocted a data collection plan that had a schedule down to the minute over the course of a month. The first day on the floor, that schedule was scrapped. Now I try to plan a day in advance, but often times I simply adjust on the fly. That’s been really good practice for me.
- Operations: A major challenge for me at IADC has been the fact that they are a “high-mix, low volume” operation. A lot of the specific Lean techniques (single piece flow, point of use supplies etc) which we have learned in class, which had been my only real exposure to operations, seem to work best on “low-mix, high volume” production. On top of this is the fact that the processes are highly regulated by the customer, with very rigorous quality requirements for every product. The end result of this situation is a high level of variability in day-to-day operations, low predictability, and a constant fear of unintentionally screwing something up because of the complexity of the system. As a result, I have reduced the problem to something manageable in six months, by either looking at a single value stream end-to-end, or concentrating on a single type of floor equipment such as vacuum ovens. The key thing I have learned from my work is that to reduce energy waste, flexibility is critical, which I think is a lesson applicable in any other product mix or operational environment.
- Workforce: On the other hand, some challenges I thought would be difficult have turned out to not be. For instance, I think the challenges resulting from a union environment is a bit overblown. Sure, at first it was a little annoying that there are contract negotiated breaks during the day, but they’re always at the same times, so you can plan around them. It’s kind of like what batters and pitchers say about homeplate umps – as long as they’re consistent, players don’t have any problem with them. What’s more important is that employees are engaged, helpful, and willing to change. On that score, I’ve had attitudes run the gamut from subdued hostility to indifferent resignation to enthusiastic support – but in no different proportion than when I speak to managers and engineers. I’m sure it can be tough in other places, but for me at least, it hasn’t been an issue.
My view of the upcoming union contract negotiations (how can you have a picture of an umpire without Bobby Cox?)
- Waking up early: This has been the most difficult part for me. First shift at IADC starts at 6 AM, and it’s a 40 minute drive from Watertown. Some people I work with on first shift get there at 5! I think the earliest I’ve made it there has been 7, but generally I get there at 8 so I miss a good 2-3 hour chunk. Fortunately, it hasn’t really been a major problem for me because I need to work with second shift as well and I’m not exactly responsible for anything, but I have massive respect for the people who do it every day.
Yeah, that's pretty much me every morning.
My thesis will go into detail on a lot of the specifics in engineering and management around the project. But suffice it to say it has been a great learning experience so far, and I just hope that I can figure out a way in the next couple months to sustain and spread the approach I’ve developed which should result in some big energy savings throughout the facility.
Google and IBM say we need to train more supercrunchers
There was an article in the New York Times today about the effort that companies like Google and IBM are making to allow university students access to very powerful computing environments to allow engineers and scientists to plow through massive data sets. Their argument is that students are being trained right now to think on a gigabyte scale (if they’re lucky enough to be trained how to analyze real data at all), when all the breakthroughs are happening with datasets in the tera and peta-byte scales.
I couldn’t agree more with this analysis. If people are serious about analyzing those “very rare events”, “long tails” or whatever that can make the difference between a profit and loss, success or failure, or even life or death, then we can’t continue running around assuming things because the model fits 80% of the time and anyways, it’s too hard to do that level of analysis. We all saw what happened with that idea.
When I was working at Lincoln, we created a highly accurate model of U.S. near mid-air collisions. We did this by analyzing about 5 terabytes worth of radar data from across the country (about 8 months worth). Nobody had ever done this before on anything close to that scale.
As a result, we had orders of magnitude more data on near mid-air collisions (a very rare event) than the last model in the early 90′s. Without this data, and the high-powered systems available at Lincoln that we used to analyze it, our model would have suffered from the same assumptions and modeling error as previous attempts, and that is just not good enough for developing something as important as the next generation of collision avoidance systems for manned and unmanned aircraft, which people are now doing at Lincoln, largely as a result of that effort.
The ability to analyze massive data sets has been proven again and again as a competitive advantage in bio-tech, finance (those who do it correctly), internet, and even marketing, making those companies who developed those competencies hundreds of billions of dollars.
Is it then a stretch to say that the next lucrative opportunity in operations management will be to develop the capabilities to harness the massive amounts of data companies already generate every day? I’m talking about everything from inventories to machine control outputs and even to intra-company emails. There are signals in that data, just as there are signals in everything from our DNA to the stock markets, if you look hard enough.
To be honest, I don’t know (I’m new to this stuff!) but that’s why I and several of my classmates are trying to start a new track for LGOs in the EECS department this year called Information and Decision Systems. The focus in this track is to develop the theoretical, practical and communication skills for students who want to take on this operations challenge in the real world, for real companies. That means not just studying and learning the algorithms, but also getting a design background in the networking, database and parallel computing systems that are critical enablers of this type of work. It also means developing specialized communication skills to explain the opportunities and the results, because like the NYT article said, most people have not been trained to think on this scale before.
I could talk for pages more about this topic, but lets just leave it at that for now. I just had to write something because I’m obsessed with this idea, and this article got me all excited. I’m definitely going to look into Hadoop…
Plug for MIT Lincoln Laboratory (“The Lab”)
As cool as it looks
For the past two years I have worked at MIT Lincoln Laboratory, or the Lab for short. Not many people outside of the research circles know about this place, and in fact I only stumbled upon it in my job search because I went to a job fair for high tech companies and got some literature. In fact, the wikipedia page on the Lab is comically short considering the amount of research that has been conducted there for the past 50+ years – while the Lab didn’t invent radar, it probably perfected it. However, because most of this research is classified for national security, 95% of the amazing work by Lab scientists goes unpublished. When I was looking around for information on the Lab, I could hardly find any online, so hopefully somebody may stumble upon this page when they are thinking about working there.
While at the Lab, I worked for the Surveillance Systems group, Group 42. When I got to the Lab, the group was called the Air Traffic Control and Surveillance Systems Group, and about half of the group’s research is sponsored by the FAA. Most of my work centered around the TCAS collision avoidance system, which is mandated by Congress to be installed on every commercial passenger aircraft above a certain size.
TCAS vertical speed indicator (displayed in cockpit)
I also worked on new collision avoidance systems for UAVs (Unmanned Air Vehicles). My major project was developing airspace encounter models for generating random, realistic encounters so that these systems can be tested in simulation. Over the course of developing these models, I learned a great deal about Bayesian networks, Monte Carlo methods, importance sampling, radar (note: the giant radar ALTAIR in that link is operated by Lincoln for missile defense in the Marshall Islands), flight dynamics, and especially air traffic control. It was a fascinating project, and only one of many fascinating projects that I was involved in (I will probably blog about my thoughts on ATC at some point in the future).
Global Hawk, one of the platforms we worked with.
I’ve worked with some amazingly smart and capable people who I now consider my friends. In particular Mykel Kochenderfer and Jim Kuchar have been my mentors, among many others who don’t have personal websites. Over half of the Lab employees have PhDs, most from MIT and other top schools, and the sheer brain power there is kind of awesome. People also work at the Lab for the love of their research, and that definately comes through as well.
I will be leaving the Lab in a couple months because I accepted an offer to be a LGO fellow at MIT. However, my work there has been very intellectually satisfying, and it has served to focus my future academic interests. For instance, I will recieve a MS in Computer Science, largely because my work at the Lab has piqued my interest in AI and machine learning. In particular, I hope to research how these principles can be applied to improve the efficiency and operation of manufacturing companies. I would say that the Lab is an excellent place to work if you are interested in doing cutting edge research, are intellectually curious and smart, and enjoy tackling difficult problems in the area of national defense, homeland security or the FAA.
