Energy and Operations Post #4: Ways to measure/estimate power use of equipment
In general, there are three ways to go about measuring/estimating power usage of a machine:
- Measuring it directly with a wattmeter.
- Measuring current only with an ammeter (and voltage with a voltmeter).
- Look at the label on the machine.
I have used all three methods during my internship, and there are advantages and disadvantages to each.
In terms of accuracy, the best option is to attach a wattmeter that can log data at frequent intervals (ideally 1 second or less), handle power factors, and leave it recording for a production cycle (usually a week).
For 120V single phase machines (i.e. office/desk equipment) I’ve used the Watts Up-Pro (say it out loud and you’ll get the joke) wattmeter. These meters are pretty inexpensive ($40-$150) and they’re easy to use.
The Watt-up-pro watt meter is very nice for your every day stuff around the house or office.
For three-phase power systems, it’s not so cheap. Easy-to-use hand-held wattmeters that can log data tend to be expensive. At Raytheon, we bought AEMC 8335 Power Quality Analyzers – one kit with three probes (I recommend the MN193BK 5/100A clamps) will run you around $5,000.
I used the AEMC 8335 Power Analyzer for my data gathering. I'll post about how to use it another day.
In my research, I found some cheaper options, though the cheapest will still cost in the thousands of dollars and may not have all the support you may need. There is also a bit of a learning curve on them, though they’ve gotten easier to use in recent years. I think use of the 8335 will require a separate blog post, in fact.
In addition to the cost of the meters themselves, there are also operational costs to use them. In order to affix them inside or on top of the disconnect, the machine should be shut down for safety, which doesn’t make the floor operators very happy. Also, for a lot of equipment, the bulk of the meter and the probes (the things hanging off the meter that attach to the wires) doesn’t fit inside the disconnect. After multiple “installations”, we got it down to about 20 minutes to put the meter on and 5 minutes to take it off, but downtime is downtime.
The next best option to a wattmeter is to have an electrician use an ammeter (~$100 for the one pictured below) to measure the current when the machine is running and when it is idle.
This Fluke ammeter is the one I've seen electricians use at Raytheon.
It is important to get both values, because energy use over a given time period is pretty much driven by how often the machine is “on”, but sometimes machines can still use significant power when it is “idle”. The current is what varies the most during machine operation, whereas the voltage (should be) constant. Therefore, if you know that the power system supplies 480V to the equipment, then you can generally assume that’s the voltage all the time – it does fluctuate, but for energy calculations you can assume it’s constant and you don’t have to use a voltmeter and waste time measuring it. Just remember to measure power on all three wires if it is three phase, because not all loads are balanced.
The advantage to this method is that it is fast, doesn’t require machine downtime, and the meter itself is relatively cheap.
One drawback to this method is that you can’t really measure the power factor easily, and as I wrote about in my last blog post, the power factor can be significant in terms of how much “real power” your machine is using, which is what you get charged for by the power company. That being said, when I use this method, I use a rule of thumb. If what you are measuring is a “resistive” load (heating elements in convection ovens) then power factor is probably pretty close to 1. If you are measuring an “inductive” load (motors, pumps and everything) then I use a power factor of 0.7.
The easiest, and least accurate way to determine how much energy your equipment uses is to look at the power supply label on the machine itself (pictured below).
The maximum power draw by the equipment on the label is often much more than it typically uses.
Energy and Operations Post #3: Calculating power usage of equipment
I gave a rough overview of a typical industrial electrical power distribution system in my second post in this series, so now that we know how power is delivered to the machine, we can talk about the formula for how much power a machine uses. All of the calculations in this post assume that the power is AC power, which is typical in the United States.
As a reminder before getting into the formulas, power is like speed – it’s a rate of use during any given instant of time. Energy is like distance – it is the cumulative power used over a given are of time. In other words, energy is the area underneath the curve drawn out by the instantaneous power use of a piece of equipment over the course of a given period of time.
Now to the math. The power formula for a single phase circuit is:
,
where 
It’s important to note that the formula just given is for “real power” or “true power” – this is the energy we actually get charged for by the electric company, so it’s the one we generally measure for a given machine. That’s why the power factor is so important. If you eliminate the
So, how do we know what the power factor is for a given piece of equipment? I’ll get into that in the next post.
As a side-note, as customers paying electricity bills, we generally don’t care so much about theoretical power but power companies certainly do. As wikipedia says:
The significance of power factor lies in the fact that utility companies supply customers with volt-amperes, but bill them for watts. Power factors below 1.0 require a utility to generate more than the minimum volt-amperes necessary to supply the real power (watts). This increases generation and transmission costs. For example, if the load power factor were as low as 0.7, the apparent power would be 1.4 times the real power used by the load. Line current in the circuit would also be 1.4 times the current required at 1.0 power factor, so the losses in the circuit would be doubled (since they are proportional to the square of the current). Alternatively all components of the system such as generators, conductors, transformers, and switchgear would be increased in size (and cost) to carry the extra current.
Utilities typically charge additional costs to customers who have a power factor below some limit, which is typically 0.9 to 0.95. Engineers are often interested in the power factor of a load as one of the factors that affect the efficiency of power transmission.
There’s hardware to correct a low overall power factor in a facility to avoid the penalty that power companies will assess to low power-factor establishments, and installing one should be an easy way to save on some energy costs if there isn’t one already at your plant.
Getting back to the formula, a lot of industrial power systems are actually three-phase power for a variety of reasons I won’t get into. In addition, there are other varieties of AC power circuits, but single phase and three-phase are the two typical ones I’ve encountered.
The way to calculate power usage is different for three phase power machines, so you need to know the number of phases your machine takes – it will usually say so on the label, pictured below.
Typical three phase motor power supply label. This one actually has the max real power required listed on the label, but that isn't always the case.
A three phase power supply will actually have three wires in the disconnect box, one for each phase. There may also be a fourth wire, for the neutral load, depending on the type of three-phase power. I will be talking about the three phase, three wire set up, since most of the machines at Raytheon that I’ve worked with are like that.
In a lot of ways, each of the three phases/wires can be treated like separate power supplies within the equipment itself – total power use is additive between the three phases. In fact, a lot of equipment will use a “balanced load” between the three phases, which means there will be the same current magnitude on each wire. Some more complicated equipment, however, will use unbalanced loads, each with a different power factor and everything. So when you want to measure power, you should measure the load on all three wires separately, you can’t assume that it is balanced.
The way to calculate total real power use in a three phase system is pretty simple. It’s just:
where the subscripts are for each phase. The square root of three factor for three phase circuits has a lot of interesting reasons behind it, but I won’t get into it here because I don’t really understand it myself.
Now that we know what the variables are in the equation, we can talk about how to measure and/or estimate them.
Energy and Operations Post #2: An idiot’s guide to the industrial power supply system
This post is intended to explain in plain terms what the power distribution system is like in a manufacturing plant. I’ve put together a glossary of terms/basic explanation so you can translate electrician-ese. Where relevant I’ve referenced a link to something more technical.
At the IADC, and as far as I understand many industrial plants around the country, there are a variety of power supplies to different types of equipment – it isn’t uniform as it tends to be in the home. As Mike Norelli described:
Electricity is delivered to the IADC with a voltage of 15 kV at two points and then distributed through itsown internal electricity grid to ten substations spread out in the facility. These substations do not alignwith departments or manufacturing value streams, rather, the substations align with geographic areas of the plant. While it is preferred to have substations align with organizational departments, especially from a reporting and accountability standpoint, it is often not realistic because equipment and departments relocate over time. From these substations, the electricity is stepped down to various voltages (typically 480V, 220V and 120V) and then delivered to equipment on the manufacturing floor or wall outlets in the offices.
The transformers in a substation are far less fun.
You will find similar circuit panels on the manufacturing floor.
Look for these bus ducts overhead - they typically run the length of a large room, or potentially the building.
This disconnect box will be attached to the piece of equipment. Inside it are the wires to deliver power to the equipment.
Not your mother's power plug.
Energy and Operations Post #1: Why should operations (and you) care about energy waste?
In my introductory post about energy and operations, more a reflection on my current internship at Raytheon in Andover, MA, I spoke about my experience in broad terms and all the things I’ve learned in the four months I’ve been at that plant. But, I realized that I hadn’t done a good job motivating anyone to care about energy or my project in the first place. In this first post in the informal weekly series which will hopefully coalesce into a decent sized portion of my thesis, I will be motivating the problem. I’ll be largely drawing on previous LGO theses, and the second chapter in Mike Norelli’s in particular.
Basically, corporations should care about energy if for no other reason than it costs money to use – duh. Electricity bills from where I work, a 1.1 million square foot manufacturing plant in Andover, MA, run about $9 million a year. Of course, there are other forms of energy than electricity – plants also use a lot of natural gas for heating, coal or a variety of other sources. But electricity, at least in Andover, is the most expensive form of energy per watt-hour, and that’s what I’ve been focusing on in my internship.
In fact, I should point out now that electricity in Andover, MA is A LOT more expensive than in other places around the country. According to the Department of Energy, retail electricity in 2010 costs about $0.14 per kilowatt-hour for industrial customers in MA ( and $0.19 for residential customers). That puts MA just behind Connecticut ($0.15) for the most expensive industrial electricity costs in the lower 48 states. For comparison, the average national retail price is half as expensive, at $0.07 , and the cheapest electricity for industrial customers was in Utah, coming in at $0.04/kWh. In China, it is approximately $0.11/kWh, which is actually surprisingly high. These estimates are all blended rate estimates, which is a weighted average of demand costs and consumption costs.
However, it’s one thing to say that electricity costs a lot, it’s another to say that a lot of it is wasted, and yet another to say that something can be done about it. But to start, we need to see where all that electricity is going in the typical manufacturing plant.
From a 1998 survey (yeah, it’s old) done by the DOE the typical electronics manufacturer in the U.S. uses 65% of its electricity on “direct uses”, which basically means the “plug load” of floor equipment (heating, cooling and machine drives) and offices (computers). The remainder is used on “indirect uses” like HVAC (, lighting and other facility support (kitchens). I’m not quite sure how much I can reveal about my specific company yet, but the numbers I’ve gathered are actually quite similar at the IADC.
Energy use of a typical electronics manufacturer in 1998
| End Use | |
|---|---|
| TOTAL FUEL CONSUMPTION | 100% |
| Direct Uses-Total Process | 65% |
| Process Heating | 18% |
| Process Cooling and Refrigeration | 4% |
| Machine Drive | 40% |
| Electro-Chemical Processes | 2% |
| Other Process Use | 2% |
| Direct Uses-Total Nonprocess | 29% |
| Facility HVAC (f) | 15% |
| Facility Lighting | 13% |
| Other Facility Support | 4% |
| End Use Not Reported | 5% |
There is a plethora of information out there about how to best reduce some of the costs for indirect uses, and even some of the direct uses. Two great resources are the Manufacturing Institute’s energy efficiency toolkit and the Environmental Protection Agency’s Lean and Energy Toolkit. Personally, I have been focusing on the direct uses – the 65%. Within that 65% at the IADC, however, is a variety of machinery and technology ranging from thirty-plus year old ovens to the latest in component placement equipment for circuit card manufacturing. I’ll drill down a little bit more into some of my findings from my data gathering over the past couple months to put some of the numbers in perspective. But as a teaser, what I have found is that the energy use of equipment greatly depends on the schedule of operations, which lends itself to some (hopefully) interesting analysis and optimization possibilities.
Also, the flavor of these posts will be on industrial and operational energy use. However, a great resource I’ve used to learn some of the basics and get some easy and (generally) very practical tips and data about reducing energy use in the home is the Mr. Electricity site. (One word of warning – he lives in Austin, TX, so when he advocates using only a space heater to heat the rooms you are in, I would take it under advisement…)
Out of darkness: my internship experience
I’ve gone dark the past few months – I have no real excuse, but my radio silence happened to roughly correspond with my internship start date, so we’ll go with that explanation.
My internship project is to figure out how to best reduce energy waste in the manufacturing process. I am following on to the work done by Mike Norelli, an on-cycle LGO ’10 who wrapped up in December. Since January, I’ve been located at Raytheon in Andover, MA, at the Integrated Air Defense Center (IADC). This plant is best known for manufacturing the majority of Patriot Missile System (the erstwhile “Scudbusters” from Operation Desert Storm), but they also make components for a variety of other advanced radar systems.
The Patriot Missile System
The IADC has about 4,400 (largely unionized) employees, and has a footprint of 1.2 million square feet. The plant includes a mix of offices, production areas, kitchens, and server rooms. Moreover, they make everything from circuit cards to giant radar systems installed on Navy vessels. From Mike Norelli’s thesis:
The IADC had an annual electricity consumption of approximately 57,574 MWhs in 2009, which is the equivalent amount of electrical energy used by 5,126 average American homes. The IADC’s peak power during this was 11,410 kW, occurring in mid August. Since the IADC is such a large energy user, it negotiates its rates directly with its electricity provider. Its approximate annual electricity bill is $9 million.
Following on to Mike’s project was great because he already had built a network of people who were familiar with the project and I could essentially hit the ground running. Unfortunately, that doesn’t mean I didn’t struggle with defining my project for about 2 months. Although I will graduate with a degree in Electrical Engineering, I really didn’t know the first thing about electricity. I never got that light bulb to light up in Physics lab in high school. I had to have somebody explain to me the difference between a Kilowatt (power) and Kilowatt-hour (energy) - the former is a rate, the latter is cumulative: like speed versus total distance traveled in your car. I also had no idea how much energy an oven used versus a laptop versus a soldering iron (or even how to spell “soldering”). And then there’s real power versus apparent power, power factors, inductive loads versus resistive loads, 3 phase versus single phase…
I still don't really get it
The funny thing is, I found a similar level of ignorance throughout the facility, including those in management who have been tasked to somehow reduce the use of energy in their departments. A big part of my project is simply mapping out, as best as we can, how much energy each piece of equipment in a particular department uses under the theory that you can’t control what you can’t measure.
As I’m writing this post, I’m realizing that I have learned so much about energy in my 4 months there – I really owe Raytheon a major debt because I think what I ultimately give the company will be far less than what I’ve taken from them in terms of my own education. I think I’ll start a series of Sunday posts about what I’ve learned so far about energy use in a manufacturing facility – maybe my perspective of ignorance will help teach others who are coming from the same perspective, that’s kind of why I started this blog in the first place. I’m still no expert, but you have to start somewhere.
I’ve also learned a lot about how to work, collaborate and lead in a production environment. My previous jobs were in a start-up and in a research lab – neither of which could be classified as production. I’ve also generally worked sitting at a desk with a computer, whereas about half of my time now is spent on the floor with people. So there have been some interesting work-related personal challenges for me at Raytheon:
- Leadership: The majority of the workforce at IADC, and particularly those I have been working with, have all been much older than me, generally in their late 40′s and 50′s. I was wondering if I could lead effectively in this situation. One piece of advice given to me is if somebody is old enough to be your parent, then they expect you to treat them like that. Another way of putting it is: don’t be a brat. Good parents want to support their kids, and maybe because of that I have gotten great support from pretty much everybody at the facility in that age group. In fact, I would say I have gotten better support from them than from some of my peers closer to my age!
- Culture: If working in a startup was chaos, and working in a research lab was pretty smooth and controlled, I would say that working in a production environment, to be a bit pithy, is controlled chaos. As an example, at one point, I concocted a data collection plan that had a schedule down to the minute over the course of a month. The first day on the floor, that schedule was scrapped. Now I try to plan a day in advance, but often times I simply adjust on the fly. That’s been really good practice for me.
- Operations: A major challenge for me at IADC has been the fact that they are a “high-mix, low volume” operation. A lot of the specific Lean techniques (single piece flow, point of use supplies etc) which we have learned in class, which had been my only real exposure to operations, seem to work best on “low-mix, high volume” production. On top of this is the fact that the processes are highly regulated by the customer, with very rigorous quality requirements for every product. The end result of this situation is a high level of variability in day-to-day operations, low predictability, and a constant fear of unintentionally screwing something up because of the complexity of the system. As a result, I have reduced the problem to something manageable in six months, by either looking at a single value stream end-to-end, or concentrating on a single type of floor equipment such as vacuum ovens. The key thing I have learned from my work is that to reduce energy waste, flexibility is critical, which I think is a lesson applicable in any other product mix or operational environment.
- Workforce: On the other hand, some challenges I thought would be difficult have turned out to not be. For instance, I think the challenges resulting from a union environment is a bit overblown. Sure, at first it was a little annoying that there are contract negotiated breaks during the day, but they’re always at the same times, so you can plan around them. It’s kind of like what batters and pitchers say about homeplate umps – as long as they’re consistent, players don’t have any problem with them. What’s more important is that employees are engaged, helpful, and willing to change. On that score, I’ve had attitudes run the gamut from subdued hostility to indifferent resignation to enthusiastic support – but in no different proportion than when I speak to managers and engineers. I’m sure it can be tough in other places, but for me at least, it hasn’t been an issue.
My view of the upcoming union contract negotiations (how can you have a picture of an umpire without Bobby Cox?)
- Waking up early: This has been the most difficult part for me. First shift at IADC starts at 6 AM, and it’s a 40 minute drive from Watertown. Some people I work with on first shift get there at 5! I think the earliest I’ve made it there has been 7, but generally I get there at 8 so I miss a good 2-3 hour chunk. Fortunately, it hasn’t really been a major problem for me because I need to work with second shift as well and I’m not exactly responsible for anything, but I have massive respect for the people who do it every day.
Yeah, that's pretty much me every morning.
My thesis will go into detail on a lot of the specifics in engineering and management around the project. But suffice it to say it has been a great learning experience so far, and I just hope that I can figure out a way in the next couple months to sustain and spread the approach I’ve developed which should result in some big energy savings throughout the facility.
Information/Decision systems in retail
Sam’s club recently started offering their customers personalized deals based on their past buying history:
Linda Vytlacil, vice president for member insights and innovation at Sam’s Club, said coupons normally had a response rate of 1 percent or 2 percent. With eValues, she said, as many as 20 percent to 30 percent of eligible customers collect the discount they are offered.
The program is called “eValues” and it is the “latest iteration in the fast-growing field known as predictive analytics, which uses vast amounts of data to spot trends and anticipate consumer behavior.”
In a related article in the New York Times (yes, occasionally I do read other stuff) there was a description of how people are being offered product deals to share their personal data on sites like mint.com and foursquare.
While taking Intro to Marketing this past semester, I was struck by how much the field could benefit from quantitative techniques like machine learning and decision/control theory, and how the best retailers (like Wal-Mart) have already harnessed the concept of real-time/individualized data analysis. The push only appears to be accelerating. After all, as a shopper, don’t you want the best deals on things you actually want to buy? It seems like a win-win.
I imagine the trick would be to judiciously use the data to “nudge” people to buy things they normally wouldn’t buy – isn’t that the point of sales in the first place? That could get pretty tricky, but I can see how the data could be used to do that in an optimal sort of way. For instance, you could use the data to group similar products and then predict whether this individual is likely to stray from his/her preferred brand on that day and then give the perfect coupon tailored to that person for a rival brand.
Of course, if that fails, free samples always work…
I prefer the chicken teryaki...
Announcement – New LGO EECS track: Information and Decision Systems
The Information and Decision Systems (IDS) track is designed for LGO students in EECS who want to both explore and develop practical skills in how to apply the latest algorithms and mathematical analysis in real operational settings.
The goal of the track is to make LGO the premier training program for leaders who will use advanced data analysis to make smarter operational decisions. The track includes five courses in four areas: 2 courses in tools/theory, 1 course in design, 1 course in communication and 1 course in an engineering elective specific to an application area.
We have a great adviser in Professor Patrick Jaillet, who is associated with LIDS and is also the new co-director of the Operations Research Center at Sloan.
In the coming months, we will be preparing track materials for the LGO Open House and fleshing out how to make this track an active group in the LGO community, much like the sustainability program in ESD. Shoot me an email if you are interested or have ideas…
Two semesters down, two to go (redux)
I just finished up final exams/projects on Wednesday. They capped a long, hard semester where I learned that graduate school is not exactly like the undergrad experience. For example, there was a lot less partying and a lot more commuting.
Um, wasn't exactly like this...
Last night I posted my highlights for the semester from the engineering perspective on my EECS blog. On this blog, I’ll list some highlights from the LGO/Sloan side (in no particular order):
- My Sloan core team. There were six of us (including myself). We really hit it off from the very beginning in the orientation activities, and we were able to keep it going the whole semester. My group included a girl from Senegal, a guy from Korea, a guy from the Dominican Republic, a girl from Peru and a girl who partially grew up in France. All very nice people, and I really learned a lot from them throughout the semester about their cultures. The group reflects the general level of multi-culturalism at Sloan (I think it is over 50% international at this point), which was something I definitely was not expecting, but was probably the best part of the whole experience.
OP class - seriously, we didn't pose for this photo...
- On that note, the one C-function I went to (Korea) was also a highlight. Done really professionally, I didn’t realize that companies actually sponsor these things for tens of thousands of dollars. Very impressive and Erika and I had a lot of fun (and it included free beer and food). I’m going to try to get to more of these next semester.
A fan dance by Sloanies
- The Sloan sponsored Oktoberfest was also awesome. It included free sausage, potato pancakes, sauerkraut, good beer (are you sensing a theme here?) and a sweet German oompah-band.
This isn't the band that played, but you get the idea...
- Out of all my Sloan classes I enjoyed Marketing, with Professor Mark Ritson, the most. Our case studies ranged from Snapple to Wal-mart, and he had a lot of real world experience, especially in luxury brands (which was fun). He told stories about working for Louis Vuitton where they would actually burn $10,000 extra handbags while drinking cognac and smoking cigars rather than put them on sale (true story!). He also explained how retail stores like Wal-Mart and Costco have so much leverage over brands. For reference, see the recent flap between Costco and Coca-Cola (guess who won?) As a direct result of this class, I’m going to get a Costco membership: they sell $160 Dom P champagne bottles for $80, and their store brand champagne ($10 a bottle) is rated a 94 out of 100 on the champagne quality scale (plus they have those trampolines up front!) Ritson will probably kill me for posting this seeing as how he works for Dom Perignon and I know he reads my blogs…
Costco brand champagne
- The SIP (Sloan Innovation Period) class put on by our LGO leadership committee on how to motivate under-performers in real organizations. This was the first year that LGOs had to take SIP classes in the fall semester, and our leadership committee really stepped up to the plate to negotiate the bureaucracy and offer this class to LGOs for credit. It ended up being a great class for me, because I had never managed a group where motivation was a problem, but that is definitely a major (and delicate) part of a typical managers job. It was great to hear from classmates (Steve Smith, Min, Steve Lee, and Tim McIntosh among others) who had actually been there talk about their approaches. Good stuff that is really practical knowledge – in general all the “leadership labs” that have been put on by our fellow LGO students have been great – a real highlight of the program for me and exactly what I wanted to get out of LGO.
- The Sales club sponsored three day Sales training class. This was also very practical and valuable outside of class knowledge, and I would recommend that everybody take it. Among other things sales related, the instructor basically taught us how to write emails to high level executives that optimize your chance of actually getting responses. Since taking the course, I have actually used his techniques and they really do work (valuable for company liaisons on the internship committee to take…)
- The trip to the Michigan-Notre Dame game that I went on with my friends Tim, Bayan and Todd. I think 8 LGO ’11s went to Michigan as undergrads, and pretty much all of them went to this game, so we had a good crew out there.
Superfans
- Last but not least, LGO ice hockey. Awesome. I can’t really skate or play hockey, but it is good times. We even had a bunch of local alums show up and play with us. I figured out that a good way to defend is to dive all over the ice – more surface area when you’re horizontal, and I turn my fat ass into an advantage that way, rather than a liability.
We're not really as good as the uniforms and gear make us appear
That’s a lot for now, there was a lot of other cool experiences this semester but didn’t quite make the cut given the time I have to write this post (on the bubble, as they say). Those include competing in the 100K elevator pitch competition, an American Airlines case competition, and all the talks and seminars that I went to. I do have a blog post about a lot of that stuff on my EECS blog.
I plan on writing a few more blog posts over the next couple weeks, definitely one about my internship at Raytheon which is really exciting.
But right now, I’m gonna go play some video games…and maybe shovel a little bit…
Big Daddy - you're going down...
Business Intelligence?
I guess I now have a new buzzword for what I’ve been thinking about for the past 8 months. Looks like its a decent place to be right now too, from a business perspective…
In the last two years, the major software companies have scooped up companies in the business intelligence market. Among the larger moves, SAP bought Business Objects for $6.8 billion, I.B.M. bought Cognos for $4.9 billion and Oracle picked up Hyperion for $3.3 billion.
This is from a New York Times article talking about the company culture at SAS.
A personal note about SAS – in the article, they claim that SAS is under attack from open-source software like R, and that they didn’t recognize it in time. I guess I was one of the assailants. I remember when I was working at a startup, we had a need for automating analysis and developing graphics (we were dealing with patient biometric data). SAS wanted to sell us a package for like $3,000 – we didn’t even have that kind of money for salary! From my perspective, R was just as good, if not better, and we made it work automatically using stat-connector (which at that time was under heavy development, it’s probably a lot better now than it was back then in 2004).
The major gripe I have with R is that there is no really good development environment. I’ve used Tinn-R, but the version I’ve been using lately hasn’t been that great. In terms of usability and support, Matlab still is the best option – but you definitely pay for it. Fortunately, I’ve had a free license at MIT – I think if I go to a company that doesn’t have a Matlab license I’ll probably go through withdrawal, but I would definitely take R as my next choice.
I wonder if anybody has been able to implement R on a grid environment?
Where Have You Gone, Bell Labs?
Really interesting article in Business Week that I just caught wind of by Adrian Slywotzky talking about how the importance of public/private research labs for America’s economic recovery.
He estimates that due to the Recession and outsourcing, we need to create 6.7 million jobs, and then to spark demand to truly “recover”, we need to create another 10 million. He says this isn’t impossible, because in the 1990′s the U.S. economy generated 22 million jobs (2.2 million a year), but between 2000-2007 (before the Recession), the economy only generated 900,000 a year.
In addition, as he says,
Of the roughly 130 million jobs in the U.S., only 20% (26 million) pay more than $60,000 a year. The other 80% pay an average of $33,000. That ratio is not a good foundation for a strong middle class and a prosperous society. Rather than a demand engine, it’s a decay curve.
His argument is that basic scientific research by both government and private labs has fueled the various “blockbuster” economies over the past hundred something years:
Cars and petroleum in the 1920s, movies and radio in the 1930s, defense in the 1940s, appliances and television in the 1950s, pharmaceuticals in the 1960s, aerospace in the 1970s, PCs in the 1980s, the Internet and cellular telephony in the 1990s.
He observes that all of these industries grew out of basic research conducted either at private research labs like Bell Labs or government labs like DARPA.
What he sees as a problem is that unlike in previous recessions, the funding for basic research has dwindled over the past decade. He cites Bell Labs as an example where as recently as 2001, there were 30,000 scientists employed and now there are only 1,000.
Underlying much of this, of course, is the oft-observed truth that I can certainly confirm personally, that most of the smart technical people (especially the ones I graduated with at Harvard) have been going into finance over the 10 years. As he says,
Science has lost its allure as the domain for our best and brightest. Much of the best technical talent has been drawn to the promise of riches from Wall Street and financial engineering. We need to reestablish a culture that rewards and celebrates the scientist who is willing to work on tough problems even if the commercial return is less certain.
He fundamentally calls for greater investment in labs and R&D in the United States. His three recommendations for how to get back on track are:
• Clear national goals in two or three key areas, such as carbon-free energy and preventive medicine.
• Government commitment of $10 billion a year above and beyond spending for national agencies to jump-start new industrial research labs
• Government tax credits for corporations that commit to spending 5% to 10% (or more) of R&D on basic research
Incidentally, I saw a while ago, that the third point sounds like something President Obama is proposing as part of his general tax reforms – namely a $74.5 billion tax cut over 10 years for R&D.
Having worked at a government research lab over the past 3 years, I can’t comment much on what it used to be like pre-2000. But I know that the people working there are brilliant. And I also know that the finance sector is not going to create 17 million jobs over the next 10 years. What’s wrong with giving scientists some love, for the good of the country?
