Which refineries participate in the benchmarking studies by Solomon Associates? In which quartile is your refinery? What are the benefits gained from participating in benchmarking?

Josh,
Many years ago, I used to coordinate the particiapation of refineries in the Solomon studies in a multi-national oil company.
Solomon Associates (SAI) do the studies every year, even years in the USA, and odd years for refineries in the rest of the world. About 150-200 refineries took part in each study but SAI kept the names of the locations confidential. So your question of who actulaay participated in each study can only be answered by SAI, and they wont tell you. Therefore you cant say who is in the top quartile or who is a pacesetter, but what you can get is their performance figures and youir own.
SAI measures a number of parameters, e.g, energy, manpower, maintenance costs, operating costs, conversion efficiency etc. They use a normalizing method, where all process units are measured in equivalent distillation capacity (EDC), so that simple and complex refineries can be compared. They also normalize for shutdown cycles and maintenance cost allocations to eliminate differences in paccounting practices. This way they manage to get a level playing field.

quote:

This way they manage to get a level playing field.

Vee,

I couldn't resist. I have to ask this question, though I don't really expect you to "answer" it.

Does SAI really get the participants on a level playing field?

I currently work for a multi-national oil company. In past years, before many mergers, the "heritage" company I worked for participated in Solomon surveys. I was NOT directly involved in either coordinating the data submitted to Solomon, nor in doing anything with the results that came back from Solomon.

In my limited exposure to those who were, two things strike me as very interesting.

1 - Two engineers on the same process unit in the same plant can work on putting together the "Solomon Survey" numbers, and end up with surveys that look totally different. It follows, then, that if two experienced people from the same unit of the same refinery can't get it the same, how standardized is the response from all the participants with respect to each other?

2 - I remember the Solomon Report saying in the equivalent of big, bold, letters DO NOT make organizational/operating changes based on these comparisons. What's the first thing a company does if they don't come out "on top", for example, in the manpower/EDC metric? They start cutting people...the very thing that Solomon warns them not to do based on these results.

Mostly I bring this up, as a reminder that just because the benchmarking says something, don't go make a knee-jerk change to emulate someone else's end-result. If you want to get where they are, you need to understand what makes them what they are, and the PATH they took to get there.

Kim Williams

quote:

DO NOT make organizational/operating changes based on these comparisons

At least it is honest recommendation.

BenchMarking is good, but in most cases it is only valid for a limited time span, when you are doing an (self) assessment. I have also read a lot of benchmarking / best practices for especially maintenance, there are a lot of good indicators, but I have seen also IMHO BS that maybe work in some situations, but would be a complete disaster in most cases.

The levelling theory would not fit in the SAI recomendation

What are the benefits of entering in a benchmarking if changes are not to be done?

Eugene,

The point here is not "don't benchmark", it's really that you need to understand.

Take a company (Company A)that's been through years and years of improvements and now they're top of the list.

A company (Company B) that hasn't been making improvements can't simply implement Company A's CURRENT organizational structure and manpower loading and expect to magically have the same results as Company A. Company B needs to understand what Company A did to get there. (What was the PATH?) We can't just emulate the end-result. We have to emulate the path. Hopefully, if we understand the path we're trying to emulate, we can short-circuit it, and not take as long as they did to get there, but there are pieces that just have to happen.

Kind of like your 8th grade algebra teacher that made you always show your work. Just trying to implement someone else's ending structure is like copying down the answer. My 8th grade algebra teacher would mark it wrong if we didn't show the work, and that's what life does to those companies.

If you understand, you can show the work, and get the "right answer", and if you show the work, but it's not exactly the "right answer" you can at least get partial credit.

Kim Williams

PS: I don't know how well companies account for economies of scale in these benchmarking studies, but I'm certain of one thing. It doesn't take many more people to operate and maintain a 200,000 barrel-per-day crude unit than it does to operate and maintain a 50,000 barrel-per-day crude unit. In fact, if the 200,000 bpd unit is a lot newer, it might even take fewer people to run it. Those are the kinds of details you need to understand when you try to make improvements based on benchmarking. You need to understand THEIR system AND their PATH, and you need to understand YOUR system and where you want it to go, and build the right path to get where you need to go, using what you've learned from them to help you.

Hullo Kim,
Nice to hear from you.

1. On level playing fields, in my personal view, SAI does a far better job than most other such organizations, e.g. McKinsey for E&P installations. SAI also cover a range of parameters, not just costs.
2. Intrestingly, I used to do a benchmarking study for the multinational oil company internally as well, every year. We had about 35 Refineries to look at, so it was not a small sample size. We had our own normalizing methods, different from SAI. The top five and the bottom five in both studies were the same, though the order sometimes varied. In the middle lot, the order was sometimes quite different. Lee Solomon of SAI used to say, a dog is a dog any way you measure it, and I think this sort of supports that statement.
3. SAI did not just give performance numbers. They did tell participants what the best operformers (pacesetters) do differently. For example, Lee used to insist that Reliability is the key, and encouranged participants to use RCA, RCM etc to help reduce failures or failure consequences.

quote:

1 - Two engineers on the same process unit in the same plant can work on putting together the "Solomon Survey" numbers, and end up with surveys that look totally different.

4. SAI give a set of detailed instructions on how to fill in the input forms. They do a limited amount of validation, including site visits occasionally. I would argue that if people followed their instructions properly, the survey inputs have to be substantially similar. However, it does not need much effort to deliberately misinterpret an input instruction. Cheating is prevelant in many benchmarking efforts, because 'looking good' is often seen as more important than knowing the truth. One cannot help those who are in denial. Manage the PI not performance!
5. I know of 2 or 3 of these refineries who used the SAI reports to make big improvements, and they shone, in subsequent SAI and internal benchmarks. One refinery for example, got pump MTBFs up from 22 months to 35 months within 2 years. One of them did a dramatic shift in the way they did their shutdowns (turnarounds) with brilliant results. The person who did that is a fellow author in a second book I am writing, and he describes this process and the benchmarking results that woke them up.
6. Unthinking managements do cut manpower numbers based on benchmarking results without paying heed to the way in which these results were achieved. Cherry-picking is a favourite nmanagement hobby-horse. What did they say in 'Looking Glass', Judgement first, never mind the ....!
I can go on about this but I will stop inflicting pain on the forum members. But I would advise you not to scoff at benchmarking; it works, it is a powerful tool if you approach it with an open mind.

Kim,

quote:

We can't just emulate the end-result. We have to emulate the path. Hopefully, if we understand the path we're trying to emulate, we can short-circuit it, and not take as long as they did to get there, but there are pieces that just have to happen.

Bingo! I wish managements would see this basic truth. But you know how Peter's principle works; many top managers would not be there if they were competent!

Kim,
Toyota remains the top auto-maker in the world, by leaps and bounds. In every benchmarking survey they come up tops. If you examine how they handle invoice payments for components delivered for (JIT) parts, you will see what I mean. No amount of massaging input sheets can make GM reach their level.

Two persons gave different results for the same benchmarking survey? Is it possible they were from different parts of the same refinery?

This message has been edited. Last edited by: Josh,

Any person know MERIT project from SGSI?

SGSI (Shell Global Solution Inc.)

quote:

Originally posted by Josh:
Two persons gave different results for the same benchmarking survey? Is it possible they were from different parts of the same refinery?

Josh,

This was a long time ago. Both engineers were experienced on the SAME process unit in the SAME refinery and did the paperwork separately. If I remember correctly, it was an issue of interpretation on the part of the two engineers. I assume that the instructions were not as well refined and specific as they are today.

Kim Williams