Don’t mention the flaw!

Once upon a time I was posted to a department in an oil company which dealt with the early-stage designs of new installations, much of which was geared towards providing enough information for a cost estimate to be carried out. To a rough order of magnitude, the cost of a new offshore installation (either floating or fixed to the seabed) can be estimated from its weight. Keeping things simple, the weight of an offshore facility comprises Equipment Weight, Piping Weight, Structural Weight, and Others. If you have enough data, it is theoretically possible to work out the total weight of a new offshore installation by taking just the Equipment Weight and applying various ratios from similar, existing facilities. Most large engineering companies do this in order to obtain order-of-magnitude weights and cost estimates, but it is very much a finger-in-the-air approach which, at the early stages of a project, is fine.

The problem with my new department was they did the equivalent of dividing 11.3 by 3.4 and writing the answer as 3.32352941. Any GCSE science or maths teacher will tell you the answer to any calculation cannot be more accurate than the initial input data. But when we did estimates using data with an accuracy of ± 30%, we’d make comparisons of estimates that were within 10% of each other and propose weight savings of 5%. If you think it’s just journalists who are innumerate, be aware there are engineers with the same affliction working in large oil companies.

Then things got a whole lot worse. Weight ratios apply to offshore facilities because they are designed as a single unit relatively unaffected by their location (I’m talking topsides or floaters here, not the jackets or other support structures). I’m simplifying massively, but the point is that the weights of floating and other offshore facilities are not primarily driven by where they are installed. By contrast, the cost and complexity of onshore installations is enormously impacted by topography and geotechnical conditions under the soil. As you can imagine, building a facility on flat, firm ground is a bit easier than doing so on the side of a granite mountain or in a marsh. Civil engineering accounts for approximately 30-40% of the cost of constructing an onshore oil and gas installation, mainly grading the site, bringing in aggregate and compacting, and building the vast underground networks of pipes and cables needed to run the thing. This is why the first things you do when you’re thinking about building an onshore plant is the topographical and geotechnical survey; it’s sort of hard to do anything without it.

But I worked with very clever people, and they came up with a way of estimating the costs of an onshore facility regardless of where it was located. Insofar as topography went we could just assume it was flat, and soil conditions could be ignored or data from a project on another continent used instead. That soil conditions can vary dramatically across a hundred metres didn’t seem to matter. Furthermore, we could use ratios to work out the weights like we did offshore. Now I spied a problem with this. Offshore, on a global basis, there is probably a relationship between Total Equipment Weight and Total Structural Weight; all equipment on such facilities is supported by structural steel, after all. But onshore equipment is generally placed on a concrete plinth sunk into the ground, the size of which is driven by the soil conditions and equipment weight. The structural steel supports some equipment and a lot of piping and cables, but it does a very different job to that on offshore facilities. In many instances, the structural steel around a piece of onshore equipment is negligible. In short, on an onshore plant there is no ratio from other facilities which can be used to estimate structural weight using equipment weight. But here were were, applying the same methodology as if it could.

Having some experience on onshore sites, I began to use my noggin a little. In one estimate, I ascertained that a vessel had no structural steel at all: it rested on its own legs and there was no maintainable valve on top which would need an access platform. But two managers queried this: they asked how the structural steel weight could be zero. I said it was because there is no structure associated with this vessel. They said this must be wrong, and I should apply a ratio of 30% vessel weight. So I asked them what structure they thought I was missing. They couldn’t say, but they told me to add the weight in, which came to several tonnes.

A little later, they got an intern with no post-graduate engineering experience to create a formal procedure for estimating the weights of onshore facilities, convinced that from such data the costs could be derived. They then passed it around all the engineers for comments. I noticed that it did not consider many components of the underground networks, which as I said comprises a huge portion of the costs. The most glaring omission was the firewater ring main, which is big, expensive, and common to all onshore oil and gas facilities. The reason this wasn’t included was because it would be designed “later”, which I found actually meant “nobody here knows anything about firewater ring mains so it’s best to pretend they don’t exist”.

I’d only been in the department a few weeks and I naively thought I’d be being helpful by pointing out, as I have done above, why this new methodology drawn up by the intern was fatally flawed. I drafted a comprehensive email with examples and explanations and sent it to my boss and the head of department, whose brainchild this new methodology was. A few days later I was called into an office where both of them were waiting and told to close the door. Their talk with me can be summarised as follows:

“We have read your email, but the decision has been made to adopt this methodology going forward. Your job is to follow it without asking questions.”

This was probably the first time it dawned on me that in many corporate departments results are meaningless, and all that matters is people obediently follow the process. I fought it for about a year, then just got with the program and pumped out absolute garbage which got wrapped up in more garbage and presented to senior management right up to the CEO. It didn’t take me long to work out whatever rubbish we were generating was not the basis on which decisions were getting made – the company wouldn’t be in business if that were the case – and the entire process, which cost millions of dollars, was merely to keep people employed. I once remarked in the wake of the oil price crash that if the company wanted to cut costs they could get rid of our entire department and employ a child to roll dice every time senior management wanted figures. That went down about as well as my critique of the estimation methodology.

The experience left me wondering how much of this sort of thing goes on in major corporations with names you’ve heard of. Quite a bit, would be my guess.

Liked it? Take a second to support Tim Newman on Patreon!
Share

23 thoughts on “Don’t mention the flaw!

  1. In most organisations a lot of formal processes exist to create a paper trail showing that some form of due diligence is theoretically being applied. That stuff gets done properly is down to having enough competent people, who are not total head cases, dotted around the actual process to make it work. It is true that there are entire cottage industries dedicated to having such processes, but that is the nature of bureaucracy.

  2. A long time ago, I worked in avionics, which was undergoing a cultural shock as software appeared. Reliability being quite important in aircraft, the issue was, how to apply reliability metrics to software?

    For hardware, there were long established rules that worked. Mostly.
    A resistor, of a particular type and manufacturing quality, would be extensively tested, and found to have an average life of a million hours (for example).
    MTBF= 1 million hours.
    All components had such figures,, and these figures were revised by live experience and retesting periodically.

    Hence a system containing lots of components is just a statistical sum of all the individual MTBF contributions. (Keeping it simple, skipping bathtubs and burn-ins etc, 1000 resistors = 1000 hours, etc).

    But software has no failure mechanisms. None at all. It’s reliability is 100%, always.
    Every time that software misbehaves, you are discovering a DESIGN problem, not a service life defect. It’s there from the beginning, and all redundant instances will behave the same way to the same input.
    (Multiple independent development is little or no help by the way, though it increases the bid cost and kept us lot employed).

    Extensive theoretical papers were written and published that would identify this issue, yet within 3 paragraphs, in the absence of any better idea, would be applying standard service life statistics to “Software Reliability”.

    The numbers looked good, never mind the validity.
    So I guess they still do this. Enjoy your flight.

  3. Large onshore plant that I was involved with fairly recently was installed on a greenfield site in the tropics with no sewage system or closed drain system on the plans. The contractor installed as per the drawings without question. Took a bit of putting right…
    Then we found that cathodic protection was missing from big sections of the pipeline… and then… etc

  4. @Tim the Coder

    Ariane 5 is probably one of the more spectacular examples of the problem you describe! IIRC there were two or three parallel software modules, unfortunately they all crashed at the same time, due to the same buggy code…

  5. You might be missing the point of the department. If your goal is to produce accurate engineering designs for people in the field, then yes, accuracy matters. But if your goal is merely to secure funding for your project (whether internal, market, or government), then the accuracy can be wildly out.

    I learned this while working for a consultancy providing questionable data models to a government agency. The models were ok, but minor tweaks to the input parameters could yield wildly different outputs, and the input data was of poor quality. We then enjoyed seeing our wild guesses turn up in the press as “expert analysis”.

    In the O&G business, I often wondered how the installation engineers could take our theoretical plans and just implement it despite our lack of on-the-ground knowledge.

    I’m now in a new field in a large corporate environment, where once again we’re making decisions with very little knowledge of the reality on the ground. Seems to be par for the course.

  6. You might be missing the point of the department.

    Yeah, maybe I’m not.

    But if your goal is merely to secure funding for your project (whether internal, market, or government), then the accuracy can be wildly out.

    That’s fine, and that’s what our department should have been doing. Instead, we were spending millions of dollars trying to design various components to a level of detail wholly inappropriate for the purpose, estimate their weights, and treat the output as if they were the results of accurate calculations. One of the worst aspects of this is it was misleading in the extreme: if you produce a drawing or datasheet with very accurate numbers on and that gets passed around, people will assume it’s been calculated properly rather than estimated or, more usually, copied from somewhere else.

  7. “The problem with my new department was they did the equivalent of dividing 11.3 by 3.4 and writing the answer as 3.32352941. Any GCSE science or maths teacher will tell you the answer to any calculation cannot be more accurate than the initial input data.”

    In an engineering situation, where the numbers are representations of actual things (and therefore are estimations of an actual reality), yes, in a pure maths sense, no.

  8. Departments gain a life after creation. The original point of the department may have been to provide orders of magnitude estimates to prioritise funding allocations.

    However, once more than one or two members of staff have been recruited, the boss has been given an important job title, an operating budget has been allocated and an HR business partner given responsibility for the team, anything that questions a pull back of responsibilities is heresy.

    The only time this can then be challenged or reversed is after a recession or an industry/company version of one AND if an outsider with little or no personal history or relationship with the department is appointed in a position above.

    Versions of this story is how old, large and seemingly solid companies find themselves in receivership.

  9. However, once more than one or two members of staff have been recruited, the boss has been given an important job title, an operating budget has been allocated and an HR business partner given responsibility for the team, anything that questions a pull back of responsibilities is heresy.

    That definitely happens, but what also clobbered the oil companies is a lot of old-school experienced people retired and were replaced with bright young things who knew what to say to their managers but not much else.

  10. It goes on a lot in IT services cost estimating, whether the cost model is top-down or bottom-up.

    It’s backed up, of course, by metrics sourced from IDC, Gartner et al, which are usually 10 years out of date by the time they make it into the model, and take no account of how things have moved on in technology and methodology.

    When I first entered that world, I used to produce my work by blood, sweat and tears, with evidence and analysis.

    By the time I left that world, I was doing it on the back of a fag packet then dreaming up gobbledegook to support it and spending the rest of the day on the golf course.

    Governance was a rigorous process of box-ticking and self-regarding-make-work-idiot mollifying.

    My change of tack was of absolutely no detriment to the business, of course, since every deal got sliced up and settled during talks between C-level stakeholders in our company and the client’s with no regard to anything that had gone before.

    And this is before we get to the fact that client and provider both seem to understand that the winning bidder would be the one that signed up to delivering the moon on a stick for a price that would buy neither component, then the game of contract management and change could begin as soon as the ink was dry.

    Literally nothing we did mattered, so I eventually responded accordingly.

  11. This problem is universal. From what I’ve seen, the primary cause is that executives ask their subordinate managers for information; and the managers – wanting to impress the executive by massively over-delivering – make their staff jump through an inordinate number of completely worthless hoops.

    In a truly dysfunctional organization, offhand comments or rhetorical questions in a meeting will spur multi-week projects by someone looking to kiss ass.

    In the end an exec spends five minutes perusing a colorful deck of slides he doesn’t really care about; completely unaware that it took weeks of man-hours and was produced entirely for his consumption.

    There can be other causes as well, I’ve noticed most people like to gather as much information as possible before making decisions; even when the information is unreliable or unrelated.

  12. In a truly dysfunctional organization, offhand comments or rhetorical questions in a meeting will spur multi-week projects by someone looking to kiss ass.

    I have seen just that. A senior manager asked a question which I thought was pretty much rhetorical and we spent a million dollars answering it. Hell, we’d spend a thousands investigating ludicrous scenarios *just in case* a senior manager asked about it. And some of them really were ludicrous.

    In the end an exec spends five minutes perusing a colorful deck of slides he doesn’t really care about; completely unaware that it took weeks of man-hours and was produced entirely for his consumption.

    Exactly.

  13. …the entire process, which cost millions of dollars, was merely to keep people employed.

    I don’t doubt that this is true, but I’m never quite sure how or why it happens or is allowed to continue.

    I’m sure we’d all like to believe that we’re the rare type of genius that can spot that this is all nonsense, but surely that’s bullshit, and most of the other employees can see it too – including the bigwigs and shareholders.

    So why doesn’t someone try to save millions of dollars? It’s a business, not a jobs program.

    I heard this author on a podcast – he’d written a book called Bullshit Jobs – and one of his ideas was that the muckity-mucks can measure their status in their number of employees, i.e. the size of their retinues (he even called it “managerial feudalism”). Is that all that’s going on?

    More and more, I’m starting to think that you can stop just about any enterprise in its tracks by asking basic questions like, “Why are we doing this, exactly?” and “What, precisely, are we hoping to accomplish here?” – because nobody actually knows what the reasons or goals are, they’re just going through the motions. Doesn’t bode well for the future if true, does it?

  14. One place I worked, the department heads would be gathered regularly, at least weekly, to discuss some or other bid. It wasn’t long before we got wise to the CEO’s tactic of demanding we shave 30% off our costs every single time. Result: everyone inflated their initial estimate to account for the demand to cut the bid by 30%.

  15. Tim, here’s a couple of links you may enjoy. The AEI article links to the Forbes article.

    https://www.aei.org/publication/we-are-not-a-dashboard-contesting-the-tyranny-of-metrics-measurement-and-managerialism/

    https://www.forbes.com/sites/davidshaywitz/2011/06/23/our-metrics-fetish-and-what-to-do-about-it/#71d438875371

    I can’t remember who it was said “All is number” but as long as you get back to 42 you’re pretty close to home plate.

  16. one of his ideas was that the muckity-mucks can measure their status in their number of employees, i.e. the size of their retinues (he even called it “managerial feudalism”). Is that all that’s going on?

    That’s not all of it, but it is part of it. Human nature wants people to be seen as big bosses, and so they embark on a campaign of empire building. I saw this in Nigeria as our subsidiary grew, as each manager campaigned for more staff they didn’t need because being a boss of 15 people looks better than managing a department of 5. Things aren’t helped by the fact that the Hay system of job evaluation, which many companies use to determine job position and hence pay, considers number of reports (the greater, the higher the job position).

    So why doesn’t someone try to save millions of dollars? It’s a business, not a jobs program.

    Most large corporations are actually both. You also need to look at the incentives each individual manager faces.

  17. In the beginning was the DEMO Project. And the Project was without form. And darkness was upon the staff members thereof. So did draw lots and he that hath received the short straw did depart and he spake unto their Division Head, and saith, “It is a crock of shit, and it stinks.”

    And the Division Head spake unto his Department Head, and saith, “It is a crock of excrement and none may abide the odor thereof.” Now, the Department Head spake unto his Directorate Head, and saith, “It is a container of excrement, and is very strong, such that none may abide before it.” And it came to pass that the Directorate Head spake unto the Assistant Technical Director, and saith, “It is a vessel of fertilizer and none may abide by its strength.”

    And the assistant Technical Director spake thus unto the Technical Director, and saith, “It containeth that which aids growth and it is very strong.” And, Lo, the Technical Director spake then unto the Captain, and saith, “The powerful new Project will help promote the growth of the Laboratories.”

    And the Captain looked down upon the Project, and He saw that
    it was Good!

  18. Most large corporations are actually both.

    Yeah, but… how? Exactly?

    This is what I can’t get my head around.

    We can surely at least agree that, to the extent that a major corporation is a jobs program, it doesn’t perceive itself as such. Right?

    I mean, the board, the shareholders, the bigwigs – they aren’t aware that they’re paying thousands of people to dig holes and then fill ’em in again, right? Or at least, they think such activity is actually important, right?

  19. We can surely at least agree that, to the extent that a major corporation is a jobs program, it doesn’t perceive itself as such. Right?

    Like I said, yes and no. 🙂

    Take Total, for example. They didn’t lay off a single French worker when the oil price crashed in 2013 when their peers were getting rid of thousands, because in France it’s not politically expedient to do so. Sure, there’s the unions to get around but it was equally the case that it would have looked very bad for the government to suddenly find a few thousand middle class, white collar engineers without a job (and unemployable elsewhere). Total maintains two large offices in France, one in Paris the other in Pau (a legacy of the Total-Elf merger). They should have closed one or the other years ago, but they can’t due to political concerns. If the one in Pau closed, unemployment in that town would go through the roof as Total dominates there. In many ways, Total exists in part to provide employment to the graduates of France’s engineering schools; they could easily get rid of a thousand people but in France a company’s role is as much to provide employment as a return to shareholders.

    Do the employees realise this? Not really. If the system is big enough, people can’t see the wood for the trees, and one thing which my observations over the years has convinced me of, it’s people’s ability to sincerely believe the non-job they are doing is a matter of life and death. Carried on long enough, I don’t think even the senior management realise it. I’ve seen *very* senior managers commission work which I knew would get cancelled as soon as anyone outside our company got wind of it, and it’s obvious it’s just an excuse to keep everyone looking busy. It’s easier than firing people, after all.

  20. It’s that old Venn diagram thing of Your Interests, Your Manager’s interests, and the Board/Company’s Interests.
    The overlaps are quite small.

    The Manager’s job, power and promotion, depend on a large team doing something. it doesn’t matter what. So he/she/xe will be sure to build such a team and keep them busy.

    The individual’s strategy need to be to identify which, of any of the tasks actually matter, and do those properly, while polishing the turds for the other make work tasks. Presentation for those is more important than content.

    That’s based on a lifetime in safety critical & life critical engineering, so I guess the same applies everywhere….Fill in the crater and redo. Lessons will be learned.

    Only small privately owned businesses have any real focus,because the Venn starts overlapping more.

  21. Re: Total: but is that the same thing?

    In that scenario, they’d like to fire their workers, but it’d be more trouble than it’s worth. It’s like when your rich client insists you give his idiot nephew a job: it’s the cost of doing business.

    With the idiot nephew, you stuff him somewhere out of the way where he can’t do too much damage and you give him something that makes him feel important.

    With thousands of trained engineers, don’t you try to find something productive for them to do? I dunno, figuring out cheaper ways to connect danglefwerters to confrabulators?

    Or is that the trap – thinking that there’s anything productive these engineers could be doing? Because, to continue the analogy, the current method for connecting danglefwerters to confrabulators works fine and costs barely anything, so rejigging it is more akin to reinventing the wheel, i.e., pointless make-work?

    To put it another way, is it akin to assuming that the make-work you’ve given to your client’s idiot nephew is actually important?

    Thinking out loud: I suppose the nephew probably thinks it is, and will drag that impression with him as he rises through the ranks – and people who join the company after he was hired will assume so – and the people who are already at the company when he’s hired will have to be told that it’s important, because otherwise either they’d be resentful and the nephew would be a laughing stock, which would upset his uncle, your biggest client… so, what, probably very few people actually understand that his job is a waste of time, then. Going back to something I said earlier:

    I’m sure we’d all like to believe that we’re the rare type of genius that can spot that this is all nonsense, but surely that’s bullshit, and most of the other employees can see it too – including the bigwigs and shareholders.

    Perhaps that’s where I’m going wrong – maybe most people can’t see it.

Comments are closed.