Thursday, October 1, 2009

Captain Fantastic and the Amazing Dew Point Controller

I don't have any extra special new material tonight, except for my unofficial promotion to instrument assembly tech - watch out Jar Jar! I thought that I would go back to my tales of Captain Fantastic. Tonight, I will explore the tale of the dew point control system.

Early in my time at the Biotech Mecca, the Captain had chosen a dispenser to put the samples into the sample holders. Given the fact that the sample is a very small amount of liquid, and the holder has a whole lot of tiny holes, this is not a trivial feat. The Captain knew what he wanted and bought two of them with confidence. That's a quarter of a million dollars spent without any real qualification to see if it would work. In fact, after two or three man years of effort, we determined that the technology was not suitable, and it was a dumb idea that should have never gotten off of the ground. The crappy design of the dispenser itself isn't what I want to talk about though.

The Captain was charged with finding a way to deal with the fact that it took so long to dispense the whopping 1/2ml of liquid (in 5 million individual drops) that the first wells were completely dried out before the last wells were even dispensed. There is a technique for dealing with this that is well known. If you balance the sample temperature and humidity, you can keep the water from evaporating. Most companies include a dew point controller. The people who made this device thought that keeping the sample hydrated was a dumb idea and did neither. We would have to add that capability to the dispenser. Captain Fantastic, however didn't want to void the warranty by adding water. Apparently, adding a thermoelectric cooler and control electronics, a PLC, some wiring, and a new sample stage would have no effect on the warranty.

The first step was to find a PLC. The only requirements were that it had to be able to interpret the data from the humidity sensor and, if possible, fit in the already full dispenser chassis. Never one to miss the critical requirements, Captain Fantastic found a PLC that fit into the chassis. Apparently, the fact that the sensor response was logarithmic, and the PLC had no log function did not matter.

The Captain contracted his favorite consultant to write the code. Unfortunately, Smee had no idea how to program the PLC. So, she hired a consultant to do the programming for her. Unfortunately, the consultant couldn't figure out how to get the communications to the thermoelectric controller to work, so he hired another consultant to do that work for him. So... the Captain hired a consultant, who hired a consultant, who hired a consultant. This seemed a bit odd to me, but he assured me that it was "the Silicon Valley Way."

And then, there was the hardware. For several weeks the Captain had the mechanical designer (Buffalo Bill's predecessor - and successor) design parts and have them made overnight, only to find that they still didn't fit the next day, and repeat the cycle. Our best machine shops will occasionally do us a favor and turn parts around in a couple of days. I wouldn't dream of asking a shop to turn multiple pieces around over night. I would be even less inclined to go back to them the next day and admit that I screwed up and asked for something that was just wrong. And, can you please forsake your family again tonight to make even more parts that will just be scrapped tomorrow morning anyway. Not so for the Captain. The customer is always right, even when he is a stupid jackass.

Eventually, the modified dispenser was due to ship. This was after I suggested to the electrical engineer that the reason why he kept blowing up thermoelectric coolers was because he was running a 12V part at 24V. Apparently, the Captain didn't read the package before handing the specs to the electrical engineer. Who would have figured.

We shipped out the dispenser (to a test sight on the other side of the continent), and the Mad Man from Down South was sent to install it. I guess he drew the short straw. He took some very lovely pictures of the device with things sticking up at funny angles. He commented on how the thing was put together and was stunned that it had worked at all. The Captain was very quick to site "tampering". Really? The VP of engineering went to our first alpha site and screwed with the device? What had he to gain? I don't know, but Captain Fantastic stuck to his guns.

When the Mad Man from Down South came back armed with pictures and props, the story changed. Then, it was blamed on "high frequency vibrations during shipping that vibrated all of the pieces apart". Really? I don't think that I will ever fly again. What keeps all of the bolts from falling out of the plane? Apparently, it's very common. I must admit that I have never hear of such a thing, and I coordinated service for what was possibly the world's most (strike that... second most) ill conceived device. Those things failed in may ways, but they never once failed because the bolts fell out during air travel. I have heard however, that the change in pressure during the flight can screw up pressure sensors. I find that one only slightly more plausible.

After several weeks of the Captain ordering more parts overnight and visiting the customer site himself (on a one-way ticket), the dew point control system kind of worked. Not that it mattered, the dispenser was unable to successfully put the liquid in the holes anyway.

The Mad Man called Captain fantastic a liar. This hurt the Captain's feelings. Nothing hurts like the truth... so they say.

No comments:

Post a Comment