HP Opens Up Its Houston Testing Labs for a Journalist Tour!

Recently Hewlett-Packard invited a group of bloggers into their Houston testing labs. Why do that? That was my question going on the tour – but it was quickly apparent that HP has a lot to be proud of in an area that adds tremendous value to their computers. Let’s take a look with HP at the Science of PC!

Before I get into the actual tour, I have to comment that I found it amazing from start to finish that HP was able to assemble such a tour! I mean, these were active labs we toured with test technicians and engineers and current-generation products and test equipment … and none of us had full NDAs that we were required to sign!

Having been in the equipment and semiconductor industries in the past and currently working for a company where intellectual property is very important – I was really struck by the efforts HP made at accessibility and disclosure. Of course, some visitors wanted to see more cutting edge stuff and were disappointed … but I knew better.

HP Houston Labs005

Kicking off the tour and setting the scene – we would have access to the actual people doing the work and see what they do.

Quick View – What I learned

If I had to list my five biggest learnings, here is what struck me the most:

  • Integration – product testing and quality are ‘in HP’s DNA’, as the saying goes. It is what allowed Compaq to quickly become the top PC maker, and for HP to build such a legacy of great products. It means that quality is considered throughout the design cycle – something more difficult when testing is outsourced like many PC makers are doing now.
  • State-of-the-Art Focus – this is different from ‘bleeding edge’, with more of a focus on understanding what is going to add value to their core customers and how to ensure it will work properly in any number of environments.
  • Technical Breadth – we met with a number of really smart people of all ages and from all types of backgrounds, people who know the science and engineering but also possess the context to apply what they have learned.
  • Passion for Quality – this sounds like a buzzword but it is much more than that. The science of quality and reliability is critical to making products that will withstand the test of time, and having people who have quality and reliability as part of their mantra means that new products will reflect learnings from old failures.
  • Global Reach – with testing a truly global endeavour for HP, the level of integration and cross-communication evident in their operations is amazing. The work of building this into their workstreams has paid off as markets have shifted over the last few decades.
HP Houston Labs009

Just some of the stuff being powered by the ZBook!

HP Houston Labs – The Science of PC

The HP Houston Lab facility is part of the old Compaq complex, and Compaq’s history of quality is what allowed them to become incredibly successful as the first maker of ‘IBM clones’ back in the 80s.

One of the first people we met on the tour was Steve Prato, who was working on Thunderbolt 2 testing and certification. He showed us a Thunderbolt 2 daisy chain set with 8 drives on the cable. Thunderbolt 2 is high performance and low latency, offering the ability for simultaneous multiple-streaming.

HP Houston Labs014

Multiple daisy-chained monitors

He showed the Thunderbolt 2 streaming working on the HP ZBook 15 and 17 – and noted that it wasn’t supported on the ‘Ultrabook’ form-factor ZBook 14 due to the space constraints of getting the daughtercard on the motherboard.

In order to get Thunderbolt into their systems, they need to license the name from Intel and get their products independently certified. This allows them to start with a small offering and evolve into other products over time as they gain more experience. They have already noted interoperability issues between Thunderbolt 1 and 2, which can cause problems and is a reminder that not all standards are standard.

HP Houston Labs016

Steve Hlawek took us further into the testing of the ZBook 15 and 17, looking at the 4k display (gorgeous!) and the Mosaic Multi-display. He showed us a stress test with 2 Intel and 3 Nvidia graphics processors powering 5 displays simultaneously streaming video over Thunderbolt; impressive stuff.

Then we were led through software validation by Pam who heads the international team (lost her last name, sorry!). One of their big metrics is that there is an estimated 133,000 hours of test before release.

HP Houston Labs017

Showing off how the new drive-based encryption works

They start with component level test – some of this is done at suppliers with verification by HP. There was discussion about validation, with Pam talking about sampling and having a long-term history of what types of components are susceptible to which kinds of failures – and what that means for failure analysis and testing. As systems have moved away from mechanical movement (hard drives and optical disks) to more densely packed circuits, the failures have evolved and so has HP’s testing and analysis.

HP Houston Labs025

This is a ‘wall of patents’ – loads of innovation in Houston!

Once the systems begin to get integrated, things get interesting – the team is divided between Houston and locations in Asia, with each having different responsibilities but all working towards the same goals. ‘WAT’ is the term for their integration test efforts, and they are handled through solutions tests with the ISV to make sure subsystems work.

HP Houston Labs027

The products that have come through here – note the Compaq heritage, no early HP stuff here.

Drivers are where things start to get tricky (as any Windows user can attest!) – HP has their components and subsystems tested individually with drivers, then as they are fully integrated into subsystems, and finally as complete product builds. What they have seen is that most of the components and subsystems have matured to the point where they are actually quite reliable.

HP Houston Labs035

Inside an acoustic test lab

Once they are well on the way with their product development they need to get the WHQL certification from Microsoft, which is a mandatory requirement for Windows certification.

The whole process can take about 9 months, and involves a large cross-functional team with leads for each component. There are 10-12 total leads, and then there are integration teams which are all in Asia. It is an incredibly complex cross-function effort that eventually leads to products on shelves.

Inside the acoustic test lab

Inside the acoustic test lab

Data protection has become more critical with each passing year – both in terms of how much critical data we handle and how much of it has potential exposure on the internet. Gordon Cross talked about how prior methods were software based and inherently slowed down the system (personally I remember having data transfers to flash drives taking minutes-per-megabyte), so HP has been working on other methods.

HP Houston Labs039

Hard drive shock and vibration tests

The current scheme is a multi-layer hybrid with the control based in the drive itself. It has an encryption key exposed to the BIOS so it is secure and robust to system failure.

HP Houston Labs042

Crush test for hard drives

Khalid Hameed told us about the HP Stratus, which features the ability to utilize the HP Sure Start technology to detect and correct BIOS corruption. It can use the HP server or a custom corporate server to bring the BIOS back up to date and correct any hardware configuration issues related to the corruption. This all happens independent of the operating system and has no visibility to the actual data on the system, meaning it cannot be used to expose sensitive data.

HP Houston Labs047

Box crush testing

Before we went on an extended tour of the environmental test facilities, Jimmy Holbert gave us some vision of how all of this fits together in the context of HP’s ZBook products. The ZBook is really a mobile workstation and is made in a collaborative effort between the desktop workstation and Elitebook groups, with the result being a sleek business computer with the heart of a desktop replacement and at a price much better than would previously have been possible.

HP Houston Labs052

Environmental chamber

The ZBook series brings in the incredible Thunderbolt 2 technology with 20GB/sec throughput, but is also partially ruggedized and goes through a series of MIL STD tests for dust, spills, vibration, drops and so on. The ZBook 14 is really a separate product, and is a compromise of space and performance, built upon an ultrabook chassis.

HP Houston Labs055

Showing off some of the full-system and packaged test stations

The environmental test facility in Houston has a focus on getting products early according to Bruce Van Alstyne. This is not a full reliability lab, but rather a small-scale initial failure test lab that grabs a small statistical sampling of parts and products to assess likelihood of failure under a number of conditions.

HP Houston Labs056

Hi frequency, low frequency, bumps and bangs – and it just keeps performing!

Some of the tests they perform include packaging tests (box crush); elevation tests (up to 118,000 ft!) which deals with air thickness, impacts of cooling and how it affects optical drives; diaphragm tests for keyboards; corrosion protection tests in a salt box (by ASTM v117 standard.

HP Houston Labs063

Tablet stress testing

Mohammed Chowdry took us into the acoustic test labs. The chamber was amazing to enter, as when he closed the door it felt like all sound was simply sucked out of the air! The room is semi-anechoic (floor is reflective), and is used to check products according to the ISO 7779, 9296 testing standards. The lab tests everything from HP, mostly notebooks but also desktop, workstations, and servers. Servers have more fans and therefore produce more noise.

HP Houston Labs066

Strain measurements in subsystems

Noise issues typically come from fans and drives, but they have historically also had problems with people using cheap-unbalanced disks in their optical drives. The room has a variety of microphones and analyzers for signal and frequency analysis, all laid out and positioned according to the standards.

HP Houston Labs072

FT-IR lab for materials analysis

The acoustic test lab has two purposes: #1 development support, #2 for market certification, data sheet information. It has also been certified as NIST 3rd party lab, independent – gets test data in 3-4 weeks. Everything in the lab is externally calibrated and audited on a 2-year schedule.

HP Houston Labs078

SEM gets to 10,000X magnification!

In the rest of the lab we looked at shock, vibration and packaging tests. Floyd Privette showed off disk drive tests for shock and voibration, as well as drop tests and Instron-based crush tests. John took us through a variety of force and frequency tests for packaged products and some newer precision drop tests for tablets and laptops.

HP Houston Labs081

Racks of circuit boards under extreme test conditions for a very long time

EMC Test Lab

EMC stands for Electro-Magnetic Compliance, and is the ability of a product to properly control & contain radio-frequency emissions leaving or entering the device. Rick Shaw took us through the labs, and showed us some of the compliance and reliability tests they performed.

HP Houston Labs082

Real time X-Ray system

One of the coolest things we saw during the entire tour was the SG-64 test environment. It was a massive EMC ring superstructure in a fully anechoic chamber. It was impossible to get the whole thing in a picture, but check this out:

HP Houston Labs141

The SG-64 EMC Test Chamber

Yeah, I know. But trust me, it is visually impressive (and bonus geek points to anyone who can guess the SG-64 name origins). The room is an advanced setup for doing a pattern test for radio transmitters. This allows for full 3D field emissions tests in a way that wasn’t previously possible. With more and more of our access coming wirelessly, it is important to understand the strengths and weaknesses of components.

HP Houston Labs089

Looking inside the circuit board with the Real-Time X-Ray system

They can also check a variety of other field-emission tests including magnetic fields, ELF, TCL and conducted emissions. There is a radiated emissions lab that tests from 30MHz to 40GHz, and also a massive semi-anechoic chamber that allows for radiated emissions testing, interference analysis, and also immunity, which is a check on what can get into the system.

HP Houston Labs112

Failure analysis testing

Most testing of displays, servers, calculators, and administration is in Houston, with facilities in Roseville and Fort Collins to handle some high-end products and other areas of testing.

HP Houston Labs115

Multi-purpose robot puts systems through the paces

One of the last things we watched was basic grounding checks – a spark-generator around the casing to try to get the system to fail. Again, simple test, but really cool watching the sparks jump.

HP Houston Labs116

Open, close, open, close, open, close, open …

Reliability Lab

Our final visit was to the Reliability lab, where we got to look at a number of tests looking at failures, as well as material analysis and life testing.

HP Houston Labs117

Weighted touchscreen tester

John Olivaris walked us through the materials labs, which are used to look at products and components either in development or after failure. They have a number of tools at their disposal including a FT-IR lab with microscopy, material identification tools and libraries, and an XRF (X-Ray Fluorescence) which is used for metal layer thickness analysis.

HP Houston Labs122

Dual trackpad abuse system!

The lab also has a cross-sectioning microscope for failure analysis as well as a SEM (scanning electron microscope) for resolutions beyond 10,000X magnification. SEM requires a coated sample, making it more difficult to look at metallized surfaces. The lab has a large number of thermal cycling chambers that Michael Fahrenbach showed us, to accelerate any potential temperature-based failures, and also have instrumented many of the tests to observe conduction and other properties in real-time as the tests are conducted.

HP Houston Labs128

Inside the dust test system

Henry Garcia showed off what was my second favorite thing – a real-time X-Ray test. With this they are able to put a full laptop display on the system, scan around, and zoom in real-time. He had a display with a broken connection embedded deep in the layers that would have been very difficult to see any other way. With this system information is gathered and fed back to engineering to determine what happened and how to deal with it in the future. Really cool stuff.

HP Houston Labs130

In-depth materials and components testing and analysis

Other tests they perform in the lab are HALT – highly accelerated life testing, as well as an Instron for normal force displacement testing to see how products perform under normal life forces as well as more aggressive shocks and bending forces.

HP Houston Labs144

Output of the EMC field test system

In the Reliability Engineering Lab, John Vijil took us through some similar tests we had seen before but done here for engineering failure analysis rather than for compliance testing. They have shaker tables for hard drives, and a room for electrostatic discharge, where it took 10000V to shut down system – which then immediately restarted (in other words, it wasn’t fried!)

HP Houston Labs161

Constant set of test packages run as the system is subjected to various external signals

There was another room for small-scale reliability testing of interface components. There was a multi-purpose robot that performs tablet-centric tests like presses, swipes and other gestures. There was also a laptop hinge open-close tester that worked on several laptops simultaneously (for context of why this is important, Google “Nintendo DS hinge crack” or “HP 200 hinge crack”).

They also had a screen press test for laptop touchscreens, a keyboard and mouse test that rubs and looks for print wear, a displacement test for keys, and a touchpad abrasion test that puts the pad through 250,000 cycles! The lab has a number of environmental test chambers, such as a dust chamber and a thermal shock chamber that can cycle from 40C to -40C in 1 second!

HP Houston Labs164

Static discharge gun in use (can just see the spark jump out of the wand coming from the tester’s hand.

These labs are all testing to standards, and work together on failure analysis to do teardowns and corrosion analysis, microscopy, and so on.

Oh, and of course they really took care of us in terms of food as well!

HP Houston Labs168

We were very well taken care of! Thanks HP!


I return to my initial question – WHY would HP want us in their labs? Simple – they are representative of both the history and the leading edge of computer testing and reliability, they have stayed true to their ideals even as the market has shifted and margins have thinned, and they have a commitment to quality and reliability that makes me happy that my company has a HP laptop on my desk at work everyday.

About the Author

Michael Anderson
I have loved technology for as long as I can remember - and have been a computer gamer since the PDP-10! Mobile Technology has played a major role in my life - I have used an electronic companion since the HP95LX more than 20 years ago, and have been a 'Laptop First' person since my Compaq LTE Lite 3/20 and Powerbook 170 back in 1991! As an avid gamer and gadget-junkie I was constantly asked for my opinions on new technology, which led to writing small blurbs ... and eventually becoming a reviewer many years ago. My family is my biggest priority in life, and they alternate between loving and tolerating my gaming and gadget hobbies ... but ultimately benefits from the addition of technology to our lives!