A flawed analysis: Withings Sleep Analyzer put to the test

Good sleep is considered the key to a healthy and longer life. It’s no wonder then that more and more gadgets to measure and monitor your sleep are coming out. The Withings sleeping mat has the advantage that it doesn’t need to be directly on your body. I tested it.

Set-up

The Sleep Analyzer comes in snazzy packaging. For a price of around 120 francs or euros (as of October 2024), you get an almost Apple-like unboxing experience. The box is sturdy, has a magnetic closure and is nicely organised, with the quick guide in a compartment in the lid. The mat’s USB cable has two plugs: one for the EU, one type G plug for the UK.

In the quick guide, a QR code leads me to the Withings Health Mate app, which I download onto my iPhone. It requires a user account to work. Some data, such as body weight, age and height, don’t have to be entered again, but can be taken from Apple’s Health app. Withings has integrated things well. Google Fit is apparently also compatible with Withings, but I haven’t tried it.

After software, I turn to the hardware. The Sleep Analyzer’s a fabric mat, a good 60 centimetres long and 20 centimetres wide. The mat is slightly thicker at one end, around five centimetres. That bit contains a flat box in a pocket. A grey cable, about two metres long, hangs from the box. It supplies power for the sleeping mat via USB-A and the supplied plug adapter.

According to the step-by-step instructions in the Withings app, you should place the Sleep Analyzer under your mattress, with the sensor box on the outside of the bed. Ideally, the mat should be at heart level. This only kind of works with my slatted frame. This is because there aren’t any slats around the shoulder area, but a different kind of construction. I’ve done this so my shoulders can sink into the mattress more. If I place the Sleep Analyzer on that bit, it won’t work as well. In the end, I have to position it about a hand’s width too far down.

Calibration begins after another click in the app. During this time you shouldn’t sit on the bed so as not to mess up the pressure sensors. The mat will probably feel my weight later when I go to bed anyway, I think to myself.

That’s it for the installation, now I can start collecting data. The mat under my mattress now measures and records the waves that my heartbeat emits. On top of that, the microphone in the little box listens to hear if I’m snoring.

Night data

After waking up, the Sleep Analyzer connects to my smartphone via Bluetooth and transmits data from the night. This results in a Sleep Score. It’s made up of four factors:

• Duration
• Recovery
• Regularity
• Interruptions

You can achieve a maximum of 100 points. Deductions are made, for example, if you had too few deep sleep phases or too many interruptions in one night. The mat under your mattress also measures whether t tookyou too much time to fall asleep. What it doesn’t know is that, although I was lying in bed, I was still reading my thriller. In these cases you have to make manual corrections if you want to know the «real» Sleep Score. Other gadgets such as smart rings or smart watches are better in this case.

If you snore, this isn’t included in the Sleep Score. On the one hand, this is understandable since there are actually studies that show that sleep with a lot of snoring can also be restful. On the other hand, snoring is an indication of possible sleep apnoea. In addition to the Sleep Score, a sleep apnoea analysis is created in the Withings app. Interruptions in breathing are measured here. Pauses in breathing or phases of greatly reduced breathing are added up. With an average of less than 15 apnoea episodes per hour, everything is fine in my case.

How good is apnoea detection really?

The screenshot above should put my mind at rest. All good, according to the Sleep Analyzer? I doubt how accurate its measurements are. For snoring, I can directly compare with the Sleep Cycle app on my iPhone lying next to me on the bedside table. On some nights, Withings claims I had a completely quiet night without any snoring. This is contradicted by measurements in the app and the most reliable and strictest snoring detector: my wife next to me in bed.

For accuracy in sleep apnoea testing, I give the sleep mat to my colleague Lorenz. He’s diagnosed with sleep apnoea and places the Sleep Analyzer under his mattress. During a night without using his breathing-supporting sleep mask, Withings actually measured more snoring and significantly more interruptions than during the night with it on.

Lorenz doesn’t trust the Withings device to provide a precise sleep apnoea diagnosis. Offers such as Sleepiz (tested here by colleague Patrick Vogt) are much more precise and reliable. For example, many also measure blood oxygen on your finger.