PREFACE: What you are about to read is not your typical review. There are already plenty of sites dedicated to reviewing the basic function of these devices, giving you a run down of the physical specifications. If that is what you are after, check out Fitbit product-related articles on Engadget or Tech Crunch.
Instead this review is going to dig deeper, show results & data compared to other options run in tandem, and discuss the practical use of such data. I’ll apologize now for the verbosity of this post.
MAKING THE PURCHASE
With the holidays around the corner not many subjects are as taboo right now as health, fitness, and diet. Most of us are going to be eating our weight in turkey, stuffing, pecan pie and the likes. This year I plan to partake as I always have, but with a bit more awareness of what is going into my system. Partly due to the calorie-counting diet which I’ve been on since August. And equally because of the gadgets my wife and I just purchased a few days ago.
After some reluctance, I went out this past Friday and picked up a Fitbit Force for myself, a Fitbit Flex for my wife, and a Fitbit Aria for us to share. Being a gadget-geek the idea of wearable tech is great! However I also believe in form following function. Will these devices prove practical? Can they provide data that data benefit our lives in some way?
Prior to the purchase I had already been using my iPhone 5S to track steps. For those unaware, the latest version of the iPhone sports their M7 coprocessor. This allows the phone to collect data just like any other pedometer, such as the Fitbit products listed above or their competitors. However, since many people don’t always have their phone on them or in their pocket, it won’t work for everyone as a good source of data. For example, my wife’s phone is often in her purse, and rarely in her pockets. On the other hand, I carry my phone on me 24/7 even around the house. My wife’s phone was capturing maybe only 10-20% of her data, where as mine was on me 90% of my day (approximations here).
|Pedometer++ Data prior to Fitbit||Argus Data prior to Fitbit|
From my standpoint then, the Fitbit needs to provide me a service better than what I already have on my phone. What do I already have? Well, as you can see from the screen shots above, the Pedometer++ App provides step data, and is currently free. If you have an iPhone 5S then I’d urge you to grab it and check it out, quite fun! While having step info alone is good, the Argus App takes things further, providing caloric data too. It does cost $1.99 — which isn’t free, but still a far cry away from entry level activity trackers.
Hence going into this purchase I had two main thoughts or seemingly simple questions:
- Will the Fitbit hardware provide me data that is somehow more accurate, higher quality, and more useful than what my phone already provides?
- Is this data worthy of the price of admission? Or to put it another way, beyond the function, what are the pros/cons versus just using the M7 processor-based apps?
ACCURACY & RAW DATA – CAN YOU TRUST IT?
Thus far with only THREE complete days of data under my belt the information is rather limited. But before we had even made our purchase I had done my research to find what others had to say. Most recently you can find articles where folks test the Fitbit data against their findings using metabolic rate test comparisons to a more basic distance-based comparison. These two articles are some of the more recent ones but are just the tip of the iceberg. Do a search and you’ll find most of the data support as much as 15% inaccuracy on these devices, with some people finding them to be VERY accurate just as likely.
Worse yet, a lack of history can cause the Fitbit to predict your remaining calories to be burned inaccurately. To avoid this, you can turn OFF the Calorie Estimation feature in settings. Going one step further you can also go into your Food Plan and change your Daily Calorie Estimate Setting from “Personalized” to “Sedentary” to be as conservative as possible. After doing this you are always looking at a worse case scenario, where your remaining estimated calories for the day are simply your BMR.
Prior to the purchase of our Fitbits, my wife and I were already using MyFitnessPal (MFP) to track our calories eaten each day. Having tested a few different calorie counting sites and iPhone apps, my favorite based on their huge database and ease of use remains MFP. And hey have a really nice iPhone App to boot!
Both the Fitbit and MFP can determine your baseline calories per day calculated using a BMR Calculator, and both use the standard Mifflin St Jeor equation. However since MFP has no way of knowing how active you are each day, they instead require you to answer what sort of lifestyle you have and create a Nutrition Plan from there. This works, but you’re not always going to be as active from one day to the next. For example, my BMR + 20% for my sedentary lifestyle awarded me around 1700 calories per day. Am I really sedentary most days? Or am I slightly higher or lower? Does it average out? That is what the Fitbit data needs to tell us.
Sidebar: Over the past four months I used MFP with no activity tracker, and did manage to lose on average a pound per week. That is proof that things do all balance in the long run. However to get a more true day-to-day picture, some sort of caloric burn device is helpful.
Before we dig into the calories burned I want to start with the steps counted. Let’s take a look at the three days of data I have, starting with steps counted by the Fitbit versus the two apps on my iPhone – and see if the issue with accuracy others found are true.
Pedometer++ 3-Days of Steps
Fitbit – Saturday Steps
Argus 3 Days of Steps
Fitbit – Sunday Steps
|Saturday / Sunday / Monday – Comparative Step Data||
Fitbit – Monday Steps
Weekends for me usually involve activities beyond my norm. During the week I’m typically just sitting at my desk at work or performing normal walking around. However, this past Sunday’s activities saw shopping at the mall, for example, which included clothes shopping. Imagine the movements of your wrist as you take clothes on/off your body, or lift items on/off shelves and clothes racks. Although Fitbit claims their sensors won’t usually snap this data, my findings saw otherwise.
When you examine Saturday/Sunday you’ll see a large variance when you compare the Fitbit’s data to the iPhone’s data. Monday, however, shows a much smaller variance.[Update: Tuesday’s data was also within 4% variance iPhone vs Fitbit – Showing that most lazy days might prove similar between devices] This variation in data over the weekend had me questioning the accuracy of the Fitbit.
However my wife pointed out a valid piece of logic: movement of your arms are still indeed movements, and as such does expend some calories. However the question might be if those calories burnt are similar to that of walking. Keep in mind that Fitbit cannot properly record non-walking-type activities. So if you plan to ride a bike you have to log that manually. Monday was the only day where the steps were close enough to call it clear, the other days the Fitbit was recording heavy compared to the M7. And it is worth noting that while there was minimal variation between Argus vs Pedometer++ — the disparity between the two iOS apps was negligible.
Yet it remains obvious as you study the photos above the the Fitbit saw 2000-3000 extra steps (as much as 40% at times!) on Saturday/Sunday even though it remained fairly accurate on Monday where I spent most of my day at a desk. And this could be an issue.
Sidebar: Update: I’ve also recently added my manual Stride information into the Fitbit, with hopes of seeing how that may improve future accuracy.
Since the Fitbit can predict your calories burned based on your activity it is paramount that this device accurately records your activity. If it is grabbing more data than it should does that mean it is also showing more calories burned than actual? Let’s take a look at some specific periods of time, the steps captured at those moments, and the resulting caloric burn.
Here we have the step graph for each of the three days in question, specifically selecting data at the highest peak of each of day. With this data we can try to determine what the Fitbit uses for caloric burn based on steps. Having reviewed what I was doing at these dates/times, I can confirm that I was indeed active and walking at these times but in some cases it was more than just walking. For example, shopping while walking (read: extra arm movements potentially).
Note: Each of these 15 minute intervals will as a minimum show your BMR calories burned. As there are a total of 96 intervals in you can take your BMR (1414 per day for me) divided over 96 periods to yield in my case 14.73 calories doing nothing, which for purposes here I will round to 15. You will see on the calorie page that the LOWEST orange bars are your BMR burn— anything over and beyond that would be extra based on steps.
Starting with Saturday you will see the 15-minute interval chosen shows 63 calories were burned by way of 836 recorded steps. That means we see an extra 48 calories accounted for by Fitbit based on my activity. Divide those numbers and the Fitbit gave me 0.057 calories per step.
Sidebar: The rate of speed you travel matters almost as much as the number of steps you take. Based on my stature it would take me 2,395 steps to walk a mile per this site. If I had walked 836 steps in 15-minutes, that is 3,344 steps per hour. Hence 3344/2395 = 1.4 miles per hour– a nice, slow pace.
Using a distance calculator, a speed of 2mph, and my current weight of 131-lbs those 836 steps should bring a caloric burn of around 21 calories., or 0.025 calories per step. In that case the Fitbit is giving me more than 2x the caloric burn for those steps…. why?!
Sunday had a difference of just 6 steps, for the same value of calories, which makes sense for consistency. And if you analyze the data from Monday in the same way (46 calories burned in 507 steps, 31 extra active calories over BMR), you’ll find that again the Fitbit rate was higher than most online calculators, at a rate of 0.061 calories per step.
Another comparison: Since I have relied heavily on MFP over the past few months, I went onto their site to gather data for steps, distance, and caloric burn. They show that I should burn 147 calories per hour walking at a “slow pace (2mph)” and for me at 2,395 steps per mile, that is 4,790 steps. Taking 147/4790 and you get 0.031 calories per step. This data is VERY similar to the other calculator used, and still once again about HALF what the Fitbit is providing me.
While 18 or 21 calories as an error does not sound like much, keep in mind that was just over a 15-minute period. As you near the end of the day however the figures become amplified. Am I really burning that many calories? If the Fitbit was that far off wouldn’t people complain?
ACTIVITY LEVELS & CALORIC BURN
Various sites provide a breakdown of how many steps slot you into what activity level. This is helpful for answering questions like MFP where it asks if you are Sedentary vs Light Active vs .. and so forth.
These breakdowns are supported by various studies, including a study from 2004, where for example “light/low active” is defined at 5,000 to 7,499 steps per day. Those sites prior and others as well take your BMR plus a percentage to come up with your calories for the day, also known as your total daily energy expenditure (TDEE).
Based on my BMR (1414 calories) my breakdown would be as follows (some rounding):
- Sedentary (<5,000 steps) at 20% = 1700 calories
- Light Active (5,000 to 7,499 steps) at 37.5% = 1945 calories
- Moderately Active (7,500 to 9999 steps) at 55% = 2200 calories
- Heavily Active (10,000+ steps) at 72.5% = 2450 calories
Given this data, here are the three days total steps and total calories per Fitbit:
- Saturday – 7,917 steps (Moderately Active) earned me 2,155 calories
- Sunday – 10,124 steps (Heavily Active) earned me 2,272 calories
- Monday – 4,480 steps (Sedentary) earned me 1,918 calories
If you compare those two lists you’ll realize that the data actually might makes sense. It may not had made sense when you analyze a single 15-minute period as we did– but the end of the day data matches the studies provided. However, how does this compare to what MFP would say for those same tiers?
- MFP’s maintain weight (TDEE) for Sedentary is 1,760 calories
- MFP’s maintain weight (TDEE) for Lightly Active is 1,900 calories
- MFP’s maintain weight (TDEE) for Active is 2,040 calories
Note that Sedentary by MFP’s standards is 160 calories behind Monday’s Fitbit data. Notice also that Sunday’s more heavily active day is 230 calories apart. Now the Fitbit DOES push an “adjustment” over to MFP to account for this, but that brings us back to the big question which is … how accurate is the Fitbit? To answer that question, let us remember that we had the phone to compare.
HOW DOES THE PHONE COMPARE
Since the pedometer app just counts steps, it is only the Argus app from Azumio that gives us any chance of competing with the Fitbit for my attention. For starters, however, it lacks integration with MyFitnessPal. Worse yet it has no calorie tracking at this time. It does let you take photos of your food and even share it to social media, but that is rather pointless if you can’t compare calories in versus out.
Based on this I will instead focus on the data we can compare. We will look at the steps it did count, how many calories it reflects burned, and how that compares to the Fitbit data prior.
Argus – Saturday Calories
Argus – Sunday Calories
Argus – Monday Calories
Based on my weight, which changed slightly when I entered it on Sunday morning, the Argus app dropped my Daily Basal dropped as you can see. For reference, I personally do like the screens on the Argus app but without a doubt it is not as polished feeling as the Fitbit’s interface.
Sidebar: Mentioned here is that I entered my body weight into Argus. Up to thos point I’ve not mentioned the Fitbit Aria — it is a great wifi scale, very nice! If you live inside the Fitbit world (or use apps that can pull data from them), you will never have to manually record your weight. Plus the nice charts/data on the Dashboard makes that device alone worthy of a purchase. For the Argus app they claim the Withings wifi scales integrates with their software, but I have not tested this. Instead my weight was manually recorded into the Argus app, which it does allow you to do.
My steps recorded by Argus using the M7 for the three days above were 5255, 7079, and 4363 as shown in images prior in thos post. That corresponds with the 1626, 1651, and 1590 calories total shown for each day above, or the extra 118, 158, or 97 “Active” calories burned above & beyond my BMR.
First off it is worth noting here that the BMR calculated by the Argus app does NOT match the data from most other sites which use the Mifflin standard. I’ve not been able to figure out what formula they are using or how they are coming to that data, but it is curious why it shows more calories than anyone else. It is not a huge difference, but still a curiosity none the less.
Again using the same distance calorie calculator used before, and assuming my same steps per mile, those three days should have earned me 131, 176, and 108 calories. These are all based on the slowest speed (under 2 mph) same as the Fitbit provided me. So where the BMR numbers run a little high, the conversion of calories seems to run a little bit low, and it may all work out. For comparison then, Argus is providing me 0.025 calories per step, which seems like the “industry standard” for calculated measurements. Is the Fitbit using some more advanced data which inflates their results to higher calories burned?
While you don’t get a web portal from the Azumio folks in comparison to the beautiful Dashboard provided by the Fitbit site, you do still get some pretty decent screen shots for your steps among other things it can view (example below).
|Argus – Steps screen viewing||Argus – Steps screen, when touched|
Both photos here show the same day. One photo is the natural look, and in the other photo showing 4674 step that is me touching the screen on the largest/tallest bar. You can run your finger side-to-side and get a snap-shot of the time of the day, where it will show you the cumulative steps and distance at that moment as well as the time of day.
However unlike the Fitbit, the Argus has no such “time period” breakdown for the daily calories burned per time interval. You can see data per day total, but not per hour, minute, or similar. In contrast, the Fitbit not only shows you a color coding of how active you were in those intervals, it also does appear to do some math with regards to intensity, at least with regards to Very Active Minutes. This may or may not also relate to calories burned, their site does not clarify.
Again this isn’t meant to be a full app review of the Argus. In short I like the app, it is nice looking even if not as clean as the Fitbit. As such there are plenty of other screens that I’m going to ignore for purpose of this article here What matters most is how the data here compares to the data of the Fitbit, for sake of accuracy and other matters.
Since we already know the step counters were off on Saturday & Sunday, it is a given that the caloric burn for those days would be off. But how about Monday? Argus had me taking 4363 steps and burning 1590 total calories. This compares to the Fitbit’s 4480 steps and burning 1918 calories. Why the huge difference?
The answer seems simple- the Fitbit is awarding too many calories for my steps. Or is perhaps the Argus too few.? When you compare the TDEE figures most sites suggest, the Argus data does not match the percentage data usually found online. If indeed sedentary is 20% over BMR then I should have seen around 1800 calories burned on sedentary days. So does that mean the Argus is the one whose figures are incorrect?
However, the calories per step figures that Argus gave were almost identical to MFP and other online sites. This means that the numbers seen on Argus for “Activity” more align with the figures matched to online sources. And the figures that I’ve seen on the Fitbit seem over-inflated in comparison, and that scares me.
If your goal is to make sure the calories you burn match the calories you ingest (plus or minus your goals with regards to weight gain/loss), then it is important as I said prior not just that the step counts are correct, but that the math used to turn that into calories is also correct.
WHERE DO WE GO FROM HERE?
For starters, I plan to share this article with both the folks at Fitbit and Azumio. I’d like to have their technical feedback and opinions.
For the folks at Fitbit:
- Why are your calories per step higher rates than most other sites/data?
- How do you determine calories per time interval, and does speed/step-rate matter?
- Do you feel the total calories burned in a given day is accurate compared to Argus?
For the folks at Azumio (Argus):
- How are you calculating your BMR, which differs from everyone else?
- Do you feel the total calories burned in a given day is accurate compared to Fitbit?
- Do you plan to integrate into MFP or other sites in the future for food recording?
As for why the step counts varied over the weekend, the most logical explanation is hand movements that were not steps. Having completed a hundred steps on my treadmill, both the iPhone and the Fitbit were within 3% accuracy. That leads me to believe that even with their various algorithms used to determine what is or is not a step, the Fitbit still grabs data for movements that are indeed not steps.
For TRUE walking/steps I did find the figures seemed pretty spot on, so I give both the iPhone and the Fitbit credit where credit is due. When you start getting into non-walking movements, however, the Fitbit is going to perhaps provide data you do not want. What matters most is how that plays into your calories, and that can make or break your diet plan.
Would I suggest the Fitbit? For now I feel that I still need more time to gather more data and also would like some answers back from the manufacturers as noted above. Obviously for many people who don’t have an iPhone 5S (or an iPhone at all), they will need a dedicated device to log steps. If that device encourages you to exercise, get up and move more than you had been then yes, go buy one today! Anything to help your health! Ideally you should just be comparing past data and attempting to walk more and exercise in general anyhow.
But as I suspect MANY people purchase this type of device with the intention as I did to record calories burned you have to ask — is the data correct? Since the math used on the Argus app differs from that of the Fitbit app, and it is unclear which is correct. So, the answer to which is better for my purpose of comparison is not so cut-and-dry.
Both options (Argus app vs Fitbit device) can help keep you mindful of your steps, though you can’t argue having a wrist mounted fashion statement is a greater reminder, at least until you start forget it is there as the light weight and small profile might allow. Never the less, there are some pros/cons that make the two differ greatly:
Fitbit Pros & Cons
- PRO: Always on you, don’t have to remember anything, it just works passively
- PRO: Integrates with MFP and other sites to give you best possible food intake data
- PRO: Interface is better on the phone, and the presence of a web dashboard is nice
- CON: Cost may be prohibitive to some
- CON: Have to take it off and charge it roughly once per week
Argus Pros & Cons
- PRO: Cost is much cheaper than most of the wearables on the market right
- PRO: With the M7 built into your phone you’ve already got the hardware needed
- CON: Does not allow you to track food data (calories) ingested
- CON: Only good for iPhone 5S owners, not applicable to anyone without M7
In conclusion, I can see a good case for both applications, and hope to be able to determine more about the accuracy of both the Argus app and the Fitbit device. Based on the information I hope to collect in the weeks to come you can expect an update on this subject right after the New Year. Happy holidays!
[UPDATE – 11/28/2013] – Extra info found…
Found a nice Fitbit Blog Post that is from three years ago. While it doesn’t fully explain how they obtained the caloric burn data, it does essentially say that steps aren’t always “steps” but in some cases calories burned from “any” activity. It also explains how different people would have different burn values.
What it fails to explain to me are two details: first, how did they determine for the figures/samples provided the caloric burn? What instrument(s) did they use to record that data? And secondly, how do they convert “non-step” movements into steps/calories burned? I realize the 2nd question dips into their proprietary algorithms but still some data might help explain WHY we should trust their info over others. Will post more if they reply to some emails I’ve sent off to them. Ciao!