Expected CdA Precision

[nikshar]

I'm curious about the kind of precision other users are seeing in their CdA testing. I've read of +-0.002 in a couple of places in this forum, but haven't seen the data that goes with it - I'm more interested in understanding all the different parameters that went with such precision.

The best I'm seeing (when my test route had 0 cars/bikes interference but had some non-gusty tail/head-winds, with a 4.5 minute out-back loop) is around +-0.005 for the 95% confidence interval. Which, roughly speaking, makes it so that I can't really tell apart configurations that are at 10-15 W of difference.

Today I tested the difference between raising my stack height by 6cm. The analyzed CdA values I got for baseline were 0.185, 0.181, 0.171, 0.172, 0.178, 0.178 and that for the higher stack height were 0.178, 0.180, 0.171, 0.175.

[lorduintah]

You might consider your reproducibility, not the device as a source of variation?

[nikshar]

The variables under my control, to ensure reproducibility, are:

• Using the same test route: I did that, made sure to start and stop at the exact point

• Holding the same position: I did that too (made sure to hold the position for all of those 4.5 minute intervals) to an extent that cannot give a 10W difference.

• Power: I rode all the 13 laps by feel. When I look at the data, the average power was within +-6W across the board, w/o any accelerations. That must be consistent enough

The point I'm trying to make is, that in my point of view, I tested in a consistent enough way that could explain maybe 2-3 W of variance out of it, but not the kind I'm seeing. Which is also why I was seeking to see data of people who've got that +-0.002 CdA variance, to see what other conditions are required to get that level of accuracy

[Velocomp]

I'm curious about the kind of precision other users are seeing in their CdA testing. I've read of +-0.002 in a couple of places in this forum, but haven't seen the data that goes with it - I'm more interested in understanding all the different parameters that went with such precision.

The best I'm seeing (when my test route had 0 cars/bikes interference but had some non-gusty tail/head-winds, with a 4.5 minute out-back loop) is around +-0.005 for the 95% confidence interval. Which, roughly speaking, makes it so that I can't really tell apart configurations that are at 10-15 W of difference.

Today I tested the difference between raising my stack height by 6cm. The analyzed CdA values I got for baseline were 0.185, 0.181, 0.171, 0.172, 0.178, 0.178 and that for the higher stack height were 0.178, 0.180, 0.171, 0.175.

Please post ride file

[nikshar]

Attaching two files, thanks John

Context:
Test-A: Baseline
Test-B: 6cm higher stack compared to Test-A
Test-C: Same as Test-A but with aero baselayer, different skinsuit, and calf sleeves

(File 1 - 27 miles)
- Laps 4, 6, 8, 10, 12 - Test-A
- Laps 14, 16, 18, 20 - Test-B

<Downloaded ride file to laptop, w/o removing aeropod>

(File 2 - 11 miles)
- Laps 2, 4, 6 - Test-A
- Laps 8, 10 - Test-C

FWIW, I've noticed two inconsistencies in the VA graphs wrt the turn-around point:
1. On some laps it does not detect/report "Braking" (in the Elevation row)
2. The slope it reports around that point is inconsistent in both, magnitude and direction

I've noticed this across a bunch of different tests, and I'm not sure if this is affecting my results

[HDPWR]

One factor is the wind speed and wind direction, especially direction with a higher yaw angle. Another factor would be tire pressure (have you got a digital pressure meter device?, is the pressure constant?)

[nikshar]

Hi John

I did some more digging into the data and found something interesting. At first I noticed that the CdA values reported in the table were more consistent than the corresponding Analyzed CdA values. Then I took a step further and split each lap into two sections - (1) from the start to just before the turn around, and (2) from a few seconds after the turn around to the end. I used the corresponding CdA values (from the table) for both sections and computed a weighted average (weighed by distance, which I got from the csv file for more precision). This new value generally seems to be more consistent than the analyzed value, especially when I compare the 2nd set of Test-A runs with the 1st set of Test-A runs.

These are the results I found, listed in the order in which the laps were performed. Each row has 3 CdA values - CdA from the new method, CdA from the table, Analyzed CdA

• Test-A, 0.187, 0.18, 0.185
• Test-A, 0.178, 0.176, 0.181
• Test-A, 0.179, 0.182, 0.171
• Test-A, 0.179, 0.177, 0.172
• Test-B, 0.191, 0.186, 0.18
• Test-B, 0.189, 0.192, 0.171
• Test-B, 0.19, 0.187, 0.175
• Test-A, 0.175, 0.182, 0.178
• Test-A, 0.183, 0.184, 0.178

Standard Deviations
Test-A: 0.0038, 0.0029, 0.2246
Test-B: 0.0008, 0.0026, 0.0037


I also tested this on my file from a ride a couple weeks ago, and I see similar trends.

• Test-A, 0.219, 0.221, 0.24
• Test-A, 0.222, 0.222, 0.236
• Test-A, 0.222, 0.222, 0.239
• Test-B, 0.182, 0.207, 0.243
• Test-B, 0.219, 0.229, 0.253
• Test-B, 0.228, 0.227, 0.244
• Test-B, 0.226, 0.226, 0.248
• Test-A, 0.22, 0.215, 0.25
• Test-A, 0.222, 0.227, 0.253
• Test-A, 0.218, 0.211, 0.239
• Test-C, 0.219, 0.226, 0.244
• Test-C, 0.219, 0.214, 0.246
• Test-C, 0.221, 0.227, 0.249
• Test-C, 0.226, 0.226, 0.253
• Test-C, 0.227, 0.228, 0.251

Standard Deviations
Test-A: 0.0016, 0.0052, 0.0063
Test-B: 0.0186, 0.0089, 0.0039
Test-C: 0.0034, 0.0052, 0.0033


Thoughts?

[nikshar]

One factor is the wind speed and wind direction, especially direction with a higher yaw angle. Another factor would be tire pressure (have you got a digital pressure meter device?, is the pressure constant?)

Yeah, it seems like wind is likely the biggest cause of variation. It's pretty much impossible to find a location that has close to no winds while simultaneously being in a place free of traffic (those places tend to be out in the open, away from the cities, so there's always wind).

For tire pressure, I pump my tires occasionally, but I don't think that should matter here. I'm only interested in A/B testing on the same day, for which the tire pressure should be ~constant for the duration of the test. It might go down by something like 2-3 psi over a couple hours, maybe, but that's not enough to impact rolling resistance in any meaningful way.
And I'm not interested in comparing CdA across days, so any CdA changes that come as a result of a tire pressure change (therefore, rolling resistance) would not matter.

[HDPWR]

One factor is the wind speed and wind direction, especially direction with a higher yaw angle. Another factor would be tire pressure (have you got a digital pressure meter device?, is the pressure constant?)

Yeah, it seems like wind is likely the biggest cause of variation. It's pretty much impossible to find a location that has close to no winds while simultaneously being in a place free of traffic (those places tend to be out in the open, away from the cities, so there's always wind).

For tire pressure, I pump my tires occasionally, but I don't think that should matter here. I'm only interested in A/B testing on the same day, for which the tire pressure should be ~constant for the duration of the test. It might go down by something like 2-3 psi over a couple hours, maybe, but that's not enough to impact rolling resistance in any meaningful way.
And I'm not interested in comparing CdA across days, so any CdA changes that come as a result of a tire pressure change (therefore, rolling resistance) would not matter.

In this case pressure cannot be a factor, that is correct. Just be sure there is no slow leakage.

Regarding your previous analyzis with cutting the laps: I did something very similar in the past and had a very long, partly private conversation with John about my findings. Only outcome was a couple of bug fixes in the VA software. John is very sure about their Profile 4 out-and-back testing protocol, while I am against the method of recording the turnarounds and using them in the calculations. All my recorded laps contain a small but long bump in the CdA graph at the turnaround. You can find the corresponding topic here: https://velocompforum.com/viewtopic.php?f=98&t=6080

[Velocomp]

I know I need to weigh-in on this topic...there are lots of good comments and observations.

I'm working on something right now that all of you will know about in a day or so...until then, keep the topic going and the ride files coming!

[nikshar]

Hi John,

Just to make sure these factors aren't contributing to any unwanted errors, can you confirm a couple of things:

1. Is my device location and angle, as shown in the photos, fine?
2. One of my testing sites is a bike path next to a freeway. I wonder if that's causing any unwanted wind turbulence. Here's a link to the street view, do you think this location is fine? https://maps.app.goo.gl/NENMgwh5SdY5Dihn7?g_st=ac

Thank you!

[nikshar]

Sharing another test file from today, with some more observations.

I only tested one position, I just wanted to test for consistency of CdA over laps.
I redid the calibration process before testing. In the attached file, relevant laps are 2 (warmup), 4, 6, 8, 11. Then I rode somewhere else and came back to the same course an hour later, for which the laps are 19 (warmup), 21, 23.

The analyzed CdA over the individual lap varies significantly (it almost has a pattern of high, low, high, low), but what I found interesting was that the analyzed CdA when grouping laps 4 to 11 (with the turn around laps) was 0.23, and that when grouping laps 21 to 23 was also 0.23.

Like I mentioned, I didn't change anything across all the test runs, so the 0.23 vs 0.23 is a perfect match, but I can't trust it given the variance of the individual analyzed lap values. Thoughts?

Another interesting observation was comparing laps 4 and 6. The average power was 178 W vs 175 W, lap time was 02:59 vs 03:00, but the analyzed CdA was 0.24 vs 0.261! That's like 20 W of difference!

[Velocomp]

Your AP is pointed down more than is recommended. I would get it eye-ball level, then lock it in to make sure it can't rotate.

[nikshar]

I know we're talking about this in the other thread, but I wanted to mention that I observe that effect of turn-around on the analyzed CdA on the file I shared above too.

Analyzed CdA of a group of laps that include test laps and intermittent turn-around laps (< 0.5 miles) is not equal to the average of the analyzed CdAs of the individual test laps.

[Velocomp]

I know we're talking about this in the other thread, but I wanted to mention that I observe that effect of turn-around on the analyzed CdA on the file I shared above too.

Analyzed CdA of a group of laps that include test laps and intermittent turn-around laps (< 0.5 miles) is not equal to the average of the analyzed CdAs of the individual test laps.

We are taking another look at this...

[nikshar]

Sharing yet another file from today.

Config A: Laps 2, 4, 6, 8, 10, <gap> Laps 23, 25, 27, 29
Config B: Laps 12, 14, 16, 18, 20, <gap> Laps 39, 41, 43, 45
Config C: Laps 31, 33, 35, 37

The intra-lap and intra-group variances of analyzed CdA, both were high.
As an example, for Config A:
Laps 2 to 10 had 0.25, 0.244, 0.252, 0.272, 0.252 (same alternate high-low pattern as from yesterday)
Laps 23 to 29 had 0.244, 0.237, 0.253, 0.26

[Velocomp]

Sharing yet another file from today.

Config A: Laps 2, 4, 6, 8, 10, <gap> Laps 23, 25, 27, 29
Config B: Laps 12, 14, 16, 18, 20, <gap> Laps 39, 41, 43, 45
Config C: Laps 31, 33, 35, 37

The intra-lap and intra-group variances of analyzed CdA, both were high.
As an example, for Config A:
Laps 2 to 10 had 0.25, 0.244, 0.252, 0.272, 0.252 (same alternate high-low pattern as from yesterday)
Laps 23 to 29 had 0.244, 0.237, 0.253, 0.26

Do you have a .FIT file for this ride? It would be nice to have GPS information...

[nikshar]

Here's the .fit

[nikshar]

Hi John,

Do you have any updates to share?

Thank you

[Velocomp]

Hi John,

Do you have any updates to share?

Thank you

Yes, actually, quite a few...

1) We just released VA version 1.026, which adds many new features for aero analysis:

A. You can select individual laps, or combination of laps, for aero analysis. Click on the "box" for each lap you want to include in the analysis. To clear the selection click "Reset" at the top right of the screen
B. Turn around laps are clearly marked with "N/A" and are not included in the calculations

1.png (136.85 KiB) Viewed 114 times

C. If your complete ride file is for one configuration only, highlight a lap (don't click on it) then right click. You will see an option to "Select all Aero Laps". Selecting this will automatically find and select all aero laps for the ride file (if a lap is selected you don't want to use, click on the lap number to deselect it)

2.png (13.17 KiB) Viewed 114 times

2) It's critical to understand that aero measurements done outdoors are complicated by variations in ambient wind speed and direction, road surface type, and road slope. If you look at competitor websites, their testing is generally taken in velodromes or even abandoned tunnels. Why? Because ambient wind, road surface, slopes are all known and highly controlled. In other words: they don't show how their products perform in the real world.

3) All of the ride files I've reviewed, both in this thread and others, are from real-world testing. I very much appreciate the effort and discipline reflected in them. I've also done more testing of my own, on my real-world outdoor route (more on that in a minute).

4) In outdoor aero testing, highest accuracy and consistency is achieved when a) the route has minimal changes in elevation; and b) when ambient winds are relatively calm.

5) Ambient wind speed and wind direction, are two factors not to be ignored. If you’re riding in ambient winds of 5 mph or more, you will get greater variation in results, both because ambient wind speed will gust to higher (or lower) values around the average, and because wind direction will likely be changing too. On a reasonably flat route, if you’re seeing significant changes in bike speed between the out and back portions of your route, then wind speed is the likely culprit.

5) A final, critical thing to understand: more data is better, particularly when riding in the open outdoors! While it’s tempting to analyze CdA variations between individual laps, they really aren’t terribly relevant. Wind gusts, small differences in ride position, bike speed variation can all have an effect on CdA measurement, resulting in lap-to-lap variations in CdA. I recommend 3 or more laps to average out these small changes.

In the aero testing I do I am able to enjoy both a flat route, and relatively low ambient winds.
A recent ride file is attached. You’ll be able to see my out-and-back route, and also the “turnaround” laps between tests. Note that lap “1” was around the loop, about a 5 minute warmup. My route was only 1.3 miles long in total; not a huge amount. I did not ride at super-fast speeds or power levels, because that is not necessary for testing. You can also see I tried to maintain consistent speed/power levels while testing. Note finally that the final lap 14 occurred after a complete stop. Because this is a different protocol from my prior laps, I deselected that lap (though it has only minor effect on overall CdA).

3.png (733.67 KiB) Viewed 114 times

My measured CdA is 0.470 for this set of tests; on a different day it was 0.471. (definitely not a podium-winning CdA).

I will comment on your other ride files in the next post.

APTTP4_10_02_2025_1053_9_Miles.ibr

(651.93 KiB) Downloaded 3 times