Reliability and Validity of Clinical Measures of Assessing Running Step Rate

Despite the numerous health benefits of running as a form of exercise, [1,2] Running Related Injuries (RRIs) are common with an incidence of upwards of 90% [3]. Many factors have been shown to contribute to RRIs including age [4], body composition [5], running experience [6], training habits [7] and movement biomechanics [8]. Step rate, defined as the number of foot contacts per minute, is a commonly studied biomechanical variable that can assist clinicians in extrapolating the forces associated with running [9,10] and help identify those at a greater injury risk [11]. Furthermore, increasing a runner’s preferred step rate by 5-10% can decrease stresses incurred at the hip and knee [12] and help in the injury management of runners with distinct biomechanical profiles [13]. 

High-speed motion analysis and instrumented treadmills are considered the gold standards of measuring running step rate. As these technologies are costly, time intensive, and require operator expertise they are often reserved for laboratory studies. Clinical alternatives for assessing step rate have been developed including fitness watches [14], wireless instrumented insoles [15], commercial grade accelerometers [16] and smart phone applications. Despite being more clinically practical, these technologies still have associated costs, may not be readily accessible to clinicians or coaches, or lack research validation. 

Real-time counting has been advocated as an acceptable and easily applied method of assessing walking step rate as it has no associated costs or instrumentation requisites. Due to the greater speeds and higher step rates associated with treadmill running, slow-motion, two-dimensional video analysis of step rate is utilized clinically. Despite it being a reliable methodology [17], slow-motion, frame-by-frame video analysis requires more time for video capture setup and asynchronous viewing. Counting step rate in real-time is a potential alternative for healthcare professionals and coaches working with runners, however limited evidence exists pertaining to its reliability and validity [18]. Furthermore, due to the nuances of real-time counting practices, it is currently unknown as to the impacts of specific counting variations, including the use of a hand-held tally counter to decrease the cognitive demands of mentally counting, attending to one or two foot contacts, or assessment duration, on measurement accuracy. Therefore, the purpose of this study was to evaluate the reliability and validity of various procedures of real-time step rate assessments during treadmill running.

Participants 

A convenience sample of 10 experienced runners within the local community was enrolled in this study. This sample size is comparable to the number of raters, sample, and data points in other running reliability and validity studies [14,15]. To be eligible to participate the runners had to be within the ages of 18 to 50 years old, run at least 60 minutes/week, and have experience running on a treadmill. Participants were excluded if they had a history of any lower limb surgery or musculoskeletal injury within the last 3 months that forced them to stop running. The protocol was approved by the New York Institute of Technology Institutional Review Board.

Procedure: Participants attended a single visit to the New York Institute of Technology research laboratory. Following written informed consent, each participant completed questionnaires pertaining to their running history including competition level, running volume, and preferred running speed. Height and weight were measured to the nearest 0.1 cm and 0.1 kg, respectively. After a warmup consisting of a three-minute walk, participants ran on a treadmill for two-minutes. During this acclimation period, each participant was permitted to adjust the treadmill speed to match their typical five-mile, moderate intensity training run pace. After the participant expressed comfort, an additional 90-second lateral view video recording, captured at a 60 frames/second frame rate (iPad 5^th Generation, Apple Inc, CA, USA), was used to assess step rate. After completion of the trial, the participant returned to a walking speed for at least two minutes to allow for recovery, which was affirmed verbally. The same procedure was repeated at +15% and -15% of the speed from the initial trial. 

After all participant enrollment and data collection, two second-year doctor of physical therapy students with no professional experience in running analysis simultaneously viewed each video on a common projector screen. Running step rate was analyzed while watching the full-speed video recordings. Each rater analyzed the videos utilizing one of the eight counting methods that were derived from combinations of three factors that are easily manipulated when clinically assessing step rate: 1) with or without a hand-held tally counter (Steel master tally counter, MMF Industries, Wheeling, IL, USA); 2) assessment of one or two limbs, and 3) counting duration of 30 or 60-seconds (Figure 1). A tally counter can be used as an alternative to mentally counting to decrease the concentration demands of the rater. Doubling steps of a single limb has also been a suggested method instead of assessing bilateral foot contacts. Additionally, similarly to previous studies on measuring heart rate, step rate evaluations can be performed over various time intervals and converted to steps per minute (spm). The counting methods used to analyze step rates included:

Figure 1: Combinations of eight real-time counting methods and their associated abbreviations. 

In each of the four tally counter conditions, raters attempted to synchronize each foot contact with pressing the tally counter with their finger. Alternatively, in the four mentally counting conditions, raters were forced to maintain a sequential count in their head without the aid of an external device. 

To decrease bias, each rater was blinded to the specific running condition and other rater’s results. Simple calculations were performed to normalize step rate to spm. One week later, the raters re-assessed the same videos with all eight methods in a randomized fashion. Slow-motion, frame-by-frame step rate analyses were performed for all videos by co-author (C.M), blinded to the findings of the two raters, which wasutilized as the reference standard. 

Statistical analysis                                    

Means and standard deviations were computed to describe demographic and step rate data for all eight counting methods. Intraclass Correlation Coefficients (ICC) were computed to assess interrater (ICC _2,1) and intrarater (ICC _3,1) reliabilities with 95% confidence intervals to estimate precision. The ICC values were classified using the method described by Fisher [19], where ICC < 0.5 was considered poor, 0.5< ICC< 0.75 was considered moderate, 0.75< ICC < 0.9 was considered good and ICC> 0.9 is considered excellent. To assess validity, ICCs were used to calculate the relative agreement between the various clinical counting methods and the slow-motion, frame by frame video analysis. Absolute agreement was assessed using mean differences, mean absolute differences, and Standard Error of Measurement (SEM). All statistical procedures were performed using SPSS version 23 (IBM SPSS Statistics., Armonk, NY). The Guidelines for Reporting Reliability and Agreement Studies (GRRAS) was used to ensure the quality of reporting the findings of this study [20].

Four males and six females participated and were on average 23.2 ± 2.5 years old, weighed 69.1 ± 10.4 kg, and were 172.5 ± 6.8 cm tall. Mean running speeds and step rates for each running condition can be found in Table 1. Intrarater reliability (ICC_3,1) for both raters for each counting condition is described in table 2. Raters had good to excellent intrarater reliability for all counting methods (ICC 0.83-1.00,p<0.001) except for the Tally/Bilat/30-sec condition which had moderate to good reliability (ICC 0.60-0.79, p<0.001). Interrater reliability calculations (ICC_2,1) for all eight counting methods can be found in table 3. All counting conditions had good to excellent interrater reliability (ICC 0.80-0.98, p<0.001) with the exception of the Tally/Bilat/30-sec condition which had poor to moderate reliability (ICC 0.45-0.75, p<0.001). 

Table 1: Participant demographics.

Abbreviations:

m/s: meters per second;

spm: steps per minute;

Table 2: Intrarater reliability for all counting methods.

Abbreviations:

Tally: Tally Counter;

NoTally: Without tally counter;

R only: Right leg counted only;

Bilat: Bilateral legs counted; sec: seconds;

ICC: Intraclass correlation coefficient;

SEM: Standard error of measurement;

spm: steps per minute;

^*ICC_{(3, 1)} with 95% confidence interval 

Table 3: Interrater reliability for counting methods.

Abbreviations:

Tally: Tally Counter;

NoTally: Without tally counter;

R only: Right leg counted only;

Bilat: Bilateral legs counted;

sec: seconds

ICC: Intraclass correlation coefficient;

CI: Confidence interval;

SEM: Standard error of measurement;

spm: steps per minute 

The relative agreement for the various counting methods when compared to the reference standard of slow-motion, frame-by-frame video analysis were good to excellent (ICC 0.74-0.96, p<0.001) except for the Tally/Bilat/30-sec condition which was poor (ICC=0.45, p<0.001) (Table 4). However, only the NoTally/Bilat/60-sec condition had 95% confidence of good to excellent relative agreement with the reference standard (ICC 0.95, 95% CI 0.83-0.98). Mean differences were less than 5 steps in all conditions except the Tally/R only/30-sec and Tall/Bilat/30-sec conditions, which were 6.2 and 9.9 steps, respectively. The positive values of all counting conditions, evident by the similar mean and absolute mean errors and Bland-Altman plots, highlighted the overestimation of step rate values of all counting conditions (Figure 2). Furthermore, there was no suggestive relationship between measurement error of real-time counting methods and magnitude of step rate; however there was a trend for greater error when utilizing the tally counter for counting bilateral foot strikes in runners with higher step rates in both the 30 and 60-second methods. 

Table 4: Mean differences and correlations between counting methods and slow-motion video analyses.

Abbreviations:

Tally: Tally Counter;

NoTally: Without tally counter;

R only: Right leg counted only;

Bilat: Bilateral; sec: seconds;

ICC: Intraclass correlation coefficient;

SEM: Standard error of measurement

*steps per min;

^†combined day 1 and day 2 for raters;

^‡positive value indicates overestimate compared slow-motion video analysis;

^§SEM based on ICC  

Figure 2: Bland-Altman plots for measurement error for the eight real-time step rate counting methods. 

The measure of the y-axes is in steps per minute and were obtained by subtracting the step rate measured by the reference standard of slow-motion video analysis from the step rate measure by utilizing the real-time counting method. The x-axes represents the step rates in steps per minute as determined by the slow-motion video analyses. Solid lines represent the mean error while dotted lines represent the lower and upper bounds of the 95% limit of agreement interval.

Reliable and valid tests and measures are needed to help inform decisions pertaining to patient management. Step rate is an essential metric for clinicians to consider when performing clinical running gait assessments. The current study highlights the nuances in various methodologies of assessing real-time, running step rate. Though recent evidence suggests that step rate is individualized and should not be used as an isolated determinant of clinical decisions [21], variations in measurement methods can result in clinical inconsistencies and suboptimal outcomes for those undergoing running gait analyses. To our knowledge, this is the first study to compare various methods of assessing running step rate in real-time when advanced instrumentation is unavailable. 

In lieu of expensive high-speed motion analysis and instrumented treadmills systems, various more cost effective, time efficient, and practical technologies are accesible for clinicians treating RRIs. When available, instrumentation, such as fitness watches, can be valuable as a reliable and accurate means of assessing step rate [14]. However, such technologies can still be costly and may not be considered financially sound to clinicians that do not regularly work with the running population. Instead, with the advancement in video capabilities of tablets and smart phones, slow-motion running analysis can be easily performed without extensive training or added cost. Slow-motion, frame-by-frame analysis was utilized as the reference standard for step rate in this study. Previous research has demonstrated excellent interrater (ICC >0.90) [18,22] and intrarater reliability (ICC 0.984-0.998) [22] of this method to analyze step rates in healthy runners and in those with anterior knee pain. Additionally, a previous study demonstrated excellent correlation between two-dimensional video analysis and two commercial grade gait analysis systems (ICC 0.901) [18]. However, this method requires space and is more time intensive to allow for equipment setup, video capture, and asynchronous slow-motion analyses, and thus inefficient in a demanding clinical environment. 

The current study assessed the reliability and validity of eight easily performed counting methods. These measures were chosen as clinicians can easily differ in their preferred assessment methods by altering the time duration, counting one limb or both limbs, or the whether or not to utilize a tally counter. Seven of the eight counting methods were found to have good or excellent intrarater (ICC 0.83-1.00) and interrater (ICC 0.8-0.98) reliabilities, with the exception of counting bilateral foot strikes for 30-seconds with the hand-held tally counter which had moderate to good intrarater (ICC 0.60-0.79) and poor to moderate interrater (ICC 0.45-0.75) reliabilities. The better performing counting methods were comparative to previous studies that assessed the reliability of a fitness watch (ICC 0.93, 95% 0.89-0.96CI) [14] and wireless instrumented insoles (ICC 0.91, 95% CI 0.78-0.96) [15]. Theoretically, the authors hypothesized that the tally counter would improve raters’ abilities of counting step rate, however this was not supported by the current findings. Furthermore, confidence intervals were generally wider in the conditions that utilized the tally counter, supporting its inferior performance compared to mentally counting without the tally counter. Though the tally counter likely decreased the cognitive demands of counting, it required the performance of an added psychomotor task of coordinating the tally counter to match foot strikes. This proved to be more difficult for the raters, especially during the conditions that were assessed with bilateral foot contacts. Therefore, when a clinician prefers to use a tally counter, it is advised to apply methods that only assess the foot contacts of one limb. 

Accurate assessments are important as step rate measures can help to identify runners at greater risk for sustaining a running injury. High school runners with step rates ≤166 were more likely to sustain a shin related injury compared to runners with step rates ≥178 spm. Additionally, when accounting for the entire sample of runners in the same study, ≤172 spm was the discriminating predictive value of incurring a shin injury during the cross-country season [11]. The conditions that demonstrated the least absolute error and greatest accuracy were counting without the use of a tally counter for 60-seconds on either both limbs (1.9 ± 1.9 spm) or one limb (2.0 ± 1.3 spm), and when using the tally counter corresponding to one limb’s foot contact for 60-seconds (2.5 ± 1.6 spm). Though these differences between counting measures seem minimal, subtle errors can negatively affect the proper identification of runners at greater risk of injury. Therefore, counting without a tally counter for 60-seconds on both limbs is the most preferable counting method to assist in injury risk stratification as it had the highest relative agreement with the reference standard (ICC=0.95, 95% CI 0.83-0.98). 

 From a rehabilitation perspective, step rate manipulation has been an easily implemented and effective form of gait retraining in the management of RRI such as tibial stress fractures and anterior knee pain [13,23]. A 5-10% increase in preferred step rate has been shown to decrease knees tresses by as much as 34%, [12] while prescriptions greater than 10% are more difficult for the runner to adopt as it increases oxygen consumption and may subsequently impair performance [24]. All of the counting measures in this study resulted in positive mean difference values, indicating an overestimation in step rate. To illustrate these consequences, a runner with a true step rate of 160 spm may be found to as have a step rate of 165 spm by utilizing one of the counting methods in this study. Based on the 5-10% recommended range for step rate increase, an appropriate step rate prescription for this particular runner should be between the range of 168 and 176 spm. However, due to the associated measurement error the runner would instead be prescribed a running step rate between 173-181 spm and thus risks the deleterious effects of over-prescription. Therefore, to ensure appropriate treatment, prescriptions based upon the studied counting measures should be more conservative, within the lower range (5%-7.5%) of current step rate manipulation recommendations. Additionally, due to the compounded effects of normalizing to spm, counting for 30-seconds doubled the amount of associated error compared to their respective 60-seconds methods. Therefore, regardless of counting method, clinicians should opt to perform step assessments for 60-seconds to improve measurement accuracy. 

This study presents with several limitations. Slow motion, frame-by-frame analysis was considered the reference standard for running step rate measurement. Though this method has been previously validated [17,18], it may present with some inherent measurement error that may have impacted findings. Secondly, having the raters watch real-time videos may not be equivalent to watching real-time treadmill running. To most accurately determine intrarater reliability, this design limitation was intentional to control for the intrinsic variations in running step rate of participants across running trials. Furthermore, though previous work found no effect of assessor experience on reliability of assessing step rate with two-dimensional video analysis [22], raters in the current study were second-year physical therapy students and may not be representative of the population of professionals performing clinical running gait analysis. Additionally, concentration fatigue from the repetitive assessment of numerous videos was not considered and could have affected rater performance. Lastly, a larger sample size with a wider range of running speeds and preferred step rates may help to further expand on the current findings. 

This study examined only eight of the many possibilities of assessing running step rate in real-time. Future research should look to identify other easily administered modes of assessing running step rate and their application in clinical gait analyses. Additionally, future studies should attempt to observe the utility of real-time counting methods during gait retraining interventions that aim to manipulate step rate.

Various technologies exist to assess running step rate in lieu of the clinically impractical gold standards of high-speed motion capture systems and instrumented treadmills. Fitness watches [14], wireless instrumented insoles [15], and commercial grade accelerometers [16] have been shown in previous research to be reliable and less expensive options. When these instrumentations are not readily available, clinicians may utilize slow-motion, frame-by-frame video analysis [22], however this method is more time intensive and requires increased space. This study highlights several real-time counting methods that are reliable, valid, and efficient and can offer practical alternatives for clinicians, coaches, and runners as a means of identifying runners with an increased injury risk or in the management following injury. Of the methods studied, simply counting bilateral foot strikes without a tally counter for 60-seconds was the most reliable and valid method and, therefore the preferred counting method of those studied. Additionally, not using a hand-held tally counter and counting foot contacts for 60-seconds compared to 30-seconds will decrease measurement error. To avoid the deleterious effects of over prescription, step rate increases based on the real-time counting measures studied should be made more conservatively (5%-7.5%) as they tended to overestimate step rate.

No financial or non-financial conflicts of interest

Institutional Review Board of the New York Institute of Technology approved this study. Protocol number: BHS-1355

There is no funding to report for this manuscript.

The authors would like to thank Anthony Joe, DanoNorceide, and Giuseppe Sposito for their dedicated time and efforts to support this research.