Internal validation of an automated system for brachial and femoral flow mediated dilation

Atherosclerosis, the pathological deterioration of endothelium that results in vessel blockage, is responsible for approximately 90% of all cardiovascular diseases [1, 2]. Endothelial dysfunction is the first step in this process, defined by the decrease in nitric oxide availability within the endothelium, developing oxidative stress. When endothelial oxidative stress is increased, endothelial NO synthase is uncoupled, and generates a decrease in NO production and bioavailabilty [3]. Flow Mediated Dilation (FMD) is a non-invasive assessment of endothelial function and vasodilatory capacity in humans [4–6] with an immense potential to become a clinical tool for cardiovascular risk assessment [1]. Therefore, FMD is normally considered as a biomarker for endothelial function and NO bioavailability in humans, relevant to study cardiovascular risk factors such as hypertension [5, 7]. There are three major sites where FMD is assessed. The most studied one is the brachial artery [4, 5]; however, femoral artery [8] and popliteal artery [9] sites have been used to study large elastic and leg conduit arteries, respectively.

In general, there is a consensus on how to perform FMD [5, 10, 11], however, there are several analysis systems that can be used to determine vessel diameters [12–17]. In addition, FMD analysis could significantly deviate in different laboratories which makes the comparison of results very difficult. To provide a better understanding of the analyzed data, each laboratory must standardize their specific protocols and provide information about technical error of the procedure.

Therefore the purpose of this study is to provide validation to the assessment of FMD analysis in our laboratory and to standardize this process as a first step in every laboratory prior to reporting results of FMD.

Thirty apparently healthy individuals, 15 males and 15 females, ages 18–35 years old, were recruited for this study. Exclusion criteria included known cardiovascular or cardiac disease, prescription medication medicines, and ‘over-the-counter’ painkillers, such as NSAIDs or aspirin, or nutritional supplements containing antioxidants [7]. Young female participants were tested within 4 days before or within 4 days after menses to standardize any influence that certain phases of the menstrual cycle may have on vascular physiology [18].

The investigation was approved by the Indiana State University Institutional Review Board for ethical practices and declared in accordance with the Declaration of Helinski. All subjects were tested at the same time of day to avoid any diurnal variations and following at least 8 h of fasting [5, 11].

Analysis

Analysis was performed by two independent testers (BR and RP) who were blind to each other’s analyses. Analysis included baseline artery diameter measurement, peak artery diameter post-ischemia, as well as FMD. Analysis was completed via Brachial Analyzer for Research® Software (Medical Imaging Applications LLC, Coralville, Iowa) by both testers after training of the methodical process of analysis to minimize variability (Fig. 1). After reviewing FMD images, a square Region of Interest (ROI) was placed on the most stable region of the artery. The ROI encompassed both sides of the endothelial lumen and measured a width of at least 3 mm ± 0.1 mm across. The ROI was then locked, and curvature of ROI to endothelial lumen was checked for consistent diameter accuracy. Images with less than 75% Confidence Index of quality control were edited with computerized assistance to best fit arterial morphology. No more than nine computerized assistances were allowed, if more were needed for accurate results, the frame was rejected. Standard deviation of the measured frame was minimized when possible, maximal value allowed was 0.15 mm.

In this experiment, FMD calculations were allometrically scaled using FMD = [Peak Diameter/(Baseline Diameter^0.89)] proposed by Atkinson et al. [19, 20] after analyzing the slope of the relationship between logarithmic transformed baseline and peak diameters for this specific sample. In addition, FMD is presented as a percentage change from baseline measurements (%FMD) using the following formula:

$$ \%FMD=\left[\left(\mathrm{Peak}\ \mathrm{Diameter}\hbox{--} \mathrm{Baseline}\ \mathrm{Diameter}\right)/\left(\mathrm{Baseline}\ \mathrm{Diameter}\right)\right]\ \mathrm{x}\ 100. $$

During the study, data collected during brachial FMD from one subject had some technical error on the first visit. In addition, data collected during femoral FMD from two subjects had some technical error. Data from these subjects were withdrawn from the intertester and intratester analyses. All data was normally distributed according to Kolmogorov-Smirnov analysis.

The present study was designed to internally validate the analysis of FMD within the same tester and between testers. The major findings in this study are two-fold: first, the FMD analysis protocol is reproducible within same tester and second, the FMD analysis protocol is reproducible between testers.

In general, FMD protocols follow international guidelines [5, 10, 11]. Our laboratory is not an exception. All images are obtained with an isonation angle of 60° and images are obtained with the help of an ECG-gated system to avoid changes of vessel diameter observed within one cardiac cycle. These technical details provide more accurate and clear images of the studied vessel. The recording duration of 180 s, 30 s prior to deflation accompanied by 150 s post-deflation, was determined to be the most adequate method in which FMD could be determined [5, 10]. The 30 s prior to deflation allows to record images of the vessel that reflect differences with baseline diameters [7, 21]. After the cuff deflation, the hyperemic blood flow will rapidly increase eliciting the vascular response, which will peak anywhere between 30 and 90 s in an average 18–35 years old adult. The additional 60 s help to account for outliers in the vasodilatory response as well as return to baseline measurements.

The present study showed that the intratester CVs from two independent testers (tester 1 and 2) were very similar in all studied variables (range 2.62–4.95%, Table 3) and there no significant difference between the CVs from both testers. Brachial baseline and peak CVs are all below 3.75%, while FMD CV was slightly higher in one of the testers (4.95%). The CVs obtained in the present study are lower than the CVs from previous reports, which are already considered as low methodological error [16, 17, 22]. Therefore, the intratester validation presented here provides data for brachial baseline and peak artery dilation, making this data an acceptable reproducible measurement of brachial artery recording. Similar results were observed for the intertester analysis (Table 4). CVs between both testers were lower or at 5.50% for all studied variables. All the studied diameters, brachial baseline and peak and femoral baseline and peak, and femoral FMD showed no significant difference between testers. Only brachial FMD showed a significant difference between testers. This difference might be the results of slightly larger peak brachial diameter observed on tester 1, which could produce a larger difference when the FMD ratio is applied. Interestingly, and according to our best knowledge, this is the first study designed to determine reproducibility between two independent testers. The results of the present study show that the FMD protocol use in our laboratory is reproducible within a ~ 5% of variation between testers.

Other studies using similar approaches have shown comparable results than the present study. For example, Avery et al. [16] used the same automated edge detection system [15] as the present study. Their CVs were slightly higher than 5% (4–10%). This difference might be explained by the difference on their acquisition system, a 10.5 MHz ultrasound probe, which could decrease the image definition [5, 15]. Woodman et al. [17], using a tailored automated edge detection system, also found slightly higher CVs than the ones presented here (6.7%). Interestingly, their image acquisition was also performed with a 10 MHz ultrasound probe, which can produce less defined images [5]. Finally, Ghiadoni et al. [12, 22] used a new automated edge detection system that does not use ECG-gated image selection, using the average vessel diameter within a cardiac cycle. In addition, they recommend using 7.5 to 10 MHz ultrasound probes, which decreases images definition [5]. These two major differences might be responsible of higher CVs (7.6–11.6%) than the ones observed in the present study [22].

The final impact that this study had was the reproducibility of the observed results. A major factor of clinical utilization of FMD as a diagnostic technique is the accurate replicability between analyzers [1, 5, 11]. In this laboratory, two individuals with different backgrounds in research performed this validation study. One tester was a research assistant for nearly 2 years in this lab, while the other had been working in the lab for only 1 month prior to this analysis. The same techniques were provided by the principal investigator to both analyzers, with the same allotted practice time and techniques given from previous studies [17, 23]. The correlation between testers provided by this data proves the reproducibility of this laboratory procedure and advances the application of FMD as a future clinical diagnostic tool.

In essence, the present study was designed to provide a basis of support and assess the capabilities of our laboratory to provide accurate analysis and a specific, reproducible protocol for FMD testing. One of the main problems with FMD becoming a clinical assessment tool is the wide variety of results between laboratories and the dependency upon experimenter knowledge and background [23]. To solve the problem, our laboratory has created a structured methodological approach, accompanied by several sessions of background information about FMD that discussed background knowledge, methods, and common misconceptions, including image acquisition, recordings storage, and analysis software training. The present approach provides a basis of support for the protocol being easily reproducible and helps laboratories performing FMD to use it as a potential biomarker.

This study was not without limitations. First, the present study used the calculation of FMD proposed by Atkinson et al. [19, 20, 24]. This method deviated from previous studies, which used %FMD = [(Peak Diameter – Baseline Diameter)/(Baseline Diameter)] × 100 and implements a mathematical correction to account for the confounding influence of allometric scaling between individuals [19, 20, 23, 24]. Atkinson et al. approach allows constructing a mathematically efficient and more accurate representation of FMD ratio, which in the present study are different from data presented from other laboratories [4–6, 11, 16, 17, 22]. Secondly, the ECG-gated trigger system produced some incorrect readings on the QRS complexes, sometimes choosing a frame picture that was not at the correct time line. However, this limitation was non-significant as an occurrence in less than 1 in 1000 frames was observed. Finally, one individual, as highlighted in the results section, had to be removed from the intratester analysis due to technical difficulties that could not be amended, limiting our number of participants (n = 9).

The present study shows that in this laboratory, the described protocol is reliable and reproducible. All Coefficients of Variation (CV) were under or approximately 5%, confirming a strong reliability of the protocol and methods associated. CV was very similar between Tester 1 and Tester 2 and there was no statistical significance between Tester 1 and Tester 2 and between both testers’ CVs. The present study should promote the concept of all laboratories validating FMD procedures before carrying out any further FMD testing in order to obtain a reproducible and validated result in any future procedure that utilizes FMD. Finally, our laboratory will use the present study as a current reference of our FMD protocol reliability in any further scientific communication.