Insulin resistance is commonly assessed using the homeostasis model assessment (HOMA) variants. HOMA is potentially insensitive to change because of its high coefficient of variation. The repeatability coefficient is an alternative means of assessing test repeatability. To be confident of clinical change, rather than biological variation, a subsequent test needs to differ from the former by more than the repeatability coefficient using the equation.
Test 1 = Test 2 ± repeatability coefficient.
The repeatability coefficients for measures of insulin resistance are unknown.
To compare the repeatability coefficient of HOMA2 variants (Beta-cell function [%B], insulin sensitivity [%S], insulin resistance [IR]) to a dynamic measure of insulin resistance, and the oral glucose insulin sensitivity (OGIS) test.
The raw data from a previously used data set were reanalysed.
Glycaemic and insulinaemic tests were performed on 32 men and women both with (
Repeatability coefficients for all participants for the HOMA2 %B, %S and IR variants were 72.91, 189.75 and 0.9, which equates to 89%, 135% and 89% of their respective grand means. By contrast, OGIS had a repeatability coefficient of 87.13, which equates to 21% of the grand mean.
Because of the high repeatability coefficient relative to the grand mean, use of HOMA2 measures for assessing insulin resistance in small population studies should be reconsidered.
It is projected that by 2030, nearly 7% of the global population will have type 2 diabetes with many of them living in developing countries.
An example of a method based on a fasting insulin and glucose sample is the homeostasis model assessment (HOMA), or the second-generation HOMA2. HOMA is the collective term for the assessment of three different aspects of insulin resistance: Beta-cell function (HOMA %B), insulin sensitivity (HOMA %S) and insulin resistance (HOMA IR). The original HOMA methods are calculated using the following equations (
Homeostasis model assessment calculations where glucose is mmol/L and insulin is mU/L.
By contrast, HOMA2 is a computer model that accounts for variations in hepatic and peripheral insulin resistance. The formulae are not available, but the calculator can be downloaded from the Diabetes Trials Unit, University of Oxford.
HOMA methods are commonly used to assess insulin resistance with studies ranging from large-scale epidemiological assessments to smaller interventional studies. HOMA methods are practical instruments as they are cost-effective and easy to use. However, previous studies have shown HOMA to have coefficients of variation (CVs) ranging from 10% to 50%.
Emerging research suggests that fasting insulin levels may not predict post-prandial insulin levels,
Instruments assessing change need an acceptable level of repeatability. Repeatability is how much variation can be expected among repeat measurements on the same subject under identical conditions. This enables subsequent test results (e.g. blood tests) to either indicate clinical change or biological variation (noise). Repeatability is often assessed using CV (the ratio of standard deviation compared to the mean) and expressed by percentage, using the equation (see
By contrast, the repeatability coefficient defines the range within which 95% of the differences between two measurements in the same subject by the same measurement method are likely to fall, assuming there is no change in clinical condition between the tests.
Therefore, to be confident of clinical change, a subsequent test needs to have changed by an amount that is greater than the repeatability coefficient (either larger or smaller). Otherwise, the result will still lie within the range of typical biological variation. All else being equal, tests with a small repeatability coefficient are more sensitive to clinical change. Conversely, tests with a large repeatability coefficient generally require greater clinical change to occur before they can confidently be taken as an indication of such change. This makes tests with a large repeatability coefficient less desirable for clinical use as they are less sensitive to clinical change. Although CV is more commonly used to assess repeatability, use of the repeatability coefficient may be more practical for clinicians as it may be easier to interpret and incorporate into clinical practice.
The aim of this study was to assess the repeatability coefficients for the HOMA2 variants and OGIS by evaluating repeated oral glucose tolerance tests with insulin assays in a small group of subjects with or without type 2 diabetes.
The raw data from a glycaemic and insulinaemic index study were reanalysed.
Full methods and methodology are available.
Originally, the 22 healthy participants were divided into ‘control’ and ‘hyperinsulinaemic’ groups based on a fasting insulin of 40 pmol/L.
Blood samples were collected according to standard protocols. Each participant had fasting blood samples drawn on eight separate mornings. On three of those mornings, they then consumed 50 g anhydrous glucose in 250 mL water and on the other five mornings they consumed 50 g available carbohydrate from sucrose, instant mashed potato, white bread, polished rice or pearled barley. Venous blood samples were then drawn at 15, 30, 45, 60, 90 and 120 min for participants without type 2 diabetes and at 30, 60, 90, 120 and 180 min for participants with type 2 diabetes. Timing commenced after starting to eat. This study analysed all results from the eight fasting tests and from the three glucose meals.
Venous blood samples were collected in BD vacutainer SST tubes. Serum glucose was measured by the glucose oxidase method (Synchron LX Systems) with inter-assay CV of 1.9%. Insulin was measured using one-step immunoenzymatic assay (Beckman Access Ultrasensitive Insulin Assay) with inter-assay CV of 2.5% – 4.3%. Insulin has no cross-reactivity with proinsulin.
As individual height and weight data were not available for each person, the standards of 1.7 m for height and 70 kg for weight were used for each person. The fasting glucose and insulin values from each test were used to calculate each of the HOMA2 variants (HOMA2 %S, HOMA2 %B and HOMA2 IR) via the available online calculator.
The glucose and insulin values from each of the three oral glucose tests were used to calculate OGIS via the available spreadsheet.
For each test, within-subject means were plotted against within-subject standard deviations to determine if there was a mean–variance relationship. Ordinary least squares regression was used to assess the strength of such relationships. If the slope coefficient was significant at the 0.05 significance level, the process was repeated for the mean and standard deviation of the natural log of the variable.
If a significant mean–variance relationship was determined, participants were divided into subgroups according to test results. The intent was to reduce the mean–variance relationship and therefore bias in the repeatability coefficient at each end of the range while maintaining a clinically meaningful result.
Repeatability was quantified by estimating repeatability coefficients according to the methods of Bland and Altman.
The following calculations defined the ranges within which two repeat measurements could be expected to fall under the assumption of no clinical change between repeat tests (
Ethical permission for data collection was previously granted by Research Ethics Boards at the University of Toronto and St Michael’s Hospital. All participants gave written informed consent.
Raw data for control (left) and diabetes (right) groups for fasting glucose, fasting insulin, HOMA2 %B, HOMA2 %S, HOMA2 IR and OGIS.
Mean–variance relationships were positive and significant for all tests with the exception of OGIS for both groups, and fasting insulin and HOMA2 IR for the Diabetes group (
Regression coefficient and
Variable | All participants |
Diabetes |
No Diabetes |
Control |
||||
---|---|---|---|---|---|---|---|---|
Reg Coef | Reg Coef | Reg Coef | Reg Coef | |||||
Fasting glucose (mmol/L) | 0.097 | < 0.001 | 0.049 | 0.035 | 0.165 | 0.039 | 0.119 | 0.17 |
log fasting glucose | 1.474 | < 0.001 | 0.54 | 0.034 | 2.219 | 0.069 | – | – |
Fasting insulin (µU/mL) | 0.156 | < 0.001 | –0.23 | 0.635 | 0.221 | < 0.001 | 0.388 | < 0.001 |
log fasting insulin | 0.942 | < 0.001 | – | – | 0.997 | < 0.001 | 1.182 | < 0.001 |
HOMA2 %B | 0.306 | < 0.001 | 0.318 | 0.001 | 0.409 | < 0.001 | 0.296 | < 0.001 |
log HOMA2 %B | 0.864 | < 0.001 | 1.06 | < 0.001 | 1.366 | < 0.001 | 1.328 | < 0.001 |
HOMA2 %S | 0.406 | < 0.001 | 0.825 | < 0.001 | 0.333 | 0.007 | 0.303 | 0.061 |
log HOMA2 %S | 1.227 | < 0.001 | 1.40 | 0.004 | 1.220 | < 0.001 | – | – |
HOMA2 IR | 0.194 | < 0.001 | 0.236 | 0.067 | 0.180 | < 0.001 | 0.363 | <0.001 |
log HOMA2 IR | 0.897 | < 0.001 | – | – | 0.921 | < 0.001 | 1.105 | < 0.001 |
OGIS (mL/min/m |
0.450 | 0.099 | 0.005 | 0.904 | –0.003 | 0.962 | –0.350 | 0.731 |
%B, beta cell function; HOMA2, homeostasis model assessment 2; IR, insulin resistance; OGIS, oral glucose insulin sensitivity test; Reg Coef, regression coefficient; %S, insulin sensitivity.
Re-examination of the mean–variance relationships identified a non-significant relationship for fasting glucose (
With the exception of log fasting glucose for the No Diabetes set, the only measures that demonstrated a non-significant, log-transformed mean–variance relationship also had a non-significant mean–variance relationship for the raw data (
The repeatability coefficients for HOMA2 variants, fasting insulin, fasting glucose and OGIS by participant sets are presented in
Repeatability coefficients for simple measures of insulin resistance (all data).
Variable | All participants ( |
Diabetes ( |
No Diabetes ( |
Control ( |
||||||||||||
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
sw | ±Rep Coef | Change % | sw | ±Rep Coef | Change% | sw | ±Rep Coef | Change% | sw | ±Rep Coef | Change% | |||||
Fasting glucose (mmol/L) | 0.59 | 1.62 | 6.08 | 26.64 | 0.93 | 2.59 | 9.20 | 28.15 | 0.32 | 0.88 | 4.65 | 18.92 | 0.30 |
0.84 | 4.06 | 20.69 |
Fasting insulin (µU/mL) | 2.85 | 7.91 | 8.61 | 91.87 | 2.86 |
7.90 | 8.59 | 91.97 | 2.85 | 7.90 | 8.62 | 91.65 | 2.14 | 5.93 | 6.52 | 90.96 |
HOMA2 %B | 26.31 | 72.91 | 82.31 | 88.57 | 13.08 | 36.26 | 36.41 | 99.45 | 30.53 | 84.61 | 103.18 | 82.00 | 20.67 | 57.31 | 90.62 | 63.24 |
HOMA2 %S | 68.46 | 189.75 | 140.05 | 135.49 | 70.02 | 194.09 | 111.19 | 174.56 | 67.72 | 187.72 | 153.17 | 122.56 | 72.68 |
201.45 | 169.33 | 118.97 |
HOMA2 IR | 0.32 | 0.90 | 1.01 | 89.11 | 0.39 |
1.07 | 1.15 | 93.04 | 0.29 | 0.81 | 0.95 | 85.26 | 0.23 | 0.64 | 0.72 | 88.89 |
OGIS (mL/min/m |
31.43 |
87.13 | 413.10 | 21.1 | 21.88 |
60.67 | 303.69 | 19.98 | 34.91 |
96.77 | 462.84 | 20.91 | 34.90 |
96.74 | 475.79 | 20.33 |
%B, beta cell function; Change % = per cent change of the repeatability coefficient relative to the Grand mean; HOMA2, homeostasis model assessment; IR, insulin resistance; OGIS, oral glucose insulin sensitivity test; Rep Coef, repeatability coefficient; %S, insulin sensitivity; sw, residual mean square error;= Grand mean.
Denotes a non-significant mean–variance relationship from
OGIS was conducted with three repeated tests. All other variables had eight repeated tests.
OGIS was the only index to have a non-significant mean–variance relationship across all four participant sets (
We examined the reproducibility of fasting and dynamic measures of insulin resistance by calculating the repeatability coefficient for the HOMA2 variants and OGIS in people with and without type 2 diabetes. HOMA2 and OGIS were chosen as representative of techniques assessing insulin resistance that did not require specialised methods or monitoring. The computer-modelled HOMA2 was also chosen above the original HOMA calculations as we believed that HOMA2 would become the more widely used model.
Although the absolute value of the repeatability coefficients varied by participant subset, measures based on fasting insulin, including all HOMA2 variants, had a large repeatability coefficient, meaning that they would require a large change relative to the population mean in order to indicate clinical change with confidence. This suggests that the main driver of the variation is fasting insulin given its wider spread as compared to fasting glucose (
We chose to express figures as percentages of the grand mean for two main reasons. Firstly, to determine whether there was consistency in the magnitude of change required when examining the different participant subsets. It was also believed that calculating percentages may be easier for clinicians than using different values based on glycaemic status.
However, when the repeatability coefficient was converted to a percentage of the grand mean of that subset, only certain measures showed this consistency. For example, in order to be confident clinical change has occurred, a subsequent test of either fasting insulin or HOMA2 IR needs to differ from a former test by approximately 90% irrespective of the participant subset. Depending on the individual’s glycaemic status, this equates to a change in fasting insulin of between 6 µU/mL and 8 µU/mL or HOMA2 IR values of 0.64–1.07. Similarly, OGIS only required a 20% (60.67 mL/min/m
These findings suggest that OGIS should be preferred over either fasting insulin or any variant of HOMA2, when assessing individuals or small populations for insulin resistance for the reason that OGIS appears to be more sensitive to clinical change.
There are limited data on the test–retest repeatability of measures of insulin resistance, especially including the use of the repeatability coefficient. The original HOMA IR has been reported as needing to change by +90% or -47% in patients with type 2 diabetes to ensure that the second sample is clinically significant when compared to a previous sample.
Using the repeatability coefficient rather than CV meant that it was harder to compare our results to the existing literature. However, CV can be derived from the repeatability coefficient using the following calculation in
Using this conversion, our results align with current CV reports. Gordon and colleagues reported the CV for OGIS to be 7.8% (range, 4.2% – 14.2%) for eight people with four repeated tests.
Another challenge with comparing repeatability studies is the assessment methods for quantifying insulin concentrations. Insulin concentrations will vary depending on analytical method, including the use of plasma or serum.
This study highlighted that for many variables, the repeatability coefficient differed according to the subset of the study population. For this reason, we converted the absolute figures of the repeatability coefficient to a percentage of the Grand mean to determine if there were consistencies throughout the subgroupings (
A notable finding of our study was the maintenance of a positive and significant mean‑variance relationship for almost all the study variables, including fasting glucose; OGIS was the only variable that consistently lacked a positive and significant mean–variance relationship. These positive mean–variance relationships mean that the repeatability coefficient may be over- or underestimated at the extremes of the ranges of observed test results. Given that measures based on fasting insulin required 60% – 175% difference in results to ensure clinical change, the influence of the bias may not matter. What was clear is that OGIS did not have a positive and significant mean–variance relationship for any subgrouping tested, and although the repeatability coefficient altered depending on the subgroup, it remained a consistent 20% of the population grand mean.
The large number of repeated tests of fasting measures (
There were a number of limitations to our study. While we were able to assess HOMA through eight repeated tests, we only had data from three repeated tests by which to assess OGIS. However, previous research has only assessed the repeatability of OGIS via duplicated tests.
The results from our study highlight that HOMA2 variants are insufficient to detect small, but clinically significant changes within an individual. This suggests that HOMA2, and potentially other measures based on fasting insulin, may be impractical for use within clinical practice or small-scale research projects. The implications of high degree of variation for larger scale research projects are not yet known. HOMA has been used to either classify participants or assess the effects of an intervention. These results suggest that using measures based on fasting insulin for baseline stratification may not be effective as participants may have different results on different testing occasions. If these variables are to be used as a primary outcome, then power calculations should ensure that the study has a sufficient sample size in order to accurately detect change. Many studies do not use HOMA as a primary outcome, and this would be reflected in sample size. In a placebo-controlled intervention study, in order to detect a 15% change in HOMA2 IR in people with normal glucose tolerance, a target sample size of 55 people in each arm is needed to provide 80% power at the 0.05 level of significance using a two-sample
We aimed this study towards clinical practice and translational research. HOMA and HOMA2 variants are widely used in many areas of research including the assessment of interventions aimed at improving insulin sensitivity. Although measuring insulin resistance is discouraged in medical practice as is does not enhance disease risk calculations,
We accept that measures based on fasting insulin are much cheaper and less demanding than those based on the results derived from an oral glucose tolerance test. This may partially explain the popularity of HOMA. We further recognise that only recommending tests based on an oral glucose tolerance test would likely result in fewer assessments of insulin resistance. But should we settle for convenience over accuracy?
There may be a number of physiological explanations for our observations. Insulin is secreted from the pancreas via a pulsatile pattern leading to oscillatory blood concentrations. These oscillations have a slow ultradian periodicity (~140 min), which is modulated by a small-amplitude, high-frequency oscillation (periodicity 3–10 min).
We believe that early detection of insulin resistance is important given it precedes or is highly associated with a range of metabolic diseases. Although HOMA measures are a convenient method for assessing insulin resistance, their high variability impedes accuracy in diagnosis and monitoring clinical change. A subsequent HOMA2 test needs to change by approximately 90% to be confident that clinical change has occurred. Dynamic methods such as OGIS may have a significantly higher degree of repeatability, although they need further study. Considering the global incidence of insulin resistance and its impact on non-communicable disease, our findings support the inclusion of OGIS in clinical settings.
Catherine Crofts was supported by a Heart Foundation (NZ) study award (Ref 1522).
Grant Schofield is the Chief Education Health and Nutrition Advisor for the New Zealand Ministry of Education and serves as a board member for Health Promotion Agency of New Zealand, a government agency.
Thomas Wolever and his wife are employees and part-owners of GI Labs, Inc., a contract research organisation, and of Glycaemic Index Testing, Inc., a corporation that provides statistical services related to calculation of the glycaemic index.
Mark C. Wheldon, Xiaomiao Lan-Pidhainy and Caryn Zinn have no conflicts of interest to declare.
C.A.P.C. was the project lead and writer and was responsible for concept and design, data analysis and interpretation. M.C.W. was the reviewer, was responsible for concept and design, and provided statistical expertise. C.Z. was the reviewer and performed data analysis and interpretation. X.L.-P. was responsible for concept and design and data acquisition. T.M.S.W. was responsible for concept and design and data acquisition. G.S. was the reviewer and performed data analysis and interpretation.