SQL Server KAPPA_COHEN_TV Function
Updated 2023-11-06 14:27:52.600000
Description
Use the table-valued function KAPPA_COHEN_TV to calculate Cohen's kappa (κ) or weighted kappa as an index of inter-rater agreement between 2 raters. For weighted kappa, ratings must be ordinal. KAPPA_COHEN_TV supports quadratic and linear weighting.
The equation for κ is:
\kappa = \frac{P_a - P_c}{1 - P_c}
Where:
Pa is the relative observed agreement among raters
Pc is the hypothetical probability of chance agreement
Syntax
SELECT * FROM [westclintech].[wct].[KAPPA_COHEN_TV](
<@InputData_RangeQuery, nvarchar(max),>
,<@Weight, nvarchar(max),>)
Arguments
@InputData_RangeQuery
a T-SQL statement, as a string, that specifies the rating 1- and rating 2- values.
@Weight
identifies the weighting algorithm to be used. Permitted values are un-weighted, linear and quadratic.
Return Type
table
| colName | colDatatype | colDesc |
|---|---|---|
| Pa | float | the relative observed agreement among raters |
| Pc | float | the hypothetical probability of chance agreement |
| K | float | the test statistic |
| P | float | the p-value |
| Z | float | K/SE |
| SE | float | the standard error |
| NS | float | the number of subjects |
| NR | float | the number of ratings |
Remarks
The function is insensitive to order; it does not matter what order the ratings are passed in.
Ratings are passed in as pairs.
For more than 2 raters, consider using KAPPA_FLEISS_TV.
If @Weight is NULL, then @Weight = 'U'.
For quadratic weighting, set @Weight = 'Q'.
For linear weighting, set @Weight = 'L'.
Examples
SELECT *
INTO #k
FROM
(
SELECT 1,
1
UNION ALL
SELECT 1,
1
UNION ALL
SELECT 1,
1
UNION ALL
SELECT 1,
1
UNION ALL
SELECT 1,
1
UNION ALL
SELECT 1,
1
UNION ALL
SELECT 1,
1
UNION ALL
SELECT 1,
1
UNION ALL
SELECT 1,
1
UNION ALL
SELECT 1,
1
UNION ALL
SELECT 1,
1
UNION ALL
SELECT 1,
1
UNION ALL
SELECT 1,
1
UNION ALL
SELECT 1,
1
UNION ALL
SELECT 1,
1
UNION ALL
SELECT 1,
1
UNION ALL
SELECT 1,
1
UNION ALL
SELECT 1,
1
UNION ALL
SELECT 1,
1
UNION ALL
SELECT 1,
1
UNION ALL
SELECT 1,
1
UNION ALL
SELECT 1,
1
UNION ALL
SELECT 1,
1
UNION ALL
SELECT 1,
1
UNION ALL
SELECT 1,
1
UNION ALL
SELECT 1,
1
UNION ALL
SELECT 1,
1
UNION ALL
SELECT 1,
1
UNION ALL
SELECT 1,
1
UNION ALL
SELECT 1,
1
UNION ALL
SELECT 1,
1
UNION ALL
SELECT 1,
1
UNION ALL
SELECT 1,
1
UNION ALL
SELECT 1,
1
UNION ALL
SELECT 1,
1
UNION ALL
SELECT 1,
1
UNION ALL
SELECT 1,
1
UNION ALL
SELECT 1,
1
UNION ALL
SELECT 1,
1
UNION ALL
SELECT 1,
1
UNION ALL
SELECT 1,
1
UNION ALL
SELECT 1,
1
UNION ALL
SELECT 1,
1
UNION ALL
SELECT 1,
1
UNION ALL
SELECT 1,
1
UNION ALL
SELECT 1,
1
UNION ALL
SELECT 1,
1
UNION ALL
SELECT 1,
1
UNION ALL
SELECT 1,
1
UNION ALL
SELECT 1,
1
UNION ALL
SELECT 1,
1
UNION ALL
SELECT 1,
1
UNION ALL
SELECT 1,
1
UNION ALL
SELECT 1,
1
UNION ALL
SELECT 1,
1
UNION ALL
SELECT 1,
1
UNION ALL
SELECT 1,
1
UNION ALL
SELECT 1,
1
UNION ALL
SELECT 1,
1
UNION ALL
SELECT 1,
1
UNION ALL
SELECT 1,
1
UNION ALL
SELECT 1,
1
UNION ALL
SELECT 1,
1
UNION ALL
SELECT 1,
1
UNION ALL
SELECT 1,
1
UNION ALL
SELECT 1,
1
UNION ALL
SELECT 1,
1
UNION ALL
SELECT 1,
1
UNION ALL
SELECT 1,
1
UNION ALL
SELECT 1,
1
UNION ALL
SELECT 1,
1
UNION ALL
SELECT 1,
1
UNION ALL
SELECT 1,
1
UNION ALL
SELECT 1,
1
UNION ALL
SELECT 1,
1
UNION ALL
SELECT 1,
1
UNION ALL
SELECT 1,
1
UNION ALL
SELECT 1,
1
UNION ALL
SELECT 1,
1
UNION ALL
SELECT 1,
1
UNION ALL
SELECT 1,
1
UNION ALL
SELECT 1,
1
UNION ALL
SELECT 1,
1
UNION ALL
SELECT 1,
1
UNION ALL
SELECT 1,
1
UNION ALL
SELECT 1,
1
UNION ALL
SELECT 1,
1
UNION ALL
SELECT 1,
1
UNION ALL
SELECT 2,
1
UNION ALL
SELECT 2,
1
UNION ALL
SELECT 2,
1
UNION ALL
SELECT 2,
1
UNION ALL
SELECT 2,
1
UNION ALL
SELECT 2,
1
UNION ALL
SELECT 2,
1
UNION ALL
SELECT 2,
1
UNION ALL
SELECT 2,
1
UNION ALL
SELECT 2,
1
UNION ALL
SELECT 3,
1
UNION ALL
SELECT 3,
1
UNION ALL
SELECT 1,
2
UNION ALL
SELECT 1,
2
UNION ALL
SELECT 1,
2
UNION ALL
SELECT 1,
2
UNION ALL
SELECT 1,
2
UNION ALL
SELECT 1,
2
UNION ALL
SELECT 1,
2
UNION ALL
SELECT 1,
2
UNION ALL
SELECT 1,
2
UNION ALL
SELECT 1,
2
UNION ALL
SELECT 1,
2
UNION ALL
SELECT 1,
2
UNION ALL
SELECT 1,
2
UNION ALL
SELECT 1,
2
UNION ALL
SELECT 2,
2
UNION ALL
SELECT 2,
2
UNION ALL
SELECT 2,
2
UNION ALL
SELECT 2,
2
UNION ALL
SELECT 2,
2
UNION ALL
SELECT 2,
2
UNION ALL
SELECT 2,
2
UNION ALL
SELECT 2,
2
UNION ALL
SELECT 2,
2
UNION ALL
SELECT 2,
2
UNION ALL
SELECT 2,
2
UNION ALL
SELECT 2,
2
UNION ALL
SELECT 2,
2
UNION ALL
SELECT 2,
2
UNION ALL
SELECT 2,
2
UNION ALL
SELECT 2,
2
UNION ALL
SELECT 2,
2
UNION ALL
SELECT 2,
2
UNION ALL
SELECT 2,
2
UNION ALL
SELECT 2,
2
UNION ALL
SELECT 2,
2
UNION ALL
SELECT 2,
2
UNION ALL
SELECT 2,
2
UNION ALL
SELECT 2,
2
UNION ALL
SELECT 2,
2
UNION ALL
SELECT 2,
2
UNION ALL
SELECT 2,
2
UNION ALL
SELECT 2,
2
UNION ALL
SELECT 2,
2
UNION ALL
SELECT 2,
2
UNION ALL
SELECT 2,
2
UNION ALL
SELECT 2,
2
UNION ALL
SELECT 2,
2
UNION ALL
SELECT 2,
2
UNION ALL
SELECT 2,
2
UNION ALL
SELECT 2,
2
UNION ALL
SELECT 2,
2
UNION ALL
SELECT 2,
2
UNION ALL
SELECT 2,
2
UNION ALL
SELECT 2,
2
UNION ALL
SELECT 3,
2
UNION ALL
SELECT 3,
2
UNION ALL
SELECT 3,
2
UNION ALL
SELECT 3,
2
UNION ALL
SELECT 3,
2
UNION ALL
SELECT 3,
2
UNION ALL
SELECT 1,
3
UNION ALL
SELECT 1,
3
UNION ALL
SELECT 1,
3
UNION ALL
SELECT 1,
3
UNION ALL
SELECT 1,
3
UNION ALL
SELECT 1,
3
UNION ALL
SELECT 1,
3
UNION ALL
SELECT 1,
3
UNION ALL
SELECT 1,
3
UNION ALL
SELECT 1,
3
UNION ALL
SELECT 1,
3
UNION ALL
SELECT 1,
3
UNION ALL
SELECT 1,
3
UNION ALL
SELECT 1,
3
UNION ALL
SELECT 1,
3
UNION ALL
SELECT 1,
3
UNION ALL
SELECT 1,
3
UNION ALL
SELECT 1,
3
UNION ALL
SELECT 2,
3
UNION ALL
SELECT 2,
3
UNION ALL
SELECT 2,
3
UNION ALL
SELECT 2,
3
UNION ALL
SELECT 2,
3
UNION ALL
SELECT 2,
3
UNION ALL
SELECT 2,
3
UNION ALL
SELECT 2,
3
UNION ALL
SELECT 2,
3
UNION ALL
SELECT 2,
3
UNION ALL
SELECT 3,
3
UNION ALL
SELECT 3,
3
UNION ALL
SELECT 3,
3
UNION ALL
SELECT 3,
3
UNION ALL
SELECT 3,
3
UNION ALL
SELECT 3,
3
UNION ALL
SELECT 3,
3
UNION ALL
SELECT 3,
3
UNION ALL
SELECT 3,
3
UNION ALL
SELECT 3,
3
UNION ALL
SELECT 3,
3
UNION ALL
SELECT 3,
3
) n(r1, r2);
SELECT p.*
FROM wct.KAPPA_COHEN_TV('SELECT r1,r2 FROM #k', 'U') k
--This CROSS APPLY UNPIVOTS the tvf columns for formatting
CROSS APPLY
(
SELECT 'Pa',
Pa
UNION ALL
SELECT 'Pc',
Pc
UNION ALL
SELECT 'K',
K
UNION ALL
SELECT 'P',
P
UNION ALL
SELECT 'Z',
Z
UNION ALL
SELECT 'SE',
SE
UNION ALL
SELECT 'NS',
NS
UNION ALL
SELECT 'NR',
NR
) p(stat, val);
This produces the following result.
| stat | val |
|---|---|
| Pa | 0.7 |
| Pc | 0.41 |
| K | 0.491525423728813 |
| P | 3.19208256584873E-21 |
| Z | 9.45624243552736 |
| SE | 0.0519789363565954 |
| NS | 200 |
| NR | 3 |
Here an example using quadratic weighting.
with mycte (subj, rater, rating)
as (SELECT 1,
'R1',
'c'
UNION ALL
SELECT 1,
'R2',
'd'
UNION ALL
SELECT 2,
'R1',
'c'
UNION ALL
SELECT 2,
'R2',
'a'
UNION ALL
SELECT 3,
'R1',
'c'
UNION ALL
SELECT 3,
'R2',
'a'
UNION ALL
SELECT 4,
'R1',
'c'
UNION ALL
SELECT 4,
'R2',
'c'
UNION ALL
SELECT 5,
'R1',
'c'
UNION ALL
SELECT 5,
'R2',
'b'
UNION ALL
SELECT 6,
'R1',
'b'
UNION ALL
SELECT 6,
'R2',
'b'
UNION ALL
SELECT 7,
'R1',
'c'
UNION ALL
SELECT 7,
'R2',
'a'
UNION ALL
SELECT 8,
'R1',
'd'
UNION ALL
SELECT 8,
'R2',
'c'
UNION ALL
SELECT 9,
'R1',
'b'
UNION ALL
SELECT 9,
'R2',
'a'
UNION ALL
SELECT 10,
'R1',
'b'
UNION ALL
SELECT 10,
'R2',
'b'
UNION ALL
SELECT 11,
'R1',
'a'
UNION ALL
SELECT 11,
'R2',
'd'
UNION ALL
SELECT 12,
'R1',
'c'
UNION ALL
SELECT 12,
'R2',
'd'
UNION ALL
SELECT 13,
'R1',
'a'
UNION ALL
SELECT 13,
'R2',
'd'
UNION ALL
SELECT 14,
'R1',
'b'
UNION ALL
SELECT 14,
'R2',
'c'
UNION ALL
SELECT 15,
'R1',
'a'
UNION ALL
SELECT 15,
'R2',
'a'
UNION ALL
SELECT 16,
'R1',
'c'
UNION ALL
SELECT 16,
'R2',
'a'
UNION ALL
SELECT 17,
'R1',
'a'
UNION ALL
SELECT 17,
'R2',
'c'
UNION ALL
SELECT 18,
'R1',
'b'
UNION ALL
SELECT 18,
'R2',
'd'
UNION ALL
SELECT 19,
'R1',
'b'
UNION ALL
SELECT 19,
'R2',
'b'
UNION ALL
SELECT 20,
'R1',
'c'
UNION ALL
SELECT 20,
'R2',
'c'
UNION ALL
SELECT 21,
'R1',
'd'
UNION ALL
SELECT 21,
'R2',
'd'
UNION ALL
SELECT 22,
'R1',
'b'
UNION ALL
SELECT 22,
'R2',
'd'
UNION ALL
SELECT 23,
'R1',
'a'
UNION ALL
SELECT 23,
'R2',
'a'
UNION ALL
SELECT 24,
'R1',
'd'
UNION ALL
SELECT 24,
'R2',
'b'
UNION ALL
SELECT 25,
'R1',
'b'
UNION ALL
SELECT 25,
'R2',
'b')
SELECT m1.rating as r1,
m2.rating as r2
INTO #c
FROM mycte m1
JOIN mycte m2
ON m1.subj = m2.subj
WHERE m1.rater = 'R1'
AND m2.rater = 'R2';
SELECT p.*
FROM wct.KAPPA_COHEN_TV('SELECT r1,r2 FROM #C', 'Q') k
--This CROSS APPLY UNPIVOTS the tvf columns for formatting
CROSS APPLY
(
SELECT 'Pa',
Pa
UNION ALL
SELECT 'Pc',
Pc
UNION ALL
SELECT 'K',
K
UNION ALL
SELECT 'P',
P
UNION ALL
SELECT 'Z',
Z
UNION ALL
SELECT 'SE',
SE
UNION ALL
SELECT 'NS',
NS
UNION ALL
SELECT 'NR',
NR
) p(stat, val);
This produces the following result.
| stat | val |
|---|---|
| Pa | 0.751111111111111 |
| Pc | 0.749333333333333 |
| K | 0.00709219858156069 |
| P | 0.970935305534129 |
| Z | 0.0364352523330701 |
| SE | 0.194652105513855 |
| NS | 25 |
| NR | 4 |