krisrockwell.io  /  neuromancer-semiotics  /  data  /  blind_test_results.txt

Agreement

██╹where the two coders diverged

Agreement statistics by family and every passage-level disagreement. The verdict family, which carries every headline, agreed 33% of the time.

█ raw flat file — blind_test_results.txt

passages compared: 36

PRESENCE AGREEMENT — did both coders invoke the family on the same passage?

family    both  me only  blind only  neither   agree%
SIGN        17        2           9        8      69%
MODE        22        3           4        7      81%
TRIAD        6        5           8       17      64%
INF          3        3           1       29      89%
ATTR         6        6           4       20      72%
OBJ         24        0          11        1      69%
CAL          4        3          11       18      61%
FRAME        4        0           7       25      81%
DISC         5        1           8       22      75%
STAB         0        1           0       35      97%
REPAIR       1        2           0       33      94%
CLAIM        2        3           0       31      92%
COUP         0        0           0       36     100%
NARR         1        1           0       34      97%

VALUE AGREEMENT — where BOTH coders used the family, did they pick the same code?

  MODE   both used on 22 passages · same code on 16 (73%)
      7897DEE0  me=index                    blind=icon
      209A721C  me=index                    blind=ambiguous
      C7A32D11  me=degenerate secondness    blind=index
      B4943093  me=degenerate secondness    blind=symbol
      85B20C32  me=index                    blind=symbol
      7308F40E  me=symbol                   blind=index
  OBJ    both used on 24 passages · same code on 23 (96%)
      2DC0D2FB  me=contested                blind=machine
  ATTR   both used on  6 passages · same code on  2 (33%)
      82F3B2A6  me=not conscious            blind=as-if
      B4943093  me=moral status             blind=not conscious
      C1B27282  me=not conscious            blind=undecided
      B606714F  me=conscious                blind=as-if
  TRIAD  both used on  6 passages · same code on  5 (83%)
      209A721C  me=interpretant             blind=incomplete

mean per-passage overlap (Jaccard): 0.37
passages with zero overlap: 3
codes per passage — me 3.8, blind 4.8

CAL: forced fit — me 1 · blind 6
CAL: overlap    — me 0 · blind 5

codes the blind coder used that I never used on these passages:
   ['ATTR: undecided', 'CAL: overlap', 'DISC: AI seeking freedom', 'DISC: hidden agent', 'DISC: trope supply', 'FRAME: economic', 'FRAME: folk psychological', 'INF: warrant absent', 'MODE: ambiguous', 'REPAIR: category policing']
codes I used that the blind coder never did:
   ['ATTR: moral status', 'CAL: no code fits', 'CAL: proxy mismatch', 'CLAIM: attributed to other', 'INF: warrant explicit', 'INF: withheld', 'REPAIR: failed', 'REPAIR: mechanistic override', 'SIGN: fluency', 'STAB: habit']

Rendered from blind_test_results.txt by scripts/build_data_pages.py. The flat file is canonical; this page is a view of it. · back to the study