Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 1889 in Layer 1

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 1.0197. Min Range: -1.0197

Max Act: 1.0197. Min Act: -0.1700

Data Index: 2953616 (Open Web Text)

Max Activating Token Index: 22

Text #1

Max Range: 1.0197. Min Range: -1.0197

Max Act: 1.0190. Min Act: -0.1700

Data Index: 7573843 (Open Web Text)

Max Activating Token Index: 27

Text #2

Max Range: 1.0197. Min Range: -1.0197

Max Act: 0.9998. Min Act: -0.1700

Data Index: 6869158 (Open Web Text)

Max Activating Token Index: 16

Text #3

Max Range: 1.0197. Min Range: -1.0197

Max Act: 0.9727. Min Act: -0.1700

Data Index: 2412958 (Open Web Text)

Max Activating Token Index: 780

Text #4

Max Range: 1.0197. Min Range: -1.0197

Max Act: 0.9796. Min Act: -0.1700

Data Index: 8160082 (Open Web Text)

Max Activating Token Index: 19

Text #5

Max Range: 1.0197. Min Range: -1.0197

Max Act: 0.9871. Min Act: -0.1696

Data Index: 1996021 (Open Web Text)

Max Activating Token Index: 136

Text #6

Max Range: 1.0197. Min Range: -1.0197

Max Act: 0.9903. Min Act: -0.1700

Data Index: 1850772 (Open Web Text)

Max Activating Token Index: 307

Text #7

Max Range: 1.0197. Min Range: -1.0197

Max Act: 0.9755. Min Act: -0.1700

Data Index: 2685275 (Open Web Text)

Max Activating Token Index: 802

Text #8

Max Range: 1.0197. Min Range: -1.0197

Max Act: 0.9657. Min Act: -0.1699

Data Index: 5670851 (Open Web Text)

Max Activating Token Index: 18

Text #9

Max Range: 1.0197. Min Range: -1.0197

Max Act: 0.9653. Min Act: -0.1700

Data Index: 7539987 (Open Web Text)

Max Activating Token Index: 13

Text #10

Max Range: 1.0197. Min Range: -1.0197

Max Act: 0.9527. Min Act: -0.1700

Data Index: 7900607 (Open Web Text)

Max Activating Token Index: 788

Text #11

Max Range: 1.0197. Min Range: -1.0197

Max Act: 0.9660. Min Act: -0.1669

Data Index: 5172999 (Open Web Text)

Max Activating Token Index: 787

Text #12

Max Range: 1.0197. Min Range: -1.0197

Max Act: 0.9687. Min Act: -0.1700

Data Index: 2503242 (Open Web Text)

Max Activating Token Index: 755

Text #13

Max Range: 1.0197. Min Range: -1.0197

Max Act: 0.9685. Min Act: -0.1700

Data Index: 2499207 (Open Web Text)

Max Activating Token Index: 281

Text #14

Max Range: 1.0197. Min Range: -1.0197

Max Act: 0.9617. Min Act: -0.1700

Data Index: 6009952 (Open Web Text)

Max Activating Token Index: 817

Text #15

Max Range: 1.0197. Min Range: -1.0197

Max Act: 0.9559. Min Act: -0.1700

Data Index: 2824404 (Open Web Text)

Max Activating Token Index: 222

Text #16

Max Range: 1.0197. Min Range: -1.0197

Max Act: 0.9468. Min Act: -0.1700

Data Index: 685217 (Open Web Text)

Max Activating Token Index: 821

Text #17

Max Range: 1.0197. Min Range: -1.0197

Max Act: 0.9738. Min Act: -0.1700

Data Index: 7662320 (Open Web Text)

Max Activating Token Index: 798

Text #18

Max Range: 1.0197. Min Range: -1.0197

Max Act: 0.9565. Min Act: -0.1700

Data Index: 3615938 (Open Web Text)

Max Activating Token Index: 50

Text #19

Max Range: 1.0197. Min Range: -1.0197

Max Act: 0.9569. Min Act: -0.1700

Data Index: 7014868 (Open Web Text)

Max Activating Token Index: 818