Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 2108 in Layer 5

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 4.3210. Min Range: -4.3210

Max Act: 4.3102. Min Act: -0.1700

Data Index: 2218018 (Open Web Text)

Max Activating Token Index: 913

Text #1

Max Range: 4.3210. Min Range: -4.3210

Max Act: 4.3210. Min Act: -0.1700

Data Index: 2022122 (Open Web Text)

Max Activating Token Index: 319

Text #2

Max Range: 4.3210. Min Range: -4.3210

Max Act: 4.2646. Min Act: -0.1700

Data Index: 2164728 (Open Web Text)

Max Activating Token Index: 488

Text #3

Max Range: 4.3210. Min Range: -4.3210

Max Act: 4.2374. Min Act: -0.1700

Data Index: 6157772 (Open Web Text)

Max Activating Token Index: 457

Text #4

Max Range: 4.3210. Min Range: -4.3210

Max Act: 4.1969. Min Act: -0.1700

Data Index: 3528627 (Open Web Text)

Max Activating Token Index: 924

Text #5

Max Range: 4.3210. Min Range: -4.3210

Max Act: 4.2175. Min Act: -0.1700

Data Index: 2908651 (Open Web Text)

Max Activating Token Index: 424

Text #6

Max Range: 4.3210. Min Range: -4.3210

Max Act: 4.1704. Min Act: -0.1700

Data Index: 8589601 (Open Web Text)

Max Activating Token Index: 666

Text #7

Max Range: 4.3210. Min Range: -4.3210

Max Act: 4.1299. Min Act: -0.1700

Data Index: 3429307 (Open Web Text)

Max Activating Token Index: 873

Text #8

Max Range: 4.3210. Min Range: -4.3210

Max Act: 4.1430. Min Act: -0.1700

Data Index: 8560403 (Open Web Text)

Max Activating Token Index: 775

Text #9

Max Range: 4.3210. Min Range: -4.3210

Max Act: 4.1559. Min Act: -0.1700

Data Index: 2499793 (Open Web Text)

Max Activating Token Index: 79

Text #10

Max Range: 4.3210. Min Range: -4.3210

Max Act: 4.1656. Min Act: -0.1700

Data Index: 376865 (Open Web Text)

Max Activating Token Index: 771

Text #11

Max Range: 4.3210. Min Range: -4.3210

Max Act: 4.1381. Min Act: -0.1700

Data Index: 8483850 (Open Web Text)

Max Activating Token Index: 767

Text #12

Max Range: 4.3210. Min Range: -4.3210

Max Act: 4.1260. Min Act: -0.1700

Data Index: 7394406 (Open Web Text)

Max Activating Token Index: 66

Text #13

Max Range: 4.3210. Min Range: -4.3210

Max Act: 4.1138. Min Act: -0.1700

Data Index: 1702286 (Open Web Text)

Max Activating Token Index: 859

Text #14

Max Range: 4.3210. Min Range: -4.3210

Max Act: 4.0897. Min Act: -0.1700

Data Index: 5168858 (Open Web Text)

Max Activating Token Index: 575

Text #15

Max Range: 4.3210. Min Range: -4.3210

Max Act: 4.1139. Min Act: -0.1700

Data Index: 4569053 (Open Web Text)

Max Activating Token Index: 463

Text #16

Max Range: 4.3210. Min Range: -4.3210

Max Act: 4.1085. Min Act: -0.1700

Data Index: 4559204 (Open Web Text)

Max Activating Token Index: 203

Text #17

Max Range: 4.3210. Min Range: -4.3210

Max Act: 4.1225. Min Act: -0.1700

Data Index: 3082793 (Open Web Text)

Max Activating Token Index: 113

Text #18

Max Range: 4.3210. Min Range: -4.3210

Max Act: 4.1271. Min Act: -0.1700

Data Index: 7317562 (Open Web Text)

Max Activating Token Index: 138

Text #19

Max Range: 4.3210. Min Range: -4.3210

Max Act: 4.1150. Min Act: -0.1700

Data Index: 6767852 (Open Web Text)

Max Activating Token Index: 272