Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 1902 in Layer 1

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 1.7159. Min Range: -1.7159

Max Act: 1.7042. Min Act: -0.1700

Data Index: 56717 (Open Web Text)

Max Activating Token Index: 300

Text #1

Max Range: 1.7159. Min Range: -1.7159

Max Act: 1.7159. Min Act: -0.1700

Data Index: 7207674 (Open Web Text)

Max Activating Token Index: 701

Text #2

Max Range: 1.7159. Min Range: -1.7159

Max Act: 1.7028. Min Act: -0.1700

Data Index: 2418294 (Open Web Text)

Max Activating Token Index: 536

Text #3

Max Range: 1.7159. Min Range: -1.7159

Max Act: 1.6829. Min Act: -0.1696

Data Index: 5189096 (Open Web Text)

Max Activating Token Index: 479

Text #4

Max Range: 1.7159. Min Range: -1.7159

Max Act: 1.6763. Min Act: -0.1690

Data Index: 6375275 (Open Web Text)

Max Activating Token Index: 101

Text #5

Max Range: 1.7159. Min Range: -1.7159

Max Act: 1.6770. Min Act: -0.1700

Data Index: 792655 (Open Web Text)

Max Activating Token Index: 629

Text #6

Max Range: 1.7159. Min Range: -1.7159

Max Act: 1.6818. Min Act: -0.1700

Data Index: 6287330 (Open Web Text)

Max Activating Token Index: 287

Text #7

Max Range: 1.7159. Min Range: -1.7159

Max Act: 1.6599. Min Act: -0.1700

Data Index: 4879698 (Open Web Text)

Max Activating Token Index: 702

Text #8

Max Range: 1.7159. Min Range: -1.7159

Max Act: 1.6711. Min Act: -0.1700

Data Index: 6306995 (Open Web Text)

Max Activating Token Index: 439

Text #9

Max Range: 1.7159. Min Range: -1.7159

Max Act: 1.6637. Min Act: -0.1700

Data Index: 4103076 (Open Web Text)

Max Activating Token Index: 915

Text #10

Max Range: 1.7159. Min Range: -1.7159

Max Act: 1.6574. Min Act: -0.1700

Data Index: 2653386 (Open Web Text)

Max Activating Token Index: 187

Text #11

Max Range: 1.7159. Min Range: -1.7159

Max Act: 1.6396. Min Act: -0.1700

Data Index: 3383135 (Open Web Text)

Max Activating Token Index: 443

Text #12

Max Range: 1.7159. Min Range: -1.7159

Max Act: 1.6414. Min Act: -0.1700

Data Index: 5337849 (Open Web Text)

Max Activating Token Index: 249

Text #13

Max Range: 1.7159. Min Range: -1.7159

Max Act: 1.6414. Min Act: -0.1700

Data Index: 7668534 (Open Web Text)

Max Activating Token Index: 48

Text #14

Max Range: 1.7159. Min Range: -1.7159

Max Act: 1.6384. Min Act: -0.1700

Data Index: 6285032 (Open Web Text)

Max Activating Token Index: 863

Text #15

Max Range: 1.7159. Min Range: -1.7159

Max Act: 1.6321. Min Act: -0.1700

Data Index: 4145627 (Open Web Text)

Max Activating Token Index: 165

Text #16

Max Range: 1.7159. Min Range: -1.7159

Max Act: 1.6400. Min Act: -0.1700

Data Index: 4276971 (Open Web Text)

Max Activating Token Index: 746

Text #17

Max Range: 1.7159. Min Range: -1.7159

Max Act: 1.6326. Min Act: -0.1700

Data Index: 2689837 (Open Web Text)

Max Activating Token Index: 216

Text #18

Max Range: 1.7159. Min Range: -1.7159

Max Act: 1.6381. Min Act: -0.1700

Data Index: 6060083 (Open Web Text)

Max Activating Token Index: 504

Text #19

Max Range: 1.7159. Min Range: -1.7159

Max Act: 1.6337. Min Act: -0.1700

Data Index: 3792105 (Open Web Text)

Max Activating Token Index: 145