Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 2928 in Layer 4

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 4.1287. Min Range: -4.1287

Max Act: 4.1161. Min Act: -0.1700

Data Index: 6022283 (Open Web Text)

Max Activating Token Index: 303

Text #1

Max Range: 4.1287. Min Range: -4.1287

Max Act: 4.1287. Min Act: -0.1700

Data Index: 7494638 (Open Web Text)

Max Activating Token Index: 749

Text #2

Max Range: 4.1287. Min Range: -4.1287

Max Act: 4.0413. Min Act: -0.1700

Data Index: 5913136 (Open Web Text)

Max Activating Token Index: 111

Text #3

Max Range: 4.1287. Min Range: -4.1287

Max Act: 4.0119. Min Act: -0.1700

Data Index: 5994105 (Open Web Text)

Max Activating Token Index: 593

Text #4

Max Range: 4.1287. Min Range: -4.1287

Max Act: 3.9966. Min Act: -0.1700

Data Index: 3681182 (Open Web Text)

Max Activating Token Index: 488

Text #5

Max Range: 4.1287. Min Range: -4.1287

Max Act: 4.0223. Min Act: -0.1700

Data Index: 4538683 (Open Web Text)

Max Activating Token Index: 182

Text #6

Max Range: 4.1287. Min Range: -4.1287

Max Act: 3.9977. Min Act: -0.1700

Data Index: 1353684 (Open Web Text)

Max Activating Token Index: 205

Text #7

Max Range: 4.1287. Min Range: -4.1287

Max Act: 3.9856. Min Act: -0.1700

Data Index: 2371606 (Open Web Text)

Max Activating Token Index: 936

Text #8

Max Range: 4.1287. Min Range: -4.1287

Max Act: 3.9654. Min Act: -0.1700

Data Index: 7036754 (Open Web Text)

Max Activating Token Index: 347

Text #9

Max Range: 4.1287. Min Range: -4.1287

Max Act: 3.9989. Min Act: -0.1700

Data Index: 4747644 (Open Web Text)

Max Activating Token Index: 67

Text #10

Max Range: 4.1287. Min Range: -4.1287

Max Act: 3.9977. Min Act: -0.1700

Data Index: 6066513 (Open Web Text)

Max Activating Token Index: 186

Text #11

Max Range: 4.1287. Min Range: -4.1287

Max Act: 3.9924. Min Act: -0.1700

Data Index: 8435316 (Open Web Text)

Max Activating Token Index: 843

Text #12

Max Range: 4.1287. Min Range: -4.1287

Max Act: 3.9728. Min Act: -0.1700

Data Index: 2558658 (Open Web Text)

Max Activating Token Index: 87

Text #13

Max Range: 4.1287. Min Range: -4.1287

Max Act: 3.9833. Min Act: -0.1700

Data Index: 7282201 (Open Web Text)

Max Activating Token Index: 219

Text #14

Max Range: 4.1287. Min Range: -4.1287

Max Act: 3.9637. Min Act: -0.1700

Data Index: 2431379 (Open Web Text)

Max Activating Token Index: 444

Text #15

Max Range: 4.1287. Min Range: -4.1287

Max Act: 3.9381. Min Act: -0.1700

Data Index: 2776272 (Open Web Text)

Max Activating Token Index: 747

Text #16

Max Range: 4.1287. Min Range: -4.1287

Max Act: 3.9725. Min Act: -0.1700

Data Index: 2131563 (Open Web Text)

Max Activating Token Index: 110

Text #17

Max Range: 4.1287. Min Range: -4.1287

Max Act: 3.9759. Min Act: -0.1700

Data Index: 6135282 (Open Web Text)

Max Activating Token Index: 341

Text #18

Max Range: 4.1287. Min Range: -4.1287

Max Act: 3.9356. Min Act: -0.1700

Data Index: 2372529 (Open Web Text)

Max Activating Token Index: 497

Text #19

Max Range: 4.1287. Min Range: -4.1287

Max Act: 3.9424. Min Act: -0.1700

Data Index: 5807767 (Open Web Text)

Max Activating Token Index: 944