Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 849 in Layer 6

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 3.5343. Min Range: -3.5343

Max Act: 3.5343. Min Act: -0.1700

Data Index: 5968607 (Open Web Text)

Max Activating Token Index: 757

Text #1

Max Range: 3.5343. Min Range: -3.5343

Max Act: 3.5277. Min Act: -0.1700

Data Index: 3474276 (Open Web Text)

Max Activating Token Index: 669

Text #2

Max Range: 3.5343. Min Range: -3.5343

Max Act: 3.5029. Min Act: -0.1700

Data Index: 5509086 (Open Web Text)

Max Activating Token Index: 105

Text #3

Max Range: 3.5343. Min Range: -3.5343

Max Act: 3.4797. Min Act: -0.1700

Data Index: 8688945 (Open Web Text)

Max Activating Token Index: 899

Text #4

Max Range: 3.5343. Min Range: -3.5343

Max Act: 3.4681. Min Act: -0.1700

Data Index: 4628365 (Open Web Text)

Max Activating Token Index: 1020

Text #5

Max Range: 3.5343. Min Range: -3.5343

Max Act: 3.4715. Min Act: -0.1700

Data Index: 2042750 (Open Web Text)

Max Activating Token Index: 327

Text #6

Max Range: 3.5343. Min Range: -3.5343

Max Act: 3.4665. Min Act: -0.1700

Data Index: 5732247 (Open Web Text)

Max Activating Token Index: 513

Text #7

Max Range: 3.5343. Min Range: -3.5343

Max Act: 3.4563. Min Act: -0.1700

Data Index: 375523 (Open Web Text)

Max Activating Token Index: 61

Text #8

Max Range: 3.5343. Min Range: -3.5343

Max Act: 3.4395. Min Act: -0.1700

Data Index: 5487470 (Open Web Text)

Max Activating Token Index: 146

Text #9

Max Range: 3.5343. Min Range: -3.5343

Max Act: 3.4243. Min Act: -0.1700

Data Index: 5610606 (Open Web Text)

Max Activating Token Index: 293

Text #10

Max Range: 3.5343. Min Range: -3.5343

Max Act: 3.4349. Min Act: -0.1700

Data Index: 4595271 (Open Web Text)

Max Activating Token Index: 530

Text #11

Max Range: 3.5343. Min Range: -3.5343

Max Act: 3.4151. Min Act: -0.1700

Data Index: 4701429 (Open Web Text)

Max Activating Token Index: 374

Text #12

Max Range: 3.5343. Min Range: -3.5343

Max Act: 3.4378. Min Act: -0.1700

Data Index: 980608 (Open Web Text)

Max Activating Token Index: 719

Text #13

Max Range: 3.5343. Min Range: -3.5343

Max Act: 3.4096. Min Act: -0.1700

Data Index: 1106402 (Open Web Text)

Max Activating Token Index: 398

Text #14

Max Range: 3.5343. Min Range: -3.5343

Max Act: 3.4231. Min Act: -0.1700

Data Index: 6880125 (Open Web Text)

Max Activating Token Index: 667

Text #15

Max Range: 3.5343. Min Range: -3.5343

Max Act: 3.4208. Min Act: -0.1700

Data Index: 6419948 (Open Web Text)

Max Activating Token Index: 121

Text #16

Max Range: 3.5343. Min Range: -3.5343

Max Act: 3.4081. Min Act: -0.1700

Data Index: 3030180 (Open Web Text)

Max Activating Token Index: 87

Text #17

Max Range: 3.5343. Min Range: -3.5343

Max Act: 3.3947. Min Act: -0.1700

Data Index: 7244504 (Open Web Text)

Max Activating Token Index: 730

Text #18

Max Range: 3.5343. Min Range: -3.5343

Max Act: 3.3986. Min Act: -0.1700

Data Index: 2150004 (Open Web Text)

Max Activating Token Index: 96

Text #19

Max Range: 3.5343. Min Range: -3.5343

Max Act: 3.4075. Min Act: -0.1700

Data Index: 6763152 (Open Web Text)

Max Activating Token Index: 79