Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 1443 in Layer 5

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 2.8031. Min Range: -2.8031

Max Act: 2.8031. Min Act: -0.1700

Data Index: 4863325 (Open Web Text)

Max Activating Token Index: 187

Text #1

Max Range: 2.8031. Min Range: -2.8031

Max Act: 2.7326. Min Act: -0.1700

Data Index: 8045460 (Open Web Text)

Max Activating Token Index: 32

Text #2

Max Range: 2.8031. Min Range: -2.8031

Max Act: 2.7211. Min Act: -0.1700

Data Index: 7232713 (Open Web Text)

Max Activating Token Index: 129

Text #3

Max Range: 2.8031. Min Range: -2.8031

Max Act: 2.6978. Min Act: -0.1700

Data Index: 4264653 (Open Web Text)

Max Activating Token Index: 813

Text #4

Max Range: 2.8031. Min Range: -2.8031

Max Act: 2.7037. Min Act: -0.1700

Data Index: 4915534 (Open Web Text)

Max Activating Token Index: 151

Text #5

Max Range: 2.8031. Min Range: -2.8031

Max Act: 2.6729. Min Act: -0.1700

Data Index: 4422762 (Open Web Text)

Max Activating Token Index: 115

Text #6

Max Range: 2.8031. Min Range: -2.8031

Max Act: 2.6788. Min Act: -0.1700

Data Index: 8447162 (Open Web Text)

Max Activating Token Index: 587

Text #7

Max Range: 2.8031. Min Range: -2.8031

Max Act: 2.6655. Min Act: -0.1700

Data Index: 4263573 (Open Web Text)

Max Activating Token Index: 637

Text #8

Max Range: 2.8031. Min Range: -2.8031

Max Act: 2.6433. Min Act: -0.1700

Data Index: 4169473 (Open Web Text)

Max Activating Token Index: 248

Text #9

Max Range: 2.8031. Min Range: -2.8031

Max Act: 2.6316. Min Act: -0.1700

Data Index: 919300 (Open Web Text)

Max Activating Token Index: 29

Text #10

Max Range: 2.8031. Min Range: -2.8031

Max Act: 2.6436. Min Act: -0.1700

Data Index: 3093265 (Open Web Text)

Max Activating Token Index: 523

Text #11

Max Range: 2.8031. Min Range: -2.8031

Max Act: 2.6417. Min Act: -0.1700

Data Index: 1251119 (Open Web Text)

Max Activating Token Index: 681

Text #12

Max Range: 2.8031. Min Range: -2.8031

Max Act: 2.6099. Min Act: -0.1700

Data Index: 2557187 (Open Web Text)

Max Activating Token Index: 371

Text #13

Max Range: 2.8031. Min Range: -2.8031

Max Act: 2.6257. Min Act: -0.1700

Data Index: 7963651 (Open Web Text)

Max Activating Token Index: 703

Text #14

Max Range: 2.8031. Min Range: -2.8031

Max Act: 2.6274. Min Act: -0.1700

Data Index: 3563699 (Open Web Text)

Max Activating Token Index: 932

Text #15

Max Range: 2.8031. Min Range: -2.8031

Max Act: 2.6452. Min Act: -0.1700

Data Index: 1146235 (Open Web Text)

Max Activating Token Index: 113

Text #16

Max Range: 2.8031. Min Range: -2.8031

Max Act: 2.6246. Min Act: -0.1700

Data Index: 792110 (Open Web Text)

Max Activating Token Index: 257

Text #17

Max Range: 2.8031. Min Range: -2.8031

Max Act: 2.6168. Min Act: -0.1700

Data Index: 2743761 (Open Web Text)

Max Activating Token Index: 569

Text #18

Max Range: 2.8031. Min Range: -2.8031

Max Act: 2.5902. Min Act: -0.1700

Data Index: 6864986 (Open Web Text)

Max Activating Token Index: 213

Text #19

Max Range: 2.8031. Min Range: -2.8031

Max Act: 2.5965. Min Act: -0.1700

Data Index: 7657832 (Open Web Text)

Max Activating Token Index: 937