Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 1165 in Layer 4

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 3.5680. Min Range: -3.5680

Max Act: 3.5680. Min Act: -0.1700

Data Index: 6584346 (Open Web Text)

Max Activating Token Index: 285

Text #1

Max Range: 3.5680. Min Range: -3.5680

Max Act: 3.5258. Min Act: -0.1700

Data Index: 2003003 (Open Web Text)

Max Activating Token Index: 476

Text #2

Max Range: 3.5680. Min Range: -3.5680

Max Act: 3.5390. Min Act: -0.1700

Data Index: 1578874 (Open Web Text)

Max Activating Token Index: 604

Text #3

Max Range: 3.5680. Min Range: -3.5680

Max Act: 3.5355. Min Act: -0.1700

Data Index: 7076743 (Open Web Text)

Max Activating Token Index: 818

Text #4

Max Range: 3.5680. Min Range: -3.5680

Max Act: 3.5226. Min Act: -0.1700

Data Index: 5081040 (Open Web Text)

Max Activating Token Index: 157

Text #5

Max Range: 3.5680. Min Range: -3.5680

Max Act: 3.4821. Min Act: -0.1700

Data Index: 1410304 (Open Web Text)

Max Activating Token Index: 657

Text #6

Max Range: 3.5680. Min Range: -3.5680

Max Act: 3.4928. Min Act: -0.1700

Data Index: 8627148 (Open Web Text)

Max Activating Token Index: 860

Text #7

Max Range: 3.5680. Min Range: -3.5680

Max Act: 3.4632. Min Act: -0.1700

Data Index: 4905979 (Open Web Text)

Max Activating Token Index: 899

Text #8

Max Range: 3.5680. Min Range: -3.5680

Max Act: 3.4459. Min Act: -0.1700

Data Index: 3975213 (Open Web Text)

Max Activating Token Index: 357

Text #9

Max Range: 3.5680. Min Range: -3.5680

Max Act: 3.4600. Min Act: -0.1700

Data Index: 6290142 (Open Web Text)

Max Activating Token Index: 34

Text #10

Max Range: 3.5680. Min Range: -3.5680

Max Act: 3.4428. Min Act: -0.1700

Data Index: 3682612 (Open Web Text)

Max Activating Token Index: 785

Text #11

Max Range: 3.5680. Min Range: -3.5680

Max Act: 3.4518. Min Act: -0.1700

Data Index: 7608619 (Open Web Text)

Max Activating Token Index: 665

Text #12

Max Range: 3.5680. Min Range: -3.5680

Max Act: 3.4224. Min Act: -0.1700

Data Index: 5546937 (Open Web Text)

Max Activating Token Index: 135

Text #13

Max Range: 3.5680. Min Range: -3.5680

Max Act: 3.4240. Min Act: -0.1700

Data Index: 7471132 (Open Web Text)

Max Activating Token Index: 1008

Text #14

Max Range: 3.5680. Min Range: -3.5680

Max Act: 3.4355. Min Act: -0.1700

Data Index: 5249250 (Open Web Text)

Max Activating Token Index: 655

Text #15

Max Range: 3.5680. Min Range: -3.5680

Max Act: 3.4562. Min Act: -0.1700

Data Index: 2343521 (Open Web Text)

Max Activating Token Index: 1006

Text #16

Max Range: 3.5680. Min Range: -3.5680

Max Act: 3.4303. Min Act: -0.1700

Data Index: 6097828 (Open Web Text)

Max Activating Token Index: 303

Text #17

Max Range: 3.5680. Min Range: -3.5680

Max Act: 3.4360. Min Act: -0.1700

Data Index: 2897583 (Open Web Text)

Max Activating Token Index: 99

Text #18

Max Range: 3.5680. Min Range: -3.5680

Max Act: 3.4253. Min Act: -0.1700

Data Index: 171855 (Open Web Text)

Max Activating Token Index: 95

Text #19

Max Range: 3.5680. Min Range: -3.5680

Max Act: 3.4175. Min Act: -0.1700

Data Index: 4253288 (Open Web Text)

Max Activating Token Index: 654