Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 1895 in Layer 5

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 4.0267. Min Range: -4.0267

Max Act: 4.0267. Min Act: -0.1700

Data Index: 3634019 (Open Web Text)

Max Activating Token Index: 537

Text #1

Max Range: 4.0267. Min Range: -4.0267

Max Act: 3.9895. Min Act: -0.1700

Data Index: 1807607 (Open Web Text)

Max Activating Token Index: 606

Text #2

Max Range: 4.0267. Min Range: -4.0267

Max Act: 3.8495. Min Act: -0.1700

Data Index: 7133648 (Open Web Text)

Max Activating Token Index: 123

Text #3

Max Range: 4.0267. Min Range: -4.0267

Max Act: 3.8211. Min Act: -0.1700

Data Index: 2523443 (Open Web Text)

Max Activating Token Index: 202

Text #4

Max Range: 4.0267. Min Range: -4.0267

Max Act: 3.8371. Min Act: -0.1700

Data Index: 5963473 (Open Web Text)

Max Activating Token Index: 798

Text #5

Max Range: 4.0267. Min Range: -4.0267

Max Act: 3.8137. Min Act: -0.1700

Data Index: 1109257 (Open Web Text)

Max Activating Token Index: 152

Text #6

Max Range: 4.0267. Min Range: -4.0267

Max Act: 3.7477. Min Act: -0.1700

Data Index: 7729587 (Open Web Text)

Max Activating Token Index: 23

Text #7

Max Range: 4.0267. Min Range: -4.0267

Max Act: 3.7729. Min Act: -0.1700

Data Index: 6667905 (Open Web Text)

Max Activating Token Index: 499

Text #8

Max Range: 4.0267. Min Range: -4.0267

Max Act: 3.7541. Min Act: -0.1700

Data Index: 165036 (Open Web Text)

Max Activating Token Index: 173

Text #9

Max Range: 4.0267. Min Range: -4.0267

Max Act: 3.7507. Min Act: -0.1700

Data Index: 2469853 (Open Web Text)

Max Activating Token Index: 569

Text #10

Max Range: 4.0267. Min Range: -4.0267

Max Act: 3.7487. Min Act: -0.1700

Data Index: 8444580 (Open Web Text)

Max Activating Token Index: 861

Text #11

Max Range: 4.0267. Min Range: -4.0267

Max Act: 3.7569. Min Act: -0.1700

Data Index: 5918026 (Open Web Text)

Max Activating Token Index: 204

Text #12

Max Range: 4.0267. Min Range: -4.0267

Max Act: 3.7375. Min Act: -0.1700

Data Index: 2579343 (Open Web Text)

Max Activating Token Index: 120

Text #13

Max Range: 4.0267. Min Range: -4.0267

Max Act: 3.7501. Min Act: -0.1700

Data Index: 7562310 (Open Web Text)

Max Activating Token Index: 426

Text #14

Max Range: 4.0267. Min Range: -4.0267

Max Act: 3.7408. Min Act: -0.1700

Data Index: 1049961 (Open Web Text)

Max Activating Token Index: 172

Text #15

Max Range: 4.0267. Min Range: -4.0267

Max Act: 3.7572. Min Act: -0.1700

Data Index: 1109256 (Open Web Text)

Max Activating Token Index: 766

Text #16

Max Range: 4.0267. Min Range: -4.0267

Max Act: 3.7131. Min Act: -0.1700

Data Index: 6637917 (Open Web Text)

Max Activating Token Index: 261

Text #17

Max Range: 4.0267. Min Range: -4.0267

Max Act: 3.7445. Min Act: -0.1700

Data Index: 4773163 (Open Web Text)

Max Activating Token Index: 60

Text #18

Max Range: 4.0267. Min Range: -4.0267

Max Act: 3.7068. Min Act: -0.1700

Data Index: 406130 (Open Web Text)

Max Activating Token Index: 706

Text #19

Max Range: 4.0267. Min Range: -4.0267

Max Act: 3.7152. Min Act: -0.1700

Data Index: 2285072 (Open Web Text)

Max Activating Token Index: 532