Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 1288 in Layer 5

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 5.6379. Min Range: -5.6379

Max Act: 5.6011. Min Act: -0.1700

Data Index: 1160253 (Open Web Text)

Max Activating Token Index: 395

Text #1

Max Range: 5.6379. Min Range: -5.6379

Max Act: 5.6379. Min Act: -0.1700

Data Index: 900335 (Open Web Text)

Max Activating Token Index: 475

Text #2

Max Range: 5.6379. Min Range: -5.6379

Max Act: 5.5787. Min Act: -0.1700

Data Index: 3097442 (Open Web Text)

Max Activating Token Index: 951

Text #3

Max Range: 5.6379. Min Range: -5.6379

Max Act: 5.5586. Min Act: -0.1700

Data Index: 6130800 (Open Web Text)

Max Activating Token Index: 923

Text #4

Max Range: 5.6379. Min Range: -5.6379

Max Act: 5.5213. Min Act: -0.1700

Data Index: 5148822 (Open Web Text)

Max Activating Token Index: 980

Text #5

Max Range: 5.6379. Min Range: -5.6379

Max Act: 5.4187. Min Act: -0.1700

Data Index: 3812511 (Open Web Text)

Max Activating Token Index: 948

Text #6

Max Range: 5.6379. Min Range: -5.6379

Max Act: 5.4684. Min Act: -0.1700

Data Index: 7410044 (Open Web Text)

Max Activating Token Index: 984

Text #7

Max Range: 5.6379. Min Range: -5.6379

Max Act: 5.3950. Min Act: -0.1700

Data Index: 5068104 (Open Web Text)

Max Activating Token Index: 103

Text #8

Max Range: 5.6379. Min Range: -5.6379

Max Act: 5.4309. Min Act: -0.1700

Data Index: 2340717 (Open Web Text)

Max Activating Token Index: 929

Text #9

Max Range: 5.6379. Min Range: -5.6379

Max Act: 5.4017. Min Act: -0.1700

Data Index: 8138269 (Open Web Text)

Max Activating Token Index: 356

Text #10

Max Range: 5.6379. Min Range: -5.6379

Max Act: 5.3891. Min Act: -0.1700

Data Index: 3305341 (Open Web Text)

Max Activating Token Index: 49

Text #11

Max Range: 5.6379. Min Range: -5.6379

Max Act: 5.4127. Min Act: -0.1700

Data Index: 8164055 (Open Web Text)

Max Activating Token Index: 788

Text #12

Max Range: 5.6379. Min Range: -5.6379

Max Act: 5.3777. Min Act: -0.1700

Data Index: 8085961 (Open Web Text)

Max Activating Token Index: 523

Text #13

Max Range: 5.6379. Min Range: -5.6379

Max Act: 5.3563. Min Act: -0.1700

Data Index: 2624502 (Open Web Text)

Max Activating Token Index: 319

Text #14

Max Range: 5.6379. Min Range: -5.6379

Max Act: 5.3461. Min Act: -0.1700

Data Index: 8563686 (Open Web Text)

Max Activating Token Index: 285

Text #15

Max Range: 5.6379. Min Range: -5.6379

Max Act: 5.3455. Min Act: -0.1700

Data Index: 7678389 (Open Web Text)

Max Activating Token Index: 410

Text #16

Max Range: 5.6379. Min Range: -5.6379

Max Act: 5.3481. Min Act: -0.1700

Data Index: 6272512 (Open Web Text)

Max Activating Token Index: 416

Text #17

Max Range: 5.6379. Min Range: -5.6379

Max Act: 5.3141. Min Act: -0.1700

Data Index: 2756425 (Open Web Text)

Max Activating Token Index: 803

Text #18

Max Range: 5.6379. Min Range: -5.6379

Max Act: 5.3184. Min Act: -0.1700

Data Index: 7776719 (Open Web Text)

Max Activating Token Index: 542

Text #19

Max Range: 5.6379. Min Range: -5.6379

Max Act: 5.3436. Min Act: -0.1700

Data Index: 504749 (Open Web Text)

Max Activating Token Index: 951