Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 754 in Layer 3

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 3.1007. Min Range: -3.1007

Max Act: 3.0712. Min Act: -0.1700

Data Index: 1431568 (Open Web Text)

Max Activating Token Index: 774

Text #1

Max Range: 3.1007. Min Range: -3.1007

Max Act: 3.1007. Min Act: -0.1700

Data Index: 3283164 (Open Web Text)

Max Activating Token Index: 19

Text #2

Max Range: 3.1007. Min Range: -3.1007

Max Act: 3.0623. Min Act: -0.1700

Data Index: 1510931 (Open Web Text)

Max Activating Token Index: 191

Text #3

Max Range: 3.1007. Min Range: -3.1007

Max Act: 3.0637. Min Act: -0.1700

Data Index: 3062838 (Open Web Text)

Max Activating Token Index: 238

Text #4

Max Range: 3.1007. Min Range: -3.1007

Max Act: 3.0158. Min Act: -0.1700

Data Index: 5081447 (Open Web Text)

Max Activating Token Index: 1009

Text #5

Max Range: 3.1007. Min Range: -3.1007

Max Act: 3.0420. Min Act: -0.1700

Data Index: 5356510 (Open Web Text)

Max Activating Token Index: 522

Text #6

Max Range: 3.1007. Min Range: -3.1007

Max Act: 3.0408. Min Act: -0.1700

Data Index: 8774423 (Open Web Text)

Max Activating Token Index: 287

Text #7

Max Range: 3.1007. Min Range: -3.1007

Max Act: 3.0364. Min Act: -0.1700

Data Index: 77718 (Open Web Text)

Max Activating Token Index: 946

Text #8

Max Range: 3.1007. Min Range: -3.1007

Max Act: 3.0036. Min Act: -0.1700

Data Index: 6815446 (Open Web Text)

Max Activating Token Index: 684

Text #9

Max Range: 3.1007. Min Range: -3.1007

Max Act: 2.9890. Min Act: -0.1700

Data Index: 6668388 (Open Web Text)

Max Activating Token Index: 762

Text #10

Max Range: 3.1007. Min Range: -3.1007

Max Act: 3.0435. Min Act: -0.1700

Data Index: 8511562 (Open Web Text)

Max Activating Token Index: 581

Text #11

Max Range: 3.1007. Min Range: -3.1007

Max Act: 3.0280. Min Act: -0.1700

Data Index: 903825 (Open Web Text)

Max Activating Token Index: 987

Text #12

Max Range: 3.1007. Min Range: -3.1007

Max Act: 3.0448. Min Act: -0.1700

Data Index: 7905057 (Open Web Text)

Max Activating Token Index: 983

Text #13

Max Range: 3.1007. Min Range: -3.1007

Max Act: 3.0163. Min Act: -0.1700

Data Index: 4255712 (Open Web Text)

Max Activating Token Index: 864

Text #14

Max Range: 3.1007. Min Range: -3.1007

Max Act: 2.9727. Min Act: -0.1700

Data Index: 8568351 (Open Web Text)

Max Activating Token Index: 558

Text #15

Max Range: 3.1007. Min Range: -3.1007

Max Act: 2.9867. Min Act: -0.1700

Data Index: 6605719 (Open Web Text)

Max Activating Token Index: 439

Text #16

Max Range: 3.1007. Min Range: -3.1007

Max Act: 3.0415. Min Act: -0.1700

Data Index: 8020282 (Open Web Text)

Max Activating Token Index: 923

Text #17

Max Range: 3.1007. Min Range: -3.1007

Max Act: 2.9866. Min Act: -0.1700

Data Index: 2728370 (Open Web Text)

Max Activating Token Index: 579

Text #18

Max Range: 3.1007. Min Range: -3.1007

Max Act: 2.9918. Min Act: -0.1700

Data Index: 6776523 (Open Web Text)

Max Activating Token Index: 793

Text #19

Max Range: 3.1007. Min Range: -3.1007

Max Act: 2.9916. Min Act: -0.1700

Data Index: 7031 (Open Web Text)

Max Activating Token Index: 208