Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 2884 in Layer 4

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 3.5760. Min Range: -3.5760

Max Act: 3.5760. Min Act: -0.1700

Data Index: 248723 (Open Web Text)

Max Activating Token Index: 33

Text #1

Max Range: 3.5760. Min Range: -3.5760

Max Act: 3.5646. Min Act: -0.1700

Data Index: 193881 (Open Web Text)

Max Activating Token Index: 153

Text #2

Max Range: 3.5760. Min Range: -3.5760

Max Act: 3.5723. Min Act: -0.1700

Data Index: 3498274 (Open Web Text)

Max Activating Token Index: 960

Text #3

Max Range: 3.5760. Min Range: -3.5760

Max Act: 3.5645. Min Act: -0.1700

Data Index: 6674082 (Open Web Text)

Max Activating Token Index: 148

Text #4

Max Range: 3.5760. Min Range: -3.5760

Max Act: 3.5393. Min Act: -0.1700

Data Index: 3476894 (Open Web Text)

Max Activating Token Index: 639

Text #5

Max Range: 3.5760. Min Range: -3.5760

Max Act: 3.5451. Min Act: -0.1700

Data Index: 2992697 (Open Web Text)

Max Activating Token Index: 606

Text #6

Max Range: 3.5760. Min Range: -3.5760

Max Act: 3.5495. Min Act: -0.1700

Data Index: 2719092 (Open Web Text)

Max Activating Token Index: 122

Text #7

Max Range: 3.5760. Min Range: -3.5760

Max Act: 3.5058. Min Act: -0.1700

Data Index: 2430517 (Open Web Text)

Max Activating Token Index: 931

Text #8

Max Range: 3.5760. Min Range: -3.5760

Max Act: 3.5247. Min Act: -0.1700

Data Index: 3439953 (Open Web Text)

Max Activating Token Index: 248

Text #9

Max Range: 3.5760. Min Range: -3.5760

Max Act: 3.5324. Min Act: -0.1700

Data Index: 8681681 (Open Web Text)

Max Activating Token Index: 390

Text #10

Max Range: 3.5760. Min Range: -3.5760

Max Act: 3.5340. Min Act: -0.1700

Data Index: 1342000 (Open Web Text)

Max Activating Token Index: 1012

Text #11

Max Range: 3.5760. Min Range: -3.5760

Max Act: 3.5023. Min Act: -0.1700

Data Index: 3495766 (Open Web Text)

Max Activating Token Index: 764

Text #12

Max Range: 3.5760. Min Range: -3.5760

Max Act: 3.5082. Min Act: -0.1700

Data Index: 163758 (Open Web Text)

Max Activating Token Index: 799

Text #13

Max Range: 3.5760. Min Range: -3.5760

Max Act: 3.5366. Min Act: -0.1700

Data Index: 8612063 (Open Web Text)

Max Activating Token Index: 257

Text #14

Max Range: 3.5760. Min Range: -3.5760

Max Act: 3.4737. Min Act: -0.1700

Data Index: 6471529 (Open Web Text)

Max Activating Token Index: 126

Text #15

Max Range: 3.5760. Min Range: -3.5760

Max Act: 3.4808. Min Act: -0.1700

Data Index: 6971402 (Open Web Text)

Max Activating Token Index: 773

Text #16

Max Range: 3.5760. Min Range: -3.5760

Max Act: 3.4727. Min Act: -0.1700

Data Index: 4808336 (Open Web Text)

Max Activating Token Index: 623

Text #17

Max Range: 3.5760. Min Range: -3.5760

Max Act: 3.5036. Min Act: -0.1700

Data Index: 626056 (Open Web Text)

Max Activating Token Index: 800

Text #18

Max Range: 3.5760. Min Range: -3.5760

Max Act: 3.5030. Min Act: -0.1700

Data Index: 6921379 (Open Web Text)

Max Activating Token Index: 972

Text #19

Max Range: 3.5760. Min Range: -3.5760

Max Act: 3.4814. Min Act: -0.1700

Data Index: 1190798 (Open Web Text)

Max Activating Token Index: 450