Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 908 in Layer 4

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 3.0050. Min Range: -3.0050

Max Act: 3.0050. Min Act: -0.1700

Data Index: 4282744 (Open Web Text)

Max Activating Token Index: 669

Text #1

Max Range: 3.0050. Min Range: -3.0050

Max Act: 2.9930. Min Act: -0.1700

Data Index: 1551706 (Open Web Text)

Max Activating Token Index: 61

Text #2

Max Range: 3.0050. Min Range: -3.0050

Max Act: 2.9832. Min Act: -0.1700

Data Index: 1791093 (Open Web Text)

Max Activating Token Index: 220

Text #3

Max Range: 3.0050. Min Range: -3.0050

Max Act: 2.9342. Min Act: -0.1700

Data Index: 5883412 (Open Web Text)

Max Activating Token Index: 406

Text #4

Max Range: 3.0050. Min Range: -3.0050

Max Act: 2.8633. Min Act: -0.1700

Data Index: 2908250 (Open Web Text)

Max Activating Token Index: 420

Text #5

Max Range: 3.0050. Min Range: -3.0050

Max Act: 2.8218. Min Act: -0.1700

Data Index: 5502019 (Open Web Text)

Max Activating Token Index: 238

Text #6

Max Range: 3.0050. Min Range: -3.0050

Max Act: 2.7849. Min Act: -0.1700

Data Index: 4725484 (Open Web Text)

Max Activating Token Index: 936

Text #7

Max Range: 3.0050. Min Range: -3.0050

Max Act: 2.7736. Min Act: -0.1700

Data Index: 2795681 (Open Web Text)

Max Activating Token Index: 365

Text #8

Max Range: 3.0050. Min Range: -3.0050

Max Act: 2.7870. Min Act: -0.1700

Data Index: 3876986 (Open Web Text)

Max Activating Token Index: 244

Text #9

Max Range: 3.0050. Min Range: -3.0050

Max Act: 2.7734. Min Act: -0.1700

Data Index: 2795692 (Open Web Text)

Max Activating Token Index: 303

Text #10

Max Range: 3.0050. Min Range: -3.0050

Max Act: 2.7500. Min Act: -0.1700

Data Index: 8166121 (Open Web Text)

Max Activating Token Index: 545

Text #11

Max Range: 3.0050. Min Range: -3.0050

Max Act: 2.7472. Min Act: -0.1700

Data Index: 1093513 (Open Web Text)

Max Activating Token Index: 314

Text #12

Max Range: 3.0050. Min Range: -3.0050

Max Act: 2.7513. Min Act: -0.1700

Data Index: 3494252 (Open Web Text)

Max Activating Token Index: 220

Text #13

Max Range: 3.0050. Min Range: -3.0050

Max Act: 2.7449. Min Act: -0.1700

Data Index: 5791105 (Open Web Text)

Max Activating Token Index: 331

Text #14

Max Range: 3.0050. Min Range: -3.0050

Max Act: 2.7324. Min Act: -0.1700

Data Index: 4417533 (Open Web Text)

Max Activating Token Index: 909

Text #15

Max Range: 3.0050. Min Range: -3.0050

Max Act: 2.7226. Min Act: -0.1700

Data Index: 3755020 (Open Web Text)

Max Activating Token Index: 199

Text #16

Max Range: 3.0050. Min Range: -3.0050

Max Act: 2.7122. Min Act: -0.1700

Data Index: 6184002 (Open Web Text)

Max Activating Token Index: 390

Text #17

Max Range: 3.0050. Min Range: -3.0050

Max Act: 2.7206. Min Act: -0.1700

Data Index: 3786255 (Open Web Text)

Max Activating Token Index: 58

Text #18

Max Range: 3.0050. Min Range: -3.0050

Max Act: 2.7301. Min Act: -0.1700

Data Index: 1312284 (Open Web Text)

Max Activating Token Index: 621

Text #19

Max Range: 3.0050. Min Range: -3.0050

Max Act: 2.7057. Min Act: -0.1700

Data Index: 4464248 (Open Web Text)

Max Activating Token Index: 517