Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 344 in Layer 3

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 3.6805. Min Range: -3.6805

Max Act: 3.6805. Min Act: -0.1700

Data Index: 3552908 (Open Web Text)

Max Activating Token Index: 881

Text #1

Max Range: 3.6805. Min Range: -3.6805

Max Act: 3.5630. Min Act: -0.1700

Data Index: 5625805 (Open Web Text)

Max Activating Token Index: 378

Text #2

Max Range: 3.6805. Min Range: -3.6805

Max Act: 3.4626. Min Act: -0.1700

Data Index: 5844598 (Open Web Text)

Max Activating Token Index: 652

Text #3

Max Range: 3.6805. Min Range: -3.6805

Max Act: 3.4245. Min Act: -0.1700

Data Index: 4098979 (Open Web Text)

Max Activating Token Index: 152

Text #4

Max Range: 3.6805. Min Range: -3.6805

Max Act: 3.3647. Min Act: -0.1700

Data Index: 5988079 (Open Web Text)

Max Activating Token Index: 395

Text #5

Max Range: 3.6805. Min Range: -3.6805

Max Act: 3.3720. Min Act: -0.1700

Data Index: 425915 (Open Web Text)

Max Activating Token Index: 495

Text #6

Max Range: 3.6805. Min Range: -3.6805

Max Act: 3.2933. Min Act: -0.1700

Data Index: 3548234 (Open Web Text)

Max Activating Token Index: 1005

Text #7

Max Range: 3.6805. Min Range: -3.6805

Max Act: 3.3499. Min Act: -0.1700

Data Index: 2864740 (Open Web Text)

Max Activating Token Index: 728

Text #8

Max Range: 3.6805. Min Range: -3.6805

Max Act: 3.2760. Min Act: -0.1700

Data Index: 1076849 (Open Web Text)

Max Activating Token Index: 578

Text #9

Max Range: 3.6805. Min Range: -3.6805

Max Act: 3.2844. Min Act: -0.1700

Data Index: 2854282 (Open Web Text)

Max Activating Token Index: 322

Text #10

Max Range: 3.6805. Min Range: -3.6805

Max Act: 3.2727. Min Act: -0.1700

Data Index: 8041548 (Open Web Text)

Max Activating Token Index: 720

Text #11

Max Range: 3.6805. Min Range: -3.6805

Max Act: 3.2436. Min Act: -0.1700

Data Index: 414133 (Open Web Text)

Max Activating Token Index: 191

Text #12

Max Range: 3.6805. Min Range: -3.6805

Max Act: 3.2946. Min Act: -0.1700

Data Index: 7423561 (Open Web Text)

Max Activating Token Index: 586

Text #13

Max Range: 3.6805. Min Range: -3.6805

Max Act: 3.2715. Min Act: -0.1700

Data Index: 3682944 (Open Web Text)

Max Activating Token Index: 913

Text #14

Max Range: 3.6805. Min Range: -3.6805

Max Act: 3.2869. Min Act: -0.1700

Data Index: 5052843 (Open Web Text)

Max Activating Token Index: 290

Text #15

Max Range: 3.6805. Min Range: -3.6805

Max Act: 3.3241. Min Act: -0.1700

Data Index: 4204551 (Open Web Text)

Max Activating Token Index: 970

Text #16

Max Range: 3.6805. Min Range: -3.6805

Max Act: 3.2614. Min Act: -0.1700

Data Index: 3282363 (Open Web Text)

Max Activating Token Index: 525

Text #17

Max Range: 3.6805. Min Range: -3.6805

Max Act: 3.2582. Min Act: -0.1700

Data Index: 149448 (Open Web Text)

Max Activating Token Index: 867

Text #18

Max Range: 3.6805. Min Range: -3.6805

Max Act: 3.2755. Min Act: -0.1700

Data Index: 5011989 (Open Web Text)

Max Activating Token Index: 439

Text #19

Max Range: 3.6805. Min Range: -3.6805

Max Act: 3.2592. Min Act: -0.1700

Data Index: 5641521 (Open Web Text)

Max Activating Token Index: 581