Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 2624 in Layer 5

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 4.1849. Min Range: -4.1849

Max Act: 4.1849. Min Act: -0.1700

Data Index: 5731174 (Open Web Text)

Max Activating Token Index: 103

Text #1

Max Range: 4.1849. Min Range: -4.1849

Max Act: 4.0949. Min Act: -0.1700

Data Index: 1756989 (Open Web Text)

Max Activating Token Index: 843

Text #2

Max Range: 4.1849. Min Range: -4.1849

Max Act: 4.0950. Min Act: -0.1700

Data Index: 8125718 (Open Web Text)

Max Activating Token Index: 523

Text #3

Max Range: 4.1849. Min Range: -4.1849

Max Act: 4.0106. Min Act: -0.1700

Data Index: 5714339 (Open Web Text)

Max Activating Token Index: 393

Text #4

Max Range: 4.1849. Min Range: -4.1849

Max Act: 4.0091. Min Act: -0.1700

Data Index: 6194794 (Open Web Text)

Max Activating Token Index: 329

Text #5

Max Range: 4.1849. Min Range: -4.1849

Max Act: 3.9851. Min Act: -0.1700

Data Index: 3077760 (Open Web Text)

Max Activating Token Index: 485

Text #6

Max Range: 4.1849. Min Range: -4.1849

Max Act: 3.9710. Min Act: -0.1700

Data Index: 6409541 (Open Web Text)

Max Activating Token Index: 393

Text #7

Max Range: 4.1849. Min Range: -4.1849

Max Act: 3.9659. Min Act: -0.1700

Data Index: 4213531 (Open Web Text)

Max Activating Token Index: 914

Text #8

Max Range: 4.1849. Min Range: -4.1849

Max Act: 3.9971. Min Act: -0.1700

Data Index: 6136702 (Open Web Text)

Max Activating Token Index: 117

Text #9

Max Range: 4.1849. Min Range: -4.1849

Max Act: 3.9321. Min Act: -0.1700

Data Index: 4465690 (Open Web Text)

Max Activating Token Index: 301

Text #10

Max Range: 4.1849. Min Range: -4.1849

Max Act: 3.9508. Min Act: -0.1700

Data Index: 7029592 (Open Web Text)

Max Activating Token Index: 345

Text #11

Max Range: 4.1849. Min Range: -4.1849

Max Act: 3.9559. Min Act: -0.1700

Data Index: 2033866 (Open Web Text)

Max Activating Token Index: 861

Text #12

Max Range: 4.1849. Min Range: -4.1849

Max Act: 3.9253. Min Act: -0.1700

Data Index: 322258 (Open Web Text)

Max Activating Token Index: 440

Text #13

Max Range: 4.1849. Min Range: -4.1849

Max Act: 3.9055. Min Act: -0.1700

Data Index: 1239798 (Open Web Text)

Max Activating Token Index: 83

Text #14

Max Range: 4.1849. Min Range: -4.1849

Max Act: 3.9070. Min Act: -0.1700

Data Index: 7652828 (Open Web Text)

Max Activating Token Index: 27

Text #15

Max Range: 4.1849. Min Range: -4.1849

Max Act: 3.8877. Min Act: -0.1700

Data Index: 6991763 (Open Web Text)

Max Activating Token Index: 4

Text #16

Max Range: 4.1849. Min Range: -4.1849

Max Act: 3.9478. Min Act: -0.1700

Data Index: 5490998 (Open Web Text)

Max Activating Token Index: 510

Text #17

Max Range: 4.1849. Min Range: -4.1849

Max Act: 3.8919. Min Act: -0.1700

Data Index: 6116099 (Open Web Text)

Max Activating Token Index: 474

Text #18

Max Range: 4.1849. Min Range: -4.1849

Max Act: 3.8798. Min Act: -0.1700

Data Index: 926154 (Open Web Text)

Max Activating Token Index: 89

Text #19

Max Range: 4.1849. Min Range: -4.1849

Max Act: 3.8693. Min Act: -0.1700

Data Index: 210124 (Open Web Text)

Max Activating Token Index: 95