Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 1128 in Layer 3

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 3.4067. Min Range: -3.4067

Max Act: 3.4067. Min Act: -0.1700

Data Index: 1120179 (Open Web Text)

Max Activating Token Index: 370

Text #1

Max Range: 3.4067. Min Range: -3.4067

Max Act: 3.3852. Min Act: -0.1700

Data Index: 8632329 (Open Web Text)

Max Activating Token Index: 701

Text #2

Max Range: 3.4067. Min Range: -3.4067

Max Act: 3.3311. Min Act: -0.1700

Data Index: 8692385 (Open Web Text)

Max Activating Token Index: 806

Text #3

Max Range: 3.4067. Min Range: -3.4067

Max Act: 3.2710. Min Act: -0.1700

Data Index: 2700855 (Open Web Text)

Max Activating Token Index: 79

Text #4

Max Range: 3.4067. Min Range: -3.4067

Max Act: 3.2751. Min Act: -0.1700

Data Index: 3855812 (Open Web Text)

Max Activating Token Index: 407

Text #5

Max Range: 3.4067. Min Range: -3.4067

Max Act: 3.2512. Min Act: -0.1700

Data Index: 5886004 (Open Web Text)

Max Activating Token Index: 13

Text #6

Max Range: 3.4067. Min Range: -3.4067

Max Act: 3.2471. Min Act: -0.1700

Data Index: 408920 (Open Web Text)

Max Activating Token Index: 729

Text #7

Max Range: 3.4067. Min Range: -3.4067

Max Act: 3.2524. Min Act: -0.1700

Data Index: 6746941 (Open Web Text)

Max Activating Token Index: 383

Text #8

Max Range: 3.4067. Min Range: -3.4067

Max Act: 3.2111. Min Act: -0.1700

Data Index: 1736663 (Open Web Text)

Max Activating Token Index: 540

Text #9

Max Range: 3.4067. Min Range: -3.4067

Max Act: 3.2127. Min Act: -0.1700

Data Index: 745571 (Open Web Text)

Max Activating Token Index: 722

Text #10

Max Range: 3.4067. Min Range: -3.4067

Max Act: 3.2132. Min Act: -0.1700

Data Index: 2275741 (Open Web Text)

Max Activating Token Index: 942

Text #11

Max Range: 3.4067. Min Range: -3.4067

Max Act: 3.1969. Min Act: -0.1700

Data Index: 5315942 (Open Web Text)

Max Activating Token Index: 503

Text #12

Max Range: 3.4067. Min Range: -3.4067

Max Act: 3.2030. Min Act: -0.1700

Data Index: 5890042 (Open Web Text)

Max Activating Token Index: 173

Text #13

Max Range: 3.4067. Min Range: -3.4067

Max Act: 3.2114. Min Act: -0.1700

Data Index: 1760144 (Open Web Text)

Max Activating Token Index: 612

Text #14

Max Range: 3.4067. Min Range: -3.4067

Max Act: 3.2049. Min Act: -0.1700

Data Index: 3306057 (Open Web Text)

Max Activating Token Index: 180

Text #15

Max Range: 3.4067. Min Range: -3.4067

Max Act: 3.1808. Min Act: -0.1700

Data Index: 3404086 (Open Web Text)

Max Activating Token Index: 818

Text #16

Max Range: 3.4067. Min Range: -3.4067

Max Act: 3.1794. Min Act: -0.1700

Data Index: 3517565 (Open Web Text)

Max Activating Token Index: 985

Text #17

Max Range: 3.4067. Min Range: -3.4067

Max Act: 3.2042. Min Act: -0.1700

Data Index: 1946260 (Open Web Text)

Max Activating Token Index: 93

Text #18

Max Range: 3.4067. Min Range: -3.4067

Max Act: 3.1756. Min Act: -0.1700

Data Index: 6686554 (Open Web Text)

Max Activating Token Index: 749

Text #19

Max Range: 3.4067. Min Range: -3.4067

Max Act: 3.1574. Min Act: -0.1700

Data Index: 7089575 (Open Web Text)

Max Activating Token Index: 12