Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 2644 in Layer 4

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 4.1907. Min Range: -4.1907

Max Act: 4.1907. Min Act: -0.1700

Data Index: 2506940 (Open Web Text)

Max Activating Token Index: 548

Text #1

Max Range: 4.1907. Min Range: -4.1907

Max Act: 4.1057. Min Act: -0.1700

Data Index: 3526434 (Open Web Text)

Max Activating Token Index: 696

Text #2

Max Range: 4.1907. Min Range: -4.1907

Max Act: 4.0269. Min Act: -0.1700

Data Index: 1945431 (Open Web Text)

Max Activating Token Index: 851

Text #3

Max Range: 4.1907. Min Range: -4.1907

Max Act: 4.0947. Min Act: -0.1700

Data Index: 4837252 (Open Web Text)

Max Activating Token Index: 935

Text #4

Max Range: 4.1907. Min Range: -4.1907

Max Act: 4.0538. Min Act: -0.1700

Data Index: 1172004 (Open Web Text)

Max Activating Token Index: 860

Text #5

Max Range: 4.1907. Min Range: -4.1907

Max Act: 4.0248. Min Act: -0.1700

Data Index: 72270 (Open Web Text)

Max Activating Token Index: 966

Text #6

Max Range: 4.1907. Min Range: -4.1907

Max Act: 3.9936. Min Act: -0.1700

Data Index: 5452893 (Open Web Text)

Max Activating Token Index: 672

Text #7

Max Range: 4.1907. Min Range: -4.1907

Max Act: 3.9860. Min Act: -0.1700

Data Index: 5861809 (Open Web Text)

Max Activating Token Index: 267

Text #8

Max Range: 4.1907. Min Range: -4.1907

Max Act: 3.9882. Min Act: -0.1700

Data Index: 2337406 (Open Web Text)

Max Activating Token Index: 149

Text #9

Max Range: 4.1907. Min Range: -4.1907

Max Act: 4.0320. Min Act: -0.1700

Data Index: 1258369 (Open Web Text)

Max Activating Token Index: 887

Text #10

Max Range: 4.1907. Min Range: -4.1907

Max Act: 3.9668. Min Act: -0.1700

Data Index: 4908250 (Open Web Text)

Max Activating Token Index: 583

Text #11

Max Range: 4.1907. Min Range: -4.1907

Max Act: 3.9434. Min Act: -0.1700

Data Index: 5017228 (Open Web Text)

Max Activating Token Index: 982

Text #12

Max Range: 4.1907. Min Range: -4.1907

Max Act: 3.9786. Min Act: -0.1700

Data Index: 6640144 (Open Web Text)

Max Activating Token Index: 625

Text #13

Max Range: 4.1907. Min Range: -4.1907

Max Act: 3.9054. Min Act: -0.1700

Data Index: 2725292 (Open Web Text)

Max Activating Token Index: 100

Text #14

Max Range: 4.1907. Min Range: -4.1907

Max Act: 3.9159. Min Act: -0.1700

Data Index: 4514166 (Open Web Text)

Max Activating Token Index: 384

Text #15

Max Range: 4.1907. Min Range: -4.1907

Max Act: 3.9032. Min Act: -0.1700

Data Index: 2242986 (Open Web Text)

Max Activating Token Index: 505

Text #16

Max Range: 4.1907. Min Range: -4.1907

Max Act: 3.8811. Min Act: -0.1700

Data Index: 4931909 (Open Web Text)

Max Activating Token Index: 216

Text #17

Max Range: 4.1907. Min Range: -4.1907

Max Act: 3.9046. Min Act: -0.1700

Data Index: 2566282 (Open Web Text)

Max Activating Token Index: 447

Text #18

Max Range: 4.1907. Min Range: -4.1907

Max Act: 3.8822. Min Act: -0.1700

Data Index: 8628574 (Open Web Text)

Max Activating Token Index: 567

Text #19

Max Range: 4.1907. Min Range: -4.1907

Max Act: 3.8591. Min Act: -0.1700

Data Index: 4494036 (Open Web Text)

Max Activating Token Index: 352