Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 1949 in Layer 5

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 5.8157. Min Range: -5.8157

Max Act: 5.8157. Min Act: -0.1700

Data Index: 7000913 (Open Web Text)

Max Activating Token Index: 92

Text #1

Max Range: 5.8157. Min Range: -5.8157

Max Act: 5.6241. Min Act: -0.1700

Data Index: 7623522 (Open Web Text)

Max Activating Token Index: 876

Text #2

Max Range: 5.8157. Min Range: -5.8157

Max Act: 5.6338. Min Act: -0.1699

Data Index: 118832 (Open Web Text)

Max Activating Token Index: 888

Text #3

Max Range: 5.8157. Min Range: -5.8157

Max Act: 5.6694. Min Act: -0.1700

Data Index: 5820277 (Open Web Text)

Max Activating Token Index: 107

Text #4

Max Range: 5.8157. Min Range: -5.8157

Max Act: 5.6135. Min Act: -0.1700

Data Index: 3532469 (Open Web Text)

Max Activating Token Index: 909

Text #5

Max Range: 5.8157. Min Range: -5.8157

Max Act: 5.5450. Min Act: -0.1700

Data Index: 4433957 (Open Web Text)

Max Activating Token Index: 553

Text #6

Max Range: 5.8157. Min Range: -5.8157

Max Act: 5.5472. Min Act: -0.1700

Data Index: 4293445 (Open Web Text)

Max Activating Token Index: 215

Text #7

Max Range: 5.8157. Min Range: -5.8157

Max Act: 5.5705. Min Act: -0.1700

Data Index: 7639112 (Open Web Text)

Max Activating Token Index: 574

Text #8

Max Range: 5.8157. Min Range: -5.8157

Max Act: 5.5139. Min Act: -0.1700

Data Index: 7370349 (Open Web Text)

Max Activating Token Index: 364

Text #9

Max Range: 5.8157. Min Range: -5.8157

Max Act: 5.5308. Min Act: -0.1700

Data Index: 4162233 (Open Web Text)

Max Activating Token Index: 548

Text #10

Max Range: 5.8157. Min Range: -5.8157

Max Act: 5.5402. Min Act: -0.1700

Data Index: 1771856 (Open Web Text)

Max Activating Token Index: 1012

Text #11

Max Range: 5.8157. Min Range: -5.8157

Max Act: 5.4654. Min Act: -0.1700

Data Index: 8706821 (Open Web Text)

Max Activating Token Index: 18

Text #12

Max Range: 5.8157. Min Range: -5.8157

Max Act: 5.4637. Min Act: -0.1699

Data Index: 275089 (Open Web Text)

Max Activating Token Index: 235

Text #13

Max Range: 5.8157. Min Range: -5.8157

Max Act: 5.4475. Min Act: -0.1700

Data Index: 8578601 (Open Web Text)

Max Activating Token Index: 936

Text #14

Max Range: 5.8157. Min Range: -5.8157

Max Act: 5.4405. Min Act: -0.1700

Data Index: 8721045 (Open Web Text)

Max Activating Token Index: 148

Text #15

Max Range: 5.8157. Min Range: -5.8157

Max Act: 5.4526. Min Act: -0.1700

Data Index: 3082249 (Open Web Text)

Max Activating Token Index: 855

Text #16

Max Range: 5.8157. Min Range: -5.8157

Max Act: 5.4794. Min Act: -0.1700

Data Index: 6780067 (Open Web Text)

Max Activating Token Index: 276

Text #17

Max Range: 5.8157. Min Range: -5.8157

Max Act: 5.4774. Min Act: -0.1700

Data Index: 4125215 (Open Web Text)

Max Activating Token Index: 144

Text #18

Max Range: 5.8157. Min Range: -5.8157

Max Act: 5.4578. Min Act: -0.1700

Data Index: 2979296 (Open Web Text)

Max Activating Token Index: 540

Text #19

Max Range: 5.8157. Min Range: -5.8157

Max Act: 5.4325. Min Act: -0.1700

Data Index: 1401269 (Open Web Text)

Max Activating Token Index: 135