Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 3048 in Layer 0

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 1.7045. Min Range: -1.7045

Max Act: 1.7045. Min Act: -0.1589

Data Index: 8432628 (Open Web Text)

Max Activating Token Index: 811

Text #1

Max Range: 1.7045. Min Range: -1.7045

Max Act: 1.6819. Min Act: -0.0843

Data Index: 7850836 (Open Web Text)

Max Activating Token Index: 406

Text #2

Max Range: 1.7045. Min Range: -1.7045

Max Act: 1.6823. Min Act: -0.0976

Data Index: 4708849 (Open Web Text)

Max Activating Token Index: 786

Text #3

Max Range: 1.7045. Min Range: -1.7045

Max Act: 1.6730. Min Act: -0.0562

Data Index: 6035837 (Open Web Text)

Max Activating Token Index: 505

Text #4

Max Range: 1.7045. Min Range: -1.7045

Max Act: 1.6724. Min Act: -0.1355

Data Index: 2206233 (Open Web Text)

Max Activating Token Index: 59

Text #5

Max Range: 1.7045. Min Range: -1.7045

Max Act: 1.6557. Min Act: -0.1227

Data Index: 7162828 (Open Web Text)

Max Activating Token Index: 679

Text #6

Max Range: 1.7045. Min Range: -1.7045

Max Act: 1.6576. Min Act: -0.0725

Data Index: 4697126 (Open Web Text)

Max Activating Token Index: 684

Text #7

Max Range: 1.7045. Min Range: -1.7045

Max Act: 1.6242. Min Act: -0.0539

Data Index: 8794513 (Open Web Text)

Max Activating Token Index: 770

Text #8

Max Range: 1.7045. Min Range: -1.7045

Max Act: 1.6270. Min Act: -0.0534

Data Index: 8794514 (Open Web Text)

Max Activating Token Index: 402

Text #9

Max Range: 1.7045. Min Range: -1.7045

Max Act: 1.6254. Min Act: -0.1273

Data Index: 6393268 (Open Web Text)

Max Activating Token Index: 315

Text #10

Max Range: 1.7045. Min Range: -1.7045

Max Act: 1.6230. Min Act: -0.1250

Data Index: 1309602 (Open Web Text)

Max Activating Token Index: 760

Text #11

Max Range: 1.7045. Min Range: -1.7045

Max Act: 1.6183. Min Act: 0.0794

Data Index: 2104705 (Open Web Text)

Max Activating Token Index: 913

Text #12

Max Range: 1.7045. Min Range: -1.7045

Max Act: 1.6235. Min Act: -0.1214

Data Index: 7330756 (Open Web Text)

Max Activating Token Index: 343

Text #13

Max Range: 1.7045. Min Range: -1.7045

Max Act: 1.6238. Min Act: -0.0886

Data Index: 3657536 (Open Web Text)

Max Activating Token Index: 808

Text #14

Max Range: 1.7045. Min Range: -1.7045

Max Act: 1.6296. Min Act: -0.1085

Data Index: 7614164 (Open Web Text)

Max Activating Token Index: 804

Text #15

Max Range: 1.7045. Min Range: -1.7045

Max Act: 1.6163. Min Act: -0.0431

Data Index: 7597182 (Open Web Text)

Max Activating Token Index: 880

Text #16

Max Range: 1.7045. Min Range: -1.7045

Max Act: 1.6168. Min Act: -0.1088

Data Index: 1921093 (Open Web Text)

Max Activating Token Index: 855

Text #17

Max Range: 1.7045. Min Range: -1.7045

Max Act: 1.6176. Min Act: -0.0568

Data Index: 7515645 (Open Web Text)

Max Activating Token Index: 552

Text #18

Max Range: 1.7045. Min Range: -1.7045

Max Act: 1.6166. Min Act: -0.1154

Data Index: 1926731 (Open Web Text)

Max Activating Token Index: 828

Text #19

Max Range: 1.7045. Min Range: -1.7045

Max Act: 1.6100. Min Act: -0.0651

Data Index: 479260 (Open Web Text)

Max Activating Token Index: 642