Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 2052 in Layer 3

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 4.4206. Min Range: -4.4206

Max Act: 4.4206. Min Act: -0.1700

Data Index: 5194984 (Open Web Text)

Max Activating Token Index: 597

Text #1

Max Range: 4.4206. Min Range: -4.4206

Max Act: 4.3470. Min Act: -0.1700

Data Index: 6654692 (Open Web Text)

Max Activating Token Index: 597

Text #2

Max Range: 4.4206. Min Range: -4.4206

Max Act: 4.3143. Min Act: -0.1700

Data Index: 5202404 (Open Web Text)

Max Activating Token Index: 202

Text #3

Max Range: 4.4206. Min Range: -4.4206

Max Act: 4.3185. Min Act: -0.1700

Data Index: 8726653 (Open Web Text)

Max Activating Token Index: 458

Text #4

Max Range: 4.4206. Min Range: -4.4206

Max Act: 4.3173. Min Act: -0.1700

Data Index: 7180124 (Open Web Text)

Max Activating Token Index: 39

Text #5

Max Range: 4.4206. Min Range: -4.4206

Max Act: 4.3340. Min Act: -0.1700

Data Index: 5342877 (Open Web Text)

Max Activating Token Index: 924

Text #6

Max Range: 4.4206. Min Range: -4.4206

Max Act: 4.3516. Min Act: -0.1700

Data Index: 3319664 (Open Web Text)

Max Activating Token Index: 467

Text #7

Max Range: 4.4206. Min Range: -4.4206

Max Act: 4.2876. Min Act: -0.1700

Data Index: 6210142 (Open Web Text)

Max Activating Token Index: 367

Text #8

Max Range: 4.4206. Min Range: -4.4206

Max Act: 4.3010. Min Act: -0.1700

Data Index: 1974778 (Open Web Text)

Max Activating Token Index: 238

Text #9

Max Range: 4.4206. Min Range: -4.4206

Max Act: 4.3035. Min Act: -0.1700

Data Index: 755325 (Open Web Text)

Max Activating Token Index: 275

Text #10

Max Range: 4.4206. Min Range: -4.4206

Max Act: 4.2994. Min Act: -0.1700

Data Index: 4193555 (Open Web Text)

Max Activating Token Index: 1000

Text #11

Max Range: 4.4206. Min Range: -4.4206

Max Act: 4.2890. Min Act: -0.1700

Data Index: 2824300 (Open Web Text)

Max Activating Token Index: 644

Text #12

Max Range: 4.4206. Min Range: -4.4206

Max Act: 4.2712. Min Act: -0.1700

Data Index: 7750254 (Open Web Text)

Max Activating Token Index: 772

Text #13

Max Range: 4.4206. Min Range: -4.4206

Max Act: 4.2606. Min Act: -0.1700

Data Index: 6151099 (Open Web Text)

Max Activating Token Index: 507

Text #14

Max Range: 4.4206. Min Range: -4.4206

Max Act: 4.2618. Min Act: -0.1700

Data Index: 3235354 (Open Web Text)

Max Activating Token Index: 302

Text #15

Max Range: 4.4206. Min Range: -4.4206

Max Act: 4.2624. Min Act: -0.1700

Data Index: 6151097 (Open Web Text)

Max Activating Token Index: 701

Text #16

Max Range: 4.4206. Min Range: -4.4206

Max Act: 4.2211. Min Act: -0.1700

Data Index: 8542586 (Open Web Text)

Max Activating Token Index: 207

Text #17

Max Range: 4.4206. Min Range: -4.4206

Max Act: 4.2435. Min Act: -0.1700

Data Index: 8437883 (Open Web Text)

Max Activating Token Index: 775

Text #18

Max Range: 4.4206. Min Range: -4.4206

Max Act: 4.2245. Min Act: -0.1700

Data Index: 635837 (Open Web Text)

Max Activating Token Index: 123

Text #19

Max Range: 4.4206. Min Range: -4.4206

Max Act: 4.2676. Min Act: -0.1700

Data Index: 2048561 (Open Web Text)

Max Activating Token Index: 592