Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 1343 in Layer 3

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 3.3552. Min Range: -3.3552

Max Act: 3.3552. Min Act: -0.1700

Data Index: 1138635 (Open Web Text)

Max Activating Token Index: 307

Text #1

Max Range: 3.3552. Min Range: -3.3552

Max Act: 3.2691. Min Act: -0.1700

Data Index: 5585535 (Open Web Text)

Max Activating Token Index: 551

Text #2

Max Range: 3.3552. Min Range: -3.3552

Max Act: 3.3002. Min Act: -0.1700

Data Index: 5273767 (Open Web Text)

Max Activating Token Index: 505

Text #3

Max Range: 3.3552. Min Range: -3.3552

Max Act: 3.2785. Min Act: -0.1700

Data Index: 6129185 (Open Web Text)

Max Activating Token Index: 1000

Text #4

Max Range: 3.3552. Min Range: -3.3552

Max Act: 3.2853. Min Act: -0.1700

Data Index: 7922888 (Open Web Text)

Max Activating Token Index: 473

Text #5

Max Range: 3.3552. Min Range: -3.3552

Max Act: 3.2538. Min Act: -0.1700

Data Index: 7540021 (Open Web Text)

Max Activating Token Index: 139

Text #6

Max Range: 3.3552. Min Range: -3.3552

Max Act: 3.2767. Min Act: -0.1700

Data Index: 1778655 (Open Web Text)

Max Activating Token Index: 505

Text #7

Max Range: 3.3552. Min Range: -3.3552

Max Act: 3.2608. Min Act: -0.1700

Data Index: 978089 (Open Web Text)

Max Activating Token Index: 391

Text #8

Max Range: 3.3552. Min Range: -3.3552

Max Act: 3.2399. Min Act: -0.1700

Data Index: 7282832 (Open Web Text)

Max Activating Token Index: 289

Text #9

Max Range: 3.3552. Min Range: -3.3552

Max Act: 3.2605. Min Act: -0.1700

Data Index: 8217059 (Open Web Text)

Max Activating Token Index: 746

Text #10

Max Range: 3.3552. Min Range: -3.3552

Max Act: 3.2580. Min Act: -0.1700

Data Index: 3564415 (Open Web Text)

Max Activating Token Index: 266

Text #11

Max Range: 3.3552. Min Range: -3.3552

Max Act: 3.2290. Min Act: -0.1700

Data Index: 5266826 (Open Web Text)

Max Activating Token Index: 977

Text #12

Max Range: 3.3552. Min Range: -3.3552

Max Act: 3.2401. Min Act: -0.1700

Data Index: 7273387 (Open Web Text)

Max Activating Token Index: 507

Text #13

Max Range: 3.3552. Min Range: -3.3552

Max Act: 3.2348. Min Act: -0.1700

Data Index: 1159784 (Open Web Text)

Max Activating Token Index: 143

Text #14

Max Range: 3.3552. Min Range: -3.3552

Max Act: 3.2538. Min Act: -0.1700

Data Index: 5433437 (Open Web Text)

Max Activating Token Index: 920

Text #15

Max Range: 3.3552. Min Range: -3.3552

Max Act: 3.2262. Min Act: -0.1700

Data Index: 3805173 (Open Web Text)

Max Activating Token Index: 837

Text #16

Max Range: 3.3552. Min Range: -3.3552

Max Act: 3.2203. Min Act: -0.1700

Data Index: 4921982 (Open Web Text)

Max Activating Token Index: 187

Text #17

Max Range: 3.3552. Min Range: -3.3552

Max Act: 3.2318. Min Act: -0.1700

Data Index: 5429708 (Open Web Text)

Max Activating Token Index: 87

Text #18

Max Range: 3.3552. Min Range: -3.3552

Max Act: 3.2382. Min Act: -0.1700

Data Index: 406989 (Open Web Text)

Max Activating Token Index: 915

Text #19

Max Range: 3.3552. Min Range: -3.3552

Max Act: 3.2232. Min Act: -0.1700

Data Index: 2683663 (Open Web Text)

Max Activating Token Index: 86