Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 139 in Layer 3

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 3.7368. Min Range: -3.7368

Max Act: 3.7368. Min Act: -0.1700

Data Index: 6312519 (Open Web Text)

Max Activating Token Index: 767

Text #1

Max Range: 3.7368. Min Range: -3.7368

Max Act: 3.5225. Min Act: -0.1700

Data Index: 7290922 (Open Web Text)

Max Activating Token Index: 651

Text #2

Max Range: 3.7368. Min Range: -3.7368

Max Act: 3.5013. Min Act: -0.1700

Data Index: 1626871 (Open Web Text)

Max Activating Token Index: 924

Text #3

Max Range: 3.7368. Min Range: -3.7368

Max Act: 3.4603. Min Act: -0.1700

Data Index: 5819599 (Open Web Text)

Max Activating Token Index: 845

Text #4

Max Range: 3.7368. Min Range: -3.7368

Max Act: 3.5012. Min Act: -0.1700

Data Index: 7323520 (Open Web Text)

Max Activating Token Index: 467

Text #5

Max Range: 3.7368. Min Range: -3.7368

Max Act: 3.4458. Min Act: -0.1700

Data Index: 7672243 (Open Web Text)

Max Activating Token Index: 371

Text #6

Max Range: 3.7368. Min Range: -3.7368

Max Act: 3.4018. Min Act: -0.1700

Data Index: 3977171 (Open Web Text)

Max Activating Token Index: 144

Text #7

Max Range: 3.7368. Min Range: -3.7368

Max Act: 3.4045. Min Act: -0.1700

Data Index: 7033056 (Open Web Text)

Max Activating Token Index: 462

Text #8

Max Range: 3.7368. Min Range: -3.7368

Max Act: 3.3664. Min Act: -0.1700

Data Index: 7243409 (Open Web Text)

Max Activating Token Index: 746

Text #9

Max Range: 3.7368. Min Range: -3.7368

Max Act: 3.3635. Min Act: -0.1700

Data Index: 5434572 (Open Web Text)

Max Activating Token Index: 63

Text #10

Max Range: 3.7368. Min Range: -3.7368

Max Act: 3.3650. Min Act: -0.1700

Data Index: 1884638 (Open Web Text)

Max Activating Token Index: 435

Text #11

Max Range: 3.7368. Min Range: -3.7368

Max Act: 3.3773. Min Act: -0.1700

Data Index: 5260923 (Open Web Text)

Max Activating Token Index: 126

Text #12

Max Range: 3.7368. Min Range: -3.7368

Max Act: 3.3495. Min Act: -0.1700

Data Index: 6930096 (Open Web Text)

Max Activating Token Index: 513

Text #13

Max Range: 3.7368. Min Range: -3.7368

Max Act: 3.3797. Min Act: -0.1700

Data Index: 4987050 (Open Web Text)

Max Activating Token Index: 941

Text #14

Max Range: 3.7368. Min Range: -3.7368

Max Act: 3.3612. Min Act: -0.1700

Data Index: 1828638 (Open Web Text)

Max Activating Token Index: 37

Text #15

Max Range: 3.7368. Min Range: -3.7368

Max Act: 3.3521. Min Act: -0.1700

Data Index: 1378079 (Open Web Text)

Max Activating Token Index: 36

Text #16

Max Range: 3.7368. Min Range: -3.7368

Max Act: 3.3406. Min Act: -0.1700

Data Index: 2513369 (Open Web Text)

Max Activating Token Index: 465

Text #17

Max Range: 3.7368. Min Range: -3.7368

Max Act: 3.3211. Min Act: -0.1700

Data Index: 1645508 (Open Web Text)

Max Activating Token Index: 164

Text #18

Max Range: 3.7368. Min Range: -3.7368

Max Act: 3.3359. Min Act: -0.1700

Data Index: 4681758 (Open Web Text)

Max Activating Token Index: 616

Text #19

Max Range: 3.7368. Min Range: -3.7368

Max Act: 3.2931. Min Act: -0.1700

Data Index: 489578 (Open Web Text)

Max Activating Token Index: 148