Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 856 in Layer 3

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 3.4961. Min Range: -3.4961

Max Act: 3.4961. Min Act: -0.1700

Data Index: 3537998 (Open Web Text)

Max Activating Token Index: 16

Text #1

Max Range: 3.4961. Min Range: -3.4961

Max Act: 3.4452. Min Act: -0.1700

Data Index: 3537997 (Open Web Text)

Max Activating Token Index: 993

Text #2

Max Range: 3.4961. Min Range: -3.4961

Max Act: 3.3460. Min Act: -0.1700

Data Index: 6843741 (Open Web Text)

Max Activating Token Index: 235

Text #3

Max Range: 3.4961. Min Range: -3.4961

Max Act: 3.3419. Min Act: -0.1700

Data Index: 7806030 (Open Web Text)

Max Activating Token Index: 845

Text #4

Max Range: 3.4961. Min Range: -3.4961

Max Act: 3.3464. Min Act: -0.1700

Data Index: 5473602 (Open Web Text)

Max Activating Token Index: 610

Text #5

Max Range: 3.4961. Min Range: -3.4961

Max Act: 3.2659. Min Act: -0.1700

Data Index: 7604924 (Open Web Text)

Max Activating Token Index: 913

Text #6

Max Range: 3.4961. Min Range: -3.4961

Max Act: 3.2545. Min Act: -0.1700

Data Index: 4272551 (Open Web Text)

Max Activating Token Index: 776

Text #7

Max Range: 3.4961. Min Range: -3.4961

Max Act: 3.2986. Min Act: -0.1700

Data Index: 3069917 (Open Web Text)

Max Activating Token Index: 632

Text #8

Max Range: 3.4961. Min Range: -3.4961

Max Act: 3.2328. Min Act: -0.1700

Data Index: 6222428 (Open Web Text)

Max Activating Token Index: 5

Text #9

Max Range: 3.4961. Min Range: -3.4961

Max Act: 3.2410. Min Act: -0.1700

Data Index: 2329040 (Open Web Text)

Max Activating Token Index: 656

Text #10

Max Range: 3.4961. Min Range: -3.4961

Max Act: 3.2201. Min Act: -0.1700

Data Index: 2536913 (Open Web Text)

Max Activating Token Index: 35

Text #11

Max Range: 3.4961. Min Range: -3.4961

Max Act: 3.1952. Min Act: -0.1700

Data Index: 691633 (Open Web Text)

Max Activating Token Index: 854

Text #12

Max Range: 3.4961. Min Range: -3.4961

Max Act: 3.2037. Min Act: -0.1700

Data Index: 1728545 (Open Web Text)

Max Activating Token Index: 16

Text #13

Max Range: 3.4961. Min Range: -3.4961

Max Act: 3.2162. Min Act: -0.1700

Data Index: 8142605 (Open Web Text)

Max Activating Token Index: 398

Text #14

Max Range: 3.4961. Min Range: -3.4961

Max Act: 3.1883. Min Act: -0.1700

Data Index: 6499672 (Open Web Text)

Max Activating Token Index: 13

Text #15

Max Range: 3.4961. Min Range: -3.4961

Max Act: 3.2019. Min Act: -0.1700

Data Index: 4357925 (Open Web Text)

Max Activating Token Index: 440

Text #16

Max Range: 3.4961. Min Range: -3.4961

Max Act: 3.1949. Min Act: -0.1700

Data Index: 6256847 (Open Web Text)

Max Activating Token Index: 996

Text #17

Max Range: 3.4961. Min Range: -3.4961

Max Act: 3.1718. Min Act: -0.1700

Data Index: 8747569 (Open Web Text)

Max Activating Token Index: 922

Text #18

Max Range: 3.4961. Min Range: -3.4961

Max Act: 3.1757. Min Act: -0.1700

Data Index: 650872 (Open Web Text)

Max Activating Token Index: 397

Text #19

Max Range: 3.4961. Min Range: -3.4961

Max Act: 3.1750. Min Act: -0.1700

Data Index: 1981478 (Open Web Text)

Max Activating Token Index: 866