Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 356 in Layer 0

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 4.4879. Min Range: -4.4879

Max Act: 4.4879. Min Act: -0.1700

Data Index: 4301703 (Open Web Text)

Max Activating Token Index: 45

Text #1

Max Range: 4.4879. Min Range: -4.4879

Max Act: 4.4352. Min Act: -0.1700

Data Index: 4734174 (Open Web Text)

Max Activating Token Index: 9

Text #2

Max Range: 4.4879. Min Range: -4.4879

Max Act: 4.4251. Min Act: -0.1700

Data Index: 1676004 (Open Web Text)

Max Activating Token Index: 998

Text #3

Max Range: 4.4879. Min Range: -4.4879

Max Act: 4.4169. Min Act: -0.1700

Data Index: 681990 (Open Web Text)

Max Activating Token Index: 4

Text #4

Max Range: 4.4879. Min Range: -4.4879

Max Act: 4.3679. Min Act: -0.1700

Data Index: 1521516 (Open Web Text)

Max Activating Token Index: 6

Text #5

Max Range: 4.4879. Min Range: -4.4879

Max Act: 4.3651. Min Act: -0.1700

Data Index: 8184796 (Open Web Text)

Max Activating Token Index: 46

Text #6

Max Range: 4.4879. Min Range: -4.4879

Max Act: 4.3702. Min Act: -0.1700

Data Index: 7174298 (Open Web Text)

Max Activating Token Index: 7

Text #7

Max Range: 4.4879. Min Range: -4.4879

Max Act: 4.3482. Min Act: -0.1700

Data Index: 5619403 (Open Web Text)

Max Activating Token Index: 904

Text #8

Max Range: 4.4879. Min Range: -4.4879

Max Act: 4.3361. Min Act: -0.1700

Data Index: 867734 (Open Web Text)

Max Activating Token Index: 8

Text #9

Max Range: 4.4879. Min Range: -4.4879

Max Act: 4.3347. Min Act: -0.1700

Data Index: 8307337 (Open Web Text)

Max Activating Token Index: 6

Text #10

Max Range: 4.4879. Min Range: -4.4879

Max Act: 4.3462. Min Act: -0.1700

Data Index: 3678409 (Open Web Text)

Max Activating Token Index: 6

Text #11

Max Range: 4.4879. Min Range: -4.4879

Max Act: 4.3520. Min Act: -0.1700

Data Index: 657634 (Open Web Text)

Max Activating Token Index: 9

Text #12

Max Range: 4.4879. Min Range: -4.4879

Max Act: 4.3609. Min Act: -0.1700

Data Index: 1340589 (Open Web Text)

Max Activating Token Index: 122

Text #13

Max Range: 4.4879. Min Range: -4.4879

Max Act: 4.3318. Min Act: -0.1700

Data Index: 1656178 (Open Web Text)

Max Activating Token Index: 6

Text #14

Max Range: 4.4879. Min Range: -4.4879

Max Act: 4.3511. Min Act: -0.1700

Data Index: 1402067 (Open Web Text)

Max Activating Token Index: 6

Text #15

Max Range: 4.4879. Min Range: -4.4879

Max Act: 4.3345. Min Act: -0.1700

Data Index: 5339995 (Open Web Text)

Max Activating Token Index: 22

Text #16

Max Range: 4.4879. Min Range: -4.4879

Max Act: 4.3152. Min Act: -0.1700

Data Index: 2782429 (Open Web Text)

Max Activating Token Index: 4

Text #17

Max Range: 4.4879. Min Range: -4.4879

Max Act: 4.3268. Min Act: -0.1700

Data Index: 4111971 (Open Web Text)

Max Activating Token Index: 15

Text #18

Max Range: 4.4879. Min Range: -4.4879

Max Act: 4.3091. Min Act: -0.1700

Data Index: 4150557 (Open Web Text)

Max Activating Token Index: 22

Text #19

Max Range: 4.4879. Min Range: -4.4879

Max Act: 4.3156. Min Act: -0.1700

Data Index: 1340224 (Open Web Text)

Max Activating Token Index: 9