Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 1357 in Layer 6

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 3.3130. Min Range: -3.3130

Max Act: 3.3130. Min Act: -0.1700

Data Index: 7054942 (Open Web Text)

Max Activating Token Index: 976

Text #1

Max Range: 3.3130. Min Range: -3.3130

Max Act: 3.1405. Min Act: -0.1700

Data Index: 8087139 (Open Web Text)

Max Activating Token Index: 987

Text #2

Max Range: 3.3130. Min Range: -3.3130

Max Act: 3.0850. Min Act: -0.1700

Data Index: 7260589 (Open Web Text)

Max Activating Token Index: 546

Text #3

Max Range: 3.3130. Min Range: -3.3130

Max Act: 3.0895. Min Act: -0.1700

Data Index: 3695696 (Open Web Text)

Max Activating Token Index: 868

Text #4

Max Range: 3.3130. Min Range: -3.3130

Max Act: 3.0663. Min Act: -0.1700

Data Index: 876329 (Open Web Text)

Max Activating Token Index: 631

Text #5

Max Range: 3.3130. Min Range: -3.3130

Max Act: 3.0965. Min Act: -0.1700

Data Index: 6753461 (Open Web Text)

Max Activating Token Index: 409

Text #6

Max Range: 3.3130. Min Range: -3.3130

Max Act: 3.0417. Min Act: -0.1700

Data Index: 4149200 (Open Web Text)

Max Activating Token Index: 314

Text #7

Max Range: 3.3130. Min Range: -3.3130

Max Act: 3.0605. Min Act: -0.1700

Data Index: 8457366 (Open Web Text)

Max Activating Token Index: 427

Text #8

Max Range: 3.3130. Min Range: -3.3130

Max Act: 3.0960. Min Act: -0.1700

Data Index: 7983769 (Open Web Text)

Max Activating Token Index: 738

Text #9

Max Range: 3.3130. Min Range: -3.3130

Max Act: 3.0618. Min Act: -0.1700

Data Index: 2341595 (Open Web Text)

Max Activating Token Index: 587

Text #10

Max Range: 3.3130. Min Range: -3.3130

Max Act: 3.0481. Min Act: -0.1700

Data Index: 3021351 (Open Web Text)

Max Activating Token Index: 570

Text #11

Max Range: 3.3130. Min Range: -3.3130

Max Act: 3.0518. Min Act: -0.1700

Data Index: 1708901 (Open Web Text)

Max Activating Token Index: 731

Text #12

Max Range: 3.3130. Min Range: -3.3130

Max Act: 3.0520. Min Act: -0.1700

Data Index: 5642381 (Open Web Text)

Max Activating Token Index: 385

Text #13

Max Range: 3.3130. Min Range: -3.3130

Max Act: 3.0780. Min Act: -0.1700

Data Index: 4263032 (Open Web Text)

Max Activating Token Index: 785

Text #14

Max Range: 3.3130. Min Range: -3.3130

Max Act: 3.0344. Min Act: -0.1700

Data Index: 5218358 (Open Web Text)

Max Activating Token Index: 156

Text #15

Max Range: 3.3130. Min Range: -3.3130

Max Act: 3.0382. Min Act: -0.1700

Data Index: 1794381 (Open Web Text)

Max Activating Token Index: 674

Text #16

Max Range: 3.3130. Min Range: -3.3130

Max Act: 3.0252. Min Act: -0.1700

Data Index: 6836401 (Open Web Text)

Max Activating Token Index: 829

Text #17

Max Range: 3.3130. Min Range: -3.3130

Max Act: 3.0055. Min Act: -0.1700

Data Index: 35001 (Open Web Text)

Max Activating Token Index: 484

Text #18

Max Range: 3.3130. Min Range: -3.3130

Max Act: 3.0220. Min Act: -0.1700

Data Index: 6783475 (Open Web Text)

Max Activating Token Index: 900

Text #19

Max Range: 3.3130. Min Range: -3.3130

Max Act: 3.0092. Min Act: -0.1700

Data Index: 3820936 (Open Web Text)

Max Activating Token Index: 974