Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 453 in Layer 4

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 3.8318. Min Range: -3.8318

Max Act: 3.8318. Min Act: -0.1700

Data Index: 5791458 (Open Web Text)

Max Activating Token Index: 425

Text #1

Max Range: 3.8318. Min Range: -3.8318

Max Act: 3.7195. Min Act: -0.1700

Data Index: 2746428 (Open Web Text)

Max Activating Token Index: 433

Text #2

Max Range: 3.8318. Min Range: -3.8318

Max Act: 3.7955. Min Act: -0.1700

Data Index: 8383613 (Open Web Text)

Max Activating Token Index: 444

Text #3

Max Range: 3.8318. Min Range: -3.8318

Max Act: 3.7451. Min Act: -0.1700

Data Index: 5061704 (Open Web Text)

Max Activating Token Index: 909

Text #4

Max Range: 3.8318. Min Range: -3.8318

Max Act: 3.7138. Min Act: -0.1700

Data Index: 4281146 (Open Web Text)

Max Activating Token Index: 352

Text #5

Max Range: 3.8318. Min Range: -3.8318

Max Act: 3.7109. Min Act: -0.1700

Data Index: 1807871 (Open Web Text)

Max Activating Token Index: 429

Text #6

Max Range: 3.8318. Min Range: -3.8318

Max Act: 3.6769. Min Act: -0.1700

Data Index: 3447225 (Open Web Text)

Max Activating Token Index: 704

Text #7

Max Range: 3.8318. Min Range: -3.8318

Max Act: 3.6121. Min Act: -0.1700

Data Index: 8125699 (Open Web Text)

Max Activating Token Index: 887

Text #8

Max Range: 3.8318. Min Range: -3.8318

Max Act: 3.6478. Min Act: -0.1700

Data Index: 5717434 (Open Web Text)

Max Activating Token Index: 347

Text #9

Max Range: 3.8318. Min Range: -3.8318

Max Act: 3.6392. Min Act: -0.1700

Data Index: 2735604 (Open Web Text)

Max Activating Token Index: 905

Text #10

Max Range: 3.8318. Min Range: -3.8318

Max Act: 3.6095. Min Act: -0.1700

Data Index: 7995836 (Open Web Text)

Max Activating Token Index: 29

Text #11

Max Range: 3.8318. Min Range: -3.8318

Max Act: 3.5770. Min Act: -0.1700

Data Index: 4281152 (Open Web Text)

Max Activating Token Index: 442

Text #12

Max Range: 3.8318. Min Range: -3.8318

Max Act: 3.6188. Min Act: -0.1700

Data Index: 3412347 (Open Web Text)

Max Activating Token Index: 97

Text #13

Max Range: 3.8318. Min Range: -3.8318

Max Act: 3.6164. Min Act: -0.1700

Data Index: 4563456 (Open Web Text)

Max Activating Token Index: 431

Text #14

Max Range: 3.8318. Min Range: -3.8318

Max Act: 3.5530. Min Act: -0.1700

Data Index: 3412351 (Open Web Text)

Max Activating Token Index: 96

Text #15

Max Range: 3.8318. Min Range: -3.8318

Max Act: 3.6267. Min Act: -0.1700

Data Index: 6627426 (Open Web Text)

Max Activating Token Index: 705

Text #16

Max Range: 3.8318. Min Range: -3.8318

Max Act: 3.5504. Min Act: -0.1700

Data Index: 5717437 (Open Web Text)

Max Activating Token Index: 564

Text #17

Max Range: 3.8318. Min Range: -3.8318

Max Act: 3.5571. Min Act: -0.1700

Data Index: 4627988 (Open Web Text)

Max Activating Token Index: 946

Text #18

Max Range: 3.8318. Min Range: -3.8318

Max Act: 3.5844. Min Act: -0.1700

Data Index: 1827396 (Open Web Text)

Max Activating Token Index: 189

Text #19

Max Range: 3.8318. Min Range: -3.8318

Max Act: 3.5659. Min Act: -0.1700

Data Index: 4238806 (Open Web Text)

Max Activating Token Index: 395