Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 1981 in Layer 5

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 3.6708. Min Range: -3.6708

Max Act: 3.6708. Min Act: -0.1700

Data Index: 7116328 (Open Web Text)

Max Activating Token Index: 419

Text #1

Max Range: 3.6708. Min Range: -3.6708

Max Act: 3.6462. Min Act: -0.1700

Data Index: 2677470 (Open Web Text)

Max Activating Token Index: 140

Text #2

Max Range: 3.6708. Min Range: -3.6708

Max Act: 3.6553. Min Act: -0.1700

Data Index: 3113093 (Open Web Text)

Max Activating Token Index: 297

Text #3

Max Range: 3.6708. Min Range: -3.6708

Max Act: 3.6318. Min Act: -0.1700

Data Index: 8279564 (Open Web Text)

Max Activating Token Index: 76

Text #4

Max Range: 3.6708. Min Range: -3.6708

Max Act: 3.5921. Min Act: -0.1700

Data Index: 2658570 (Open Web Text)

Max Activating Token Index: 354

Text #5

Max Range: 3.6708. Min Range: -3.6708

Max Act: 3.6346. Min Act: -0.1700

Data Index: 2131147 (Open Web Text)

Max Activating Token Index: 492

Text #6

Max Range: 3.6708. Min Range: -3.6708

Max Act: 3.6299. Min Act: -0.1700

Data Index: 8360498 (Open Web Text)

Max Activating Token Index: 800

Text #7

Max Range: 3.6708. Min Range: -3.6708

Max Act: 3.6097. Min Act: -0.1700

Data Index: 8063344 (Open Web Text)

Max Activating Token Index: 942

Text #8

Max Range: 3.6708. Min Range: -3.6708

Max Act: 3.6121. Min Act: -0.1700

Data Index: 3410145 (Open Web Text)

Max Activating Token Index: 340

Text #9

Max Range: 3.6708. Min Range: -3.6708

Max Act: 3.6036. Min Act: -0.1700

Data Index: 263530 (Open Web Text)

Max Activating Token Index: 519

Text #10

Max Range: 3.6708. Min Range: -3.6708

Max Act: 3.5910. Min Act: -0.1700

Data Index: 7839198 (Open Web Text)

Max Activating Token Index: 985

Text #11

Max Range: 3.6708. Min Range: -3.6708

Max Act: 3.5298. Min Act: -0.1700

Data Index: 8322394 (Open Web Text)

Max Activating Token Index: 195

Text #12

Max Range: 3.6708. Min Range: -3.6708

Max Act: 3.5748. Min Act: -0.1700

Data Index: 5537030 (Open Web Text)

Max Activating Token Index: 821

Text #13

Max Range: 3.6708. Min Range: -3.6708

Max Act: 3.5688. Min Act: -0.1700

Data Index: 350559 (Open Web Text)

Max Activating Token Index: 323

Text #14

Max Range: 3.6708. Min Range: -3.6708

Max Act: 3.5957. Min Act: -0.1700

Data Index: 8011182 (Open Web Text)

Max Activating Token Index: 765

Text #15

Max Range: 3.6708. Min Range: -3.6708

Max Act: 3.5790. Min Act: -0.1700

Data Index: 8559455 (Open Web Text)

Max Activating Token Index: 855

Text #16

Max Range: 3.6708. Min Range: -3.6708

Max Act: 3.5513. Min Act: -0.1700

Data Index: 7989458 (Open Web Text)

Max Activating Token Index: 67

Text #17

Max Range: 3.6708. Min Range: -3.6708

Max Act: 3.5400. Min Act: -0.1700

Data Index: 2163021 (Open Web Text)

Max Activating Token Index: 955

Text #18

Max Range: 3.6708. Min Range: -3.6708

Max Act: 3.5495. Min Act: -0.1700

Data Index: 7144293 (Open Web Text)

Max Activating Token Index: 841

Text #19

Max Range: 3.6708. Min Range: -3.6708

Max Act: 3.5651. Min Act: -0.1700

Data Index: 1470876 (Open Web Text)

Max Activating Token Index: 572