Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 2201 in Layer 6

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 3.1306. Min Range: -3.1306

Max Act: 3.1306. Min Act: -0.1700

Data Index: 4812674 (Open Web Text)

Max Activating Token Index: 484

Text #1

Max Range: 3.1306. Min Range: -3.1306

Max Act: 3.0660. Min Act: -0.1700

Data Index: 7306253 (Open Web Text)

Max Activating Token Index: 630

Text #2

Max Range: 3.1306. Min Range: -3.1306

Max Act: 3.0814. Min Act: -0.1700

Data Index: 4678760 (Open Web Text)

Max Activating Token Index: 492

Text #3

Max Range: 3.1306. Min Range: -3.1306

Max Act: 3.0615. Min Act: -0.1700

Data Index: 5447791 (Open Web Text)

Max Activating Token Index: 602

Text #4

Max Range: 3.1306. Min Range: -3.1306

Max Act: 3.0432. Min Act: -0.1700

Data Index: 2491770 (Open Web Text)

Max Activating Token Index: 894

Text #5

Max Range: 3.1306. Min Range: -3.1306

Max Act: 3.0371. Min Act: -0.1700

Data Index: 5919909 (Open Web Text)

Max Activating Token Index: 519

Text #6

Max Range: 3.1306. Min Range: -3.1306

Max Act: 3.0495. Min Act: -0.1700

Data Index: 2386625 (Open Web Text)

Max Activating Token Index: 323

Text #7

Max Range: 3.1306. Min Range: -3.1306

Max Act: 3.0705. Min Act: -0.1700

Data Index: 2436688 (Open Web Text)

Max Activating Token Index: 537

Text #8

Max Range: 3.1306. Min Range: -3.1306

Max Act: 3.0131. Min Act: -0.1700

Data Index: 5730776 (Open Web Text)

Max Activating Token Index: 973

Text #9

Max Range: 3.1306. Min Range: -3.1306

Max Act: 3.0115. Min Act: -0.1700

Data Index: 3965139 (Open Web Text)

Max Activating Token Index: 674

Text #10

Max Range: 3.1306. Min Range: -3.1306

Max Act: 3.0642. Min Act: -0.1700

Data Index: 1220134 (Open Web Text)

Max Activating Token Index: 483

Text #11

Max Range: 3.1306. Min Range: -3.1306

Max Act: 3.0322. Min Act: -0.1700

Data Index: 2178849 (Open Web Text)

Max Activating Token Index: 592

Text #12

Max Range: 3.1306. Min Range: -3.1306

Max Act: 3.0092. Min Act: -0.1700

Data Index: 1894079 (Open Web Text)

Max Activating Token Index: 58

Text #13

Max Range: 3.1306. Min Range: -3.1306

Max Act: 2.9434. Min Act: -0.1700

Data Index: 1295571 (Open Web Text)

Max Activating Token Index: 497

Text #14

Max Range: 3.1306. Min Range: -3.1306

Max Act: 2.9349. Min Act: -0.1700

Data Index: 306951 (Open Web Text)

Max Activating Token Index: 204

Text #15

Max Range: 3.1306. Min Range: -3.1306

Max Act: 2.9461. Min Act: -0.1700

Data Index: 7419278 (Open Web Text)

Max Activating Token Index: 439

Text #16

Max Range: 3.1306. Min Range: -3.1306

Max Act: 2.9583. Min Act: -0.1700

Data Index: 786021 (Open Web Text)

Max Activating Token Index: 340

Text #17

Max Range: 3.1306. Min Range: -3.1306

Max Act: 2.9679. Min Act: -0.1700

Data Index: 8645140 (Open Web Text)

Max Activating Token Index: 376

Text #18

Max Range: 3.1306. Min Range: -3.1306

Max Act: 2.9401. Min Act: -0.1700

Data Index: 8318706 (Open Web Text)

Max Activating Token Index: 893

Text #19

Max Range: 3.1306. Min Range: -3.1306

Max Act: 2.9336. Min Act: -0.1700

Data Index: 8630818 (Open Web Text)

Max Activating Token Index: 361