Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 2887 in Layer 6

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 3.3565. Min Range: -3.3565

Max Act: 3.3565. Min Act: -0.1700

Data Index: 5107129 (Open Web Text)

Max Activating Token Index: 269

Text #1

Max Range: 3.3565. Min Range: -3.3565

Max Act: 3.2590. Min Act: -0.1700

Data Index: 4995810 (Open Web Text)

Max Activating Token Index: 221

Text #2

Max Range: 3.3565. Min Range: -3.3565

Max Act: 3.2603. Min Act: -0.1700

Data Index: 1895461 (Open Web Text)

Max Activating Token Index: 901

Text #3

Max Range: 3.3565. Min Range: -3.3565

Max Act: 3.2369. Min Act: -0.1700

Data Index: 2340898 (Open Web Text)

Max Activating Token Index: 112

Text #4

Max Range: 3.3565. Min Range: -3.3565

Max Act: 3.2413. Min Act: -0.1700

Data Index: 2952637 (Open Web Text)

Max Activating Token Index: 895

Text #5

Max Range: 3.3565. Min Range: -3.3565

Max Act: 3.2288. Min Act: -0.1700

Data Index: 7620445 (Open Web Text)

Max Activating Token Index: 484

Text #6

Max Range: 3.3565. Min Range: -3.3565

Max Act: 3.2338. Min Act: -0.1700

Data Index: 5108921 (Open Web Text)

Max Activating Token Index: 644

Text #7

Max Range: 3.3565. Min Range: -3.3565

Max Act: 3.1929. Min Act: -0.1700

Data Index: 6833651 (Open Web Text)

Max Activating Token Index: 976

Text #8

Max Range: 3.3565. Min Range: -3.3565

Max Act: 3.1769. Min Act: -0.1700

Data Index: 820072 (Open Web Text)

Max Activating Token Index: 787

Text #9

Max Range: 3.3565. Min Range: -3.3565

Max Act: 3.1944. Min Act: -0.1700

Data Index: 5465254 (Open Web Text)

Max Activating Token Index: 915

Text #10

Max Range: 3.3565. Min Range: -3.3565

Max Act: 3.1684. Min Act: -0.1700

Data Index: 3485115 (Open Web Text)

Max Activating Token Index: 478

Text #11

Max Range: 3.3565. Min Range: -3.3565

Max Act: 3.1800. Min Act: -0.1700

Data Index: 6427364 (Open Web Text)

Max Activating Token Index: 703

Text #12

Max Range: 3.3565. Min Range: -3.3565

Max Act: 3.1655. Min Act: -0.1700

Data Index: 4294793 (Open Web Text)

Max Activating Token Index: 411

Text #13

Max Range: 3.3565. Min Range: -3.3565

Max Act: 3.1422. Min Act: -0.1700

Data Index: 2803851 (Open Web Text)

Max Activating Token Index: 687

Text #14

Max Range: 3.3565. Min Range: -3.3565

Max Act: 3.1429. Min Act: -0.1700

Data Index: 3028042 (Open Web Text)

Max Activating Token Index: 671

Text #15

Max Range: 3.3565. Min Range: -3.3565

Max Act: 3.1373. Min Act: -0.1700

Data Index: 3802775 (Open Web Text)

Max Activating Token Index: 66

Text #16

Max Range: 3.3565. Min Range: -3.3565

Max Act: 3.1381. Min Act: -0.1700

Data Index: 312913 (Open Web Text)

Max Activating Token Index: 258

Text #17

Max Range: 3.3565. Min Range: -3.3565

Max Act: 3.1356. Min Act: -0.1700

Data Index: 6672784 (Open Web Text)

Max Activating Token Index: 642

Text #18

Max Range: 3.3565. Min Range: -3.3565

Max Act: 3.1347. Min Act: -0.1700

Data Index: 7351704 (Open Web Text)

Max Activating Token Index: 412

Text #19

Max Range: 3.3565. Min Range: -3.3565

Max Act: 3.1170. Min Act: -0.1700

Data Index: 361423 (Open Web Text)

Max Activating Token Index: 943