Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 2412 in Layer 3

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 3.4613. Min Range: -3.4613

Max Act: 3.4613. Min Act: -0.1700

Data Index: 5977491 (Open Web Text)

Max Activating Token Index: 448

Text #1

Max Range: 3.4613. Min Range: -3.4613

Max Act: 3.4367. Min Act: -0.1700

Data Index: 5198755 (Open Web Text)

Max Activating Token Index: 461

Text #2

Max Range: 3.4613. Min Range: -3.4613

Max Act: 3.3592. Min Act: -0.1700

Data Index: 6190995 (Open Web Text)

Max Activating Token Index: 718

Text #3

Max Range: 3.4613. Min Range: -3.4613

Max Act: 3.3599. Min Act: -0.1700

Data Index: 5968729 (Open Web Text)

Max Activating Token Index: 562

Text #4

Max Range: 3.4613. Min Range: -3.4613

Max Act: 3.2981. Min Act: -0.1700

Data Index: 7292663 (Open Web Text)

Max Activating Token Index: 567

Text #5

Max Range: 3.4613. Min Range: -3.4613

Max Act: 3.3183. Min Act: -0.1700

Data Index: 7351276 (Open Web Text)

Max Activating Token Index: 539

Text #6

Max Range: 3.4613. Min Range: -3.4613

Max Act: 3.2973. Min Act: -0.1700

Data Index: 8168699 (Open Web Text)

Max Activating Token Index: 57

Text #7

Max Range: 3.4613. Min Range: -3.4613

Max Act: 3.2849. Min Act: -0.1700

Data Index: 7354022 (Open Web Text)

Max Activating Token Index: 701

Text #8

Max Range: 3.4613. Min Range: -3.4613

Max Act: 3.2964. Min Act: -0.1700

Data Index: 3881319 (Open Web Text)

Max Activating Token Index: 769

Text #9

Max Range: 3.4613. Min Range: -3.4613

Max Act: 3.3120. Min Act: -0.1700

Data Index: 3841587 (Open Web Text)

Max Activating Token Index: 321

Text #10

Max Range: 3.4613. Min Range: -3.4613

Max Act: 3.3096. Min Act: -0.1700

Data Index: 7007993 (Open Web Text)

Max Activating Token Index: 370

Text #11

Max Range: 3.4613. Min Range: -3.4613

Max Act: 3.2389. Min Act: -0.1700

Data Index: 6435566 (Open Web Text)

Max Activating Token Index: 802

Text #12

Max Range: 3.4613. Min Range: -3.4613

Max Act: 3.2397. Min Act: -0.1700

Data Index: 7236950 (Open Web Text)

Max Activating Token Index: 632

Text #13

Max Range: 3.4613. Min Range: -3.4613

Max Act: 3.2607. Min Act: -0.1700

Data Index: 1688150 (Open Web Text)

Max Activating Token Index: 342

Text #14

Max Range: 3.4613. Min Range: -3.4613

Max Act: 3.2545. Min Act: -0.1700

Data Index: 6935147 (Open Web Text)

Max Activating Token Index: 485

Text #15

Max Range: 3.4613. Min Range: -3.4613

Max Act: 3.2603. Min Act: -0.1700

Data Index: 6898105 (Open Web Text)

Max Activating Token Index: 259

Text #16

Max Range: 3.4613. Min Range: -3.4613

Max Act: 3.2554. Min Act: -0.1700

Data Index: 833078 (Open Web Text)

Max Activating Token Index: 635

Text #17

Max Range: 3.4613. Min Range: -3.4613

Max Act: 3.2470. Min Act: -0.1700

Data Index: 1436872 (Open Web Text)

Max Activating Token Index: 67

Text #18

Max Range: 3.4613. Min Range: -3.4613

Max Act: 3.2436. Min Act: -0.1700

Data Index: 3662539 (Open Web Text)

Max Activating Token Index: 289

Text #19

Max Range: 3.4613. Min Range: -3.4613

Max Act: 3.2534. Min Act: -0.1700

Data Index: 1726363 (Open Web Text)

Max Activating Token Index: 760