Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 701 in Layer 0

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 3.1805. Min Range: -3.1805

Max Act: 3.1805. Min Act: -0.1700

Data Index: 7779674 (Open Web Text)

Max Activating Token Index: 483

Text #1

Max Range: 3.1805. Min Range: -3.1805

Max Act: 3.1482. Min Act: -0.1700

Data Index: 7648430 (Open Web Text)

Max Activating Token Index: 564

Text #2

Max Range: 3.1805. Min Range: -3.1805

Max Act: 3.0861. Min Act: -0.1700

Data Index: 8455959 (Open Web Text)

Max Activating Token Index: 904

Text #3

Max Range: 3.1805. Min Range: -3.1805

Max Act: 3.0877. Min Act: -0.1700

Data Index: 6680031 (Open Web Text)

Max Activating Token Index: 564

Text #4

Max Range: 3.1805. Min Range: -3.1805

Max Act: 3.0753. Min Act: -0.1700

Data Index: 4507354 (Open Web Text)

Max Activating Token Index: 153

Text #5

Max Range: 3.1805. Min Range: -3.1805

Max Act: 3.0894. Min Act: -0.1700

Data Index: 5775882 (Open Web Text)

Max Activating Token Index: 977

Text #6

Max Range: 3.1805. Min Range: -3.1805

Max Act: 3.0577. Min Act: -0.1700

Data Index: 2260218 (Open Web Text)

Max Activating Token Index: 395

Text #7

Max Range: 3.1805. Min Range: -3.1805

Max Act: 3.0656. Min Act: -0.1700

Data Index: 8421047 (Open Web Text)

Max Activating Token Index: 791

Text #8

Max Range: 3.1805. Min Range: -3.1805

Max Act: 3.0560. Min Act: -0.1700

Data Index: 5091589 (Open Web Text)

Max Activating Token Index: 916

Text #9

Max Range: 3.1805. Min Range: -3.1805

Max Act: 3.0486. Min Act: -0.1700

Data Index: 4267455 (Open Web Text)

Max Activating Token Index: 652

Text #10

Max Range: 3.1805. Min Range: -3.1805

Max Act: 3.0578. Min Act: -0.1700

Data Index: 3988290 (Open Web Text)

Max Activating Token Index: 731

Text #11

Max Range: 3.1805. Min Range: -3.1805

Max Act: 3.0537. Min Act: -0.1700

Data Index: 7772828 (Open Web Text)

Max Activating Token Index: 762

Text #12

Max Range: 3.1805. Min Range: -3.1805

Max Act: 3.0466. Min Act: -0.1700

Data Index: 758702 (Open Web Text)

Max Activating Token Index: 805

Text #13

Max Range: 3.1805. Min Range: -3.1805

Max Act: 3.0239. Min Act: -0.1700

Data Index: 155784 (Open Web Text)

Max Activating Token Index: 487

Text #14

Max Range: 3.1805. Min Range: -3.1805

Max Act: 3.0328. Min Act: -0.1700

Data Index: 4829436 (Open Web Text)

Max Activating Token Index: 909

Text #15

Max Range: 3.1805. Min Range: -3.1805

Max Act: 3.0339. Min Act: -0.1700

Data Index: 60012 (Open Web Text)

Max Activating Token Index: 976

Text #16

Max Range: 3.1805. Min Range: -3.1805

Max Act: 3.0313. Min Act: -0.1700

Data Index: 5560188 (Open Web Text)

Max Activating Token Index: 430

Text #17

Max Range: 3.1805. Min Range: -3.1805

Max Act: 3.0418. Min Act: -0.1700

Data Index: 3505463 (Open Web Text)

Max Activating Token Index: 356

Text #18

Max Range: 3.1805. Min Range: -3.1805

Max Act: 3.0287. Min Act: -0.1700

Data Index: 6296785 (Open Web Text)

Max Activating Token Index: 458

Text #19

Max Range: 3.1805. Min Range: -3.1805

Max Act: 3.0327. Min Act: -0.1700

Data Index: 386785 (Open Web Text)

Max Activating Token Index: 948