Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 1144 in Layer 3

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 3.8754. Min Range: -3.8754

Max Act: 3.8754. Min Act: -0.1700

Data Index: 232822 (Open Web Text)

Max Activating Token Index: 846

Text #1

Max Range: 3.8754. Min Range: -3.8754

Max Act: 3.8108. Min Act: -0.1700

Data Index: 7779222 (Open Web Text)

Max Activating Token Index: 431

Text #2

Max Range: 3.8754. Min Range: -3.8754

Max Act: 3.7389. Min Act: -0.1700

Data Index: 3819143 (Open Web Text)

Max Activating Token Index: 392

Text #3

Max Range: 3.8754. Min Range: -3.8754

Max Act: 3.7349. Min Act: -0.1700

Data Index: 2870387 (Open Web Text)

Max Activating Token Index: 705

Text #4

Max Range: 3.8754. Min Range: -3.8754

Max Act: 3.6938. Min Act: -0.1700

Data Index: 6255115 (Open Web Text)

Max Activating Token Index: 656

Text #5

Max Range: 3.8754. Min Range: -3.8754

Max Act: 3.7085. Min Act: -0.1700

Data Index: 7416344 (Open Web Text)

Max Activating Token Index: 605

Text #6

Max Range: 3.8754. Min Range: -3.8754

Max Act: 3.6470. Min Act: -0.1700

Data Index: 8466400 (Open Web Text)

Max Activating Token Index: 617

Text #7

Max Range: 3.8754. Min Range: -3.8754

Max Act: 3.6319. Min Act: -0.1700

Data Index: 8653718 (Open Web Text)

Max Activating Token Index: 801

Text #8

Max Range: 3.8754. Min Range: -3.8754

Max Act: 3.6682. Min Act: -0.1700

Data Index: 2714638 (Open Web Text)

Max Activating Token Index: 12

Text #9

Max Range: 3.8754. Min Range: -3.8754

Max Act: 3.6636. Min Act: -0.1700

Data Index: 3847528 (Open Web Text)

Max Activating Token Index: 765

Text #10

Max Range: 3.8754. Min Range: -3.8754

Max Act: 3.6377. Min Act: -0.1700

Data Index: 5162017 (Open Web Text)

Max Activating Token Index: 826

Text #11

Max Range: 3.8754. Min Range: -3.8754

Max Act: 3.6617. Min Act: -0.1700

Data Index: 3669303 (Open Web Text)

Max Activating Token Index: 496

Text #12

Max Range: 3.8754. Min Range: -3.8754

Max Act: 3.6586. Min Act: -0.1700

Data Index: 4654617 (Open Web Text)

Max Activating Token Index: 218

Text #13

Max Range: 3.8754. Min Range: -3.8754

Max Act: 3.6541. Min Act: -0.1700

Data Index: 5610504 (Open Web Text)

Max Activating Token Index: 767

Text #14

Max Range: 3.8754. Min Range: -3.8754

Max Act: 3.6116. Min Act: -0.1700

Data Index: 3395089 (Open Web Text)

Max Activating Token Index: 759

Text #15

Max Range: 3.8754. Min Range: -3.8754

Max Act: 3.6257. Min Act: -0.1700

Data Index: 183124 (Open Web Text)

Max Activating Token Index: 141

Text #16

Max Range: 3.8754. Min Range: -3.8754

Max Act: 3.6344. Min Act: -0.1700

Data Index: 8156274 (Open Web Text)

Max Activating Token Index: 591

Text #17

Max Range: 3.8754. Min Range: -3.8754

Max Act: 3.6029. Min Act: -0.1700

Data Index: 1346858 (Open Web Text)

Max Activating Token Index: 192

Text #18

Max Range: 3.8754. Min Range: -3.8754

Max Act: 3.6297. Min Act: -0.1700

Data Index: 3278429 (Open Web Text)

Max Activating Token Index: 228

Text #19

Max Range: 3.8754. Min Range: -3.8754

Max Act: 3.6292. Min Act: -0.1700

Data Index: 1239687 (Open Web Text)

Max Activating Token Index: 365