Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 1027 in Layer 1

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 1.9447. Min Range: -1.9447

Max Act: 1.9343. Min Act: -0.1700

Data Index: 1261756 (Open Web Text)

Max Activating Token Index: 818

Text #1

Max Range: 1.9447. Min Range: -1.9447

Max Act: 1.9447. Min Act: -0.1700

Data Index: 5573096 (Open Web Text)

Max Activating Token Index: 348

Text #2

Max Range: 1.9447. Min Range: -1.9447

Max Act: 1.9280. Min Act: -0.1700

Data Index: 3009681 (Open Web Text)

Max Activating Token Index: 565

Text #3

Max Range: 1.9447. Min Range: -1.9447

Max Act: 1.9236. Min Act: -0.1700

Data Index: 6277701 (Open Web Text)

Max Activating Token Index: 771

Text #4

Max Range: 1.9447. Min Range: -1.9447

Max Act: 1.9298. Min Act: -0.1700

Data Index: 2860452 (Open Web Text)

Max Activating Token Index: 863

Text #5

Max Range: 1.9447. Min Range: -1.9447

Max Act: 1.9302. Min Act: -0.1700

Data Index: 3479051 (Open Web Text)

Max Activating Token Index: 622

Text #6

Max Range: 1.9447. Min Range: -1.9447

Max Act: 1.9102. Min Act: -0.1700

Data Index: 2318885 (Open Web Text)

Max Activating Token Index: 245

Text #7

Max Range: 1.9447. Min Range: -1.9447

Max Act: 1.9044. Min Act: -0.1700

Data Index: 6712290 (Open Web Text)

Max Activating Token Index: 56

Text #8

Max Range: 1.9447. Min Range: -1.9447

Max Act: 1.9028. Min Act: -0.1700

Data Index: 352096 (Open Web Text)

Max Activating Token Index: 668

Text #9

Max Range: 1.9447. Min Range: -1.9447

Max Act: 1.8987. Min Act: -0.1700

Data Index: 5036435 (Open Web Text)

Max Activating Token Index: 454

Text #10

Max Range: 1.9447. Min Range: -1.9447

Max Act: 1.9083. Min Act: -0.1700

Data Index: 1478009 (Open Web Text)

Max Activating Token Index: 76

Text #11

Max Range: 1.9447. Min Range: -1.9447

Max Act: 1.9092. Min Act: -0.1700

Data Index: 1096909 (Open Web Text)

Max Activating Token Index: 476

Text #12

Max Range: 1.9447. Min Range: -1.9447

Max Act: 1.8916. Min Act: -0.1700

Data Index: 8175747 (Open Web Text)

Max Activating Token Index: 173

Text #13

Max Range: 1.9447. Min Range: -1.9447

Max Act: 1.8893. Min Act: -0.1699

Data Index: 8458076 (Open Web Text)

Max Activating Token Index: 526

Text #14

Max Range: 1.9447. Min Range: -1.9447

Max Act: 1.8988. Min Act: -0.1700

Data Index: 4370373 (Open Web Text)

Max Activating Token Index: 491

Text #15

Max Range: 1.9447. Min Range: -1.9447

Max Act: 1.8870. Min Act: -0.1700

Data Index: 6738574 (Open Web Text)

Max Activating Token Index: 898

Text #16

Max Range: 1.9447. Min Range: -1.9447

Max Act: 1.8694. Min Act: -0.1700

Data Index: 3517942 (Open Web Text)

Max Activating Token Index: 909

Text #17

Max Range: 1.9447. Min Range: -1.9447

Max Act: 1.8770. Min Act: -0.1700

Data Index: 387091 (Open Web Text)

Max Activating Token Index: 293

Text #18

Max Range: 1.9447. Min Range: -1.9447

Max Act: 1.8660. Min Act: -0.1700

Data Index: 312679 (Open Web Text)

Max Activating Token Index: 210

Text #19

Max Range: 1.9447. Min Range: -1.9447

Max Act: 1.8699. Min Act: -0.1700

Data Index: 216006 (Open Web Text)

Max Activating Token Index: 513