Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 1798 in Layer 3

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 4.2752. Min Range: -4.2752

Max Act: 4.2752. Min Act: -0.1700

Data Index: 2893602 (Open Web Text)

Max Activating Token Index: 302

Text #1

Max Range: 4.2752. Min Range: -4.2752

Max Act: 4.2015. Min Act: -0.1700

Data Index: 1505448 (Open Web Text)

Max Activating Token Index: 580

Text #2

Max Range: 4.2752. Min Range: -4.2752

Max Act: 4.1814. Min Act: -0.1700

Data Index: 527430 (Open Web Text)

Max Activating Token Index: 972

Text #3

Max Range: 4.2752. Min Range: -4.2752

Max Act: 4.1584. Min Act: -0.1700

Data Index: 8130390 (Open Web Text)

Max Activating Token Index: 579

Text #4

Max Range: 4.2752. Min Range: -4.2752

Max Act: 4.1714. Min Act: -0.1700

Data Index: 6772171 (Open Web Text)

Max Activating Token Index: 578

Text #5

Max Range: 4.2752. Min Range: -4.2752

Max Act: 4.1478. Min Act: -0.1700

Data Index: 830655 (Open Web Text)

Max Activating Token Index: 562

Text #6

Max Range: 4.2752. Min Range: -4.2752

Max Act: 4.1224. Min Act: -0.1700

Data Index: 2269998 (Open Web Text)

Max Activating Token Index: 100

Text #7

Max Range: 4.2752. Min Range: -4.2752

Max Act: 4.0992. Min Act: -0.1700

Data Index: 4926445 (Open Web Text)

Max Activating Token Index: 584

Text #8

Max Range: 4.2752. Min Range: -4.2752

Max Act: 4.1028. Min Act: -0.1700

Data Index: 3670738 (Open Web Text)

Max Activating Token Index: 468

Text #9

Max Range: 4.2752. Min Range: -4.2752

Max Act: 4.1098. Min Act: -0.1700

Data Index: 4464993 (Open Web Text)

Max Activating Token Index: 380

Text #10

Max Range: 4.2752. Min Range: -4.2752

Max Act: 4.0636. Min Act: -0.1700

Data Index: 1353849 (Open Web Text)

Max Activating Token Index: 505

Text #11

Max Range: 4.2752. Min Range: -4.2752

Max Act: 4.1027. Min Act: -0.1700

Data Index: 7608223 (Open Web Text)

Max Activating Token Index: 603

Text #12

Max Range: 4.2752. Min Range: -4.2752

Max Act: 4.0566. Min Act: -0.1700

Data Index: 7170118 (Open Web Text)

Max Activating Token Index: 7

Text #13

Max Range: 4.2752. Min Range: -4.2752

Max Act: 4.0572. Min Act: -0.1700

Data Index: 3484401 (Open Web Text)

Max Activating Token Index: 508

Text #14

Max Range: 4.2752. Min Range: -4.2752

Max Act: 4.0679. Min Act: -0.1700

Data Index: 987260 (Open Web Text)

Max Activating Token Index: 883

Text #15

Max Range: 4.2752. Min Range: -4.2752

Max Act: 4.0738. Min Act: -0.1700

Data Index: 8145723 (Open Web Text)

Max Activating Token Index: 153

Text #16

Max Range: 4.2752. Min Range: -4.2752

Max Act: 4.0197. Min Act: -0.1700

Data Index: 3482823 (Open Web Text)

Max Activating Token Index: 656

Text #17

Max Range: 4.2752. Min Range: -4.2752

Max Act: 4.0378. Min Act: -0.1700

Data Index: 4772078 (Open Web Text)

Max Activating Token Index: 1021

Text #18

Max Range: 4.2752. Min Range: -4.2752

Max Act: 4.0130. Min Act: -0.1700

Data Index: 8072158 (Open Web Text)

Max Activating Token Index: 346

Text #19

Max Range: 4.2752. Min Range: -4.2752

Max Act: 4.0088. Min Act: -0.1700

Data Index: 4358212 (Open Web Text)

Max Activating Token Index: 335