Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 49 in Layer 6

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 3.1082. Min Range: -3.1082

Max Act: 3.1082. Min Act: -0.1700

Data Index: 3228521 (Open Web Text)

Max Activating Token Index: 526

Text #1

Max Range: 3.1082. Min Range: -3.1082

Max Act: 3.0448. Min Act: -0.1700

Data Index: 93678 (Open Web Text)

Max Activating Token Index: 991

Text #2

Max Range: 3.1082. Min Range: -3.1082

Max Act: 3.0106. Min Act: -0.1700

Data Index: 4423036 (Open Web Text)

Max Activating Token Index: 472

Text #3

Max Range: 3.1082. Min Range: -3.1082

Max Act: 2.9752. Min Act: -0.1700

Data Index: 5614464 (Open Web Text)

Max Activating Token Index: 394

Text #4

Max Range: 3.1082. Min Range: -3.1082

Max Act: 2.9450. Min Act: -0.1700

Data Index: 2827846 (Open Web Text)

Max Activating Token Index: 177

Text #5

Max Range: 3.1082. Min Range: -3.1082

Max Act: 2.9327. Min Act: -0.1700

Data Index: 2261515 (Open Web Text)

Max Activating Token Index: 438

Text #6

Max Range: 3.1082. Min Range: -3.1082

Max Act: 2.9130. Min Act: -0.1700

Data Index: 4107522 (Open Web Text)

Max Activating Token Index: 594

Text #7

Max Range: 3.1082. Min Range: -3.1082

Max Act: 2.8887. Min Act: -0.1700

Data Index: 199740 (Open Web Text)

Max Activating Token Index: 567

Text #8

Max Range: 3.1082. Min Range: -3.1082

Max Act: 2.9018. Min Act: -0.1700

Data Index: 1612681 (Open Web Text)

Max Activating Token Index: 565

Text #9

Max Range: 3.1082. Min Range: -3.1082

Max Act: 2.8698. Min Act: -0.1700

Data Index: 8619558 (Open Web Text)

Max Activating Token Index: 614

Text #10

Max Range: 3.1082. Min Range: -3.1082

Max Act: 2.8766. Min Act: -0.1700

Data Index: 6772719 (Open Web Text)

Max Activating Token Index: 255

Text #11

Max Range: 3.1082. Min Range: -3.1082

Max Act: 2.8590. Min Act: -0.1700

Data Index: 4234386 (Open Web Text)

Max Activating Token Index: 312

Text #12

Max Range: 3.1082. Min Range: -3.1082

Max Act: 2.8521. Min Act: -0.1700

Data Index: 4334855 (Open Web Text)

Max Activating Token Index: 719

Text #13

Max Range: 3.1082. Min Range: -3.1082

Max Act: 2.8484. Min Act: -0.1700

Data Index: 8384827 (Open Web Text)

Max Activating Token Index: 40

Text #14

Max Range: 3.1082. Min Range: -3.1082

Max Act: 2.8235. Min Act: -0.1700

Data Index: 8119366 (Open Web Text)

Max Activating Token Index: 187

Text #15

Max Range: 3.1082. Min Range: -3.1082

Max Act: 2.8340. Min Act: -0.1700

Data Index: 185839 (Open Web Text)

Max Activating Token Index: 380

Text #16

Max Range: 3.1082. Min Range: -3.1082

Max Act: 2.8256. Min Act: -0.1700

Data Index: 8138495 (Open Web Text)

Max Activating Token Index: 465

Text #17

Max Range: 3.1082. Min Range: -3.1082

Max Act: 2.7895. Min Act: -0.1700

Data Index: 3343759 (Open Web Text)

Max Activating Token Index: 690

Text #18

Max Range: 3.1082. Min Range: -3.1082

Max Act: 2.7779. Min Act: -0.1700

Data Index: 7549846 (Open Web Text)

Max Activating Token Index: 60

Text #19

Max Range: 3.1082. Min Range: -3.1082

Max Act: 2.8059. Min Act: -0.1700

Data Index: 1340285 (Open Web Text)

Max Activating Token Index: 572