Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 1583 in Layer 0

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 3.0580. Min Range: -3.0580

Max Act: 3.0580. Min Act: -0.1700

Data Index: 2320580 (Open Web Text)

Max Activating Token Index: 5

Text #1

Max Range: 3.0580. Min Range: -3.0580

Max Act: 3.0362. Min Act: -0.1700

Data Index: 3010979 (Open Web Text)

Max Activating Token Index: 3

Text #2

Max Range: 3.0580. Min Range: -3.0580

Max Act: 3.0242. Min Act: -0.1700

Data Index: 7075024 (Open Web Text)

Max Activating Token Index: 1

Text #3

Max Range: 3.0580. Min Range: -3.0580

Max Act: 3.0242. Min Act: -0.1700

Data Index: 4981758 (Open Web Text)

Max Activating Token Index: 1

Text #4

Max Range: 3.0580. Min Range: -3.0580

Max Act: 2.9658. Min Act: -0.1700

Data Index: 1249992 (Open Web Text)

Max Activating Token Index: 1021

Text #5

Max Range: 3.0580. Min Range: -3.0580

Max Act: 2.9224. Min Act: -0.1700

Data Index: 5624157 (Open Web Text)

Max Activating Token Index: 553

Text #6

Max Range: 3.0580. Min Range: -3.0580

Max Act: 2.9230. Min Act: -0.1700

Data Index: 5401288 (Open Web Text)

Max Activating Token Index: 686

Text #7

Max Range: 3.0580. Min Range: -3.0580

Max Act: 2.9134. Min Act: -0.1700

Data Index: 70360 (Open Web Text)

Max Activating Token Index: 731

Text #8

Max Range: 3.0580. Min Range: -3.0580

Max Act: 2.9183. Min Act: -0.1700

Data Index: 5535178 (Open Web Text)

Max Activating Token Index: 63

Text #9

Max Range: 3.0580. Min Range: -3.0580

Max Act: 2.8951. Min Act: -0.1700

Data Index: 3955780 (Open Web Text)

Max Activating Token Index: 244

Text #10

Max Range: 3.0580. Min Range: -3.0580

Max Act: 2.9029. Min Act: -0.1700

Data Index: 7398371 (Open Web Text)

Max Activating Token Index: 163

Text #11

Max Range: 3.0580. Min Range: -3.0580

Max Act: 2.9005. Min Act: -0.1700

Data Index: 5624159 (Open Web Text)

Max Activating Token Index: 423

Text #12

Max Range: 3.0580. Min Range: -3.0580

Max Act: 2.9005. Min Act: -0.1700

Data Index: 8600089 (Open Web Text)

Max Activating Token Index: 612

Text #13

Max Range: 3.0580. Min Range: -3.0580

Max Act: 2.9165. Min Act: -0.1700

Data Index: 6287035 (Open Web Text)

Max Activating Token Index: 642

Text #14

Max Range: 3.0580. Min Range: -3.0580

Max Act: 2.8860. Min Act: -0.1700

Data Index: 6075557 (Open Web Text)

Max Activating Token Index: 511

Text #15

Max Range: 3.0580. Min Range: -3.0580

Max Act: 2.8899. Min Act: -0.1700

Data Index: 5535177 (Open Web Text)

Max Activating Token Index: 710

Text #16

Max Range: 3.0580. Min Range: -3.0580

Max Act: 2.8830. Min Act: -0.1700

Data Index: 2229026 (Open Web Text)

Max Activating Token Index: 675

Text #17

Max Range: 3.0580. Min Range: -3.0580

Max Act: 2.8917. Min Act: -0.1700

Data Index: 6075556 (Open Web Text)

Max Activating Token Index: 496

Text #18

Max Range: 3.0580. Min Range: -3.0580

Max Act: 2.8827. Min Act: -0.1700

Data Index: 4341714 (Open Web Text)

Max Activating Token Index: 72

Text #19

Max Range: 3.0580. Min Range: -3.0580

Max Act: 2.8836. Min Act: -0.1700

Data Index: 2640822 (Open Web Text)

Max Activating Token Index: 661