Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 1518 in Layer 3

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 2.8712. Min Range: -2.8712

Max Act: 2.8712. Min Act: -0.1700

Data Index: 7188814 (Open Web Text)

Max Activating Token Index: 502

Text #1

Max Range: 2.8712. Min Range: -2.8712

Max Act: 2.8568. Min Act: -0.1700

Data Index: 4911593 (Open Web Text)

Max Activating Token Index: 89

Text #2

Max Range: 2.8712. Min Range: -2.8712

Max Act: 2.7831. Min Act: -0.1700

Data Index: 6912813 (Open Web Text)

Max Activating Token Index: 2

Text #3

Max Range: 2.8712. Min Range: -2.8712

Max Act: 2.6768. Min Act: -0.1700

Data Index: 1155197 (Open Web Text)

Max Activating Token Index: 770

Text #4

Max Range: 2.8712. Min Range: -2.8712

Max Act: 2.7039. Min Act: -0.1700

Data Index: 5246200 (Open Web Text)

Max Activating Token Index: 522

Text #5

Max Range: 2.8712. Min Range: -2.8712

Max Act: 2.6717. Min Act: -0.1700

Data Index: 2008345 (Open Web Text)

Max Activating Token Index: 625

Text #6

Max Range: 2.8712. Min Range: -2.8712

Max Act: 2.6897. Min Act: -0.1700

Data Index: 5923709 (Open Web Text)

Max Activating Token Index: 176

Text #7

Max Range: 2.8712. Min Range: -2.8712

Max Act: 2.6727. Min Act: -0.1700

Data Index: 1709570 (Open Web Text)

Max Activating Token Index: 548

Text #8

Max Range: 2.8712. Min Range: -2.8712

Max Act: 2.6935. Min Act: -0.1700

Data Index: 2728694 (Open Web Text)

Max Activating Token Index: 4

Text #9

Max Range: 2.8712. Min Range: -2.8712

Max Act: 2.6285. Min Act: -0.1700

Data Index: 2857240 (Open Web Text)

Max Activating Token Index: 504

Text #10

Max Range: 2.8712. Min Range: -2.8712

Max Act: 2.6914. Min Act: -0.1700

Data Index: 6306243 (Open Web Text)

Max Activating Token Index: 1000

Text #11

Max Range: 2.8712. Min Range: -2.8712

Max Act: 2.6388. Min Act: -0.1700

Data Index: 4671421 (Open Web Text)

Max Activating Token Index: 1022

Text #12

Max Range: 2.8712. Min Range: -2.8712

Max Act: 2.6271. Min Act: -0.1700

Data Index: 4267583 (Open Web Text)

Max Activating Token Index: 721

Text #13

Max Range: 2.8712. Min Range: -2.8712

Max Act: 2.6403. Min Act: -0.1700

Data Index: 8266913 (Open Web Text)

Max Activating Token Index: 829

Text #14

Max Range: 2.8712. Min Range: -2.8712

Max Act: 2.6658. Min Act: -0.1700

Data Index: 2561839 (Open Web Text)

Max Activating Token Index: 2

Text #15

Max Range: 2.8712. Min Range: -2.8712

Max Act: 2.6685. Min Act: -0.1700

Data Index: 2072053 (Open Web Text)

Max Activating Token Index: 143

Text #16

Max Range: 2.8712. Min Range: -2.8712

Max Act: 2.6529. Min Act: -0.1700

Data Index: 7932345 (Open Web Text)

Max Activating Token Index: 800

Text #17

Max Range: 2.8712. Min Range: -2.8712

Max Act: 2.6377. Min Act: -0.1700

Data Index: 6740261 (Open Web Text)

Max Activating Token Index: 180

Text #18

Max Range: 2.8712. Min Range: -2.8712

Max Act: 2.6448. Min Act: -0.1700

Data Index: 2570901 (Open Web Text)

Max Activating Token Index: 3

Text #19

Max Range: 2.8712. Min Range: -2.8712

Max Act: 2.6448. Min Act: -0.1700

Data Index: 293838 (Open Web Text)

Max Activating Token Index: 3