Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 2524 in Layer 2

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 4.2539. Min Range: -4.2539

Max Act: 4.2539. Min Act: -0.1700

Data Index: 7141774 (Open Web Text)

Max Activating Token Index: 679

Text #1

Max Range: 4.2539. Min Range: -4.2539

Max Act: 4.1828. Min Act: -0.1700

Data Index: 2284665 (Open Web Text)

Max Activating Token Index: 154

Text #2

Max Range: 4.2539. Min Range: -4.2539

Max Act: 4.1951. Min Act: -0.1700

Data Index: 8352190 (Open Web Text)

Max Activating Token Index: 714

Text #3

Max Range: 4.2539. Min Range: -4.2539

Max Act: 4.1518. Min Act: -0.1700

Data Index: 8093207 (Open Web Text)

Max Activating Token Index: 1001

Text #4

Max Range: 4.2539. Min Range: -4.2539

Max Act: 4.1661. Min Act: -0.1700

Data Index: 5064832 (Open Web Text)

Max Activating Token Index: 667

Text #5

Max Range: 4.2539. Min Range: -4.2539

Max Act: 4.1410. Min Act: -0.1700

Data Index: 6235221 (Open Web Text)

Max Activating Token Index: 976

Text #6

Max Range: 4.2539. Min Range: -4.2539

Max Act: 4.1359. Min Act: -0.1700

Data Index: 6751330 (Open Web Text)

Max Activating Token Index: 75

Text #7

Max Range: 4.2539. Min Range: -4.2539

Max Act: 4.1159. Min Act: -0.1700

Data Index: 3106077 (Open Web Text)

Max Activating Token Index: 67

Text #8

Max Range: 4.2539. Min Range: -4.2539

Max Act: 4.1208. Min Act: -0.1700

Data Index: 5671823 (Open Web Text)

Max Activating Token Index: 441

Text #9

Max Range: 4.2539. Min Range: -4.2539

Max Act: 4.1305. Min Act: -0.1700

Data Index: 77976 (Open Web Text)

Max Activating Token Index: 16

Text #10

Max Range: 4.2539. Min Range: -4.2539

Max Act: 4.0853. Min Act: -0.1700

Data Index: 13434 (Open Web Text)

Max Activating Token Index: 8

Text #11

Max Range: 4.2539. Min Range: -4.2539

Max Act: 4.1014. Min Act: -0.1700

Data Index: 5599031 (Open Web Text)

Max Activating Token Index: 280

Text #12

Max Range: 4.2539. Min Range: -4.2539

Max Act: 4.0872. Min Act: -0.1700

Data Index: 8623314 (Open Web Text)

Max Activating Token Index: 838

Text #13

Max Range: 4.2539. Min Range: -4.2539

Max Act: 4.0910. Min Act: -0.1700

Data Index: 4769137 (Open Web Text)

Max Activating Token Index: 1003

Text #14

Max Range: 4.2539. Min Range: -4.2539

Max Act: 4.0945. Min Act: -0.1700

Data Index: 3984476 (Open Web Text)

Max Activating Token Index: 34

Text #15

Max Range: 4.2539. Min Range: -4.2539

Max Act: 4.0924. Min Act: -0.1700

Data Index: 1543121 (Open Web Text)

Max Activating Token Index: 943

Text #16

Max Range: 4.2539. Min Range: -4.2539

Max Act: 4.1157. Min Act: -0.1700

Data Index: 3306612 (Open Web Text)

Max Activating Token Index: 608

Text #17

Max Range: 4.2539. Min Range: -4.2539

Max Act: 4.0829. Min Act: -0.1700

Data Index: 6229284 (Open Web Text)

Max Activating Token Index: 665

Text #18

Max Range: 4.2539. Min Range: -4.2539

Max Act: 4.0601. Min Act: -0.1700

Data Index: 5012755 (Open Web Text)

Max Activating Token Index: 960

Text #19

Max Range: 4.2539. Min Range: -4.2539

Max Act: 4.0645. Min Act: -0.1700

Data Index: 2048723 (Open Web Text)

Max Activating Token Index: 984