Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 549 in Layer 1

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 1.5325. Min Range: -1.5325

Max Act: 1.5325. Min Act: -0.1700

Data Index: 8545918 (Open Web Text)

Max Activating Token Index: 8

Text #1

Max Range: 1.5325. Min Range: -1.5325

Max Act: 1.5305. Min Act: -0.1700

Data Index: 3378639 (Open Web Text)

Max Activating Token Index: 7

Text #2

Max Range: 1.5325. Min Range: -1.5325

Max Act: 1.5208. Min Act: -0.1700

Data Index: 4149691 (Open Web Text)

Max Activating Token Index: 166

Text #3

Max Range: 1.5325. Min Range: -1.5325

Max Act: 1.5218. Min Act: -0.1700

Data Index: 588601 (Open Web Text)

Max Activating Token Index: 732

Text #4

Max Range: 1.5325. Min Range: -1.5325

Max Act: 1.5306. Min Act: -0.1700

Data Index: 3794823 (Open Web Text)

Max Activating Token Index: 874

Text #5

Max Range: 1.5325. Min Range: -1.5325

Max Act: 1.5184. Min Act: -0.1700

Data Index: 3396770 (Open Web Text)

Max Activating Token Index: 691

Text #6

Max Range: 1.5325. Min Range: -1.5325

Max Act: 1.5146. Min Act: -0.1696

Data Index: 5001126 (Open Web Text)

Max Activating Token Index: 947

Text #7

Max Range: 1.5325. Min Range: -1.5325

Max Act: 1.5193. Min Act: -0.1657

Data Index: 361376 (Open Web Text)

Max Activating Token Index: 691

Text #8

Max Range: 1.5325. Min Range: -1.5325

Max Act: 1.5105. Min Act: -0.1700

Data Index: 1159588 (Open Web Text)

Max Activating Token Index: 138

Text #9

Max Range: 1.5325. Min Range: -1.5325

Max Act: 1.5151. Min Act: -0.1700

Data Index: 2028571 (Open Web Text)

Max Activating Token Index: 4

Text #10

Max Range: 1.5325. Min Range: -1.5325

Max Act: 1.5016. Min Act: -0.1695

Data Index: 6361108 (Open Web Text)

Max Activating Token Index: 105

Text #11

Max Range: 1.5325. Min Range: -1.5325

Max Act: 1.5031. Min Act: -0.1700

Data Index: 8356462 (Open Web Text)

Max Activating Token Index: 546

Text #12

Max Range: 1.5325. Min Range: -1.5325

Max Act: 1.5035. Min Act: -0.1698

Data Index: 6728556 (Open Web Text)

Max Activating Token Index: 333

Text #13

Max Range: 1.5325. Min Range: -1.5325

Max Act: 1.5046. Min Act: -0.1700

Data Index: 5288984 (Open Web Text)

Max Activating Token Index: 303

Text #14

Max Range: 1.5325. Min Range: -1.5325

Max Act: 1.4950. Min Act: -0.1699

Data Index: 588573 (Open Web Text)

Max Activating Token Index: 236

Text #15

Max Range: 1.5325. Min Range: -1.5325

Max Act: 1.4936. Min Act: -0.1700

Data Index: 4283941 (Open Web Text)

Max Activating Token Index: 139

Text #16

Max Range: 1.5325. Min Range: -1.5325

Max Act: 1.4883. Min Act: -0.1651

Data Index: 3671156 (Open Web Text)

Max Activating Token Index: 249

Text #17

Max Range: 1.5325. Min Range: -1.5325

Max Act: 1.4847. Min Act: -0.1700

Data Index: 8549655 (Open Web Text)

Max Activating Token Index: 954

Text #18

Max Range: 1.5325. Min Range: -1.5325

Max Act: 1.4937. Min Act: -0.1700

Data Index: 2118512 (Open Web Text)

Max Activating Token Index: 464

Text #19

Max Range: 1.5325. Min Range: -1.5325

Max Act: 1.4981. Min Act: -0.1700

Data Index: 515948 (Open Web Text)

Max Activating Token Index: 422