Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 1157 in Layer 0

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 1.4452. Min Range: -1.4452

Max Act: 1.4452. Min Act: -0.1385

Data Index: 7331647 (Open Web Text)

Max Activating Token Index: 589

Text #1

Max Range: 1.4452. Min Range: -1.4452

Max Act: 1.4360. Min Act: -0.0110

Data Index: 5966898 (Open Web Text)

Max Activating Token Index: 243

Text #2

Max Range: 1.4452. Min Range: -1.4452

Max Act: 1.4354. Min Act: -0.0110

Data Index: 5966903 (Open Web Text)

Max Activating Token Index: 392

Text #3

Max Range: 1.4452. Min Range: -1.4452

Max Act: 1.4298. Min Act: -0.1277

Data Index: 8647831 (Open Web Text)

Max Activating Token Index: 716

Text #4

Max Range: 1.4452. Min Range: -1.4452

Max Act: 1.4310. Min Act: -0.1419

Data Index: 7483149 (Open Web Text)

Max Activating Token Index: 712

Text #5

Max Range: 1.4452. Min Range: -1.4452

Max Act: 1.4267. Min Act: -0.0110

Data Index: 5966899 (Open Web Text)

Max Activating Token Index: 357

Text #6

Max Range: 1.4452. Min Range: -1.4452

Max Act: 1.4305. Min Act: -0.0110

Data Index: 5966911 (Open Web Text)

Max Activating Token Index: 347

Text #7

Max Range: 1.4452. Min Range: -1.4452

Max Act: 1.4268. Min Act: -0.0881

Data Index: 832593 (Open Web Text)

Max Activating Token Index: 823

Text #8

Max Range: 1.4452. Min Range: -1.4452

Max Act: 1.4220. Min Act: -0.1272

Data Index: 4605962 (Open Web Text)

Max Activating Token Index: 866

Text #9

Max Range: 1.4452. Min Range: -1.4452

Max Act: 1.4249. Min Act: -0.0596

Data Index: 5966913 (Open Web Text)

Max Activating Token Index: 317

Text #10

Max Range: 1.4452. Min Range: -1.4452

Max Act: 1.4157. Min Act: -0.0110

Data Index: 5966897 (Open Web Text)

Max Activating Token Index: 771

Text #11

Max Range: 1.4452. Min Range: -1.4452

Max Act: 1.4241. Min Act: -0.0110

Data Index: 5966904 (Open Web Text)

Max Activating Token Index: 269

Text #12

Max Range: 1.4452. Min Range: -1.4452

Max Act: 1.4207. Min Act: -0.1683

Data Index: 267026 (Open Web Text)

Max Activating Token Index: 837

Text #13

Max Range: 1.4452. Min Range: -1.4452

Max Act: 1.4160. Min Act: -0.1648

Data Index: 5825877 (Open Web Text)

Max Activating Token Index: 914

Text #14

Max Range: 1.4452. Min Range: -1.4452

Max Act: 1.4172. Min Act: -0.0110

Data Index: 5966900 (Open Web Text)

Max Activating Token Index: 281

Text #15

Max Range: 1.4452. Min Range: -1.4452

Max Act: 1.4179. Min Act: -0.0110

Data Index: 5966902 (Open Web Text)

Max Activating Token Index: 298

Text #16

Max Range: 1.4452. Min Range: -1.4452

Max Act: 1.4255. Min Act: -0.1579

Data Index: 2167888 (Open Web Text)

Max Activating Token Index: 797

Text #17

Max Range: 1.4452. Min Range: -1.4452

Max Act: 1.4235. Min Act: -0.0110

Data Index: 5966906 (Open Web Text)

Max Activating Token Index: 374

Text #18

Max Range: 1.4452. Min Range: -1.4452

Max Act: 1.4245. Min Act: -0.0110

Data Index: 5966901 (Open Web Text)

Max Activating Token Index: 332

Text #19

Max Range: 1.4452. Min Range: -1.4452

Max Act: 1.4220. Min Act: -0.0110

Data Index: 5966909 (Open Web Text)

Max Activating Token Index: 273