Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 2428 in Layer 2

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 6.0894. Min Range: -6.0894

Max Act: 6.0894. Min Act: -0.1700

Data Index: 1903674 (Open Web Text)

Max Activating Token Index: 289

Text #1

Max Range: 6.0894. Min Range: -6.0894

Max Act: 6.0611. Min Act: -0.1700

Data Index: 8809744 (Open Web Text)

Max Activating Token Index: 55

Text #2

Max Range: 6.0894. Min Range: -6.0894

Max Act: 6.0169. Min Act: -0.1700

Data Index: 6704670 (Open Web Text)

Max Activating Token Index: 108

Text #3

Max Range: 6.0894. Min Range: -6.0894

Max Act: 6.0244. Min Act: -0.1700

Data Index: 4573455 (Open Web Text)

Max Activating Token Index: 70

Text #4

Max Range: 6.0894. Min Range: -6.0894

Max Act: 5.9591. Min Act: -0.1700

Data Index: 4874176 (Open Web Text)

Max Activating Token Index: 189

Text #5

Max Range: 6.0894. Min Range: -6.0894

Max Act: 5.9291. Min Act: -0.1700

Data Index: 2624894 (Open Web Text)

Max Activating Token Index: 704

Text #6

Max Range: 6.0894. Min Range: -6.0894

Max Act: 5.7771. Min Act: -0.1700

Data Index: 1000851 (Open Web Text)

Max Activating Token Index: 194

Text #7

Max Range: 6.0894. Min Range: -6.0894

Max Act: 5.8014. Min Act: -0.1700

Data Index: 848399 (Open Web Text)

Max Activating Token Index: 800

Text #8

Max Range: 6.0894. Min Range: -6.0894

Max Act: 5.7319. Min Act: -0.1700

Data Index: 8482137 (Open Web Text)

Max Activating Token Index: 464

Text #9

Max Range: 6.0894. Min Range: -6.0894

Max Act: 5.7000. Min Act: -0.1700

Data Index: 7967338 (Open Web Text)

Max Activating Token Index: 63

Text #10

Max Range: 6.0894. Min Range: -6.0894

Max Act: 5.6858. Min Act: -0.1700

Data Index: 4828223 (Open Web Text)

Max Activating Token Index: 98

Text #11

Max Range: 6.0894. Min Range: -6.0894

Max Act: 5.7181. Min Act: -0.1700

Data Index: 3415539 (Open Web Text)

Max Activating Token Index: 20

Text #12

Max Range: 6.0894. Min Range: -6.0894

Max Act: 5.7146. Min Act: -0.1700

Data Index: 956059 (Open Web Text)

Max Activating Token Index: 998

Text #13

Max Range: 6.0894. Min Range: -6.0894

Max Act: 5.6811. Min Act: -0.1700

Data Index: 259973 (Open Web Text)

Max Activating Token Index: 21

Text #14

Max Range: 6.0894. Min Range: -6.0894

Max Act: 5.6643. Min Act: -0.1700

Data Index: 5825345 (Open Web Text)

Max Activating Token Index: 263

Text #15

Max Range: 6.0894. Min Range: -6.0894

Max Act: 5.6241. Min Act: -0.1700

Data Index: 1323183 (Open Web Text)

Max Activating Token Index: 769

Text #16

Max Range: 6.0894. Min Range: -6.0894

Max Act: 5.6622. Min Act: -0.1700

Data Index: 6834749 (Open Web Text)

Max Activating Token Index: 98

Text #17

Max Range: 6.0894. Min Range: -6.0894

Max Act: 5.6070. Min Act: -0.1700

Data Index: 3180365 (Open Web Text)

Max Activating Token Index: 153

Text #18

Max Range: 6.0894. Min Range: -6.0894

Max Act: 5.6184. Min Act: -0.1700

Data Index: 2660176 (Open Web Text)

Max Activating Token Index: 775

Text #19

Max Range: 6.0894. Min Range: -6.0894

Max Act: 5.5853. Min Act: -0.1700

Data Index: 6865951 (Open Web Text)

Max Activating Token Index: 198