Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 2904 in Layer 0

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 3.2867. Min Range: -3.2867

Max Act: 3.2867. Min Act: -0.1700

Data Index: 3352911 (Open Web Text)

Max Activating Token Index: 911

Text #1

Max Range: 3.2867. Min Range: -3.2867

Max Act: 3.2526. Min Act: -0.1700

Data Index: 6278803 (Open Web Text)

Max Activating Token Index: 951

Text #2

Max Range: 3.2867. Min Range: -3.2867

Max Act: 3.1890. Min Act: -0.1700

Data Index: 583904 (Open Web Text)

Max Activating Token Index: 1007

Text #3

Max Range: 3.2867. Min Range: -3.2867

Max Act: 3.1976. Min Act: -0.1700

Data Index: 1842816 (Open Web Text)

Max Activating Token Index: 27

Text #4

Max Range: 3.2867. Min Range: -3.2867

Max Act: 3.1646. Min Act: -0.1700

Data Index: 2653900 (Open Web Text)

Max Activating Token Index: 25

Text #5

Max Range: 3.2867. Min Range: -3.2867

Max Act: 3.1334. Min Act: -0.1700

Data Index: 6269786 (Open Web Text)

Max Activating Token Index: 40

Text #6

Max Range: 3.2867. Min Range: -3.2867

Max Act: 3.1333. Min Act: -0.1700

Data Index: 8415233 (Open Web Text)

Max Activating Token Index: 855

Text #7

Max Range: 3.2867. Min Range: -3.2867

Max Act: 3.1353. Min Act: -0.1700

Data Index: 4294718 (Open Web Text)

Max Activating Token Index: 570

Text #8

Max Range: 3.2867. Min Range: -3.2867

Max Act: 3.1376. Min Act: -0.1700

Data Index: 6852359 (Open Web Text)

Max Activating Token Index: 19

Text #9

Max Range: 3.2867. Min Range: -3.2867

Max Act: 3.1291. Min Act: -0.1700

Data Index: 1593732 (Open Web Text)

Max Activating Token Index: 1020

Text #10

Max Range: 3.2867. Min Range: -3.2867

Max Act: 3.1182. Min Act: -0.1700

Data Index: 6946776 (Open Web Text)

Max Activating Token Index: 275

Text #11

Max Range: 3.2867. Min Range: -3.2867

Max Act: 3.1090. Min Act: -0.1700

Data Index: 835337 (Open Web Text)

Max Activating Token Index: 839

Text #12

Max Range: 3.2867. Min Range: -3.2867

Max Act: 3.1178. Min Act: -0.1700

Data Index: 6268215 (Open Web Text)

Max Activating Token Index: 27

Text #13

Max Range: 3.2867. Min Range: -3.2867

Max Act: 3.1261. Min Act: -0.1700

Data Index: 4472663 (Open Web Text)

Max Activating Token Index: 234

Text #14

Max Range: 3.2867. Min Range: -3.2867

Max Act: 3.1203. Min Act: -0.1700

Data Index: 2533921 (Open Web Text)

Max Activating Token Index: 497

Text #15

Max Range: 3.2867. Min Range: -3.2867

Max Act: 3.1151. Min Act: -0.1700

Data Index: 583907 (Open Web Text)

Max Activating Token Index: 387

Text #16

Max Range: 3.2867. Min Range: -3.2867

Max Act: 3.1027. Min Act: -0.1700

Data Index: 7852744 (Open Web Text)

Max Activating Token Index: 41

Text #17

Max Range: 3.2867. Min Range: -3.2867

Max Act: 3.1138. Min Act: -0.1700

Data Index: 1749255 (Open Web Text)

Max Activating Token Index: 258

Text #18

Max Range: 3.2867. Min Range: -3.2867

Max Act: 3.1039. Min Act: -0.1700

Data Index: 2393377 (Open Web Text)

Max Activating Token Index: 56

Text #19

Max Range: 3.2867. Min Range: -3.2867

Max Act: 3.0940. Min Act: -0.1700

Data Index: 8457268 (Open Web Text)

Max Activating Token Index: 697