Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 1894 in Layer 3

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 3.3195. Min Range: -3.3195

Max Act: 3.3116. Min Act: -0.1700

Data Index: 2947815 (Open Web Text)

Max Activating Token Index: 8

Text #1

Max Range: 3.3195. Min Range: -3.3195

Max Act: 3.3195. Min Act: -0.1700

Data Index: 5761282 (Open Web Text)

Max Activating Token Index: 6

Text #2

Max Range: 3.3195. Min Range: -3.3195

Max Act: 3.2747. Min Act: -0.1700

Data Index: 2542898 (Open Web Text)

Max Activating Token Index: 8

Text #3

Max Range: 3.3195. Min Range: -3.3195

Max Act: 3.2296. Min Act: -0.1700

Data Index: 7526668 (Open Web Text)

Max Activating Token Index: 9

Text #4

Max Range: 3.3195. Min Range: -3.3195

Max Act: 3.2379. Min Act: -0.1700

Data Index: 8728867 (Open Web Text)

Max Activating Token Index: 7

Text #5

Max Range: 3.3195. Min Range: -3.3195

Max Act: 3.2196. Min Act: -0.1700

Data Index: 2749741 (Open Web Text)

Max Activating Token Index: 10

Text #6

Max Range: 3.3195. Min Range: -3.3195

Max Act: 3.1980. Min Act: -0.1700

Data Index: 8260946 (Open Web Text)

Max Activating Token Index: 11

Text #7

Max Range: 3.3195. Min Range: -3.3195

Max Act: 3.2037. Min Act: -0.1700

Data Index: 6197945 (Open Web Text)

Max Activating Token Index: 12

Text #8

Max Range: 3.3195. Min Range: -3.3195

Max Act: 3.1944. Min Act: -0.1700

Data Index: 5247360 (Open Web Text)

Max Activating Token Index: 6

Text #9

Max Range: 3.3195. Min Range: -3.3195

Max Act: 3.1794. Min Act: -0.1700

Data Index: 702212 (Open Web Text)

Max Activating Token Index: 9

Text #10

Max Range: 3.3195. Min Range: -3.3195

Max Act: 3.1838. Min Act: -0.1700

Data Index: 6427924 (Open Web Text)

Max Activating Token Index: 12

Text #11

Max Range: 3.3195. Min Range: -3.3195

Max Act: 3.1607. Min Act: -0.1700

Data Index: 8813296 (Open Web Text)

Max Activating Token Index: 5

Text #12

Max Range: 3.3195. Min Range: -3.3195

Max Act: 3.1478. Min Act: -0.1700

Data Index: 4039356 (Open Web Text)

Max Activating Token Index: 14

Text #13

Max Range: 3.3195. Min Range: -3.3195

Max Act: 3.1621. Min Act: -0.1700

Data Index: 1749758 (Open Web Text)

Max Activating Token Index: 12

Text #14

Max Range: 3.3195. Min Range: -3.3195

Max Act: 3.1353. Min Act: -0.1700

Data Index: 7402612 (Open Web Text)

Max Activating Token Index: 80

Text #15

Max Range: 3.3195. Min Range: -3.3195

Max Act: 3.1335. Min Act: -0.1700

Data Index: 5971339 (Open Web Text)

Max Activating Token Index: 15

Text #16

Max Range: 3.3195. Min Range: -3.3195

Max Act: 3.1334. Min Act: -0.1700

Data Index: 7636583 (Open Web Text)

Max Activating Token Index: 13

Text #17

Max Range: 3.3195. Min Range: -3.3195

Max Act: 3.1431. Min Act: -0.1700

Data Index: 1096817 (Open Web Text)

Max Activating Token Index: 13

Text #18

Max Range: 3.3195. Min Range: -3.3195

Max Act: 3.1329. Min Act: -0.1700

Data Index: 380407 (Open Web Text)

Max Activating Token Index: 653

Text #19

Max Range: 3.3195. Min Range: -3.3195

Max Act: 3.1258. Min Act: -0.1700

Data Index: 2794840 (Open Web Text)

Max Activating Token Index: 12