Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 1512 in Layer 5

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 4.1971. Min Range: -4.1971

Max Act: 4.1971. Min Act: -0.1700

Data Index: 6319068 (Open Web Text)

Max Activating Token Index: 219

Text #1

Max Range: 4.1971. Min Range: -4.1971

Max Act: 4.1953. Min Act: -0.1700

Data Index: 1127227 (Open Web Text)

Max Activating Token Index: 1022

Text #2

Max Range: 4.1971. Min Range: -4.1971

Max Act: 4.1658. Min Act: -0.1700

Data Index: 8189663 (Open Web Text)

Max Activating Token Index: 683

Text #3

Max Range: 4.1971. Min Range: -4.1971

Max Act: 4.1567. Min Act: -0.1700

Data Index: 7305655 (Open Web Text)

Max Activating Token Index: 219

Text #4

Max Range: 4.1971. Min Range: -4.1971

Max Act: 4.1227. Min Act: -0.1700

Data Index: 5203430 (Open Web Text)

Max Activating Token Index: 472

Text #5

Max Range: 4.1971. Min Range: -4.1971

Max Act: 4.1488. Min Act: -0.1700

Data Index: 2342328 (Open Web Text)

Max Activating Token Index: 764

Text #6

Max Range: 4.1971. Min Range: -4.1971

Max Act: 4.1370. Min Act: -0.1700

Data Index: 158134 (Open Web Text)

Max Activating Token Index: 295

Text #7

Max Range: 4.1971. Min Range: -4.1971

Max Act: 4.1193. Min Act: -0.1700

Data Index: 1189471 (Open Web Text)

Max Activating Token Index: 462

Text #8

Max Range: 4.1971. Min Range: -4.1971

Max Act: 4.1295. Min Act: -0.1700

Data Index: 7179304 (Open Web Text)

Max Activating Token Index: 711

Text #9

Max Range: 4.1971. Min Range: -4.1971

Max Act: 4.1204. Min Act: -0.1700

Data Index: 3639999 (Open Web Text)

Max Activating Token Index: 435

Text #10

Max Range: 4.1971. Min Range: -4.1971

Max Act: 4.1343. Min Act: -0.1700

Data Index: 6448907 (Open Web Text)

Max Activating Token Index: 516

Text #11

Max Range: 4.1971. Min Range: -4.1971

Max Act: 4.1078. Min Act: -0.1700

Data Index: 3178932 (Open Web Text)

Max Activating Token Index: 296

Text #12

Max Range: 4.1971. Min Range: -4.1971

Max Act: 4.1152. Min Act: -0.1700

Data Index: 5454606 (Open Web Text)

Max Activating Token Index: 553

Text #13

Max Range: 4.1971. Min Range: -4.1971

Max Act: 4.1356. Min Act: -0.1700

Data Index: 7693713 (Open Web Text)

Max Activating Token Index: 744

Text #14

Max Range: 4.1971. Min Range: -4.1971

Max Act: 4.0893. Min Act: -0.1700

Data Index: 6602743 (Open Web Text)

Max Activating Token Index: 452

Text #15

Max Range: 4.1971. Min Range: -4.1971

Max Act: 4.1060. Min Act: -0.1700

Data Index: 2252139 (Open Web Text)

Max Activating Token Index: 856

Text #16

Max Range: 4.1971. Min Range: -4.1971

Max Act: 4.0878. Min Act: -0.1700

Data Index: 3848240 (Open Web Text)

Max Activating Token Index: 731

Text #17

Max Range: 4.1971. Min Range: -4.1971

Max Act: 4.0922. Min Act: -0.1700

Data Index: 6844070 (Open Web Text)

Max Activating Token Index: 889

Text #18

Max Range: 4.1971. Min Range: -4.1971

Max Act: 4.0884. Min Act: -0.1700

Data Index: 6596805 (Open Web Text)

Max Activating Token Index: 627

Text #19

Max Range: 4.1971. Min Range: -4.1971

Max Act: 4.1260. Min Act: -0.1700

Data Index: 3943379 (Open Web Text)

Max Activating Token Index: 949