Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 1241 in Layer 4

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 4.1315. Min Range: -4.1315

Max Act: 4.1315. Min Act: -0.1700

Data Index: 6167252 (Open Web Text)

Max Activating Token Index: 929

Text #1

Max Range: 4.1315. Min Range: -4.1315

Max Act: 4.1216. Min Act: -0.1700

Data Index: 7585656 (Open Web Text)

Max Activating Token Index: 515

Text #2

Max Range: 4.1315. Min Range: -4.1315

Max Act: 4.0728. Min Act: -0.1700

Data Index: 6674735 (Open Web Text)

Max Activating Token Index: 361

Text #3

Max Range: 4.1315. Min Range: -4.1315

Max Act: 4.0686. Min Act: -0.1700

Data Index: 5117666 (Open Web Text)

Max Activating Token Index: 983

Text #4

Max Range: 4.1315. Min Range: -4.1315

Max Act: 4.1123. Min Act: -0.1700

Data Index: 2212257 (Open Web Text)

Max Activating Token Index: 873

Text #5

Max Range: 4.1315. Min Range: -4.1315

Max Act: 4.0617. Min Act: -0.1700

Data Index: 8327590 (Open Web Text)

Max Activating Token Index: 646

Text #6

Max Range: 4.1315. Min Range: -4.1315

Max Act: 4.0169. Min Act: -0.1700

Data Index: 5624721 (Open Web Text)

Max Activating Token Index: 856

Text #7

Max Range: 4.1315. Min Range: -4.1315

Max Act: 4.0154. Min Act: -0.1700

Data Index: 8777741 (Open Web Text)

Max Activating Token Index: 826

Text #8

Max Range: 4.1315. Min Range: -4.1315

Max Act: 4.0228. Min Act: -0.1700

Data Index: 5341912 (Open Web Text)

Max Activating Token Index: 545

Text #9

Max Range: 4.1315. Min Range: -4.1315

Max Act: 4.0175. Min Act: -0.1700

Data Index: 1165569 (Open Web Text)

Max Activating Token Index: 35

Text #10

Max Range: 4.1315. Min Range: -4.1315

Max Act: 3.9905. Min Act: -0.1700

Data Index: 1393481 (Open Web Text)

Max Activating Token Index: 486

Text #11

Max Range: 4.1315. Min Range: -4.1315

Max Act: 4.0176. Min Act: -0.1700

Data Index: 3341670 (Open Web Text)

Max Activating Token Index: 743

Text #12

Max Range: 4.1315. Min Range: -4.1315

Max Act: 4.0045. Min Act: -0.1700

Data Index: 2696233 (Open Web Text)

Max Activating Token Index: 283

Text #13

Max Range: 4.1315. Min Range: -4.1315

Max Act: 3.9661. Min Act: -0.1700

Data Index: 1790424 (Open Web Text)

Max Activating Token Index: 546

Text #14

Max Range: 4.1315. Min Range: -4.1315

Max Act: 4.0149. Min Act: -0.1700

Data Index: 8151570 (Open Web Text)

Max Activating Token Index: 272

Text #15

Max Range: 4.1315. Min Range: -4.1315

Max Act: 3.9588. Min Act: -0.1700

Data Index: 2732427 (Open Web Text)

Max Activating Token Index: 785

Text #16

Max Range: 4.1315. Min Range: -4.1315

Max Act: 3.9709. Min Act: -0.1700

Data Index: 2033388 (Open Web Text)

Max Activating Token Index: 354

Text #17

Max Range: 4.1315. Min Range: -4.1315

Max Act: 3.9553. Min Act: -0.1700

Data Index: 5409305 (Open Web Text)

Max Activating Token Index: 964

Text #18

Max Range: 4.1315. Min Range: -4.1315

Max Act: 3.9682. Min Act: -0.1700

Data Index: 3197765 (Open Web Text)

Max Activating Token Index: 530

Text #19

Max Range: 4.1315. Min Range: -4.1315

Max Act: 3.9496. Min Act: -0.1700

Data Index: 5426665 (Open Web Text)

Max Activating Token Index: 520