Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 1563 in Layer 4

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 3.6365. Min Range: -3.6365

Max Act: 3.6365. Min Act: -0.1700

Data Index: 4438246 (Open Web Text)

Max Activating Token Index: 155

Text #1

Max Range: 3.6365. Min Range: -3.6365

Max Act: 3.4320. Min Act: -0.1700

Data Index: 3769232 (Open Web Text)

Max Activating Token Index: 657

Text #2

Max Range: 3.6365. Min Range: -3.6365

Max Act: 3.3492. Min Act: -0.1700

Data Index: 8592068 (Open Web Text)

Max Activating Token Index: 189

Text #3

Max Range: 3.6365. Min Range: -3.6365

Max Act: 3.3125. Min Act: -0.1700

Data Index: 2409689 (Open Web Text)

Max Activating Token Index: 900

Text #4

Max Range: 3.6365. Min Range: -3.6365

Max Act: 3.2967. Min Act: -0.1700

Data Index: 5793472 (Open Web Text)

Max Activating Token Index: 170

Text #5

Max Range: 3.6365. Min Range: -3.6365

Max Act: 3.2487. Min Act: -0.1700

Data Index: 1901207 (Open Web Text)

Max Activating Token Index: 319

Text #6

Max Range: 3.6365. Min Range: -3.6365

Max Act: 3.2596. Min Act: -0.1700

Data Index: 4534666 (Open Web Text)

Max Activating Token Index: 179

Text #7

Max Range: 3.6365. Min Range: -3.6365

Max Act: 3.2969. Min Act: -0.1700

Data Index: 1300398 (Open Web Text)

Max Activating Token Index: 1018

Text #8

Max Range: 3.6365. Min Range: -3.6365

Max Act: 3.2200. Min Act: -0.1700

Data Index: 7034318 (Open Web Text)

Max Activating Token Index: 340

Text #9

Max Range: 3.6365. Min Range: -3.6365

Max Act: 3.2428. Min Act: -0.1700

Data Index: 2763768 (Open Web Text)

Max Activating Token Index: 108

Text #10

Max Range: 3.6365. Min Range: -3.6365

Max Act: 3.2465. Min Act: -0.1700

Data Index: 1415727 (Open Web Text)

Max Activating Token Index: 946

Text #11

Max Range: 3.6365. Min Range: -3.6365

Max Act: 3.1906. Min Act: -0.1700

Data Index: 4846482 (Open Web Text)

Max Activating Token Index: 237

Text #12

Max Range: 3.6365. Min Range: -3.6365

Max Act: 3.1869. Min Act: -0.1700

Data Index: 6936425 (Open Web Text)

Max Activating Token Index: 732

Text #13

Max Range: 3.6365. Min Range: -3.6365

Max Act: 3.1700. Min Act: -0.1700

Data Index: 3645535 (Open Web Text)

Max Activating Token Index: 979

Text #14

Max Range: 3.6365. Min Range: -3.6365

Max Act: 3.1607. Min Act: -0.1700

Data Index: 3380733 (Open Web Text)

Max Activating Token Index: 427

Text #15

Max Range: 3.6365. Min Range: -3.6365

Max Act: 3.1618. Min Act: -0.1700

Data Index: 2813692 (Open Web Text)

Max Activating Token Index: 823

Text #16

Max Range: 3.6365. Min Range: -3.6365

Max Act: 3.1657. Min Act: -0.1700

Data Index: 956107 (Open Web Text)

Max Activating Token Index: 505

Text #17

Max Range: 3.6365. Min Range: -3.6365

Max Act: 3.1848. Min Act: -0.1700

Data Index: 4069826 (Open Web Text)

Max Activating Token Index: 782

Text #18

Max Range: 3.6365. Min Range: -3.6365

Max Act: 3.1434. Min Act: -0.1700

Data Index: 2534032 (Open Web Text)

Max Activating Token Index: 896

Text #19

Max Range: 3.6365. Min Range: -3.6365

Max Act: 3.1248. Min Act: -0.1700

Data Index: 575455 (Open Web Text)

Max Activating Token Index: 378