Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 1416 in Layer 1

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 4.3621. Min Range: -4.3621

Max Act: 4.3621. Min Act: -0.1700

Data Index: 830881 (Open Web Text)

Max Activating Token Index: 343

Text #1

Max Range: 4.3621. Min Range: -4.3621

Max Act: 4.3360. Min Act: -0.1700

Data Index: 20317 (Open Web Text)

Max Activating Token Index: 649

Text #2

Max Range: 4.3621. Min Range: -4.3621

Max Act: 4.2742. Min Act: -0.1700

Data Index: 2701663 (Open Web Text)

Max Activating Token Index: 236

Text #3

Max Range: 4.3621. Min Range: -4.3621

Max Act: 4.2868. Min Act: -0.1700

Data Index: 1594202 (Open Web Text)

Max Activating Token Index: 209

Text #4

Max Range: 4.3621. Min Range: -4.3621

Max Act: 4.3017. Min Act: -0.1700

Data Index: 6614630 (Open Web Text)

Max Activating Token Index: 205

Text #5

Max Range: 4.3621. Min Range: -4.3621

Max Act: 4.2593. Min Act: -0.1700

Data Index: 3665009 (Open Web Text)

Max Activating Token Index: 194

Text #6

Max Range: 4.3621. Min Range: -4.3621

Max Act: 4.2625. Min Act: -0.1700

Data Index: 6865119 (Open Web Text)

Max Activating Token Index: 132

Text #7

Max Range: 4.3621. Min Range: -4.3621

Max Act: 4.2451. Min Act: -0.1700

Data Index: 6614638 (Open Web Text)

Max Activating Token Index: 358

Text #8

Max Range: 4.3621. Min Range: -4.3621

Max Act: 4.2384. Min Act: -0.1700

Data Index: 7912889 (Open Web Text)

Max Activating Token Index: 689

Text #9

Max Range: 4.3621. Min Range: -4.3621

Max Act: 4.2594. Min Act: -0.1700

Data Index: 5114351 (Open Web Text)

Max Activating Token Index: 583

Text #10

Max Range: 4.3621. Min Range: -4.3621

Max Act: 4.2422. Min Act: -0.1700

Data Index: 8593657 (Open Web Text)

Max Activating Token Index: 372

Text #11

Max Range: 4.3621. Min Range: -4.3621

Max Act: 4.2276. Min Act: -0.1700

Data Index: 5273214 (Open Web Text)

Max Activating Token Index: 737

Text #12

Max Range: 4.3621. Min Range: -4.3621

Max Act: 4.2333. Min Act: -0.1700

Data Index: 7674879 (Open Web Text)

Max Activating Token Index: 434

Text #13

Max Range: 4.3621. Min Range: -4.3621

Max Act: 4.2244. Min Act: -0.1700

Data Index: 7549473 (Open Web Text)

Max Activating Token Index: 330

Text #14

Max Range: 4.3621. Min Range: -4.3621

Max Act: 4.2213. Min Act: -0.1700

Data Index: 7549466 (Open Web Text)

Max Activating Token Index: 359

Text #15

Max Range: 4.3621. Min Range: -4.3621

Max Act: 4.2330. Min Act: -0.1700

Data Index: 6297456 (Open Web Text)

Max Activating Token Index: 267

Text #16

Max Range: 4.3621. Min Range: -4.3621

Max Act: 4.2234. Min Act: -0.1700

Data Index: 6297455 (Open Web Text)

Max Activating Token Index: 598

Text #17

Max Range: 4.3621. Min Range: -4.3621

Max Act: 4.2346. Min Act: -0.1700

Data Index: 7050375 (Open Web Text)

Max Activating Token Index: 434

Text #18

Max Range: 4.3621. Min Range: -4.3621

Max Act: 4.1969. Min Act: -0.1700

Data Index: 6614645 (Open Web Text)

Max Activating Token Index: 216

Text #19

Max Range: 4.3621. Min Range: -4.3621

Max Act: 4.1987. Min Act: -0.1700

Data Index: 234637 (Open Web Text)

Max Activating Token Index: 563