Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 3064 in Layer 1

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 4.2788. Min Range: -4.2788

Max Act: 4.2640. Min Act: -0.1700

Data Index: 8158396 (Open Web Text)

Max Activating Token Index: 273

Text #1

Max Range: 4.2788. Min Range: -4.2788

Max Act: 4.2625. Min Act: -0.1700

Data Index: 7592959 (Open Web Text)

Max Activating Token Index: 345

Text #2

Max Range: 4.2788. Min Range: -4.2788

Max Act: 4.2778. Min Act: -0.1700

Data Index: 7065019 (Open Web Text)

Max Activating Token Index: 166

Text #3

Max Range: 4.2788. Min Range: -4.2788

Max Act: 4.2788. Min Act: -0.1700

Data Index: 2823459 (Open Web Text)

Max Activating Token Index: 295

Text #4

Max Range: 4.2788. Min Range: -4.2788

Max Act: 4.2270. Min Act: -0.1700

Data Index: 4898435 (Open Web Text)

Max Activating Token Index: 345

Text #5

Max Range: 4.2788. Min Range: -4.2788

Max Act: 4.2363. Min Act: -0.1700

Data Index: 3586953 (Open Web Text)

Max Activating Token Index: 340

Text #6

Max Range: 4.2788. Min Range: -4.2788

Max Act: 4.2474. Min Act: -0.1700

Data Index: 2106713 (Open Web Text)

Max Activating Token Index: 329

Text #7

Max Range: 4.2788. Min Range: -4.2788

Max Act: 4.2436. Min Act: -0.1700

Data Index: 1000554 (Open Web Text)

Max Activating Token Index: 336

Text #8

Max Range: 4.2788. Min Range: -4.2788

Max Act: 4.2326. Min Act: -0.1700

Data Index: 6749127 (Open Web Text)

Max Activating Token Index: 294

Text #9

Max Range: 4.2788. Min Range: -4.2788

Max Act: 4.2431. Min Act: -0.1700

Data Index: 1605818 (Open Web Text)

Max Activating Token Index: 360

Text #10

Max Range: 4.2788. Min Range: -4.2788

Max Act: 4.2334. Min Act: -0.1700

Data Index: 631974 (Open Web Text)

Max Activating Token Index: 440

Text #11

Max Range: 4.2788. Min Range: -4.2788

Max Act: 4.2318. Min Act: -0.1700

Data Index: 6494378 (Open Web Text)

Max Activating Token Index: 472

Text #12

Max Range: 4.2788. Min Range: -4.2788

Max Act: 4.2449. Min Act: -0.1700

Data Index: 2219977 (Open Web Text)

Max Activating Token Index: 978

Text #13

Max Range: 4.2788. Min Range: -4.2788

Max Act: 4.2347. Min Act: -0.1700

Data Index: 4770035 (Open Web Text)

Max Activating Token Index: 112

Text #14

Max Range: 4.2788. Min Range: -4.2788

Max Act: 4.1852. Min Act: -0.1700

Data Index: 3065677 (Open Web Text)

Max Activating Token Index: 460

Text #15

Max Range: 4.2788. Min Range: -4.2788

Max Act: 4.2057. Min Act: -0.1700

Data Index: 3518305 (Open Web Text)

Max Activating Token Index: 572

Text #16

Max Range: 4.2788. Min Range: -4.2788

Max Act: 4.2070. Min Act: -0.1700

Data Index: 1855682 (Open Web Text)

Max Activating Token Index: 424

Text #17

Max Range: 4.2788. Min Range: -4.2788

Max Act: 4.2061. Min Act: -0.1700

Data Index: 3568649 (Open Web Text)

Max Activating Token Index: 545

Text #18

Max Range: 4.2788. Min Range: -4.2788

Max Act: 4.1823. Min Act: -0.1700

Data Index: 2064625 (Open Web Text)

Max Activating Token Index: 346

Text #19

Max Range: 4.2788. Min Range: -4.2788

Max Act: 4.1944. Min Act: -0.1700

Data Index: 2159583 (Open Web Text)

Max Activating Token Index: 319