Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 1628 in Layer 4

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 4.4297. Min Range: -4.4297

Max Act: 4.4297. Min Act: -0.1700

Data Index: 7737546 (Open Web Text)

Max Activating Token Index: 441

Text #1

Max Range: 4.4297. Min Range: -4.4297

Max Act: 4.3922. Min Act: -0.1700

Data Index: 2980914 (Open Web Text)

Max Activating Token Index: 448

Text #2

Max Range: 4.4297. Min Range: -4.4297

Max Act: 4.3543. Min Act: -0.1700

Data Index: 2282784 (Open Web Text)

Max Activating Token Index: 534

Text #3

Max Range: 4.4297. Min Range: -4.4297

Max Act: 4.3057. Min Act: -0.1700

Data Index: 6048419 (Open Web Text)

Max Activating Token Index: 636

Text #4

Max Range: 4.4297. Min Range: -4.4297

Max Act: 4.3399. Min Act: -0.1700

Data Index: 8244351 (Open Web Text)

Max Activating Token Index: 369

Text #5

Max Range: 4.4297. Min Range: -4.4297

Max Act: 4.3339. Min Act: -0.1700

Data Index: 7394246 (Open Web Text)

Max Activating Token Index: 625

Text #6

Max Range: 4.4297. Min Range: -4.4297

Max Act: 4.2773. Min Act: -0.1700

Data Index: 3242443 (Open Web Text)

Max Activating Token Index: 389

Text #7

Max Range: 4.4297. Min Range: -4.4297

Max Act: 4.2812. Min Act: -0.1700

Data Index: 8186169 (Open Web Text)

Max Activating Token Index: 591

Text #8

Max Range: 4.4297. Min Range: -4.4297

Max Act: 4.2184. Min Act: -0.1700

Data Index: 3723843 (Open Web Text)

Max Activating Token Index: 563

Text #9

Max Range: 4.4297. Min Range: -4.4297

Max Act: 4.2465. Min Act: -0.1700

Data Index: 8356176 (Open Web Text)

Max Activating Token Index: 623

Text #10

Max Range: 4.4297. Min Range: -4.4297

Max Act: 4.2700. Min Act: -0.1700

Data Index: 8527222 (Open Web Text)

Max Activating Token Index: 350

Text #11

Max Range: 4.4297. Min Range: -4.4297

Max Act: 4.2727. Min Act: -0.1700

Data Index: 8753619 (Open Web Text)

Max Activating Token Index: 382

Text #12

Max Range: 4.4297. Min Range: -4.4297

Max Act: 4.2630. Min Act: -0.1700

Data Index: 6656723 (Open Web Text)

Max Activating Token Index: 559

Text #13

Max Range: 4.4297. Min Range: -4.4297

Max Act: 4.2328. Min Act: -0.1700

Data Index: 8813107 (Open Web Text)

Max Activating Token Index: 390

Text #14

Max Range: 4.4297. Min Range: -4.4297

Max Act: 4.2508. Min Act: -0.1700

Data Index: 3264567 (Open Web Text)

Max Activating Token Index: 585

Text #15

Max Range: 4.4297. Min Range: -4.4297

Max Act: 4.2754. Min Act: -0.1700

Data Index: 7730889 (Open Web Text)

Max Activating Token Index: 935

Text #16

Max Range: 4.4297. Min Range: -4.4297

Max Act: 4.2478. Min Act: -0.1700

Data Index: 2618007 (Open Web Text)

Max Activating Token Index: 624

Text #17

Max Range: 4.4297. Min Range: -4.4297

Max Act: 4.2340. Min Act: -0.1700

Data Index: 138149 (Open Web Text)

Max Activating Token Index: 202

Text #18

Max Range: 4.4297. Min Range: -4.4297

Max Act: 4.2076. Min Act: -0.1700

Data Index: 340564 (Open Web Text)

Max Activating Token Index: 867

Text #19

Max Range: 4.4297. Min Range: -4.4297

Max Act: 4.2347. Min Act: -0.1700

Data Index: 7399528 (Open Web Text)

Max Activating Token Index: 268