Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 2637 in Layer 6

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 3.7605. Min Range: -3.7605

Max Act: 3.7605. Min Act: -0.1700

Data Index: 6588604 (Open Web Text)

Max Activating Token Index: 571

Text #1

Max Range: 3.7605. Min Range: -3.7605

Max Act: 3.6514. Min Act: -0.1700

Data Index: 206564 (Open Web Text)

Max Activating Token Index: 1016

Text #2

Max Range: 3.7605. Min Range: -3.7605

Max Act: 3.5109. Min Act: -0.1700

Data Index: 6570538 (Open Web Text)

Max Activating Token Index: 139

Text #3

Max Range: 3.7605. Min Range: -3.7605

Max Act: 3.5150. Min Act: -0.1700

Data Index: 6278654 (Open Web Text)

Max Activating Token Index: 113

Text #4

Max Range: 3.7605. Min Range: -3.7605

Max Act: 3.4505. Min Act: -0.1700

Data Index: 5489704 (Open Web Text)

Max Activating Token Index: 468

Text #5

Max Range: 3.7605. Min Range: -3.7605

Max Act: 3.4143. Min Act: -0.1700

Data Index: 1634472 (Open Web Text)

Max Activating Token Index: 878

Text #6

Max Range: 3.7605. Min Range: -3.7605

Max Act: 3.4130. Min Act: -0.1700

Data Index: 3944838 (Open Web Text)

Max Activating Token Index: 513

Text #7

Max Range: 3.7605. Min Range: -3.7605

Max Act: 3.3499. Min Act: -0.1700

Data Index: 3420191 (Open Web Text)

Max Activating Token Index: 460

Text #8

Max Range: 3.7605. Min Range: -3.7605

Max Act: 3.4100. Min Act: -0.1700

Data Index: 3654200 (Open Web Text)

Max Activating Token Index: 875

Text #9

Max Range: 3.7605. Min Range: -3.7605

Max Act: 3.3623. Min Act: -0.1700

Data Index: 3076939 (Open Web Text)

Max Activating Token Index: 328

Text #10

Max Range: 3.7605. Min Range: -3.7605

Max Act: 3.3961. Min Act: -0.1700

Data Index: 54733 (Open Web Text)

Max Activating Token Index: 356

Text #11

Max Range: 3.7605. Min Range: -3.7605

Max Act: 3.3520. Min Act: -0.1700

Data Index: 1634473 (Open Web Text)

Max Activating Token Index: 209

Text #12

Max Range: 3.7605. Min Range: -3.7605

Max Act: 3.3553. Min Act: -0.1700

Data Index: 3644341 (Open Web Text)

Max Activating Token Index: 594

Text #13

Max Range: 3.7605. Min Range: -3.7605

Max Act: 3.3681. Min Act: -0.1700

Data Index: 7156140 (Open Web Text)

Max Activating Token Index: 184

Text #14

Max Range: 3.7605. Min Range: -3.7605

Max Act: 3.3324. Min Act: -0.1700

Data Index: 802453 (Open Web Text)

Max Activating Token Index: 899

Text #15

Max Range: 3.7605. Min Range: -3.7605

Max Act: 3.3300. Min Act: -0.1700

Data Index: 8811217 (Open Web Text)

Max Activating Token Index: 44

Text #16

Max Range: 3.7605. Min Range: -3.7605

Max Act: 3.3008. Min Act: -0.1700

Data Index: 3308914 (Open Web Text)

Max Activating Token Index: 242

Text #17

Max Range: 3.7605. Min Range: -3.7605

Max Act: 3.3321. Min Act: -0.1700

Data Index: 7435811 (Open Web Text)

Max Activating Token Index: 240

Text #18

Max Range: 3.7605. Min Range: -3.7605

Max Act: 3.2897. Min Act: -0.1700

Data Index: 3477369 (Open Web Text)

Max Activating Token Index: 527

Text #19

Max Range: 3.7605. Min Range: -3.7605

Max Act: 3.3355. Min Act: -0.1700

Data Index: 6597030 (Open Web Text)

Max Activating Token Index: 289