Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 976 in Layer 0

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 1.3711. Min Range: -1.3711

Max Act: 1.3711. Min Act: -0.0330

Data Index: 1587238 (Open Web Text)

Max Activating Token Index: 1017

Text #1

Max Range: 1.3711. Min Range: -1.3711

Max Act: 1.3499. Min Act: -0.0715

Data Index: 7231250 (Open Web Text)

Max Activating Token Index: 319

Text #2

Max Range: 1.3711. Min Range: -1.3711

Max Act: 1.3452. Min Act: -0.1144

Data Index: 1587237 (Open Web Text)

Max Activating Token Index: 712

Text #3

Max Range: 1.3711. Min Range: -1.3711

Max Act: 1.3308. Min Act: -0.0618

Data Index: 335961 (Open Web Text)

Max Activating Token Index: 213

Text #4

Max Range: 1.3711. Min Range: -1.3711

Max Act: 1.3242. Min Act: -0.0493

Data Index: 256982 (Open Web Text)

Max Activating Token Index: 762

Text #5

Max Range: 1.3711. Min Range: -1.3711

Max Act: 1.3206. Min Act: -0.0723

Data Index: 6050948 (Open Web Text)

Max Activating Token Index: 109

Text #6

Max Range: 1.3711. Min Range: -1.3711

Max Act: 1.3159. Min Act: -0.0949

Data Index: 107504 (Open Web Text)

Max Activating Token Index: 215

Text #7

Max Range: 1.3711. Min Range: -1.3711

Max Act: 1.2993. Min Act: -0.0932

Data Index: 5235481 (Open Web Text)

Max Activating Token Index: 747

Text #8

Max Range: 1.3711. Min Range: -1.3711

Max Act: 1.3078. Min Act: -0.0912

Data Index: 5964458 (Open Web Text)

Max Activating Token Index: 598

Text #9

Max Range: 1.3711. Min Range: -1.3711

Max Act: 1.2990. Min Act: -0.0330

Data Index: 1587239 (Open Web Text)

Max Activating Token Index: 311

Text #10

Max Range: 1.3711. Min Range: -1.3711

Max Act: 1.2926. Min Act: -0.1332

Data Index: 1508192 (Open Web Text)

Max Activating Token Index: 168

Text #11

Max Range: 1.3711. Min Range: -1.3711

Max Act: 1.2825. Min Act: -0.0635

Data Index: 2633991 (Open Web Text)

Max Activating Token Index: 784

Text #12

Max Range: 1.3711. Min Range: -1.3711

Max Act: 1.2682. Min Act: -0.0798

Data Index: 4221286 (Open Web Text)

Max Activating Token Index: 192

Text #13

Max Range: 1.3711. Min Range: -1.3711

Max Act: 1.2745. Min Act: -0.1008

Data Index: 8667235 (Open Web Text)

Max Activating Token Index: 366

Text #14

Max Range: 1.3711. Min Range: -1.3711

Max Act: 1.2757. Min Act: -0.0873

Data Index: 6507669 (Open Web Text)

Max Activating Token Index: 604

Text #15

Max Range: 1.3711. Min Range: -1.3711

Max Act: 1.2758. Min Act: -0.0584

Data Index: 466528 (Open Web Text)

Max Activating Token Index: 169

Text #16

Max Range: 1.3711. Min Range: -1.3711

Max Act: 1.2711. Min Act: -0.0939

Data Index: 4381537 (Open Web Text)

Max Activating Token Index: 203

Text #17

Max Range: 1.3711. Min Range: -1.3711

Max Act: 1.2699. Min Act: -0.0958

Data Index: 1781634 (Open Web Text)

Max Activating Token Index: 226

Text #18

Max Range: 1.3711. Min Range: -1.3711

Max Act: 1.2738. Min Act: -0.0330

Data Index: 4719136 (Open Web Text)

Max Activating Token Index: 1020

Text #19

Max Range: 1.3711. Min Range: -1.3711

Max Act: 1.2682. Min Act: -0.0948

Data Index: 4089525 (Open Web Text)

Max Activating Token Index: 192