Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 114 in Layer 2

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 3.7565. Min Range: -3.7565

Max Act: 3.7565. Min Act: -0.1700

Data Index: 2683553 (Open Web Text)

Max Activating Token Index: 4

Text #1

Max Range: 3.7565. Min Range: -3.7565

Max Act: 3.5750. Min Act: -0.1700

Data Index: 4597785 (Open Web Text)

Max Activating Token Index: 297

Text #2

Max Range: 3.7565. Min Range: -3.7565

Max Act: 3.5256. Min Act: -0.1700

Data Index: 1083339 (Open Web Text)

Max Activating Token Index: 719

Text #3

Max Range: 3.7565. Min Range: -3.7565

Max Act: 3.5467. Min Act: -0.1700

Data Index: 6753509 (Open Web Text)

Max Activating Token Index: 205

Text #4

Max Range: 3.7565. Min Range: -3.7565

Max Act: 3.5238. Min Act: -0.1700

Data Index: 497634 (Open Web Text)

Max Activating Token Index: 102

Text #5

Max Range: 3.7565. Min Range: -3.7565

Max Act: 3.5537. Min Act: -0.1700

Data Index: 857889 (Open Web Text)

Max Activating Token Index: 568

Text #6

Max Range: 3.7565. Min Range: -3.7565

Max Act: 3.5395. Min Act: -0.1700

Data Index: 3025461 (Open Web Text)

Max Activating Token Index: 125

Text #7

Max Range: 3.7565. Min Range: -3.7565

Max Act: 3.4974. Min Act: -0.1700

Data Index: 207905 (Open Web Text)

Max Activating Token Index: 38

Text #8

Max Range: 3.7565. Min Range: -3.7565

Max Act: 3.5141. Min Act: -0.1700

Data Index: 3050713 (Open Web Text)

Max Activating Token Index: 457

Text #9

Max Range: 3.7565. Min Range: -3.7565

Max Act: 3.5103. Min Act: -0.1700

Data Index: 8672685 (Open Web Text)

Max Activating Token Index: 287

Text #10

Max Range: 3.7565. Min Range: -3.7565

Max Act: 3.5127. Min Act: -0.1700

Data Index: 8557610 (Open Web Text)

Max Activating Token Index: 371

Text #11

Max Range: 3.7565. Min Range: -3.7565

Max Act: 3.4820. Min Act: -0.1700

Data Index: 5903820 (Open Web Text)

Max Activating Token Index: 398

Text #12

Max Range: 3.7565. Min Range: -3.7565

Max Act: 3.4818. Min Act: -0.1700

Data Index: 5962548 (Open Web Text)

Max Activating Token Index: 296

Text #13

Max Range: 3.7565. Min Range: -3.7565

Max Act: 3.4686. Min Act: -0.1700

Data Index: 3016500 (Open Web Text)

Max Activating Token Index: 271

Text #14

Max Range: 3.7565. Min Range: -3.7565

Max Act: 3.4735. Min Act: -0.1700

Data Index: 439404 (Open Web Text)

Max Activating Token Index: 941

Text #15

Max Range: 3.7565. Min Range: -3.7565

Max Act: 3.4613. Min Act: -0.1700

Data Index: 6057790 (Open Web Text)

Max Activating Token Index: 474

Text #16

Max Range: 3.7565. Min Range: -3.7565

Max Act: 3.4883. Min Act: -0.1700

Data Index: 1077975 (Open Web Text)

Max Activating Token Index: 686

Text #17

Max Range: 3.7565. Min Range: -3.7565

Max Act: 3.4746. Min Act: -0.1700

Data Index: 1397828 (Open Web Text)

Max Activating Token Index: 147

Text #18

Max Range: 3.7565. Min Range: -3.7565

Max Act: 3.4800. Min Act: -0.1700

Data Index: 6546215 (Open Web Text)

Max Activating Token Index: 1022

Text #19

Max Range: 3.7565. Min Range: -3.7565

Max Act: 3.4683. Min Act: -0.1700

Data Index: 3653811 (Open Web Text)

Max Activating Token Index: 110