Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 979 in Layer 4

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 3.5627. Min Range: -3.5627

Max Act: 3.5627. Min Act: -0.1700

Data Index: 4307565 (Open Web Text)

Max Activating Token Index: 412

Text #1

Max Range: 3.5627. Min Range: -3.5627

Max Act: 3.5118. Min Act: -0.1700

Data Index: 1506693 (Open Web Text)

Max Activating Token Index: 942

Text #2

Max Range: 3.5627. Min Range: -3.5627

Max Act: 3.5181. Min Act: -0.1700

Data Index: 7476429 (Open Web Text)

Max Activating Token Index: 134

Text #3

Max Range: 3.5627. Min Range: -3.5627

Max Act: 3.4855. Min Act: -0.1700

Data Index: 6574189 (Open Web Text)

Max Activating Token Index: 509

Text #4

Max Range: 3.5627. Min Range: -3.5627

Max Act: 3.4575. Min Act: -0.1700

Data Index: 3615668 (Open Web Text)

Max Activating Token Index: 996

Text #5

Max Range: 3.5627. Min Range: -3.5627

Max Act: 3.4792. Min Act: -0.1700

Data Index: 7475977 (Open Web Text)

Max Activating Token Index: 606

Text #6

Max Range: 3.5627. Min Range: -3.5627

Max Act: 3.4344. Min Act: -0.1700

Data Index: 4718021 (Open Web Text)

Max Activating Token Index: 509

Text #7

Max Range: 3.5627. Min Range: -3.5627

Max Act: 3.4236. Min Act: -0.1700

Data Index: 6927290 (Open Web Text)

Max Activating Token Index: 217

Text #8

Max Range: 3.5627. Min Range: -3.5627

Max Act: 3.4221. Min Act: -0.1700

Data Index: 5041975 (Open Web Text)

Max Activating Token Index: 816

Text #9

Max Range: 3.5627. Min Range: -3.5627

Max Act: 3.3642. Min Act: -0.1700

Data Index: 6421946 (Open Web Text)

Max Activating Token Index: 792

Text #10

Max Range: 3.5627. Min Range: -3.5627

Max Act: 3.4241. Min Act: -0.1700

Data Index: 3387077 (Open Web Text)

Max Activating Token Index: 1013

Text #11

Max Range: 3.5627. Min Range: -3.5627

Max Act: 3.3555. Min Act: -0.1700

Data Index: 3807069 (Open Web Text)

Max Activating Token Index: 309

Text #12

Max Range: 3.5627. Min Range: -3.5627

Max Act: 3.3917. Min Act: -0.1700

Data Index: 7381222 (Open Web Text)

Max Activating Token Index: 423

Text #13

Max Range: 3.5627. Min Range: -3.5627

Max Act: 3.3662. Min Act: -0.1700

Data Index: 3705905 (Open Web Text)

Max Activating Token Index: 245

Text #14

Max Range: 3.5627. Min Range: -3.5627

Max Act: 3.3577. Min Act: -0.1700

Data Index: 7972417 (Open Web Text)

Max Activating Token Index: 796

Text #15

Max Range: 3.5627. Min Range: -3.5627

Max Act: 3.3544. Min Act: -0.1700

Data Index: 7622805 (Open Web Text)

Max Activating Token Index: 274

Text #16

Max Range: 3.5627. Min Range: -3.5627

Max Act: 3.2890. Min Act: -0.1700

Data Index: 713445 (Open Web Text)

Max Activating Token Index: 514

Text #17

Max Range: 3.5627. Min Range: -3.5627

Max Act: 3.3524. Min Act: -0.1700

Data Index: 7931066 (Open Web Text)

Max Activating Token Index: 515

Text #18

Max Range: 3.5627. Min Range: -3.5627

Max Act: 3.3447. Min Act: -0.1700

Data Index: 363608 (Open Web Text)

Max Activating Token Index: 375

Text #19

Max Range: 3.5627. Min Range: -3.5627

Max Act: 3.3065. Min Act: -0.1700

Data Index: 6966221 (Open Web Text)

Max Activating Token Index: 397