Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 2436 in Layer 6

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 3.2262. Min Range: -3.2262

Max Act: 3.2262. Min Act: -0.1700

Data Index: 446079 (Open Web Text)

Max Activating Token Index: 256

Text #1

Max Range: 3.2262. Min Range: -3.2262

Max Act: 3.1600. Min Act: -0.1700

Data Index: 8680041 (Open Web Text)

Max Activating Token Index: 711

Text #2

Max Range: 3.2262. Min Range: -3.2262

Max Act: 3.1406. Min Act: -0.1700

Data Index: 1925622 (Open Web Text)

Max Activating Token Index: 216

Text #3

Max Range: 3.2262. Min Range: -3.2262

Max Act: 3.1128. Min Act: -0.1700

Data Index: 6094386 (Open Web Text)

Max Activating Token Index: 119

Text #4

Max Range: 3.2262. Min Range: -3.2262

Max Act: 3.0929. Min Act: -0.1700

Data Index: 5082957 (Open Web Text)

Max Activating Token Index: 353

Text #5

Max Range: 3.2262. Min Range: -3.2262

Max Act: 3.0958. Min Act: -0.1700

Data Index: 1047986 (Open Web Text)

Max Activating Token Index: 319

Text #6

Max Range: 3.2262. Min Range: -3.2262

Max Act: 3.0924. Min Act: -0.1700

Data Index: 859725 (Open Web Text)

Max Activating Token Index: 1002

Text #7

Max Range: 3.2262. Min Range: -3.2262

Max Act: 3.0877. Min Act: -0.1700

Data Index: 8121500 (Open Web Text)

Max Activating Token Index: 990

Text #8

Max Range: 3.2262. Min Range: -3.2262

Max Act: 3.0627. Min Act: -0.1700

Data Index: 1651332 (Open Web Text)

Max Activating Token Index: 180

Text #9

Max Range: 3.2262. Min Range: -3.2262

Max Act: 3.0447. Min Act: -0.1700

Data Index: 5692196 (Open Web Text)

Max Activating Token Index: 388

Text #10

Max Range: 3.2262. Min Range: -3.2262

Max Act: 3.0142. Min Act: -0.1700

Data Index: 3534488 (Open Web Text)

Max Activating Token Index: 253

Text #11

Max Range: 3.2262. Min Range: -3.2262

Max Act: 3.0300. Min Act: -0.1700

Data Index: 5246157 (Open Web Text)

Max Activating Token Index: 6

Text #12

Max Range: 3.2262. Min Range: -3.2262

Max Act: 2.9795. Min Act: -0.1700

Data Index: 3992568 (Open Web Text)

Max Activating Token Index: 353

Text #13

Max Range: 3.2262. Min Range: -3.2262

Max Act: 3.0100. Min Act: -0.1700

Data Index: 7539611 (Open Web Text)

Max Activating Token Index: 554

Text #14

Max Range: 3.2262. Min Range: -3.2262

Max Act: 3.0355. Min Act: -0.1700

Data Index: 4773788 (Open Web Text)

Max Activating Token Index: 421

Text #15

Max Range: 3.2262. Min Range: -3.2262

Max Act: 2.9832. Min Act: -0.1700

Data Index: 5313585 (Open Web Text)

Max Activating Token Index: 763

Text #16

Max Range: 3.2262. Min Range: -3.2262

Max Act: 2.9950. Min Act: -0.1700

Data Index: 6296532 (Open Web Text)

Max Activating Token Index: 737

Text #17

Max Range: 3.2262. Min Range: -3.2262

Max Act: 3.0292. Min Act: -0.1700

Data Index: 5197733 (Open Web Text)

Max Activating Token Index: 463

Text #18

Max Range: 3.2262. Min Range: -3.2262

Max Act: 3.0034. Min Act: -0.1700

Data Index: 3548337 (Open Web Text)

Max Activating Token Index: 426

Text #19

Max Range: 3.2262. Min Range: -3.2262

Max Act: 2.9736. Min Act: -0.1700

Data Index: 1288214 (Open Web Text)

Max Activating Token Index: 868