Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 2390 in Layer 5

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 5.1448. Min Range: -5.1448

Max Act: 5.1448. Min Act: -0.1700

Data Index: 2926106 (Open Web Text)

Max Activating Token Index: 61

Text #1

Max Range: 5.1448. Min Range: -5.1448

Max Act: 5.0827. Min Act: -0.1700

Data Index: 5216238 (Open Web Text)

Max Activating Token Index: 732

Text #2

Max Range: 5.1448. Min Range: -5.1448

Max Act: 5.0725. Min Act: -0.1700

Data Index: 3971664 (Open Web Text)

Max Activating Token Index: 189

Text #3

Max Range: 5.1448. Min Range: -5.1448

Max Act: 5.0589. Min Act: -0.1700

Data Index: 5583912 (Open Web Text)

Max Activating Token Index: 573

Text #4

Max Range: 5.1448. Min Range: -5.1448

Max Act: 5.0192. Min Act: -0.1700

Data Index: 8495460 (Open Web Text)

Max Activating Token Index: 114

Text #5

Max Range: 5.1448. Min Range: -5.1448

Max Act: 5.0072. Min Act: -0.1700

Data Index: 4798712 (Open Web Text)

Max Activating Token Index: 339

Text #6

Max Range: 5.1448. Min Range: -5.1448

Max Act: 4.9707. Min Act: -0.1700

Data Index: 8232764 (Open Web Text)

Max Activating Token Index: 263

Text #7

Max Range: 5.1448. Min Range: -5.1448

Max Act: 4.9469. Min Act: -0.1700

Data Index: 7912471 (Open Web Text)

Max Activating Token Index: 241

Text #8

Max Range: 5.1448. Min Range: -5.1448

Max Act: 4.9884. Min Act: -0.1700

Data Index: 3378675 (Open Web Text)

Max Activating Token Index: 176

Text #9

Max Range: 5.1448. Min Range: -5.1448

Max Act: 4.9675. Min Act: -0.1700

Data Index: 3659513 (Open Web Text)

Max Activating Token Index: 973

Text #10

Max Range: 5.1448. Min Range: -5.1448

Max Act: 4.9331. Min Act: -0.1700

Data Index: 6835189 (Open Web Text)

Max Activating Token Index: 674

Text #11

Max Range: 5.1448. Min Range: -5.1448

Max Act: 4.9400. Min Act: -0.1700

Data Index: 4497049 (Open Web Text)

Max Activating Token Index: 278

Text #12

Max Range: 5.1448. Min Range: -5.1448

Max Act: 4.8808. Min Act: -0.1700

Data Index: 5544270 (Open Web Text)

Max Activating Token Index: 404

Text #13

Max Range: 5.1448. Min Range: -5.1448

Max Act: 4.8672. Min Act: -0.1700

Data Index: 2143655 (Open Web Text)

Max Activating Token Index: 259

Text #14

Max Range: 5.1448. Min Range: -5.1448

Max Act: 4.8825. Min Act: -0.1700

Data Index: 2902169 (Open Web Text)

Max Activating Token Index: 587

Text #15

Max Range: 5.1448. Min Range: -5.1448

Max Act: 4.8973. Min Act: -0.1700

Data Index: 7287133 (Open Web Text)

Max Activating Token Index: 778

Text #16

Max Range: 5.1448. Min Range: -5.1448

Max Act: 4.8517. Min Act: -0.1700

Data Index: 7242849 (Open Web Text)

Max Activating Token Index: 713

Text #17

Max Range: 5.1448. Min Range: -5.1448

Max Act: 4.8722. Min Act: -0.1700

Data Index: 5729839 (Open Web Text)

Max Activating Token Index: 274

Text #18

Max Range: 5.1448. Min Range: -5.1448

Max Act: 4.9052. Min Act: -0.1700

Data Index: 5618705 (Open Web Text)

Max Activating Token Index: 644

Text #19

Max Range: 5.1448. Min Range: -5.1448

Max Act: 4.8607. Min Act: -0.1700

Data Index: 3867596 (Open Web Text)

Max Activating Token Index: 875