Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 923 in Layer 2

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 3.4573. Min Range: -3.4573

Max Act: 3.4573. Min Act: -0.1700

Data Index: 2778673 (Open Web Text)

Max Activating Token Index: 349

Text #1

Max Range: 3.4573. Min Range: -3.4573

Max Act: 3.3607. Min Act: -0.1700

Data Index: 99260 (Open Web Text)

Max Activating Token Index: 20

Text #2

Max Range: 3.4573. Min Range: -3.4573

Max Act: 3.3478. Min Act: -0.1700

Data Index: 6772829 (Open Web Text)

Max Activating Token Index: 8

Text #3

Max Range: 3.4573. Min Range: -3.4573

Max Act: 3.2765. Min Act: -0.1700

Data Index: 7669186 (Open Web Text)

Max Activating Token Index: 90

Text #4

Max Range: 3.4573. Min Range: -3.4573

Max Act: 3.2823. Min Act: -0.1700

Data Index: 1978355 (Open Web Text)

Max Activating Token Index: 413

Text #5

Max Range: 3.4573. Min Range: -3.4573

Max Act: 3.2483. Min Act: -0.1700

Data Index: 1997652 (Open Web Text)

Max Activating Token Index: 588

Text #6

Max Range: 3.4573. Min Range: -3.4573

Max Act: 3.2250. Min Act: -0.1700

Data Index: 8111339 (Open Web Text)

Max Activating Token Index: 160

Text #7

Max Range: 3.4573. Min Range: -3.4573

Max Act: 3.2294. Min Act: -0.1700

Data Index: 5160485 (Open Web Text)

Max Activating Token Index: 454

Text #8

Max Range: 3.4573. Min Range: -3.4573

Max Act: 3.2459. Min Act: -0.1700

Data Index: 6952667 (Open Web Text)

Max Activating Token Index: 135

Text #9

Max Range: 3.4573. Min Range: -3.4573

Max Act: 3.2037. Min Act: -0.1700

Data Index: 374179 (Open Web Text)

Max Activating Token Index: 387

Text #10

Max Range: 3.4573. Min Range: -3.4573

Max Act: 3.2121. Min Act: -0.1700

Data Index: 5277393 (Open Web Text)

Max Activating Token Index: 35

Text #11

Max Range: 3.4573. Min Range: -3.4573

Max Act: 3.2065. Min Act: -0.1700

Data Index: 6341510 (Open Web Text)

Max Activating Token Index: 148

Text #12

Max Range: 3.4573. Min Range: -3.4573

Max Act: 3.1860. Min Act: -0.1700

Data Index: 2653841 (Open Web Text)

Max Activating Token Index: 843

Text #13

Max Range: 3.4573. Min Range: -3.4573

Max Act: 3.1857. Min Act: -0.1700

Data Index: 2361428 (Open Web Text)

Max Activating Token Index: 83

Text #14

Max Range: 3.4573. Min Range: -3.4573

Max Act: 3.1802. Min Act: -0.1700

Data Index: 6011369 (Open Web Text)

Max Activating Token Index: 539

Text #15

Max Range: 3.4573. Min Range: -3.4573

Max Act: 3.1755. Min Act: -0.1700

Data Index: 3958995 (Open Web Text)

Max Activating Token Index: 457

Text #16

Max Range: 3.4573. Min Range: -3.4573

Max Act: 3.1832. Min Act: -0.1700

Data Index: 4816934 (Open Web Text)

Max Activating Token Index: 223

Text #17

Max Range: 3.4573. Min Range: -3.4573

Max Act: 3.1562. Min Act: -0.1700

Data Index: 5577106 (Open Web Text)

Max Activating Token Index: 307

Text #18

Max Range: 3.4573. Min Range: -3.4573

Max Act: 3.1539. Min Act: -0.1700

Data Index: 668645 (Open Web Text)

Max Activating Token Index: 905

Text #19

Max Range: 3.4573. Min Range: -3.4573

Max Act: 3.1326. Min Act: -0.1700

Data Index: 4687333 (Open Web Text)

Max Activating Token Index: 507