Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 2411 in Layer 4

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 3.8674. Min Range: -3.8674

Max Act: 3.8674. Min Act: -0.1700

Data Index: 8216496 (Open Web Text)

Max Activating Token Index: 216

Text #1

Max Range: 3.8674. Min Range: -3.8674

Max Act: 3.7914. Min Act: -0.1700

Data Index: 8287565 (Open Web Text)

Max Activating Token Index: 884

Text #2

Max Range: 3.8674. Min Range: -3.8674

Max Act: 3.8232. Min Act: -0.1700

Data Index: 2193857 (Open Web Text)

Max Activating Token Index: 381

Text #3

Max Range: 3.8674. Min Range: -3.8674

Max Act: 3.7547. Min Act: -0.1700

Data Index: 4298682 (Open Web Text)

Max Activating Token Index: 422

Text #4

Max Range: 3.8674. Min Range: -3.8674

Max Act: 3.7867. Min Act: -0.1700

Data Index: 2278202 (Open Web Text)

Max Activating Token Index: 508

Text #5

Max Range: 3.8674. Min Range: -3.8674

Max Act: 3.7841. Min Act: -0.1700

Data Index: 952010 (Open Web Text)

Max Activating Token Index: 6

Text #6

Max Range: 3.8674. Min Range: -3.8674

Max Act: 3.7514. Min Act: -0.1700

Data Index: 6346069 (Open Web Text)

Max Activating Token Index: 242

Text #7

Max Range: 3.8674. Min Range: -3.8674

Max Act: 3.7465. Min Act: -0.1700

Data Index: 7840425 (Open Web Text)

Max Activating Token Index: 575

Text #8

Max Range: 3.8674. Min Range: -3.8674

Max Act: 3.7296. Min Act: -0.1700

Data Index: 1415546 (Open Web Text)

Max Activating Token Index: 735

Text #9

Max Range: 3.8674. Min Range: -3.8674

Max Act: 3.7607. Min Act: -0.1700

Data Index: 4180165 (Open Web Text)

Max Activating Token Index: 325

Text #10

Max Range: 3.8674. Min Range: -3.8674

Max Act: 3.7292. Min Act: -0.1700

Data Index: 5358586 (Open Web Text)

Max Activating Token Index: 410

Text #11

Max Range: 3.8674. Min Range: -3.8674

Max Act: 3.6727. Min Act: -0.1700

Data Index: 8287566 (Open Web Text)

Max Activating Token Index: 221

Text #12

Max Range: 3.8674. Min Range: -3.8674

Max Act: 3.7153. Min Act: -0.1700

Data Index: 4734578 (Open Web Text)

Max Activating Token Index: 1015

Text #13

Max Range: 3.8674. Min Range: -3.8674

Max Act: 3.7033. Min Act: -0.1700

Data Index: 1906227 (Open Web Text)

Max Activating Token Index: 531

Text #14

Max Range: 3.8674. Min Range: -3.8674

Max Act: 3.6787. Min Act: -0.1700

Data Index: 2398059 (Open Web Text)

Max Activating Token Index: 999

Text #15

Max Range: 3.8674. Min Range: -3.8674

Max Act: 3.6592. Min Act: -0.1700

Data Index: 2418924 (Open Web Text)

Max Activating Token Index: 4

Text #16

Max Range: 3.8674. Min Range: -3.8674

Max Act: 3.6464. Min Act: -0.1700

Data Index: 7779456 (Open Web Text)

Max Activating Token Index: 282

Text #17

Max Range: 3.8674. Min Range: -3.8674

Max Act: 3.6667. Min Act: -0.1700

Data Index: 410739 (Open Web Text)

Max Activating Token Index: 598

Text #18

Max Range: 3.8674. Min Range: -3.8674

Max Act: 3.6473. Min Act: -0.1700

Data Index: 5291107 (Open Web Text)

Max Activating Token Index: 992

Text #19

Max Range: 3.8674. Min Range: -3.8674

Max Act: 3.6811. Min Act: -0.1700

Data Index: 3792624 (Open Web Text)

Max Activating Token Index: 1002