Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 589 in Layer 3

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 4.0588. Min Range: -4.0588

Max Act: 4.0588. Min Act: -0.1700

Data Index: 2950240 (Open Web Text)

Max Activating Token Index: 2

Text #1

Max Range: 4.0588. Min Range: -4.0588

Max Act: 3.9284. Min Act: -0.1700

Data Index: 1685554 (Open Web Text)

Max Activating Token Index: 572

Text #2

Max Range: 4.0588. Min Range: -4.0588

Max Act: 3.8948. Min Act: -0.1700

Data Index: 5072693 (Open Web Text)

Max Activating Token Index: 952

Text #3

Max Range: 4.0588. Min Range: -4.0588

Max Act: 3.8314. Min Act: -0.1700

Data Index: 2347130 (Open Web Text)

Max Activating Token Index: 519

Text #4

Max Range: 4.0588. Min Range: -4.0588

Max Act: 3.8415. Min Act: -0.1700

Data Index: 47786 (Open Web Text)

Max Activating Token Index: 940

Text #5

Max Range: 4.0588. Min Range: -4.0588

Max Act: 3.8279. Min Act: -0.1700

Data Index: 6730430 (Open Web Text)

Max Activating Token Index: 90

Text #6

Max Range: 4.0588. Min Range: -4.0588

Max Act: 3.8051. Min Act: -0.1700

Data Index: 6736336 (Open Web Text)

Max Activating Token Index: 609

Text #7

Max Range: 4.0588. Min Range: -4.0588

Max Act: 3.8024. Min Act: -0.1700

Data Index: 555644 (Open Web Text)

Max Activating Token Index: 366

Text #8

Max Range: 4.0588. Min Range: -4.0588

Max Act: 3.7917. Min Act: -0.1700

Data Index: 7801294 (Open Web Text)

Max Activating Token Index: 917

Text #9

Max Range: 4.0588. Min Range: -4.0588

Max Act: 3.7430. Min Act: -0.1700

Data Index: 6982885 (Open Web Text)

Max Activating Token Index: 865

Text #10

Max Range: 4.0588. Min Range: -4.0588

Max Act: 3.7621. Min Act: -0.1700

Data Index: 3201995 (Open Web Text)

Max Activating Token Index: 847

Text #11

Max Range: 4.0588. Min Range: -4.0588

Max Act: 3.7523. Min Act: -0.1700

Data Index: 5497944 (Open Web Text)

Max Activating Token Index: 245

Text #12

Max Range: 4.0588. Min Range: -4.0588

Max Act: 3.7479. Min Act: -0.1700

Data Index: 1585283 (Open Web Text)

Max Activating Token Index: 9

Text #13

Max Range: 4.0588. Min Range: -4.0588

Max Act: 3.7221. Min Act: -0.1700

Data Index: 7246937 (Open Web Text)

Max Activating Token Index: 721

Text #14

Max Range: 4.0588. Min Range: -4.0588

Max Act: 3.7284. Min Act: -0.1700

Data Index: 8195814 (Open Web Text)

Max Activating Token Index: 94

Text #15

Max Range: 4.0588. Min Range: -4.0588

Max Act: 3.6980. Min Act: -0.1700

Data Index: 3061736 (Open Web Text)

Max Activating Token Index: 438

Text #16

Max Range: 4.0588. Min Range: -4.0588

Max Act: 3.6983. Min Act: -0.1700

Data Index: 7576701 (Open Web Text)

Max Activating Token Index: 783

Text #17

Max Range: 4.0588. Min Range: -4.0588

Max Act: 3.6990. Min Act: -0.1700

Data Index: 6260275 (Open Web Text)

Max Activating Token Index: 575

Text #18

Max Range: 4.0588. Min Range: -4.0588

Max Act: 3.7005. Min Act: -0.1700

Data Index: 3633387 (Open Web Text)

Max Activating Token Index: 215

Text #19

Max Range: 4.0588. Min Range: -4.0588

Max Act: 3.6654. Min Act: -0.1700

Data Index: 6652782 (Open Web Text)

Max Activating Token Index: 820