Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 1545 in Layer 5

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 3.0205. Min Range: -3.0205

Max Act: 3.0205. Min Act: -0.1700

Data Index: 3796564 (Open Web Text)

Max Activating Token Index: 882

Text #1

Max Range: 3.0205. Min Range: -3.0205

Max Act: 2.9005. Min Act: -0.1700

Data Index: 2202852 (Open Web Text)

Max Activating Token Index: 130

Text #2

Max Range: 3.0205. Min Range: -3.0205

Max Act: 2.8736. Min Act: -0.1700

Data Index: 2513643 (Open Web Text)

Max Activating Token Index: 94

Text #3

Max Range: 3.0205. Min Range: -3.0205

Max Act: 2.8608. Min Act: -0.1700

Data Index: 7521617 (Open Web Text)

Max Activating Token Index: 987

Text #4

Max Range: 3.0205. Min Range: -3.0205

Max Act: 2.8927. Min Act: -0.1700

Data Index: 3146261 (Open Web Text)

Max Activating Token Index: 53

Text #5

Max Range: 3.0205. Min Range: -3.0205

Max Act: 2.8753. Min Act: -0.1700

Data Index: 6963181 (Open Web Text)

Max Activating Token Index: 679

Text #6

Max Range: 3.0205. Min Range: -3.0205

Max Act: 2.8904. Min Act: -0.1700

Data Index: 8071195 (Open Web Text)

Max Activating Token Index: 740

Text #7

Max Range: 3.0205. Min Range: -3.0205

Max Act: 2.8479. Min Act: -0.1700

Data Index: 7192768 (Open Web Text)

Max Activating Token Index: 82

Text #8

Max Range: 3.0205. Min Range: -3.0205

Max Act: 2.8501. Min Act: -0.1700

Data Index: 157494 (Open Web Text)

Max Activating Token Index: 194

Text #9

Max Range: 3.0205. Min Range: -3.0205

Max Act: 2.8393. Min Act: -0.1700

Data Index: 5917308 (Open Web Text)

Max Activating Token Index: 953

Text #10

Max Range: 3.0205. Min Range: -3.0205

Max Act: 2.8225. Min Act: -0.1700

Data Index: 4802023 (Open Web Text)

Max Activating Token Index: 665

Text #11

Max Range: 3.0205. Min Range: -3.0205

Max Act: 2.8308. Min Act: -0.1700

Data Index: 3356950 (Open Web Text)

Max Activating Token Index: 191

Text #12

Max Range: 3.0205. Min Range: -3.0205

Max Act: 2.8012. Min Act: -0.1700

Data Index: 7002227 (Open Web Text)

Max Activating Token Index: 704

Text #13

Max Range: 3.0205. Min Range: -3.0205

Max Act: 2.7972. Min Act: -0.1700

Data Index: 3561711 (Open Web Text)

Max Activating Token Index: 709

Text #14

Max Range: 3.0205. Min Range: -3.0205

Max Act: 2.7906. Min Act: -0.1700

Data Index: 2445038 (Open Web Text)

Max Activating Token Index: 903

Text #15

Max Range: 3.0205. Min Range: -3.0205

Max Act: 2.7820. Min Act: -0.1700

Data Index: 5776005 (Open Web Text)

Max Activating Token Index: 376

Text #16

Max Range: 3.0205. Min Range: -3.0205

Max Act: 2.7799. Min Act: -0.1700

Data Index: 7139343 (Open Web Text)

Max Activating Token Index: 608

Text #17

Max Range: 3.0205. Min Range: -3.0205

Max Act: 2.7803. Min Act: -0.1700

Data Index: 6539575 (Open Web Text)

Max Activating Token Index: 868

Text #18

Max Range: 3.0205. Min Range: -3.0205

Max Act: 2.7953. Min Act: -0.1700

Data Index: 5559158 (Open Web Text)

Max Activating Token Index: 381

Text #19

Max Range: 3.0205. Min Range: -3.0205

Max Act: 2.7716. Min Act: -0.1700

Data Index: 3849938 (Open Web Text)

Max Activating Token Index: 341