Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 1807 in Layer 4

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 3.2544. Min Range: -3.2544

Max Act: 3.2544. Min Act: -0.1700

Data Index: 358431 (Open Web Text)

Max Activating Token Index: 939

Text #1

Max Range: 3.2544. Min Range: -3.2544

Max Act: 3.2509. Min Act: -0.1700

Data Index: 6770803 (Open Web Text)

Max Activating Token Index: 580

Text #2

Max Range: 3.2544. Min Range: -3.2544

Max Act: 3.2190. Min Act: -0.1700

Data Index: 5584304 (Open Web Text)

Max Activating Token Index: 311

Text #3

Max Range: 3.2544. Min Range: -3.2544

Max Act: 3.1722. Min Act: -0.1700

Data Index: 3342103 (Open Web Text)

Max Activating Token Index: 288

Text #4

Max Range: 3.2544. Min Range: -3.2544

Max Act: 3.1677. Min Act: -0.1700

Data Index: 854195 (Open Web Text)

Max Activating Token Index: 821

Text #5

Max Range: 3.2544. Min Range: -3.2544

Max Act: 3.1889. Min Act: -0.1700

Data Index: 5963998 (Open Web Text)

Max Activating Token Index: 807

Text #6

Max Range: 3.2544. Min Range: -3.2544

Max Act: 3.1425. Min Act: -0.1700

Data Index: 7602945 (Open Web Text)

Max Activating Token Index: 848

Text #7

Max Range: 3.2544. Min Range: -3.2544

Max Act: 3.1455. Min Act: -0.1700

Data Index: 8357206 (Open Web Text)

Max Activating Token Index: 85

Text #8

Max Range: 3.2544. Min Range: -3.2544

Max Act: 3.1642. Min Act: -0.1700

Data Index: 3841066 (Open Web Text)

Max Activating Token Index: 899

Text #9

Max Range: 3.2544. Min Range: -3.2544

Max Act: 3.1549. Min Act: -0.1700

Data Index: 4642772 (Open Web Text)

Max Activating Token Index: 992

Text #10

Max Range: 3.2544. Min Range: -3.2544

Max Act: 3.1261. Min Act: -0.1700

Data Index: 5343707 (Open Web Text)

Max Activating Token Index: 607

Text #11

Max Range: 3.2544. Min Range: -3.2544

Max Act: 3.0957. Min Act: -0.1700

Data Index: 6153184 (Open Web Text)

Max Activating Token Index: 502

Text #12

Max Range: 3.2544. Min Range: -3.2544

Max Act: 3.1101. Min Act: -0.1700

Data Index: 3757512 (Open Web Text)

Max Activating Token Index: 153

Text #13

Max Range: 3.2544. Min Range: -3.2544

Max Act: 3.1176. Min Act: -0.1700

Data Index: 1309861 (Open Web Text)

Max Activating Token Index: 331

Text #14

Max Range: 3.2544. Min Range: -3.2544

Max Act: 3.1013. Min Act: -0.1700

Data Index: 8507127 (Open Web Text)

Max Activating Token Index: 226

Text #15

Max Range: 3.2544. Min Range: -3.2544

Max Act: 3.1030. Min Act: -0.1700

Data Index: 7002138 (Open Web Text)

Max Activating Token Index: 689

Text #16

Max Range: 3.2544. Min Range: -3.2544

Max Act: 3.1049. Min Act: -0.1700

Data Index: 8665418 (Open Web Text)

Max Activating Token Index: 217

Text #17

Max Range: 3.2544. Min Range: -3.2544

Max Act: 3.0916. Min Act: -0.1700

Data Index: 5396373 (Open Web Text)

Max Activating Token Index: 776

Text #18

Max Range: 3.2544. Min Range: -3.2544

Max Act: 3.0858. Min Act: -0.1700

Data Index: 6284844 (Open Web Text)

Max Activating Token Index: 334

Text #19

Max Range: 3.2544. Min Range: -3.2544

Max Act: 3.0733. Min Act: -0.1700

Data Index: 748495 (Open Web Text)

Max Activating Token Index: 636