Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 1599 in Layer 0

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 5.9972. Min Range: -5.9972

Max Act: 5.9972. Min Act: -0.1700

Data Index: 5851623 (Open Web Text)

Max Activating Token Index: 939

Text #1

Max Range: 5.9972. Min Range: -5.9972

Max Act: 5.7372. Min Act: -0.1700

Data Index: 2573622 (Open Web Text)

Max Activating Token Index: 820

Text #2

Max Range: 5.9972. Min Range: -5.9972

Max Act: 5.7043. Min Act: -0.1699

Data Index: 5101118 (Open Web Text)

Max Activating Token Index: 314

Text #3

Max Range: 5.9972. Min Range: -5.9972

Max Act: 5.6456. Min Act: -0.1700

Data Index: 919505 (Open Web Text)

Max Activating Token Index: 288

Text #4

Max Range: 5.9972. Min Range: -5.9972

Max Act: 5.6334. Min Act: -0.1700

Data Index: 7513803 (Open Web Text)

Max Activating Token Index: 568

Text #5

Max Range: 5.9972. Min Range: -5.9972

Max Act: 5.6196. Min Act: -0.1700

Data Index: 7644805 (Open Web Text)

Max Activating Token Index: 516

Text #6

Max Range: 5.9972. Min Range: -5.9972

Max Act: 5.6181. Min Act: -0.1700

Data Index: 2379231 (Open Web Text)

Max Activating Token Index: 516

Text #7

Max Range: 5.9972. Min Range: -5.9972

Max Act: 5.5636. Min Act: -0.1700

Data Index: 2491825 (Open Web Text)

Max Activating Token Index: 645

Text #8

Max Range: 5.9972. Min Range: -5.9972

Max Act: 5.5401. Min Act: -0.1700

Data Index: 8690199 (Open Web Text)

Max Activating Token Index: 325

Text #9

Max Range: 5.9972. Min Range: -5.9972

Max Act: 5.5340. Min Act: -0.1700

Data Index: 7161237 (Open Web Text)

Max Activating Token Index: 905

Text #10

Max Range: 5.9972. Min Range: -5.9972

Max Act: 5.5310. Min Act: -0.1700

Data Index: 6165909 (Open Web Text)

Max Activating Token Index: 463

Text #11

Max Range: 5.9972. Min Range: -5.9972

Max Act: 5.5299. Min Act: -0.1700

Data Index: 3775510 (Open Web Text)

Max Activating Token Index: 114

Text #12

Max Range: 5.9972. Min Range: -5.9972

Max Act: 5.5375. Min Act: -0.1700

Data Index: 5646767 (Open Web Text)

Max Activating Token Index: 280

Text #13

Max Range: 5.9972. Min Range: -5.9972

Max Act: 5.5189. Min Act: -0.1700

Data Index: 7631972 (Open Web Text)

Max Activating Token Index: 15

Text #14

Max Range: 5.9972. Min Range: -5.9972

Max Act: 5.4882. Min Act: -0.1700

Data Index: 5999361 (Open Web Text)

Max Activating Token Index: 654

Text #15

Max Range: 5.9972. Min Range: -5.9972

Max Act: 5.5281. Min Act: -0.1700

Data Index: 3437970 (Open Web Text)

Max Activating Token Index: 444

Text #16

Max Range: 5.9972. Min Range: -5.9972

Max Act: 5.5235. Min Act: -0.1700

Data Index: 272256 (Open Web Text)

Max Activating Token Index: 555

Text #17

Max Range: 5.9972. Min Range: -5.9972

Max Act: 5.5071. Min Act: -0.1700

Data Index: 5677729 (Open Web Text)

Max Activating Token Index: 391

Text #18

Max Range: 5.9972. Min Range: -5.9972

Max Act: 5.5052. Min Act: -0.1700

Data Index: 583057 (Open Web Text)

Max Activating Token Index: 879

Text #19

Max Range: 5.9972. Min Range: -5.9972

Max Act: 5.4738. Min Act: -0.1700

Data Index: 7396904 (Open Web Text)

Max Activating Token Index: 507