Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 2218 in Layer 2

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 1.8564. Min Range: -1.8564

Max Act: 1.8564. Min Act: -0.1700

Data Index: 312593 (Open Web Text)

Max Activating Token Index: 215

Text #1

Max Range: 1.8564. Min Range: -1.8564

Max Act: 1.8544. Min Act: -0.1700

Data Index: 2907569 (Open Web Text)

Max Activating Token Index: 783

Text #2

Max Range: 1.8564. Min Range: -1.8564

Max Act: 1.8352. Min Act: -0.1700

Data Index: 997429 (Open Web Text)

Max Activating Token Index: 92

Text #3

Max Range: 1.8564. Min Range: -1.8564

Max Act: 1.8266. Min Act: -0.1700

Data Index: 159013 (Open Web Text)

Max Activating Token Index: 748

Text #4

Max Range: 1.8564. Min Range: -1.8564

Max Act: 1.8197. Min Act: -0.1700

Data Index: 7400856 (Open Web Text)

Max Activating Token Index: 390

Text #5

Max Range: 1.8564. Min Range: -1.8564

Max Act: 1.8321. Min Act: -0.1700

Data Index: 2354220 (Open Web Text)

Max Activating Token Index: 161

Text #6

Max Range: 1.8564. Min Range: -1.8564

Max Act: 1.8305. Min Act: -0.1700

Data Index: 7838206 (Open Web Text)

Max Activating Token Index: 937

Text #7

Max Range: 1.8564. Min Range: -1.8564

Max Act: 1.8423. Min Act: -0.1700

Data Index: 4772395 (Open Web Text)

Max Activating Token Index: 862

Text #8

Max Range: 1.8564. Min Range: -1.8564

Max Act: 1.8258. Min Act: -0.1700

Data Index: 2766766 (Open Web Text)

Max Activating Token Index: 530

Text #9

Max Range: 1.8564. Min Range: -1.8564

Max Act: 1.8039. Min Act: -0.1700

Data Index: 6916809 (Open Web Text)

Max Activating Token Index: 830

Text #10

Max Range: 1.8564. Min Range: -1.8564

Max Act: 1.8275. Min Act: -0.1700

Data Index: 3269206 (Open Web Text)

Max Activating Token Index: 439

Text #11

Max Range: 1.8564. Min Range: -1.8564

Max Act: 1.8194. Min Act: -0.1700

Data Index: 4849259 (Open Web Text)

Max Activating Token Index: 788

Text #12

Max Range: 1.8564. Min Range: -1.8564

Max Act: 1.8067. Min Act: -0.1700

Data Index: 6802197 (Open Web Text)

Max Activating Token Index: 1019

Text #13

Max Range: 1.8564. Min Range: -1.8564

Max Act: 1.8151. Min Act: -0.1700

Data Index: 4341389 (Open Web Text)

Max Activating Token Index: 722

Text #14

Max Range: 1.8564. Min Range: -1.8564

Max Act: 1.8035. Min Act: -0.1700

Data Index: 4050384 (Open Web Text)

Max Activating Token Index: 402

Text #15

Max Range: 1.8564. Min Range: -1.8564

Max Act: 1.7904. Min Act: -0.1700

Data Index: 3947351 (Open Web Text)

Max Activating Token Index: 320

Text #16

Max Range: 1.8564. Min Range: -1.8564

Max Act: 1.7853. Min Act: -0.1700

Data Index: 3492425 (Open Web Text)

Max Activating Token Index: 99

Text #17

Max Range: 1.8564. Min Range: -1.8564

Max Act: 1.7906. Min Act: -0.1700

Data Index: 441830 (Open Web Text)

Max Activating Token Index: 912

Text #18

Max Range: 1.8564. Min Range: -1.8564

Max Act: 1.7980. Min Act: -0.1700

Data Index: 2850438 (Open Web Text)

Max Activating Token Index: 29

Text #19

Max Range: 1.8564. Min Range: -1.8564

Max Act: 1.7988. Min Act: -0.1700

Data Index: 6082414 (Open Web Text)

Max Activating Token Index: 888