Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 744 in Layer 6

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 3.9532. Min Range: -3.9532

Max Act: 3.9532. Min Act: -0.1700

Data Index: 929649 (Open Web Text)

Max Activating Token Index: 360

Text #1

Max Range: 3.9532. Min Range: -3.9532

Max Act: 3.9242. Min Act: -0.1700

Data Index: 7937762 (Open Web Text)

Max Activating Token Index: 221

Text #2

Max Range: 3.9532. Min Range: -3.9532

Max Act: 3.9024. Min Act: -0.1700

Data Index: 2061678 (Open Web Text)

Max Activating Token Index: 124

Text #3

Max Range: 3.9532. Min Range: -3.9532

Max Act: 3.8950. Min Act: -0.1700

Data Index: 2196864 (Open Web Text)

Max Activating Token Index: 1003

Text #4

Max Range: 3.9532. Min Range: -3.9532

Max Act: 3.8564. Min Act: -0.1700

Data Index: 5375146 (Open Web Text)

Max Activating Token Index: 318

Text #5

Max Range: 3.9532. Min Range: -3.9532

Max Act: 3.8197. Min Act: -0.1700

Data Index: 5307865 (Open Web Text)

Max Activating Token Index: 126

Text #6

Max Range: 3.9532. Min Range: -3.9532

Max Act: 3.8283. Min Act: -0.1700

Data Index: 4890861 (Open Web Text)

Max Activating Token Index: 661

Text #7

Max Range: 3.9532. Min Range: -3.9532

Max Act: 3.8277. Min Act: -0.1700

Data Index: 4638331 (Open Web Text)

Max Activating Token Index: 941

Text #8

Max Range: 3.9532. Min Range: -3.9532

Max Act: 3.8066. Min Act: -0.1700

Data Index: 5635053 (Open Web Text)

Max Activating Token Index: 773

Text #9

Max Range: 3.9532. Min Range: -3.9532

Max Act: 3.8179. Min Act: -0.1700

Data Index: 8737668 (Open Web Text)

Max Activating Token Index: 874

Text #10

Max Range: 3.9532. Min Range: -3.9532

Max Act: 3.8035. Min Act: -0.1700

Data Index: 4205837 (Open Web Text)

Max Activating Token Index: 805

Text #11

Max Range: 3.9532. Min Range: -3.9532

Max Act: 3.8077. Min Act: -0.1700

Data Index: 7563841 (Open Web Text)

Max Activating Token Index: 1022

Text #12

Max Range: 3.9532. Min Range: -3.9532

Max Act: 3.8103. Min Act: -0.1700

Data Index: 2893634 (Open Web Text)

Max Activating Token Index: 356

Text #13

Max Range: 3.9532. Min Range: -3.9532

Max Act: 3.8191. Min Act: -0.1700

Data Index: 4020006 (Open Web Text)

Max Activating Token Index: 200

Text #14

Max Range: 3.9532. Min Range: -3.9532

Max Act: 3.7660. Min Act: -0.1700

Data Index: 3177008 (Open Web Text)

Max Activating Token Index: 303

Text #15

Max Range: 3.9532. Min Range: -3.9532

Max Act: 3.7873. Min Act: -0.1700

Data Index: 7300928 (Open Web Text)

Max Activating Token Index: 67

Text #16

Max Range: 3.9532. Min Range: -3.9532

Max Act: 3.7295. Min Act: -0.1700

Data Index: 990311 (Open Web Text)

Max Activating Token Index: 454

Text #17

Max Range: 3.9532. Min Range: -3.9532

Max Act: 3.7463. Min Act: -0.1700

Data Index: 1752575 (Open Web Text)

Max Activating Token Index: 718

Text #18

Max Range: 3.9532. Min Range: -3.9532

Max Act: 3.7602. Min Act: -0.1700

Data Index: 7945905 (Open Web Text)

Max Activating Token Index: 431

Text #19

Max Range: 3.9532. Min Range: -3.9532

Max Act: 3.8009. Min Act: -0.1700

Data Index: 268185 (Open Web Text)

Max Activating Token Index: 536