Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 1070 in Layer 5

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 3.6201. Min Range: -3.6201

Max Act: 3.6201. Min Act: -0.1700

Data Index: 6004881 (Open Web Text)

Max Activating Token Index: 4

Text #1

Max Range: 3.6201. Min Range: -3.6201

Max Act: 3.4779. Min Act: -0.1700

Data Index: 1344783 (Open Web Text)

Max Activating Token Index: 3

Text #2

Max Range: 3.6201. Min Range: -3.6201

Max Act: 3.3717. Min Act: -0.1700

Data Index: 5887709 (Open Web Text)

Max Activating Token Index: 324

Text #3

Max Range: 3.6201. Min Range: -3.6201

Max Act: 3.3508. Min Act: -0.1700

Data Index: 4641715 (Open Web Text)

Max Activating Token Index: 731

Text #4

Max Range: 3.6201. Min Range: -3.6201

Max Act: 3.3521. Min Act: -0.1700

Data Index: 5358126 (Open Web Text)

Max Activating Token Index: 186

Text #5

Max Range: 3.6201. Min Range: -3.6201

Max Act: 3.3606. Min Act: -0.1700

Data Index: 6668913 (Open Web Text)

Max Activating Token Index: 6

Text #6

Max Range: 3.6201. Min Range: -3.6201

Max Act: 3.3483. Min Act: -0.1700

Data Index: 4066231 (Open Web Text)

Max Activating Token Index: 93

Text #7

Max Range: 3.6201. Min Range: -3.6201

Max Act: 3.3671. Min Act: -0.1700

Data Index: 5857069 (Open Web Text)

Max Activating Token Index: 103

Text #8

Max Range: 3.6201. Min Range: -3.6201

Max Act: 3.3462. Min Act: -0.1700

Data Index: 7159917 (Open Web Text)

Max Activating Token Index: 27

Text #9

Max Range: 3.6201. Min Range: -3.6201

Max Act: 3.3351. Min Act: -0.1700

Data Index: 2270192 (Open Web Text)

Max Activating Token Index: 634

Text #10

Max Range: 3.6201. Min Range: -3.6201

Max Act: 3.3316. Min Act: -0.1700

Data Index: 2580344 (Open Web Text)

Max Activating Token Index: 24

Text #11

Max Range: 3.6201. Min Range: -3.6201

Max Act: 3.3390. Min Act: -0.1700

Data Index: 5042606 (Open Web Text)

Max Activating Token Index: 624

Text #12

Max Range: 3.6201. Min Range: -3.6201

Max Act: 3.3105. Min Act: -0.1700

Data Index: 7101309 (Open Web Text)

Max Activating Token Index: 331

Text #13

Max Range: 3.6201. Min Range: -3.6201

Max Act: 3.3172. Min Act: -0.1700

Data Index: 594992 (Open Web Text)

Max Activating Token Index: 173

Text #14

Max Range: 3.6201. Min Range: -3.6201

Max Act: 3.3289. Min Act: -0.1700

Data Index: 5354545 (Open Web Text)

Max Activating Token Index: 655

Text #15

Max Range: 3.6201. Min Range: -3.6201

Max Act: 3.3158. Min Act: -0.1700

Data Index: 7200507 (Open Web Text)

Max Activating Token Index: 927

Text #16

Max Range: 3.6201. Min Range: -3.6201

Max Act: 3.2883. Min Act: -0.1700

Data Index: 6179653 (Open Web Text)

Max Activating Token Index: 16

Text #17

Max Range: 3.6201. Min Range: -3.6201

Max Act: 3.2890. Min Act: -0.1700

Data Index: 1091793 (Open Web Text)

Max Activating Token Index: 681

Text #18

Max Range: 3.6201. Min Range: -3.6201

Max Act: 3.2990. Min Act: -0.1700

Data Index: 8282143 (Open Web Text)

Max Activating Token Index: 102

Text #19

Max Range: 3.6201. Min Range: -3.6201

Max Act: 3.2660. Min Act: -0.1700

Data Index: 1757497 (Open Web Text)

Max Activating Token Index: 10