Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 2645 in Layer 1

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 5.4325. Min Range: -5.4325

Max Act: 5.4325. Min Act: -0.1700

Data Index: 5340604 (Open Web Text)

Max Activating Token Index: 618

Text #1

Max Range: 5.4325. Min Range: -5.4325

Max Act: 5.3955. Min Act: -0.1700

Data Index: 8272609 (Open Web Text)

Max Activating Token Index: 330

Text #2

Max Range: 5.4325. Min Range: -5.4325

Max Act: 5.3261. Min Act: -0.1700

Data Index: 531212 (Open Web Text)

Max Activating Token Index: 431

Text #3

Max Range: 5.4325. Min Range: -5.4325

Max Act: 5.3388. Min Act: -0.1700

Data Index: 4908741 (Open Web Text)

Max Activating Token Index: 48

Text #4

Max Range: 5.4325. Min Range: -5.4325

Max Act: 5.3364. Min Act: -0.1700

Data Index: 1974425 (Open Web Text)

Max Activating Token Index: 192

Text #5

Max Range: 5.4325. Min Range: -5.4325

Max Act: 5.3315. Min Act: -0.1700

Data Index: 6702901 (Open Web Text)

Max Activating Token Index: 176

Text #6

Max Range: 5.4325. Min Range: -5.4325

Max Act: 5.3250. Min Act: -0.1700

Data Index: 5782917 (Open Web Text)

Max Activating Token Index: 234

Text #7

Max Range: 5.4325. Min Range: -5.4325

Max Act: 5.2919. Min Act: -0.1700

Data Index: 5226034 (Open Web Text)

Max Activating Token Index: 195

Text #8

Max Range: 5.4325. Min Range: -5.4325

Max Act: 5.2972. Min Act: -0.1700

Data Index: 3568960 (Open Web Text)

Max Activating Token Index: 123

Text #9

Max Range: 5.4325. Min Range: -5.4325

Max Act: 5.2907. Min Act: -0.1700

Data Index: 3361027 (Open Web Text)

Max Activating Token Index: 308

Text #10

Max Range: 5.4325. Min Range: -5.4325

Max Act: 5.2861. Min Act: -0.1700

Data Index: 6187230 (Open Web Text)

Max Activating Token Index: 552

Text #11

Max Range: 5.4325. Min Range: -5.4325

Max Act: 5.3220. Min Act: -0.1700

Data Index: 5782918 (Open Web Text)

Max Activating Token Index: 235

Text #12

Max Range: 5.4325. Min Range: -5.4325

Max Act: 5.2806. Min Act: -0.1700

Data Index: 2411846 (Open Web Text)

Max Activating Token Index: 297

Text #13

Max Range: 5.4325. Min Range: -5.4325

Max Act: 5.3199. Min Act: -0.1700

Data Index: 1575695 (Open Web Text)

Max Activating Token Index: 636

Text #14

Max Range: 5.4325. Min Range: -5.4325

Max Act: 5.2571. Min Act: -0.1700

Data Index: 396393 (Open Web Text)

Max Activating Token Index: 55

Text #15

Max Range: 5.4325. Min Range: -5.4325

Max Act: 5.2651. Min Act: -0.1700

Data Index: 5910483 (Open Web Text)

Max Activating Token Index: 454

Text #16

Max Range: 5.4325. Min Range: -5.4325

Max Act: 5.2876. Min Act: -0.1700

Data Index: 2856506 (Open Web Text)

Max Activating Token Index: 391

Text #17

Max Range: 5.4325. Min Range: -5.4325

Max Act: 5.2920. Min Act: -0.1700

Data Index: 5686857 (Open Web Text)

Max Activating Token Index: 740

Text #18

Max Range: 5.4325. Min Range: -5.4325

Max Act: 5.2874. Min Act: -0.1700

Data Index: 5226506 (Open Web Text)

Max Activating Token Index: 400

Text #19

Max Range: 5.4325. Min Range: -5.4325

Max Act: 5.2749. Min Act: -0.1700

Data Index: 2072701 (Open Web Text)

Max Activating Token Index: 58