Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 1465 in Layer 1

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 1.8465. Min Range: -1.8465

Max Act: 1.8465. Min Act: -0.1700

Data Index: 7423665 (Open Web Text)

Max Activating Token Index: 162

Text #1

Max Range: 1.8465. Min Range: -1.8465

Max Act: 1.8224. Min Act: -0.1700

Data Index: 2721854 (Open Web Text)

Max Activating Token Index: 116

Text #2

Max Range: 1.8465. Min Range: -1.8465

Max Act: 1.8232. Min Act: -0.1700

Data Index: 141242 (Open Web Text)

Max Activating Token Index: 139

Text #3

Max Range: 1.8465. Min Range: -1.8465

Max Act: 1.8061. Min Act: -0.1700

Data Index: 6434071 (Open Web Text)

Max Activating Token Index: 71

Text #4

Max Range: 1.8465. Min Range: -1.8465

Max Act: 1.7939. Min Act: -0.1700

Data Index: 7936655 (Open Web Text)

Max Activating Token Index: 567

Text #5

Max Range: 1.8465. Min Range: -1.8465

Max Act: 1.7915. Min Act: -0.1700

Data Index: 4584977 (Open Web Text)

Max Activating Token Index: 175

Text #6

Max Range: 1.8465. Min Range: -1.8465

Max Act: 1.7789. Min Act: -0.1700

Data Index: 2269976 (Open Web Text)

Max Activating Token Index: 290

Text #7

Max Range: 1.8465. Min Range: -1.8465

Max Act: 1.7801. Min Act: -0.1700

Data Index: 7690071 (Open Web Text)

Max Activating Token Index: 613

Text #8

Max Range: 1.8465. Min Range: -1.8465

Max Act: 1.7880. Min Act: -0.1700

Data Index: 5449960 (Open Web Text)

Max Activating Token Index: 141

Text #9

Max Range: 1.8465. Min Range: -1.8465

Max Act: 1.7770. Min Act: -0.1700

Data Index: 5995423 (Open Web Text)

Max Activating Token Index: 237

Text #10

Max Range: 1.8465. Min Range: -1.8465

Max Act: 1.7754. Min Act: -0.1700

Data Index: 6257656 (Open Web Text)

Max Activating Token Index: 59

Text #11

Max Range: 1.8465. Min Range: -1.8465

Max Act: 1.7804. Min Act: -0.1699

Data Index: 4436472 (Open Web Text)

Max Activating Token Index: 73

Text #12

Max Range: 1.8465. Min Range: -1.8465

Max Act: 1.7677. Min Act: -0.1700

Data Index: 4449581 (Open Web Text)

Max Activating Token Index: 492

Text #13

Max Range: 1.8465. Min Range: -1.8465

Max Act: 1.7719. Min Act: -0.1700

Data Index: 6904639 (Open Web Text)

Max Activating Token Index: 739

Text #14

Max Range: 1.8465. Min Range: -1.8465

Max Act: 1.7650. Min Act: -0.1700

Data Index: 5066813 (Open Web Text)

Max Activating Token Index: 18

Text #15

Max Range: 1.8465. Min Range: -1.8465

Max Act: 1.7640. Min Act: -0.1700

Data Index: 5696765 (Open Web Text)

Max Activating Token Index: 147

Text #16

Max Range: 1.8465. Min Range: -1.8465

Max Act: 1.7555. Min Act: -0.1700

Data Index: 1975985 (Open Web Text)

Max Activating Token Index: 297

Text #17

Max Range: 1.8465. Min Range: -1.8465

Max Act: 1.7459. Min Act: -0.1700

Data Index: 7383907 (Open Web Text)

Max Activating Token Index: 330

Text #18

Max Range: 1.8465. Min Range: -1.8465

Max Act: 1.7612. Min Act: -0.1700

Data Index: 36055 (Open Web Text)

Max Activating Token Index: 197

Text #19

Max Range: 1.8465. Min Range: -1.8465

Max Act: 1.7547. Min Act: -0.1700

Data Index: 363530 (Open Web Text)

Max Activating Token Index: 48