Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 240 in Layer 3

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 3.7993. Min Range: -3.7993

Max Act: 3.7993. Min Act: -0.1700

Data Index: 5626132 (Open Web Text)

Max Activating Token Index: 484

Text #1

Max Range: 3.7993. Min Range: -3.7993

Max Act: 3.7703. Min Act: -0.1700

Data Index: 1590690 (Open Web Text)

Max Activating Token Index: 158

Text #2

Max Range: 3.7993. Min Range: -3.7993

Max Act: 3.7490. Min Act: -0.1700

Data Index: 6643836 (Open Web Text)

Max Activating Token Index: 139

Text #3

Max Range: 3.7993. Min Range: -3.7993

Max Act: 3.7521. Min Act: -0.1700

Data Index: 7611688 (Open Web Text)

Max Activating Token Index: 750

Text #4

Max Range: 3.7993. Min Range: -3.7993

Max Act: 3.7768. Min Act: -0.1700

Data Index: 2949654 (Open Web Text)

Max Activating Token Index: 5

Text #5

Max Range: 3.7993. Min Range: -3.7993

Max Act: 3.7078. Min Act: -0.1700

Data Index: 1823603 (Open Web Text)

Max Activating Token Index: 570

Text #6

Max Range: 3.7993. Min Range: -3.7993

Max Act: 3.6965. Min Act: -0.1700

Data Index: 5626128 (Open Web Text)

Max Activating Token Index: 130

Text #7

Max Range: 3.7993. Min Range: -3.7993

Max Act: 3.7177. Min Act: -0.1700

Data Index: 5626137 (Open Web Text)

Max Activating Token Index: 443

Text #8

Max Range: 3.7993. Min Range: -3.7993

Max Act: 3.7046. Min Act: -0.1700

Data Index: 6305709 (Open Web Text)

Max Activating Token Index: 147

Text #9

Max Range: 3.7993. Min Range: -3.7993

Max Act: 3.7017. Min Act: -0.1700

Data Index: 5523507 (Open Web Text)

Max Activating Token Index: 175

Text #10

Max Range: 3.7993. Min Range: -3.7993

Max Act: 3.6923. Min Act: -0.1700

Data Index: 5603890 (Open Web Text)

Max Activating Token Index: 385

Text #11

Max Range: 3.7993. Min Range: -3.7993

Max Act: 3.6592. Min Act: -0.1700

Data Index: 5332145 (Open Web Text)

Max Activating Token Index: 645

Text #12

Max Range: 3.7993. Min Range: -3.7993

Max Act: 3.6948. Min Act: -0.1700

Data Index: 8267280 (Open Web Text)

Max Activating Token Index: 865

Text #13

Max Range: 3.7993. Min Range: -3.7993

Max Act: 3.6539. Min Act: -0.1700

Data Index: 8210891 (Open Web Text)

Max Activating Token Index: 179

Text #14

Max Range: 3.7993. Min Range: -3.7993

Max Act: 3.6651. Min Act: -0.1700

Data Index: 7542252 (Open Web Text)

Max Activating Token Index: 329

Text #15

Max Range: 3.7993. Min Range: -3.7993

Max Act: 3.6785. Min Act: -0.1700

Data Index: 2887197 (Open Web Text)

Max Activating Token Index: 457

Text #16

Max Range: 3.7993. Min Range: -3.7993

Max Act: 3.6313. Min Act: -0.1700

Data Index: 4328892 (Open Web Text)

Max Activating Token Index: 921

Text #17

Max Range: 3.7993. Min Range: -3.7993

Max Act: 3.6185. Min Act: -0.1700

Data Index: 3332085 (Open Web Text)

Max Activating Token Index: 735

Text #18

Max Range: 3.7993. Min Range: -3.7993

Max Act: 3.6335. Min Act: -0.1700

Data Index: 1304808 (Open Web Text)

Max Activating Token Index: 908

Text #19

Max Range: 3.7993. Min Range: -3.7993

Max Act: 3.6381. Min Act: -0.1700

Data Index: 5574438 (Open Web Text)

Max Activating Token Index: 314