Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 440 in Layer 5

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 3.8822. Min Range: -3.8822

Max Act: 3.8822. Min Act: -0.1700

Data Index: 2484239 (Open Web Text)

Max Activating Token Index: 443

Text #1

Max Range: 3.8822. Min Range: -3.8822

Max Act: 3.8665. Min Act: -0.1700

Data Index: 4306913 (Open Web Text)

Max Activating Token Index: 419

Text #2

Max Range: 3.8822. Min Range: -3.8822

Max Act: 3.7773. Min Act: -0.1700

Data Index: 1972175 (Open Web Text)

Max Activating Token Index: 294

Text #3

Max Range: 3.8822. Min Range: -3.8822

Max Act: 3.6627. Min Act: -0.1700

Data Index: 2218901 (Open Web Text)

Max Activating Token Index: 457

Text #4

Max Range: 3.8822. Min Range: -3.8822

Max Act: 3.6543. Min Act: -0.1700

Data Index: 7224954 (Open Web Text)

Max Activating Token Index: 848

Text #5

Max Range: 3.8822. Min Range: -3.8822

Max Act: 3.6694. Min Act: -0.1700

Data Index: 7183698 (Open Web Text)

Max Activating Token Index: 629

Text #6

Max Range: 3.8822. Min Range: -3.8822

Max Act: 3.5983. Min Act: -0.1700

Data Index: 2044707 (Open Web Text)

Max Activating Token Index: 824

Text #7

Max Range: 3.8822. Min Range: -3.8822

Max Act: 3.5844. Min Act: -0.1700

Data Index: 1903732 (Open Web Text)

Max Activating Token Index: 776

Text #8

Max Range: 3.8822. Min Range: -3.8822

Max Act: 3.5384. Min Act: -0.1700

Data Index: 4040164 (Open Web Text)

Max Activating Token Index: 523

Text #9

Max Range: 3.8822. Min Range: -3.8822

Max Act: 3.5778. Min Act: -0.1700

Data Index: 4940059 (Open Web Text)

Max Activating Token Index: 487

Text #10

Max Range: 3.8822. Min Range: -3.8822

Max Act: 3.5356. Min Act: -0.1700

Data Index: 1093855 (Open Web Text)

Max Activating Token Index: 680

Text #11

Max Range: 3.8822. Min Range: -3.8822

Max Act: 3.5411. Min Act: -0.1700

Data Index: 5103608 (Open Web Text)

Max Activating Token Index: 483

Text #12

Max Range: 3.8822. Min Range: -3.8822

Max Act: 3.5323. Min Act: -0.1700

Data Index: 1972166 (Open Web Text)

Max Activating Token Index: 234

Text #13

Max Range: 3.8822. Min Range: -3.8822

Max Act: 3.5164. Min Act: -0.1700

Data Index: 4486348 (Open Web Text)

Max Activating Token Index: 284

Text #14

Max Range: 3.8822. Min Range: -3.8822

Max Act: 3.5207. Min Act: -0.1700

Data Index: 5548821 (Open Web Text)

Max Activating Token Index: 234

Text #15

Max Range: 3.8822. Min Range: -3.8822

Max Act: 3.5070. Min Act: -0.1700

Data Index: 4586754 (Open Web Text)

Max Activating Token Index: 243

Text #16

Max Range: 3.8822. Min Range: -3.8822

Max Act: 3.5257. Min Act: -0.1700

Data Index: 1053843 (Open Web Text)

Max Activating Token Index: 280

Text #17

Max Range: 3.8822. Min Range: -3.8822

Max Act: 3.5025. Min Act: -0.1700

Data Index: 6442161 (Open Web Text)

Max Activating Token Index: 301

Text #18

Max Range: 3.8822. Min Range: -3.8822

Max Act: 3.5138. Min Act: -0.1700

Data Index: 6037264 (Open Web Text)

Max Activating Token Index: 80

Text #19

Max Range: 3.8822. Min Range: -3.8822

Max Act: 3.5266. Min Act: -0.1700

Data Index: 3211276 (Open Web Text)

Max Activating Token Index: 972