Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 1216 in Layer 6

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 3.3267. Min Range: -3.3267

Max Act: 3.3267. Min Act: -0.1700

Data Index: 1540209 (Open Web Text)

Max Activating Token Index: 384

Text #1

Max Range: 3.3267. Min Range: -3.3267

Max Act: 3.3003. Min Act: -0.1700

Data Index: 2502192 (Open Web Text)

Max Activating Token Index: 510

Text #2

Max Range: 3.3267. Min Range: -3.3267

Max Act: 3.2700. Min Act: -0.1700

Data Index: 7381474 (Open Web Text)

Max Activating Token Index: 508

Text #3

Max Range: 3.3267. Min Range: -3.3267

Max Act: 3.2257. Min Act: -0.1700

Data Index: 6291932 (Open Web Text)

Max Activating Token Index: 876

Text #4

Max Range: 3.3267. Min Range: -3.3267

Max Act: 3.1841. Min Act: -0.1700

Data Index: 3128285 (Open Web Text)

Max Activating Token Index: 80

Text #5

Max Range: 3.3267. Min Range: -3.3267

Max Act: 3.1884. Min Act: -0.1700

Data Index: 8131196 (Open Web Text)

Max Activating Token Index: 990

Text #6

Max Range: 3.3267. Min Range: -3.3267

Max Act: 3.2053. Min Act: -0.1700

Data Index: 3099458 (Open Web Text)

Max Activating Token Index: 146

Text #7

Max Range: 3.3267. Min Range: -3.3267

Max Act: 3.1866. Min Act: -0.1700

Data Index: 7215884 (Open Web Text)

Max Activating Token Index: 275

Text #8

Max Range: 3.3267. Min Range: -3.3267

Max Act: 3.1840. Min Act: -0.1700

Data Index: 1371086 (Open Web Text)

Max Activating Token Index: 627

Text #9

Max Range: 3.3267. Min Range: -3.3267

Max Act: 3.1806. Min Act: -0.1700

Data Index: 6856465 (Open Web Text)

Max Activating Token Index: 778

Text #10

Max Range: 3.3267. Min Range: -3.3267

Max Act: 3.1820. Min Act: -0.1700

Data Index: 6283497 (Open Web Text)

Max Activating Token Index: 136

Text #11

Max Range: 3.3267. Min Range: -3.3267

Max Act: 3.1860. Min Act: -0.1700

Data Index: 6106995 (Open Web Text)

Max Activating Token Index: 724

Text #12

Max Range: 3.3267. Min Range: -3.3267

Max Act: 3.1712. Min Act: -0.1700

Data Index: 5956504 (Open Web Text)

Max Activating Token Index: 615

Text #13

Max Range: 3.3267. Min Range: -3.3267

Max Act: 3.1916. Min Act: -0.1700

Data Index: 341153 (Open Web Text)

Max Activating Token Index: 378

Text #14

Max Range: 3.3267. Min Range: -3.3267

Max Act: 3.2021. Min Act: -0.1700

Data Index: 2534792 (Open Web Text)

Max Activating Token Index: 511

Text #15

Max Range: 3.3267. Min Range: -3.3267

Max Act: 3.1841. Min Act: -0.1700

Data Index: 3397048 (Open Web Text)

Max Activating Token Index: 200

Text #16

Max Range: 3.3267. Min Range: -3.3267

Max Act: 3.1673. Min Act: -0.1700

Data Index: 5475220 (Open Web Text)

Max Activating Token Index: 176

Text #17

Max Range: 3.3267. Min Range: -3.3267

Max Act: 3.1854. Min Act: -0.1700

Data Index: 4091726 (Open Web Text)

Max Activating Token Index: 599

Text #18

Max Range: 3.3267. Min Range: -3.3267

Max Act: 3.1633. Min Act: -0.1700

Data Index: 1415213 (Open Web Text)

Max Activating Token Index: 486

Text #19

Max Range: 3.3267. Min Range: -3.3267

Max Act: 3.1675. Min Act: -0.1700

Data Index: 1550704 (Open Web Text)

Max Activating Token Index: 431