Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 1676 in Layer 2

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 6.0614. Min Range: -6.0614

Max Act: 6.0614. Min Act: -0.1700

Data Index: 5649439 (Open Web Text)

Max Activating Token Index: 101

Text #1

Max Range: 6.0614. Min Range: -6.0614

Max Act: 5.9290. Min Act: -0.1700

Data Index: 3071703 (Open Web Text)

Max Activating Token Index: 373

Text #2

Max Range: 6.0614. Min Range: -6.0614

Max Act: 5.8295. Min Act: -0.1700

Data Index: 1533664 (Open Web Text)

Max Activating Token Index: 6

Text #3

Max Range: 6.0614. Min Range: -6.0614

Max Act: 5.8111. Min Act: -0.1700

Data Index: 3573670 (Open Web Text)

Max Activating Token Index: 422

Text #4

Max Range: 6.0614. Min Range: -6.0614

Max Act: 5.8212. Min Act: -0.1700

Data Index: 3006451 (Open Web Text)

Max Activating Token Index: 16

Text #5

Max Range: 6.0614. Min Range: -6.0614

Max Act: 5.7703. Min Act: -0.1700

Data Index: 6062481 (Open Web Text)

Max Activating Token Index: 729

Text #6

Max Range: 6.0614. Min Range: -6.0614

Max Act: 5.6325. Min Act: -0.1700

Data Index: 1004169 (Open Web Text)

Max Activating Token Index: 18

Text #7

Max Range: 6.0614. Min Range: -6.0614

Max Act: 5.6197. Min Act: -0.1700

Data Index: 1456906 (Open Web Text)

Max Activating Token Index: 760

Text #8

Max Range: 6.0614. Min Range: -6.0614

Max Act: 5.6500. Min Act: -0.1700

Data Index: 273338 (Open Web Text)

Max Activating Token Index: 17

Text #9

Max Range: 6.0614. Min Range: -6.0614

Max Act: 5.6220. Min Act: -0.1700

Data Index: 3006450 (Open Web Text)

Max Activating Token Index: 19

Text #10

Max Range: 6.0614. Min Range: -6.0614

Max Act: 5.6029. Min Act: -0.1700

Data Index: 2365175 (Open Web Text)

Max Activating Token Index: 395

Text #11

Max Range: 6.0614. Min Range: -6.0614

Max Act: 5.6036. Min Act: -0.1700

Data Index: 1625682 (Open Web Text)

Max Activating Token Index: 179

Text #12

Max Range: 6.0614. Min Range: -6.0614

Max Act: 5.5665. Min Act: -0.1700

Data Index: 4958000 (Open Web Text)

Max Activating Token Index: 10

Text #13

Max Range: 6.0614. Min Range: -6.0614

Max Act: 5.5800. Min Act: -0.1700

Data Index: 5369572 (Open Web Text)

Max Activating Token Index: 18

Text #14

Max Range: 6.0614. Min Range: -6.0614

Max Act: 5.5646. Min Act: -0.1700

Data Index: 104620 (Open Web Text)

Max Activating Token Index: 14

Text #15

Max Range: 6.0614. Min Range: -6.0614

Max Act: 5.5895. Min Act: -0.1700

Data Index: 869197 (Open Web Text)

Max Activating Token Index: 1000

Text #16

Max Range: 6.0614. Min Range: -6.0614

Max Act: 5.5232. Min Act: -0.1699

Data Index: 1625681 (Open Web Text)

Max Activating Token Index: 929

Text #17

Max Range: 6.0614. Min Range: -6.0614

Max Act: 5.5165. Min Act: -0.1700

Data Index: 1681707 (Open Web Text)

Max Activating Token Index: 26

Text #18

Max Range: 6.0614. Min Range: -6.0614

Max Act: 5.4582. Min Act: -0.1700

Data Index: 6657199 (Open Web Text)

Max Activating Token Index: 18

Text #19

Max Range: 6.0614. Min Range: -6.0614

Max Act: 5.4210. Min Act: -0.1700

Data Index: 3585447 (Open Web Text)

Max Activating Token Index: 11