Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 545 in Layer 0

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 3.4859. Min Range: -3.4859

Max Act: 3.4812. Min Act: -0.1700

Data Index: 7503275 (Open Web Text)

Max Activating Token Index: 2

Text #1

Max Range: 3.4859. Min Range: -3.4859

Max Act: 3.4859. Min Act: -0.1700

Data Index: 4771640 (Open Web Text)

Max Activating Token Index: 3

Text #2

Max Range: 3.4859. Min Range: -3.4859

Max Act: 3.4552. Min Act: -0.1700

Data Index: 5711736 (Open Web Text)

Max Activating Token Index: 6

Text #3

Max Range: 3.4859. Min Range: -3.4859

Max Act: 3.4195. Min Act: -0.1700

Data Index: 5965414 (Open Web Text)

Max Activating Token Index: 2

Text #4

Max Range: 3.4859. Min Range: -3.4859

Max Act: 3.3985. Min Act: -0.1700

Data Index: 6757694 (Open Web Text)

Max Activating Token Index: 3

Text #5

Max Range: 3.4859. Min Range: -3.4859

Max Act: 3.3719. Min Act: -0.1700

Data Index: 7178654 (Open Web Text)

Max Activating Token Index: 7

Text #6

Max Range: 3.4859. Min Range: -3.4859

Max Act: 3.3752. Min Act: -0.1700

Data Index: 1646109 (Open Web Text)

Max Activating Token Index: 10

Text #7

Max Range: 3.4859. Min Range: -3.4859

Max Act: 3.3559. Min Act: -0.1700

Data Index: 6509228 (Open Web Text)

Max Activating Token Index: 5

Text #8

Max Range: 3.4859. Min Range: -3.4859

Max Act: 3.3509. Min Act: -0.1700

Data Index: 4175638 (Open Web Text)

Max Activating Token Index: 2

Text #9

Max Range: 3.4859. Min Range: -3.4859

Max Act: 3.3509. Min Act: -0.1700

Data Index: 5218021 (Open Web Text)

Max Activating Token Index: 2

Text #10

Max Range: 3.4859. Min Range: -3.4859

Max Act: 3.3583. Min Act: -0.1700

Data Index: 7676178 (Open Web Text)

Max Activating Token Index: 17

Text #11

Max Range: 3.4859. Min Range: -3.4859

Max Act: 3.3523. Min Act: -0.1700

Data Index: 5511836 (Open Web Text)

Max Activating Token Index: 6

Text #12

Max Range: 3.4859. Min Range: -3.4859

Max Act: 3.3424. Min Act: -0.1700

Data Index: 5154009 (Open Web Text)

Max Activating Token Index: 2

Text #13

Max Range: 3.4859. Min Range: -3.4859

Max Act: 3.3329. Min Act: -0.1700

Data Index: 2024448 (Open Web Text)

Max Activating Token Index: 213

Text #14

Max Range: 3.4859. Min Range: -3.4859

Max Act: 3.3071. Min Act: -0.1700

Data Index: 5724563 (Open Web Text)

Max Activating Token Index: 12

Text #15

Max Range: 3.4859. Min Range: -3.4859

Max Act: 3.3222. Min Act: -0.1700

Data Index: 6755181 (Open Web Text)

Max Activating Token Index: 7

Text #16

Max Range: 3.4859. Min Range: -3.4859

Max Act: 3.3095. Min Act: -0.1700

Data Index: 3436033 (Open Web Text)

Max Activating Token Index: 636

Text #17

Max Range: 3.4859. Min Range: -3.4859

Max Act: 3.2973. Min Act: -0.1700

Data Index: 4260033 (Open Web Text)

Max Activating Token Index: 4

Text #18

Max Range: 3.4859. Min Range: -3.4859

Max Act: 3.2899. Min Act: -0.1700

Data Index: 2295660 (Open Web Text)

Max Activating Token Index: 510

Text #19

Max Range: 3.4859. Min Range: -3.4859

Max Act: 3.2816. Min Act: -0.1700

Data Index: 6631314 (Open Web Text)

Max Activating Token Index: 9