Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 552 in Layer 6

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 3.4071. Min Range: -3.4071

Max Act: 3.4029. Min Act: -0.1700

Data Index: 5649551 (Open Web Text)

Max Activating Token Index: 293

Text #1

Max Range: 3.4071. Min Range: -3.4071

Max Act: 3.3905. Min Act: -0.1700

Data Index: 5847272 (Open Web Text)

Max Activating Token Index: 636

Text #2

Max Range: 3.4071. Min Range: -3.4071

Max Act: 3.4071. Min Act: -0.1700

Data Index: 3315194 (Open Web Text)

Max Activating Token Index: 285

Text #3

Max Range: 3.4071. Min Range: -3.4071

Max Act: 3.3987. Min Act: -0.1700

Data Index: 3146660 (Open Web Text)

Max Activating Token Index: 299

Text #4

Max Range: 3.4071. Min Range: -3.4071

Max Act: 3.3508. Min Act: -0.1700

Data Index: 5108860 (Open Web Text)

Max Activating Token Index: 185

Text #5

Max Range: 3.4071. Min Range: -3.4071

Max Act: 3.3546. Min Act: -0.1700

Data Index: 2440661 (Open Web Text)

Max Activating Token Index: 918

Text #6

Max Range: 3.4071. Min Range: -3.4071

Max Act: 3.3490. Min Act: -0.1700

Data Index: 5063370 (Open Web Text)

Max Activating Token Index: 349

Text #7

Max Range: 3.4071. Min Range: -3.4071

Max Act: 3.3488. Min Act: -0.1700

Data Index: 5148156 (Open Web Text)

Max Activating Token Index: 231

Text #8

Max Range: 3.4071. Min Range: -3.4071

Max Act: 3.3185. Min Act: -0.1700

Data Index: 6091272 (Open Web Text)

Max Activating Token Index: 407

Text #9

Max Range: 3.4071. Min Range: -3.4071

Max Act: 3.3037. Min Act: -0.1700

Data Index: 7797467 (Open Web Text)

Max Activating Token Index: 410

Text #10

Max Range: 3.4071. Min Range: -3.4071

Max Act: 3.3091. Min Act: -0.1700

Data Index: 1770271 (Open Web Text)

Max Activating Token Index: 201

Text #11

Max Range: 3.4071. Min Range: -3.4071

Max Act: 3.2784. Min Act: -0.1700

Data Index: 5221712 (Open Web Text)

Max Activating Token Index: 375

Text #12

Max Range: 3.4071. Min Range: -3.4071

Max Act: 3.3157. Min Act: -0.1700

Data Index: 2275892 (Open Web Text)

Max Activating Token Index: 731

Text #13

Max Range: 3.4071. Min Range: -3.4071

Max Act: 3.2976. Min Act: -0.1700

Data Index: 3856562 (Open Web Text)

Max Activating Token Index: 1022

Text #14

Max Range: 3.4071. Min Range: -3.4071

Max Act: 3.2999. Min Act: -0.1700

Data Index: 7505146 (Open Web Text)

Max Activating Token Index: 356

Text #15

Max Range: 3.4071. Min Range: -3.4071

Max Act: 3.3034. Min Act: -0.1700

Data Index: 7637147 (Open Web Text)

Max Activating Token Index: 16

Text #16

Max Range: 3.4071. Min Range: -3.4071

Max Act: 3.3067. Min Act: -0.1700

Data Index: 970432 (Open Web Text)

Max Activating Token Index: 704

Text #17

Max Range: 3.4071. Min Range: -3.4071

Max Act: 3.2916. Min Act: -0.1700

Data Index: 763935 (Open Web Text)

Max Activating Token Index: 768

Text #18

Max Range: 3.4071. Min Range: -3.4071

Max Act: 3.2914. Min Act: -0.1700

Data Index: 4364349 (Open Web Text)

Max Activating Token Index: 602

Text #19

Max Range: 3.4071. Min Range: -3.4071

Max Act: 3.2878. Min Act: -0.1700

Data Index: 457441 (Open Web Text)

Max Activating Token Index: 125