Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 718 in Layer 1

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 4.7127. Min Range: -4.7127

Max Act: 4.7127. Min Act: -0.1700

Data Index: 1925255 (Open Web Text)

Max Activating Token Index: 332

Text #1

Max Range: 4.7127. Min Range: -4.7127

Max Act: 4.6687. Min Act: -0.1700

Data Index: 4042205 (Open Web Text)

Max Activating Token Index: 277

Text #2

Max Range: 4.7127. Min Range: -4.7127

Max Act: 4.6593. Min Act: -0.1700

Data Index: 5526230 (Open Web Text)

Max Activating Token Index: 308

Text #3

Max Range: 4.7127. Min Range: -4.7127

Max Act: 4.6250. Min Act: -0.1700

Data Index: 7826342 (Open Web Text)

Max Activating Token Index: 550

Text #4

Max Range: 4.7127. Min Range: -4.7127

Max Act: 4.6155. Min Act: -0.1700

Data Index: 5190712 (Open Web Text)

Max Activating Token Index: 221

Text #5

Max Range: 4.7127. Min Range: -4.7127

Max Act: 4.6274. Min Act: -0.1700

Data Index: 1229330 (Open Web Text)

Max Activating Token Index: 380

Text #6

Max Range: 4.7127. Min Range: -4.7127

Max Act: 4.5771. Min Act: -0.1700

Data Index: 81820 (Open Web Text)

Max Activating Token Index: 398

Text #7

Max Range: 4.7127. Min Range: -4.7127

Max Act: 4.6029. Min Act: -0.1700

Data Index: 1262316 (Open Web Text)

Max Activating Token Index: 686

Text #8

Max Range: 4.7127. Min Range: -4.7127

Max Act: 4.5838. Min Act: -0.1700

Data Index: 8392283 (Open Web Text)

Max Activating Token Index: 267

Text #9

Max Range: 4.7127. Min Range: -4.7127

Max Act: 4.5848. Min Act: -0.1700

Data Index: 1201336 (Open Web Text)

Max Activating Token Index: 213

Text #10

Max Range: 4.7127. Min Range: -4.7127

Max Act: 4.6014. Min Act: -0.1700

Data Index: 2091678 (Open Web Text)

Max Activating Token Index: 602

Text #11

Max Range: 4.7127. Min Range: -4.7127

Max Act: 4.5570. Min Act: -0.1700

Data Index: 7360015 (Open Web Text)

Max Activating Token Index: 403

Text #12

Max Range: 4.7127. Min Range: -4.7127

Max Act: 4.5591. Min Act: -0.1700

Data Index: 2971712 (Open Web Text)

Max Activating Token Index: 263

Text #13

Max Range: 4.7127. Min Range: -4.7127

Max Act: 4.5677. Min Act: -0.1700

Data Index: 3592084 (Open Web Text)

Max Activating Token Index: 397

Text #14

Max Range: 4.7127. Min Range: -4.7127

Max Act: 4.5556. Min Act: -0.1700

Data Index: 312933 (Open Web Text)

Max Activating Token Index: 237

Text #15

Max Range: 4.7127. Min Range: -4.7127

Max Act: 4.5800. Min Act: -0.1700

Data Index: 5328265 (Open Web Text)

Max Activating Token Index: 170

Text #16

Max Range: 4.7127. Min Range: -4.7127

Max Act: 4.5502. Min Act: -0.1700

Data Index: 8494415 (Open Web Text)

Max Activating Token Index: 392

Text #17

Max Range: 4.7127. Min Range: -4.7127

Max Act: 4.5731. Min Act: -0.1700

Data Index: 3941341 (Open Web Text)

Max Activating Token Index: 295

Text #18

Max Range: 4.7127. Min Range: -4.7127

Max Act: 4.5433. Min Act: -0.1700

Data Index: 5868830 (Open Web Text)

Max Activating Token Index: 722

Text #19

Max Range: 4.7127. Min Range: -4.7127

Max Act: 4.5391. Min Act: -0.1700

Data Index: 3933782 (Open Web Text)

Max Activating Token Index: 183