Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 995 in Layer 4

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 4.5342. Min Range: -4.5342

Max Act: 4.5342. Min Act: -0.1700

Data Index: 8377918 (Open Web Text)

Max Activating Token Index: 207

Text #1

Max Range: 4.5342. Min Range: -4.5342

Max Act: 4.3121. Min Act: -0.1700

Data Index: 4602265 (Open Web Text)

Max Activating Token Index: 628

Text #2

Max Range: 4.5342. Min Range: -4.5342

Max Act: 4.2789. Min Act: -0.1700

Data Index: 4887965 (Open Web Text)

Max Activating Token Index: 361

Text #3

Max Range: 4.5342. Min Range: -4.5342

Max Act: 4.2885. Min Act: -0.1700

Data Index: 8680826 (Open Web Text)

Max Activating Token Index: 280

Text #4

Max Range: 4.5342. Min Range: -4.5342

Max Act: 4.2069. Min Act: -0.1700

Data Index: 2908574 (Open Web Text)

Max Activating Token Index: 493

Text #5

Max Range: 4.5342. Min Range: -4.5342

Max Act: 4.1868. Min Act: -0.1700

Data Index: 6902223 (Open Web Text)

Max Activating Token Index: 640

Text #6

Max Range: 4.5342. Min Range: -4.5342

Max Act: 4.1543. Min Act: -0.1700

Data Index: 3453359 (Open Web Text)

Max Activating Token Index: 158

Text #7

Max Range: 4.5342. Min Range: -4.5342

Max Act: 4.1503. Min Act: -0.1700

Data Index: 7930596 (Open Web Text)

Max Activating Token Index: 109

Text #8

Max Range: 4.5342. Min Range: -4.5342

Max Act: 4.1327. Min Act: -0.1700

Data Index: 8682109 (Open Web Text)

Max Activating Token Index: 580

Text #9

Max Range: 4.5342. Min Range: -4.5342

Max Act: 4.0752. Min Act: -0.1700

Data Index: 2549773 (Open Web Text)

Max Activating Token Index: 568

Text #10

Max Range: 4.5342. Min Range: -4.5342

Max Act: 4.0937. Min Act: -0.1700

Data Index: 5366707 (Open Web Text)

Max Activating Token Index: 397

Text #11

Max Range: 4.5342. Min Range: -4.5342

Max Act: 4.0704. Min Act: -0.1700

Data Index: 2188506 (Open Web Text)

Max Activating Token Index: 377

Text #12

Max Range: 4.5342. Min Range: -4.5342

Max Act: 4.0658. Min Act: -0.1700

Data Index: 8703649 (Open Web Text)

Max Activating Token Index: 457

Text #13

Max Range: 4.5342. Min Range: -4.5342

Max Act: 4.0358. Min Act: -0.1700

Data Index: 2589641 (Open Web Text)

Max Activating Token Index: 582

Text #14

Max Range: 4.5342. Min Range: -4.5342

Max Act: 4.0291. Min Act: -0.1700

Data Index: 4166542 (Open Web Text)

Max Activating Token Index: 601

Text #15

Max Range: 4.5342. Min Range: -4.5342

Max Act: 3.9998. Min Act: -0.1700

Data Index: 3920693 (Open Web Text)

Max Activating Token Index: 345

Text #16

Max Range: 4.5342. Min Range: -4.5342

Max Act: 3.9940. Min Act: -0.1700

Data Index: 2642636 (Open Web Text)

Max Activating Token Index: 709

Text #17

Max Range: 4.5342. Min Range: -4.5342

Max Act: 3.9606. Min Act: -0.1700

Data Index: 1710755 (Open Web Text)

Max Activating Token Index: 459

Text #18

Max Range: 4.5342. Min Range: -4.5342

Max Act: 3.9929. Min Act: -0.1700

Data Index: 198818 (Open Web Text)

Max Activating Token Index: 940

Text #19

Max Range: 4.5342. Min Range: -4.5342

Max Act: 3.9529. Min Act: -0.1700

Data Index: 3438833 (Open Web Text)

Max Activating Token Index: 919