Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 2297 in Layer 4

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 3.6907. Min Range: -3.6907

Max Act: 3.6907. Min Act: -0.1700

Data Index: 2309171 (Open Web Text)

Max Activating Token Index: 981

Text #1

Max Range: 3.6907. Min Range: -3.6907

Max Act: 3.6666. Min Act: -0.1700

Data Index: 8758635 (Open Web Text)

Max Activating Token Index: 579

Text #2

Max Range: 3.6907. Min Range: -3.6907

Max Act: 3.6252. Min Act: -0.1700

Data Index: 1322819 (Open Web Text)

Max Activating Token Index: 793

Text #3

Max Range: 3.6907. Min Range: -3.6907

Max Act: 3.6651. Min Act: -0.1700

Data Index: 8732251 (Open Web Text)

Max Activating Token Index: 526

Text #4

Max Range: 3.6907. Min Range: -3.6907

Max Act: 3.6140. Min Act: -0.1700

Data Index: 4289440 (Open Web Text)

Max Activating Token Index: 488

Text #5

Max Range: 3.6907. Min Range: -3.6907

Max Act: 3.6387. Min Act: -0.1700

Data Index: 4252113 (Open Web Text)

Max Activating Token Index: 754

Text #6

Max Range: 3.6907. Min Range: -3.6907

Max Act: 3.5886. Min Act: -0.1700

Data Index: 3090224 (Open Web Text)

Max Activating Token Index: 712

Text #7

Max Range: 3.6907. Min Range: -3.6907

Max Act: 3.6192. Min Act: -0.1700

Data Index: 1694883 (Open Web Text)

Max Activating Token Index: 659

Text #8

Max Range: 3.6907. Min Range: -3.6907

Max Act: 3.5885. Min Act: -0.1700

Data Index: 3192243 (Open Web Text)

Max Activating Token Index: 212

Text #9

Max Range: 3.6907. Min Range: -3.6907

Max Act: 3.5910. Min Act: -0.1700

Data Index: 5601566 (Open Web Text)

Max Activating Token Index: 342

Text #10

Max Range: 3.6907. Min Range: -3.6907

Max Act: 3.5723. Min Act: -0.1700

Data Index: 1033102 (Open Web Text)

Max Activating Token Index: 552

Text #11

Max Range: 3.6907. Min Range: -3.6907

Max Act: 3.5840. Min Act: -0.1700

Data Index: 1549356 (Open Web Text)

Max Activating Token Index: 585

Text #12

Max Range: 3.6907. Min Range: -3.6907

Max Act: 3.5727. Min Act: -0.1700

Data Index: 8126356 (Open Web Text)

Max Activating Token Index: 446

Text #13

Max Range: 3.6907. Min Range: -3.6907

Max Act: 3.5833. Min Act: -0.1700

Data Index: 7512148 (Open Web Text)

Max Activating Token Index: 423

Text #14

Max Range: 3.6907. Min Range: -3.6907

Max Act: 3.5578. Min Act: -0.1700

Data Index: 4692026 (Open Web Text)

Max Activating Token Index: 802

Text #15

Max Range: 3.6907. Min Range: -3.6907

Max Act: 3.5630. Min Act: -0.1700

Data Index: 7387340 (Open Web Text)

Max Activating Token Index: 753

Text #16

Max Range: 3.6907. Min Range: -3.6907

Max Act: 3.5506. Min Act: -0.1700

Data Index: 5652322 (Open Web Text)

Max Activating Token Index: 731

Text #17

Max Range: 3.6907. Min Range: -3.6907

Max Act: 3.5430. Min Act: -0.1700

Data Index: 5184391 (Open Web Text)

Max Activating Token Index: 187

Text #18

Max Range: 3.6907. Min Range: -3.6907

Max Act: 3.5126. Min Act: -0.1700

Data Index: 2013615 (Open Web Text)

Max Activating Token Index: 863

Text #19

Max Range: 3.6907. Min Range: -3.6907

Max Act: 3.5293. Min Act: -0.1700

Data Index: 5097531 (Open Web Text)

Max Activating Token Index: 838