Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 1672 in Layer 5

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 3.0495. Min Range: -3.0495

Max Act: 3.0495. Min Act: -0.1700

Data Index: 561003 (Open Web Text)

Max Activating Token Index: 658

Text #1

Max Range: 3.0495. Min Range: -3.0495

Max Act: 3.0001. Min Act: -0.1700

Data Index: 2786273 (Open Web Text)

Max Activating Token Index: 550

Text #2

Max Range: 3.0495. Min Range: -3.0495

Max Act: 2.9858. Min Act: -0.1700

Data Index: 215030 (Open Web Text)

Max Activating Token Index: 606

Text #3

Max Range: 3.0495. Min Range: -3.0495

Max Act: 2.9811. Min Act: -0.1700

Data Index: 1577573 (Open Web Text)

Max Activating Token Index: 710

Text #4

Max Range: 3.0495. Min Range: -3.0495

Max Act: 2.9715. Min Act: -0.1700

Data Index: 5350683 (Open Web Text)

Max Activating Token Index: 221

Text #5

Max Range: 3.0495. Min Range: -3.0495

Max Act: 2.9575. Min Act: -0.1700

Data Index: 5537862 (Open Web Text)

Max Activating Token Index: 257

Text #6

Max Range: 3.0495. Min Range: -3.0495

Max Act: 2.9252. Min Act: -0.1700

Data Index: 4246615 (Open Web Text)

Max Activating Token Index: 739

Text #7

Max Range: 3.0495. Min Range: -3.0495

Max Act: 2.8990. Min Act: -0.1700

Data Index: 3646047 (Open Web Text)

Max Activating Token Index: 422

Text #8

Max Range: 3.0495. Min Range: -3.0495

Max Act: 2.9090. Min Act: -0.1700

Data Index: 731691 (Open Web Text)

Max Activating Token Index: 252

Text #9

Max Range: 3.0495. Min Range: -3.0495

Max Act: 2.9211. Min Act: -0.1700

Data Index: 3734713 (Open Web Text)

Max Activating Token Index: 324

Text #10

Max Range: 3.0495. Min Range: -3.0495

Max Act: 2.9076. Min Act: -0.1700

Data Index: 7357227 (Open Web Text)

Max Activating Token Index: 320

Text #11

Max Range: 3.0495. Min Range: -3.0495

Max Act: 2.8697. Min Act: -0.1700

Data Index: 5019876 (Open Web Text)

Max Activating Token Index: 456

Text #12

Max Range: 3.0495. Min Range: -3.0495

Max Act: 2.8719. Min Act: -0.1700

Data Index: 7272826 (Open Web Text)

Max Activating Token Index: 298

Text #13

Max Range: 3.0495. Min Range: -3.0495

Max Act: 2.8595. Min Act: -0.1700

Data Index: 1523169 (Open Web Text)

Max Activating Token Index: 448

Text #14

Max Range: 3.0495. Min Range: -3.0495

Max Act: 2.8729. Min Act: -0.1700

Data Index: 202319 (Open Web Text)

Max Activating Token Index: 738

Text #15

Max Range: 3.0495. Min Range: -3.0495

Max Act: 2.8675. Min Act: -0.1700

Data Index: 6324577 (Open Web Text)

Max Activating Token Index: 207

Text #16

Max Range: 3.0495. Min Range: -3.0495

Max Act: 2.8643. Min Act: -0.1700

Data Index: 5848367 (Open Web Text)

Max Activating Token Index: 296

Text #17

Max Range: 3.0495. Min Range: -3.0495

Max Act: 2.8542. Min Act: -0.1700

Data Index: 8636019 (Open Web Text)

Max Activating Token Index: 487

Text #18

Max Range: 3.0495. Min Range: -3.0495

Max Act: 2.8557. Min Act: -0.1700

Data Index: 7140785 (Open Web Text)

Max Activating Token Index: 180

Text #19

Max Range: 3.0495. Min Range: -3.0495

Max Act: 2.8569. Min Act: -0.1700

Data Index: 2258758 (Open Web Text)

Max Activating Token Index: 350