Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 2505 in Layer 3

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 5.7831. Min Range: -5.7831

Max Act: 5.7831. Min Act: -0.1700

Data Index: 2813031 (Open Web Text)

Max Activating Token Index: 38

Text #1

Max Range: 5.7831. Min Range: -5.7831

Max Act: 5.7542. Min Act: -0.1700

Data Index: 1529789 (Open Web Text)

Max Activating Token Index: 434

Text #2

Max Range: 5.7831. Min Range: -5.7831

Max Act: 5.7585. Min Act: -0.1700

Data Index: 1208254 (Open Web Text)

Max Activating Token Index: 555

Text #3

Max Range: 5.7831. Min Range: -5.7831

Max Act: 5.7376. Min Act: -0.1700

Data Index: 8710699 (Open Web Text)

Max Activating Token Index: 530

Text #4

Max Range: 5.7831. Min Range: -5.7831

Max Act: 5.6936. Min Act: -0.1700

Data Index: 3023148 (Open Web Text)

Max Activating Token Index: 324

Text #5

Max Range: 5.7831. Min Range: -5.7831

Max Act: 5.6925. Min Act: -0.1700

Data Index: 586145 (Open Web Text)

Max Activating Token Index: 510

Text #6

Max Range: 5.7831. Min Range: -5.7831

Max Act: 5.7141. Min Act: -0.1700

Data Index: 2360643 (Open Web Text)

Max Activating Token Index: 97

Text #7

Max Range: 5.7831. Min Range: -5.7831

Max Act: 5.6624. Min Act: -0.1700

Data Index: 6739440 (Open Web Text)

Max Activating Token Index: 444

Text #8

Max Range: 5.7831. Min Range: -5.7831

Max Act: 5.6973. Min Act: -0.1700

Data Index: 3767225 (Open Web Text)

Max Activating Token Index: 354

Text #9

Max Range: 5.7831. Min Range: -5.7831

Max Act: 5.6591. Min Act: -0.1700

Data Index: 6713192 (Open Web Text)

Max Activating Token Index: 242

Text #10

Max Range: 5.7831. Min Range: -5.7831

Max Act: 5.6359. Min Act: -0.1700

Data Index: 3940449 (Open Web Text)

Max Activating Token Index: 393

Text #11

Max Range: 5.7831. Min Range: -5.7831

Max Act: 5.6419. Min Act: -0.1700

Data Index: 36846 (Open Web Text)

Max Activating Token Index: 459

Text #12

Max Range: 5.7831. Min Range: -5.7831

Max Act: 5.6445. Min Act: -0.1700

Data Index: 5833174 (Open Web Text)

Max Activating Token Index: 265

Text #13

Max Range: 5.7831. Min Range: -5.7831

Max Act: 5.6388. Min Act: -0.1700

Data Index: 6369603 (Open Web Text)

Max Activating Token Index: 498

Text #14

Max Range: 5.7831. Min Range: -5.7831

Max Act: 5.6506. Min Act: -0.1700

Data Index: 5313971 (Open Web Text)

Max Activating Token Index: 431

Text #15

Max Range: 5.7831. Min Range: -5.7831

Max Act: 5.6347. Min Act: -0.1700

Data Index: 5011813 (Open Web Text)

Max Activating Token Index: 465

Text #16

Max Range: 5.7831. Min Range: -5.7831

Max Act: 5.6635. Min Act: -0.1700

Data Index: 7086561 (Open Web Text)

Max Activating Token Index: 469

Text #17

Max Range: 5.7831. Min Range: -5.7831

Max Act: 5.6087. Min Act: -0.1700

Data Index: 1779332 (Open Web Text)

Max Activating Token Index: 378

Text #18

Max Range: 5.7831. Min Range: -5.7831

Max Act: 5.6202. Min Act: -0.1700

Data Index: 4672445 (Open Web Text)

Max Activating Token Index: 500

Text #19

Max Range: 5.7831. Min Range: -5.7831

Max Act: 5.6414. Min Act: -0.1700

Data Index: 7225756 (Open Web Text)

Max Activating Token Index: 759